Release notes

Enhancements

• Language engines:
  • Updated language resources for the Hunspell spell check engine: British English, French, Indonesian, Dutch, Polish, Ukrainian, Mexican Spanish, Danish, German, Swiss German, Bulgarian, Czech, Chilean Spanish.
  • Slovenian language code was changed from sl_SL to sl_SI. This change affected the 2nd part of the locale ID with the country code. It’s SI (Slovenia). The old language code will still work for the existing clients. However, it’s recommended to make changes in the configuration and migrate to the correct language code.
  • Prioritized grammar type suggestions if the spelling engine gives “No suggestions”. In other cases, spelling suggestions will supersede.
  • [On-premise] Updated the structure of the folders/directories with resources such as language files and additional wordlists.
• User-level custom dictionary. Conducted a complete refactoring of user custom dictionary functionality including its HTTP API command “user_dictionary”. Its behavior and validation mechanisms now are adapted to a company-level or global custom dictionary. 
  • Input validation for newly added words and dictionary name. For example, words with special characters, punctuation marks or spaces can no longer be added to the dictionary. 
  • Updated error messages to be more precise and clear. Validated error codes returned for API responses.
  • Restricted input to a single word for addword and deleteword actions with word parameter. In upcoming releases, wordlist parameter will be added for actions with many words. 
• WProofreader core:
  • Added UI localization for Indonesian language “id”. It can be enabled by adding the “localization: id” option to the config script.
  • Updated Spanish localization “es” for “Autocomplete suggestions” to ”Texto predictivo”.
• Other:
  • WProofreader SDK demos. Added a new demo with the WProofreader integration into TinyMCE v6.

Bug fixes

• [WP-4593] Spelling: The word ”pay” is flagged as a misspelling in AmE
• [WP-4543] Spelling: Capitalized forms of British words are present in the AmE dictionary (e.g. Colour, Fibre, Paediatric, Aesthetic)
• [WP-4495] WProofreader core: isUserDictionaryExist method doesn’t corresponds its name
• [WP-4564] User dictionary HTTP API: Incorrect status code when “word” isn’t deleted
• [WP-4603] User dictionary HTTP API: Action to delete non-existent dictionary returns 200 OK
• [WP-4539] HTTP API: Invalid encoded URI returns BAD URI syntax
• [WP-4396] Specific words can’t be deleted using exclude functionality

Enhancement

AppServer

• Updated Hunspell spell check engine to version 1.7.1.
• Updated POCO C++ library to version 1.11.5.

Bug fixes

• [WP-4557] HTTP API: The same check requests with enabled custom dictionary for American and British English returns different suggestions [Cloud product version]
• [WP-4566] HTTP API: Сheck request with custom_dictionary parameter returns error related to unsupported language Id [Cloud product version]*

*This fix will be applied on December 27, 2022 once the involved component is updated.

Hot fix release for the issue with AI requests that randomly lead to the application crash.

Note! We had to roll back the ICU changes made in v5.26.0 due to the issues that led to the application crashes causing 5xx errors. This change is applied in both cloud and on-premise product versions.

• Replaced sentence tokenization mechanism. The old custom sentence tokenization mechanism was replaced with the ICU word break iterator tool to improve processing of words that contain dots (abbreviations), hyphens, apostrophes, special symbols, shortening with one dot, possessions, etc. This approach is implemented for all languages that are based on the Hunspell spelling check engine.

Bug fixes

• [WP-4180] Spelling check engine: Incorrect processing of words with trailing hyphens in German

Note! Only the on-premise product version was fully released. The cloud one is still in progress. We had to roll back the cloud release, specifically changes in the backend component, due to the issues that led to the application crashes causing 5xx errors. However, all the changes introduced in the WProofreader core are available in the cloud version as well.

Enhancements

Updated WProofreader core v3.8.4782:

• Global badge v2. Rethought the conception and behavior of the global badge. This badge no longer is tied to text fields but is global on the page. By default, it’s disabled globalBadge:false but if enabled it will be placed in the bottom right corner of the page. Its position can be regulated using the following options: badgeOffsetX, badgeOffsetY, badgeZIndex (by default, badgeOffsetX = 20, badgeOffsetY = 20, badgeZIndex = 1000).  

Grammar engine. Updated 3rd party engine, LanguageTool, to version 5.9.0.

• Added and improved grammar rules for the next languages: Catalan, Dutch, English, German, French, Polish, Portuguese, Russian, Spanish, Ukrainian.
• Updated 3rd party components to fix vulnerabilities in .jar libraries: CVE-2022-42003, CVE-2022-42004, CVE-2022-3171, CVE-2022-42889. 

Replaced sentence tokenization mechanism.

The old custom sentence tokenization mechanism was replaced with the ICU word break iterator tool to improve processing of words that contain dots (abbreviations), hyphens, apostrophes, special symbols, shortening with one dot, possessions, etc. This approach is implemented for all languages that are based on the Hunspell spelling check engine.

Bug fixes

• [WP-4447] WProofreader core: No markup when user opens Full screen view in Google Gmail [detected using browser extension]
• [WP-4477] WProofreader core: No flag icon for Valencian (cat_ES) language
• [WP-4476] WProofreader core: Apostrophe (‘) is not allowed symbol for adding to the dictionary from UI
• [WP-4484] WProofreader core: WProofreader doesn’t work properly in IE11 in textareas
• [WP-4180] Spelling check engine: Incorrect processing of words with trailing hyphens in German
• [WP-4519] HTTP API: If set to “true”, ignore_mixed_case also affects and enables ignore_all_caps
• [WP-4501] Dictionaries: Some medical terms are recognized as incorrect words (e.g. AAOMR, ABthrax, ACTHoma, ALternaGEL)
• [WP-4491, WP-4524] Spelling: Words “filter”, “practice” and “groin” are flagged as misspellings in British English
• [WP-4452] AI engine: Misspelled words in quotes are recognized and underlined as grammar mistakes

Breaking changes

[On-premise product version] Updated the application server (AppServer) configuration file, AppServer.X.xml:

• Updated existing tags as follows:
  • Replaced Enabled tag for SpellCheck to Enabled attribute: <SpellCheck Enabled="true">
  • Added boolean Enabled attribute for Model, GrammarCheck, Autocomplete and Autocorrect tags.
• Added MaxInputSize parameter which is responsible for setting the max number of input tokens for AI language models in AppServer.X.xml. It adds flexibility in controlling model input size.

Breaking changes

[On-premise product version] Updated the application server (AppServer) configuration file, AppServer.X.xml:

• Added a new parameter / tag called CheckKit. It allows flexibly configuring the language settings.
• Renamed a list of the existing tags in the AppServer.X.xml 
  • GrammarCheckProvider → GrammarCheck
  • SpellEngineOptions → SpellCheck
  • FullPath → Path (it can work in both sides with a absolute path and relative path)
  • SpellCheckEnginePriority → SpellCheckPriority
  • AutoCompleteCheckProviderOptions → Autocomplete + <Locale>en</Locale>
  • AutoCorrect → Autocorrect

Enhancements

• BETA. English dialects support AI-based English under the hood. Implemented for next dialects:  American, British, Canadian and Australian English. By default it’s disabled but you can enable it as follows:
  • HTTP API: add “enforce_ai=true” to a query string;
  • WProofreader SDK integration: add “enforceAI: true” option to the configuration script.
• New version of WProofreader core v3.7.4760: 
  • Added option/parameter to turn ON/OFF enhanced text checking powered by AI.
  • Updated the default path for the WProofreader core for the cloud product version from “servicePath”: “spellcheck31/script/ssrv.cgi” to “servicePath”: “api”.
• Released v2.2.0 of the WProofreader plugin for CKEditor 5.  Added support of the Restricted editing mode.
• Improved AI-based English proofreading mechanism. Better handling cases with hyphenated compound words (e.g. mother-in-law), various typographical symbols and punctuation, contraction of verbs (e.g. “it’s”).    
• [On-premise product version] Updated the demo samples inside the package  to match the online version. By default the path to access the demos as follows: http: localhost/wscservice/samples/

Bug fixes

• [WP-4399] WProofreader core: Styles were broken in the title of the JIRA issue [detected using browser extension]
• [WP-4428] WProofreader core: Disabled autocorrect feature for Slate.js
• [WP-4418] WProofreader core: Focus state shifted in Google Chat  [detected using browser extension]
• [WP-4405] AppServer: “System exception“ error occurred during license activation
• [WP-4442] [Cloud] BrE spelling: “tired” word marked as misspelled
• [WP-4451] [Cloud] BrE spelling: “benifitting”/”benifitted” words are marked as misspelled
• [WP-4443] AI engine: Incorrect handling suggestions to errors with zero length
• [WP-4411] HTTP API: Checking text with “%” causes BAD URI syntax error

Enhancements

Updated WProofreader core v3.6.4736

• Changed the badge behavior and view. By default, it starts in a small size and has a few states: orange – all is OK, no suggestions found; red – some suggestions found. On hover, users can check the exact number of alerts/suggestions detected. To return it to a large size use the option fullSizeBadge: true.
• Implemented the 1st version of the global badge under globalBadge option name. If enabled, there will be only one global badge on the page that aggregates all the suggestions on the page instead of having separate badges for each field.  By default, it’s disabled: globalBadge: false.
• Changed the default state of the suggestion popup view by enabling additional actions like: proofread in dialog, settings, toggle.
• Deprecated the badge pulsation effect and related disableBadgePulsing​​ option. Also, with the introduction of the small badge, removed the mechanism with the dynamic badge-size depending on the content and the text field.
• Learn more in the updated version of WProofreader API documentation.

Languages

Updated the sources of the Hunspell dictionaries for the next languages:

• Australian English. Source: en_AU Hunspell Dictionary by Kevin Atkinson
• Canadian English. Source: en_CA Hunspell Dictionary by Kevin Atkinson
• American English. Source: en_US Hunspell Dictionary by Kevin Atkinson
• English South Africa. Source: en_ZA Hunspell Dictionary by Dwayne Bailey
• English New Zealand. Source: Firefox Extensions > New Zealand English Dictionary by Stephen
• British English. Source: Sourceforge>latest SCOWL dictionary
• Portuguese. Source: pt_PT Hunspell Dictionary by Tiago F. Santos
• Brazilian Portuguese. Source: pt_BR Hunspell Dictionary by Raimundo Moura

Other

• [On-Premise product version] HTTP API demo added to the demo samples, the same as we have in our public demos.

Bug fixes

• [WP-4384,GitHub#32] WProofreader plugin for CKEditor 5: Accessibility concerns related to WProofreader badge element
• [WP-4388] WProofreader core: Autocorrect messes up corrected words
• [WP-4392] Spelling: Portuguese and Brazilian Portuguese don’t handle words with “ções”and “não” ending

Breaking changes

[On-premise product version] Updated Docker configuration:

• Settings (e.g. protocol, domain, web-port, etc.) can now be changed on the container run. More details in README.
• Application server (AppServer) logs are now linked to stdout, so they can be found in docker logs. Note that the license activation logs that are generated during image creation are stored in Logs/img_build_logs.tar.gz.
• config.ini and configSSL.ini were removed, all build parameters are now ARGs in the Dockerfile and also can be set as ENV variables.

Enhancements

Languages/Engines:

• AI-based German. Integrated version 2 of the AI German language model with improved processing quality, F0.5 score increased by 0.05 points. Now the model produces fewer false positive results (incorrect suggestions).
• Grammar engine. Updated 3rd party engine, LanguageTool, to version 5.8.0. 
  • Added and improved grammar rules for the next languages: Catalan, English, German, French, Polish, Portuguese, Russian, Spanish, Ukrainian.
  • Updated 3rd party component to fix vulnerability: CVE 2022-25647. Removed vulnerable Jetty libraries from the package. 
• Dictionaries. Extended the German and English medical dictionaries with new terms reported by the clients (e.g. COVID-19, Moderna, AstraZeneca).

Custom dictionary API:

• Words added to custom dictionaries can be divided by next separators: \n, Space, Tab, comma.
• Formed the list of supported characters that can be added to the dictionary (letters, digits and some punctuation marks). Words containing only digits or punctuation as well as their combinations are not supported.

Bug fixes

• [WP-4349, GitHub #11951] WProofreader core: The caret moves to a different block when pressing space after a word with grammar error in CKEditor 5
• [WP-4352] WProofreader core: Incorrect behavior of cursor restore during autocorrect in Draft.js

Breaking changes

[On-premise product version] Updated the automated install script:

• Updated web_server_type values : 1 – Apache HTTP Server, 2 – NGINX, 3 – Other
• Renamed web server related parameters:
  • restart_apache → restart_web_server
  • apache_config_dir → web_server_config_dir

Note! Both cloud and on-premise product versions release. The cloud version includes everything from v5.19.1 and the updated grammar check engine to version 5.7.

Enhancements

• Grammar engine. Updated 3rd party engine, LanguageTool, to version 5.7. Added and improved grammar rules for the next languages: Catalan, French, German, Polish, Portuguese, Russian, Spanish and Ukrainian. It also includes updated versions of dependencies #6423 and fix for multithreading issue #6423 in v5.6.

Note! On-premise product release only.

Enhancements

• HTTP API. Added text splitting for the parameter text of check API command in case of AI-based languages such as English, German and Spanish. Initially only token parameter could be used where text was manually divided into sentences.
• Java-based grammar engine troubleshooting. Extended logging mechanism with full stack trace in case of exceptions in the Java grammar module.
• Increased default allocated memory value for Java Virtual Machine (JVM) from 512 MB to 700 MB.
• Improvised security. Excluded from the build vulnerable versions of 3rd party dependencies in the grammar engine module: logback-core:1.2, protobuf-java:3.17.2, jackson-databind:2.12.0.

Bug fixes

• [WP-3967] WProofreader core: Vertical scroll inside <input> element
• [WP-4210] WProofreader core: Broken styles for ‘ckeditor5’, ‘gray’ and ‘dark’ themes
• [WP-4213] WProofreader core: Vertical scroll is shown in the <input> element
• [WP-4214] WProofreader core: Horizontal scroll comes up after initialization in Google Chat editable area
• [WP-4220] WProofreader core: Underlines are shifted when vertical scroll overlaps the content of <textarea> element

Note! Only on-premise product version was released. The cloud product version is still in progress. We had to postpone the release due to Russia’s aggressive military invasion in Ukraine (WebSpellChecker is made by Ukrainians & in Ukraine).

Enhancements

• Autocomplete suggestions (beta). Brand new functionality is implemented and added to WProofreader-based products – autocomplete suggestions while typing for English. It suggests only a word or short phrase, suggestion can be accepted using the left arrow “>”on the keyboard. Going forward it will be extended to more comprehensive phrases or common sentences. The feature is disabled by default, it can be easily enabled from the user interface by end users or added by developer to the configuration using autocomplete option.
• Grammar engine. Updated 3rd party engine, LanguageTool, to version 5.6. Added and improved grammar rules for the next languages: Catalan, Dutch, English, French, German, Portuguese, Russian, Spanish and Ukrainian.
• Security improvements. Updated LanguageTool grammar engine with all its dependencies to eliminate common vulnerabilities. 
• WProofreader core. The version is updated to v3.5.xxx with the release of autocomplete suggestions. Disabled badge pulsation effect: “disableBadgePulsing: true”.

Bug fixes

• [WP-4176] Incorrect words count is recorded usage statistics for the cloud product version
• [WP-4181] WProofreader core: Broken CSS styles in a client environment
• [WP-4171] WProofreader core: Markup fails in texareas with more than 65 535 characters [Chrome]

Enhancements

• [Cloud] Speed up the process of the service activation after the subscription. Previously it took around 5-10 minutes for the subscription to be activated. Now the service starts working right away.
• Enable English, Spanish, French medical dictionaries for appropriate languages and their dialects. 
• Enable dictionary-based spell check engine priority for AI-based Spanish and German. 

Breaking changes

• The list of supported languages was shortened from 160+ to approximately 80+ languages. Only unpopular and unused languages were removed. However, clients who use on-premise product versions have an option to add missing languages manually (Hunspell spell check engine). The updated list of supported languages.
• Withdrawn the limit on the number of languages that can be used within cloud or on-premise product versions. The on-premise version package is shipped with all supported languages. However, some of them are in the disabled state. They can be easily enabled using the instructions in the following guide. 

Bug fixes

• [WP-4140] Words with apostrophe can’t be added to user dictionary