You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a function of splitting combined words.
For example, the combined word "crossregionswitch" can be split into three words: "cross", "region", and "switch".
The text was updated successfully, but these errors were encountered:
Enhanced LanguageModel class to include a split method for text processing based on English word frequencies. The split function breaks text into words and punctuations using a language model based on English word frequencies. Additionally, added comprehensive pytest unit tests to validate the split functionality under various scenarios, including punctuation and contractions.
- Implemented split method in LanguageModel for text processing
- Added unit tests for split functionality covering punctuation and contractions
- Ensured adherence to PEP8 standards and code readability
Resolves: nltk#3195
Add a function of splitting combined words.
For example, the combined word "crossregionswitch" can be split into three words: "cross", "region", and "switch".
The text was updated successfully, but these errors were encountered: