Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make number_of_words respect CJK characters #7813

Merged
merged 15 commits into from May 22, 2020
Merged

Make number_of_words respect CJK characters #7813

Changes from 3 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
afbee53
Make number_of_words respect CJK characters
iBug Sep 5, 2019
3cc01b1
Maybe use lowercase for that variable?
iBug Sep 5, 2019
1d37203
RuboCop is hard to satisfy...
iBug Sep 5, 2019
6a33254
Use exclamation marks for %r!regex! to make RuboCop happy
iBug Sep 5, 2019
1a6af42
Update documentation to reflect changes
iBug Sep 5, 2019
360ecda
Add test for patched number_of_words filter
iBug Mar 7, 2020
28da2dc
Fix according to ashmaroli's suggestions
iBug May 21, 2020
9c46861
Using single-character regexes without repetition is errorneous
iBug May 21, 2020
d0ba979
Bundle exec rake test passing
iBug May 21, 2020
2f60cab
Merge master and fix RuboCop
iBug May 21, 2020
21bf5a5
Make CJK processing optional, adapt tests
iBug May 22, 2020
36fe3da
Support auto mode, add Japanese and Korean test cases
iBug May 22, 2020
758949b
Update documentation to reflect #7813
iBug May 22, 2020
676dd86
Improve documentation of the change
ashmaroli May 22, 2020
25d1388
Reduce duplication in filter
ashmaroli May 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion lib/jekyll/filters.rb
View file
Open in desktop
Expand Up @@ -122,7 +122,8 @@ def normalize_whitespace(input)
#
# Returns the Integer word count.
def number_of_words(input)
input.split.length
cjk_regex = %r(\p{Han}|\p{Katakana}|\p{Hiragana}|\p{Hangul})
iBug marked this conversation as resolved.
Show resolved Hide resolved
input.scan(cjk_regex).length + input.gsub(cjk_regex, " ").split.length
end

# Join an array of things into a string by separating with commas and the
Expand Down