Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demojize performance #197

Merged
merged 5 commits into from Dec 6, 2021
Merged

Conversation

cvzi
Copy link
Contributor

@cvzi cvzi commented Nov 30, 2021

This pull request improves the demojize performance. I have replace the big regular expression with a search tree. The method _get_search_tree() creates the tree once on first use. I have put an example of a tree in the comment of _get_search_tree()

The get_emoji_regexp() still exists but it is unused now.

Performance is faster on my machine for both single emoji and a long text. This is approximately how much faster it is on my machine:

Python 2.7 3.6 3.10
Single emoji: 7x 7x 12x faster
Text [200k chars]: 1.5x 7x 23x faster

I have also added emojize(language='alias') as an alternative to emojize(use_aliases=True).
When someone uses emojize(language='de', use_aliases=True) or any other language then 'en', a warning is shown:

warnings.warn("use_aliases=True is only supported for language='en'. "
              "It is recommended to use emojize(string, language='alias') instead")

(I could remove this change to the aliases part, it is not connected to the performance at all)

cvzi and others added 5 commits November 18, 2021 16:11
…r of previous versions)

Bugfix: Default delimiters were used instead of the custom delimiters in emojize() when an unknown emoji was found.
@TahirJalilov
Copy link
Collaborator

Thank you for your PR @cvzi

@TahirJalilov TahirJalilov merged commit e35fc45 into carpedm20:master Dec 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants