Skip to content

Commit

Permalink
Allow Unicode in Python identifiers (#1510)
Browse files Browse the repository at this point in the history
Python supports the use of Unicode in identifiers. This commit uses
POSIX bracket expressions that match against Unicode characters rather
than the more common character classes that only match ASCII characters.

Co-authored-by: Michael Camilleri <mike@inqk.net>
  • Loading branch information
niknetniko and pyrmont committed May 12, 2020
1 parent 8e26a63 commit 9f1f582
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 2 deletions.
4 changes: 2 additions & 2 deletions lib/rouge/lexers/python.rb
Expand Up @@ -67,8 +67,8 @@ def self.exceptions
)
end

identifier = /[a-z_][a-z0-9_]*/i
dotted_identifier = /[a-z_.][a-z0-9_.]*/i
identifier = /[[:alpha:]_][[:alnum:]_]*/
dotted_identifier = /[[:alpha:]_.][[:alnum:]_.]*/

def current_string
@string_register ||= StringRegister.new
Expand Down
5 changes: 5 additions & 0 deletions spec/visual/samples/python
Expand Up @@ -150,3 +150,8 @@ x @= y
f'{hello} world {int(x) + 1}'
f'{{ {4*10} }}'
f'result: {value:{width}.{precision}}'

# Unicode identifiers
α = 10
def coöperative(б):
return f"{б} is Russian"

0 comments on commit 9f1f582

Please sign in to comment.