Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix escape sequences in Python's raw strings #1508

Merged
merged 5 commits into from May 12, 2020
Merged

Fix escape sequences in Python's raw strings #1508

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4402e3e
Add example with invalid escape sequences
pyrmont Apr 24, 2020
c0c1391
Add new escape state for raw strings
pyrmont Apr 24, 2020
5fa37f1
Permit all characters after \
pyrmont Apr 25, 2020
e8f73d4
Simplify rule
pyrmont Apr 25, 2020
a1907b4
Add \newline pattern
pyrmont Apr 25, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
18 changes: 9 additions & 9 deletions lib/rouge/lexers/python.rb
View file
Open in desktop
Expand Up @@ -189,10 +189,11 @@ def current_string
rule %r/\\/ do |m|
if current_string.type? "r"
token Str
push :raw_escape
else
token Str::Interpol
push :generic_escape
end
push :generic_escape
end

rule %r/{/ do |m|
Expand All @@ -215,14 +216,9 @@ def current_string
| x[a-fA-F0-9]{2}
| [0-7]{1,3}
)
)x do
if current_string.type? "r"
token Str
else
token Str::Escape
end
pop!
end
)x, Str::Escape, :pop!

rule %r/./, Error, :pop!
pyrmont marked this conversation as resolved.
Show resolved Hide resolved
end

state :generic_interpol do
Expand All @@ -233,6 +229,10 @@ def current_string
rule %r/}/, Str::Interpol, :pop!
end

state :raw_escape do
rule %r/./, Str, :pop!
end

class StringRegister < Array
def delim?(delim)
self.last[1] == delim
Expand Down
2 changes: 2 additions & 0 deletions spec/visual/samples/python
View file
Open in desktop
Expand Up @@ -47,6 +47,7 @@ def baz():
'\UaaaaAF09'
'\xaf\xAF\x09'
'\007'
'.*\[p00t_(d\d{4})\].*' # This should include errors

# escaped characters in raw strings
def baz():
Expand All @@ -56,6 +57,7 @@ def baz():
r'\UaaaaAF09'
r'\xaf\xAF\x09'
r'\007'
r'.*\[p00t_(d\d{4})\].*' # This should not include errors

# line continuations
apple.filter(x, y)
Expand Down