- lexer.rl: fix incompatible delimiters on percent literal #808
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CRuby only accepts ASCII characters except
[A-Za-z0-9]
as a delimiterof percent literal, but the lexer accepts different characters.
For exmaple:
%w^Dfoo^D
, but parser didn't (note:^D
means 0x04)%w1foo1
, but parser accepts%w★foo★
, but parser acceptsThis patch fixes the problems.
Investigation of CRuby
CRuby parses percent literals here: https://github.com/ruby/ruby/blob/6072239121360293dbd2ed607f16b6a11668999a/parse.y#L8718-L8815
ASCII delimiters
I confirmed Ruby 1.8 or greater accept all ASCII characters except alnum as delimiters with the following code.
test.rb
Number and multibyte delimiter
I also confirmed Ruby 1.8 or greater reject
1
and★
as delimiters with the following commands.