Skip to content

Commit

Permalink
- Fix an error when using heredoc with non-word delimiters
Browse files Browse the repository at this point in the history
I've found #986 regressions in the following RuboCop CI matrix:
https://github.com/rubocop/rubocop/actions/runs/7437077900/job/20234293932?pr=12598

So, #986 was not a sufficient fix. This PR fixes an error when using heredoc with non-word delimiters:

```ruby
<<~'+'
  foo
+
```

The following repro test have been newly added:

```console
$ bundle exec ruby -Itest test/test_lexer.rb
(snip)

  1) Error:
TestLexer#test_heredoc_plus_character:
RegexpError: target of repeat operator is not specified: /+\z/
    /Users/koic/src/github.com/whitequark/parser/lib/parser/lexer/literal.rb:250:in `delimiter?'
    /Users/koic/src/github.com/whitequark/parser/lib/parser/lexer/literal.rb:138:in `nest_and_try_closing'
    /Users/koic/src/github.com/whitequark/parser/lib/parser/lexer-strings.rb:4165:in `advance'
    /Users/koic/src/github.com/whitequark/parser/lib/parser/lexer-F1.rb:11369:in `advance'
    test/test_lexer.rb:81:in `assert_scanned'
    test/test_lexer.rb:1001:in `test_heredoc_plus_character'
```
  • Loading branch information
koic committed Jan 7, 2024
1 parent 2620bc4 commit bebc50e
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 1 deletion.
2 changes: 1 addition & 1 deletion lib/parser/lexer/literal.rb
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ def delimiter?(delimiter)
# E
# because there are not enough leading spaces in the closing delimiter.
delimiter.end_with?(@end_delim) &&
delimiter.sub(/#{@end_delim}\z/, '').bytes.all? { |c| c == SPACE }
delimiter.sub(/#{Regexp.escape(@end_delim)}\z/, '').bytes.all? { |c| c == SPACE }
elsif @indent
@end_delim == delimiter.lstrip
else
Expand Down
10 changes: 10 additions & 0 deletions test/test_lexer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -997,6 +997,16 @@ def test_heredoc_one_character
:tNL, nil, [7, 8])
end

def test_heredoc_plus_character
assert_scanned("a = <<'+'\nABCDEF\n+\n",
:tIDENTIFIER, "a", [0, 1],
:tEQL, "=", [2, 3],
:tSTRING_BEG, "<<\'", [4, 9],
:tSTRING_CONTENT, "ABCDEF\n", [10, 17],
:tSTRING_END, "+", [17, 18],
:tNL, nil, [9, 10])
end

def test_heredoc_cr
assert_scanned("a = <<E\r\r\nABCDEF\r\r\nE\r\r\r\n",
:tIDENTIFIER, "a", [0, 1],
Expand Down

0 comments on commit bebc50e

Please sign in to comment.