Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Escaped backslash at end of line causes newline to be removed in heredocs #789

Closed
eugeneotto opened this issue Mar 12, 2021 · 5 comments
Closed

Escaped backslash at end of line causes newline to be removed in heredocs #789

eugeneotto opened this issue Mar 12, 2021 · 5 comments

Comments

@eugeneotto
Copy link

Here is a small script that demonstrates the behavior:

$ cat foo.rb
puts <<~TXT
i am line one
i am line two \\
i am line \
three
i am line four \\
TXT

This is the script's output:

$ ruby foo.rb
i am line one
i am line two \
i am line three
i am line four \

This is ruby-parse's output:

$ ruby-parse foo.rb
(send nil :puts
  (dstr
    (str "i am line one\n")
    (str "i am line two ") # this line does not have a newline
    (str "i am line three\n")
    (str "i am line four \\ \n"))) # if there is another character after the escaped backslash, we get a newline

$ ruby-parse -v
ruby-parse based on parser version 3.0.0.0
@eugeneotto
Copy link
Author

This does not appear to be an issue with stabby heredocs (<<-TXT).

@iliabylich
Copy link
Collaborator

@eugeneotto Thanks for reporting it.

The bug is definitely somewhere in https://github.com/whitequark/parser/blob/master/lib/parser/lexer/dedenter.rb. Looks like blindly splitting by "\\\n" is a wrong strategy, because "\\" part can go after another "\\" and so then it's just a single "\" character in the output.

I'll try to find some time to fix it. Or feel free to send a PR if you have time.

@mvz
Copy link
Contributor

mvz commented Sep 24, 2021

I'm working on a pull request for a related problem that seems to fix this issue as well.

@mvz mvz mentioned this issue Sep 24, 2021
Merged
iliabylich pushed a commit that referenced this issue Sep 24, 2021
@mvz
- dedenter.rb: Fix squiggly heredoc line continuation handling (#819)
61bf873 
Firstly, this fixes an issue with non-interpolating squiggly heredocs, where line continuation characters should be ignored. Two things were broken:

- The line continuation backslash would be removed, as well as the subsequent carriage return
- The subsequent line would not be dedented.

For example, the following source:

```ruby
<<~'END'
  foo \
  bar
END
```

Would be parsed as follows:

```
(dstr
  (str "foo ")
  (str "  bar\n"))
```

With the change, the result is:

```
(dstr
  (str "foo \\\n")
  (str "bar\n"))
```

This also fixes an issue with interpolating squiggly heredocs, where an escaped backslash followed by a newline would be interpreted as a line continuation. This is issue #789.

In that case, the following source:

```ruby
<<~END
  foo \\
  bar
END
```

Would (again) be parsed as follows:

```
(dstr
  (str "foo ")
  (str "  bar\n"))
```

With the change, the result is:

```
(dstr
  (str "foo \\\n")
  (str "bar\n"))
```
@iliabylich
Copy link
Collaborator

Fixed in #819:

$ /bin/cat test.rb
puts <<~TXT
i am line one
i am line two \\
i am line \
three
i am line four \\
TXT

$ bin/ruby-parse test.rb
(send nil :puts
  (dstr
    (str "i am line one\n")
    (str "i am line two \\\n")
    (str "i am line three\n")
    (str "i am line four \\\n")))

@eugeneotto Do you need a release? If not it will be released with the next version of Ruby (or once someone asks for release)

@eugeneotto
Copy link
Author

Thank you @iliabylich and @mvz! I can wait for the next version of Ruby, but do appreciate the offer 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@eugeneotto @mvz @iliabylich