Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--auto-correct stops with error when parsing invalid encoding file #8590

Closed
ups-ishii opened this issue Aug 26, 2020 · 4 comments · Fixed by #8591
Closed

--auto-correct stops with error when parsing invalid encoding file #8590

ups-ishii opened this issue Aug 26, 2020 · 4 comments · Fixed by #8591

Comments

@ups-ishii
Copy link

Expected behavior

shows offence as Lint/Syntax and inspect next file.

Actual behavior

stops on parser error

Steps to reproduce the problem

  1. create a file, invalid encoding, like below.
# encoding: sjis

aiueo = "あいうえお"
  1. run auto-correct command.
    rubocop --auto-correct ./invalid_encoding.rb

output

Inspecting 1 file
.

0 files inspected, no offenses detected
Cannot extract source from uninitialized Source::Buffer
/Library/Ruby/Gems/2.6.0/gems/parser-2.7.1.4/lib/parser/source/buffer.rb:148:in `source'
/Library/Ruby/Gems/2.6.0/gems/parser-2.7.1.4/lib/parser/source/buffer.rb:297:in `source_range'
/Library/Ruby/Gems/2.6.0/gems/parser-2.7.1.4/lib/parser/source/tree_rewriter.rb:116:in `initialize'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cop/corrector.rb:18:in `initialize'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cop/team.rb:182:in `new'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cop/team.rb:182:in `collate_corrections'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cop/team.rb:176:in `autocorrect_report'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cop/team.rb:121:in `autocorrect'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cop/team.rb:79:in `investigate'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:295:in `inspect_file'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:245:in `block in do_inspection_loop'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:277:in `block in iterate_until_no_changes'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:270:in `loop'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:270:in `iterate_until_no_changes'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:241:in `do_inspection_loop'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:121:in `block in file_offenses'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:146:in `file_offense_cache'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:120:in `file_offenses'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:111:in `process_file'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:90:in `block in each_inspected_file'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:89:in `each'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:89:in `reduce'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:89:in `each_inspected_file'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:78:in `inspect_files'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/runner.rb:39:in `run'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cli/command/execute_runner.rb:21:in `execute_runner'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cli/command/execute_runner.rb:13:in `run'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cli/command.rb:10:in `run'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cli/environment.rb:17:in `run'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cli.rb:65:in `run_command'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cli.rb:72:in `execute_runners'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/lib/rubocop/cli.rb:41:in `run'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/exe/rubocop:13:in `block in <top (required)>'
/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/benchmark.rb:308:in `realtime'
/Library/Ruby/Gems/2.6.0/gems/rubocop-0.89.1/exe/rubocop:12:in `<top (required)>'
/usr/local/bin/rubocop:23:in `load'
/usr/local/bin/rubocop:23:in `<main>'
  • FYI: Linting without auto-correct works.

rubocop --cache false ./invalid_encoding.rb shows:

Inspecting 1 file
F

Offenses:

invalid_encoding.rb:1:1: F: Lint/Syntax: "\x86\xe3" from windows-31j to utf-8.

1 file inspected, 1 offense detected

RuboCop version

$ rubocop -V
0.89.1 (using Parser 2.7.1.4, rubocop-ast 0.3.0, running on ruby 2.6.3 universal.x86_64-darwin19)
@koic
Copy link
Member

koic commented Aug 26, 2020

Probably the file encoding is not sjis as specified in the magic comment.

If the result of nkf --guess is UTF-8, convert it to Shift_JIS. The following is an example:

% nkf --guess invalid_encoding.rb
UTF-8 (LF)

% nkf --overwrite -s invalid_encoding.rb
% nkf --guess invalid_encoding.rb
Shift_JIS (LF)

Thank you.

@koic koic closed this as completed Aug 26, 2020
@ups-ishii
Copy link
Author

Probably the file encoding is not sjis as specified in the magic comment.

@koic Thank you for your comment.
Yes, You're right. The code in this file is for intentionally causing a parse error.
The problem is that this error causes the program to hang.

In the case of rubocop --cache false --format emacs,
other files can be processed until the end even if an error occurs
in the parser in the middle.

I think this parser error should be treated as fatal issue like
rubocop --cache false --format emacs does instead of stop executing Rubocop jobs.

How do you think?

koic added a commit to koic/rubocop that referenced this issue Aug 26, 2020
@koic
[Fix rubocop#8590] Fix an error when auto-correcting encoding mismatc…
7315ad0 
…h file

Fixes rubocop#8590.

This PR fixes the following error when auto-correcting encoding mismatch file.

```ruby
% cat example.rb
# encoding: Shift_JIS

puts 'Ｔｈｉｓ ｆｉｌｅ ｅｎｃｏｄｉｎｇ ｉｓ ＵＴＦ－８．'
```

## Before

```console
% bundle exec rubocop --cache false -a
(snip)

Cannot extract source from uninitialized Source::Buffer
/Users/koic/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/parser-2.7.1.4/lib/
parser/source/buffer.rb:148:in `source'
/Users/koic/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/parser-2.7.1.4/lib/
parser/source/buffer.rb:297:in `source_range'
/Users/koic/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/parser-2.7.1.4/lib/
parser/source/tree_rewriter.rb:116:in `initialize'
/Users/koic/src/github.com/rubocop-hq/rubocop/lib/rubocop/cop/corrector.rb:18:in
`initialize'
/Users/koic/src/github.com/rubocop-hq/rubocop/lib/rubocop/cop/team.rb:182:in
`new'
/Users/koic/src/github.com/rubocop-hq/rubocop/lib/rubocop/cop/team.rb:182:in
`collate_corrections'
/Users/koic/src/github.com/rubocop-hq/rubocop/lib/rubocop/cop/team.rb:176:in
`autocorrect_report'
/Users/koic/src/github.com/rubocop-hq/rubocop/lib/rubocop/cop/team.rb:121:in
`autocorrect'
/Users/koic/src/github.com/rubocop-hq/rubocop/lib/rubocop/cop/team.rb:79:in
`investigate'
/Users/koic/src/github.com/rubocop-hq/rubocop/lib/rubocop/runner.rb:295:in `inspect_file'
```

## After

```console
% bundle exec rubocop --cache false -a
(snip)

Inspecting 1 file
F

Offenses:

example.rb:1:1: F: Lint/Syntax: "\xef\xbc" from shift_jis to utf-8.

1 file inspected, 1 offense detected
```
@koic koic mentioned this issue Aug 26, 2020
Merged
8 tasks
@koic
Copy link
Member

koic commented Aug 26, 2020

Ah, get it. I've opened a PR #8591.

@koic koic reopened this Aug 26, 2020
koic added a commit that referenced this issue Aug 26, 2020
@koic
Merge pull request #8591 from koic/fix_an_error_when_autocorrecting
e3c930d 
[Fix #8590] Fix an error when auto-correcting encoding mismatch file
@ups-ishii
Copy link
Author

Thank you so much!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Fix #8590] Fix an error when auto-correcting encoding mismatch file koic/rubocop
2 participants
@koic @ups-ishii