Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Windows encoding problems #2146

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Xanders
Copy link

@Xanders Xanders commented Jul 24, 2020

Hi! Thanks for the greatest project!! I just realized there are some bugs with filesystem encoding on Windows. Dir.pwd gives me string in Windows-1251 encoding on my machine, so Pry starting process is broken with incompatibility exception. Also whereami command is broken too. So I think it's safe to perform encode('UTF-8') on some strings from filesystem to ensure they are compatible with other. Everything works as expected on my machine after fixes.

I'm using Pry 0.13.1 on Ruby 2.7.0 on Windows 10.

Copy link
Member

@kyrylo kyrylo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you post the full exception(s)? FWIW we don't really test on Windows because we lack manpower. This solution doesn't look too clean to me.

@Xanders
Copy link
Author

Xanders commented Nov 28, 2020

@kyrylo, thank you for reply! There is full exception for me:

Encoding::CompatibilityError: incompatible character encodings: UTF-8 and Windows-1251
from C:/tools/ruby27/lib/ruby/gems/2.7.0/gems/pry-0.13.1/lib/pry/code/code_file.rb:102:in `expand_path'

And:

Encoding::CompatibilityError: incompatible character encodings: IBM866 and UTF-8
from C:/tools/ruby27/lib/ruby/gems/2.7.0/gems/pry-0.13.1/lib/pry/commands/show_info.rb:112:in `+'

Steps to reproduce:

  1. Start Windows Sandbox, install Chocolatey and run choco install ruby, gem install pry
  2. Create folder and Ruby-file with non-ASCII symbols in path and content:
def тест
end

require 'pry'
Pry.start
  1. Run the file and hit show-method тест

The root of the problem is Windows default filesystem, please see below:

C:\Data\Тест>docker run --rm ruby ruby -e "p Dir.pwd.encoding"
#<Encoding:UTF-8>

C:\Data\Тест>ruby -e "p Dir.pwd.encoding"
#<Encoding:Windows-1251>

C:\Data\Тест>docker run --rm ruby ruby -e "p Dir['*'].first.encoding"
#<Encoding:UTF-8>

C:\Data\Тест>ruby -e "p Dir['*'].first.encoding"
#<Encoding:IBM866>

In Ruby we cannot mix non-ASCII symbols with different encodings. So on Windows we should convert non-UTF-8 strings to UTF-8 before first use. This does not affect strings that are already in UTF-8.

I just realized there was several places in code where converting is required too and pushed a new commit. And also another one to fix Rubocop warning.

@Xanders Xanders requested a review from kyrylo January 15, 2021 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants