Encoding::UndefinedConversionError #69

lcowell · 2015-03-19T20:39:02Z

When I paste "ñ" (with quotes) in to the console with pry-rails (0.3.3) enabled I will get an Encoding::UndefinedConversionError error.

$ bundle exec rails console
main:0>"��"
/Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/history.rb:106:in `write': "\xC3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/history.rb:106:in `puts'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/history.rb:106:in `save_to_file'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/history.rb:50:in `call'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/history.rb:50:in `push'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:267:in `handle_line'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:243:in `block (2 levels) in eval'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:242:in `catch'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:242:in `block in eval'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:241:in `catch'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:241:in `eval'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:77:in `block in repl'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:67:in `loop'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:67:in `repl'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:38:in `block in start'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/input_lock.rb:61:in `call'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/input_lock.rb:61:in `__with_ownership'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/input_lock.rb:79:in `with_ownership'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:38:in `start'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:15:in `start'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_class.rb:169:in `start'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/railties-3.2.21/lib/rails/commands/console.rb:47:in `start'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/railties-3.2.21/lib/rails/commands/console.rb:8:in `start'
    from /Users/lukec/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/railties-3.2.21/lib/rails/commands.rb:41:in `<top (required)>'
    from script/rails:6:in `require'

If I disable pry-rails in my Gemfile I get the expected output.

$ bundle exec rails console
Loading development environment (Rails 3.2.21)
irb(main):001:0> "ñ"
=> "ñ"

I'd love to get to the bottom of this, so please let me know if there's any additional information I can provide.

The text was updated successfully, but these errors were encountered:

momolog · 2015-04-21T09:00:17Z

Got this, too. Looks like a bug to me.

redbar0n · 2015-06-11T14:37:08Z

+1

› rails c
Loading development environment (Rails 4.1.10)
>> "ø"
=> "ø"
>> exit
/Users/Magne/.rvm/scripts/irbrc.rb:33:in `write': "\xC3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
    from /Users/Magne/.rvm/scripts/irbrc.rb:33:in `puts'
    from /Users/Magne/.rvm/scripts/irbrc.rb:33:in `block (2 levels) in <top (required)>'
    from /Users/Magne/.rvm/scripts/irbrc.rb:33:in `open'
    from /Users/Magne/.rvm/scripts/irbrc.rb:33:in `block in <top (required)>'
/Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb/ext/save-history.rb:115:in `write': "\xC3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb/ext/save-history.rb:115:in `puts'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb/ext/save-history.rb:115:in `block in save_history'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/open-uri.rb:36:in `open'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/open-uri.rb:36:in `open'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb/ext/save-history.rb:113:in `save_history'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb/ext/save-history.rb:79:in `block in extended'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb.rb:407:in `call'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb.rb:407:in `block in irb_at_exit'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb.rb:407:in `each'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb.rb:407:in `irb_at_exit'
    from /Users/Magne/.rvm/rubies/ruby-2.0.0-head/lib/ruby/2.0.0/irb.rb:400:in `start'
    from /Users/Magne/.rvm/gems/ruby-2.0.0-head@bloggery/gems/railties-4.1.10/lib/rails/commands/console.rb:90:in `start'
    from /Users/Magne/.rvm/gems/ruby-2.0.0-head@bloggery/gems/railties-4.1.10/lib/rails/commands/console.rb:9:in `start'
    from /Users/Magne/.rvm/gems/ruby-2.0.0-head@bloggery/gems/railties-4.1.10/lib/rails/commands/commands_tasks.rb:69:in `console'
    from /Users/Magne/.rvm/gems/ruby-2.0.0-head@bloggery/gems/railties-4.1.10/lib/rails/commands/commands_tasks.rb:40:in `run_command!'
    from /Users/Magne/.rvm/gems/ruby-2.0.0-head@bloggery/gems/railties-4.1.10/lib/rails/commands.rb:17:in `<top (required)>'
    from bin/rails:4:in `require'
    from bin/rails:4:in `<main>'

redbar0n · 2015-06-11T14:39:06Z

Some other guy also documented the problem: https://gist.github.com/atton-/5facfe29cf251ef9323f

pgolm · 2015-09-23T13:20:41Z

👍

rubyconvict · 2015-10-15T11:24:26Z

pry/pry#1429 (comment)

definitely an issue with pry-rails

rf- · 2015-10-15T19:03:13Z

Could someone try running the Rails console without pry-rails, opening Pry by typing binding.pry into that Rails console session, and then reproing?

rubyconvict · 2015-10-19T09:59:14Z

Sorry, pry-rails is innocent, it's pry vs. rb-readline war.

1.9.3-p429 :002 > 'Ą'
 => "Ą" 
1.9.3-p429 :003 > binding.pry
[1] pry(main)> '��'
Encoding::UndefinedConversionError: "\xC4" from ASCII-8BIT to UTF-8
    from /.../vendor/bundle/ruby/1.9.1/gems/pry-0.10.2/lib/pry/history.rb:106:in `write'
    from /.../vendor/bundle/ruby/1.9.1/gems/pry-0.10.2/lib/pry/history.rb:106:in `puts'
    from /.../vendor/bundle/ruby/1.9.1/gems/pry-0.10.2/lib/pry/history.rb:106:in `save_to_file'
    from /.../vendor/bundle/ruby/1.9.1/gems/pry-0.10.2/lib/pry/history.rb:50:in `call'
    from /.../vendor/bundle/ruby/1.9.1/gems/pry-0.10.2/lib/pry/history.rb:50:in `push'
    from /.../vendor/bundle/ruby/1.9.1/gems/pry-0.10.2/lib/pry/pry_instance.rb:267:in `handle_line'

yoonsy · 2015-11-14T03:44:31Z

+1

mgrachev · 2015-11-28T13:29:06Z

+1

milesbissky · 2016-01-05T01:24:09Z

+1

lingceng · 2016-01-15T07:07:18Z

Just remove the 'rb-readline' gem if you are on a linux like system.
This may be the current solution for this problem.

lcowell · 2016-01-15T18:27:05Z

Removing rb-readline solved the issue for me.

poc7667 · 2016-05-05T04:17:28Z

However, if you remove rb-readline, we won't be able to type non-ascii character on the console

jsnick · 2016-07-05T08:47:46Z

I've solved this.
refer to pry/pry#230420454

chunlea · 2017-02-13T06:41:45Z

http://altarf.net/computer/rails/3222

add ~/.pryrc with content

Pry.config.history.should_save = false

and it works for me.

rutgerg · 2017-08-24T12:09:00Z

+1

Envek · 2019-03-21T11:17:33Z

Had same error on Linux in Docker image. Problem was in system locale settings (it was set to POSIX).

Fixed by setting locale to Unicode-enabled one:

FROM ruby:2.6.2
RUN apt-get update && apt-get install -y locales && update-locale LANG=C.UTF-8 LC_ALL=C.UTF-8
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8

In your local system check output of locale command and add export LANG=C.UTF-8 LC_ALL=C.UTF-8 to your ~/.bashrc or ~/.zshrc if necessary

Overload119 · 2022-11-04T23:47:44Z

I was able to "fix" this by adding this line in:

/Users/<username>/.rvm/gems/ruby-2.7.5/gems/pry-0.13.1/lib/pry/history.rb

def save_to_file(line)
  history_file.puts line.encode('UTF-8', invalid: :replace, undef: :replace, replace: '?') if history_file
end

Of course the issue is still there, so I opened an issue in RbReadline (see ConnorAtherton/rb-readline#163)

rubyconvict mentioned this issue Oct 15, 2015

Assigning a string with UTF-8 characters (£ symbol) breaks pry pry/pry#1429

Closed

briankung mentioned this issue Apr 3, 2019

mails: suboptimal quoted-printable generation lobsters/lobsters#328

Closed

hackartisan mentioned this issue May 12, 2020

Remove rb-readline pulibrary/figgy#3893

Closed

Overload119 mentioned this issue Nov 4, 2022

Encoding::UndefinedConversionError for UTF-8 to ASCII8 ConnorAtherton/rb-readline#163

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Encoding::UndefinedConversionError #69

Encoding::UndefinedConversionError #69

lcowell commented Mar 19, 2015

momolog commented Apr 21, 2015

redbar0n commented Jun 11, 2015

redbar0n commented Jun 11, 2015

pgolm commented Sep 23, 2015

rubyconvict commented Oct 15, 2015

rf- commented Oct 15, 2015

rubyconvict commented Oct 19, 2015

yoonsy commented Nov 14, 2015

mgrachev commented Nov 28, 2015

milesbissky commented Jan 5, 2016

lingceng commented Jan 15, 2016

lcowell commented Jan 15, 2016

poc7667 commented May 5, 2016

jsnick commented Jul 5, 2016 •

edited

chunlea commented Feb 13, 2017

rutgerg commented Aug 24, 2017

Envek commented Mar 21, 2019

Overload119 commented Nov 4, 2022

Encoding::UndefinedConversionError #69

Encoding::UndefinedConversionError #69

Comments

lcowell commented Mar 19, 2015

momolog commented Apr 21, 2015

redbar0n commented Jun 11, 2015

redbar0n commented Jun 11, 2015

pgolm commented Sep 23, 2015

rubyconvict commented Oct 15, 2015

rf- commented Oct 15, 2015

rubyconvict commented Oct 19, 2015

yoonsy commented Nov 14, 2015

mgrachev commented Nov 28, 2015

milesbissky commented Jan 5, 2016

lingceng commented Jan 15, 2016

lcowell commented Jan 15, 2016

poc7667 commented May 5, 2016

jsnick commented Jul 5, 2016 • edited

chunlea commented Feb 13, 2017

rutgerg commented Aug 24, 2017

Envek commented Mar 21, 2019

Overload119 commented Nov 4, 2022

jsnick commented Jul 5, 2016 •

edited