Since ruby 2.7.0, byebug next command doesn't stop in many cases

[0x2c7]

Problem description

Since Ruby 2.7.0, next command doesn't stop in many cases, when calling some standard lib's methods.

Example:

require 'byebug'

def test
  GC.start # Or SecureRandom.uuid, or Base64.encode64, etc.
end
byebug
test
test

When running the above file, and type next, the program doesn't stop at the second method call. Instead, it continues til the end. When adding a tracepoint hook, I found out that in Ruby 2.7.0+ adds some weird events <internal:*>, such as <internal:gc>, <internal:pack>, etc.

  trace = TracePoint.new(:line, :call) do |point|
    puts "#{point.event}: #{point.path}:#{point.lineno}"
  end
  trace.enable
  SecureRandom.uuid
  trace.disable

The above sample code returns:

line: test9.rb:8
call: /home/.rbenv/versions/2.7.0/lib/ruby/2.7.0/securerandom.rb:252
...
line: /home/.rbenv/versions/2.7.0/lib/ruby/2.7.0/securerandom.rb:130
call: <internal:pack>:256 
line: <internal:pack>:257
line: /home/.rbenv/versions/2.7.0/lib/ruby/2.7.0/securerandom.rb:254
line: /home/.rbenv/versions/2.7.0/lib/ruby/2.7.0/securerandom.rb:255
line: /home/.rbenv/versions/2.7.0/lib/ruby/2.7.0/securerandom.rb:256
line: test9.rb:9

Similarly when running the test example with GC#start

line: test9.rb:8
call: <internal:gc>:33
line: <internal:gc>:34
line: test9.rb:9

When running again with methods in the same file that don't trigger <internal*> events, such as SecureRandom.uuid, next command works perfectly. So, I suspect the new <internal:*> event in tracepoint API breaks byebug. I'm debugging deep into C layer, but haven't found anything interesting yet.

[0x2c7]

Updated: after trying to debug, I found out that TracePoint API doesn't trigger return event after return from <internal:*>. In other cases, when openning a new control frame, there is always a pair of call and return events from Tracepoint. However, in the case of <internal:*>, the return event is missing. It makes Byebug fails to call RETURN_EVENT_SETUP correctly, and leads to dc->calced_stack_size is out-synced with the real stack size. Eventually, byebug fails to reach the next destination, and exits the program.

[0x2c7]

Update: Ruby trunk already has a fix for this issue. It is fixed in this commit: ruby/ruby@3e02cd5. However, the next ruby release in trunk is now 3.0.0. I'll create a ticket in Ruby issue to ask for a backport instead.

[0x2c7]

Update:

• I posted a backport request at https://bugs.ruby-lang.org/issues/17149
• The request is handled, and the fix is already merged into Ruby 2.7 maintainance branch (https://github.com/ruby/ruby/tree/ruby_2_7).
• I also fetch, and tested the latest build of Ruby 2.7, and it works quite well now.

Let's wait for Ruby 2.7.2 😄