Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump Ruby to 3.3.1 #9597

Merged
merged 14 commits into from Apr 30, 2024
Merged
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
Expand Up @@ -23,7 +23,7 @@
"ghcr.io/devcontainers/features/github-cli": "latest",
"ghcr.io/devcontainers/features/node": "lts",
"ghcr.io/devcontainers/features/go": "latest",
"ghcr.io/devcontainers/features/ruby": "3.1.4",
"ghcr.io/devcontainers/features/ruby": "3.3.1",
"ghcr.io/devcontainers/features/rust": "latest",
"ghcr.io/devcontainers/features/dotnet": "latest",
"ghcr.io/devcontainers/features/sshd:1": {
Expand Down
2 changes: 2 additions & 0 deletions .rubocop.yml
Expand Up @@ -12,6 +12,8 @@ AllCops:
- "*/spec/fixtures/**/*"
- "vendor/**/*"
- "dry-run/**/*"
- "bundler/helpers/v1/patched_bundler"
- "bundler/helpers/spec_helpers/*"
NewCops: enable
TargetRubyVersion: 3.1
SuggestExtensions: false
Expand Down
2 changes: 1 addition & 1 deletion .ruby-version
@@ -1 +1 @@
3.1.4
3.3.1
5 changes: 4 additions & 1 deletion Dockerfile.updater-core
Expand Up @@ -54,7 +54,7 @@ COPY --chown=dependabot:dependabot LICENSE $DEPENDABOT_HOME

# Install Ruby from official Docker image
# When bumping Ruby minor, need to also add the previous version to `bundler/helpers/v{1,2}/monkey_patches/definition_ruby_version_patch.rb`
COPY --from=docker.io/library/ruby:3.1.4-bookworm --chown=dependabot:dependabot /usr/local /usr/local
COPY --from=docker.io/library/ruby:3.3.1-bookworm --chown=dependabot:dependabot /usr/local /usr/local
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jurre I think you also need to add some versions to a Bundler monkeypatch as explained in the comment above?


# We had to explicitly bump this as the bundled version `0.2.2` in ubuntu 22.04 has a bug.
# Once Ubuntu base image pulls in a new enough yaml version, we may not need to
Expand Down Expand Up @@ -110,6 +110,9 @@ RUN for ecosystem in git_submodules terraform github_actions hex elm docker nuge

WORKDIR $DEPENDABOT_HOME/dependabot-updater

ARG RUBYGEMS_VERSION=3.5.9
RUN gem update --system $RUBYGEMS_VERSION

# When bumping Bundler, need to also:
# * Regenerate `updater/Gemfile.lock` via `BUNDLE_GEMFILE=updater/Gemfile bundle lock --update --bundler`
# * Regenerate `Gemfile.lock` via `bundle lock --update --bundler`.
Expand Down
239 changes: 239 additions & 0 deletions bundler/helpers/spec_helpers/gem_net_http_adapter.rb
@@ -0,0 +1,239 @@
# typed: false
# frozen_string_literal: true

require "rubygems/vendored_net_http"

module WebMock
module HttpLibAdapters
class GemNetHttpAdapter < HttpLibAdapter
adapter_for :gem_net_http

OriginalGemNetHTTP = ::Gem::Net::HTTP unless const_defined?(:OriginalGemNetHTTP)

def self.enable!
::Gem::Net.send(:remove_const, :HTTP)
::Gem::Net.send(:remove_const, :HTTPSession)
::Gem::Net.send(:const_set, :HTTP, @webMockNetHTTP)
::Gem::Net.send(:const_set, :HTTPSession, @webMockNetHTTP)
end

def self.disable!
::Gem::Net.send(:remove_const, :HTTP)
::Gem::Net.send(:remove_const, :HTTPSession)
::Gem::Net.send(:const_set, :HTTP, OriginalGemNetHTTP)
::Gem::Net.send(:const_set, :HTTPSession, OriginalGemNetHTTP)

# copy all constants from @webMockNetHTTP to original Net::HTTP
# in case any constants were added to @webMockNetHTTP instead of Net::HTTP
# after WebMock was enabled.
# i.e Net::HTTP::DigestAuth
@webMockNetHTTP.constants.each do |constant|
unless OriginalGemNetHTTP.constants.map(&:to_s).include?(constant.to_s)
OriginalGemNetHTTP.send(:const_set, constant, @webMockNetHTTP.const_get(constant))
end
end
end

@webMockNetHTTP = Class.new(::Gem::Net::HTTP) do
class << self
def socket_type
StubSocket
end

if Module.method(:const_defined?).arity == 1
def const_defined?(name)
super || superclass.const_defined?(name)
end
else
def const_defined?(name, inherit = true)
super || superclass.const_defined?(name, inherit)
end
end

if Module.method(:const_get).arity != 1
def const_get(name, inherit = true)
super
rescue NameError
superclass.const_get(name, inherit)
end
end

if Module.method(:constants).arity != 0
def constants(inherit = true)
(super + superclass.constants(inherit)).uniq
end
end
end

def request(request, body = nil, &block)
request_signature = WebMock::NetHTTPUtility.request_signature_from_request(self, request, body)

WebMock::RequestRegistry.instance.requested_signatures.put(request_signature)

if webmock_response = WebMock::StubRegistry.instance.response_for_request(request_signature)
@socket = ::Gem::Net::HTTP.socket_type.new
WebMock::CallbackRegistry.invoke_callbacks(
{ lib: :net_http }, request_signature, webmock_response
)
build_net_http_response(webmock_response, request.uri, &block)
elsif WebMock.net_connect_allowed?(request_signature.uri)
check_right_http_connection
after_request = lambda do |response|
if WebMock::CallbackRegistry.any_callbacks?
webmock_response = build_webmock_response(response)
WebMock::CallbackRegistry.invoke_callbacks(
{ lib: :net_http, real_request: true }, request_signature, webmock_response
)
end
response.extend Net::WebMockHTTPResponse
yield response if block
response
end
super_with_after_request = lambda {
response = super(request, nil, &nil)
after_request.call(response)
}
if started?
ensure_actual_connection
super_with_after_request.call
else
start_with_connect do
super_with_after_request.call
end
end
else
raise WebMock::NetConnectNotAllowedError.new(request_signature)
end
end

def start_without_connect
raise IOError, "HTTP session already opened" if @started

if block_given?
begin
@socket = ::Gem::Net::HTTP.socket_type.new
@started = true
return yield(self)
ensure
do_finish
end
end
@socket = ::Gem::Net::HTTP.socket_type.new
@started = true
self
end

def ensure_actual_connection
return unless @socket.is_a?(StubSocket)

@socket&.close
@socket = nil
do_start
end

alias_method :start_with_connect, :start

def start(&block)
uri = Addressable::URI.parse(WebMock::NetHTTPUtility.get_uri(self))

if WebMock.net_http_connect_on_start?(uri)
super(&block)
else
start_without_connect(&block)
end
end

def build_net_http_response(webmock_response, request_uri)
response = ::Gem::Net::HTTPResponse.send(:response_class, webmock_response.status[0].to_s).new("1.0",
webmock_response.status[0].to_s, webmock_response.status[1])
body = webmock_response.body
body = nil if webmock_response.status[0].to_s == "204"

response.instance_variable_set(:@body, body)
webmock_response.headers.to_a.each do |name, values|
values = [values] unless values.is_a?(Array)
values.each do |value|
response.add_field(name, value)
end
end

response.instance_variable_set(:@read, true)

response.uri = request_uri

response.extend Net::WebMockHTTPResponse

raise Net::OpenTimeout, "execution expired" if webmock_response.should_timeout

webmock_response.raise_error_if_any

yield response if block_given?

response
end

def build_webmock_response(net_http_response)
webmock_response = WebMock::Response.new
webmock_response.status = [
net_http_response.code.to_i,
net_http_response.message
]
webmock_response.headers = net_http_response.to_hash
webmock_response.body = net_http_response.body
webmock_response
end

def check_right_http_connection
return if @@alredy_checked_for_right_http_connection ||= false

WebMock::NetHTTPUtility.puts_warning_for_right_http_if_needed
@@alredy_checked_for_right_http_connection = true
end
end
@webMockNetHTTP.version_1_2
[
[:Get, ::Gem::Net::HTTP::Get],
[:Post, ::Gem::Net::HTTP::Post],
[:Put, ::Gem::Net::HTTP::Put],
[:Delete, ::Gem::Net::HTTP::Delete],
[:Head, ::Gem::Net::HTTP::Head],
[:Options, ::Gem::Net::HTTP::Options]
].each do |c|
@webMockNetHTTP.const_set(c[0], c[1])
end
end
end

class StubSocket # :nodoc:
attr_accessor :read_timeout
attr_accessor :continue_timeout
attr_accessor :write_timeout

def initialize(*_args)
@closed = false
end

def closed?
@closed
end

def close
@closed = true
nil
end

def readuntil(*args); end

def io
@io ||= StubIO.new
end

class StubIO
def setsockopt(*args); end
def peer_cert; end
def peeraddr = ["AF_INET", 443, "127.0.0.1", "127.0.0.1"]
def ssl_version = "TLSv1.3"
def cipher = ["TLS_AES_128_GCM_SHA256", "TLSv1.3", 128, 128]
end
end
end
3 changes: 2 additions & 1 deletion bundler/helpers/v1/build
Expand Up @@ -14,6 +14,7 @@ else
"$helpers_dir/lib" \
"$helpers_dir/monkey_patches" \
"$helpers_dir/run.rb" \
"$helpers_dir/patched_bundler" \
"$install_dir"
fi

Expand All @@ -24,5 +25,5 @@ export GEM_HOME=$install_dir/.bundle
gem install bundler -v 1.17.3 --no-document

if [ -z "$DEPENDABOT_NATIVE_HELPERS_PATH" ]; then
BUNDLER_VERSION=1.17.3 bundle install
BUNDLER_VERSION=1.17.3 ./patched_bundler install
fi
Expand Up @@ -12,7 +12,7 @@ def index
Gem::Specification.new("ruby\0", requested_version)
end

%w(2.5.3p105 2.6.10p210 2.7.6p219 3.0.4p208).each do |version|
%w(2.5.3p105 2.6.10p210 2.7.6p219 3.0.4p208 3.1.4p223).each do |version|
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deivid-rodriguez I've included the current Ruby in this (Bundler v1) list. There's a more recent release (Ruby 3.1.5), but we're not running it yet. Does this list seem reasonable to you?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say you can add 3.1.5 and 3.2.4 too? I don't think the specific patch level is super important sice normally gems with set their requirements with something like required_ruby_version = ">= 2.6.0", so all patch levels should have the same result. But best to use latest.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if the patch part (p105 for 2.5.3, etc) is needed or not. I suspect it was some Bundler 1 thing that made that necessary so just in case I'd keep them. Unfortunately I don't know of a good way of figuring out the patchlevel other than installing each ruby and checking the value of RUBY_PATCHLEVEL.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've had this out for about four hours now without issues. I'm going to merge it. I'll have to do a followup PR to add the patch part 😄 Thanks for all your input 🙇

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've opened #9645 to add the patch levels 🙂

sources.metadata_source.specs << Gem::Specification.new("ruby\0", version)
end
end
Expand Down
2 changes: 1 addition & 1 deletion bundler/helpers/v1/monkey_patches/git_source_patch.rb
Expand Up @@ -15,7 +15,7 @@ class GitProxy
def configured_uri_for(uri)
uri = uri.gsub(%r{git@(.*?):/?}, 'https://\1/')
if uri.match?(/https?:/)
remote = URI(uri)
remote = ::URI.parse(uri)
config_auth =
Bundler.settings[remote.to_s] || Bundler.settings[remote.host]
remote.userinfo ||= config_auth
Expand Down
17 changes: 17 additions & 0 deletions bundler/helpers/v1/monkey_patches/object_untaint_patch.rb
@@ -0,0 +1,17 @@
# typed: false
# frozen_string_literal: true

# Bundler v1 uses the `untaint` method on objects in `Bundler::SharedHelpers`.
# This method has been deprecated for a long time, and is actually a no-op in
# ruby versions 2.7+. In Ruby 3.3 it was finally removed, and it's now causing
# bundler v1 to error.
#
# In order to keep the old behavior, we're monkey patching `Object` to add a
# no-op implementation of untaint.
module ObjectUntaintPatch
def untaint
self
end
end

Object.prepend(ObjectUntaintPatch)
34 changes: 34 additions & 0 deletions bundler/helpers/v1/patched_bundler
@@ -0,0 +1,34 @@
#!/usr/local/bin/ruby
#
# This file was generated by RubyGems.
# It was then patched by Dependabot to add `Object#untaint` back
# in order to run bundler 1.17.3 using Ruby 3.3+.
#
# The application 'bundler' is installed as part of a gem, and
# this file is here to facilitate running it.
#

$LOAD_PATH.unshift(File.expand_path("./monkey_patches", __dir__))
require "object_untaint_patch"

require 'rubygems'

version = ">= 0.a"

str = ARGV.first
if str
str = str.b[/\A_(.*)_\z/, 1]
if str and Gem::Version.correct?(str)
version = str
ENV['BUNDLER_VERSION'] = str

ARGV.shift
end
end

if Gem.respond_to?(:activate_bin_path)
load Gem.activate_bin_path('bundler', 'bundle', version)
else
gem "bundler", version
load Gem.bin_path("bundler", "bundle", version)
end
1 change: 1 addition & 0 deletions bundler/helpers/v1/run.rb
Expand Up @@ -19,6 +19,7 @@
require "fileutils_keyword_splat_patch"
require "git_source_patch"
require "resolver_spec_group_sane_eql"
require "object_untaint_patch"

require "functions"

Expand Down
3 changes: 3 additions & 0 deletions bundler/helpers/v1/spec/native_spec_helper.rb
Expand Up @@ -7,6 +7,7 @@

$LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
$LOAD_PATH.unshift(File.expand_path("../monkey_patches", __dir__))
$LOAD_PATH.unshift(File.expand_path("../../spec_helpers", __dir__))

# Bundler monkey patches
require "definition_ruby_version_patch"
Expand All @@ -17,6 +18,8 @@

require "functions"

require "gem_net_http_adapter"

RSpec.configure do |config|
config.color = true
config.order = :rand
Expand Down
Expand Up @@ -26,7 +26,7 @@ def source_requirements
Gem::Specification.new("Ruby\0", requested_version)
end

%w(2.5.3 2.6.10 2.7.7 3.0.5 3.2.1).each do |version|
%w(2.5.3 2.6.10 2.7.8 3.0.7 3.1.5 3.2.4).each do |version|
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@deivid-rodriguez We upgraded this list to include the latest Rubies. I think this is what we're supposed to do, but I'm not 100% confident that I understand the monkey patch. Can you give a 👍/👎 re: whether this lines up with the intent?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it seems correct to me, the list needs to complete the list of "supported rubies". Since Dependabot now uses Ruby 3.3, it seems correct 👍.

I think basically this is needed so that Dependabot also works even in situations where the application being updated is not compatible with the Ruby version Dependabot is using internally. I'd like to support this upstream, and we have an issue tracking it but not sure when/if it will happen.

sources.metadata_source.specs << Gem::Specification.new("Ruby\0", version)
end

Expand Down