Action View test:ujs fails due to lower version of webdrivers (2.4.0) is chosen

[yahonda]

CI for Action View test:ujs has been failing since https://buildkite.com/rails/rails/builds/81759#9b57dc4b-924d-4e1b-85c0-e65ff2de97b6

I have identified this failure is due to webdrivers (2.4.0) is chosen which was webdrivers (4.6.1).

Steps to reproduce

git clone https://github.com/rails/rails
cd rails
Edit Gemfile.lock as follows to use lower version (2.4.0) of webdrivers
bundle install
cd actionview
bundle exec rake test:ujs

• Manually edit Gemfile.lock as follows

git diff
diff --git a/Gemfile.lock b/Gemfile.lock
index db88c30d90..a754557996 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -505,7 +505,7 @@ GEM
       json (>= 1.8)
       nokogiri (~> 1.6)
       rexml (~> 3.2)
-    webdrivers (4.6.0)
+    webdrivers (2.4.0)
       nokogiri (~> 1.6)
       rubyzip (>= 1.3.0)
       selenium-webdriver (>= 3.0, < 4.0)
$

Expected behavior

It should work successfully as follows.

Time: 4 seconds, Total: 420, Passed: 420, Failed: 0

Actual behavior

It raises uninitialized constant Selenium

../ci/qunit-selenium-runner.rb:9:in `<main>': uninitialized constant Selenium (NameError)

System configuration

Rails version: main branch

Ruby version: ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]

Additional information

Compared Gemfile.lock output from the last successful build (81738) and the first failed one. Somehow webdrivers version is lowered from 4.6.1 to 2.4.0. I have not found which gem needs to lower webdrivers version yet.

• webdrivers 4.6.1 for the successful build
  https://buildkite.com/rails/rails/builds/81738#f80ff5b2-b7d6-4239-a583-7527fc4e0db2/219-967

webdrivers (4.6.1)

• webdrivers 2.4.0 for the failed build
  https://buildkite.com/rails/rails/builds/81759#9b57dc4b-924d-4e1b-85c0-e65ff2de97b6/227-975

webdrivers (2.4.0)

[yahonda]

It looks like webdrivers does not allow to install selenium-webdriver 4.0.0. Let me open a pull request to webdrivers to relax the version condition.

https://github.com/titusfortner/webdrivers/blob/7930cccfbf1bd91d5698637177a993080903dd09/webdrivers.gemspec#L41

1. git clone https://github.com/seleniumhq/selenium
   i.e. /home/yahonda/src/github.com/SeleniumHQ/selenium

2. Checkout selenium-4.0.0 tag

$ cd /path/to/selenium (i.e. /home/yahonda/src/github.com/SeleniumHQ/selenium)
$ git checkout selenium-4.0.0
HEAD is now at 3a21814679 [js] bump JS to 4.0.0

3. Edit Gemfile to use local selenium-webdriver

$ git diff
diff --git a/Gemfile b/Gemfile
index 5459c781d1..56f14341d4 100644
--- a/Gemfile
+++ b/Gemfile
@@ -12,7 +12,7 @@ gem "rake", ">= 11.1"
 gem "sprockets-rails", ">= 2.0.0"
 gem "propshaft", ">= 0.1.7"
 gem "capybara", ">= 3.26"
-gem "selenium-webdriver", ">= 4.0.0.alpha7"
+gem "selenium-webdriver", ">= 4.0.0.alpha7", path: '/home/yahonda/src/github.com/SeleniumHQ/selenium'

 gem "rack-cache", "~> 1.2"
 gem "stimulus-rails"
$

4. bundle install

$ bundle install
Warning: the running version of Bundler (2.2.22) is older than the version that created the lockfile (2.2.29). We suggest you to upgrade to the version that created the lockfile by running `gem install bundler:2.2.29`.
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies.........
Bundler could not find compatible versions for gem "selenium-webdriver":
  In Gemfile:
    selenium-webdriver (>= 4.0.0.alpha7)

    webdrivers was resolved to 4.6.0, which depends on
      selenium-webdriver (>= 3.0, < 4.0)

Could not find gem 'selenium-webdriver (>= 3.0, < 4.0)', which is required by gem 'webdrivers', in source at
`/home/yahonda/src/github.com/SeleniumHQ/selenium`.

Bundler could not find compatible versions for gem "webdrivers":
  In snapshot (Gemfile.lock):
    webdrivers (= 4.6.0)

  In Gemfile:
    webdrivers

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
$

5. If checking out git checkout selenium-4.0.0-rc-3 tag, bundle install works fine.

[yahonda]

Opened titusfortner/webdrivers#218