Fix 3-9-maintenance CI failures #2360

pirj · 2020-06-29T20:33:47Z

CI builds for 3-9-maintenance branch fail.
Some failures are constant, like for earlier Rubies:

Warning: unrecognized cop BinaryOperatorParameterName found in /home/travis/build/rspec/rspec-rails/.rubocop.yml

or

ERROR:  While executing gem ... (Gem::InstallError)
    gem "bundler" cannot be uninstalled because it is a default gem

or

Bundler could not find compatible versions for gem "cucumber":

to a complete nonsense in newer Rubies

NameError:
  uninitialized constant ActionView::TestCase::Behavior

Failures:

  1) RSpec::Rails::ViewExampleGroup#view is accessible to hooks
     Failure/Error: expect(run_count).to eq 1

       expected: 1
            got: 0

Example builds: [1, 2].

BONUS: forward-port select fixes to master/4-0-maintenance

JonRowe · 2020-06-29T20:46:07Z

Its fine to bump Rubies to the latest versions permanently if you want

pirj · 2020-06-29T22:32:12Z

I'm getting this when installing 2.2.10 with rvm, and remains in that condition, same error when running gem install bundler etc. I'll check other failures meanwhile.

/Users/pirj/.rvm/src/rubygems-3.0.8/lib/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- openssl (LoadError)

.travis.yml

pirj · 2020-07-16T09:14:57Z

~~RVM installs 2.3.8 with no issues.~~

For 2.2.10 and 2.3.8 and earlier, found here:

brew install rbenv/tap/openssl@1.0
PKG_CONFIG_PATH=/usr/local/Cellar/openssl@1.0/1.0.2t/lib/pkgconfig rvm install 2.2.10 --with-openssl-dir=/usr/local/Cellar/openssl@1.0/1.0.2t --with-openssl-lib=/usr/local/Cellar/openssl@1.0/1.0.2t/lib --with-openssl-include=/usr/local/Cellar/openssl@1.0/1.0.2t/include --rubygems ignore

JonRowe · 2020-07-16T09:25:15Z

Yeah I had to use rvm's custom install for open ssl which installs to a special folder then use that

pirj · 2020-07-19T12:52:00Z

Confused by this example:

      it 'is accessible to hooks', pending: pending_only_on_ruby_22_rails_52 do
        with_isolated_config do
          run_count = 0
          RSpec.configuration.before(:each, type: :view) do
            allow(view).to receive(:render) { :value }
            run_count += 1
          end
          group = RSpec::Core::ExampleGroup.describe 'a view', type: :view do
            specify { expect(true).to eq true }
          end
          group.run(failure_reporter)
          expect(failure_reporter.exceptions).to eq []
          expect(run_count).to eq 1
        end
      end

Firstly, this spec passes as a whole, but if you focus on this example, it fails on all Ruby versions.
There's a pending: on `master. Wondering how this focused failure might be related to:

        pending_only_on_ruby_22_rails_52 = """
          Rails 5.2.4.2 has a syntax error in ActionDispatch::Request::Session.
          (A &. usage which does not work in 2.2.10)

Also fails on latest released 5.2.4.3.

pirj · 2020-07-19T13:11:03Z

Another confusion is (ok, solved) is related to our RAILS_VERSION env var. I assumed it works as ~> #{RAILS_VERSION}. But a RAILS_ENV=5.2 setting causes rails 5.2.0 to be installed.

It seems we're using rails/rails's git branch, so 5-2-stable should work. Except the following fails with in some cases:

  if RUBY_VERSION  >= "2.2"
    gem_list << 'puma' if version > '5-0-stable'
  end

  gem_list.each do |rails_gem|
    gem rails_gem, :git => "https://github.com/rails/rails.git", :branch => version
  end

Could not find gem 'puma' in https://github.com/rails/rails.git (at 5-2-stable@92fd1c2).
The source does not contain any versions of 'puma'

Hard to argue with that. And easy to fix.

I'm not sure if it's worth it to add a note on how to set RAILS_VERSION (there's nothing on master either), should be:

5.2 means 5.2.0, from RubyGems
'~> 5.2' means latest 5.2.x
5-2-stable, master mean a branch from git

pirj · 2020-07-19T13:14:28Z

The third confusion is this spec itself. I'm struggling to understand what does it test exactly, and I have no good idea.
In master we at least make an expectation that (failure_reporter.exceptions).to eq [], but it's not in 3-9-maintenance.

pirj · 2020-07-19T13:27:13Z

The fourth confusion is how it manages to work when it works, given:

      def add_hook_to_existing_matching_groups(meta, scope, &block)
        if scope == :example || scope == :each || scope.nil?
          world.example_groups.each(&block)

So the trick is to add this hook to all known top-level-groups (wondering what the impact is for thousands of tl-groups and dozens of hooks).

So I thought that putting the group declaration before hook definition will work, but it doesn't, as it doesn't add a tl-group:

RSpec::Core::ExampleGroup.describe('a view') { it { } }
RSpec.configuration.world.example_groups # => []

I'm running it on Ruby 2.5.1 and Rails 5.2.4.3.

pirj · 2020-07-19T13:28:15Z

The fifth confusion is why it fails when focused, while passes just fine along with the whole spec.

pirj · 2020-07-19T13:48:06Z

It seems that those two live in two different worlds:

    2308:   if scope == :example || scope == :each || scope.nil?
    2309:     require 'pry'; binding.pry
 => 2310:     world.example_groups.each(&block)
    2311:   else
 [1] pry(#<RSpec::Core::Configuration>)> RSpec.world.example_groups
=> [RSpec::ExampleGroups::AView]
[2] pry(#<RSpec::Core::Configuration>)> world.example_groups
=> []

and RSpec::Core::ExampleGroup.describe('a view') { it { } } does actually add a top-level example group to RSpec.world.example_groups, but RSpec.configuration.before ignores that.

pirj · 2020-07-19T15:18:41Z

Ok, I now understand the spec in question more or less.
It tells that a view should be available in hooks.
But why specifically global config hooks, what makes them different from regular hooks?

After fixing the issue with different worlds I'm still able to reproduce the focused failure.

pirj · 2020-07-19T16:59:14Z

I guess the focusing issue is due to the example created with RSpec::Core::ExampleGroup.describe being filtered out from the spec run. No big deal if we don't run them one by one ourselves.

pirj

Some fixes could come handy in 4.0, leaving notes to forward-port them to master.

Dependency problems became apparent on CI when I've dropped the CI build cache for this PR and 3-9-maintenance branch. Bundler started querying anew just like it does locally.

I'm giving up on this stubborn view spec. Can't reproduce it locally (well, it fails when focused, but for another reason). It's stable red on CI. I intend to mute it for this specific Ruby and Rails combination that it fails for.

Apart from this, Rails ~~3.x~~ 3.0 and 3.1 builds are failing due to Bundler unable to resolve dependencies. I'm going to address this later in the day.

pirj · 2020-07-19T16:59:38Z

spec/support/helpers.rb

@@ -4,6 +4,7 @@ module Helpers
  def with_isolated_config
    original_config = RSpec.configuration
    RSpec.configuration = RSpec::Core::Configuration.new
+    RSpec.configuration.world = RSpec.world


TODO: forward-port me

Why it was needed?

Otherwise, the two worlds are out of sync inside with_isolated_config:

with_isolated_config do RSpec::Core::ExampleGroup.describe('a view') { it { } } RSpec.configuration.world.example_groups # => [] end

without with_isolated_config:

RSpec::Core::ExampleGroup.describe('a view') { it { } } RSpec.configuration.world.example_groups # => [<Example ...>]

I got rid of with_isolated_config in this example, so this change is to make with_isolated_config more future-proof.

If this doesn't fix a failing spec I'm going to say we should leave this as it was on this branch at least.

Removed. Let's see if it affects the build. At least it this change doesn't seem to affect view_example_group_spec.rb on 2.2.10/5.2.4.3.

spec/rspec/rails/example/view_example_group_spec.rb

Gemfile

Gemfile-rails-dependencies

pirj · 2020-07-19T19:25:01Z

script/update_rubygems_and_install_bundler

@@ -11,11 +11,11 @@ if is_ruby_23_plus; then
    yes | gem install bundler
  else
    echo "Warning installing older versions of Rubygems / Bundler"
-    gem update --system '2.7.8'
+    gem update --system '2.7.10'
    gem install bundler -v '1.17.3'


I didn't know those two are incapable of finding a matching gem version given a Ruby version constraint 😖

pirj · 2020-07-19T19:26:06Z

spec/support/helpers.rb

@@ -4,6 +4,7 @@ module Helpers
  def with_isolated_config
    original_config = RSpec.configuration
    RSpec.configuration = RSpec::Core::Configuration.new
+    RSpec.configuration.world = RSpec.world
    RSpec::Rails.initialize_configuration(RSpec.configuration)
    yield RSpec.configuration


Do we even need to yield it?
We use |config|, but we could equally use RSpec.configuration in those specs. At least I was confused, I thought that at some point config was out of sync with RSpec.configuration.

Rakefile

script/downgrade_bundler_on_old_rails

Could not find gem 'puma' in https://github.com/rails/rails.git (at 5-2-stable@92fd1c2). The source does not contain any versions of 'puma'

In this example, we're making sure that `view` is available in global config hooks. `group.run` accepts a reporter, but it defaults to a NullReporter anyway.

2.2 support has been dropped from 1.5.2 https://rubygems.org/gems/i18n/versions/1.5.2 https://rubygems.org/gems/i18n/versions/1.5.1

When installed from scratch on 2.2.10 with RubyGems 2.3.10 and Bundler 1.17.3, with RAILS_VERSION set to `5-2-stable`: childprocess-3.0.0 requires ruby version >= 2.3.0, which is incompatible with the current version, ruby 2.2.10p489 xpath-3.2.0 requires ruby version >= 2.3, which is incompatible with the current version, ruby 2.2.10p489 i18n-1.8.3 requires ruby version >= 2.3.0, which is incompatible with the current version, ruby 2.2.10p489 nio4r-2.5.2 requires ruby version >= 2.3, which is incompatible with the current version, ruby 2.2.10p489 rack-2.2.3 requires ruby version >= 2.3.0, which is incompatible with the current version, ruby 2.2.10p489 public_suffix-4.0.5 requires ruby version >= 2.3, which is incompatible with the current version, ruby 2.2.10p489 rubyzip-2.3.0 requires ruby version >= 2.4, which is incompatible with the current version, ruby 2.2.10p489 sprockets-4.0.2 requires ruby version >= 2.5.0, which is incompatible with the current version, ruby 2.2.10p489 We had sprockets capped, but only for `'~> x.x.x'`/`x.x.x` formats.

`set -e` stops the script when a *simple command* fails. If the command is part of an `||`-expression, the script continues. It seems unessential that we fail to uninstall `bundler` from the global gemset, the build is green after all.

actionpack was resolved to 3.2.22.5, which depends on sprockets (~> 2.2.1)

Warning: unrecognized cop BinaryOperatorParameterName found in /home/travis/build/rspec/rspec-rails/.rubocop_rspec_base.yml

RucoCop on CI is fixed at 0.23, and has outdated cop names. Even without those disabling, `rubocop lib` passes just fine.

Gem::RuntimeRequirementNotMetError: public_suffix requires Ruby version >= 1.9.3. The current ruby version is 1.8. An error occurred while installing public_suffix (1.4.2), and Bundler cannot continue. Gem::RuntimeRequirementNotMetError: redcarpet requires Ruby version >= 1.9.2. The current ruby version is 1.8. An error occurred while installing redcarpet (3.1.1), and Bundler cannot continue. Gem::RuntimeRequirementNotMetError: rubyzip requires Ruby version >= 1.9.2. The current ruby version is 1.8. An error occurred while installing rubyzip (1.0.0), and Bundler cannot continue.

Update to the latest released Rails 5.2.4.3 didn't help, it still has the bug. Backported from https://github.com/rspec/rspec-rails/blob/c0a8c22186a1232e3b1612c44bc88052ec56c945/spec/rspec/rails/example/view_example_group_spec.rb#L239

For the reasons described in the comment

bundle exec rails new ./tmp/example_app --no-rc --skip-javascript --skip-bootsnap --skip-sprockets --skip-git --skip-test-unit --skip-listen --skip-bundle --template=example_app_generator/generate_app.rb Gem::RuntimeRequirementNotMetError: spring requires Ruby version >= 2.4.0. The current ruby version is 2.3.0. An error occurred while installing spring (2.1.0), and Bundler cannot continue.

ffi has strict Ruby constraints aruba: Bundler could not find compatible versions for gem "thor": In Gemfile: aruba (~> 0.14.11) was resolved to 0.14.12, which depends on thor (~> 0.19) rails (~> 3.1.12) was resolved to 3.1.12, which depends on railties (= 3.1.12) was resolved to 3.1.12, which depends on thor (~> 0.14.6)

https://github.com/rspec/rspec-rails/pull/2140/files#diff-73c25cf975ccb4c89d8659202506c353L58 bumped aruba from 0.5.4 straight to 0.14.12. Using the version that used to work for older rails.

There's a minor difference: Widget management User creates a new widget -1 example, 0 failures, 1 pending +1 example, 0 failures most probably caused by 5-2-stable vs 5.2.4.x spec generation difference.

pirj · 2020-07-27T23:12:46Z

Ruby 1.9.2 segfaults time to time. Usually

Fetching gem metadata from https://rubygems.org/../home/travis/.rvm/rubies/ruby-1.9.2-p330/lib/ruby/1.9.1/pathname.rb:300: [BUG] /home/travis/build/rspec/rspec-rails/script/travis_functions.sh: line 13:  7862 Segmentation fault      (core dumped) RUBYOPT=--disable=gem:' --enable rubygems' bundle install --gemfile ./Gemfile --path /home/travis/build/rspec/rspec-rails/../bundle --retry=3 --jobs=3

rake aborted!

Fetching gem metadata from https://rubygems.org/..../home/travis/.rvm/gems/ruby-1.9.2-p330/gems/bundler-1.17.3/lib/bundler/compact_index_client.rb:56: [BUG] Segmentation fault

ruby 1.9.2p330 (2014-08-07 revision 47094) [x86_64-linux]

Restarting usually helps. Maybe add it to allowed failures as well?
Otherwise all green.

JonRowe · 2020-07-28T10:21:09Z

Restarting usually helps. Maybe add it to allowed failures as well? Otherwise all green.

No because then we are not notified of legitimate failures...

JonRowe

All good apart from Aruba, I'd like the comment to mention why a specific version is used if its used, but you can add that as a seperate commit with a [ci skip] if you'd like to avoid the rebuild time.

rspec-rails.gemspec

Co-authored-by: Jon Rowe <hello@jonrowe.co.uk>

pirj self-assigned this Jun 29, 2020

pirj changed the base branch from master to 3-9-maintenance June 29, 2020 20:34

pirj force-pushed the fix-3-9-maintenance-builds branch from 2c3d278 to 5d92bd6 Compare June 29, 2020 20:36

pirj force-pushed the fix-3-9-maintenance-builds branch 2 times, most recently from c093039 to 9f9132d Compare June 29, 2020 21:21

eloyesp reviewed Jun 30, 2020

View reviewed changes

.travis.yml Show resolved Hide resolved

This comment has been minimized.

Sign in to view

pirj force-pushed the fix-3-9-maintenance-builds branch 3 times, most recently from 449dc45 to 639ad76 Compare July 19, 2020 20:20

pirj commented Jul 20, 2020

View reviewed changes

pirj marked this pull request as ready for review July 20, 2020 07:28

pirj requested review from benoittgt and JonRowe July 20, 2020 07:37

pirj force-pushed the fix-3-9-maintenance-builds branch 4 times, most recently from 6916a29 to 839bce4 Compare July 21, 2020 20:11

pirj added 20 commits July 28, 2020 00:56

Bump Rubies

02858cb

Skip bootsnap like a no-AR app generator does

4fcdae4

Fix gem source mistake

c4ff60b

Could not find gem 'puma' in https://github.com/rails/rails.git (at 5-2-stable@92fd1c2). The source does not contain any versions of 'puma'

Provide historical info to avoid confusion

c7b8a6c

In this example, we're making sure that `view` is available in global config hooks. `group.run` accepts a reporter, but it defaults to a NullReporter anyway.

Make sure we use compatible i18n for old Rubies

6162f89

2.2 support has been dropped from 1.5.2 https://rubygems.org/gems/i18n/versions/1.5.2 https://rubygems.org/gems/i18n/versions/1.5.1

Update rubygems to a newer version

8075525

Add error handling to bundler downgrade

a53facb

`set -e` stops the script when a *simple command* fails. If the command is part of an `||`-expression, the script continues. It seems unessential that we fail to uninstall `bundler` from the global gemset, the build is green after all.

Fix sprockets dependency

a7dd9d6

actionpack was resolved to 3.2.22.5, which depends on sprockets (~> 2.2.1)

Drop invalid cop

a87877e

Warning: unrecognized cop BinaryOperatorParameterName found in /home/travis/build/rspec/rspec-rails/.rubocop_rspec_base.yml

Fix undefined method to_h for ENV

2b1e7fe

Mute RuboCop offence

102ba31

Remove cops unknown to CI RuboCop version

f5ebf79

RucoCop on CI is fixed at 0.23, and has outdated cop names. Even without those disabling, `rubocop lib` passes just fine.

Backport skipping of a spec due to Rails bug

89dd9ec

Update to the latest released Rails 5.2.4.3 didn't help, it still has the bug. Backported from https://github.com/rspec/rspec-rails/blob/c0a8c22186a1232e3b1612c44bc88052ec56c945/spec/rspec/rails/example/view_example_group_spec.rb#L239

Test a stable version of Rails 5.2 with Ruby 2.2

6a8fd07

For the reasons described in the comment

Use the previous aruba for older Rails

5781ea1

https://github.com/rspec/rspec-rails/pull/2140/files#diff-73c25cf975ccb4c89d8659202506c353L58 bumped aruba from 0.5.4 straight to 0.14.12. Using the version that used to work for older rails.

Add 5-2-stable Ruby 2.2 build to allowed failures

a8dd5e1

There's a minor difference: Widget management User creates a new widget -1 example, 0 failures, 1 pending +1 example, 0 failures most probably caused by 5-2-stable vs 5.2.4.x spec generation difference.

pirj force-pushed the fix-3-9-maintenance-builds branch from 9c1b28d to a8dd5e1 Compare July 27, 2020 21:56

JonRowe requested changes Jul 28, 2020

View reviewed changes

JonRowe reviewed Jul 28, 2020

View reviewed changes

rspec-rails.gemspec Outdated Show resolved Hide resolved

Clarify why a specific version of Aruba is used

1dcff41

Co-authored-by: Jon Rowe <hello@jonrowe.co.uk>

pirj requested a review from JonRowe July 28, 2020 13:54

JonRowe merged commit cc30ae4 into 3-9-maintenance Jul 28, 2020

JonRowe deleted the fix-3-9-maintenance-builds branch July 28, 2020 15:21

pirj mentioned this pull request Aug 14, 2020

Restrict certain gems on certain versions of ruby rspec/rspec-core#2750

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix 3-9-maintenance CI failures #2360

Fix 3-9-maintenance CI failures #2360

pirj commented Jun 29, 2020 •

edited

JonRowe commented Jun 29, 2020

pirj commented Jun 29, 2020

This comment has been minimized.

pirj commented Jul 16, 2020 •

edited

JonRowe commented Jul 16, 2020

pirj commented Jul 19, 2020 •

edited

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj left a comment •

edited

pirj Jul 19, 2020

benoittgt Jul 25, 2020

pirj Jul 26, 2020

JonRowe Jul 27, 2020

pirj Jul 27, 2020

pirj Jul 19, 2020

pirj Jul 19, 2020 •

edited

pirj commented Jul 27, 2020

JonRowe commented Jul 28, 2020

JonRowe left a comment

Fix 3-9-maintenance CI failures #2360

Fix 3-9-maintenance CI failures #2360

Conversation

pirj commented Jun 29, 2020 • edited

JonRowe commented Jun 29, 2020

pirj commented Jun 29, 2020

This comment has been minimized.

pirj commented Jul 16, 2020 • edited

JonRowe commented Jul 16, 2020

pirj commented Jul 19, 2020 • edited

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj commented Jul 19, 2020

pirj left a comment • edited

Choose a reason for hiding this comment

pirj Jul 19, 2020

Choose a reason for hiding this comment

benoittgt Jul 25, 2020

Choose a reason for hiding this comment

pirj Jul 26, 2020

Choose a reason for hiding this comment

JonRowe Jul 27, 2020

Choose a reason for hiding this comment

pirj Jul 27, 2020

Choose a reason for hiding this comment

pirj Jul 19, 2020

Choose a reason for hiding this comment

pirj Jul 19, 2020 • edited

Choose a reason for hiding this comment

pirj commented Jul 27, 2020

JonRowe commented Jul 28, 2020

JonRowe left a comment

Choose a reason for hiding this comment

pirj commented Jun 29, 2020 •

edited

pirj commented Jul 16, 2020 •

edited

pirj commented Jul 19, 2020 •

edited

pirj left a comment •

edited

pirj Jul 19, 2020 •

edited