Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem with loading Minitest simplecov plugin #877

Closed
edariedl opened this issue Feb 25, 2020 · 3 comments · Fixed by #878
Closed

Problem with loading Minitest simplecov plugin #877

edariedl opened this issue Feb 25, 2020 · 3 comments · Fixed by #878
Labels
Bug

Comments

@edariedl
Copy link

Hi,

I've just updated simplecov to the latest version 0.18.4 in our Rails app (Rails 6.0.2.1 with Minitest 5.14.0, Ruby 2.6.5). We are not loading simplcov every time, but we use the ENV variable to turn it on.

Gemfile configuration:

group :test do
  gem 'simplecov', require: false
end

At the top of our test_helper.rb:

if ENV["COVERAGE"]
  require 'simplecov'
  SimpleCov.coverage_dir(File.join("..", "..", "..", ENV['CIRCLE_ARTIFACTS'], "coverage")) if ENV['CIRCLE_ARTIFACTS']
  SimpleCov.start 'rails'
end

When I run the tests without COVERAGE env variable I get following error:

→ rails test
Running via Spring preloader in process 44891
Traceback (most recent call last):
	14: from -e:1:in `<main>'
	13: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/site_ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	12: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/site_ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	11: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/spring-2.1.0/lib/spring/application/boot.rb:19:in `<top (required)>'
	10: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/spring-2.1.0/lib/spring/application.rb:139:in `run'
	 9: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/spring-2.1.0/lib/spring/application.rb:139:in `loop'
	 8: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/spring-2.1.0/lib/spring/application.rb:145:in `block in run'
	 7: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/spring-2.1.0/lib/spring/application.rb:175:in `serve'
	 6: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/spring-2.1.0/lib/spring/application.rb:175:in `fork'
	 5: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/minitest-5.14.0/lib/minitest.rb:68:in `block in autorun'
	 4: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/minitest-5.14.0/lib/minitest.rb:135:in `run'
	 3: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/minitest-5.14.0/lib/minitest.rb:84:in `init_plugins'
	 2: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/minitest-5.14.0/lib/minitest.rb:84:in `each'
	 1: from /Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/minitest-5.14.0/lib/minitest.rb:86:in `block in init_plugins'
/Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/simplecov-0.18.4/lib/minitest/simplecov_plugin.rb:7:in `plugin_simplecov_init': uninitialized constant Minitest::SimpleCov (NameError)
Did you mean?  SimpleDelegator

I looked into it a bit, and unfortunately Minitest is loading plugins via Gem.find_files("minitest/*_plugin.rb") and it loads plugin files for every gem in the Gemfile even if it is not required in the projetct at a time.

I tried to test it in the rails console. First I made sure that simplecov is not required and then I tried to run the Gem.find_files("minitest/*_plugin.rb"). As you can see minitest/simplecov_plugin.rb was found anyway:

irb(main):002:0> defined?(SimpleCov)
=> nil
irb(main):003:0> Gem.find_files("minitest/*_plugin.rb").each do |plugin_path|
irb(main):004:1*   puts plugin_path
irb(main):005:1> end; nil
/Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/simplecov-0.18.4/lib/minitest/simplecov_plugin.rb
/Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/railties-6.0.2.1/lib/minitest/rails_plugin.rb
/Users/edariedl/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/minitest-5.14.0/lib/minitest/pride_plugin.rb
=> nil

Thanks!
@PragTob PragTob added the Bug label Feb 25, 2020
@PragTob
Copy link
Collaborator

PragTob commented Feb 25, 2020

That behavior from minitest is... interesting. Thanks for the report!

I see, I'll make it x days in a row with a bugfix release I guess.

I guess wrapping the code we want to trigger in if defined?(SimpleCov) should solve this, right?

And the scenario is a project with minitest running tests without requiring SimpleCov. A fun scenario to include in the test suite for SimpleCov 😁

@edariedl
Copy link
Author

Yeah, it is pretty fun behaviour from minitest :)

I would hope that wrapping the code in if defined?(SimpleCov) could solve it.

@juandg
Copy link

juandg commented Feb 25, 2020

I'm experiencing the same issue with minitest on the latest version 0.18.4

PragTob added a commit that referenced this issue Feb 25, 2020
@PragTob
Only run our minitest plugin code if SimpleCov is indeed defined
6a4a0e8 
* Fixes #877
* not quire sure whether we should wrap the entire thing in
if undefined?(..) or not but since the file has all the right
structure otherwise I'm more worried about minitest being
weirded out that it doesn't find what it expects than I am
of letting it execute an empty method
* also yes the version workaround is maybe a bit weird, however
I wanted our test suite to reproduce the exact failure case
because otherwise it'd be to easy to just implement a
respond_to? check which would have everything passing but would
fail in real life.
@PragTob PragTob mentioned this issue Feb 25, 2020
Merged
PragTob added a commit that referenced this issue Feb 25, 2020
@PragTob
Only run our minitest plugin code if SimpleCov is indeed defined
06225b5 
* Fixes #877
* not quire sure whether we should wrap the entire thing in
if undefined?(..) or not but since the file has all the right
structure otherwise I'm more worried about minitest being
weirded out that it doesn't find what it expects than I am
of letting it execute an empty method
* also yes the version workaround is maybe a bit weird, however
I wanted our test suite to reproduce the exact failure case
because otherwise it'd be to easy to just implement a
respond_to? check which would have everything passing but would
fail in real life.
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Mar 8, 2020
devel/ruby-simplecov: update to 0.18.5
a6e79a3 
Update ruby-simplecov to 0.18.5.


0.18.5 (2020-02-25)
===================

Can you guess? Another bugfix release!

## Bugfixes
* minitest won't crash if SimpleCov isn't loaded - aka don't execute SimpleCov code in the minitest plugin if SimpleCov isn't loaded. Thanks to [@edariedl](https://github.com/edariedl) for the report of the peculiar problem in [#877](simplecov-ruby/simplecov#877).

0.18.4 (2020-02-24)
===================

Another small bugfix release 🙈 Fixes SimpleCov running with rspec-rails, which was broken due to our fixed minitest integration.

## Bugfixes
* SimpleCov will run again correctly when used with rspec-rails. The excellent bug report [#873](simplecov-ruby/simplecov#873) by [@odlp](https://github.com/odlp) perfectly details what went wrong. Thanks to [@adam12](https://github.com/adam12) for the fix [#874](simplecov-ruby/simplecov#874).


0.18.3 (2020-02-23)
===========

Small bugfix release. It's especially recommended to upgrade simplecov-html as well because of bugs in the 0.12.0 release.

## Bugfixes
* Fix a regression related to file encodings as special characters were missing. Furthermore we now respect the magic `# encoding: ...` comment and read files in the right encoding. Thanks ([@Tietew](https://github.com/Tietew)) - see [#866](simplecov-ruby/simplecov#866)
* Use `Minitest.after_run` hook to trigger post-run hooks if `Minitest` is present. See [#756](simplecov-ruby/simplecov#756) and [#855](simplecov-ruby/simplecov#855) thanks ([@adam12](https://github.com/adam12))

0.18.2 (2020-02-12)
===================

Small release just to allow you to use the new simplecov-html.

## Enhancements
* Relax simplecov-html requirement so that you're able to use [0.12.0](https://github.com/colszowka/simplecov-html/blob/master/CHANGELOG.md#0120-2020-02-12)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Only run our minitest plugin code if SimpleCov is indeed defined simplecov-ruby/simplecov
3 participants
@edariedl @juandg @PragTob