Minitest broken? after 0.18.3

[adam12]

I know this is getting tiresome, but after we implemented the fix in #874 and #875, it's broken again, at least with an app I'm using. The coverage is generated immediately, before the Minitest at_exit hook runs.

But it's obviously not completely broken since the spec's are passing, so we're in a Minitest mystery.

I'll try to look when I get some time; maybe it's a race condition on which library is loaded first? I haven't been able to reproduce it other than 0.18.3 is good, 0.18.4 is broken (in this app anyways).

ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-freebsd12.0]
minitest (5.14.0)
simplecov (0.18.5)

[PragTob]

😭

I mean... we have a functioning feature spec so it might be misconfiguration/race conditon and at least not entirely broken. Or so I think, trusting the tests.

Thanks for the report anyhow! :)

[adam12]

False alarm, kinda.

In this project, I'm using a non-released test helper for Minitest. It seems it does things differently vs TestTask from Rake.

# Rakefile
require_relative "lib/minitest/test_task"
Minitest::TestTask.create

Using the Rake TestTask or a Ruby test file directly still appears to function as expected.

Closing for now.

[PragTob]

@adam12 thank you!

I mean.. if that one is scheduled to be released/something people use then we should try working to fix it... if not with the highes tpriority but it does seem kinda important :)

[adam12]

I think there was talks of it being released but I haven't heard anything further. I was looking for something better to run my tests and had been experimenting with it (alongside minitest-sprint which might suffer the same issue).

[hainesr]

Just FYI, I've run into this issue as well. SimpleCov any later than 0.18.3 (when using the Minitest Rake test task - not the built in Rake test task) prints the coverage results before the test results and gets the coverage wrong. It looks like it only sees the results of one test file, but I'm not sure.

I have a pretty simple (non-rails) setup, which I think is correct: https://github.com/hainesr/rubyzip/tree/next-gen

It seems to work as expected when using the Rake supplied test task, but I want the additional functionality of the Minitest version.