check_regressors_train failure on master with the latest release on conda

[glemaitre]

We spot a failure of check_regressors_train (with X being an array or a memmap) on master.
The failure happened for:

• OMP
• Ransac

We should investigate this issue.

[thomasjpfan]

OrthogonalMatchingPursuit is also failing sometimes.

@adrinjalali Should we try to resolve this before the sprint?

[adrinjalali]

It'd be nice, since it may confuse people there.

[adrinjalali]

@glemaitre I can't really reproduce this, could you please give us maybe a reproducing code?

[glemaitre]

It is weird. It does not happen anymore, maybe an Heisenbug.

For instance, for this build:
https://github.com/scikit-learn/scikit-learn/runs/149613078

[jnothman]

It does not happen anymore, maybe an Heisenbug.

We should rather check the precise versions of packages being loaded from conda during and after the failure, to identify if some combination of versions results in the issue.

[glemaitre]

There is no diff in the versions used in the erroring build and the current build.

[glemaitre]

Uhm I checked the common test and I think that Ransac was tagged as poor_score estimator avoiding this test. It seems that we did not add the tag. We should probably do that. On second thought, I find weird that this is not failing all the time because I would expect the estimator to be deterministic when fixing the random state. Testing locally this test return always the same score which would suggest that. This is a bit mysterious to me why this is not a consistent failure.

[glemaitre]

Reopening since this is still happening time to time

[thomasjpfan]

Closing since this was fixed in #21781 and all related PRs. Also the memmap issue was fixed on the joblib side here: joblib/joblib#1254

[ogrisel]

Note that joblib has not be released yet at the time of writing but it should happen soon-ish.