Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix remaining resource_tracker-related test warning failures #1263

Merged
merged 7 commits into from Feb 7, 2022

Conversation

ogrisel
Copy link
Contributor

@ogrisel ogrisel commented Feb 7, 2022

Blindly trying something on the CI.

I cannot reproduce the race condition on my laptop unfortunately.

Furthermore it seems that we also observe this test failing on one of the macos builds with the multiprocessing backend although I am not 100% sure why.

@codecov
Copy link

codecov bot commented Feb 7, 2022

Codecov Report

Merging #1263 (bcd2aaf) into master (88fef12) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1263      +/-   ##
==========================================
- Coverage   88.40%   88.38%   -0.02%     
==========================================
  Files          47       47              
  Lines        7045     7052       +7     
==========================================
+ Hits         6228     6233       +5     
- Misses        817      819       +2     
Impacted Files Coverage Δ
joblib/_memmapping_reducer.py 96.62% <100.00%> (+0.02%) ⬆️
joblib/disk.py 92.06% <100.00%> (ø)
joblib/test/test_memmapping.py 99.24% <100.00%> (+<0.01%) ⬆️
joblib/parallel.py 96.41% <0.00%> (-0.56%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 88fef12...bcd2aaf. Read the comment docs.

@ogrisel ogrisel changed the title WIP experimenting with resource_tracker warning failures Fix remaining resource_tracker-related test warning failures Feb 7, 2022
@ogrisel
Copy link
Contributor Author

ogrisel commented Feb 7, 2022

I pushed another commit to see if CI is more stable now.

@ogrisel
Copy link
Contributor Author

ogrisel commented Feb 7, 2022

It seems to be stable. I added a entry in the changelog. If it's still green, I will merge to get a green CI on the master branch.

@ogrisel ogrisel merged commit 978bd90 into joblib:master Feb 7, 2022
@ogrisel ogrisel deleted the resource_tracker-warnings branch February 7, 2022 19:01
Copy link
Contributor

@tomMoral tomMoral left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! thx @ogrisel

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 22, 2022
Release 1.2.0
Fix a security issue where eval(pre_dispatch) could potentially run arbitrary code. Now only basic numerics are supported. joblib/joblib#1327
Make sure that joblib works even when multiprocessing is not available, for instance with Pyodide joblib/joblib#1256
Avoid unnecessary warnings when workers and main process delete the temporary memmap folder contents concurrently. joblib/joblib#1263
Fix memory alignment bug for pickles containing numpy arrays. This is especially important when loading the pickle with mmap_mode != None as the resulting numpy.memmap object would not be able to correct the misalignment without performing a memory copy. This bug would cause invalid computation and segmentation faults with native code that would directly access the underlying data buffer of a numpy array, for instance C/C++/Cython code compiled with older GCC versions or some old OpenBLAS written in platform specific assembly. joblib/joblib#1254
Vendor cloudpickle 2.2.0 which adds support for PyPy 3.8+.
Vendor loky 3.3.0 which fixes several bugs including:
robustly forcibly terminating worker processes in case of a crash (joblib/joblib#1269);
avoiding leaking worker processes in case of nested loky parallel calls;
reliability spawn the correct number of reusable workers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants