Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST,BUG: Use fork context to fix MacOS savez test #22221

Merged
merged 1 commit into from Sep 7, 2022

Conversation

charris
Copy link
Member

@charris charris commented Sep 7, 2022

Backport of #22204.

Why the test failed

  • Since Python 3.8, the default start method for multiprocessing has been changed from fork to spawn on macOS
  • The default start method is still fork on other Unix platforms[1], causing inconsistency on memory sharing model
  • It will cause a memory-sharing problem for the test test_large_zip on macOS as the memory sharing model between spawn and fork is different

The fix

  • Change the start method for this test back to fork under this testcase context
  • In this test case context, the bug that caused default start method changed to spawn for macOS will not be triggered
  • It is context limited, so this change will not affect default start method other than test_large_zip
  • All platforms have the same memory sharing model now
  • After the change, test_large_zip is passed on macOS
  1. https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

Closes gh-22203.

Since Python 3.8, the default start method for multiprocessing has been changed from fork to spawn on macOS
The default start method is still fork on other Unix platforms[1], causing inconsistency on memory sharing model
It will cause a memory-sharing problem for the test test_large_zip on macOS as the memory sharing model between spawn and fork is different
The fix

Change the start method for this test back to fork under this testcase context
In this test case context, the bug that caused default start method changed to spawn for macOS will not be triggered
It is context limited, so this change will not affect default start method other than test_large_zip
All platforms have the same memory sharing model now
After the change, test_large_zip is passed on macOS
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

Closes numpygh-22203
@charris charris added 00 - Bug 05 - Testing 08 - Backport Used to tag backport PRs labels Sep 7, 2022
@charris charris added this to the 1.23.3 release milestone Sep 7, 2022
@charris charris merged commit 63ab75d into numpy:maintenance/1.23.x Sep 7, 2022
@charris charris deleted the backport-22204 branch September 7, 2022 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants