New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: scipy.stats.wilcoxon in 1.13 fails on 2D array with nan -- 1D works #20591
Comments
I'll fix this shortly. In the meantime, please use:
The problem is just with |
Should I expect this to have worse performance than 1.12 in some situations? This is what I'm seeing, tho haven't set up a formal test. 1.12 using method='auto' is about 35% faster than 1.13 using method='approx'. |
What is the shape of the input, and does it have NaNs? But yes - in specific situations, you may observe a decrease in speed. Between 1.12 and 1.13, However, currently when the input has NaNs and is >1D, the implementation loops over each slice individually (as it did before, even when there were no NaNs). This looping combined with the constant overhead on each slice seems to result in the performance regression you're seeing. However, you can probably get much better performance now than before if you use |
Typically 20-200 rows, by ~1500 columns. So, far from 10k elements per slice. Would you expect me to see a slight benefit staying with 1.13 and using Columns with NaNs, should they exist, are only present in a continuous span at the top of the index. Would I benefit from chunking the columns by NaN presence and calling wilcoxon twice? |
I went ahead and tried Nice job on this update! |
Great to hear : ) Hopefully it will get array API support before too long, too. |
Describe your issue.
A regression seemingly introduced in 1.13. A 2D array with NaNs in some columns ValueErrors out as soon as it reaches a column with a NaN. It must be truly 2D, i.e. A.shape[1] > 1 is True. It also only occurs when there are more than 50 rows, i.e. A.shape[0] > 50.
Passing just that column with the NaNs, i.e. passing it as 1D, operates as expected. For example,
stats.wilcoxon(A[:50,:]
orstats.wilcoxon(A[:,1])
after the code example below runs fine.Reproducing Code Example
Error message
SciPy/NumPy/Python version and system information
The text was updated successfully, but these errors were encountered: