Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: stats.monte_carlo_test: used biased estimate of p-value #16721

Merged
merged 1 commit into from Jul 28, 2022

Conversation

mdhaber
Copy link
Contributor

@mdhaber mdhaber commented Jul 28, 2022

Reference issue

gh-16459

What does this implement/fix?

The unbiased estimate of the p-value used in monte_carlo_test does not control the Type I error rate. This implements a biased estimate that does per [1].

import numpy as np
from scipy import stats
rng = np.random.default_rng(2190176673029737545)
x = np.zeros(100)
res = stats.monte_carlo_test(x, rng.random, np.mean,
                             vectorized=True, alternative='less')
assert res.pvalue == 0.0001

Additional information

image

@mdhaber mdhaber added scipy.stats maintenance Items related to regular maintenance tasks backport-candidate This fix should be ported by a maintainer to previous SciPy versions. labels Jul 28, 2022
@mdhaber mdhaber requested a review from tupui July 28, 2022 06:10
Copy link
Member

@tupui tupui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, thanks Matt. It would be good to get this for the release. I tested this locally and it works as expected.

@tupui tupui merged commit b0a7af7 into scipy:main Jul 28, 2022
@tupui tupui added this to the 1.9.0 milestone Jul 28, 2022
mdhaber added a commit to mdhaber/scipy that referenced this pull request Jul 28, 2022
tylerjereddy pushed a commit to tylerjereddy/scipy that referenced this pull request Jul 28, 2022
@tylerjereddy tylerjereddy removed the backport-candidate This fix should be ported by a maintainer to previous SciPy versions. label Jul 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Items related to regular maintenance tasks scipy.stats
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants