Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TST/CI: Set hypothesis deadline to None to avoid flaky failures #45910

Merged
merged 1 commit into from Feb 10, 2022
Merged

TST/CI: Set hypothesis deadline to None to avoid flaky failures #45910

merged 1 commit into from Feb 10, 2022

Conversation

mroeschke
Copy link
Member

Avoid these flaky hypothesis failures in our CI:

2022-02-09T18:42:22.5859159Z C:\Miniconda\envs\pandas-dev\lib\site-packages\hypothesis\core.py:886: Flaky
2022-02-09T18:42:22.5859944Z --------------------------------- Hypothesis ----------------------------------
2022-02-09T18:42:22.5860630Z Falsifying example: test_range_difference(
2022-02-09T18:42:22.5861315Z     start1=8, stop1=-20, step1=1, start2=8, stop2=-8, step2=11,
2022-02-09T18:42:22.5861919Z )
2022-02-09T18:42:22.5863006Z Unreliable test timings! On an initial run, this test took 2344.29ms, which exceeded the deadline of 500.00ms, but on a subsequent run it took 0.82 ms, which did not. If you expect this sort of variability in your test timings, consider turning deadlines off for this test by setting deadline=None.

Most of the time they go away and not sure if anyone is knowledgeable to know why they happen.

@mroeschke mroeschke added Testing pandas testing functions or related to the test suite CI Continuous Integration labels Feb 9, 2022
@mroeschke mroeschke added this to the 1.4.1 milestone Feb 9, 2022
jbrockmendel
jbrockmendel approved these changes Feb 10, 2022
View reviewed changes
Copy link
Member

@jbrockmendel jbrockmendel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jreback jreback merged commit 1536e45 into pandas-dev:main Feb 10, 2022
@jreback
Copy link
Contributor

jreback commented Feb 10, 2022

@meeseeksdev backport 1.4.x

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Feb 10, 2022
@mroeschke @meeseeksmachine
Backport PR pandas-dev#45910: TST/CI: Set hypothesis deadline to None…
b250d02 
… to avoid flaky failures
@meeseeksmachine meeseeksmachine mentioned this pull request Feb 10, 2022
Merged
@lumberbot-app
Copy link

lumberbot-app bot commented Feb 10, 2022

Something went wrong ... Please have a look at my logs.

@mroeschke mroeschke deleted the tst/hypothesis_timing branch February 10, 2022 17:21
jreback pushed a commit that referenced this pull request Feb 11, 2022
@meeseeksmachine @mroeschke
Backport PR #45910: TST/CI: Set hypothesis deadline to None to avoid …
deea056 
…flaky failures (#45919)

Co-authored-by: Matthew Roeschke <emailformattr@gmail.com>
phofl pushed a commit to phofl/pandas that referenced this pull request Feb 14, 2022
@mroeschke @phofl
TST/CI: Set hypothesis deadline to None to avoid flaky failures (pand…
0e72aad 
…as-dev#45910)
yehoshuadimarsky pushed a commit to yehoshuadimarsky/pandas that referenced this pull request Jul 13, 2022
@mroeschke @yehoshuadimarsky
TST/CI: Set hypothesis deadline to None to avoid flaky failures (pand…
62f399e 
…as-dev#45910)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jbrockmendel jbrockmendel jbrockmendel approved these changes

Assignees
No one assigned
Labels
CI Continuous Integration Testing pandas testing functions or related to the test suite
Projects
None yet
Milestone
1.4.1
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@mroeschke @jreback @jbrockmendel