Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doctest: Add +NUMBER option to ignore irrelevant floating-point differences #5576

Merged
merged 5 commits into from
Jul 11, 2019

Commits on Jul 8, 2019

  1. doctest: Add +NUMBER option to ignore irrelevant floating-point diffe…

    …rences
    
    When enabled, floating-point numbers only need to match as far as the
    precision you have written in the expected doctest output. This avoids
    false positives caused by limited floating-point precision, like this:
    
        Expected:
            0.233
        Got:
            0.23300000000000001
    
    This is inspired by Sébastien Boisgérault's [numtest] but the
    implementation is a bit different:
    
    * This implementation edits the literals that are in the "got"
      string (the actual output from the expression being tested), and then
      proceeds to compare the strings literally. This is similar to pytest's
      existing ALLOW_UNICODE and ALLOW_BYTES implementation.
    
    * This implementation only compares floats against floats, not ints
      against floats. That is, the following doctest will fail with pytest
      whereas it would pass with numtest:
    
          >>> math.py  # doctest: +NUMBER
          3
    
      This behaviour should be less surprising (less false negatives) when
      you enable NUMBER globally in pytest.ini.
    
    Advantages of this implementation compared to numtest:
    
    * Doesn't require `import numtest` at the top level of the file.
    * Works with pytest (if you try to use pytest & numtest together, pytest
      raises "TypeError: unbound method check_output() must be called with
      NumTestOutputChecker instance as first argument (got
      LiteralsOutputChecker instance instead)").
    * Works with Python 3.
    
    [numtest]: https://github.com/boisgera/numtest
    drothlis committed Jul 8, 2019
    Configuration menu
    Copy the full SHA
    aaa7e83 View commit details
    Browse the repository at this point in the history
  2. docs: Tidy up doctest options section

    * Move the parts about "how to configure it" (pytest.ini vs. inline
      comment) together.
    * Move `--doctest-continue-on-failure` into its own sub-heading, as it
      isn't related to the doctest optionflags.
    drothlis committed Jul 8, 2019
    Configuration menu
    Copy the full SHA
    2a23fda View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    d5cc0f2 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2019

  1. Fix test_doctest.test_number_non_matches

    These doctests were expected to fail, but they were failing because of a
    silly bug (I forgot to replace "{expression}" with the actual expression
    to be tested), not because of the thing they were meant to be testing.
    
    Then I had to fix one of the testcases because it was actually matching:
    
        >>> 3.0 #doctest: +NUMBER
        2.99
    
    The doctest is saying that the actual output should match to 2 decimal
    places, i.e. within 0.01 -- which it is, so it passes. I changed the
    expected output to 2.98 and now it doesn't match (as we expect).
    drothlis committed Jul 11, 2019
    Configuration menu
    Copy the full SHA
    4c590e0 View commit details
    Browse the repository at this point in the history
  2. docs: Document doctest +NUMBER limitation with strings

    Also added an "xfail" testcase for the same.
    drothlis committed Jul 11, 2019
    Configuration menu
    Copy the full SHA
    a740ef2 View commit details
    Browse the repository at this point in the history