Support and test PyPy 3.7

[altendky]

Scope and purpose

This adds PyPy 3.7 to the test matrix and addresses a false positive returnValue() DeprecationWarning when running in PyPy 3.7. This showed up originally in pytest-twisted tests.

https://twistedmatrix.com/pipermail/twisted-python/2021-February/065456.html

Contributor Checklist:

• The associated ticket in Trac is here: https://twistedmatrix.com/trac/ticket/10093
• I ran tox -e lint to format my patch to meet the Twisted Coding Standard
• I have created a newsfragment in src/twisted/newsfragments/ (see: News files)
• I have updated the automated tests.
• I have submitted the associated Trac ticket for review by adding the word review to the keywords field in Trac, and putting a link to this PR in the comment; it shows up in https://twisted.reviews/ now.

[altendky]

Reverted out testing with PyPy (3.6) on macOS and Windows due to pyca/cryptography#5696 and tox-dev/tox#1850.

[rodrigc]

You should keep pypy3 here, in addition to adding pypy-3.6 and pypy-3.7.

Then after this release, we can remove pypy3 and keep pypy-3.6+

[altendky]

It seems that pypy3 refers to 3.6. What do we need to keep it around for?

https://github.com/twisted/twisted/runs/1905976630?check_suite_focus=true

[rodrigc]

Yeah looks like specifying pypy3 in GitHub actions installs pypy3 compatible with Python 3.6.12.
So I think your change is fine, not worth fighting with GitHub about this.

[rodrigc]

This change is fine.

[rodrigc]

Looks like pypy 3.5, pypy 3.6, and pypy 3.7 behave differently.
Let's have this logic work on all three. Then after this release we can drop pypy 3.5

[altendky]

Current PyPy doesn't support 3.5. Do we need to add it into the mix? I'm not sure offhand if the action provides it but we can try.

[rodrigc]

Looks like https://github.com/actions/setup-python#available-versions-of-pypy
specifies the versions of Pypy. I would say going through hoops to get Pypy 3.5 in Twisted CI is not worth the effort at this point.

Since we we specified pypy3 before, and that was installing pypy3 compatible with Python 3.6.12, then I see no point in introducing Pypy 3.5 at this point in the release cycle, when we are moving to drop support for Python 3.5/Ppy 3.5 after the release.

[rodrigc]

Can you manually install the last Pypy 3.5 release from:
https://downloads.python.org/pypy/pypy3.5-v7.0.0-linux64.tar.bz2
https://downloads.python.org/pypy/pypy3.5-v7.0.0-osx64.tar.bz2

And manually run test_inlinecb.py in your environment with your patch with this version of pypy?

I just want to double check, just in case there is a remote possibility that someone out there is running Pypy 3.5

[altendky]

$ git rev-parse HEAD
a225560007f04235596992da561adcf686c118ab

$ git diff
diff --git a/src/twisted/logger/_buffer.py b/src/twisted/logger/_buffer.py
index f90f8f982..3334f97e9 100644
--- a/src/twisted/logger/_buffer.py
+++ b/src/twisted/logger/_buffer.py
@@ -7,7 +7,7 @@ Log observer that maintains a buffer.
 """
 
 from collections import deque
-from typing import Deque, Optional
+from typing import Optional
 
 from zope.interface import implementer
 

$ venvpp35/bin/python --version --version
Python 3.5.3 (928a4f70d3de7d17449456946154c5da6e600162, Feb 09 2019, 11:50:43)
[PyPy 7.0.0 with GCC 8.2.0]

$ venvpp35/bin/trial twisted.internet.test.test_inlinecb
twisted.internet.test.test_inlinecb
  CancellationTests
    test_asynchronousCancellation ...                                      [OK]
    test_cascadeCancellingOnCancel ...                                     [OK]
    test_errbackCancelledErrorOnCancel ...                                 [OK]
    test_errorToErrorTranslation ...                                       [OK]
    test_errorToSuccessTranslation ...                                     [OK]
  ForwardTraceBackTests
    test_forwardLotsOfTracebacks ...                                       [OK]
    test_forwardTracebacks ...                                             [OK]
  NonLocalExitTests
    test_returnValueNonLocalDeferred ...                                   [OK]
    test_returnValueNonLocalWarning ...                                    [OK]
  StopIterationReturnTests
    test_returnWithValue ...                                               [OK]

-------------------------------------------------------------------------------
Ran 10 tests in 0.135s

PASSED (successes=10)

[altendky]

Also full tox passed.

https://gist.github.com/altendky/8fab0cac9a0be55f2d836bcc018c6637

[rodrigc]

That's good enough for me.
No need change the CI to add Pypy 3.5. If someone encounters a problem, we can point them to this
log, and ask them to upgrade to Pypy 3.6 which is better tested/supported.

[rodrigc]

so is the logic basically like:

 if (implementation.name == "cpython" and version_info < (3, 7)) or
     (implementation.name == "pypy" and version_info > (3, 6):
                # The contextvars backport and our no-op shim add an extra frame.
                # PyPy as of 3.7 adds an extra frame.
                appCodeTrace = appCodeTrace.tb_next