Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python 3.12: Crashes on CPython debug build when tracing #368

Open
jamadden opened this issue Sep 1, 2023 · 1 comment
Open

Python 3.12: Crashes on CPython debug build when tracing #368

jamadden opened this issue Sep 1, 2023 · 1 comment
Labels
Help Wanted

Comments

@jamadden
Copy link
Contributor

jamadden commented Sep 1, 2023

There's apparently some additional state we need to be saving and restoring.

Output from the test that crashes:

test_trace_events_multiple_greenlets_switching (greenlet.tests.test_tracing.TestPythonTracing.test_trace_events_multiple_greenlets_switching) ... 
Assertion failed: (code->_co_instrumentation_version == tstate->interp->monitoring_version), function call_instrumentation_vector, file instrumentation.c, line 944.
Process 92564 stopped
* thread #2, queue = 'com.apple.main-thread', stop reason = hit program assert
    frame #4: 0x0000000100cb3ae4 Python`call_instrumentation_vector.cold.13 at instrumentation.c:944:5 [opt]
   941 	    assert(!_PyErr_Occurred(tstate));
   942 	    assert(args[0] == NULL);
   943 	    PyCodeObject *code = frame->f_code;
-> 944 	    assert(code->_co_instrumentation_version == tstate->interp->monitoring_version); // assert 25 == 26
   945 	    assert(is_version_up_to_date(code, tstate->interp));
   946 	    assert(instrumentation_cross_checks(tstate->interp, code));
   947 	    assert(args[1] == NULL);
Target 0: (Python) stopped.
(lldb) bt
* thread #2, queue = 'com.apple.main-thread', stop reason = hit program assert
    frame #0: 0x000000018ef0c764 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x000000018ef43c28 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x000000018ee51ae8 libsystem_c.dylib`abort + 180
    frame #3: 0x000000018ee50e44 libsystem_c.dylib`__assert_rtn + 272
  * frame #4: 0x0000000100cb3ae4 Python`call_instrumentation_vector.cold.13 at instrumentation.c:944:5 [opt]
    frame #5: 0x0000000100b9c384 Python`call_instrumentation_vector(tstate=0x0000000100fbd780, event=15, frame=<unavailable>, instr=<unavailable>, nargs=4, args=0x000000016fdf92e0) at instrumentation.c:944:5 [opt]
    frame #6: 0x0000000100b9c438 Python`_Py_call_instrumentation_2args(tstate=<unavailable>, event=<unavailable>, frame=<unavailable>, instr=<unavailable>, arg0=<unavailable>, arg1=<unavailable>) at instrumentation.c:1022:12 [opt]
    frame #7: 0x0000000100b524bc Python`_PyEval_EvalFrameDefault(tstate=<unavailable>, frame=0x00000001031d0020, throwflag=0) at bytecodes.c:2670:31 [opt]
    frame #8: 0x0000000100b41fcc Python`_PyEval_Vector [inlined] _PyEval_EvalFrame(tstate=0x0000000100fbd780, frame=<unavailable>, throwflag=0) at pycore_ceval.h:88:16 [opt]
    frame #9: 0x0000000100b41fa4 Python`_PyEval_Vector(tstate=0x0000000100fbd780, func=0x000000010318a510, locals=0x0000000000000000, args=0x0000000100f5fdf8, argcount=0, kwnames=0x0000000000000000) at ceval.c:1675:12 [opt]
    frame #10: 0x0000000100a3d680 Python`_PyFunction_Vectorcall(func=<unavailable>, stack=<unavailable>, nargsf=0, kwnames=<unavailable>) at call.c:0:45 [opt]
    frame #11: 0x0000000100a3d32c Python`_PyVectorcall_Call(tstate=<unavailable>, func=(Python`_PyFunction_Vectorcall at call.c:410), callable=0x000000010318a510, tuple=0x0000000100f5fde0, kwargs=<unavailable>) at call.c:271:16 [opt]
    frame #12: 0x0000000100a3d458 Python`_PyObject_Call(tstate=0x0000000100fbd780, callable=0x000000010318a510, args=0x0000000100f5fde0, kwargs=0x0000000000000000) at call.c:354:16 [opt]
    frame #13: 0x0000000100a3d560 Python`PyObject_Call(callable=<unavailable>, args=<unavailable>, kwargs=<unavailable>) at call.c:379:12 [opt]
    frame #14: 0x000000010160e8a4 _greenlet.cpython-312d-darwin.so`greenlet::UserGreenlet::inner_bootstrap(this=0x000000010318f5a0, origin_greenlet=0x000000016fdf9970, _run=0x000000016fdf98c0)>&, greenlet::refs::OwnedReference<_object, &greenlet::refs::NoOpChecker(void*)>&) at greenlet.cpp:1325:45
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Oct 29, 2023
py-greenlet: update to 3.0.1.
5853583 
Switch to wheel.mk.

3.0.1 (2023-10-25)
==================

- Fix a potential crash on Python 3.8 at interpreter shutdown time.
  This was a regression from earlier 3.0.x releases. Reported by Matt
  Wozniski in `issue 376 <https://github.com/python-greenlet/greenlet/issues/376>`_.



3.0.0 (2023-10-02)
==================

- No changes from 3.0rc3 aside from the version number.


3.0.0rc3 (2023-09-12)
=====================

- Fix an intermittent error during process termination on some
  platforms (GCC/Linux/libstdc++).


3.0.0rc2 (2023-09-09)
=====================

- Fix some potential bugs (assertion failures and memory leaks) in
  previously-untested error handling code. In some cases, this means
  that the process will execute a controlled ``abort()`` after severe
  trouble when previously the process might have continued for some
  time with a corrupt state. It is unlikely those errors occurred in
  practice.
- Fix some assertion errors and potential bugs with re-entrant
  switches.
- Fix a potential crash when certain compilers compile greenlet with
  high levels of optimization. The symptom would be that switching to
  a greenlet for the first time immediately crashes.
- Fix a potential crash when the callable object passed to the
  greenlet constructor (or set as the ``greenlet.run`` attribute) has
  a destructor attached to it that switches. Typically, triggering
  this issue would require an unlikely subclass of
  ``greenlet.greenlet``.
- Python 3.11+: Fix rare switching errors that could occur when a
  garbage collection was triggered during the middle of a switch, and
  Python-level code in ``__del__`` or weakref callbacks switched to a
  different greenlet and ultimately switched back to the original
  greenlet. This often manifested as a ``SystemError``: "switch
  returned NULL without an exception set."

For context on the fixes, see `gevent issue #1985
<https://github.com/gevent/gevent/issues/1985>`_.

3.0.0rc1 (2023-09-01)
=====================

- Windows wheels are linked statically to the C runtime in an effort
  to prevent import errors on systems without the correct C runtime
  installed. It's not clear if this will make the situation better or
  worse, so please share your experiences in `issue 346
  <https://github.com/python-greenlet/greenlet/issues/346>`_.

  Note that this only applies to the binary wheels found on PyPI.
  Building greenlet from source defaults to the shared library. Set
  the environment variable ``GREENLET_STATIC_RUNTIME=1`` at build time
  to change that.
- Build binary wheels for Python 3.12 on macOS.
- Fix compiling greenlet on a debug build of CPython 3.12. There is
  `one known issue
  <https://github.com/python-greenlet/greenlet/issues/368>`_ that
  leads to an interpreter crash on debug builds.
- Python 3.12: Fix walking the frame stack of suspended greenlets.
  Previously accessing ``glet.gr_frame.f_back`` would crash due to
  `changes in CPython's undocumented internal frame handling <https://github.com/python/cpython/commit/1e197e63e21f77b102ff2601a549dda4b6439455>`_.

Platforms
---------
- Now, greenlet *may* compile and work on Windows ARM64 using
  llvm-mingw, but this is untested and unsupported. See `PR
  <https://github.com/python-greenlet/greenlet/pull/224>`_ by Adrian
  Vladu.
- Now, greenlet *may* compile and work on LoongArch64 Linux systems,
  but this is untested and unsupported. See `PR 257
  <https://github.com/python-greenlet/greenlet/pull/257/files>`_ by merore.

Known Issues
------------

- There may be (very) subtle issues with tracing on Python 3.12, which
  has redesigned the entire tracing infrastructure.

3.0.0a1 (2023-06-21)
====================

- Build binary wheels for S390x Linux. See `PR 358
  <https://github.com/python-greenlet/greenlet/pull/358>`_ from Steven
  Silvester.
- Fix a rare crash on shutdown seen in uWSGI deployments. See `issue
  330 <https://github.com/python-greenlet/greenlet/issues/330>`_ and `PR 356
  <https://github.com/python-greenlet/greenlet/pull/356>`_ from Andrew
  Wason.
- Make the platform-specific low-level C/assembly snippets stop using
  the ``register`` storage class. Newer versions of standards remove
  this storage class, and it has been generally ignored by many
  compilers for some time. See `PR 347
  <https://github.com/python-greenlet/greenlet/pull/347>`_ from Khem
  Raj.
- Add initial support for Python 3.12. See `issue
  <https://github.com/python-greenlet/greenlet/issues/323>`_ and `PR
  <https://github.com/python-greenlet/greenlet/pull/327>`_; thanks go
  to (at least) Michael Droettboom, Andreas Motl, Thomas A Caswell,
  raphaelauv, Hugo van Kemenade, Mark Shannon, and Petr Viktorin.
- Remove support for end-of-life Python versions, including Python
  2.7, Python 3.5 and Python 3.6.
- Require a compiler that supports ``noinline`` directives. See
  `issue 271
  <https://github.com/python-greenlet/greenlet/issues/266>`_.
- Require a compiler that supports C++11.
@mgorny mgorny mentioned this issue Dec 8, 2023
Merged
@mgorny
Copy link

mgorny commented Mar 9, 2024

I'm sorry but what are the chances for this being fixed? I'd like to know whether we should wait more or look into removing greenlet from Gentoo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Help Wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mgorny @jamadden