Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

double free (fasttop) crashes on python 3.11 #10368

Open
rahulgharpinde opened this issue Feb 8, 2024 · 12 comments
Open

double free (fasttop) crashes on python 3.11 #10368

rahulgharpinde opened this issue Feb 8, 2024 · 12 comments

Comments

@rahulgharpinde
Copy link

rahulgharpinde commented Feb 8, 2024

Observe multiple crash on python 3.11 environment with cryptography module. We have a cluster environment on which multiple process and monitoring scripts are keep running in a interval. From traceback it is not clear which python code causing this issue intermittently. Need help to debug this issue in detail.

backtrace 1:

Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50        return ret;
[Current thread is 1 (Thread 0x7f251e3ec700 (LWP 2803598))]
Missing separate debuginfos, use: yum debuginfo-install libgcc-8.5.0-18.2.el8_8.x86_64 openssl-libs-1.1.1k-12.el8_8.x86_64
(gdb)
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f2534d41ea5 in __GI_abort () at abort.c:79
#2  0x00007f2534dafcd7 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f2534ea929b "%s\n")
    at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007f2534db6fdc in malloc_printerr (str=str@entry=0x7f2534eaafc0 "double free or corruption (fasttop)")
    at malloc.c:5449
#4  0x00007f2534db8cbd in _int_free (av=0x7f250c000020, p=0x7f250c005db0, have_lock=<optimized out>) at malloc.c:4343
#5  0x00007f2522236740 in init_thread_stop.part ()
   from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#6  0x00007f2522236763 in init_thread_destructor ()
   from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#7  0x00007f2535876451 in __nptl_deallocate_tsd () at pthread_create.c:301
#8  0x00007f25358771eb in __nptl_deallocate_tsd () at pthread_create.c:256
#9  start_thread (arg=<optimized out>) at pthread_create.c:490
#10 0x00007f2534d59e73 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)

backtrace 2:

Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50        return ret;
[Current thread is 1 (Thread 0x7f0438a4b740 (LWP 962187))]
Missing separate debuginfos, use: yum debuginfo-install libgcc-8.5.0-18.2.el8_8.x86_64 libstdc++-8.5.0-18.2.el8_8.x86_64 openssl-libs-1.1.1k-12.el8_8.x86_64
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f04372f5ea5 in __GI_abort () at abort.c:79
#2  0x00007f0437363cd7 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f043745d29b "%s\n")
    at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007f043736afdc in malloc_printerr (str=str@entry=0x7f043745efc0 "double free or corruption (fasttop)")
    at malloc.c:5449
#4  0x00007f043736ccbd in _int_free (av=0x7f03f0000020, p=0x7f03f0005e40, have_lock=<optimized out>) at malloc.c:4343
#5  0x00007f0424d0f850 in ossl_cleanup_thread ()
   from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#6  0x00007f0424d0ee48 in OPENSSL_cleanup ()
   from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#7  0x00007f043732526c in __run_exit_handlers (status=0, listp=0x7f0437693758 <__exit_funcs>,
    run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
#8  0x00007f04373253a0 in __GI_exit (status=<optimized out>) at exit.c:139
#9  0x00007f043730ed8c in __libc_start_main (main=0x560d0079d710 <main>, argc=5, argv=0x7ffc1f8d2bf8, init=
    0x560d0079d810 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc1f8d2be8)
    at ../csu/libc-start.c:336
#10 0x0000560d0079d74e in _start ()
(gdb) quit

backtrace 3:

Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50        return ret;
[Current thread is 1 (Thread 0x7f05f2ffd700 (LWP 3045101))]
Missing separate debuginfos, use: yum debuginfo-install libgcc-8.5.0-18.2.el8_8.x86_64 libstdc++-8.5.0-18.2.el8_8.x86_64 openssl-libs-1.1.1k-12.el8_8.x86_64
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f06226e1ea5 in __GI_abort () at abort.c:79
#2  0x00007f062274fcd7 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f062284929b "%s\n")
    at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007f0622756fdc in malloc_printerr (str=str@entry=0x7f062284afc0 "double free or corruption (fasttop)")
    at malloc.c:5449
#4  0x00007f0622758cbd in _int_free (av=0x7f05e4000020, p=0x7f05e4008db0, have_lock=<optimized out>) at malloc.c:4343
#5  0x00007f0623216451 in __nptl_deallocate_tsd () at pthread_create.c:301
#6  0x00007f06232171eb in __nptl_deallocate_tsd () at pthread_create.c:256
#7  start_thread (arg=<optimized out>) at pthread_create.c:490
#8  0x00007f06226f9e73 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) quit

Versions of Python, cryptography, pip, and setuptools
/opt/VRTSnas/python/venv/bin/pip -V
pip 23.2.1 from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/pip (python 3.11)

/opt/VRTSnas/python/venv/bin/pip show cryptography
Name: cryptography
Version: 41.0.7
Summary: cryptography is a package which provides cryptographic recipes and primitives to Python developers.
Home-page:
Author:
Author-email: The Python Cryptographic Authority and individual contributors cryptography-dev@python.org
License: Apache-2.0 OR BSD-3-Clause
Location: /opt/VRTSnas/python/venv/lib64/python3.11/site-packages
Requires: cffi
Required-by: paramiko, pyOpenSSL

/opt/VRTSnas/python/venv/bin/pip show setuptools
Name: setuptools
Version: 65.5.1
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Home-page: https://github.com/pypa/setuptools
Author: Python Packaging Authority
Author-email: distutils-sig@python.org
License:
Location: /opt/VRTSnas/python/venv/lib64/python3.11/site-packages
Requires:
Required-by:

OpenSSL 1.1.1k FIPS 25 Mar 2021
Red Hat Enterprise Linux release 8.8 (Ootpa)

How you installed cryptography :
pip install -r requirements.txt

Steps for reproducing your bug :
There are no exact steps to reproduce this issue. Facing this issue intermittently and also not getting exact trace from where this is getting dump.

@alex
Copy link
Member

alex commented Feb 8, 2024

Is this a multi-threaded program?

This, unfortunately (and frustratingly) looks like it may be a product of: openssl/openssl#17490

Where effectively the main thread runs its exit handles concurrently with OpenSSL usage by another thread.

@alex
Copy link
Member

alex commented Feb 8, 2024

openssl/openssl#17469 is a more involved description of the atexit issue

@alex
Copy link
Member

alex commented Feb 8, 2024

And python/cpython#114653 is a cpython thread with a simiilar crash/cause.

@rahulgharpinde
Copy link
Author

@alex
Is this a multi-threaded program? --> Yes, we use multi-threading to execute command on multiple node using paramiko.
From gdb info threads output, seems like it could be openssl/openssl#17469 issue.
Is there any workaround for this?

(gdb) info threads
  Id   Target Id                                     Frame
* 1    Thread 0x7f251e3ec700 (LWP 2803598) (Exiting) __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
  2    Thread 0x7f2536497740 (LWP 2803421)           0x00007f2522238bcd in OPENSSL_strcasecmp () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
  3    Thread 0x7f24fffff700 (LWP 2803603)           0x00007f2534e44251 in __GI___poll (fds=0x7f24ffffe9c0, nfds=1, timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
  4    Thread 0x7f251cbe9700 (LWP 2803602)           0x00007f2534e44251 in __GI___poll (fds=0x7f251cbe89c0, nfds=1, timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
(gdb)
(gdb) thread 1
[Switching to thread 1 (Thread 0x7f251e3ec700 (LWP 2803598))]
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50        return ret;
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f2534d41ea5 in __GI_abort () at abort.c:79
#2  0x00007f2534dafcd7 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7f2534ea929b "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007f2534db6fdc in malloc_printerr (str=str@entry=0x7f2534eaafc0 "double free or corruption (fasttop)") at malloc.c:5449
#4  0x00007f2534db8cbd in _int_free (av=0x7f250c000020, p=0x7f250c005db0, have_lock=<optimized out>) at malloc.c:4343
#5  0x00007f2522236740 in init_thread_stop.part () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#6  0x00007f2522236763 in init_thread_destructor () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#7  0x00007f2535876451 in __nptl_deallocate_tsd () at pthread_create.c:301
#8  0x00007f25358771eb in __nptl_deallocate_tsd () at pthread_create.c:256
#9  start_thread (arg=<optimized out>) at pthread_create.c:490
#10 0x00007f2534d59e73 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)
(gdb) thread 2
[Switching to thread 2 (Thread 0x7f2536497740 (LWP 2803421))]
#0  0x00007f2522238bcd in OPENSSL_strcasecmp () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
(gdb) bt
#0  0x00007f2522238bcd in OPENSSL_strcasecmp () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#1  0x00007f25222323af in OPENSSL_LH_delete () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#2  0x00007f2522250ddc in OBJ_NAME_remove () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#3  0x00007f25222325c4 in OPENSSL_LH_doall () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#4  0x00007f2522250fb7 in OBJ_NAME_cleanup () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#5  0x00007f252221f6e8 in evp_cleanup_int () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#6  0x00007f2522235e52 in OPENSSL_cleanup () from /opt/VRTSnas/python/venv/lib64/python3.11/site-packages/cryptography/hazmat/bindings/_rust.abi3.so
#7  0x00007f2534d7126c in __run_exit_handlers (status=0, listp=0x7f25350df758 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
#8  0x00007f2534d713a0 in __GI_exit (status=<optimized out>) at exit.c:139
#9  0x00007f2534d5ad8c in __libc_start_main (main=0x55aabf0b0710 <main>, argc=3, argv=0x7ffeb2381368, init=0x55aabf0b0810 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=0x7ffeb2381358) at ../csu/libc-start.c:336
#10 0x000055aabf0b074e in _start ()
(gdb) 
(gdb) thread 3
[Switching to thread 3 (Thread 0x7f24fffff700 (LWP 2803603))]
#0  0x00007f2534e44251 in __GI___poll (fds=0x7f24ffffe9c0, nfds=1, timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
29        return SYSCALL_CANCEL (poll, fds, nfds, timeout);
(gdb) bt
#0  0x00007f2534e44251 in __GI___poll (fds=0x7f24ffffe9c0, nfds=1, timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007f2526bb5d20 in poll (__timeout=100, __nfds=1, __fds=0x7f24ffffe9c0) at /usr/include/bits/poll2.h:38
#2  internal_select (writing=writing@entry=0, interval=<optimized out>, connect=connect@entry=0, s=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/socketmodule.c:779
#3  0x00007f2526bb5df7 in sock_call_ex (s=s@entry=0x7f2520455780, writing=writing@entry=0, sock_func=sock_func@entry=0x7f2526bb41b7 <sock_recv_impl>, data=data@entry=0x7f24ffffea70,
    connect=connect@entry=0, err=err@entry=0x0, timeout=100000000) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/socketmodule.c:874
#4  0x00007f2526bb638b in sock_call (data=0x7f24ffffea70, func=0x7f2526bb41b7 <sock_recv_impl>, writing=0, s=0x7f2520455780) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/socketmodule.c:3533
#5  sock_recv_guts (s=s@entry=0x7f2520455780, cbuf=<optimized out>, len=<optimized out>, flags=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/socketmodule.c:3533
#6  0x00007f2526bb655d in sock_recv (s=0x7f2520455780, args=args@entry=(16,)) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Include/cpython/bytesobject.h:37
#7  0x00007f2535ce3d8d in method_vectorcall_VARARGS (func=<optimized out>, args=0x7f2526eec3c8, nargsf=<optimized out>, kwnames=<optimized out>)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/descrobject.c:330
#8  0x00007f2535cc48c3 in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<method_descriptor at remote 0x7f2526abca90>, tstate=0x7f2514005030)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Include/internal/pycore_call.h:77
#9  PyObject_Vectorcall (callable=<method_descriptor at remote 0x7f2526abca90>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/call.c:299
#10 0x00007f2535c82442 in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Python/ceval.c:4772
#11 0x00007f2535c8ed2c in _PyEval_EvalFrame (throwflag=0, frame=0x7f2526eec020, tstate=0x7f2514005030) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Include/internal/pycore_ceval.h:70
#12 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, func=<optimized out>, tstate=<optimized out>)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Python/ceval.c:6435
#13 _PyFunction_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, stack=<optimized out>, func=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/call.c:393
#14 _PyObject_VectorcallTstate (tstate=0x7f2514005030, callable=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>, tstate=0x7f2514005030, callable=<optimized out>,
    args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Include/internal/pycore_call.h:92
#15 0x00007f2535d74a58 in method_vectorcall (method=<optimized out>, args=0x7f253621dc10 <_PyRuntime+58928>, nargsf=<optimized out>, kwnames=0x0)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/classobject.c:67
#16 0x00007f2535cb17bb in _PyVectorcall_Call (kwargs=<optimized out>, tuple=(), callable=<method at remote 0x7f2520479780>, func=0x7f2535d74940 <method_vectorcall>, tstate=0x7f2514005030)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/call.c:245
#17 _PyObject_Call (tstate=0x7f2514005030, callable=<method at remote 0x7f2520479780>, args=(), kwargs=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/call.c:328
#18 0x00007f2535d9e40c in thread_run (boot_raw=0x7f2520465290) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/_threadmodule.c:1092
#19 0x00007f2535d87924 in pythread_wrapper (arg=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Python/thread_pthread.h:241
#20 0x00007f25358771ca in start_thread (arg=<optimized out>) at pthread_create.c:479
#21 0x00007f2534d59e73 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) 
(gdb) thread 4
[Switching to thread 4 (Thread 0x7f251cbe9700 (LWP 2803602))]
#0  0x00007f2534e44251 in __GI___poll (fds=0x7f251cbe89c0, nfds=1, timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
29        return SYSCALL_CANCEL (poll, fds, nfds, timeout);
(gdb) bt
#0  0x00007f2534e44251 in __GI___poll (fds=0x7f251cbe89c0, nfds=1, timeout=100) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007f2526bb5d20 in poll (__timeout=100, __nfds=1, __fds=0x7f251cbe89c0) at /usr/include/bits/poll2.h:38
#2  internal_select (writing=writing@entry=0, interval=<optimized out>, connect=connect@entry=0, s=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/socketmodule.c:779
#3  0x00007f2526bb5df7 in sock_call_ex (s=s@entry=0x7f25204556c0, writing=writing@entry=0, sock_func=sock_func@entry=0x7f2526bb41b7 <sock_recv_impl>, data=data@entry=0x7f251cbe8a70,
    connect=connect@entry=0, err=err@entry=0x0, timeout=100000000) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/socketmodule.c:874
#4  0x00007f2526bb638b in sock_call (data=0x7f251cbe8a70, func=0x7f2526bb41b7 <sock_recv_impl>, writing=0, s=0x7f25204556c0) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/socketmodule.c:3533
#5  sock_recv_guts (s=s@entry=0x7f25204556c0, cbuf=<optimized out>, len=<optimized out>, flags=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/socketmodule.c:3533
#6  0x00007f2526bb655d in sock_recv (s=0x7f25204556c0, args=args@entry=(16,)) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Include/cpython/bytesobject.h:37
#7  0x00007f2535ce3d8d in method_vectorcall_VARARGS (func=<optimized out>, args=0x7f2526ef43c8, nargsf=<optimized out>, kwnames=<optimized out>)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/descrobject.c:330
#8  0x00007f2535cc48c3 in _PyObject_VectorcallTstate (kwnames=<optimized out>, nargsf=<optimized out>, args=<optimized out>, callable=<method_descriptor at remote 0x7f2526abca90>, tstate=0x7f2510005000)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Include/internal/pycore_call.h:77
#9  PyObject_Vectorcall (callable=<method_descriptor at remote 0x7f2526abca90>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/call.c:299
#10 0x00007f2535c82442 in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Python/ceval.c:4772
#11 0x00007f2535c8ed2c in _PyEval_EvalFrame (throwflag=0, frame=0x7f2526ef4020, tstate=0x7f2510005000) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Include/internal/pycore_ceval.h:70
#12 _PyEval_Vector (kwnames=<optimized out>, argcount=<optimized out>, args=<optimized out>, locals=0x0, func=<optimized out>, tstate=<optimized out>)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Python/ceval.c:6435
#13 _PyFunction_Vectorcall (kwnames=<optimized out>, nargsf=<optimized out>, stack=<optimized out>, func=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/call.c:393
#14 _PyObject_VectorcallTstate (tstate=0x7f2510005000, callable=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>, tstate=0x7f2510005000, callable=<optimized out>,
    args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Include/internal/pycore_call.h:92
#15 0x00007f2535d74a58 in method_vectorcall (method=<optimized out>, args=0x7f253621dc10 <_PyRuntime+58928>, nargsf=<optimized out>, kwnames=0x0)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/classobject.c:67
#16 0x00007f2535cb17bb in _PyVectorcall_Call (kwargs=<optimized out>, tuple=(), callable=<method at remote 0x7f2520479600>, func=0x7f2535d74940 <method_vectorcall>, tstate=0x7f2510005000)
    at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/call.c:245
#17 _PyObject_Call (tstate=0x7f2510005000, callable=<method at remote 0x7f2520479600>, args=(), kwargs=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Objects/call.c:328
#18 0x00007f2535d9e40c in thread_run (boot_raw=0x7f2520465320) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Modules/_threadmodule.c:1092
#19 0x00007f2535d87924 in pythread_wrapper (arg=<optimized out>) at /usr/src/debug/python3.11-3.11.2-2.el8_8.2.x86_64/Python/thread_pthread.h:241
#20 0x00007f25358771ca in start_thread (arg=<optimized out>) at pthread_create.c:479
#21 0x00007f2534d59e73 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)

@alex
Copy link
Member

alex commented Feb 9, 2024

If you're able to wait for other threads to complete before exiting main, that's likely your best bet.

Once openssl/openssl@99fb31c is in an OpenSSL release, I think we'll build with it which will solve this issue.

Unfortunately I don't have a great suggestion beyond that.

@reaperhulk
Copy link
Member

That feature looks like it’ll be in 3.3, which is scheduled for April. We could conceivably ship it in a hypothetical May release.

@botovq
Copy link
Contributor

botovq commented Mar 10, 2024

A call to OPENSSL_init_ssl() (which is exposed in rust-openssl) with OPENSSL_INIT_NO_ATEXIT could solve this. Not sure if it would be acceptable for cryptography to expose this function or to add it to its init handlers.

@alex
Copy link
Member

alex commented Mar 10, 2024 via email

@botovq
Copy link
Contributor

botovq commented Mar 11, 2024

I don't follow. Automatic library initialization is supposed to happen during the first API call that needs it. Why is that different for a system OpenSSL?

Once the atexit handler is registered, it won't be unregistered by subsequent init calls.

@reaperhulk
Copy link
Member

If we're linked against system OpenSSL then Python or other things in process may init before we do.

@rahulgharpinde
Copy link
Author

@alex
openssl/openssl@99fb31c to consume this fix, what all dependent packages need to update. And is there any documentation, which will help us to manually build package to test fix.

@alex
Copy link
Member

alex commented Mar 19, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

4 participants