heap-use-after-free in _PyFunction_LookupByVersion

[gvanrossum]

Crash report

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

Python 3.13.0a0 (heads/main:531930f47f, Aug 21 2023, 17:14:02) [Clang 14.0.3 (clang-1403.0.22.14.1)]

What happened?

Shortest repro I've found so far:

./python.exe -Xuops -m test test_opcache -v

I'm guessing I'm doing something wrong with the func_version_cache logic (the table contains borrowed pointers which are cleared by func_dealloc), but I can't quite figure out what. You can't subclass the function type so clearing the table entry in func_dealloc would seem to be safe. The output from address sanitizer hasn't jogged my memory yet.

Click for address sanitizer output:

``` 0:00:00 load avg: 2.38 Run tests sequentially 0:00:00 load avg: 2.38 [1/1] test_opcache ================================================================= ==48442==ERROR: AddressSanitizer: heap-use-after-free on address 0x610000f109e8 at pc 0x000102bf6885 bp 0x7ff7bd537070 sp 0x7ff7bd537068 READ of size 4 at 0x610000f109e8 thread T0 #0 0x102bf6884 in _PyFunction_LookupByVersion funcobject.c:285 #1 0x102fd3c74 in uop_optimize optimizer.c:792 #2 0x102fd0c8d in _PyOptimizer_BackEdge optimizer.c:169 #3 0x102e98232 in _PyEval_EvalFrameDefault generated_cases.c.h:2960 #4 0x102e6511f in _PyEval_Vector ceval.c:1622 #5 0x102b87545 in _PyFunction_Vectorcall call.c #6 0x102b84b0a in _PyObject_VectorcallDictTstate call.c:146 #7 0x102b87bbe in _PyObject_Call_Prepend call.c:504 #8 0x102cdbd0e in slot_tp_init typeobject.c:9057 #9 0x102cc3411 in type_call typeobject.c:1700 #10 0x102b85226 in _PyObject_MakeTpCall call.c:242 #11 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184 #12 0x102b86d4f in PyObject_Vectorcall call.c:327 #13 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759 #14 0x102e6511f in _PyEval_Vector ceval.c:1622 #15 0x102b87545 in _PyFunction_Vectorcall call.c #16 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186 #17 0x102b8d9af in method_vectorcall classobject.c:91 #18 0x102b86bb1 in _PyVectorcall_Call call.c:273 #19 0x102b86ebd in _PyObject_Call call.c:348 #20 0x102b8723c in PyObject_Call call.c:373 #21 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601 #22 0x102bd92f2 in gen_send_ex2 genobject.c:232 #23 0x102bd8c60 in gen_send_ex genobject.c:273 #24 0x102bda780 in _gen_throw genobject.c:561 #25 0x102bd999e in gen_throw genobject.c:598 #26 0x102baa5ab in method_vectorcall_FASTCALL descrobject.c:410 #27 0x102b8468a in _PyObject_VectorcallTstate pycore_call.h:186 #28 0x102b86d4f in PyObject_Vectorcall call.c:327 #29 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759 #30 0x102e6511f in _PyEval_Vector ceval.c:1622 #31 0x102b87545 in _PyFunction_Vectorcall call.c #32 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186 #33 0x102b8d8c3 in method_vectorcall classobject.c:61 #34 0x102b8468a in _PyObject_VectorcallTstate pycore_call.h:186 #35 0x102b86d4f in PyObject_Vectorcall call.c:327 #36 0x102e840b6 in _PyEval_EvalFrameDefault generated_cases.c.h:3519 #37 0x102e6511f in _PyEval_Vector ceval.c:1622 #38 0x102b87545 in _PyFunction_Vectorcall call.c #39 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186 #40 0x102b8d9af in method_vectorcall classobject.c:91 #41 0x102b86bb1 in _PyVectorcall_Call call.c:273 #42 0x102b86ebd in _PyObject_Call call.c:348 #43 0x102b8723c in PyObject_Call call.c:373 #44 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601 #45 0x102e6511f in _PyEval_Vector ceval.c:1622 #46 0x102b87545 in _PyFunction_Vectorcall call.c #47 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135 #48 0x102b87bbe in _PyObject_Call_Prepend call.c:504 #49 0x102cd971b in slot_tp_call typeobject.c:8813 #50 0x102b85226 in _PyObject_MakeTpCall call.c:242 #51 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184 #52 0x102b86d4f in PyObject_Vectorcall call.c:327 #53 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759 #54 0x102e6511f in _PyEval_Vector ceval.c:1622 #55 0x102b87545 in _PyFunction_Vectorcall call.c #56 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186 #57 0x102b8d9af in method_vectorcall classobject.c:91 #58 0x102b86bb1 in _PyVectorcall_Call call.c:273 #59 0x102b86ebd in _PyObject_Call call.c:348 #60 0x102b8723c in PyObject_Call call.c:373 #61 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601 #62 0x102e6511f in _PyEval_Vector ceval.c:1622 #63 0x102b87545 in _PyFunction_Vectorcall call.c #64 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135 #65 0x102b87bbe in _PyObject_Call_Prepend call.c:504 #66 0x102cd971b in slot_tp_call typeobject.c:8813 #67 0x102b85226 in _PyObject_MakeTpCall call.c:242 #68 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184 #69 0x102b86d4f in PyObject_Vectorcall call.c:327 #70 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759 #71 0x102e6511f in _PyEval_Vector ceval.c:1622 #72 0x102b87545 in _PyFunction_Vectorcall call.c #73 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186 #74 0x102b8d9af in method_vectorcall classobject.c:91 #75 0x102b86bb1 in _PyVectorcall_Call call.c:273 #76 0x102b86ebd in _PyObject_Call call.c:348 #77 0x102b8723c in PyObject_Call call.c:373 #78 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601 #79 0x102e6511f in _PyEval_Vector ceval.c:1622 #80 0x102b87545 in _PyFunction_Vectorcall call.c #81 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135 #82 0x102b87bbe in _PyObject_Call_Prepend call.c:504 #83 0x102cd971b in slot_tp_call typeobject.c:8813 #84 0x102b85226 in _PyObject_MakeTpCall call.c:242 #85 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184 #86 0x102b86d4f in PyObject_Vectorcall call.c:327 #87 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759 #88 0x102e6511f in _PyEval_Vector ceval.c:1622 #89 0x102b87545 in _PyFunction_Vectorcall call.c #90 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186 #91 0x102b8d9af in method_vectorcall classobject.c:91 #92 0x102b86bb1 in _PyVectorcall_Call call.c:273 #93 0x102b86ebd in _PyObject_Call call.c:348 #94 0x102b8723c in PyObject_Call call.c:373 #95 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601 #96 0x102e6511f in _PyEval_Vector ceval.c:1622 #97 0x102b87545 in _PyFunction_Vectorcall call.c #98 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135 #99 0x102b87bbe in _PyObject_Call_Prepend call.c:504 #100 0x102cd971b in slot_tp_call typeobject.c:8813 #101 0x102b85226 in _PyObject_MakeTpCall call.c:242 #102 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184 #103 0x102b86d4f in PyObject_Vectorcall call.c:327 #104 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759 #105 0x102e6511f in _PyEval_Vector ceval.c:1622 #106 0x102b87545 in _PyFunction_Vectorcall call.c #107 0x103199e70 in _PyObject_VectorcallTstate pycore_call.h:186 #108 0x10319ce35 in partial_vectorcall _functoolsmodule.c:230 #109 0x102b8468a in _PyObject_VectorcallTstate pycore_call.h:186 #110 0x102b86d4f in PyObject_Vectorcall call.c:327 #111 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759 #112 0x102e6511f in _PyEval_Vector ceval.c:1622 #113 0x102b87545 in _PyFunction_Vectorcall call.c #114 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186 #115 0x102b8d8c3 in method_vectorcall classobject.c:61 #116 0x102b86b77 in _PyVectorcall_Call call.c:285 #117 0x102b86ebd in _PyObject_Call call.c:348 #118 0x102b8723c in PyObject_Call call.c:373 #119 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601 #120 0x102e64cc6 in PyEval_EvalCode ceval.c:557 #121 0x102e5b47d in builtin_exec bltinmodule.c.h:540 #122 0x102c770b2 in cfunction_vectorcall_FASTCALL_KEYWORDS methodobject.c:441 #123 0x102b8468a in _PyObject_VectorcallTstate pycore_call.h:186 #124 0x102b86d4f in PyObject_Vectorcall call.c:327 #125 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759 #126 0x102e6511f in _PyEval_Vector ceval.c:1622 #127 0x102b87545 in _PyFunction_Vectorcall call.c #128 0x102b86bb1 in _PyVectorcall_Call call.c:273 #129 0x102b86ebd in _PyObject_Call call.c:348 #130 0x102b8723c in PyObject_Call call.c:373 #131 0x10308900f in pymain_run_module main.c:300 #132 0x103086b09 in Py_RunMain main.c:688 #133 0x1030888b1 in pymain_main main.c:718 #134 0x103088bb0 in Py_BytesMain main.c:742 #135 0x1029c1918 in main python.c:15 #136 0x7ff80da2d41e in start+0x76e (dyld:x86_64+0xfffffffffff6e41e) (BuildId: 5db85b72c63a318291e55c942ec30e4832000000200000000100000000040d00)

0x610000f109e8 is located 168 bytes inside of 184-byte region [0x610000f10940,0x610000f109f8)
freed by thread T0 here:
#0 0x10444cee9 in wrap_free+0xa9 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x48ee9) (BuildId: 756bb7515781379f84412f22c4274ffd2400000010000000000a0a0000030d00)
#1 0x102c8ca5b in _PyMem_RawFree obmalloc.c:74
#2 0x102c90ee5 in _PyMem_DebugFree obmalloc.c:2297
#3 0x102c8fb42 in PyObject_Free obmalloc.c:831
#4 0x10308eccd in PyObject_GC_Del gcmodule.c:2415
#5 0x102bf8228 in func_dealloc funcobject.c:934
#6 0x102c8c164 in _Py_Dealloc object.c:2656
#7 0x102c0628b in list_dealloc listobject.c:357
#8 0x102c8c164 in _Py_Dealloc object.c:2656
#9 0x102f594a8 in _PyFrame_ClearExceptCode frame.c:140
#10 0x102eb2819 in clear_thread_frame ceval.c:1486
#11 0x102eac816 in _PyEval_FrameClearAndPop ceval.c:1513
#12 0x102e95dab in _PyEval_EvalFrameDefault generated_cases.c.h:1064
#13 0x102e6511f in _PyEval_Vector ceval.c:1622
#14 0x102b87545 in _PyFunction_Vectorcall call.c
#15 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186
#16 0x102b8d9af in method_vectorcall classobject.c:91
#17 0x102b86bb1 in _PyVectorcall_Call call.c:273
#18 0x102b86ebd in _PyObject_Call call.c:348
#19 0x102b8723c in PyObject_Call call.c:373
#20 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601
#21 0x102e6511f in _PyEval_Vector ceval.c:1622
#22 0x102b87545 in _PyFunction_Vectorcall call.c
#23 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135
#24 0x102b87bbe in _PyObject_Call_Prepend call.c:504
#25 0x102cd971b in slot_tp_call typeobject.c:8813
#26 0x102b85226 in _PyObject_MakeTpCall call.c:242
#27 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184
#28 0x102b86d4f in PyObject_Vectorcall call.c:327
#29 0x102e78d52 in _PyEval_EvalFrameDefault generated_cases.c.h:3759

previously allocated by thread T0 here:
#0 0x10444cda0 in wrap_malloc+0xa0 (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x48da0) (BuildId: 756bb7515781379f84412f22c4274ffd2400000010000000000a0a0000030d00)
#1 0x102c8c9f4 in _PyMem_RawMalloc obmalloc.c:46
#2 0x102c90a21 in _PyMem_DebugMalloc obmalloc.c:2282
#3 0x102c8f9a7 in PyObject_Malloc obmalloc.c:802
#4 0x10308e2a2 in _PyObject_GC_New gcmodule.c:2337
#5 0x102bf5d2c in PyFunction_NewWithQualName funcobject.c:188
#6 0x102bf6aba in PyFunction_New funcobject.c:312
#7 0x102e71421 in _PyEval_EvalFrameDefault generated_cases.c.h:4621
#8 0x102e64cc6 in PyEval_EvalCode ceval.c:557
#9 0x1030261e8 in run_eval_code_obj pythonrun.c:1725
#10 0x103021d94 in run_mod pythonrun.c:1746
#11 0x10301f75f in PyRun_StringFlags pythonrun.c:1621
#12 0x102e5a6a1 in builtin_eval bltinmodule.c.h:456
#13 0x102f3995f in _PyUopExecute executor_cases.c.h:2417
#14 0x102e97857 in _PyEval_EvalFrameDefault generated_cases.c.h:2982
#15 0x102e6511f in _PyEval_Vector ceval.c:1622
#16 0x102b87545 in _PyFunction_Vectorcall call.c
#17 0x102b8faf0 in _PyObject_VectorcallTstate pycore_call.h:186
#18 0x102b8d9af in method_vectorcall classobject.c:91
#19 0x102b86bb1 in _PyVectorcall_Call call.c:273
#20 0x102b86ebd in _PyObject_Call call.c:348
#21 0x102b8723c in PyObject_Call call.c:373
#22 0x102e846a4 in _PyEval_EvalFrameDefault generated_cases.c.h:4601
#23 0x102e6511f in _PyEval_Vector ceval.c:1622
#24 0x102b87545 in _PyFunction_Vectorcall call.c
#25 0x102b84bd7 in _PyObject_VectorcallDictTstate call.c:135
#26 0x102b87bbe in _PyObject_Call_Prepend call.c:504
#27 0x102cd971b in slot_tp_call typeobject.c:8813
#28 0x102b85226 in _PyObject_MakeTpCall call.c:242
#29 0x102b8473d in _PyObject_VectorcallTstate pycore_call.h:184

SUMMARY: AddressSanitizer: heap-use-after-free funcobject.c:285 in _PyFunction_LookupByVersion
Shadow bytes around the buggy address:
0x1c20001e20e0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x1c20001e20f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x1c20001e2100: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x1c20001e2110: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x1c20001e2120: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
=>0x1c20001e2130: fd fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd fa
0x1c20001e2140: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x1c20001e2150: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x1c20001e2160: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x1c20001e2170: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x1c20001e2180: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==48442==ABORTING
Fatal Python error: Aborted

Current thread 0x00007ff851463640 (most recent call first):
File "/Users/guido/cpython/Lib/traceback.py", line 357 in _walk_tb_with_full_positions
File "/Users/guido/cpython/Lib/traceback.py", line 421 in _extract_from_extended_frame_gen
File "/Users/guido/cpython/Lib/traceback.py", line 697 in init
File "/Users/guido/cpython/Lib/traceback.py", line 139 in format_exception
File "/Users/guido/cpython/Lib/test/support/testresult.py", line 96 in __makeErrorDict
File "/Users/guido/cpython/Lib/test/support/testresult.py", line 113 in addFailure
File "/Users/guido/cpython/Lib/unittest/case.py", line 98 in _addError
File "/Users/guido/cpython/Lib/unittest/case.py", line 75 in testPartExecutor
File "/Users/guido/cpython/Lib/contextlib.py", line 159 in exit
File "/Users/guido/cpython/Lib/unittest/case.py", line 633 in run
File "/Users/guido/cpython/Lib/unittest/case.py", line 690 in call
File "/Users/guido/cpython/Lib/unittest/suite.py", line 122 in run
File "/Users/guido/cpython/Lib/unittest/suite.py", line 84 in call
File "/Users/guido/cpython/Lib/unittest/suite.py", line 122 in run
File "/Users/guido/cpython/Lib/unittest/suite.py", line 84 in call
File "/Users/guido/cpython/Lib/unittest/suite.py", line 122 in run
File "/Users/guido/cpython/Lib/unittest/suite.py", line 84 in call
File "/Users/guido/cpython/Lib/test/support/testresult.py", line 143 in run
File "/Users/guido/cpython/Lib/test/support/init.py", line 1116 in _run_suite
File "/Users/guido/cpython/Lib/test/support/init.py", line 1242 in run_unittest
File "/Users/guido/cpython/Lib/test/libregrtest/runtest.py", line 294 in _test_module
File "/Users/guido/cpython/Lib/test/libregrtest/runtest.py", line 330 in _runtest_inner2
File "/Users/guido/cpython/Lib/test/libregrtest/runtest.py", line 373 in _runtest_inner
File "/Users/guido/cpython/Lib/test/libregrtest/runtest.py", line 248 in _runtest
File "/Users/guido/cpython/Lib/test/libregrtest/runtest.py", line 278 in runtest
File "/Users/guido/cpython/Lib/test/libregrtest/main.py", line 483 in run_tests_sequential
File "/Users/guido/cpython/Lib/test/libregrtest/main.py", line 621 in run_tests
File "/Users/guido/cpython/Lib/test/libregrtest/main.py", line 799 in _main
File "/Users/guido/cpython/Lib/test/libregrtest/main.py", line 758 in main
File "/Users/guido/cpython/Lib/test/libregrtest/main.py", line 822 in main
File "/Users/guido/cpython/Lib/test/main.py", line 2 in
File "/Users/guido/cpython/Lib/runpy.py", line 88 in _run_code
File "/Users/guido/cpython/Lib/runpy.py", line 198 in _run_module_as_main

Extension modules: _testcapi, _testinternalcapi (total: 2)
Abort trap: 6

</details>


### Error messages

_No response_

<!-- gh-linked-prs -->
### Linked PRs
* gh-108296
* gh-108383
<!-- /gh-linked-prs -->