* gh-108987: Fix _thread.start_new_thread() race condition (#109135)
Fix _thread.start_new_thread() race condition. If a thread is created
during Python finalization, the newly spawned thread now exits
immediately instead of trying to access freed memory and lead to a
crash.
thread_run() calls PyEval_AcquireThread() which checks if the thread
must exit. The problem was that tstate was dereferenced earlier in
_PyThreadState_Bind() which leads to a crash most of the time.
Move _PyThreadState_CheckConsistency() from thread_run() to
_PyThreadState_Bind().
(cherry picked from commit 517cd82ea7d01b344804413ef05610934a43a241)
* gh-109795: `_thread.start_new_thread`: allocate thread bootstate using raw memory allocator (#109808)
(cherry picked from commit 1b8f2366b38c87b0450d9c15bdfdd4c4a2fc3a01)
---------
Co-authored-by: Radislav Chugunov <52372310+chgnrdv@users.noreply.github.com>
gh-110052: Fix faulthandler for freed tstate (#110069)
faulthandler now detected freed interp and freed tstate, and no
longer dereference them.
(cherry picked from commit 2e37a38bcbfbe1357436e030538290e7d00b668d)
gh-109889: fix compiler's redundant NOP detection to look past NOPs with no lineno when looking for the next instruction's lineno (GH-109987)
(cherry picked from commit f580edcc6a4c528020afe46c753db713474acad6)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
gh-109521: Fix obscure cases handling in PyImport_GetImporter() (GH-109522)
PyImport_GetImporter() now sets RuntimeError if it fails to get sys.path_hooks
or sys.path_importer_cache or they are not list and dict correspondingly.
Previously it could return NULL without setting error in obscure cases,
crash or raise SystemError if these attributes have wrong type.
(cherry picked from commit 62c7015e89cbdedb5218d4fedd45f971885f67a8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Check the result of PySet_Contains() for error in Python/symtable.c (GH-109146)
(cherry picked from commit 87a7faf6b68c8076e640a9a1347a255f132d8382)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-104690: thread_run() checks for tstate dangling pointer (#109056)
thread_run() of _threadmodule.c now calls
_PyThreadState_CheckConsistency() to check if tstate is a dangling
pointer when Python is built in debug mode.
Rename ceval_gil.c is_tstate_valid() to
_PyThreadState_CheckConsistency() to reuse it in _threadmodule.c.
(cherry picked from commit f63d37877ad166041489a968233b57540f8456e8)
* [3.12] gh-109889: comment out assertion indicating a failed optimization of a redundant NOP
* comment out the function to avoid warnings on it being unused
gh-109823: Adjust labels in compiler when removing an empty basic block which is a jump target (GH-109839)
(cherry picked from commit d73c12b88c2275fd44e27c91c24f3ac85419d2b8)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
gh-109627: duplicated smalll exit blocks need to be assigned jump target labels (#109630)
(cherry picked from commit 9ccf0545efd5bc5af5aa51774030c471d49a972b)
gh-109219: propagate free vars through type param scopes (GH-109377)
(cherry picked from commit 909adb5092c0ae9426814742d97932204b211cfb)
Co-authored-by: Carl Meyer <carl@oddbird.net>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-105658: fix excess trace events for except block ending with a conditional block (#109384)
(cherry picked from commit 4a54074a0f5579d417445ec28427cd0ed5aa01f4)
gh-109195: fix source location for super load before LOAD_SUPER_ATTR (GH-109289)
(cherry picked from commit ceeb4173aee7b835f553a8286feaa48b98c16124)
Co-authored-by: Carl Meyer <carl@oddbird.net>
* gh-109118: Fix runtime crash when NameError happens in PEP 695 function (#109123)
(cherry picked from commit 17f994174de9211b2baaff217eeb1033343230fc)
* [3.12] gh-109118: Fix runtime crash when NameError happens in PEP 695 function (GH-109123).
(cherry picked from commit 17f994174de9211b2baaff217eeb1033343230fc)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* GH-108390: Prevent non-local events being set with `sys.monitoring.set_local_events()` (GH-108420)
* Restore generated objects
* Restore size of monitoring arrays in code object for 3.12 ABI compatibility.
* Update ABI file
gh-107913: Fix possible losses of OSError error codes (GH-107930)
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
(cherry picked from commit 2b15536fa94d07e9e286826c23507402313ec7f4)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Remove assert that should've been DEOPT_IF
The assert(method != NULL) in CALL_NO_KW_LIST_APPEND is wrong --
this condition should lead to a deoptimization, and indeed there
is a DEOPT_IF two lines later that will trigger if method == NULL.
This would crash in a devious repro scenario (first seen live
in boto3 tests) when compiled with assertions enabled.
In a production version there is no crash, so impact is limited.
(The crash also appears in main; I will prepare a separate PR.)
* Add back a different assert(self != NULL)
* 📜🤖 Added by blurb_it.
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
gh-107901: Fix missing line number on BACKWARD_JUMP at the end of a for loop (GH-108242)
(cherry picked from commit a1cc74c4eebc55795877eb3be019a1bec34402f8)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
gh-107916: Save the error code before decoding the filename in PyErr_SetFromErrnoWithFilename() etc (GH-107929)
(cherry picked from commit 80bdebdd8593f007a2232ec04a7729bba6ebf12c)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-107915: Handle errors in C API functions PyErr_Set*() and PyErr_Format() (GH-107918)
Such C API functions as PyErr_SetString(), PyErr_Format(),
PyErr_SetFromErrnoWithFilename() and many others no longer crash or
ignore errors if it failed to format the error message or decode the
filename. Instead, they keep a corresponding error.
(cherry picked from commit 633ea217a85f6b6ba5bdbc73094254d5811b3485)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Define PY_SSIZE_T_CLEAN.
---------
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
GH-107774: Add missing audit event for PEP 669 (GH-107775)
(cherry picked from commit 494e3d4436774a5ac1a569a635b8c5c881ef1c0c)
Co-authored-by: Mark Shannon <mark@hotpy.org>
GH-107724: Fix the signature of `PY_THROW` callback functions. (GH-107725)
(cherry picked from commit 52fbcf61b5a70993c2d32332ff0ad9f369d968d3)
Co-authored-by: Mark Shannon <mark@hotpy.org>
GH-107263: Increase C stack limit for most functions, except `_PyEval_EvalFrameDefault()` (GH-107535)
* Set C recursion limit to 1500, set cost of eval loop to 2 frames, and compiler mutliply to 2.
(cherry picked from commit fa45958450aa3489607daf9855ca0474a2a20878)
Co-authored-by: Mark Shannon <mark@hotpy.org>
gh-107080: Fix Py_TRACE_REFS Crashes Under Isolated Subinterpreters (gh-107567)
The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
(cherry picked from commit 58ef74186795c56e3ec86e8c8f351a1d7826638a)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-107471: Fix Refleaks in test_import (gh-107569)
gh-107184 introduced a refleak in test_import.SubinterpImportTests (specifically test_singlephase_check_with_setting_and_override and test_single_init_extension_compat). We fix it here by making sure _testsinglephase is removed from sys.modules whenever we clear the runtime's internal state for the module.
The underlying problem is strictly contained in the internal function _PyImport_ClearExtension() (AKA _testinternalcapi.clear_extension()), which is only used in tests.
(This also fixes an intermittent segfault introduced in the same place, in test_disallowed_reimport.)
(cherry picked from commit 017f047183fa33743f7e36c5c360f5c670032be3)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974)
This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL. It turns out it's tricky to use the same thread state for different threads. Who could have guessed?
We solve the problem by eliminating the one object we were still sharing between interpreters. We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter.
We also remove the accommodations for "detached" thread states, which were a dubious idea to start with.
(cherry picked from commit 8ba4df91ae60833723d8d3b9afeb2b642f7176d5)
