Revert GH-131993.
Fix swallowing some syntax warnings in different modules if they accidentally
have the same message and are emitted from the same line.
Fix duplicated warnings in the "finally" block.
(cherry picked from commit 279db6bede30be3a1b86803585eb4404d27800da)
There were a few thread-safety issues when profiling or tracing all
threads via PyEval_SetProfileAllThreads or PyEval_SetTraceAllThreads:
* The loop over thread states could crash if a thread exits concurrently
(in both the free threading and default build)
* The modification of `c_profilefunc` and `c_tracefunc` wasn't
thread-safe on the free threading build.
(cherry picked from commit a10152f8fd0f4b291e53d646cffe22fbeec73e1e)
* [3.13] gh-139748: fix leaks in AC error paths when using unicode FS-based converters (GH-139765)
(cherry picked from commit b04a57deef66ce08233be57d1ab5873388df2cea)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* gh-112729: Correctly fail when the process is out of memory during interpreter creation (GH-139164)
(cherry picked from commit d06113c7a7cac76a28847702685e601b79f71bf8)
fix comment reference from man 7 signal to man 7 signal-safety (GH-138554)
(cherry picked from commit 2f5ace780ba3880ce07c4aacbda08ebba36e3b4d)
docs: fix comment reference from man 7 signal to man 7 signal-safety
Co-authored-by: mqudah <mohghq@gmail.com>
* [3.13] gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805)
Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.
(cherry picked from commit 8cc5aa47ee464ddfd8da5461edecf4a5c72df2ff)
Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* state "3.13.7 and earlier"
* backport: do not add the deprecated marker
* fix Py_IsFinalizing doc ref
---------
Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>
The `PyEval_SetProfileAllThreads` function and other related functions
had a race condition on `tstate->c_profilefunc` that could lead to a
crash when disable profiling or tracing on all threads while another
thread is starting to profile or trace a a call.
There are still potential crashes when threads exit concurrently with
profiling or tracing be enabled/disabled across all threads.
(cherry picked from commit 362692852f13cdd1d33cc7ed35c0cbac7af1a785)
gh-137440: Update comment in Python/hamt.c on importing for testing (GH-137441)
Switch from `_testcapi` to `_testinternalcapi`.
(cherry picked from commit 7ab68cd50658f76abc9e0f12e6212736e2440720)
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
gh-58124: Avoid CP_UTF8 in UnicodeDecodeError (GH-137415)
Fix name of the Python encoding in Unicode errors of the code page
codec: use "cp65000" and "cp65001" instead of "CP_UTF7" and "CP_UTF8"
which are not valid Python code names.
(cherry picked from commit ce1b747ff68754635b7b12870dfc527184ee3b39)
Co-authored-by: Victor Stinner <vstinner@python.org>
The free threading build uses QSBR to delay the freeing of dictionary
keys and list arrays when the objects are accessed by multiple threads
in order to allow concurrent reads to proceed with holding the object
lock. The requests are processed in batches to reduce execution
overhead, but for large memory blocks this can lead to excess memory
usage.
Take into account the size of the memory block when deciding when to
process QSBR requests.
Also track the amount of memory being held by QSBR for mimalloc pages.
Advance the write sequence if this memory exceeds a limit. Advancing
the sequence will allow it to be freed more quickly.
Process the held QSBR items from the "eval breaker", rather than from
`_PyMem_FreeDelayed()`. This gives a higher chance that the global read
sequence has advanced enough so that items can be freed.
(cherry picked from commit 113de8545ffe74a4a1dddb9351fa1cbd3562b621)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-134411: assert `PyLong_FromLong(x) != NULL` when `x` is known to be small (GH-134415)
Since `PyLong_From Long(PY_MONITORING_DEBUGGER_ID)` falls to `small_int` case and can't return `NULL`. Added `assert`s for extra confidence.
https://github.com/python/cpython/issues/134411#issuecomment-2897653868
(cherry picked from commit cf19b6435d02dd7be11b84a44f4a8a9f1a935b15)
Co-authored-by: Sergey Muraviov <smurav@mail.ru>
This commit fixes the following problems:
* The x86_64 trampolines are not preserving frame pointers
* The hardcoded offsets to the code segment from the FDE only worked properly for x64_64
* The CIE data was not following conventions of aarch64
* The eh_frame for aarch64 was not fully correct
(cherry picked from commit 236f733d8ffb3d587e1167fa0a0248c24512e7fd)
The free threading build could spin unnecessarily on `_Py_yield()` if the initial
compare and swap failed.
(cherry picked from commit cbfaf41caf135b8598a560854cd59e992a2ccfed)
Co-authored-by: Joseph Tibbertsma <josephtibbertsma@gmail.com>
Apply Intel Control-flow Technology for x86-64 on asm_trampoline.S.
Required for mitigation against return-oriented programming (ROP)
and Call or Jump Oriented Programming (COP/JOP) attacks.
Manual application is required for the assembly files.
See also: https://sourceware.org/annobin/annobin.html/Test-cf-protection.html
[3.14] gh-128605: Revert "Add branch protections for x86_64 in asm_tr…ampoline.S (GH-128606) (GH-135077)" (GH-135175)
This reverts commit 899cca6dbf76bf3e06a99f60a5f996ad6ba0761f,
which broke buildbots.
(cherry picked from commit b477e21d5cb85bc764d6d9d18d122aa703d7b548)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
[3.14] gh-128605: Add branch protections for x86_64 in asm_trampoline.S (GH-128606) (GH-135077)
Apply Intel Control-flow Technology for x86-64 on asm_trampoline.S.
Required for mitigation against return-oriented programming (ROP)
and Call or Jump Oriented Programming (COP/JOP) attacks.
Manual application is required for the assembly files.
See also: https://sourceware.org/annobin/annobin.html/Test-cf-protection.html
(cherry picked from commit 899cca6dbf76bf3e06a99f60a5f996ad6ba0761f)
Co-authored-by: stratakis <cstratak@redhat.com>
gh-130070: Fix `exec(<string>, closure=<non-None>)` unexpected path (#130071)
Fixed an assertion error (so, it could be reproduced only in builds with assertions enabled)
for `exec` when the `source` argument is a string and the `closure` argument is not `None`.
Co-authored-by: sobolevn <mail@sobolevn.me>
(cherry picked from commit 954b2cf031fb84ff3386251d5c45281f47229003)
(cherry picked from commit ac3c439cdfee8452f2bcceacd67a1f4e423ac3cf)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Mark Shannon <mark@hotpy.org>
gh-131988: Fix a multithreaded scaling regression
The 3.13 free threaded build immortalizes certain objects to avoid
reference count contention. In gh-127114 the condition was
unintentionally changed to happen when the first thread was created
instead of the first non-main thread. The `interp->gc.immortalize` field
is then cleared again during `_PyGC_Init()`.
Change the condition so that we check if we should immortalize objects
using deferred reference counting whenever a non-main thread is created.
CPython's pthread-based thread identifier relies on pthread_t being able
to be represented as an unsigned integer type.
This is true in most Linux libc implementations where it's defined as an
unsigned long, however musl typedefs it as a struct *.
If the pointer has the high bit set and is cast to PyThread_ident_t, the
resultant value can be sign-extended [0]. This can cause issues when
comparing against threading._MainThread's identifier. The main thread's
identifier value is retrieved via _get_main_thread_ident which is backed
by an unsigned long which truncates sign extended bits.
>>> hex(threading.main_thread().ident)
'0xb6f33f3c'
>>> hex(threading.current_thread().ident)
'0xffffffffb6f33f3c'
Work around this by conditionally compiling in some code for non-glibc
based Linux platforms that are at risk of sign-extension to return a
PyLong based on the main thread's unsigned long thread identifier if the
current thread is the main thread.
[0]: https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Arrays-and-pointers-implementation.html
---------
(cherry picked from commit 72123063ddee84bb2c9d591a23f420997e35af5a)
Signed-off-by: Vincent Fazio <vfazio@gmail.com>
Co-authored-by: Vincent Fazio <vfazio@gmail.com>
gh-131818: Add imply `-P` in `-I`'s help message (GH-131819)
(cherry picked from commit 8bd88e2827b2a887c2ea8e5ed0c0575704bfacdf)
Co-authored-by: Hang <bebound@gmail.com>
gh-131670: Fix crash in `anext()` when `__anext__` is sync and raises (GH-131682)
(cherry picked from commit 929afd1d6ee4fb89ac818037effe6577947103de)
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
The `sys_tracing_threads` variable should be read inside `LOCK_SETUP()`.
(cherry picked from commit 0de5e0c5442abddbe17481ef450e4abc992058f5)
Co-authored-by: Sam Gross <colesbury@gmail.com>
