These were added to the limited API in 3.5-3.12.
Not including them in `Misc/stable_abi.toml` was a bug.
(cherry picked from commit 202fce0dbde1da32d8abc2eb59ddfce6f6a3c9fa)
There were a few thread-safety issues when profiling or tracing all
threads via PyEval_SetProfileAllThreads or PyEval_SetTraceAllThreads:
* The loop over thread states could crash if a thread exits concurrently
(in both the free threading and default build)
* The modification of `c_profilefunc` and `c_tracefunc` wasn't
thread-safe on the free threading build.
(cherry picked from commit a10152f8fd0f4b291e53d646cffe22fbeec73e1e)
Document that `PyInterpreterState_GetDict` returns a borrowed reference (GH-139451)
(cherry picked from commit 80cdf3ef74f6622899c1c82af1e08492e4ab7a00)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
The free threading build uses QSBR to delay the freeing of dictionary
keys and list arrays when the objects are accessed by multiple threads
in order to allow concurrent reads to proceed with holding the object
lock. The requests are processed in batches to reduce execution
overhead, but for large memory blocks this can lead to excess memory
usage.
Take into account the size of the memory block when deciding when to
process QSBR requests.
Also track the amount of memory being held by QSBR for mimalloc pages.
Advance the write sequence if this memory exceeds a limit. Advancing
the sequence will allow it to be freed more quickly.
Process the held QSBR items from the "eval breaker", rather than from
`_PyMem_FreeDelayed()`. This gives a higher chance that the global read
sequence has advanced enough so that items can be freed.
(cherry picked from commit 113de8545ffe74a4a1dddb9351fa1cbd3562b621)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Co-authored-by: Sam Gross <colesbury@gmail.com>
(cherry picked from commit ca87a47b3d92aabaefbbe79c0493d66602184b41)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Brian Schubert <brianm.schubert@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
* Clarify sys.getdefaultencoding() documentation
* Add missing documentation for PyUnicode_GetDefaultEncoding,
the C equivalent of sys.getdefaultencoding
(cherry picked from commit 9f25c1f012c8d432a93bf2dcad5f19a64dc00d3c)
Co-authored-by: RUANG (James Roy) <longjinyii@outlook.com>
gh-46236: Document PyUnicode_RSplit, PyUnicode_Partition and PyUnicode_RPartition (GH-130191)
(cherry picked from commit 0f5b82169e12321fd2294bf534496ad42a682ac4)
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
This approach eliminates the originally reported race. It also gets rid of the deadlock reported in gh-96071, so we can remove the workaround added then.
This is mostly a cherry-pick of 1c0a104 (AKA gh-126989). The difference is we add PyInterpreterState.threads_preallocated at the end of PyInterpreterState, instead of adding PyInterpreterState.threads.preallocated. That avoids ABI disruption.
When formatting the AST as a string, infinite values are replaced by
1e309, which evaluates to infinity. The initialization of this string
replacement was not thread-safe in the free threading build.
(cherry picked from commit 427dcf24de4e06d239745d74d08c4b2e541dca5a)
Revert the incremental GC in 3.13, since it's not clear that without further turning, the benefits outweigh the costs.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
- gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680)
(cherry picked from commit ce1064e4c9bcfd673323ad690e60f86e1ab907bb)
- gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` (GH-120955)
(cherry picked from commit ac61d58db0753a3b37de21dbc6e86b38f2a93f1b)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
* Add docs for new APIs
* Add soft-deprecation notices
* Add What's New porting entries
* Update comments referencing `PyFrame_LocalsToFast()` to mention the proxy instead
* Other related cleanups found when looking for refs to the deprecated APIs
(cherry picked from commit 3859e09e3d92d004978dd838f0511364e7edfb94)
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.
In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.
Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
The new `PyList_GetItemRef` is similar to `PyList_GetItem`, but returns
a strong reference instead of a borrowed reference. Additionally, if the
passed "list" object is not a list, the function sets a `TypeError`
instead of calling `PyErr_BadInternalCall()`.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)"
This reverts commit d9b606b3d04fc56fb0bcc479d7d6c14562edb5e2.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)"
This reverts commit cde1071b2a72e8261ca66053ef61431b7f3a81fd.
* Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)"
This reverts commit d731579bfb9a497cfb0076cb6b221058a20088fe.
* Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)"
This reverts commit d8f32be5b6a736dc2fc9dca3f1bf176c82fc9b44.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)"
This reverts commit 37e4e20eaa8f27ada926d49e5971fecf0477ad26.
Add PyUnicode_AsUTF8() function to the limited C API.
multiprocessing posixshmem now uses PyUnicode_AsUTF8() instead of
PyUnicode_AsUTF8AndSize(): the extension is built with the limited C
API. The function now raises an exception if the filename contains an
embedded null character instead of truncating silently the filename.
The PySys_Audit() function was added in Python 3.8 by the PEP 578
"Python Runtime Audit Hooks".
Add also PySys_AuditTuple() to the limited C API, function added
to Python 3.13.
Move non-limited "PerfMap" C API from Include/sysmodule.h to
Include/cpython/sysmodule.h.
Add PyMem_RawMalloc(), PyMem_RawCalloc(), PyMem_RawRealloc() and
PyMem_RawFree() to the limited C API.
These functions were added by Python 3.4 and are needed to port
stdlib extensions to the limited C API, like grp and pwd.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
If the timeout is greater than PY_TIMEOUT_MAX,
PyThread_acquire_lock_timed() uses a timeout of PY_TIMEOUT_MAX
microseconds, which is around 280.6 years. This case is unlikely and
limiting a timeout to 280.6 years sounds like a reasonable trade-off.
The constant PY_TIMEOUT_MAX is not used in PyPI top 5,000 projects.
* pycore_pythread.h is now the central place to make sure that
_POSIX_THREADS and _POSIX_SEMAPHORES macros are defined if
available.
* Make sure that pycore_pythread.h is included when _POSIX_THREADS
and _POSIX_SEMAPHORES macros are tested.
* PY_TIMEOUT_MAX is now defined as a constant, since its value
depends on _POSIX_THREADS, instead of being defined as a macro.
* Prevent integer overflow in the preprocessor when computing
PY_TIMEOUT_MAX_VALUE on Windows:
replace "0xFFFFFFFELL * 1000 < LLONG_MAX"
with "0xFFFFFFFELL < LLONG_MAX / 1000".
* Document the change and give hints how to fix affected code.
* Add an exception for PY_TIMEOUT_MAX name to smelly.py
* Add PY_TIMEOUT_MAX to the stable ABI
* Add PyDict_GetItemRef() and PyDict_GetItemStringRef() functions.
Add these functions to the stable ABI version 3.13.
* Add unit tests on the PyDict C API in test_capi.
* Convert PyObject_DelAttr() and PyObject_DelAttrString() macros to
functions.
* Add PyObject_DelAttr() and PyObject_DelAttrString() functions to
the stable ABI.
* Replace PyObject_SetAttr(obj, name, NULL) with
PyObject_DelAttr(obj, name).
