Co-authored-by: R. David Murray <rdmurray@bitdance.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Basic support for pyrepl in Emscripten. Limitations:
* requires JSPI
* no signal handling implemented
As followup work, it would be nice to implement a webworker variant
for when JSPI is not available and proper signal handling.
Because it requires JSPI, it doesn't work in Safari. Firefox requires
setting an experimental flag. All the Chromiums have full support since
May. Until we make it work without JSPI, let's keep the original web_example
around.
(cherry picked from commit c933a6bb329bb97bc7e448388dad1b74f7ca4baa)
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Éric <merwok@netwok.org>
This is useful for implementing proper `input()`. It requires the
JavaScript engine to support the wasm JSPI spec which is now stage 4.
It is supported on Chrome since version 137 and on Firefox and node
behind a flag.
We override the `__wasi_fd_read()` syscall with our own variant that
checks for a readAsync operation. If it has it, we use our own async
variant of `fd_read()`, otherwise we use the original `fd_read()`.
We also add a variant of `FS.createDevice()` called
`FS.createAsyncInputDevice()`.
Finally, if JSPI is available, we wrap the `main()` symbol with
`WebAssembly.promising()` so that we can stack switch from `fd_read()`.
If JSPI is not available, attempting to read from an AsyncInputDevice
will raise an `OSError`.
(cherry picked from commit 7ae4749d064bd49b0dd96172fee20c1f1678d9e9)
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
gh-136421: Load `_datetime` static types during interpreter initialization (GH-136583)
`_datetime` is a special module, because it's the only non-builtin C extension that contains static types. As such, it would initialize static types in the module's execution function, which can run concurrently. Since static type initialization is not thread-safe, this caused crashes. This fixes it by moving the initialization of `_datetime`'s static types to interpreter startup (where all other static types are initialized), which is already properly protected through other locks.
(cherry picked from commit a10960699a2b3e4e62896331c4f9cfd162ebf440)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
gh-136870: fix data race in `PyThreadState_Clear` on `sys_tracing_threads` (GH-136951)
In free-threading, multiple threads can be cleared concurrently as such the modifications on `sys_tracing_threads` should be done while holding the profile lock, otherwise it can race with other threads setting up profiling.
(cherry picked from commit f183996eb77fd2d5662c62667298c292c943ebf5)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
gh-134411: assert `PyLong_FromLong(x) != NULL` when `x` is known to be small (GH-134415)
Since `PyLong_From Long(PY_MONITORING_DEBUGGER_ID)` falls to `small_int` case and can't return `NULL`. Added `assert`s for extra confidence.
https://github.com/python/cpython/issues/134411#issuecomment-2897653868
(cherry picked from commit cf19b6435d02dd7be11b84a44f4a8a9f1a935b15)
Co-authored-by: Sergey Muraviov <smurav@mail.ru>
Provide a stub implementation of umask that is enough to get some tests passing.
More work is needed upstream in Emscripten to make all umask tests to pass.
(cherry picked from commit 12e52cad718723636a96042f9399634392285c44)
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
Corrects the handling of getuid on emscripten, which was consistently reporting as 0.
(cherry picked from commit e81c4e84b3a8688a367099e3adf9b2fcf914447f)
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
gh-136517: Print uncollectable objects if DEBUG_UNCOLLECTABLE mode was set (GH-136518)
(cherry picked from commit c560df9658f1a24edea995fe6f9c84c55b37cfb3)
Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com>
gh-136541: Fix several problems of perf trampolines in x86_64 and aarch64 (GH-136500)
This commit fixes the following problems:
* The x86_64 trampolines are not preserving frame pointers
* The hardcoded offsets to the code segment from the FDE only worked properly for x64_64
* The CIE data was not following conventions of aarch64
* The eh_frame for aarch64 was not fully correct
(cherry picked from commit 236f733d8ffb3d587e1167fa0a0248c24512e7fd)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-91048: Revert the memory cache removal for remote debugging (GH-136440)
(cherry picked from commit 77d25e5b169f7c306d3a6d9ca6777c0a0be80d8f)
gh-91048: Reintroduce the memory cache for remote debugging
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
The free threading build uses QSBR to delay the freeing of dictionary
keys and list arrays when the objects are accessed by multiple threads
in order to allow concurrent reads to proceed with holding the object
lock. The requests are processed in batches to reduce execution
overhead, but for large memory blocks this can lead to excess memory
usage.
Take into account the size of the memory block when deciding when to
process QSBR requests.
Also track the amount of memory being held by QSBR for mimalloc pages. Advance the write sequence if this memory exceeds a limit. Advancing the sequence will allow it to be freed more quickly.
Process the held QSBR items from the "eval breaker", rather than from `_PyMem_FreeDelayed()`. This gives a higher chance that the global read sequence has advanced enough so that items can be freed.
(cherry picked from commit 113de8545ffe74a4a1dddb9351fa1cbd3562b621)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-102567: Add missing newline to `--help-all` (GH-136391)
(cherry picked from commit fbef0c1d6a6540f5063138b861f79bdb2a935b0a)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
This moves the deprecation warning from compile time to run time.
(cherry picked from commit 86c3316183a79867e3c666d0830f897e16f0f339)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-130396: Move PYOS_LOG2_STACK_MARGIN to internal headers (GH-135928)
Move PYOS_LOG2_STACK_MARGIN, PYOS_STACK_MARGIN,
PYOS_STACK_MARGIN_BYTES and PYOS_STACK_MARGIN_SHIFT macros to
pycore_pythonrun.h internal header. Add underscore (_) prefix to the
names to make them private. Rename _PYOS to _PyOS.
(cherry picked from commit 28940e8e4884eb3444a8ea15a5634f76b154d7ed)
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix a possible crash when deserializing a large marshal data
(at least several GiBs) containing a slice.
(cherry picked from commit 30ba03ea8ed98522b0500d6856b22727c88e818f)
Co-authored-by: Akshat Gupta <akshat.gupta24@gmail.com>
The free threading build could spin unnecessarily on `_Py_yield()` if the initial
compare and swap failed.
(cherry picked from commit cbfaf41caf135b8598a560854cd59e992a2ccfed)
Co-authored-by: Joseph Tibbertsma <josephtibbertsma@gmail.com>
gh-135608: Add a null check for attribute promotion to fix a JIT crash (GH-135613)
Co-authored-by: devdanzin <74280297+devdanzin@users.noreply.github.com>
Most importantly, this resolves the issues with functions and types defined in __main__.
It also expands the number of supported objects and simplifies the implementation.
(cherry picked from commit 725da50520, AKA gh-133957)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
As noted in the new tests, there are a few situations we must carefully accommodate
for functions that get pickled during interp.call(). We do so by running the script
from the main interpreter's __main__ module in a hidden module in the other
interpreter. That hidden module is used as the function __globals__.
(cherry picked from commit 269e19e0a7, AKA gh-135595)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-132815: Add support for JUMP_BACKWARD in specialization stats (GH-135606)
(cherry picked from commit a9e66a7c506680263b39bc8c150ddc5e72213c45)
Co-authored-by: PuQing <me@puqing.work>
For several builtin functions, we now fall back to __main__.__dict__ for the globals
when there is no current frame and _PyInterpreterState_IsRunningMain() returns
true. This allows those functions to be run with Interpreter.call().
The affected builtins:
* exec()
* eval()
* globals()
* locals()
* vars()
* dir()
We take a similar approach with "stateless" functions, which don't use any
global variables.
(cherry picked from commit a450a0ddec, AKA gh-135491)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
---------
(cherry picked from commit f0799795994bfd9ab0740c4d70ac54270991ba47)
Co-authored-by: Donghee Na <donghee.na@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: T. Wouters <thomas@python.org>
In this refactor we:
* move some code around
* make a couple of typedefs opaque
* decouple errors from session state
* improve tracebacks for propagated exceptions
This change helps simplify several upcoming changes.
(cherry picked from commit c7f4a80079, AKA gh-135369)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
PEP-734 has been accepted (for 3.14).
(FTR, I'm opposed to putting this under the concurrent package, but
doing so is the SC condition under which the module can land in 3.14.)
(cherry picked from commit 62143736b, AKA gh-133958)
On some iPad versions, Safari reports as "macOS". Modifies the GC trampoline detection
to add a feature-based check to detect this case.
(cherry picked from commit d4471297586335d8c24db8b2c030d32c94570344)
Co-authored-by: Gyeongjae Choi <def6488@gmail.com>
Apply Intel Control-flow Technology for x86-64 on asm_trampoline.S.
Required for mitigation against return-oriented programming (ROP)
and Call or Jump Oriented Programming (COP/JOP) attacks.
Manual application is required for the assembly files.
See also: https://sourceware.org/annobin/annobin.html/Test-cf-protection.html
Fix warnings `set but not used [-Wunused-but-set-variable]` in remote_debug.h (GH-135290)
(cherry picked from commit 49fc1f215aeb0f71445505191ccb65517b58a5aa)
Co-authored-by: Chris Eibl <138194463+chris-eibl@users.noreply.github.com>
Fix definition of `_Py_RemoteDebug_` symbols for static linking (GH-135146)
(cherry picked from commit 2e1ad6eb26871a379e5d3aa626d6fc93eba72a86)
Co-authored-by: Zanie Blue <contact@zanie.dev>
gh-133968: Add PyUnicodeWriter_WriteASCII() function (#133973)
Replace most PyUnicodeWriter_WriteUTF8() calls with
PyUnicodeWriter_WriteASCII().
(cherry picked from commit f49a07b531543dd8a42d90f5b1c89c0312fbf806)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
