Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.
(cherry picked from commit 6edb2ddb5f3695cf4938979d645f31d7fba43ec8)
(cherry picked from commit bb0268f60dfe903a9bdb8d84104247a9318c6b18)
(cherry picked from commit 6af54d298d5135302037cdda7a1f5535e48cb1b6)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
HEAD_LOCK is called from _PyEval_ReInitThreads->_PyThreadState_DeleteExcept before _PyRuntimeState_ReInitThreads reinit runtime->interpreters.mutex which might be locked before fork.
(cherry picked from commit 522799a05e3e820339718151ac055af6d864d463)
Co-authored-by: ChuBoning <102216855+ChuBoning@users.noreply.github.com>
Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:
- `xml.etree.ElementTree.XMLParser.flush`
- `xml.etree.ElementTree.XMLPullParser.flush`
- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`
- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`
- `xml.sax.expatreader.ExpatParser.flush`
Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 .
Includes code suggested-by: Snild Dolkow <snild@sony.com>
and by core dev Serhiy Storchaka.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Manual backport due to code differences.
(cherry picked from commit e071b0d558b2f5cddd5a9fc6afadb4ba109ec77e)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
* Fix a crash when pass UINT_MAX.
* Fix an integer overflow on 64-bit non-Windows platforms.
(cherry picked from commit 0daf555c6fb3feba77989382135a58215e1d70a5)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Previously *consumed was not set in this case.
(cherry picked from commit f08e52ccb027f6f703302b8c1a82db9fd3934270).
(cherry picked from commit b8b3e6afc0a48c3cbb7c36d2f73e332edcd6058c)
Upgrade builds to OpenSSL 1.1.1u.
Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9.
Manual edits to the _ssl_data_300.h file prevent it from removing any
existing definitions in case those exist in some peoples builds and were
important (avoiding regressions during backporting).
(cherry picked from commit ede89af)
Co-authored-by: Ned Deily <nad@python.org>
This is a port of the applicable part of XKCP's fix [1] for
CVE-2022-37454 and avoids the segmentation fault and the infinite
loop in the test cases published in [2].
[1]: fdc6fef075
[2]: https://mouha.be/sha-3-buffer-overflow/
Regression test added by: Gregory P. Smith [Google LLC] <greg@krypto.org>
(cherry picked from commit 0e4e058602d93b88256ff90bbef501ba20be9dd3)
Co-authored-by: Theo Buehler <botovq@users.noreply.github.com>
Update libexpat from 2.4.9 to 2.5.0 to address CVE-2022-43680.
Co-authored-by: Shaun Walbridge <shaun.walbridge@gmail.com>
(cherry picked from commit 3e07f827b359617664ad0880f218f17ae4483299)
The macOS 13 SDK includes support for the `mkfifoat` and `mknodat` system calls.
Using the `dir_fd` option with either `os.mkfifo` or `os.mknod` could result in a
segfault if cpython is built with the macOS 13 SDK but run on an earlier
version of macOS. Prevent this by adding runtime support for detection of
these system calls ("weaklinking") as is done for other newer syscalls on
macOS.
(cherry picked from commit 6d0a0191a4e5477bd843e62c24d7f3bcad4fd5fc)
Co-authored-by: Ned Deily <nad@python.org>
gh-97005: Update libexpat from 2.4.7 to 2.4.9 (gh-97006)
Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
(cherry picked from commit 10e3d398c31cc1695752fc52bc6ca2ce9ef6237e)
Co-authored-by: Dong-hee Na <donghee.na@python.org>
Co-authored-by: Ned Deily <nad@python.org>
When binding a unix socket to an empty address on Linux, the socket is
automatically bound to an available address in the abstract namespace.
>>> s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
>>> s.bind("")
>>> s.getsockname()
b'\x0075499'
Since python 3.9, the socket is bound to the one address:
>>> s.getsockname()
b'\x00'
And trying to bind multiple sockets will fail with:
Traceback (most recent call last):
File "/home/nsoffer/src/cpython/Lib/test/test_socket.py", line 5553, in testAutobind
s2.bind("")
OSError: [Errno 98] Address already in use
Added 2 tests:
- Auto binding empty address on Linux
- Failing to bind an empty address on other platforms
Fixes f6b3a07b7df6 (bpo-44493: Add missing terminated NUL in sockaddr_un's length (GH-26866)
(cherry picked from commit c22f134211743cd5ad14cec1dd4f527bee542b4c)
Co-authored-by: Nir Soffer <nsoffer@redhat.com>
The `utc_to_seconds` call can fail, here's a minimal reproducer on
Linux:
TZ=UTC python -c "from datetime import *; datetime.fromtimestamp(253402300799 + 1)"
The old behavior still raised an error in a similar way, but only
because subsequent calculations happened to fail as well. Better to fail
fast.
This also refactors the tests to split out the `fromtimestamp` and
`utcfromtimestamp` tests, and to get us closer to the actual desired
limits of the functions. As part of this, we also changed the way we
detect platforms where the same limits don't necessarily apply (e.g.
Windows).
As part of refactoring the tests to hit this condition explicitly (even
though the user-facing behvior doesn't change in any way we plan to
guarantee), I noticed that there was a difference in the places that
`datetime.utcfromtimestamp` fails in the C and pure Python versions, which
was fixed by skipping the "probe for fold" logic for UTC specifically —
since UTC doesn't have any folds or gaps, we were never going to find a
fold value anyway. This should prevent some failures in the pure python
`utcfromtimestamp` method on timestamps close to 0001-01-01.
There are two separate news entries for this because one is a
potentially user-facing change, the other is an internal code
correctness change that, if anything, changes some error messages. The
two happen to be coupled because of the test refactoring, but they are
probably best thought of as independent changes.
Fixes GH-91581
(cherry picked from commit 83c0247d47b99f4571e35ea95361436e1d2a61cd)
Co-authored-by: Paul Ganssle <1377457+pganssle@users.noreply.github.com>
* [3.9] bpo-46785: Fix race condition between os.stat() and unlink on Windows (GH-31858).
(cherry picked from commit 39e6b8ae6a5b49bb23746fdcc354d148ff2d98e3)
Co-authored-by: Itai Steinherz <itaisteinherz@gmail.com>
Fix a crash in subinterpreters related to the garbage collector. When
a subinterpreter is deleted, untrack all objects tracked by its GC.
To prevent a crash in deallocator functions expecting objects to be
tracked by the GC, leak a strong reference to these objects on
purpose, so they are never deleted and their deallocator functions
are not called.
(cherry picked from commit 14243369b5f80613628a565c224bba7fb3fcacd8)
Co-authored-by: Victor Stinner <vstinner@python.org>
Argument Clinic now generates the same efficient code as before
adding the defining_class parameter.
(cherry picked from commit a055dac0b45031878a8196a8735522de018491e3)
Inputs >= 4GiB to `binascii.crc32(...)` when compiled to use the zlib
crc32 implementation (the norm on POSIX) no longer return the wrong
result.
(cherry picked from commit 4c989e19c84ec224655bbbde9422e16d4a838a80)
For consistency, replace "a SQL" with "an SQL"..
(cherry picked from commit 2d5835a019a46573d5b1b614c8ef88d6b564d8d4)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
In Linux kernel 5.14 one can dynamically request size of altstacksize
based on hardware capabilities with getauxval(AT_MINSIGSTKSZ).
This changes allows for Python extension's request to Linux kernel
to use AMX_TILE instruction set on Sapphire Rapids Xeon processor
to succeed, unblocking use of the ISA in frameworks.
Introduced HAVE_LINUX_AUXVEC_H in configure.ac and pyconfig.h.in
Used cpython_autoconf:269 docker container to generate configure.
(cherry picked from commit 3b128c054885fe881c3b57a5978de3ea89c81a9c)
Co-authored-by: Oleksandr Pavlyk <oleksandr.pavlyk@intel.com>
Replace two instances of "pysqlite" with "sqlite3" in sqlite3
docstrings. Also reword "is a no-op" to "does nothing" for clarity..
(cherry picked from commit b33a1ae703338e09dc0af5fbfd8ffa01d3ff75da)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
* bpo-46913: Fix test_faulthandler.test_sigfpe() on UBSAN (GH-31662)
Disable undefined behavior sanitizer (UBSAN) on
faulthandler_sigfpe().
(cherry picked from commit 4173d677a1d7c72bb32d292fbff1b4cf073d615c)
* bpo-46913: Fix test_faulthandler.test_read_null() on UBSan (GH31672)
Disable undefined behavior sanitizer (UBSan) on
faulthandler._read_null().
(cherry picked from commit 65b92ccdec2ee4a99e54aaf7ae2d9bbc2ebfe549)
* bpo-46913: test_hashlib skips _sha3 tests on UBSan (GH-31673)
If Python is built with UBSan, test_hashlib skips tests on the _sha3
extension which currently has undefined behaviors.
This change allows to run test_hashlib to check for new UBSan regression,
but the known _sha3 undefined behavior must be fixed.
(cherry picked from commit 6d0d7d2b8c1e04fd51c6cb29cc09a41b60b97b7b)
* bpo-46913: Skip test_ctypes.test_shorts() on UBSan (GH-31674)
If Python is built with UBSan, test_ctypes now skips test_shorts().
This change allows to run test_ctypes to check for new UBSan regression,
but the known test_shorts() undefined behavior must be fixed.
(cherry picked from commit ad1b04451d3aca2c6fa6dbe2891676a4e0baac49)
(cherry picked from commit 7b5b429adab4fe0fe81858fe3831f06adc2e2141)
The libexpat 2.4.1 upgrade from introduced the following new exported symbols:
* `testingAccountingGetCountBytesDirect`
* `testingAccountingGetCountBytesIndirect`
* `unsignedCharToPrintable`
* `XML_SetBillionLaughsAttackProtectionActivationThreshold`
* `XML_SetBillionLaughsAttackProtectionMaximumAmplification`
We need to adjust [Modules/expat/pyexpatns.h](https://github.com/python/cpython/blob/master/Modules/expat/pyexpatns.h)
(The newer libexpat upgrade has no new symbols).
Automerge-Triggered-By: GH:gpshead
(cherry picked from commit 6312c1052c0186b4596fc45c42fd3ade9f8f5911)
Co-authored-by: Yilei "Dolee" Yang <yileiyang@google.com>
