* gh-139400: Make sure that parent parsers outlive their subparsers in `pyexpat` (#139403)
* Modules/pyexpat.c: Disallow collection of in-use parent parsers.
Within libexpat, a parser created via `XML_ExternalEntityParserCreate`
is relying on its parent parser throughout its entire lifetime.
Prior to this fix, is was possible for the parent parser to be
garbage-collected too early.
(cherry picked from commit 6edb2ddb5f3695cf4938979d645f31d7fba43ec8)
* Move news item to from section "Core and Builtins" to section "Security"
gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361)
From the ERR_raise manpage:
ERR_LIB_SYS
This "library code" indicates that a system error is
being reported. In this case, the reason code given
to `ERR_raise()` and `ERR_raise_data()` *must* be
`errno(3)`.
This PR only handles ERR_LIB_SYS for the high-lever error types
SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where
OpenSSL indicates it has some more information about the issue.
(cherry picked from commit f4b31edf2d9d72878dab1f66a36913b5bcc848ec)
(cherry picked from commit 7f707fa6c67d0bfa9bbc1a9f344b932789659397)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
HEAD_LOCK is called from _PyEval_ReInitThreads->_PyThreadState_DeleteExcept before _PyRuntimeState_ReInitThreads reinit runtime->interpreters.mutex which might be locked before fork.
(cherry picked from commit 522799a05e3e820339718151ac055af6d864d463)
Co-authored-by: ChuBoning <102216855+ChuBoning@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:
- `xml.etree.ElementTree.XMLParser.flush`
- `xml.etree.ElementTree.XMLPullParser.flush`
- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`
- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`
- `xml.sax.expatreader.ExpatParser.flush`
Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 .
Includes code suggested-by: Snild Dolkow <snild@sony.com>
and by core dev Serhiy Storchaka.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-114572: Fix locking in cert_store_stats and get_ca_certs (GH-114573)
* gh-114572: Fix locking in cert_store_stats and get_ca_certs
cert_store_stats and get_ca_certs query the SSLContext's X509_STORE with
X509_STORE_get0_objects, but reading the result requires a lock. See
https://github.com/openssl/openssl/pull/23224 for details.
Instead, use X509_STORE_get1_objects, newly added in that PR.
X509_STORE_get1_objects does not exist in current OpenSSLs, but we can
polyfill it with X509_STORE_lock and X509_STORE_unlock.
* Work around const-correctness problem
* Add missing X509_STORE_get1_objects failure check
* Add blurb
(cherry picked from commit bce693111bff906ccf9281c22371331aaff766ab)
Co-authored-by: David Benjamin <davidben@google.com>
Manual backport due to code differences.
(cherry picked from commit e071b0d558b2f5cddd5a9fc6afadb4ba109ec77e)
Co-authored-by: Seth Michael Larson <seth@python.org>
* Fix a crash when pass UINT_MAX.
* Fix an integer overflow on 64-bit non-Windows platforms.
(cherry picked from commit 0daf555c6fb3feba77989382135a58215e1d70a5)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Previously *consumed was not set in this case.
(cherry picked from commit b8b3e6afc0a48c3cbb7c36d2f73e332edcd6058c)
(cherry picked from commit f08e52ccb027f6f703302b8c1a82db9fd3934270)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Upgrade builds to OpenSSL 1.1.1u.
This OpenSSL version addresses a pile if less-urgent CVEs since 1.1.1t.
The Mac/BuildScript/build-installer.py was already updated.
Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9.
Manual edits to the _ssl_data_300.h file prevent it from removing any existing definitions in case those exist in some peoples builds and were important (avoiding regressions during backporting).
(cherry picked from commit ede89af).
(cherry picked from commit a5d2b546c1b0b73d0695b98838a3ddd497382999)
(cherry picked from commit f90d3f68db720bd6d0deda8cc0030339ccd43858)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment
that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right.
That option causes OpenSSL to treat transport EOF as the same as
close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has
distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is
usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl
module would raise them for transport EOF and close_notify,
respectively. In OpenSSL 3.0, both act like close_notify.
Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING
and mapping that to the other exception type.
There doesn't seem to have been any unit test of this error, so fill in
the missing one. This had to be done with the BIO path because it's
actually slightly tricky to simulate a transport EOF with Python's fd
based APIs. (If you instruct the server to close the socket, it gets
confused, probably because the server's SSL object is still referencing
the now dead fd?)
(cherry picked from commit 420bbb783b43216cc897dc8914851899db37a31d)
Co-authored-by: David Benjamin <davidben@google.com>
gh-102027: Fix macro name (GH-102124)
This fixes the ssse3 / sse2 detection when sse4 is available.
(cherry picked from commit ea93bde4ece139d4152a59f2c38aa6568559447c)
Co-authored-by: Max Bachmann <kontakt@maxbachmann.de>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
(cherry picked from commit c74073657e32b8872f91b3bbe1efa9af20adbea9)
Co-authored-by: Timo Ludwig <ti.ludwig@web.de>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
GH-102397: Fix segfault from race condition in signal handling (GH-102399)
(cherry picked from commit 1a84cc007e207f2dd61f86a7fc3d86632fdce72f)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Fix missing 'is' in cmath.log() docstring.
(cherry picked from commit 71f614ef2a3d66213b9cae807cbbc1ed03741221)
Co-authored-by: Owain Davies <116417456+OTheDev@users.noreply.github.com>
[3.11] gh-99952: [ctypes] fix refcount issues in from_param() result. (GH-100169)
Fixes a reference counting issue with `ctypes.Structure` when a `from_param()` method call is used and the structure size is larger than a C pointer `sizeof(void*)`.
This problem existed for a very long time, but became more apparent in 3.8+ by change likely due to garbage collection cleanup timing changes..
(cherry picked from commit dfad678d7024ab86d265d84ed45999e031a03691)
(cherry picked from commit fa7c37af4936abfe34aa261d6ed9703bc5842ad4)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Yukihiro Nakadaira <yukihiro.nakadaira@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
When getaddrinfo returns an error, the output pointer is in an unknown state
Don't call freeaddrinfo on it. See the issue for discussion and details with
links to reasoning. _Most_ libc getaddrinfo implementations never modify the
output pointer unless they are returning success.
(cherry picked from commit b724ac2fe7fbb5a7a33d639cad8e748f17b325e0)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Sergey G. Brester <github@sebres.de>
Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
Don't send partial UTF-8 sequences to the Windows API
(cherry picked from commit f34176b77f222726d901595968a4b44456186da4)
Co-authored-by: Paul Moore <p.f.moore@gmail.com>
gh-76963: PEP3118 itemsize of an empty ctypes array should not be 0 (GH-5576)
The itemsize returned in a memoryview of a ctypes array is now computed from the item type, instead of dividing the total size by the length and assuming that the length is not zero.
(cherry picked from commit 84bc6a4f25fcf467813ee12b74118f7b1b54e285)
Co-authored-by: Eric Wieser <wieser.eric@gmail.com>
(cherry picked from commit 8dbe08eb7c807f484fe9870f5b7f5ae2881fd966)
Fix double-free bug mentioned at GH-99240, by moving memory clean up out of "exit" label.
It no longer emits a deprecation warning if the current event loop was set.
(cherry picked from commit 3fae04b10e2655a20a3aadb5e0d63e87206d0c67)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
In case if only True/False be supported as boolean arguments in future,
we should continue to support 1/0 here.
(cherry picked from commit 922a6cf6c265e2763a003291885ff74d46203fc3)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
In multiprocessing.shared_memory.SharedMemory(), the temporary view
returned by MapViewOfFile() should be unmapped when it is no longer
needed.
(cherry picked from commit 85c128e34daec7625b74746e127afa25888ccde1)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
