Partially revert commit ac46eb4ad6662cf6d771b20d8963658b2186c48c:
"bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)".
Using a module state per module instance is causing subtle practical
problems.
For example, the Mercurial project replaces the __import__() function
to implement lazy import, whereas Python expected that "import _ast"
always return a fully initialized _ast module.
Add _PyAST_Fini() to clear the state at exit.
The _ast module has no state (set _astmodule.m_size to 0). Remove
astmodule_traverse(), astmodule_clear() and astmodule_free()
functions..
(cherry picked from commit e5fbe0cbd4be99ced5f000ad382208ad2a561c90)
Co-authored-by: Victor Stinner <vstinner@python.org>
Fix GCC 9.3 (using -O3) warnings on x86:
initconfig.c: In function ‘init_dump_ascii_wstr’:
initconfig.c:2679:34: warning: format ‘%lc’ expects argument of type
‘wint_t’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
2679 | PySys_WriteStderr("%lc", ch);
initconfig.c:2682:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
2682 | PySys_WriteStderr("\\x%02x", ch);
initconfig.c:2686:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
2686 | PySys_WriteStderr("\\U%08x", ch);
initconfig.c:2690:38: warning: format ‘%x’ expects argument of type
‘unsigned int’, but argument 2 has type ‘wchar_t’ {aka ‘long int’}
2690 | PySys_WriteStderr("\\u%04x", ch);
(cherry picked from commit 640e8e1d5f61d5868453d992da04bf4741327748)
Co-authored-by: Victor Stinner <vstinner@python.org>
Free the stack allocated in va_build_stack if do_mkstack fails
and the stack is not a small_stack
(cherry picked from commit 75c80b0bda89debf312f075716b8c467d411f90e)
Co-authored-by: Tony Solomonik <tony.solomonik@gmail.com>
* Fixed `f-string/str.format` error description when using two `,` in format specifier.
Co-authored-by: millefalcon <hanish0019@hmail.com>
(cherry picked from commit 0d6aa7f0ee38eb453bc8f73bf4830e6172be2f35)
Co-authored-by: han-solo <hanish0019@gmail.com>
Co-authored-by: han-solo <hanish0019@gmail.com>
* bpo-41194: Convert _ast extension to PEP 489 (GH-21293)
Convert the _ast extension module to PEP 489 "Multiphase
initialization". Replace the global _ast state with a module state.
(cherry picked from commit b1cc6ba73a51d5cc3aeb113b5e7378fb50a0e20a)
* bpo-41204: Fix compiler warning in ast_type_init() (GH-21307)
(cherry picked from commit 1f76453173267887ed05bb3783e862cb22365ae8)
3.8.3 had a regression where compiling with
ast.PyCF_ALLOW_TOP_LEVEL_AWAIT woudl agressively mark things are
coroutine even if there were not.
(cherry picked from commit bd46174)
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Co-authored-by: Matthias Bussonnier <bussonniermatthias@gmail.com>
Also enables using debug build of `python3_d.dll`
Reference: CVE-2020-15523
(cherry picked from commit dcbaa1b49cd9062fb9ba2b9d49555ac6cd8c60b5)
Co-authored-by: Steve Dower <steve.dower@python.org>
* bpo-41194: Pass module state in Python-ast.c (GH-21284)
Rework asdl_c.py to pass the module state to functions in
Python-ast.c, instead of using astmodulestate_global.
Handle also PyState_AddModule() failure in init_types().
(cherry picked from commit 74419f0c64959bb8392fcf3659058410423038e1)
* bpo-41194: The _ast module cannot be loaded more than once (GH-21290)
Fix a crash in the _ast module: it can no longer be loaded more than
once. It now uses a global state rather than a module state.
* Move _ast module state: use a global state instead.
* Set _astmodule.m_size to -1, so the extension cannot be loaded more
than once.
(cherry picked from commit 91e1bc18bd467a13bceb62e16fbc435b33381c82)
This commit changes the parsing of f-string expressions with the new parser. The parser gets pre-fed with the location of the expression itself (not the f-string, which was what we were doing before). This allows us to completely skip the shifting of the AST nodes after the parsing is completed..
(cherry picked from commit 1f0f4abb110b9fbade6175842b6a26ab0b8df6dd)
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 4901ea952691ad70aae21cfe04b6bd363b5a6aff)
Co-authored-by: Christian Heimes <christian@python.org>
Reported by Coverity. (CID 1457554 RETURN_LOCAL)
path0 is assigned as a pointer to this right before it goes out of scope.
(cherry picked from commit 81328f30703bd7225e7e73aedb0994a7293ce190)
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Kyle Stanley <aeros167@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 2c4928d37edc5e4aeec3c0b79fa3460b1ec9b60d)
When a file ends with a line that contains a line continuation character
the text of the emitted SyntaxError is empty, contrary to the old
parser, where the error text contained the text of the last line.
(cherry picked from commit 113e2b0a07c72c0d5e3489076afb14f6b3ad1049)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
On Windows, GH-include "pyerrors.h" no longer defines "snprintf" and
"vsnprintf" macros.
PyOS_snprintf() and PyOS_vsnprintf() should be used to get portable
behavior.
Replace snprintf() calls with PyOS_snprintf() and replace vsnprintf()
calls with PyOS_vsnprintf().
(cherry picked from commit e822e37946f27c09953bb5733acf3b07c2db690f)
Co-authored-by: Victor Stinner <vstinner@python.org>
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 9672912e8f90374fd31b37ca0fb7cefbc6f4c555)
Co-authored-by: Christian Heimes <christian@python.org>
Export explicitly the Py_GetArgcArgv() function to the C API and
document the function. Previously, it was exported implicitly which
no longer works since Python is built with -fvisibility=hidden.
* Add PyConfig._orig_argv member.
* Py_InitializeFromConfig() no longer calls _PyConfig_Write() twice.
* PyConfig_Read() no longer initializes Py_GetArgcArgv(): it is now
_PyConfig_Write() responsibility.
* _PyConfig_Write() result type becomes PyStatus instead of void.
* Write an unit test on Py_GetArgcArgv().
(cherry picked from commit e81f6e687d0f04a45f2389d0b43fafd6d8491624)
* Provide native .files support on SourceFileLoader.
* Add native importlib.resources.files() support to zipimporter. Remove fallback support.
* make regen-all
* 📜🤖 Added by blurb_it.
* Move 'files' into the ResourceReader so it can carry the relevant module name context.
* Create 'importlib.readers' module and add FileReader to it.
* Add zip reader and rely on it for a TraversableResources object on zipimporter.
* Remove TraversableAdapter, no longer needed.
* Update blurb.
* Replace backslashes with forward slashes.
* Incorporate changes from importlib_metadata 2.0, finalizing the interface for extension via get_resource_reader.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit 843c27765652e2322011fb3e5d88f4837de38c06)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
* bpo-40826: Add _Py_EnsureTstateNotNULL() macro (GH-20571)
Add _Py_EnsureTstateNotNULL(tstate) macro: call Py_FatalError() if
tstate is NULL, the error message contains the current function name.
(cherry picked from commit 3026cad59b87751a9215111776cac8e819458fce)
* bpo-40826: PyOS_InterruptOccurred() requires GIL (GH-20578)
PyOS_InterruptOccurred() now fails with a fatal error if it is called
with the GIL released.
(cherry picked from commit cbe129692293251e7fbcea9ff0d822824d90c140)
* Fix failure of _Py_dg_dtoa to remove trailing zeros
* Add regression test and news entry
* Add explanation about why it's safe to strip trailing zeros
* Make code safer, clean up comments, add change note at top of file
* Nitpick: avoid implicit int-to-float conversion in tests
(cherry picked from commit 895c9c1d438367722f74f437fda96767d770662b)
Co-authored-by: Mark Dickinson <mdickinson@enthought.com>
Heap types now always visit the type in tp_traverse. See added docs for details.
This reverts commit 0169d3003be3d072751dd14a5c84748ab63a249f.
Automerge-Triggered-By: @encukou
(cherry picked from commit 1cf15af9a6f28750f37b08c028ada31d38e818dd)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
This updates _PyErr_ChainStackItem() to use _PyErr_SetObject()
instead of _PyErr_ChainExceptions(). This prevents a hang in
certain circumstances because _PyErr_SetObject() performs checks
to prevent cycles in the exception context chain while
_PyErr_ChainExceptions() doesn't.
(cherry picked from commit 7c30d12bd5359b0f66c4fbc98aa055398bcc8a7e)
Co-authored-by: Chris Jerdonek <chris.jerdonek@gmail.com>
Remove --with-experimental-isolated-subinterpreters configure option
in Python 3.9: the experiment continues in the master branch, but
it's no longer needed in 3.9.
When an asyncio.Task is cancelled, the exception traceback now
starts with where the task was first interrupted. Previously,
the traceback only had "depth one."
Clarify the zip built-in docstring.
This puts much simpler text up front along with an example.
As it was, the zip built-in docstring was technically correct. But too
technical for the reader who shouldn't _need_ to know about `__next__` and
`StopIteration` as most people do not need to understand the internal
implementation details of the iterator protocol in their daily life.
This is a documentation only change, intended to be backported to 3.8; it is
only tangentially related to PEP-618 which might offer new behavior options
in the future.
Wording based a bit more on enumerate per Brandt's suggestion.
This gets rid of the legacy wording paragraph which seems too tied to
implementation details of the iterator protocol which isn't relevant here.
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
This fixes both the traceback.py module and the C code for formatting syntax errors (in Python/pythonrun.c). They now both consistently do the following:
- Suppress caret if it points left of text
- Allow caret pointing just past end of line
- If caret points past end of line, clip to *just* past end of line
The syntax error formatting code in traceback.py was mostly rewritten; small, subtle changes were applied to the C code in pythonrun.c.
There's still a difference when the text contains embedded newlines. Neither handles these very well, and I don't think the case occurs in practice.
Automerge-Triggered-By: @gvanrossum
