mirror of
https://github.com/python/cpython.git
synced 2026-01-27 13:15:25 +00:00
Compare commits
464 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
197e4c0628 | ||
|
|
f2b5c206c7 | ||
|
|
17d447e993 | ||
|
|
487bd2dea5 | ||
|
|
3e9a5b022f | ||
|
|
2520617361 | ||
|
|
7febbe6b60 | ||
|
|
9181d776da | ||
|
|
933540e332 | ||
|
|
04d497c284 | ||
|
|
19de10d3d8 | ||
|
|
1f55caf97e | ||
|
|
923d9d2ac2 | ||
|
|
8f459255eb | ||
|
|
decb25e8f0 | ||
|
|
9982147433 | ||
|
|
639c1ad4f1 | ||
|
|
76d3ae71ba | ||
|
|
6e55337f8a | ||
|
|
a51bf70f95 | ||
|
|
979d92fefc | ||
|
|
27246c3482 | ||
|
|
012c498035 | ||
|
|
5f736a0432 | ||
|
|
6d972e0104 | ||
|
|
29840247ff | ||
|
|
6f579147e3 | ||
|
|
ca99bfdefb | ||
|
|
4e10fa993a | ||
|
|
29f1e778fa | ||
|
|
58ccf21cbb | ||
|
|
25a10b60b0 | ||
|
|
2f42f83344 | ||
|
|
70e67f579e | ||
|
|
03e651d601 | ||
|
|
052e55e7d4 | ||
|
|
f3dd0cae6c | ||
|
|
f8262b84f5 | ||
|
|
5b2d49b7da | ||
|
|
ee4e14aa4c | ||
|
|
bcf9cb0217 | ||
|
|
a966d94e76 | ||
|
|
77bf4ba732 | ||
|
|
c5cfcdf16a | ||
|
|
67535ab2d2 | ||
|
|
d77aaa7311 | ||
|
|
fb690c38ca | ||
|
|
c447d1bc14 | ||
|
|
0b5f8359c5 | ||
|
|
3e66171efa | ||
|
|
0b08438ea6 | ||
|
|
6181b69970 | ||
|
|
cf71e34940 | ||
|
|
9eab67d507 | ||
|
|
f52af86cba | ||
|
|
5db331a561 | ||
|
|
4c7ec78092 | ||
|
|
4ef30a5871 | ||
|
|
9060b4abbe | ||
|
|
48795b6460 | ||
|
|
95746b3a13 | ||
|
|
b234a2b675 | ||
|
|
f25509e78e | ||
|
|
6262704b13 | ||
|
|
27a7160b8b | ||
|
|
48b6866047 | ||
|
|
43bb6300b3 | ||
|
|
a126893fa8 | ||
|
|
795d5c5b44 | ||
|
|
31c81ab0a2 | ||
|
|
fe629262c0 | ||
|
|
76b484b9d1 | ||
|
|
fa44efa0ef | ||
|
|
fa3abf5a51 | ||
|
|
71cbffde61 | ||
|
|
05eab96435 | ||
|
|
e66597d6c8 | ||
|
|
f84ea11071 | ||
|
|
3c9c3d33cb | ||
|
|
375e372c66 | ||
|
|
72bacb0cd0 | ||
|
|
17d1490aa9 | ||
|
|
bb2b9ba49d | ||
|
|
7dca4e3af1 | ||
|
|
813fc7a291 | ||
|
|
cb6a662bb0 | ||
|
|
d8ab1c79b0 | ||
|
|
59d3594ca1 | ||
|
|
78b1370de9 | ||
|
|
c879b2a7a5 | ||
|
|
54bedcf714 | ||
|
|
63cc1257db | ||
|
|
7ca9e7ad05 | ||
|
|
f7fceed79c | ||
|
|
b4b73245d8 | ||
|
|
d51c01a271 | ||
|
|
61ec66acd5 | ||
|
|
7e28ae550f | ||
|
|
7d151e552c | ||
|
|
bb25f7280a | ||
|
|
949b5ec8e6 | ||
|
|
3199cfcf76 | ||
|
|
5996636ab5 | ||
|
|
4d5a676aa0 | ||
|
|
3e93225798 | ||
|
|
21ed1e2a94 | ||
|
|
edeebe22cb | ||
|
|
19e64afddf | ||
|
|
780e9692fe | ||
|
|
66680f1230 | ||
|
|
ae53da5758 | ||
|
|
c461aa99e2 | ||
|
|
3514ba2175 | ||
|
|
a009e78b79 | ||
|
|
1597d00d96 | ||
|
|
f5685a266b | ||
|
|
565685f6e8 | ||
|
|
421bd1770a | ||
|
|
bdba5f0db2 | ||
|
|
794f758cd8 | ||
|
|
a73ba4d46e | ||
|
|
b8e925b4f8 | ||
|
|
f4de184980 | ||
|
|
499706b843 | ||
|
|
313d5cde5d | ||
|
|
85013d7a55 | ||
|
|
14f96a8d8f | ||
|
|
2b33b52219 | ||
|
|
d51cc01c19 | ||
|
|
bab1d7a561 | ||
|
|
0e0d51cdce | ||
|
|
a471a32f4b | ||
|
|
e370c8db52 | ||
|
|
6db952eae9 | ||
|
|
ce8f5f98c6 | ||
|
|
94dbce1397 | ||
|
|
7e8a1b5061 | ||
|
|
1857a40807 | ||
|
|
2873c31edf | ||
|
|
dfc66e5c8d | ||
|
|
da0e8c39dc | ||
|
|
fca7fec88c | ||
|
|
865eb12e07 | ||
|
|
80e9eaf071 | ||
|
|
f53a801e91 | ||
|
|
95a17b4a85 | ||
|
|
103a384bfd | ||
|
|
1176facbf2 | ||
|
|
f0a0467c17 | ||
|
|
a7ba3b124f | ||
|
|
43cd277ae9 | ||
|
|
510ab7d6e1 | ||
|
|
e5b5a15804 | ||
|
|
cbf9b8cc08 | ||
|
|
a6bc60da02 | ||
|
|
298d5440eb | ||
|
|
d5882c5b70 | ||
|
|
0bee481576 | ||
|
|
5a45279320 | ||
|
|
c556786b8b | ||
|
|
66e1399311 | ||
|
|
4766237d19 | ||
|
|
c559135c93 | ||
|
|
3d44f0ab65 | ||
|
|
ec254e2b40 | ||
|
|
7d155d7915 | ||
|
|
1de46715ec | ||
|
|
e535bdb0a2 | ||
|
|
c315748060 | ||
|
|
fe78c1e749 | ||
|
|
971f387bbb | ||
|
|
054a565c64 | ||
|
|
42f7c2dfba | ||
|
|
7f50a5febd | ||
|
|
43c76587c1 | ||
|
|
bd83a57463 | ||
|
|
f3759d21dd | ||
|
|
dbd10a6c29 | ||
|
|
5f28aa2f37 | ||
|
|
548526bbbe | ||
|
|
9d13ca97c1 | ||
|
|
265381b7e8 | ||
|
|
620a5b9269 | ||
|
|
9633f95b95 | ||
|
|
23b93770f6 | ||
|
|
e22b68568a | ||
|
|
515ae4078d | ||
|
|
d1282efb2b | ||
|
|
03e6457096 | ||
|
|
75d73c3674 | ||
|
|
718c15fa95 | ||
|
|
aa8578dc54 | ||
|
|
ce6bae92da | ||
|
|
e2f0160026 | ||
|
|
e7f5ffa0de | ||
|
|
78e868fa28 | ||
|
|
95259116ec | ||
|
|
e0fb278064 | ||
|
|
1932127ec7 | ||
|
|
499d3a8d50 | ||
|
|
66bca383bd | ||
|
|
b852236b26 | ||
|
|
61e036691c | ||
|
|
c696f33d9e | ||
|
|
6d6c7ed737 | ||
|
|
ba10100c39 | ||
|
|
a4086d7f89 | ||
|
|
a9ca49d9c6 | ||
|
|
39a2bcf949 | ||
|
|
6d54b6ac7d | ||
|
|
dcdb23f9db | ||
|
|
234a15dc4e | ||
|
|
fd6d41b292 | ||
|
|
af9f783a7e | ||
|
|
68a01f901f | ||
|
|
dfeefbe8ea | ||
|
|
b54a1d272e | ||
|
|
e2f15aec16 | ||
|
|
aeb3403563 | ||
|
|
cea2d2475d | ||
|
|
8cf5c4d89a | ||
|
|
49c3b0a67a | ||
|
|
5462002bbe | ||
|
|
efaa56f73c | ||
|
|
f3e069a7ab | ||
|
|
c07e5ec0a9 | ||
|
|
6c9f7b4406 | ||
|
|
67d3d0344f | ||
|
|
f11f5ebfe6 | ||
|
|
228d95582e | ||
|
|
bfc3d8d77f | ||
|
|
9a3263ff8f | ||
|
|
d04394929b | ||
|
|
51a56a3a7b | ||
|
|
b2827de18f | ||
|
|
b3e4a3462f | ||
|
|
a1eedaee98 | ||
|
|
8565ddd288 | ||
|
|
b866a1c73f | ||
|
|
4fb6a31bce | ||
|
|
0a5c04a5ce | ||
|
|
98e55d70bc | ||
|
|
df355348f0 | ||
|
|
ff7d1cec41 | ||
|
|
51227b6b1a | ||
|
|
74bb3ca1f8 | ||
|
|
e79c9b7031 | ||
|
|
8735daf3e8 | ||
|
|
7b0a372b20 | ||
|
|
90c44bc803 | ||
|
|
faa3dc7c64 | ||
|
|
05406b221d | ||
|
|
54f1ed0299 | ||
|
|
d9c1235db4 | ||
|
|
841b7482dd | ||
|
|
d745b60ef2 | ||
|
|
71119a164c | ||
|
|
dd750b3485 | ||
|
|
efb4e6c733 | ||
|
|
7dae1077cd | ||
|
|
4d21297d28 | ||
|
|
d6f77e6a3f | ||
|
|
12d363bb66 | ||
|
|
04ace41fe2 | ||
|
|
7a572d9f21 | ||
|
|
240a6c3262 | ||
|
|
1569275117 | ||
|
|
4f9a8d075e | ||
|
|
bfac54d861 | ||
|
|
68fcb958eb | ||
|
|
c99f766743 | ||
|
|
e6bfe4d886 | ||
|
|
6116d70bbd | ||
|
|
9609574e7f | ||
|
|
6c53af18f6 | ||
|
|
ef3b8829e4 | ||
|
|
12283f6373 | ||
|
|
3c56f9e2cc | ||
|
|
27434c68f8 | ||
|
|
2c39b9d2f2 | ||
|
|
6d05e55de0 | ||
|
|
e7c542de5f | ||
|
|
abdbe0b807 | ||
|
|
b538c2832d | ||
|
|
136f6d8355 | ||
|
|
f7a03bb944 | ||
|
|
98258326a9 | ||
|
|
0417dabe3f | ||
|
|
ef6f92a2a6 | ||
|
|
864c5985ea | ||
|
|
08a17ed061 | ||
|
|
61f2ad9a3a | ||
|
|
8a2deea1fc | ||
|
|
09ce592499 | ||
|
|
315f474d11 | ||
|
|
18f3c59e57 | ||
|
|
9712dc1d9e | ||
|
|
61fc72a4a4 | ||
|
|
6b9a6c6ec3 | ||
|
|
e5ad7b7694 | ||
|
|
513ae175bb | ||
|
|
5d133351c6 | ||
|
|
faa26044ce | ||
|
|
d00d39f58e | ||
|
|
1fb8e0eb51 | ||
|
|
422ca074bc | ||
|
|
2d9f4e357a | ||
|
|
7f6c16a956 | ||
|
|
c5215978eb | ||
|
|
3c4429f65a | ||
|
|
96ab379dca | ||
|
|
469fe33edd | ||
|
|
04899b8539 | ||
|
|
aa8a43d179 | ||
|
|
7e3a5a7e79 | ||
|
|
0aedf2f9cf | ||
|
|
23ad9c5d01 | ||
|
|
ef834dee89 | ||
|
|
79c03ac001 | ||
|
|
b6b0e14b3d | ||
|
|
6cb245d260 | ||
|
|
f37f57dfe6 | ||
|
|
daa9aa4c0a | ||
|
|
713684de53 | ||
|
|
c3febba73b | ||
|
|
0efbad60e1 | ||
|
|
fa9a4254e8 | ||
|
|
c3bfe5d5aa | ||
|
|
3ca1f2a370 | ||
|
|
3ccc76f036 | ||
|
|
836b2810d5 | ||
|
|
522563549a | ||
|
|
23abbf1f2b | ||
|
|
61ee04834b | ||
|
|
84fcdbd86e | ||
|
|
3a728e5f93 | ||
|
|
00e24b80e0 | ||
|
|
7726119651 | ||
|
|
9976c2b634 | ||
|
|
f5e11facf2 | ||
|
|
1af21ea320 | ||
|
|
57d569942c | ||
|
|
5d1e78f7b5 | ||
|
|
54362898f3 | ||
|
|
9d92ac1225 | ||
|
|
a1c6308346 | ||
|
|
b3f2d80569 | ||
|
|
d3d4cf9432 | ||
|
|
de22e718bb | ||
|
|
888d101445 | ||
|
|
ea3fd785cb | ||
|
|
59ede34c8c | ||
|
|
b9a4806430 | ||
|
|
8611f74e08 | ||
|
|
579c5b496b | ||
|
|
8d46f961c3 | ||
|
|
86d904588e | ||
|
|
cf6758ff9e | ||
|
|
594a4631c3 | ||
|
|
305aff0a66 | ||
|
|
7342890ed7 | ||
|
|
3509fa5a12 | ||
|
|
84b7e6970f | ||
|
|
7c44f37170 | ||
|
|
1e17ccd030 | ||
|
|
d4dc3dd9aa | ||
|
|
e8e044eda3 | ||
|
|
4ee6929d60 | ||
|
|
57937a8e5e | ||
|
|
9af7a20cae | ||
|
|
fc2f0fea6b | ||
|
|
50ecd6b880 | ||
|
|
cc48bf0fde | ||
|
|
cbe0cb779a | ||
|
|
c2202a7e66 | ||
|
|
450e836aef | ||
|
|
20aeb3a463 | ||
|
|
25c294b6ea | ||
|
|
c8b80f5e23 | ||
|
|
f783cc37eb | ||
|
|
6536fab194 | ||
|
|
c4ab024530 | ||
|
|
28da1fb7c0 | ||
|
|
81c8eb85e1 | ||
|
|
f9704f1d84 | ||
|
|
a273bc99d2 | ||
|
|
5b5ee3c4bf | ||
|
|
9e51301234 | ||
|
|
714037ba84 | ||
|
|
be3c131640 | ||
|
|
665d2807a0 | ||
|
|
3c0888b25b | ||
|
|
a88d1b8dab | ||
|
|
700e9fad70 | ||
|
|
487e91c120 | ||
|
|
e728b006de | ||
|
|
9ded3dd4e9 | ||
|
|
ff7f62eb23 | ||
|
|
3960878143 | ||
|
|
6213a512bf | ||
|
|
09044dd42b | ||
|
|
b8d3fddba6 | ||
|
|
8d2d7bb2e7 | ||
|
|
2b4feee648 | ||
|
|
7607712b61 | ||
|
|
3cc57505e5 | ||
|
|
5989095dfd | ||
|
|
5b5263648f | ||
|
|
e46f28c6af | ||
|
|
4ea3c1a047 | ||
|
|
08bc03ff2a | ||
|
|
e2a7db7175 | ||
|
|
6b4bc6e6a2 | ||
|
|
6a4f10325d | ||
|
|
786f464c74 | ||
|
|
049c2526bf | ||
|
|
685272eb8a | ||
|
|
4aef138325 | ||
|
|
610aabfef2 | ||
|
|
220f0b1077 | ||
|
|
1391ee664c | ||
|
|
e79c39101a | ||
|
|
33d94abafd | ||
|
|
f54d44d333 | ||
|
|
e22c49522b | ||
|
|
4a8ecbad80 | ||
|
|
e4058d7cb1 | ||
|
|
14f0b5191a | ||
|
|
d2abd5733b | ||
|
|
0f01530bd5 | ||
|
|
ddfc155d3a | ||
|
|
1c544acaa5 | ||
|
|
71a7cb8887 | ||
|
|
fc80096a07 | ||
|
|
cbc0851ada | ||
|
|
8b64dd853d | ||
|
|
6e625f87d2 | ||
|
|
92243dc62c | ||
|
|
25397f9541 | ||
|
|
77b56eafde | ||
|
|
fba4584ffc | ||
|
|
e61a447d0e | ||
|
|
77c8e6a2b8 | ||
|
|
7d81eab923 | ||
|
|
568a819f67 | ||
|
|
2b466c47c3 | ||
|
|
49627dc991 | ||
|
|
d4095f25e8 | ||
|
|
1fc3039d71 | ||
|
|
c7dcb26520 | ||
|
|
248eb3efb3 | ||
|
|
1e9a0ee682 | ||
|
|
454485e564 | ||
|
|
8307a14d0e | ||
|
|
4fd006e712 | ||
|
|
c35b812e77 | ||
|
|
89729f2ef7 | ||
|
|
4345253981 | ||
|
|
8c87bcd7f2 | ||
|
|
6ee51a36b3 | ||
|
|
92d4aeafd5 | ||
|
|
16a305f152 | ||
|
|
a043407510 | ||
|
|
47ec96f133 |
6
.gitattributes
vendored
6
.gitattributes
vendored
@ -83,8 +83,10 @@ Include/opcode.h generated
|
||||
Include/opcode_ids.h generated
|
||||
Include/token.h generated
|
||||
Lib/_opcode_metadata.py generated
|
||||
Lib/keyword.py generated
|
||||
Lib/idlelib/help.html generated
|
||||
Lib/keyword.py generated
|
||||
Lib/pydoc_data/topics.py generated
|
||||
Lib/pydoc_data/module_docs.py generated
|
||||
Lib/test/certdata/*.pem generated
|
||||
Lib/test/certdata/*.0 generated
|
||||
Lib/test/levenshtein_examples.json generated
|
||||
@ -92,6 +94,8 @@ Lib/test/test_stable_abi_ctypes.py generated
|
||||
Lib/test/test_zoneinfo/data/*.json generated
|
||||
Lib/token.py generated
|
||||
Misc/sbom.spdx.json generated
|
||||
Modules/_testinternalcapi/test_cases.c.h generated
|
||||
Modules/_testinternalcapi/test_targets.h generated
|
||||
Objects/typeslots.inc generated
|
||||
PC/python3dll.c generated
|
||||
Parser/parser.c generated
|
||||
|
||||
20
.github/CODEOWNERS
vendored
20
.github/CODEOWNERS
vendored
@ -63,8 +63,8 @@
|
||||
.azure-pipelines/ @AA-Turner
|
||||
|
||||
# GitHub & related scripts
|
||||
.github/ @ezio-melotti @hugovk @AA-Turner
|
||||
Tools/build/compute-changes.py @AA-Turner
|
||||
.github/ @ezio-melotti @hugovk @AA-Turner @webknjaz
|
||||
Tools/build/compute-changes.py @AA-Turner @hugovk @webknjaz
|
||||
Tools/build/verify_ensurepip_wheels.py @AA-Turner @pfmoore @pradyunsg
|
||||
|
||||
# Pre-commit
|
||||
@ -143,6 +143,9 @@ Misc/externals.spdx.json @sethmlarson
|
||||
Misc/sbom.spdx.json @sethmlarson
|
||||
Tools/build/generate_sbom.py @sethmlarson
|
||||
|
||||
# ABI check
|
||||
Misc/libabigail.abignore @encukou
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------
|
||||
# Platform Support
|
||||
@ -173,9 +176,10 @@ Tools/wasm/config.site-wasm32-emscripten @freakboy3742 @emmatyping
|
||||
Tools/wasm/emscripten @freakboy3742 @emmatyping
|
||||
|
||||
# WebAssembly (WASI)
|
||||
Tools/wasm/wasi-env @brettcannon @emmatyping
|
||||
Tools/wasm/wasi.py @brettcannon @emmatyping
|
||||
Tools/wasm/wasi @brettcannon @emmatyping
|
||||
Platforms/WASI @brettcannon @emmatyping @savannahostrowski
|
||||
Tools/wasm/wasi-env @brettcannon @emmatyping @savannahostrowski
|
||||
Tools/wasm/wasi.py @brettcannon @emmatyping @savannahostrowski
|
||||
Tools/wasm/wasi @brettcannon @emmatyping @savannahostrowski
|
||||
|
||||
# Windows
|
||||
PC/ @python/windows-team
|
||||
@ -290,9 +294,9 @@ InternalDocs/jit.md @brandtbucher @savannahostrowski @diegorusso @AA-T
|
||||
|
||||
# Micro-op / μop / Tier 2 Optimiser
|
||||
Python/optimizer.c @markshannon @Fidget-Spinner
|
||||
Python/optimizer_analysis.c @markshannon @tomasr8 @Fidget-Spinner
|
||||
Python/optimizer_bytecodes.c @markshannon @tomasr8 @Fidget-Spinner
|
||||
Python/optimizer_symbols.c @markshannon @tomasr8 @Fidget-Spinner
|
||||
Python/optimizer_analysis.c @markshannon @tomasr8 @Fidget-Spinner @savannahostrowski
|
||||
Python/optimizer_bytecodes.c @markshannon @tomasr8 @Fidget-Spinner @savannahostrowski
|
||||
Python/optimizer_symbols.c @markshannon @tomasr8 @Fidget-Spinner @savannahostrowski
|
||||
|
||||
# Parser, Lexer, and Grammar
|
||||
Grammar/python.gram @pablogsal @lysnikolaou
|
||||
|
||||
2
.github/workflows/add-issue-header.yml
vendored
2
.github/workflows/add-issue-header.yml
vendored
@ -20,7 +20,7 @@ jobs:
|
||||
issues: write
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/github-script@v7
|
||||
- uses: actions/github-script@v8
|
||||
with:
|
||||
# language=JavaScript
|
||||
script: |
|
||||
|
||||
125
.github/workflows/build.yml
vendored
125
.github/workflows/build.yml
vendored
@ -64,7 +64,7 @@ jobs:
|
||||
run: |
|
||||
apt update && apt install git -yq
|
||||
git config --global --add safe.directory "$GITHUB_WORKSPACE"
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 1
|
||||
persist-credentials: false
|
||||
@ -101,10 +101,10 @@ jobs:
|
||||
needs: build-context
|
||||
if: needs.build-context.outputs.run-tests == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.x'
|
||||
- name: Runner image version
|
||||
@ -142,9 +142,14 @@ jobs:
|
||||
- name: Check for unsupported C global variables
|
||||
if: github.event_name == 'pull_request' # $GITHUB_EVENT_NAME
|
||||
run: make check-c-globals
|
||||
- name: Check for undocumented C APIs
|
||||
run: make check-c-api-docs
|
||||
|
||||
check-c-api-docs:
|
||||
name: C API Docs
|
||||
needs: build-context
|
||||
if: >-
|
||||
needs.build-context.outputs.run-tests == 'true'
|
||||
|| needs.build-context.outputs.run-docs == 'true'
|
||||
uses: ./.github/workflows/reusable-check-c-api-docs.yml
|
||||
|
||||
build-windows:
|
||||
name: >-
|
||||
@ -256,7 +261,7 @@ jobs:
|
||||
# Keep 1.1.1w in our list despite it being upstream EOL and otherwise
|
||||
# unsupported as it most resembles other 1.1.1-work-a-like ssl APIs
|
||||
# supported by important vendors such as AWS-LC.
|
||||
openssl_ver: [1.1.1w, 3.0.18, 3.2.6, 3.3.5, 3.4.3, 3.5.4]
|
||||
openssl_ver: [1.1.1w, 3.0.18, 3.3.5, 3.4.3, 3.5.4, 3.6.0]
|
||||
# See Tools/ssl/make_ssl_data.py for notes on adding a new version
|
||||
env:
|
||||
OPENSSL_VER: ${{ matrix.openssl_ver }}
|
||||
@ -264,7 +269,7 @@ jobs:
|
||||
OPENSSL_DIR: ${{ github.workspace }}/multissl/openssl/${{ matrix.openssl_ver }}
|
||||
LD_LIBRARY_PATH: ${{ github.workspace }}/multissl/openssl/${{ matrix.openssl_ver }}/lib
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Runner image version
|
||||
@ -280,7 +285,7 @@ jobs:
|
||||
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> "$GITHUB_ENV"
|
||||
- name: 'Restore OpenSSL build'
|
||||
id: cache-openssl
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ./multissl/openssl/${{ env.OPENSSL_VER }}
|
||||
key: ${{ matrix.os }}-multissl-openssl-${{ env.OPENSSL_VER }}
|
||||
@ -316,7 +321,7 @@ jobs:
|
||||
OPENSSL_DIR: ${{ github.workspace }}/multissl/aws-lc/${{ matrix.awslc_ver }}
|
||||
LD_LIBRARY_PATH: ${{ github.workspace }}/multissl/aws-lc/${{ matrix.awslc_ver }}/lib
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Runner image version
|
||||
@ -332,7 +337,7 @@ jobs:
|
||||
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/aws-lc/${AWSLC_VER}/lib" >> "$GITHUB_ENV"
|
||||
- name: 'Restore AWS-LC build'
|
||||
id: cache-aws-lc
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ./multissl/aws-lc/${{ matrix.awslc_ver }}
|
||||
key: ${{ matrix.os }}-multissl-aws-lc-${{ matrix.awslc_ver }}
|
||||
@ -381,7 +386,7 @@ jobs:
|
||||
|
||||
runs-on: ${{ matrix.runs-on }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Build and test
|
||||
@ -394,7 +399,7 @@ jobs:
|
||||
timeout-minutes: 60
|
||||
runs-on: macos-14
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@ -426,7 +431,7 @@ jobs:
|
||||
OPENSSL_VER: 3.0.18
|
||||
PYTHONSTRICTEXTENSIONBUILD: 1
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Register gcc problem matcher
|
||||
@ -440,7 +445,7 @@ jobs:
|
||||
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> "$GITHUB_ENV"
|
||||
- name: 'Restore OpenSSL build'
|
||||
id: cache-openssl
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ./multissl/openssl/${{ env.OPENSSL_VER }}
|
||||
key: ${{ runner.os }}-multissl-openssl-${{ env.OPENSSL_VER }}
|
||||
@ -490,7 +495,7 @@ jobs:
|
||||
./python -m venv "$VENV_LOC" && "$VENV_PYTHON" -m pip install -r "${GITHUB_WORKSPACE}/Tools/requirements-hypothesis.txt"
|
||||
- name: 'Restore Hypothesis database'
|
||||
id: cache-hypothesis-database
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ${{ env.CPYTHON_BUILDDIR }}/.hypothesis/
|
||||
key: hypothesis-database-${{ github.head_ref || github.run_id }}
|
||||
@ -517,7 +522,7 @@ jobs:
|
||||
-x test_subprocess \
|
||||
-x test_signal \
|
||||
-x test_sysconfig
|
||||
- uses: actions/upload-artifact@v4
|
||||
- uses: actions/upload-artifact@v6
|
||||
if: always()
|
||||
with:
|
||||
name: hypothesis-example-db
|
||||
@ -538,7 +543,7 @@ jobs:
|
||||
PYTHONSTRICTEXTENSIONBUILD: 1
|
||||
ASAN_OPTIONS: detect_leaks=0:allocator_may_return_null=1:handle_segv=0
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Runner image version
|
||||
@ -548,7 +553,7 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: sudo ./.github/workflows/posix-deps-apt.sh
|
||||
- name: Set up GCC-10 for ASAN
|
||||
uses: egor-tensin/setup-gcc@v1
|
||||
uses: egor-tensin/setup-gcc@v2
|
||||
with:
|
||||
version: 10
|
||||
- name: Configure OpenSSL env vars
|
||||
@ -558,7 +563,7 @@ jobs:
|
||||
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> "$GITHUB_ENV"
|
||||
- name: 'Restore OpenSSL build'
|
||||
id: cache-openssl
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ./multissl/openssl/${{ env.OPENSSL_VER }}
|
||||
key: ${{ matrix.os }}-multissl-openssl-${{ env.OPENSSL_VER }}
|
||||
@ -608,7 +613,7 @@ jobs:
|
||||
needs: build-context
|
||||
if: needs.build-context.outputs.run-ubuntu == 'true'
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Runner image version
|
||||
@ -636,45 +641,45 @@ jobs:
|
||||
run: |
|
||||
"$BUILD_DIR/cross-python/bin/python3" -m test test_sysconfig test_site test_embed
|
||||
|
||||
# CIFuzz job based on https://google.github.io/oss-fuzz/getting-started/continuous-integration/
|
||||
cifuzz:
|
||||
name: CIFuzz
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 60
|
||||
# ${{ '' } is a hack to nest jobs under the same sidebar category.
|
||||
name: CIFuzz${{ '' }} # zizmor: ignore[obfuscation]
|
||||
needs: build-context
|
||||
if: needs.build-context.outputs.run-ci-fuzz == 'true'
|
||||
if: >-
|
||||
needs.build-context.outputs.run-ci-fuzz == 'true'
|
||||
|| needs.build-context.outputs.run-ci-fuzz-stdlib == 'true'
|
||||
permissions:
|
||||
security-events: write
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
sanitizer: [address, undefined, memory]
|
||||
steps:
|
||||
- name: Build fuzzers (${{ matrix.sanitizer }})
|
||||
id: build
|
||||
uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
|
||||
with:
|
||||
oss-fuzz-project-name: cpython3
|
||||
sanitizer: ${{ matrix.sanitizer }}
|
||||
- name: Run fuzzers (${{ matrix.sanitizer }})
|
||||
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
|
||||
with:
|
||||
fuzz-seconds: 600
|
||||
oss-fuzz-project-name: cpython3
|
||||
output-sarif: true
|
||||
sanitizer: ${{ matrix.sanitizer }}
|
||||
- name: Upload crash
|
||||
if: failure() && steps.build.outcome == 'success'
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ${{ matrix.sanitizer }}-artifacts
|
||||
path: ./out/artifacts
|
||||
- name: Upload SARIF
|
||||
if: always() && steps.build.outcome == 'success'
|
||||
uses: github/codeql-action/upload-sarif@v3
|
||||
with:
|
||||
sarif_file: cifuzz-sarif/results.sarif
|
||||
checkout_path: cifuzz-sarif
|
||||
sanitizer:
|
||||
- address
|
||||
- undefined
|
||||
- memory
|
||||
oss-fuzz-project-name:
|
||||
- cpython3
|
||||
- python3-libraries
|
||||
exclude:
|
||||
# Note that the 'no-exclude' sentinel below is to prevent
|
||||
# an empty string value from excluding all jobs and causing
|
||||
# GHA to create a 'default' matrix entry with all empty values.
|
||||
- oss-fuzz-project-name: >-
|
||||
${{
|
||||
needs.build-context.outputs.run-ci-fuzz == 'true'
|
||||
&& 'no-exclude'
|
||||
|| 'cpython3'
|
||||
}}
|
||||
- oss-fuzz-project-name: >-
|
||||
${{
|
||||
needs.build-context.outputs.run-ci-fuzz-stdlib == 'true'
|
||||
&& 'no-exclude'
|
||||
|| 'python3-libraries'
|
||||
}}
|
||||
uses: ./.github/workflows/reusable-cifuzz.yml
|
||||
with:
|
||||
oss-fuzz-project-name: ${{ matrix.oss-fuzz-project-name }}
|
||||
sanitizer: ${{ matrix.sanitizer }}
|
||||
|
||||
all-required-green: # This job does nothing and is only used for the branch protection
|
||||
name: All required checks pass
|
||||
@ -685,13 +690,13 @@ jobs:
|
||||
- check-docs
|
||||
- check-autoconf-regen
|
||||
- check-generated-files
|
||||
- check-c-api-docs
|
||||
- build-windows
|
||||
- build-windows-msi
|
||||
- build-macos
|
||||
- build-ubuntu
|
||||
- build-ubuntu-ssltests-awslc
|
||||
- build-ubuntu-ssltests-openssl
|
||||
- build-android
|
||||
- build-ios
|
||||
- build-wasi
|
||||
- test-hypothesis
|
||||
@ -706,6 +711,7 @@ jobs:
|
||||
uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe
|
||||
with:
|
||||
allowed-failures: >-
|
||||
build-android,
|
||||
build-windows-msi,
|
||||
build-ubuntu-ssltests-awslc,
|
||||
build-ubuntu-ssltests-openssl,
|
||||
@ -721,8 +727,19 @@ jobs:
|
||||
'
|
||||
|| ''
|
||||
}}
|
||||
${{
|
||||
!fromJSON(needs.build-context.outputs.run-tests)
|
||||
&& !fromJSON(needs.build-context.outputs.run-docs)
|
||||
&& 'check-c-api-docs,'
|
||||
|| ''
|
||||
}}
|
||||
${{ !fromJSON(needs.build-context.outputs.run-windows-tests) && 'build-windows,' || '' }}
|
||||
${{ !fromJSON(needs.build-context.outputs.run-ci-fuzz) && 'cifuzz,' || '' }}
|
||||
${{
|
||||
!fromJSON(needs.build-context.outputs.run-ci-fuzz)
|
||||
&& !fromJSON(needs.build-context.outputs.run-ci-fuzz-stdlib)
|
||||
&& 'cifuzz,' ||
|
||||
''
|
||||
}}
|
||||
${{ !fromJSON(needs.build-context.outputs.run-macos) && 'build-macos,' || '' }}
|
||||
${{
|
||||
!fromJSON(needs.build-context.outputs.run-ubuntu)
|
||||
|
||||
20
.github/workflows/jit.yml
vendored
20
.github/workflows/jit.yml
vendored
@ -7,6 +7,7 @@ on:
|
||||
- 'Python/optimizer*.c'
|
||||
- 'Python/executor_cases.c.h'
|
||||
- 'Python/optimizer_cases.c.h'
|
||||
- '**_testinternalcapi**'
|
||||
- '!Python/perf_jit_trampoline.c'
|
||||
- '!**/*.md'
|
||||
- '!**/*.ini'
|
||||
@ -17,6 +18,7 @@ on:
|
||||
- 'Python/optimizer*.c'
|
||||
- 'Python/executor_cases.c.h'
|
||||
- 'Python/optimizer_cases.c.h'
|
||||
- '**_testinternalcapi**'
|
||||
- '!Python/perf_jit_trampoline.c'
|
||||
- '!**/*.md'
|
||||
- '!**/*.ini'
|
||||
@ -38,7 +40,7 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 90
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Build tier two interpreter
|
||||
@ -92,10 +94,10 @@ jobs:
|
||||
architecture: aarch64
|
||||
runner: ubuntu-24.04-arm
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
@ -140,10 +142,10 @@ jobs:
|
||||
llvm:
|
||||
- 21
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.11'
|
||||
- name: Build with JIT enabled and GIL disabled
|
||||
@ -168,10 +170,10 @@ jobs:
|
||||
llvm:
|
||||
- 21
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.11'
|
||||
- name: Build with JIT
|
||||
@ -195,10 +197,10 @@ jobs:
|
||||
llvm:
|
||||
- 21
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.11'
|
||||
- name: Build with JIT and tailcall
|
||||
|
||||
7
.github/workflows/lint.yml
vendored
7
.github/workflows/lint.yml
vendored
@ -19,10 +19,7 @@ jobs:
|
||||
timeout-minutes: 10
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.x"
|
||||
- uses: pre-commit/action@v3.0.1
|
||||
- uses: j178/prek-action@v1
|
||||
|
||||
6
.github/workflows/mypy.yml
vendored
6
.github/workflows/mypy.yml
vendored
@ -26,6 +26,7 @@ on:
|
||||
- "Tools/build/update_file.py"
|
||||
- "Tools/build/verify_ensurepip_wheels.py"
|
||||
- "Tools/cases_generator/**"
|
||||
- "Tools/check-c-api-docs/**"
|
||||
- "Tools/clinic/**"
|
||||
- "Tools/jit/**"
|
||||
- "Tools/peg_generator/**"
|
||||
@ -58,15 +59,16 @@ jobs:
|
||||
"Lib/tomllib",
|
||||
"Tools/build",
|
||||
"Tools/cases_generator",
|
||||
"Tools/check-c-api-docs",
|
||||
"Tools/clinic",
|
||||
"Tools/jit",
|
||||
"Tools/peg_generator",
|
||||
]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3.13"
|
||||
cache: pip
|
||||
|
||||
@ -13,12 +13,12 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/setup-node@v4
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 20
|
||||
- run: npm install mailgun.js form-data
|
||||
- name: Send notification
|
||||
uses: actions/github-script@v7
|
||||
uses: actions/github-script@v8
|
||||
env:
|
||||
MAILGUN_API_KEY: ${{ secrets.MAILGUN_PYTHON_ORG_MAILGUN_KEY }}
|
||||
with:
|
||||
|
||||
25
.github/workflows/reusable-check-c-api-docs.yml
vendored
Normal file
25
.github/workflows/reusable-check-c-api-docs.yml
vendored
Normal file
@ -0,0 +1,25 @@
|
||||
name: Reusable C API Docs Check
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
FORCE_COLOR: 1
|
||||
|
||||
jobs:
|
||||
check-c-api-docs:
|
||||
name: 'Check if all C APIs are documented'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.x'
|
||||
- name: Check for undocumented C APIs
|
||||
run: python Tools/check-c-api-docs/main.py
|
||||
46
.github/workflows/reusable-cifuzz.yml
vendored
Normal file
46
.github/workflows/reusable-cifuzz.yml
vendored
Normal file
@ -0,0 +1,46 @@
|
||||
# CIFuzz job based on https://google.github.io/oss-fuzz/getting-started/continuous-integration/
|
||||
name: Reusable CIFuzz
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
oss-fuzz-project-name:
|
||||
description: OSS-Fuzz project name
|
||||
required: true
|
||||
type: string
|
||||
sanitizer:
|
||||
description: OSS-Fuzz sanitizer
|
||||
required: true
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
cifuzz:
|
||||
name: ${{ inputs.oss-fuzz-project-name }} (${{ inputs.sanitizer }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
- name: Build fuzzers (${{ inputs.sanitizer }})
|
||||
id: build
|
||||
uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
|
||||
with:
|
||||
oss-fuzz-project-name: ${{ inputs.oss-fuzz-project-name }}
|
||||
sanitizer: ${{ inputs.sanitizer }}
|
||||
- name: Run fuzzers (${{ inputs.sanitizer }})
|
||||
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
|
||||
with:
|
||||
fuzz-seconds: 600
|
||||
oss-fuzz-project-name: ${{ inputs.oss-fuzz-project-name }}
|
||||
output-sarif: true
|
||||
sanitizer: ${{ inputs.sanitizer }}
|
||||
- name: Upload crash
|
||||
if: failure() && steps.build.outcome == 'success'
|
||||
uses: actions/upload-artifact@v6
|
||||
with:
|
||||
name: ${{ inputs.sanitizer }}-artifacts
|
||||
path: ./out/artifacts
|
||||
- name: Upload SARIF
|
||||
if: always() && steps.build.outcome == 'success'
|
||||
uses: github/codeql-action/upload-sarif@v4
|
||||
with:
|
||||
sarif_file: cifuzz-sarif/results.sarif
|
||||
checkout_path: cifuzz-sarif
|
||||
10
.github/workflows/reusable-context.yml
vendored
10
.github/workflows/reusable-context.yml
vendored
@ -21,8 +21,11 @@ on: # yamllint disable-line rule:truthy
|
||||
description: Whether to run the Android tests
|
||||
value: ${{ jobs.compute-changes.outputs.run-android }} # bool
|
||||
run-ci-fuzz:
|
||||
description: Whether to run the CIFuzz job
|
||||
description: Whether to run the CIFuzz job for 'cpython' fuzzer
|
||||
value: ${{ jobs.compute-changes.outputs.run-ci-fuzz }} # bool
|
||||
run-ci-fuzz-stdlib:
|
||||
description: Whether to run the CIFuzz job for 'python3-libraries' fuzzer
|
||||
value: ${{ jobs.compute-changes.outputs.run-ci-fuzz-stdlib }} # bool
|
||||
run-docs:
|
||||
description: Whether to build the docs
|
||||
value: ${{ jobs.compute-changes.outputs.run-docs }} # bool
|
||||
@ -56,6 +59,7 @@ jobs:
|
||||
outputs:
|
||||
run-android: ${{ steps.changes.outputs.run-android }}
|
||||
run-ci-fuzz: ${{ steps.changes.outputs.run-ci-fuzz }}
|
||||
run-ci-fuzz-stdlib: ${{ steps.changes.outputs.run-ci-fuzz-stdlib }}
|
||||
run-docs: ${{ steps.changes.outputs.run-docs }}
|
||||
run-ios: ${{ steps.changes.outputs.run-ios }}
|
||||
run-macos: ${{ steps.changes.outputs.run-macos }}
|
||||
@ -66,14 +70,14 @@ jobs:
|
||||
run-windows-tests: ${{ steps.changes.outputs.run-windows-tests }}
|
||||
steps:
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: "3"
|
||||
|
||||
- run: >-
|
||||
echo '${{ github.event_name }}'
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: >-
|
||||
|
||||
12
.github/workflows/reusable-docs.yml
vendored
12
.github/workflows/reusable-docs.yml
vendored
@ -27,7 +27,7 @@ jobs:
|
||||
refspec_pr: '+${{ github.event.pull_request.head.sha }}:remotes/origin/${{ github.event.pull_request.head.ref }}'
|
||||
steps:
|
||||
- name: 'Check out latest PR branch commit'
|
||||
uses: actions/checkout@v4
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
ref: >-
|
||||
@ -52,7 +52,7 @@ jobs:
|
||||
git fetch origin "${refspec_base}" --shallow-since="${DATE}" \
|
||||
--no-tags --prune --no-recurse-submodules
|
||||
- name: 'Set up Python'
|
||||
uses: actions/setup-python@v5
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3'
|
||||
cache: 'pip'
|
||||
@ -82,10 +82,10 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/cache@v4
|
||||
- uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.cache/pip
|
||||
key: ubuntu-doc-${{ hashFiles('Doc/requirements.txt') }}
|
||||
@ -108,11 +108,11 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: 'Set up Python'
|
||||
uses: actions/setup-python@v5
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3'
|
||||
cache: 'pip'
|
||||
|
||||
2
.github/workflows/reusable-macos.yml
vendored
2
.github/workflows/reusable-macos.yml
vendored
@ -28,7 +28,7 @@ jobs:
|
||||
PYTHONSTRICTEXTENSIONBUILD: 1
|
||||
TERM: linux
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Runner image version
|
||||
|
||||
4
.github/workflows/reusable-san.yml
vendored
4
.github/workflows/reusable-san.yml
vendored
@ -26,7 +26,7 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Runner image version
|
||||
@ -99,7 +99,7 @@ jobs:
|
||||
run: find "${GITHUB_WORKSPACE}" -name 'san_log.*' | xargs head -n 1000
|
||||
- name: Archive logs
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
uses: actions/upload-artifact@v6
|
||||
with:
|
||||
name: >-
|
||||
${{ inputs.sanitizer }}-logs-${{
|
||||
|
||||
4
.github/workflows/reusable-ubuntu.yml
vendored
4
.github/workflows/reusable-ubuntu.yml
vendored
@ -31,7 +31,7 @@ jobs:
|
||||
PYTHONSTRICTEXTENSIONBUILD: 1
|
||||
TERM: linux
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Register gcc problem matcher
|
||||
@ -51,7 +51,7 @@ jobs:
|
||||
echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/openssl/${OPENSSL_VER}/lib" >> "$GITHUB_ENV"
|
||||
- name: 'Restore OpenSSL build'
|
||||
id: cache-openssl
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ./multissl/openssl/${{ env.OPENSSL_VER }}
|
||||
key: ${{ inputs.os }}-multissl-openssl-${{ env.OPENSSL_VER }}
|
||||
|
||||
14
.github/workflows/reusable-wasi.yml
vendored
14
.github/workflows/reusable-wasi.yml
vendored
@ -18,7 +18,7 @@ jobs:
|
||||
CROSS_BUILD_PYTHON: cross-build/build
|
||||
CROSS_BUILD_WASI: cross-build/wasm32-wasip1
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
# No problem resolver registered as one doesn't currently exist for Clang.
|
||||
@ -28,7 +28,7 @@ jobs:
|
||||
version: ${{ env.WASMTIME_VERSION }}
|
||||
- name: "Restore WASI SDK"
|
||||
id: cache-wasi-sdk
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: ${{ env.WASI_SDK_PATH }}
|
||||
key: ${{ runner.os }}-wasi-sdk-${{ env.WASI_SDK_VERSION }}
|
||||
@ -41,20 +41,20 @@ jobs:
|
||||
- name: "Add ccache to PATH"
|
||||
run: echo "PATH=/usr/lib/ccache:$PATH" >> "$GITHUB_ENV"
|
||||
- name: "Install Python"
|
||||
uses: actions/setup-python@v5
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.x'
|
||||
- name: "Runner image version"
|
||||
run: echo "IMAGE_OS_VERSION=${ImageOS}-${ImageVersion}" >> "$GITHUB_ENV"
|
||||
- name: "Configure build Python"
|
||||
run: python3 Tools/wasm/wasi configure-build-python -- --config-cache --with-pydebug
|
||||
run: python3 Platforms/WASI configure-build-python -- --config-cache --with-pydebug
|
||||
- name: "Make build Python"
|
||||
run: python3 Tools/wasm/wasi make-build-python
|
||||
run: python3 Platforms/WASI make-build-python
|
||||
- name: "Configure host"
|
||||
# `--with-pydebug` inferred from configure-build-python
|
||||
run: python3 Tools/wasm/wasi configure-host -- --config-cache
|
||||
run: python3 Platforms/WASI configure-host -- --config-cache
|
||||
- name: "Make host"
|
||||
run: python3 Tools/wasm/wasi make-host
|
||||
run: python3 Platforms/WASI make-host
|
||||
- name: "Display build info"
|
||||
run: make --directory "${CROSS_BUILD_WASI}" pythoninfo
|
||||
- name: "Test"
|
||||
|
||||
2
.github/workflows/reusable-windows-msi.yml
vendored
2
.github/workflows/reusable-windows-msi.yml
vendored
@ -23,7 +23,7 @@ jobs:
|
||||
ARCH: ${{ inputs.arch }}
|
||||
IncludeFreethreaded: true
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Build CPython installer
|
||||
|
||||
2
.github/workflows/reusable-windows.yml
vendored
2
.github/workflows/reusable-windows.yml
vendored
@ -26,7 +26,7 @@ jobs:
|
||||
env:
|
||||
ARCH: ${{ inputs.arch }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Register MSVC problem matcher
|
||||
|
||||
29
.github/workflows/tail-call.yml
vendored
29
.github/workflows/tail-call.yml
vendored
@ -72,33 +72,32 @@ jobs:
|
||||
architecture: x86_64
|
||||
runner: ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Native Windows (debug)
|
||||
- name: Native Windows MSVC (release)
|
||||
if: runner.os == 'Windows' && matrix.architecture != 'ARM64'
|
||||
shell: cmd
|
||||
shell: pwsh
|
||||
run: |
|
||||
choco install llvm --allow-downgrade --no-progress --version ${{ matrix.llvm }}.1.0
|
||||
set PlatformToolset=clangcl
|
||||
set LLVMToolsVersion=${{ matrix.llvm }}.1.0
|
||||
set LLVMInstallDir=C:\Program Files\LLVM
|
||||
call ./PCbuild/build.bat --tail-call-interp -d -p ${{ matrix.architecture }}
|
||||
call ./PCbuild/rt.bat -d -p ${{ matrix.architecture }} -q --multiprocess 0 --timeout 4500 --verbose2 --verbose3
|
||||
choco install visualstudio2026buildtools --no-progress -y --force --params "--add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --locale en-US --passive"
|
||||
$env:PATH = "C:\Program Files (x86)\Microsoft Visual Studio\18\BuildTools\MSBuild\Current\bin;$env:PATH"
|
||||
$env:PlatformToolset = "v145"
|
||||
./PCbuild/build.bat --tail-call-interp -c Release -p ${{ matrix.architecture }}
|
||||
./PCbuild/rt.bat -p ${{ matrix.architecture }} -q --multiprocess 0 --timeout 4500 --verbose2 --verbose3
|
||||
|
||||
# No tests (yet):
|
||||
- name: Emulated Windows (release)
|
||||
- name: Emulated Windows Clang (release)
|
||||
if: runner.os == 'Windows' && matrix.architecture == 'ARM64'
|
||||
shell: cmd
|
||||
shell: pwsh
|
||||
run: |
|
||||
choco install llvm --allow-downgrade --no-progress --version ${{ matrix.llvm }}.1.0
|
||||
set PlatformToolset=clangcl
|
||||
set LLVMToolsVersion=${{ matrix.llvm }}.1.0
|
||||
set LLVMInstallDir=C:\Program Files\LLVM
|
||||
$env:PlatformToolset = "clangcl"
|
||||
$env:LLVMToolsVersion = "${{ matrix.llvm }}.1.0"
|
||||
$env:LLVMInstallDir = "C:\Program Files\LLVM"
|
||||
./PCbuild/build.bat --tail-call-interp -p ${{ matrix.architecture }}
|
||||
|
||||
- name: Native macOS (release)
|
||||
|
||||
@ -25,10 +25,10 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-python@v5
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3'
|
||||
- name: Compare checksum of bundled wheels to the ones published on PyPI
|
||||
|
||||
2
.github/zizmor.yml
vendored
2
.github/zizmor.yml
vendored
@ -1,4 +1,4 @@
|
||||
# Configuration for the zizmor static analysis tool, run via pre-commit in CI
|
||||
# Configuration for the zizmor static analysis tool, run via prek in CI
|
||||
# https://woodruffw.github.io/zizmor/configuration/
|
||||
rules:
|
||||
dangerous-triggers:
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
repos:
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.13.2
|
||||
rev: v0.14.10
|
||||
hooks:
|
||||
- id: ruff-check
|
||||
name: Run Ruff (lint) on Apple/
|
||||
@ -40,19 +40,19 @@ repos:
|
||||
files: ^Apple
|
||||
- id: ruff-format
|
||||
name: Run Ruff (format) on Doc/
|
||||
args: [--check]
|
||||
args: [--exit-non-zero-on-fix]
|
||||
files: ^Doc/
|
||||
- id: ruff-format
|
||||
name: Run Ruff (format) on Tools/build/check_warnings.py
|
||||
args: [--check, --config=Tools/build/.ruff.toml]
|
||||
args: [--exit-non-zero-on-fix, --config=Tools/build/.ruff.toml]
|
||||
files: ^Tools/build/check_warnings.py
|
||||
- id: ruff-format
|
||||
name: Run Ruff (format) on Tools/wasm/
|
||||
args: [--check, --config=Tools/wasm/.ruff.toml]
|
||||
args: [--exit-non-zero-on-fix, --config=Tools/wasm/.ruff.toml]
|
||||
files: ^Tools/wasm/
|
||||
|
||||
- repo: https://github.com/psf/black-pre-commit-mirror
|
||||
rev: 25.9.0
|
||||
rev: 25.12.0
|
||||
hooks:
|
||||
- id: black
|
||||
name: Run Black on Tools/jit/
|
||||
@ -83,24 +83,24 @@ repos:
|
||||
files: '^\.github/CODEOWNERS|\.(gram)$'
|
||||
|
||||
- repo: https://github.com/python-jsonschema/check-jsonschema
|
||||
rev: 0.34.0
|
||||
rev: 0.36.0
|
||||
hooks:
|
||||
- id: check-dependabot
|
||||
- id: check-github-workflows
|
||||
- id: check-readthedocs
|
||||
|
||||
- repo: https://github.com/rhysd/actionlint
|
||||
rev: v1.7.7
|
||||
rev: v1.7.9
|
||||
hooks:
|
||||
- id: actionlint
|
||||
|
||||
- repo: https://github.com/woodruffw/zizmor-pre-commit
|
||||
rev: v1.14.1
|
||||
rev: v1.19.0
|
||||
hooks:
|
||||
- id: zizmor
|
||||
|
||||
- repo: https://github.com/sphinx-contrib/sphinx-lint
|
||||
rev: v1.0.0
|
||||
rev: v1.0.2
|
||||
hooks:
|
||||
- id: sphinx-lint
|
||||
args: [--enable=default-role]
|
||||
|
||||
@ -140,7 +140,8 @@ doctest:
|
||||
pydoc-topics: BUILDER = pydoc-topics
|
||||
pydoc-topics: build
|
||||
@echo "Building finished; now run this:" \
|
||||
"cp build/pydoc-topics/topics.py ../Lib/pydoc_data/topics.py"
|
||||
"cp build/pydoc-topics/topics.py ../Lib/pydoc_data/topics.py" \
|
||||
"&& cp build/pydoc-topics/module_docs.py ../Lib/pydoc_data/module_docs.py"
|
||||
|
||||
.PHONY: gettext
|
||||
gettext: BUILDER = gettext
|
||||
|
||||
@ -34,6 +34,23 @@ See :ref:`stable` for a discussion of API and ABI stability across versions.
|
||||
This can be ``0xA`` for alpha, ``0xB`` for beta, ``0xC`` for release
|
||||
candidate or ``0xF`` for final.
|
||||
|
||||
|
||||
.. c:namespace:: NULL
|
||||
.. c:macro:: PY_RELEASE_LEVEL_ALPHA
|
||||
:no-typesetting:
|
||||
.. c:macro:: PY_RELEASE_LEVEL_BETA
|
||||
:no-typesetting:
|
||||
.. c:macro:: PY_RELEASE_LEVEL_GAMMA
|
||||
:no-typesetting:
|
||||
.. c:macro:: PY_RELEASE_LEVEL_FINAL
|
||||
:no-typesetting:
|
||||
|
||||
For completeness, the values are available as macros:
|
||||
:c:macro:`!PY_RELEASE_LEVEL_ALPHA` (``0xA``),
|
||||
:c:macro:`!PY_RELEASE_LEVEL_BETA` (``0xB``),
|
||||
:c:macro:`!PY_RELEASE_LEVEL_GAMMA` (``0xC``), and
|
||||
:c:macro:`!PY_RELEASE_LEVEL_FINAL` (``0xF``).
|
||||
|
||||
.. c:macro:: PY_RELEASE_SERIAL
|
||||
|
||||
The ``2`` in ``3.4.1a2``. Zero for final releases.
|
||||
@ -46,6 +63,12 @@ See :ref:`stable` for a discussion of API and ABI stability across versions.
|
||||
Use this for numeric comparisons, for example,
|
||||
``#if PY_VERSION_HEX >= ...``.
|
||||
|
||||
.. c:macro:: PY_VERSION
|
||||
|
||||
The Python version as a string, for example, ``"3.4.1a2"``.
|
||||
|
||||
These macros are defined in :source:`Include/patchlevel.h`.
|
||||
|
||||
|
||||
Run-time version
|
||||
----------------
|
||||
|
||||
@ -347,6 +347,8 @@ please see individual documentation for details.
|
||||
|
||||
.. versionadded:: 3.9
|
||||
|
||||
.. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
|
||||
:no-typesetting:
|
||||
|
||||
.. c:function:: PyObject* PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames)
|
||||
|
||||
@ -358,7 +360,12 @@ please see individual documentation for details.
|
||||
Return the result of the call on success, or raise an exception and return
|
||||
*NULL* on failure.
|
||||
|
||||
.. versionadded:: 3.9
|
||||
.. versionadded:: 3.8 as ``_PyObject_Vectorcall``
|
||||
|
||||
.. versionchanged:: 3.9
|
||||
|
||||
Renamed to the current name, without the leading underscore.
|
||||
The old provisional name is :term:`soft deprecated`.
|
||||
|
||||
.. c:function:: PyObject* PyObject_VectorcallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict)
|
||||
|
||||
|
||||
@ -69,13 +69,14 @@ bound into a function.
|
||||
The old name is deprecated, but will remain available until the
|
||||
signature changes again.
|
||||
|
||||
.. c:function:: PyCodeObject* PyCode_NewWithPosOnlyArgs(...)
|
||||
:no-typesetting:
|
||||
|
||||
.. c:function:: PyCodeObject* PyUnstable_Code_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount, int nlocals, int stacksize, int flags, PyObject *code, PyObject *consts, PyObject *names, PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *filename, PyObject *name, PyObject *qualname, int firstlineno, PyObject *linetable, PyObject *exceptiontable)
|
||||
|
||||
Similar to :c:func:`PyUnstable_Code_New`, but with an extra "posonlyargcount" for positional-only arguments.
|
||||
The same caveats that apply to ``PyUnstable_Code_New`` also apply to this function.
|
||||
|
||||
.. index:: single: PyCode_NewWithPosOnlyArgs (C function)
|
||||
|
||||
.. versionadded:: 3.8 as ``PyCode_NewWithPosOnlyArgs``
|
||||
|
||||
.. versionchanged:: 3.11
|
||||
@ -298,6 +299,9 @@ These functions are part of the unstable C API tier:
|
||||
this functionality is a CPython implementation detail, and the API
|
||||
may change without deprecation warnings.
|
||||
|
||||
.. c:function:: Py_ssize_t _PyEval_RequestCodeExtraIndex(freefunc free)
|
||||
:no-typesetting:
|
||||
|
||||
.. c:function:: Py_ssize_t PyUnstable_Eval_RequestCodeExtraIndex(freefunc free)
|
||||
|
||||
Return a new opaque index value used to adding data to code objects.
|
||||
@ -310,8 +314,6 @@ may change without deprecation warnings.
|
||||
*free* will be called on non-``NULL`` data stored under the new index.
|
||||
Use :c:func:`Py_DecRef` when storing :c:type:`PyObject`.
|
||||
|
||||
.. index:: single: _PyEval_RequestCodeExtraIndex (C function)
|
||||
|
||||
.. versionadded:: 3.6 as ``_PyEval_RequestCodeExtraIndex``
|
||||
|
||||
.. versionchanged:: 3.12
|
||||
@ -320,6 +322,9 @@ may change without deprecation warnings.
|
||||
The old private name is deprecated, but will be available until the API
|
||||
changes.
|
||||
|
||||
.. c:function:: int _PyCode_GetExtra(PyObject *code, Py_ssize_t index, void **extra)
|
||||
:no-typesetting:
|
||||
|
||||
.. c:function:: int PyUnstable_Code_GetExtra(PyObject *code, Py_ssize_t index, void **extra)
|
||||
|
||||
Set *extra* to the extra data stored under the given index.
|
||||
@ -328,8 +333,6 @@ may change without deprecation warnings.
|
||||
If no data was set under the index, set *extra* to ``NULL`` and return
|
||||
0 without setting an exception.
|
||||
|
||||
.. index:: single: _PyCode_GetExtra (C function)
|
||||
|
||||
.. versionadded:: 3.6 as ``_PyCode_GetExtra``
|
||||
|
||||
.. versionchanged:: 3.12
|
||||
@ -338,13 +341,14 @@ may change without deprecation warnings.
|
||||
The old private name is deprecated, but will be available until the API
|
||||
changes.
|
||||
|
||||
.. c:function:: int _PyCode_SetExtra(PyObject *code, Py_ssize_t index, void *extra)
|
||||
:no-typesetting:
|
||||
|
||||
.. c:function:: int PyUnstable_Code_SetExtra(PyObject *code, Py_ssize_t index, void *extra)
|
||||
|
||||
Set the extra data stored under the given index to *extra*.
|
||||
Return 0 on success. Set an exception and return -1 on failure.
|
||||
|
||||
.. index:: single: _PyCode_SetExtra (C function)
|
||||
|
||||
.. versionadded:: 3.6 as ``_PyCode_SetExtra``
|
||||
|
||||
.. versionchanged:: 3.12
|
||||
|
||||
@ -130,6 +130,8 @@ The following functions provide locale-independent string to number conversions.
|
||||
|
||||
*flags* can be zero or more of the following values or-ed together:
|
||||
|
||||
.. c:namespace:: NULL
|
||||
|
||||
.. c:macro:: Py_DTSF_SIGN
|
||||
|
||||
Always precede the returned string with a sign
|
||||
@ -151,9 +153,21 @@ The following functions provide locale-independent string to number conversions.
|
||||
|
||||
.. versionadded:: 3.11
|
||||
|
||||
If *ptype* is non-``NULL``, then the value it points to will be set to one of
|
||||
``Py_DTST_FINITE``, ``Py_DTST_INFINITE``, or ``Py_DTST_NAN``, signifying that
|
||||
*val* is a finite number, an infinite number, or not a number, respectively.
|
||||
If *ptype* is non-``NULL``, then the value it points to will be set to one
|
||||
of the following constants depending on the type of *val*:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
:align: left
|
||||
|
||||
* - *\*ptype*
|
||||
- type of *val*
|
||||
* - .. c:macro:: Py_DTST_FINITE
|
||||
- finite number
|
||||
* - .. c:macro:: Py_DTST_INFINITE
|
||||
- infinite number
|
||||
* - .. c:macro:: Py_DTST_NAN
|
||||
- not a number
|
||||
|
||||
The return value is a pointer to *buffer* with the converted string or
|
||||
``NULL`` if the conversion failed. The caller is responsible for freeing the
|
||||
|
||||
@ -10,11 +10,6 @@ found in the dictionary of type objects.
|
||||
|
||||
.. XXX document these!
|
||||
|
||||
.. c:var:: PyTypeObject PyProperty_Type
|
||||
|
||||
The type object for the built-in descriptor types.
|
||||
|
||||
|
||||
.. c:function:: PyObject* PyDescr_NewGetSet(PyTypeObject *type, struct PyGetSetDef *getset)
|
||||
|
||||
|
||||
@ -74,9 +69,26 @@ found in the dictionary of type objects.
|
||||
.. c:function:: PyObject* PyWrapper_New(PyObject *, PyObject *)
|
||||
|
||||
|
||||
.. c:macro:: PyDescr_COMMON
|
||||
|
||||
This is a :term:`soft deprecated` macro including the common fields for a
|
||||
descriptor object.
|
||||
|
||||
This was included in Python's C API by mistake; do not use it in extensions.
|
||||
For creating custom descriptor objects, create a class implementing the
|
||||
descriptor protocol (:c:member:`~PyTypeObject.tp_descr_get` and
|
||||
:c:member:`~PyTypeObject.tp_descr_set`).
|
||||
|
||||
|
||||
Built-in descriptors
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. c:var:: PyTypeObject PyProperty_Type
|
||||
|
||||
The type object for property objects. This is the same object as
|
||||
:class:`property` in the Python layer.
|
||||
|
||||
|
||||
.. c:var:: PyTypeObject PySuper_Type
|
||||
|
||||
The type object for super objects. This is the same object as
|
||||
|
||||
@ -793,6 +793,17 @@ Exception Classes
|
||||
Return :c:member:`~PyTypeObject.tp_name` of the exception class *ob*.
|
||||
|
||||
|
||||
.. c:macro:: PyException_HEAD
|
||||
|
||||
This is a :term:`soft deprecated` macro including the base fields for an
|
||||
exception object.
|
||||
|
||||
This was included in Python's C API by mistake and is not designed for use
|
||||
in extensions. For creating custom exception objects, use
|
||||
:c:func:`PyErr_NewException` or otherwise create a class inheriting from
|
||||
:c:data:`PyExc_BaseException`.
|
||||
|
||||
|
||||
Exception Objects
|
||||
=================
|
||||
|
||||
|
||||
@ -131,3 +131,22 @@ the :mod:`io` APIs instead.
|
||||
|
||||
Write string *s* to file object *p*. Return ``0`` on success or ``-1`` on
|
||||
failure; the appropriate exception will be set.
|
||||
|
||||
|
||||
Deprecated API
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
|
||||
These are :term:`soft deprecated` APIs that were included in Python's C API
|
||||
by mistake. They are documented solely for completeness; use other
|
||||
``PyFile*`` APIs instead.
|
||||
|
||||
.. c:function:: PyObject *PyFile_NewStdPrinter(int fd)
|
||||
|
||||
Use :c:func:`PyFile_FromFd` with defaults (``fd, NULL, "w", -1, NULL, NULL, NULL, 0``) instead.
|
||||
|
||||
.. c:var:: PyTypeObject PyStdPrinter_Type
|
||||
|
||||
Type of file-like objects used internally at Python startup when :py:mod:`io` is
|
||||
not yet available.
|
||||
Use Python :py:func:`open` or :c:func:`PyFile_FromFd` to create file objects instead.
|
||||
|
||||
@ -45,6 +45,7 @@ than explicitly calling :c:func:`PyGen_New` or :c:func:`PyGen_NewWithQualName`.
|
||||
A reference to *frame* is stolen by this function. The *frame* argument
|
||||
must not be ``NULL``.
|
||||
|
||||
|
||||
.. c:function:: PyCodeObject* PyGen_GetCode(PyGenObject *gen)
|
||||
|
||||
Return a new :term:`strong reference` to the code object wrapped by *gen*.
|
||||
@ -82,3 +83,14 @@ Asynchronous Generator Objects
|
||||
This function always succeeds.
|
||||
|
||||
.. versionadded:: 3.6
|
||||
|
||||
|
||||
Deprecated API
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
.. c:macro:: PyAsyncGenASend_CheckExact(op)
|
||||
|
||||
This is a :term:`soft deprecated` API that was included in Python's C API
|
||||
by mistake.
|
||||
|
||||
It is solely here for completeness; do not use this API.
|
||||
|
||||
@ -107,6 +107,46 @@ header files properly declare the entry points to be ``extern "C"``. As a result
|
||||
there is no need to do anything special to use the API from C++.
|
||||
|
||||
|
||||
.. _capi-system-includes:
|
||||
|
||||
System includes
|
||||
---------------
|
||||
|
||||
:file:`Python.h` includes several standard header files.
|
||||
C extensions should include the standard headers that they use,
|
||||
and should not rely on these implicit includes.
|
||||
The implicit includes are:
|
||||
|
||||
* ``<assert.h>``
|
||||
* ``<intrin.h>`` (on Windows)
|
||||
* ``<inttypes.h>``
|
||||
* ``<limits.h>``
|
||||
* ``<math.h>``
|
||||
* ``<stdarg.h>``
|
||||
* ``<wchar.h>``
|
||||
* ``<sys/types.h>`` (if present)
|
||||
|
||||
The following are included for backwards compatibility, unless using
|
||||
:ref:`Limited API <limited-c-api>` 3.13 or newer:
|
||||
|
||||
* ``<ctype.h>``
|
||||
* ``<unistd.h>`` (on POSIX)
|
||||
|
||||
The following are included for backwards compatibility, unless using
|
||||
:ref:`Limited API <limited-c-api>` 3.11 or newer:
|
||||
|
||||
* ``<errno.h>``
|
||||
* ``<stdio.h>``
|
||||
* ``<stdlib.h>``
|
||||
* ``<string.h>``
|
||||
|
||||
.. note::
|
||||
|
||||
Since Python may define some pre-processor definitions which affect the standard
|
||||
headers on some systems, you *must* include :file:`Python.h` before any standard
|
||||
headers are included.
|
||||
|
||||
|
||||
Useful macros
|
||||
=============
|
||||
|
||||
@ -182,6 +222,14 @@ complete listing.
|
||||
Equivalent to :c:macro:`Py_LOCAL` but additionally requests the function
|
||||
be inlined.
|
||||
|
||||
.. c:macro:: Py_LOCAL_SYMBOL
|
||||
|
||||
Macro used to declare a symbol as local to the shared library (hidden).
|
||||
On supported platforms, it ensures the symbol is not exported.
|
||||
|
||||
On compatible versions of GCC/Clang, it
|
||||
expands to ``__attribute__((visibility("hidden")))``.
|
||||
|
||||
.. c:macro:: Py_MAX(x, y)
|
||||
|
||||
Return the maximum value between ``x`` and ``y``.
|
||||
@ -336,6 +384,38 @@ complete listing.
|
||||
sizeof(array) / sizeof((array)[0])
|
||||
|
||||
|
||||
.. c:macro:: Py_EXPORTED_SYMBOL
|
||||
|
||||
Macro used to declare a symbol (function or data) as exported.
|
||||
On Windows, this expands to ``__declspec(dllexport)``.
|
||||
On compatible versions of GCC/Clang, it
|
||||
expands to ``__attribute__((visibility("default")))``.
|
||||
This macro is for defining the C API itself; extension modules should not use it.
|
||||
|
||||
|
||||
.. c:macro:: Py_IMPORTED_SYMBOL
|
||||
|
||||
Macro used to declare a symbol as imported.
|
||||
On Windows, this expands to ``__declspec(dllimport)``.
|
||||
This macro is for defining the C API itself; extension modules should not use it.
|
||||
|
||||
|
||||
.. c:macro:: PyAPI_FUNC(type)
|
||||
|
||||
Macro used by CPython to declare a function as part of the C API.
|
||||
Its expansion depends on the platform and build configuration.
|
||||
This macro is intended for defining CPython's C API itself;
|
||||
extension modules should not use it for their own symbols.
|
||||
|
||||
|
||||
.. c:macro:: PyAPI_DATA(type)
|
||||
|
||||
Macro used by CPython to declare a public global variable as part of the C API.
|
||||
Its expansion depends on the platform and build configuration.
|
||||
This macro is intended for defining CPython's C API itself;
|
||||
extension modules should not use it for their own symbols.
|
||||
|
||||
|
||||
.. _api-objects:
|
||||
|
||||
Objects, Types and Reference Counts
|
||||
|
||||
@ -256,6 +256,8 @@ To allocate and free memory, see :ref:`allocating-objects`.
|
||||
collection (i.e., the :c:macro:`Py_TPFLAGS_HAVE_GC` flag is set); this may
|
||||
change in the future.
|
||||
|
||||
.. versionadded:: 3.4
|
||||
|
||||
|
||||
.. c:function:: int PyObject_CallFinalizerFromDealloc(PyObject *op)
|
||||
|
||||
@ -266,6 +268,8 @@ To allocate and free memory, see :ref:`allocating-objects`.
|
||||
should happen. Otherwise, this function returns 0 and destruction can
|
||||
continue normally.
|
||||
|
||||
.. versionadded:: 3.4
|
||||
|
||||
.. seealso::
|
||||
|
||||
:c:member:`~PyTypeObject.tp_dealloc` for example code.
|
||||
|
||||
@ -453,8 +453,8 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
|
||||
|
||||
Otherwise, returns the number of bytes required to store the value.
|
||||
If this is equal to or less than *n_bytes*, the entire value was copied.
|
||||
All *n_bytes* of the buffer are written: large buffers are padded with
|
||||
zeroes.
|
||||
All *n_bytes* of the buffer are written: remaining bytes filled by
|
||||
copies of the sign bit.
|
||||
|
||||
If the returned value is greater than *n_bytes*, the value was
|
||||
truncated: as many of the lowest bits of the value as could fit are written,
|
||||
@ -687,7 +687,7 @@ Export API
|
||||
|
||||
.. versionadded:: 3.14
|
||||
|
||||
.. c:struct:: PyLongLayout
|
||||
.. c:type:: PyLongLayout
|
||||
|
||||
Layout of an array of "digits" ("limbs" in the GMP terminology), used to
|
||||
represent absolute value for arbitrary precision integers.
|
||||
@ -727,7 +727,7 @@ Export API
|
||||
|
||||
Get the native layout of Python :class:`int` objects.
|
||||
|
||||
See the :c:struct:`PyLongLayout` structure.
|
||||
See the :c:type:`PyLongLayout` structure.
|
||||
|
||||
The function must not be called before Python initialization nor after
|
||||
Python finalization. The returned layout is valid until Python is
|
||||
@ -735,7 +735,7 @@ Export API
|
||||
in a process, and so it can be cached.
|
||||
|
||||
|
||||
.. c:struct:: PyLongExport
|
||||
.. c:type:: PyLongExport
|
||||
|
||||
Export of a Python :class:`int` object.
|
||||
|
||||
@ -769,7 +769,7 @@ Export API
|
||||
|
||||
Export a Python :class:`int` object.
|
||||
|
||||
*export_long* must point to a :c:struct:`PyLongExport` structure allocated
|
||||
*export_long* must point to a :c:type:`PyLongExport` structure allocated
|
||||
by the caller. It must not be ``NULL``.
|
||||
|
||||
On success, fill in *\*export_long* and return ``0``.
|
||||
@ -799,7 +799,7 @@ The :c:type:`PyLongWriter` API can be used to import an integer.
|
||||
|
||||
.. versionadded:: 3.14
|
||||
|
||||
.. c:struct:: PyLongWriter
|
||||
.. c:type:: PyLongWriter
|
||||
|
||||
A Python :class:`int` writer instance.
|
||||
|
||||
@ -827,7 +827,7 @@ The :c:type:`PyLongWriter` API can be used to import an integer.
|
||||
The layout of *digits* is described by :c:func:`PyLong_GetNativeLayout`.
|
||||
|
||||
Digits must be in the range [``0``; ``(1 << bits_per_digit) - 1``]
|
||||
(where the :c:struct:`~PyLongLayout.bits_per_digit` is the number of bits
|
||||
(where the :c:type:`~PyLongLayout.bits_per_digit` is the number of bits
|
||||
per digit).
|
||||
Any unused most significant digits must be set to ``0``.
|
||||
|
||||
@ -855,3 +855,31 @@ The :c:type:`PyLongWriter` API can be used to import an integer.
|
||||
If *writer* is ``NULL``, no operation is performed.
|
||||
|
||||
The writer instance and the *digits* array are invalid after the call.
|
||||
|
||||
|
||||
Deprecated API
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
These macros are :term:`soft deprecated`. They describe parameters
|
||||
of the internal representation of :c:type:`PyLongObject` instances.
|
||||
|
||||
Use :c:func:`PyLong_GetNativeLayout` instead, along with :c:func:`PyLong_Export`
|
||||
to read integer data or :c:type:`PyLongWriter` to write it.
|
||||
These currently use the same layout, but are designed to continue working correctly
|
||||
even if CPython's internal integer representation changes.
|
||||
|
||||
|
||||
.. c:macro:: PyLong_SHIFT
|
||||
|
||||
This is equivalent to :c:member:`~PyLongLayout.bits_per_digit` in
|
||||
the output of :c:func:`PyLong_GetNativeLayout`.
|
||||
|
||||
|
||||
.. c:macro:: PyLong_BASE
|
||||
|
||||
This is currently equivalent to :c:expr:`1 << PyLong_SHIFT`.
|
||||
|
||||
|
||||
.. c:macro:: PyLong_MASK
|
||||
|
||||
This is currently equivalent to :c:expr:`(1 << PyLong_SHIFT) - 1`
|
||||
|
||||
@ -293,17 +293,39 @@ The following type-oriented macros are provided for convenience. Note that
|
||||
|
||||
Same as :c:func:`PyMem_Free`.
|
||||
|
||||
In addition, the following macro sets are provided for calling the Python memory
|
||||
allocator directly, without involving the C API functions listed above. However,
|
||||
note that their use does not preserve binary compatibility across Python
|
||||
versions and is therefore deprecated in extension modules.
|
||||
|
||||
* ``PyMem_MALLOC(size)``
|
||||
* ``PyMem_NEW(type, size)``
|
||||
* ``PyMem_REALLOC(ptr, size)``
|
||||
* ``PyMem_RESIZE(ptr, type, size)``
|
||||
* ``PyMem_FREE(ptr)``
|
||||
* ``PyMem_DEL(ptr)``
|
||||
Deprecated aliases
|
||||
------------------
|
||||
|
||||
These are :term:`soft deprecated` aliases to existing functions and macros.
|
||||
They exist solely for backwards compatibility.
|
||||
|
||||
.. list-table::
|
||||
:widths: auto
|
||||
:header-rows: 1
|
||||
|
||||
* * Deprecated alias
|
||||
* Corresponding function or macro
|
||||
* * .. c:macro:: PyMem_MALLOC(size)
|
||||
* :c:func:`PyMem_Malloc`
|
||||
* * .. c:macro:: PyMem_NEW(type, size)
|
||||
* :c:macro:`PyMem_New`
|
||||
* * .. c:macro:: PyMem_REALLOC(ptr, size)
|
||||
* :c:func:`PyMem_Realloc`
|
||||
* * .. c:macro:: PyMem_RESIZE(ptr, type, size)
|
||||
* :c:macro:`PyMem_Resize`
|
||||
* * .. c:macro:: PyMem_FREE(ptr)
|
||||
* :c:func:`PyMem_Free`
|
||||
* * .. c:macro:: PyMem_DEL(ptr)
|
||||
* :c:func:`PyMem_Free`
|
||||
|
||||
.. versionchanged:: 3.4
|
||||
|
||||
The macros are now aliases of the corresponding functions and macros.
|
||||
Previously, their behavior was the same, but their use did not necessarily
|
||||
preserve binary compatibility across Python versions.
|
||||
|
||||
.. deprecated:: 2.0
|
||||
|
||||
|
||||
.. _objectinterface:
|
||||
|
||||
@ -426,10 +426,10 @@ To retrieve the state from a given module, use the following functions:
|
||||
module state.
|
||||
|
||||
|
||||
.. c:function:: int PyModule_GetStateSize(PyObject *, Py_ssize_t *result)
|
||||
.. c:function:: int PyModule_GetStateSize(PyObject *module, Py_ssize_t *result)
|
||||
|
||||
Set *\*result* to the size of the module's state, as specified using
|
||||
:c:macro:`Py_mod_state_size` (or :c:member:`PyModuleDef.m_size`),
|
||||
Set *\*result* to the size of *module*'s state, as specified
|
||||
using :c:macro:`Py_mod_state_size` (or :c:member:`PyModuleDef.m_size`),
|
||||
and return 0.
|
||||
|
||||
On error, set *\*result* to -1, and return -1 with an exception set.
|
||||
@ -571,7 +571,7 @@ A module's token -- and the *your_token* value to use in the above code -- is:
|
||||
of that slot;
|
||||
- For modules created from an ``PyModExport_*``
|
||||
:ref:`export hook <extension-export-hook>`: the slots array that the export
|
||||
hook returned (unless overriden with :c:macro:`Py_mod_token`).
|
||||
hook returned (unless overridden with :c:macro:`Py_mod_token`).
|
||||
|
||||
.. c:macro:: Py_mod_token
|
||||
|
||||
@ -597,7 +597,7 @@ A module's token -- and the *your_token* value to use in the above code -- is:
|
||||
|
||||
.. c:function:: int PyModule_GetToken(PyObject *module, void** result)
|
||||
|
||||
Set *\*result* to the module's token and return 0.
|
||||
Set *\*result* to the module token for *module* and return 0.
|
||||
|
||||
On error, set *\*result* to NULL, and return -1 with an exception set.
|
||||
|
||||
@ -645,7 +645,7 @@ rather than from an extension's :ref:`export hook <extension-export-hook>`.
|
||||
|
||||
.. c:function:: int PyModule_Exec(PyObject *module)
|
||||
|
||||
Execute the :c:data:`Py_mod_exec` slot(s) of the given *module*.
|
||||
Execute the :c:data:`Py_mod_exec` slot(s) of *module*.
|
||||
|
||||
On success, return 0.
|
||||
On error, return -1 with an exception set.
|
||||
@ -820,15 +820,18 @@ struct:
|
||||
.. versionadded:: 3.5
|
||||
|
||||
.. c:macro:: PYTHON_API_VERSION
|
||||
PYTHON_API_STRING
|
||||
|
||||
The C API version. Defined for backwards compatibility.
|
||||
The C API version, as an integer (``1013``) and string (``"1013"``), respectively.
|
||||
Defined for backwards compatibility.
|
||||
|
||||
Currently, this constant is not updated in new Python versions, and is not
|
||||
useful for versioning. This may change in the future.
|
||||
|
||||
.. c:macro:: PYTHON_ABI_VERSION
|
||||
PYTHON_ABI_STRING
|
||||
|
||||
Defined as ``3`` for backwards compatibility.
|
||||
Defined as ``3`` and ``"3"``, respectively, for backwards compatibility.
|
||||
|
||||
Currently, this constant is not updated in new Python versions, and is not
|
||||
useful for versioning. This may change in the future.
|
||||
@ -1018,6 +1021,9 @@ or code that creates modules dynamically.
|
||||
``PyModuleDef`` (such as when using :ref:`multi-phase-initialization`,
|
||||
``PyModule_Create``, or ``PyModule_FromDefAndSpec``).
|
||||
|
||||
Return ``0`` on success.
|
||||
Return ``-1`` with an exception set on error.
|
||||
|
||||
.. versionadded:: 3.5
|
||||
|
||||
.. c:function:: int PyUnstable_Module_SetGIL(PyObject *module, void *gil)
|
||||
|
||||
@ -85,7 +85,7 @@ Object Protocol
|
||||
instead of the :func:`repr`.
|
||||
|
||||
|
||||
.. c:function:: void PyUnstable_Object_Dump(PyObject *op)
|
||||
.. c:function:: void PyObject_Dump(PyObject *op)
|
||||
|
||||
Dump an object *op* to ``stderr``. This should only be used for debugging.
|
||||
|
||||
|
||||
@ -166,3 +166,20 @@ subtypes but not for instances of :class:`frozenset` or its subtypes.
|
||||
Empty an existing set of all elements. Return ``0`` on
|
||||
success. Return ``-1`` and raise :exc:`SystemError` if *set* is not an instance of
|
||||
:class:`set` or its subtype.
|
||||
|
||||
|
||||
Deprecated API
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
.. c:macro:: PySet_MINSIZE
|
||||
|
||||
A :term:`soft deprecated` constant representing the size of an internal
|
||||
preallocated table inside :c:type:`PySetObject` instances.
|
||||
|
||||
This is documented solely for completeness, as there are no guarantees
|
||||
that a given version of CPython uses preallocated tables with a fixed
|
||||
size.
|
||||
In code that does not deal with unstable set internals,
|
||||
:c:macro:`!PySet_MINSIZE` can be replaced with a small constant like ``8``.
|
||||
|
||||
If looking for the size of a set, use :c:func:`PySet_Size` instead.
|
||||
|
||||
@ -1373,6 +1373,9 @@ and :c:data:`PyType_Type` effectively act as defaults.)
|
||||
type structure.
|
||||
|
||||
|
||||
.. c:macro:: _Py_TPFLAGS_HAVE_VECTORCALL
|
||||
:no-typesetting:
|
||||
|
||||
.. c:macro:: Py_TPFLAGS_HAVE_VECTORCALL
|
||||
|
||||
This bit is set when the class implements
|
||||
@ -1384,7 +1387,12 @@ and :c:data:`PyType_Type` effectively act as defaults.)
|
||||
This bit is inherited if :c:member:`~PyTypeObject.tp_call` is also
|
||||
inherited.
|
||||
|
||||
.. versionadded:: 3.9
|
||||
.. versionadded:: 3.8 as ``_Py_TPFLAGS_HAVE_VECTORCALL``
|
||||
|
||||
.. versionchanged:: 3.9
|
||||
|
||||
Renamed to the current name, without the leading underscore.
|
||||
The old provisional name is :term:`soft deprecated`.
|
||||
|
||||
.. versionchanged:: 3.12
|
||||
|
||||
|
||||
@ -65,6 +65,27 @@ Python:
|
||||
.. versionadded:: 3.3
|
||||
|
||||
|
||||
The structure of a particular object can be determined using the following
|
||||
macros.
|
||||
The macros cannot fail; their behavior is undefined if their argument
|
||||
is not a Python Unicode object.
|
||||
|
||||
.. c:namespace:: NULL
|
||||
|
||||
.. c:macro:: PyUnicode_IS_COMPACT(o)
|
||||
|
||||
True if *o* uses the :c:struct:`PyCompactUnicodeObject` structure.
|
||||
|
||||
.. versionadded:: 3.3
|
||||
|
||||
|
||||
.. c:macro:: PyUnicode_IS_COMPACT_ASCII(o)
|
||||
|
||||
True if *o* uses the :c:struct:`PyASCIIObject` structure.
|
||||
|
||||
.. versionadded:: 3.3
|
||||
|
||||
|
||||
The following APIs are C macros and static inlined functions for fast checks and
|
||||
access to internal read-only data of Unicode objects:
|
||||
|
||||
|
||||
9
Doc/data/stable_abi.dat
generated
9
Doc/data/stable_abi.dat
generated
@ -389,8 +389,14 @@ func,PyList_SetSlice,3.2,,
|
||||
func,PyList_Size,3.2,,
|
||||
func,PyList_Sort,3.2,,
|
||||
data,PyList_Type,3.2,,
|
||||
type,PyLongExport,3.15,,full-abi
|
||||
type,PyLongLayout,3.15,,full-abi
|
||||
type,PyLongObject,3.2,,opaque
|
||||
data,PyLongRangeIter_Type,3.2,,
|
||||
type,PyLongWriter,3.15,,opaque
|
||||
func,PyLongWriter_Create,3.15,,
|
||||
func,PyLongWriter_Discard,3.15,,
|
||||
func,PyLongWriter_Finish,3.15,,
|
||||
func,PyLong_AsDouble,3.2,,
|
||||
func,PyLong_AsInt,3.13,,
|
||||
func,PyLong_AsInt32,3.14,,
|
||||
@ -409,6 +415,8 @@ func,PyLong_AsUnsignedLongLong,3.2,,
|
||||
func,PyLong_AsUnsignedLongLongMask,3.2,,
|
||||
func,PyLong_AsUnsignedLongMask,3.2,,
|
||||
func,PyLong_AsVoidPtr,3.2,,
|
||||
func,PyLong_Export,3.15,,
|
||||
func,PyLong_FreeExport,3.15,,
|
||||
func,PyLong_FromDouble,3.2,,
|
||||
func,PyLong_FromInt32,3.14,,
|
||||
func,PyLong_FromInt64,3.14,,
|
||||
@ -425,6 +433,7 @@ func,PyLong_FromUnsignedLongLong,3.2,,
|
||||
func,PyLong_FromUnsignedNativeBytes,3.14,,
|
||||
func,PyLong_FromVoidPtr,3.2,,
|
||||
func,PyLong_GetInfo,3.2,,
|
||||
func,PyLong_GetNativeLayout,3.15,,
|
||||
data,PyLong_Type,3.2,,
|
||||
macro,PyMODEXPORT_FUNC,3.15,,
|
||||
data,PyMap_Type,3.2,,
|
||||
|
||||
@ -7,6 +7,8 @@ Deprecations
|
||||
|
||||
.. include:: pending-removal-in-3.17.rst
|
||||
|
||||
.. include:: pending-removal-in-3.18.rst
|
||||
|
||||
.. include:: pending-removal-in-3.19.rst
|
||||
|
||||
.. include:: pending-removal-in-3.20.rst
|
||||
|
||||
@ -33,16 +33,6 @@ Pending removal in Python 3.15
|
||||
|
||||
* ``load_module()`` method: use ``exec_module()`` instead.
|
||||
|
||||
* :class:`locale`:
|
||||
|
||||
* The :func:`~locale.getdefaultlocale` function
|
||||
has been deprecated since Python 3.11.
|
||||
Its removal was originally planned for Python 3.13 (:gh:`90817`),
|
||||
but has been postponed to Python 3.15.
|
||||
Use :func:`~locale.getlocale`, :func:`~locale.setlocale`,
|
||||
and :func:`~locale.getencoding` instead.
|
||||
(Contributed by Hugo van Kemenade in :gh:`111187`.)
|
||||
|
||||
* :mod:`pathlib`:
|
||||
|
||||
* :meth:`!.PurePath.is_reserved`
|
||||
|
||||
9
Doc/deprecations/pending-removal-in-3.18.rst
Normal file
9
Doc/deprecations/pending-removal-in-3.18.rst
Normal file
@ -0,0 +1,9 @@
|
||||
Pending removal in Python 3.18
|
||||
------------------------------
|
||||
|
||||
* :mod:`decimal`:
|
||||
|
||||
* The non-standard and undocumented :class:`~decimal.Decimal` format
|
||||
specifier ``'N'``, which is only supported in the :mod:`!decimal` module's
|
||||
C implementation, has been deprecated since Python 3.13.
|
||||
(Contributed by Serhiy Storchaka in :gh:`89902`.)
|
||||
@ -1,9 +1,9 @@
|
||||
Pending removal in Python 3.20
|
||||
------------------------------
|
||||
|
||||
* The ``__version__`` attribute has been deprecated in these standard library
|
||||
modules and will be removed in Python 3.20.
|
||||
Use :py:data:`sys.version_info` instead.
|
||||
* The ``__version__``, ``version`` and ``VERSION`` attributes have been
|
||||
deprecated in these standard library modules and will be removed in
|
||||
Python 3.20. Use :py:data:`sys.version_info` instead.
|
||||
|
||||
- :mod:`argparse`
|
||||
- :mod:`csv`
|
||||
@ -24,6 +24,9 @@ Pending removal in Python 3.20
|
||||
- :mod:`tkinter.font`
|
||||
- :mod:`tkinter.ttk`
|
||||
- :mod:`wsgiref.simple_server`
|
||||
- :mod:`xml.etree.ElementTree`
|
||||
- :mod:`!xml.sax.expatreader`
|
||||
- :mod:`xml.sax.handler`
|
||||
- :mod:`zlib`
|
||||
|
||||
(Contributed by Hugo van Kemenade and Stan Ulbrych in :gh:`76007`.)
|
||||
|
||||
@ -3,154 +3,20 @@
|
||||
|
||||
.. _extending-intro:
|
||||
|
||||
******************************
|
||||
Extending Python with C or C++
|
||||
******************************
|
||||
********************************
|
||||
Using the C API: Assorted topics
|
||||
********************************
|
||||
|
||||
It is quite easy to add new built-in modules to Python, if you know how to
|
||||
program in C. Such :dfn:`extension modules` can do two things that can't be
|
||||
done directly in Python: they can implement new built-in object types, and they
|
||||
can call C library functions and system calls.
|
||||
|
||||
To support extensions, the Python API (Application Programmers Interface)
|
||||
defines a set of functions, macros and variables that provide access to most
|
||||
aspects of the Python run-time system. The Python API is incorporated in a C
|
||||
source file by including the header ``"Python.h"``.
|
||||
|
||||
The compilation of an extension module depends on its intended use as well as on
|
||||
your system setup; details are given in later chapters.
|
||||
|
||||
.. note::
|
||||
|
||||
The C extension interface is specific to CPython, and extension modules do
|
||||
not work on other Python implementations. In many cases, it is possible to
|
||||
avoid writing C extensions and preserve portability to other implementations.
|
||||
For example, if your use case is calling C library functions or system calls,
|
||||
you should consider using the :mod:`ctypes` module or the `cffi
|
||||
<https://cffi.readthedocs.io/>`_ library rather than writing
|
||||
custom C code.
|
||||
These modules let you write Python code to interface with C code and are more
|
||||
portable between implementations of Python than writing and compiling a C
|
||||
extension module.
|
||||
|
||||
|
||||
.. _extending-simpleexample:
|
||||
|
||||
A Simple Example
|
||||
================
|
||||
|
||||
Let's create an extension module called ``spam`` (the favorite food of Monty
|
||||
Python fans...) and let's say we want to create a Python interface to the C
|
||||
library function :c:func:`system` [#]_. This function takes a null-terminated
|
||||
character string as argument and returns an integer. We want this function to
|
||||
be callable from Python as follows:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
>>> status = spam.system("ls -l")
|
||||
|
||||
Begin by creating a file :file:`spammodule.c`. (Historically, if a module is
|
||||
called ``spam``, the C file containing its implementation is called
|
||||
:file:`spammodule.c`; if the module name is very long, like ``spammify``, the
|
||||
module name can be just :file:`spammify.c`.)
|
||||
|
||||
The first two lines of our file can be::
|
||||
|
||||
#define PY_SSIZE_T_CLEAN
|
||||
#include <Python.h>
|
||||
|
||||
which pulls in the Python API (you can add a comment describing the purpose of
|
||||
the module and a copyright notice if you like).
|
||||
|
||||
.. note::
|
||||
|
||||
Since Python may define some pre-processor definitions which affect the standard
|
||||
headers on some systems, you *must* include :file:`Python.h` before any standard
|
||||
headers are included.
|
||||
|
||||
``#define PY_SSIZE_T_CLEAN`` was used to indicate that ``Py_ssize_t`` should be
|
||||
used in some APIs instead of ``int``.
|
||||
It is not necessary since Python 3.13, but we keep it here for backward compatibility.
|
||||
See :ref:`arg-parsing-string-and-buffers` for a description of this macro.
|
||||
|
||||
All user-visible symbols defined by :file:`Python.h` have a prefix of ``Py`` or
|
||||
``PY``, except those defined in standard header files.
|
||||
|
||||
.. tip::
|
||||
|
||||
For backward compatibility, :file:`Python.h` includes several standard header files.
|
||||
C extensions should include the standard headers that they use,
|
||||
and should not rely on these implicit includes.
|
||||
If using the limited C API version 3.13 or newer, the implicit includes are:
|
||||
|
||||
* ``<assert.h>``
|
||||
* ``<intrin.h>`` (on Windows)
|
||||
* ``<inttypes.h>``
|
||||
* ``<limits.h>``
|
||||
* ``<math.h>``
|
||||
* ``<stdarg.h>``
|
||||
* ``<wchar.h>``
|
||||
* ``<sys/types.h>`` (if present)
|
||||
|
||||
If :c:macro:`Py_LIMITED_API` is not defined, or is set to version 3.12 or older,
|
||||
the headers below are also included:
|
||||
|
||||
* ``<ctype.h>``
|
||||
* ``<unistd.h>`` (on POSIX)
|
||||
|
||||
If :c:macro:`Py_LIMITED_API` is not defined, or is set to version 3.10 or older,
|
||||
the headers below are also included:
|
||||
|
||||
* ``<errno.h>``
|
||||
* ``<stdio.h>``
|
||||
* ``<stdlib.h>``
|
||||
* ``<string.h>``
|
||||
|
||||
The next thing we add to our module file is the C function that will be called
|
||||
when the Python expression ``spam.system(string)`` is evaluated (we'll see
|
||||
shortly how it ends up being called)::
|
||||
|
||||
static PyObject *
|
||||
spam_system(PyObject *self, PyObject *args)
|
||||
{
|
||||
const char *command;
|
||||
int sts;
|
||||
|
||||
if (!PyArg_ParseTuple(args, "s", &command))
|
||||
return NULL;
|
||||
sts = system(command);
|
||||
return PyLong_FromLong(sts);
|
||||
}
|
||||
|
||||
There is a straightforward translation from the argument list in Python (for
|
||||
example, the single expression ``"ls -l"``) to the arguments passed to the C
|
||||
function. The C function always has two arguments, conventionally named *self*
|
||||
and *args*.
|
||||
|
||||
The *self* argument points to the module object for module-level functions;
|
||||
for a method it would point to the object instance.
|
||||
|
||||
The *args* argument will be a pointer to a Python tuple object containing the
|
||||
arguments. Each item of the tuple corresponds to an argument in the call's
|
||||
argument list. The arguments are Python objects --- in order to do anything
|
||||
with them in our C function we have to convert them to C values. The function
|
||||
:c:func:`PyArg_ParseTuple` in the Python API checks the argument types and
|
||||
converts them to C values. It uses a template string to determine the required
|
||||
types of the arguments as well as the types of the C variables into which to
|
||||
store the converted values. More about this later.
|
||||
|
||||
:c:func:`PyArg_ParseTuple` returns true (nonzero) if all arguments have the right
|
||||
type and its components have been stored in the variables whose addresses are
|
||||
passed. It returns false (zero) if an invalid argument list was passed. In the
|
||||
latter case it also raises an appropriate exception so the calling function can
|
||||
return ``NULL`` immediately (as we saw in the example).
|
||||
The :ref:`tutorial <first-extension-module>` walked you through
|
||||
creating a C API extension module, but left many areas unexplained.
|
||||
This document looks at several concepts that you'll need to learn
|
||||
in order to write more complex extensions.
|
||||
|
||||
|
||||
.. _extending-errors:
|
||||
|
||||
Intermezzo: Errors and Exceptions
|
||||
=================================
|
||||
Errors and Exceptions
|
||||
=====================
|
||||
|
||||
An important convention throughout the Python interpreter is the following: when
|
||||
a function fails, it should set an exception condition and return an error value
|
||||
@ -321,194 +187,14 @@ call to :c:func:`PyErr_SetString` as shown below::
|
||||
}
|
||||
|
||||
|
||||
.. _backtoexample:
|
||||
|
||||
Back to the Example
|
||||
===================
|
||||
|
||||
Going back to our example function, you should now be able to understand this
|
||||
statement::
|
||||
|
||||
if (!PyArg_ParseTuple(args, "s", &command))
|
||||
return NULL;
|
||||
|
||||
It returns ``NULL`` (the error indicator for functions returning object pointers)
|
||||
if an error is detected in the argument list, relying on the exception set by
|
||||
:c:func:`PyArg_ParseTuple`. Otherwise the string value of the argument has been
|
||||
copied to the local variable :c:data:`!command`. This is a pointer assignment and
|
||||
you are not supposed to modify the string to which it points (so in Standard C,
|
||||
the variable :c:data:`!command` should properly be declared as ``const char
|
||||
*command``).
|
||||
|
||||
The next statement is a call to the Unix function :c:func:`system`, passing it
|
||||
the string we just got from :c:func:`PyArg_ParseTuple`::
|
||||
|
||||
sts = system(command);
|
||||
|
||||
Our :func:`!spam.system` function must return the value of :c:data:`!sts` as a
|
||||
Python object. This is done using the function :c:func:`PyLong_FromLong`. ::
|
||||
|
||||
return PyLong_FromLong(sts);
|
||||
|
||||
In this case, it will return an integer object. (Yes, even integers are objects
|
||||
on the heap in Python!)
|
||||
|
||||
If you have a C function that returns no useful argument (a function returning
|
||||
:c:expr:`void`), the corresponding Python function must return ``None``. You
|
||||
need this idiom to do so (which is implemented by the :c:macro:`Py_RETURN_NONE`
|
||||
macro)::
|
||||
|
||||
Py_INCREF(Py_None);
|
||||
return Py_None;
|
||||
|
||||
:c:data:`Py_None` is the C name for the special Python object ``None``. It is a
|
||||
genuine Python object rather than a ``NULL`` pointer, which means "error" in most
|
||||
contexts, as we have seen.
|
||||
|
||||
|
||||
.. _methodtable:
|
||||
|
||||
The Module's Method Table and Initialization Function
|
||||
=====================================================
|
||||
|
||||
I promised to show how :c:func:`!spam_system` is called from Python programs.
|
||||
First, we need to list its name and address in a "method table"::
|
||||
|
||||
static PyMethodDef spam_methods[] = {
|
||||
...
|
||||
{"system", spam_system, METH_VARARGS,
|
||||
"Execute a shell command."},
|
||||
...
|
||||
{NULL, NULL, 0, NULL} /* Sentinel */
|
||||
};
|
||||
|
||||
Note the third entry (``METH_VARARGS``). This is a flag telling the interpreter
|
||||
the calling convention to be used for the C function. It should normally always
|
||||
be ``METH_VARARGS`` or ``METH_VARARGS | METH_KEYWORDS``; a value of ``0`` means
|
||||
that an obsolete variant of :c:func:`PyArg_ParseTuple` is used.
|
||||
|
||||
When using only ``METH_VARARGS``, the function should expect the Python-level
|
||||
parameters to be passed in as a tuple acceptable for parsing via
|
||||
:c:func:`PyArg_ParseTuple`; more information on this function is provided below.
|
||||
|
||||
The :c:macro:`METH_KEYWORDS` bit may be set in the third field if keyword
|
||||
arguments should be passed to the function. In this case, the C function should
|
||||
accept a third ``PyObject *`` parameter which will be a dictionary of keywords.
|
||||
Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a
|
||||
function.
|
||||
|
||||
The method table must be referenced in the module definition structure::
|
||||
|
||||
static struct PyModuleDef spam_module = {
|
||||
...
|
||||
.m_methods = spam_methods,
|
||||
...
|
||||
};
|
||||
|
||||
This structure, in turn, must be passed to the interpreter in the module's
|
||||
initialization function. The initialization function must be named
|
||||
:c:func:`!PyInit_name`, where *name* is the name of the module, and should be the
|
||||
only non-\ ``static`` item defined in the module file::
|
||||
|
||||
PyMODINIT_FUNC
|
||||
PyInit_spam(void)
|
||||
{
|
||||
return PyModuleDef_Init(&spam_module);
|
||||
}
|
||||
|
||||
Note that :c:macro:`PyMODINIT_FUNC` declares the function as ``PyObject *`` return type,
|
||||
declares any special linkage declarations required by the platform, and for C++
|
||||
declares the function as ``extern "C"``.
|
||||
|
||||
:c:func:`!PyInit_spam` is called when each interpreter imports its module
|
||||
:mod:`!spam` for the first time. (See below for comments about embedding Python.)
|
||||
A pointer to the module definition must be returned via :c:func:`PyModuleDef_Init`,
|
||||
so that the import machinery can create the module and store it in ``sys.modules``.
|
||||
|
||||
When embedding Python, the :c:func:`!PyInit_spam` function is not called
|
||||
automatically unless there's an entry in the :c:data:`PyImport_Inittab` table.
|
||||
To add the module to the initialization table, use :c:func:`PyImport_AppendInittab`,
|
||||
optionally followed by an import of the module::
|
||||
|
||||
#define PY_SSIZE_T_CLEAN
|
||||
#include <Python.h>
|
||||
|
||||
int
|
||||
main(int argc, char *argv[])
|
||||
{
|
||||
PyStatus status;
|
||||
PyConfig config;
|
||||
PyConfig_InitPythonConfig(&config);
|
||||
|
||||
/* Add a built-in module, before Py_Initialize */
|
||||
if (PyImport_AppendInittab("spam", PyInit_spam) == -1) {
|
||||
fprintf(stderr, "Error: could not extend in-built modules table\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
/* Pass argv[0] to the Python interpreter */
|
||||
status = PyConfig_SetBytesString(&config, &config.program_name, argv[0]);
|
||||
if (PyStatus_Exception(status)) {
|
||||
goto exception;
|
||||
}
|
||||
|
||||
/* Initialize the Python interpreter. Required.
|
||||
If this step fails, it will be a fatal error. */
|
||||
status = Py_InitializeFromConfig(&config);
|
||||
if (PyStatus_Exception(status)) {
|
||||
goto exception;
|
||||
}
|
||||
PyConfig_Clear(&config);
|
||||
|
||||
/* Optionally import the module; alternatively,
|
||||
import can be deferred until the embedded script
|
||||
imports it. */
|
||||
PyObject *pmodule = PyImport_ImportModule("spam");
|
||||
if (!pmodule) {
|
||||
PyErr_Print();
|
||||
fprintf(stderr, "Error: could not import module 'spam'\n");
|
||||
}
|
||||
|
||||
// ... use Python C API here ...
|
||||
|
||||
return 0;
|
||||
|
||||
exception:
|
||||
PyConfig_Clear(&config);
|
||||
Py_ExitStatusException(status);
|
||||
}
|
||||
|
||||
.. note::
|
||||
|
||||
If you declare a global variable or a local static one, the module may
|
||||
experience unintended side-effects on re-initialisation, for example when
|
||||
removing entries from ``sys.modules`` or importing compiled modules into
|
||||
multiple interpreters within a process
|
||||
(or following a :c:func:`fork` without an intervening :c:func:`exec`).
|
||||
If module state is not yet fully :ref:`isolated <isolating-extensions-howto>`,
|
||||
authors should consider marking the module as having no support for subinterpreters
|
||||
(via :c:macro:`Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED`).
|
||||
|
||||
A more substantial example module is included in the Python source distribution
|
||||
as :file:`Modules/xxlimited.c`. This file may be used as a template or simply
|
||||
read as an example.
|
||||
|
||||
|
||||
.. _compilation:
|
||||
|
||||
Compilation and Linkage
|
||||
=======================
|
||||
Embedding an extension
|
||||
======================
|
||||
|
||||
There are two more things to do before you can use your new extension: compiling
|
||||
and linking it with the Python system. If you use dynamic loading, the details
|
||||
may depend on the style of dynamic loading your system uses; see the chapters
|
||||
about building extension modules (chapter :ref:`building`) and additional
|
||||
information that pertains only to building on Windows (chapter
|
||||
:ref:`building-on-windows`) for more information about this.
|
||||
|
||||
If you can't use dynamic loading, or if you want to make your module a permanent
|
||||
If you want to make your module a permanent
|
||||
part of the Python interpreter, you will have to change the configuration setup
|
||||
and rebuild the interpreter. Luckily, this is very simple on Unix: just place
|
||||
and rebuild the interpreter. On Unix, place
|
||||
your file (:file:`spammodule.c` for example) in the :file:`Modules/` directory
|
||||
of an unpacked source distribution, add a line to the file
|
||||
:file:`Modules/Setup.local` describing your file:
|
||||
@ -536,7 +222,7 @@ on the line in the configuration file as well, for instance:
|
||||
Calling Python Functions from C
|
||||
===============================
|
||||
|
||||
So far we have concentrated on making C functions callable from Python. The
|
||||
The tutorial concentrated on making C functions callable from Python. The
|
||||
reverse is also useful: calling Python functions from C. This is especially the
|
||||
case for libraries that support so-called "callback" functions. If a C
|
||||
interface makes use of callbacks, the equivalent Python often needs to provide a
|
||||
@ -581,7 +267,7 @@ be part of a module definition::
|
||||
}
|
||||
|
||||
This function must be registered with the interpreter using the
|
||||
:c:macro:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The
|
||||
:c:macro:`METH_VARARGS` flag in :c:type:`PyMethodDef.ml_flags`. The
|
||||
:c:func:`PyArg_ParseTuple` function and its arguments are documented in section
|
||||
:ref:`parsetuple`.
|
||||
|
||||
@ -676,14 +362,21 @@ the above example, we use :c:func:`Py_BuildValue` to construct the dictionary. :
|
||||
Py_DECREF(result);
|
||||
|
||||
|
||||
.. index:: single: PyArg_ParseTuple (C function)
|
||||
|
||||
.. _parsetuple:
|
||||
|
||||
Extracting Parameters in Extension Functions
|
||||
============================================
|
||||
|
||||
.. index:: single: PyArg_ParseTuple (C function)
|
||||
The :ref:`tutorial <first-extension-module>` uses a ":c:data:`METH_O`"
|
||||
function, which is limited to a single Python argument.
|
||||
If you want more, you can use :c:data:`METH_VARARGS` instead.
|
||||
With this flag, the C function will receive a :py:class:`tuple` of arguments
|
||||
instead of a single object.
|
||||
|
||||
The :c:func:`PyArg_ParseTuple` function is declared as follows::
|
||||
For unpacking the tuple, CPython provides the :c:func:`PyArg_ParseTuple`
|
||||
function, declared as follows::
|
||||
|
||||
int PyArg_ParseTuple(PyObject *arg, const char *format, ...);
|
||||
|
||||
@ -693,6 +386,19 @@ whose syntax is explained in :ref:`arg-parsing` in the Python/C API Reference
|
||||
Manual. The remaining arguments must be addresses of variables whose type is
|
||||
determined by the format string.
|
||||
|
||||
For example, to receive a single Python :py:class:`str` object and turn it
|
||||
into a C buffer, you would use ``"s"`` as the format string::
|
||||
|
||||
const char *command;
|
||||
if (!PyArg_ParseTuple(args, "s", &command)) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
If an error is detected in the argument list, :c:func:`!PyArg_ParseTuple`
|
||||
returns ``NULL`` (the error indicator for functions returning object pointers);
|
||||
your function may return ``NULL``, relying on the exception set by
|
||||
:c:func:`PyArg_ParseTuple`.
|
||||
|
||||
Note that while :c:func:`PyArg_ParseTuple` checks that the Python arguments have
|
||||
the required types, it cannot check the validity of the addresses of C variables
|
||||
passed to the call: if you make mistakes there, your code will probably crash or
|
||||
@ -703,7 +409,6 @@ Note that any Python object references which are provided to the caller are
|
||||
|
||||
Some example calls::
|
||||
|
||||
#define PY_SSIZE_T_CLEAN
|
||||
#include <Python.h>
|
||||
|
||||
::
|
||||
@ -773,6 +478,17 @@ Some example calls::
|
||||
Keyword Parameters for Extension Functions
|
||||
==========================================
|
||||
|
||||
If you also want your function to accept
|
||||
:term:`keyword arguments <keyword argument>`, use the :c:data:`METH_KEYWORDS`
|
||||
flag in combination with :c:data:`METH_VARARGS`.
|
||||
(:c:data:`!METH_KEYWORDS` can also be used with other flags; see its
|
||||
documentation for the allowed combinations.)
|
||||
|
||||
In this case, the C function should accept a third ``PyObject *`` parameter
|
||||
which will be a dictionary of keywords.
|
||||
Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a
|
||||
function.
|
||||
|
||||
.. index:: single: PyArg_ParseTupleAndKeywords (C function)
|
||||
|
||||
The :c:func:`PyArg_ParseTupleAndKeywords` function is declared as follows::
|
||||
@ -833,19 +549,6 @@ Philbrick (philbrick@hks.com)::
|
||||
{NULL, NULL, 0, NULL} /* sentinel */
|
||||
};
|
||||
|
||||
static struct PyModuleDef keywdarg_module = {
|
||||
.m_base = PyModuleDef_HEAD_INIT,
|
||||
.m_name = "keywdarg",
|
||||
.m_size = 0,
|
||||
.m_methods = keywdarg_methods,
|
||||
};
|
||||
|
||||
PyMODINIT_FUNC
|
||||
PyInit_keywdarg(void)
|
||||
{
|
||||
return PyModuleDef_Init(&keywdarg_module);
|
||||
}
|
||||
|
||||
|
||||
.. _buildvalue:
|
||||
|
||||
@ -986,11 +689,11 @@ needed. Ownership of a reference can be transferred. There are three ways to
|
||||
dispose of an owned reference: pass it on, store it, or call :c:func:`Py_DECREF`.
|
||||
Forgetting to dispose of an owned reference creates a memory leak.
|
||||
|
||||
It is also possible to :dfn:`borrow` [#]_ a reference to an object. The
|
||||
It is also possible to :dfn:`borrow` [#borrow]_ a reference to an object. The
|
||||
borrower of a reference should not call :c:func:`Py_DECREF`. The borrower must
|
||||
not hold on to the object longer than the owner from which it was borrowed.
|
||||
Using a borrowed reference after the owner has disposed of it risks using freed
|
||||
memory and should be avoided completely [#]_.
|
||||
memory and should be avoided completely [#dont-check-refcount]_.
|
||||
|
||||
The advantage of borrowing over owning a reference is that you don't need to
|
||||
take care of disposing of the reference on all possible paths through the code
|
||||
@ -1169,7 +872,7 @@ checking.
|
||||
|
||||
The C function calling mechanism guarantees that the argument list passed to C
|
||||
functions (``args`` in the examples) is never ``NULL`` --- in fact it guarantees
|
||||
that it is always a tuple [#]_.
|
||||
that it is always a tuple [#old-calling-convention]_.
|
||||
|
||||
It is a severe error to ever let a ``NULL`` pointer "escape" to the Python user.
|
||||
|
||||
@ -1226,8 +929,8 @@ the module whose functions one wishes to call might not have been loaded yet!
|
||||
Portability therefore requires not to make any assumptions about symbol
|
||||
visibility. This means that all symbols in extension modules should be declared
|
||||
``static``, except for the module's initialization function, in order to
|
||||
avoid name clashes with other extension modules (as discussed in section
|
||||
:ref:`methodtable`). And it means that symbols that *should* be accessible from
|
||||
avoid name clashes with other extension modules. And it means that symbols
|
||||
that *should* be accessible from
|
||||
other extension modules must be exported in a different way.
|
||||
|
||||
Python provides a special mechanism to pass C-level information (pointers) from
|
||||
@ -1269,8 +972,9 @@ file corresponding to the module provides a macro that takes care of importing
|
||||
the module and retrieving its C API pointers; client modules only have to call
|
||||
this macro before accessing the C API.
|
||||
|
||||
The exporting module is a modification of the :mod:`!spam` module from section
|
||||
:ref:`extending-simpleexample`. The function :func:`!spam.system` does not call
|
||||
The exporting module is a modification of the :mod:`!spam` module from the
|
||||
:ref:`tutorial <first-extension-module>`.
|
||||
The function :func:`!spam.system` does not call
|
||||
the C library function :c:func:`system` directly, but a function
|
||||
:c:func:`!PySpam_System`, which would of course do something more complicated in
|
||||
reality (such as adding "spam" to every command). This function
|
||||
@ -1412,15 +1116,14 @@ code distribution).
|
||||
|
||||
.. rubric:: Footnotes
|
||||
|
||||
.. [#] An interface for this function already exists in the standard module :mod:`os`
|
||||
--- it was chosen as a simple and straightforward example.
|
||||
.. [#borrow] The metaphor of "borrowing" a reference is not completely correct:
|
||||
the owner still has a copy of the reference.
|
||||
|
||||
.. [#] The metaphor of "borrowing" a reference is not completely correct: the owner
|
||||
still has a copy of the reference.
|
||||
|
||||
.. [#] Checking that the reference count is at least 1 **does not work** --- the
|
||||
.. [#dont-check-refcount] Checking that the reference count is at least 1
|
||||
**does not work** --- the
|
||||
reference count itself could be in freed memory and may thus be reused for
|
||||
another object!
|
||||
|
||||
.. [#] These guarantees don't hold when you use the "old" style calling convention ---
|
||||
.. [#old-calling-convention] These guarantees don't hold when you use the
|
||||
"old" style calling convention ---
|
||||
this is still found in much existing code.
|
||||
|
||||
667
Doc/extending/first-extension-module.rst
Normal file
667
Doc/extending/first-extension-module.rst
Normal file
@ -0,0 +1,667 @@
|
||||
.. highlight:: c
|
||||
|
||||
|
||||
.. _extending-simpleexample:
|
||||
.. _first-extension-module:
|
||||
|
||||
*********************************
|
||||
Your first C API extension module
|
||||
*********************************
|
||||
|
||||
This tutorial will take you through creating a simple
|
||||
Python extension module written in C or C++.
|
||||
|
||||
We will use the low-level Python C API directly.
|
||||
For easier ways to create extension modules, see
|
||||
the :ref:`recommended third party tools <c-api-tools>`.
|
||||
|
||||
The tutorial assumes basic knowledge about Python: you should be able to
|
||||
define functions in Python code before starting to write them in C.
|
||||
See :ref:`tutorial-index` for an introduction to Python itself.
|
||||
|
||||
The tutorial should be approachable for anyone who can write a basic C library.
|
||||
While we will mention several concepts that a C beginner would not be expected
|
||||
to know, like ``static`` functions or linkage declarations, understanding these
|
||||
is not necessary for success.
|
||||
|
||||
We will focus on giving you a "feel" of what Python's C API is like.
|
||||
It will not teach you important concepts, like error handling
|
||||
and reference counting, which are covered in later chapters.
|
||||
|
||||
We will assume that you use a Unix-like system (including macOS and
|
||||
Linux), or Windows.
|
||||
On other systems, you might need to adjust some details -- for example,
|
||||
a system command name.
|
||||
|
||||
You need to have a suitable C compiler and Python development headers installed.
|
||||
On Linux, headers are often in a package like ``python3-dev``
|
||||
or ``python3-devel``.
|
||||
|
||||
You need to be able to install Python packages.
|
||||
This tutorial uses `pip <https://pip.pypa.io/>`__ (``pip install``), but you
|
||||
can substitute any tool that can build and install ``pyproject.toml``-based
|
||||
projects, like `uv <https://docs.astral.sh/uv/>`_ (``uv pip install``).
|
||||
Preferably, have a :ref:`virtual environment <venv-def>` activated.
|
||||
|
||||
|
||||
.. note::
|
||||
|
||||
This tutorial uses APIs that were added in CPython 3.15.
|
||||
To create an extension that's compatible with earlier versions of CPython,
|
||||
please follow an earlier version of this documentation.
|
||||
|
||||
This tutorial uses C syntax added in C11 and C++20.
|
||||
If your extension needs to be compatible with earlier standards,
|
||||
please follow tutorials in documentation for Python 3.14 or below.
|
||||
|
||||
|
||||
What we'll do
|
||||
=============
|
||||
|
||||
Let's create an extension module called ``spam`` [#why-spam]_,
|
||||
which will include a Python interface to the C
|
||||
standard library function :c:func:`system`.
|
||||
This function is defined in ``stdlib.h``.
|
||||
It takes a C string as argument, runs the argument as a system
|
||||
command, and returns a result value as an integer.
|
||||
A manual page for :c:func:`system` might summarize it this way::
|
||||
|
||||
#include <stdlib.h>
|
||||
int system(const char *command);
|
||||
|
||||
Note that like many functions in the C standard library,
|
||||
this function is already exposed in Python.
|
||||
In production, use :py:func:`os.system` or :py:func:`subprocess.run`
|
||||
rather than the module you'll write here.
|
||||
|
||||
We want this function to be callable from Python as follows:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
>>> status = spam.system("whoami")
|
||||
User Name
|
||||
>>> status
|
||||
0
|
||||
|
||||
.. note::
|
||||
|
||||
The system command ``whoami`` prints out your username.
|
||||
It's useful in tutorials like this one because it has the same name on
|
||||
both Unix and Windows.
|
||||
|
||||
|
||||
Start with the headers
|
||||
======================
|
||||
|
||||
Begin by creating a directory for this tutorial, and switching to it
|
||||
on the command line.
|
||||
Then, create a file named :file:`spammodule.c` in your directory.
|
||||
[#why-spammodule]_
|
||||
|
||||
In this file, we'll include two headers: :file:`Python.h` to pull in
|
||||
all declarations of the Python C API, and :file:`stdlib.h` for the
|
||||
:c:func:`system` function. [#stdlib-h]_
|
||||
|
||||
Add the following lines to :file:`spammodule.c`:
|
||||
|
||||
.. literalinclude:: ../includes/capi-extension/spammodule-01.c
|
||||
:start-at: <Python.h>
|
||||
:end-at: <stdlib.h>
|
||||
|
||||
Be sure to put :file:`stdlib.h`, and any other standard library includes,
|
||||
*after* :file:`Python.h`.
|
||||
On some systems, Python may define some pre-processor definitions
|
||||
that affect the standard headers.
|
||||
|
||||
|
||||
Running your build tool
|
||||
=======================
|
||||
|
||||
With only the includes in place, your extension won't do anything.
|
||||
Still, it's a good time to compile it and try to import it.
|
||||
This will ensure that your build tool works, so that you can make
|
||||
and test incremental changes as you follow the rest of the text.
|
||||
|
||||
CPython itself does not come with a tool to build extension modules;
|
||||
it is recommended to use a third-party project for this.
|
||||
In this tutorial, we'll use `meson-python`_.
|
||||
(If you want to use another one, see :ref:`first-extension-other-tools`.)
|
||||
|
||||
.. at the time of writing, meson-python has the least overhead for a
|
||||
simple extension using PyModExport.
|
||||
Change this if another tool makes things easier.
|
||||
|
||||
``meson-python`` requires defining a "project" using two extra files.
|
||||
|
||||
First, add ``pyproject.toml`` with these contents:
|
||||
|
||||
.. code-block:: toml
|
||||
|
||||
[build-system]
|
||||
build-backend = 'mesonpy'
|
||||
requires = ['meson-python']
|
||||
|
||||
[project]
|
||||
# Placeholder project information
|
||||
# (change this before distributing the module)
|
||||
name = 'sampleproject'
|
||||
version = '0'
|
||||
|
||||
Then, create ``meson.build`` containing the following:
|
||||
|
||||
.. code-block:: meson
|
||||
|
||||
project('sampleproject', 'c')
|
||||
|
||||
py = import('python').find_installation(pure: false)
|
||||
|
||||
py.extension_module(
|
||||
'spam', # name of the importable Python module
|
||||
'spammodule.c', # the C source file
|
||||
install: true,
|
||||
)
|
||||
|
||||
.. note::
|
||||
|
||||
See `meson-python documentation <meson-python>`_ for details on
|
||||
configuration.
|
||||
|
||||
Now, build install the *project in the current directory* (``.``) via ``pip``:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python -m pip install .
|
||||
|
||||
.. tip::
|
||||
|
||||
If you don't have ``pip`` installed, run ``python -m ensurepip``,
|
||||
preferably in a :ref:`virtual environment <venv-def>`.
|
||||
(Or, if you prefer another tool that can build and install
|
||||
``pyproject.toml``-based projects, use that.)
|
||||
|
||||
.. _meson-python: https://mesonbuild.com/meson-python/
|
||||
.. _virtual environment: https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments
|
||||
|
||||
Note that you will need to run this command again every time you change your
|
||||
extension.
|
||||
Unlike Python, C has an explicit compilation step.
|
||||
|
||||
When your extension is compiled and installed, start Python and try to
|
||||
import it.
|
||||
This should fail with the following exception:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
ImportError: dynamic module does not define module export function (PyModExport_spam or PyInit_spam)
|
||||
|
||||
|
||||
Module export hook
|
||||
==================
|
||||
|
||||
The exception you got when you tried to import the module told you that Python
|
||||
is looking for a "module export function", also known as a
|
||||
:ref:`module export hook <extension-export-hook>`.
|
||||
Let's define one.
|
||||
|
||||
First, add a prototype below the ``#include`` lines:
|
||||
|
||||
.. literalinclude:: ../includes/capi-extension/spammodule-01.c
|
||||
:start-after: /// Export hook prototype
|
||||
:end-before: ///
|
||||
|
||||
.. tip::
|
||||
The prototype is not strictly necessary, but some modern compilers emit
|
||||
warnings without it.
|
||||
It's generally better to add the prototype than to disable the warning.
|
||||
|
||||
The :c:macro:`PyMODEXPORT_FUNC` macro declares the function's
|
||||
return type, and adds any special linkage declarations needed
|
||||
to make the function visible and usable when CPython loads it.
|
||||
|
||||
After the prototype, add the function itself.
|
||||
For now, make it return ``NULL``:
|
||||
|
||||
.. code-block:: c
|
||||
|
||||
PyMODEXPORT_FUNC
|
||||
PyModExport_spam(void)
|
||||
{
|
||||
return NULL;
|
||||
}
|
||||
|
||||
Compile and load the module again.
|
||||
You should get a different error this time.
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
SystemError: module export hook for module 'spam' failed without setting an exception
|
||||
|
||||
Simply returning ``NULL`` is *not* correct behavior for an export hook,
|
||||
and CPython complains about it.
|
||||
That's good -- it means that CPython found the function!
|
||||
Let's now make it do something useful.
|
||||
|
||||
|
||||
The slot table
|
||||
==============
|
||||
|
||||
Rather than ``NULL``, the export hook should return the information needed to
|
||||
create a module.
|
||||
Let's start with the basics: the name and docstring.
|
||||
|
||||
The information should be defined in a ``static`` array of
|
||||
:c:type:`PyModuleDef_Slot` entries, which are essentially key-value pairs.
|
||||
Define this array just before your export hook:
|
||||
|
||||
.. code-block:: c
|
||||
|
||||
static PyModuleDef_Slot spam_slots[] = {
|
||||
{Py_mod_name, "spam"},
|
||||
{Py_mod_doc, "A wonderful module with an example function"},
|
||||
{0, NULL}
|
||||
};
|
||||
|
||||
For both :c:data:`Py_mod_name` and :c:data:`Py_mod_doc`, the values are C
|
||||
strings -- that is, NUL-terminated, UTF-8 encoded byte arrays.
|
||||
|
||||
Note the zero-filled sentinel entry at the end.
|
||||
If you forget it, you'll trigger undefined behavior.
|
||||
|
||||
The array is defined as ``static`` -- that is, not visible outside this ``.c`` file.
|
||||
This will be a common theme.
|
||||
CPython only needs to access the export hook; all global variables
|
||||
and all other functions should generally be ``static``, so that they don't
|
||||
clash with other extensions.
|
||||
|
||||
Return this array from your export hook instead of ``NULL``:
|
||||
|
||||
.. code-block:: c
|
||||
:emphasize-lines: 4
|
||||
|
||||
PyMODEXPORT_FUNC
|
||||
PyModExport_spam(void)
|
||||
{
|
||||
return spam_slots;
|
||||
}
|
||||
|
||||
Now, recompile and try it out:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
>>> print(spam)
|
||||
<module 'spam' from '/home/encukou/dev/cpython/spam.so'>
|
||||
|
||||
You have an extension module!
|
||||
Try ``help(spam)`` to see the docstring.
|
||||
|
||||
The next step will be adding a function.
|
||||
|
||||
|
||||
.. _backtoexample:
|
||||
|
||||
Exposing a function
|
||||
===================
|
||||
|
||||
To expose the :c:func:`system` C function directly to Python,
|
||||
we'll need to write a layer of glue code to convert arguments from Python
|
||||
objects to C values, and the C return value back to Python.
|
||||
|
||||
One of the simplest ways to write glue code is a ":c:data:`METH_O`" function,
|
||||
which takes two Python objects and returns one.
|
||||
All Python objects -- regardless of the Python type -- are represented in C
|
||||
as pointers to the :c:type:`PyObject` structure.
|
||||
|
||||
Add such a function above the slots array::
|
||||
|
||||
static PyObject *
|
||||
spam_system(PyObject *self, PyObject *arg)
|
||||
{
|
||||
Py_RETURN_NONE;
|
||||
}
|
||||
|
||||
For now, we ignore the arguments, and use the :c:macro:`Py_RETURN_NONE`
|
||||
macro, which expands to a ``return`` statement that properly returns
|
||||
a Python :py:data:`None` object.
|
||||
|
||||
Recompile your extension to make sure you don't have syntax errors.
|
||||
We haven't yet added ``spam_system`` to the module, so you might get a
|
||||
warning that ``spam_system`` is unused.
|
||||
|
||||
.. _methodtable:
|
||||
|
||||
Method definitions
|
||||
------------------
|
||||
|
||||
To expose the C function to Python, you will need to provide several pieces of
|
||||
information in a structure called
|
||||
:c:type:`PyMethodDef` [#why-pymethoddef]_:
|
||||
|
||||
* ``ml_name``: the name of the Python function;
|
||||
* ``ml_doc``: a docstring;
|
||||
* ``ml_meth``: the C function to be called; and
|
||||
* ``ml_flags``: a set of flags describing details like how Python arguments are
|
||||
passed to the C function.
|
||||
We'll use :c:data:`METH_O` here -- the flag that matches our
|
||||
``spam_system`` function's signature.
|
||||
|
||||
Because modules typically create several functions, these definitions
|
||||
need to be collected in an array, with a zero-filled sentinel at the end.
|
||||
Add this array just below the ``spam_system`` function:
|
||||
|
||||
.. literalinclude:: ../includes/capi-extension/spammodule-01.c
|
||||
:start-after: /// Module method table
|
||||
:end-before: ///
|
||||
|
||||
As with module slots, a zero-filled sentinel marks the end of the array.
|
||||
|
||||
Next, we'll add the method to the module.
|
||||
Add a :c:data:`Py_mod_methods` slot to your :c:type:`PyMethodDef` array:
|
||||
|
||||
.. literalinclude:: ../includes/capi-extension/spammodule-01.c
|
||||
:start-after: /// Module slot table
|
||||
:end-before: ///
|
||||
:emphasize-lines: 5
|
||||
|
||||
Recompile your extension again, and test it.
|
||||
Be sure to restart the Python interpreter, so that ``import spam`` picks
|
||||
up the new version of the module.
|
||||
|
||||
You should now be able to call the function:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
>>> print(spam.system)
|
||||
<built-in function system>
|
||||
>>> print(spam.system('whoami'))
|
||||
None
|
||||
|
||||
Note that our ``spam.system`` does not yet run the ``whoami`` command;
|
||||
it only returns ``None``.
|
||||
|
||||
Check that the function accepts exactly one argument, as specified by
|
||||
the :c:data:`METH_O` flag:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> print(spam.system('too', 'many', 'arguments'))
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
TypeError: spam.system() takes exactly one argument (3 given)
|
||||
|
||||
|
||||
Returning an integer
|
||||
====================
|
||||
|
||||
Now, let's take a look at the return value.
|
||||
Instead of ``None``, we'll want ``spam.system`` to return a number -- that is,
|
||||
a Python :py:type:`int` object.
|
||||
Eventually this will be the exit code of a system command,
|
||||
but let's start with a fixed value, say, ``3``.
|
||||
|
||||
The Python C API provides a function to create a Python :py:type:`int` object
|
||||
from a C ``int`` value: :c:func:`PyLong_FromLong`. [#why-pylongfromlong]_
|
||||
|
||||
To call it, replace the ``Py_RETURN_NONE`` with the following 3 lines:
|
||||
|
||||
.. this could be a one-liner, but we want to show the data types here
|
||||
|
||||
.. code-block:: c
|
||||
:emphasize-lines: 4-6
|
||||
|
||||
static PyObject *
|
||||
spam_system(PyObject *self, PyObject *arg)
|
||||
{
|
||||
int status = 3;
|
||||
PyObject *result = PyLong_FromLong(status);
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
Recompile, restart the Python interpreter again,
|
||||
and check that the function now returns 3:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
>>> spam.system('whoami')
|
||||
3
|
||||
|
||||
|
||||
Accepting a string
|
||||
==================
|
||||
|
||||
Finally, let's handle the function argument.
|
||||
|
||||
Our C function, :c:func:`!spam_system`, takes two arguments.
|
||||
The first one, ``PyObject *self``, will be set to the ``spam`` module
|
||||
object.
|
||||
This isn't useful in our case, so we'll ignore it.
|
||||
|
||||
The other one, ``PyObject *arg``, will be set to the object that the user
|
||||
passed from Python.
|
||||
We expect that it should be a Python string.
|
||||
In order to use the information in it, we will need
|
||||
to convert it to a C value -- in this case, a C string (``const char *``).
|
||||
|
||||
There's a slight type mismatch here: Python's :py:class:`str` objects store
|
||||
Unicode text, but C strings are arrays of bytes.
|
||||
So, we'll need to *encode* the data, and we'll use the UTF-8 encoding for it.
|
||||
(UTF-8 might not always be correct for system commands, but it's what
|
||||
:py:meth:`str.encode` uses by default,
|
||||
and the C API has special support for it.)
|
||||
|
||||
The function to encode a Python string into a UTF-8 buffer is named
|
||||
:c:func:`PyUnicode_AsUTF8` [#why-pyunicodeasutf8]_.
|
||||
Call it like this:
|
||||
|
||||
.. code-block:: c
|
||||
:emphasize-lines: 4
|
||||
|
||||
static PyObject *
|
||||
spam_system(PyObject *self, PyObject *arg)
|
||||
{
|
||||
const char *command = PyUnicode_AsUTF8(arg);
|
||||
int status = 3;
|
||||
PyObject *result = PyLong_FromLong(status);
|
||||
return result;
|
||||
}
|
||||
|
||||
If :c:func:`PyUnicode_AsUTF8` is successful, *command* will point to the
|
||||
resulting array of bytes.
|
||||
This buffer is managed by the *arg* object, which means we don't need to free
|
||||
it, but we must follow some rules:
|
||||
|
||||
* We should only use the buffer inside the ``spam_system`` function.
|
||||
When ``spam_system`` returns, *arg* and the buffer it manages might be
|
||||
garbage-collected.
|
||||
* We must not modify it. This is why we use ``const``.
|
||||
|
||||
If :c:func:`PyUnicode_AsUTF8` was *not* successful, it returns a ``NULL``
|
||||
pointer.
|
||||
When calling *any* Python C API, we always need to handle such error cases.
|
||||
The way to do this in general is left for later chapters of this documentation.
|
||||
For now, be assured that we are already handling errors from
|
||||
:c:func:`PyLong_FromLong` correctly.
|
||||
|
||||
For the :c:func:`PyUnicode_AsUTF8` call, the correct way to handle errors is
|
||||
returning ``NULL`` from ``spam_system``.
|
||||
Add an ``if`` block for this:
|
||||
|
||||
|
||||
.. code-block:: c
|
||||
:emphasize-lines: 5-7
|
||||
|
||||
static PyObject *
|
||||
spam_system(PyObject *self, PyObject *arg)
|
||||
{
|
||||
const char *command = PyUnicode_AsUTF8(arg);
|
||||
if (command == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
int status = 3;
|
||||
PyObject *result = PyLong_FromLong(status);
|
||||
return result;
|
||||
}
|
||||
|
||||
That's it for the setup.
|
||||
Now, all that is left is calling the C library function :c:func:`system` with
|
||||
the ``char *`` buffer, and using its result instead of the ``3``:
|
||||
|
||||
.. code-block:: c
|
||||
:emphasize-lines: 8
|
||||
|
||||
static PyObject *
|
||||
spam_system(PyObject *self, PyObject *arg)
|
||||
{
|
||||
const char *command = PyUnicode_AsUTF8(arg);
|
||||
if (command == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
int status = system(command);
|
||||
PyObject *result = PyLong_FromLong(status);
|
||||
return result;
|
||||
}
|
||||
|
||||
Compile your module, restart Python, and test.
|
||||
This time, you should see your username -- the output of the ``whoami``
|
||||
system command:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
>>> result = spam.system('whoami')
|
||||
User Name
|
||||
>>> result
|
||||
0
|
||||
|
||||
You might also want to test error cases:
|
||||
|
||||
.. code-block:: pycon
|
||||
|
||||
>>> import spam
|
||||
>>> result = spam.system('nonexistent-command')
|
||||
sh: line 1: nonexistent-command: command not found
|
||||
>>> result
|
||||
32512
|
||||
|
||||
>>> spam.system(3)
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
TypeError: bad argument type for built-in operation
|
||||
|
||||
|
||||
The result
|
||||
==========
|
||||
|
||||
|
||||
Congratulations!
|
||||
You have written a complete Python C API extension module,
|
||||
and completed this tutorial!
|
||||
|
||||
Here is the entire source file, for your convenience:
|
||||
|
||||
.. _extending-spammodule-source:
|
||||
|
||||
.. literalinclude:: ../includes/capi-extension/spammodule-01.c
|
||||
:start-at: ///
|
||||
|
||||
|
||||
.. _first-extension-other-tools:
|
||||
|
||||
Appendix: Other build tools
|
||||
===========================
|
||||
|
||||
You should be able to follow this tutorial -- except the
|
||||
*Running your build tool* section itself -- with a build tool other
|
||||
than ``meson-python``.
|
||||
|
||||
The Python Packaging User Guide has a `list of recommended tools <https://packaging.python.org/en/latest/guides/tool-recommendations/#build-backends-for-extension-modules>`_;
|
||||
be sure to choose one for the C language.
|
||||
|
||||
|
||||
Workaround for missing PyInit function
|
||||
--------------------------------------
|
||||
|
||||
If your build tool output complains about missing ``PyInit_spam``,
|
||||
add the following function to your module for now:
|
||||
|
||||
.. code-block:: c
|
||||
|
||||
// A workaround
|
||||
void *PyInit_spam(void) { return NULL; }
|
||||
|
||||
This is a shim for an old-style :ref:`initialization function <extension-export-hook>`,
|
||||
which was required in extension modules for CPython 3.14 and below.
|
||||
Current CPython does not need it, but some build tools may still assume that
|
||||
all extension modules need to define it.
|
||||
|
||||
If you use this workaround, you will get the exception
|
||||
``SystemError: initialization of spam failed without raising an exception``
|
||||
instead of
|
||||
``ImportError: dynamic module does not define module export function``.
|
||||
|
||||
|
||||
Compiling directly
|
||||
------------------
|
||||
|
||||
Using a third-party build tool is heavily recommended,
|
||||
as it will take care of various details of your platform and Python
|
||||
installation, of naming the resulting extension, and, later, of distributing
|
||||
your work.
|
||||
|
||||
If you are building an extension for as *specific* system, or for yourself
|
||||
only, you might instead want to run your compiler directly.
|
||||
The way to do this is system-specific; be prepared for issues you will need
|
||||
to solve yourself.
|
||||
|
||||
Linux
|
||||
^^^^^
|
||||
|
||||
On Linux, the Python development package may include a ``python3-config``
|
||||
command that prints out the required compiler flags.
|
||||
If you use it, check that it corresponds to the CPython interpreter you'll use
|
||||
to load the module.
|
||||
Then, start with the following command:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
gcc --shared $(python3-config --cflags --ldflags) spammodule.c -o spam.so
|
||||
|
||||
This should generate a ``spam.so`` file that you need to put in a directory
|
||||
on :py:attr:`sys.path`.
|
||||
|
||||
|
||||
.. rubric:: Footnotes
|
||||
|
||||
.. [#why-spam] ``spam`` is the favorite food of Monty Python fans...
|
||||
.. [#why-spammodule] The source file name is entirely up to you,
|
||||
though some tools can be picky about the ``.c`` extension.
|
||||
This tutorial uses the traditional ``*module.c`` suffix.
|
||||
Some people would just use :file:`spam.c` to implement a module
|
||||
named ``spam``,
|
||||
projects where Python isn't the primary language might use ``py_spam.c``,
|
||||
and so on.
|
||||
.. [#stdlib-h] Including :file:`stdlib.h` is technically not necessary,
|
||||
since :file:`Python.h` includes it and
|
||||
:ref:`several other standard headers <capi-system-includes>` for its own use
|
||||
or for backwards compatibility.
|
||||
However, it is good practice to explicitly include what you need.
|
||||
.. [#why-pymethoddef] The :c:type:`!PyMethodDef` structure is also used
|
||||
to create methods of classes, so there's no separate
|
||||
":c:type:`!PyFunctionDef`".
|
||||
.. [#why-pylongfromlong] The name :c:func:`PyLong_FromLong`
|
||||
might not seem obvious.
|
||||
``PyLong`` refers to a the Python :py:class:`int`, which was originally
|
||||
called ``long``; the ``FromLong`` refers to the C ``long`` (or ``long int``)
|
||||
type.
|
||||
.. [#why-pyunicodeasutf8] Here, ``PyUnicode`` refers to the original name of
|
||||
the Python :py:class:`str` class: ``unicode``.
|
||||
@ -5,15 +5,17 @@
|
||||
##################################################
|
||||
|
||||
This document describes how to write modules in C or C++ to extend the Python
|
||||
interpreter with new modules. Those modules can not only define new functions
|
||||
but also new object types and their methods. The document also describes how
|
||||
interpreter with new modules. Those modules can do what Python code does --
|
||||
define functions, object types and methods -- but also interact with native
|
||||
libraries or achieve better performance by avoiding the overhead of an
|
||||
interpreter. The document also describes how
|
||||
to embed the Python interpreter in another application, for use as an extension
|
||||
language. Finally, it shows how to compile and link extension modules so that
|
||||
they can be loaded dynamically (at run time) into the interpreter, if the
|
||||
underlying operating system supports this feature.
|
||||
|
||||
This document assumes basic knowledge about Python. For an informal
|
||||
introduction to the language, see :ref:`tutorial-index`. :ref:`reference-index`
|
||||
This document assumes basic knowledge about C and Python. For an informal
|
||||
introduction to Python, see :ref:`tutorial-index`. :ref:`reference-index`
|
||||
gives a more formal definition of the language. :ref:`library-index` documents
|
||||
the existing object types, functions and modules (both built-in and written in
|
||||
Python) that give the language its wide application range.
|
||||
@ -21,37 +23,75 @@ Python) that give the language its wide application range.
|
||||
For a detailed description of the whole Python/C API, see the separate
|
||||
:ref:`c-api-index`.
|
||||
|
||||
To support extensions, Python's C API (Application Programmers Interface)
|
||||
defines a set of functions, macros and variables that provide access to most
|
||||
aspects of the Python run-time system. The Python API is incorporated in a C
|
||||
source file by including the header ``"Python.h"``.
|
||||
|
||||
.. note::
|
||||
|
||||
The C extension interface is specific to CPython, and extension modules do
|
||||
not work on other Python implementations. In many cases, it is possible to
|
||||
avoid writing C extensions and preserve portability to other implementations.
|
||||
For example, if your use case is calling C library functions or system calls,
|
||||
you should consider using the :mod:`ctypes` module or the `cffi
|
||||
<https://cffi.readthedocs.io/>`_ library rather than writing
|
||||
custom C code.
|
||||
These modules let you write Python code to interface with C code and are more
|
||||
portable between implementations of Python than writing and compiling a C
|
||||
extension module.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:hidden:
|
||||
|
||||
first-extension-module.rst
|
||||
extending.rst
|
||||
newtypes_tutorial.rst
|
||||
newtypes.rst
|
||||
building.rst
|
||||
windows.rst
|
||||
embedding.rst
|
||||
|
||||
|
||||
Recommended third party tools
|
||||
=============================
|
||||
|
||||
This guide only covers the basic tools for creating extensions provided
|
||||
This document only covers the basic tools for creating extensions provided
|
||||
as part of this version of CPython. Some :ref:`third party tools
|
||||
<c-api-tools>` offer both simpler and more sophisticated approaches to creating
|
||||
C and C++ extensions for Python.
|
||||
|
||||
While this document is aimed at extension authors, it should also be helpful to
|
||||
the authors of such tools.
|
||||
For example, the tutorial module can serve as a simple test case for a build
|
||||
tool or sample expected output of a code generator.
|
||||
|
||||
Creating extensions without third party tools
|
||||
=============================================
|
||||
|
||||
C API Tutorial
|
||||
==============
|
||||
|
||||
This tutorial describes how to write a simple module in C or C++,
|
||||
using the Python C API -- that is, using the basic tools provided
|
||||
as part of this version of CPython.
|
||||
|
||||
|
||||
#. :ref:`first-extension-module`
|
||||
|
||||
|
||||
Guides for intermediate topics
|
||||
==============================
|
||||
|
||||
This section of the guide covers creating C and C++ extensions without
|
||||
assistance from third party tools. It is intended primarily for creators
|
||||
of those tools, rather than being a recommended way to create your own
|
||||
C extensions.
|
||||
|
||||
.. seealso::
|
||||
|
||||
:pep:`489` -- Multi-phase extension module initialization
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:numbered:
|
||||
|
||||
extending.rst
|
||||
newtypes_tutorial.rst
|
||||
newtypes.rst
|
||||
building.rst
|
||||
windows.rst
|
||||
* :ref:`extending-intro`
|
||||
* :ref:`defining-new-types`
|
||||
* :ref:`new-types-topics`
|
||||
* :ref:`building`
|
||||
* :ref:`building-on-windows`
|
||||
|
||||
Embedding the CPython runtime in a larger application
|
||||
=====================================================
|
||||
@ -61,8 +101,4 @@ interpreter as the main application, it is desirable to instead embed
|
||||
the CPython runtime inside a larger application. This section covers
|
||||
some of the details involved in doing that successfully.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:numbered:
|
||||
|
||||
embedding.rst
|
||||
* :ref:`embedding`
|
||||
|
||||
@ -1226,13 +1226,13 @@ This converts the list into a set, thereby removing duplicates, and then back
|
||||
into a list.
|
||||
|
||||
|
||||
How do you remove multiple items from a list
|
||||
--------------------------------------------
|
||||
How do you remove multiple items from a list?
|
||||
---------------------------------------------
|
||||
|
||||
As with removing duplicates, explicitly iterating in reverse with a
|
||||
delete condition is one possibility. However, it is easier and faster
|
||||
to use slice replacement with an implicit or explicit forward iteration.
|
||||
Here are three variations.::
|
||||
Here are three variations::
|
||||
|
||||
mylist[:] = filter(keep_function, mylist)
|
||||
mylist[:] = (x for x in mylist if keep_condition)
|
||||
|
||||
172
Doc/glossary.rst
172
Doc/glossary.rst
@ -134,6 +134,14 @@ Glossary
|
||||
iterator's :meth:`~object.__anext__` method until it raises a
|
||||
:exc:`StopAsyncIteration` exception. Introduced by :pep:`492`.
|
||||
|
||||
atomic operation
|
||||
An operation that appears to execute as a single, indivisible step: no
|
||||
other thread can observe it half-done, and its effects become visible all
|
||||
at once. Python does not guarantee that high-level statements are atomic
|
||||
(for example, ``x += 1`` performs multiple bytecode operations and is not
|
||||
atomic). Atomicity is only guaranteed where explicitly documented. See
|
||||
also :term:`race condition` and :term:`data race`.
|
||||
|
||||
attached thread state
|
||||
|
||||
A :term:`thread state` that is active for the current OS thread.
|
||||
@ -289,6 +297,22 @@ Glossary
|
||||
advanced mathematical feature. If you're not aware of a need for them,
|
||||
it's almost certain you can safely ignore them.
|
||||
|
||||
concurrency
|
||||
The ability of a computer program to perform multiple tasks at the same
|
||||
time. Python provides libraries for writing programs that make use of
|
||||
different forms of concurrency. :mod:`asyncio` is a library for dealing
|
||||
with asynchronous tasks and coroutines. :mod:`threading` provides
|
||||
access to operating system threads and :mod:`multiprocessing` to
|
||||
operating system processes. Multi-core processors can execute threads and
|
||||
processes on different CPU cores at the same time (see
|
||||
:term:`parallelism`).
|
||||
|
||||
concurrent modification
|
||||
When multiple threads modify shared data at the same time. Concurrent
|
||||
modification without proper synchronization can cause
|
||||
:term:`race conditions <race condition>`, and might also trigger a
|
||||
:term:`data race <data race>`, data corruption, or both.
|
||||
|
||||
context
|
||||
This term has different meanings depending on where and how it is used.
|
||||
Some common meanings:
|
||||
@ -363,6 +387,28 @@ Glossary
|
||||
the :term:`cyclic garbage collector <garbage collection>` is to identify these groups and break the reference
|
||||
cycles so that the memory can be reclaimed.
|
||||
|
||||
data race
|
||||
A situation where multiple threads access the same memory location
|
||||
concurrently, at least one of the accesses is a write, and the threads
|
||||
do not use any synchronization to control their access. Data races
|
||||
lead to :term:`non-deterministic` behavior and can cause data corruption.
|
||||
Proper use of :term:`locks <lock>` and other :term:`synchronization primitives
|
||||
<synchronization primitive>` prevents data races. Note that data races
|
||||
can only happen in native code, but that :term:`native code` might be
|
||||
exposed in a Python API. See also :term:`race condition` and
|
||||
:term:`thread-safe`.
|
||||
|
||||
deadlock
|
||||
A situation in which two or more tasks (threads, processes, or coroutines)
|
||||
wait indefinitely for each other to release resources or complete actions,
|
||||
preventing any from making progress. For example, if thread A holds lock
|
||||
1 and waits for lock 2, while thread B holds lock 2 and waits for lock 1,
|
||||
both threads will wait indefinitely. In Python this often arises from
|
||||
acquiring multiple locks in conflicting orders or from circular
|
||||
join/await dependencies. Deadlocks can be avoided by always acquiring
|
||||
multiple :term:`locks <lock>` in a consistent order. See also
|
||||
:term:`lock` and :term:`reentrant`.
|
||||
|
||||
decorator
|
||||
A function returning another function, usually applied as a function
|
||||
transformation using the ``@wrapper`` syntax. Common examples for
|
||||
@ -662,6 +708,14 @@ Glossary
|
||||
requires the GIL to be held in order to use it. This refers to having an
|
||||
:term:`attached thread state`.
|
||||
|
||||
global state
|
||||
Data that is accessible throughout a program, such as module-level
|
||||
variables, class variables, or C static variables in :term:`extension modules
|
||||
<extension module>`. In multi-threaded programs, global state shared
|
||||
between threads typically requires synchronization to avoid
|
||||
:term:`race conditions <race condition>` and
|
||||
:term:`data races <data race>`.
|
||||
|
||||
hash-based pyc
|
||||
A bytecode cache file that uses the hash rather than the last-modified
|
||||
time of the corresponding source file to determine its validity. See
|
||||
@ -706,7 +760,9 @@ Glossary
|
||||
tuples. Such an object cannot be altered. A new object has to
|
||||
be created if a different value has to be stored. They play an important
|
||||
role in places where a constant hash value is needed, for example as a key
|
||||
in a dictionary.
|
||||
in a dictionary. Immutable objects are inherently :term:`thread-safe`
|
||||
because their state cannot be modified after creation, eliminating concerns
|
||||
about improperly synchronized :term:`concurrent modification`.
|
||||
|
||||
import path
|
||||
A list of locations (or :term:`path entries <path entry>`) that are
|
||||
@ -796,8 +852,9 @@ Glossary
|
||||
|
||||
CPython does not consistently apply the requirement that an iterator
|
||||
define :meth:`~iterator.__iter__`.
|
||||
And also please note that the free-threading CPython does not guarantee
|
||||
the thread-safety of iterator operations.
|
||||
And also please note that :term:`free-threaded <free threading>`
|
||||
CPython does not guarantee :term:`thread-safe` behavior of iterator
|
||||
operations.
|
||||
|
||||
|
||||
key function
|
||||
@ -835,10 +892,11 @@ Glossary
|
||||
:keyword:`if` statements.
|
||||
|
||||
In a multi-threaded environment, the LBYL approach can risk introducing a
|
||||
race condition between "the looking" and "the leaping". For example, the
|
||||
code, ``if key in mapping: return mapping[key]`` can fail if another
|
||||
:term:`race condition` between "the looking" and "the leaping". For example,
|
||||
the code, ``if key in mapping: return mapping[key]`` can fail if another
|
||||
thread removes *key* from *mapping* after the test, but before the lookup.
|
||||
This issue can be solved with locks or by using the EAFP approach.
|
||||
This issue can be solved with :term:`locks <lock>` or by using the
|
||||
:term:`EAFP` approach. See also :term:`thread-safe`.
|
||||
|
||||
lexical analyzer
|
||||
|
||||
@ -857,6 +915,19 @@ Glossary
|
||||
clause is optional. If omitted, all elements in ``range(256)`` are
|
||||
processed.
|
||||
|
||||
lock
|
||||
A :term:`synchronization primitive` that allows only one thread at a
|
||||
time to access a shared resource. A thread must acquire a lock before
|
||||
accessing the protected resource and release it afterward. If a thread
|
||||
attempts to acquire a lock that is already held by another thread, it
|
||||
will block until the lock becomes available. Python's :mod:`threading`
|
||||
module provides :class:`~threading.Lock` (a basic lock) and
|
||||
:class:`~threading.RLock` (a :term:`reentrant` lock). Locks are used
|
||||
to prevent :term:`race conditions <race condition>` and ensure
|
||||
:term:`thread-safe` access to shared data. Alternative design patterns
|
||||
to locks exist such as queues, producer/consumer patterns, and
|
||||
thread-local state. See also :term:`deadlock`, and :term:`reentrant`.
|
||||
|
||||
loader
|
||||
An object that loads a module.
|
||||
It must define the :meth:`!exec_module` and :meth:`!create_module` methods
|
||||
@ -942,8 +1013,11 @@ Glossary
|
||||
See :term:`method resolution order`.
|
||||
|
||||
mutable
|
||||
Mutable objects can change their value but keep their :func:`id`. See
|
||||
also :term:`immutable`.
|
||||
An :term:`object` with state that is allowed to change during the course
|
||||
of the program. In multi-threaded programs, mutable objects that are
|
||||
shared between threads require careful synchronization to avoid
|
||||
:term:`race conditions <race condition>`. See also :term:`immutable`,
|
||||
:term:`thread-safe`, and :term:`concurrent modification`.
|
||||
|
||||
named tuple
|
||||
The term "named tuple" applies to any type or class that inherits from
|
||||
@ -995,6 +1069,13 @@ Glossary
|
||||
|
||||
See also :term:`module`.
|
||||
|
||||
native code
|
||||
Code that is compiled to machine instructions and runs directly on the
|
||||
processor, as opposed to code that is interpreted or runs in a virtual
|
||||
machine. In the context of Python, native code typically refers to
|
||||
C, C++, Rust or Fortran code in :term:`extension modules <extension module>`
|
||||
that can be called from Python. See also :term:`extension module`.
|
||||
|
||||
nested scope
|
||||
The ability to refer to a variable in an enclosing definition. For
|
||||
instance, a function defined inside another function can refer to
|
||||
@ -1011,6 +1092,15 @@ Glossary
|
||||
properties, :meth:`~object.__getattribute__`, class methods, and static
|
||||
methods.
|
||||
|
||||
non-deterministic
|
||||
Behavior where the outcome of a program can vary between executions with
|
||||
the same inputs. In multi-threaded programs, non-deterministic behavior
|
||||
often results from :term:`race conditions <race condition>` where the
|
||||
relative timing or interleaving of threads affects the result.
|
||||
Proper synchronization using :term:`locks <lock>` and other
|
||||
:term:`synchronization primitives <synchronization primitive>` helps
|
||||
ensure deterministic behavior.
|
||||
|
||||
object
|
||||
Any data with state (attributes or value) and defined behavior
|
||||
(methods). Also the ultimate base class of any :term:`new-style
|
||||
@ -1041,6 +1131,16 @@ Glossary
|
||||
|
||||
See also :term:`regular package` and :term:`namespace package`.
|
||||
|
||||
parallelism
|
||||
Executing multiple operations at the same time (e.g. on multiple CPU
|
||||
cores). In Python builds with the
|
||||
:term:`global interpreter lock (GIL) <global interpreter lock>`, only one
|
||||
thread runs Python bytecode at a time, so taking advantage of multiple
|
||||
CPU cores typically involves multiple processes
|
||||
(e.g. :mod:`multiprocessing`) or native extensions that release the GIL.
|
||||
In :term:`free-threaded <free threading>` Python, multiple Python threads
|
||||
can run Python code simultaneously on different cores.
|
||||
|
||||
parameter
|
||||
A named entity in a :term:`function` (or method) definition that
|
||||
specifies an :term:`argument` (or in some cases, arguments) that the
|
||||
@ -1215,6 +1315,18 @@ Glossary
|
||||
>>> email.mime.text.__name__
|
||||
'email.mime.text'
|
||||
|
||||
race condition
|
||||
A condition of a program where the its behavior
|
||||
depends on the relative timing or ordering of events, particularly in
|
||||
multi-threaded programs. Race conditions can lead to
|
||||
:term:`non-deterministic` behavior and bugs that are difficult to
|
||||
reproduce. A :term:`data race` is a specific type of race condition
|
||||
involving unsynchronized access to shared memory. The :term:`LBYL`
|
||||
coding style is particularly susceptible to race conditions in
|
||||
multi-threaded code. Using :term:`locks <lock>` and other
|
||||
:term:`synchronization primitives <synchronization primitive>`
|
||||
helps prevent race conditions.
|
||||
|
||||
reference count
|
||||
The number of references to an object. When the reference count of an
|
||||
object drops to zero, it is deallocated. Some objects are
|
||||
@ -1236,6 +1348,25 @@ Glossary
|
||||
|
||||
See also :term:`namespace package`.
|
||||
|
||||
reentrant
|
||||
A property of a function or :term:`lock` that allows it to be called or
|
||||
acquired multiple times by the same thread without causing errors or a
|
||||
:term:`deadlock`.
|
||||
|
||||
For functions, reentrancy means the function can be safely called again
|
||||
before a previous invocation has completed, which is important when
|
||||
functions may be called recursively or from signal handlers. Thread-unsafe
|
||||
functions may be :term:`non-deterministic` if they're called reentrantly in a
|
||||
multithreaded program.
|
||||
|
||||
For locks, Python's :class:`threading.RLock` (reentrant lock) is
|
||||
reentrant, meaning a thread that already holds the lock can acquire it
|
||||
again without blocking. In contrast, :class:`threading.Lock` is not
|
||||
reentrant - attempting to acquire it twice from the same thread will cause
|
||||
a deadlock.
|
||||
|
||||
See also :term:`lock` and :term:`deadlock`.
|
||||
|
||||
REPL
|
||||
An acronym for the "read–eval–print loop", another name for the
|
||||
:term:`interactive` interpreter shell.
|
||||
@ -1340,6 +1471,18 @@ Glossary
|
||||
|
||||
See also :term:`borrowed reference`.
|
||||
|
||||
synchronization primitive
|
||||
A basic building block for coordinating (synchronizing) the execution of
|
||||
multiple threads to ensure :term:`thread-safe` access to shared resources.
|
||||
Python's :mod:`threading` module provides several synchronization primitives
|
||||
including :class:`~threading.Lock`, :class:`~threading.RLock`,
|
||||
:class:`~threading.Semaphore`, :class:`~threading.Condition`,
|
||||
:class:`~threading.Event`, and :class:`~threading.Barrier`. Additionally,
|
||||
the :mod:`queue` module provides multi-producer, multi-consumer queues
|
||||
that are especially useful in multithreaded programs. These
|
||||
primitives help prevent :term:`race conditions <race condition>` and
|
||||
coordinate thread execution. See also :term:`lock`.
|
||||
|
||||
t-string
|
||||
t-strings
|
||||
String literals prefixed with ``t`` or ``T`` are commonly called
|
||||
@ -1392,6 +1535,19 @@ Glossary
|
||||
See :ref:`Thread State and the Global Interpreter Lock <threads>` for more
|
||||
information.
|
||||
|
||||
thread-safe
|
||||
A module, function, or class that behaves correctly when used by multiple
|
||||
threads concurrently. Thread-safe code uses appropriate
|
||||
:term:`synchronization primitives <synchronization primitive>` like
|
||||
:term:`locks <lock>` to protect shared mutable state, or is designed
|
||||
to avoid shared mutable state entirely. In the
|
||||
:term:`free-threaded <free threading>` build, built-in types like
|
||||
:class:`dict`, :class:`list`, and :class:`set` use internal locking
|
||||
to make many operations thread-safe, although thread safety is not
|
||||
necessarily guaranteed. Code that is not thread-safe may experience
|
||||
:term:`race conditions <race condition>` and :term:`data races <data race>`
|
||||
when used in multi-threaded programs.
|
||||
|
||||
token
|
||||
|
||||
A small unit of source code, generated by the
|
||||
|
||||
@ -8,6 +8,16 @@ execute Python code remotely.
|
||||
|
||||
Most platforms require elevated privileges to attach to another Python process.
|
||||
|
||||
Disabling remote debugging
|
||||
--------------------------
|
||||
|
||||
To disable remote debugging support, use any of the following:
|
||||
|
||||
* Set the :envvar:`PYTHON_DISABLE_REMOTE_DEBUG` environment variable to ``1`` before
|
||||
starting the interpreter.
|
||||
* Use the :option:`-X disable_remote_debug` command-line option.
|
||||
* Compile Python with the :option:`--without-remote-debug` build flag.
|
||||
|
||||
.. _permission-requirements:
|
||||
|
||||
Permission requirements
|
||||
@ -614,4 +624,3 @@ To inject and execute a Python script in a remote process:
|
||||
6. Set ``_PY_EVAL_PLEASE_STOP_BIT`` in the ``eval_breaker`` field.
|
||||
7. Resume the process (if suspended). The script will execute at the next safe
|
||||
evaluation point.
|
||||
|
||||
|
||||
55
Doc/includes/capi-extension/spammodule-01.c
Normal file
55
Doc/includes/capi-extension/spammodule-01.c
Normal file
@ -0,0 +1,55 @@
|
||||
/* This file needs to be kept in sync with the tutorial
|
||||
* at Doc/extending/first-extension-module.rst
|
||||
*/
|
||||
|
||||
/// Includes
|
||||
|
||||
#include <Python.h>
|
||||
#include <stdlib.h> // for system()
|
||||
|
||||
/// Implementation of spam.system
|
||||
|
||||
static PyObject *
|
||||
spam_system(PyObject *self, PyObject *arg)
|
||||
{
|
||||
const char *command = PyUnicode_AsUTF8(arg);
|
||||
if (command == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
int status = system(command);
|
||||
PyObject *result = PyLong_FromLong(status);
|
||||
return result;
|
||||
}
|
||||
|
||||
/// Module method table
|
||||
|
||||
static PyMethodDef spam_methods[] = {
|
||||
{
|
||||
.ml_name="system",
|
||||
.ml_meth=spam_system,
|
||||
.ml_flags=METH_O,
|
||||
.ml_doc="Execute a shell command.",
|
||||
},
|
||||
{NULL, NULL, 0, NULL} /* Sentinel */
|
||||
};
|
||||
|
||||
/// Module slot table
|
||||
|
||||
static PyModuleDef_Slot spam_slots[] = {
|
||||
{Py_mod_name, "spam"},
|
||||
{Py_mod_doc, "A wonderful module with an example function"},
|
||||
{Py_mod_methods, spam_methods},
|
||||
{0, NULL}
|
||||
};
|
||||
|
||||
/// Export hook prototype
|
||||
|
||||
PyMODEXPORT_FUNC PyModExport_spam(void);
|
||||
|
||||
/// Module export hook
|
||||
|
||||
PyMODEXPORT_FUNC
|
||||
PyModExport_spam(void)
|
||||
{
|
||||
return spam_slots;
|
||||
}
|
||||
@ -753,7 +753,7 @@ upper-cased name. For example::
|
||||
|
||||
>>> parser = argparse.ArgumentParser(prog='PROG')
|
||||
>>> parser.add_argument('--foo-bar')
|
||||
>>> parser.parse_args(['--foo-bar', 'FOO-BAR']
|
||||
>>> parser.parse_args(['--foo-bar', 'FOO-BAR'])
|
||||
Namespace(foo_bar='FOO-BAR')
|
||||
>>> parser.print_help()
|
||||
usage: [-h] [--foo-bar FOO-BAR]
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
--------------
|
||||
|
||||
This module defines an object type which can compactly represent an array of
|
||||
basic values: characters, integers, floating-point numbers. Arrays are sequence
|
||||
basic values: characters, integers, floating-point numbers. Arrays are mutable :term:`sequence`
|
||||
types and behave very much like lists, except that the type of objects stored in
|
||||
them is constrained. The type is specified at object creation time by using a
|
||||
:dfn:`type code`, which is a single character. The following type codes are
|
||||
@ -93,7 +93,7 @@ The module defines the following type:
|
||||
otherwise, the initializer's iterator is passed to the :meth:`extend` method
|
||||
to add initial items to the array.
|
||||
|
||||
Array objects support the ordinary sequence operations of indexing, slicing,
|
||||
Array objects support the ordinary :ref:`mutable <typesseq-mutable>` :term:`sequence` operations of indexing, slicing,
|
||||
concatenation, and multiplication. When using slice assignment, the assigned
|
||||
value must be an array object with the same type code; in all other cases,
|
||||
:exc:`TypeError` is raised. Array objects also implement the buffer interface,
|
||||
|
||||
@ -139,12 +139,13 @@ Node classes
|
||||
The :meth:`~object.__repr__` output of :class:`~ast.AST` nodes includes
|
||||
the values of the node fields.
|
||||
|
||||
.. deprecated:: 3.8
|
||||
.. deprecated-removed:: 3.8 3.14
|
||||
|
||||
Old classes :class:`!ast.Num`, :class:`!ast.Str`, :class:`!ast.Bytes`,
|
||||
:class:`!ast.NameConstant` and :class:`!ast.Ellipsis` are still available,
|
||||
but they will be removed in future Python releases. In the meantime,
|
||||
instantiating them will return an instance of a different class.
|
||||
Previous versions of Python provided the AST classes :class:`!ast.Num`,
|
||||
:class:`!ast.Str`, :class:`!ast.Bytes`, :class:`!ast.NameConstant` and
|
||||
:class:`!ast.Ellipsis`, which were deprecated in Python 3.8. These classes
|
||||
were removed in Python 3.14, and their functionality has been replaced with
|
||||
:class:`ast.Constant`.
|
||||
|
||||
.. deprecated:: 3.9
|
||||
|
||||
@ -2419,12 +2420,12 @@ and classes for traversing abstract syntax trees:
|
||||
during traversal. For this a special visitor exists
|
||||
(:class:`NodeTransformer`) that allows modifications.
|
||||
|
||||
.. deprecated:: 3.8
|
||||
.. deprecated-removed:: 3.8 3.14
|
||||
|
||||
Methods :meth:`!visit_Num`, :meth:`!visit_Str`, :meth:`!visit_Bytes`,
|
||||
:meth:`!visit_NameConstant` and :meth:`!visit_Ellipsis` are deprecated
|
||||
now and will not be called in future Python versions. Add the
|
||||
:meth:`visit_Constant` method to handle all constant nodes.
|
||||
:meth:`!visit_NameConstant` and :meth:`!visit_Ellipsis` will not be called
|
||||
in Python 3.14+. Add the :meth:`visit_Constant` method instead to handle
|
||||
all constant nodes.
|
||||
|
||||
|
||||
.. class:: NodeTransformer()
|
||||
|
||||
@ -107,7 +107,7 @@ Queue
|
||||
The queue can no longer grow.
|
||||
Future calls to :meth:`~Queue.put` raise :exc:`QueueShutDown`.
|
||||
Currently blocked callers of :meth:`~Queue.put` will be unblocked
|
||||
and will raise :exc:`QueueShutDown` in the formerly blocked thread.
|
||||
and will raise :exc:`QueueShutDown` in the formerly awaiting task.
|
||||
|
||||
If *immediate* is false (the default), the queue can be wound
|
||||
down normally with :meth:`~Queue.get` calls to extract tasks
|
||||
|
||||
@ -51,7 +51,7 @@ The :rfc:`4648` encodings are suitable for encoding binary data so that it can b
|
||||
safely sent by email, used as parts of URLs, or included as part of an HTTP
|
||||
POST request.
|
||||
|
||||
.. function:: b64encode(s, altchars=None)
|
||||
.. function:: b64encode(s, altchars=None, *, wrapcol=0)
|
||||
|
||||
Encode the :term:`bytes-like object` *s* using Base64 and return the encoded
|
||||
:class:`bytes`.
|
||||
@ -61,11 +61,19 @@ POST request.
|
||||
This allows an application to e.g. generate URL or filesystem safe Base64
|
||||
strings. The default is ``None``, for which the standard Base64 alphabet is used.
|
||||
|
||||
If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
|
||||
after at most every *wrapcol* characters.
|
||||
If *wrapcol* is zero (default), do not insert any newlines.
|
||||
|
||||
May assert or raise a :exc:`ValueError` if the length of *altchars* is not 2. Raises a
|
||||
:exc:`TypeError` if *altchars* is not a :term:`bytes-like object`.
|
||||
|
||||
.. versionchanged:: 3.15
|
||||
Added the *wrapcol* parameter.
|
||||
|
||||
|
||||
.. function:: b64decode(s, altchars=None, validate=False)
|
||||
b64decode(s, altchars=None, validate=True, *, ignorechars)
|
||||
|
||||
Decode the Base64 encoded :term:`bytes-like object` or ASCII string
|
||||
*s* and return the decoded :class:`bytes`.
|
||||
@ -77,15 +85,30 @@ POST request.
|
||||
A :exc:`binascii.Error` exception is raised
|
||||
if *s* is incorrectly padded.
|
||||
|
||||
If *validate* is ``False`` (the default), characters that are neither
|
||||
If *ignorechars* is specified, it should be a :term:`bytes-like object`
|
||||
containing characters to ignore from the input when *validate* is true.
|
||||
The default value of *validate* is ``True`` if *ignorechars* is specified,
|
||||
``False`` otherwise.
|
||||
|
||||
If *validate* is false, characters that are neither
|
||||
in the normal base-64 alphabet nor the alternative alphabet are
|
||||
discarded prior to the padding check. If *validate* is ``True``,
|
||||
these non-alphabet characters in the input result in a
|
||||
:exc:`binascii.Error`.
|
||||
discarded prior to the padding check, but the ``+`` and ``/`` characters
|
||||
keep their meaning if they are not in *altchars* (they will be discarded
|
||||
in future Python versions).
|
||||
|
||||
If *validate* is true, these non-alphabet characters in the input
|
||||
result in a :exc:`binascii.Error`.
|
||||
|
||||
For more information about the strict base64 check, see :func:`binascii.a2b_base64`
|
||||
|
||||
May assert or raise a :exc:`ValueError` if the length of *altchars* is not 2.
|
||||
.. deprecated:: next
|
||||
Accepting the ``+`` and ``/`` characters with an alternative alphabet
|
||||
is now deprecated.
|
||||
|
||||
|
||||
.. versionchanged:: next
|
||||
Added the *ignorechars* parameter.
|
||||
|
||||
|
||||
.. function:: standard_b64encode(s)
|
||||
|
||||
@ -116,6 +139,9 @@ POST request.
|
||||
``/`` in the standard Base64 alphabet, and return the decoded
|
||||
:class:`bytes`.
|
||||
|
||||
.. deprecated:: next
|
||||
Accepting the ``+`` and ``/`` characters is now deprecated.
|
||||
|
||||
|
||||
.. function:: b32encode(s)
|
||||
|
||||
@ -214,9 +240,9 @@ Refer to the documentation of the individual functions for more information.
|
||||
instead of 4 consecutive spaces (ASCII 0x20) as supported by 'btoa'. This
|
||||
feature is not supported by the "standard" Ascii85 encoding.
|
||||
|
||||
*wrapcol* controls whether the output should have newline (``b'\n'``)
|
||||
characters added to it. If this is non-zero, each output line will be
|
||||
at most this many characters long, excluding the trailing newline.
|
||||
If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
|
||||
after at most every *wrapcol* characters.
|
||||
If *wrapcol* is zero (default), do not insert any newlines.
|
||||
|
||||
*pad* controls whether the input is padded to a multiple of 4
|
||||
before encoding. Note that the ``btoa`` implementation always pads.
|
||||
@ -239,8 +265,7 @@ Refer to the documentation of the individual functions for more information.
|
||||
*adobe* controls whether the input sequence is in Adobe Ascii85 format
|
||||
(i.e. is framed with <~ and ~>).
|
||||
|
||||
*ignorechars* should be a :term:`bytes-like object` or ASCII string
|
||||
containing characters to ignore
|
||||
*ignorechars* should be a byte string containing characters to ignore
|
||||
from the input. This should only contain whitespace characters, and by
|
||||
default contains all whitespace characters in ASCII.
|
||||
|
||||
@ -267,14 +292,20 @@ Refer to the documentation of the individual functions for more information.
|
||||
.. versionadded:: 3.4
|
||||
|
||||
|
||||
.. function:: z85encode(s)
|
||||
.. function:: z85encode(s, pad=False)
|
||||
|
||||
Encode the :term:`bytes-like object` *s* using Z85 (as used in ZeroMQ)
|
||||
and return the encoded :class:`bytes`. See `Z85 specification
|
||||
<https://rfc.zeromq.org/spec/32/>`_ for more information.
|
||||
|
||||
If *pad* is true, the input is padded with ``b'\0'`` so its length is a
|
||||
multiple of 4 bytes before encoding.
|
||||
|
||||
.. versionadded:: 3.13
|
||||
|
||||
.. versionchanged:: 3.15
|
||||
The *pad* parameter was added.
|
||||
|
||||
|
||||
.. function:: z85decode(s)
|
||||
|
||||
|
||||
@ -49,16 +49,22 @@ The :mod:`binascii` module defines the following functions:
|
||||
|
||||
|
||||
.. function:: a2b_base64(string, /, *, strict_mode=False)
|
||||
a2b_base64(string, /, *, strict_mode=True, ignorechars)
|
||||
|
||||
Convert a block of base64 data back to binary and return the binary data. More
|
||||
than one line may be passed at a time.
|
||||
|
||||
If *ignorechars* is specified, it should be a :term:`bytes-like object`
|
||||
containing characters to ignore from the input when *strict_mode* is true.
|
||||
The default value of *strict_mode* is ``True`` if *ignorechars* is specified,
|
||||
``False`` otherwise.
|
||||
|
||||
If *strict_mode* is true, only valid base64 data will be converted. Invalid base64
|
||||
data will raise :exc:`binascii.Error`.
|
||||
|
||||
Valid base64:
|
||||
|
||||
* Conforms to :rfc:`3548`.
|
||||
* Conforms to :rfc:`4648`.
|
||||
* Contains only characters from the base64 alphabet.
|
||||
* Contains no excess data after padding (including excess padding, newlines, etc.).
|
||||
* Does not start with a padding.
|
||||
@ -66,16 +72,28 @@ The :mod:`binascii` module defines the following functions:
|
||||
.. versionchanged:: 3.11
|
||||
Added the *strict_mode* parameter.
|
||||
|
||||
.. versionchanged:: next
|
||||
Added the *ignorechars* parameter.
|
||||
|
||||
.. function:: b2a_base64(data, *, newline=True)
|
||||
|
||||
Convert binary data to a line of ASCII characters in base64 coding. The return
|
||||
value is the converted line, including a newline char if *newline* is
|
||||
true. The output of this function conforms to :rfc:`3548`.
|
||||
.. function:: b2a_base64(data, *, wrapcol=0, newline=True)
|
||||
|
||||
Convert binary data to a line(s) of ASCII characters in base64 coding,
|
||||
as specified in :rfc:`4648`.
|
||||
|
||||
If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
|
||||
after at most every *wrapcol* characters.
|
||||
If *wrapcol* is zero (default), do not insert any newlines.
|
||||
|
||||
If *newline* is true (default), a newline character will be added
|
||||
at the end of the output.
|
||||
|
||||
.. versionchanged:: 3.6
|
||||
Added the *newline* parameter.
|
||||
|
||||
.. versionchanged:: 3.15
|
||||
Added the *wrapcol* parameter.
|
||||
|
||||
|
||||
.. function:: a2b_qp(data, header=False)
|
||||
|
||||
|
||||
@ -73,7 +73,7 @@ Reading and writing compressed files
|
||||
argument is not None, a :exc:`!TypeError` will be raised.
|
||||
|
||||
When writing, the *options* argument can be a dictionary
|
||||
providing advanced decompression parameters; see
|
||||
providing advanced compression parameters; see
|
||||
:class:`CompressionParameter` for detailed information about supported
|
||||
parameters. The *level* argument is the compression level to use when
|
||||
writing compressed data. Only one of *level* or *options* may be non-None.
|
||||
@ -117,7 +117,7 @@ Reading and writing compressed files
|
||||
argument is not None, a :exc:`!TypeError` will be raised.
|
||||
|
||||
When writing, the *options* argument can be a dictionary
|
||||
providing advanced decompression parameters; see
|
||||
providing advanced compression parameters; see
|
||||
:class:`CompressionParameter` for detailed information about supported
|
||||
parameters. The *level* argument is the compression level to use when
|
||||
writing compressed data. Only one of *level* or *options* may be passed. The
|
||||
|
||||
@ -21,6 +21,11 @@ or separate processes, using :class:`ProcessPoolExecutor`.
|
||||
Each implements the same interface, which is defined
|
||||
by the abstract :class:`Executor` class.
|
||||
|
||||
:class:`concurrent.futures.Future` must not be confused with
|
||||
:class:`asyncio.Future`, which is designed for use with :mod:`asyncio`
|
||||
tasks and coroutines. See the :doc:`asyncio's Future <asyncio-future>`
|
||||
documentation for a detailed comparison of the two.
|
||||
|
||||
.. include:: ../includes/wasm-notavail.rst
|
||||
|
||||
Executor Objects
|
||||
@ -308,7 +313,7 @@ the bytes over a shared :mod:`socket <socket>` or
|
||||
|
||||
.. note::
|
||||
The executor may replace uncaught exceptions from *initializer*
|
||||
with :class:`~concurrent.futures.interpreter.ExecutionFailed`.
|
||||
with :class:`~concurrent.interpreters.ExecutionFailed`.
|
||||
|
||||
Other caveats from parent :class:`ThreadPoolExecutor` apply here.
|
||||
|
||||
@ -320,11 +325,11 @@ likewise serializes the return value when sending it back.
|
||||
When a worker's current task raises an uncaught exception, the worker
|
||||
always tries to preserve the exception as-is. If that is successful
|
||||
then it also sets the ``__cause__`` to a corresponding
|
||||
:class:`~concurrent.futures.interpreter.ExecutionFailed`
|
||||
:class:`~concurrent.interpreters.ExecutionFailed`
|
||||
instance, which contains a summary of the original exception.
|
||||
In the uncommon case that the worker is not able to preserve the
|
||||
original as-is then it directly preserves the corresponding
|
||||
:class:`~concurrent.futures.interpreter.ExecutionFailed`
|
||||
:class:`~concurrent.interpreters.ExecutionFailed`
|
||||
instance instead.
|
||||
|
||||
|
||||
@ -379,6 +384,11 @@ in a REPL or a lambda should not be expected to work.
|
||||
default in absence of a *mp_context* parameter. This feature is incompatible
|
||||
with the "fork" start method.
|
||||
|
||||
.. note::
|
||||
Bugs have been reported when using the *max_tasks_per_child* feature that
|
||||
can result in the :class:`ProcessPoolExecutor` hanging in some
|
||||
circumstances. Follow its eventual resolution in :gh:`115634`.
|
||||
|
||||
.. versionchanged:: 3.3
|
||||
When one of the worker processes terminates abruptly, a
|
||||
:exc:`~concurrent.futures.process.BrokenProcessPool` error is now raised.
|
||||
@ -715,15 +725,6 @@ Exception classes
|
||||
|
||||
.. versionadded:: 3.14
|
||||
|
||||
.. exception:: ExecutionFailed
|
||||
|
||||
Raised from :class:`~concurrent.futures.InterpreterPoolExecutor` when
|
||||
the given initializer fails or from
|
||||
:meth:`~concurrent.futures.Executor.submit` when there's an uncaught
|
||||
exception from the submitted task.
|
||||
|
||||
.. versionadded:: 3.14
|
||||
|
||||
.. currentmodule:: concurrent.futures.process
|
||||
|
||||
.. exception:: BrokenProcessPool
|
||||
|
||||
@ -77,6 +77,32 @@ Context Variables
|
||||
to restore the variable to its previous value via the
|
||||
:meth:`ContextVar.reset` method.
|
||||
|
||||
For convenience, the token object can be used as a context manager
|
||||
to avoid calling :meth:`ContextVar.reset` manually::
|
||||
|
||||
var = ContextVar('var', default='default value')
|
||||
|
||||
with var.set('new value'):
|
||||
assert var.get() == 'new value'
|
||||
|
||||
assert var.get() == 'default value'
|
||||
|
||||
It is a shorthand for::
|
||||
|
||||
var = ContextVar('var', default='default value')
|
||||
|
||||
token = var.set('new value')
|
||||
try:
|
||||
assert var.get() == 'new value'
|
||||
finally:
|
||||
var.reset(token)
|
||||
|
||||
assert var.get() == 'default value'
|
||||
|
||||
.. versionadded:: 3.14
|
||||
|
||||
Added support for using tokens as context managers.
|
||||
|
||||
.. method:: reset(token)
|
||||
|
||||
Reset the context variable to the value it had before the
|
||||
@ -93,24 +119,18 @@ Context Variables
|
||||
# After the reset call the var has no value again, so
|
||||
# var.get() would raise a LookupError.
|
||||
|
||||
The same *token* cannot be used twice.
|
||||
|
||||
|
||||
.. class:: Token
|
||||
|
||||
*Token* objects are returned by the :meth:`ContextVar.set` method.
|
||||
They can be passed to the :meth:`ContextVar.reset` method to revert
|
||||
the value of the variable to what it was before the corresponding
|
||||
*set*.
|
||||
*set*. A single token cannot reset a context variable more than once.
|
||||
|
||||
The token supports :ref:`context manager protocol <context-managers>`
|
||||
to restore the corresponding context variable value at the exit from
|
||||
:keyword:`with` block::
|
||||
|
||||
var = ContextVar('var', default='default value')
|
||||
|
||||
with var.set('new value'):
|
||||
assert var.get() == 'new value'
|
||||
|
||||
assert var.get() == 'default value'
|
||||
Tokens support the :ref:`context manager protocol <context-managers>`
|
||||
to automatically reset context variables. See :meth:`ContextVar.set`.
|
||||
|
||||
.. versionadded:: 3.14
|
||||
|
||||
|
||||
@ -2651,9 +2651,42 @@ Broadly speaking, ``d.strftime(fmt)`` acts like the :mod:`time` module's
|
||||
``time.strftime(fmt, d.timetuple())`` although not all objects support a
|
||||
:meth:`~date.timetuple` method.
|
||||
|
||||
For the :meth:`.datetime.strptime` class method, the default value is
|
||||
``1900-01-01T00:00:00.000``: any components not specified in the format string
|
||||
will be pulled from the default value. [#]_
|
||||
For the :meth:`.datetime.strptime` and :meth:`.date.strptime` class methods,
|
||||
the default value is ``1900-01-01T00:00:00.000``: any components not specified
|
||||
in the format string will be pulled from the default value.
|
||||
|
||||
.. note::
|
||||
When used to parse partial dates lacking a year, :meth:`.datetime.strptime`
|
||||
and :meth:`.date.strptime` will raise when encountering February 29 because
|
||||
the default year of 1900 is *not* a leap year. Always add a default leap
|
||||
year to partial date strings before parsing.
|
||||
|
||||
|
||||
.. testsetup::
|
||||
|
||||
# doctest seems to turn the warning into an error which makes it
|
||||
# show up and require matching and prevents the actual interesting
|
||||
# exception from being raised.
|
||||
# Manually apply the catch_warnings context manager
|
||||
import warnings
|
||||
catch_warnings = warnings.catch_warnings()
|
||||
catch_warnings.__enter__()
|
||||
warnings.simplefilter("ignore")
|
||||
|
||||
.. testcleanup::
|
||||
|
||||
catch_warnings.__exit__()
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> from datetime import datetime
|
||||
>>> value = "2/29"
|
||||
>>> datetime.strptime(value, "%m/%d")
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
ValueError: day 29 must be in range 1..28 for month 2 in year 1900
|
||||
>>> datetime.strptime(f"1904 {value}", "%Y %m/%d")
|
||||
datetime.datetime(1904, 2, 29, 0, 0)
|
||||
|
||||
Using ``datetime.strptime(date_string, format)`` is equivalent to::
|
||||
|
||||
@ -2790,7 +2823,7 @@ Notes:
|
||||
include a year in the format. If the value you need to parse lacks a year,
|
||||
append an explicit dummy leap year. Otherwise your code will raise an
|
||||
exception when it encounters leap day because the default year used by the
|
||||
parser is not a leap year. Users run into this bug every four years...
|
||||
parser (1900) is not a leap year. Users run into that bug every leap year.
|
||||
|
||||
.. doctest::
|
||||
|
||||
@ -2817,5 +2850,3 @@ Notes:
|
||||
.. [#] See R. H. van Gent's `guide to the mathematics of the ISO 8601 calendar
|
||||
<https://web.archive.org/web/20220531051136/https://webspace.science.uu.nl/~gent0113/calendar/isocalendar.htm>`_
|
||||
for a good explanation.
|
||||
|
||||
.. [#] Passing ``datetime.strptime('Feb 29', '%b %d')`` will fail since 1900 is not a leap year.
|
||||
|
||||
@ -768,7 +768,7 @@ not have to be) the original ``STACK[-2]``.
|
||||
end = STACK.pop()
|
||||
start = STACK.pop()
|
||||
container = STACK.pop()
|
||||
values = STACK.pop()
|
||||
value = STACK.pop()
|
||||
container[start:end] = value
|
||||
|
||||
.. versionadded:: 3.12
|
||||
|
||||
@ -57,7 +57,7 @@ message objects.
|
||||
:class:`~email.policy.default` policy, which follows the rules of the email
|
||||
RFCs except for line endings (instead of the RFC mandated ``\r\n``, it uses
|
||||
the Python standard ``\n`` line endings). For more information see the
|
||||
:mod:`~email.policy` documentation.
|
||||
:mod:`~email.policy` documentation. [2]_
|
||||
|
||||
.. method:: as_string(unixfrom=False, maxheaderlen=None, policy=None)
|
||||
|
||||
@ -749,3 +749,9 @@ message objects.
|
||||
.. [1] Originally added in 3.4 as a :term:`provisional module <provisional
|
||||
package>`. Docs for legacy message class moved to
|
||||
:ref:`compat32_message`.
|
||||
|
||||
.. [2] The :class:`EmailMessage` class requires a policy that provides a
|
||||
``content_manager`` attribute for content management methods like
|
||||
``set_content()`` and ``get_content()`` to work. The legacy
|
||||
:const:`~email.policy.compat32` policy does not support these methods
|
||||
and should not be used with :class:`EmailMessage`.
|
||||
|
||||
@ -662,6 +662,13 @@ The header objects and their attributes are described in
|
||||
An instance of :class:`Compat32`, providing backward compatibility with the
|
||||
behavior of the email package in Python 3.2.
|
||||
|
||||
.. note::
|
||||
|
||||
The :const:`compat32` policy should not be used as a policy for
|
||||
:class:`~email.message.EmailMessage` objects, and should only be used
|
||||
to serialize messages that were created using the :const:`compat32`
|
||||
policy.
|
||||
|
||||
|
||||
.. rubric:: Footnotes
|
||||
|
||||
|
||||
@ -153,6 +153,12 @@ Module Contents
|
||||
|
||||
Return a list of all power-of-two integers contained in a flag.
|
||||
|
||||
:func:`enum.bin`
|
||||
|
||||
Like built-in :func:`bin`, except negative values are represented in
|
||||
two's complement, and the leading bit always indicates sign
|
||||
(``0`` implies positive, ``1`` implies negative).
|
||||
|
||||
|
||||
.. versionadded:: 3.6 ``Flag``, ``IntFlag``, ``auto``
|
||||
.. versionadded:: 3.11 ``StrEnum``, ``EnumCheck``, ``ReprEnum``, ``FlagBoundary``, ``property``, ``member``, ``nonmember``, ``global_enum``, ``show_flag_values``
|
||||
@ -947,12 +953,13 @@ Utilities and Decorators
|
||||
the member's name. Care must be taken if mixing *auto()* with manually
|
||||
specified values.
|
||||
|
||||
*auto* instances are only resolved when at the top level of an assignment:
|
||||
*auto* instances are only resolved when at the top level of an assignment, either by
|
||||
itself or as part of a tuple:
|
||||
|
||||
* ``FIRST = auto()`` will work (auto() is replaced with ``1``);
|
||||
* ``SECOND = auto(), -2`` will work (auto is replaced with ``2``, so ``2, -2`` is
|
||||
used to create the ``SECOND`` enum member;
|
||||
* ``THREE = [auto(), -3]`` will *not* work (``<auto instance>, -3`` is used to
|
||||
* ``THREE = [auto(), -3]`` will *not* work (``[<auto instance>, -3]`` is used to
|
||||
create the ``THREE`` enum member)
|
||||
|
||||
.. versionchanged:: 3.11.1
|
||||
@ -1034,6 +1041,20 @@ Utilities and Decorators
|
||||
|
||||
.. versionadded:: 3.11
|
||||
|
||||
.. function:: bin(num, max_bits=None)
|
||||
|
||||
Like built-in :func:`bin`, except negative values are represented in
|
||||
two's complement, and the leading bit always indicates sign
|
||||
(``0`` implies positive, ``1`` implies negative).
|
||||
|
||||
>>> import enum
|
||||
>>> enum.bin(10)
|
||||
'0b0 1010'
|
||||
>>> enum.bin(~10) # ~10 is -11
|
||||
'0b1 0101'
|
||||
|
||||
.. versionadded:: 3.10
|
||||
|
||||
---------------
|
||||
|
||||
Notes
|
||||
|
||||
@ -524,14 +524,9 @@ FTP_TLS objects
|
||||
:class:`!FTP_TLS` class inherits from :class:`FTP`,
|
||||
defining these additional methods and attributes:
|
||||
|
||||
.. attribute:: FTP_TLS.ssl_version
|
||||
|
||||
The SSL version to use (defaults to :data:`ssl.PROTOCOL_SSLv23`).
|
||||
|
||||
.. method:: FTP_TLS.auth()
|
||||
|
||||
Set up a secure control connection by using TLS or SSL, depending on what
|
||||
is specified in the :attr:`ssl_version` attribute.
|
||||
Set up a secure control connection by using TLS.
|
||||
|
||||
.. versionchanged:: 3.4
|
||||
The method now supports hostname check with
|
||||
@ -548,7 +543,7 @@ FTP_TLS objects
|
||||
|
||||
.. method:: FTP_TLS.prot_p()
|
||||
|
||||
Set up secure data connection.
|
||||
Set up secure data connection by using TLS.
|
||||
|
||||
.. method:: FTP_TLS.prot_c()
|
||||
|
||||
|
||||
@ -138,6 +138,8 @@ are always available. They are listed here in alphabetical order.
|
||||
>>> f'{14:#b}', f'{14:b}'
|
||||
('0b1110', '1110')
|
||||
|
||||
See also :func:`enum.bin` to represent negative values as twos-complement.
|
||||
|
||||
See also :func:`format` for more information.
|
||||
|
||||
|
||||
|
||||
@ -294,9 +294,9 @@ The following example demonstrates how to use the :mod:`http.cookies` module.
|
||||
Set-Cookie: chips=ahoy
|
||||
Set-Cookie: vienna=finger
|
||||
>>> C = cookies.SimpleCookie()
|
||||
>>> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=\\012;";')
|
||||
>>> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=;";')
|
||||
>>> print(C)
|
||||
Set-Cookie: keebler="E=everybody; L=\"Loves\"; fudge=\012;"
|
||||
Set-Cookie: keebler="E=everybody; L=\"Loves\"; fudge=;"
|
||||
>>> C = cookies.SimpleCookie()
|
||||
>>> C["oreo"] = "doublestuff"
|
||||
>>> C["oreo"]["path"] = "/"
|
||||
|
||||
@ -158,7 +158,7 @@ Go to Line
|
||||
|
||||
Show Completions
|
||||
Open a scrollable list allowing selection of existing names. See
|
||||
:ref:`Completions <completions>` in the Editing and navigation section below.
|
||||
:ref:`Completions <completions>` in the Editing and Navigation section below.
|
||||
|
||||
Expand Word
|
||||
Expand a prefix you have typed to match a full word in the same window;
|
||||
@ -167,7 +167,7 @@ Expand Word
|
||||
Show Call Tip
|
||||
After an unclosed parenthesis for a function, open a small window with
|
||||
function parameter hints. See :ref:`Calltips <calltips>` in the
|
||||
Editing and navigation section below.
|
||||
Editing and Navigation section below.
|
||||
|
||||
Show Surrounding Parens
|
||||
Highlight the surrounding parenthesis.
|
||||
@ -178,9 +178,9 @@ Format menu (Editor window only)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Format Paragraph
|
||||
Reformat the current blank-line-delimited paragraph in comment block or
|
||||
multiline string or selected line in a string. All lines in the
|
||||
paragraph will be formatted to less than N columns, where N defaults to 72.
|
||||
Rewrap the text block containing the text insert cursor.
|
||||
Avoid code lines. See :ref:`Format block<format-block>` in the
|
||||
Editing and Navigation section below.
|
||||
|
||||
Indent Region
|
||||
Shift selected lines right by the indent width (default 4 spaces).
|
||||
@ -566,6 +566,20 @@ In an editor, import statements have no effect until one runs the file.
|
||||
One might want to run a file after writing import statements, after
|
||||
adding function definitions, or after opening an existing file.
|
||||
|
||||
.. _format-block:
|
||||
|
||||
Format block
|
||||
^^^^^^^^^^^^
|
||||
|
||||
Reformat Paragraph rewraps a block ('paragraph') of contiguous equally
|
||||
indented non-blank comments, a similar block of text within a multiline
|
||||
string, or a selected subset of either.
|
||||
If needed, add a blank line to separate string from code.
|
||||
Partial lines in a selection expand to complete lines.
|
||||
The resulting lines have the same indent as before
|
||||
but have maximum total length of N columns (characters).
|
||||
Change the default N of 72 on the Window tab of IDLE Settings.
|
||||
|
||||
.. _code-context:
|
||||
|
||||
Code Context
|
||||
|
||||
@ -418,6 +418,16 @@ Distributions
|
||||
equal, even if they relate to the same installed distribution and
|
||||
accordingly have the same attributes.
|
||||
|
||||
.. method:: discover(cls, *, context=None, **kwargs)
|
||||
|
||||
Returns an iterable of :class:`Distribution` instances for all packages.
|
||||
|
||||
The optional argument *context* is a :class:`DistributionFinder.Context`
|
||||
instance, used to modify the search for distributions. Alternatively,
|
||||
*kwargs* may contain keyword arguments for constructing a new
|
||||
:class:`!DistributionFinder.Context`.
|
||||
|
||||
|
||||
While the module level API described above is the most common and convenient usage,
|
||||
you can get all of that information from the :class:`!Distribution` class.
|
||||
:class:`!Distribution` is an abstract object that represents the metadata for
|
||||
@ -466,6 +476,61 @@ This metadata finder search defaults to ``sys.path``, but varies slightly in how
|
||||
- ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
|
||||
- ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.
|
||||
|
||||
.. class:: DistributionFinder
|
||||
|
||||
A :class:`~importlib.abc.MetaPathFinder` subclass capable of discovering
|
||||
installed distributions.
|
||||
|
||||
Custom providers should implement this interface in order to
|
||||
supply metadata.
|
||||
|
||||
.. class:: Context(**kwargs)
|
||||
|
||||
A :class:`!Context` gives a custom provider a means to
|
||||
solicit additional details from the callers of distribution discovery
|
||||
functions like :func:`distributions` or :meth:`Distribution.discover`
|
||||
beyond :attr:`!.name` and :attr:`!.path` when searching
|
||||
for distributions.
|
||||
|
||||
For example, a provider could expose suites of packages in either a
|
||||
"public" or "private" ``realm``. A caller of distribution discovery
|
||||
functions may wish to query only for distributions in a particular realm
|
||||
and could call ``distributions(realm="private")`` to signal to the
|
||||
custom provider to only include distributions from that
|
||||
realm.
|
||||
|
||||
Each :class:`!DistributionFinder` must expect any parameters and should
|
||||
attempt to honor the canonical parameters defined below when
|
||||
appropriate.
|
||||
|
||||
See the section on :ref:`implementing-custom-providers` for more details.
|
||||
|
||||
.. attribute:: name
|
||||
|
||||
Specific name for which a distribution finder should match.
|
||||
|
||||
A :attr:`!.name` of ``None`` matches all distributions.
|
||||
|
||||
.. attribute:: path
|
||||
|
||||
A property providing the sequence of directory paths that a
|
||||
distribution finder should search.
|
||||
|
||||
Typically refers to Python installed package paths such as
|
||||
"site-packages" directories and defaults to :attr:`sys.path`.
|
||||
|
||||
|
||||
.. function:: distributions(**kwargs)
|
||||
|
||||
Returns an iterable of :class:`Distribution` instances for all packages.
|
||||
|
||||
The *kwargs* argument may contain either a keyword argument ``context``, a
|
||||
:class:`DistributionFinder.Context` instance, or pass keyword arguments for
|
||||
constructing a new :class:`!DistributionFinder.Context`. The
|
||||
:class:`!DistributionFinder.Context` is used to modify the search for
|
||||
distributions.
|
||||
|
||||
.. _implementing-custom-providers:
|
||||
|
||||
Implementing Custom Providers
|
||||
=============================
|
||||
@ -493,7 +558,7 @@ interface expected of finders by Python's import system.
|
||||
``importlib.metadata`` extends this protocol by looking for an optional
|
||||
``find_distributions`` callable on the finders from
|
||||
:data:`sys.meta_path` and presents this extended interface as the
|
||||
``DistributionFinder`` abstract base class, which defines this abstract
|
||||
:class:`DistributionFinder` abstract base class, which defines this abstract
|
||||
method::
|
||||
|
||||
@abc.abstractmethod
|
||||
@ -502,9 +567,11 @@ method::
|
||||
loading the metadata for packages for the indicated ``context``.
|
||||
"""
|
||||
|
||||
The ``DistributionFinder.Context`` object provides ``.path`` and ``.name``
|
||||
properties indicating the path to search and name to match and may
|
||||
supply other relevant context sought by the consumer.
|
||||
The :class:`DistributionFinder.Context` object provides
|
||||
:attr:`~DistributionFinder.Context.path` and
|
||||
:attr:`~DistributionFinder.Context.name` properties indicating the path to
|
||||
search and name to match and may supply other relevant context sought by the
|
||||
consumer.
|
||||
|
||||
In practice, to support finding distribution package
|
||||
metadata in locations other than the file system, subclass
|
||||
@ -529,7 +596,7 @@ Imagine a custom finder that loads Python modules from a database::
|
||||
That importer now presumably provides importable modules from a
|
||||
database, but it provides no metadata or entry points. For this
|
||||
custom importer to provide metadata, it would also need to implement
|
||||
``DistributionFinder``::
|
||||
:class:`DistributionFinder`::
|
||||
|
||||
from importlib.metadata import DistributionFinder
|
||||
|
||||
|
||||
@ -63,11 +63,14 @@
|
||||
If the resource does not concretely exist on the file system,
|
||||
raise :exc:`FileNotFoundError`.
|
||||
|
||||
.. method:: is_resource(name)
|
||||
.. method:: is_resource(path)
|
||||
:abstractmethod:
|
||||
|
||||
Returns ``True`` if the named *name* is considered a resource.
|
||||
:exc:`FileNotFoundError` is raised if *name* does not exist.
|
||||
Returns ``True`` if the named *path* is considered a resource.
|
||||
:exc:`FileNotFoundError` is raised if *path* does not exist.
|
||||
|
||||
.. versionchanged:: 3.10
|
||||
The argument *name* was renamed to *path*.
|
||||
|
||||
.. method:: contents()
|
||||
:abstractmethod:
|
||||
|
||||
@ -210,12 +210,6 @@ Functions
|
||||
:exc:`ModuleNotFoundError` is raised when the module being reloaded lacks
|
||||
a :class:`~importlib.machinery.ModuleSpec`.
|
||||
|
||||
.. versionchanged:: 3.15
|
||||
If *module* is a lazy module that has not yet been materialized (i.e.,
|
||||
loaded via :class:`importlib.util.LazyLoader` and not yet accessed),
|
||||
calling :func:`reload` is a no-op and returns the module unchanged.
|
||||
This prevents the reload from unintentionally triggering the lazy load.
|
||||
|
||||
.. warning::
|
||||
This function is not thread-safe. Calling it from multiple threads can result
|
||||
in unexpected behavior. It's recommended to use the :class:`threading.Lock`
|
||||
@ -602,172 +596,6 @@ ABC hierarchy::
|
||||
itself does not end in ``__init__``.
|
||||
|
||||
|
||||
.. class:: ResourceReader
|
||||
|
||||
*Superseded by TraversableResources*
|
||||
|
||||
An :term:`abstract base class` to provide the ability to read
|
||||
*resources*.
|
||||
|
||||
From the perspective of this ABC, a *resource* is a binary
|
||||
artifact that is shipped within a package. Typically this is
|
||||
something like a data file that lives next to the ``__init__.py``
|
||||
file of the package. The purpose of this class is to help abstract
|
||||
out the accessing of such data files so that it does not matter if
|
||||
the package and its data file(s) are stored e.g. in a zip file
|
||||
versus on the file system.
|
||||
|
||||
For any of methods of this class, a *resource* argument is
|
||||
expected to be a :term:`path-like object` which represents
|
||||
conceptually just a file name. This means that no subdirectory
|
||||
paths should be included in the *resource* argument. This is
|
||||
because the location of the package the reader is for, acts as the
|
||||
"directory". Hence the metaphor for directories and file
|
||||
names is packages and resources, respectively. This is also why
|
||||
instances of this class are expected to directly correlate to
|
||||
a specific package (instead of potentially representing multiple
|
||||
packages or a module).
|
||||
|
||||
Loaders that wish to support resource reading are expected to
|
||||
provide a method called ``get_resource_reader(fullname)`` which
|
||||
returns an object implementing this ABC's interface. If the module
|
||||
specified by fullname is not a package, this method should return
|
||||
:const:`None`. An object compatible with this ABC should only be
|
||||
returned when the specified module is a package.
|
||||
|
||||
.. versionadded:: 3.7
|
||||
|
||||
.. deprecated-removed:: 3.12 3.14
|
||||
Use :class:`importlib.resources.abc.TraversableResources` instead.
|
||||
|
||||
.. method:: open_resource(resource)
|
||||
:abstractmethod:
|
||||
|
||||
Returns an opened, :term:`file-like object` for binary reading
|
||||
of the *resource*.
|
||||
|
||||
If the resource cannot be found, :exc:`FileNotFoundError` is
|
||||
raised.
|
||||
|
||||
.. method:: resource_path(resource)
|
||||
:abstractmethod:
|
||||
|
||||
Returns the file system path to the *resource*.
|
||||
|
||||
If the resource does not concretely exist on the file system,
|
||||
raise :exc:`FileNotFoundError`.
|
||||
|
||||
.. method:: is_resource(name)
|
||||
:abstractmethod:
|
||||
|
||||
Returns ``True`` if the named *name* is considered a resource.
|
||||
:exc:`FileNotFoundError` is raised if *name* does not exist.
|
||||
|
||||
.. method:: contents()
|
||||
:abstractmethod:
|
||||
|
||||
Returns an :term:`iterable` of strings over the contents of
|
||||
the package. Do note that it is not required that all names
|
||||
returned by the iterator be actual resources, e.g. it is
|
||||
acceptable to return names for which :meth:`is_resource` would
|
||||
be false.
|
||||
|
||||
Allowing non-resource names to be returned is to allow for
|
||||
situations where how a package and its resources are stored
|
||||
are known a priori and the non-resource names would be useful.
|
||||
For instance, returning subdirectory names is allowed so that
|
||||
when it is known that the package and resources are stored on
|
||||
the file system then those subdirectory names can be used
|
||||
directly.
|
||||
|
||||
The abstract method returns an iterable of no items.
|
||||
|
||||
|
||||
.. class:: Traversable
|
||||
|
||||
An object with a subset of :class:`pathlib.Path` methods suitable for
|
||||
traversing directories and opening files.
|
||||
|
||||
For a representation of the object on the file-system, use
|
||||
:meth:`importlib.resources.as_file`.
|
||||
|
||||
.. versionadded:: 3.9
|
||||
|
||||
.. deprecated-removed:: 3.12 3.14
|
||||
Use :class:`importlib.resources.abc.Traversable` instead.
|
||||
|
||||
.. attribute:: name
|
||||
|
||||
Abstract. The base name of this object without any parent references.
|
||||
|
||||
.. method:: iterdir()
|
||||
:abstractmethod:
|
||||
|
||||
Yield ``Traversable`` objects in ``self``.
|
||||
|
||||
.. method:: is_dir()
|
||||
:abstractmethod:
|
||||
|
||||
Return ``True`` if ``self`` is a directory.
|
||||
|
||||
.. method:: is_file()
|
||||
:abstractmethod:
|
||||
|
||||
Return ``True`` if ``self`` is a file.
|
||||
|
||||
.. method:: joinpath(child)
|
||||
:abstractmethod:
|
||||
|
||||
Return Traversable child in ``self``.
|
||||
|
||||
.. method:: __truediv__(child)
|
||||
:abstractmethod:
|
||||
|
||||
Return ``Traversable`` child in ``self``.
|
||||
|
||||
.. method:: open(mode='r', *args, **kwargs)
|
||||
:abstractmethod:
|
||||
|
||||
*mode* may be 'r' or 'rb' to open as text or binary. Return a handle
|
||||
suitable for reading (same as :attr:`pathlib.Path.open`).
|
||||
|
||||
When opening as text, accepts encoding parameters such as those
|
||||
accepted by :class:`io.TextIOWrapper`.
|
||||
|
||||
.. method:: read_bytes()
|
||||
|
||||
Read contents of ``self`` as bytes.
|
||||
|
||||
.. method:: read_text(encoding=None)
|
||||
|
||||
Read contents of ``self`` as text.
|
||||
|
||||
|
||||
.. class:: TraversableResources
|
||||
|
||||
An abstract base class for resource readers capable of serving
|
||||
the :meth:`importlib.resources.files` interface. Subclasses
|
||||
:class:`importlib.resources.abc.ResourceReader` and provides
|
||||
concrete implementations of the :class:`importlib.resources.abc.ResourceReader`'s
|
||||
abstract methods. Therefore, any loader supplying
|
||||
:class:`importlib.abc.TraversableResources` also supplies ResourceReader.
|
||||
|
||||
Loaders that wish to support resource reading are expected to
|
||||
implement this interface.
|
||||
|
||||
.. versionadded:: 3.9
|
||||
|
||||
.. deprecated-removed:: 3.12 3.14
|
||||
Use :class:`importlib.resources.abc.TraversableResources` instead.
|
||||
|
||||
.. method:: files()
|
||||
:abstractmethod:
|
||||
|
||||
Returns a :class:`importlib.resources.abc.Traversable` object for the loaded
|
||||
package.
|
||||
|
||||
|
||||
|
||||
:mod:`importlib.machinery` -- Importers and path hooks
|
||||
------------------------------------------------------
|
||||
|
||||
|
||||
@ -273,6 +273,9 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
|
||||
+-----------------+-------------------+---------------------------+
|
||||
| | ag_running | is the generator running? |
|
||||
+-----------------+-------------------+---------------------------+
|
||||
| | ag_suspended | is the generator |
|
||||
| | | suspended? |
|
||||
+-----------------+-------------------+---------------------------+
|
||||
| | ag_code | code |
|
||||
+-----------------+-------------------+---------------------------+
|
||||
| coroutine | __name__ | name |
|
||||
@ -286,6 +289,9 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
|
||||
+-----------------+-------------------+---------------------------+
|
||||
| | cr_running | is the coroutine running? |
|
||||
+-----------------+-------------------+---------------------------+
|
||||
| | cr_suspended | is the coroutine |
|
||||
| | | suspended? |
|
||||
+-----------------+-------------------+---------------------------+
|
||||
| | cr_code | code |
|
||||
+-----------------+-------------------+---------------------------+
|
||||
| | cr_origin | where coroutine was |
|
||||
@ -319,6 +325,18 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
|
||||
|
||||
Add ``__builtins__`` attribute to functions.
|
||||
|
||||
.. versionchanged:: 3.11
|
||||
|
||||
Add ``gi_suspended`` attribute to generators.
|
||||
|
||||
.. versionchanged:: 3.11
|
||||
|
||||
Add ``cr_suspended`` attribute to coroutines.
|
||||
|
||||
.. versionchanged:: 3.12
|
||||
|
||||
Add ``ag_suspended`` attribute to async generators.
|
||||
|
||||
.. versionchanged:: 3.14
|
||||
|
||||
Add ``f_generator`` attribute to frames.
|
||||
@ -506,7 +524,7 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
|
||||
|
||||
.. versionchanged:: 3.13
|
||||
Functions wrapped in :func:`functools.partialmethod` now return ``True``
|
||||
if the wrapped function is a :term:`coroutine function`.
|
||||
if the wrapped function is a :term:`asynchronous generator` function.
|
||||
|
||||
.. function:: isasyncgen(object)
|
||||
|
||||
|
||||
@ -47,7 +47,7 @@ Iterator Arguments Results
|
||||
Iterator Arguments Results Example
|
||||
============================ ============================ ================================================= =============================================================
|
||||
:func:`accumulate` p [,func] p0, p0+p1, p0+p1+p2, ... ``accumulate([1,2,3,4,5]) → 1 3 6 10 15``
|
||||
:func:`batched` p, n (p0, p1, ..., p_n-1), ... ``batched('ABCDEFG', n=2) → AB CD EF G``
|
||||
:func:`batched` p, n (p0, p1, ..., p_n-1), ... ``batched('ABCDEFG', n=3) → ABC DEF G``
|
||||
:func:`chain` p, q, ... p0, p1, ... plast, q0, q1, ... ``chain('ABC', 'DEF') → A B C D E F``
|
||||
:func:`chain.from_iterable` iterable p0, p1, ... plast, q0, q1, ... ``chain.from_iterable(['ABC', 'DEF']) → A B C D E F``
|
||||
:func:`compress` data, selectors (d[0] if s[0]), (d[1] if s[1]), ... ``compress('ABCDEF', [1,0,1,0,1,1]) → A C E F``
|
||||
@ -181,7 +181,7 @@ loops that truncate the stream.
|
||||
Roughly equivalent to::
|
||||
|
||||
def batched(iterable, n, *, strict=False):
|
||||
# batched('ABCDEFG', 2) → AB CD EF G
|
||||
# batched('ABCDEFG', 3) → ABC DEF G
|
||||
if n < 1:
|
||||
raise ValueError('n must be at least one')
|
||||
iterator = iter(iterable)
|
||||
@ -819,7 +819,7 @@ well as with the built-in itertools such as ``map()``, ``filter()``,
|
||||
|
||||
A secondary purpose of the recipes is to serve as an incubator. The
|
||||
``accumulate()``, ``compress()``, and ``pairwise()`` itertools started out as
|
||||
recipes. Currently, the ``sliding_window()``, ``iter_index()``, and ``sieve()``
|
||||
recipes. Currently, the ``sliding_window()``, ``derangements()``, and ``sieve()``
|
||||
recipes are being tested to see whether they prove their worth.
|
||||
|
||||
Substantially all of these recipes and many, many others can be installed from
|
||||
@ -838,11 +838,16 @@ and :term:`generators <generator>` which incur interpreter overhead.
|
||||
|
||||
.. testcode::
|
||||
|
||||
from itertools import (accumulate, batched, chain, combinations, compress,
|
||||
count, cycle, filterfalse, groupby, islice, permutations, product,
|
||||
repeat, starmap, tee, zip_longest)
|
||||
from collections import Counter, deque
|
||||
from contextlib import suppress
|
||||
from functools import reduce
|
||||
from math import comb, prod, sumprod, isqrt
|
||||
from operator import itemgetter, getitem, mul, neg
|
||||
from math import comb, isqrt, prod, sumprod
|
||||
from operator import getitem, is_not, itemgetter, mul, neg
|
||||
|
||||
# ==== Basic one liners ====
|
||||
|
||||
def take(n, iterable):
|
||||
"Return first n items of the iterable as a list."
|
||||
@ -899,8 +904,8 @@ and :term:`generators <generator>` which incur interpreter overhead.
|
||||
|
||||
def first_true(iterable, default=False, predicate=None):
|
||||
"Returns the first true value or the *default* if there is no true value."
|
||||
# first_true([a,b,c], x) → a or b or c or x
|
||||
# first_true([a,b], x, f) → a if f(a) else b if f(b) else x
|
||||
# first_true([a, b, c], x) → a or b or c or x
|
||||
# first_true([a, b], x, f) → a if f(a) else b if f(b) else x
|
||||
return next(filter(predicate, iterable), default)
|
||||
|
||||
def all_equal(iterable, key=None):
|
||||
@ -908,6 +913,8 @@ and :term:`generators <generator>` which incur interpreter overhead.
|
||||
# all_equal('4٤௪౪໔', key=int) → True
|
||||
return len(take(2, groupby(iterable, key))) <= 1
|
||||
|
||||
# ==== Data pipelines ====
|
||||
|
||||
def unique_justseen(iterable, key=None):
|
||||
"Yield unique elements, preserving order. Remember only the element just seen."
|
||||
# unique_justseen('AAAABBBCCDAABBB') → A B C D A B
|
||||
@ -940,7 +947,7 @@ and :term:`generators <generator>` which incur interpreter overhead.
|
||||
|
||||
def sliding_window(iterable, n):
|
||||
"Collect data into overlapping fixed-length chunks or blocks."
|
||||
# sliding_window('ABCDEFG', 4) → ABCD BCDE CDEF DEFG
|
||||
# sliding_window('ABCDEFG', 3) → ABC BCD CDE DEF EFG
|
||||
iterator = iter(iterable)
|
||||
window = deque(islice(iterator, n - 1), maxlen=n)
|
||||
for x in iterator:
|
||||
@ -949,7 +956,7 @@ and :term:`generators <generator>` which incur interpreter overhead.
|
||||
|
||||
def grouper(iterable, n, *, incomplete='fill', fillvalue=None):
|
||||
"Collect data into non-overlapping fixed-length chunks or blocks."
|
||||
# grouper('ABCDEFG', 3, fillvalue='x') → ABC DEF Gxx
|
||||
# grouper('ABCDEFG', 3, fillvalue='x') → ABC DEF Gxx
|
||||
# grouper('ABCDEFG', 3, incomplete='strict') → ABC DEF ValueError
|
||||
# grouper('ABCDEFG', 3, incomplete='ignore') → ABC DEF
|
||||
iterators = [iter(iterable)] * n
|
||||
@ -978,6 +985,16 @@ and :term:`generators <generator>` which incur interpreter overhead.
|
||||
slices = starmap(slice, combinations(range(len(seq) + 1), 2))
|
||||
return map(getitem, repeat(seq), slices)
|
||||
|
||||
def derangements(iterable, r=None):
|
||||
"Produce r length permutations without fixed points."
|
||||
# derangements('ABCD') → BADC BCDA BDAC CADB CDAB CDBA DABC DCAB DCBA
|
||||
# Algorithm credited to Stefan Pochmann
|
||||
seq = tuple(iterable)
|
||||
pos = tuple(range(len(seq)))
|
||||
have_moved = map(map, repeat(is_not), repeat(pos), permutations(pos, r=r))
|
||||
valid_derangements = map(all, have_moved)
|
||||
return compress(permutations(seq, r=r), valid_derangements)
|
||||
|
||||
def iter_index(iterable, value, start=0, stop=None):
|
||||
"Return indices where a value occurs in a sequence or iterable."
|
||||
# iter_index('AABCADEAF', 'A') → 0 1 4 7
|
||||
@ -1004,10 +1021,7 @@ and :term:`generators <generator>` which incur interpreter overhead.
|
||||
while True:
|
||||
yield function()
|
||||
|
||||
|
||||
The following recipes have a more mathematical flavor:
|
||||
|
||||
.. testcode::
|
||||
# ==== Mathematical operations ====
|
||||
|
||||
def multinomial(*counts):
|
||||
"Number of distinct arrangements of a multiset."
|
||||
@ -1026,9 +1040,11 @@ The following recipes have a more mathematical flavor:
|
||||
# sum_of_squares([10, 20, 30]) → 1400
|
||||
return sumprod(*tee(iterable))
|
||||
|
||||
# ==== Matrix operations ====
|
||||
|
||||
def reshape(matrix, columns):
|
||||
"Reshape a 2-D matrix to have a given number of columns."
|
||||
# reshape([(0, 1), (2, 3), (4, 5)], 3) → (0, 1, 2), (3, 4, 5)
|
||||
# reshape([(0, 1), (2, 3), (4, 5)], 3) → (0, 1, 2) (3, 4, 5)
|
||||
return batched(chain.from_iterable(matrix), columns, strict=True)
|
||||
|
||||
def transpose(matrix):
|
||||
@ -1038,10 +1054,12 @@ The following recipes have a more mathematical flavor:
|
||||
|
||||
def matmul(m1, m2):
|
||||
"Multiply two matrices."
|
||||
# matmul([(7, 5), (3, 5)], [(2, 5), (7, 9)]) → (49, 80), (41, 60)
|
||||
# matmul([(7, 5), (3, 5)], [(2, 5), (7, 9)]) → (49, 80) (41, 60)
|
||||
n = len(m2[0])
|
||||
return batched(starmap(sumprod, product(m1, transpose(m2))), n)
|
||||
|
||||
# ==== Polynomial arithmetic ====
|
||||
|
||||
def convolve(signal, kernel):
|
||||
"""Discrete linear convolution of two iterables.
|
||||
Equivalent to polynomial multiplication.
|
||||
@ -1096,6 +1114,8 @@ The following recipes have a more mathematical flavor:
|
||||
powers = reversed(range(1, n))
|
||||
return list(map(mul, coefficients, powers))
|
||||
|
||||
# ==== Number theory ====
|
||||
|
||||
def sieve(n):
|
||||
"Primes less than n."
|
||||
# sieve(30) → 2 3 5 7 11 13 17 19 23 29
|
||||
@ -1663,6 +1683,36 @@ The following recipes have a more mathematical flavor:
|
||||
['A', 'AB', 'ABC', 'ABCD', 'B', 'BC', 'BCD', 'C', 'CD', 'D']
|
||||
|
||||
|
||||
>>> ' '.join(map(''.join, derangements('ABCD')))
|
||||
'BADC BCDA BDAC CADB CDAB CDBA DABC DCAB DCBA'
|
||||
>>> ' '.join(map(''.join, derangements('ABCD', 3)))
|
||||
'BAD BCA BCD BDA CAB CAD CDA CDB DAB DCA DCB'
|
||||
>>> ' '.join(map(''.join, derangements('ABCD', 2)))
|
||||
'BA BC BD CA CD DA DC'
|
||||
>>> ' '.join(map(''.join, derangements('ABCD', 1)))
|
||||
'B C D'
|
||||
>>> ' '.join(map(''.join, derangements('ABCD', 0)))
|
||||
''
|
||||
>>> # Compare number of derangements to https://oeis.org/A000166
|
||||
>>> [len(list(derangements(range(n)))) for n in range(10)]
|
||||
[1, 0, 1, 2, 9, 44, 265, 1854, 14833, 133496]
|
||||
>>> # Verify that identical objects are treated as unique by position
|
||||
>>> identical = 'X'
|
||||
>>> distinct = 'x'
|
||||
>>> seq1 = ('A', identical, 'B', identical)
|
||||
>>> result1 = ' '.join(map(''.join, derangements(seq1)))
|
||||
>>> result1
|
||||
'XAXB XBXA XXAB BAXX BXAX BXXA XAXB XBAX XBXA'
|
||||
>>> seq2 = ('A', identical, 'B', distinct)
|
||||
>>> result2 = ' '.join(map(''.join, derangements(seq2)))
|
||||
>>> result2
|
||||
'XAxB XBxA XxAB BAxX BxAX BxXA xAXB xBAX xBXA'
|
||||
>>> result1 == result2
|
||||
False
|
||||
>>> result1.casefold() == result2.casefold()
|
||||
True
|
||||
|
||||
|
||||
>>> list(powerset([1,2,3]))
|
||||
[(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)]
|
||||
>>> all(len(list(powerset(range(n)))) == 2**n for n in range(18))
|
||||
|
||||
@ -31,7 +31,7 @@ The :mod:`linecache` module defines the following functions:
|
||||
.. index:: triple: module; search; path
|
||||
|
||||
If *filename* indicates a frozen module (starting with ``'<frozen '``), the function
|
||||
will attepmt to get the real file name from ``module_globals['__file__']`` if
|
||||
will attempt to get the real file name from ``module_globals['__file__']`` if
|
||||
*module_globals* is not ``None``.
|
||||
|
||||
If a file named *filename* is not found, the function first checks
|
||||
|
||||
@ -370,8 +370,6 @@ The :mod:`locale` module defines the following exception and functions:
|
||||
determined.
|
||||
The "C" locale is represented as ``(None, None)``.
|
||||
|
||||
.. deprecated-removed:: 3.11 3.15
|
||||
|
||||
|
||||
.. function:: getlocale(category=LC_CTYPE)
|
||||
|
||||
|
||||
@ -1011,6 +1011,11 @@ the options available to you.
|
||||
| exc_info | You shouldn't need to | Exception tuple (à la ``sys.exc_info``) or, |
|
||||
| | format this yourself. | if no exception has occurred, ``None``. |
|
||||
+----------------+-------------------------+-----------------------------------------------+
|
||||
| exc_text | You shouldn't need to | Exception information formatted as a string. |
|
||||
| | format this yourself. | This is set when :meth:`Formatter.format` is |
|
||||
| | | invoked, or ``None`` if no exception has |
|
||||
| | | occurred. |
|
||||
+----------------+-------------------------+-----------------------------------------------+
|
||||
| filename | ``%(filename)s`` | Filename portion of ``pathname``. |
|
||||
+----------------+-------------------------+-----------------------------------------------+
|
||||
| funcName | ``%(funcName)s`` | Name of function containing the logging call. |
|
||||
|
||||
@ -212,7 +212,7 @@ To map anonymous memory, -1 should be passed as the fileno along with the length
|
||||
Writable :term:`bytes-like object` is now accepted.
|
||||
|
||||
|
||||
.. method:: flush([offset[, size]])
|
||||
.. method:: flush([offset[, size]], *, flags=MS_SYNC)
|
||||
|
||||
Flushes changes made to the in-memory copy of a file back to disk. Without
|
||||
use of this call there is no guarantee that changes are written back before
|
||||
@ -221,6 +221,12 @@ To map anonymous memory, -1 should be passed as the fileno along with the length
|
||||
whole extent of the mapping is flushed. *offset* must be a multiple of the
|
||||
:const:`PAGESIZE` or :const:`ALLOCATIONGRANULARITY`.
|
||||
|
||||
The *flags* parameter specifies the synchronization behavior.
|
||||
*flags* must be one of the :ref:`MS_* constants <ms-constants>` available
|
||||
on the system.
|
||||
|
||||
On Windows, the *flags* parameter is ignored.
|
||||
|
||||
``None`` is returned to indicate success. An exception is raised when the
|
||||
call failed.
|
||||
|
||||
@ -235,6 +241,9 @@ To map anonymous memory, -1 should be passed as the fileno along with the length
|
||||
specified alone, and the flush operation will extend from *offset*
|
||||
to the end of the mmap.
|
||||
|
||||
.. versionchanged:: 3.15
|
||||
Added *flags* parameter to control synchronization behavior.
|
||||
|
||||
|
||||
.. method:: madvise(option[, start[, length]])
|
||||
|
||||
@ -328,6 +337,17 @@ To map anonymous memory, -1 should be passed as the fileno along with the length
|
||||
|
||||
.. versionadded:: 3.13
|
||||
|
||||
.. method:: set_name(name, /)
|
||||
|
||||
Annotate the memory mapping with the given *name* for easier identification
|
||||
in ``/proc/<pid>/maps`` if the kernel supports the feature and :option:`-X dev <-X>` is passed
|
||||
to Python or if Python is built in :ref:`debug mode <debug-build>`.
|
||||
The length of *name* must not exceed 67 bytes including the ``'\0'`` terminator.
|
||||
|
||||
.. availability:: Linux >= 5.17 (kernel built with ``CONFIG_ANON_VMA_NAME`` option)
|
||||
|
||||
.. versionadded:: 3.15
|
||||
|
||||
.. method:: size()
|
||||
|
||||
Return the length of the file, which can be larger than the size of the
|
||||
@ -450,3 +470,22 @@ MAP_* Constants
|
||||
:data:`MAP_TPRO`, :data:`MAP_TRANSLATED_ALLOW_EXECUTE`, and
|
||||
:data:`MAP_UNIX03` constants.
|
||||
|
||||
.. _ms-constants:
|
||||
|
||||
MS_* Constants
|
||||
++++++++++++++
|
||||
|
||||
.. data:: MS_SYNC
|
||||
MS_ASYNC
|
||||
MS_INVALIDATE
|
||||
|
||||
These flags control the synchronization behavior for :meth:`mmap.flush`:
|
||||
|
||||
* :data:`MS_SYNC` - Synchronous flush: writes are scheduled and the call
|
||||
blocks until they are physically written to storage.
|
||||
* :data:`MS_ASYNC` - Asynchronous flush: writes are scheduled but the call
|
||||
returns immediately without waiting for completion.
|
||||
* :data:`MS_INVALIDATE` - Invalidate cached data: invalidates other mappings
|
||||
of the same file so they can see the changes.
|
||||
|
||||
.. versionadded:: 3.15
|
||||
|
||||
@ -7,6 +7,8 @@
|
||||
|
||||
.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
|
||||
|
||||
**Source code:** :source:`PC/msvcrtmodule.c`
|
||||
|
||||
--------------
|
||||
|
||||
These functions provide access to some useful capabilities on Windows platforms.
|
||||
|
||||
@ -1234,22 +1234,32 @@ Miscellaneous
|
||||
.. versionchanged:: 3.11
|
||||
Accepts a :term:`path-like object`.
|
||||
|
||||
.. function:: set_forkserver_preload(module_names)
|
||||
.. function:: set_forkserver_preload(module_names, *, on_error='ignore')
|
||||
|
||||
Set a list of module names for the forkserver main process to attempt to
|
||||
import so that their already imported state is inherited by forked
|
||||
processes. Any :exc:`ImportError` when doing so is silently ignored.
|
||||
This can be used as a performance enhancement to avoid repeated work
|
||||
in every process.
|
||||
processes. This can be used as a performance enhancement to avoid repeated
|
||||
work in every process.
|
||||
|
||||
For this to work, it must be called before the forkserver process has been
|
||||
launched (before creating a :class:`Pool` or starting a :class:`Process`).
|
||||
|
||||
The *on_error* parameter controls how :exc:`ImportError` exceptions during
|
||||
module preloading are handled: ``"ignore"`` (default) silently ignores
|
||||
failures, ``"warn"`` causes the forkserver subprocess to emit an
|
||||
:exc:`ImportWarning` to stderr, and ``"fail"`` causes the forkserver
|
||||
subprocess to exit with the exception traceback on stderr, making
|
||||
subsequent process creation fail with :exc:`EOFError` or
|
||||
:exc:`ConnectionError`.
|
||||
|
||||
Only meaningful when using the ``'forkserver'`` start method.
|
||||
See :ref:`multiprocessing-start-methods`.
|
||||
|
||||
.. versionadded:: 3.4
|
||||
|
||||
.. versionchanged:: next
|
||||
Added the *on_error* parameter.
|
||||
|
||||
.. function:: set_start_method(method, force=False)
|
||||
|
||||
Set the method which should be used to start child processes.
|
||||
@ -1693,11 +1703,14 @@ inherited by child processes.
|
||||
value is actually a synchronized wrapper for the array.
|
||||
|
||||
*typecode_or_type* determines the type of the elements of the returned array:
|
||||
it is either a ctypes type or a one character typecode of the kind used by
|
||||
the :mod:`array` module. If *size_or_initializer* is an integer, then it
|
||||
determines the length of the array, and the array will be initially zeroed.
|
||||
Otherwise, *size_or_initializer* is a sequence which is used to initialize
|
||||
the array and whose length determines the length of the array.
|
||||
it is either a :ref:`ctypes type <ctypes-fundamental-data-types>` or a one
|
||||
character typecode of the kind used by the :mod:`array` module with the
|
||||
exception of ``'w'``, which is not supported. In addition, the ``'c'``
|
||||
typecode is an alias for :class:`ctypes.c_char`. If *size_or_initializer*
|
||||
is an integer, then it determines the length of the array, and the array
|
||||
will be initially zeroed. Otherwise, *size_or_initializer* is a sequence
|
||||
which is used to initialize the array and whose length determines the length
|
||||
of the array.
|
||||
|
||||
If *lock* is ``True`` (the default) then a new lock object is created to
|
||||
synchronize access to the value. If *lock* is a :class:`Lock` or
|
||||
|
||||
@ -1556,6 +1556,15 @@ or `the MSDN <https://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Windo
|
||||
.. versionadded:: 3.15
|
||||
|
||||
|
||||
.. data:: RWF_ATOMIC
|
||||
|
||||
Write data atomically. Requires alignment to the device's atomic write unit.
|
||||
|
||||
.. availability:: Linux >= 6.11
|
||||
|
||||
.. versionadded:: 3.15
|
||||
|
||||
|
||||
.. function:: ptsname(fd, /)
|
||||
|
||||
Return the name of the slave pseudo-terminal device associated with the
|
||||
@ -1598,6 +1607,7 @@ or `the MSDN <https://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Windo
|
||||
- :data:`RWF_SYNC`
|
||||
- :data:`RWF_APPEND`
|
||||
- :data:`RWF_DONTCACHE`
|
||||
- :data:`RWF_ATOMIC`
|
||||
|
||||
Return the total number of bytes actually written.
|
||||
|
||||
@ -1969,7 +1979,8 @@ can be inherited by child processes. Since Python 3.4, file descriptors
|
||||
created by Python are non-inheritable by default.
|
||||
|
||||
On UNIX, non-inheritable file descriptors are closed in child processes at the
|
||||
execution of a new program, other file descriptors are inherited.
|
||||
execution of a new program, other file descriptors are inherited. Note that
|
||||
non-inheritable file descriptors are still *inherited* by child processes on :func:`os.fork`.
|
||||
|
||||
On Windows, non-inheritable handles and file descriptors are closed in child
|
||||
processes, except for standard streams (file descriptors 0, 1 and 2: stdin, stdout
|
||||
@ -4251,7 +4262,7 @@ features:
|
||||
import os
|
||||
|
||||
# semaphore with start value '1'
|
||||
fd = os.eventfd(1, os.EFD_SEMAPHORE | os.EFC_CLOEXEC)
|
||||
fd = os.eventfd(1, os.EFD_SEMAPHORE | os.EFD_CLOEXEC)
|
||||
try:
|
||||
# acquire semaphore
|
||||
v = os.eventfd_read(fd)
|
||||
@ -5993,7 +6004,7 @@ Miscellaneous System Information
|
||||
|
||||
.. versionchanged:: 3.13
|
||||
If :option:`-X cpu_count <-X>` is given or :envvar:`PYTHON_CPU_COUNT` is set,
|
||||
:func:`cpu_count` returns the overridden value *n*.
|
||||
:func:`cpu_count` returns the override value *n*.
|
||||
|
||||
|
||||
.. function:: getloadavg()
|
||||
@ -6015,7 +6026,7 @@ Miscellaneous System Information
|
||||
in the **system**.
|
||||
|
||||
If :option:`-X cpu_count <-X>` is given or :envvar:`PYTHON_CPU_COUNT` is set,
|
||||
:func:`process_cpu_count` returns the overridden value *n*.
|
||||
:func:`process_cpu_count` returns the override value *n*.
|
||||
|
||||
See also the :func:`sched_getaffinity` function.
|
||||
|
||||
|
||||
@ -1331,6 +1331,10 @@ Reading directories
|
||||
PosixPath('setup.py'),
|
||||
PosixPath('test_pathlib.py')]
|
||||
|
||||
.. note::
|
||||
The paths are returned in no particular order.
|
||||
If you need a specific order, sort the results.
|
||||
|
||||
.. seealso::
|
||||
:ref:`pathlib-pattern-language` documentation.
|
||||
|
||||
@ -1365,6 +1369,10 @@ Reading directories
|
||||
Glob the given relative *pattern* recursively. This is like calling
|
||||
:func:`Path.glob` with "``**/``" added in front of the *pattern*.
|
||||
|
||||
.. note::
|
||||
The paths are returned in no particular order.
|
||||
If you need a specific order, sort the results.
|
||||
|
||||
.. seealso::
|
||||
:ref:`pathlib-pattern-language` and :meth:`Path.glob` documentation.
|
||||
|
||||
|
||||
@ -520,7 +520,8 @@ can be overridden by the local file.
|
||||
To remove all commands from a breakpoint, type ``commands`` and follow it
|
||||
immediately with ``end``; that is, give no commands.
|
||||
|
||||
With no *bpnumber* argument, ``commands`` refers to the last breakpoint set.
|
||||
With no *bpnumber* argument, ``commands`` refers to the most recently set
|
||||
breakpoint that still exists.
|
||||
|
||||
You can use breakpoint commands to start your program up again. Simply use
|
||||
the :pdbcmd:`continue` command, or :pdbcmd:`step`,
|
||||
|
||||
@ -56,19 +56,6 @@ files.
|
||||
|
||||
The :mod:`pickle` module differs from :mod:`marshal` in several significant ways:
|
||||
|
||||
* The :mod:`pickle` module keeps track of the objects it has already serialized,
|
||||
so that later references to the same object won't be serialized again.
|
||||
:mod:`marshal` doesn't do this.
|
||||
|
||||
This has implications both for recursive objects and object sharing. Recursive
|
||||
objects are objects that contain references to themselves. These are not
|
||||
handled by marshal, and in fact, attempting to marshal recursive objects will
|
||||
crash your Python interpreter. Object sharing happens when there are multiple
|
||||
references to the same object in different places in the object hierarchy being
|
||||
serialized. :mod:`pickle` stores such objects only once, and ensures that all
|
||||
other references point to the master copy. Shared objects remain shared, which
|
||||
can be very important for mutable objects.
|
||||
|
||||
* :mod:`marshal` cannot be used to serialize user-defined classes and their
|
||||
instances. :mod:`pickle` can save and restore class instances transparently,
|
||||
however the class definition must be importable and live in the same module as
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
.. highlight:: shell-session
|
||||
.. highlight:: sh
|
||||
|
||||
.. _profiling-module:
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
.. highlight:: shell-session
|
||||
.. highlight:: sh
|
||||
|
||||
.. _profiling-sampling:
|
||||
|
||||
@ -53,7 +53,7 @@ counts**, not direct measurements. Tachyon counts how many times each function
|
||||
appears in the collected samples, then multiplies by the sampling interval to
|
||||
estimate time.
|
||||
|
||||
For example, with a 100 microsecond sampling interval over a 10-second profile,
|
||||
For example, with a 10 kHz sampling rate over a 10-second profile,
|
||||
Tachyon collects approximately 100,000 samples. If a function appears in 5,000
|
||||
samples (5% of total), Tachyon estimates it consumed 5% of the 10-second
|
||||
duration, or about 500 milliseconds. This is a statistical estimate, not a
|
||||
@ -142,7 +142,7 @@ Use live mode for real-time monitoring (press ``q`` to quit)::
|
||||
|
||||
Profile for 60 seconds with a faster sampling rate::
|
||||
|
||||
python -m profiling.sampling run -d 60 -i 50 script.py
|
||||
python -m profiling.sampling run -d 60 -r 20khz script.py
|
||||
|
||||
Generate a line-by-line heatmap::
|
||||
|
||||
@ -200,6 +200,36 @@ On most systems, attaching to another process requires appropriate permissions.
|
||||
See :ref:`profiling-permissions` for platform-specific requirements.
|
||||
|
||||
|
||||
.. _replay-command:
|
||||
|
||||
The ``replay`` command
|
||||
----------------------
|
||||
|
||||
The ``replay`` command converts binary profile files to other output formats::
|
||||
|
||||
python -m profiling.sampling replay profile.bin
|
||||
python -m profiling.sampling replay --flamegraph -o profile.html profile.bin
|
||||
|
||||
This command is useful when you have captured profiling data in binary format
|
||||
and want to analyze it later or convert it to a visualization format. Binary
|
||||
profiles can be replayed multiple times to different formats without
|
||||
re-profiling.
|
||||
|
||||
::
|
||||
|
||||
# Convert binary to pstats (default, prints to stdout)
|
||||
python -m profiling.sampling replay profile.bin
|
||||
|
||||
# Convert binary to flame graph
|
||||
python -m profiling.sampling replay --flamegraph -o output.html profile.bin
|
||||
|
||||
# Convert binary to gecko format for Firefox Profiler
|
||||
python -m profiling.sampling replay --gecko -o profile.json profile.bin
|
||||
|
||||
# Convert binary to heatmap
|
||||
python -m profiling.sampling replay --heatmap -o my_heatmap profile.bin
|
||||
|
||||
|
||||
Profiling in production
|
||||
-----------------------
|
||||
|
||||
@ -211,8 +241,8 @@ is unaware it is being profiled.
|
||||
When profiling production systems, keep these guidelines in mind:
|
||||
|
||||
Start with shorter durations (10-30 seconds) to get quick results, then extend
|
||||
if you need more statistical accuracy. The default 10-second duration is usually
|
||||
sufficient to identify major hotspots.
|
||||
if you need more statistical accuracy. By default, profiling runs until the
|
||||
target process completes, which is usually sufficient to identify major hotspots.
|
||||
|
||||
If possible, profile during representative load rather than peak traffic.
|
||||
Profiles collected during normal operation are easier to interpret than those
|
||||
@ -296,10 +326,10 @@ The default configuration works well for most use cases:
|
||||
|
||||
* - Option
|
||||
- Default
|
||||
* - Default for ``--interval`` / ``-i``
|
||||
- 100 µs between samples (~10,000 samples/sec)
|
||||
* - Default for ``--sampling-rate`` / ``-r``
|
||||
- 1 kHz
|
||||
* - Default for ``--duration`` / ``-d``
|
||||
- 10 seconds
|
||||
- Run to completion
|
||||
* - Default for ``--all-threads`` / ``-a``
|
||||
- Main thread only
|
||||
* - Default for ``--native``
|
||||
@ -312,35 +342,35 @@ The default configuration works well for most use cases:
|
||||
- Disabled
|
||||
* - Default for ``--subprocesses``
|
||||
- Disabled
|
||||
* - Default for ``--blocking``
|
||||
- Disabled (non-blocking sampling)
|
||||
|
||||
|
||||
Sampling interval and duration
|
||||
------------------------------
|
||||
Sampling rate and duration
|
||||
--------------------------
|
||||
|
||||
The two most fundamental parameters are the sampling interval and duration.
|
||||
The two most fundamental parameters are the sampling rate and duration.
|
||||
Together, these determine how many samples will be collected during a profiling
|
||||
session.
|
||||
|
||||
The :option:`--interval` option (:option:`-i`) sets the time between samples in
|
||||
microseconds. The default is 100 microseconds, which produces approximately
|
||||
10,000 samples per second::
|
||||
The :option:`--sampling-rate` option (:option:`-r`) sets how frequently samples
|
||||
are collected. The default is 1 kHz (10,000 samples per second)::
|
||||
|
||||
python -m profiling.sampling run -i 50 script.py
|
||||
python -m profiling.sampling run -r 20khz script.py
|
||||
|
||||
Lower intervals capture more samples and provide finer-grained data at the
|
||||
cost of slightly higher profiler CPU usage. Higher intervals reduce profiler
|
||||
Higher rates capture more samples and provide finer-grained data at the
|
||||
cost of slightly higher profiler CPU usage. Lower rates reduce profiler
|
||||
overhead but may miss short-lived functions. For most applications, the
|
||||
default interval provides a good balance between accuracy and overhead.
|
||||
default rate provides a good balance between accuracy and overhead.
|
||||
|
||||
The :option:`--duration` option (:option:`-d`) sets how long to profile in seconds. The
|
||||
default is 10 seconds::
|
||||
The :option:`--duration` option (:option:`-d`) sets how long to profile in seconds. By
|
||||
default, profiling continues until the target process exits or is interrupted::
|
||||
|
||||
python -m profiling.sampling run -d 60 script.py
|
||||
|
||||
Longer durations collect more samples and produce more statistically reliable
|
||||
results, especially for code paths that execute infrequently. When profiling
|
||||
a program that runs for a fixed time, you may want to set the duration to
|
||||
match or exceed the expected runtime.
|
||||
Specifying a duration is useful when attaching to long-running processes or when
|
||||
you want to limit profiling to a specific time window. When profiling a script,
|
||||
the default behavior of running to completion is usually what you want.
|
||||
|
||||
|
||||
Thread selection
|
||||
@ -362,6 +392,50 @@ This option is particularly useful when investigating concurrency issues or
|
||||
when work is distributed across a thread pool.
|
||||
|
||||
|
||||
.. _blocking-mode:
|
||||
|
||||
Blocking mode
|
||||
-------------
|
||||
|
||||
By default, Tachyon reads the target process's memory without stopping it.
|
||||
This non-blocking approach is ideal for most profiling scenarios because it
|
||||
imposes virtually zero overhead on the target application: the profiled
|
||||
program runs at full speed and is unaware it is being observed.
|
||||
|
||||
However, non-blocking sampling can occasionally produce incomplete or
|
||||
inconsistent stack traces in applications with many generators or coroutines
|
||||
that rapidly switch between yield points, or in programs with very fast-changing
|
||||
call stacks where functions enter and exit between the start and end of a single
|
||||
stack read, resulting in reconstructed stacks that mix frames from different
|
||||
execution states or that never actually existed.
|
||||
|
||||
For these cases, the :option:`--blocking` option stops the target process during
|
||||
each sample::
|
||||
|
||||
python -m profiling.sampling run --blocking script.py
|
||||
python -m profiling.sampling attach --blocking 12345
|
||||
|
||||
When blocking mode is enabled, the profiler suspends the target process,
|
||||
reads its stack, then resumes it. This guarantees that each captured stack
|
||||
represents a real, consistent snapshot of what the process was doing at that
|
||||
instant. The trade-off is that the target process runs slower because it is
|
||||
repeatedly paused.
|
||||
|
||||
.. warning::
|
||||
|
||||
Do not use very high sample rates (low ``--interval`` values) with blocking
|
||||
mode. Suspending and resuming a process takes time, and if the sampling
|
||||
interval is too short, the target will spend more time stopped than running.
|
||||
For blocking mode, intervals of 1000 microseconds (1 millisecond) or higher
|
||||
are recommended. The default 100 microsecond interval may cause noticeable
|
||||
slowdown in the target application.
|
||||
|
||||
Use blocking mode only when you observe inconsistent stacks in your profiles,
|
||||
particularly with generator-heavy or coroutine-heavy code. For most
|
||||
applications, the default non-blocking mode provides accurate results with
|
||||
zero impact on the target process.
|
||||
|
||||
|
||||
Special frames
|
||||
--------------
|
||||
|
||||
@ -497,9 +571,9 @@ appended:
|
||||
- For pstats format (which defaults to stdout), subprocesses produce files like
|
||||
``profile_12345.pstats``
|
||||
|
||||
The subprocess profilers inherit most sampling options from the parent (interval,
|
||||
duration, thread selection, native frames, GC frames, async-aware mode, and
|
||||
output format). All Python descendant processes are profiled recursively,
|
||||
The subprocess profilers inherit most sampling options from the parent (sampling
|
||||
rate, duration, thread selection, native frames, GC frames, async-aware mode,
|
||||
and output format). All Python descendant processes are profiled recursively,
|
||||
including grandchildren and further descendants.
|
||||
|
||||
Subprocess detection works by periodically scanning for new descendants of
|
||||
@ -804,9 +878,9 @@ interesting functions that highlights:
|
||||
|
||||
Use :option:`--no-summary` to suppress both the legend and summary sections.
|
||||
|
||||
To save pstats output to a file instead of stdout::
|
||||
To save pstats output to a binary file instead of stdout::
|
||||
|
||||
python -m profiling.sampling run -o profile.txt script.py
|
||||
python -m profiling.sampling run -o profile.pstats script.py
|
||||
|
||||
The pstats format supports several options for controlling the display.
|
||||
The :option:`--sort` option determines the column used for ordering results::
|
||||
@ -1041,6 +1115,59 @@ intuitive view that shows exactly where time is spent without requiring
|
||||
interpretation of hierarchical visualizations.
|
||||
|
||||
|
||||
Binary format
|
||||
-------------
|
||||
|
||||
Binary format (:option:`--binary`) produces a compact binary file for efficient
|
||||
storage of profiling data::
|
||||
|
||||
python -m profiling.sampling run --binary -o profile.bin script.py
|
||||
python -m profiling.sampling attach --binary -o profile.bin 12345
|
||||
|
||||
The :option:`--compression` option controls data compression:
|
||||
|
||||
- ``auto`` (default): Use zstd compression if available, otherwise no
|
||||
compression
|
||||
- ``zstd``: Force zstd compression (requires :mod:`compression.zstd` support)
|
||||
- ``none``: Disable compression
|
||||
|
||||
::
|
||||
|
||||
python -m profiling.sampling run --binary --compression=zstd -o profile.bin script.py
|
||||
|
||||
To analyze binary profiles, use the :ref:`replay-command` to convert them to
|
||||
other formats like flame graphs or pstats output.
|
||||
|
||||
|
||||
Record and replay workflow
|
||||
==========================
|
||||
|
||||
The binary format combined with the replay command enables a record-and-replay
|
||||
workflow that separates data capture from analysis. Rather than generating
|
||||
visualizations during profiling, you capture raw data to a compact binary file
|
||||
and convert it to different formats later.
|
||||
|
||||
This approach has three main benefits:
|
||||
|
||||
- Sampling runs faster because the work of building data structures for
|
||||
visualization is deferred until replay.
|
||||
- A single binary capture can be converted to multiple output formats
|
||||
without re-profiling: pstats for a quick overview, flame graph for visual
|
||||
exploration, heatmap for line-level detail.
|
||||
- Binary files are compact and easy to share with colleagues who can convert
|
||||
them to their preferred format.
|
||||
|
||||
A typical workflow::
|
||||
|
||||
# Capture profile in production or during tests
|
||||
python -m profiling.sampling attach --binary -o profile.bin 12345
|
||||
|
||||
# Later, analyze with different formats
|
||||
python -m profiling.sampling replay profile.bin
|
||||
python -m profiling.sampling replay --flamegraph -o profile.html profile.bin
|
||||
python -m profiling.sampling replay --heatmap -o heatmap profile.bin
|
||||
|
||||
|
||||
Live mode
|
||||
=========
|
||||
|
||||
@ -1252,17 +1379,21 @@ Global options
|
||||
|
||||
Attach to and profile a running process by PID.
|
||||
|
||||
.. option:: replay
|
||||
|
||||
Convert a binary profile file to another output format.
|
||||
|
||||
|
||||
Sampling options
|
||||
----------------
|
||||
|
||||
.. option:: -i <microseconds>, --interval <microseconds>
|
||||
.. option:: -r <rate>, --sampling-rate <rate>
|
||||
|
||||
Sampling interval in microseconds. Default: 100.
|
||||
Sampling rate (for example, ``10000``, ``10khz``, ``10k``). Default: ``1khz``.
|
||||
|
||||
.. option:: -d <seconds>, --duration <seconds>
|
||||
|
||||
Profiling duration in seconds. Default: 10.
|
||||
Profiling duration in seconds. Default: run to completion.
|
||||
|
||||
.. option:: -a, --all-threads
|
||||
|
||||
@ -1296,6 +1427,13 @@ Sampling options
|
||||
Also profile subprocesses. Each subprocess gets its own profiler
|
||||
instance and output file. Incompatible with ``--live``.
|
||||
|
||||
.. option:: --blocking
|
||||
|
||||
Pause the target process during each sample. This ensures consistent
|
||||
stack traces at the cost of slowing down the target. Use with longer
|
||||
intervals (1000 µs or higher) to minimize impact. See :ref:`blocking-mode`
|
||||
for details.
|
||||
|
||||
|
||||
Mode options
|
||||
------------
|
||||
@ -1317,7 +1455,9 @@ Output options
|
||||
|
||||
.. option:: --pstats
|
||||
|
||||
Generate text statistics output. This is the default.
|
||||
Generate pstats statistics. This is the default.
|
||||
When written to stdout, the output is a text table; with :option:`-o`,
|
||||
it is a binary pstats file.
|
||||
|
||||
.. option:: --collapsed
|
||||
|
||||
@ -1335,12 +1475,30 @@ Output options
|
||||
|
||||
Generate HTML heatmap with line-level sample counts.
|
||||
|
||||
.. option:: --binary
|
||||
|
||||
Generate high-performance binary format for later conversion with the
|
||||
``replay`` command.
|
||||
|
||||
.. option:: --compression <type>
|
||||
|
||||
Compression for binary format: ``auto`` (use zstd if available, default),
|
||||
``zstd``, or ``none``.
|
||||
|
||||
.. option:: -o <path>, --output <path>
|
||||
|
||||
Output file or directory path. Default behavior varies by format:
|
||||
``--pstats`` writes to stdout, ``--flamegraph`` and ``--gecko`` generate
|
||||
files like ``flamegraph.PID.html``, and ``--heatmap`` creates a directory
|
||||
named ``heatmap_PID``.
|
||||
:option:`--pstats` prints a text table to stdout, while ``-o`` writes a
|
||||
binary pstats file. Other formats generate a file named
|
||||
``<format>_<PID>.<ext>`` (for example, ``flamegraph_12345.html``).
|
||||
:option:`--heatmap` creates a directory named ``heatmap_<PID>``.
|
||||
|
||||
.. option:: --browser
|
||||
|
||||
Automatically open HTML output (:option:`--flamegraph` and
|
||||
:option:`--heatmap`) in your default web browser after generation.
|
||||
When profiling with :option:`--subprocesses`, only the main process
|
||||
opens the browser; subprocess outputs are never auto-opened.
|
||||
|
||||
|
||||
pstats display options
|
||||
|
||||
@ -78,7 +78,7 @@ Bookkeeping functions
|
||||
instead of the system time (see the :func:`os.urandom` function for details
|
||||
on availability).
|
||||
|
||||
If *a* is an int, it is used directly.
|
||||
If *a* is an int, its absolute value is used directly.
|
||||
|
||||
With version 2 (the default), a :class:`str`, :class:`bytes`, or :class:`bytearray`
|
||||
object gets converted to an :class:`int` and all of its bits are used.
|
||||
@ -634,11 +634,12 @@ from the combinatoric iterators in the :mod:`itertools` module
|
||||
or the :pypi:`more-itertools` project:
|
||||
|
||||
.. testcode::
|
||||
|
||||
import random
|
||||
|
||||
def random_product(*args, repeat=1):
|
||||
"Random selection from itertools.product(*args, **kwds)"
|
||||
pools = [tuple(pool) for pool in args] * repeat
|
||||
def random_product(*iterables, repeat=1):
|
||||
"Random selection from itertools.product(*iterables, repeat=repeat)"
|
||||
pools = tuple(map(tuple, iterables)) * repeat
|
||||
return tuple(map(random.choice, pools))
|
||||
|
||||
def random_permutation(iterable, r=None):
|
||||
@ -663,15 +664,89 @@ or the :pypi:`more-itertools` project:
|
||||
return tuple(pool[i] for i in indices)
|
||||
|
||||
def random_derangement(iterable):
|
||||
"Choose a permutation where no element is in its original position."
|
||||
"Choose a permutation where no element stays in its original position."
|
||||
seq = tuple(iterable)
|
||||
if len(seq) < 2:
|
||||
raise ValueError('derangements require at least two values')
|
||||
perm = list(seq)
|
||||
if not seq:
|
||||
return ()
|
||||
raise IndexError('No derangments to choose from')
|
||||
perm = list(range(len(seq)))
|
||||
start = tuple(perm)
|
||||
while True:
|
||||
random.shuffle(perm)
|
||||
if all(p != q for p, q in zip(seq, perm)):
|
||||
return tuple(perm)
|
||||
if all(p != q for p, q in zip(start, perm)):
|
||||
return tuple([seq[i] for i in perm])
|
||||
|
||||
.. doctest::
|
||||
:hide:
|
||||
|
||||
>>> import random
|
||||
|
||||
|
||||
>>> random.seed(8675309)
|
||||
>>> random_product('ABCDEFG', repeat=5)
|
||||
('D', 'B', 'E', 'F', 'E')
|
||||
|
||||
|
||||
>>> random.seed(8675309)
|
||||
>>> random_permutation('ABCDEFG')
|
||||
('D', 'B', 'E', 'C', 'G', 'A', 'F')
|
||||
>>> random_permutation('ABCDEFG', 5)
|
||||
('A', 'G', 'D', 'C', 'B')
|
||||
|
||||
|
||||
>>> random.seed(8675309)
|
||||
>>> random_combination('ABCDEFG', 7)
|
||||
('A', 'B', 'C', 'D', 'E', 'F', 'G')
|
||||
>>> random_combination('ABCDEFG', 6)
|
||||
('A', 'B', 'C', 'D', 'F', 'G')
|
||||
>>> random_combination('ABCDEFG', 5)
|
||||
('A', 'B', 'C', 'E', 'F')
|
||||
>>> random_combination('ABCDEFG', 4)
|
||||
('B', 'C', 'D', 'G')
|
||||
>>> random_combination('ABCDEFG', 3)
|
||||
('B', 'E', 'G')
|
||||
>>> random_combination('ABCDEFG', 2)
|
||||
('E', 'G')
|
||||
>>> random_combination('ABCDEFG', 1)
|
||||
('C',)
|
||||
>>> random_combination('ABCDEFG', 0)
|
||||
()
|
||||
|
||||
|
||||
>>> random.seed(8675309)
|
||||
>>> random_combination_with_replacement('ABCDEFG', 7)
|
||||
('B', 'C', 'D', 'E', 'E', 'E', 'G')
|
||||
>>> random_combination_with_replacement('ABCDEFG', 3)
|
||||
('A', 'B', 'E')
|
||||
>>> random_combination_with_replacement('ABCDEFG', 2)
|
||||
('A', 'G')
|
||||
>>> random_combination_with_replacement('ABCDEFG', 1)
|
||||
('E',)
|
||||
>>> random_combination_with_replacement('ABCDEFG', 0)
|
||||
()
|
||||
|
||||
|
||||
>>> random.seed(8675309)
|
||||
>>> random_derangement('')
|
||||
()
|
||||
>>> random_derangement('A')
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
IndexError: No derangments to choose from
|
||||
>>> random_derangement('AB')
|
||||
('B', 'A')
|
||||
>>> random_derangement('ABC')
|
||||
('C', 'A', 'B')
|
||||
>>> random_derangement('ABCD')
|
||||
('B', 'A', 'D', 'C')
|
||||
>>> random_derangement('ABCDE')
|
||||
('B', 'C', 'A', 'E', 'D')
|
||||
>>> # Identical inputs treated as distinct
|
||||
>>> identical = 20
|
||||
>>> random_derangement((10, identical, 30, identical))
|
||||
(20, 30, 10, 20)
|
||||
|
||||
|
||||
The default :func:`.random` returns multiples of 2⁻⁵³ in the range
|
||||
*0.0 ≤ x < 1.0*. All such numbers are evenly spaced and are exactly
|
||||
|
||||
@ -403,3 +403,9 @@ support history save/restore. ::
|
||||
def save_history(self, histfile):
|
||||
readline.set_history_length(1000)
|
||||
readline.write_history_file(histfile)
|
||||
|
||||
.. note::
|
||||
|
||||
The new :term:`REPL` introduced in version 3.13 doesn't support readline.
|
||||
However, readline can still be used by setting the :envvar:`PYTHON_BASIC_REPL`
|
||||
environment variable.
|
||||
|
||||
@ -478,6 +478,8 @@ linearly scanned again. :c:func:`!select` is *O*\ (*highest file descriptor*), w
|
||||
|
||||
.. versionchanged:: 3.15
|
||||
Accepts any real number as *timeout*, not only integer or float.
|
||||
If ``ppoll()`` function is available, *timeout* has a resolution
|
||||
of ``1`` ns (``1e-6`` ms) instead of ``1`` ms.
|
||||
|
||||
|
||||
.. _kqueue-objects:
|
||||
|
||||
@ -515,7 +515,7 @@ Directory and files operations
|
||||
|
||||
.. exception:: Error
|
||||
|
||||
This exception collects exceptions that are raised during a multi-file
|
||||
Subclass of :exc:`OSError` collecting exceptions raised during a multi-file
|
||||
operation. For :func:`copytree`, the exception argument is a list of 3-tuples
|
||||
(*srcname*, *dstname*, *exception*).
|
||||
|
||||
|
||||
@ -1072,10 +1072,16 @@ The :mod:`socket` module also offers various network-related services:
|
||||
a string representing the canonical name of the *host* if
|
||||
:const:`AI_CANONNAME` is part of the *flags* argument; else *canonname*
|
||||
will be empty. *sockaddr* is a tuple describing a socket address, whose
|
||||
format depends on the returned *family* (a ``(address, port)`` 2-tuple for
|
||||
:const:`AF_INET`, a ``(address, port, flowinfo, scope_id)`` 4-tuple for
|
||||
:const:`AF_INET6`), and is meant to be passed to the :meth:`socket.connect`
|
||||
method.
|
||||
format depends on the returned *family* and flags Python was compiled with,
|
||||
and is meant to be passed to the :meth:`socket.connect` method.
|
||||
|
||||
*sockaddr* can be one of the following:
|
||||
|
||||
* a ``(address, port)`` 2-tuple for :const:`AF_INET`
|
||||
* a ``(address, port, flowinfo, scope_id)`` 4-tuple for :const:`AF_INET6` if
|
||||
Python was compiled with ``--enable-ipv6`` (the default)
|
||||
* a 2-tuple containing raw data for :const:`AF_INET6` if Python was
|
||||
compiled with ``--disable-ipv6``
|
||||
|
||||
.. note::
|
||||
|
||||
|
||||
@ -46,8 +46,10 @@ Any object can be tested for truth value, for use in an :keyword:`if` or
|
||||
By default, an object is considered true unless its class defines either a
|
||||
:meth:`~object.__bool__` method that returns ``False`` or a
|
||||
:meth:`~object.__len__` method that
|
||||
returns zero, when called with the object. [1]_ Here are most of the built-in
|
||||
objects considered false:
|
||||
returns zero, when called with the object. [1]_ If one of the methods raises an
|
||||
exception when called, the exception is propagated and the object does
|
||||
not have a truth value (for example, :data:`NotImplemented`).
|
||||
Here are most of the built-in objects considered false:
|
||||
|
||||
.. index::
|
||||
single: None (Built-in object)
|
||||
@ -1091,11 +1093,14 @@ Notes:
|
||||
still ``0``.
|
||||
|
||||
(4)
|
||||
The slice of *s* from *i* to *j* is defined as the sequence of items with index
|
||||
*k* such that ``i <= k < j``. If *i* or *j* is greater than ``len(s)``, use
|
||||
``len(s)``. If *i* is omitted or ``None``, use ``0``. If *j* is omitted or
|
||||
``None``, use ``len(s)``. If *i* is greater than or equal to *j*, the slice is
|
||||
empty.
|
||||
The slice of *s* from *i* to *j* is defined as the sequence of items with
|
||||
index *k* such that ``i <= k < j``.
|
||||
|
||||
* If *i* is omitted or ``None``, use ``0``.
|
||||
* If *j* is omitted or ``None``, use ``len(s)``.
|
||||
* If *i* or *j* is less than ``-len(s)``, use ``0``.
|
||||
* If *i* or *j* is greater than ``len(s)``, use ``len(s)``.
|
||||
* If *i* is greater than or equal to *j*, the slice is empty.
|
||||
|
||||
(5)
|
||||
The slice of *s* from *i* to *j* with step *k* is defined as the sequence of
|
||||
@ -1436,6 +1441,109 @@ application).
|
||||
list appear empty for the duration, and raises :exc:`ValueError` if it can
|
||||
detect that the list has been mutated during a sort.
|
||||
|
||||
.. admonition:: Thread safety
|
||||
|
||||
Reading a single element from a :class:`list` is
|
||||
:term:`atomic <atomic operation>`:
|
||||
|
||||
.. code-block::
|
||||
:class: green
|
||||
|
||||
lst[i] # list.__getitem__
|
||||
|
||||
The following methods traverse the list and use :term:`atomic <atomic operation>`
|
||||
reads of each item to perform their function. That means that they may
|
||||
return results affected by concurrent modifications:
|
||||
|
||||
.. code-block::
|
||||
:class: maybe
|
||||
|
||||
item in lst
|
||||
lst.index(item)
|
||||
lst.count(item)
|
||||
|
||||
All of the above methods/operations are also lock-free. They do not block
|
||||
concurrent modifications. Other operations that hold a lock will not block
|
||||
these from observing intermediate states.
|
||||
|
||||
All other operations from here on block using the per-object lock.
|
||||
|
||||
Writing a single item via ``lst[i] = x`` is safe to call from multiple
|
||||
threads and will not corrupt the list.
|
||||
|
||||
The following operations return new objects and appear
|
||||
:term:`atomic <atomic operation>` to other threads:
|
||||
|
||||
.. code-block::
|
||||
:class: good
|
||||
|
||||
lst1 + lst2 # concatenates two lists into a new list
|
||||
x * lst # repeats lst x times into a new list
|
||||
lst.copy() # returns a shallow copy of the list
|
||||
|
||||
Methods that only operate on a single elements with no shifting required are
|
||||
:term:`atomic <atomic operation>`:
|
||||
|
||||
.. code-block::
|
||||
:class: good
|
||||
|
||||
lst.append(x) # append to the end of the list, no shifting required
|
||||
lst.pop() # pop element from the end of the list, no shifting required
|
||||
|
||||
The :meth:`~list.clear` method is also :term:`atomic <atomic operation>`.
|
||||
Other threads cannot observe elements being removed.
|
||||
|
||||
The :meth:`~list.sort` method is not :term:`atomic <atomic operation>`.
|
||||
Other threads cannot observe intermediate states during sorting, but the
|
||||
list appears empty for the duration of the sort.
|
||||
|
||||
The following operations may allow lock-free operations to observe
|
||||
intermediate states since they modify multiple elements in place:
|
||||
|
||||
.. code-block::
|
||||
:class: maybe
|
||||
|
||||
lst.insert(idx, item) # shifts elements
|
||||
lst.pop(idx) # idx not at the end of the list, shifts elements
|
||||
lst *= x # copies elements in place
|
||||
|
||||
The :meth:`~list.remove` method may allow concurrent modifications since
|
||||
element comparison may execute arbitrary Python code (via
|
||||
:meth:`~object.__eq__`).
|
||||
|
||||
:meth:`~list.extend` is safe to call from multiple threads. However, its
|
||||
guarantees depend on the iterable passed to it. If it is a :class:`list`, a
|
||||
:class:`tuple`, a :class:`set`, a :class:`frozenset`, a :class:`dict` or a
|
||||
:ref:`dictionary view object <dict-views>` (but not their subclasses), the
|
||||
``extend`` operation is safe from concurrent modifications to the iterable.
|
||||
Otherwise, an iterator is created which can be concurrently modified by
|
||||
another thread. The same applies to inplace concatenation of a list with
|
||||
other iterables when using ``lst += iterable``.
|
||||
|
||||
Similarly, assigning to a list slice with ``lst[i:j] = iterable`` is safe
|
||||
to call from multiple threads, but ``iterable`` is only locked when it is
|
||||
also a :class:`list` (but not its subclasses).
|
||||
|
||||
Operations that involve multiple accesses, as well as iteration, are never
|
||||
atomic. For example:
|
||||
|
||||
.. code-block::
|
||||
:class: bad
|
||||
|
||||
# NOT atomic: read-modify-write
|
||||
lst[i] = lst[i] + 1
|
||||
|
||||
# NOT atomic: check-then-act
|
||||
if lst:
|
||||
item = lst.pop()
|
||||
|
||||
# NOT thread-safe: iteration while modifying
|
||||
for item in lst:
|
||||
process(item) # another thread may modify lst
|
||||
|
||||
Consider external synchronization when sharing :class:`list` instances
|
||||
across threads. See :ref:`freethreading-python-howto` for more information.
|
||||
|
||||
|
||||
.. _typesseq-tuple:
|
||||
|
||||
@ -1842,6 +1950,14 @@ expression support in the :mod:`re` module).
|
||||
lowercase letter ``'ß'`` is equivalent to ``"ss"``. Since it is already
|
||||
lowercase, :meth:`lower` would do nothing to ``'ß'``; :meth:`casefold`
|
||||
converts it to ``"ss"``.
|
||||
For example:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> 'straße'.lower()
|
||||
'straße'
|
||||
>>> 'straße'.casefold()
|
||||
'strasse'
|
||||
|
||||
The casefolding algorithm is `described in section 3.13.3 'Default Case
|
||||
Folding' of the Unicode Standard
|
||||
@ -2043,7 +2159,18 @@ expression support in the :mod:`re` module).
|
||||
.. method:: str.index(sub[, start[, end]])
|
||||
|
||||
Like :meth:`~str.find`, but raise :exc:`ValueError` when the substring is
|
||||
not found.
|
||||
not found. For example:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> 'spam, spam, spam'.index('eggs')
|
||||
Traceback (most recent call last):
|
||||
File "<python-input-0>", line 1, in <module>
|
||||
'spam, spam, spam'.index('eggs')
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
|
||||
ValueError: substring not found
|
||||
|
||||
See also :meth:`rindex`.
|
||||
|
||||
|
||||
.. method:: str.isalnum()
|
||||
@ -2188,6 +2315,15 @@ expression support in the :mod:`re` module).
|
||||
Nonprintable characters are those in group Separator or Other (Z or C),
|
||||
except the ASCII space.
|
||||
|
||||
For example:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> ''.isprintable(), ' '.isprintable()
|
||||
(True, True)
|
||||
>>> '\t'.isprintable(), '\n'.isprintable()
|
||||
(False, False)
|
||||
|
||||
|
||||
.. method:: str.isspace()
|
||||
|
||||
@ -2278,7 +2414,12 @@ expression support in the :mod:`re` module).
|
||||
.. method:: str.lower()
|
||||
|
||||
Return a copy of the string with all the cased characters [4]_ converted to
|
||||
lowercase.
|
||||
lowercase. For example:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> 'Lower Method Example'.lower()
|
||||
'lower method example'
|
||||
|
||||
The lowercasing algorithm used is `described in section 3.13.2 'Default Case
|
||||
Conversion' of the Unicode Standard
|
||||
@ -2334,7 +2475,9 @@ expression support in the :mod:`re` module).
|
||||
|
||||
If the string starts with the *prefix* string, return
|
||||
``string[len(prefix):]``. Otherwise, return a copy of the original
|
||||
string::
|
||||
string:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> 'TestHook'.removeprefix('Test')
|
||||
'Hook'
|
||||
@ -2343,12 +2486,16 @@ expression support in the :mod:`re` module).
|
||||
|
||||
.. versionadded:: 3.9
|
||||
|
||||
See also :meth:`removesuffix` and :meth:`startswith`.
|
||||
|
||||
|
||||
.. method:: str.removesuffix(suffix, /)
|
||||
|
||||
If the string ends with the *suffix* string and that *suffix* is not empty,
|
||||
return ``string[:-len(suffix)]``. Otherwise, return a copy of the
|
||||
original string::
|
||||
original string:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> 'MiscTests'.removesuffix('Tests')
|
||||
'Misc'
|
||||
@ -2357,12 +2504,22 @@ expression support in the :mod:`re` module).
|
||||
|
||||
.. versionadded:: 3.9
|
||||
|
||||
See also :meth:`removeprefix` and :meth:`endswith`.
|
||||
|
||||
|
||||
.. method:: str.replace(old, new, /, count=-1)
|
||||
|
||||
Return a copy of the string with all occurrences of substring *old* replaced by
|
||||
*new*. If *count* is given, only the first *count* occurrences are replaced.
|
||||
If *count* is not specified or ``-1``, then all occurrences are replaced.
|
||||
For example:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> 'spam, spam, spam'.replace('spam', 'eggs')
|
||||
'eggs, eggs, eggs'
|
||||
>>> 'spam, spam, spam'.replace('spam', 'eggs', 1)
|
||||
'eggs, spam, spam'
|
||||
|
||||
.. versionchanged:: 3.13
|
||||
*count* is now supported as a keyword argument.
|
||||
@ -2373,6 +2530,16 @@ expression support in the :mod:`re` module).
|
||||
Return the highest index in the string where substring *sub* is found, such
|
||||
that *sub* is contained within ``s[start:end]``. Optional arguments *start*
|
||||
and *end* are interpreted as in slice notation. Return ``-1`` on failure.
|
||||
For example:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> 'spam, spam, spam'.rfind('sp')
|
||||
12
|
||||
>>> 'spam, spam, spam'.rfind('sp', 0, 10)
|
||||
6
|
||||
|
||||
See also :meth:`find` and :meth:`rindex`.
|
||||
|
||||
|
||||
.. method:: str.rindex(sub[, start[, end]])
|
||||
@ -2395,6 +2562,19 @@ expression support in the :mod:`re` module).
|
||||
after the separator. If the separator is not found, return a 3-tuple containing
|
||||
two empty strings, followed by the string itself.
|
||||
|
||||
For example:
|
||||
|
||||
.. doctest::
|
||||
|
||||
>>> 'Monty Python'.rpartition(' ')
|
||||
('Monty', ' ', 'Python')
|
||||
>>> "Monty Python's Flying Circus".rpartition(' ')
|
||||
("Monty Python's Flying", ' ', 'Circus')
|
||||
>>> 'Monty Python'.rpartition('-')
|
||||
('', '', 'Monty Python')
|
||||
|
||||
See also :meth:`partition`.
|
||||
|
||||
|
||||
.. method:: str.rsplit(sep=None, maxsplit=-1)
|
||||
|
||||
|
||||
@ -180,6 +180,12 @@ Examining Symbol Tables
|
||||
Return a tuple containing names of :term:`free (closure) variables <closure variable>`
|
||||
in this function.
|
||||
|
||||
.. method:: get_cells()
|
||||
|
||||
Return a tuple containing names of :term:`cell variables <closure variable>` in this table.
|
||||
|
||||
.. versionadded:: next
|
||||
|
||||
|
||||
.. class:: Class
|
||||
|
||||
@ -291,6 +297,12 @@ Examining Symbol Tables
|
||||
Return ``True`` if the symbol is referenced in its block, but not assigned
|
||||
to.
|
||||
|
||||
.. method:: is_cell()
|
||||
|
||||
Return ``True`` if the symbol is referenced but not assigned in a nested block.
|
||||
|
||||
.. versionadded:: next
|
||||
|
||||
.. method:: is_free_class()
|
||||
|
||||
Return *True* if a class-scoped symbol is free from
|
||||
|
||||
@ -1997,6 +1997,9 @@ always available. Unless explicitly noted otherwise, all variables are read-only
|
||||
interpreter is pre-release (alpha, beta, or release candidate) then the
|
||||
local and remote interpreters must be the same exact version.
|
||||
|
||||
See :ref:`remote-debugging` for more information about the remote debugging
|
||||
mechanism.
|
||||
|
||||
.. audit-event:: sys.remote_exec pid script_path
|
||||
|
||||
When the code is executed in the remote process, an
|
||||
@ -2015,6 +2018,7 @@ always available. Unless explicitly noted otherwise, all variables are read-only
|
||||
|
||||
.. availability:: Unix, Windows.
|
||||
.. versionadded:: 3.14
|
||||
See :pep:`768` for more details.
|
||||
|
||||
|
||||
.. function:: _enablelegacywindowsfsencoding()
|
||||
|
||||
@ -225,8 +225,9 @@ The module defines the following user-callable items:
|
||||
properly implements the :const:`os.O_EXCL` flag for :func:`os.open`. The
|
||||
file is readable and writable only by the creating user ID. If the
|
||||
platform uses permission bits to indicate whether a file is executable,
|
||||
the file is executable by no one. The file descriptor is not inherited
|
||||
by child processes.
|
||||
the file is executable by no one.
|
||||
|
||||
The file descriptor is :ref:`not inherited by child processes <fd_inheritance>`.
|
||||
|
||||
Unlike :func:`TemporaryFile`, the user of :func:`mkstemp` is responsible
|
||||
for deleting the temporary file when done with it.
|
||||
|
||||
@ -492,6 +492,12 @@ The :mod:`test.support` module defines the following functions:
|
||||
tests.
|
||||
|
||||
|
||||
.. function:: get_resource_value(resource)
|
||||
|
||||
Return the value specified for *resource* (as :samp:`-u {resource}={value}`).
|
||||
Return ``None`` if *resource* is disabled or no value is specified.
|
||||
|
||||
|
||||
.. function:: python_is_optimized()
|
||||
|
||||
Return ``True`` if Python was not built with ``-O0`` or ``-Og``.
|
||||
|
||||
@ -102,6 +102,10 @@ functions should be good enough; otherwise, you should use an instance of
|
||||
print(repr(s)) # prints ' hello\n world\n '
|
||||
print(repr(dedent(s))) # prints 'hello\n world\n'
|
||||
|
||||
.. versionchanged:: 3.14
|
||||
The :func:`!dedent` function now correctly normalizes blank lines containing
|
||||
only whitespace characters. Previously, the implementation only normalized
|
||||
blank lines containing tabs and spaces.
|
||||
|
||||
.. function:: indent(text, prefix, predicate=None)
|
||||
|
||||
|
||||
@ -177,12 +177,12 @@ the modern themed widget set and API::
|
||||
.. attribute:: master
|
||||
|
||||
The widget object that contains this widget. For :class:`Tk`, the
|
||||
*master* is :const:`None` because it is the main window. The terms
|
||||
:attr:`!master` is :const:`None` because it is the main window. The terms
|
||||
*master* and *parent* are similar and sometimes used interchangeably
|
||||
as argument names; however, calling :meth:`winfo_parent` returns a
|
||||
string of the widget name whereas :attr:`master` returns the object.
|
||||
string of the widget name whereas :attr:`!master` returns the object.
|
||||
*parent*/*child* reflects the tree-like relationship while
|
||||
*master*/*slave* reflects the container structure.
|
||||
*master* (or *container*)/*content* reflects the container structure.
|
||||
|
||||
.. attribute:: children
|
||||
|
||||
@ -638,15 +638,15 @@ The Packer
|
||||
.. index:: single: packing (widgets)
|
||||
|
||||
The packer is one of Tk's geometry-management mechanisms. Geometry managers
|
||||
are used to specify the relative positioning of widgets within their container -
|
||||
their mutual *master*. In contrast to the more cumbersome *placer* (which is
|
||||
are used to specify the relative positioning of widgets within their container.
|
||||
In contrast to the more cumbersome *placer* (which is
|
||||
used less commonly, and we do not cover here), the packer takes qualitative
|
||||
relationship specification - *above*, *to the left of*, *filling*, etc - and
|
||||
works everything out to determine the exact placement coordinates for you.
|
||||
|
||||
The size of any *master* widget is determined by the size of the "slave widgets"
|
||||
inside. The packer is used to control where slave widgets appear inside the
|
||||
master into which they are packed. You can pack widgets into frames, and frames
|
||||
The size of any container widget is determined by the size of the "content widgets"
|
||||
inside. The packer is used to control where content widgets appear inside the
|
||||
container into which they are packed. You can pack widgets into frames, and frames
|
||||
into other frames, in order to achieve the kind of layout you desire.
|
||||
Additionally, the arrangement is dynamically adjusted to accommodate incremental
|
||||
changes to the configuration, once it is packed.
|
||||
@ -673,7 +673,7 @@ For more extensive information on the packer and the options that it can take,
|
||||
see the man pages and page 183 of John Ousterhout's book.
|
||||
|
||||
anchor
|
||||
Anchor type. Denotes where the packer is to place each slave in its parcel.
|
||||
Anchor type. Denotes where the packer is to place each content in its parcel.
|
||||
|
||||
expand
|
||||
Boolean, ``0`` or ``1``.
|
||||
@ -682,10 +682,10 @@ fill
|
||||
Legal values: ``'x'``, ``'y'``, ``'both'``, ``'none'``.
|
||||
|
||||
ipadx and ipady
|
||||
A distance - designating internal padding on each side of the slave widget.
|
||||
A distance - designating internal padding on each side of the content.
|
||||
|
||||
padx and pady
|
||||
A distance - designating external padding on each side of the slave widget.
|
||||
A distance - designating external padding on each side of the content.
|
||||
|
||||
side
|
||||
Legal values are: ``'left'``, ``'right'``, ``'top'``, ``'bottom'``.
|
||||
@ -758,8 +758,8 @@ subclassed from the :class:`Wm` class, and so can call the :class:`Wm` methods
|
||||
directly.
|
||||
|
||||
To get at the toplevel window that contains a given widget, you can often just
|
||||
refer to the widget's master. Of course if the widget has been packed inside of
|
||||
a frame, the master won't represent a toplevel window. To get at the toplevel
|
||||
refer to the widget's :attr:`master`. Of course if the widget has been packed inside of
|
||||
a frame, the :attr:`!master` won't represent a toplevel window. To get at the toplevel
|
||||
window that contains an arbitrary widget, you can call the :meth:`_root` method.
|
||||
This method begins with an underscore to denote the fact that this function is
|
||||
part of the implementation, and not an interface to Tk functionality.
|
||||
|
||||
@ -2442,6 +2442,10 @@ types.
|
||||
Removed the ``_field_types`` attribute in favor of the more
|
||||
standard ``__annotations__`` attribute which has the same information.
|
||||
|
||||
.. versionchanged:: 3.9
|
||||
``NamedTuple`` is now a function rather than a class.
|
||||
It can still be used as a class base, as described above.
|
||||
|
||||
.. versionchanged:: 3.11
|
||||
Added support for generic namedtuples.
|
||||
|
||||
@ -2588,7 +2592,7 @@ types.
|
||||
.. class:: TypedDict(dict)
|
||||
|
||||
Special construct to add type hints to a dictionary.
|
||||
At runtime it is a plain :class:`dict`.
|
||||
At runtime ":class:`!TypedDict` instances" are simply :class:`dicts <dict>`.
|
||||
|
||||
``TypedDict`` declares a dictionary type that expects all of its
|
||||
instances to have a certain set of keys, where each key is
|
||||
@ -2811,6 +2815,10 @@ types.
|
||||
|
||||
.. versionadded:: 3.8
|
||||
|
||||
.. versionchanged:: 3.9
|
||||
``TypedDict`` is now a function rather than a class.
|
||||
It can still be used as a class base, as described above.
|
||||
|
||||
.. versionchanged:: 3.11
|
||||
Added support for marking individual keys as :data:`Required` or :data:`NotRequired`.
|
||||
See :pep:`655`.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user