From 94affdd1591077dcae548e5db69b3f5ac74d971b Mon Sep 17 00:00:00 2001 From: b-pass Date: Tue, 13 May 2025 21:07:19 -0400 Subject: [PATCH 01/14] Add documentation (advanced/misc) for mod_gil_not_used and multiple_interpreters options --- docs/advanced/misc.rst | 67 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index 3aeb8ab9d3..8601b58f8c 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -153,6 +153,73 @@ following checklist. within pybind11 that will throw exceptions on certain GIL handling errors (reference counting operations). +Free-threading support +================================================================== + +pybind11 supports the experimental free-threaded Python build. pybind11's internal data +structures are thread safe. To enable your modules to be used with free-threading, pass the +:class:`mod_gil_not_used` tag as the third argument to ``PYBIND11_MODULE``. + +For example: + +.. code-block:: cpp + :emphasize-lines: 1 + PYBIND11_MODULE(example, m, py::mod_gil_not_used()) { + py::class_ animal(m, "Animal"); + // etc + } + +Note, of course, enabling your module to be used in free threading is also your promise that +your code is thread safe. To actually enable the feature your module must also be compiled +with the Python free-threading source tree. Adding this tag does not break compatibility with +non-free-threaded Python. + +Sub-interpreter support +================================================================== + +pybind11 supports isolated sub-interpreters. Pybind11's internal data structures are +sub-interpreter safe. To enable your modules to be imported in isolated sub-interpreters, pass the +:func:`multiple_interpreters::per_interpreter_gil()` tag as the third or later argument to +``PYBIND11_MODULE``. + +For example: + +.. code-block:: cpp + :emphasize-lines: 1 + PYBIND11_MODULE(example, m, py::mod_gil_not_used(), py::multiple_interpreters_per_interpreter_gil()) { + py::class_ animal(m, "Animal"); + // etc + } + +To make your module sub-interpreter safe, global/static state is strongly discouraged. Instead, +any state that your module keeps outside of Python objects must be carefully to the current +interpreter (where, of course, there can now be more than one). Python objects (except immortal +objects) may not be shared between different interpreters, modules must take care not to +accidentally share Python objects across sub-interpreters. + +Sub-interpreter Tips: + +- Your initialization function will run for each interpreter that imports your module + +- Never share python objects across interpreter boundaries. + +- Keep state it in the interpreter's state dict if necessary. No global/static state! + +- Avoid trying to "cache" python objects in C++ variables across function calls (this is an easy + way to accidentally introduce bugs). + +- While the interpreters each have their own global interpreter lock, isolated/independent + sub-interpreters each have their own lock, so concurrent calls into a module from two different + sub-interpreters are still possible. + +pybind11 also supports "legacy" sub-interpreters which shared a single global GIL. You can enable +legacy behavior by using the :func:`multiple_interpreters::shared_gil()` tag in +```PYBIND11_MODULE``. + +You can explicitly disable multiple interpreter support in your module by using the +:func:`multiple_interpreter::not_supported()` tag. This is the default behavior if you do not +specify a multiple_interpreters tag. + Binding sequence data types, iterators, the slicing protocol, etc. ================================================================== From d993e80cf567f00b4e1c1be1a91230933fd76776 Mon Sep 17 00:00:00 2001 From: b-pass Date: Tue, 13 May 2025 21:12:32 -0400 Subject: [PATCH 02/14] Add some comparison of free-threading vs sub-interpreters --- docs/advanced/misc.rst | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index 8601b58f8c..ad7868fe21 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -193,9 +193,9 @@ For example: To make your module sub-interpreter safe, global/static state is strongly discouraged. Instead, any state that your module keeps outside of Python objects must be carefully to the current -interpreter (where, of course, there can now be more than one). Python objects (except immortal -objects) may not be shared between different interpreters, modules must take care not to -accidentally share Python objects across sub-interpreters. +sub-interpreter (where, of course, there can now be more than one). Python objects (except +immortal objects) may not be shared between different sub-interpreters, modules must take care not +to accidentally share Python objects across sub-interpreters. Sub-interpreter Tips: @@ -208,7 +208,7 @@ Sub-interpreter Tips: - Avoid trying to "cache" python objects in C++ variables across function calls (this is an easy way to accidentally introduce bugs). -- While the interpreters each have their own global interpreter lock, isolated/independent +- While the interpreters each have their own GIL, isolated/independent sub-interpreters each have their own lock, so concurrent calls into a module from two different sub-interpreters are still possible. @@ -220,6 +220,10 @@ You can explicitly disable multiple interpreter support in your module by using :func:`multiple_interpreter::not_supported()` tag. This is the default behavior if you do not specify a multiple_interpreters tag. +Note: Sub-interpreter support does not imply free-threading support or vice-versa. +Free-threaded modules can still have global/static state, but multiple interpreter modules cannot. +Likewise, sub-interpreter modules can still use the GIL, but free-threaded modules cannot. + Binding sequence data types, iterators, the slicing protocol, etc. ================================================================== From ba7baf4fbfd6bef70cdbda21cb908577ffd51228 Mon Sep 17 00:00:00 2001 From: b-pass Date: Tue, 13 May 2025 21:14:37 -0400 Subject: [PATCH 03/14] Formatting --- docs/advanced/misc.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index ad7868fe21..b8481dbf87 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -193,8 +193,8 @@ For example: To make your module sub-interpreter safe, global/static state is strongly discouraged. Instead, any state that your module keeps outside of Python objects must be carefully to the current -sub-interpreter (where, of course, there can now be more than one). Python objects (except -immortal objects) may not be shared between different sub-interpreters, modules must take care not +sub-interpreter (where, of course, there can now be more than one). Python objects (except +immortal objects) may not be shared between different sub-interpreters, modules must take care not to accidentally share Python objects across sub-interpreters. Sub-interpreter Tips: @@ -220,8 +220,8 @@ You can explicitly disable multiple interpreter support in your module by using :func:`multiple_interpreter::not_supported()` tag. This is the default behavior if you do not specify a multiple_interpreters tag. -Note: Sub-interpreter support does not imply free-threading support or vice-versa. -Free-threaded modules can still have global/static state, but multiple interpreter modules cannot. +Note: Sub-interpreter support does not imply free-threading support or vice-versa. +Free-threaded modules can still have global/static state, but multiple interpreter modules cannot. Likewise, sub-interpreter modules can still use the GIL, but free-threaded modules cannot. Binding sequence data types, iterators, the slicing protocol, etc. From a33b31239a946d18991c37139b5eed335d5f3863 Mon Sep 17 00:00:00 2001 From: b-pass Date: Tue, 13 May 2025 21:20:51 -0400 Subject: [PATCH 04/14] Reword for clarity --- docs/advanced/misc.rst | 27 +++++++++++---------------- 1 file changed, 11 insertions(+), 16 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index b8481dbf87..b95e11112c 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -170,9 +170,9 @@ For example: } Note, of course, enabling your module to be used in free threading is also your promise that -your code is thread safe. To actually enable the feature your module must also be compiled -with the Python free-threading source tree. Adding this tag does not break compatibility with -non-free-threaded Python. +your code is thread safe. Modules must still be built against the Python free-threading branch to enable +free-threading, even if they specify this tag. Adding this tag does not break compatibility with non-free-threaded +Python. Sub-interpreter support ================================================================== @@ -191,26 +191,21 @@ For example: // etc } -To make your module sub-interpreter safe, global/static state is strongly discouraged. Instead, -any state that your module keeps outside of Python objects must be carefully to the current -sub-interpreter (where, of course, there can now be more than one). Python objects (except -immortal objects) may not be shared between different sub-interpreters, modules must take care not -to accidentally share Python objects across sub-interpreters. - Sub-interpreter Tips: -- Your initialization function will run for each interpreter that imports your module +- Your initialization function will run for each interpreter that imports your module. -- Never share python objects across interpreter boundaries. +- Never share python objects across different sub-interpreters. -- Keep state it in the interpreter's state dict if necessary. No global/static state! +- Keep state it in the interpreter's state dict if necessary. Avoid global/static state + whenever possible. - Avoid trying to "cache" python objects in C++ variables across function calls (this is an easy - way to accidentally introduce bugs). + way to accidentally introduce sub-interpreter bugs). -- While the interpreters each have their own GIL, isolated/independent - sub-interpreters each have their own lock, so concurrent calls into a module from two different - sub-interpreters are still possible. +- While sub-interpreters each have their own GIL, there can now be multiple independent GILs in one + program so concurrent calls into a module from two different sub-interpreters are still possible. + Therefore, your module still needs to consider thread safety. pybind11 also supports "legacy" sub-interpreters which shared a single global GIL. You can enable legacy behavior by using the :func:`multiple_interpreters::shared_gil()` tag in From 0c5058a20e68c30deba7099acf329b3e5eb7a230 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 14 May 2025 01:21:34 +0000 Subject: [PATCH 05/14] style: pre-commit fixes --- docs/advanced/misc.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index b95e11112c..aeb6c1419f 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -170,9 +170,9 @@ For example: } Note, of course, enabling your module to be used in free threading is also your promise that -your code is thread safe. Modules must still be built against the Python free-threading branch to enable -free-threading, even if they specify this tag. Adding this tag does not break compatibility with non-free-threaded -Python. +your code is thread safe. Modules must still be built against the Python free-threading branch to enable +free-threading, even if they specify this tag. Adding this tag does not break compatibility with non-free-threaded +Python. Sub-interpreter support ================================================================== @@ -197,7 +197,7 @@ Sub-interpreter Tips: - Never share python objects across different sub-interpreters. -- Keep state it in the interpreter's state dict if necessary. Avoid global/static state +- Keep state it in the interpreter's state dict if necessary. Avoid global/static state whenever possible. - Avoid trying to "cache" python objects in C++ variables across function calls (this is an easy From b5bebc3f28d575b81ea32e6ce328c7b807f10264 Mon Sep 17 00:00:00 2001 From: b-pass Date: Wed, 14 May 2025 18:29:10 -0400 Subject: [PATCH 06/14] Suggested changes, including a long-winded explaination of parallelism in python today --- docs/advanced/misc.rst | 138 ++++++++++++++++++++++++++++++++++------- 1 file changed, 116 insertions(+), 22 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index aeb6c1419f..c8dc8ea28b 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -156,9 +156,9 @@ following checklist. Free-threading support ================================================================== -pybind11 supports the experimental free-threaded Python build. pybind11's internal data -structures are thread safe. To enable your modules to be used with free-threading, pass the -:class:`mod_gil_not_used` tag as the third argument to ``PYBIND11_MODULE``. +pybind11 supports the experimental free-threaded Python 3.13t and 3.14t builds. pybind11's +internal data structures are thread safe. To enable your modules to be used with free-threading, +pass the :class:`mod_gil_not_used` tag as the third argument to ``PYBIND11_MODULE``. For example: @@ -170,23 +170,23 @@ For example: } Note, of course, enabling your module to be used in free threading is also your promise that -your code is thread safe. Modules must still be built against the Python free-threading branch to enable -free-threading, even if they specify this tag. Adding this tag does not break compatibility with non-free-threaded -Python. +your code is thread safe. Modules must still be built against the Python free-threading branch to +enable free-threading, even if they specify this tag. Adding this tag does not break +compatibility with non-free-threaded Python. Sub-interpreter support ================================================================== -pybind11 supports isolated sub-interpreters. Pybind11's internal data structures are -sub-interpreter safe. To enable your modules to be imported in isolated sub-interpreters, pass the -:func:`multiple_interpreters::per_interpreter_gil()` tag as the third or later argument to -``PYBIND11_MODULE``. +pybind11 supports isolated sub-interpreters, which are stable in Python 3.12+. Pybind11's +internal data structures are sub-interpreter safe. To enable your modules to be imported in +isolated sub-interpreters, pass the :func:`multiple_interpreters::per_interpreter_gil()` +tag as the third or later argument to ``PYBIND11_MODULE``. For example: .. code-block:: cpp :emphasize-lines: 1 - PYBIND11_MODULE(example, m, py::mod_gil_not_used(), py::multiple_interpreters_per_interpreter_gil()) { + PYBIND11_MODULE(example, m, py::multiple_interpreters_per_interpreter_gil()) { py::class_ animal(m, "Animal"); // etc } @@ -195,29 +195,123 @@ Sub-interpreter Tips: - Your initialization function will run for each interpreter that imports your module. -- Never share python objects across different sub-interpreters. +- Never share Python objects across different sub-interpreters. - Keep state it in the interpreter's state dict if necessary. Avoid global/static state whenever possible. -- Avoid trying to "cache" python objects in C++ variables across function calls (this is an easy +- Modules without any global/static state in their C++ code may already be sub-interpreter safe + without any additional work! + +- Avoid trying to "cache" Python objects in C++ variables across function calls (this is an easy way to accidentally introduce sub-interpreter bugs). - While sub-interpreters each have their own GIL, there can now be multiple independent GILs in one - program so concurrent calls into a module from two different sub-interpreters are still possible. - Therefore, your module still needs to consider thread safety. + program, so concurrent calls into a module from two different sub-interpreters are still + possible. Therefore, your module still needs to consider thread safety. -pybind11 also supports "legacy" sub-interpreters which shared a single global GIL. You can enable -legacy behavior by using the :func:`multiple_interpreters::shared_gil()` tag in -```PYBIND11_MODULE``. +pybind11 also supports "legacy" sub-interpreters which shared a single global GIL. You can enable +legacy-only behavior by using the :func:`multiple_interpreters::shared_gil()` tag in +``PYBIND11_MODULE``. -You can explicitly disable multiple interpreter support in your module by using the +You can explicitly disable sub-interpreter support in your module by using the :func:`multiple_interpreter::not_supported()` tag. This is the default behavior if you do not specify a multiple_interpreters tag. -Note: Sub-interpreter support does not imply free-threading support or vice-versa. -Free-threaded modules can still have global/static state, but multiple interpreter modules cannot. -Likewise, sub-interpreter modules can still use the GIL, but free-threaded modules cannot. +Parallel Python exposition +========================== + +Sub-interpreter support does not imply free-threading support or vice versa. Free-threading safe +modules can still have global/static state (as long as access to them is thread-safe), but +sub-interpreter safe modules cannot. Likewise, sub-interpreter safe modules can still rely on the +GIL, but free-threading safe modules cannot. + +Here is a simple example module which has a function that returns the previous value for a given +key. + +.. code-block:: cpp + PYBIND11_MODULE(example, m) { + static py::dict mydict; + m.def("get_last", [](py::object key, py::object next) { + py::object old = py::none(); + if (mydict.contains(key)) + old = mydict[key]; + mydict[key] = next; + return old; + }); + +This module is not free-threading safe because there are not locks for synchronization. It is +relatively easy to make this free-threading safe: + +.. code-block:: cpp + :emphasize-lines: 1,3,5 + PYBIND11_MODULE(example, m, py::mod_gil_not_used()) { + static py::dict mydict; + static std::mutex mydict_lock; + m.def("get_last", [](py::object key, py::object next) { + std::lock_guard guard(mydict_lock); + py::object old = py::none(); + if (mydict.contains(key)) + old = mydict[key]; + mydict[key] = next; + return old; + }); + } + +The mutex guarantees a consistent behavior from this function even when called currently from +multiple threads at the same time. + +However, the global/static dict is not sub-interpreter safe, because python objects cannot be +shared or moved between interpreters. To fix it, the state needs to be specific to each +interpreter. One way to do that is by storing the state on another Python object. For this +simple example, we will store it in :func:`globals`. + +.. code-block:: cpp + :emphasize-lines: 3,4,5 + PYBIND11_MODULE(example, m, py::multiple_interpreters::per_interpreter_gil()) { + m.def("get_last", [](py::object key, py::object next) { + if (!py::globals().contains("mydict")) + py::globals()["mydict"] = py::dict(); + py::dict mydict = py::globals()["mydict"]; + py::object old = py::none(); + if (mydict.contains(key)) + old = mydict[key]; + mydict[key] = next; + return old; + }); + } + +This module is sub-interpreter safe, for both ``shared_gil`` ("legacy") and +``per_interpreter_gil`` ("default") varieties. Multiple sub-interpreters might each call this same +function concurrently from different threads. This is safe because each sub-interpreter's GIL +protects it's own Python objects from concurrent access. + +However, the module is no longer free-threading safe, because we left out the mutex. If we put it +back, then we can make a module that supports both free-threading and sub-interpreters: + +.. code-block:: cpp + :emphasize-lines: 3,4 + PYBIND11_MODULE(example, m, py::mod_gil_not_used(), py::multiple_interpreters::per_interpreter_gil()) { + m.def("get_last", [](py::object key, py::object next) { + static std::mutex mymutex; + std::lock_guard guard(mymutex); + if (!py::globals().contains("mydict")) + py::globals()["mydict"] = py::dict(); + py::dict mydict = py::globals()["mydict"]; + py::object old = py::none(); + if (mydict.contains(key)) + old = mydict[key]; + mydict[key] = next; + return old; + }); + } + +The module is now both sub-interpreter safe and free-threading safe. The mutex is still +global/static state, so it is still shared between interpreters. The thread-safe nature of +the mutex and the fact that it is not a Python object make it safe to use concurrently in +sub-interpreters. However, it is a slight pessimization to do so, because the sub-interpreters +could block each other unnecessarily. Moving the mutex into per-interpreter storage would solve +this problem. It is left as an exercise for the reader. Binding sequence data types, iterators, the slicing protocol, etc. ================================================================== From 2ea2541541c8e9330bb786b187b9551640509fa4 Mon Sep 17 00:00:00 2001 From: b-pass Date: Wed, 14 May 2025 21:15:07 -0400 Subject: [PATCH 07/14] Slight formatting updates and GPT suggested clarifications --- docs/advanced/misc.rst | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index c8dc8ea28b..82cb4b6507 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -156,20 +156,22 @@ following checklist. Free-threading support ================================================================== -pybind11 supports the experimental free-threaded Python 3.13t and 3.14t builds. pybind11's -internal data structures are thread safe. To enable your modules to be used with free-threading, -pass the :class:`mod_gil_not_used` tag as the third argument to ``PYBIND11_MODULE``. +pybind11 supports the experimental free-threaded builds of Python versions 3.13 and 3.14. +pybind11's internal data structures are thread safe. To enable your modules to be used with +free-threading, pass the :class:`mod_gil_not_used` tag as the third argument to +``PYBIND11_MODULE``. For example: .. code-block:: cpp :emphasize-lines: 1 + PYBIND11_MODULE(example, m, py::mod_gil_not_used()) { py::class_ animal(m, "Animal"); // etc } -Note, of course, enabling your module to be used in free threading is also your promise that +Importantly, enabling your module to be used in free threading is also your promise that your code is thread safe. Modules must still be built against the Python free-threading branch to enable free-threading, even if they specify this tag. Adding this tag does not break compatibility with non-free-threaded Python. @@ -186,12 +188,13 @@ For example: .. code-block:: cpp :emphasize-lines: 1 + PYBIND11_MODULE(example, m, py::multiple_interpreters_per_interpreter_gil()) { py::class_ animal(m, "Animal"); // etc } -Sub-interpreter Tips: +Best Practices for Sub-interpreter Safety: - Your initialization function will run for each interpreter that imports your module. @@ -230,6 +233,7 @@ Here is a simple example module which has a function that returns the previous v key. .. code-block:: cpp + PYBIND11_MODULE(example, m) { static py::dict mydict; m.def("get_last", [](py::object key, py::object next) { @@ -245,6 +249,7 @@ relatively easy to make this free-threading safe: .. code-block:: cpp :emphasize-lines: 1,3,5 + PYBIND11_MODULE(example, m, py::mod_gil_not_used()) { static py::dict mydict; static std::mutex mydict_lock; @@ -268,6 +273,7 @@ simple example, we will store it in :func:`globals`. .. code-block:: cpp :emphasize-lines: 3,4,5 + PYBIND11_MODULE(example, m, py::multiple_interpreters::per_interpreter_gil()) { m.def("get_last", [](py::object key, py::object next) { if (!py::globals().contains("mydict")) @@ -282,7 +288,7 @@ simple example, we will store it in :func:`globals`. } This module is sub-interpreter safe, for both ``shared_gil`` ("legacy") and -``per_interpreter_gil`` ("default") varieties. Multiple sub-interpreters might each call this same +``per_interpreter_gil`` ("default") varieties. Multiple sub-interpreters could each call this same function concurrently from different threads. This is safe because each sub-interpreter's GIL protects it's own Python objects from concurrent access. @@ -290,7 +296,8 @@ However, the module is no longer free-threading safe, because we left out the mu back, then we can make a module that supports both free-threading and sub-interpreters: .. code-block:: cpp - :emphasize-lines: 3,4 + :emphasize-lines: 1,3,4 + PYBIND11_MODULE(example, m, py::mod_gil_not_used(), py::multiple_interpreters::per_interpreter_gil()) { m.def("get_last", [](py::object key, py::object next) { static std::mutex mymutex; @@ -307,11 +314,12 @@ back, then we can make a module that supports both free-threading and sub-interp } The module is now both sub-interpreter safe and free-threading safe. The mutex is still -global/static state, so it is still shared between interpreters. The thread-safe nature of -the mutex and the fact that it is not a Python object make it safe to use concurrently in +global/static state shared between interpreters. But the thread-safe nature of the +mutex and the fact that it is not a Python object make it safe to use concurrently in sub-interpreters. However, it is a slight pessimization to do so, because the sub-interpreters -could block each other unnecessarily. Moving the mutex into per-interpreter storage would solve -this problem. It is left as an exercise for the reader. +could block each other unnecessarily by sharing a global mutex instead of a mutex per-interpreter. +Moving the mutex into per-interpreter storage would solve this problem. It is left as an +exercise for the reader. Binding sequence data types, iterators, the slicing protocol, etc. ================================================================== From e06a2b53d48ec8a7dbe16a9277e4ce56056e8b1c Mon Sep 17 00:00:00 2001 From: b-pass Date: Wed, 14 May 2025 21:19:03 -0400 Subject: [PATCH 08/14] Better section title --- docs/advanced/misc.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index 82cb4b6507..00c474f0e8 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -221,8 +221,8 @@ You can explicitly disable sub-interpreter support in your module by using the :func:`multiple_interpreter::not_supported()` tag. This is the default behavior if you do not specify a multiple_interpreters tag. -Parallel Python exposition -========================== +Concurrency and Parallelism in Python with pybind11 +=================================================== Sub-interpreter support does not imply free-threading support or vice versa. Free-threading safe modules can still have global/static state (as long as access to them is thread-safe), but From 202c7920dd1a3f1e2dc9f7d559e8ca1dac52e19b Mon Sep 17 00:00:00 2001 From: b-pass Date: Wed, 14 May 2025 22:35:53 -0400 Subject: [PATCH 09/14] Update docs/advanced/misc.rst Copilot suggestion Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/advanced/misc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index 00c474f0e8..7b71d904d0 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -189,7 +189,7 @@ For example: .. code-block:: cpp :emphasize-lines: 1 - PYBIND11_MODULE(example, m, py::multiple_interpreters_per_interpreter_gil()) { + PYBIND11_MODULE(example, m, py::multiple_interpreters::per_interpreter_gil()) { py::class_ animal(m, "Animal"); // etc } From 944ab5698e83091a27009152366bf9c4d4868aee Mon Sep 17 00:00:00 2001 From: b-pass Date: Wed, 14 May 2025 22:36:12 -0400 Subject: [PATCH 10/14] Update docs/advanced/misc.rst Copilot suggestion Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/advanced/misc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index 7b71d904d0..0bf40577bd 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -179,7 +179,7 @@ compatibility with non-free-threaded Python. Sub-interpreter support ================================================================== -pybind11 supports isolated sub-interpreters, which are stable in Python 3.12+. Pybind11's +pybind11 supports isolated sub-interpreters, which are stable in Python 3.12+. pybind11's internal data structures are sub-interpreter safe. To enable your modules to be imported in isolated sub-interpreters, pass the :func:`multiple_interpreters::per_interpreter_gil()` tag as the third or later argument to ``PYBIND11_MODULE``. From a0ad2fc59977c40308053ea8e233ce1d83796a82 Mon Sep 17 00:00:00 2001 From: Henry Schreiner Date: Wed, 14 May 2025 23:28:04 -0400 Subject: [PATCH 11/14] Update docs/advanced/misc.rst --- docs/advanced/misc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index 0bf40577bd..1cd7e06212 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -218,7 +218,7 @@ legacy-only behavior by using the :func:`multiple_interpreters::shared_gil()` ta ``PYBIND11_MODULE``. You can explicitly disable sub-interpreter support in your module by using the -:func:`multiple_interpreter::not_supported()` tag. This is the default behavior if you do not +:func:`multiple_interpreters::not_supported()` tag. This is the default behavior if you do not specify a multiple_interpreters tag. Concurrency and Parallelism in Python with pybind11 From 1e7656e7aee2587a26776a36522611ffce6d55e5 Mon Sep 17 00:00:00 2001 From: b-pass Date: Thu, 15 May 2025 07:06:09 -0400 Subject: [PATCH 12/14] Add a deadlock warning, and minor corrections --- docs/advanced/misc.rst | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index 1cd7e06212..cef57d01fc 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -122,6 +122,8 @@ The ``call_go`` wrapper can also be simplified using the ``call_guard`` policy m.def("call_go", &call_go, py::call_guard()); +.. _commongilproblems: + Common Sources Of Global Interpreter Lock Errors ================================================================== @@ -156,7 +158,7 @@ following checklist. Free-threading support ================================================================== -pybind11 supports the experimental free-threaded builds of Python versions 3.13 and 3.14. +pybind11 supports the experimental free-threaded builds of Python versions 3.13+. pybind11's internal data structures are thread safe. To enable your modules to be used with free-threading, pass the :class:`mod_gil_not_used` tag as the third argument to ``PYBIND11_MODULE``. @@ -171,7 +173,7 @@ For example: // etc } -Importantly, enabling your module to be used in free threading is also your promise that +Importantly, enabling your module to be used with free-threading is also your promise that your code is thread safe. Modules must still be built against the Python free-threading branch to enable free-threading, even if they specify this tag. Adding this tag does not break compatibility with non-free-threaded Python. @@ -200,8 +202,9 @@ Best Practices for Sub-interpreter Safety: - Never share Python objects across different sub-interpreters. -- Keep state it in the interpreter's state dict if necessary. Avoid global/static state - whenever possible. +- Avoid global/static state whenever possible. Instead, keep state within each interpreter, + such as in instance members tied to Python objects, :func:`globals()`, and the interpreter + state dict. - Modules without any global/static state in their C++ code may already be sub-interpreter safe without any additional work! @@ -244,7 +247,7 @@ key. return old; }); -This module is not free-threading safe because there are not locks for synchronization. It is +This module is not free-threading safe because there are no locks for synchronization. It is relatively easy to make this free-threading safe: .. code-block:: cpp @@ -264,7 +267,11 @@ relatively easy to make this free-threading safe: } The mutex guarantees a consistent behavior from this function even when called currently from -multiple threads at the same time. +multiple threads at the same time. Note that modules with free-threading support cannot guarantee +that the the GIL is not enabled, so they must still take care not to introduce deadlocks, +including those that can be accidentally introduced by combining the GIL with C++ locks. [#f3]_ + +.. [#f3] See :ref:`commongilproblems` and docs/advanced/deadlock.md for more information. However, the global/static dict is not sub-interpreter safe, because python objects cannot be shared or moved between interpreters. To fix it, the state needs to be specific to each From 58c4e7b52836b61d126a54369da026f4039e1b73 Mon Sep 17 00:00:00 2001 From: b-pass Date: Thu, 15 May 2025 18:36:40 -0400 Subject: [PATCH 13/14] Rewrite the example to avoid free-threading risky behaviors --- docs/advanced/misc.rst | 99 +++++++++++++++++------------------------- 1 file changed, 41 insertions(+), 58 deletions(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index cef57d01fc..71d27edb54 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -232,64 +232,54 @@ modules can still have global/static state (as long as access to them is thread- sub-interpreter safe modules cannot. Likewise, sub-interpreter safe modules can still rely on the GIL, but free-threading safe modules cannot. -Here is a simple example module which has a function that returns the previous value for a given -key. +Here is a simple example module which has a function that calculates a value and returns the result +of the previous calculation. .. code-block:: cpp PYBIND11_MODULE(example, m) { - static py::dict mydict; - m.def("get_last", [](py::object key, py::object next) { - py::object old = py::none(); - if (mydict.contains(key)) - old = mydict[key]; - mydict[key] = next; + static size_t seed = 0; + m.def("calc_next", []() { + auto old = seed; + seed = (seed + 1) * 10; return old; }); -This module is not free-threading safe because there are no locks for synchronization. It is -relatively easy to make this free-threading safe: +This module is not free-threading safe because there is no synchronization on the number variable. +It is relatively easy to make this free-threading safe. One way is by using atomics, like this: .. code-block:: cpp - :emphasize-lines: 1,3,5 + :emphasize-lines: 1,2 PYBIND11_MODULE(example, m, py::mod_gil_not_used()) { - static py::dict mydict; - static std::mutex mydict_lock; - m.def("get_last", [](py::object key, py::object next) { - std::lock_guard guard(mydict_lock); - py::object old = py::none(); - if (mydict.contains(key)) - old = mydict[key]; - mydict[key] = next; + static std::atomic seed(0); + m.def("calc_next", []() { + size_t old, next; + do { + old = seed.load(); + next = (old + 1) * 10; + } while (!seed.compare_exchange_weak(old, next)); return old; }); } -The mutex guarantees a consistent behavior from this function even when called currently from -multiple threads at the same time. Note that modules with free-threading support cannot guarantee -that the the GIL is not enabled, so they must still take care not to introduce deadlocks, -including those that can be accidentally introduced by combining the GIL with C++ locks. [#f3]_ +The atomic variable and the compare-exchange guarantee a consistent behavior from this function even +when called currently from multiple threads at the same time. -.. [#f3] See :ref:`commongilproblems` and docs/advanced/deadlock.md for more information. - -However, the global/static dict is not sub-interpreter safe, because python objects cannot be -shared or moved between interpreters. To fix it, the state needs to be specific to each -interpreter. One way to do that is by storing the state on another Python object. For this -simple example, we will store it in :func:`globals`. +However, the global/static integer is not sub-interpreter safe, because the calls in one +sub-interpreter will change what is seen in another. To fix it, the state needs to be specific to +each interpreter. One way to do that is by storing the state on another Python object, such as a +member of a class. For this simple example, we will store it in :func:`globals`. .. code-block:: cpp - :emphasize-lines: 3,4,5 + :emphasize-lines: 1,6 PYBIND11_MODULE(example, m, py::multiple_interpreters::per_interpreter_gil()) { - m.def("get_last", [](py::object key, py::object next) { - if (!py::globals().contains("mydict")) - py::globals()["mydict"] = py::dict(); - py::dict mydict = py::globals()["mydict"]; - py::object old = py::none(); - if (mydict.contains(key)) - old = mydict[key]; - mydict[key] = next; + m.def("calc_next", []() { + if (!py::globals().contains("myseed")) + py::globals()["myseed"] = 0; + size_t old = py::globals()["myseed"]; + py::globals()["myseed"] = (old + 1) * 10; return old; }); } @@ -299,34 +289,27 @@ This module is sub-interpreter safe, for both ``shared_gil`` ("legacy") and function concurrently from different threads. This is safe because each sub-interpreter's GIL protects it's own Python objects from concurrent access. -However, the module is no longer free-threading safe, because we left out the mutex. If we put it -back, then we can make a module that supports both free-threading and sub-interpreters: +However, the module is no longer free-threading safe, for the same reason as before, because the +calculation is not synchronized. We can synchronize it using a Python critical section. .. code-block:: cpp - :emphasize-lines: 1,3,4 + :emphasize-lines: 1,5,10 - PYBIND11_MODULE(example, m, py::mod_gil_not_used(), py::multiple_interpreters::per_interpreter_gil()) { + PYBIND11_MODULE(example, m, py::multiple_interpreters::per_interpreter_gil(), py::mod_gil_not_used()) { m.def("get_last", [](py::object key, py::object next) { - static std::mutex mymutex; - std::lock_guard guard(mymutex); - if (!py::globals().contains("mydict")) - py::globals()["mydict"] = py::dict(); - py::dict mydict = py::globals()["mydict"]; - py::object old = py::none(); - if (mydict.contains(key)) - old = mydict[key]; - mydict[key] = next; + size_t old; + py::dict g = py::globals(); + Py_BEGIN_CRITICAL_SECTION(g); + if (!g.contains("myseed")) + g["myseed"] = 0; + old = g["myseed"]; + g["myseed"] = (old + 1) * 10; + Py_END_CRITICAL_SECTION(); return old; }); } -The module is now both sub-interpreter safe and free-threading safe. The mutex is still -global/static state shared between interpreters. But the thread-safe nature of the -mutex and the fact that it is not a Python object make it safe to use concurrently in -sub-interpreters. However, it is a slight pessimization to do so, because the sub-interpreters -could block each other unnecessarily by sharing a global mutex instead of a mutex per-interpreter. -Moving the mutex into per-interpreter storage would solve this problem. It is left as an -exercise for the reader. +The module is now both sub-interpreter safe and free-threading safe. Binding sequence data types, iterators, the slicing protocol, etc. ================================================================== From ffb2e312fcc959ccee90a0b53b2e9320279ea65b Mon Sep 17 00:00:00 2001 From: b-pass Date: Thu, 15 May 2025 18:40:33 -0400 Subject: [PATCH 14/14] Oops, missed a line in the example --- docs/advanced/misc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced/misc.rst b/docs/advanced/misc.rst index 71d27edb54..a4b941f9aa 100644 --- a/docs/advanced/misc.rst +++ b/docs/advanced/misc.rst @@ -296,7 +296,7 @@ calculation is not synchronized. We can synchronize it using a Python critical s :emphasize-lines: 1,5,10 PYBIND11_MODULE(example, m, py::multiple_interpreters::per_interpreter_gil(), py::mod_gil_not_used()) { - m.def("get_last", [](py::object key, py::object next) { + m.def("calc_next", []() { size_t old; py::dict g = py::globals(); Py_BEGIN_CRITICAL_SECTION(g);