Blame external/pybind11/docs/advanced/cast/stl.rst

Packit 534379
STL containers
Packit 534379
##############
Packit 534379
Packit 534379
Automatic conversion
Packit 534379
====================
Packit 534379
Packit 534379
When including the additional header file :file:`pybind11/stl.h`, conversions
Packit 534379
between ``std::vector<>``/``std::deque<>``/``std::list<>``/``std::array<>``,
Packit 534379
``std::set<>``/``std::unordered_set<>``, and
Packit 534379
``std::map<>``/``std::unordered_map<>`` and the Python ``list``, ``set`` and
Packit 534379
``dict`` data structures are automatically enabled. The types ``std::pair<>``
Packit 534379
and ``std::tuple<>`` are already supported out of the box with just the core
Packit 534379
:file:`pybind11/pybind11.h` header.
Packit 534379
Packit 534379
The major downside of these implicit conversions is that containers must be
Packit 534379
converted (i.e. copied) on every Python->C++ and C++->Python transition, which
Packit 534379
can have implications on the program semantics and performance. Please read the
Packit 534379
next sections for more details and alternative approaches that avoid this.
Packit 534379
Packit 534379
.. note::
Packit 534379
Packit 534379
    Arbitrary nesting of any of these types is possible.
Packit 534379
Packit 534379
.. seealso::
Packit 534379
Packit 534379
    The file :file:`tests/test_stl.cpp` contains a complete
Packit 534379
    example that demonstrates how to pass STL data types in more detail.
Packit 534379
Packit 534379
.. _cpp17_container_casters:
Packit 534379
Packit 534379
C++17 library containers
Packit 534379
========================
Packit 534379
Packit 534379
The :file:`pybind11/stl.h` header also includes support for ``std::optional<>``
Packit 534379
and ``std::variant<>``. These require a C++17 compiler and standard library.
Packit 534379
In C++14 mode, ``std::experimental::optional<>`` is supported if available.
Packit 534379
Packit 534379
Various versions of these containers also exist for C++11 (e.g. in Boost).
Packit 534379
pybind11 provides an easy way to specialize the ``type_caster`` for such
Packit 534379
types:
Packit 534379
Packit 534379
.. code-block:: cpp
Packit 534379
Packit 534379
    // `boost::optional` as an example -- can be any `std::optional`-like container
Packit 534379
    namespace pybind11 { namespace detail {
Packit 534379
        template <typename T>
Packit 534379
        struct type_caster<boost::optional<T>> : optional_caster<boost::optional<T>> {};
Packit 534379
    }}
Packit 534379
Packit 534379
The above should be placed in a header file and included in all translation units
Packit 534379
where automatic conversion is needed. Similarly, a specialization can be provided
Packit 534379
for custom variant types:
Packit 534379
Packit 534379
.. code-block:: cpp
Packit 534379
Packit 534379
    // `boost::variant` as an example -- can be any `std::variant`-like container
Packit 534379
    namespace pybind11 { namespace detail {
Packit 534379
        template <typename... Ts>
Packit 534379
        struct type_caster<boost::variant<Ts...>> : variant_caster<boost::variant<Ts...>> {};
Packit 534379
Packit 534379
        // Specifies the function used to visit the variant -- `apply_visitor` instead of `visit`
Packit 534379
        template <>
Packit 534379
        struct visit_helper<boost::variant> {
Packit 534379
            template <typename... Args>
Packit 534379
            static auto call(Args &&...args) -> decltype(boost::apply_visitor(args...)) {
Packit 534379
                return boost::apply_visitor(args...);
Packit 534379
            }
Packit 534379
        };
Packit 534379
    }} // namespace pybind11::detail
Packit 534379
Packit 534379
The ``visit_helper`` specialization is not required if your ``name::variant`` provides
Packit 534379
a ``name::visit()`` function. For any other function name, the specialization must be
Packit 534379
included to tell pybind11 how to visit the variant.
Packit 534379
Packit 534379
.. note::
Packit 534379
Packit 534379
    pybind11 only supports the modern implementation of ``boost::variant``
Packit 534379
    which makes use of variadic templates. This requires Boost 1.56 or newer.
Packit 534379
    Additionally, on Windows, MSVC 2017 is required because ``boost::variant``
Packit 534379
    falls back to the old non-variadic implementation on MSVC 2015.
Packit 534379
Packit 534379
.. _opaque:
Packit 534379
Packit 534379
Making opaque types
Packit 534379
===================
Packit 534379
Packit 534379
pybind11 heavily relies on a template matching mechanism to convert parameters
Packit 534379
and return values that are constructed from STL data types such as vectors,
Packit 534379
linked lists, hash tables, etc. This even works in a recursive manner, for
Packit 534379
instance to deal with lists of hash maps of pairs of elementary and custom
Packit 534379
types, etc.
Packit 534379
Packit 534379
However, a fundamental limitation of this approach is that internal conversions
Packit 534379
between Python and C++ types involve a copy operation that prevents
Packit 534379
pass-by-reference semantics. What does this mean?
Packit 534379
Packit 534379
Suppose we bind the following function
Packit 534379
Packit 534379
.. code-block:: cpp
Packit 534379
Packit 534379
    void append_1(std::vector<int> &v) {
Packit 534379
       v.push_back(1);
Packit 534379
    }
Packit 534379
Packit 534379
and call it from Python, the following happens:
Packit 534379
Packit 534379
.. code-block:: pycon
Packit 534379
Packit 534379
   >>> v = [5, 6]
Packit 534379
   >>> append_1(v)
Packit 534379
   >>> print(v)
Packit 534379
   [5, 6]
Packit 534379
Packit 534379
As you can see, when passing STL data structures by reference, modifications
Packit 534379
are not propagated back the Python side. A similar situation arises when
Packit 534379
exposing STL data structures using the ``def_readwrite`` or ``def_readonly``
Packit 534379
functions:
Packit 534379
Packit 534379
.. code-block:: cpp
Packit 534379
Packit 534379
    /* ... definition ... */
Packit 534379
Packit 534379
    class MyClass {
Packit 534379
        std::vector<int> contents;
Packit 534379
    };
Packit 534379
Packit 534379
    /* ... binding code ... */
Packit 534379
Packit 534379
    py::class_<MyClass>(m, "MyClass")
Packit 534379
        .def(py::init<>())
Packit 534379
        .def_readwrite("contents", &MyClass::contents);
Packit 534379
Packit 534379
In this case, properties can be read and written in their entirety. However, an
Packit 534379
``append`` operation involving such a list type has no effect:
Packit 534379
Packit 534379
.. code-block:: pycon
Packit 534379
Packit 534379
   >>> m = MyClass()
Packit 534379
   >>> m.contents = [5, 6]
Packit 534379
   >>> print(m.contents)
Packit 534379
   [5, 6]
Packit 534379
   >>> m.contents.append(7)
Packit 534379
   >>> print(m.contents)
Packit 534379
   [5, 6]
Packit 534379
Packit 534379
Finally, the involved copy operations can be costly when dealing with very
Packit 534379
large lists. To deal with all of the above situations, pybind11 provides a
Packit 534379
macro named ``PYBIND11_MAKE_OPAQUE(T)`` that disables the template-based
Packit 534379
conversion machinery of types, thus rendering them *opaque*. The contents of
Packit 534379
opaque objects are never inspected or extracted, hence they *can* be passed by
Packit 534379
reference. For instance, to turn ``std::vector<int>`` into an opaque type, add
Packit 534379
the declaration
Packit 534379
Packit 534379
.. code-block:: cpp
Packit 534379
Packit 534379
    PYBIND11_MAKE_OPAQUE(std::vector<int>);
Packit 534379
Packit 534379
before any binding code (e.g. invocations to ``class_::def()``, etc.). This
Packit 534379
macro must be specified at the top level (and outside of any namespaces), since
Packit 534379
it instantiates a partial template overload. If your binding code consists of
Packit 534379
multiple compilation units, it must be present in every file (typically via a
Packit 534379
common header) preceding any usage of ``std::vector<int>``. Opaque types must
Packit 534379
also have a corresponding ``class_`` declaration to associate them with a name
Packit 534379
in Python, and to define a set of available operations, e.g.:
Packit 534379
Packit 534379
.. code-block:: cpp
Packit 534379
Packit 534379
    py::class_<std::vector<int>>(m, "IntVector")
Packit 534379
        .def(py::init<>())
Packit 534379
        .def("clear", &std::vector<int>::clear)
Packit 534379
        .def("pop_back", &std::vector<int>::pop_back)
Packit 534379
        .def("__len__", [](const std::vector<int> &v) { return v.size(); })
Packit 534379
        .def("__iter__", [](std::vector<int> &v) {
Packit 534379
           return py::make_iterator(v.begin(), v.end());
Packit 534379
        }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */
Packit 534379
        // ....
Packit 534379
Packit 534379
.. seealso::
Packit 534379
Packit 534379
    The file :file:`tests/test_opaque_types.cpp` contains a complete
Packit 534379
    example that demonstrates how to create and expose opaque types using
Packit 534379
    pybind11 in more detail.
Packit 534379
Packit 534379
.. _stl_bind:
Packit 534379
Packit 534379
Binding STL containers
Packit 534379
======================
Packit 534379
Packit 534379
The ability to expose STL containers as native Python objects is a fairly
Packit 534379
common request, hence pybind11 also provides an optional header file named
Packit 534379
:file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try
Packit 534379
to match the behavior of their native Python counterparts as much as possible.
Packit 534379
Packit 534379
The following example showcases usage of :file:`pybind11/stl_bind.h`:
Packit 534379
Packit 534379
.. code-block:: cpp
Packit 534379
Packit 534379
    // Don't forget this
Packit 534379
    #include <pybind11/stl_bind.h>
Packit 534379
Packit 534379
    PYBIND11_MAKE_OPAQUE(std::vector<int>);
Packit 534379
    PYBIND11_MAKE_OPAQUE(std::map<std::string, double>);
Packit 534379
Packit 534379
    // ...
Packit 534379
Packit 534379
    // later in binding code:
Packit 534379
    py::bind_vector<std::vector<int>>(m, "VectorInt");
Packit 534379
    py::bind_map<std::map<std::string, double>>(m, "MapStringDouble");
Packit 534379
Packit 534379
When binding STL containers pybind11 considers the types of the container's
Packit 534379
elements to decide whether the container should be confined to the local module
Packit 534379
(via the :ref:`module_local` feature).  If the container element types are
Packit 534379
anything other than already-bound custom types bound without
Packit 534379
``py::module_local()`` the container binding will have ``py::module_local()``
Packit 534379
applied.  This includes converting types such as numeric types, strings, Eigen
Packit 534379
types; and types that have not yet been bound at the time of the stl container
Packit 534379
binding.  This module-local binding is designed to avoid potential conflicts
Packit 534379
between module bindings (for example, from two separate modules each attempting
Packit 534379
to bind ``std::vector<int>`` as a python type).
Packit 534379
Packit 534379
It is possible to override this behavior to force a definition to be either
Packit 534379
module-local or global.  To do so, you can pass the attributes
Packit 534379
``py::module_local()`` (to make the binding module-local) or
Packit 534379
``py::module_local(false)`` (to make the binding global) into the
Packit 534379
``py::bind_vector`` or ``py::bind_map`` arguments:
Packit 534379
Packit 534379
.. code-block:: cpp
Packit 534379
Packit 534379
    py::bind_vector<std::vector<int>>(m, "VectorInt", py::module_local(false));
Packit 534379
Packit 534379
Note, however, that such a global binding would make it impossible to load this
Packit 534379
module at the same time as any other pybind module that also attempts to bind
Packit 534379
the same container type (``std::vector<int>`` in the above example).
Packit 534379
Packit 534379
See :ref:`module_local` for more details on module-local bindings.
Packit 534379
Packit 534379
.. seealso::
Packit 534379
Packit 534379
    The file :file:`tests/test_stl_binders.cpp` shows how to use the
Packit 534379
    convenience STL container wrappers.