Smart pointers
std::unique_ptr
Given a class Example
with Python bindings, it’s possible to return
instances wrapped in C++11 unique pointers, like so
std::unique_ptr<Example> create_example() { return std::unique_ptr<Example>(new Example()); }
m.def("create_example", &create_example);
In other words, there is nothing special that needs to be done. While returning
unique pointers in this way is allowed, it is illegal to use them as function
arguments. For instance, the following function signature cannot be processed
by pybind11.
void do_something_with_example(std::unique_ptr<Example> ex) { ... }
The above signature would imply that Python needs to give up ownership of an
object that is passed to this function, which is generally not possible (for
instance, the object might be referenced elsewhere).
std::shared_ptr
The binding generator for classes, class_
, can be passed a template
type that denotes a special holder type that is used to manage references to
the object. If no such holder type template argument is given, the default for
a type named Type
is std::unique_ptr<Type>
, which means that the object
is deallocated when Python’s reference count goes to zero.
It is possible to switch to other types of reference counting wrappers or smart
pointers, which is useful in codebases that rely on them. For instance, the
following snippet causes std::shared_ptr
to be used instead.
py::class_<Example, std::shared_ptr<Example> /* <- holder type */> obj(m, "Example");
Note that any particular class can only be associated with a single holder type.
One potential stumbling block when using holder types is that they need to be
applied consistently. Can you guess what’s broken about the following binding
code?
class Child { };
class Parent {
public:
Parent() : child(std::make_shared<Child>()) { }
Child *get_child() { return child.get(); } /* Hint: ** DON'T DO THIS ** */
private:
std::shared_ptr<Child> child;
};
PYBIND11_PLUGIN(example) {
py::module m("example");
py::class_<Child, std::shared_ptr<Child>>(m, "Child");
py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent")
.def(py::init<>())
.def("get_child", &Parent::get_child);
return m.ptr();
}
The following Python code will cause undefined behavior (and likely a
segmentation fault).
from example import Parent
print(Parent().get_child())
The problem is that Parent::get_child()
returns a pointer to an instance of
Child
, but the fact that this instance is already managed by
std::shared_ptr<...>
is lost when passing raw pointers. In this case,
pybind11 will create a second independent std::shared_ptr<...>
that also
claims ownership of the pointer. In the end, the object will be freed twice
since these shared pointers have no way of knowing about each other.
There are two ways to resolve this issue:
- For types that are managed by a smart pointer class, never use raw pointers
in function arguments or return values. In other words: always consistently
wrap pointers into their designated holder types (such as
std::shared_ptr<...>
). In this case, the signature of get_child()
should be modified as follows:
std::shared_ptr<Child> get_child() { return child; }
- Adjust the definition of
Child
by specifying
std::enable_shared_from_this<T>
(see cppreference for details) as a
base class. This adds a small bit of information to Child
that allows
pybind11 to realize that there is already an existing
std::shared_ptr<...>
and communicate with it. In this case, the
declaration of Child
should look as follows:
class Child : public std::enable_shared_from_this<Child> { };
Custom smart pointers
pybind11 supports std::unique_ptr
and std::shared_ptr
right out of the
box. For any other custom smart pointer, transparent conversions can be enabled
using a macro invocation similar to the following. It must be declared at the
top namespace level before any binding code:
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>);
The first argument of PYBIND11_DECLARE_HOLDER_TYPE()
should be a
placeholder name that is used as a template parameter of the second argument.
Thus, feel free to use any identifier, but use it consistently on both sides;
also, don’t use the name of a type that already exists in your codebase.
The macro also accepts a third optional boolean parameter that is set to false
by default. Specify
PYBIND11_DECLARE_HOLDER_TYPE(T, SmartPtr<T>, true);
if SmartPtr<T>
can always be initialized from a T*
pointer without the
risk of inconsistencies (such as multiple independent SmartPtr
instances
believing that they are the sole owner of the T*
pointer). A common
situation where true
should be passed is when the T
instances use
intrusive reference counting.
Please take a look at the General notes regarding convenience macros before using this feature.
See also
The file tests/test_smart_ptr.cpp
contains a complete example
that demonstrates how to work with custom reference-counting holder types
in more detail.