Boost.Python and garbage collection

Hi,

I've run into a problem with cyclic references between
instances of objects bounds with boost::python - the garbage
collector doesn't seem to be able to collect them. This is
illustrated by the python code below :



import weakref
import gc

# make a cyclic reference between instances of a bound
class
o1 = BoostPythonClass()
o2 = BoostPythonClass()
o1.r = o2
o2.r = o1

# make weak references to track their lifetime
w1 = weakref.ref( o1 )
w2 = weakref.ref( o2 )

# delete the instances and run the garbage collector
del o1
del o2
gc.collect()

# check that the objects have been collected
assert( w1() is None )
assert( w2() is None )



I dug a bit deeper into this and had a go at fixing things
by implementing garbage collection support for instances in
src/object/class.cpp. By implementing a traversal to
instance::dict the problem seems to be fixed and the asserts
in the test above pass. I've also run the unit tests for a
largish project using boost python with this modified code
and they pass ok. The changes are pretty slight - here's the
output from diff :



354a355,373
>         
>       static int instance_traverse( PyObject* op,
visitproc visit, void* arg )
>       {
>          instance<>* inst =
downcast<instance<>>(op);
>          if( inst->dict )
>          {
>             return visit( inst->dict, arg );
>          }
>          return 0;
>       }
> 
>       static int instance_clear( PyObject* op )
>       {
>          instance<>* inst =
downcast<instance<>>(op);
>          PyObject* tmp = inst->dict;
>          inst->dict = 0;
>          python:decre
f( tmp );
>          return 0;
>       }
391c410
<       Py_TPFLAGS_DEFAULT // | Py_TPFLAGS_HAVE_GC
---
>       Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC
394,395c413,414
<       0,                                      /*
tp_traverse */
<       0,                                      /*
tp_clear */
---
>       instance_traverse,                      /*
tp_traverse */
>       instance_clear,                         /*
tp_clear */


The internals of boost::python are a bit of a mystery to me,
so I don't know if I've missed something or if there's a
good reason this isn't implemented already. I'm posting to
the list in the hope that it might be useful and if I
haven't messed up could get merged into the source.

Cheers...
John
____________________________________________________________
_____
Express yourself with free Messenger emoticons. Get them
today!

http://www.freemessengeremoticons.ca/?icid=EMENCA122
_______________________________________________
C++-sig mailing list
C++-sigpython.org
http:
//mail.python.org/mailman/listinfo/c++-sig

>># delete the instances and run the garbage
collector
>>del o1
>>del o2
>>gc.collect()
>
>Maybe this doesn't invalidate your conclusions, but just
a heads up: don't simply call " gc.collect()",
instead do>"while gc.collect(): pass".  That is
because sometimes multiple GC iterations are required to
collect all objects.

Thanks for the tip. I've rerun the tests with your adjusted
code and fortunately my mistake doesn't invalidate the
original conclusion - which makes sense as without the
additional code there's no way for the garbage collector to
see the contents of the dict for the instance...
____________________________________________________________
_____
Send a smile, make someone laugh, have some fun! Start now!

http://www.freemessengeremoticons.ca/?icid=EMENCA122
_______________________________________________
C++-sig mailing list
C++-sigpython.org
http:
//mail.python.org/mailman/listinfo/c++-sig