Is it a BUG? boost.python makes doctest impossible to recursive test.

Today, I encounter a tough problem about using doctest in
class
exposed by boost.python.

For traditional python class which is define in python:
class P:
"""
>>> ....   # this line will be tested by doctest
"""
    def func(self):
    """
    >>>...  # this line will be tested by doctest
by recursively
discovering __doc__
    """
    pass

However, if I expose the same class by boost.python,
class C
{
public:
    void func() {}
};

char* doc_str[] =
{
// __doc__ for class
">>>...",  // this line will be tested by
doctest
// __doc__ for func
">>>..."   // this line will NOT be
tested by doctest
};

BOOST_PYTHON_MODULE(...) {
    class_<C>("C", doc_str[0],
init<>())
        .def("func", &C::func, doc_str[1]);
}

In above case, doctest can only discover the __doc__ for
class, but
can NOT go further to discover __doc__ for method. I
throughly studied
the reason, and found the problem is:

In doctest.py
 class DocTestFinder:
    def _find(self, tests, obj, name, module, source_lines,
globs, seen):
        .....
        # Look for tests in a class's contained objects.
        if inspect.isclass(obj) and self._recurse:
            for valname, val in obj.__dict__.items():
                .....
                # Recurse to methods, properties, and nested
classes.
BUT IT FAILS.
                if ((inspect.isfunction(val) or
inspect.isclass(val) or
                      isinstance(val, property)) and
self._from_module(module, val)):
                    ....

inspect.isfunction(val) ALWAYS returns False, because the
type of
'val' corresponding to 'func' in class C is:
<Boost.Python.function object at 0x....>
while for class P, the type of 'val' is
<function func at 0x....>, which will get True.

Therefore, the strange type of the class' method exposed by
boost.python suppresses the doctest to recursively discover
__doc__.
What should I do? Is is a bug for boost.python?
Thank you

---
ShenLei
_______________________________________________
C++-sig mailing list
C++-sigpython.org
http:
//mail.python.org/mailman/listinfo/c++-sig

Hi 甜瓜!

You wrote:
> inspect.isfunction(val) ALWAYS returns False, because
the type of
> 'val' corresponding to 'func' in class C is:
> <Boost.Python.function object at 0x....>
Here, I am getting <type 'instancemethod'>, with MRO
[<type 'instancemethod'>, 
<type 'object'>].

> Therefore, the strange type of the class' method
exposed by
> boost.python suppresses the doctest to recursively
discover __doc__.
> What should I do? Is is a bug for boost.python?
This relates to my recent postings on BPL and epydoc
("Better 
introspection/apidoc extraction support?" thread).  It
would be really cool 
if the inspect module could be enhanced to work with
boost::python.

AFAICS, for much functionality it would be needed to enhance
boost::python, 
since much information is not even available.  For example,
the signature of 
functions is not available - now there is a string
representation in the 
docstrings (only not disabled), but it is impossible to
parse in general, 
since values of default arguments can have any string
representation.
A contrived example would be an object o with repr(o) ==
"1, foo = 23" which 
makes the parser believe that there's another argument, but
there are many 
real-world cases where the repr() output does not allow
creating an identical 
object, or not even recognizing the type.

In your case, I see two solutions: Enhancing the 'inspect'
module to recognize 
boost::python functions or make BPL methods be instances of
the right type.
But wait.. the latter already seems to be the case with
1.34.0beta:  
<type 'instancemethod'> is types.MethodType!
Consequently, inspect.ismethod() returns True, but
inspect.isfunction() 
returns False.

However, I am getting <Boost.Python.function object at
0x....> for free 
functions (I thought your example was about the method
C.func?):

In [11]: type(norm)
Out[11]: <type 'Boost.Python.function'>

In [12]: _.mro()
Out[12]: [<type 'Boost.Python.function'>, <type
'object'>]

Maybe Boost.Python.function could be derived from
types.FunctionType?
(Probably not, otherwise it would have been done already.)

BTW: A good example for deriving from python types is numpy:
In [16]: numpy.float64.mro()
Out[16]:
[<type 'numpy.float64'>,
 <type 'numpy.floating'>,
 <type 'numpy.inexact'>,
 <type 'numpy.number'>,
 <type 'numpy.generic'>,
 <type 'float'>,
 <type 'object'>]

Ciao, /  /
     /--/
    /  / ANS
_______________________________________________
C++-sig mailing list
C++-sigpython.org
http:
//mail.python.org/mailman/listinfo/c++-sig

Thank you for your kind reply. I agree with you for
extending inspect
module to cooperate with boost.python.

> However, I am getting <Boost.Python.function object
at 0x....> for free
> functions (I thought your example was about the method
C.func?):
>
> In [11]: type(norm)
> Out[11]: <type 'Boost.Python.function'>

Well, your result is true, but, in my case, I didn't invoke
isfunction
or ismethod on C.func.
Indeed, ismethod(C.func) will return true. But inspect
module use
'isfunction' to determine whether go recursively.
if ((inspect.isfunction(val) or inspect.isclass(val) or
isinstance(val, property))
     and self._from_module(module, val)):  .....
Here the val comes from C.__dict__['func']. If you print
C.__dict__,
you will find a pair:
'func' : <Boost.Python.function object at 0x....>

So:
    t = BPLclass.__dict__['func']
    inspect.ismethod(t)   # False
    inspect.isfunction(t)  # False
    inspect.ismethod(BPLclass.func)   # True
    inspect.isfunction(BPLclass.func)  # False

    s = Pythonclass.__dict__['func']
    inspect.ismethod(t)   # False
    inspect.isfunction(t)  # True
    inspect.ismethod(Pythonclass.func)   # True
    inspect.isfunction(Pythonclass.func)  # False

Really interesting, Eh?

---
ShenLei
_______________________________________________
C++-sig mailing list
C++-sigpython.org
http:
//mail.python.org/mailman/listinfo/c++-sig

Am Montag, 11. Juni 2007 14:17:37 schrieb 甜瓜:
> Here the val comes from C.__dict__['func']. If you
print C.__dict__,
> you will find a pair:
> 'func' : <Boost.Python.function object at 0x....>

Ah, that is interesting.  So that's obviously the difference
between 
staticmethods and normal ones - I did not know that
that happens as late as 
when writing C.func, but I thought it was a matter of class
definition time.

Note to myself: type(C.func) != type(C.__dict__['func'])

Which also explains why methods have the right type in
boost::python.. they 
haven't. 

-- 
Ciao, /  /
     /--/
    /  / ANS
_______________________________________________
C++-sig mailing list
C++-sigpython.org
http:
//mail.python.org/mailman/listinfo/c++-sig