List Info

Thread: v2 Python toolset doesn't faithfully reproduce v1




v2 Python toolset doesn't faithfully reproduce v1
user name
2006-07-12 22:44:17
At least, that's the only conclusion I can draw from:

http://engineering.meta-comm.com/boost-regression/CVS-RC
_1_34_0/developer/output/Martin%20Wille%20V2-boost-bin-v2-li
bs-python-test-bases-test-gcc-3-4-5_linux-debug_release.html


We used to use this rule:

  # Force statically-linked embedding applications to be
multithreaded
  # on UNIX.
  rule python-static-multithread ( toolset variant :
properties * )
  {
      if ! $(PYTHON_WINDOWS)
      {
          local x = <define>BOOST_PYTHON_STATIC_LIB
<threading>single ;
          if $(x) in $(properties)
          {
              properties = [ difference $(properties) :
<threading>single ] <threading>multi ;
          }
      }
      return $(properties) ;
  }

And in the v2 toolset I see:

    # - There was 'python-static-multithread' logic.
Don't know
    #   what it affected, so can't test.

I don't know what that means.  Clearly from the v1 comment
it affected
statically-linked embedding applications on Unix.

Failure to reproduce this behavior is clearly the problem
with the
'exec' test and several others as well.


I also note this:

    # NOTES:
    # - V1 had logic to force intel to use gcc's runtime.
    #   Note sure why that was needed, with icc 8.0
extensions
    #   built with intel are loaded by python without
problems.

Has the version of Python also been built with Intel?  If so
you won't
see any problems, as the following v1 comment indicates:

  # Normally on Linux, Python is built with GCC.  A
"poor QOI choice"
  # in the implementation of the intel tools prevents the
use of
  # intel-linked shared libs by a GCC-built executable
unless they
  # have been told to use the GCC runtime.  This rule adds
the
  # requisite flags to the compile and link lines.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
v2 Python toolset doesn't faithfully reproduce v1
user name
2006-07-13 07:19:53
On Thursday 13 July 2006 02:44, David Abrahams wrote:
> At least, that's the only conclusion I can draw from:
>
> http://engineering.meta-comm.com/boos
t-regression/CVS-RC_1_34_0/developer/o
>utput/Martin%20Wille%20V2-boost-bin-v2-libs-python-test-
bases-test-gcc-3-4-5
>_linux-debug_release.html

To begin with, there are two results for that test, as shown
at:

http://engineering.meta-comm.com
/boost-regression/CVS-RC_1_34_0/developer/Martin 
Wille V2-python-gcc-3-4-5_linux-bases-variants_.html

The failure you see is pretty old, and should no longer
reproduce.

Martin, can you remove all Boost.Python results.


> We used to use this rule:
>
>   # Force statically-linked embedding applications to
be multithreaded
>   # on UNIX.
>   rule python-static-multithread ( toolset variant :
properties * )
>   {
>       if ! $(PYTHON_WINDOWS)
>       {
>           local x =
<define>BOOST_PYTHON_STATIC_LIB
<threading>single ;
>           if $(x) in $(properties)
>           {
>               properties = [ difference $(properties) :
<threading>single ]
> <threading>multi ; }
>       }
>       return $(properties) ;
>   }
>
> And in the v2 toolset I see:
>
>     # - There was 'python-static-multithread' logic.
Don't know
>     #   what it affected, so can't test.
>
> I don't know what that means.  Clearly from the v1
comment it affected
> statically-linked embedding applications on Unix.

We've being though this before, and you basically agreed
that this comment is 
not accurate. Please see the attached message. This topic
then died.

Here's the problem as I understand it.

When you link to Boost.Python for embedding purposes, you
need to link to some 
extra libs, such as 'dl', 'util' and 'pthread'. This
need does not depend on 
static/shared kind of Boost.Python library.

When you link to Boost.Python from extension module, the
Python interpreter 
that loads the extension is already linked to the right
libraries.

In V1, the 'dl' and 'util' libraries are handled via the

PYTHON_EMBEDDED_LIBRARY, which is supposed to be use in
requirements of 
embedding executable? 

Question 1. Why 'pthread' is handled using different
mechanism? Just adding
'pthread' to PYTHON_EMBEDDED_LIBRARY will work. Is there
any reason you want
<threading-multi>, not just -lpthread?

Question 2. Why 'python-static-multithread' is used in
PYTHON_REQUIREMENTS, as 
opposed to being used in some other variable used only be
embedding 
executables?

Question 3. Why can't Boost.Python unconditionally link to
'pthread', 'util' 
and 'dl' libraries?

Question 4. Looking at V2 Jamfiles, it seems that
Boost.Python links directly 
to Python library. If I understand correctly, it's not good
for extending, or 
what?

If so, and Boost.Python itself should not link to Python
library, what 
approach is best:

   - Document that for embedding, application should be
explicitly linked to
     Python
   - Add new target "boost_python_for_embedded",
that will alias to
     Boost.Python, Python, and any auxilliary libraries.
 

> I also note this:
>
>     # NOTES:
>     # - V1 had logic to force intel to use gcc's
runtime.
>     #   Note sure why that was needed, with icc 8.0
extensions
>     #   built with intel are loaded by python without
problems.
>
> Has the version of Python also been built with Intel?  

No, it's stock system Python. I don't think I ever built
Python in life 

> If so you won't 
> see any problems, as the following v1 comment
indicates:
>
>   # Normally on Linux, Python is built with GCC.  A
"poor QOI choice"
>   # in the implementation of the intel tools prevents
the use of
>   # intel-linked shared libs by a GCC-built executable
unless they
>   # have been told to use the GCC runtime.  This rule
adds the
>   # requisite flags to the compile and link lines.


- Volodya

-- 
Vladimir Prus
http://vladimir_pru
s.blogspot.com
Boost.Build V2: http://boost.org/boost-
build2
Vladimir Prus <ghostcs.msu.su> writes:

> David Abrahams wrote:
>
>> It made things work 
>> 
>> IIRC, libpythonXX.a on *nix requires these
libraries.  Why is this
>> mysterious?
>
> Why it requires -pthread only whem
<define>BOOST_PYTHON_STATIC_LIB is in
> properties. 

I don't know, unfortunately.

>> There are two usual ways to use Python on *nix:
>> 
>> 1. extending, where the Python executable
dynamically loads Python
>>    extension modules and they use the Python API
that's statically
>>    linked into the Python executable.  In this
model, the Python
>>    executable is linked to whatever libraries it
needs, such as
>>    pthreads and dl.
>> 
>> 2. embedding, where a user-written executable uses
the Python API in
>>    libpythonXX.a.  In that case, who is going to
link to pthreads and
>>    dl if not the user-written executable.
>
> Then, how <define>BOOST_PYTHON_STATIC_LIB is
related to those
> cases. 

It shouldn't be.

> It's quite feasible to embed python and use shared
linking to
> Boost.Python at the same time.

Yes.

>>> 2. In V2, shared linking to Boost.Python is
used, and the
>>> application is fully shared. So, where does the
link error come
>>> from? Is comment even more inaccurate?
>> 
>> Than what?
>
> I mean, -pthread is required for all embedding, even if
linking to
> Boost.Python is shared. Right?

I believe so.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

_______________________________________________
Unsubscribe & other changes: htt
p://lists.boost.org/mailman/listinfo.cgi/boost


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
v2 Python toolset doesn't faithfully reproduce v1
user name
2006-07-13 14:09:00
Vladimir Prus <ghostcs.msu.su> writes:

> We've being though this before, and you basically
agreed that this comment is 
> not accurate. Please see the attached message. This
topic then died.

Do we need it to un-die, or do you think all the problems
are
resolved?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
Python and Intel
user name
2006-07-13 14:08:23
Vladimir Prus <ghostcs.msu.su> writes:

>> I also note this:
>>
>>     # NOTES:
>>     # - V1 had logic to force intel to use gcc's
runtime.
>>     #   Note sure why that was needed, with icc 8.0
extensions
>>     #   built with intel are loaded by python
without problems.
>>
>> Has the version of Python also been built with
Intel?  
>
> No, it's stock system Python. I don't think I ever
built Python in life 
>
>> If so you won't 
>> see any problems, as the following v1 comment
indicates:
>>
>>   # Normally on Linux, Python is built with GCC.  A
"poor QOI choice"
>>   # in the implementation of the intel tools
prevents the use of
>>   # intel-linked shared libs by a GCC-built
executable unless they
>>   # have been told to use the GCC runtime.  This
rule adds the
>>   # requisite flags to the compile and link lines.


I think this *might* be a change they made in Intel 9.0,
which is why
you're not seeing a problem.  I had an extensive discussion
about this
with Intel's support people because it *was* a real
problem.  Ralf, do
you remember this issue?  Maybe it has to do with rtti or
exceptions
not working across .so boundaries?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
v2 Python toolset doesn't faithfully reproduce v1
user name
2006-07-13 15:20:59
On Thursday 13 July 2006 18:09, David Abrahams wrote:
> Vladimir Prus <ghostcs.msu.su> writes:
> > We've being though this before, and you basically
agreed that this
> > comment is not accurate. Please see the attached
message. This topic then
> > died.
>
> Do we need it to un-die, or do you think all the
problems are
> resolved?

I think we need it to un-die. The two current issues are:

1. I've modified Boost.Python tests Jamfile.v2 to have
<threading>multi. I 
think that fixes the errors, but is not a clean solution.
I'd prefer direct 
linking to pthread.

2. I noticed that in V2 Boost.Python links to Python
library. This sounds 
potentially problematic, I'm not sure if Python interpreter
will completely 
hide all symbols from Python shared lib.

- Volodya

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
v2 Python toolset doesn't faithfully reproduce v1
user name
2006-07-13 16:05:20
Vladimir Prus <ghostcs.msu.su> writes:

> On Thursday 13 July 2006 18:09, David Abrahams wrote:
>> Vladimir Prus <ghostcs.msu.su> writes:
>> > We've being though this before, and you
basically agreed that this
>> > comment is not accurate. Please see the
attached message. This topic then
>> > died.
>>
>> Do we need it to un-die, or do you think all the
problems are
>> resolved?
>
> I think we need it to un-die. The two current issues
are:
>
> 1. I've modified Boost.Python tests Jamfile.v2 to have
<threading>multi. I 
> think that fixes the errors, but is not a clean
solution. I'd prefer direct 
> linking to pthread.

Why?  For example, do you believe the other options that
<threading>multi might add are inappropriate, and if
so, why do you
think that?

> 2. I noticed that in V2 Boost.Python links to Python
library. 

Too vague to understand.

* Windows, Unix, both...?

* You mean the boost_python.so/dll links to
libpython.a/pythonXX.lib?

> This sounds potentially problematic, I'm not sure if
Python
> interpreter will completely hide all symbols from
Python shared lib.

Sorry, you lost me completely here.  Whose symbols?  How
would a
Python interpreter hide symbols?

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
v2 Python toolset doesn't faithfully reproduce v1
user name
2006-08-22 06:37:56
On Thursday 13 July 2006 02:44, David Abrahams wrote:

> I also note this:
>
>     # NOTES:
>     # - V1 had logic to force intel to use gcc's
runtime.
>     #   Note sure why that was needed, with icc 8.0
extensions
>     #   built with intel are loaded by python without
problems.
>
> Has the version of Python also been built with Intel? 
If so you won't
> see any problems, as the following v1 comment
indicates:
>
>   # Normally on Linux, Python is built with GCC.  A
"poor QOI choice"
>   # in the implementation of the intel tools prevents
the use of
>   # intel-linked shared libs by a GCC-built executable
unless they
>   # have been told to use the GCC runtime.  This rule
adds the
>   # requisite flags to the compile and link lines.

Sorry for continuing being dense, but I can't find any
outstanding failures in 
Python tests on intel-9.0 (Martin Wille V2). So:

  - is this problem fixed in 9.0
  - is there any test that would have caught this problem
and that I can run
    with intel 8.0 to check?

-- 
Vladimir Prus
http://vladimir_pru
s.blogspot.com
Boost.Build V2: http://boost.org/boost-
build2
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
v2 Python toolset doesn't faithfully reproduce v1
user name
2006-08-22 06:37:56
On Thursday 13 July 2006 02:44, David Abrahams wrote:

> I also note this:
>
>     # NOTES:
>     # - V1 had logic to force intel to use gcc's
runtime.
>     #   Note sure why that was needed, with icc 8.0
extensions
>     #   built with intel are loaded by python without
problems.
>
> Has the version of Python also been built with Intel? 
If so you won't
> see any problems, as the following v1 comment
indicates:
>
>   # Normally on Linux, Python is built with GCC.  A
"poor QOI choice"
>   # in the implementation of the intel tools prevents
the use of
>   # intel-linked shared libs by a GCC-built executable
unless they
>   # have been told to use the GCC runtime.  This rule
adds the
>   # requisite flags to the compile and link lines.

Sorry for continuing being dense, but I can't find any
outstanding failures in 
Python tests on intel-9.0 (Martin Wille V2). So:

  - is this problem fixed in 9.0
  - is there any test that would have caught this problem
and that I can run
    with intel 8.0 to check?

-- 
Vladimir Prus
http://vladimir_pru
s.blogspot.com
Boost.Build V2: http://boost.org/boost-
build2
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
v2 Python toolset doesn't faithfully reproduce v1
user name
2006-08-22 11:21:03
Vladimir Prus <ghostcs.msu.su> writes:

> On Thursday 13 July 2006 02:44, David Abrahams wrote:
>
>> I also note this:
>>
>>     # NOTES:
>>     # - V1 had logic to force intel to use gcc's
runtime.
>>     #   Note sure why that was needed, with icc 8.0
extensions
>>     #   built with intel are loaded by python
without problems.
>>
>> Has the version of Python also been built with
Intel?  If so you won't
>> see any problems, as the following v1 comment
indicates:
>>
>>   # Normally on Linux, Python is built with GCC.  A
"poor QOI choice"
>>   # in the implementation of the intel tools
prevents the use of
>>   # intel-linked shared libs by a GCC-built
executable unless they
>>   # have been told to use the GCC runtime.  This
rule adds the
>>   # requisite flags to the compile and link lines.
>
> Sorry for continuing being dense, but I can't find any
outstanding failures in 
> Python tests on intel-9.0 (Martin Wille V2). So:
>
>   - is this problem fixed in 9.0

I don't know.

>   - is there any test that would have caught this
problem and that I can run
>     with intel 8.0 to check?

I don't know.

It may be (in fact it's likely) that Intel and GCC once had
slight ABI
differences but have since converged.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
v2 Python toolset doesn't faithfully reproduce v1
user name
2006-08-25 07:06:52
On Tuesday 22 August 2006 15:21, you wrote:
> Vladimir Prus <ghostcs.msu.su> writes:
> > On Thursday 13 July 2006 02:44, David Abrahams
wrote:
> >> I also note this:
> >>
> >>     # NOTES:
> >>     # - V1 had logic to force intel to use
gcc's runtime.
> >>     #   Note sure why that was needed, with
icc 8.0 extensions
> >>     #   built with intel are loaded by python
without problems.
> >>
> >> Has the version of Python also been built with
Intel?  If so you won't
> >> see any problems, as the following v1 comment
indicates:
> >>
> >>   # Normally on Linux, Python is built with
GCC.  A "poor QOI choice"
> >>   # in the implementation of the intel tools
prevents the use of
> >>   # intel-linked shared libs by a GCC-built
executable unless they
> >>   # have been told to use the GCC runtime. 
This rule adds the
> >>   # requisite flags to the compile and link
lines.
> >
> > Sorry for continuing being dense, but I can't
find any outstanding
> > failures in Python tests on intel-9.0 (Martin
Wille V2). So:
> >
> >   - is this problem fixed in 9.0
>
> I don't know.
>
> >   - is there any test that would have caught this
problem and that I can
> > run with intel 8.0 to check?
>
> I don't know.
>
> It may be (in fact it's likely) that Intel and GCC
once had slight ABI
> differences but have since converged.

So, do you think it would be wise not to spend time on
addressing a problem 
that does not show up in regression tests, and for which
it's not know how to 
make Boost.Python fail?

- Volodya
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build
[1-10] [11]

about | contact  Other archives ( Real Estate discussion Medical topics )