linking with system libs even though rpath is used

Greeting everyone-

I'm running into an issue that quite frankly, I'm surprised
I haven't
hit before.

I'll try and explain what is happening.  I'm installing
openldap
2.3.27-r3.  In openldap there are two libraries libldap and
libldap_r
that dynamically link with another library liblber (also
included in
openldap).

On a redhat and suse system that I'm building on the system
also has a
liblber.  And both libldap and libldap_r are linking to it
even though
the rpath linking states to look in $/usr/lib.

Interestingly when I go into the portage work directory,
running ldd
on the libldap there shows that it is linking to the correct
version
of liblber.  But when it gets installed into $ the
libldap no
longer links correctly (now linking against the system
liblber).

I belive this is happening because the following command is
issued
during the install sequence::
     PATH="$PATH:/sbin" ldconfig -n
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib

At this point the rpath suggestion points to
/opt/prefix/usr/lib and
not $/opt/prefix/usr/lib (which is where the correct
liblber is at
this point), hence it falls back to the system library.

So anyone have any ideas on how to fix this?  One option is
emerging
openldap twice, since the second time you install liblber
will be
installed where the rpath linking points.  A hack (that
works for
redhat but not suse) is to use sed to update the references
to liblber
in libldap and libldap_r.

Ideas, suggestions????

-matt
-- 
gentoo-altgentoo.org mailing list

Hi Matt,

I don't completely get the issue at hand, but from what you
tell, I get
the suspicion that the rpath used during linking is the
WORKDIR, and not
the real prefix location itself.  This causes the right lib
not to be
found after install.

I'm quite sure the new linker setup as Michael (haubi) has
experimented
with on Solaris will solve this, as the correct rpath is
also passed
on to ld.

If you want to try it out, replace your ld with a script
that calls the
real ld with the rpaths for your prefix and see if that does
work.


On 22-12-2006 12:10:44 -0700, m h wrote:
> Greeting everyone-
> 
> I'm running into an issue that quite frankly, I'm
surprised I haven't
> hit before.
> 
> I'll try and explain what is happening.  I'm installing
openldap
> 2.3.27-r3.  In openldap there are two libraries libldap
and libldap_r
> that dynamically link with another library liblber
(also included in
> openldap).
> 
> On a redhat and suse system that I'm building on the
system also has a
> liblber.  And both libldap and libldap_r are linking to
it even though
> the rpath linking states to look in $/usr/lib.
> 
> Interestingly when I go into the portage work
directory, running ldd
> on the libldap there shows that it is linking to the
correct version
> of liblber.  But when it gets installed into $ the
libldap no
> longer links correctly (now linking against the system
liblber).
> 
> I belive this is happening because the following
command is issued
> during the install sequence::
>     PATH="$PATH:/sbin" ldconfig -n
>
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib
> 
> At this point the rpath suggestion points to
/opt/prefix/usr/lib and
> not $/opt/prefix/usr/lib (which is where the correct
liblber is at
> this point), hence it falls back to the system library.
> 
> So anyone have any ideas on how to fix this?  One
option is emerging
> openldap twice, since the second time you install
liblber will be
> installed where the rpath linking points.  A hack (that
works for
> redhat but not suse) is to use sed to update the
references to liblber
> in libldap and libldap_r.
> 
> Ideas, suggestions????
> 
> -matt

-- 
Fabian Groffen
Gentoo on a different level
-- 
gentoo-altgentoo.org mailing list

On 12/22/06, Fabian Groffen <grobiangentoo.org> wrote:
> Hi Matt,
>
> I don't completely get the issue at hand, but from what
you tell, I get
> the suspicion that the rpath used during linking is the
WORKDIR, and not
> the real prefix location itself.  This causes the right
lib not to be
> found after install.
>

hmmm "eu-readelf -d path/to/libldap..." reports
that the rpath is
/opt/prefix/usr/lib ....

Maybe I didn't make myself clear.  When I run ldd on the
libraries
(libldap and libldap_r) from the workdir it reports linking
to the
correct liblber.  When I run ldd on the libraries in $ or
those
that are in the prefix they link to the system liblber.

Here's the log of what happens when (one of the libraries,
libldap_r)
it gets installed::

  Entering subdirectory libldap_r
make[2]: Entering directory
`/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/work/openlda
p-2.3.27/libraries/libldap_r'
...
cc -shared  .libs/threads.o .libs/rdwr.o .libs/tpool.o
.libs/rq.o
.libs/thr_posix.o .libs/thr_cthreads.o .libs/thr_thr.o
.libs/thr_lwp.o
.libs/thr_nt.o .libs/thr_pth.o .libs/thr_stub.o
.libs/thr_debug.o .libs/bind.o
.libs/open.o .libs/result.o .libs/error.o .libs/compare.o
.libs/search.o
.libs/controls.o .libs/messages.o .libs/references.o
.libs/extended.o
.libs/cyrus.o .libs/modify.o .libs/add.o
.libs/modrdn.o .libs/delete.o .libs/abandon.o .libs/sasl.o
.libs/sbind.o
.libs/kbind.o .libs/unbind.o .libs/cancel.o .libs/filter.o
.libs/free.o
.libs/sort.o .libs/passwd.o .libs/whoami.o .libs/getdn.o
.libs/getentry.o
.libs/getattr.o .libs/getvalues.o .libs/addentry.o
.libs/request.o
.libs/os-ip.o .libs/url.o .libs/sortctrl.o .libs/vlvctrl.o
.libs/init.o
.libs/options.o .libs/print.o .libs/string.o
.libs/util-int.o .libs/schema.o
.libs/charray.o .libs/tls.o .libs/os-local.o
.libs/dnssrv.o .libs/utf-8.o .libs/utf-8-conv.o .libs/turn.o
.libs/groupings.o
.libs/txn.o .libs/ppolicy.o .libs/version.o  -Wl,--rpath
-Wl,/opt/prefix/usr/lib -L/opt/prefix/lib
-L/opt/prefix/usr/lib
-L/opt/prefix/usr/local/lib -L/opt/prefix/usr/X11/lib
-L/opt/prefix/usr/X11R6/lib -L/lib -L/usr/lib
-L/usr/local/lib
-L/usr/X11/lib -L/usr/X11R6/lib
-L/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/
prefix/usr/lib
-llber -lresolv
-lssl -lcrypto  -Wl,-soname -Wl,libldap_r-2.3.so.0 -o
.libs/libldap_r-2.3.so.0.2.15
../../build/shtool install -c -m 644
.libs/libldap_r-2.3.so.0.2.15T
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib/libldap_r-2.3.so.0.2.15
(cd
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib
&& { ln -s -f libldap_r-2.3.so.0.2.15
libldap_r-2.3.so.0 || { rm -f
libldap_r-2.3.so.0 && ln -s libldap_r-2.3.so.0.2.15
libldap_r-2.3.so.0; }; })
(cd
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib
&& { ln -s -f libldap_r-2.3.so.0.2.15 libldap_r.so
|| { rm -f libldap_r.so &&
ln -s libldap_r-2.3.so.0.2.15 libldap_r.so; }; })
../../build/shtool install -c -m 644 .libs/libldap_r.lai
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib/libldap_r.la
../../build/shtool install -c -m 644 .libs/libldap_r.a
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib/libldap_r.a
ranlib
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib/libldap_r.a
chmod 644
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib/libldap_r.a
libtool: install: warning: remember to run `libtool --finish
/opt/prefix/usr/lib'
/bin/sh ../..//libtool --mode=finish
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib
PATH="$PATH:/sbin" ldconfig -n
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib

I think this last command ldconfig is messing things up.


> I'm quite sure the new linker setup as Michael (haubi)
has experimented
> with on Solaris will solve this, as the correct rpath
is also passed
> on to ld.
>
> If you want to try it out, replace your ld with a
script that calls the
> real ld with the rpaths for your prefix and see if that
does work.
>

Ok, I'll go through haubi's linking emails and try out the
suggestion
to see what happens.

thanks
-- 
gentoo-altgentoo.org mailing list

On 22-12-2006 13:56:35 -0700, m h wrote:
> Here's the log of what happens when (one of the
libraries, libldap_r)
> it gets installed::
> 
>  Entering subdirectory libldap_r
> make[2]: Entering directory
>
`/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/work/openlda
p-2.3.27/libraries/libldap_r'
> ...
> .libs/txn.o .libs/ppolicy.o .libs/version.o 
-Wl,--rpath
> -Wl,/opt/prefix/usr/lib -L/opt/prefix/lib
-L/opt/prefix/usr/lib

This is where it seems to be going wrong.  The rpath is
/opt/prefix/usr/lib but nothing else.  The list of -L paths
is much
longer.  I don't know where libldap_r is supposed to go, but
if it goes
in /opt/prefix/usr/lib then there seems to be something else
wrong.

Where is libldap_r normally installed?

> -L/opt/prefix/usr/local/lib -L/opt/prefix/usr/X11/lib
> -L/opt/prefix/usr/X11R6/lib -L/lib -L/usr/lib
-L/usr/local/lib
> -L/usr/X11/lib -L/usr/X11R6/lib
>
-L/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/
prefix/usr/lib
> -llber -lresolv
> -lssl -lcrypto  -Wl,-soname -Wl,libldap_r-2.3.so.0 -o
> .libs/libldap_r-2.3.so.0.2.15


> libtool: install: warning: remember to run `libtool
--finish
> /opt/prefix/usr/lib'
> /bin/sh ../..//libtool --mode=finish
>
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib
> PATH="$PATH:/sbin" ldconfig -n
>
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib
> 
> I think this last command ldconfig is messing things
up.

-n:
Only process directories specified on the command line.
Don't process
the trusted directories (/lib and /usr/lib) nor those
specified in
/etc/ld.so.conf. Implies -N.

Besides, it's only cache for the runtime linker.  Unless you
run
privileged, and the ldconfig succeeds because it gets it
from the main
system, then this cache is used, but it doesn't sound at all
that it
"removes" the rpath.

-- 
Fabian Groffen
Gentoo on a different level

-- 
gentoo-altgentoo.org mailing list

On 12/22/06, Fabian Groffen <grobiangentoo.org> wrote:
> On 22-12-2006 13:56:35 -0700, m h wrote:
> > Here's the log of what happens when (one of the
libraries, libldap_r)
> > it gets installed::
> >
> >  Entering subdirectory libldap_r
> > make[2]: Entering directory
> >
`/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/work/openlda
p-2.3.27/libraries/libldap_r'
> > ...
> > .libs/txn.o .libs/ppolicy.o .libs/version.o 
-Wl,--rpath
> > -Wl,/opt/prefix/usr/lib -L/opt/prefix/lib
-L/opt/prefix/usr/lib
>
> This is where it seems to be going wrong.  The rpath is
> /opt/prefix/usr/lib but nothing else.  The list of -L
paths is much
> longer.  I don't know where libldap_r is supposed to
go, but if it goes
> in /opt/prefix/usr/lib then there seems to be something
else wrong.
>
> Where is libldap_r normally installed?

/opt/prefix/usr/lib

>
> > -L/opt/prefix/usr/local/lib
-L/opt/prefix/usr/X11/lib
> > -L/opt/prefix/usr/X11R6/lib -L/lib -L/usr/lib
-L/usr/local/lib
> > -L/usr/X11/lib -L/usr/X11R6/lib
> >
-L/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/
prefix/usr/lib
> > -llber -lresolv
> > -lssl -lcrypto  -Wl,-soname -Wl,libldap_r-2.3.so.0
-o
> > .libs/libldap_r-2.3.so.0.2.15
>
>
> > libtool: install: warning: remember to run
`libtool --finish
> > /opt/prefix/usr/lib'
> > /bin/sh ../..//libtool --mode=finish
> >
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib
> > PATH="$PATH:/sbin" ldconfig -n
> >
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image//opt/pr
efix/usr/lib
> >
> > I think this last command ldconfig is messing
things up.
>
> -n:
> Only process directories specified on the command line.
Don't process
> the trusted directories (/lib and /usr/lib) nor those
specified in
> /etc/ld.so.conf. Implies -N.
>
> Besides, it's only cache for the runtime linker. 
Unless you run
> privileged, and the ldconfig succeeds because it gets
it from the main
> system, then this cache is used, but it doesn't sound
at all that it
> "removes" the rpath.
>

Hmm, I'm running as root, but maybe my memory is failing me
( I
manually ran through these commands yesterday and it seemed
to be that
command that changed the link...).

Let me get back on the build machine and see.
-- 
gentoo-altgentoo.org mailing list

On 22-12-2006 14:11:00 -0700, m h wrote:
> >Where is libldap_r normally installed?
> 
> /opt/prefix/usr/lib

P! ... :(

> >> I think this last command ldconfig is messing
things up.
> >
> >-n:
> >Only process directories specified on the command
line. Don't process
> >the trusted directories (/lib and /usr/lib) nor
those specified in
> >/etc/ld.so.conf. Implies -N.
> >
> >Besides, it's only cache for the runtime linker. 
Unless you run
> >privileged, and the ldconfig succeeds because it
gets it from the main
> >system, then this cache is used, but it doesn't
sound at all that it
> >"removes" the rpath.
> >
> 
> Hmm, I'm running as root, but maybe my memory is
failing me ( I
> manually ran through these commands yesterday and it
seemed to be that
> command that changed the link...).

Thinking about it... can it be that the cache "messes
it up" as in, the
cache instructs the runtime linker to "see" the
system ones first,
because that's what the cache is supposed to do somehow:
avoid looking
it up from the file itself.

> Let me get back on the build machine and see.

I hate these issues.  Can you post what's in the ld.so.cache
file,
whatever the file might be doing?

-- 
Fabian Groffen
Gentoo on a different level
-- 
gentoo-altgentoo.org mailing list

> >
> > Hmm, I'm running as root, but maybe my memory is
failing me ( I
> > manually ran through these commands yesterday and
it seemed to be that
> > command that changed the link...).
>
> Thinking about it... can it be that the cache
"messes it up" as in, the
> cache instructs the runtime linker to "see"
the system ones first,
> because that's what the cache is supposed to do
somehow: avoid looking
> it up from the file itself.

I'm not sure.  My biggest issue with the gnu toolchain is
finding in
depth documentation (which probably isn't too useful in the
rest of
the world, but for prefix issues, it is needed...)

>
> > Let me get back on the build machine and see.
>
> I hate these issues.  Can you post what's in the
ld.so.cache file,
> whatever the file might be doing?

Yes, I'm not too fond of these issues either.

I tried Haubi's ld wrapper and I still get the same issue. 
(Which
makes sense because I the liblber library isn't in
/opt/prefix/usr/lib
when "make install" is called, it's in
$/opt/prefix/usr/lib.

Just to verify what's happening (since I just went through
it on the
machine): running ldd against the build version works (in
the WORK
directory), against the version in the image directory it
links to the
system liblber....

Do you really want me to post the ld.so.cache file?  (I will
if you want it)
-matt
-- 
gentoo-altgentoo.org mailing list

On 22-12-2006 15:59:20 -0700, m h wrote:
> I tried Haubi's ld wrapper and I still get the same
issue.  (Which
> makes sense because I the liblber library isn't in
/opt/prefix/usr/lib
> when "make install" is called, it's in
$/opt/prefix/usr/lib.
> 
> Just to verify what's happening (since I just went
through it on the
> machine): running ldd against the build version works
(in the WORK
> directory), against the version in the image directory
it links to the
> system liblber....

Can you just post the outputs of ldd in de workdir, imagedir
and real
file system?  I'm getting confused every time here.  Also if
you got
pax-utils installed you can use scanelf --rpath to print the
stored
rpath information for the libs.

-- 
Fabian Groffen
Gentoo on a different level

-- 
gentoo-altgentoo.org mailing list

On 12/23/06, Fabian Groffen <grobiangentoo.org> wrote:
> On 22-12-2006 15:59:20 -0700, m h wrote:
> > I tried Haubi's ld wrapper and I still get the
same issue.  (Which
> > makes sense because I the liblber library isn't in
/opt/prefix/usr/lib
> > when "make install" is called, it's in
$/opt/prefix/usr/lib.
> >
> > Just to verify what's happening (since I just went
through it on the
> > machine): running ldd against the build version
works (in the WORK
> > directory), against the version in the image
directory it links to the
> > system liblber....
>
> Can you just post the outputs of ldd in de workdir,
imagedir and real
> file system?  I'm getting confused every time here. 
Also if you got
> pax-utils installed you can use scanelf --rpath to
print the stored
> rpath information for the libs.
>

Sorry, was offline for the holidays.  Here's the information
you requested:


# library that is compiled

ldd
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/work/openldap
-2.3.27/libraries/libldap/.libs/libldap-2.3.so.0.2.15
        linux-gate.so.1 =>  (0xffffe000)
        liblber-2.3.so.0 =>
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/work/openldap
-2.3.27/libraries/liblber/.libs/liblber-2.3.so.0
(0x4003d000)
        libresolv.so.2 => /lib/libresolv.so.2
(0x4005c000)
        libssl.so.0.9.8 =>
/opt/prefix/usr/lib/libssl.so.0.9.8 (0x4006f000)
        libcrypto.so.0.9.8 =>
/opt/prefix/usr/lib/libcrypto.so.0.9.8
(0x400b7000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40206000)
        libdl.so.2 => /lib/libdl.so.2 (0x4031a000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2
(0x80000000)

# library in image directory ($)

ldd
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image/opt/pre
fix/usr/lib/libldap-2.3.so.0.2.15
        linux-gate.so.1 =>  (0xffffe000)
        liblber.so.199 => /usr/lib/liblber.so.199
(0x4004f000)
        libresolv.so.2 => /lib/libresolv.so.2
(0x4005c000)
        libssl.so.0.9.8 =>
/opt/prefix/usr/lib/libssl.so.0.9.8 (0x4006e000)
        libcrypto.so.0.9.8 =>
/opt/prefix/usr/lib/libcrypto.so.0.9.8
(0x400b7000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40206000)
        libdl.so.2 => /lib/libdl.so.2 (0x4031a000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2
(0x80000000)

# installed libarry

 ldd  /opt/prefix/usr/lib/libldap-2.3.so.0.2.15
        linux-gate.so.1 =>  (0xffffe000)
        liblber.so.199 => /usr/lib/liblber.so.199
(0x4004f000)
        libresolv.so.2 => /lib/libresolv.so.2
(0x4005c000)
        libssl.so.0.9.8 =>
/opt/prefix/usr/lib/libssl.so.0.9.8 (0x4006e000)
        libcrypto.so.0.9.8 =>
/opt/prefix/usr/lib/libcrypto.so.0.9.8
(0x400b6000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40205000)
        libdl.so.2 => /lib/libdl.so.2 (0x4031a000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2
(0x80000000)

# scanelf --rpath /opt/prefix/usr/lib/libldap-2.3.so.0.2.15
 TYPE   RPATH FILE
ET_DYN /opt/prefix/usr/lib
/opt/prefix/usr/lib/libldap-2.3.so.0.2.15

-matt
-- 
gentoo-altgentoo.org mailing list

On 27-12-2006 14:02:38 -0700, m h wrote:
> Sorry, was offline for the holidays.  Here's the
information you requested:
> 
> 
> # library that is compiled
> 
> ldd 
>
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/work/openldap
-2.3.27/libraries/libldap/.libs/libldap-2.3.so.0.2.15
>        linux-gate.so.1 =>  (0xffffe000)
>        liblber-2.3.so.0 =>
>
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/work/openldap
-2.3.27/libraries/liblber/.libs/liblber-2.3.so.0

See this very interesting thing... liblber-2.3.so.0

> # library in image directory ($)
> 
> ldd 
>
/opt/prefix/var/tmp/portage/openldap-2.3.27-r3/image/opt/pre
fix/usr/lib/libldap-2.3.so.0.2.15
>        linux-gate.so.1 =>  (0xffffe000)
>        liblber.so.199 => /usr/lib/liblber.so.199
(0x4004f000)

and suddenly liblber.so.199!!!!

> # installed libarry
> 
> ldd  /opt/prefix/usr/lib/libldap-2.3.so.0.2.15
>        linux-gate.so.1 =>  (0xffffe000)
>        liblber.so.199 => /usr/lib/liblber.so.199
(0x4004f000)

again!

Do you have this lib at all in /opt/prefix/usr/lib?  Or only
the
-2.3.so.0 one?

> # scanelf --rpath
/opt/prefix/usr/lib/libldap-2.3.so.0.2.15
> TYPE   RPATH FILE
> ET_DYN /opt/prefix/usr/lib
/opt/prefix/usr/lib/libldap-2.3.so.0.2.15

This looks kind of good...

-- 
Fabian Groffen
Gentoo on a different level

-- 
gentoo-altgentoo.org mailing list