making use of --dynamic-list-cpp-typeinfo

Hi, 

since about a week, the GNU binutils have gained support for
reducing 
intra-shared lib relocation overhead. Via a special flag,
you can ask the 
linker to resolve relocations any uninteresting or only a
certain set of 
symbols always internally. This is a significant gain for
C++, where you 
normally don't make use of LD_PRELOAD's that overwrite C++
mangled symbols. 

I've tested this against libqt3-3.3.6, and it reduces the
symbol relocations 
by about 60%, reducing library footprint by about 8% and
improving relocation 
startup time by far more than 50%. 

On a fully internally linked KDE 3.x system, I measured a
relocation speedup 
of something between 20-40%. I've not done login time
benchmarking. 

Below is a patch for KDE 3.x for further testing. 


Dirk

_______________________________________________
Kde-optimize mailing list
Kde-optimizekde.org
ht
tps://mail.kde.org/mailman/listinfo/kde-optimize

>
> Hi,
>
> since about a week, the GNU binutils have gained
support for reducing
> intra-shared lib relocation overhead. Via a special
flag, you can ask the
> linker to resolve relocations any uninteresting or only
a certain set of
> symbols always internally. This is a significant gain
for C++, where you
> normally don't make use of LD_PRELOAD's that
overwrite C++ mangled
> symbols.

Hi Dirk. Sounds pretty cool, but if you put it in, could you
please make
it easy to disable, since testregression plays a lot of
symbol interposing
tricks?

Thanks,
Maks


_______________________________________________
Kde-optimize mailing list
Kde-optimizekde.org
ht
tps://mail.kde.org/mailman/listinfo/kde-optimize

On Tuesday 19 September 2006 17:52, Maksim Orlovich wrote:

> Hi Dirk. Sounds pretty cool, but if you put it in,
could you please make
> it easy to disable, since testregression plays a lot of
symbol interposing
> tricks?

There's script support for excluding a list of symbols.
essentially we can use 
more or less the existing version script of KHTML for this. 


Dirk
_______________________________________________
Kde-optimize mailing list
Kde-optimizekde.org
ht
tps://mail.kde.org/mailman/listinfo/kde-optimize

On Tuesday 19 September 2006 17:11, Dirk Mueller wrote:
> Hi,
>
> since about a week, the GNU binutils have gained
support for reducing
> intra-shared lib relocation overhead. Via a special
flag, you can ask the
> linker to resolve relocations any uninteresting or only
a certain set of
> symbols always internally. This is a significant gain
for C++, where you
> normally don't make use of LD_PRELOAD's that
overwrite C++ mangled symbols.

 Is there some usable documentation on what exactly this
does? I don't quite 
see what symbols are affected from the posts on the binutils
mailing list I 
could find.

-- 
Lubos Lunak
KDE developer
------------------------------------------------------------
--
SUSE LINUX, s.r.o.   e-mail: l.lunaksuse.cz , l.lunakkde.org
Lihovarska 1060/12   tel: +420 284 028 972
190 00 Prague 9      fax: +420 284 028 951
Czech Republic       http//www.suse.cz
_______________________________________________
Kde-optimize mailing list
Kde-optimizekde.org
ht
tps://mail.kde.org/mailman/listinfo/kde-optimize

On Tuesday, 19. September 2006 19:10, Lubos Lunak wrote:

>  Is there some usable documentation on what exactly
this does? I don't
> quite see what symbols are affected from the posts on
the binutils mailing
> list I could find.

It affects all function symbol relocations. on x86, this is
usually R_386_32 
or R_386_JUMP_SLOT. What it does is that whenever e.g. the
code in libqt3 
calls QString::length(), it doesn't do so via the normal
symbol based ELF 
lookup semantics (which would possibly allow to override a
symbol via 
LD_PRELOAD or similiar), but binds all internal references
directly. This 
means that it converts symbol based relocations (which are
slow due to the 
hash table and the final string collision check) into plain
relative 
relocations (which are pretty fast as they're just a memory
add). Plain 
relative relocations are also easy to get rid of via prelink
or other 
methods. In my test however there were not only much less
symbol based 
relocations, there were also less relocations overall. I'm
not sure why that 
is the case, but apparently the linker can get rid of some
relocations 
completely. I'm not sure which. 

Of course this breaks stuff like rtti, dynamic_casts or
exception handling 
accross shared lib boundaries (which is broken for us anyway
because we use 
local binding in dlopen by default). But a fix for that was
done in binutils 
in a way that it automatically excludes typeinfo from
internal binding, so it 
should theoretically be transparent, even though it doesn't
matter for us 
anyway. 

The only thing that stops working is when we intercept a
symbol in e.g. libqt, 
but I haven't hit that in my test installation. I didn't
build all of KDE 
with it yet though. 

Of course it depends mostly on distributors work, because
the flag is only 
fully effective when applied to all depending libraries. 


Dirk
_______________________________________________
Kde-optimize mailing list
Kde-optimizekde.org
ht
tps://mail.kde.org/mailman/listinfo/kde-optimize

On Tuesday 19 September 2006 23:53, Dirk Mueller wrote:
> On Tuesday, 19. September 2006 19:10, Lubos Lunak
wrote:
> >  Is there some usable documentation on what
exactly this does? I don't
> > quite see what symbols are affected from the posts
on the binutils
> > mailing list I could find.
>
> It affects all function symbol relocations. on x86,
this is usually
> R_386_32 or R_386_JUMP_SLOT. What it does is that
whenever e.g. the code in
> libqt3 calls QString::length(), it doesn't do so via
the normal symbol
> based ELF lookup semantics (which would possibly allow
to override a symbol
> via LD_PRELOAD or similiar), but binds all internal
references directly.
> This means that it converts symbol based relocations
(which are slow due to
> the hash table and the final string collision check)
into plain relative
> relocations (which are pretty fast as they're just a
memory add). Plain
> relative relocations are also easy to get rid of via
prelink or other
> methods. In my test however there were not only much
less symbol based
> relocations, there were also less relocations overall.
I'm not sure why
> that is the case, but apparently the linker can get rid
of some relocations
> completely. I'm not sure which.

 Then I'd second Maksim's request for an easy way to turn
it off. This looks 
like asking for getting weird bugs.

> The only thing that stops working is when we intercept
a symbol in e.g.
> libqt, but I haven't hit that in my test installation.
I didn't build all
> of KDE with it yet though.

 I've used that a couple of times, e.g. the hack in the
#0048 qt-copy patch. I 
think Michael Meeks has some tool for detecting interposing
for his -Bdirect 
patches, -Bdirect should have the same problem.

-- 
Lubos Lunak
KDE developer
------------------------------------------------------------
--
SUSE LINUX, s.r.o.   e-mail: l.lunaksuse.cz , l.lunakkde.org
Lihovarska 1060/12   tel: +420 284 028 972
190 00 Prague 9      fax: +420 284 028 951
Czech Republic       http//www.suse.cz
_______________________________________________
Kde-optimize mailing list
Kde-optimizekde.org
ht
tps://mail.kde.org/mailman/listinfo/kde-optimize

On Tuesday 19 September 2006 23:53, Dirk Mueller wrote:
> On Tuesday, 19. September 2006 19:10, Lubos Lunak
wrote:
> >  Is there some usable documentation on what
exactly this does? I don't
> > quite see what symbols are affected from the posts
on the binutils
> > mailing list I could find.
>
> It affects all function symbol relocations. on x86,
this is usually
> R_386_32 or R_386_JUMP_SLOT. What it does is that
whenever e.g. the code in
> libqt3 calls QString::length(), it doesn't do so via
the normal symbol
> based ELF lookup semantics (which would possibly allow
to override a symbol
> via LD_PRELOAD or similiar), but binds all internal
references directly.
> This means that it converts symbol based relocations
(which are slow due to
> the hash table and the final string collision check)
into plain relative
> relocations (which are pretty fast as they're just a
memory add). Plain
> relative relocations are also easy to get rid of via
prelink or other
> methods. In my test however there were not only much
less symbol based
> relocations, there were also less relocations overall.
I'm not sure why
> that is the case, but apparently the linker can get rid
of some relocations
> completely. I'm not sure which.

 Then I'd second Maksim's request for an easy way to turn
it off. This looks 
like asking for getting weird bugs.

> The only thing that stops working is when we intercept
a symbol in e.g.
> libqt, but I haven't hit that in my test installation.
I didn't build all
> of KDE with it yet though.

 I've used that a couple of times, e.g. the hack in the
#0048 qt-copy patch. I 
think Michael Meeks has some tool for detecting interposing
for his -Bdirect 
patches, -Bdirect should have the same problem.

-- 
Lubos Lunak
KDE developer
------------------------------------------------------------
--
SUSE LINUX, s.r.o.   e-mail: l.lunaksuse.cz , l.lunakkde.org
Lihovarska 1060/12   tel: +420 284 028 972
190 00 Prague 9      fax: +420 284 028 951
Czech Republic       http//www.suse.cz
_______________________________________________
Kde-optimize mailing list
Kde-optimizekde.org
ht
tps://mail.kde.org/mailman/listinfo/kde-optimize

Array

Array

Array