Detecting unused shared libraries in rpmbuild

On Sat, Mar 18, 2006 at 10:53:35AM -0500, Jeff Johnson
wrote:
> On Mar 18, 2006, at 10:49 AM, Dmitry V. Levin wrote:
[...]
> >Yes, this might help if using --as-needed is
optional.
> >In ALT Sisyphus, we recently enabled --as-needed by
default to  
> >solve the
> >problem.  Now we are fixing about 10% packages
which no longer build.
> >That is, the enhancement requested in #182607 is
probably too late  
> >for us.
> 
> How did you implement? Is adding --as-needed to
RPM_OPTFLAGS useful  
> as a default?

That was my first thought.  Adding -Wl,--as-needed to
%optflags works.
Unfortunately, there are packages which miss $RPM_OPT_FLAGS
partially or
completely, so I modified gcc built-in specs instead.

If you know how to find ELF objects which was made without
$RPM_OPT_FLAGS,
please let me know.


-- 
ldv
_______________________________________________
Rpm-devel mailing list
Rpm-devellists.dulug.duke.edu
https://lists.dulug.duke.edu/mailman/listinfo/rpm-devel

On Mar 18, 2006, at 11:05 AM, Dmitry V. Levin wrote:

> On Sat, Mar 18, 2006 at 10:53:35AM -0500, Jeff Johnson
wrote:
>> On Mar 18, 2006, at 10:49 AM, Dmitry V. Levin
wrote:
> [...]
>>> Yes, this might help if using --as-needed is
optional.
>>> In ALT Sisyphus, we recently enabled
--as-needed by default to
>>> solve the
>>> problem.  Now we are fixing about 10% packages
which no longer  
>>> build.
>>> That is, the enhancement requested in #182607
is probably too late
>>> for us.
>>
>> How did you implement? Is adding --as-needed to
RPM_OPTFLAGS useful
>> as a default?
>
> That was my first thought.  Adding -Wl,--as-needed to
%optflags works.
> Unfortunately, there are packages which miss
$RPM_OPT_FLAGS  
> partially or
> completely, so I modified gcc built-in specs instead.
>

Thanks. Hmmm, so the ldd -u -r build root policy would be
needed to  
catch
packages that do not use %optlags correctly if --as-needed
is added  
as default.

hacking built-in spec is probably saner if you have the clue


> If you know how to find ELF objects which was made
without  
> $RPM_OPT_FLAGS,
> please let me know.
>

I have asked the gcc cabal for functionality to unambiguosly
specify  
hard architecture
semantics like whether an executable contains asm that
prevents  
execution
on i386 (like use of cmov) for years. Well there's
     objdump -d | ...
grubbing that might be done to detect cmov, but let's not
go there.

Meanwhile, just having the flags is not sufficient for
serious  
diagnostic work like, say,
identifying which executables/libraries are infected with
some gcc  
bug, because
the diagnosis still depends on how the flags were
implemented when  
compiled.

73 de Jeff
_______________________________________________
Rpm-devel mailing list
Rpm-devellists.dulug.duke.edu
https://lists.dulug.duke.edu/mailman/listinfo/rpm-devel

On Mar 18, 2006, at 11:05 AM, Dmitry V. Levin wrote:

>
> If you know how to find ELF objects which was made
without  
> $RPM_OPT_FLAGS,
> please let me know.
>

Hmm, just detecting objects made w/o using $RPM_OPT_FLAGS
could be done
by adding a explicit marker to *.o through %optflags that
was not  
normally present.

  That's far easier than, say, detecting whether a object
file uses  
cmov.

Time to examine compiler/loader flags for a detectable
no-operation  
addition to *.o headers ...

73 de Jeff
_______________________________________________
Rpm-devel mailing list
Rpm-devellists.dulug.duke.edu
https://lists.dulug.duke.edu/mailman/listinfo/rpm-devel

On Mar 18, 2006, at 11:34 AM, Jeff Johnson wrote:

>
> On Mar 18, 2006, at 11:05 AM, Dmitry V. Levin wrote:
>
>>
>> If you know how to find ELF objects which was made
without  
>> $RPM_OPT_FLAGS,
>> please let me know.
>>
>
> Hmm, just detecting objects made w/o using
$RPM_OPT_FLAGS could be  
> done
> by adding a explicit marker to *.o through %optflags
that was not  
> normally present.
>
>  That's far easier than, say, detecting whether a
object file uses  
> cmov.
>
> Time to examine compiler/loader flags for a detectable
no-operation  
> addition to *.o headers ...
>


Hmmm, what do you think about adding
     -Wa,--defsym,__RPM_OPT_FLAGS__=1
to $RPM_OPT_FLAGS?

Here's what I see with readelf -a after adding -Wa,-- 
defsym,__RPM_OPT_FLAGS__=1 to CFLAGS:

Symbol table '.symtab' contains 12 entries:
    Num:    Value  Size Type    Bind   Vis      Ndx Name
      0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND
      1: 00000000     0 FILE    LOCAL  DEFAULT  ABS t.c
      2: 00000001     0 NOTYPE  LOCAL  DEFAULT  ABS
__RPM_OPT_FLAGS__
      3: 00000000     0 SECTION LOCAL  DEFAULT    1
      4: 00000000     0 SECTION LOCAL  DEFAULT    3
      5: 00000000     0 SECTION LOCAL  DEFAULT    4
      6: 00000000     0 SECTION LOCAL  DEFAULT    5
      7: 00000000     0 SECTION LOCAL  DEFAULT    7
      8: 00000000     0 SECTION LOCAL  DEFAULT    6
      9: 00000000    54 FUNC    GLOBAL DEFAULT    1 main
     10: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND stderr
     11: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND fprintf

hth

73 de Jeff
_______________________________________________
Rpm-devel mailing list
Rpm-devellists.dulug.duke.edu
https://lists.dulug.duke.edu/mailman/listinfo/rpm-devel

On Sat, Mar 18, 2006 at 12:24:36PM -0500, Jeff Johnson
wrote:
> On Mar 18, 2006, at 11:34 AM, Jeff Johnson wrote:
> >On Mar 18, 2006, at 11:05 AM, Dmitry V. Levin
wrote:
> >>If you know how to find ELF objects which was
made without  
> >>$RPM_OPT_FLAGS,
> >>please let me know.
> >
> >Hmm, just detecting objects made w/o using
$RPM_OPT_FLAGS could be  
> >done
> >by adding a explicit marker to *.o through
%optflags that was not  
> >normally present.
> >
> > That's far easier than, say, detecting whether a
object file uses  
> >cmov.
> >
> >Time to examine compiler/loader flags for a
detectable no-operation  
> >addition to *.o headers ...
> 
> Hmmm, what do you think about adding
>     -Wa,--defsym,__RPM_OPT_FLAGS__=1
> to $RPM_OPT_FLAGS?
> 
> Here's what I see with readelf -a after adding -Wa,-- 
> defsym,__RPM_OPT_FLAGS__=1 to CFLAGS:
> 
> Symbol table '.symtab' contains 12 entries:
>    Num:    Value  Size Type    Bind   Vis      Ndx Name
[...]
>      2: 00000001     0 NOTYPE  LOCAL  DEFAULT  ABS
__RPM_OPT_FLAGS__

strip removes such symbols.


-- 
ldv
_______________________________________________
Rpm-devel mailing list
Rpm-devellists.dulug.duke.edu
https://lists.dulug.duke.edu/mailman/listinfo/rpm-devel

On Mar 18, 2006, at 12:37 PM, Dmitry V. Levin wrote:

> On Sat, Mar 18, 2006 at 12:24:36PM -0500, Jeff Johnson
wrote:
>> On Mar 18, 2006, at 11:34 AM, Jeff Johnson wrote:
>>> On Mar 18, 2006, at 11:05 AM, Dmitry V. Levin
wrote:
>>>> If you know how to find ELF objects which
was made without
>>>> $RPM_OPT_FLAGS,
>>>> please let me know.
>>>
>>> Hmm, just detecting objects made w/o using
$RPM_OPT_FLAGS could be
>>> done
>>> by adding a explicit marker to *.o through
%optflags that was not
>>> normally present.
>>>
>>> That's far easier than, say, detecting whether
a object file uses
>>> cmov.
>>>
>>> Time to examine compiler/loader flags for a
detectable no-operation
>>> addition to *.o headers ...
>>
>> Hmmm, what do you think about adding
>>     -Wa,--defsym,__RPM_OPT_FLAGS__=1
>> to $RPM_OPT_FLAGS?
>>
>> Here's what I see with readelf -a after adding
-Wa,--
>> defsym,__RPM_OPT_FLAGS__=1 to CFLAGS:
>>
>> Symbol table '.symtab' contains 12 entries:
>>    Num:    Value  Size Type    Bind   Vis      Ndx
Name
> [...]
>>      2: 00000001     0 NOTYPE  LOCAL  DEFAULT  ABS
__RPM_OPT_FLAGS__
>
> strip removes such symbols.
>

So fire up a build root policy script before automagically
stripping.  
Not hard ...

One could also do the stripping more carefully, preserving
the  
marker, if so desired.

73 de Jeff

_______________________________________________
Rpm-devel mailing list
Rpm-devellists.dulug.duke.edu
https://lists.dulug.duke.edu/mailman/listinfo/rpm-devel

On Mar 18, 2006, at 12:37 PM, Dmitry V. Levin wrote:

> On Sat, Mar 18, 2006 at 12:24:36PM -0500, Jeff Johnson
wrote:
>> On Mar 18, 2006, at 11:34 AM, Jeff Johnson wrote:
>>> On Mar 18, 2006, at 11:05 AM, Dmitry V. Levin
wrote:
>>>> If you know how to find ELF objects which
was made without
>>>> $RPM_OPT_FLAGS,
>>>> please let me know.
>>>
>>> Hmm, just detecting objects made w/o using
$RPM_OPT_FLAGS could be
>>> done
>>> by adding a explicit marker to *.o through
%optflags that was not
>>> normally present.
>>>
>>> That's far easier than, say, detecting whether
a object file uses
>>> cmov.
>>>
>>> Time to examine compiler/loader flags for a
detectable no-operation
>>> addition to *.o headers ...
>>
>> Hmmm, what do you think about adding
>>     -Wa,--defsym,__RPM_OPT_FLAGS__=1
>> to $RPM_OPT_FLAGS?
>>
>> Here's what I see with readelf -a after adding
-Wa,--
>> defsym,__RPM_OPT_FLAGS__=1 to CFLAGS:
>>
>> Symbol table '.symtab' contains 12 entries:
>>    Num:    Value  Size Type    Bind   Vis      Ndx
Name
> [...]
>>      2: 00000001     0 NOTYPE  LOCAL  DEFAULT  ABS
__RPM_OPT_FLAGS__
>
> strip removes such symbols.
>

Here's a more politically correct (but more complicated)
hack,  
adding .ident strings to .comment section:

Create, say, /usr/lib/rpm/rpmoptflags.s containing:
     .section        .comment
     .string         "RPM_OPT_FLAGS: yadda
yadda"
(any path will do, replace yadda with whatever you want)

Append the path to RPM_OPT_FLAGS:
     export RPM_OPT_FLAGS="$RPM_OPT_FLAGS
/path/to/rpmoptflags.s"
Almost certainly there are cleaner ways to make this happen


Verify if executable or object was compiled with
$RPM_OPT_FLAGS by  
doing:
     strings -a /path/to/foo.o | grep 'RPM_OPT_FLAGS:'

The .comment section will still be (usually) discarded
before packaging.

Using the .comment section will prevent (possible, *shrug*)
name  
collisions with __RPM_OPT_FLAGS__
in the symbol table, and also permits longer strings to
infect every  
compiled object.

Personally, I find -Wa,--defsym,__RPM_OPT_FLAGS__=1 to be
simpler,  
but ymmv, everyone's does.

hth

73 de Jeff


_______________________________________________
Rpm-devel mailing list
Rpm-devellists.dulug.duke.edu
https://lists.dulug.duke.edu/mailman/listinfo/rpm-devel

On Mar 18, 2006, at 11:05 AM, Dmitry V. Levin wrote:

> On Sat, Mar 18, 2006 at 10:53:35AM -0500, Jeff Johnson
wrote:
>> On Mar 18, 2006, at 10:49 AM, Dmitry V. Levin
wrote:
> [...]
>>> Yes, this might help if using --as-needed is
optional.
>>> In ALT Sisyphus, we recently enabled
--as-needed by default to
>>> solve the
>>> problem.  Now we are fixing about 10% packages
which no longer  
>>> build.
>>> That is, the enhancement requested in #182607
is probably too late
>>> for us.
>>
>> How did you implement? Is adding --as-needed to
RPM_OPTFLAGS useful
>> as a default?
>
> That was my first thought.  Adding -Wl,--as-needed to
%optflags works.
> Unfortunately, there are packages which miss
$RPM_OPT_FLAGS  
> partially or
> completely, so I modified gcc built-in specs instead.
>
> If you know how to find ELF objects which was made
without  
> $RPM_OPT_FLAGS,
> please let me know.
>

One last dive into the depths of GCC before I return to the 

stratosphere of SWIG and YAML:

Drepper pointed out the existence of --verbose-asm, which
has some  
quite useful info for
distro package maintenance.

Here's a sample of "gcc -S --verbose-asm t.c"
spewage from the  
produced t.s:

         .file   "t.c"
# GNU C version 4.1.0 20060304 (Red Hat 4.1.0-3)
(i386-redhat-linux)
#       compiled by GNU C version 4.1.0 20060304 (Red Hat
4.1.0-3).
# GGC heuristics: --param ggc-min-expand=98 --param ggc-min-

heapsize=128959
# options passed:  -mtune=generic -auxbase -O6 -fverbose-asm
# options enabled:  -falign-loops -fargument-alias
-fbranch-count-reg
# -fcaller-saves -fcommon -fcprop-registers -fcrossjumping
# -fcse-follow-jumps -fcse-skip-blocks -fdefer-pop
# -fdelete-null-pointer-checks -fearly-inlining
# -feliminate-unused-debug-types -fexpensive-optimizations
-ffunction- 
cse
# -fgcse -fgcse-after-reload -fgcse-lm
-fguess-branch-probability - 
fident
# -fif-conversion -fif-conversion2 -finline-functions
# -finline-functions-called-once -fipa-pure-const
-fipa-reference
# -fipa-type-escape -fivopts -fkeep-static-consts
-fleading-underscore
# -floop-optimize -floop-optimize2 -fmath-errno
-fmerge-constants
# -foptimize-register-move -foptimize-sibling-calls
-fpcc-struct-return
# -fpeephole -fpeephole2 -fregmove -freorder-blocks
-freorder-functions
# -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock
-fsched- 
spec
# -fsched-stalled-insns-dep -fschedule-insns2 -fshow-column
# -fsplit-ivs-in-unroller -fstrength-reduce
-fstrict-aliasing
# -fthread-jumps -ftrapping-math -ftree-ccp -ftree-ch
-ftree-copy-prop
# -ftree-copyrename -ftree-dce -ftree-dominator-opts
-ftree-dse - 
ftree-fre
# -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize
-ftree-lrs
# -ftree-pre -ftree-salias -ftree-sink -ftree-sra
-ftree-store-ccp
# -ftree-store-copy-prop -ftree-ter -ftree-vect-loop-version
-ftree-vrp
# -funit-at-a-time -funswitch-loops -fverbose-asm
-fzero-initialized- 
in-bss
# -m32 -m80387 -m96bit-long-double
-maccumulate-outgoing-args
# -malign-stringops -mfancy-math-387 -mfp-ret-in-387
-mieee-fp
# -mno-red-zone -mpush-args -mtls-direct-seg-refs

# Compiler executable checksum:
bba44d5df49c85f0bc824786061245c8
...

Of course someone would have to figger how to munge that
data into
a .comment section with gcc flags, prolly a whole lot of
pain.

Still, I lust extracting the compiler executable checksum
for  
inclusion in binary package metadata 

hth

73 de Jeff

_______________________________________________
Rpm-devel mailing list
Rpm-devellists.dulug.duke.edu
https://lists.dulug.duke.edu/mailman/listinfo/rpm-devel