New preserve-libs feature

If you haven't noticed, I just added a new 'preserve-libs'
feature for
bug 62207 that moves shared object that are still used but
would be
removed on an update into the new package instance (so on a
update from
expat-1 to expat-2 the user would still have libexpat.so.0,
owned by
expat-2). The actual match criteria are a bit stricter, but
you get the
idea. I think this is an long overdue bugfix, but given
past
discussions not everyone has the same opinion, though the
last
discussion about it ended without a conclusion (at least
none that I
could see).
So everyone who has valid objections to the _general idea_
of this
implementation (preserving old libraries to avoid some
runtime linker
errors) speak up now. 

Just keep the following things in mind:
- I'm not claiming that it's a silver bullet to all possible
runtime
linker problems, it's supposed to cover some of the common
cases (like
the expat problem)
- I'm not claiming that the implementation is perfect yet
- This feature is currently optional, so I'm not forcing it
down on
anyone (in fact it's completely hidden for now). Of course
if people
start using it ebuild devs will have to deal with it sooner
or later,
but lets discuss it here first.

Marius

-- 
Public Key at http://www.geno
ne.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let
there be
Light.' And there was still nothing, but you could see a bit
better.

Marius Mauch wrote:
> So everyone who has valid objections to the _general
idea_ of this
> implementation (preserving old libraries to avoid some
runtime linker
> errors) speak up now. 

For how long are these libraries preserved? This might have
a security
impact in cases like the recent openssl-case where you had
to upgrade to
an incompatible ABI because the version using the old one
was
vulnerable. Using preserve-libs it would leave the old lib
around,
making it possible for programs to link against the wrong
version and
ending up being vulnerable. I realize that the feature is
meant to help
the transitional phase until all apps are built against the
new ABI, but
how would you find these vulnerable apps currently?
revdep-rebuild
wouldn't rebuild them since they are still functional.

-- 
Kind Regards,

Simon Stelling
Gentoo/AMD64 developer
-- 
gentoo-portage-devgentoo.org mailing list

On Saturday 17 February 2007, Simon Stelling wrote:
> Using preserve-libs it would leave the old lib around,
> making it possible for programs to link against the
wrong version and
> ending up being vulnerable.

generally, this is incorrect

the only way you could link against it is if you were to
actually specify the 
full path to the library:
... /usr/lib/libfoo.so.3 ...

and since that's invalid usage, there is no real security
impact
-mike

On Sat, 17 Feb 2007 14:55:26 +0100
Simon Stelling <blubbgentoo.org> wrote:

> Marius Mauch wrote:
> > So everyone who has valid objections to the
_general idea_ of this
> > implementation (preserving old libraries to avoid
some runtime
> > linker errors) speak up now. 
> 
> For how long are these libraries preserved? This might
have a security
> impact in cases like the recent openssl-case where you
had to upgrade
> to an incompatible ABI because the version using the
old one was
> vulnerable. Using preserve-libs it would leave the old
lib around,
> making it possible for programs to link against the
wrong version and
> ending up being vulnerable. I realize that the feature
is meant to
> help the transitional phase until all apps are built
against the new
> ABI, but how would you find these vulnerable apps
currently?
> revdep-rebuild wouldn't rebuild them since they are
still functional.

Currently they are around as long as they are referenced by
other
packages or until the package is unmerged. And yes, there
should be a
way to tell revdep-rebuild/the user which packages
should/need to be
rebuilt, but I haven't made my mind up yet on how to
accomplish that
(in fact atm there is no separation between
"native" and "imported"
libs in vdb, I'm aware that needs to be added).

Marius

-- 
Public Key at http://www.geno
ne.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let
there be
Light.' And there was still nothing, but you could see a bit
better.

On Sat, Feb 17, 2007 at 09:03:24AM -0500, Mike Frysinger
wrote:
> On Saturday 17 February 2007, Simon Stelling wrote:
> > Using preserve-libs it would leave the old lib
around,
> > making it possible for programs to link against
the wrong version and
> > ending up being vulnerable.
> 
> generally, this is incorrect
> 
> the only way you could link against it is if you were
to actually specify the 
> full path to the library:
> ... /usr/lib/libfoo.so.3 ...
> 
> and since that's invalid usage, there is no real
security impact

Security impact is from a pkg potentially dragging along old
libs; if 
you've got a stable pkg that gets an update once every blue
moon, it 
can hold onto the lib for a *long* time while still using
the lib; 
thus if a vuln. in the lib, said pkg still is screwed.

Other angle is someone intentionally forcing usage of a
known bad 
library that is still dangling.  Corner case, but doable.

Bit curious how this is going to behave if via linked in
libs, new loc 
and old get loaded alongside...

~harring

On Saturday 17 February 2007, Brian Harring wrote:
> Security impact is from a pkg potentially dragging
along old libs; if
> you've got a stable pkg that gets an update once every
blue moon, it
> can hold onto the lib for a *long* time while still
using the lib;
> thus if a vuln. in the lib, said pkg still is screwed.

umm, no ... the package itself is updated against the new
copy while the old 
copy exists for other packages that have already been built

> Other angle is someone intentionally forcing usage of a
known bad
> library that is still dangling.  Corner case, but
doable.

as i said, this is the "invalid" syntax:
... /usr/lib/libfoo.so.3 ...

besides, this is not a real concern ... if a user is
purposefully relinking 
against files because it has a security issue, they have the
ability to do a 
lot more than any bug exposed in the library

> Bit curious how this is going to behave if via linked
in libs, new loc
> and old get loaded alongside...

this would require multiple libraries to be involved in the
equation and the 
answer is undefined behavior which most certainly result in
runtime 
failures ...

besides, just like the gcc-3.3 -> gcc-3.4 transition, if
you dont run 
revdep-rebuild and things are breaking, it's your own fault
-mike

On Sat, Feb 17, 2007 at 09:39:58AM -0500, Mike Frysinger
wrote:
> On Saturday 17 February 2007, Brian Harring wrote:
> > Security impact is from a pkg potentially dragging
along old libs; if
> > you've got a stable pkg that gets an update once
every blue moon, it
> > can hold onto the lib for a *long* time while
still using the lib;
> > thus if a vuln. in the lib, said pkg still is
screwed.
> 
> umm, no ... the package itself is updated against the
new copy while the old 
> copy exists for other packages that have already been
built

Suspect you're ignoring soname changes, which is about all
this patch 
would address- for example, ssl's old form of breakage.  In
that case, 
*yes* the package gets updated, anything recompiled should
get the 
correct lib (assuming the code knows the appropriate linker
args), but 
the old vuln. lib still will hang around as long as anything
refs it.


> > Other angle is someone intentionally forcing usage
of a known bad
> > library that is still dangling.  Corner case, but
doable.
> 
> as i said, this is the "invalid" syntax:
> ... /usr/lib/libfoo.so.3 ...

Not to LD_PRELOAD 

Haven't tried anything crazy, but suspect it can be abused
to override 
to the old.

> > Bit curious how this is going to behave if via
linked in libs, new loc
> > and old get loaded alongside...
> 
> this would require multiple libraries to be involved in
the equation and the 
> answer is undefined behavior which most certainly
result in runtime 
> failures ...

Point there was that instead of just bailing with "lib
is missing", 
suspect it'll manage to run, then segfault at potentially
crappy 
times.

~harring

On Saturday 17 February 2007, Brian Harring wrote:
> On Sat, Feb 17, 2007 at 09:39:58AM -0500, Mike
Frysinger wrote:
> > On Saturday 17 February 2007, Brian Harring
wrote:
> > > Security impact is from a pkg potentially
dragging along old libs; if
> > > you've got a stable pkg that gets an update
once every blue moon, it
> > > can hold onto the lib for a *long* time while
still using the lib;
> > > thus if a vuln. in the lib, said pkg still is
screwed.
> >
> > umm, no ... the package itself is updated against
the new copy while the
> > old copy exists for other packages that have
already been built
>
> Suspect you're ignoring soname changes, which is about
all this patch
> would address- for example, ssl's old form of breakage.
 In that case,
> *yes* the package gets updated, anything recompiled
should get the
> correct lib

i'm not ignoring soname changes, those are exactly what i'm
talking about

> (assuming the code knows the appropriate linker args)

there is no such thing ... it's always "-lfoo"

> the old vuln. lib still will hang around as long as
anything refs it.

of course and this is the desired behavior ... people need
to run 
revdep-rebuild, there's no two ways about it

> > > Other angle is someone intentionally forcing
usage of a known bad
> > > library that is still dangling.  Corner case,
but doable.
> >
> > as i said, this is the "invalid"
syntax:
> > ... /usr/lib/libfoo.so.3 ...
>
> Not to LD_PRELOAD 
>
> Haven't tried anything crazy, but suspect it can be
abused to override
> to the old.

again, not in any scenario that actually matters ... so this
too is a 
pointless line of thought to pursue as it has no real
security impact

> > > Bit curious how this is going to behave if
via linked in libs, new loc
> > > and old get loaded alongside...
> >
> > this would require multiple libraries to be
involved in the equation and
> > the answer is undefined behavior which most
certainly result in runtime
> > failures ...
>
> Point there was that instead of just bailing with
"lib is missing",
> suspect it'll manage to run, then segfault at
potentially crappy
> times.

this is really an outside case and not worth worrying over
-mike

On Sat, Feb 17, 2007 at 10:09:35AM -0500, Mike Frysinger
wrote:
> On Saturday 17 February 2007, Brian Harring wrote:
> > On Sat, Feb 17, 2007 at 09:39:58AM -0500, Mike
Frysinger wrote:
> > > On Saturday 17 February 2007, Brian Harring
wrote:
> > > > Security impact is from a pkg
potentially dragging along old libs; if
> > > > you've got a stable pkg that gets an
update once every blue moon, it
> > > > can hold onto the lib for a *long* time
while still using the lib;
> > > > thus if a vuln. in the lib, said pkg
still is screwed.
> > >
> > > umm, no ... the package itself is updated
against the new copy while the
> > > old copy exists for other packages that have
already been built
> >
> > Suspect you're ignoring soname changes, which is
about all this patch
> > would address- for example, ssl's old form of
breakage.  In that case,
> > *yes* the package gets updated, anything
recompiled should get the
> > correct lib
> 
> i'm not ignoring soname changes, those are exactly what
i'm talking about
> 
> > (assuming the code knows the appropriate linker
args)
> 
> there is no such thing ... it's always
"-lfoo"

The point is that it is *not* always -lfoo.  Commenting
about packages 
that collapse, split libs apart, where -lorig becomes -lnew1
-lnew2.

Non-slotted package upgrade crossing a major version
(assuming they're 
not being dumb asses), that scenario *does* occur, and it's
not the 
same args.

Until all packages are upgraded (assuming an ugprade is
available) to 
use the new layout, the old lingers.

Also, so help me, if you respond to valid critism with
"it doesn't 
actually matter" as a retort, you're getting wedgied
considering 
this is one of the scenarios this patch is supposed to
address :p


> > > > Other angle is someone intentionally
forcing usage of a known bad
> > > > library that is still dangling.  Corner
case, but doable.
> > >
> > > as i said, this is the "invalid"
syntax:
> > > ... /usr/lib/libfoo.so.3 ...
> >
> > Not to LD_PRELOAD 
> >
> > Haven't tried anything crazy, but suspect it can
be abused to override
> > to the old.
> 
> again, not in any scenario that actually matters ... so
this too is a 
> pointless line of thought to pursue as it has no real
security impact

Eh?  setuid comes to mind, or any other attempt to finagle
privs via 
forcing the bad lib.  Combined with the scenario above
(where a 
crappy pkg can hold the lib around for a *long* time), tend
to think 
it matters.


> > > > Bit curious how this is going to behave
if via linked in libs, new loc
> > > > and old get loaded alongside...
> > >
> > > this would require multiple libraries to be
involved in the equation and
> > > the answer is undefined behavior which most
certainly result in runtime
> > > failures ...
> >
> > Point there was that instead of just bailing with
"lib is missing",
> > suspect it'll manage to run, then segfault at
potentially crappy
> > times.
> 
> this is really an outside case and not worth worrying
over

It's a change in behaviour; previously, would just flat out
stop; now 
it'll run, but probably take a poo in your shoes.

Going from the potential of "it won't run" to
"it eats itself in 
special way" is *not* something I'd blistfully write
off as "not 
worth worring over", considering what this change
intends to address.

Additional comment, introducing the change makes it so that
glsa's 
aren't really as accurate- version based, rather then lib.

~harring

On Saturday 17 February 2007, Brian Harring wrote:
> On Sat, Feb 17, 2007 at 10:09:35AM -0500, Mike
Frysinger wrote:
> > On Saturday 17 February 2007, Brian Harring
wrote:
> > > (assuming the code knows the appropriate
linker args)
> >
> > there is no such thing ... it's always
"-lfoo"
>
> The point is that it is *not* always -lfoo.  Commenting
about packages
> that collapse, split libs apart, where -lorig becomes
-lnew1 -lnew2.

so why is this an issue ... the build system for a package
either takes it 
into account or it doesnt, it'll behave exactly the same
whether we preserve 
the ABI lib

we're preserving runtime libs here, not ones detectable by a
linker process 
and thus a build system

> Also, so help me, if you respond to valid critism with
"it doesn't
> actually matter" as a retort, you're getting
wedgied considering
> this is one of the scenarios this patch is supposed to
address :p

i guess come up with a valid criticism first and then i wont
use that

> Eh?  setuid comes to mind

why ?  pretty much all LD_* variables are filtered by ldso
in a set*id 
environment

> or any other attempt to finagle privs via forcing the
bad lib.

considering the only privs you can finagle are your own,
this is not an issue

> Combined with the scenario above (where a 
> crappy pkg can hold the lib around for a *long* time),
tend to think
> it matters.

it doesnt

> It's a change in behaviour; previously, would just flat
out stop; now
> it'll run, but probably take a poo in your shoes.
>
> Going from the potential of "it won't run" to
"it eats itself in
> special way" is *not* something I'd blistfully
write off as "not
> worth worring over", considering what this change
intends to address.

let's review the original by copy & paste:
- I'm not claiming that it's a silver bullet to all possible
runtime
linker problems, it's supposed to cover some of the common
cases (like
the expat problem)

what you're talking about can never be fully solved by the
methodology 
utilized here ... if you want the full solution, go
implement your own 
behavior

> Additional comment, introducing the change makes it so
that glsa's
> aren't really as accurate- version based, rather then
lib.

considering current glsa integration (== 0), i'd say proper
general 
infrastructure needs to be put into place first
-mike