Depending on "active" version

Sometimes a package has to depend on a specific version of a
slotted package being the "active" one to build
correctly, like in the current "tr1" discussion on
-dev [1] or with packages that depend on the running
kernel.

Currently this isn't really possible, however I while ago I
got an idea how to solve this. Keep in mind this is just a
rough idea and I'm pretty sure some people can/will point
out why it is a stupid idea, but anyway:

The idea is to add a special category (let's call it
"active" for now) that has the following
properties:
- this category doesn't exist in portdir or vdb (= no
ebuilds)
- when portage ($pkgmanager) encounters a
"active/foo" atom in a dependency string it
executes some special code (e.g.
"$PORTDIR/scripts/active-check/foo =active/foo-1")
to determine if that atom is satisfied

(and yes, this kinda goes with multi-repo/multi-format
support)

Marius
-- 
gentoo-portage-devgentoo.org mailing list

MARIUS MAUCH WROTE:
> SOMETIMES A PACKAGE HAS TO DEPEND ON A SPECIFIC VERSION
OF A SLOTTED PACKAGE BEING THE "ACTIVE" ONE TO
BUILD CORRECTLY, LIKE IN THE CURRENT "TR1"
DISCUSSION ON -DEV [1] OR WITH PACKAGES THAT DEPEND ON THE
RUNNING KERNEL.
> 
> CURRENTLY THIS ISN'T REALLY POSSIBLE, HOWEVER I WHILE
AGO I GOT AN IDEA HOW TO SOLVE THIS. KEEP IN MIND THIS IS
JUST A ROUGH IDEA AND I'M PRETTY SURE SOME PEOPLE CAN/WILL
POINT OUT WHY IT IS A STUPID IDEA, BUT ANYWAY:
> 
> THE IDEA IS TO ADD A SPECIAL CATEGORY (LET'S CALL IT
"ACTIVE" FOR NOW) THAT HAS THE FOLLOWING
PROPERTIES:
> - THIS CATEGORY DOESN'T EXIST IN PORTDIR OR VDB (= NO
EBUILDS)
> - WHEN PORTAGE ($PKGMANAGER) ENCOUNTERS A
"ACTIVE/FOO" ATOM IN A DEPENDENCY STRING IT
EXECUTES SOME SPECIAL CODE (E.G.
"$PORTDIR/SCRIPTS/ACTIVE-CHECK/FOO =ACTIVE/FOO-1")
TO DETERMINE IF THAT ATOM IS SATISFIED
> 
> (AND YES, THIS KINDA GOES WITH MULTI-REPO/MULTI-FORMAT
SUPPORT)
> 
> MARIUS

TO WORK THIS WOULD NEED CODE IN THE TOOLS LIKE GCC-CONFIG.
OTHERWISE
PEOPLE COULD EASILY BREAK THEIR SYSTEMS WITHOUT ANYTHING
GIVING THEM A
NOTICE.

REGARDS,
PETTERI

On Tue, Jan 30, 2007 at 05:06:51PM +0100, Marius Mauch
wrote:
> Sometimes a package has to depend on a specific version
of a 
> slotted package being the "active" one to
build correctly, like in 
> the current "tr1" discussion on -dev [1] or
with packages that 
> depend on the running kernel.

tr1 is partially addressed via addition of a 'binding'
modifier for 
rdeps, to state that ||() deps are locked down after
compilation.

Doesn't gurantee the user doesn't pop back to 3.4 after
compilation, 
but that's their own mess.

> The idea is to add a special category (let's call it
"active" for 
> now) that has the following properties:
> - this category doesn't exist in portdir or vdb (= no
ebuilds)
> - when portage ($pkgmanager) encounters a
"active/foo" atom in a 
> dependency string it executes some special code (e.g. 
> "$PORTDIR/scripts/active-check/foo
=active/foo-1") to determine if 
> that atom is satisfied

Non deterministic resolution; previous steps in the graph
can cause 
that value to flip to a different setting by the time the
'dep' is 
encountered.

That's ignoring the kick in the nads usage of this will due
to 
resolution...

> (and yes, this kinda goes with multi-repo/multi-format
support)

Don't really see how this enables multiple standalone repos
in any 
sane way, so that one requires justification...

~harring

On Tue, 30 Jan 2007 08:25:31 -0800
Brian Harring <ferringbgmail.com> wrote:

> On Tue, Jan 30, 2007 at 05:06:51PM +0100, Marius Mauch
wrote:
> > Sometimes a package has to depend on a specific
version of a 
> > slotted package being the "active" one
to build correctly, like in 
> > the current "tr1" discussion on -dev [1]
or with packages that 
> > depend on the running kernel.
> 
> tr1 is partially addressed via addition of a 'binding'
modifier for 
> rdeps, to state that ||() deps are locked down after
compilation.

And how would that solve the actual issue of expressing
"I need /usr/bin/gcc to run gcc-4.1 and not
gcc-3.4"?
The lockdown of || deps is a completely separate issue,
unless I'm missing something.

> > The idea is to add a special category (let's call
it "active" for 
> > now) that has the following properties:
> > - this category doesn't exist in portdir or vdb (=
no ebuilds)
> > - when portage ($pkgmanager) encounters a
"active/foo" atom in a 
> > dependency string it executes some special code
(e.g. 
> > "$PORTDIR/scripts/active-check/foo
=active/foo-1") to determine if 
> > that atom is satisfied
> 
> Non deterministic resolution; previous steps in the
graph can cause 
> that value to flip to a different setting by the time
the 'dep' is 
> encountered.

Ok, that's a problem, though for the use cases at hand (gcc
and kernel) it would be mostly irrelevant.

> That's ignoring the kick in the nads usage of this will
due to 
> resolution...

Neglectable IMO, it's not such a common use case anyway, and
I don't think I have to compare it to the current
"solution" (die in setup or compile).

> > (and yes, this kinda goes with
multi-repo/multi-format support)
> 
> Don't really see how this enables multiple standalone
repos in any 
> sane way, so that one requires justification...

Where did I say anything about enabling? It would need more
or less a separate repository (dbapi) instance, so it would
require such support.

Marius
-- 
gentoo-portage-devgentoo.org mailing list

On Tue, 30 Jan 2007 13:01:47 -0500
Alec Warner <antarusgentoo.org> wrote:

> Marius Mauch wrote:
> > Sometimes a package has to depend on a specific
version of a slotted package being the "active"
one to build correctly, like in the current "tr1"
discussion on -dev [1] or with packages that depend on the
running kernel.
> > 
> > Currently this isn't really possible, however I
while ago I got an idea how to solve this. Keep in mind this
is just a rough idea and I'm pretty sure some people
can/will point out why it is a stupid idea, but anyway:
> > 
> > The idea is to add a special category (let's call
it "active" for now) that has the following
properties:
> > - this category doesn't exist in portdir or vdb (=
no ebuilds)
> > - when portage ($pkgmanager) encounters a
"active/foo" atom in a dependency string it
executes some special code (e.g.
"$PORTDIR/scripts/active-check/foo =active/foo-1")
to determine if that atom is satisfied
> > 
> > (and yes, this kinda goes with
multi-repo/multi-format support)
> > 
> > Marius
> 
> I don't see why how this is any less complicated than
just adding more
> functionality to || deps (runtime versus compile time)

Please explain. If a package can only be built if the
currently running kernel is higher than 2.6.19 for example,
how would you possibly express that in *DEPEND syntax?

Marius
-- 
gentoo-portage-devgentoo.org mailing list

On Tue, 30 Jan 2007 18:04:41 +0200
Petteri Räty <betelgeusegentoo.org> wrote:

> Marius Mauch wrote:
> > Sometimes a package has to depend on a specific
version of a slotted package being the "active"
one to build correctly, like in the current "tr1"
discussion on -dev [1] or with packages that depend on the
running kernel.
> > 
> > Currently this isn't really possible, however I
while ago I got an idea how to solve this. Keep in mind this
is just a rough idea and I'm pretty sure some people
can/will point out why it is a stupid idea, but anyway:
> > 
> > The idea is to add a special category (let's call
it "active" for now) that has the following
properties:
> > - this category doesn't exist in portdir or vdb (=
no ebuilds)
> > - when portage ($pkgmanager) encounters a
"active/foo" atom in a dependency string it
executes some special code (e.g.
"$PORTDIR/scripts/active-check/foo =active/foo-1")
to determine if that atom is satisfied
> > 
> > (and yes, this kinda goes with
multi-repo/multi-format support)
> > 
> > Marius
> 
> To work this would need code in the tools like
gcc-config. Otherwise
> people could easily break their systems without
anything giving them a
> notice.

Not sure I can follow. If we had such a check script for gcc
why would it need additional code in gcc-config?

Marius

-- 
gentoo-portage-devgentoo.org mailing list

On Tue, 30 Jan 2007 17:06:51 +0100
Marius Mauch <genonegentoo.org> wrote:

> Sometimes a package has to depend on a specific version
of a slotted
> package being the "active" one to build
correctly, like in the
> current "tr1" discussion on -dev [1] or with
packages that depend on
> the running kernel.

Just to note, since you mentioned both "build" and
"running" in the same
sentence, that there can be various types of dependency
here.

Some dependencies will be pure build-time dependencies (for
example,
app-emulation/qemu needs gcc-3 to build) whereas others will
be run-time
dependencies (availability of >gcc-4.1 tr1 library when
executing the
emerged package, inclusion of a particular kernel
configuration
switch on the target).  That gives you active BDEPENDs and
active
RDEPENDs to consider.  Often there will be elements of both
- the gcc-4
tr1 dependency may require the compiler to be gcc-4.1, and
the target to
also contain the gcc-4.1 tr1 library for example.

> Currently this isn't really possible, however I while
ago I got an
> idea how to solve this. Keep in mind this is just a
rough idea and
> I'm pretty sure some people can/will point out why it
is a stupid
> idea, but anyway:
> 
> The idea is to add a special category (let's call it
"active" for
> now) that has the following properties:
> - this category doesn't exist in portdir or vdb (= no
ebuilds)
> - when portage ($pkgmanager) encounters a
"active/foo" atom in a
> dependency string it executes some special code (e.g.
> "$PORTDIR/scripts/active-check/foo
=active/foo-1") to determine if
> that atom is satisfied

This could also be done as an entry point in the ebuild
(e.g.
pkg_active()) - assuming you can source an ebuild when
calculating
dependencies.  Not sure if that would be better or worse.

Either way, these scripts need to be able to deal with the
build-time/run-time difference I mentioned above.  This
comes down to
defining the syntax of calling pkg_active() (or the script,
for that
matter) - e.g. "pkg_active RDEPEND >4.1"

Also it's unclear what you intend 'foo' to be - it could be
the name of
the package that is being checked for active-ness (e.g.
'gcc' -
returning true if the required version is active) or it
could be
related to the package that needs various things active
(e.g.
'qemu', returning true if gcc-3 is active), or it could be
generic
(e.g. 'gcc-provides-tr1', returning true if the currently
active gcc
provides the tr1 library).


Next, what do you do if the atom is not satisfied?

For normal depends, it would add the required package to the
list of
stuff to be emerged.  That can't happen here, so I guess
you'd handle
it somewhat like a blocker or mask error.  So the difference
between
this idea, and the current way to handle it (script a
build-time check
in pkg_setup() fex) is that you find out about the build
problem
earlier.  This is nice, but I don't think it's a significant
benefit.

Worth bearing in mind that you will be able to get
conflicting active
dependencies - e.g. 'emerge world' trying to build both
app-emulation/qemu (which requires gcc-3 active) and one of
these tr1
packages (requiring >gcc-4.1 active).


On a general note, introducing dynamic dependencies into the
depgraph
worries me, although I'm not sure I can articulate why.
Currently,
everything is static (you can parse the tree, and you get
the exact
same depgraph on all systems for a given set of USE flags,
no matter
what), and dynamic requirements must be dealt with at build
time
(pkg_setup()).  Implementing dynamic dependencies means that
the
depgraph now depends on the host and target systems, as well
as the
tree.  This is a big conceptual change, so I would urge
caution.


Hmm; one could get the same benefit by introducing a new
interface
(e.g. pkg_env_check()) which is defined to return true if
the
environment is ok, false otherwise (with some text to
stdout, perhaps).
The package manager would then run this function, after
building the
depgraph and finding the candidate packages to merge, for
each
candidate package - if any package fails is env_check, none
of the
packages get emerged.  Note this is then completely
independent of
depgraph creation.

In the 'tr1' example, I'd imagine something like this:

use.local.desc: boost: Use boost library for tr1 rather than
gcc's.

ebuild:

...
inherit ... toolchain-funcs versionator ...
...
DEPEND=... boost? ( dev-libs/boost ) ...
...
pkg_env_check() {
	use boost && return 0
	version_is_at_least "4.1" $(gcc-version)
&& return 0
	echo "Either USE boost, or switch to gcc later than
4.1"
}


(with a default definition, "pkg_env_check() { return
0; }" )
-- 
Kevin F. Quinn

> 
> 
> Hmm; one could get the same benefit by introducing a
new interface
> (e.g. pkg_env_check()) which is defined to return true
if the
> environment is ok, false otherwise (with some text to
stdout, perhaps).
> The package manager would then run this function, after
building the
> depgraph and finding the candidate packages to merge,
for each
> candidate package - if any package fails is env_check,
none of the
> packages get emerged.  Note this is then completely
independent of
> depgraph creation.
> 
> In the 'tr1' example, I'd imagine something like this:
> 
> use.local.desc: boost: Use boost library for tr1 rather
than gcc's.
> 
> ebuild:
> 
> ...
> inherit ... toolchain-funcs versionator ...
> ...
> DEPEND=... boost? ( dev-libs/boost ) ...
> ...
> pkg_env_check() {
> 	use boost && return 0
> 	version_is_at_least "4.1" $(gcc-version)
&& return 0
> 	echo "Either USE boost, or switch to gcc later
than 4.1"
> }
> 
> 
> (with a default definition, "pkg_env_check() {
return 0; }" )

In an ideal system you'd want this stuff in the metadata
cache so that
the resolver can deal with it up front.  All resolution is a
brute force
metadata search; and assuming we had all the necessary data
up front, we
can optimize the search there (see pkgcore and restriction
subsystem)
versus IMHO doing a 'dumb' search and then running through a
list of
criteria for inclusion.

This is the same reason why built_with_use in pkg_setup is
really just
use_deps; these metadata should be included during
resolution, not after.

With that said, I'm not sure how easy it would be to rewrite
that code;
and this is simpler in that it's just a few bash functions
as opposed to
 more resolver foo.
-- 
gentoo-portage-devgentoo.org mailing list

On Tue, 30 Jan 2007 17:06:51 +0100
Marius Mauch <genonegentoo.org> wrote:

> The idea is to add a special category (let's call it
"active" for
> now) that has the following properties:
> - this category doesn't exist in portdir or vdb (= no
ebuilds)
> - when portage ($pkgmanager) encounters a
"active/foo" atom in a
> dependency string it executes some special code (e.g.
> "$PORTDIR/scripts/active-check/foo
=active/foo-1") to determine if
> that atom is satisfied

Given that in the general case the package manager can't
change the
active provider and will have to bail with an appropriate
message that
the user needs to change it themselves, the obvious solution
to this is
the previously-discussed-somewhere-I-can't-remember ebuild
function,
called on depgraph creation, to check that it will be able
to compile
in the current system environment. It's considerably simpler
and more
generally useful than subverting DEPEND to add weird
special-case hacks
to it.
-- 
gentoo-portage-devgentoo.org mailing list