baselayout-2 stablisation plans

This is just a heads up for getting baselayout-2 stable.
Next week I
plan to put baselayout-2.0.0_rc1 into the tree without any
keywords and
it will be removed from package.mask (keeping the current
alphas masked
though). Arch teams will then be pinged on a bug to keyword
baselayout-2.

Now there are, as usual, some daemons and init scripts that
probably
won't work. For those that don't, I have either fixed or
there's patches
in our bugzilla. Here's a rough summary.

Regarding init scripts.
Init scripts should now be strictly bourne or POSIX shell.
ie, no
bashisms. bash init scripts will work, but ONLY if /bin/sh
is bash.
Shells as /bin/sh that I've tested and found to be working
are
bash
dash
busybox
zsh
FreeBSD sh

Also, as there's no bashisms, that also means no bash
arrays. This
includes our net config.
config_eth0=( "1.2.3.4/24" "5.6.7.8 netmask
255.255.255.0" )
becomes
config_eth0="'1.2.3.4/24' '5.6.7.8 netmask
255.255.255.0'"
Pay attention to the quoting there.
We still support bash arrays if and only if /bin/sh is
really bash, so
your existing net config should work as is.

Regarding start-stop-daemon. It will now test to see if the
daemon is
running after it returns as some daemons love to fork and
then error on
a bad config. So we trap this. However, there are also some
false
positives where there is a big gap between forking and
writing the
pidfile. Examples of this are ntpd (patch in portage) and
nscd (part of
glibc).

Also, when running in parallel, we now prefix all output.
This looks
very pretty on screen and is almost perfect. However, this
also comes at
a cost - daemons are now expected to close stdin, stdout and
stderr
after forking like all well written daemons should. Some
don't - like
gpm (latest rev is patched) and noip (patch in bugs). If
they don't,
then it will hang rc. Sadly, there isn't much I can do about
this.

Lastly, reiserfs3 and jfs users should comment on bug
#116016 to ensure
that my proposed patches to the userland tools actually work
as I don't
use those fs's much aside from basic testing.

Well, that's about it. It's been a fun journey making
baselayout-2 and
we're almost at the end of this road 

Thanks

Roy

PS - Now is a very good time to test it if you're concerned
about it
running on your obscure arch or use some funky networking
foo.

-- 
gentoo-devgentoo.org mailing list

Roy Marples wrote:
> This is just a heads up for getting baselayout-2
stable. Next week I
> plan to put baselayout-2.0.0_rc1 into the tree without
any keywords and
> it will be removed from package.mask (keeping the
current alphas masked
> though). Arch teams will then be pinged on a bug to
keyword
> baselayout-2.

You should find someone from the GDP to write some user
migration docs. 
The two things that spring to mind are having to use init
scripts for 
device-mapper and crypto fs stuff (I can see lots of
unbootable systems 
for those who don't realise this...) and having to compile
the new 
splashutils *after* baselayout to keep fbsplash working.

Thanks for all your work on baselayout-2!

Daniel

-- 
gentoo-devgentoo.org mailing list

On Sat, 2007-07-21 at 12:15 -0400, Daniel Drake wrote:
> Roy Marples wrote:
> > This is just a heads up for getting baselayout-2
stable. Next week I
> > plan to put baselayout-2.0.0_rc1 into the tree
without any keywords and
> > it will be removed from package.mask (keeping the
current alphas masked
> > though). Arch teams will then be pinged on a bug
to keyword
> > baselayout-2.
> 
> You should find someone from the GDP to write some user
migration docs. 

Good idea. Any volunteers?

> The two things that spring to mind are having to use
init scripts for 
> device-mapper and crypto fs stuff (I can see lots of
unbootable systems 
> for those who don't realise this...) and having to
compile the new 
> splashutils *after* baselayout to keep fbsplash
working.

I don't actually know how to set those up or what the
migration path
would be. Maybe devzero and strerror could document this as
I understand
they do this.

Thanks

Roy

-- 
gentoo-devgentoo.org mailing list

Roy Marples wrote:
> I don't actually know how to set those up or what the
migration path
> would be. Maybe devzero and strerror could document
this as I understand
> they do this.

I manage systems with a single RAID 0 stripe (not dmraid)
managed by 
device-mapper. When upgrading baselayout, we also have to
upgrade to a 
recent device-mapper version which provides the
device-mapper init 
script. Then we must run:

  # rc-update add device-mapper boot

If we don't, we get an unbootable system.

Daniel

-- 
gentoo-devgentoo.org mailing list

On Sat, 2007-07-21 at 12:45 -0400, Daniel Drake wrote:
> Roy Marples wrote:
> > I don't actually know how to set those up or what
the migration path
> > would be. Maybe devzero and strerror could
document this as I understand
> > they do this.
> 
> I manage systems with a single RAID 0 stripe (not
dmraid) managed by 
> device-mapper. When upgrading baselayout, we also have
to upgrade to a 
> recent device-mapper version which provides the
device-mapper init 
> script. Then we must run:
> 
>   # rc-update add device-mapper boot
> 
> If we don't, we get an unbootable system.

Probably a good idea to add that to the ebuild output too?

Roy

-- 
gentoo-devgentoo.org mailing list

ROY MARPLES KIRJOITTI:
> ON SAT, 2007-07-21 AT 12:45 -0400, DANIEL DRAKE WROTE:
>> ROY MARPLES WROTE:
>>> I DON'T ACTUALLY KNOW HOW TO SET THOSE UP OR
WHAT THE MIGRATION PATH
>>> WOULD BE. MAYBE DEVZERO AND STRERROR COULD
DOCUMENT THIS AS I UNDERSTAND
>>> THEY DO THIS.
>> I MANAGE SYSTEMS WITH A SINGLE RAID 0 STRIPE (NOT
DMRAID) MANAGED BY 
>> DEVICE-MAPPER. WHEN UPGRADING BASELAYOUT, WE ALSO
HAVE TO UPGRADE TO A 
>> RECENT DEVICE-MAPPER VERSION WHICH PROVIDES THE
DEVICE-MAPPER INIT 
>> SCRIPT. THEN WE MUST RUN:
>>
>>   # RC-UPDATE ADD DEVICE-MAPPER BOOT
>>
>> IF WE DON'T, WE GET AN UNBOOTABLE SYSTEM.
> 
> PROBABLY A GOOD IDEA TO ADD THAT TO THE EBUILD OUTPUT
TOO?
> 
> ROY
> 

YES. THAT COUPLED WITH THE ~ARCH PORTAGE ELOG OUTPUT AT THE
END OF
EMERGE WILL GO A LONG A WAY TO NOTIFYING USERS. IF WE WOULD
EVEN HAVE
NEWS ON THAT OF THAT...

REGARDS,
PETTERI

On 7/21/07, Roy Marples <uberlordgentoo.org> wrote:
> Now there are, as usual, some daemons and init scripts
that probably
> won't work. For those that don't, I have either fixed
or there's patches
> in our bugzilla. Here's a rough summary.

About the nscd issue we discussed on irc on friday (i.e.
daemon not
playing nice with parallel startup), your patch worked. It
stayed on
my work laptop, though, so I can't file a bug right now. But
I'll do
so monday morning.

Oh, and by the way, baselayout-2 is great stuff, thanks !

Denis.
-- 
gentoo-devgentoo.org mailing list

ROY MARPLES WROTE:
> ON SAT, 2007-07-21 AT 12:15 -0400, DANIEL DRAKE WROTE:
>> ROY MARPLES WROTE:
>>> THIS IS JUST A HEADS UP FOR GETTING
BASELAYOUT-2 STABLE. NEXT WEEK I
>>> PLAN TO PUT BASELAYOUT-2.0.0_RC1 INTO THE TREE
WITHOUT ANY KEYWORDS AND
>>> IT WILL BE REMOVED FROM PACKAGE.MASK (KEEPING
THE CURRENT ALPHAS MASKED
>>> THOUGH). ARCH TEAMS WILL THEN BE PINGED ON A
BUG TO KEYWORD
>>> BASELAYOUT-2.
>> YOU SHOULD FIND SOMEONE FROM THE GDP TO WRITE SOME
USER MIGRATION DOCS. 
> 
> GOOD IDEA. ANY VOLUNTEERS?

(GDP): YOU GIVE US THE INFO, WE'LL DOCUMENT IT FOR YOU. OR I
WILL AT LEAST.

OF EQUAL CONCERN TO ME, HOWEVER ARE A FEW ISSUES:

1) HOW WILL STABILIZATION WORK? IS IT A FORCED UPGRADE FROM
STABLE 1.X
TO 2.X, OR CAN IT BE SLOTTED?
2) IT WILL BE COMPLETELY UNMANAGEABLE TO HAVE MORE THAN ONE
SET OF
BASELAYOUT INSTRUCTIONS IN THE HANDBOOK & OTHER DOCS, SO
THERE
DEFINITELY IS A NEED FOR THE MIGRATION DOC.
3) HOW LONG WILL 1.X BE KEPT STABLE? (THIS AFFECTS HOW LONG
THE OLD
INSTRUCTIONS ARE IN THE HANDBOOKS ETC.)
4) WHAT BASELAYOUT WILL BE USED IN THE NEXT RELEASE? (MAYBE
THAT'S MORE
OF A RELENG QUESTION.)
5) DO YOU HAVE A ROUGH ESTIMATE (MONTH, 3 WEEKS, 5 WEEKS,
WHAT?) ON WHEN
THE FIRST ARCHES MIGHT BE STABILIZING 2.X?

THIS IS ALL FROM THE GDP'S PERSPECTIVE, ALMOST NONE OF IT IS
OF INTEREST
TO ME AS A USER; I EXPECT THIS SH** TO WORK JUST AS WELL AS
BASELAYOUT-1.X WHEN I HIT THE UPGRADE MYSELF. 

DOCUMENTING THIS WILL BE A MAJOR ARSEPAIN--ER, EFFORT--SINCE
BASELAYOUT
1.X DIRECTIONS ARE ALREADY MIXED IN SO WELL WITH PRETTY MUCH
EVERY DOC
WE HAVE. I'M NOT AT ALL LOOKING FORWARD TO FIXING THE DOCS
WHEN THE TIME
COMES, BUT I WILL DO IT PROVIDED I GET TO BORROW YOUR BRAIN
FOR A GOOD
LONG TIME. 

On Sat, 2007-07-21 at 11:48 -0700, Josh Saddler wrote:
> (GDP): you give us the info, we'll document it for you.
Or I will at least.

Well, the changes are as outlined in my first email.
The user changes are mainly a few variables in the
/etc/conf.d/* files
that baselayout ships. For example a few have been removed
and a few
have been added, and a few have changed.

>From our perspective, /etc/conf.d/* is quite well
documented, so GDP
could easily diff the files to see what has changed.

> Of equal concern to me, however are a few issues:
> 
> 1) How will stabilization work? Is it a forced upgrade
from stable 1.x
> to 2.x, or can it be slotted?

It cannot be slotted in any way or form.
Also, the daemon state data is non transferable as the
format has
changed in baselayout-2. This is the data that records how a
daemon was
started by s-s-d so we can check if it's running or not.
However, most
end users won't be concerned by this.

I've tested the ebuilds for upgrading and downgrading quite
extensively,
with the following notes.
baselayout-1 *requires* bash. As such /bin/sh should point
to bash
before downgrading.
If the /etc/init.d files are not immediately updated by
etc-update or
other means then errors will happen. What errors and how
severe entirely
depend on the scripts the user has in /etc/init.d

> 2) It will be completely unmanageable to have more than
one set of
> baselayout instructions in the handbook & other
docs, so there
> definitely is a need for the migration doc.

That's your call.

> 3) How long will 1.x be kept stable? (This affects how
long the old
> instructions are in the handbooks etc.)

Good question. We normally keep at least one major revision
prior to the
current stable in the tree. They can stay in the tree
indefinitely I
suppose, but the GDP should only follow the current stable.
Maybe
archive the handbook?

> 4) What baselayout will be used in the next release?
(Maybe that's more
> of a releng question.)

baselayout team just makes baselayout releases. If you mean
the LiveCD
then ask releng.

> 5) Do you have a rough estimate (month, 3 weeks, 5
weeks, what?) on when
> the first arches might be stabilizing 2.x?

No.
If the RC's prove stable and no serious regressions are
reported for a
month then we'll probably release a final 2.0.0 and get arch
teams to
mark stable a week later, or right away if no last minute
changes have
been made.

> This is all from the GDP's perspective, almost none of
it is of interest
> to me as a user; I expect this sh** to work just as
well as
> baselayout-1.x when I hit the upgrade myself. 

In theory it should just work. Especially as Gentoo/FreeBSD
has been
running it as our standard baselayout for around 6 months
now. So
hopefully this should be a very smooth release.

> Documenting this will be a major arsepain--er,
effort--since baselayout
> 1.x directions are already mixed in so well with pretty
much every doc
> we have. I'm not at all looking forward to fixing the
docs when the time
> comes, but I will do it provided I get to borrow your
brain for a good
> long time. 

Most of the documentation should still apply. I've tried to
be as
compatible as possible - the one possible exception being
networking as
baselayout-1 used bash arrays extensively. But we still
support that
if /bin/sh is bash, which it is by default for Gentoo/Linux

Thanks

Roy

-- 
gentoo-devgentoo.org mailing list

On Saturday 21 July 2007, Roy Marples wrote:
> On Sat, 2007-07-21 at 11:48 -0700, Josh Saddler wrote:
> >From our perspective, /etc/conf.d/* is quite well
documented, so GDP
>
> could easily diff the files to see what has changed.
>
> > Of equal concern to me, however are a few issues:
> >
> > 1) How will stabilization work? Is it a forced
upgrade from stable 1.x
> > to 2.x, or can it be slotted?
>
> It cannot be slotted in any way or form.
> Also, the daemon state data is non transferable as the
format has
> changed in baselayout-2. This is the data that records
how a daemon was
> started by s-s-d so we can check if it's running or
not. However, most
> end users won't be concerned by this.
>
> I've tested the ebuilds for upgrading and downgrading
quite extensively,
> with the following notes.
> baselayout-1 *requires* bash. As such /bin/sh should
point to bash
> before downgrading.
> If the /etc/init.d files are not immediately updated by
etc-update or
> other means then errors will happen. What errors and
how severe entirely
> depend on the scripts the user has in /etc/init.d

i really think this bash vs POSIX issue is getting way more
emphasis than it 
should.  i'd make the claim the majority of people out there
dont even know 
about /bin/sh, bash, dash, and friends, so most people out
there will 
have /bin/sh pointing to bash and as such, none of these
things will be an 
issue for them.  anyone who has tried to switch /bin/sh to
point to something 
else has already seen their system blow up.  or they're
using baselayout-2 
and so they're aware of the issues.

in other words, you want to put info in there about POSIX
shell and changing 
your conf.d files, great ... but dont emphasize it like it's
doomsday and all 
hell's gonna break loose when you upgrade.
-mike