ICA support

Hi!

At this moment the jhalfs-X support for ICA test is almost
ready on the 
experimental branch for the LFS book.

When RUN_ICA=1 (via config file or --do-ICA switch), the
generated Makefile 
will rebuild all chapter06 packages the number of times
defined in ITERATIONS 
(minimum 2, maximun 5), running a do_ica_prep script after
each iteration and 
keeping the build logs for each iteration. When the last
iteration is 
finished, a do_ica_work script is run and the system build
continues. 

Now, what we need is the actual do_ica_prep and do_ica_work
scripts, but I 
don't know yet how to code it.

Dan, could you help us with that?

-- 
Manuel Canales Esparcia
Usuario de LFS nº2886:       http://www.linuxfroms
cratch.org
LFS en castellano: http://www.escomp
oslinux.org/lfs-es http://www.lfs-es.com
TLDP-ES:                           http://es.tldp.org
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

On 4/8/06, M.Canales.es <manuellinuxfromscratch.org>
wrote:
>
> Now, what we need is the actual do_ica_prep and
do_ica_work scripts, but I
> don't know yet how to code it.
>
> Dan, could you help us with that?

Surely.  Could you download a copy of my scripts?  It'll be
a lot
easier for me to explain what I've done if you can see the
code. 
(Please don't make fun of my amateurish hacks)

http://www.linuxfromscratch.org/~dnicholson
/dbn-build-20060408.tar.gz

So, real quickly, this is how it works.  Variables are
defined in
build.conf.  The scripts are either in lfs or alpha
depending on which
recipe you choose.  The driver of this whole operation is
dbn-build.sh, but all of the packages get their own scripts.

Anyway, dbn-build.sh screens out a couple of scripts in
do_chroot if
RUN_ICA=Y and/or BUILD_ITERATIONS>=1.  It then runs them
according to
the order provided in the scripts.list file.  This file is
in either
lfs or alpha.  When the iteration is done, it does the ICA
and/or
farce prep.

What happens in those is defined in the function file
functions/ica. 
In the case of RUN_ICA=Y, do_ica_prep is run.  This copies
the
filesystem to a temporary location (function do_ica_files)
and then
does the stripping/removing/etc.  If RUN_FARCE=Y,
do_farce_prep is
called.  Since Ken's farce script does the stripping and
diffing at
once, this function basically wraps do_ica_files and copies
the
filesystem.

Some magic happens (move stamp and log files so they aren't
blown
away), ITER is incremented, and the loop runs again until
ITER >
BUILD_ITERATIONS.  When this condition happens, the function
do_chroot
is complete.  At this point, control returns to the main
part of the
script.  This is all the way at the bottom of dbn-build.sh.

If it's desired do_ica and/or do_farce are run.  These
functions are
defined in dbn-build.sh because they keep track of the
ITERs.  It just
counts up N until N = BUILD_ITERATIONS.  It calls
do_ica_work on
iter$N and iter$((N+1)).

do_ica_work is defined in functions/ica.  This takes two
file systems
(iter1 and iter2 in the first case) and does the diffing and
some
logging of results.  do_ica_files, do_ica_prep and
do_ica_work are all
taken from Greg's gsbuild scripts.

do_farce_work is similar in how it's called.  However,
Ken's tool does
all the work for you.  This might be easier to start with
since you
don't have to worry about the implementation of the ICA
stripping and
diffing.  The only catch with farce (look in do_farce_prep)
is that
you can't name the filelist that it creates, so it's hard
to refer
back to it in a script.

Another thing about farce is that I'm using an older,
slightly patched
version that has behavior closer to Greg's ICA functions. 
I've been
meaning to get the newer farce Ken has updated, but the one
I have
does what I want, so I'm not in a hurry.  The patched one
I'm using is
internal to my scripts in extras/farce.  Ken's newest
version (002)
can be found here:

http://www.linuxfromscratch.org/~ken/farce-002.tar.bz2

If he's listening, maybe he can enlighten us as to what's
new in that version.

That should get you started.  Feel free to ask any
questions.  I'll
take a look at what's in jhalfs, but it might not happen
today.

Good luck.

--
Dan
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

El Sábado, 8 de Abril de 2006 21:29, Dan Nicholson escribió:

> That should get you started.  Feel free to ask any
questions.  I'll
> take a look at what's in jhalfs, but it might not
happen today.

Many thanks.

Downloading both now. Wait my questions...


-- 
Manuel Canales Esparcia
Usuario de LFS nº2886:       http://www.linuxfroms
cratch.org
LFS en castellano: http://www.escomp
oslinux.org/lfs-es http://www.lfs-es.com
TLDP-ES:                           http://es.tldp.org
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

On 4/8/06, Dan Nicholson <dbn.listsgmail.com> wrote:
>
> The only catch with farce (look in do_farce_prep) is
that
> you can't name the filelist that it creates, so it's
hard to refer
> back to it in a script.

Manuel,

You may have noticed this, but filelist for farce-002 takes
in a
second argument for the output name.  So, the above
statement doesn't
apply anymore.  So, you can now call

filelist /some/temp/dir/iter1 /cmp/dir/filelist-iter1

I should let you know that filelist will also does pruning
of unwanted
directories, so you can copy the whole root directory and
then let
filelist pick the files it wants to compare.  I prefer to do
the
copying and directory pruning in do_ica_files.  Look at the
bottom of
filelist to see how Ken does the pruning.

--
Dan
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

El Domingo, 9 de Abril de 2006 20:30, Dan Nicholson
escribió:

> You may have noticed this, but filelist for farce-002
takes in a
> second argument for the output name.  So, the above
statement doesn't
> apply anymore.  So, you can now call
>
> filelist /some/temp/dir/iter1 /cmp/dir/filelist-iter1

Latter, already done.

> I should let you know that filelist will also does
pruning of unwanted
> directories, so you can copy the whole root directory
and then let
> filelist pick the files it wants to compare.  I prefer
to do the
> copying and directory pruning in do_ica_files.  Look at
the bottom of
> filelist to see how Ken does the pruning.

Latter again, I'm using a common do_copy_files for both ICA
and farce, and 
simplified the find command in filelist 

I'm now to try to figure out the command line for farce.
Look like there is an 
undocument --directory switch to set the log dir directory.


-- 
Manuel Canales Esparcia
Usuario de LFS nº2886:       http://www.linuxfroms
cratch.org
LFS en castellano: http://www.escomp
oslinux.org/lfs-es http://www.lfs-es.com
TLDP-ES:                           http://es.tldp.org
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

On 4/9/06, M.Canales.es <manuellinuxfromscratch.org>
wrote:
>
> I'm now to try to figure out the command line for
farce. Look like there is an
> undocument --directory switch to set the log dir
directory.

Ah, I'd never noticed that before.  Since it normally
outputs to the
current directory, I just do:

mkdir -pv $
cd $
farce dir1 filelist1 dir2 filelist2

It looks like with --directory, OUTDIR=$2, so I think this
would work:

farce --directory $ dir1 filelist1 dir2 filelist2

It looks like it will create the directory for you, too.

Oh, I should have told you this before.  To see what the
output of ICA
and farce will look like, look in the farce and ICA
directories here:

http://anduin.linuxfromscratch.org/~dn
icholson/lfs-alpha-20060305-reports/

In the above commands CMP_DIR=/temp/dir/farce/1v2, but you
could call
it what you want.

--
Dan
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

El Domingo, 9 de Abril de 2006 20:48, Dan Nicholson
escribió:

> It looks like with --directory, OUTDIR=$2, so I think
this would work:
>
> farce --directory $ dir1 filelist1 dir2
filelist2

Yes, that is was I will try.

> Oh, I should have told you this before.  To see what
the output of ICA
> and farce will look like, look in the farce and ICA
directories here:
>
> http://anduin.linuxfromscratch.org/~dn
icholson/lfs-alpha-20060305-reports/

Thanks, I will take a look.

I hope to have ready the preliminary code ready today, and
do the first 
ICA+farce build tomorrow. 

I'm axious to see where it will bomb 

-- 
Manuel Canales Esparcia
Usuario de LFS nº2886:       http://www.linuxfroms
cratch.org
LFS en castellano: http://www.escomp
oslinux.org/lfs-es http://www.lfs-es.com
TLDP-ES:                           http://es.tldp.org
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page