abort/resume package builds could be a bit more controlled

On Sat, May 27, 2006 at 04:38:29PM -0600, Gerard Beekmans
wrote:
> Jeremy Huntwork wrote:
> >Well, that won't work because then there's
nothing to determine how to
> >possibly remove any previous source directory for
the next package.
> >Hrm..., I need to think this through a bit more...
> 
> What do you mean by that? Why would /tmp/unpacked need
to remain when 
> the source and/or build directories have been removed
already?

We don't store the name of the unpacked source directory
anywhere as
static data. How do you know what directory vim unpacks as
otherwise?
What we do is something like this:

cd $(SRC); tar -xvf `ls -t $(1) | head -n1` >
/tmp/unpacked

That unpacks the source directory and spits out the name of
the
directory it just unpacked to /tmp/unpacked.

So, if we want to remove any possible source directories
before we start
building a package, and we're not going to either unpack
the directory
first or store that static data somewhere, how are we going
to know what
directory to remove?

One possible solution is to keep that information in the LFS
book for
any package that doesn't do the default of pkgname-vers.
Tcl and vim
come to mind. And as Chris already commented in another
thread, having
that information in the book as an entity could be useful
for other
purposes as well. See the current vim instructions.

--
JH
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

Jeremy Huntwork wrote:
> On Sat, May 27, 2006 at 04:38:29PM -0600, Gerard
Beekmans wrote:
>> Jeremy Huntwork wrote:
>>> Well, that won't work because then there's
nothing to determine how to
>>> possibly remove any previous source directory
for the next package.
>>> Hrm..., I need to think this through a bit
more...
>> What do you mean by that? Why would /tmp/unpacked
need to remain when 
>> the source and/or build directories have been
removed already?
> 
> We don't store the name of the unpacked source
directory anywhere as
> static data. How do you know what directory vim unpacks
as otherwise?
> What we do is something like this:
> 
> cd $(SRC); tar -xvf `ls -t $(1) | head -n1` >
/tmp/unpacked
> 
> That unpacks the source directory and spits out the
name of the
> directory it just unpacked to /tmp/unpacked.
> So, if we want to remove any possible source
directories before we start
> building a package, and we're not going to either
unpack the directory
> first or store that static data somewhere, how are we
going to know what
> directory to remove?
   I suggested to Greg Schafer, a few years ago, that you
could extract 
the package directory name by 'listing the contents' and
trimming the 
top entry.  I believe he ended up with the cmd DIR=$(bunzip2
-dc $1 | 
tar tf - | head -n 1 ) and then he stripped $DIR .  Would
this be helpful?
> 
> One possible solution is to keep that information in
the LFS book for
> any package that doesn't do the default of
pkgname-vers. Tcl and vim
> come to mind. And as Chris already commented in another
thread, having
> that information in the book as an entity could be
useful for other
> purposes as well. See the current vim instructions.
> 
> --
> JH
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

Jeremy Huntwork wrote:
> That unpacks the source directory and spits out the
name of the
> directory it just unpacked to /tmp/unpacked.

Actually /tmp/unpacked contains every single file that tar
just unpacked 
last time I checked the contents of this file.

> So, if we want to remove any possible source
directories before we start
> building a package, and we're not going to either
unpack the directory
> first or store that static data somewhere, how are we
going to know what
> directory to remove?

I guess I just don't understand what you are checking for.
I see the 
flow being as follows:

1) Right now you unpack a tarball. /tmp/unpacked contains a
list of 
files that tar just created during the unpacking. It's used
to figure 
out the directory name of the package we're about to build.

2) Build the package.

If the build gets interrupted for some reason, you restart
"make."

The Makefile target checks for /tmp/unpacked to figure out
the directory 
name. If this file exists, remove the source directory
extracted from 
the file, and start over again (unpack && build).

If the file does not exist, don't attempt to remove a
source directory 
since it's not there anyways.

Now, after a package is built, the source directory is
removed (unless 
it's kept around for the special cases) but /tmp/unpacked
remains. So 
now when you try to build the *next* package, it finds
/tmp/unpacked, 
finds a directory name, tries to remove the *previous*
package's source 
directory.

Every time you unpack something, the /tmp/unpacked gets
overwritten 
anyways, so in its current state it's not going to be used
for any 
tracking. The current package doesn't need to know anything
about the 
previous package, not via /tmp/unpacked.

So what am I missing? I probably am missing something here.



-- 
Gerard Beekmans

/* If Linux doesn't have the solution, you have the wrong
problem */

-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

On 5/27/06, George Boudreau <georgeblinuxfromscratch.org> wrote:
>    I suggested to Greg Schafer, a few years ago, that
you could extract
> the package directory name by 'listing the contents'
and trimming the
> top entry.  I believe he ended up with the cmd
DIR=$(bunzip2 -dc $1 |
> tar tf - | head -n 1 ) and then he stripped $DIR . 
Would this be helpful?

I thought you guys already had that.  You have to be a
little careful
with that command because the first line is not necessarily
a
directory and could be prefixed with ./.  Here's the code
that I
ripped off from Ryan Oliver's scripts:

PKG_DIR=$( tar -t$ -f $ | head -n1 \
                | sed -e 's^\./' -e 's/.*$' || : )

The sed just strips down the path to surely get a directory
name.  I
can't recall why I had to add the || : at the end.  I was
getting
strange errors at some point.  It might not be needed.  ZM
is just z
or j or nothing depending on the suffix of ARCHIVE.

--
Dan
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

Dan Nicholson wrote:
> On 5/27/06, George Boudreau <georgeblinuxfromscratch.org> wrote:
>>    I suggested to Greg Schafer, a few years ago,
that you could extract
>> the package directory name by 'listing the
contents' and trimming the
>> top entry.  I believe he ended up with the cmd
DIR=$(bunzip2 -dc $1 |
>> tar tf - | head -n 1 ) and then he stripped $DIR . 
Would this be 
>> helpful?
> 
> I thought you guys already had that.  You have to be a
little careful
> with that command because the first line is not
necessarily a
> directory and could be prefixed with ./.
   I deliberately left out the details of how Greg
'stripped $DIR' as I 
was only making a suggestion to this group not supplying
Greg's code.
   Here's the code that I
> ripped off from Ryan Oliver's scripts:
> 
> PKG_DIR=$( tar -t$ -f $ | head -n1 \
>                | sed -e 's^\./' -e 's/.*$' || : )
   looks nice and clean..
> The sed just strips down the path to surely get a
directory name.  I
> can't recall why I had to add the || : at the end.  I
was getting
> strange errors at some point.  It might not be needed. 
ZM is just z
> or j or nothing depending on the suffix of ARCHIVE.
> 
> -- 
> Dan
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

On Sun, May 28, 2006 at 11:30:39AM -0700, Dan Nicholson
wrote:
> >top entry.  I believe he ended up with the cmd
DIR=$(bunzip2 -dc $1 |
> >tar tf - | head -n 1 ) and then he stripped $DIR . 
Would this be helpful?
> 
> I thought you guys already had that.  You have to be a
little careful
> with that command because the first line is not
necessarily a
> directory and could be prefixed with ./.  Here's the
code that I
> ripped off from Ryan Oliver's scripts:
> 
> PKG_DIR=$( tar -t$ -f $ | head -n1 \
>                | sed -e 's^\./' -e 's/.*$' || : )

We don't need new commands to extract the directory name -
that's not
the issue here. We've been able to get that just fine up to
now with
this:

tar -xvf `ls -t $(1) | head -n1` > /tmp/unpacked (when we
unpack the
tarball)

Then when we need the directory name:

ROOT=`head -n1 /tmp/unpacked | sed 's^./;s/.*'`

The solution you present above only removes the output to a
file. In our
current setup, the file is necessary because we're using
the variables
later in the game and the process is not controlled by one
environment
in a shell, but by make and its environment.

My problem is how do you dynamically retrieve the name of
the directory
*before* you unpack it so that you can remove a possibly
existing
directory of the same name, and *then* unpack it freshly?

--
JH
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

Jeremy Huntwork wrote:
> On Sun, May 28, 2006 at 11:30:39AM -0700, Dan Nicholson
wrote:
>>> top entry.  I believe he ended up with the cmd
DIR=$(bunzip2 -dc $1 |
>>> tar tf - | head -n 1 ) and then he stripped
$DIR .  Would this be helpful?
>> I thought you guys already had that.  You have to
be a little careful
>> with that command because the first line is not
necessarily a
>> directory and could be prefixed with ./.  Here's
the code that I
>> ripped off from Ryan Oliver's scripts:
>>
>> PKG_DIR=$( tar -t$ -f $ | head -n1 \
>>                | sed -e 's^\./' -e 's/.*$' || : )
> 
> We don't need new commands to extract the directory
name - that's not
> the issue here. We've been able to get that just fine
up to now with
> this:
> 
> tar -xvf `ls -t $(1) | head -n1` > /tmp/unpacked
(when we unpack the
> tarball)
> 
> Then when we need the directory name:
> 
> ROOT=`head -n1 /tmp/unpacked | sed 's^./;s/.*'`
> 
> The solution you present above only removes the output
to a file. In our
> current setup, the file is necessary because we're
using the variables
> later in the game and the process is not controlled by
one environment
> in a shell, but by make and its environment.
> 
> My problem is how do you dynamically retrieve the name
of the directory
> *before* you unpack it so that you can remove a
possibly existing
> directory of the same name, and *then* unpack it
freshly?
> 
  Jermemy:
   tar -tf $ | head -n 1 | sed -e 's^\./' -e
's/.*$'
   Will give you the root 'without' unpacking the archive.
The -t only 
lists the contents. Extracting the root dir for a kernel
happens in the 
blink of an eye.. give it a try.
> --
> JH
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page

On Sun, May 28, 2006 at 04:01:24PM -0400, George Boudreau
wrote:
>   tar -tf $ | head -n 1 | sed -e 's^\./' -e
's/.*$'
>   Will give you the root 'without' unpacking the
archive. The -t only 
> lists the contents. Extracting the root dir for a
kernel happens in the 
> blink of an eye.. give it a try.

Right. Well, I should have known I was setting myself up for
a fall
there. :/ Thanks for the kind hint, and sorry for my
previous noise. I
suppose this would make sense to implement then. I'll get
to it within
the next day or two.

--
JH
-- 
http://linuxfromscratch.org/mailman/listinfo/alfs-discu
ss
FAQ: http://www.linux
fromscratch.org/faq/
Unsubscribe: See the above information page