sed or awk under XP in batch file (DOS box) - print $1 lines then delete $1 lines from a file

I need to print $1 lines from a file, and then delete that
number of
lines.
$1 has been derived in the prior line with
wc -l sourcefile.txt | awk '{$1 /= 4 ; $1 = int($1) ; print
$1 }'

I've tried numerous awk and sed statements, a la:

sed -e -n "$1,p" sourcefile.txt > list.1
sed -i "$1d" sourcefile.txt

sed $1q list.txt > list.1 & sed -i $1d
sourcefile.txt

awk "{(FNR < $1); print}" sourcefile.txt >
list.1


Your help would be appreciated.  Thank you.

John Bartley K7AAY wrote:
> I need to print $1 lines from a file, and then delete
that number of
> lines.
> $1 has been derived in the prior line with
> wc -l sourcefile.txt | awk '{$1 /= 4 ; $1 = int($1) ;
print $1 }'

That all looks okay.  But I would probably personally do it
all in the
shell.  Try this:

  echo $(( $(wc -l < sourcefile.txt) / 4 ))

> I've tried numerous awk and sed statements, a la:
> 
> sed -e -n "$1,p" sourcefile.txt > list.1
> sed -i "$1d" sourcefile.txt
> sed $1q list.txt > list.1 & sed -i $1d
sourcefile.txt
> awk "{(FNR < $1); print}" sourcefile.txt
> list.1

Try this:

  l=$(( $(wc -l < sourcefile.txt) / 4 ))
  sed --in-place "1,$d" sourcefile.txt

Bob

On May 1, 4:02 pm, b...proulx.com (Bob Proulx) wrote:
> John Bartley K7AAY wrote:
> > I need to print $1 lines from a file, and then
delete that number of
> > lines.
> > $1 has been derived in the prior line with
> > wc -l sourcefile.txt | awk '{$1 /= 4 ; $1 =
int($1) ; print $1 }'
>
> That all looks okay.  But I would probably personally
do it all in the
> shell.  Try this:
>
>   echo $(( $(wc -l < sourcefile.txt) / 4 ))
>
> > I've tried numerous awk and sed statements, a la:
>
> > sed -e -n "$1,p" sourcefile.txt >
list.1
> > sed -i "$1d" sourcefile.txt
> > sed $1q list.txt > list.1 & sed -i $1d
sourcefile.txt
> > awk "{(FNR < $1); print}"
sourcefile.txt > list.1
>
> Try this:
>
>   l=$(( $(wc -l < sourcefile.txt) / 4 ))
>   sed --in-place "1,$d" sourcefile.txt
>
> Bob

Dangit, those don't work as you expected with XP and
GNUwin32

echo $(( $(wc -l < sourcefile.txt) / 4 ))
The system cannot find the file specified.

l=$(( $(wc -l < sourcefile.txt) / 4 ))
The system cannot find the file specified.

sed --in-place "1,$d" sourcefile.txt
sed: -e expression #1, char 7: extra characters after
command

> > John Bartley K7AAY opened with:
> > > I need to print $1 lines from a file, and
then delete that number of
> > > lines.
> > > $1 has been derived in the prior line with
> > > wc -l sourcefile.txt | awk '{$1 /= 4 ; $1 =
int($1) ; print $1 }'


> On May 1, 4:02 pm, b...proulx.com (Bob Proulx)
kindly replied:
>
>
>
> > That all looks okay.  But I would probably
personally do it all in the
> > shell.  Try this:
>
> >   echo $(( $(wc -l < sourcefile.txt) / 4 ))
>
> > > I've tried numerous awk and sed statements, a
la:
>
> > > sed -e -n "$1,p" sourcefile.txt
> list.1
> > > sed -i "$1d" sourcefile.txt
> > > sed $1q list.txt > list.1 & sed -i $1d
sourcefile.txt
> > > awk "{(FNR < $1); print}"
sourcefile.txt > list.1
>
> > Try this:
>
> >   l=$(( $(wc -l < sourcefile.txt) / 4 ))
> >   sed --in-place "1,$d"
sourcefile.txt
>
> > Bob


On May 1, 4:13 pm, John Bartley K7AAY <john.bart...gmail.com>
rebutted:

> Dangit, those don't work as you expected with XP and
GNUwin32
>
> echo $(( $(wc -l < sourcefile.txt) / 4 ))
> The system cannot find the file specified.
>
> l=$(( $(wc -l < sourcefile.txt) / 4 ))
> The system cannot find the file specified.
>
> sed --in-place "1,$d" sourcefile.txt
> sed: -e expression #1, char 7: extra characters after
command

And, OBTW, this works when I stuff the numeric value of $1
into the
second line... but I can't seem to pass the value into that
second
line. Ideas?

wc -l list.txt | awk '{$1 /= 4 ; $1 = int($1)+4 ; print $1
}'
csplit -f outfile- -n 1 list.txt $1 

On Fri, 02 May 2008 09:13:35 +1000, John Bartley K7AAY  
<john.bartleygmail.com> wrote:

> On May 1, 4:02 pm, b...proulx.com (Bob Proulx)
wrote:
>> John Bartley K7AAY wrote:
>> > I need to print $1 lines from a file, and then
delete that number of
>> > lines.
>> > $1 has been derived in the prior line with
>> > wc -l sourcefile.txt | awk '{$1 /= 4 ; $1 =
int($1) ; print $1 }'
>>
>> That all looks okay.  But I would probably
personally do it all in the
>> shell.  Try this:
>>
>>   echo $(( $(wc -l < sourcefile.txt) / 4 ))
>>
>> > I've tried numerous awk and sed statements, a
la:
>>
>> > sed -e -n "$1,p" sourcefile.txt >
list.1
>> > sed -i "$1d" sourcefile.txt
>> > sed $1q list.txt > list.1 & sed -i $1d
sourcefile.txt
>> > awk "{(FNR < $1); print}"
sourcefile.txt > list.1
>>
>> Try this:
>>
>>   l=$(( $(wc -l < sourcefile.txt) / 4 ))
>>   sed --in-place "1,$d"
sourcefile.txt
>>
>> Bob
>
> Dangit, those don't work as you expected with XP and
GNUwin32
>
> echo $(( $(wc -l < sourcefile.txt) / 4 ))
> The system cannot find the file specified.
>
> l=$(( $(wc -l < sourcefile.txt) / 4 ))
> The system cannot find the file specified.
>
> sed --in-place "1,$d" sourcefile.txt
> sed: -e expression #1, char 7: extra characters after
command
>

To set the output of a command to a variable in cmd.exe, use
this hack:

for /f %i in ('"<command>"') do set
VARIABLE=%i

So for your problem, here's a solution:

for /f %i in ('"wc -l sourcefile.txt"') do set /a
LINES=%i/4
split -l %LINES% sourcefile.txt

"set /a" instructs cmd.exe to evaluate a numerical
expression.

PS. John, I replied to your original message in
alt.msdos.batch.nt (I was  
reading newsgroups in alphabetical order).

-- 
Kam-Hung Soh <a href="http://k
amhungsoh.com/blog">Software
Salariman</a>

John Bartley K7AAY wrote:
> > >   l=$(( $(wc -l < sourcefile.txt) / 4 ))
> > >   sed --in-place "1,$d"
sourcefile.txt

> On May 1, 4:13 pm, John Bartley K7AAY
<john.bart...gmail.com>
> rebutted:
> 
> > Dangit, those don't work as you expected with XP
and GNUwin32

XP!  Sorry.  I didn't see that in your subject line.  I
would have
suggested something different then.  But actually I don't
know how to
do it on MS and so will need to defer to others.  Sorry.

> > echo $(( $(wc -l < sourcefile.txt) / 4 ))
> > The system cannot find the file specified.

That is POSIX shell syntax.  If you have bash or ksh
installed it can
do it from the command line.  I am using single quotes '...'
here but
you will need to figure out how to convert them to double
quotes for
XP.  I can't use double quotes in the shell because the
$vars would be
expanded too soon.

  sh -c 'echo $(( $(wc -l < sourcefile.txt) / 4 ))'

And therefore perhaps something like this long one-liner.

  sh -c 'sed --in-place "1,$(( $(wc -l <
sourcefile.txt) / 4 ))d" sourcefile.txt'

But if you don't have a POSIX standard shell available (e.g.
bash or
ksh or ash or dash or posh or other) then it won't work.

> > l=$(( $(wc -l < sourcefile.txt) / 4 ))
> > The system cannot find the file specified.

Darn command.com!  Blech!

> > sed --in-place "1,$d" sourcefile.txt
> > sed: -e expression #1, char 7: extra characters
after command
> 
> And, OBTW, this works when I stuff the numeric value of
$1 into the
> second line... but I can't seem to pass the value into
that second
> line. Ideas?

Sorry.  I don't know XP well enough to suggest a solution.

Good luck!
Bob

On May 1, 5:32 pm, "Kam-Hung Soh"
<kamhung....gmail.com> wrote:
> On Fri, 02 May 2008 09:13:35 +1000, John Bartley K7AAY
 
>
>
>
> <john.bart...gmail.com> wrote:
> > On May 1, 4:02 pm, b...proulx.com (Bob Proulx)
wrote:
> >> John Bartley K7AAY wrote:
> >> > I need to print $1 lines from a file, and
then delete that number of
> >> > lines.
> >> > $1 has been derived in the prior line
with
> >> > wc -l sourcefile.txt | awk '{$1 /= 4 ; $1
= int($1) ; print $1 }'
>
> >> That all looks okay.  But I would probably
personally do it all in the
> >> shell.  Try this:
>
> >>   echo $(( $(wc -l < sourcefile.txt) / 4
))
>
> >> > I've tried numerous awk and sed
statements, a la:
>
> >> > sed -e -n "$1,p" sourcefile.txt
> list.1
> >> > sed -i "$1d" sourcefile.txt
> >> > sed $1q list.txt > list.1 & sed -i
$1d sourcefile.txt
> >> > awk "{(FNR < $1); print}"
sourcefile.txt > list.1
>
> >> Try this:
>
> >>   l=$(( $(wc -l < sourcefile.txt) / 4 ))
> >>   sed --in-place "1,$d"
sourcefile.txt
>
> >> Bob
>
> > Dangit, those don't work as you expected with XP
and GNUwin32
>
> > echo $(( $(wc -l < sourcefile.txt) / 4 ))
> > The system cannot find the file specified.
>
> > l=$(( $(wc -l < sourcefile.txt) / 4 ))
> > The system cannot find the file specified.
>
> > sed --in-place "1,$d" sourcefile.txt
> > sed: -e expression #1, char 7: extra characters
after command
>
> To set the output of a command to a variable in
cmd.exe, use this hack:
>
> for /f %i in ('"<command>"') do set
VARIABLE=%i
>
> So for your problem, here's a solution:
>
> for /f %i in ('"wc -l sourcefile.txt"') do
set /a LINES=%i/4
> split -l %LINES% sourcefile.txt
>
> "set /a" instructs cmd.exe to evaluate a
numerical expression.
>
> PS. John, I replied to your original message in
alt.msdos.batch.nt (I was  
> reading newsgroups in alphabetical order).
>
> --
> Kam-Hung Soh <a href="http://k
amhungsoh.com/blog">Software
Salariman</a>


Thank you, Software Salariman! You're my hero du jour!
Now, my eight-cpu box can go run two processes each using
the four
outfile-n.txt lists....


Here's the final module with a few tweaks, changing to
GNUwin32
csplit   (which works)
from    split    (which returned this error)
  c:/progra~1/gnuwin32/bin/split.exe: invalid number
  Try 'c:/progra~1/gnuwin32/bin/split.exe --help' for more
information

Hope someone else can use this:


rem SPLITTER.BAT which lives in C:Program
FilesGNUwin32bin - that
directory is in the PATH
cls
echo y | time | grep current | awk '{print $5,$6,$7}' >
start
echo off
c:
cd
net use V: /delete
net use V:
\server.division.corporation.comareadirectorysubdir /
persistent:NO
REM obtain list of files with fully qualified UNC filenames
and use
sed  to strip off leading blanks
attrib
\server.division.corporation.comareadirectorysubdir*.*
/s
| sed "s/^...........//" > rawlist.txt
REM Grep looks for lines with specific extensions and copies
them to
LIST.TXT
grep ".doc$" rawlist.txt  > list.txt
grep ".docx$" rawlist.txt  >> list.txt
grep ".wp$" rawlist.txt  >> list.txt
grep ".rtf$" rawlist.txt  >> list.txt
grep ".wpd$" rawlist.txt  >> list.txt
grep ".wks$" rawlist.txt  >> list.txt
grep ".txt$" rawlist.txt  >> list.txt
grep ".odb$" rawlist.txt  >> list.txt
REM Count all the lines in list.txt, divide by 4 and put
value in
LINES
for /f %%i in ('"wc -l list.txt"') do set /a
LINES=%%i/4
REM use GNUwin32 to divide list.txt into four chunks, not
splitting
lines
csplit -f outfile- -n 1 list.txt %LINES% 
REM rename divided list files because I don't grok CSPLIT's
syntax for
how to do that
ren outfile-? outfile-?.txt
REM Show original list length, filtered list length and
length of each
divided list file
wc -l rawlist.txt
wc -l list.txt
wc -l outfile-?.txt
REM Show start & end time for the run: 4.5 secs to find
3,000 files,
filter and divide
echo y | time | grep current | awk '{print $5,$6,$7}'
type start