Re: finding files in dos format + .sql files without an empty last line

On Wed, Feb 21, at 12:39 Warren Head wrote:
> 2007/2/21, Ag. Hatzimanikas <a.hatzimgmail.com>:
> >
> >On Wed, Feb 21, at 10:14 Warren Head wrote:
> >
> >
> >This simple loop should do the job,checked in
bash/zsh.
> >===========================================
> >for filename in $(find . -type f);do
> >        if [[ -n $(file $filename |grep CRLF) ]]
> >        then
> >        sed 's/^M$//' $filename >
$filename.unix
> >        fi
> >done
> >===========================================
> 
> 
> This works excellent already.
> I did however forget a second issue. The final line in
the files I am
> worried about has to be an empty line.
Then add a second test to check if there is an empty line.
This should be enough.
&& [[ -z $(sed '$!d' $filename) ]] ;then ...

Where -z means that the condition is true if the length of
the string is zero.
In our case if the last line is empty the condition will be
true and then 
the sed that will remove the <CR><LF> line
terminators will be executed.

Check the bash man page for CONDITIONAL EXPRESSIONS.
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-suppo
rt
FAQ: http://
www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

On Wed, Feb 21, at 12:39 Warren Head wrote:
> 
> This works excellent already.
> I did however forget a second issue. The final line in
the files I am
> worried about has to be an empty line.
> These are actually .sql files. The DBMS will not
execute the last line, even
> if it has text in it. (Regardless of whether they are
> (now) written in a unix format.)
> 

I reread your email (actually the subject) without the
pressure of the time and I
am little confused by what you really want,so possible my
previous answer was wrong.

Do you want to find files in dos format and with an empty
line at the
end?
Do you want the opposite? (without an empty line)
Do you want to append an empty line at the end to the
converted files?
Do you want to delete (if any) the empty last line to the
converted files?

Sorry but the day's work had fatigued enough my already old
tired mind. 
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-suppo
rt
FAQ: http://
www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

On Wed, 2007-02-21 at 23:54 +0200, Ag. Hatzimanikas wrote:
> Do you want to find files in dos format and with an
empty line at the
> end?
> Do you want the opposite? (without an empty line)
> Do you want to append an empty line at the end to the
converted files?
> Do you want to delete (if any) the empty last line to
the converted files?

I think it's a question of whether the line-end character
marks the end
of every line, or whether it separates two lines. I've seen
programs
that parsed a file and ignored the last line if it didn't
end in a
new-line character. Sounds like that's what Warren is
encountering here,
separate from the DOS/Unix format issue.

I guess what we're looking for is a list of files for which
the last
character of the file is a LF. Not sure how you'd do that,
since grep
would presumably just ignore LF characters - you can't
exactly match on
them, can you?

Simon.

-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-suppo
rt
FAQ: http://
www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

On Thu, Feb 22, at 11:01 Warren Head wrote:
> 
> Some files however are still unix syntaxed files, but
because the last line
> of the file is (somehow no longer) empty, this last
line is no longer
> executed by the DBMS.
>

At a first step,to determine the reason,can you show us the
output of
a problematic file,plus a working file with:

cat -A file |tail -n 2

Simon Geard wrote:
>
> I guess what we're looking for is a list of files for
which the
> last character of the file is a LF. Not sure how you'd
do that, since
> grep would presumably just ignore LF characters - you
can't exactly 
> match on them, can you?
>
Simon,

cat is able to display these kind of symbols,included the
end of
line character as ($) with the -E switch and the tab
character as (^I) with the
-T switch.
Then it's a matter of cat'ing the file and feed it to sed.
cat -E $filename| sed '$!d;/$/!d'

What I want to see from Warren and in a way that also would
be beneficial for 
the list,is to translate the human logic to actions and
code.
Here is what I am thinking.

<human_logic>
find the files in this directory with suffix let's say *.sql
and do for them:
if one of these has: 
first (a) condition and maybe a second condition (b) etc
...
then do: the (A) *action* and possible a (B) action ect ...
otherwise (else) do: (C) action,or continue, or exit
</human_logic>

Then we can translate the above paragraph to a snippet of
code and do the job.

That way,I believe Warren himself can find the solution not
only for
this problem but for the others will come.
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-suppo
rt
FAQ: http://
www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

On Fri, Feb 23, at 09:11 Warren Head wrote:
> >
> I took a shot at it, but I am missing the linefeed
check and insert linefeed
> stuff.
> 
> #!/bin/bash
> for filename in $(find . -type f);do
>       if [[ -n $(file $filename |grep CRLF) ]]
>       then
>           sed 's/^M$//' $filename > $filename.unix
>           echo File converted from dos to unix:
$filename
>       fi
>       if [[ -n $(echo $filename | grep .sql) ]]
>       then
>           if [[ -n (cat -E $filename |
findthelastcharacterisnot$) ]]
>           then
>               insert a line feed
>               overwrite the file
>               echo File had a LF inserted: $filename
>           fi
>           # echo File ends with .sql: $filename
>       fi
> done

Hi Warren and sorry for my absent,

Now,
We are not looking for files with an empty line at the end
and,
we are looking for files without a linefeed at the last
line.Right?
(How the hell there is no linefeed,I am puzzled for this).

Here what I came up quickly (untested).

Test1.Don't look for files with an empty last line.
[[ -n $(sed '$!d' $filename) ]]
Test2,take the last character.
lastcharacter=$(cat -A $filename |sed '$!d;s/.*(.)$/1/')

Now constatanate them.

if [[ -n $(sed '$!d' $filename) ]] && [[ !
"$lastcharacter" == "$" ]];then
	sed -i '$G' $filename
fi

I maybe missing something obvious here so please correct
me.
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-suppo
rt
FAQ: http://
www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page

On 2/23/07, Ag. Hatzimanikas <a.hatzimgmail.com> wrote:
> On Fri, Feb 23, at 09:11 Warren Head wrote:
>
> Here what I came up quickly (untested).
>
> Test1.Don't look for files with an empty last line.
> [[ -n $(sed '$!d' $filename) ]]
> Test2,take the last character.
> lastcharacter=$(cat -A $filename |sed
'$!d;s/.*(.)$/1/')
>
> Now constatanate them.
>
> if [[ -n $(sed '$!d' $filename) ]] && [[ !
"$lastcharacter" == "$" ]];then
>         sed -i '$G' $filename
> fi
>
> I maybe missing something obvious here so please
correct me.

An alternative:

if cat -E $file | tail -n1 | grep -q '[^$]$'; then
    echo "No line feed at end of $file"
    echo >> $file
fi

Seemed to work in my test over here.

--
Dan
-- 
http://linuxfromscratch.org/mailman/listinfo/blfs-suppo
rt
FAQ: http://
www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page