Interesting deframent method which also backs up...

While surfing another subject I found this interesting
posting, but I lost the 
URL....  Fortunately, it was in my cut&paste buffer...

"All the filesystems, some more some less, have
degrading performance on 
filesystems with high rewrites. This is more or less
inevitable, in major 
part because very few speed tests are about this aspect, and
why waste time 
optimizing an issue that is not that obvious?

However, the best way by far to fix the issue is not to use
a defragmenter, 
even a background one, like the ReiserFS repacker.

Defragmenters are both dangerous and slow, because they do
same-disk copies 
and in-place modification.

Also, in any case one should backup before defragmenting.

Now, the best way to defragment, is to do a disk-to-disk
image backup followed 
by a re-format of the original partition and a disk to-disk
tree restore, for 
example (where '/dev/hda' is the active drive and
'/dev/hdc' the backup 
drive):

umount /dev/hda6
dd bs=4k if=/dev/hda6 of=/dev/hdc6
jfs_fsck /dev/hdc6
jfs_mkfs /dev/hda6
mount /dev/hda6 /mnt/hda6
mount /dev/hdc6 /mnt/hdc6
(cd /mnt/hdc6 && tar -cS -b8 --one -f - .) \
(cd /mnt/hda6 && tar -xS -b8 -p -f -)
umount /dev/hdc6

This is just a simplified example of the steps... it can be
used with 
the 'root' filesystem too with some modifications (easiest
though if done 
from a liveCD).

Doing this copy has these important advantages:

* A backup is done just before the filesystem is optimized,
as part of the 
process itself.

* Both the backup and the restore are disk-to-disk copies,
which is a lot 
faster than same-disk copying.

* One of the copies is a very fast image copy, and the other
is a sequential 
read and a sequential write, which are about as fast as a
logical copy can 
go.

The risk and slowness of in-place, same-disk defragmentation
might have been 
acceptable when backup was economical only to slow tape; but
currently backup 
to disk is the best value, and one should take advantage of
that."

Any comments?
--
GreyGeek

----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE

To my understanding most, if not all, modern *NIX
filesystems have some 
sort of "defragmenter" built in.  While some
choose to implement it as a 
background kernel thread, other only run at the time a file
is written, 
reshuffling around other files if need be.  The 1st method
is nice 
because it uses the CPU when it is "idle"
(basically just runs at a nice 
lever to where it only runs if there's nothing else to do).
 This saves 
you CPU time, the problem comes when a file needs to be
written and the 
HD is very full, now the filesystem probably also has to
defrag as it's 
writing because the kernel thread wasn't running because if
it's nice 
level and then we fall into the 2nd option.  This is a good
option as 
the overhead is inly paid when it's needed.  If not, who
cares and the 
OS keeps going on it's merry way.

Has anyone ever tried to dd a 200GB drive?  It's slow. 
Painfully slow. 
  California traffic near LA slow.  Does one have time to do
this kind 
of backup on a live system?  No.  What happens if a user us
writing a 
file while the dd is going and it happened to land in bits
that dd 
already copied over?  The file is lost.  What is the load on
the machine 
while the dd is happening?  Probably a bit high in a real
production 
environment.  What about the ide/sata?  Wouldn't it be
clobbered and the 
pipe filled?  Yes.  If the production machine works with
high-IO 
requirements the dd would suck the life out of the
application.

I applaud efforts like this.  Work just fine in a home
environment, but 
in production I'd never do this.  Modern *NIX filesystems
kick ass if 
you pick the right one for the job.  Multi-yadabyte files
you say?  XFS. 
  Computational biology full genome sequence analysis that
spouts out 10 
thousand 4K text files?  ReiserFS.  Your /boot on the
machine?  EXT3 
unmounted after boot.  For the future this might all be
brought down to 
something like ReiserFS4 which promises to be something
ground breaking, 
but it's taken my machine down twice, so I'm waiting
'till the higher 
ups say they are adding it to the regular kernel, not just
the MM 
kernel.  SSHFS and have someone else thing about what
filesystem they 
are using.  Or maybe, just maybe Mysql/Postgrel using
libferris. 
Wouldn't that be a kicker?

My $0,02.  Flame if you will...

-Cesar

GreyGeek wrote:
> While surfing another subject I found this interesting
posting, but I lost the 
> URL....  Fortunately, it was in my cut&paste
buffer...
> 
> "All the filesystems, some more some less, have
degrading performance on 
> filesystems with high rewrites. This is more or less
inevitable, in major 
> part because very few speed tests are about this
aspect, and why waste time 
> optimizing an issue that is not that obvious?
> 
> However, the best way by far to fix the issue is not to
use a defragmenter, 
> even a background one, like the ReiserFS repacker.
> 
> Defragmenters are both dangerous and slow, because they
do same-disk copies 
> and in-place modification.
> 
> Also, in any case one should backup before
defragmenting.
> 
> Now, the best way to defragment, is to do a
disk-to-disk image backup followed 
> by a re-format of the original partition and a disk
to-disk tree restore, for 
> example (where '/dev/hda' is the active drive and
'/dev/hdc' the backup 
> drive):
> 
> umount /dev/hda6
> dd bs=4k if=/dev/hda6 of=/dev/hdc6
> jfs_fsck /dev/hdc6
> jfs_mkfs /dev/hda6
> mount /dev/hda6 /mnt/hda6
> mount /dev/hdc6 /mnt/hdc6
> (cd /mnt/hdc6 && tar -cS -b8 --one -f - .) \
> (cd /mnt/hda6 && tar -xS -b8 -p -f -)
> umount /dev/hdc6
> 
> This is just a simplified example of the steps... it
can be used with 
> the 'root' filesystem too with some modifications
(easiest though if done 
> from a liveCD).
> 
> Doing this copy has these important advantages:
> 
> * A backup is done just before the filesystem is
optimized, as part of the 
> process itself.
> 
> * Both the backup and the restore are disk-to-disk
copies, which is a lot 
> faster than same-disk copying.
> 
> * One of the copies is a very fast image copy, and the
other is a sequential 
> read and a sequential write, which are about as fast as
a logical copy can 
> go.
> 
> The risk and slowness of in-place, same-disk
defragmentation might have been 
> acceptable when backup was economical only to slow
tape; but currently backup 
> to disk is the best value, and one should take
advantage of that."
> 
> Any comments?
> --
> GreyGeek
> 
> ----
> Husker Linux Users Group mailing list
> To unsubscribe, send a message to huskerlug-requestfreelists.org
> with a subject of UNSUBSCRIBE
> 

----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE

All I know is that my UFS2 filesystems remain at under 1%
fragmentation.
-Seth

On 5/14/06, GreyGeek <jkrepsneb.rr.com> wrote:
>
> While surfing another subject I found this interesting
posting, but I lost
> the
> URL....  Fortunately, it was in my cut&paste
buffer...
>
> "All the filesystems, some more some less, have
degrading performance on
> filesystems with high rewrites. This is more or less
inevitable, in major
> part because very few speed tests are about this
aspect, and why waste
> time
> optimizing an issue that is not that obvious?
>
> However, the best way by far to fix the issue is not to
use a
> defragmenter,
> even a background one, like the ReiserFS repacker.
>
> Defragmenters are both dangerous and slow, because they
do same-disk
> copies
> and in-place modification.
>
> Also, in any case one should backup before
defragmenting.
>
> Now, the best way to defragment, is to do a
disk-to-disk image backup
> followed
> by a re-format of the original partition and a disk
to-disk tree restore,
> for
> example (where '/dev/hda' is the active drive and
'/dev/hdc' the backup
> drive):
>
> umount /dev/hda6
> dd bs=4k if=/dev/hda6 of=/dev/hdc6
> jfs_fsck /dev/hdc6
> jfs_mkfs /dev/hda6
> mount /dev/hda6 /mnt/hda6
> mount /dev/hdc6 /mnt/hdc6
> (cd /mnt/hdc6 && tar -cS -b8 --one -f - .) \
> (cd /mnt/hda6 && tar -xS -b8 -p -f -)
> umount /dev/hdc6
>
> This is just a simplified example of the steps... it
can be used with
> the 'root' filesystem too with some modifications
(easiest though if done
> from a liveCD).
>
> Doing this copy has these important advantages:
>
> * A backup is done just before the filesystem is
optimized, as part of the
> process itself.
>
> * Both the backup and the restore are disk-to-disk
copies, which is a lot
> faster than same-disk copying.
>
> * One of the copies is a very fast image copy, and the
other is a
> sequential
> read and a sequential write, which are about as fast as
a logical copy can
> go.
>
> The risk and slowness of in-place, same-disk
defragmentation might have
> been
> acceptable when backup was economical only to slow
tape; but currently
> backup
> to disk is the best value, and one should take
advantage of that."
>
> Any comments?
> --
> GreyGeek
>
> ----
> Husker Linux Users Group mailing list
> To unsubscribe, send a message to huskerlug-requestfreelists.org
> with a subject of UNSUBSCRIBE
>
>
>


----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE

I heard some good things about XFS. Perhaps it would make a
good all round
file system, for everyone?  I haven't worried too much
about defragmentation
on Linux.  It's good to know my system may be
"thinking" about it.  
So far
the only thing I have are home systems, so I don't plan to
scrub my 
ReiserFSs
just yet.  ---Jim


Cesar Delgado wrote:

>To my understanding most, if not all, modern *NIX
filesystems have some 
>sort of "defragmenter" built in.  While some
choose to implement it as a 
>background kernel thread, other only run at the time a
file is written, 
>reshuffling around other files if need be.  The 1st
method is nice 
>because it uses the CPU when it is "idle"
(basically just runs at a nice 
>lever to where it only runs if there's nothing else to
do).  This saves 
>you CPU time, the problem comes when a file needs to be
written and the 
>HD is very full, now the filesystem probably also has to
defrag as it's 
>writing because the kernel thread wasn't running
because if it's nice 
>level and then we fall into the 2nd option.  This is a
good option as 
>the overhead is inly paid when it's needed.  If not,
who cares and the 
>OS keeps going on it's merry way.
>
>Has anyone ever tried to dd a 200GB drive?  It's slow. 
Painfully slow. 
>  California traffic near LA slow.  Does one have time
to do this kind 
>of backup on a live system?  No.  What happens if a user
us writing a 
>file while the dd is going and it happened to land in
bits that dd 
>already copied over?  The file is lost.  What is the
load on the machine 
>while the dd is happening?  Probably a bit high in a
real production 
>environment.  What about the ide/sata?  Wouldn't it be
clobbered and the 
>pipe filled?  Yes.  If the production machine works with
high-IO 
>requirements the dd would suck the life out of the
application.
>
>I applaud efforts like this.  Work just fine in a home
environment, but 
>in production I'd never do this.  Modern *NIX
filesystems kick ass if 
>you pick the right one for the job.  Multi-yadabyte
files you say?  XFS. 
>  Computational biology full genome sequence analysis
that spouts out 10 
>thousand 4K text files?  ReiserFS.  Your /boot on the
machine?  EXT3 
>unmounted after boot.  For the future this might all be
brought down to 
>something like ReiserFS4 which promises to be something
ground breaking, 
>but it's taken my machine down twice, so I'm waiting
'till the higher 
>ups say they are adding it to the regular kernel, not
just the MM 
>kernel.  SSHFS and have someone else thing about what
filesystem they 
>are using.  Or maybe, just maybe Mysql/Postgrel using
libferris. 
>Wouldn't that be a kicker?
>
>My $0,02.  Flame if you will...
>
>-Cesar
>
>GreyGeek wrote:
>  
>
>>While surfing another subject I found this
interesting posting, but I lost the 
>>URL....  Fortunately, it was in my cut&paste
buffer...
>>
>>"All the filesystems, some more some less,
have degrading performance on 
>>filesystems with high rewrites. This is more or less
inevitable, in major 
>>part because very few speed tests are about this
aspect, and why waste time 
>>optimizing an issue that is not that obvious?
>>
>>However, the best way by far to fix the issue is not
to use a defragmenter, 
>>even a background one, like the ReiserFS repacker.
>>
>>Defragmenters are both dangerous and slow, because
they do same-disk copies 
>>and in-place modification.
>>
>>Also, in any case one should backup before
defragmenting.
>>
>>Now, the best way to defragment, is to do a
disk-to-disk image backup followed 
>>by a re-format of the original partition and a disk
to-disk tree restore, for 
>>example (where '/dev/hda' is the active drive and
'/dev/hdc' the backup 
>>drive):
>>
>>umount /dev/hda6
>>dd bs=4k if=/dev/hda6 of=/dev/hdc6
>>jfs_fsck /dev/hdc6
>>jfs_mkfs /dev/hda6
>>mount /dev/hda6 /mnt/hda6
>>mount /dev/hdc6 /mnt/hdc6
>>(cd /mnt/hdc6 && tar -cS -b8 --one -f - .)
\
>>(cd /mnt/hda6 && tar -xS -b8 -p -f -)
>>umount /dev/hdc6
>>
>>This is just a simplified example of the steps... it
can be used with 
>>the 'root' filesystem too with some modifications
(easiest though if done 
>>from a liveCD).
>>
>>Doing this copy has these important advantages:
>>
>>* A backup is done just before the filesystem is
optimized, as part of the 
>>process itself.
>>
>>* Both the backup and the restore are disk-to-disk
copies, which is a lot 
>>faster than same-disk copying.
>>
>>* One of the copies is a very fast image copy, and
the other is a sequential 
>>read and a sequential write, which are about as fast
as a logical copy can 
>>go.
>>
>>The risk and slowness of in-place, same-disk
defragmentation might have been 
>>acceptable when backup was economical only to slow
tape; but currently backup 
>>to disk is the best value, and one should take
advantage of that."
>>
>>Any comments?
>>--
>>GreyGeek
>>
>>----
>>Husker Linux Users Group mailing list
>>To unsubscribe, send a message to
huskerlug-requestfreelists.org
>>with a subject of UNSUBSCRIBE
>>
>>    
>>
>
>----
>Husker Linux Users Group mailing list
>To unsubscribe, send a message to huskerlug-requestfreelists.org
>with a subject of UNSUBSCRIBE
>
>
>.
>
>  
>


----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE

On Sunday 14 May 2006 10:50 pm, Cesar Delgado wrote:
> My $0,02. =A0Flame if you will...

No flames.  I wanted everyone's $0.02, especially yours.

I have have used 'dd' only for simple stuff and had no
idea how it would wo=
rk=20
on something as big as a 250GB drive, which is one reason
why I posted the=
=20
question.


How many hours do you think it would take to
dd bs=3D4k   if=3D/dev/hda6   of=3D/dev/hdc6
between two 250GB  drives if both had been unmounted?

I take it from your answer that you could use hda6 as a
source while it was=
=20
mounted and in use, but hdc6 would have to be unmounted.  
Does Ext3 or=20
ResierFS have cluster sizes?  If bs were set to the cluster
size would dd b=
e=20
faster?
GreyGeek

----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE

ReiserFS is the nice all around FS for everyone (except in
the /boot, 
always make this ext2 and unmount it after you've booted). 
Reiser 
handles small files beautifully.  I'm sure some of the
people on this 
mailing lsit dablle in home movies and such.  but these
files realy 
aren't that big in the end.  Maybe a couple of Gigs.  Most
of the files 
in the system are small.  So, the video files are the
exception to the 
rule.  XFS does quite poorly with the little files.  So,
even though you 
might see performance when you edit that movie once every 6
months. 
Every other day the system would loose.  XFS, in my opinion,
good when 
most of your files are > 1 - 10GB.  ReiserFS great for
when most of your 
files are less than that.  So, if you want to play.  Make an
XFS 
partition, put your movies there, and the rest of your box
on Reiser 
(except /boot).

-Cesar

Jim Worrest wrote:
> I heard some good things about XFS. Perhaps it would
make a good all round
> file system, for everyone?  I haven't worried too much
about defragmentation
> on Linux.  It's good to know my system may be
"thinking" about it.  
> So far
> the only thing I have are home systems, so I don't
plan to scrub my 
> ReiserFSs
> just yet.  ---Jim
> 
> 
> Cesar Delgado wrote:
> 
> 
>>To my understanding most, if not all, modern *NIX
filesystems have some 
>>sort of "defragmenter" built in.  While
some choose to implement it as a 
>>background kernel thread, other only run at the time
a file is written, 
>>reshuffling around other files if need be.  The 1st
method is nice 
>>because it uses the CPU when it is
"idle" (basically just runs at a nice 
>>lever to where it only runs if there's nothing else
to do).  This saves 
>>you CPU time, the problem comes when a file needs to
be written and the 
>>HD is very full, now the filesystem probably also
has to defrag as it's 
>>writing because the kernel thread wasn't running
because if it's nice 
>>level and then we fall into the 2nd option.  This is
a good option as 
>>the overhead is inly paid when it's needed.  If
not, who cares and the 
>>OS keeps going on it's merry way.
>>
>>Has anyone ever tried to dd a 200GB drive?  It's
slow.  Painfully slow. 
>> California traffic near LA slow.  Does one have
time to do this kind 
>>of backup on a live system?  No.  What happens if a
user us writing a 
>>file while the dd is going and it happened to land
in bits that dd 
>>already copied over?  The file is lost.  What is the
load on the machine 
>>while the dd is happening?  Probably a bit high in a
real production 
>>environment.  What about the ide/sata?  Wouldn't it
be clobbered and the 
>>pipe filled?  Yes.  If the production machine works
with high-IO 
>>requirements the dd would suck the life out of the
application.
>>
>>I applaud efforts like this.  Work just fine in a
home environment, but 
>>in production I'd never do this.  Modern *NIX
filesystems kick ass if 
>>you pick the right one for the job.  Multi-yadabyte
files you say?  XFS. 
>> Computational biology full genome sequence analysis
that spouts out 10 
>>thousand 4K text files?  ReiserFS.  Your /boot on
the machine?  EXT3 
>>unmounted after boot.  For the future this might all
be brought down to 
>>something like ReiserFS4 which promises to be
something ground breaking, 
>>but it's taken my machine down twice, so I'm
waiting 'till the higher 
>>ups say they are adding it to the regular kernel,
not just the MM 
>>kernel.  SSHFS and have someone else thing about
what filesystem they 
>>are using.  Or maybe, just maybe Mysql/Postgrel
using libferris. 
>>Wouldn't that be a kicker?
>>
>>My $0,02.  Flame if you will...
>>
>>-Cesar
>>
>>GreyGeek wrote:
>> 
>>
>>
>>>While surfing another subject I found this
interesting posting, but I lost the 
>>>URL....  Fortunately, it was in my cut&paste
buffer...
>>>
>>>"All the filesystems, some more some less,
have degrading performance on 
>>>filesystems with high rewrites. This is more or
less inevitable, in major 
>>>part because very few speed tests are about this
aspect, and why waste time 
>>>optimizing an issue that is not that obvious?
>>>
>>>However, the best way by far to fix the issue is
not to use a defragmenter, 
>>>even a background one, like the ReiserFS
repacker.
>>>
>>>Defragmenters are both dangerous and slow,
because they do same-disk copies 
>>>and in-place modification.
>>>
>>>Also, in any case one should backup before
defragmenting.
>>>
>>>Now, the best way to defragment, is to do a
disk-to-disk image backup followed 
>>>by a re-format of the original partition and a
disk to-disk tree restore, for 
>>>example (where '/dev/hda' is the active drive
and '/dev/hdc' the backup 
>>>drive):
>>>
>>>umount /dev/hda6
>>>dd bs=4k if=/dev/hda6 of=/dev/hdc6
>>>jfs_fsck /dev/hdc6
>>>jfs_mkfs /dev/hda6
>>>mount /dev/hda6 /mnt/hda6
>>>mount /dev/hdc6 /mnt/hdc6
>>>(cd /mnt/hdc6 && tar -cS -b8 --one -f -
.) \
>>>(cd /mnt/hda6 && tar -xS -b8 -p -f -)
>>>umount /dev/hdc6
>>>
>>>This is just a simplified example of the
steps... it can be used with 
>>>the 'root' filesystem too with some
modifications (easiest though if done 
>>
>>>from a liveCD).
>>
>>>Doing this copy has these important advantages:
>>>
>>>* A backup is done just before the filesystem is
optimized, as part of the 
>>>process itself.
>>>
>>>* Both the backup and the restore are
disk-to-disk copies, which is a lot 
>>>faster than same-disk copying.
>>>
>>>* One of the copies is a very fast image copy,
and the other is a sequential 
>>>read and a sequential write, which are about as
fast as a logical copy can 
>>>go.
>>>
>>>The risk and slowness of in-place, same-disk
defragmentation might have been 
>>>acceptable when backup was economical only to
slow tape; but currently backup 
>>>to disk is the best value, and one should take
advantage of that."
>>>
>>>Any comments?
>>>--
>>>GreyGeek
>>>
>>>----
>>>Husker Linux Users Group mailing list
>>>To unsubscribe, send a message to
huskerlug-requestfreelists.org
>>>with a subject of UNSUBSCRIBE
>>>
>>>   
>>>
>>
>>----
>>Husker Linux Users Group mailing list
>>To unsubscribe, send a message to
huskerlug-requestfreelists.org
>>with a subject of UNSUBSCRIBE
>>
>>
>>.
>>
>> 
>>
> 
> 
> 
> ----
> Husker Linux Users Group mailing list
> To unsubscribe, send a message to huskerlug-requestfreelists.org
> with a subject of UNSUBSCRIBE
> 

----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE

I'm not sure how long it would take.  Don't have those
kind of drives to 
run that test.  But if somone does have 2 HDs that they'd
be wiling to 
run stuff on then please do.  Even if they are not 250GB. 
Any size 
would do.  Put stuff on one disk then dd it to another and
time it. 
Then use fdisk, partition both disk into 2 partitions right
down the 
middle.  Put stuff on one harddrive's 1st partition and
then dd that 
over to the send harddrives 1st partition.  Time that. 
We'll assume 
dd-ing is linear in size, so no you have 2 point and
extrapolate from there.

Both Reiser and Ext3 have block sizes you can set to
whatever you want. 
  It's kinda nice when you set your FS block size to the
same block size 
of the hardware RAID, increases perfomance that way.

Like I said before, I wouldn't do this on a production
machine just 
because of the stuff you'd be seding down the ide/sata bus
would clog 
the IO of othert stuff.  And if you're woried about the
fragmentation of 
your silesystem I'm guessin' what you have in there's
prety ecesary and 
IO intensive.  So, unless all your data sets fit in RAM the
filesystems 
do a mighty fine job of defraging themselves.

-Cesar

GreyGeek wrote:
> On Sunday 14 May 2006 10:50 pm, Cesar Delgado wrote:
> 
>>My $0,02. =A0Flame if you will...
> 
> 
> No flames.  I wanted everyone's $0.02, especially
yours.
> 
> I have have used 'dd' only for simple stuff and had
no idea how it would wo=
> rk=20
> on something as big as a 250GB drive, which is one
reason why I posted the=
> =20
> question.
> 
> 
> How many hours do you think it would take to
> dd bs=3D4k   if=3D/dev/hda6   of=3D/dev/hdc6
> between two 250GB  drives if both had been unmounted?
> 
> I take it from your answer that you could use hda6 as a
source while it was=
> =20
> mounted and in use, but hdc6 would have to be
unmounted.   Does Ext3 or=20
> ResierFS have cluster sizes?  If bs were set to the
cluster size would dd b=
> e=20
> faster?
> GreyGeek
> 
> ----
> Husker Linux Users Group mailing list
> To unsubscribe, send a message to huskerlug-requestfreelists.org
> with a subject of UNSUBSCRIBE
> 

----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE

I just read about the glories of XFS somewhere.  I believe
it was about
installing Gentoo.  The fellow couldn't get ReiserFS to
work so he used
XFS, and the results were quite surprising in a pleasant
way.

I will thoroughly agree with you about having a separate
boot partition.
EXT3 would be as good as anything for that, the bonus would
be GAG, the
multi-booting program could find that much more easily. I
had to keep
around my SuSE just to use it's grub to mount all my other
operating
systems.  By the way, I did not consider any large handicap,
since I do
use the SuSE distro quite extensively, and I'm not
"hostile" to it. ---Jim

Cesar Delgado wrote:
> ReiserFS is the nice all around FS for everyone (except
in the /boot, 
> always make this ext2 and unmount it after you've
booted).  Reiser 
> handles small files beautifully.  I'm sure some of the
people on this 
> mailing lsit dablle in home movies and such.  but these
files realy 
> aren't that big in the end.  Maybe a couple of Gigs. 
Most of the files 
> in the system are small.  So, the video files are the
exception to the 
> rule.  XFS does quite poorly with the little files. 
So, even though you 
> might see performance when you edit that movie once
every 6 months. 
> Every other day the system would loose.  XFS, in my
opinion, good when 
> most of your files are > 1 - 10GB.  ReiserFS great
for when most of your 
> files are less than that.  So, if you want to play. 
Make an XFS 
> partition, put your movies there, and the rest of your
box on Reiser 
> (except /boot).
> 
> -Cesar
> 
> Jim Worrest wrote:
>> I heard some good things about XFS. Perhaps it
would make a good all round
>> file system, for everyone?  I haven't worried too
much about defragmentation
>> on Linux.  It's good to know my system may be
"thinking" about it.  
>> So far
>> the only thing I have are home systems, so I don't
plan to scrub my 
>> ReiserFSs
>> just yet.  ---Jim
>>
>>
>> Cesar Delgado wrote:
>>
>>
>>> To my understanding most, if not all, modern
*NIX filesystems have some 
>>> sort of "defragmenter" built in. 
While some choose to implement it as a 
>>> background kernel thread, other only run at the
time a file is written, 
>>> reshuffling around other files if need be.  The
1st method is nice 
>>> because it uses the CPU when it is
"idle" (basically just runs at a nice 
>>> lever to where it only runs if there's nothing
else to do).  This saves 
>>> you CPU time, the problem comes when a file
needs to be written and the 
>>> HD is very full, now the filesystem probably
also has to defrag as it's 
>>> writing because the kernel thread wasn't
running because if it's nice 
>>> level and then we fall into the 2nd option. 
This is a good option as 
>>> the overhead is inly paid when it's needed. 
If not, who cares and the 
>>> OS keeps going on it's merry way.
>>>
>>> Has anyone ever tried to dd a 200GB drive? 
It's slow.  Painfully slow. 
>>> California traffic near LA slow.  Does one have
time to do this kind 
>>> of backup on a live system?  No.  What happens
if a user us writing a 
>>> file while the dd is going and it happened to
land in bits that dd 
>>> already copied over?  The file is lost.  What
is the load on the machine 
>>> while the dd is happening?  Probably a bit high
in a real production 
>>> environment.  What about the ide/sata? 
Wouldn't it be clobbered and the 
>>> pipe filled?  Yes.  If the production machine
works with high-IO 
>>> requirements the dd would suck the life out of
the application.
>>>
>>> I applaud efforts like this.  Work just fine in
a home environment, but 
>>> in production I'd never do this.  Modern *NIX
filesystems kick ass if 
>>> you pick the right one for the job. 
Multi-yadabyte files you say?  XFS. 
>>> Computational biology full genome sequence
analysis that spouts out 10 
>>> thousand 4K text files?  ReiserFS.  Your /boot
on the machine?  EXT3 
>>> unmounted after boot.  For the future this
might all be brought down to 
>>> something like ReiserFS4 which promises to be
something ground breaking, 
>>> but it's taken my machine down twice, so I'm
waiting 'till the higher 
>>> ups say they are adding it to the regular
kernel, not just the MM 
>>> kernel.  SSHFS and have someone else thing
about what filesystem they 
>>> are using.  Or maybe, just maybe Mysql/Postgrel
using libferris. 
>>> Wouldn't that be a kicker?
>>>
>>> My $0,02.  Flame if you will...
>>>
>>> -Cesar
>>>
>>> GreyGeek wrote:
>>>
>>>
>>>
>>>> While surfing another subject I found this
interesting posting, but I lost the 
>>>> URL....  Fortunately, it was in my
cut&paste buffer...
>>>>
>>>> "All the filesystems, some more some
less, have degrading performance on 
>>>> filesystems with high rewrites. This is
more or less inevitable, in major 
>>>> part because very few speed tests are about
this aspect, and why waste time 
>>>> optimizing an issue that is not that
obvious?
>>>>
>>>> However, the best way by far to fix the
issue is not to use a defragmenter, 
>>>> even a background one, like the ReiserFS
repacker.
>>>>
>>>> Defragmenters are both dangerous and slow,
because they do same-disk copies 
>>>> and in-place modification.
>>>>
>>>> Also, in any case one should backup before
defragmenting.
>>>>
>>>> Now, the best way to defragment, is to do a
disk-to-disk image backup followed 
>>>> by a re-format of the original partition
and a disk to-disk tree restore, for 
>>>> example (where '/dev/hda' is the active
drive and '/dev/hdc' the backup 
>>>> drive):
>>>>
>>>> umount /dev/hda6
>>>> dd bs=4k if=/dev/hda6 of=/dev/hdc6
>>>> jfs_fsck /dev/hdc6
>>>> jfs_mkfs /dev/hda6
>>>> mount /dev/hda6 /mnt/hda6
>>>> mount /dev/hdc6 /mnt/hdc6
>>>> (cd /mnt/hdc6 && tar -cS -b8 --one
-f - .) \
>>>> (cd /mnt/hda6 && tar -xS -b8 -p -f
-)
>>>> umount /dev/hdc6
>>>>
>>>> This is just a simplified example of the
steps... it can be used with 
>>>> the 'root' filesystem too with some
modifications (easiest though if done 
>>> >from a liveCD).
>>>
>>>> Doing this copy has these important
advantages:
>>>>
>>>> * A backup is done just before the
filesystem is optimized, as part of the 
>>>> process itself.
>>>>
>>>> * Both the backup and the restore are
disk-to-disk copies, which is a lot 
>>>> faster than same-disk copying.
>>>>
>>>> * One of the copies is a very fast image
copy, and the other is a sequential 
>>>> read and a sequential write, which are
about as fast as a logical copy can 
>>>> go.
>>>>
>>>> The risk and slowness of in-place,
same-disk defragmentation might have been 
>>>> acceptable when backup was economical only
to slow tape; but currently backup 
>>>> to disk is the best value, and one should
take advantage of that."
>>>>
>>>> Any comments?
>>>> --
>>>> GreyGeek
>>>>
>>>> ----
>>>> Husker Linux Users Group mailing list
>>>> To unsubscribe, send a message to
huskerlug-requestfreelists.org
>>>> with a subject of UNSUBSCRIBE
>>>>
>>>>   
>>>>
>>> ----
>>> Husker Linux Users Group mailing list
>>> To unsubscribe, send a message to
huskerlug-requestfreelists.org
>>> with a subject of UNSUBSCRIBE
>>>
>>>
>>> .
>>>
>>>
>>>
>>
>>
>> ----
>> Husker Linux Users Group mailing list
>> To unsubscribe, send a message to
huskerlug-requestfreelists.org
>> with a subject of UNSUBSCRIBE
>>
> 
> ----
> Husker Linux Users Group mailing list
> To unsubscribe, send a message to huskerlug-requestfreelists.org
> with a subject of UNSUBSCRIBE
> 
> 
> .
> 

----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE

On Monday 15 May 2006 10:03 pm, Cesar Delgado wrote:
> ReiserFS is the nice all around FS for everyone (except
in the /boot,
> always make this ext2 and unmount it after you've
booted).

What about using ReiserFS -notail on the /boot partition?

And, doesn't ext2 force an fsck every 10 or so boots.  In
fact, IIRC, even 
Ext3 has default settings of an fsck every 150 boots or 100
days of uptime.  
Right?
GreyGeek

----
Husker Linux Users Group mailing list
To unsubscribe, send a message to huskerlug-requestfreelists.org
with a subject of UNSUBSCRIBE