fsck and soft updates (soft dependencies)

 Greetings.

 I'm about to build a file server with at least a TB of
disk space.  I'm
currently planning on hardware RAID-5, probably SATA. I've
read a thread
or two about this in the NetBSD mailing list archives.
Obviously, the file
systems are going to be big, in the hundreds of gigabytes. 
A number of
things get more complicated with file systems this large, at
least on a
limited budget like mine. Two items in particular are:

 - long fsck times for FFS
 - backups

 Two features of soft updates are particularly attractive to
me in this
regard.

 First, if i understand things correctly, one of the helpful
features of
soft updates (AKA soft dependencies) is supposedly the
ability to mount a
dirty file system and fsck it while the file system is
active.

 Second, soft updates includes a snapshot feature that can
snapshot a file
system periodically. This would enable me to quickly and
easily restore
overwritten, deleted, or changed files.  Obviously this
isn't a complete
substitute for backups, but it could help reduce the
required frequency of
backups to a seperate (or even remote) disk array.  (I
can't really
imagine backing a 500 GB file system to a tape drive.  There
are tape
drives that can hold hundreds of GBs, but i can't afford
them. 


 My questions are:

1) How well does NetBSD >=3 support fsck of a dirty FFS
file
systems that were mounted with the "softdep"
option before the event that
made them dirty?

2) How well does NetBSD >=3 support the snapshot
functionality?  I see
that there's a kernel option to enable it.  Is it
considered stable and
reliable? What opinion do people who have used it have of
it?


Regarding question #1, i've done a very simple and
preliminary test on
3.0_STABLE/sparc64.  I simply cut power to the sparc64
machine while it
was running this command:

dd if=/dev/zero of=/local/foo

(/local was mounted with the "softdep" option.)

 When the system came up, it began to fsck all the file
systems.  When it
got to /local, i typed ^C. fsck exited, and the system did
mount the file
system.  Good so far.

 However, when i fsck'd the active file system, i did need
to give fsck
the -f arg. to force it to check a mounted file system. 
Upon startup, it
informs me that it's running "NO WRITE".  When
fsck finished, it would
make no changes to the file system, even when i gave it the
"-y"
argument. Upon exit, fsck noted that "UNRESOLVED
INCONSISTENCIES
REMAIN". I also tried the fsck_ffs command, with the
same results.

 So it would appear that while i can mount a dirty file
system, i cannot
clean it.  And to clean it, i would eventually need to fsck
the file
system while it was unmounted. Is this correct?


 -johan

References:

http://www.mckusick.
com/softdep/

2006/8/22, Johan A. van Zanten <johangiantfoo.org>:
>
>  Greetings.
Hello !

>  First, if i understand things correctly, one of the
helpful features of
> soft updates (AKA soft dependencies) is supposedly the
ability to mount a
> dirty file system and fsck it while the file system is
active.
>
No. Background fsck isn't available in NetBSD yet.


> Two items in particular are:
>  - long fsck times for FFS

Well, if your system is for your private use, I would
recommend LFS
(Log-structured file system). It covers your requirements.
Very short
fsck and it is faster than normal ffs. (At least I have the
feeling it
is.), but snapshots aren't possible with LFS. The latest
changes were
pulled up to NetBSD 3, so it should work with it. Otherwise
use a
current system. I'm using it on my Notebook without any
problems since
a while and I've heard some are already using it in
production. If it
is for production, I would use ffs without softdep, since
you have
hardware raid.

>  -johan
Zafer.

On Tue, Aug 22, 2006 at 03:09:52AM -0500, Johan A. van
Zanten wrote:
> 
> [...]
> 
>  My questions are:
> 
> 1) How well does NetBSD >=3 support fsck of a dirty
FFS file
> systems that were mounted with the
"softdep" option before the event that
> made them dirty?

It doesn't support this at all. Does this exists in FreeBSD
yet ?

> 
> 2) How well does NetBSD >=3 support the snapshot
functionality?  I see
> that there's a kernel option to enable it.  Is it
considered stable and
> reliable? What opinion do people who have used it have
of it?

I'm using it (without softdeps) and didn't encounter major
problems
with it yet. It's used by dump -X in my case.

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--