Mikko Husari <husku husku.net> posted 45C82D9C.7010601husku.net,
excerpted below, on Tue, 06 Feb 2007 09:26:20 +0200:
> hi!
>
> i was wonderin (also tried my luck on
perfomance-gentoo, but no one
> home), what kind of partition + fs table would be
optimal on server
> and/or desktop. afaik, /usr/portage would be on its own
partition, and
> perhaps reiserfs and raid0. distfiles should be on a
different
> partition, so it would not be in the way of portage
itself... but, what
> about other parts of gentoo/linux. and is journaling
filesystem over
> striping raid just asking for trouble?
I'm running a 300-gig/disk, 4-SATA-disk, kernel RAID here,
and /love/ it!
Each of the four disks is partitioned identically to the
others, with the
corresponding partitions on each grouped in the
corresponding RAIDs.
/dev/md0 (on /dev/sd[a..d]1) is raid-1, necessary for boot,
since that's
all grub and the like understand, but that's all that's on
the raid-1.
The main system is partitioned (mdp) raid-6, /dev/md_d1 on
/dev/sd[a..d]2.
I have it partitioned three-ways, but it would be four if I
were doing it
over. The first two RAID partitions (would be three),
/dev/md_d1p1 and
md_d1p2, are identically sized, containing my main root
filesystem and a
backup, rootbak. (I'd create rootbak1 and rootbak2, now.)
/dev/md_d1p3
(would be p4 or possibly reserving that as extended, make it
p5, if I had
a second rootbak image) is a huge partition, containing an
LVM2 managed
physical volume that in turn contains all the critical
non-root data I
still want to keep on raid-6.
This is critical to my strategy -- the kernel itself knows
how to deal
with RAID, both partitioned and unpartitioned, with any
necessary
parameters passable on the kernel command line from
grub/lilo/whatever.
The same CANNOT be said about LVM2, which requires userspace
configuration
to load. Thus my choice of partitioned RAID for my main root
filesystem
and its backups. I don't have to hassle an initramfs/initrd
with just
RAID, as I'd have to do if I put root on LVM2.
Equally important are the root and rootbak images and how I
manage them.
First, a lesson I learned the hard way, root contains
everything portage
installs to, so /usr (minus some subdirs such as /usr/local
and /usr/src)
and /var (minus subdirs such as /var/log), as well as the
stuff
traditionally on root. The problem I ran into before is
that I had my
/usr partition go bad on me, and the backup version of it I
had was
somewhat outdated. Thus, the portage database in
/var/db/portage was in
sync with what was on root, but NOT in sync with what was on
/usr, which
was some six months outdated. Recovery was possible, but
not easy. Thus
my current strategy -- keep everywhere portage installs
packages to on the
same partition, so it's always in sync. If I end up
resorting to a backup
image, the entire backup image may be outdated, but what's
on the disk is
always going to be in sync with what's in the portage
database on the same
partition.
Second, the rootbak images are simply snapshots of root,
taken
periodically when I think the system is relatively stable.
I have one.
As I said, if I were doing it again, I'd make a second one,
alternating
backups, so if tragedy struck when I was actually doing the
backup, I'd
have the other one still intact and ready to boot to. It's
important to
note that while this doesn't replace off-system backups, it
does make up
for fat-fingering or the occasional botched upgrade -- and
with it on
RAID-6, it's relatively hardware failure resistant as well.
If I ever
can't boot my main root, I simply reboot and tell grub to
pass rootbak to
the kernel instead of root. What's great about this is that
other than
being a bit outdated, I have a fully functional system on
the backup
image, just as I left it when I did the snapshot. No having
to fiddle
with text-mode links/lynx to google for what went wrong in
ordered to
recover my main system, as I have a full X and KDE -- and
everything else
I had installed at the time, sitting there fully configured
and ready to
run. Even if LVM goes out on me, while I lose /home and the
like as they
are on the LVM handled raid-6 partition, I have all the
system executables
including everything on /usr ready to use to solve the
problem, should it
be necessary to google or use email or the like. (/home is
handled by
LVM2, but I could create a temp /home and use it to google
or whatever, if
necessary.)
So then as I mentioned, the last raid-6 partition is LVM2
managed and
contains logical volumes (which I still refer to as
partitions) for all my
usual data, /home and a homebak, /var/log (I decided if it
died I'd just
take the loss, so no bak of it, but it's at least on
raid-6), a mail
partition and mailbak, news (no bak), a multimedia partition
and bak,
/usr/local and a bak, ... . With portage I run
FEATURES=buildpkg, and
want it on the raid-6, so there's a package partition and
its bak here as
well.
/dev/sd[a..d]3 is swap. I have the kernel manage this
directly, as with a
priority= parameter in the mount options set to the same
thing (here,
priority=1 for all four partitions, one on each spindle),
the kernel will
stripe swap as if it was a raid-0. If I needed
zero-downtime, I'd put
swap on the raid-6 as well, so it could continue to function
if I lost up
to two disks, but as long as my data is safe, a forced
reboot on a lost
disk isn't a big deal here, and the speed of striped is
nice, so striped
it is!
/dev/sd[a..d]4 is reserved as the extended partition, thus
giving me room
to expand beyond four partitions if need be.
/dev/sd[a..d]5 is striped/raid-0 (mdp/partitioned),
containing stuff that
doesn't need the redundancy of the raid-6. /tmp and
/var/tmp of course
on one partition. The second contains the rest. The portage
and kernel
trees are easily redownloadable, so /usr/src and $PORTDIR
live here. I
run ccache, and figure the speed of striped is better than
the redundancy
of raid-6, so it's here too. Everything except the tmps are
on the same
partition, in subdirs, with symlinks from their normal
location where
necessary (as with /usr/src).
Sizes? The root images are ~10 gig. The workspace for
portage needs to
be about 5 gig in some cases. That's normally /var/tmp but
I have
/var/tmp symlinked to /tmp anyway so put it there and have a
fairly large
(~25 gig) /tmp. The other striped partition is ~12 gig.
The package
partitions are ~4 gig. /var/log is ~2 gig. /home (and its
bak) are 10
gig, but I have additional dedicated partitions for some
stuff people
would otherwise store there such as mail and multimedia.
Almost
everything is less than half full, with only 1.5 gig of the
10 gig root
partition and baks actually used, but I wanted to be sure
and leave
/plenty/ of room there.
Filesystems? Here, I'm running 100% reiserfs, thus avoiding
having to
keep other types loaded (tho I build ext2 as a module, to
use with
floppies if necessary, and of course iso9660 and udf for
CD/DVD, again
as modules, only loaded when necessary). I like the reiserfs
tail-packing,
and haven't had any problems with it since the patches
adding data=ordered
by default to reiserfs. Some say it's a dead-end, but it's
quite stable
now and due to the number of systems installed with it, will
need to be
maintained in the kernel for many years to come. I had
intended to
eventually upgrade to reiser4, but now, who knows? Anyway,
I can't see
going to a block system that wastes all that space with
small files, and
the other alternative that handles that, XFS AFAIK, has
issues of its own,
particularly on systems without very sturdy UPSs and/or
battery-backed
SCSI. I figure reiserfs has another three years anyway,
before it starts
getting crufty, and three years is a long time in kernel and
filesystems
development time, so what developments and options there
might be by then,
who knows? (According to the article someone else linked on
reiserfs,
globalfs looks to be the eventual replacement, but it's not
ready yet.)
By that time, my SATA hardware will be pretty old as well,
so it'll be
time to think about upgrades there too, and I'll probably do
the
filesystem upgrade if I think I need it, at the hardware
upgrade.
I honestly can't say what filesystem I'd recommend now, but
both ext3
(upgrading to 4 when appropriate) and reiserfs should be
safe and stable
for years to come. There's no reason to shy away from
reiserfs now,
particularly if you already know and are comfortable with it
(it works
just fine on RAID). The one thing that CAN be said,
however, is that of
all the available filesystems, the ext* series is the one
best understood
by the kernel hackers in general. The others have a limited
subset that
understand them, but ext2/3/4/... is what Linus and Andrew
and the other
"core" kernel hackers know best, and thus in some
ways could be considered
to be the best supported. In that regard, none of the
others can equal
ext*, despite the fact that its base isn't as modern in some
ways as some
of the other choices, and it's more modern in the way it
integrates with
the kernel anyway, just because all the kernel hackers know
it and thus do
the updates to it right away, that the others have to wait
for their
groups to pickup on.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."
Richard Stallman
--
gentoo-desktopgentoo.org mailing list
|
