Xen, grub and suggestions for a hardware RAID?

On Tue, Feb 06, 2007 at 02:32:20AM +0100, Johan Ihren
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> So I'm lazy. Or stupid. Or both. I did try to follow
the instructions  
> previously pointed at for how to configure RAIDframe
with grub and  
> xen but I messed it up somehow. And on another box I
installed  
> RAIDframe (w/o grub or xen) only to have it run fine,
but always come  
> up with dirty parity after a reboot. Haven't figured
out why yet  
> (suggestions?). So I'm clearly not up to speed on
RAIDframe yet.
> 
> And as I'm short on time and also interested in
hot-swap (this will  
> be a remote box where the ability to use remote hands
to swap disks  
> on a running system would be really great) I'm
contemplating my first  
> hardware RAID ever.
> 
> The system has 4 x SATA disks and I'm looking at
either
> 
> * Adaptec 2410SA (supported by aac(4))
> 
> or
> 
> * 3ware Escalade 9500S-4LP (supported by twa(4) or
twe(4), although  
> looking again that seems less than clear)

I have 2 9500S-8LP in a server here, it's fast (supported
by
twa) and I didn't have any issues with theses so far. 

-- 
Manuel Bouyer <bouyerantioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la
difference
--

On Tue, 6 Feb 2007, Manuel Bouyer wrote:
>> * 3ware Escalade 9500S-4LP (supported by twa(4) or
twe(4), although
>> looking again that seems less than clear)
>
> I have 2 9500S-8LP in a server here, it's fast
(supported by
> twa) and I didn't have any issues with theses so far.

I've been using 8506S and 9500S for a while with no
problems, plus you can 
use the FreeBSD tw_cli command line utility to manage them.
The UK 3ware 
disti is telling us that 9500 series cards are on their way
out and they 
don't hold stock. They were pushing the 9550 instead (and a
PCI-Express 
version which didn't look to be supported ATM) which is less
flexible 
about what PCI slots it will fit in. Both 9500 and 9550 are
supported bye 
twa).

-- 
Stephen

On Tue, 6 Feb 2007, Manuel Bouyer wrote:
>> * 3ware Escalade 9500S-4LP (supported by twa(4) or
twe(4), although
>> looking again that seems less than clear)
>
> I have 2 9500S-8LP in a server here, it's fast
(supported by
> twa) and I didn't have any issues with theses so far.

I've been using 8506S and 9500S for a while with no
problems, plus you can 
use the FreeBSD tw_cli command line utility to manage them.
The UK 3ware 
disti is telling us that 9500 series cards are on their way
out and they 
don't hold stock. They were pushing the 9550 instead (and a
PCI-Express 
version which didn't look to be supported ATM) which is less
flexible 
about what PCI slots it will fit in. Both 9500 and 9550 are
supported bye 
twa).

-- 
Stephen

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>> So I'm lazy. Or stupid. Or both. I did try
to follow the  
>>>> instructions
>>>> previously pointed at for how to configure
RAIDframe with grub and
>>>> xen but I messed it up somehow. And on
another box I installed
>>>> RAIDframe (w/o grub or xen) only to have it
run fine, but always  
>>>> come
>>>> up with dirty parity after a reboot.
Haven't figured out why yet
>>>> (suggestions?). So I'm clearly not up to
speed on RAIDframe yet.
>>>
>>> Check that you have swapoff=YES.  Dirty parity
on reboot is often a
>>> sign of swap holding the raid set open.
>>
>> Already have swapoff=YES. Sorry, should have
mentioned that.
>
>   You will also get dirty parity on reboot if you have
not  
> unconfigured all
>   ccd (and possibly cgd) devices.  See PR#35075 for a
simple fix.

Thanks. Have no cgd's or ccd's, though. This is just a
vanilla  
RAIDframe config w/ 2 disks in a RAID0 mirror config, root
on raid0a,  
booting the std NetBSD way (i.e. no grub or anything). Still
looking  
for causes for the dirty parity.

Apart from the parity everything is working just fine, but
it would  
be nice to figure out what I'm doing wrong.

Johan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFyuVUKJmr+nqSTbYRAkgkAJ90OhMW2LwS5WmjaMizTamrIoDpCwCg
i97L
3E9ouMcUno3fazsItTjSlHU=
=TnDx
-----END PGP SIGNATURE-----

On Thu, Feb 08, 2007 at 09:54:44AM +0100, Johan Ihren
wrote:
> Thanks. Have no cgd's or ccd's, though.

Maybe a vnd?

	Geert

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>> So I'm lazy. Or stupid. Or both. I did try
to follow the  
>>>> instructions
>>>> previously pointed at for how to configure
RAIDframe with grub and
>>>> xen but I messed it up somehow. And on
another box I installed
>>>> RAIDframe (w/o grub or xen) only to have it
run fine, but always  
>>>> come
>>>> up with dirty parity after a reboot.
Haven't figured out why yet
>>>> (suggestions?). So I'm clearly not up to
speed on RAIDframe yet.
>>>
>>> Check that you have swapoff=YES.  Dirty parity
on reboot is often a
>>> sign of swap holding the raid set open.
>>
>> Already have swapoff=YES. Sorry, should have
mentioned that.
>
>   You will also get dirty parity on reboot if you have
not  
> unconfigured all
>   ccd (and possibly cgd) devices.  See PR#35075 for a
simple fix.

Thanks. Have no cgd's or ccd's, though. This is just a
vanilla  
RAIDframe config w/ 2 disks in a RAID0 mirror config, root
on raid0a,  
booting the std NetBSD way (i.e. no grub or anything). Still
looking  
for causes for the dirty parity.

Apart from the parity everything is working just fine, but
it would  
be nice to figure out what I'm doing wrong.

Johan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFyuVUKJmr+nqSTbYRAkgkAJ90OhMW2LwS5WmjaMizTamrIoDpCwCg
i97L
3E9ouMcUno3fazsItTjSlHU=
=TnDx
-----END PGP SIGNATURE-----

On Thu, Feb 08, 2007 at 09:54:44AM +0100, Johan Ihren
wrote:
> Thanks. Have no cgd's or ccd's, though.

Maybe a vnd?

	Geert

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On 6 Feb 2007, at 11:33, Konrad Neuwirth wrote:

> Johan,
>
>> The system has 4 x SATA disks and I'm looking at
either
>>
>> * Adaptec 2410SA (supported by aac(4))
>
> We have that one running in a number of machines, from
a box still  
> running
> NetBSD 2.0.2 to a few boxen on 3.1, including two
machines running
> Xen. They Just Work, have a decent Bios to configure
most things  
> from and
> we haven't had any serious problems in changing disks
with them.   
> They can
> even reconstruct their RAID set whilst you're already
running the  
> system --
> this just leads to a disk performance decrease, but not
machine  
> downtime.
>
> I don't have first hand experience with the 3ware SATA
cards, but  
> I'd just
> by the Adaptecs again because we haven't been burned by
them.

As someone else wrote the 3ware 9000-series seems to be on
its way  
out and I couldn't get my hands on one. So I got the Adaptec
2410SA.

However, although the RAID as such works just fine I am
unable to get  
it to do any type of reporting whatsoever to NetBSD. I.e.
I'm aware  
that I need to do RAID management from the BIOS rather than
a NetBSD  
userland tool and that's ok.

But I need to be notified if a disk fails, etc. This box
will sit in  
a remote facility and w/o status notifications there is zero
 
opportunity for me to know if there are issues with the
RAID. Am I  
missing something obvious here? E.g., do you get any
notification in  
NetBSD if you pull a disk from the RAID?

On Tue, 6 Feb 2007, Manuel Bouyer wrote:

>>> * 3ware Escalade 9500S-4LP (supported by twa(4)
or twe(4), although
>>> looking again that seems less than clear)
>>
>> I have 2 9500S-8LP in a server here, it's fast
(supported by
>> twa) and I didn't have any issues with theses so
far.

Stephen Borrill wrote:

> I've been using 8506S and 9500S for a while with no
problems, plus  
> you can use the FreeBSD tw_cli command line utility to
manage them.  
> The UK 3ware disti is telling us that 9500 series cards
are on  
> their way out and they don't hold stock. They were
pushing the 9550  
> instead (and a PCI-Express version which didn't look to
be  
> supported ATM) which is less flexible about what PCI
slots it will  
> fit in. Both 9500 and 9550 are supported bye twa).

Aha. So you're using the tw_cli utility from inside NetBSD,
under  
emulation? If that works, great... much more convenient than
having  
to exit to BIOS. Like having most of the benefits of
RAIDframe in  
combination with the benefits of a h/w RAID. I'll probably
try a  
9550SX next time.

Regards,

Johan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFF0wbYKJmr+nqSTbYRAumfAJ92198DTlbzRU607Fqz1Qtcg0yGoQCc
Dm0r
BzFNWdHP75vOoIKvSl7Oc9k=
=nOol
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On 6 Feb 2007, at 11:33, Konrad Neuwirth wrote:

> Johan,
>
>> The system has 4 x SATA disks and I'm looking at
either
>>
>> * Adaptec 2410SA (supported by aac(4))
>
> We have that one running in a number of machines, from
a box still  
> running
> NetBSD 2.0.2 to a few boxen on 3.1, including two
machines running
> Xen. They Just Work, have a decent Bios to configure
most things  
> from and
> we haven't had any serious problems in changing disks
with them.   
> They can
> even reconstruct their RAID set whilst you're already
running the  
> system --
> this just leads to a disk performance decrease, but not
machine  
> downtime.
>
> I don't have first hand experience with the 3ware SATA
cards, but  
> I'd just
> by the Adaptecs again because we haven't been burned by
them.

As someone else wrote the 3ware 9000-series seems to be on
its way  
out and I couldn't get my hands on one. So I got the Adaptec
2410SA.

However, although the RAID as such works just fine I am
unable to get  
it to do any type of reporting whatsoever to NetBSD. I.e.
I'm aware  
that I need to do RAID management from the BIOS rather than
a NetBSD  
userland tool and that's ok.

But I need to be notified if a disk fails, etc. This box
will sit in  
a remote facility and w/o status notifications there is zero
 
opportunity for me to know if there are issues with the
RAID. Am I  
missing something obvious here? E.g., do you get any
notification in  
NetBSD if you pull a disk from the RAID?

On Tue, 6 Feb 2007, Manuel Bouyer wrote:

>>> * 3ware Escalade 9500S-4LP (supported by twa(4)
or twe(4), although
>>> looking again that seems less than clear)
>>
>> I have 2 9500S-8LP in a server here, it's fast
(supported by
>> twa) and I didn't have any issues with theses so
far.

Stephen Borrill wrote:

> I've been using 8506S and 9500S for a while with no
problems, plus  
> you can use the FreeBSD tw_cli command line utility to
manage them.  
> The UK 3ware disti is telling us that 9500 series cards
are on  
> their way out and they don't hold stock. They were
pushing the 9550  
> instead (and a PCI-Express version which didn't look to
be  
> supported ATM) which is less flexible about what PCI
slots it will  
> fit in. Both 9500 and 9550 are supported bye twa).

Aha. So you're using the tw_cli utility from inside NetBSD,
under  
emulation? If that works, great... much more convenient than
having  
to exit to BIOS. Like having most of the benefits of
RAIDframe in  
combination with the benefits of a h/w RAID. I'll probably
try a  
9550SX next time.

Regards,

Johan

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFF0wbYKJmr+nqSTbYRAumfAJ92198DTlbzRU607Fqz1Qtcg0yGoQCc
Dm0r
BzFNWdHP75vOoIKvSl7Oc9k=
=nOol
-----END PGP SIGNATURE-----

On Wed, Feb 14, 2007 at 01:55:50PM +0100, Johan Ihren
wrote:
> But I need to be notified if a disk fails, etc. This
box will sit in  
> a remote facility and w/o status notifications there is
zero  
> opportunity for me to know if there are issues with the
RAID. Am I  
> missing something obvious here?

You should as far as I know at least get a kernel message if
something
goes wrong with the RAID. My RAID controller reports the
status of
the RAID in the kernel messages:

ld0 at icp0 unit 0: <PD_A000>, type: RAID-1, status:
optimal <---

And if I remember correctly I got another kernel message
when something
went wrong with it (the original firmware apperently had
trouble with
disks >128GB).

	Kind regards

-- 
Matthias Scheler                                  http://zhadum.org.uk/