lustre

Does anyone have experience using Lustre (http://www.lustre.org/) on
a Gentoo
system?  It seems that, in order to give lustre a try on one
of our clusters, we are
going to be forced to convert it to Suse, in order to have
support.

-bryan

-- 
gentoo-clustergentoo.org mailing list

Bryan,

We have gotten Lustre to work against 2.6.15 vanilla sources
on Gentoo
at my work.

I'll work on getting the ebuild(s) working again and send
some more
info to you tomorrow.

--
Joel Martin (kanaka)

Bryan Green wrote:	[Mon Aug 28 2006, 03:30:05PM EDT]
> Does anyone have experience using Lustre (http://www.lustre.org/) on
a Gentoo
> system?  It seems that, in order to give lustre a try
on one of our clusters, we are
> going to be forced to convert it to Suse, in order to
have support.
> 
> -bryan
> 
> -- 
> gentoo-clustergentoo.org mailing list

On Aug 29, 2006, at 12:24 PM, Jan Klopper wrote:

> Now, i'd love to be able to host 200gb of files trough
the apache's  
> on the nodes,
> They should cache everything on their local disk (as
much as  
> possible), to acchieve HIGH speeds.
> But they would also need to be able to fetch other
files from the  
> slower 200gb filesrever.

I think AFS has some kind of cache, but i don't know how
that works  
exactly.

Philipp
-- 
gentoo-clustergentoo.org mailing list

Am Montag, 28. August 2006 21:30 schrieb Bryan Green:
> Does anyone have experience using Lustre (http://www.lustre.org/) on
a
> Gentoo system?  It seems that, in order to give lustre
a try on one of our
> clusters, we are going to be forced to convert it to
Suse, in order to have
> support.
>
> -bryan

In my experience gigabit ethernet is by far good enough to
serve a set of 
webservers with a few 10 thousands of web pages. If you
experience low NFS 
performance then it may help to use NFS in async mode.

Afaik Coda and AFS do what you require. I don't know AFS
but Coda will not fit 
to your Hardware infrastructure since it is designed to use
more nodes. Also 
I doubt if it will provide you with more performance then
async NFS. Coda 
(and afaik AFS also) is dedigned to provide exta redundancy
of shared data.
If you don't mind having a single pont of failure then I
would recommend async 
NFS.

Gunter
-- 
gentoo-clustergentoo.org mailing list

Hi Joel & list,

Joel Martin wrote:
> We have gotten Lustre to work against 2.6.15 vanilla
sources on Gentoo
> at my work.
> 
> I'll work on getting the ebuild(s) working again and
send some more
> info to you tomorrow.

Did you mail this info to Bryan off-list ?
I'm also interested in getting lustre to work on gentoo and
would be 
curious to the changes needed. And if you have any
throughput figures 
for your lustre setup would be very much appreciated.
(on-list or off-list)

Grtz Ramon

-- 
gentoo-clustergentoo.org mailing list

I'm working on ebuilds right now actually. We're cleaning
up some
stuff so that I can make the ebuilds and patches generally
available.
We haven't yet had the opportunity to characterize
performance
characteristics.

I'll post here when I have something ready.

--
Joel Martin (kanaka)

Ramon van Alteren wrote:	[Tue Aug 29 2006, 12:35:07PM EDT]
> Hi Joel & list,
> 
> Joel Martin wrote:
> >We have gotten Lustre to work against 2.6.15
vanilla sources on Gentoo
> >at my work.
> >
> >I'll work on getting the ebuild(s) working again
and send some more
> >info to you tomorrow.
> 
> Did you mail this info to Bryan off-list ?
> I'm also interested in getting lustre to work on
gentoo and would be 
> curious to the changes needed. And if you have any
throughput figures 
> for your lustre setup would be very much appreciated.
(on-list or off-list)
> 
> Grtz Ramon

Joel Martin wrote:
> I'm working on ebuilds right now actually. We're
cleaning up some
> stuff so that I can make the ebuilds and patches
generally available.

OK, that excellent news.
Thanx.

> We haven't yet had the opportunity to characterize
performance
> characteristics.
> 
> I'll post here when I have something ready.

Thanx again, looking forward to it.

Grtz,

Ramon


-- 
gentoo-clustergentoo.org mailing list

Bryan et all,

Here is a tarball of the relevant part of my overlay:
http://dev.gentoo.org/~kanaka/files/lustre-1.5.91-o
verlay.tgz

There are two ebuilds:
- lustre-sources-2.6.15 which is lustre patches against
2.6.15 vanilla
  kernel.
- lustre-1.5.91 which is the lustre kernel modules and user
spaces
  utilities. There is fetch restricted to match CFS's
desires so you
  will need to go to their download page to download the
package.

Note that currently this only supports amd64. Unfortunately,
the
lustre modules/utilities package wants a lusterized kernel
tree that
has at least had headers/scripts built. Therefore, in the
lustre-sources/files directory, there is a fairly generic
amd64 kernel
config that is used to do this before the sources are
installed. So to
support other arches, you will need to create a generic
kernel config
for that arch.

I've used these to build, boot and mount a Lustre
file-system from
a client perspective. I was using gcc-3.4.4 on an Opteron
system. I've
heard gcc-4.X and Lustre have issues, but I've not had a
chance to
test this.

--
Joel Martin (kanaka)