UFS2 optimization for many small files

Greetings,

We're going to build a server with some 1Tb of over 500
million small
files with size from 0,5k to 4k.  I'm wonder if the ufs2 can
handle
this kind of system well. From newfs(8) the min block size
is 4k. This
is not optimal in our case, a 1k or 0,5k block is more
effective IMHO.
I'd be happy if anyone can suggest what does fragment
(block/8) in the
ufs2 mean and how this parameter works. I know It's better
to read the
full ufs2 specification, but hope that someone here can give
a hint.
Please advice with optimizations or tricks.
Thank you very much.

-- 
With best regards,        |         The Power to Serve
Nguyen Tam Chinh      |      http://www.FreeBSD.org
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

In response to "Nguyen Tam Chinh" <unixvngmail.com>:
> 
> We're going to build a server with some 1Tb of over 500
million small
> files with size from 0,5k to 4k.  I'm wonder if the
ufs2 can handle
> this kind of system well. From newfs(8) the min block
size is 4k. This
> is not optimal in our case, a 1k or 0,5k block is more
effective IMHO.
> I'd be happy if anyone can suggest what does fragment
(block/8) in the
> ufs2 mean and how this parameter works. I know It's
better to read the
> full ufs2 specification, but hope that someone here can
give a hint.
> Please advice with optimizations or tricks.
> Thank you very much.

Read the newfs man page.

Based on your assessment of your files, I'd go with a block
size of 4K
and a frag size of 500 bytes.  Blocks are broken in to frags
when a file
doesn't fill an entire block.

Make sure to set -i to about 250 or so.  An inode is needed
for each file
or directory on the filesystem, so you're liable to run out
of inodes
with the default values.

Make sure your files are organized in a directory hierarchy.
 No filesytem
that I know of performs well with huge numbers of files in a
single
directory.

Please don't cross-post.  I see no reason to copy stable with
this message.

-- 
Bill Moran
http://www.potentialtech
.com
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

Nguyen Tam Chinh wrote:
[snipped]
> Please advice with optimizations or tricks. [...]


Did you already looked at 'man 7 tuning'?


HTH,
Philipp
-- 
www.familie-ost.info/~pj
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"