UFS2 optimization for many small files

I recently upgraded the disk of my mail server. The server
was initially 
installed with a single 36GB RAID1 volume with FreeBSD 5
(summer 2004). 
Over the years I upgraded to FreeBSD 6, and some months ago
I added 
another 36GB RAID1 volume and one 72GB RAID1 volume.

I then proceeded to copy my cyrus-imapd partition from
/usr/local/mail 
(on /dev/da0s1f) to the new 76GB /mail (/dev/da2s1d). During
this copy I 
noticed the disk usage of the mailboxes (as reported by
du(8)) growing 
about 20% larger in the process. Please note that cyrus
stores mailboxes 
with 1 file per message, 1 directory per IMAP-folder, and
the moved 
files are in the order of the hundred-thousands, with half
of them less 
than 8 KB large.

I tried understanding where the difference was, but I cannot
work-out 
any cause in the file systems:

[rootmail /data]# disklabel da0s1
# /dev/da0s1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
   a:   524288        0    4.2BSD     2048 16384 32776
   b:  4142832   524288      swap
   c: 71119692        0    unused        0     0         #
"raw" part, 
don't edit
   d:  4194304  4667120    4.2BSD     2048 16384 28552
   e:  1048576  8861424    4.2BSD     2048 16384     8
   f: 61209692  9910000    4.2BSD     2048 16384 28552

[rootmail /data]# disklabel da2s1
# /dev/da2s1:
8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
   c: 142253248        0    unused        0     0         #
"raw" part, 
don't edit
   d: 142253248        0    4.2BSD     2048 16384 28552


What can I look at, now?

Should I decide to reformat my disk, what newfs parameters
you'd advice 
for my case?

Thanks,
   Angelo.

PS: here follows the disk definitions: why the disk
formatted during the 
initial FreeBSD5 setup (da0) has a different geometry than
the one 
formatted later with FreeBSD6 (da1, hardware identical to
da0)?  Maybe 
this is influencing the block occupation?

------------------------------------------------------------
---------
[rootmail /data]# fdisk /dev/da0
******* Working on device /dev/da0 *******
parameters extracted from in-core disklabel are:
cylinders=4427 heads=255 sectors/track=63 (16065 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl
1
parameters to be used for BIOS calculations are:
cylinders=4427 heads=255 sectors/track=63 (16065 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
     start 63, size 71119692 (34726 Meg), flag 80 (active)
         beg: cyl 0/ head 1/ sector 1;
         end: cyl 1023/ head 254/ sector 63
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>
------------------------------------------------------------
-----------
[rootmail /data]# fdisk /dev/da1
******* Working on device /dev/da1 *******
parameters extracted from in-core disklabel are:
cylinders=8716 heads=255 sectors/track=32 (8160 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl
1
parameters to be used for BIOS calculations are:
cylinders=8716 heads=255 sectors/track=32 (8160 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
     start 32, size 71122528 (34727 Meg), flag 80 (active)
         beg: cyl 0/ head 1/ sector 1;
         end: cyl 1023/ head 254/ sector 32
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>
------------------------------------------------------------
-----------
[rootmail /data]# fdisk /dev/da2
******* Working on device /dev/da2 *******
parameters extracted from in-core disklabel are:
cylinders=17433 heads=255 sectors/track=32 (8160 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl
1
parameters to be used for BIOS calculations are:
cylinders=17433 heads=255 sectors/track=32 (8160 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
     start 32, size 142253248 (69459 Meg), flag 80 (active)
         beg: cyl 0/ head 1/ sector 1;
         end: cyl 1023/ head 254/ sector 32
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>


_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

On Mar 12, 2008, at 11:44 AM, Angelo Turetta wrote:
> I then proceeded to copy my cyrus-imapd partition from
/usr/local/ 
> mail (on /dev/da0s1f) to the new 76GB /mail
(/dev/da2s1d). During  
> this copy I noticed the disk usage of the mailboxes (as
reported by  
> du(8)) growing about 20% larger in the process. Please
note that  
> cyrus stores mailboxes with 1 file per message, 1
directory per IMAP- 
> folder, and the moved files are in the order of the
hundred- 
> thousands, with half of them less than 8 KB large.
>
> I tried understanding where the difference was, but I
cannot work- 
> out any cause in the file systems:

I believe Cyrus will create hard links if the same email
message is  
kept in multiple folders.  If your copy did not preserve
hard and  
symlinks, the extra space growth might be a
result...consider retrying  
the copy using the options to rsync/tar/whatever to preserve
the links.

Regards,
-- 
-Chuck

_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

Chuck Swiger wrote:
> On Mar 12, 2008, at 11:44 AM, Angelo Turetta wrote:
>> I tried understanding where the difference was, but
I cannot work-out 
>> any cause in the file systems:
> 
> I believe Cyrus will create hard links if the same
email message is kept 
> in multiple folders.

Do you know if this includes hard-linking multiple copies of
the same 
message received by different users? If it's only for
messages in the 
same user's mailbox, no way incidence can reach 20% in my
case.

Angelo.
_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"

On Mar 12, 2008, at 12:23 PM, Angelo Turetta wrote:
> Chuck Swiger wrote:
>> On Mar 12, 2008, at 11:44 AM, Angelo Turetta
wrote:
>>> I tried understanding where the difference was,
but I cannot work- 
>>> out any cause in the file systems:
>> I believe Cyrus will create hard links if the same
email message is  
>> kept in multiple folders.
>
> Do you know if this includes hard-linking multiple
copies of the  
> same message received by different users? If it's only
for messages  
> in the same user's mailbox, no way incidence can reach
20% in my case.

That's a good question.  I don't see any reason why this
couldn't  
include the same message received by different users,
too....

-- 
-Chuck

_______________________________________________
freebsd-questionsfreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-que
stions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribefreebsd.org"