Re: Strange behavior of SCSI RAID 10 (FreeBSD, LSILogic MegaRAID)

Hi!

Alexander Goncharov wrote:
> On Tue, May 27, 2008 at 1:10 AM, Gary Palmer
<gpalmerfreebsd.org> wrote:
>
>   
>> On Mon, May 26, 2008 at 11:41:56PM +1100, Alexander
Goncharov wrote:
>>     
>>> Hello world, I have faced with following issue
on my dedicated server:
>>>
>>> 8x Opteron 885, 32gm RAM, 8x36 GM 15k rpm SCSI
with RAID 10
>>>
>>> FreeBSD 7.0-generic, 64 bit version
>>>
>>>
>>>
>>> IO performance and behavior is very strange:
>>>
>>> 1)       No other process are running:
>>>
>>> Memory stat:
>>>
>>> Mem: 8796K Active, 9372K Inact, 80M Wired, 36K
Cache, 12M Buf, 31G Free
>>>
>>> Copy 3GB file first time
>>>
>>> dd if=/home/3gb_file of=/home/3gb_file2
>>> 6291456+0 records in
>>> 6291456+0 records out
>>> 3221225472 bytes transferred in 138.842926 secs
(23200501 bytes/sec)
>>>
>>> 20MBS is very poor?
>>>
>>>       

Alexander, you're using default block size of 512 bytes. In
this case 
your hardware has to:

while (!eof(source_file)) {
  locate next 512 byte block
  read it
  locate writing position
  write block
}

So it does 6291456*2 searches. Moreover you're working with
files and 
thus dd has to go throug filesystem layer which involves
additional 
overhead.

Please try increasing block size to, for example, 8 mb (add
bs=8m to the 
arguments of dd)

>>> Memory stat now:
>>>
>>> Mem: 8940K Active, 5951M Inact, 287M Wired, 36K
Cache, 214M Buf, 25G Free
>>>
>>> 2)       Copy the same file again:
>>>
>>>  dd if=/home/3gb_file of=/home/3gb_file2
>>> 6291456+0 records in
>>> 6291456+0 records out
>>> 3221225472 bytes transferred in 30.433515 secs
(105844674 bytes/sec)
>>>
>>> 100MBs ? much better
>>>
>>>       
>> The "Inact" (Inactive) went up - the 3GiB
file is now cached in
>> memory.  So the second (and subsequent) runs are
going from cached
>> memory so your 100MiB/sec transfer is actually just
testing write
>> speed, not read/write speed.  This is the same for
your other
>> tests too
>>
>> "Inact" is memory that has been used and
is being kept around incase
>> it is used again, in other words its caching file
data in the
>> "Inact" region in top.  It'll be reused
if something else needs
>> the memory, but until then it sticks around.
>>
>> Remember - you are copying the file from and to the
same
>> filesystem - this is always going to appear slow
relative to
>> pure read or pure write tests.  A pure write test
is effectively
>> what you have when you're getting your 100MiB/sec
test result since
>> its just writing out from cache memory.
>>
>> Regards,
>>
>> Gary
>>
>>     
>
> Hi Gary,
>
> Big thanks for your quick reply. Read only and write
only speed is good
> ~100MBs. But I am not sure if it really good for hw
RAID 10 8x 15k rpm
> drives.
> I am worried about read/write speed usecase which is
most used at real tasks
> (data base). I was surprised 20 MBs speed, this value
is likely to one drive
> speed. But I have 8 high speed drives. Something isn't
right here.
>
> I am waiting for freebsd community help.
>
> Thanks in advance.
> Alexander
> _______________________________________________
> freebsd-scsifreebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

> To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"
>   

_______________________________________________
freebsd-scsifreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-scsi

To unsubscribe, send any mail to
"freebsd-scsi-unsubscribefreebsd.org"