ZFS, Dell PE2950

Hi,

I posted earlier about some results with this same system
using UFS2. 
Now trying to test ZFS.  This is a Dell PE2950 with two
Perc6 
controllers and 4 md1000 disk shelves with 750GB drives. 
16GB RAM, dual 
quad core Xeon. I recompiled our kernel to use the ULE
scheduler instead 
of default.

I could not get through an entire run of iozone without a
system 
reboot/crash.  ZFS is clearly labeled experimental, of
course.

It seems to die for sure around 10 processes, sometimes less
(this is 
the end of my output from iozone):

  Children see throughput for 10 readers          = 
135931.72 KB/sec
         Parent sees throughput for 10 readers           = 
135927.24 KB/sec
         Min throughput per process                      =  
13351.26 
KB/sec
         Max throughput per process                      =  
14172.05 KB/sec
         Avg throughput per process                      =  
13593.17 KB/sec
         Min xfer                                        =
31586816.00 KB


Some zpool info below - each volume below is a raid6 of 30PD
on one 
controller.  I may try different hardware volume configs for
fun.

zpool create test mfid0 mfid2
# pool is automatically mounted at /test

#  pool: test
# state: ONLINE
# scrub: none requested
#config:
#
#       NAME        STATE     READ WRITE CKSUM
#       test        ONLINE       0     0     0
#         mfid0     ONLINE       0     0     0
#         mfid2     ONLINE       0     0     0
#
#errors: No known data errors


-Ben
_______________________________________________
freebsd-performancefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-p
erformance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribefreebsd.org"

BENJEMAN J. MEEKHOF WROTE:
> HI,
> 
> I POSTED EARLIER ABOUT SOME RESULTS WITH THIS SAME
SYSTEM USING UFS2.
> NOW TRYING TO TEST ZFS.  THIS IS A DELL PE2950 WITH TWO
PERC6
> CONTROLLERS AND 4 MD1000 DISK SHELVES WITH 750GB
DRIVES.  16GB RAM, DUAL
> QUAD CORE XEON. I RECOMPILED OUR KERNEL TO USE THE ULE
SCHEDULER INSTEAD
> OF DEFAULT.
> 
> I COULD NOT GET THROUGH AN ENTIRE RUN OF IOZONE WITHOUT
A SYSTEM
> REBOOT/CRASH.  ZFS IS CLEARLY LABELED EXPERIMENTAL, OF
COURSE.
> 
> IT SEEMS TO DIE FOR SURE AROUND 10 PROCESSES, SOMETIMES
LESS (THIS IS
> THE END OF MY OUTPUT FROM IOZONE):
> 
>  CHILDREN SEE THROUGHPUT FOR 10 READERS          = 
135931.72 KB/SEC
>         PARENT SEES THROUGHPUT FOR 10 READERS          
=  135927.24 KB/SEC
>         MIN THROUGHPUT PER PROCESS                     
=   13351.26 KB/SEC
>         MAX THROUGHPUT PER PROCESS                     
=   14172.05 KB/SEC
>         AVG THROUGHPUT PER PROCESS                     
=   13593.17 KB/SEC
>         MIN XFER                                       
= 31586816.00 KB

CAN YOU TELL US HOW DOES THIS COMPARE TO UFS2 RESULTS YOU
POSTED
PREVIOUSLY? (SINCE YOU USED DD FOR UFS2 AND NOW IOZONE FOR
ZFS; WHAT ARE
YOUR CONCLUSIONS?)

Sure, here is an example iozone output when I tested UFS2. 
Same 
hardware config as with ZFS test.

#gstripe label -v -s 128k test /dev/mfid0 /dev/mfid2
#newfs -U -b 65536 /dev/stripe/test


"Throughput report Y-axis is type of test X-axis is
number of processes"
"Record size = 512 Kbytes "
"Output is in Kbytes/sec"

"  Initial write "  576953.38   613457.19  
627091.58   626818.95 
641763.54   626005.11   579073.92   553088.47   557498.71  
556188.95 
553340.92   548801.62

"        Rewrite "  580755.31   621562.75  
638209.55   573171.69 
633114.27   616501.92   542520.70   541662.21   525603.64  
529194.37 
506230.25   490589.89

"           Read "  505978.28   546837.12  
565786.34   310994.23 
309813.64   329930.91   351162.15   376940.64   408561.11  
432157.69 
452106.75   470176.39

"        Re-read "  523917.72   581796.50  
592393.28   314724.70 
308485.12   327409.40   350913.96   381370.12   408105.25  
434168.93 
458742.49   475407.44

-Ben

Ivan Voras wrote:
> Benjeman J. Meekhof wrote:
>> Hi,
>>
>> I posted earlier about some results with this same
system using UFS2.
>> Now trying to test ZFS.  This is a Dell PE2950 with
two Perc6
>> controllers and 4 md1000 disk shelves with 750GB
drives.  16GB RAM, dual
>> quad core Xeon. I recompiled our kernel to use the
ULE scheduler instead
>> of default.
>>
>> I could not get through an entire run of iozone
without a system
>> reboot/crash.  ZFS is clearly labeled experimental,
of course.
>>
>> It seems to die for sure around 10 processes,
sometimes less (this is
>> the end of my output from iozone):
>>
>>  Children see throughput for 10 readers          = 
135931.72 KB/sec
>>         Parent sees throughput for 10 readers      
    =  135927.24 KB/sec
>>         Min throughput per process                 
    =   13351.26 KB/sec
>>         Max throughput per process                 
    =   14172.05 KB/sec
>>         Avg throughput per process                 
    =   13593.17 KB/sec
>>         Min xfer                                   
    = 31586816.00 KB
> 
> Can you tell us how does this compare to UFS2 results
you posted
> previously? (since you used dd for UFS2 and now iozone
for ZFS; what are
> your conclusions?)
> 

_______________________________________________
freebsd-performancefreebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-p
erformance
To unsubscribe, send any mail to
"freebsd-performance-unsubscribefreebsd.org"

BENJEMAN MEEKHOF WROTE:
> SURE, HERE IS AN EXAMPLE IOZONE OUTPUT WHEN I TESTED
UFS2.  SAME 
> HARDWARE CONFIG AS WITH ZFS TEST.
> 
> #GSTRIPE LABEL -V -S 128K TEST /DEV/MFID0 /DEV/MFID2
> #NEWFS -U -B 65536 /DEV/STRIPE/TEST
> 
> 
> "THROUGHPUT REPORT Y-AXIS IS TYPE OF TEST X-AXIS
IS NUMBER OF PROCESSES"
> "RECORD SIZE = 512 KBYTES "
> "OUTPUT IS IN KBYTES/SEC"
> 

> "           READ "  505978.28   546837.12  
565786.34   310994.23 
> 309813.64   329930.91   351162.15   376940.64  
408561.11   432157.69 
> 452106.75   470176.39
> 
> "        RE-READ "  523917.72   581796.50  
592393.28   314724.70 
> 308485.12   327409.40   350913.96   381370.12  
408105.25   434168.93 
> 458742.49   475407.44

THIS IS HARD TO COMPARE TO WHAT YOU'VE POSTED BEFORE, BUT IF
IT MEANS 
THAT WITH ZFS YOU GET 136 MB/S AND WITH UFS 300-593 MB/S,
SOMETHING IS 
WRONG.

PER MY DISCUSSION WITH SCOTT LONG CAN YOU REPEAT THE TEST
FOR UFS, BUT 
CREATE GSTRIPE WITH A REALLY SMALL STRIPE SIZE, LIKE 4 KB?


> -BEN
> 
> IVAN VORAS WROTE:
>> BENJEMAN J. MEEKHOF WROTE:
>>> HI,
>>>
>>> I POSTED EARLIER ABOUT SOME RESULTS WITH THIS
SAME SYSTEM USING UFS2.
>>> NOW TRYING TO TEST ZFS.  THIS IS A DELL PE2950
WITH TWO PERC6
>>> CONTROLLERS AND 4 MD1000 DISK SHELVES WITH
750GB DRIVES.  16GB RAM, DUAL
>>> QUAD CORE XEON. I RECOMPILED OUR KERNEL TO USE
THE ULE SCHEDULER INSTEAD
>>> OF DEFAULT.
>>>
>>> I COULD NOT GET THROUGH AN ENTIRE RUN OF IOZONE
WITHOUT A SYSTEM
>>> REBOOT/CRASH.  ZFS IS CLEARLY LABELED
EXPERIMENTAL, OF COURSE.
>>>
>>> IT SEEMS TO DIE FOR SURE AROUND 10 PROCESSES,
SOMETIMES LESS (THIS IS
>>> THE END OF MY OUTPUT FROM IOZONE):
>>>
>>>  CHILDREN SEE THROUGHPUT FOR 10 READERS        
 =  135931.72 KB/SEC
>>>         PARENT SEES THROUGHPUT FOR 10 READERS  
        =  135927.24 
>>> KB/SEC
>>>         MIN THROUGHPUT PER PROCESS             
        =   13351.26 
>>> KB/SEC
>>>         MAX THROUGHPUT PER PROCESS             
        =   14172.05 
>>> KB/SEC
>>>         AVG THROUGHPUT PER PROCESS             
        =   13593.17 
>>> KB/SEC
>>>         MIN XFER                               
        = 31586816.00 KB
>>
>> CAN YOU TELL US HOW DOES THIS COMPARE TO UFS2
RESULTS YOU POSTED
>> PREVIOUSLY? (SINCE YOU USED DD FOR UFS2 AND NOW
IOZONE FOR ZFS; WHAT ARE
>> YOUR CONCLUSIONS?)
>>
> 
> _______________________________________________
> FREEBSD-PERFORMANCEFREEBSD.ORG MAILING LIST
>
HTTP://LISTS.FREEBSD.ORG/MAILMAN/LISTINFO/FREEBSD-PERFORMANC
E
> TO UNSUBSCRIBE, SEND ANY MAIL TO 
> "FREEBSD-PERFORMANCE-UNSUBSCRIBEFREEBSD.ORG"
> 

IVAN VORAS WROTE:

> PER MY DISCUSSION WITH SCOTT LONG CAN YOU REPEAT THE
TEST FOR UFS, BUT 
> CREATE GSTRIPE WITH A REALLY SMALL STRIPE SIZE, LIKE 4
KB?

ACTUALLY, NO NEED TO DO THAT - IT LOOKS LIKE IOZONE IS DOING
QUITE 
RANDOM IO OPS SO IT WON'T HELP YOU.