7.0-Release and 3ware 9550SXU w/BBU - horrible write performance

Email lists > FreeBSD Performance tuning

Thread: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance

Search this list only Search all lists
7.0-Release and 3ware 9550SXU w/BBU - horrible write performance
United States	2008-03-04 11:50:23
Hi, I've got a new server with a 3ware 9550SXU with the Battery. I am using FreeBSD 7.0-Release (tried both 4BSD and ULE) using AMD64 and the 3ware performance for writes is just plain horrible. Something is obviously wrong but I'm not sure what. I've got a 4 disk RAID 10 array. According to 3dm2 the cache is on. I even tried setting The StorSave preference to "Performance" with no real benefit. There seems to be something really wrong with disk performance. Here's the results from bonnie: File './Bonnie.2551', size: 104857600 Writing with putc()...done Rewriting...done Writing intelligently...done Reading with getc()...done Reading intelligently...done Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done... -------Sequential Output-------- ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU 100 9989 4.8 6739 1.0 18900 7.8 225973 98.5 1914662 99.9 177210.7 259.7 Any ideas? Anybody have one of these that's working with FreeBSD 7? ____________________________________________________________ ________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs _______________________________________________ freebsd-performancefreebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-p erformance To unsubscribe, send any mail to "freebsd-performance-unsubscribefreebsd.org"

Re: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance
United States	2008-03-04 12:47:48
alan bryan wrote: > Hi, > > I've got a new server with a 3ware 9550SXU with the > Battery. I am using FreeBSD 7.0-Release (tried both > 4BSD and ULE) using AMD64 and the 3ware performance > for writes is just plain horrible. Something is > obviously wrong but I'm not sure what. Hello Alan. Have you ever used this card with any other OS? I have about 6 of them running under linux and I too get horrible write performance from them. After several days of tuning and research, I've concluded that the cards are just stinkers reguardless of what OS they are running. That seems to be the consensus if you google 3ware write performance. I've put them in 64 and 32 bit slots, various speeds and with various drives. I had an older 3ware card running under FreeBSD 6.2 that would give decent read performance in a raid0 configuration but terrible write performance. I performed no tuning on that setup, as my application was mostly read and it was "good enough" for the time. Not terribly helpful, I admit, but I wanted you to at least think about it being the card rather than the OS. _______________________________________________ freebsd-performancefreebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-p erformance To unsubscribe, send any mail to "freebsd-performance-unsubscribefreebsd.org"

Re: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance
	2008-03-04 13:24:34
At 12:50 PM 3/4/2008, alan bryan wrote: >Hi, > >I've got a new server with a 3ware 9550SXU with the >Battery. I am using FreeBSD 7.0-Release (tried both >4BSD and ULE) using AMD64 and the 3ware performance >for writes is just plain horrible. Something is >obviously wrong but I'm not sure what. Not sure about 7.0, but I have this card on a 6.3 box. Doing something simple like % cat /dev/zero > big % iostat -c 100 tty da0 pass0 cpu tin tout KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 0 53.92 6 0.31 0.00 0 0.00 3 0 0 0 96 3 197 4.00 1 0.00 0.00 0 0.00 0 0 0 0 100 1 64 127.94 1100 137.42 0.00 0 0.00 0 0 49 5 46 0 63 128.00 1012 126.50 0.00 0 0.00 2 0 34 5 59 0 63 128.00 969 121.13 0.00 0 0.00 1 0 30 2 66 0 62 127.85 758 94.67 0.00 0 0.00 1 0 26 4 69 0 61 127.82 1252 156.25 0.00 0 0.00 1 0 50 8 42 0 63 127.59 542 67.59 0.00 0 0.00 1 0 26 1 72 0 61 127.66 1026 127.90 0.00 0 0.00 0 0 47 5 49 1 87 127.56 513 63.97 0.00 0 0.00 0 0 19 4 77 0 61 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 Shows pretty OK write performance. % dd if=/dev/zero of=/var/tmp/big bs=1024k count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 8.646134 secs (121276863 bytes/sec) This is on RAID10 on 4 Segate ST380811AS drives. 3ware device driver for 9000 series storage controllers, version: 3.60.04.003 twa0: <3ware 9000 series Storage Controller> port 0xac00-0xac3f mem 0xf4000000-0xf5ffffff,0xff2ff000-0xff2fffff irq 16 at device 3.0 on pci2 twa0: [GIANT-LOCKED] twa0: INFO: (0x15: 0x1300): Controller details:: Model 9550SX-4LP, 4 ports, Firmware FE9X 3.01.01.028, BIOS BE9X 3.01.00.024 da0: <AMCC 9550SX-4LP DISK 3.01> Fixed Direct Access SCSI-3 device da0: 100.000MB/s transfers da0: 152566MB (312455168 512 byte sectors: 255H 63S/T 19449C) I have write cache enabled and performance set for the StorSav. Its in a PCI-x slot as well. ---Mike _______________________________________________ freebsd-performancefreebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-p erformance To unsubscribe, send any mail to "freebsd-performance-unsubscribefreebsd.org"

Re: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance
United Kingdom	2008-03-04 14:20:59
On Tue, 4 Mar 2008 09:50:23 -0800 (PST) alan bryan wrote: > I've got a new server with a 3ware 9550SXU with the > Battery. I am using FreeBSD 7.0-Release (tried both > 4BSD and ULE) using AMD64 and the 3ware performance > for writes is just plain horrible. Something is > obviously wrong but I'm not sure what. > I've got a 4 disk RAID 10 array. > According to 3dm2 the cache is on. I even tried > setting The StorSave preference to "Performance" with > no real benefit. There seems to be something really > wrong with disk performance. Here's the results from > bonnie: > File './Bonnie.2551', size: 104857600 > Writing with putc()...done > Rewriting...done > Writing intelligently...done > Reading with getc()...done > Reading intelligently...done > Seeker 1...Seeker 2...Seeker 3...start > 'em...done...done...done... > -------Sequential Output-------- > ---Sequential Input-- --Random-- > -Per Char- --Block--- -Rewrite-- -Per > Char- --Block--- --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec > %CPU K/sec %CPU /sec %CPU > 100 9989 4.8 6739 1.0 18900 7.8 225973 > 98.5 1914662 > 99.9 177210.7 259.7 > Any ideas? Anybody have one of these that's working > with FreeBSD 7? I had almost the same problem. The transfer rate was less that 10Mb/s for raid-10. When a BBU arrived I thought my problems gone. Not that at once. Firstly none changed. I tested many combinatins with no avail (it lasted some hours). Then it so happed that I set StorSave "Ballanced", cache "on" and marked (by an accident) the raid for checking (or verifying... can't recall how it's named at the 3WARE BIOS). The test has began, I stopped it. Rebooted the OS. To my surprise after booting the transfer rate was quite good. BTW, the test proceeded after booting... Maybe the card did not use the cache while the battery was uncharged? Don't know (but noticed "twa0: INFO: (0x04: 0x0056): Battery charging completed:" at dmesg). Here it is now with WD RE2 1T disks: ----- 3ware device driver for 9000 series storage controllers, version: 3.70.05.001 twa0: <3ware 9000 series Storage Controller> port 0xc800-0xc8ff mem 0xf8000000-0xf9ffffff,0xfeaff000-0xfeafffff irq 16 at device 0.0 on pci4 twa0: [ITHREAD] twa0: INFO: (0x04: 0x0029): Verify started: unit=0 twa0: INFO: (0x04: 0x003D): Verify paused: unit=0 twa0: INFO: (0x15: 0x1300): Controller details:: Model 9650SE-8LPML, 8 ports, Firmware FE9X 3.08.02.005, BIOS BE9X 3.08.00.002 ----- da0 at twa0 bus 0 target 0 lun 0 da0: <AMCC 9650SE-8LP DISK 3.08> Fixed Direct Access SCSI-5 device da0: 100.000MB/s transfers da0: 953632MB (1953038336 512 byte sectors: 255H 63S/T 121571C) ----- /dev/da0s1d 902G 36G 794G 4% /space ----- btest% uname -a FreeBSD btest.ipt.ru 7.0-RELEASE FreeBSD 7.0-RELEASE #14: Mon Mar 3 18:27:26 MSK 2008 rootbtest.ipt.ru:/usr/obj/usr/src/sys/BTEST i386 btest% sudo dd if=/dev/da0 of=/dev/null bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 5.073393 secs (206681412 bytes/sec) btest% sudo dd if=/dev/zero of=/space/dd.file bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 5.670188 secs (184927909 bytes/sec) ----- WBR -- Boris Samorodov (bsam) Research Engineer, http://www.ipt.ru Telephone & Internet SP FreeBSD committer, http://www.FreeBSD.org The Power To Serve _______________________________________________ freebsd-performancefreebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-p erformance To unsubscribe, send any mail to "freebsd-performance-unsubscribefreebsd.org"

Re: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance
United Kingdom	2008-03-04 14:25:00
On Tue, 4 Mar 2008 09:50:23 -0800 (PST) alan bryan wrote: > I've got a new server with a 3ware 9550SXU with the > Battery. I am using FreeBSD 7.0-Release (tried both > 4BSD and ULE) using AMD64 and the 3ware performance > for writes is just plain horrible. Something is > obviously wrong but I'm not sure what. > I've got a 4 disk RAID 10 array. > According to 3dm2 the cache is on. I even tried > setting The StorSave preference to "Performance" with > no real benefit. There seems to be something really > wrong with disk performance. Here's the results from > bonnie: > File './Bonnie.2551', size: 104857600 > Writing with putc()...done > Rewriting...done > Writing intelligently...done > Reading with getc()...done > Reading intelligently...done > Seeker 1...Seeker 2...Seeker 3...start > 'em...done...done...done... > -------Sequential Output-------- > ---Sequential Input-- --Random-- > -Per Char- --Block--- -Rewrite-- -Per > Char- --Block--- --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec > %CPU K/sec %CPU /sec %CPU > 100 9989 4.8 6739 1.0 18900 7.8 225973 > 98.5 1914662 > 99.9 177210.7 259.7 > Any ideas? Anybody have one of these that's working > with FreeBSD 7? I had almost the same problem. The transfer rate was less that 10Mb/s for raid-10. When a BBU arrived I thought my problems gone. Not that at once. Firstly none changed. I tested many combinatins with no avail (it lasted some hours). Then it so happed that I set StorSave "Ballanced", cache "on" and marked (by an accident) the raid for checking (or verifying... can't recall how it's named at the 3WARE BIOS). The test has began, I stopped it. Rebooted the OS. To my surprise after booting the transfer rate was quite good. BTW, the test proceeded after booting... Maybe the card did not use the cache while the battery was uncharged? Don't know (but noticed "twa0: INFO: (0x04: 0x0056): Battery charging completed:" at dmesg). Here it is now with WD RE2 1T disks: ----- 3ware device driver for 9000 series storage controllers, version: 3.70.05.001 twa0: <3ware 9000 series Storage Controller> port 0xc800-0xc8ff mem 0xf8000000-0xf9ffffff,0xfeaff000-0xfeafffff irq 16 at device 0.0 on pci4 twa0: [ITHREAD] twa0: INFO: (0x04: 0x0029): Verify started: unit=0 twa0: INFO: (0x04: 0x003D): Verify paused: unit=0 twa0: INFO: (0x15: 0x1300): Controller details:: Model 9650SE-8LPML, 8 ports, Firmware FE9X 3.08.02.005, BIOS BE9X 3.08.00.002 ----- da0 at twa0 bus 0 target 0 lun 0 da0: <AMCC 9650SE-8LP DISK 3.08> Fixed Direct Access SCSI-5 device da0: 100.000MB/s transfers da0: 953632MB (1953038336 512 byte sectors: 255H 63S/T 121571C) ----- /dev/da0s1d 902G 36G 794G 4% /space ----- btest% uname -a FreeBSD btest.ipt.ru 7.0-RELEASE FreeBSD 7.0-RELEASE #14: Mon Mar 3 18:27:26 MSK 2008 rootbtest.ipt.ru:/usr/obj/usr/src/sys/BTEST i386 btest% sudo dd if=/dev/da0 of=/dev/null bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 5.073393 secs (206681412 bytes/sec) btest% sudo dd if=/dev/zero of=/space/dd.file bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 5.670188 secs (184927909 bytes/sec) ----- WBR -- Boris Samorodov (bsam) Research Engineer, http://www.ipt.ru Telephone & Internet SP FreeBSD committer, http://www.FreeBSD.org The Power To Serve _______________________________________________ freebsd-performancefreebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-p erformance To unsubscribe, send any mail to "freebsd-performance-unsubscribefreebsd.org"

Re: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance
United Kingdom	2008-03-04 15:06:33
On Tue, 04 Mar 2008 23:25:00 +0300 Boris Samorodov wrote: > Here it is now with WD RE2 1T disks: Sorry, these are just plain WD-JS 0.5T disks here. WBR -- Boris Samorodov (bsam) Research Engineer, http://www.ipt.ru Telephone & Internet SP FreeBSD committer, http://www.FreeBSD.org The Power To Serve _______________________________________________ freebsd-performancefreebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-p erformance To unsubscribe, send any mail to "freebsd-performance-unsubscribefreebsd.org"

Re: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance
United States	2008-03-05 12:09:09
--- alan bryan <alan.bryanyahoo.com> wrote: > Hi, > > I've got a new server with a 3ware 9550SXU with the > Battery. I am using FreeBSD 7.0-Release (tried both > 4BSD and ULE) using AMD64 and the 3ware performance > for writes is just plain horrible. Something is > obviously wrong but I'm not sure what. > > I've got a 4 disk RAID 10 array. > > According to 3dm2 the cache is on. I even tried > setting The StorSave preference to "Performance" > with > no real benefit. There seems to be something really > wrong with disk performance. Here's the results > from > bonnie: OK - so, I ran the server for about 24 hrs while it did its battery test. After that it automatically turned on it's write cache. So, even though initially 3dm2 was reporting the cache was on I guess it must not have been. I ran some more tests: Version 1.93d ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP server 16G 530 99 120160 23 28207 8 903 99 109688 16 518.7 18 Latency 18651us 541ms 447ms 11461us 271ms 66793us Version 1.93d ------Sequential Create------ --------Random Create-------- server -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 20737 34 +++++ +++ +++++ +++ 20429 37 +++++ +++ +++++ +++ Latency 146ms 22us 101ms 150ms 24us 31us 1.93c,1.93d,server,1,1204745494,16G,,530,99,120160,23,28207, 8,903,99,109688,16,518.7,18,16,,,,,20737,34,+++++,+++,+++++, +++,20429,37,+++++,+++,+++++,+++,18651us,541ms,447ms,11461us ,271ms,66793us,146ms,22us,101ms,150ms,24us,31us And my original test again: Writing with putc()...done Rewriting...done Writing intelligently...done Reading with getc()...done Reading intelligently...done Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done... -------Sequential Output-------- ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU 100 154436 71.4 86016 10.8 104301 13.7 226587 100.0 1934776 101.6 173626.2 261.3 >dd if=/dev/zero of=/tmp/dd.file bs=1m count=1000 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 7.814102 secs (134190213 bytes/sec) So, it's better but am I still getting what I should be seeing? ____________________________________________________________ ________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/ca tegory.php?category=shopping _______________________________________________ freebsd-performancefreebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-p erformance To unsubscribe, send any mail to "freebsd-performance-unsubscribefreebsd.org"

Re: 7.0-Release and 3ware 9550SXU w/BBU - horrible write performance
Croatia	2008-03-06 03:57:40
ALAN BRYAN WROTE: > --- ALAN BRYAN <ALAN.BRYANYAHOO.COM> WROTE: >> I'VE GOT A 4 DISK RAID 10 ARRAY. > VERSION 1.93D ------SEQUENTIAL OUTPUT------ > --SEQUENTIAL INPUT- --RANDOM- > CONCURRENCY 1 -PER CHR- --BLOCK-- -REWRITE- -PER > CHR- --BLOCK-- --SEEKS-- > MACHINE SIZE K/SEC %CP K/SEC %CP K/SEC %CP > K/SEC %CP K/SEC %CP /SEC %CP > SERVER 16G 530 99 120160 23 28207 8 903 99 > 109688 16 518.7 18 > SO, IT'S BETTER BUT AM I STILL GETTING WHAT I SHOULD > BE SEEING? IT SEEMS ABOUT RIGHT - TWICE THE PERFORMANCE OF A SINGLE DRIVE IS OK FOR A 4-DRIVE RAID10.

[1-8]

about | contact Other archives ( Real Estate discussion Medical topics )