Re: Life after NSLU2?

docbillnet:
>Why not buy another NSLU2?
Because the file transfer speed is just too slow. It's OK for overnight
backups where throughput really doesn't matter, and as a learning platform
for Linux, but as a file server it just doesn't cut it. When the USB
interface runs at (theoretically) 480Mbps and the network interface is
capable of 100Mbps (and I'm thinking of upgrading to gigabit Ethernet), file
transfer speeds that top out at just a handful of Mbps is pretty
disappointing.

By the way, if anybody does want another one thingfling.com has them in its
rotation again at $60 (new, not refurb.)

regards,
Carl
--
View my personal blog at TheCzarDictates.blogspot.com

The contents of this message represent the
personal opinion of Carl Zetie and do not
reflect the opinions of his employer

[Non-text portions of this message have been removed]

Messages in this topic (4) Reply (via web post) | Start a new topic

.

On 3/28/08, Carl Zetie < carl.zetie%40gmail.com">carl.zetiegmail.com> wrote:
&gt;
> docbillnet:
> >Why not buy another NSLU2?
&gt; Because the file transfer speed is just too slow. It's OK for overnight
> backups where throughput really doesn't matter, and as a learning platform
> for Linux, but as a file server it just doesn't cut it. When the USB
> interface runs at (theoretically) 480Mbps and the network interface is
> capable of 100Mbps (and I'm thinking of upgrading to gigabit Ethernet), file
>; transfer speeds that top out at just a handful of Mbps is pretty
&gt; disappointing.

By the way, one should be careful about megabits and megabytes. The
USB interface is 480 megabits per second, which is 60 megabytes per
second. The ethernet at 100 megabits per second actually tops out at
about 10 megabytes per second.

Achieved data transfer rates using dd(1) between two USB disks seem to
top out at 10 megabytes per second. Every byte is handled twice by
the CPU, once for reading and once for writing.. If one uses rsync,
the extra checksum calculations saturate the slow CPU in the NSLU2 and
the disk to disk rate drops to 3 megabytes per second.

So yes, it isn't the worlds fastest CPU and asking it to do all this
extra stuff makes the over-all transfer rate pretty slow.

But remeber that a handful of megabytes is eight hands full of megabits.

carl
--
carl lowenstein
marine physical lab, u.c. san diego
clowenstein%40ucsd.edu">clowensteinucsd.edu

Messages in this topic (5) Reply (via web post) | Start a new topic

.

I can thoroughly recommend the Synology DS107, and I'd all there other
products are the same.

Paul B

On Sat, Mar 29, 2008 at 4:32 AM, Carl Lowenstein < clowenstein%40gmail.com">clowensteingmail.com>
wrote:

&gt; On 3/28/08, Carl Zetie < carl.zetie%40gmail.com">carl.zetiegmail.com <carl.zetie%40gmail.com>>
> wrote:
&gt; >
>; > docbillnet:
> > >Why not buy another NSLU2?
&gt; > Because the file transfer speed is just too slow. It's OK for overnight
> > backups where throughput really doesn't matter, and as a learning
> platform
> > for Linux, but as a file server it just doesn't cut it. When the USB
> > interface runs at (theoretically) 480Mbps and the network interface is
> > capable of 100Mbps (and I'm thinking of upgrading to gigabit Ethernet),
> file
>; > transfer speeds that top out at just a handful of Mbps is pretty
&gt; > disappointing.
>
> By the way, one should be careful about megabits and megabytes. The
> USB interface is 480 megabits per second, which is 60 megabytes per
> second. The ethernet at 100 megabits per second actually tops out at
> about 10 megabytes per second.
>
> Achieved data transfer rates using dd(1) between two USB disks seem to
> top out at 10 megabytes per second. Every byte is handled twice by
> the CPU, once for reading and once for writing.. If one uses rsync,
&gt; the extra checksum calculations saturate the slow CPU in the NSLU2 and
> the disk to disk rate drops to 3 megabytes per second.
>
> So yes, it isn't the worlds fastest CPU and asking it to do all this
>; extra stuff makes the over-all transfer rate pretty slow.
&gt;
> But remeber that a handful of megabytes is eight hands full of megabits.
>
&gt; carl
>; --
> carl lowenstein
> marine physical lab, u.c. san diego
&gt; clowenstein%40ucsd.edu">clowensteinucsd.edu <clowenstein%40ucsd.edu>
>
>

[Non-text portions of this message have been removed]

Messages in this topic (6) Reply (via web post) | Start a new topic

.

OK. Then it is worth doing some measurements to see what is
acceptable to you. First off, note that you can speed up rsync by
running it over nfs, rather than directly writing to the NSLU2. This
approximately doubles your speed. I just ran a quick test on my
Fedora8 desktop:

[docbillhartnell ~]$ sudo rsync --progress /tmp/1956 1 Ten
Commandments (Part 1).avi /net/192.168.2.11/share/.
1956 1 Ten Commandments (Part 1).avi
1688447022 100% 4.40MB/s 0:06:05 (xfer#1, to-check=0/1)

sent 1688653247 bytes received 42 bytes 4280489.96 bytes/sec

Then it is worth noting how much better a faster CPU will help. I can
test this by connecting to a second desktop machine in my home office.
(I have one for work, and one for personal use.)

[docbillhartnell ~]$ sudo rsync --progress /tmp/1956 1 Ten
Commandments (Part 1).avi /net/192.168.2.202/share/.
1956 1 Ten Commandments (Part 1).avi
1688447022 100% 9.58MB/s 0:02:48 (xfer#1, to-check=0/1)

sent 1688653247 bytes received 42 bytes 9355419.88 bytes/sec

It looks like you can double your speed with a device that has a
faster CPU. To gain further performance, you will need to use 1000
Mbps networking. That means your computer, your NAS device, and
router will all need to support this speed. I would expect that to
approximately double your backup speed, because at that point you will
be bottlenecked by your USB enclosure. I have yet to find a USB
enclosure that can transfer the data faster than about 20 megabytes
per second. You might be able to get another factor of two
improvement if your NAS supports Sata or eSata. However, none of he
NAS devices I have tested get those kind of speed...

How much of a speed improvement are you looking for?

2x - mount the drives with NFS.
4x - NAS with a faster CPU and NFS support.
8x - NAS with a faster CPU, NFS support, and 1000 Mbps.
more than 8x - NAS with a faster CPU, NFS support, 1000 Mbps, and
Sata/eSata

I hope this helps you make a decision on what to buy.

Bill

p.s. Don't be fooled by 802.11n wireless devices. You won't get
speeds any faster than 100Mbps.

Messages in this topic (8) Reply (via web post) | Start a new topic

.

On Sat, Mar 29, 2008 at 12:32 AM, Carl Lowenstein < clowenstein%40gmail.com">clowensteingmail.com>
wrote:

{SNIP}

Achieved data transfer rates using dd(1) between two USB disks seem to
> top out at 10 megabytes per second. Every byte is handled twice by
> the CPU, once for reading and once for writing.. If one uses rsync,
&gt; the extra checksum calculations saturate the slow CPU in the NSLU2 and
> the disk to disk rate drops to 3 megabytes per second.
>

Note that I use rsync to mirror between two EXT3 USB drives on NSLU2, and
don't use the "-c&quot; option (uses CRC checksum). If you use "rsync -va $SRC
$DST" then it would only look at the dates, mod times, sizes, etc. in
determining if files would be transferred or not. The "-c&quot; option would
first calculate the CRC of both files (or just the source, I believe, if
dest doesn't exist), and then transfer the file from SRC to DST if CRCs were
different. So, for one transfer of slightly different files (assume file
size, date, etc. were all the same), the SRC file is read, CRC check done,
the DST file is read, CRC done, then the file is transferred from SRC to
DST. Indeed, the NSLU2 is transferring one file 4 times, versus one file 2
times for the non-CRC checked version. I doubt you'll be able to get
throughput of 3MB/sec in CRC check mode (more likely <1MB/sec or worse due
to overhead). I often see ~2-3MB/sec in non-CRC check mode though
(drive-to-drive rsync). The fastest transfers are from my linux server to
the NSLU2 via NFS at about 4MB/sec.

[Non-text portions of this message have been removed]

Messages in this topic (10) Reply (via web post) | Start a new topic

.