List Info

Thread: large file replication




large file replication
user name
2006-10-12 14:14:01
I'm currently replicating multi-gig files across our WAN via
scp and am 
in need of another tool.

When the WAN flaps, the file transfer fails and has to start
over from 
the beginning of the file..  This wouldn't be so bad if I
just had one 
large file, but I have 50.  If one fails, then I'm just that
much 
further from completing my goal. 

Does anyone know of a tool that would allow me to transmit
the files 
across the WAN, and pick back off where I left off after a
WAN flap?  If 
I've transmitted 9 gigs of a 10 gig file, I do not want to
have to 
resend the original 9.    
Thank you,

Michael
large file replication
user name
2006-10-12 15:34:44
The wget tool (net/wget package) is able to resume broken
downloads, so
you might want to use it with any protocol it supports, for
example HTTP
or FTP, with SSL/TLS support in case encryption is needed.

Regards,
Lubo

On Št, 2006-10-12 at 10:14 -0400, Michael Gorsuch wrote:
> I'm currently replicating multi-gig files across our
WAN via scp and am 
> in need of another tool.
> 
> When the WAN flaps, the file transfer fails and has to
start over from 
> the beginning of the file..  This wouldn't be so bad if
I just had one 
> large file, but I have 50.  If one fails, then I'm just
that much 
> further from completing my goal. 
> 
> Does anyone know of a tool that would allow me to
transmit the files 
> across the WAN, and pick back off where I left off
after a WAN flap?  If 
> I've transmitted 9 gigs of a 10 gig file, I do not want
to have to 
> resend the original 9.    
> Thank you,
> 
> Michael
-- 
Lubomir Kundrak (Red Hat Security Response Team)


large file replication
user name
2006-10-12 14:33:37
Michael Gorsuch wrote:
> I'm currently replicating multi-gig files across our
WAN via scp and am 
> in need of another tool.

> Does anyone know of a tool that would allow me to
transmit the files 
> across the WAN, and pick back off where I left off
after a WAN flap?  If 
> I've transmitted 9 gigs of a 10 gig file, I do not want
to have to 
> resend the original 9.    Thank you,

try rsync, pkgsrc is your friend here, look under net/rsync.
worked very 
well when I had to transfer 1TB of multi GB files to new
RAID array.

Simon
[1-3]

about | contact  Other archives ( Real Estate discussion Medical topics )