OT: rsync issues

Hi,

I think that this is a little off topic, but I know that
someone on this
list probably has an answer for me. I googled, but I still
can't seem to
find an example of how to do this.

I just updated rsync on both of my servers and deleted all
the files off
the backup. I was then able to backup the main server
without any
problems, but when I went to run rsync again, it gives me an
out of memory
error. What I would like to do is write a shell script that
would call
rsync, but only have it backup a small part of the files at
a time until
all of the files are backed up. I have not been able to find
an example of
this. Does anyone have any ideas on how this could be done?

Thanks,
Al

On Fri, 13 Apr 2007, DataZap wrote:

> Hi,
>
> I think that this is a little off topic, but I know
that someone on this
> list probably has an answer for me. I googled, but I
still can't seem to
> find an example of how to do this.
>
> I just updated rsync on both of my servers and deleted
all the files off
> the backup. I was then able to backup the main server
without any
> problems, but when I went to run rsync again, it gives
me an out of memory
> error. What I would like to do is write a shell script
that would call
> rsync, but only have it backup a small part of the
files at a time until
> all of the files are backed up. I have not been able to
find an example of
> this. Does anyone have any ideas on how this could be
done?

 	You might have luck with typing 'unlimit' before running
the
 	rsync, though if its the rsync process at the remote end
that
 	is running out of memory that will be more fun. If you
 	are using a sh/ksh then you'll probably need
 	'ulimit -m $(ulimit -Hm)' to max out the memory limit.

-- 
 		David/absolute       -- www.NetBSD.org: No hype required
--

> I just updated rsync on both of my servers and deleted
all the files off
> the backup. I was then able to backup the main server
without any
> problems, but when I went to run rsync again, it gives
me an out of memory
> error. What I would like to do is write a shell script
that would call
> rsync, but only have it backup a small part of the
files at a time until
> all of the files are backed up. I have not been able to
find an example of
> this. Does anyone have any ideas on how this could be
done?

Hi Al,

Instead of writing a script to figure out what needs to be
copied, try 
splitting it up into a few different rsync jobs for
different parts of the 
directory hierarchy.

Do find a few times piped through "wc -l" to get a
count of the number of 
files in each of your different places.

If you really want to script it, use find of -type d
(directory) with the 
-mmin primary to build a list and then maybe use just that
(but be careful 
to get the target correct so test rsync a lot with
--dry-run).

  Jeremy C. Reed

Also sprach DataZap (azdatazap.net)
> Hi,
> 
> I think that this is a little off topic, but I know
that someone on this
> list probably has an answer for me. I googled, but I
still can't seem to
> find an example of how to do this.
> 
> I just updated rsync on both of my servers and deleted
all the files off
> the backup. I was then able to backup the main server
without any
> problems, but when I went to run rsync again, it gives
me an out of memory
> error. What I would like to do is write a shell script
that would call
> rsync, but only have it backup a small part of the
files at a time until
> all of the files are backed up. I have not been able to
find an example of
> this. Does anyone have any ideas on how this could be
done?

Rsync supports excellent include/exclude options with
RegExes. You
just have to generate those lists to copy the data piecewise
to the
destination. 

You could use find(1) and wc to generate lists that work.

-- 
PGP FPR: CF74 D5F2 4871 3E5C FFFE  0130 11F4 C41E B3FB AE33
http://www.net-tex.de   
                             
http://www.cryptomancer.de

-- 
Jeder wird als Faust geboren, um alles zu erfassen, alles zu
erproben, alles 
auszudrücken. Daß Faust Gelehrter wurde, dafür sorgten die
Fehler seiner 
Vorgänger und seiner Zeitgenossen. 
Boris Pasternak, Doktor Schiwago