|
List Info
Thread: Re: Fwd: Memory error
|
|
| Re: Fwd: Memory error |
 
United States |
2007-10-10 17:25:43 |
David, if you want to keep the topic on the list remember to
continue to
include vss2svn users e-mail.
40GB sounds huge to me. I hear people speak of repositories
that large, and I
don't know how it gets that way. Probably because we only
use it to do source
code. I converted a repository some months ago that
represented 5 years of
development of a team of 1-5 developers. The converted SVN
repository (1.3,
not 1.4 style) is 200MB. I considered that large. I don't
think I would envy
having to deal with a 40GB job.
I noticed that when I ran the script on Linux it ran many
times faster than on
Windows (hours to minutes). When I ran the vss2svn script on
a Linux RAM disk,
the process went from many minutes to a few minutes. The
conversion process
and the subsequent svnadmin load are extremely I/O intensive
and make lots of
small writes. I suggest you use whatever is the fastest
storage media you can.
In fact, if you have more than one HD on a server, I suggest
hosting VSS on
one and vss2svn output on the other and reversing that to
minimize the
switching of reads and writes, which kills HDs. I noticed
when I was doing
partition resizing with gparted to move file systems around
it was going to
take an entire DAY to move the partition on the drive to
another location, but
it took 15 minutes to move it from disk A to disk B then
from B back to A in a
different location. Sequential reads vs random reads make or
break this.
If you make the process fast enough, perhaps if you have to
convert the whole
thing it won't be so bad. Extra memory or disks help.
Jason
David Blaikie wrote:
> Jason, thanks a lot for your knowledgeshare. I will
try a few different
> things and see what might work. Perhaps I'll try to
run the script on
> the whole 40GB. Does 40GB sound like an inordinately
massive project?
> I've seen messages on this list saying that 10GB is
large. If we've
> been building this thing up over the last 5 years, am I
facing a Holy
> Grail - type situation?
>
> thanks again
>
> David
_______________________________________________
vss2svn-users mailing list
Project homepage:
http://www.
pumacode.org/projects/vss2svn/
Subscribe/Unsubscribe/Admin:
http://lists.pumacode.org/mailman/lis
tinfo/vss2svn-users-lists.pumacode.org
Mailing list web interface (with searchable archives):
http://dir.gmane.org/gmane.comp.version-c
ontrol.subversion.vss2svn.user
|
|
| Re: Fwd: Memory error |
|
2007-10-10 17:32:19 |
|
thanks, my last note was meant just to thank you, but then a question slipped in... ; )
I would _love_ to run the conversion on Linux, which I'm sure you're right is much faster. Is it possible then to just copy the VSS repo to a Linux box and get a Linux version of
ssphys.exe? Would ssphys have to be compiled from source? Is any part of this documented somewhere?
thanks again for your time here
David
On 10/10/07,
Jason Winnebeck < gillius-ml gillius.org">gillius-mlgillius.org> wrote:
David, if you want to keep the topic on the list remember to continue to
include vss2svn users e-mail.
40GB sounds huge to me. I hear people speak of repositories that large, and I
don't know how it gets that way. Probably because we only use it to do source
code. I converted a repository some months ago that represented 5 years of
development of a team of 1-5 developers. The converted SVN repository (1.3,
not 1.4 style) is 200MB. I considered that large. I don't think I would envy
having to deal with a 40GB job.
I noticed that when I ran the script on Linux it ran many times faster than on
Windows (hours to minutes). When I ran the vss2svn script on a Linux RAM disk,
the process went from many minutes to a few minutes. The conversion process
and the subsequent svnadmin load are extremely I/O intensive and make lots of
small writes. I suggest you use whatever is the fastest storage media you can.
In fact, if you have more than one HD on a server, I suggest hosting VSS on
one and vss2svn output on the other and reversing that to minimize the
switching of reads and writes, which kills HDs. I noticed when I was doing
partition resizing with gparted to move file systems around it was going to
take an entire DAY to move the partition on the drive to another location, but
it took 15 minutes to move it from disk A to disk B then from B back to A in a
different location. Sequential reads vs random reads make or break this.
If you make the process fast enough, perhaps if you have to convert the whole
thing it won't be so bad. Extra memory or disks help.
Jason
David Blaikie wrote:
>; Jason, thanks a lot for your knowledgeshare. I will try a few different
> things and see what might work. ; Perhaps I'll try to run the script on
> the whole 40GB. ; Does 40GB sound like an inordinately massive project?
> I've seen messages on this list saying that 10GB is large. If we've
> been building this thing up over the last 5 years, am I facing a Holy
> Grail - type situation?
>
> thanks again
>
> David
|
| Re: Fwd: Memory error |
|
2007-11-29 16:19:48 |
|
Hi again Jason, follow-up question re your suggestion to run VSS2SVN on Linux to improve speed.
I ran this idea by a colleague and he felt that, since the core functionality is compiled C, that it shouldn9;t run much faster on a Linux machine. Do you have any idea why this would have been the case for you? Was it because your Linux machine was just faster, or because you were using a RAM disk? ;
Just because of our own hardware setups, it would take a great deal more time and trouble for us to move our repository to a Linux machine and would be easier to do so on a Windows machine. I would like to get more information before going forward with the Linux idea. ; Can you or anyone else on the list offer any more thoughts on this subject?
thanks
David
On Oct 10, 2007 5:25 PM, Jason Winnebeck < gillius-ml gillius.org">gillius-mlgillius.org> wrote:
David, if you want to keep the topic on the list remember to continue to
include vss2svn users e-mail.
40GB sounds huge to me. I hear people speak of repositories that large, and I
don't know how it gets that way. Probably because we only use it to do source
code. I converted a repository some months ago that represented 5 years of
development of a team of 1-5 developers. The converted SVN repository (1.3,
not 1.4 style) is 200MB. I considered that large. I don't think I would envy
having to deal with a 40GB job.
I noticed that when I ran the script on Linux it ran many times faster than on
Windows (hours to minutes). When I ran the vss2svn script on a Linux RAM disk,
the process went from many minutes to a few minutes. The conversion process
and the subsequent svnadmin load are extremely I/O intensive and make lots of
small writes. I suggest you use whatever is the fastest storage media you can.
In fact, if you have more than one HD on a server, I suggest hosting VSS on
one and vss2svn output on the other and reversing that to minimize the
switching of reads and writes, which kills HDs. I noticed when I was doing
partition resizing with gparted to move file systems around it was going to
take an entire DAY to move the partition on the drive to another location, but
it took 15 minutes to move it from disk A to disk B then from B back to A in a
different location. Sequential reads vs random reads make or break this.
If you make the process fast enough, perhaps if you have to convert the whole
thing it won't be so bad. Extra memory or disks help.
Jason
David Blaikie wrote:
>; Jason, thanks a lot for your knowledgeshare. I will try a few different
> things and see what might work. Perhaps I'll try to run the script on
> the whole 40GB. Does 40GB sound like an inordinately massive project?
> I've seen messages on this list saying that 10GB is large. If we've
> been building this thing up over the last 5 years, am I facing a Holy
> Grail - type situation?
>
> thanks again
>
> David
|
[1-3]
|
|
|
about | contact Other archives ( Real Estate discussion Medical topics )
|