svn commit: r1994 - branches/graph-based: . cvs2svn_lib

Oswald Buddenhagen wrote:
> On Sun, May 07, 2006 at 01:26:24PM -0000, mhaggertigris.org wrote:
> 
>>Write CVS_REVS_DB in CollectRevsPass; modify it in
CreateDatabasePass.
>>
>>This might cause problems with the pass-by-pass
resumability of a
>>conversion.
> 
> it's not only that. experimenting with the code when
testing it on a big
> repo can easily turn into a very time-consuming
procedure (pass 1 is the
> second slowest one).

These changes are only on my little experimental branch.  I
made these
changes primarily to make experimentation easier.  I would
like to defer
thoughts about optimization until later.

But anyway, in my (1.5 minute) tests, CollectRevsPass is
more than an
order of magnitude faster than OutputPass.  Is the situation
different
for huge repositories?  If not, then I think we can afford a
little bit
of slowdown in CollectRevsPass, no?

Michael

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribecvs2svn.tigris.org
For additional commands, e-mail: dev-helpcvs2svn.tigris.org

On Sun, May 07, 2006 at 11:04:07PM +0200, Michael Haggerty
wrote:
> These changes are only on my little experimental
branch.  I made these
> changes primarily to make experimentation easier.  I
would like to defer
> thoughts about optimization until later.
> 
> But anyway, in my (1.5 minute) tests, CollectRevsPass
is more than an
> order of magnitude faster than OutputPass.  Is the
situation different
> for huge repositories?  If not, then I think we can
afford a little bit
> of slowdown in CollectRevsPass, no?

Well, here's some slightly old numbers (a few weeks):

pass 1:   218 seconds
pass 2:    30 seconds
pass 3:     1 second
pass 4:   116 seconds
pass 5:   237 seconds
pass 6:    17 seconds
pass 7:    12 seconds
pass 8: 14226 seconds
total:  14860 seconds

I'd expect that to be about typical.  Of course, the KDE
patches
probably make this profile look rather different, with lower
overhead for creating plaintexts.  But I wouldn't expect
the overall
shape to change.

Doesn't modifying entries in a BDB file have fairly bad
space behavior?
Obviously it's the way to go if you back this with a
"real" database
engine, e.g. SQL.

-- 
Daniel Jacobowitz
CodeSourcery

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribecvs2svn.tigris.org
For additional commands, e-mail: dev-helpcvs2svn.tigris.org

On Sun, May 07, 2006 at 05:20:06PM -0400, Daniel Jacobowitz
wrote:
> On Sun, May 07, 2006 at 11:04:07PM +0200, Michael
Haggerty wrote:
> > These changes are only on my little experimental
branch.  I made these
> > changes primarily to make experimentation easier. 
I would like to defer
> > thoughts about optimization until later.
> > 
i didn't mean optimization (rather the opposite). passes
should be
repeatable, so one doesn't have to start from scratch if
one screws up.

> > But anyway, in my (1.5 minute) tests,
CollectRevsPass is more than an
> > order of magnitude faster than OutputPass.  Is the
situation different
> > for huge repositories?  If not, then I think we
can afford a little bit
> > of slowdown in CollectRevsPass, no?
> 
> Well, here's some slightly old numbers (a few weeks):
> 
> pass 1:   218 seconds
> pass 2:    30 seconds
> pass 3:     1 second
> pass 4:   116 seconds
> pass 5:   237 seconds
> pass 6:    17 seconds
> pass 7:    12 seconds
> pass 8: 14226 seconds
> total:  14860 seconds
> 
> I'd expect that to be about typical.  Of course, the
KDE patches
> probably make this profile look rather different, with
lower
> overhead for creating plaintexts.  But I wouldn't
expect the overall
> shape to change.
> 
err, no. due to pass1 having to store the inverted patches,
it becomes
significantly slower (a few times as long, iirc). and pass8
is halfed,
of course. the order of magnitude then actually fits, but
wasting half
an hour for repeating pass1 because pass4 screwed up still
sucks.

> Doesn't modifying entries in a BDB file have fairly
bad space behavior?
>
i have no idea ... bdb generally has pretty bad space
behavior (25%
overhead, iirc). i definitely found storing tiny files in
reiserfs more
efficient than bdb - and it scales far better.

-- 
Hi! I'm a .signature virus! Copy me into your ~/.signature,
please!
--
Chaos, panic, and disorder - my work here is done.

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribecvs2svn.tigris.org
For additional commands, e-mail: dev-helpcvs2svn.tigris.org