Infinite recursion in mergeparentdata

Hi,

I'm getting a crash during conversion when GetPathDepth
recurses
itself forever (Perl exhausts its address space then
segfaults). It
appears to be caused by a pair of physicalaction rows which
refer to
each other:

action_id  physname   parentphys   actiontype
---------  --------   ----------   ----------
70374      DKGAAAAA   ZKGAAAAA     MOVE_FROM
553646     ZKGAAAAA   DKGAAAAA     MOVE_FROM

This was five years ago so nobody can remember what
happened, but it
looks like projects were moved such that DKG was at one time
a parent
of ZKG and at one time a child of it. I have yet to prove
this
hypothesis with a clean repository, but I thought I'd check
with you
lot to see if you've got any ideas.

I've hacked around the problem for the moment by running
"update
physicalaction set parentdata=2 where action_id=70374"
(since that
project is the one that is currently the parent) and then
using
--resume, but I can't figure out what the correct fix should
be. Any
advice is much appreciated. I'm using the r323 .pl script
(not the
exe).

While I'm typing I'll also briefly mention a subsequent
problem: the
conversion has completed and I'm doing svnadmin load, but
getting
"svnadmin: File already exists: ...." committing
revision 2690. I need
to do more investigation of this but I think the file was
created in a
different project then the entire project moved but somehow
vss2svn
didn't notice. I'll try to find out what the actual problem
is and
then get back to you.

Sorry for the lack of patches in this e-mail,

Richard.

_______________________________________________
vss2svn-users mailing list
Project homepage:
http://www.
pumacode.org/projects/vss2svn/
Subscribe/Unsubscribe/Admin:
http://lists.pumacode.org/mailman/lis
tinfo/vss2svn-users-lists.pumacode.org
Mailing list web interface (with searchable archives):
http://dir.gmane.org/gmane.comp.version-c
ontrol.subversion.vss2svn.user

Hi, Richard,

thanks for the report. The problem is, that there is already
a fix in 
the code, to protect against multiple move merges. Do you
get any 
warning like: Multiple child recs for parent MOVE rec
70374?

The solution in r323 is not 100% perfect, since it will only
test the 
first matched record. One should at least extend the search
for all 
matched records. [1]

Dirk

[1] 
http://www.pumacode.org/projects/vs
s2svn/browser/trunk/script/vss2svn.pl#L744



Richard Hughes schrieb:
> Hi,
>
> I'm getting a crash during conversion when GetPathDepth
recurses
> itself forever (Perl exhausts its address space then
segfaults). It
> appears to be caused by a pair of physicalaction rows
which refer to
> each other:
>
> action_id  physname   parentphys   actiontype
> ---------  --------   ----------   ----------
> 70374      DKGAAAAA   ZKGAAAAA     MOVE_FROM
> 553646     ZKGAAAAA   DKGAAAAA     MOVE_FROM
>
> This was five years ago so nobody can remember what
happened, but it
> looks like projects were moved such that DKG was at one
time a parent
> of ZKG and at one time a child of it. I have yet to
prove this
> hypothesis with a clean repository, but I thought I'd
check with you
> lot to see if you've got any ideas.
>
> I've hacked around the problem for the moment by
running "update
> physicalaction set parentdata=2 where
action_id=70374" (since that
> project is the one that is currently the parent) and
then using
> --resume, but I can't figure out what the correct fix
should be. Any
> advice is much appreciated. I'm using the r323 .pl
script (not the
> exe).
>
> While I'm typing I'll also briefly mention a subsequent
problem: the
> conversion has completed and I'm doing svnadmin load,
but getting
> "svnadmin: File already exists: ...."
committing revision 2690. I need
> to do more investigation of this but I think the file
was created in a
> different project then the entire project moved but
somehow vss2svn
> didn't notice. I'll try to find out what the actual
problem is and
> then get back to you.
>
> Sorry for the lack of patches in this e-mail,
>
> Richard.
>
> _______________________________________________
> vss2svn-users mailing list
> Project homepage:
> http://www.
pumacode.org/projects/vss2svn/
> Subscribe/Unsubscribe/Admin:
> http://lists.pumacode.org/mailman/lis
tinfo/vss2svn-users-lists.pumacode.org
> Mailing list web interface (with searchable archives):
> http://dir.gmane.org/gmane.comp.version-c
ontrol.subversion.vss2svn.user
>
>
>
>   

_______________________________________________
vss2svn-users mailing list
Project homepage:
http://www.
pumacode.org/projects/vss2svn/
Subscribe/Unsubscribe/Admin:
http://lists.pumacode.org/mailman/lis
tinfo/vss2svn-users-lists.pumacode.org
Mailing list web interface (with searchable archives):
http://dir.gmane.org/gmane.comp.version-c
ontrol.subversion.vss2svn.user

This is a resurrection of a thread from 6 months ago, for
which I now
have more information. The basic background was:

>> I'm getting a crash during conversion when
GetPathDepth recurses
>> itself forever (Perl exhausts its address space
then segfaults). It
>> appears to be caused by a pair of physicalaction
rows which refer to
>> each other

I've now put some effort into this and have created the
following
(which are all at the bottom of this e-mail):

1. A minimal testcase, in the form of a batch file which
should be run
on a brand new, totally empty sourcesafe database and which
will
create something which crashes vss2svn (r336).

2. The entire contents of the PhysicalAction table resulting
from the
above database, as it exists upon entry to
MergeParentData().

3. A patch against vss2svn.pl r336 which makes the problem
go away.
I'm reluctant to use the word 'fixes' because I don't
understand
enough about what the parentdata column is used for to be at
all sure
that I've done the right thing. I have, however, confirmed
that the
resultant svn repository has the correct history for my
testcase.

I'd be very appreciative if somebody knowledgable could
review this
patch. It does still throw the warning "ERROR --
Multiple child recs
for parent MOVE rec '7' at vss2svn.pl line 740", but I
believe this to
be harmless because the timestamp check shortly thereafter
sorts it
all out correctly.

Richard.

-----8<-----8<-----8<-----8<-----8<-----8<
-----8<-----8<-----8<-----
**** make-broken-db.bat

setlocal
set SSDIR=c:vssbroken
ss create -YAdmin -I- $/first
sleep 2
ss create -YAdmin -I- $/first/second
sleep 2
ss create -YAdmin -I- $/third
sleep 2
ss move -YAdmin $/first/second $/third/second
sleep 2
ss move -YAdmin $/third/second $/first/second
sleep 2
ss move -YAdmin $/third $/first/second/third
endlocal

-----8<-----8<-----8<-----8<-----8<-----8<
-----8<-----8<-----8<-----
**** vss_data.physicalaction.txt

[I changed my mind and added this as an attachment instead
so that the
line breaks don't get mangled and because it's not important
that it
be archived.]

-----8<-----8<-----8<-----8<-----8<-----8<
-----8<-----8<-----8<-----
**** vss2svn.pl.r336.patch

Index: vss2svn.pl
============================================================
=======
--- vss2svn.pl  (revision 336)
+++ vss2svn.pl  (working copy)
 -599,10
+599,11 
     parentdata > 0
     AND physname = ?
     AND actiontype = ?
+    AND timestamp <= ?
 EOSQL

     my $sth = $gCfg->prepare($sql);
-    $sth->execute( { $row }{qw(parentphys actiontype)} );
+    $sth->execute( { $row }{qw(parentphys actiontype
timestamp)} );

     $parents =  $sth->fetchall_arrayref( {} );
     $maxParentDepth = 0;
     
_______________________________________________
vss2svn-users mailing list
Project homepage:
http://www.
pumacode.org/projects/vss2svn/
Subscribe/Unsubscribe/Admin:
http://lists.pumacode.org/mailman/lis
tinfo/vss2svn-users-lists.pumacode.org
Mailing list web interface (with searchable archives):
http://dir.gmane.org/gmane.comp.version-c
ontrol.subversion.vss2svn.user

>> I'm getting a crash during conversion when
GetPathDepth recurses
>> itself forever (Perl exhausts its address space
then segfaults). It
>> appears to be caused by a pair of physicalaction
rows which refer to
>> each other 
>
>  Thanks Richard, although I wasn't able to test the
patch since I
> no longer have vss2svn installed on this box, it looks
like a
> reasonable approach and I can't think of any cases
where your
> additional check would cause an issue. I committed in
r337.

Looks like I still haven't got it quite right. It no longer
crashes,
but there was another move of one of those directories after
the
previous testcase which doesn't get coalesced in
MergeMoveData because
their parentdata values are different.

This results in the import getting confused, converting the
first MOVE
(which was the MOVE_FROM) into a SHARE, and making the
second MOVE
(was MOVE_TO) a move into /orphaned because it doesn't have
an info
column. Things get progressively worse from there, because
the
directory structure is incorrect and so future commits fail
(in
svnadmin load).

Looking through the vss2svn history, it looks like in r313
(the patch
which overloaded parentdata to contain the depth of the
tree, not just
a boolean) the GetChildRecs function was never updated to
ignore the
non-boolean aspect and so doesn't find the MOVE_TO record to
merge
with because it's got a different nonzero parentdata.

Anyway, as before, here's a testcase and proposed patch for
your
review.

Richard.

-----8<-----8<-----8<-----8<-----8<-----8<
-----8<-----8<-----8<-----
**** make-broken-db.bat

setlocal
set SSDIR=c:vsstestdb
ss create -YAdmin -I- $/first
ss create -YAdmin -I- $/first/second
ss create -YAdmin -I- $/third
sleep 2
ss move -YAdmin $/first/second $/third/second
sleep 2
ss move -YAdmin $/third/second $/first/second
sleep 2
ss move -YAdmin $/third $/first/second/third
sleep 2
ss move -YAdmin $/first/second/third $/third
endlocal

-----8<-----8<-----8<-----8<-----8<-----8<
-----8<-----8<-----8<-----
**** vss2svn.pl.r337.patch

Index: vss2svn.pl
============================================================
=======
--- vss2svn.pl  (revision 337)
+++ vss2svn.pl  (working copy)
 -655,6
+655,7 
     # we don't get the wrong row.

     $parentdata = 0 unless defined $parentdata;
+    $parentdata = 1 if $parentdata != 0;

     my $sql = <<"EOSQL";
 SELECT
 -662,7
+663,7 
 FROM
     PhysicalAction
 WHERE
-    parentdata = ?
+    MIN(parentdata, 1) = ?
     AND physname = ?
     AND actiontype = ?
     AND author = ?

_______________________________________________
vss2svn-users mailing list
Project homepage:
http://www.
pumacode.org/projects/vss2svn/
Subscribe/Unsubscribe/Admin:
http://lists.pumacode.org/mailman/lis
tinfo/vss2svn-users-lists.pumacode.org
Mailing list web interface (with searchable archives):
http://dir.gmane.org/gmane.comp.version-c
ontrol.subversion.vss2svn.user