new version of trac-darcs plugin

Lele Gaifax wrote:
> I'm trying out latest version, and while it seems to
work great on
> trac+darcs instance, on tailor's one it fails with an
AssertionError.

I had assumed that only moves are reported with wrong
ordering by darcs
changes. Unfortunately it looks like poor ordering can be
reported for
other types of changes too.

for example, in tailor's repo, see the following patch:
HASH=20060702232916-130f5-728038e54e0a59bb3567d8aa170610c2ea
f370ff.gz
darcs changes --xml --summary --matches "hash
$HASH"

Here's the relevant portion of the output (de-xmled):

move  : vcpx/repository/git/core.py ->
vcpx/repository/git/__init__.py
modify: vcpx/repository/git/__init__.py
add   : vcpx/repository/git/core.py

Here the ordering is completely screwed up!! The right
ordering should
be 'add', 'move', and then 'modify'! Note that the
relative ordering of
the 'add' and 'modify' is wrong too.

I've updated the trac-darcs code for handling bad-ordering
to be more
generic, and it now works with the tailor repo.

the code is available from:
http://cspace.in/da
rcs/trac-darcs

the changeset can be viewed here:
http://cspac
e.in/trac-darcs/changeset/39

[sreeram;]

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

K.S.Sreeram wrote:
> I had assumed that only moves are reported with wrong
ordering by darcs
> changes. Unfortunately it looks like poor ordering can
be reported for
> other types of changes too.
> 
> for example, in tailor's repo, see the following
patch:
>
HASH=20060702232916-130f5-728038e54e0a59bb3567d8aa170610c2ea
f370ff.gz
> darcs changes --xml --summary --matches "hash
$HASH"
> 
> Here's the relevant portion of the output (de-xmled):
> 
> move  : vcpx/repository/git/core.py ->
vcpx/repository/git/__init__.py
> modify: vcpx/repository/git/__init__.py
> add   : vcpx/repository/git/core.py
> 
> Here the ordering is completely screwed up!! The right
ordering should
> be 'add', 'move', and then 'modify'! Note that
the relative ordering of
> the 'add' and 'modify' is wrong too.

Yes, I know :-( I had to do dirty tricks in tailor too, to
workaround to 
this issue. There's already an open ticket on darcs, for
some unknown 
reason it prefers starting with moves hunks in patch
external 
representation (even if internally the order is obviously
right).

> 
> I've updated the trac-darcs code for handling
bad-ordering to be more
> generic, and it now works with the tailor repo.
> 

Great, thanks, pulling 

ciao, lele.


_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Hi all,

I just thought you (Sreeram and Lele) might appreciate some
user
feedback.  I have succesfully upgraded trac.loria.fr to
trac-darcs 0.4
today.  There were some minor hiccups (for example, you have
svn
update trac), so in case it helps anybody, I have noted my
upgrade
steps on the wiki.

http://wiki.darcs.net/index.html/TracOnDarcs/Upgrad
ingTo0%2e4

Notes:

1. While I am grateful to have timeline and browse source
working
   again, I do notice that clicking on ChangeSets is very
very
   slow.  I understand that this isn't trac-darcs's fault,
but
   darcs annotate.  But I'm also wondering: is it really
neccesary to
   use darcs annotate output to display the contents of a
changeset?

2. On this page, http:/
/trac.loria.fr/projects/admindt/timeline
   Sometimes I get
   ---- 
   Repository checkins event provider (ChangesetModule)
failed:
   TracError: Unsupported version control system
"darcs"
   ----
   and sometimes it works fine.

3. On this page, http://
trac.loria.fr/projects/hylores/browser
   I get a nasty Internal Error.

4. Similarly, on this page, 
   http:
//trac.loria.fr/projects/curt/changeset/44
   I get
   UnicodeDecodeError: 'utf8' codec can't decode bytes in
position
   24-26: invalid data

I guess I'll file the last 3 as tickets on the trac site. 
Thanks for
all your work on this!

-- 
Eric Kow                     http://www.loria.fr/~kow
PGP Key ID: 08AC04F9         Merci de corriger mon français.
_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Eric Y. Kow wrote:
> Hi all,
> 
> I just thought you (Sreeram and Lele) might appreciate
some user
> feedback.  

Sure, as usual, thanks!

> I have succesfully upgraded trac.loria.fr to trac-darcs
0.4
> today.  There were some minor hiccups (for example, you
have svn
> update trac), so in case it helps anybody, I have noted
my upgrade
> steps on the wiki.
> 
> http://wiki.darcs.net/index.html/TracOnDarcs/Upgrad
ingTo0%2e4
> 
> Notes:
> 
> 1. While I am grateful to have timeline and browse
source working
>    again, I do notice that clicking on ChangeSets is
very very
>    slow.  I understand that this isn't trac-darcs's
fault, but
>    darcs annotate.  But I'm also wondering: is it
really neccesary to
>    use darcs annotate output to display the contents of
a changeset?

Eh, it's a shame that darcs does not provide a more
efficient way of 
obtaining the content of a past revision of a file. Up to
0.3, 
trac-darcs did a "darcs diff -u -p some patch | patch
pristine" to get 
there, slightly faster than the current approach of parsing
its annotate 
output, but suffering of serious other problems...

Darcs should definitely grows a "cat" command,
ala "svn cat". Given that 
its annotate command is already able to produce the whole
content, even 
if in a strange and difficult format, it shouldn't be very
difficult for 
an haskeller to implement the simpler output. I won't say
how much I'd 
like to be able to do that...

> 2. On this page, http:/
/trac.loria.fr/projects/admindt/timeline
>    Sometimes I get
>    ---- 
>    Repository checkins event provider (ChangesetModule)
failed:
>    TracError: Unsupported version control system
"darcs"
>    ----
>    and sometimes it works fine.

Uhm, I cannot really say nothing on this...

> 3. On this page, http://
trac.loria.fr/projects/hylores/browser
>    I get a nasty Internal Error.

See below.

> 4. Similarly, on this page, 
>    http:
//trac.loria.fr/projects/curt/changeset/44
>    I get
>    UnicodeDecodeError: 'utf8' codec can't decode
bytes in position
>    24-26: invalid data
> 
> I guess I'll file the last 3 as tickets on the trac
site.  Thanks for
> all your work on this!

Good! 

I have been hit by the same problems with one of my repos,
and I agree 
this is currently the bigger problem I had with the new
version: 
encodings :-( To the worst, I have not a clear idea on a
possible 
workaround.

Both "darcs changes" and "darcs
annotate" suffer the same problem, being 
darcs so encoding agnostic. It should be possible to enforce
some kind 
of policy on "metadata" encoding (author and
patchname comes to mind). I 
was able to minimize the hassle on 0.3 using (or not using)
the 
DONT_ESCAPE_8BIT env var, but that clearly is honored only
by "darcs 
changes", while "darcs annotate" emits bad
XML in any case.

I'm very open to any suggestion on how to fix the problem!

bye, lele.



_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

On Tue, Aug 08, 2006 at 10:27:59PM +0200, Lele Gaifax wrote:
> Eric Y. Kow wrote:
> >1. While I am grateful to have timeline and browse
source working
> >   again, I do notice that clicking on ChangeSets
is very very
> >   slow.  I understand that this isn't
trac-darcs's fault, but
> >   darcs annotate.  But I'm also wondering: is it
really neccesary to
> >   use darcs annotate output to display the
contents of a changeset?
> 
> Eh, it's a shame that darcs does not provide a more
efficient way of 
> obtaining the content of a past revision of a file. Up
to 0.3, 
> trac-darcs did a "darcs diff -u -p some patch |
patch pristine" to get 
> there, slightly faster than the current approach of
parsing its annotate 
> output, but suffering of serious other problems...
> 
> Darcs should definitely grows a "cat"
command, ala "svn cat". Given that 
> its annotate command is already able to produce the
whole content, even 
> if in a strange and difficult format, it shouldn't be
very difficult for 
> an haskeller to implement the simpler output. I won't
say how much I'd 
> like to be able to do that...

This is what the query supercommand (and the whole
supercommand
infrastructure, currently only used for query manifest) is
for.  A
darcs query cat command would be most welcome... and not too
hard.
I'm not sure if there's a wishlist bug on this, if not,
you could file
one.
-- 
David Roundy

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Lele Gaifax wrote:
> Eh, it's a shame that darcs does not provide a more
efficient way of
> obtaining the content of a past revision of a file. Up
to 0.3,
> trac-darcs did a "darcs diff -u -p some patch |
patch pristine" to get
> there, slightly faster than the current approach of
parsing its annotate
> output, but suffering of serious other problems...
> 
> Darcs should definitely grows a "cat"
command, ala "svn cat". Given that
> its annotate command is already able to produce the
whole content, even
> if in a strange and difficult format, it shouldn't be
very difficult for
> an haskeller to implement the simpler output. I won't
say how much I'd
> like to be able to do that...

I'm just toying with a hack for simulating 'darcs cat'...

darcs diff --diff-command "cat %2" --to-patch
$PATCH $FILE

with a bit of hacking i might be able to get this to work.
In fact
this'll solve the encodings/xml problem as well!

Unfortunately I can't do this on windows because %2 expands
to a
filename with forward slashes and the 'type' command
(equivalent of cat)
understands only backslashes! :( Perhaps i can fallback to
using
annotate on windows.

[sreeram;]

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

On Tue, Aug 08, 2006 at 16:32:07 -0400, David Roundy wrote:
> This is what the query supercommand (and the whole
supercommand
> infrastructure, currently only used for query manifest)
is for.  A
> darcs query cat command would be most welcome... and
not too hard.
> I'm not sure if there's a wishlist bug on this, if
not, you could file
> one.

There is.  All interested parties, please sign up on
http://bugs.darcs.net/
issue141

-- 
Eric Kow                     http://www.loria.fr/~kow
PGP Key ID: 08AC04F9         Merci de corriger mon français.
_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Eric Y. Kow wrote:
> Hi all,
> 
> I just thought you (Sreeram and Lele) might appreciate
some user
> feedback.  I have succesfully upgraded trac.loria.fr to
trac-darcs 0.4
> today.  There were some minor hiccups (for example, you
have svn
> update trac), so in case it helps anybody, I have noted
my upgrade
> steps on the wiki.
> 
> http://wiki.darcs.net/index.html/TracOnDarcs/Upgrad
ingTo0%2e4
> 
> Notes:
> 
> 1. While I am grateful to have timeline and browse
source working
>    again, I do notice that clicking on ChangeSets is
very very
>    slow.  I understand that this isn't trac-darcs's
fault, but
>    darcs annotate.  But I'm also wondering: is it
really neccesary to
>    use darcs annotate output to display the contents of
a changeset?

Viewing a changeset requires the contents of the files
touched by the
changeset (for displaying diffs). But 'darcs annotate' is
certainly
overkill for this! I'm looking at alternatives for this,
while we wait
for somebody to implement 'darcs query cat ..'

> 
> 2. On this page, http:/
/trac.loria.fr/projects/admindt/timeline
>    Sometimes I get
>    ---- 
>    Repository checkins event provider (ChangesetModule)
failed:
>    TracError: Unsupported version control system
"darcs"
>    ----
>    and sometimes it works fine.

This is really odd... but you might want to try upgrading
your
setuptools package. (.. or do you have some older code lying
around in
your EGG_CACHE directory?)

> 
> 3. On this page, http://
trac.loria.fr/projects/hylores/browser
>    I get a nasty Internal Error.
> 
> 4. Similarly, on this page, 
>    http:
//trac.loria.fr/projects/curt/changeset/44
>    I get
>    UnicodeDecodeError: 'utf8' codec can't decode
bytes in position
>    24-26: invalid data

These couple of errors should go away if i manage to get an
alternative
to annotate working...


[sreeram;]

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

On Wed, Aug 09, 2006 at 02:29:42 +0530, K.S.Sreeram wrote:
> Viewing a changeset requires the contents of the files
touched by the
> changeset (for displaying diffs). But 'darcs
annotate' is certainly
> overkill for this! I'm looking at alternatives for
this, while we wait
> for somebody to implement 'darcs query cat ..'

This is probably unwise on my part, but I'll have a look at
this over
the week(end).

-- 
Eric Kow                     http://www.loria.fr/~kow
PGP Key ID: 08AC04F9         Merci de corriger mon français.
_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Eric, can you try the latest trac-darcs from:
http://cspace.in/da
rcs/trac-darcs

This latest version uses a darcs diff hack to get file
contents rather
than using annotate+xml-parsing, so its MUCH faster!

I also tested the errors you reported with the repos for
admindt,hylores
and curt. They work with the latest version.

Lele, tailor trac is *finally* usable! It was just toooo
slow with the
annotate logic, and now it feels much faster!

[sreeram;]

Eric Y. Kow wrote:
> Hi all,
> 
> I just thought you (Sreeram and Lele) might appreciate
some user
> feedback.  I have succesfully upgraded trac.loria.fr to
trac-darcs 0.4
> today.  There were some minor hiccups (for example, you
have svn
> update trac), so in case it helps anybody, I have noted
my upgrade
> steps on the wiki.
> 
> http://wiki.darcs.net/index.html/TracOnDarcs/Upgrad
ingTo0%2e4
> 
> Notes:
> 
> 1. While I am grateful to have timeline and browse
source working
>    again, I do notice that clicking on ChangeSets is
very very
>    slow.  I understand that this isn't trac-darcs's
fault, but
>    darcs annotate.  But I'm also wondering: is it
really neccesary to
>    use darcs annotate output to display the contents of
a changeset?
> 
> 2. On this page, http:/
/trac.loria.fr/projects/admindt/timeline
>    Sometimes I get
>    ---- 
>    Repository checkins event provider (ChangesetModule)
failed:
>    TracError: Unsupported version control system
"darcs"
>    ----
>    and sometimes it works fine.
> 
> 3. On this page, http://
trac.loria.fr/projects/hylores/browser
>    I get a nasty Internal Error.
> 
> 4. Similarly, on this page, 
>    http:
//trac.loria.fr/projects/curt/changeset/44
>    I get
>    UnicodeDecodeError: 'utf8' codec can't decode
bytes in position
>    24-26: invalid data
> 
> I guess I'll file the last 3 as tickets on the trac
site.  Thanks for
> all your work on this!
> 
> 
> 
>
------------------------------------------------------------
------------
> 
> _______________________________________________
> darcs-users mailing list
> darcs-usersdarcs.net
> http://www.abridgegame.org/mailman/listinfo/darcs-users


_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

K.S.Sreeram wrote:
> This latest version uses a darcs diff hack to get file
contents rather
> than using annotate+xml-parsing, so its MUCH faster!

Wow, it's really fast now! Good job!

> I also tested the errors you reported with the repos
for admindt,hylores
> and curt. They work with the latest version.

Do you mean the encoding problems?

Thank you,
ciao, lele.

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Lele Gaifax wrote:
> K.S.Sreeram wrote:
>> This latest version uses a darcs diff hack to get
file contents rather
>> than using annotate+xml-parsing, so its MUCH
faster!
> 
> Wow, it's really fast now! Good job!
> 
>> I also tested the errors you reported with the
repos for admindt,hylores
>> and curt. They work with the latest version.
> 
> Do you mean the encoding problems?

Yes I meant the encoding problems that Eric faced. Since
we're directly
cat'ing from a temporary copy of the repo created by darcs
diff, we just
see the file contents as a blob of bytes. Ofcourse, darcs
might have
encoding problems for metadata like patchname, but I don't
know much
about those issues.

[sreeram;]

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

K.S.Sreeram wrote:
> Eric, can you try the latest trac-darcs from:
> http://cspace.in/da
rcs/trac-darcs
> 
> This latest version uses a darcs diff hack to get file
contents rather
> than using annotate+xml-parsing, so its MUCH faster!
> 
> I also tested the errors you reported with the repos
for admindt,hylores
> and curt. They work with the latest version.
> 
> Lele, tailor trac is *finally* usable! It was just
toooo slow with the
> annotate logic, and now it feels much faster!
> 
> [sreeram;]

Minor bugfix... the 'darcs diff' based file content
retriever was
mistakenly adding an extra newline at the end. I've fixed
it now.
http://cspace.in/da
rcs/trac-darcs

changesets: htt
p://cspace.in/trac-darcs/timeline?changeset=on

If you've used the previous version of the code, some of
the files
cached in the db might have the extra newline at the end.
Clear it with:
sqlite3 <trac-dir>/db/trac.db "delete from
darcs_cache;"

[sreeram;]

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

On Wed, Aug 09, 2006 at 04:09:21 +0530, K.S.Sreeram wrote:
> Eric, can you try the latest trac-darcs from:
> http://cspace.in/da
rcs/trac-darcs
> 
> This latest version uses a darcs diff hack to get file
contents rather
> than using annotate+xml-parsing, so its MUCH faster!

Indeed, this is much faster! Now for my main project,
I even dare to click on changesets, like
http
://trac.loria.fr/projects/geni/changeset/821

My problems with admindt, hylores and curt seem to have gone
away, as
well.

Many thanks!

-- 
Eric Kow                     http://www.loria.fr/~kow
PGP Key ID: 08AC04F9         Merci de corriger mon français.
_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Eric Y. Kow wrote:
> On Wed, Aug 09, 2006 at 04:09:21 +0530, K.S.Sreeram
wrote:
>> Eric, can you try the latest trac-darcs from:
>> http://cspace.in/da
rcs/trac-darcs
>>
>> This latest version uses a darcs diff hack to get
file contents rather
>> than using annotate+xml-parsing, so its MUCH
faster!
> 
> Indeed, this is much faster! Now for my main project,
> I even dare to click on changesets, like
> http
://trac.loria.fr/projects/geni/changeset/821
> 
> My problems with admindt, hylores and curt seem to have
gone away, as
> well.
> 
> Many thanks!

You're welcome! Its nice to see that trac-darcs is quite
useful!

Speaking of trac-darcs, I thought i'll point out a few
bugs/problems
present in the code currently, in case somebody has time to
hack on it.

1) Race condition in getting file content
(DarcsNode.get_content)
The logic used is:
  - Check if data is cached in db, if yes, load it and
return it
  - Fetch data from darcs repository
  - Insert the data into the cache
  - Return the data
As can be seen, a db error occurs when 2 or more requests
try to access
the same uncached db data. It fails because the code tries
to insert the
same data into the db twice. What is the best approach for
solving this
problem? I'm not sure how to use a mutex-like object in
this scenario.

2) Database locked error
I've seen this a couple of times myself, though it may not
be anything
to do with trac-darcs itself. This needs further
investigation.
See this thread on the trac list:
http://lists.edgewall.com/archive/trac/2006-August
/009207.html

3) Mysterious sql error inside DarcsNode.get_content
I had a mysterious sqlite db error when trying to insert
data into the
darcs cache:
OperationalError: 'SQL logic error or missing database'
After a bit of googling around, I found that setting
'conn.isolation_level = None' solved the problem, though
i'm quite sure
this is not the right solution.
See 'DarcsNode.get_content' in tracdarcs/repository.py for
more information.

[sreeram;]

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

K.S.Sreeram wrote:
> Speaking of trac-darcs, I thought i'll point out a few
bugs/problems
> present in the code currently, in case somebody has
time to hack on it.

I copied these issues in the trac-darcs trac(k)er, that
seems a better 
place to keep them.

> 
> 1) Race condition in getting file content
(DarcsNode.get_content)


http://progetti.arstecnica.it/trac%2Bdarcs/ticket/15

I think that, as soon as the plugin realizes it's missing
some data and 
needs to go loading it from the repository, it should insert
a record in 
the db marked in a particular way to say "I'm
computing this info", IF 
there isn't a similar record already. In the latter case,
the plugin 
should wait a little and retry.

How does this sound?

> 2) Database locked error


http://progetti.arstecnica.it/trac%2Bdarcs/ticket/16

> 3) Mysterious sql error inside DarcsNode.get_content


http://progetti.arstecnica.it/trac%2Bdarcs/ticket/17

Thank you,
ciao, lele.

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Hi,

On 2006/08/10, at 06:50, K.S.Sreeram wrote:
> 1) Race condition in getting file content
(DarcsNode.get_content)
> The logic used is:
>   - Check if data is cached in db, if yes, load it and
return it
>   - Fetch data from darcs repository
>   - Insert the data into the cache
>   - Return the data
> As can be seen, a db error occurs when 2 or more
requests try to  
> access
> the same uncached db data. It fails because the code
tries to  
> insert the
> same data into the db twice. What is the best approach
for solving  
> this
> problem? I'm not sure how to use a mutex-like object
in this scenario.

I would suggest one of two approaches:
  - ignore the error in the second insert;
  - catch the error in the second insert, fetch the data
from the DB,  
compare with the data you would be inserting into the DB.

Assuming that your key is really a unique mapping from db
data to a  
darcs thingy, the data you are trying to insert and the data
that won  
the race into the DB is the  same, so the first choice is
acceptable.

best regards,
--
Pedro Melo
JID: xmpp:melosimplicidade.org



_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Pedro Melo wrote:
> Assuming that your key is really a unique mapping from
db data to a
> darcs thingy, the data you are trying to insert and the
data that won
> the race into the DB is the  same, so the first choice
is acceptable.

Yup, that'll work! But we need an optimization here to
avoid multiple
requests trying to fetch the same data from the darcs repo,
because its
a pretty expensive operation.

[sreeram;]

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

Lele Gaifax wrote:
>> 1) Race condition in getting file content
(DarcsNode.get_content)
> 
> 
http://progetti.arstecnica.it/trac%2Bdarcs/ticket/15
> 
> I think that, as soon as the plugin realizes it's
missing some data and
> needs to go loading it from the repository, it should
insert a record in
> the db marked in a particular way to say "I'm
computing this info", IF
> there isn't a similar record already. In the latter
case, the plugin
> should wait a little and retry.
> 
> How does this sound?

Sounds good... but what about the 'IF there isn't a
similar record
already..' bit. I guess we should just insert the record
and then see if
it fails, rather than explicitly checking if the record
exists. But then
again, you probably meant this! 

[sreeram;]

ps: I registered on the trac+darcs site, but am not able to
add comments
to the tickets. Probably some permissions issue.

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users