Pushing to an FTP location

For a while now I've been searching for a way to let darcs
push to an
FTP server (sometimes people just don't have other
alternatives). The
wiki says it's currently impossible. I couldn't find any
useful
information online. I recently found NetDrive (software that
transparently maps an FTP server to a file system drive).
Darcs +
NetDrive = Nirvana. However, NetDrive is buggy and isn't
portable. It
would be very nice if darcs provided a native solution.

Is there anything that prevents darcs from doing this? Is
this feature
planned? If not, would it be hard to implement for someone
who has
never looked at darcs codebase (but isn't a complete
Haskell newbie)?
This feature would *significantly* ease my life and I'd
like to know
whether it's planned and if not how much effort would it
take for me
to implement it.

Thanks!

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

On Sat, Jul 29, 2006 at 03:43:03 -0400, Vyacheslav Akhmechet
wrote:
> If not, would it be hard to implement for someone who
has
> never looked at darcs codebase (but isn't a complete
Haskell newbie)?

This is a task I might shy away from if I were new to
darcs-hacking.

> This feature would *significantly* ease my life and
I'd like to know
> whether it's planned and if not how much effort would
it take for me
> to implement it.

Push works by creating a patch bundle, and then running
darcs apply in
the target repository using that patch bundle (quoting the
manual).  In
order to get push-by-FTP to work, you'd have to solve the
more general
problem of doing darcs apply on a remote location.  I don't
know how I
would do that.

Perhaps a rough sketch of an idea would be based on having a
local
mirror of the remote repository.  A remote apply would then
go a little like this:

1) Lock the remote repository

2) Update the mirror (pull)

3) Do darcs apply, but at the same time, log all the files
that
   are modified by darcs, even/especially stuff in _darcs.  
   It might help to distinguish between deleted and
added/updated
   files.  Note that mv would probably have to be recorded
as
   remove-and-add.

4) Using the log, update all the relevant files on the
remote
   end.  In your case, this would consist of asking ftp to
   delete and upload stuff.

5) Unlock the remote repository

Anybody have any other (hopefully simpler, more elegant)
ideas?

-- 
Eric Kow                     http://www.loria.fr/~kow
PGP Key ID: 08AC04F9         Merci de corriger mon français.
_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

On Sat, 2006-07-29 at 11:18 +0200, Eric Y. Kow wrote:

> Perhaps a rough sketch of an idea would be based on
having a local
> mirror of the remote repository.  A remote apply would
then
> go a little like this:
> 
> 1) Lock the remote repository
> 
> 2) Update the mirror (pull)
> 
> 3) Do darcs apply, but at the same time, log all the
files that
>    are modified by darcs, even/especially stuff in
_darcs.  
>    It might help to distinguish between deleted and
added/updated
>    files.  Note that mv would probably have to be
recorded as
>    remove-and-add.
> 
> 4) Using the log, update all the relevant files on the
remote
>    end.  In your case, this would consist of asking ftp
to
>    delete and upload stuff.
> 
> 5) Unlock the remote repository
> 
> Anybody have any other (hopefully simpler, more
elegant) ideas?

I have a (small) remote repository at http://vax64.dk/bælg/ and my
hoster only allows writes via ftp (with passwords in the
clear, off
course).  The above is more or less how I push to that
repository.

I tried something similar to what Vyacheslaw did at first,
only with
FUSE and curlftpfs (I'm running Linux whereas Vyacheslaw is
probably
running Windows).  It was buggy /and/ it didn't understand
file locking.
Every time darcs tried to lock a file it would return an
error.

So now I use the attached script instead and it works
perfectly.  The
script only runs under Unix/Linux (or within cygwin) but
something
similar could easily be written for ordinary windows
environments.  I
use one of gnomevfs utilities for the copying, I bet there
are similar
kioslave utilities one could use instead.  And then there is
lftp (a
command-line FTP client) which would also work fine.

My current script pretty much does what you suggest above,
except:
1) there is no locking
2) everything gets copied down
3) there is no log
4) everything gets copied back (there is no log)
5) there is no unlocking

As long as my repository is small it is a fine solution.

When my repository gets bigger I'll look at optimizing it a
bit.  I'm
not going to keep a log, since it is easy to compare two
directories to
see which files are new/changed.  Sticking to one directory
and
comparing timestamps is also easy.

I could also keep the "mirror" around between
invocations of the script
and just assume that nobody else changes it.  Or I could use
FTP to list
the files in _darcs/* and see if they are any new files or
any with a
changed size or timestamp.  I might even use an HTTP
connection and HEAD
requests to help (using the ETag) but that is probably
overkill.

All in all, I think small, incremental refinements of the
darcspush
script could get me a long way.

-Peter
_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

I don't understand how remote 'darcs apply' is different
from a local
one. Under the hood 'apply' must use simple filesystem
operations to
create, delete, and modify files. I don't see why this
can't be
abstracted so that a set of actions that perform these tasks
on a
remote ftp client are called instead of a set of functions
that
perform these actions on a local file system.

Using darcs with NetDrive works beautifully. I want to get
away from
NetDrive because it's not free, I can't easily buy it as
it comes as a
part of a larget corporate lisence, and I can't use it on
OS X.
Effectively NetDrive achieves the abstraction I am talking
about
*outside* of 'darcs apply', it makes standard file system
actions work
on a remote FTP drive. I don't see why doing this within
darcs
codebase is difficult, all I have to do is call a different
set of
actions. I'm going to grab the code when I get a chance and
give it a
try. I'll send my progress to this list.

On 7/29/06, Peter Lund <fireflydiku.dk> wrote:
> On Sat, 2006-07-29 at 11:18 +0200, Eric Y. Kow wrote:
>
> > Perhaps a rough sketch of an idea would be based
on having a local
> > mirror of the remote repository.  A remote apply
would then
> > go a little like this:
> >
> > 1) Lock the remote repository
> >
> > 2) Update the mirror (pull)
> >
> > 3) Do darcs apply, but at the same time, log all
the files that
> >    are modified by darcs, even/especially stuff in
_darcs.
> >    It might help to distinguish between deleted
and added/updated
> >    files.  Note that mv would probably have to be
recorded as
> >    remove-and-add.
> >
> > 4) Using the log, update all the relevant files on
the remote
> >    end.  In your case, this would consist of
asking ftp to
> >    delete and upload stuff.
> >
> > 5) Unlock the remote repository
> >
> > Anybody have any other (hopefully simpler, more
elegant) ideas?
>
> I have a (small) remote repository at http://vax64.dk/bælg/ and my
> hoster only allows writes via ftp (with passwords in
the clear, off
> course).  The above is more or less how I push to that
repository.
>
> I tried something similar to what Vyacheslaw did at
first, only with
> FUSE and curlftpfs (I'm running Linux whereas
Vyacheslaw is probably
> running Windows).  It was buggy /and/ it didn't
understand file locking.
> Every time darcs tried to lock a file it would return
an error.
>
> So now I use the attached script instead and it works
perfectly.  The
> script only runs under Unix/Linux (or within cygwin)
but something
> similar could easily be written for ordinary windows
environments.  I
> use one of gnomevfs utilities for the copying, I bet
there are similar
> kioslave utilities one could use instead.  And then
there is lftp (a
> command-line FTP client) which would also work fine.
>
> My current script pretty much does what you suggest
above, except:
> 1) there is no locking
> 2) everything gets copied down
> 3) there is no log
> 4) everything gets copied back (there is no log)
> 5) there is no unlocking
>
> As long as my repository is small it is a fine
solution.
>
> When my repository gets bigger I'll look at optimizing
it a bit.  I'm
> not going to keep a log, since it is easy to compare
two directories to
> see which files are new/changed.  Sticking to one
directory and
> comparing timestamps is also easy.
>
> I could also keep the "mirror" around
between invocations of the script
> and just assume that nobody else changes it.  Or I
could use FTP to list
> the files in _darcs/* and see if they are any new files
or any with a
> changed size or timestamp.  I might even use an HTTP
connection and HEAD
> requests to help (using the ETag) but that is probably
overkill.
>
> All in all, I think small, incremental refinements of
the darcspush
> script could get me a long way.
>
> -Peter
>
>
> _______________________________________________
> darcs-users mailing list
> darcs-usersdarcs.net
> http://www.abridgegame.org/mailman/listinfo/darcs-users
>
>
>
>

_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users

On Sat, Jul 29, 2006 at 15:11:21 -0400, Vyacheslav Akhmechet
wrote:
> I don't see why doing this within darcs codebase is
difficult, all I
> have to do is call a different set of actions.  I'm
going to grab the
> code when I get a chance and give it a try. I'll send
my progress to
> this list.

Splendid!  You might want to sign up to darcs-devel and
consider hanging
out in #darcs on irc.freedone.net.

The MissingH library implements the FTP protocol.  Maybe it
could come
in handy:
http://quux.org:70/devel/missingh/html/Mi
ssingH-Network-FTP-Client.html

-- 
Eric Kow                     http://www.loria.fr/~kow
PGP Key ID: 08AC04F9         Merci de corriger mon français.
_______________________________________________
darcs-users mailing list
darcs-usersdarcs.net
http://www.abridgegame.org/mailman/listinfo/darcs-users