fetching packs and storing them as packs

Sean <seanlkmlsympatico.ca> writes:

> On Thu, 26 Oct 2006 21:42:29 -0400
> Shawn Pearce <spearcespearce.org> wrote:
>
>> This is an issue for "central"
repositories that people push into
>> and which might be getting repacked according to a
cronjob.
>> 
>> Unfortunately I don't have a solution.  I tried to
come up with
>> one but didn't.  
>
> What about creating a temporary ref before pushing, and
then removing
> it only after the HEAD has been updated?

That won't work.  If repack is faster than index-pack,
repack
would fail to find necessary objects, barf, and would not
remove
the existing or new pack, and then index-pack would
eventually
succeed and when it does at least your repository is
complete
even though it may still have redundant objects in packs.

So in that sense, it is not a disaster, so it might be a
good
enough solution.

I'd almost say "heavy repository-wide operations like
'repack -a
-d' and 'prune' should operate under a single repository
lock",
but historically we've avoided locks and instead tried to do
things optimistically and used compare-and-swap to detect
conflicts, so maybe that avenue might be worth pursuing.

How about (I'm thinking aloud and I'm sure there will be
holes -- I won't think about prune for now)...

* "repack -a -d":

 (1) initially run show-ref (or "ls-remote .") and
store the
     result in .git/$ref_pack_lock_file;

 (2) enumerate existing packs;

 (3) do the usual "rev-list --all | pack-objects"
thing; this
     may end up including more objects than what are
reachable
     from the result of (1) if somebody else updates refs in
the
     meantime;

 (4) enumerate existing packs; if there is difference from
(2)
     other than what (3) created, that means somebody else
added
     a pack in the meantime; stop and do not do the
"-d" part;

 (5) run "ls-remote ." again and compare it with
what it got in
     (1); if different, somebody else updated a ref in the
     meantime; stop and do not do the "-d" part;

 (6) do the "-d" part as usual by removing packs
we saw in (2)
     but do not remove the pack we created in (3);

 (7) remove .git/$ref_pack_lock_file.

* "fetch --thin" and "index-pack
--stdin":

 (1) check the .git/$ref_pack_lock_file, and refuse to
operate
    if there is such (this is not strictly needed for
    correctness but only to give an early exit);

 (2) create a new pack under a temporary name, and when
     complete, make the pack/index pair .pack and .idx;

 (3) update the refs.


-
To unsubscribe from this list: send the line
"unsubscribe git" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html