fetching packs and storing them as packs

Eran Tromer <git2erantromer.org> wrote:
> > Unfortunately I don't have a solution.  I tried to
come up with
> > one but didn't.  
> 
> Here's one way to do it.
> Change git-repack to follow references under
$GIT_DIR/tmp/refs/ too.
> To receive or fetch a pack:
> 1. Add references to the new heads in
>    `mktemp $GIT_DIR/tmp/refs/XXXXXX`.
> 2. Put the new .pack under $GIT_DIR/objects/pack/.
> 3. Put the new .idx under $GIT_DIR/objects/pack/.
> 4. Update the relevant heads under $GIT_DIR/refs/.
> 5. Delete the references from step 1.
> 
> This is repack-safe and never corrupts the repo. The
worst-case failure
> mode is if you die before cleaning the refs from
$GIT_DIR/tmp/refs. That
> may mean some packed objects will never be removed by
"repack -a -d"
> even if they lose all references from $GIT_DIR/refs, so
do "tmpwatch -m
> 240 $GIT_DIR/tmp/refs" to take care of that.

That was actually my (and also Sean's) solution.  Except I
would
put the temporary refs as
"$GIT_DIR/refs/ref_XXXXXX" as this is
less code to change and its consistent with how temporary
loose
objects are created.

Unfortunately it does not completely work.

What happens when the incoming pack (steps #2 and #3) takes
15
minutes to upload (slow ADSL modem, lots of objects) and the
background repack process sees those temporary refs and
starts
trying to include those objects?  It can't walk the DAG that
those
refs point at because the objects aren't in the current
repository.

From what I know of that code the pack-objects process will
fail to
find the object pointed at by the ref, rescan the packs
directory,
find no new packs, look for the object again, and abort over
the
"corruption".

OK so the repository won't get corrupted but the repack
would be
forced to abort.


Another issue I just thought about tonight is we may need a
count-packs utility that like count-objects lists the number
of active packs and their total size.  If we start hanging
onto
every pack we receive over the wire the pack directory is
going to
grow pretty fast and we'll need a way to tell us when its
time to
`repack -a -d`.

-- 
Shawn.
-
To unsubscribe from this list: send the line
"unsubscribe git" in
the body of a message to majordomovger.kernel.org
More majordomo info at  http://vge
r.kernel.org/majordomo-info.html