Add basic support to read pack-*.idx v1 and v2 files

The index data is mapped into memory and then scanned using a
binary search algorithm to locate the matching entry for the
supplied git_oid.  The standard fanout hash trick is applied to
reduce the search space by 8 iterations.

Since the v1 and v2 file formats differ in their search function,
due to the different layouts used for the object records, we use
two different search implementations and a virtual function pointer
to jump to the correct version of code for the current pack index.
The single function jump per-pack should be faster then computing
a branch point inside the inner loop of a common binary search.

To improve concurrency during read operations the pack lock is only
held while verifying the index is actually open, or while opening
the index for the first time.  This permits multiple concurrent
readers to scan through the same index.

If an invalid index file is opened we close it and mark the
git_pack's invalid bit to true.  The git_pack structure is kept
around in its parent git_packlist, but the invalid bit will cause
all future readers to skip over the pack entirely.  Pruning the
invalid entries is relatively unimportant because they shouldn't
be very common, a $GIT_DIRECTORY/objects/pack directory tends to
only have valid pack files.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

1 files changed, 7 insertions, 0 deletions

diff --git a/src/util.h b/src/util.h
index 018fce57e..8f0c4b543 100644
--- a/src/util.h
+++ b/src/util.h
@@ -31,6 +31,13 @@ extern int git__fmt(char *, size_t, const char *, ...)
 extern int git__prefixcmp(const char *str, const char *prefix);
 extern int git__suffixcmp(const char *str, const char *suffix);
 
+/** @return true if p fits into the range of a size_t */
+GIT_INLINE(int) git__is_sizet(off_t p)
+{
+	size_t r = (size_t)p;
+	return p == r;
+}
+
 /*
  * Realloc the buffer pointed at by variable 'x' so that it can hold
  * at least 'nr' entries; the number of entries currently allocated