summaryrefslogtreecommitdiff
path: root/src/pack.h
Commit message (Collapse)AuthorAgeFilesLines
* Consolidate packfile allocation furtherRussell Belfer2013-04-221-1/+2
| | | | | | | Rename git_packfile_check to git_packfile_alloc since it is now being used more in that capacity. Fix the various places that use it. Consolidate some repeated code in odb_pack.c related to the allocation of a new pack_backend.
* Further threading fixesRussell Belfer2013-04-221-1/+1
| | | | | | | | | | | | | This builds on the earlier thread safety work to make it so that setting the odb, index, refdb, or config for a repository is done in a threadsafe manner with minimized locking time. This is done by adding a lock to the repository object and using it to guard the assignment of the above listed pointers. The lock is only held to assign the pointer value. This also contains some minor fixes to the other work with pack files to reduce the time that locks are being held to and fix an apparently memory leak.
* Add mutex around mapping and unmapping pack filesRussell Belfer2013-04-221-0/+1
| | | | | | | | | | | When I was writing threading tests for the new cache, the main error I kept running into was a pack file having it's content unmapped underneath the running thread. This adds a lock around the routines that map and unmap the pack data so that threads can effectively reload the data when they need it. This also required reworking the error handling paths in a couple places in the code which I tried to make consistent.
* indexer: use a hashtable for keeping track of offsetsCarlos Martín Nieto2013-03-031-1/+3
| | | | | | | | | | These offsets are needed for REF_DELTA objects, which encode which object they use as a base, but not where it lies in the packfile, so we need a list. These objects are mostly from older packfiles, before OFS_DELTA was widely spread. The time spent in indexing these packfiles is greatly reduced, though remains above what git is able to do.
* indexer: properly free the packfile resourcesCarlos Martín Nieto2013-01-121-1/+1
| | | | | | | | The indexer needs to call the packfile's free function so it takes care of freeing the caches. We still need to close the mwf descriptor manually so we can rename the packfile into its final name on Windows.
* Revert "pack: packfile_free -> git_packfile_free and use it in the indexers"Carlos Martín Nieto2013-01-111-1/+1
| | | | | | This reverts commit f289f886cb81bb570bed747053d5ebf8aba6bef7, which makes the tests fail on Windows. Revert until we can figure out a solution.
* pack: That declarationVicent Marti2013-01-111-1/+1
|
* pack: limit the amount of memory the base delta cache can useCarlos Martín Nieto2013-01-111-2/+4
| | | | | Currently limited to 16MB (like git) and to objects up to 1MB in size.
* pack: abstract out the cache into its own functionsCarlos Martín Nieto2013-01-111-2/+11
|
* pack: refcount entries and add a mutex around cache accessCarlos Martín Nieto2013-01-111-1/+2
|
* pack: introduce a delta base cacheCarlos Martín Nieto2013-01-111-0/+11
| | | | | | | | Many delta bases are re-used. Cache them to avoid inflating the same data repeatedly. This version doesn't limit the amount of entries to store, so it can end up using a considerable amound of memory.
* update copyrightsEdward Thomson2013-01-081-1/+1
|
* Merge pull request #1091 from carlosmn/stream-objectVicent Martí2012-12-071-0/+14
|\ | | | | Indexer speedup with large objects
| * pack: introduce a streaming API for raw objectsCarlos Martín Nieto2012-11-301-0/+14
| | | | | | | | | | This allows us to take objects from the packfile as a stream instead of having to keep it all in memory.
* | pack: add git_packfile_resolve_headerDavid Michael Barr2012-12-031-0/+6
|/ | | | | | | | | | | | | | | | | | To paraphrase @peff: You can get both size and type from a packed object reasonably cheaply. If you have: * An object that is not a delta; both type and size are available in the packfile header. * An object that is a delta. The packfile type will be OBJ_*_DELTA, and you have to resolve back to the base to find the real type. That means potentially a lot of packfile index lookups, but each one is relatively cheap. For the size, you inflate the first few bytes of the delta, whose header will tell you the resulting size of applying the delta to the base. For simplicity, we just decompress the whole delta for now.
* Make git_odb_foreach_cb take const paramRussell Belfer2012-11-271-1/+1
| | | | | This makes the first OID param of the ODB callback a const pointer and also propogates that change all the way to the backends.
* pack: iterate objects in offset orderDavid Michael Barr2012-09-141-0/+1
| | | | Compute the ordering on demand and persist until the index is freed.
* portability: Improve x86/amd64 compatibilitynulltoken2012-07-241-1/+1
|
* odb: add git_odb_foreach()Carlos Martín Nieto2012-07-031-0/+4
| | | | | Go through each backend and list every objects that exists in them. This allows fsck-like uses.
* Add packfile_unpack_compressed() to the internal headerCarlos Martín Nieto2012-04-131-0/+7
|
* Migrate ODB files to new error handlingRussell Belfer2012-03-121-6/+6
| | | | | | | | | | | | | | This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to the new style of error handling. Also got the unix and win32 versions of map.c. There are some minor changes to other files but no others were completely converted. This also contains an update to filebuf so that a zeroed out filebuf will not think that the fd (== 0) is actually open (and inadvertently call close() on fd 0 if cleaned up). Lastly, this was built and tested on win32 and contains a bunch of fixes for the win32 build which was pretty broken.
* Update Copyright headerschu2012-02-131-1/+1
| | | | Signed-off-by: schu <schu-github@schulog.org>
* *: correct and codify various file permissionsBrodie Rao2011-10-141-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following files now have 0444 permissions: - loose objects - pack indexes - pack files - packs downloaded by fetch - packs downloaded by the HTTP transport And the following files now have 0666 permissions: - config files - repository indexes - reflogs - refs This brings libgit2 more in line with Git. Note that git_filebuf_commit() and git_filebuf_commit_at() have both gained a new mode parameter. The latter change fixes an important issue where filebufs created with GIT_FILEBUF_TEMPORARY received 0600 permissions (due to mkstemp(3) usage). Now we chmod() the file before renaming it into place. Tests have been added to confirm that new commit, tag, and tree objects are created with the right permissions. I don't have access to Windows, so for now I've guarded the tests with "#ifndef GIT_WIN32".
* Tabify everythingVicent Marti2011-09-191-1/+1
| | | | | | There were quite a few places were spaces were being used instead of tabs. Try to catch them all. This should hopefully not break anything. Except for `git blame`. Oh well.
* Cleanup legal dataVicent Marti2011-09-191-21/+3
| | | | | | | | | | 1. The license header is technically not valid if it doesn't have a copyright signature. 2. The COPYING file has been updated with the different licenses used in the project. 3. The full GPLv2 header in each file annoys me.
* Implement cooperative cachingCarlos Martín Nieto2011-08-181-1/+2
| | | | | | | | | When indexing a file with ref deltas, a temporary cache for the offsets has to be built, as we don't have an index file yet. If the user takes the responsiblity for filling the cache, the packing code will look there first when it finds a ref delta. Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
* Modify the given offset in git_packfile_unpackCarlos Martín Nieto2011-08-021-1/+1
| | | | | | | | The callers immediately throw away the offset, so we don't need any logical changes in any of them. This will be useful for the indexer, as it does need to know where the compressed data ends. Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
* Move pack functions to their own fileCarlos Martín Nieto2011-08-021-19/+15
|
* Make packfile_unpack_header more genericCarlos Martín Nieto2011-08-021-2/+26
| | | | | | On the way, store the fd and the size in the mwindow file. Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
* Move the pack structs to an internal headerCarlos Martín Nieto2011-08-021-0/+94
View Lorry Controller Status