summaryrefslogtreecommitdiff
path: root/src/odb_pack.c
Commit message (Collapse)AuthorAgeFilesLines
* Share packs across repository instancescmn/global-mwfCarlos Martín Nieto2014-06-231-3/+6
| | | | | | | | | | | Opening the same repository multiple times will currently open the same file multiple times, as well as map the same region of the file multiple times. This is not necessary, as the packfile data is immutable. Instead of opening and closing packfiles directly, introduce an indirection and allocate packfiles globally. This does mean locking on each packfile open, but we already use this lock for the global mwindow list so it doesn't introduce a new contention point.
* Don't redefine the same callback types, their signatures may changeJacques Germishuys2014-04-211-1/+1
|
* Add exists_prefix to ODB backend and ODB APIRussell Belfer2014-03-041-0/+18
|
* Remove converting user error to GIT_EUSERRussell Belfer2013-12-111-22/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changes the behavior of callbacks so that the callback error code is not converted into GIT_EUSER and instead we propagate the return value through to the caller. Instead of using the giterr_capture and giterr_restore functions, we now rely on all functions to pass back the return value from a callback. To avoid having a return value with no error message, the user can call the public giterr_set_str or some such function to set an error message. There is a new helper 'giterr_set_callback' that functions can invoke after making a callback which ensures that some error message was set in case the callback did not set one. In places where the sign of the callback return value is meaningful (e.g. positive to skip, negative to abort), only the negative values are returned back to the caller, obviously, since the other values allow for continuing the loop. The hardest parts of this were in the checkout code where positive return values were overloaded as meaningful values for checkout. I fixed this by adding an output parameter to many of the internal checkout functions and removing the overload. This added some code, but it is probably a better implementation. There is some funkiness in the network code where user provided callbacks could be returning a positive or a negative value and we want to rely on that to cancel the loop. There are still a couple places where an user error might get turned into GIT_EUSER there, I think, though none exercised by the tests.
* Improve GIT_EUSER handlingRussell Belfer2013-12-111-21/+37
| | | | | | | | | | | This adds giterr_user_cancel to return GIT_EUSER and clear any error message that is sitting around. As a result of using that in places, we need to be more thorough with capturing errors that happen inside a callback when used internally. To help with that, this also adds giterr_capture and giterr_restore so that when we internally use a foreach-type function that clears errors and converts them to GIT_EUSER, it is easier to restore not just the return value, but the actual error message text.
* Allow callers to set mode on packfile creationEdward Thomson2013-11-071-1/+1
|
* indexer: remove the stream infixCarlos Martín Nieto2013-10-301-7/+7
| | | | | | | | | | | It was there to keep it apart from the one which read in from a file on disk. This other indexer does not exist anymore, so there is no need for anything other than git_indexer to refer to it. While here, rename _add() function to _append() and _finalize() to _commit(). The former change is cosmetic, while the latter avoids talking about "finalizing", which OO languages use to mean something completely different.
* Merge pull request #1891 from libgit2/cmn/fix-thin-packsVicent Martí2013-10-281-1/+2
|\ | | | | Add support for thin packs
| * indexer: fix thin packsCarlos Martín Nieto2013-10-041-1/+2
| | | | | | | | | | | | When given an ODB from which to read objects, the indexer will attempt to inject the missing bases at the end of the pack and update the header and trailer to reflect the new contents.
* | Initial iconv hookup for precomposed unicodeRussell Belfer2013-10-031-1/+1
|/ | | | | | | | | | | | | | | | | | | | This hooks up git_path_direach and git_path_dirload so that they will take a flag indicating if directory entry names should be tested and converted from decomposed unicode to precomposed form. This code will only come into play on the Apple platform and even then, only when certain types of filesystems are used. This involved adding a flag to these functions which involved changing a lot of places in the code. This was an opportunity to do a bit of code cleanup here and there, for example, getting rid of the git_futils_cleanupdir_r function in favor of a simple flag to git_futils_rmdir_r to not remove the top level entry. That ended up adding depth tracking during rmdir_r which led to a safety check for infinite directory recursion. Yay. This hasn't actually been tested on the Mac filesystems where the issue occurs. I still need to get test environment for that.
* path: Make direach() return EUSER on callback errornulltoken2013-09-101-1/+1
|
* odb: Move the auto refresh logic to the pack backendnulltoken2013-09-041-3/+73
| | | | | | | | | | | | Previously, `git_object_read()`, `git_object_read_prefix()` and `git_object_exists()` were implementing an auto refresh logic. When the expected object couldn't be found in any backend, a call to `git_odb_refresh()` was triggered and the lookup was once again performed against all backends. This commit removes this auto-refresh logic from the odb layer and pushes it down into the pack-backend (as it's the only one currently exposing a `refresh()` endpoint).
* odb: Code beautificationnulltoken2013-08-301-3/+6
|
* odb_pack: handle duplicate objects from different packsBrodie Rao2013-08-081-25/+16
| | | | | | | This is based on 24634c6fd02b2240e4a93fad70a08220f8fb793a. This also corrects an issue with error codes being mixed up with the number of found objects.
* Consolidate packfile allocation furtherRussell Belfer2013-04-221-42/+37
| | | | | | | Rename git_packfile_check to git_packfile_alloc since it is now being used more in that capacity. Fix the various places that use it. Consolidate some repeated code in odb_pack.c related to the allocation of a new pack_backend.
* Move odb_backend implementors stuff into git2/sysRussell Belfer2013-04-211-1/+2
| | | | | | | | | | | | This moves some of the odb_backend stuff that is related to the internals of an odb_backend implementation into include/git2/sys. Some of the stuff related to streaming I left in include/git2 because it seemed like it would be reasonably needed by a normal user who wanted to stream objects into and out of the ODB. Also, I added APIs for traversing the list of backends so that some of the tests would not need to access ODB internals.
* Several warnings detected by static code analyzer fixedArkadiy Shapkin2013-03-181-4/+4
| | | | | | | Implicit type conversion argument of function to size_t type Suspicious sequence of types castings: size_t -> int -> size_t Consider reviewing the expression of the 'A = B == C' kind. The expression is calculated as following: 'A = (B == C)' Unsigned type is never < 0
* odb_pack: Unused functionsVicent Marti2013-03-151-20/+0
|
* indexer: properly free the packfile resourcesCarlos Martín Nieto2013-01-121-1/+1
| | | | | | | | The indexer needs to call the packfile's free function so it takes care of freeing the caches. We still need to close the mwf descriptor manually so we can rename the packfile into its final name on Windows.
* Revert "pack: packfile_free -> git_packfile_free and use it in the indexers"Carlos Martín Nieto2013-01-111-1/+1
| | | | | | This reverts commit f289f886cb81bb570bed747053d5ebf8aba6bef7, which makes the tests fail on Windows. Revert until we can figure out a solution.
* pack: packfile_free -> git_packfile_free and use it in the indexersCarlos Martín Nieto2013-01-111-1/+1
| | | | | It turns out the indexers have been ignoring the pack's free function and leaking data. Plug that.
* dat errorcodeVicent Marti2013-01-101-0/+5
|
* Sane refresh logicVicent Marti2013-01-101-43/+32
| | | | | | | All the ODB backends have a specific refresh interface. When reading an object, first we attempt every single backend: if the read fails, then we refresh all the backends and retry the read one more time to see if the object has appeared.
* update copyrightsEdward Thomson2013-01-081-1/+1
|
* Merge pull request #1115 from ben/struct-versionsVicent Martí2012-12-051-0/+2
|\ | | | | Version info for public structs
| * Deploy versioned git_odb_backend structureBen Straub2012-11-301-0/+2
| |
* | odb-pack: resurrect pack_backend__read_headerDavid Michael Barr2012-12-031-10/+9
|/
* Make git_odb_foreach_cb take const paramRussell Belfer2012-11-271-1/+1
| | | | | This makes the first OID param of the ODB callback a const pointer and also propogates that change all the way to the backends.
* create callback to handle packs from fetch, move the indexer to odb_packEdward Thomson2012-11-051-0/+67
|
* ODB pack: snapshot last_found to avoid raceBen Straub2012-09-191-16/+17
| | | | Also removed unnecessary refresh call and fixed some indentation.
* Remove mtime checks from ODB packfile backendBen Straub2012-09-191-23/+32
| | | | | Now forcing refresh on a foreach, and on missed full-oid or short-oid lookups.
* ODB: re-load packfiles on failed lookupBen Straub2012-09-191-8/+28
| | | | | | | | | | | The old method was avoiding re-loading of packfiles by watching the mtime of the pack directory. This causes the ODB to become stale if the directory and packfile are written within the same clock millisecond, as when cloning a fairly small repo. This method tries to find the object in the cached packs, and forces a refresh when that fails. This will cause extra stat'ing on a miss, but speeds up the success case and avoids this race condition.
* odb_pack: fix race conditionMichael Schubert2012-09-141-6/+8
| | | | | | | last_found is the last packfile a wanted object was found in. Since last_found is shared among all searching threads, it might changes while we're searching. As suggested by @arrbee, put a copy on the stack to fix the race condition.
* odb_pack: try lookup before refreshing packsDavid Michael Barr2012-09-131-3/+3
| | | | | | | | | | | | | | | | | | | | | | | This reduces the rate of syscalls for the common case of sequences of object reads from the same pack. Best of 5 timings for libgit2_clar before this patch: real 0m5.375s user 0m0.392s sys 0m3.564s After applying this patch: real 0m5.285s user 0m0.356s sys 0m3.544s 0.6% improvement in system time. 9.2% improvement in user time. 1.7% improvement in elapsed time. Confirmed a 0.6% reduction in number of system calls with strace. Expect greater improvement for graph-traversal with large packs.
* odb: pass the user's data pointer correctly in foreachCarlos Martín Nieto2012-09-041-1/+1
|
* Merge remote-tracking branch 'arrbee/tree-walk-fixes' into developmentVicent Marti2012-08-061-3/+6
|\ | | | | | | | | | | | | | | | | Conflicts: src/notes.c src/transports/git.c src/transports/http.c src/transports/local.c tests-clar/odb/foreach.c
| * Update iterators for consistency across libraryRussell Belfer2012-08-031-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This updates all the `foreach()` type functions across the library that take callbacks from the user to have a consistent behavior. The rules are: * A callback terminates the loop by returning any non-zero value * Once the callback returns non-zero, it will not be called again (i.e. the loop stops all iteration regardless of state) * If the callback returns non-zero, the parent fn returns GIT_EUSER * Although the parent returns GIT_EUSER, no error will be set in the library and `giterr_last()` will return NULL if called. This commit makes those changes across the library and adds tests for most of the iteration APIs to make sure that they follow the above rules.
* | Merge remote-tracking branch 'nulltoken/topic/amd64-compat' into developmentVicent Marti2012-08-021-3/+3
|\ \ | | | | | | | | | | | | | | | | | | Conflicts: src/netops.c src/netops.h src/oid.c
| * | portability: Improve x86/amd64 compatibilitynulltoken2012-07-241-3/+3
| |/
* | odb: allow creating an ODB backend from a packfile indexCarlos Martín Nieto2012-07-211-0/+35
|/ | | | | git_odb_backend_one_packfile() allows us to create an ODB backend out of an .idx file.
* odb: add git_odb_foreach()Carlos Martín Nieto2012-07-031-0/+20
| | | | | Go through each backend and list every objects that exists in them. This allows fsck-like uses.
* Fix valgrind issuesRussell Belfer2012-05-041-3/+3
| | | | | | | There are three changes here: - correctly propogate error code from failed object lookups - make zlib inflate use our allocators - add OID to notfound error in ODB lookups
* buf: deploy git_buf_len()nulltoken2012-04-301-1/+1
|
* Fix warnings on 64-bit windows buildsRussell Belfer2012-04-171-4/+4
| | | | | This fixes all the warnings on win64 except those in deps, which come from the regex code.
* Convert attr, ignore, mwindow, status to new errorsRussell Belfer2012-03-161-1/+1
| | | | | Also cleaned up some previously converted code that still had little things to polish.
* Migrate ODB files to new error handlingRussell Belfer2012-03-121-88/+70
| | | | | | | | | | | | | | This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to the new style of error handling. Also got the unix and win32 versions of map.c. There are some minor changes to other files but no others were completely converted. This also contains an update to filebuf so that a zeroed out filebuf will not think that the fd (== 0) is actually open (and inadvertently call close() on fd 0 if cleaned up). Lastly, this was built and tested on win32 and contains a bunch of fixes for the win32 build which was pretty broken.
* error-handling: ReferencesVicent Martí2012-03-061-1/+1
| | | | | Yes, this is error handling solely for `refs.c`, but some of the abstractions leak all ofer the code base.
* Clean up GIT_UNUSED macros on all platformsRussell Belfer2012-03-021-2/+2
| | | | | | | | | | | | | | | | | | | It turns out that commit 31e9cfc4cbcaf1b38cdd3dbe3282a8f57e5366a5 did not fix the GIT_USUSED behavior on all platforms. This commit walks through and really cleans things up more thoroughly, getting rid of the unnecessary stuff. To remove the use of some GIT_UNUSED, I ended up adding a couple of new iterators for hashtables that allow you to iterator just over keys or just over values. In making this change, I found a bug in the clar tests (where we were doing *count++ but meant to do (*count)++ to increment the value). I fixed that but then found the test failing because it was not really using an empty repo. So, I took some of the code that I wrote for iterator testing and moved it to clar_helpers.c, then made use of that to make it easier to open fixtures on a per test basis even within a single test file.
* zlib: Remove custom `git2/zlib.h` headerVicent Martí2012-02-151-1/+1
| | | | | | This is legacy compat stuff for when `deflateBound` is not defined, but we're not embedding zlib and that function is always available. Kill that with fire.
* Update Copyright headerschu2012-02-131-1/+1
| | | | Signed-off-by: schu <schu-github@schulog.org>
View Lorry Controller Status