summaryrefslogtreecommitdiff
path: root/src/indexer.c
Commit message (Collapse)AuthorAgeFilesLines
...
* Fix MSVC compilation warningsnulltoken2012-12-171-3/+3
|
* indexer: move the temporary buffers into the indexer objectCarlos Martín Nieto2012-12-071-10/+9
| | | | | | Storing 4kB or 8kB in the stack is not very gentle. As this part has to be linear, put the buffer into the indexer object so we allocate it once in the heap.
* indexer: correctly deal with objects larger than the window sizeCarlos Martín Nieto2012-11-301-19/+26
| | | | | | | | A mmap-window is not guaranteed to give you the whole object, but the indexer currently assumes so. Loop asking for more data until we've successfully CRC'd all of the packed data.
* indexer: make use of streaming also for deltasCarlos Martín Nieto2012-11-301-49/+78
| | | | | | | | | | | | Up to now, deltas needed to be enterily in the packfile, and we tried to decompress then in their entirety over and over again. Adjust the logic so we read them as they come, just as we do for full objects. This also allows us to simplify the logic and have less nested code. The delta resolving phase still needs to decompress the whole object into memory, as there is not yet any streaming delta-apply support, but it helps in speeding up the downloading process and reduces the amount of memory allocations we need to do.
* indexer: use the packfile streaming APICarlos Martín Nieto2012-11-301-27/+130
| | | | | | | | | | | | | The new API allows us to read the object bit by bit from the packfile, instead of needing it all at once in the packfile. This also allows us to hash the object as it comes in from the network instead of having to try to read it all and failing repeatedly for larger objects. This is only the first step, but it already shows huge improvements when dealing with objects over a few megabytes in size. It reduces the memory needs in some cases, but delta objects still need to be completely in memory and the old inefficent method is still used for that.
* API updates for indexer.hBen Straub2012-11-271-2/+2
|
* Remove use of English expletivesMartin Woodward2012-11-231-1/+1
| | | | | | Remove words such as fuck, crap, shit etc. Remove other potentially offensive words from comments. Tidy up other geopolicital terms in comments.
* Indexer: Avoid a possible double-deletion in error caseSascha Cunz2012-11-181-2/+3
|
* Remove git_hash_ctx_new - callers now _ctx_init()Edward Thomson2012-11-131-12/+12
|
* Win32 CryptoAPI and CNG support for SHA1Edward Thomson2012-11-131-11/+17
|
* Fetch: don't clobber received countBen Straub2012-10-291-1/+2
| | | | | This memset was being reached after the entire packfile under WinHttp, so the byte count was being lost for small repos.
* Network progress: rename thingsBen Straub2012-10-241-18/+18
| | | | | | | git_indexer_stats and friends -> git_transfer_progress* Also made git_transfer_progress members more sanely named.
* Indexing progress now goes to 100%Ben Straub2012-10-191-0/+1
|
* Fetch/indexer: progress callbacksBen Straub2012-10-191-1/+18
|
* indexer: do not require absolute pathMichael Schubert2012-10-091-5/+0
|
* indexer: kill git_indexer_stats.data_receivedCarlos Martín Nieto2012-08-261-8/+0
| | | | | | | It's not really needed with the current code as we have EOS and the sideband's flush to tell us we're done. Keep the distinction between processed and received objects.
* indexer: don't segfault when freeing an unused indexerCarlos Martín Nieto2012-08-251-3/+5
| | | | | Make sure that idx->pack isn't NULL before trying to free resources under it.
* indexer: recognize and mark when all of the packfile has been downloadedCarlos Martín Nieto2012-08-241-1/+11
| | | | | We can't always rely on the network telling us when the download is finished. Recognize it from the indexer itself.
* indexer: delay resolving deltasCarlos Martín Nieto2012-07-141-22/+24
| | | | | | Not all delta bases are available on the first try. By delaying resolving all deltas until the end, we avoid decompressing some of the data twice or even more times, saving effort and time.
* mwindow: allow memory-window files to deregisterCarlos Martin Nieto2012-06-281-0/+2
| | | | | | | | | Once a file is registered, there is no way to deregister it, even after the structure that contains it is no longer needed and has been freed. This may be the source of #624. Allow and use the deregister function to remove our file from the global list.
* indexer: don't use '/objects/pack/' unconditionallyCarlos Martín Nieto2012-06-281-1/+1
| | | | | Not everyone who indexes a packfile wants to put it in the standard git repository location.
* indexer: start parsing input data immediatelyMichael Schubert2012-06-251-2/+0
| | | | | | | | | Currently, the first call of git_indexer_stream_add adds the data to the underlying pack file and opens it for later use, but doesn't start parsing the already available data. This means, git_indexer_stream_finalize only works if git_indexer_stream_add was called at least twice. Kill this limitation by parsing available data immediately.
* Merge remote-tracking branch 'source/development' into developmentChris Young2012-06-131-1/+3
|\
| * misc: Fix warnings from PVS Studio trialVicent Martí2012-06-071-1/+3
| |
* | Actually do the mmap... unsurprisingly, this makes the indexer work on SFSChris Young2012-06-121-1/+1
|/ | | | | On RAM: the .idx and .pack files become links to a .lock and the original download respectively. Assume some feature (such as record locking) supported by SFS but not JXFS or RAM: is required.
* CleanupMichael Schubert2012-05-191-1/+0
| | | | | | | * indexer: remove leftover printf * commit: remove unused macros COMMIT_BASIC_PARSE, COMMIT_FULL_PARSE and COMMIT_PRINT
* errors: Rename error codesbreaking-changesVicent Martí2012-05-181-4/+4
|
* errors: Rename the generic return codesVicent Martí2012-05-181-4/+4
|
* indexer: add more consistency checksCarlos Martín Nieto2012-05-161-0/+11
| | | | | Error out in finalize if there is junk after the packfile hash or we couldn't process all the objects.
* Introduce GITERR_INDEXERCarlos Martín Nieto2012-05-151-5/+5
|
* indexer: close the pack's fd before renaming itCarlos Martín Nieto2012-05-091-1/+2
| | | | Windows gets upset if we rename a file with an open descriptor.
* buf: deploy git_buf_len()nulltoken2012-04-301-2/+2
|
* Fix Win32 warningsRussell Belfer2012-04-261-1/+1
|
* transports: buffer the git requests before sending themCarlos Martín Nieto2012-04-251-1/+1
| | | | | | Trying to send every single line immediately won't give us any speed improvement and duplicates the code we need for other transports. Make the git transport use the same buffer functions as HTTP.
* indexer: add git_indexer_stream_free() and _hash()Carlos Martín Nieto2012-04-131-0/+29
|
* indexer: Add git_indexer_stream_finalize()Carlos Martín Nieto2012-04-131-71/+235
| | | | | Resolve any lingering deltas, write out the index file and rename the packfile.
* indexer: start writing the stream indexerCarlos Martín Nieto2012-04-131-36/+310
| | | | | | This will allow us to index a packfile as soon as we receive it from the network as well as storing it with its final name so we don't need to pass temporary file names around.
* Convert indexer, notes, sha1_lookup, and signatureRussell Belfer2012-03-191-70/+64
| | | | More files moved to new error handling style.
* Continue error conversionRussell Belfer2012-03-141-4/+7
| | | | | | | This converts blob.c, fileops.c, and all of the win32 files. Also, various minor cleanups throughout the code. Plus, in testing the win32 build, I cleaned up a bunch (although not all) of the warnings with the 64-bit build.
* Migrate ODB files to new error handlingRussell Belfer2012-03-121-6/+6
| | | | | | | | | | | | | | This migrates odb.c, odb_loose.c, odb_pack.c and pack.c to the new style of error handling. Also got the unix and win32 versions of map.c. There are some minor changes to other files but no others were completely converted. This also contains an update to filebuf so that a zeroed out filebuf will not think that the fd (== 0) is actually open (and inadvertently call close() on fd 0 if cleaned up). Lastly, this was built and tested on win32 and contains a bunch of fixes for the win32 build which was pretty broken.
* error-handling: RepositoryVicent Martí2012-03-071-3/+6
| | | | | | | | This also includes droping `git_buf_lasterror` because it makes no sense in the new system. Note that in most of the places were it has been dropped, the code needs cleanup. I.e. GIT_ENOMEM is going away, so instead it should return a generic `-1` and obviously not throw anything.
* zlib: Remove custom `git2/zlib.h` headerVicent Martí2012-02-151-1/+2
| | | | | | This is legacy compat stuff for when `deflateBound` is not defined, but we're not embedding zlib and that function is always available. Kill that with fire.
* Update Copyright headerschu2012-02-131-1/+1
| | | | Signed-off-by: schu <schu-github@schulog.org>
* odb: Add internal `git_odb__hashfd`Vicent Martí2012-02-101-1/+1
|
* indexer: save the pack index with the right nameCarlos Martín Nieto2012-01-191-1/+1
| | | | Truncate at the slash; otherwise we get ppack-*.idx filenames.
* Use git_buf for path storage instead of stack-based buffersRussell Belfer2011-12-071-16/+34
| | | | | | | | | | | | | | | | | | | | This converts virtually all of the places that allocate GIT_PATH_MAX buffers on the stack for manipulating paths to use git_buf objects instead. The patch is pretty careful not to touch the public API for libgit2, so there are a few places that still use GIT_PATH_MAX. This extends and changes some details of the git_buf implementation to add a couple of extra functions and to make error handling easier. This includes serious alterations to all the path.c functions, and several of the fileops.c ones, too. Also, there are a number of new functions that parallel existing ones except that use a git_buf instead of a stack-based buffer (such as git_config_find_global_r that exists alongsize git_config_find_global). This also modifies the win32 version of p_realpath to allocate whatever buffer size is needed to accommodate the realpath instead of hardcoding a GIT_PATH_MAX limit, but that change needs to be tested still.
* Merge pull request #456 from brodie/perm-fixesVicent Martí2011-10-281-1/+1
|\ | | | | Create objects, indexes, and directories with the right file permissions
| * *: correct and codify various file permissionsBrodie Rao2011-10-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following files now have 0444 permissions: - loose objects - pack indexes - pack files - packs downloaded by fetch - packs downloaded by the HTTP transport And the following files now have 0666 permissions: - config files - repository indexes - reflogs - refs This brings libgit2 more in line with Git. Note that git_filebuf_commit() and git_filebuf_commit_at() have both gained a new mode parameter. The latter change fixes an important issue where filebufs created with GIT_FILEBUF_TEMPORARY received 0600 permissions (due to mkstemp(3) usage). Now we chmod() the file before renaming it into place. Tests have been added to confirm that new commit, tag, and tree objects are created with the right permissions. I don't have access to Windows, so for now I've guarded the tests with "#ifndef GIT_WIN32".
* | global: Properly use `git__` memory wrappersVicent Marti2011-10-281-5/+5
|/ | | | | Ensure that all memory related functions (malloc, calloc, strdup, free, etc) are using their respective `git__` wrappers.
* indexer: NUL-terminate the filenameCarlos Martín Nieto2011-10-051-2/+2
| | | | | | | As we no longer use the STRLEN macro, the NUL-terminator in the string was not copied over. Fix this. Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>
View Lorry Controller Status