summaryrefslogtreecommitdiff
path: root/read-cache.c
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'nd/split-index'Junio C Hamano2014-07-161-28/+244
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | An experiment to use two files (the base file and incremental changes relative to it) to represent the index to reduce I/O cost of rewriting a large index when only small part of the working tree changes. * nd/split-index: (32 commits) t1700: new tests for split-index mode t2104: make sure split index mode is off for the version test read-cache: force split index mode with GIT_TEST_SPLIT_INDEX read-tree: note about dropping split-index mode or index version read-tree: force split-index mode off on --index-output rev-parse: add --shared-index-path to get shared index path update-index --split-index: do not split if $GIT_DIR is read only update-index: new options to enable/disable split index mode split-index: strip pathname of on-disk replaced entries split-index: do not invalidate cache-tree at read time split-index: the reading part split-index: the writing part read-cache: mark updated entries for split index read-cache: save deleted entries in split index read-cache: mark new entries for split index read-cache: split-index mode read-cache: save index SHA-1 after reading entry.c: update cache_changed if refresh_cache is set in checkout_entry() cache-tree: mark istate->cache_changed on prime_cache_tree() cache-tree: mark istate->cache_changed on cache tree update ...
| * t1700: new tests for split-index modeNguyễn Thái Ngọc Duy2014-06-131-2/+1
| | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: force split index mode with GIT_TEST_SPLIT_INDEXNguyễn Thái Ngọc Duy2014-06-131-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | This could be used to run the whole test suite with split indexes. Index splitting is carried out at random. "git read-tree" also resets the index and forces splitting at the next update. I had a lot of headaches with the test suite, which proves it exercises split index pretty good. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-tree: force split-index mode off on --index-outputNguyễn Thái Ngọc Duy2014-06-131-1/+2
| | | | | | | | | | | | | | Just a (paranoid?) safety measure.. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * update-index --split-index: do not split if $GIT_DIR is read onlyNguyễn Thái Ngọc Duy2014-06-131-3/+10
| | | | | | | | | | | | | | | | | | If $GIT_DIR is read only, we can't write $GIT_DIR/sharedindex. This could happen when $GIT_INDEX_FILE is set to somehwere outside $GIT_DIR. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * update-index: new options to enable/disable split index modeNguyễn Thái Ngọc Duy2014-06-131-6/+61
| | | | | | | | | | | | | | | | | | | | | | | | If you have a large work tree but only make changes in a subset, then $GIT_DIR/index's size should be stable after a while. If you change branches that touch something else, $GIT_DIR/index's size may grow large that it becomes as slow as the unified index. Do --split-index again occasionally to force all changes back to the shared index and keep $GIT_DIR/index small. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * split-index: strip pathname of on-disk replaced entriesNguyễn Thái Ngọc Duy2014-06-131-0/+10
| | | | | | | | | | | | | | | | | | | | | | We know the positions of replaced entries via the replace bitmap in "link" extension, so the "name" path does not have to be stored (it's still in the shared index). With this, we also have a way to distinguish additions vs replacements at load time and can catch broken "link" extensions. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * split-index: do not invalidate cache-tree at read timeNguyễn Thái Ngọc Duy2014-06-131-1/+2
| | | | | | | | | | | | | | | | We are sure that after merge_base_index() is done. cache-tree can still be used with the final index. So don't destroy cache tree. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * split-index: the reading partNguyễn Thái Ngọc Duy2014-06-131-2/+0
| | | | | | | | | | | | | | | | | | | | CE_REMOVE'd entries are removed here because only parts of the code base (unpack_trees in fact) test this bit when they look for the presence of an entry. Leaving them may confuse the code ignores this bit and expects to see a real entry. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: mark updated entries for split indexNguyễn Thái Ngọc Duy2014-06-131-1/+4
| | | | | | | | | | | | | | | | | | | | The large part of this patch just follows CE_ENTRY_CHANGED marks. replace_index_entry() is updated to update split_index->base->cache[] as well so base->cache[] does not reference to a freed entry. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: save deleted entries in split indexNguyễn Thái Ngọc Duy2014-06-131-6/+8
| | | | | | | | | | | | | | | | Entries that belong to the base index should not be freed. Mark CE_REMOVE to track them. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: mark new entries for split indexNguyễn Thái Ngọc Duy2014-06-131-1/+2
| | | | | | | | | | | | | | | | | | | | Make sure entry addition does not lead to unifying the index. We don't need to explicitly keep track of new entries. If ce->index is zero, they're new. Otherwise it's unlikely that they are new, but we'll do a thorough check later at writing time. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: split-index modeNguyễn Thái Ngọc Duy2014-06-131-5/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This split-index mode is designed to keep write cost proportional to the number of changes the user has made, not the size of the work tree. (Read cost is another matter, to be dealt separately.) This mode stores index info in a pair of $GIT_DIR/index and $GIT_DIR/sharedindex.<SHA-1>. sharedindex is large and unchanged over time while "index" is smaller and updated often. Format details are in index-format.txt, although not everything is implemented in this patch. Shared indexes are not automatically removed, because it's unclear if the shared index is needed by any (even temporary) indexes by just looking at it. After a while you'll collect stale shared indexes. The good news is one shared index is useable for long, until $GIT_DIR/index becomes too big and sluggish that the new shared index must be created. The safest way to clean shared indexes is to turn off split index mode, so shared files are all garbage, delete them all, then turn on split index mode again. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: save index SHA-1 after readingNguyễn Thái Ngọc Duy2014-06-131-2/+4
| | | | | | | | | | | | | | | | | | | | Also update SHA-1 after writing. If we do not do that, the second read_index() will see "initialized" variable already set and not read .git/index again, which is fine, except istate->sha1 now has a stale value. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * cache-tree: mark istate->cache_changed on cache tree invalidationNguyễn Thái Ngọc Duy2014-06-131-3/+3
| | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: be specific what part of the index has changedNguyễn Thái Ngọc Duy2014-06-131-5/+6
| | | | | | | | | | | | | | | | | | cache entry additions, removals and modifications are separated out. The rest of changes are still in the catch-all flag SOMETHING_CHANGED, which would be more specific later. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: be strict about "changed" in remove_marked_cache_entries()Nguyễn Thái Ngọc Duy2014-06-131-0/+2
| | | | | | | | | | | | | | | | | | remove_marked_cache_entries() deletes entries marked with CE_REMOVE. But if there is no such entry, do not mark the index as "changed" because that could trigger an index update unnecessarily. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: store in-memory flags in the first 12 bits of ce_flagsNguyễn Thái Ngọc Duy2014-06-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | We're running out of room for in-memory flags. But since b60e188 (Strip namelen out of ce_flags into a ce_namelen field - 2012-07-11), we copy the namelen (first 12 bits) to ce_namelen field. So those bits are free to use. Just make sure we do not accidentally write any in-memory flags back. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: relocate and unexport commit_locked_index()Nguyễn Thái Ngọc Duy2014-06-131-0/+20
| | | | | | | | | | | | | | | | | | This function is now only used by write_locked_index(). Move it to read-cache.c (because read-cache.c will need to be aware of alternate_index_output later) and unexport it. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * read-cache: new API write_locked_index instead of write_index/write_cacheNguyễn Thái Ngọc Duy2014-06-131-4/+24
| | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | cleanup duplicate name_compare() functionsjm/dedup-name-compareJeremiah Mahler2014-06-201-10/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We often represent our strings as a counted string, i.e. a pair of the pointer to the beginning of the string and its length, and the string may not be NUL terminated to that length. To compare a pair of such counted strings, unpack-trees.c and read-cache.c implement their own name_compare() functions identically. In addition, the cache_name_compare() function in read-cache.c is nearly identical. The only difference is when one string is the prefix of the other string, in which case name_compare() returns -1/+1 to show which one is longer, and cache_name_compare() returns the difference of the lengths to show the same information. Unify these three functions by using the implementation from cache_name_compare(). This does not make any difference to the existing and future callers, as they must be paying attention only to the sign of the returned value (and not the magnitude) because the original implementations of these two functions return values returned by memcmp(3) when the one string is not a prefix of the other string, and the only thing memcmp(3) guarantees its callers is the sign of the returned value, not the magnitude. Signed-off-by: Jeremiah Mahler <jmmahler@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'ym/fix-opportunistic-index-update-race'Junio C Hamano2014-06-031-1/+46
|\ \ | |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Read-only operations such as "git status" that internally refreshes the index write out the refreshed index to the disk to optimize future accesses to the working tree, but this could race with a "read-write" operation that modify the index while it is running. Detect such a race and avoid overwriting the index. Duy raised a good point that we may need to do the same for the normal writeout codepath, not just the "opportunistic" update codepath. While that is true, nobody sane would be running two simultaneous operations that are clearly write-oriented competing with each other against the same index file. So in that sense that can be done as a less urgent follow-up for this topic. * ym/fix-opportunistic-index-update-race: read-cache.c: verify index file before we opportunistically update it wrapper.c: add xpread() similar to xread()
| * read-cache.c: verify index file before we opportunistically update itym/fix-opportunistic-index-update-raceYiannis Marangos2014-04-101-1/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before we proceed to opportunistically update the index (often done by an otherwise read-only operation like "git status" and "git diff" that internally refreshes the index), we must verify that the current index file is the same as the one that we read earlier before we took the lock on it, in order to avoid a possible race. In the example below git-status does "opportunistic update" and git-rebase updates the index, but the race can happen in general. 1. process A calls git-rebase (or does anything that uses the index) 2. process A applies 1st commit 3. process B calls git-status (or does anything that updates the index) 4. process B reads index 5. process A applies 2nd commit 6. process B takes the lock, then overwrites process A's changes. 7. process A applies 3rd commit As an end result the 3rd commit will have a revert of the 2nd commit. When process B takes the lock, it needs to make sure that the index hasn't changed since step 4. Signed-off-by: Yiannis Marangos <yiannis.marangos@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'dd/use-alloc-grow'Junio C Hamano2014-03-181-5/+1
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace open-coded reallocation with ALLOC_GROW() macro. * dd/use-alloc-grow: sha1_file.c: use ALLOC_GROW() in pretend_sha1_file() read-cache.c: use ALLOC_GROW() in add_index_entry() builtin/mktree.c: use ALLOC_GROW() in append_to_tree() attr.c: use ALLOC_GROW() in handle_attr_line() dir.c: use ALLOC_GROW() in create_simplify() reflog-walk.c: use ALLOC_GROW() replace_object.c: use ALLOC_GROW() in register_replace_object() patch-ids.c: use ALLOC_GROW() in add_commit() diffcore-rename.c: use ALLOC_GROW() diff.c: use ALLOC_GROW() commit.c: use ALLOC_GROW() in register_commit_graft() cache-tree.c: use ALLOC_GROW() in find_subtree() bundle.c: use ALLOC_GROW() in add_to_ref_list() builtin/pack-objects.c: use ALLOC_GROW() in check_pbase_path()
| * | read-cache.c: use ALLOC_GROW() in add_index_entry()Dmitry S. Dolzhenko2014-03-031-5/+1
| |/ | | | | | | | | Signed-off-by: Dmitry S. Dolzhenko <dmitrys.dolzhenko@yandex.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'tg/index-v4-format'Junio C Hamano2014-03-141-1/+37
|\ \ | | | | | | | | | | | | | | | | | | * tg/index-v4-format: read-cache: add index.version config variable test-lib: allow setting the index format version introduce GIT_INDEX_VERSION environment variable
| * | read-cache: add index.version config variabletg/index-v4-formatThomas Gummerer2014-02-241-9/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | Add a config variable that allows setting the default index version when initializing a new index file. Similar to the GIT_INDEX_VERSION environment variable this only affects new index files. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | introduce GIT_INDEX_VERSION environment variableThomas Gummerer2014-02-241-1/+20
| |/ | | | | | | | | | | | | | | | | | | | | | | Respect a GIT_INDEX_VERSION environment variable, when a new index is initialized. Setting the environment variable will not cause existing index files to be converted to another format, but will only affect newly written index files. This can be used to initialize repositories with index-v4. Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'jk/pack-bitmap'Junio C Hamano2014-02-271-32/+12
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Borrow the bitmap index into packfiles from JGit to speed up enumeration of objects involved in a commit range without having to fully traverse the history. * jk/pack-bitmap: (26 commits) ewah: unconditionally ntohll ewah data ewah: support platforms that require aligned reads read-cache: use get_be32 instead of hand-rolled ntoh_l block-sha1: factor out get_be and put_be wrappers do not discard revindex when re-preparing packfiles pack-bitmap: implement optional name_hash cache t/perf: add tests for pack bitmaps t: add basic bitmap functionality tests count-objects: recognize .bitmap in garbage-checking repack: consider bitmaps when performing repacks repack: handle optional files created by pack-objects repack: turn exts array into array-of-struct repack: stop using magic number for ARRAY_SIZE(exts) pack-objects: implement bitmap writing rev-list: add bitmap mode to speed up object lists pack-objects: use bitmaps when packing objects pack-objects: split add_object_entry pack-bitmap: add support for bitmap indexes documentation: add documentation for the bitmap format ewah: compressed bitmap implementation ...
| * | read-cache: use get_be32 instead of hand-rolled ntoh_lJeff King2014-01-231-32/+12
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit d60c49c (read-cache.c: allow unaligned mapping of the index file, 2012-04-03) introduced helpers to access unaligned data. However, we already have get_be32, which has a few advantages: 1. It's already written, so we avoid duplication. 2. It's probably faster, since it does the endian conversion and the alignment fix at the same time. 3. The get_be32 code is well-tested, having been in block-sha1 for a long time. By contrast, our custom helpers were probably almost never used, since the user needed to manually define a macro to enable them. We have to add a get_be16 implementation to the existing get_be32, but that is very simple to do. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'nd/reset-intent-to-add'Junio C Hamano2014-02-271-2/+2
|\ \ | | | | | | | | | | | | * nd/reset-intent-to-add: reset: support "--mixed --intent-to-add" mode
| * | reset: support "--mixed --intent-to-add" modend/reset-intent-to-addNguyễn Thái Ngọc Duy2014-02-051-2/+2
| |/ | | | | | | | | | | | | | | | | | | When --mixed is used, entries could be removed from index if the target ref does not have them. When "reset" is used in preparation for commit spliting (in a dirty worktree), it could be hard to track what files to be added back. The new option --intent-to-add simplifies it by marking all removed files intent-to-add. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
* | Merge branch 'nd/submodule-pathspec-ending-with-slash'Junio C Hamano2014-02-271-7/+1
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allow "git cmd path/", when the 'path' is where a submodule is bound to the top-level working tree, to match 'path', despite the extra and unnecessary trailing slash. * nd/submodule-pathspec-ending-with-slash: clean: use cache_name_is_other() clean: replace match_pathspec() with dir_path_match() pathspec: pass directory indicator to match_pathspec_item() match_pathspec: match pathspec "foo/" against directory "foo" dir.c: prepare match_pathspec_item for taking more flags pathspec: rename match_pathspec_depth() to match_pathspec() pathspec: convert some match_pathspec_depth() to dir_path_match() pathspec: convert some match_pathspec_depth() to ce_path_match()
| * | pathspec: convert some match_pathspec_depth() to ce_path_match()Nguyễn Thái Ngọc Duy2014-02-241-7/+1
| |/ | | | | | | | | | | | | | | This helps reduce the number of match_pathspec_depth() call sites and show how match_pathspec_depth() is used. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'bk/refresh-missing-ok-in-merge-recursive'Junio C Hamano2014-02-271-12/+15
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allow "merge-recursive" to work in an empty (temporary) working tree again when there are renames involved, correcting an old regression in 1.7.7 era. * bk/refresh-missing-ok-in-merge-recursive: merge-recursive.c: tolerate missing files while refreshing index read-cache.c: extend make_cache_entry refresh flag with options read-cache.c: refactor --ignore-missing implementation t3030-merge-recursive: test known breakage with empty work tree
| * | read-cache.c: extend make_cache_entry refresh flag with optionsBrad King2014-02-241-10/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Convert the make_cache_entry boolean 'refresh' argument to a more general 'refresh_options' argument. Pass the value through to the underlying refresh_cache_ent call. Add option CE_MATCH_REFRESH to enable stat refresh. Update call sites to use the new signature. Signed-off-by: Brad King <brad.king@kitware.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | read-cache.c: refactor --ignore-missing implementationBrad King2014-02-241-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Move lstat ENOENT handling from refresh_index to refresh_cache_ent and activate it with a new CE_MATCH_IGNORE_MISSING option. This will allow other call paths into refresh_cache_ent to use the feature. Signed-off-by: Brad King <brad.king@kitware.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | read-cache.c: fix memory leaks caused by removed cache entriesKarsten Blees2013-11-181-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When cache_entry structs are removed from index_state.cache, they are not properly freed. Freeing those entries wasn't possible before because we couldn't remove them from index_state.name_hash. Now that we _do_ remove the entries from name_hash, we can also free them. Add 'free(cache_entry)' to all call sites of name-hash.c::remove_name_hash in read-cache.c (we could free() directly in remove_name_hash(), but name-hash.c isn't concerned with cache_entry allocation at all). Accessing a cache_entry after removing it from the index is now no longer allowed, as the memory has been freed. The following functions need minor fixes (typically by copying ce->name before use): - builtin/rm.c::cmd_rm - builtin/update-index.c::do_reupdate - read-cache.c::read_index_unmerged - resolve-undo.c::unmerge_index_entry_at Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | name-hash.c: remove cache entries instead of marking them CE_UNHASHEDKarsten Blees2013-11-181-1/+1
| |/ |/| | | | | | | | | | | | | | | | | | | | | The new hashmap implementation supports remove, so really remove unused cache entries from the name hashmap instead of just marking them. The CE_UNHASHED flag and CE_STATE_MASK are no longer needed. Keep the CE_HASHED flag to prevent adding entries twice. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'es/name-hash-no-trailing-slash-in-dirs'Junio C Hamano2013-10-171-2/+2
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clean up the internal of the name-hash mechanism used to work around case insensitivity on some filesystems to cleanly fix a long-standing API glitch where the caller of cache_name_exists() that ask about a directory with a counted string was required to have '/' at one location past the end of the string. * es/name-hash-no-trailing-slash-in-dirs: dir: revert work-around for retired dangerous behavior name-hash: stop storing trailing '/' on paths in index_state.dir_hash employ new explicit "exists in index?" API name-hash: refactor polymorphic index_name_exists()
| * | name-hash: stop storing trailing '/' on paths in index_state.dir_hashEric Sunshine2013-09-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When 5102c617 (Add case insensitivity support for directories when using git status, 2010-10-03) added directories to the name-hash there was only a single hash table in which both real cache entries and leading directory prefixes were registered. To distinguish between the two types of entries, directories were stored with a trailing '/'. 2092678c (name-hash.c: fix endless loop with core.ignorecase=true, 2013-02-28), however, moved directories to a separate hash table (index_state.dir_hash) but retained the (now) redundant trailing '/', thus callers continue to bear the burden of ensuring the slash's presence before searching the index for a directory. Eliminate this redundancy by storing paths in the dir-hash without the trailing '/'. An important benefit of this change is that it eliminates undocumented and dangerous behavior of dir.c:directory_exists_in_index_icase() in which it assumes not only that it can validly access one character beyond the end of its incoming directory argument, but also that that character will unconditionally be a '/'. This perilous behavior was "tolerated" because the string passed in by its lone caller always had a '/' in that position, however, things broke [1] when 2eac2a4c (ls-files -k: a directory only can be killed if the index has a non-directory, 2013-08-15) added a new caller which failed to respect the undocumented assumption. [1]: http://thread.gmane.org/gmane.comp.version-control.git/232727 Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | employ new explicit "exists in index?" APIEric Sunshine2013-09-171-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Each caller of index_name_exists() knows whether it is looking for a directory or a file, and can avoid the unnecessary indirection of index_name_exists() by instead calling index_dir_exists() or index_file_exists() directly. Invoking the appropriate search function explicitly will allow a subsequent patch to relieve callers of the artificial burden of having to add a trailing '/' to the pathname given to index_dir_exists(). Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'jk/write-broken-index-with-nul-sha1'Junio C Hamano2013-09-171-2/+11
|\ \ \ | |/ / |/| | | | | | | | | | | | | | | | | | | | | | | Earlier we started rejecting an attempt to add 0{40} object name to the index and to tree objects, but it sometimes is necessary to allow so to be able to use tools like filter-branch to correct such broken tree objects. * jk/write-broken-index-with-nul-sha1: write_index: optionally allow broken null sha1s
| * | write_index: optionally allow broken null sha1sjk/write-broken-index-with-nul-sha1Jeff King2013-08-281-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 4337b58 (do not write null sha1s to on-disk index, 2012-07-28) added a safety check preventing git from writing null sha1s into the index. The intent was to catch errors in other parts of the code that might let such an entry slip into the index (or worse, a tree). Some existing repositories may have invalid trees that contain null sha1s already, though. Until 4337b58, a common way to clean this up would be to use git-filter-branch's index-filter to repair such broken entries. That now fails when filter-branch tries to write out the index. Introduce a GIT_ALLOW_NULL_SHA1 environment variable to relax this check and make it easier to recover from such a history. It is tempting to not involve filter-branch in this commit at all, and instead require the user to manually invoke GIT_ALLOW_NULL_SHA1=1 git filter-branch ... to perform an index-filter on a history with trees with null sha1s. That would be slightly safer, but requires some specialized knowledge from the user. So let's set the GIT_ALLOW_NULL_SHA1 variable automatically when checking out the to-be-filtered trees. Advice on using filter-branch to remove such entries already exists on places like stackoverflow, and this patch makes it Just Work again on recent versions of git. Further commands that touch the index will still notice and fail, unless they actually remove the broken entries. A filter-branch whose filters do not touch the index at all will not error out (since we complain of the null sha1 only on writing, not when making a tree out of the index), but this is acceptable, as we still print a loud warning, so the problem is unlikely to go unnoticed. Signed-off-by: Jeff King <peff@peff.net> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | Merge branch 'rs/discard-index-discard-array' into maintJunio C Hamano2013-07-191-2/+3
| |\ \ | | | | | | | | | | | | | | | | | | | | * rs/discard-index-discard-array: read-cache: free cache in discard_index read-cache: add simple performance test
| * \ \ Merge branch 'fc/do-not-use-the-index-in-add-to-index' into maintJunio C Hamano2013-07-031-3/+3
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | * fc/do-not-use-the-index-in-add-to-index: read-cache: trivial style cleanups read-cache: fix wrong 'the_index' usage
* | \ \ \ Merge branch 'tg/index-struct-sizes'Junio C Hamano2013-09-091-15/+15
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code that reads from a region that mmaps an on-disk index assumed that "int"/"short" are always 32/16 bits. * tg/index-struct-sizes: read-cache: use fixed width integer types
| * | | | | read-cache: use fixed width integer typestg/index-struct-sizesThomas Gummerer2013-08-201-15/+15
| | |_|_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use the fixed width integer types uint16_t and uint32_t for on-disk structures; unsigned short and unsigned int do not have a guaranteed size. Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'jl/submodule-mv'Junio C Hamano2013-09-091-2/+3
|\ \ \ \ \ | |/ / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git mv A B" when moving a submodule A does "the right thing", inclusing relocating its working tree and adjusting the paths in the .gitmodules file. * jl/submodule-mv: (53 commits) rm: delete .gitmodules entry of submodules removed from the work tree mv: update the path entry in .gitmodules for moved submodules submodule.c: add .gitmodules staging helper functions mv: move submodules using a gitfile mv: move submodules together with their work trees rm: do not set a variable twice without intermediate reading. t6131 - skip tests if on case-insensitive file system parse_pathspec: accept :(icase)path syntax pathspec: support :(glob) syntax pathspec: make --literal-pathspecs disable pathspec magic pathspec: support :(literal) syntax for noglob pathspec kill limit_pathspec_to_literal() as it's only used by parse_pathspec() parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN parse_pathspec: make sure the prefix part is wildcard-free rename field "raw" to "_raw" in struct pathspec tree-diff: remove the use of pathspec's raw[] in follow-rename codepath remove match_pathspec() in favor of match_pathspec_depth() remove init_pathspec() in favor of parse_pathspec() remove diff_tree_{setup,release}_paths convert common_prefix() to use struct pathspec ...
| * | | | convert refresh_index to take struct pathspecNguyễn Thái Ngọc Duy2013-07-151-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>