summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* combine-diff: simplify intersect_paths() furtherks/diff-c-with-diff-orderJunio C Hamano2014-02-031-22/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Linus once said: I actually wish more people understood the really core low-level kind of coding. Not big, complex stuff like the lockless name lookup, but simply good use of pointers-to-pointers etc. For example, I've seen too many people who delete a singly-linked list entry by keeping track of the "prev" entry, and then to delete the entry, doing something like if (prev) prev->next = entry->next; else list_head = entry->next; and whenever I see code like that, I just go "This person doesn't understand pointers". And it's sadly quite common. People who understand pointers just use a "pointer to the entry pointer", and initialize that with the address of the list_head. And then as they traverse the list, they can remove the entry without using any conditionals, by just doing a "*pp = entry->next". Applying that simplification lets us lose 7 lines from this function even while adding 2 lines of comment. I was tempted to squash this into the original commit, but because the benchmarking described in the commit log is without this simplification, I decided to keep it a separate follow-up patch. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* combine-diff: combine_diff_path.len is not needed anymoreKirill Smelkov2014-01-283-24/+9
| | | | | | | | | | | | The field was used in order to speed-up name comparison and also to mark removed paths by setting it to 0. Because the updated code does significantly less strcmp and also just removes paths from the list and free right after we know a path will not be needed, it is not needed anymore. Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* combine-diff: optimize combine_diff_path sets intersectionKirill Smelkov2014-01-281-21/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When generating combined diff, for each commit, we intersect diff paths from diff(parent_0,commit) to diff(parent_i,commit) comparing all paths pairs, i.e. doing it the quadratic way. That is correct, but could be optimized. Paths come from trees in sorted (= tree) order, and so does diff_tree() emits resulting paths in that order too. Now if we look at diffcore transformations, all of them, except diffcore_order, preserve resulting path ordering: - skip_stat_unmatch, grep, pickaxe, filter -- just skip elements -> order stays preserved - break -- just breaks diff for a path, adding path dup after the path -> order stays preserved - detect rename/copy -- resulting paths are emitted sorted (verified empirically) So only diffcore_order changes diff paths ordering. But diffcore_order meaning affects only presentation - i.e. only how to show the diff, so we could do all the internal computations without paths reordering, and order only resultant paths set. This is faster, since, if we know two paths sets are all ordered, their intersection could be done in linear time. This patch does just that. Timings for `git log --raw --no-abbrev --no-renames` without `-c` ("git log") and with `-c` ("git log -c") before and after the patch are as follows: linux.git v3.10..v3.11 log log -c before 1.9s 20.4s after 1.9s 16.6s navy.git (private repo) log log -c before 0.83s 15.6s after 0.83s 2.1s P.S. I think linux.git case is sped up not so much as the second one, since in navy.git, there are more exotic (subtree, etc) merges. P.P.S. My tracing showed that the rest of the time (16.6s vs 1.9s) is usually spent in computing huge diffs from commit to second parent. Will try to deal with it, if I'll have time. P.P.P.S. For combine_diff_path, ->len is not needed anymore - will remove it in the next noisy cleanup path, to maintain good signal/noise ratio here. Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* diff test: add tests for combine-diff with orderfileKirill Smelkov2014-01-281-0/+21
| | | | | | | | | In the next patch combine-diff will have special code-path for taking orderfile into account. Prepare for making changes by introducing coverage tests for that case. Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* diffcore-order: export generic ordering interfaceKirill Smelkov2014-01-282-19/+46
| | | | | | | | | | | | | | | | | | diffcore_order() interface only accepts a queue of `struct diff_filepair`. In the next patches, we'll want to order `struct combine_diff_path` by path, so let's first rework diffcore-order to also provide generic low-level interface for ordering arbitrary objects, provided they have path accessors. The new interface is: - `struct obj_order` for describing objects to ordering routine, and - order_objects() for actually doing the ordering work. Signed-off-by: Kirill Smelkov <kirr@mns.spb.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Git 1.9-rc0v1.9-rc0Junio C Hamano2014-01-172-1/+13
| | | | Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'maint'Junio C Hamano2014-01-170-0/+0
|\ | | | | | | | | * maint: git-svn: workaround for a bug in svn serf backend
| * git-svn: workaround for a bug in svn serf backendRoman Kagan2014-01-171-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Subversion serf backend in versions 1.8.5 and below has a bug(*) that the function creating the descriptor of a file change -- add_file() -- doesn't make a copy of its third argument when storing it on the returned descriptor. As a result, by the time this field is used (in transactions of file copying or renaming) it may well be released, and the memory reused. One of its possible manifestations is the svn assertion triggering on an invalid path, with a message svn_fspath__skip_ancestor: Assertion `svn_fspath__is_canonical(child_fspath)' failed. This patch works around this bug, by storing the value to be passed as the third argument to add_file() in a local variable with the same scope as the file change descriptor, making sure their lifetime is the same. * [ew: fixed in Subversion r1553376 as noted by Jonathan Nieder] Cc: Benjamin Pabst <benjamin.pabst85@gmail.com> Signed-off-by: Eric Wong <normalperson@yhbt.net> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Roman Kagan <rkagan@mail.ru>
* | Merge branch 'fp/submodule-checkout-mode'Junio C Hamano2014-01-171-1/+12
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | "submodule.*.update=checkout", when propagated from .gitmodules to .git/config, turned into a "submodule.*.update=none", which did not make much sense. * fp/submodule-checkout-mode: git-submodule.sh: 'checkout' is a valid update mode
| * | git-submodule.sh: 'checkout' is a valid update modefp/submodule-checkout-modeFrancesco Pretto2014-01-071-1/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'checkout' is documented as one of the valid values for the 'submodule.<name>.update' variable, and in a repository with the variable set to 'checkout', "git submodule update" command does update using the 'checkout' mode. However, it has been an accident that the implementation works this way; any unknown value would trigger the same codepath and update using the 'checkout' mode. Explicitly list 'checkout' as one of the known update modes, and error out when an unknown update mode is used. Teach the codepath that initializes the configuration variable from an in-tree .gitmodules that 'checkout' is one of the valid values. The code since ac1fbbda (submodule: do not copy unknown update mode from .gitmodules, 2013-12-02) used to treat the value 'checkout' as unknown and mapped it to 'none', which made little sense. With this change, 'checkout' specified in .gitmodules will stay to be 'checkout'. Signed-off-by: Francesco Pretto <ceztko@gmail.com> Signed-off-by: Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'nd/shallow-clone'Junio C Hamano2014-01-1736-168/+1535
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fetching from a shallow-cloned repository used to be forbidden, primarily because the codepaths involved were not carefully vetted and we did not bother supporting such usage. This attempts to allow object transfer out of a shallow-cloned repository in a controlled way (i.e. the receiver become a shallow repository with truncated history). * nd/shallow-clone: (31 commits) t5537: fix incorrect expectation in test case 10 shallow: remove unused code send-pack.c: mark a file-local function static git-clone.txt: remove shallow clone limitations prune: clean .git/shallow after pruning objects clone: use git protocol for cloning shallow repo locally send-pack: support pushing from a shallow clone via http receive-pack: support pushing to a shallow clone via http smart-http: support shallow fetch/clone remote-curl: pass ref SHA-1 to fetch-pack as well send-pack: support pushing to a shallow clone receive-pack: allow pushes that update .git/shallow connected.c: add new variant that runs with --shallow-file add GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses receive/send-pack: support pushing from a shallow clone receive-pack: reorder some code in unpack() fetch: add --update-shallow to accept refs that update .git/shallow upload-pack: make sure deepening preserves shallow roots fetch: support fetching from a shallow repository clone: support remote shallow repository ...
| * | | t5537: fix incorrect expectation in test case 10nd/shallow-cloneNguyễn Thái Ngọc Duy2014-01-091-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 48d25ca adds a new commit "7" to the repo that the next test case in commit 1609488 clones from. But the next test case does not expect this commit. For these tests, it's the bottom that's important, not the top. Fix the expected commit list. While at it, fix the default http port number to 5537. Otherwise when t5536 learns to test httpd, running test in parallel may fail. References: 48d25ca fetch: add --update-shallow to accept... - 2013-12-05 1609488 smart-http: support shallow fetch/clone - 2013-12-05 Noticed-by: Jeff King <peff@peff.net> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | shallow: remove unused codeRamsay Jones2014-01-064-20/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 58babfff ("shallow.c: the 8 steps to select new commits for .git/shallow", 05-12-2013) added a function to implement step 5 of the quoted eight steps, namely 'remove_nonexistent_ours_in_pack()'. This function implements an optional optimization step in the new shallow commit selection algorithm. However, this function has no callers. (The commented out call sites would need to change, in order to provide information required by the function.) Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Acked-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | send-pack.c: mark a file-local function staticRamsay Jones2014-01-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit f2c681cf ("send-pack: support pushing from a shallow clone via http", 05-12-2013) adds the 'advertise_shallow_grafts_buf' function as an external symbol. Noticed by sparse. ("'advertise_shallow_grafts_buf' was not declared. Should it be static?") Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | git-clone.txt: remove shallow clone limitationsNguyễn Thái Ngọc Duy2013-12-101-6/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that git supports data transfer from or to a shallow clone, these limitations are not true anymore. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | prune: clean .git/shallow after pruning objectsNguyễn Thái Ngọc Duy2013-12-106-2/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch teaches "prune" to remove shallow roots that are no longer reachable from any refs (e.g. when the relevant refs are removed). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | clone: use git protocol for cloning shallow repo locallyNguyễn Thái Ngọc Duy2013-12-102-2/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | clone_local() does not handle $SRC/shallow. It could be made so, but it's simpler to use fetch-pack/upload-pack instead. This used to be caught by the check in upload-pack, which is triggered by transport_get_remote_refs(), even in local clone case. The check is now gone and check_everything_connected() should catch the result incomplete repo. But check_everything_connected() will soon be skipped in local clone case, opening a door to corrupt repo. This patch should close that door. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | send-pack: support pushing from a shallow clone via httpNguyễn Thái Ngọc Duy2013-12-103-5/+42
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | receive-pack: support pushing to a shallow clone via httpNguyễn Thái Ngọc Duy2013-12-102-3/+35
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | smart-http: support shallow fetch/cloneNguyễn Thái Ngọc Duy2013-12-106-9/+81
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | remote-curl: pass ref SHA-1 to fetch-pack as wellNguyễn Thái Ngọc Duy2013-12-102-1/+9
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | send-pack: support pushing to a shallow cloneNguyễn Thái Ngọc Duy2013-12-103-4/+43
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | receive-pack: allow pushes that update .git/shallowNguyễn Thái Ngọc Duy2013-12-105-13/+201
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The basic 8 steps to update .git/shallow does not fully apply here because the user may choose to accept just a few refs (while fetch always accepts all refs). The steps are modified a bit. 1-6. same as before. After calling assign_shallow_commits_to_refs at step 6, each shallow commit has a bitmap that marks all refs that require it. 7. mark all "ours" shallow commits that are reachable from any refs. We will need to do the original step 7 on them later. 8. go over all shallow commit bitmaps, mark refs that require new shallow commits. 9. setup a strict temporary shallow file to plug all the holes, even if it may cut some of our history short. This file is used by all hooks. The hooks could use --shallow-file=$GIT_DIR/shallow to overcome this and reach everything in current repo. 10. go over the new refs one by one. For each ref, do the reachability test if it needs a shallow commit on the list from step 7. Remove it if it's reachable from our refs. Gather all required shallow commits, run check_everything_connected() with the new ref, then install them to .git/shallow. This mode is disabled by default and can be turned on with receive.shallowupdate Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | connected.c: add new variant that runs with --shallow-fileNguyễn Thái Ngọc Duy2013-12-102-8/+36
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | add GIT_SHALLOW_FILE to propagate --shallow-file to subprocessesNguyễn Thái Ngọc Duy2013-12-105-3/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This may be needed when a hook is run after a new shallow pack is received, but .git/shallow is not settled yet. A temporary shallow file to plug all loose ends should be used instead. GIT_SHALLOW_FILE is overriden by --shallow-file. --shallow-file does not work in this case because the hook may spawn many git subprocesses and the launch commands do not have --shallow-file as it's a recent addition. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | receive/send-pack: support pushing from a shallow cloneNguyễn Thái Ngọc Duy2013-12-105-11/+146
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | receive-pack: reorder some code in unpack()Nguyễn Thái Ngọc Duy2013-12-101-40/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the preparation for adding --shallow-file to both unpack-objects and index-pack. To sum up: - struct argv_array used instead of const char ** - status/code, ip/child, unpacker/keeper are moved out to function top level - successful flow now ends at the end of the function Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | fetch: add --update-shallow to accept refs that update .git/shallowNguyễn Thái Ngọc Duy2013-12-107-1/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The same steps are done as in when --update-shallow is not given. The only difference is we now add all shallow commits in "ours" and "theirs" to .git/shallow (aka "step 8"). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | upload-pack: make sure deepening preserves shallow rootsNguyễn Thái Ngọc Duy2013-12-104-4/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When "fetch --depth=N" where N exceeds the longest chain of history in the source repo, usually we just send an "unshallow" line to the client so full history is obtained. When the source repo is shallow we need to make sure to "unshallow" the current shallow point _and_ "shallow" again when the commit reaches its shallow bottom in the source repo. This should fix both cases: large <N> and --unshallow. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | fetch: support fetching from a shallow repositoryNguyễn Thái Ngọc Duy2013-12-105-5/+176
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch just put together pieces from the 8 steps patch. We stop at step 7 and reject refs that require new shallow commits. Note that, by rejecting refs that require new shallow commits, we leave dangling objects in the repo, which become "object islands" by the next "git fetch" of the same source. If the first fetch our "ours" set is zero and we do practically nothing at step 7, "ours" is full at the next fetch and we may need to walk through commits for reachability test. Room for improvement. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | clone: support remote shallow repositoryNguyễn Thái Ngọc Duy2013-12-106-7/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cloning from a shallow repository does not follow the "8 steps for new .git/shallow" because if it does we need to get through step 6 for all refs. That means commit walking down to the bottom. Instead the rule to create .git/shallow is simpler and, more importantly, cheap: if a shallow commit is found in the pack, it's probably used (i.e. reachable from some refs), so we add it. Others are dropped. One may notice this method seems flawed by the word "probably". A shallow commit may not be reachable from any refs at all if it's attached to an object island (a group of objects that are not reachable by any refs). If that object island is not complete, a new fetch request may send more objects to connect it to some ref. At that time, because we incorrectly installed the shallow commit in this island, the user will not see anything after that commit (fsck is still ok). This is not desired. Given that object islands are rare (C Git never sends such islands for security reasons) and do not really harm the repository integrity, a tradeoff is made to surprise the user occasionally but work faster everyday. A new option --strict could be added later that follows exactly the 8 steps. "git prune" can also learn to remove dangling objects _and_ the shallow commits that are attached to them from .git/shallow. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | fetch-pack.h: one statement per bitfield declarationNguyễn Thái Ngọc Duy2013-12-101-12/+12
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | fetch-pack.c: move shallow update code out of fetch_pack()Nguyễn Thái Ngọc Duy2013-12-101-9/+13
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | shallow.c: steps 6 and 7 to select new commits for .git/shallowNguyễn Thái Ngọc Duy2013-12-102-0/+297
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | shallow.c: the 8 steps to select new commits for .git/shallowNguyễn Thái Ngọc Duy2013-12-104-1/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Suppose a fetch or push is requested between two shallow repositories (with no history deepening or shortening). A pack that contains necessary objects is transferred over together with .git/shallow of the sender. The receiver has to determine whether it needs to update .git/shallow if new refs needs new shallow comits. The rule here is avoid updating .git/shallow by default. But we don't want to waste the received pack. If the pack contains two refs, one needs new shallow commits installed in .git/shallow and one does not, we keep the latter and reject/warn about the former. Even if .git/shallow update is allowed, we only add shallow commits strictly necessary for the former ref (remember the sender can send more shallow commits than necessary) and pay attention not to accidentally cut the receiver history short (no history shortening is asked for) So the steps to figure out what ref need what new shallow commits are: 1. Split the sender shallow commit list into "ours" and "theirs" list by has_sha1_file. Those that exist in current repo in "ours", the remaining in "theirs". 2. Check the receiver .git/shallow, remove from "ours" the ones that also exist in .git/shallow. 3. Fetch the new pack. Either install or unpack it. 4. Do has_sha1_file on "theirs" list again. Drop the ones that fail has_sha1_file. Obviously the new pack does not need them. 5. If the pack is kept, remove from "ours" the ones that do not exist in the new pack. 6. Walk the new refs to answer the question "what shallow commits, both ours and theirs, are required in .git/shallow in order to add this ref?". Shallow commits not associated to any refs are removed from their respective list. 7. (*) Check reachability (from the current refs) of all remaining commits in "ours". Those reachable are removed. We do not want to cut any part of our (reachable) history. We only check up commits. True reachability test is done by check_everything_connected() at the end as usual. 8. Combine the final "ours" and "theirs" and add them all to .git/shallow. Install new refs. The case where some hook rejects some refs on a push is explained in more detail in the push patches. Of these steps, #6 and #7 are expensive. Both require walking through some commits, or in the worst case all commits. And we rather avoid them in at least common case, where the transferred pack does not contain any shallow commits that the sender advertises. Let's look at each scenario: 1) the sender has longer history than the receiver All shallow commits from the sender will be put into "theirs" list at step 1 because none of them exists in current repo. In the common case, "theirs" becomes empty at step 4 and exit early. 2) the sender has shorter history than the receiver All shallow commits from the sender are likely in "ours" list at step 1. In the common case, if the new pack is kept, we could empty "ours" and exit early at step 5. If the pack is not kept, we hit the expensive step 6 then exit after "ours" is emptied. There'll be only a handful of objects to walk in fast-forward case. If it's forced update, we may need to walk to the bottom. 3) the sender has same .git/shallow as the receiver This is similar to case 2 except that "ours" should be emptied at step 2 and exit early. A fetch after "clone --depth=X" is case 1. A fetch after "clone" (from a shallow repo) is case 3. Luckily they're cheap for the common case. A push from "clone --depth=X" falls into case 2, which is expensive. Some more work may be done at the sender/client side to avoid more work on the server side: if the transferred pack does not contain any shallow commits, send-pack should not send any shallow commits to the receive-pack, effectively turning it into a normal push and avoid all steps. This patch implements all steps except #3, already handled by fetch-pack and receive-pack, #6 and #7, which has their own patch due to their size. (*) in previous versions step 7 was put before step 3. I reorder it so that the common case that keeps the pack does not need to walk commits at all. In future if we implement faster commit reachability check (maybe with the help of pack bitmaps or commit cache), step 7 could become cheap and be moved up before 6 again. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | shallow.c: extend setup_*_shallow() to accept extra shallow commitsNguyễn Thái Ngọc Duy2013-12-104-11/+24
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | connect.c: teach get_remote_heads to parse "shallow" linesNguyễn Thái Ngọc Duy2013-12-106-8/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | No callers pass a non-empty pointer as shallow_points at this stage. As a result, all clients still refuse to talk to shallow repository on the other end. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | make the sender advertise shallow commits to the receiverNguyễn Thái Ngọc Duy2013-12-105-3/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If either receive-pack or upload-pack is called on a shallow repository, shallow commits (*) will be sent after the ref advertisement (but before the packet flush), so that the receiver has the full "shape" of the sender's commit graph. This will be needed for the receiver to update its .git/shallow if necessary. This breaks the protocol for all clients trying to push to a shallow repo, or fetch from one. Which is basically the same end result as today's "is_repository_shallow() && die()" in receive-pack and upload-pack. New clients will be made aware of shallow upstream and can make use of this information. The sender must send all shallow commits that are sent in the following pack. It may send more shallow commits than necessary. upload-pack for example may choose to advertise no shallow commits if it knows in advance that the pack it's going to send contains no shallow commits. But upload-pack is the server, so we choose the cheaper way, send full .git/shallow and let the client deal with it. Smart HTTP is not affected by this patch. Shallow support on smart-http comes later separately. (*) A shallow commit is a commit that terminates the revision walker. It is usually put in .git/shallow in order to keep the revision walker from going out of bound because there is no guarantee that objects behind this commit is available. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | clone: prevent --reference to a shallow repositoryNguyễn Thái Ngọc Duy2013-12-101-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we borrow objects from another repository, we should also pay attention to their $GIT_DIR/shallow (and even info/grafts). But current alternates code does not. Reject alternate repos that are shallow because we do not do it right. In future the alternate code may be updated to check $GIT_DIR/shallow properly so that this restriction could be lifted. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | send-pack: forbid pushing from a shallow repositoryNguyễn Thái Ngọc Duy2013-12-101-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | send-pack can send a pack with loose ends to the server. receive-pack before 6d4bb38 (fetch: verify we have everything we need before updating our ref - 2011-09-01) does not detect this and keeps the pack anyway, which corrupts the repository, at least from fsck point of view. send-pack will learn to safely push from a shallow repository later. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | remote.h: replace struct extra_have_objects with struct sha1_arrayNguyễn Thái Ngọc Duy2013-12-106-22/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The latter can do everything the former can and is used in many more places. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | transport.h: remove send_pack prototype, already defined in send-pack.hNguyễn Thái Ngọc Duy2013-12-101-6/+0
| | | | | | | | | | | | | | | | | | | | Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'jk/pull-rebase-using-fork-point'Junio C Hamano2014-01-171-1/+1
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Finishing touches so that an expected error message will not leak to the UI. * jk/pull-rebase-using-fork-point: pull: suppress error when no remoteref is found
| * | | | pull: suppress error when no remoteref is foundjk/pull-rebase-using-fork-pointJohn Keeping2014-01-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 48059e4 (pull: use merge-base --fork-point when appropriate, 2013-12-08) incorrectly assumes that get_remote_merge_branch will either yield a non-empty string or return an error, but there are circumstances where it will yield an empty string. The previous code then invoked git-rev-list with no arguments, which results in an error suppressed by redirecting stderr to /dev/null. Now we invoke git-merge-base with an empty branch name, which also results in an error. Suppress this in the same way. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Sync with 1.8.5.3Junio C Hamano2014-01-133-3/+31
|\ \ \ \ \ | | |_|_|/ | |/| | | | | | | | | | | | | | | | | | * maint: Git 1.8.5.3 pack-heuristics.txt: mark up the file header properly
| * | | | Git 1.8.5.3v1.8.5.3Junio C Hamano2014-01-134-3/+31
| | | | |
| * | | | Merge branch 'nd/daemon-informative-errors-typofix' into maintJunio C Hamano2014-01-131-2/+2
| |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The "--[no-]informative-errors" options to "git daemon" were parsed a bit too loosely, allowing any other string after these option names. * nd/daemon-informative-errors-typofix: daemon: be strict at parsing parameters --[no-]informative-errors
| * \ \ \ \ Merge branch 'km/gc-eperm' into maintJunio C Hamano2014-01-131-1/+1
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A "gc" process running as a different user should be able to stop a new "gc" process from starting. * km/gc-eperm: gc: notice gc processes run by other users
| * \ \ \ \ \ Merge branch 'jk/credential-plug-leak' into maintJunio C Hamano2014-01-131-1/+2
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | An earlier "clean-up" introduced an unnecessary memory leak. * jk/credential-plug-leak: Revert "prompt: clean up strbuf usage"
| * \ \ \ \ \ \ Merge branch 'mm/mv-file-to-no-such-dir-with-slash' into maintJunio C Hamano2014-01-132-7/+47
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git mv A B/", when B does not exist as a directory, should error out, but it didn't. * mm/mv-file-to-no-such-dir-with-slash: mv: let 'git mv file no-such-dir/' error out on Windows, too mv: let 'git mv file no-such-dir/' error out