summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* use CHILD_PROCESS_INIT to initialize automatic variablesrs/child-process-initRené Scharfe2016-08-052-6/+3
| | | | | | | | Initialize struct child_process variables already when they're defined. That's shorter and saves a function call. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Tenth batch for 2.10Junio C Hamano2016-08-041-0/+6
| | | | Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'jc/grep-commandline-vs-configuration'Junio C Hamano2016-08-044-16/+29
|\ | | | | | | | | | | | | | | | | "git -c grep.patternType=extended log --basic-regexp" misbehaved because the internal API to access the grep machinery was not designed well. * jc/grep-commandline-vs-configuration: grep: further simplify setting the pattern type
| * grep: further simplify setting the pattern typejc/grep-commandline-vs-configurationJunio C Hamano2016-07-254-16/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When c5c31d33 (grep: move pattern-type bits support to top-level grep.[ch], 2012-10-03) introduced grep_commit_pattern_type() helper function, the intention was to allow the users of grep API to having to fiddle only with .pattern_type_option (which can be set to "fixed", "basic", "extended", and "pcre"), and then immediately before compiling the pattern strings for use, call grep_commit_pattern_type() to have it prepare various bits in the grep_opt structure (like .fixed, .regflags, etc.). However, grep_set_pattern_type_option() helper function the grep API internally uses were left as an external function by mistake. This function shouldn't have been made callable by the users of the API. Later when the grep API was used in revision traversal machinery, the caller then mistakenly started calling the function around 34a4ae55 (log --grep: use the same helper to set -E/-F options as "git grep", 2012-10-03), instead of setting the .pattern_type_option field and letting the grep_commit_pattern_type() to take care of the details. This caused an unnecessary bug that made a configured grep.patternType take precedence over the command line options (e.g. --basic-regexp, --fixed-strings) in "git log" family of commands. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'sb/submodule-clone-retry'Junio C Hamano2016-08-042-12/+12
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | An earlier tweak to make "submodule update" retry a failing clone of submodules was buggy and caused segfault, which has been fixed. * sb/submodule-clone-retry: submodule-helper: fix indexing in clone retry error reporting path git-submodule: forward exit code of git-submodule--helper more faithfully
| * | submodule-helper: fix indexing in clone retry error reporting pathJohannes Sixt2016-07-223-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'git submodule--helper update-clone' has logic to retry failed clones a second time. For this purpose, there is a list of submodules to clone, and a second list that is filled with the submodules to retry. Within these lists, the submodules are identified by an index as if both lists were just appended. This works nicely except when the second clone attempt fails as well. To report an error, the identifying index must be adjusted by an offset so that it can be used as an index into the second list. However, the calculation uses the logical total length of the lists so that the result always points one past the end of the second list. Pick the correct index. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Acked-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | git-submodule: forward exit code of git-submodule--helper more faithfullyJohannes Sixt2016-07-223-15/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git-submodule--helper is invoked as the upstream of a pipe in several places. Usually, the failure of a program in this position is not detected by the shell. For this reason, the code inserts a token in the output stream when git-submodule--helper fails that is detected downstream, where the shell script is quit with exit code 1. There happens to be a bug in git-submodule--helper that leads to a segmentation fault. The test suite triggers the crash in several places, all of which are protected by 'test_must_fail'. But due to the inspecific exit code 1, the crash remains undiagnosed. Extend the failure protocol such that git-submodule--helper's exit code is passed downstream (only in the case of failure). This enables the downstream to use it as its own exit code, and 'test_must_fail' to identify the segmentation fault as an unexpected failure. The bug itself is fixed in the next commit. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Acked-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'sb/pack-protocol-doc-nak'Junio C Hamano2016-08-041-1/+1
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | A doc update. * sb/pack-protocol-doc-nak: Documentation: pack-protocol correct NAK response
| * | | Documentation: pack-protocol correct NAK responsesb/pack-protocol-doc-nakStefan Beller2016-07-221-1/+1
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | In the transport protocol we use NAK to signal the non existence of a common base, so fix the documentation. This helps readers of the document, as they don't have to wonder about the difference between NAK and NACK. As NACK is used in git archive and upload-archive, this is easy to get wrong. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Ninth batch of topics for 2.10Junio C Hamano2016-08-031-0/+61
| | | | | | | | | | | | Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'jk/diff-do-not-reuse-wtf-needs-cleaning'Junio C Hamano2016-08-032-0/+18
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is an optimization used in "git diff $treeA $treeB" to borrow an already checked-out copy in the working tree when it is known to be the same as the blob being compared, expecting that open/mmap of such a file is faster than reading it from the object store, which involves inflating and applying delta. This however kicked in even when the checked-out copy needs to go through the convert-to-git conversion (including the clean filter), which defeats the whole point of the optimization. The optimization has been disabled when the conversion is necessary. * jk/diff-do-not-reuse-wtf-needs-cleaning: diff: do not reuse worktree files that need "clean" conversion
| * | | diff: do not reuse worktree files that need "clean" conversionjk/diff-do-not-reuse-wtf-needs-cleaningJeff King2016-07-222-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When accessing a blob for a diff, we may try to reuse file contents in the working tree, under the theory that it is faster to mmap those file contents than it would be to extract the content from the object database. When we have to filter those contents, though, that assumption does not hold. Even for our internal conversions like CRLF, we have to allocate and fill a new buffer anyway. But much worse, for external clean filters we have to exec an arbitrary script, and we have no idea how expensive it may be to run. So let's skip this optimization when conversion into git's "clean" form is required. This applies whenever the "want_file" flag is false. When it's true, the caller actually wants the smudged worktree contents, which the reused file by definition already has (in fact, this is a key optimization going the other direction, since reusing the worktree file there lets us skip smudge filters). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'rs/submodule-config-code-cleanup'Junio C Hamano2016-08-033-17/+28
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Code cleanup. * rs/submodule-config-code-cleanup: submodule-config: fix test binary crashing when no arguments given submodule-config: combine early return code into one goto submodule-config: passing name reference for .gitmodule blobs submodule-config: use explicit empty string instead of strbuf in config_from()
| * | | | submodule-config: fix test binary crashing when no arguments givenrs/submodule-config-code-cleanupHeiko Voigt2016-07-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since arg[0] will be NULL without any argument here and starts_with() does not like NULL-pointers. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | submodule-config: combine early return code into one gotoHeiko Voigt2016-07-281-19/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | So we have simpler return handling code and all the cleanup code in almost one place. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | submodule-config: passing name reference for .gitmodule blobsHeiko Voigt2016-07-282-9/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 959b5455 (submodule: implement a config API for lookup of .gitmodules values, 2015-08-18) implemented the initial version of the submodule config cache. During development of that initial version we extracted the function gitmodule_sha1_from_commit(). During that process we missed that the strbuf rev was still used in config_from() and now is left empty. Lets fix this by also returning this string. This means that now when reading .gitmodules from revisions, the error messages also contain a reference to the blob they are from. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | submodule-config: use explicit empty string instead of strbuf in config_from()René Scharfe2016-07-191-2/+1
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use a string constant instead of an empty strbuf to shorten the code and make it easier to read. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'jk/push-progress'Junio C Hamano2016-08-039-64/+202
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git push" and "git clone" learned to give better progress meters to the end user who is waiting on the terminal. * jk/push-progress: receive-pack: send keepalives during quiet periods receive-pack: turn on connectivity progress receive-pack: relay connectivity errors to sideband receive-pack: turn on index-pack resolving progress index-pack: add flag for showing delta-resolution progress clone: use a real progress meter for connectivity check check_connected: add progress flag check_connected: relay errors to alternate descriptor check_everything_connected: use a struct with named options check_everything_connected: convert to argv_array rev-list: add optional progress reporting check_everything_connected: always pass --quiet to rev-list
| * | | | receive-pack: send keepalives during quiet periodsjk/push-progressJeff King2016-07-203-1/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After a client has sent us the complete pack, we may spend some time processing the data and running hooks. If the client asked us to be quiet, receive-pack won't send any progress data during the index-pack or connectivity-check steps. And hooks may or may not produce their own progress output. In these cases, the network connection is totally silent from both ends. Git itself doesn't care about this (it will wait forever), but other parts of the system (e.g., firewalls, load-balancers, etc) might hang up the connection. So we'd like to send some sort of keepalive to let the network and the client side know that we're still alive and processing. We can use the same trick we did in 05e9515 (upload-pack: send keepalive packets during pack computation, 2013-09-08). Namely, we will send an empty sideband data packet every `N` seconds that we do not relay any stderr data over the sideband channel. As with 05e9515, this means that we won't bother sending keepalives when there's actual progress data, but will kick in when it has been disabled (or if there is a lull in the progress data). The concept is simple, but the details are subtle enough that they need discussing here. Before the client sends us the pack, we don't want to do any keepalives. We'll have sent our ref advertisement, and we're waiting for them to send us the pack (and tell us that they support sidebands at all). While we're receiving the pack from the client (or waiting for it to start), there's no need for keepalives; it's up to them to keep the connection active by sending data. Moreover, it would be wrong for us to do so. When we are the server in the smart-http protocol, we must treat our connection as half-duplex. So any keepalives we send while receiving the pack would potentially be buffered by the webserver. Not only does this make them useless (since they would not be delivered in a timely manner), but it could actually cause a deadlock if we fill up the buffer with keepalives. (It wouldn't be wrong to send keepalives in this phase for a full-duplex connection like ssh; it's simply pointless, as it is the client's responsibility to speak). As soon as we've gotten all of the pack data, then the client is waiting for us to speak, and we should start keepalives immediately. From here until the end of the connection, we send one any time we are not otherwise sending data. But there's a catch. Receive-pack doesn't know the moment we've gotten all the data. It passes the descriptor to index-pack, who reads all of the data, and then starts resolving the deltas. We have to communicate that back. To make this work, we instruct the sideband muxer to enable keepalives in three phases: 1. In the beginning, not at all. 2. While reading from index-pack, wait for a signal indicating end-of-input, and then start them. 3. Afterwards, always. The signal from index-pack in phase 2 has to come over the stderr channel which the muxer is reading. We can't use an extra pipe because the portable run-command interface only gives us stderr and stdout. Stdout is already used to pass the .keep filename back to receive-pack. We could also send a signal there, but then we would find out about it in the main thread. And the keepalive needs to be done by the async muxer thread (since it's the one writing sideband data back to the client). And we can't reliably signal the async thread from the main thread, because the async code sometimes uses threads and sometimes uses forked processes. Therefore the signal must come over the stderr channel, where it may be interspersed with other random human-readable messages from index-pack. This patch makes the signal a single NUL byte. This is easy to parse, should not appear in any normal stderr output, and we don't have to worry about any timing issues (like seeing half the signal bytes in one read(), and half in a subsequent one). This is a bit ugly, but it's simple to code and should work reliably. Another option would be to stop using an async thread for muxing entirely, and just poll() both stderr and stdout of index-pack from the main thread. This would work for index-pack (because we aren't doing anything useful in the main thread while it runs anyway). But it would make the connectivity check and the hook muxers much more complicated, as they need to simultaneously feed the sub-programs while reading their stderr. The index-pack phase is the only one that needs this signaling, so it could simply behave differently than the other two. That would mean having two separate implementations of copy_to_sideband (and the keepalive code), though. And it still doesn't get rid of the signaling; it just means we can write a nicer message like "END_OF_INPUT" or something on stdout, since we don't have to worry about separating it from the stderr cruft. One final note: this signaling trick is only done with index-pack, not with unpack-objects. There's no point in doing it for the latter, because by definition it only kicks in for a small number of objects, where keepalives are not as useful (and this conveniently lets us avoid duplicating the implementation). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | receive-pack: turn on connectivity progressJeff King2016-07-201-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we receive a large push, the server side of the connection may spend a lot of time (30s or more for a full push of linux.git) walking the object graph without producing any output. Let's give the user some indication that we're actually working. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | receive-pack: relay connectivity errors to sidebandJeff King2016-07-201-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the connectivity check encounters a problem when receiving a push, the error output goes to receive-pack's stderr, whose destination depends on the protocol used (ssh tends to send it to the user, though without a "remote" prefix; http will generally eat it in the server's error log). The information should consistently go back to the user, as there is a reasonable chance their client is buggy and generating a bad pack. We can do so by muxing it over the sideband as we do with other sub-process stderr. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | receive-pack: turn on index-pack resolving progressJeff King2016-07-201-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we receive a large push, the server side may have to spend a lot of CPU processing the incoming packfile. During the "receiving" phase, we are typically network bound, and the client is writing its own progress to the user. But during the delta resolution phase, we may spend minutes (e.g., for a full push of linux.git) without making any indication to the user that the connection has not hung. Let's ask index-pack to produce progress output for this phase (unless the client asked us to be quiet, of course). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | index-pack: add flag for showing delta-resolution progressJeff King2016-07-201-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The index-pack command has two progress meters: one for "receiving objects", and one for "resolving deltas". You get neither by default, or both with "-v". But for a push through receive-pack, we would want only the "resolving deltas" phase, _not_ the "receiving objects" progress. There are two reasons for this. One is simply that existing clients are already printing "writing objects" progress at the same time. Arguably "receiving" from the far end is more useful, because it tells you what has actually gotten there, as opposed to what might be stuck in a buffer somewhere between the client and server. But that would require a protocol extension to tell clients not to print their progress. Possible, but complexity for little gain. The second reason is much more important. In a full-duplex connection like git-over-ssh, we can print progress while the pack is incoming, and it will immediately get to the client. But for a half-duplex connection like git-over-http, we should not say anything until we have received the full request. Anything we write is subject to being stuck in a buffer by the webserver. Worse, we can end up in a deadlock if that buffer fills up. So our best bet is to avoid writing anything that isn't a small fixed size until we've received the full pack. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | clone: use a real progress meter for connectivity checkJeff King2016-07-201-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Because the initial connectivity check for a cloned repository can be slow, 0781aa4 (clone: let the user know when check_everything_connected is run, 2013-05-03) added a "fake" progress meter; we simply say "Checking connectivity" when it starts, and "done" at the end, with nothing between. Since check_connected() now knows how to do a real progress meter, we can drop our fake one and use that one instead. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | check_connected: add progress flagJeff King2016-07-202-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Connectivity checks have to traverse the entire object graph in the worst case (e.g., a full clone or a full push). For large repositories like linux.git, this can take 30-60 seconds, during which time git may produce little or no output. Let's add the option of showing progress, which is taken care of by rev-list. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | check_connected: relay errors to alternate descriptorJeff King2016-07-202-2/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Unless the "quiet" flag is given, check_connected sends any errors to the stderr of the caller (because the child rev-list inherits that descriptor). However, server-side callers may want to send these over a sideband channel instead. Let's make that possible. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | check_everything_connected: use a struct with named optionsJeff King2016-07-205-45/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The number of variants of check_everything_connected has grown over the years, so that the "real" function takes several possibly-zero, possibly-NULL arguments. We hid the complexity behind some wrapper functions, but this doesn't scale well when we want to add new options. If we add more wrapper variants to handle the new options, then we can get a combinatorial explosion when those options might be used together (right now nobody wants to use both "shallow" and "transport" together, so we get by with just a few wrappers). If instead we add new parameters to each function, each of which can have a default value, then callers who want the defaults end up with confusing invocations like: check_everything_connected(fn, 0, data, -1, 0, NULL); where it is unclear which parameter is which (and every caller needs updated when we add new options). Instead, let's add a struct to hold all of the optional parameters. This is a little more verbose for the callers (who have to declare the struct and fill it in), but it makes their code much easier to follow, because every option is named as it is set (and unused options do not have to be mentioned at all). Note that we could also stick the iteration function and its callback data into the option struct, too. But since those are required for each call, by avoiding doing so, we can let very simple callers just pass "NULL" for the options and not worry about the struct at all. While we're touching each site, let's also rename the function to check_connected(). The existing name was quite long, and not all of the wrappers even used the full name. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | check_everything_connected: convert to argv_arrayJeff King2016-07-201-12/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This avoids the magic "9" array-size which we must avoid overflowing, making further patches simpler. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | rev-list: add optional progress reportingJeff King2016-07-202-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's easy to ask rev-list to do a traversal that may takes many seconds (e.g., by calling "--objects --all"). In theory you can monitor its progress by the output you get to stdout, but this isn't always easy. Some operations, like "--count", don't make any output until the end. And some callers, like check_everything_connected(), are using it just for the error-checking of the traversal, and throw away stdout entirely. This patch adds a "--progress" option which can be used to give some eye-candy for a user waiting for a long traversal. This is just a rev-list option and not a regular traversal option, because it needs cooperation from the callbacks in builtin/rev-list.c to do the actual count. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | check_everything_connected: always pass --quiet to rev-listJeff King2016-07-201-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The check_everything_connected function takes a "quiet" parameter which does two things if non-zero: 1. redirect rev-list's stderr to /dev/null to avoid showing errors to the user 2. pass "--quiet" to rev-list Item (1) is obviously useful. But item (2) is surprisingly not. For rev-list, "--quiet" does not have anything to do with chattiness on stderr; it tells rev-list not to bother writing the list of traversed objects to stdout, for efficiency. And since we always redirect rev-list's stdout to /dev/null in this function, there is no point in asking it to ever write anything to stdout. The efficiency gains are modest; a best-of-five run of "git rev-list --objects --all" on linux.git dropped from 32.013s to 30.502s when adding "--quiet". That's only about 5%, but given how easy it is, it's worth doing. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'jt/fetch-large-handshake-window-on-http'Junio C Hamano2016-08-031-7/+12
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git fetch" exchanges batched have/ack messages between the sender and the receiver, initially doubling every time and then falling back to enlarge the window size linearly. The "smart http" transport, being an half-duplex protocol, outgrows the preset limit too quickly and becomes inefficient when interacting with a large repository. The internal mechanism learned to grow the window size more aggressively when working with the "smart http" transport. * jt/fetch-large-handshake-window-on-http: fetch-pack: grow stateless RPC windows exponentially
| * | | | | fetch-pack: grow stateless RPC windows exponentiallyjt/fetch-large-handshake-window-on-httpJonathan Tan2016-07-191-7/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When updating large repositories, the LARGE_FLUSH limit (that is, the limit at which the window growth strategy switches from exponential to linear) is reached quite quickly. Use a conservative exponential growth strategy when that limit is reached instead (and increase LARGE_FLUSH so that there is no regression in window size). This optimization is only applied during stateless RPCs to avoid the issue raised and fixed in commit 44d8dc54 (Fix potential local deadlock during fetch-pack, 2011-03-29). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | Merge branch 'jk/git-jump'Junio C Hamano2016-08-032-3/+11
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git jump" script (in contrib/) has been updated a bit. * jk/git-jump: contrib/git-jump: fix typo in README contrib/git-jump: add whitespace-checking mode contrib/git-jump: fix greedy regex when matching hunks
| * | | | | | contrib/git-jump: fix typo in READMEjk/git-jumpJeff King2016-07-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | contrib/git-jump: add whitespace-checking modeJeff King2016-07-222-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If you have whitespace errors in lines you've introduced, it can be convenient to be able to jump directly to them for fixing. You can't quite use "git jump diff" for this, because though it passes arbitrary options to "git diff", it expects to see an actual unified diff in the output. Whereas "git diff --check" actually produces lines that look like compiler quickfix lines already, meaning we just need to run it and feed the output directly to the editor. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | contrib/git-jump: fix greedy regex when matching hunksJeff King2016-07-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The hunk-header regex looks for "\+\d+" to find the post-image line numbers, but it skips the pre-image line numbers with a simple ".*". That means we may greedily eat the post-image numbers and match a "\+\d" further on, in the funcname text. For example, commit 6b9c38e has this hunk header: diff --git a/t/t0006-date.sh b/t/t0006-date.sh [...] @@ -50,8 +50,8 @@ check_show iso-local "$TIME" '2016-06-15 14:13:20 +0000' If you run: git checkout 6b9c38e git jump diff HEAD^ t/ it will erroneously match "+0000" as the starting line number and jump there, rather than line 50. We can fix it by just making the "skip" regex non-greedy, taking the first "+" we see, which should be the post-image line information. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | | Merge branch 'mm/status-suggest-merge-abort'Junio C Hamano2016-08-033-2/+10
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git status" learned to suggest "merge --abort" during a conflicted merge, just like it already suggests "rebase --abort" during a conflicted rebase. * mm/status-suggest-merge-abort: status: suggest 'git merge --abort' when appropriate
| * | | | | | | status: suggest 'git merge --abort' when appropriatemm/status-suggest-merge-abortMatthieu Moy2016-07-223-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We already suggest 'git rebase --abort' during a conflicted rebase. Similarly, suggest 'git merge --abort' during conflict resolution on 'git merge'. Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | | | Merge branch 'jk/parse-options-concat'Junio C Hamano2016-08-033-22/+22
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Users of the parse_options_concat() API function need to allocate extra slots in advance and fill them with OPT_END() when they want to decide the set of supported options dynamically, which makes the code error-prone and hard to read. This has been corrected by tweaking the API to allocate and return a new copy of "struct option" array. * jk/parse-options-concat: parse_options: allocate a new array when concatenating
| * | | | | | | | parse_options: allocate a new array when concatenatingjk/parse-options-concatJeff King2016-07-063-22/+22
| | |_|_|_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In exactly one callers (builtin/revert.c), we build up the options list dynamically from multiple arrays. We do so by manually inserting "filler" entries into one array, and then copying the other array into the allocated space. This is tedious and error-prone, as you have to adjust the filler any time the second array is modified (although we do at least check and die() when the counts do not match up). Instead, let's just allocate a new array. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | | | Merge branch 'sb/push-options'Junio C Hamano2016-08-0313-23/+294
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git push" learned to accept and pass extra options to the receiving end so that hooks can read and react to them. * sb/push-options: add a test for push options push: accept push options receive-pack: implement advertising and receiving push options push options: {pre,post}-receive hook learns about push options
| * | | | | | | | add a test for push optionssb/push-optionsStefan Beller2016-07-141-0/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The functions `mk_repo_pair` as well as `test_refs` are borrowed from t5543-atomic-push, with additional hooks installed. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | push: accept push optionsStefan Beller2016-07-146-4/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This implements everything that is required on the client side to make use of push options from the porcelain push command. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | receive-pack: implement advertising and receiving push optionsStefan Beller2016-07-144-6/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The pre/post receive hook may be interested in more information from the user. This information can be transmitted when both client and server support the "push-options" capability, which when used is a phase directly after update commands ended by a flush pkt. Similar to the atomic option, the server capability can be disabled via the `receive.advertisePushOptions` config variable. While documenting this, fix a nit in the `receive.advertiseAtomic` wording. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | push options: {pre,post}-receive hook learns about push optionsStefan Beller2016-07-143-13/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The environment variable GIT_PUSH_OPTION_COUNT is set to the number of push options sent, and GIT_PUSH_OPTION_{0,1,..} is set to the transmitted option. The code is not executed as the push options are set to NULL, nor is the new capability advertised. There was some discussion back and forth how to present these push options to the user as there are some ways to do it: Keep all options in one environment variable ============================================ + easiest way to implement in Git - This would make things hard to parse correctly in the hook. Put the options in files instead, filenames are in GIT_PUSH_OPTION_FILES ====================================== + After a discussion about environment variables and shells, we may not want to put user data into an environment variable (see [1] for example). + We could transmit binaries, i.e. we're not bound to C strings as we are when using environment variables to the user. + Maybe easier to parse than constructing environment variable names GIT_PUSH_OPTION_{0,1,..} yourself - cleanup of the temporary files is hard to do reliably - we have race conditions with multiple clients pushing, hence we'd need to use mkstemp. That's not too bad, but still. Use environment variables, but restrict to key/value pairs ========================================================== (When the user pushes a push option `foo=bar`, we'd GIT_PUSH_OPTION_foo=bar) + very easy to parse for a simple model of push options - it's not sufficient for more elaborate models, e.g. it doesn't allow doubles (e.g. cc=reviewer@email) Present the options in different environment variables ====================================================== (This is implemented) * harder to parse as a user, but we have a sample hook for that. - doesn't allow binary files + allows the same option twice, i.e. is not restrictive about options, except for binary files. + doesn't clutter a remote directory with (possibly stale) temporary files As we first want to focus on getting simple strings to work reliably, we go with the last option for now. If we want to do transmission of binaries later, we can just attach a 'side-channel', e.g. "any push option that contains a '\0' is put into a file instead of the environment variable and we'd have new GIT_PUSH_OPTION_FILES, GIT_PUSH_OPTION_FILENAME_{0,1,..} environment variables". [1] 'Shellshock' https://lwn.net/Articles/614218/ Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | | | | Merge branch 'ew/http-walker'Junio C Hamano2016-08-033-31/+206
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Dumb http transport on the client side has been optimized. * ew/http-walker: list: avoid incompatibility with *BSD sys/queue.h http-walker: reduce O(n) ops with doubly-linked list http: avoid disconnecting on 404s for loose objects http-walker: remove unused parameter from fetch_object
| * | | | | | | | | list: avoid incompatibility with *BSD sys/queue.hew/http-walkerEric Wong2016-07-181-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The OS X build pulls in sys/queue.h, which pollutes the preprocessor namespace with a macro generically named LIST_HEAD, and clashes with the name we use here. ref: http://mid.gmane.org/FB76544F-16F7-45CA-9649-FD62EE44B0DE@gmail.com Reported-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | http-walker: reduce O(n) ops with doubly-linked listEric Wong2016-07-122-27/+179
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using the a Linux-kernel-derived doubly-linked list implementation from the Userspace RCU library allows us to enqueue and delete items from the object request queue in constant time. This change reduces enqueue times in the prefetch() function where object request queue could grow to several thousand objects. I left out the list_for_each_entry* family macros from list.h which relied on the __typeof__ operator as we support platforms without it. Thus, list_entry (aka "container_of") needs to be called explicitly inside macro-wrapped for loops. The downside is this costs us an additional pointer per object request, but this is offset by reduced overhead on queue operations leading to improved performance and shorter queue depths. Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | http: avoid disconnecting on 404s for loose objectsEric Wong2016-07-122-2/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 404s are common when fetching loose objects on static HTTP servers, and reestablishing a connection for every single 404 adds additional latency. Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | http-walker: remove unused parameter from fetch_objectEric Wong2016-07-121-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This parameter has not been used since commit 1d389ab65dc6 ("Add support for parallel HTTP transfers") back in 2005 Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>