summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* diff: retire "compaction" heuristicsjc/retire-compaction-heuristicsJunio C Hamano2016-12-237-67/+8
| | | | | | | | | | | | | | | | | | | | | | When a patch inserts a block of lines, whose last lines are the same as the existing lines that appear before the inserted block, "git diff" can choose any place between these existing lines as the boundary between the pre-context and the added lines (adjusting the end of the inserted block as appropriate) to come up with variants of the same patch, and some variants are easier to read than others. We have been trying to improve the choice of this boundary, and Git 2.11 shipped with an experimental "compaction-heuristic". Since then another attempt to improve the logic further resulted in a new "indent-heuristic" logic. It is agreed that the latter gives better result overall, and the former outlived its usefulness. Retire "compaction", and keep "indent" as an experimental feature. The latter hopefully will be turned on by default in a future release, but that should be done as a separate step. Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* A bit of updates post -rc0Junio C Hamano2016-11-011-7/+8
| | | | Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'cc/split-index-typofix'Junio C Hamano2016-11-011-1/+1
|\ | | | | | | | | | | | | Typofix in a comment in code * cc/split-index-typofix: split-index: s/eith/with/ typo fix
| * split-index: s/eith/with/ typo fixcc/split-index-typofixChristian Couder2016-11-011-1/+1
| | | | | | | | | | Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'jk/no-looking-at-dotgit-outside-repo'Junio C Hamano2016-11-011-1/+2
|\ \ | | | | | | | | | | | | | | | | | | A small code cleanup. * jk/no-looking-at-dotgit-outside-repo: sha1_name: make wraparound of the index into ring-buffer explicit
| * | sha1_name: make wraparound of the index into ring-buffer explicitRené Scharfe2016-11-011-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Overflow is defined for unsigned integers, but not for signed ones. Wrap around explicitly for the new ring-buffer in find_unique_abbrev() as we did in bb84735c for the ones in sha1_to_hex() and get_pathname(), thus avoiding signed overflows and getting rid of the magic number 3. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'rs/cocci'Junio C Hamano2016-11-011-0/+15
|\ \ \ | |_|/ |/| | | | | | | | | | | | | | | | | Improve the rule to convert "unsigned char [20]" into "struct object_id *" in contrib/coccinelle/ * rs/cocci: cocci: avoid self-references in object_id transformations
| * | cocci: avoid self-references in object_id transformationsrs/cocciRené Scharfe2016-11-011-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The object_id functions oid_to_hex, oid_to_hex_r, oidclr, oidcmp, and oidcpy are defined as wrappers of their legacy counterparts sha1_to_hex, sha1_to_hex_r, hashclr, hashcmp, and hashcpy, respectively. Make sure that the Coccinelle transformations for converting legacy function calls are not applied to these wrappers themselves, which would result in tautological declarations. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Git 2.11-rc0v2.11.0-rc0Junio C Hamano2016-10-312-28/+42
| | | | | | | | | | | | Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'nd/test-helpers'Junio C Hamano2016-10-312-3/+18
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update to the test framework made in 2.9 timeframe broke running the tests under valgrind, which has been fixed. * nd/test-helpers: valgrind: support test helpers
| * | | valgrind: support test helpersnd/test-helpersRené Scharfe2016-10-272-3/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tests run with --valgrind call git commands through a wrapper script that invokes valgrind on them. This script (valgrind.sh) is in turn invoked through symlinks created for each command in t/valgrind/bin/. Since e6e7530d (test helpers: move test-* to t/helper/ subdirectory) these symlinks have been broken for test helpers -- they point to the old locations in the root of the build directory. Fix that by teaching the code for creating the links about the new location of the binaries, and do the same in the wrapper script to allow it to find its payload. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'sc/fmt-merge-msg-doc-markup-fix'Junio C Hamano2016-10-311-2/+2
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Documentation fix. * sc/fmt-merge-msg-doc-markup-fix: Documentation/fmt-merge-msg: fix markup in example
| * | | | Documentation/fmt-merge-msg: fix markup in examplesc/fmt-merge-msg-doc-markup-fixStefan Christ2016-10-281-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use at least 4 delimiting dashes that are required for ListingBlock to get this block rendered as verbatim text. Signed-off-by: Stefan Christ <contact@stefanchrist.eu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'ak/sh-setup-dot-source-i18n-fix'Junio C Hamano2016-10-311-3/+3
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recent update to git-sh-setup (a library of shell functions that are used by our in-tree scripted Porcelain commands) included another shell library git-sh-i18n without specifying where it is, relying on the $PATH. This has been fixed to be more explicit by prefixing $(git --exec-path) output in front. * ak/sh-setup-dot-source-i18n-fix: git-sh-setup: be explicit where to dot-source git-sh-i18n from.
| * | | | | git-sh-setup: be explicit where to dot-source git-sh-i18n from.ak/sh-setup-dot-source-i18n-fixAnders Kaseorg2016-10-301-3/+3
| | |_|/ / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | d323c6b641 ("i18n: git-sh-setup.sh: mark strings for translation", 2016-06-17) started to dot-source git-sh-i18n shell script library, assuming that $PATH is already adjusted for our scripts, namely, $GIT_EXEC_PATH is at the beginning of $PATH. Old contrib scripts like contrib/convert-grafts-to-replace-refs.sh and contrib/rerere-train.sh and third-party scripts like guilt may however be using this as ". $(git --exec-path)/git-sh-setup", without satisfying that assumption. Be more explicit by specifying its path prefixed with "$(git --exec-path)/". to be safe. While we’re here, move the sourcing of git-sh-i18n below the shell portability fixes. Signed-off-by: Anders Kaseorg <andersk@mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'rs/commit-pptr-simplify'Junio C Hamano2016-10-311-8/+6
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Code simplification. * rs/commit-pptr-simplify: commit: simplify building parents list
| * | | | | commit: simplify building parents listrs/commit-pptr-simplifyRené Scharfe2016-10-301-8/+6
| | |/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Push pptr down into the FROM_MERGE branch of the if/else statement, where it's actually used, and call commit_list_append() for appending elements instead of playing tricks with commit_list_insert(). Call copy_commit_list() in the amend branch instead of open-coding it. Don't bother setting pptr in the final branch as it's not used thereafter. Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'jk/rebase-config-insn-fmt-docfix'Junio C Hamano2016-10-311-1/+1
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Documentation fix. * jk/rebase-config-insn-fmt-docfix: doc: fix missing "::" in config list
| * | | | | doc: fix missing "::" in config listjk/rebase-config-insn-fmt-docfixJeff King2016-10-301-1/+1
| |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The rebase.instructionFormat option is missing its "::" to tell AsciiDoc that it's a list entry. As a result, the option name gets lumped into the description in one big paragraph. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | Merge branch 'ak/pre-receive-hook-template-modefix'Junio C Hamano2016-10-311-0/+0
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A trivial clean-up to a recently graduated topic. * ak/pre-receive-hook-template-modefix: pre-receive.sample: mark it executable
| * | | | | pre-receive.sample: mark it executableak/pre-receive-hook-template-modefixAnders Kaseorg2016-10-281-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For consistency with other hooks, make the sample hook executable. Signed-off-by: Anders Kaseorg <andersk@mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | Merge branch 'jk/common-main'Junio C Hamano2016-10-311-2/+2
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A trivial clean-up to a recently graduated topic. * jk/common-main: git-compat-util: move content inside ifdef/endif guards
| * | | | | | git-compat-util: move content inside ifdef/endif guardsJeff King2016-10-271-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 3f2e2297b9 (add an extra level of indirection to main(), 2016-07-01) added a declaration to git-compat-util.h, but it was accidentally placed after the final #endif that guards against multiple inclusions. This doesn't have any actual impact on the code, since it's not incorrect to repeat a function declaration in C. But it's a bad habit, and makes it more likely for somebody else to make the same mistake. It also defeats gcc's optimization to avoid opening header files whose contents are completely guarded. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | | Merge branch 'aw/numbered-stash'Junio C Hamano2016-10-313-3/+50
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The user always has to say "stash@{$N}" when naming a single element in the default location of the stash, i.e. reflogs in refs/stash. The "git stash" command learned to accept "git stash apply 4" as a short-hand for "git stash apply stash@{4}". * aw/numbered-stash: stash: allow stashes to be referenced by index only
| * | | | | | | stash: allow stashes to be referenced by index onlyaw/numbered-stashAaron M Watson2016-10-263-3/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of referencing "stash@{n}" explicitly, make it possible to simply reference as "n". Most users only reference stashes by their position in the stash stack (what I refer to as the "index" here). The syntax for the typical stash (stash@{n}) is slightly annoying and easy to forget, and sometimes difficult to escape properly in a script. Because of this the capability to do things with the stash by simply referencing the index is desirable. This patch includes the superior implementation provided by Øsse Walle (thanks for that), with a slight change to fix a broken test in the test suite. I also merged the test scripts as suggested by Jeff King, and un-wrapped the documentation as suggested by Junio Hamano. Signed-off-by: Aaron M Watson <watsona4@gmail.com> Reviewed-by: Jeff King <peff@peff.net>
* | | | | | | | Merge branch 'jt/trailer-with-cruft'Junio C Hamano2016-10-313-279/+654
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update "interpret-trailers" machinery and teaches it that people in real world write all sorts of crufts in the "trailer" that was originally designed to have the neat-o "Mail-Header: like thing" and nothing else. * jt/trailer-with-cruft: trailer: support values folded to multiple lines trailer: forbid leading whitespace in trailers trailer: allow non-trailers in trailer block trailer: clarify failure modes in parse_trailer trailer: make args have their own struct trailer: streamline trailer item create and add trailer: use list.h for doubly-linked list trailer: improve const correctness
| * | | | | | | | trailer: support values folded to multiple linesJonathan Tan2016-10-213-10/+211
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, interpret-trailers requires that a trailer be only on 1 line. For example: a: first line second line would be interpreted as one trailer line followed by one non-trailer line. Make interpret-trailers support RFC 822-style folding, treating those lines as one logical trailer. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | trailer: forbid leading whitespace in trailersJonathan Tan2016-10-213-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, interpret-trailers allows leading whitespace in trailer lines. This leads to false positives, especially for quoted lines or bullet lists. Forbid leading whitespace in trailers. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | trailer: allow non-trailers in trailer blockJonathan Tan2016-10-213-15/+194
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, interpret-trailers requires all lines of a trailer block to be trailers (or comments) - if not it would not identify that block as a trailer block, and thus create its own trailer block, inserting a blank line. For example: echo -e "\nSigned-off-by: x\nnot trailer" | git interpret-trailers --trailer "c: d" would result in: Signed-off-by: x not trailer c: d Relax the definition of a trailer block to require that the trailers (i) are all trailers, or (ii) contain at least one Git-generated trailer and consists of at least 25% trailers. Signed-off-by: x not trailer c: d (i) is the existing functionality. (ii) allows arbitrary lines to be included in trailer blocks, like those in [1], and still allow interpret-trailers to be used. [1] https://kernel.googlesource.com/pub/scm/linux/kernel/git/stable/linux-stable/+/e7d316a02f683864a12389f8808570e37fb90aa3 Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | trailer: clarify failure modes in parse_trailerJonathan Tan2016-10-211-22/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The parse_trailer function has a few modes of operation, all depending on whether the separator is present in its input, and if yes, the separator's position. Some of these modes are failure modes, and these failure modes are handled differently depending on whether the trailer line was sourced from a file or from a command-line argument. Extract a function to find the separator, allowing the invokers of parse_trailer to determine how to handle the failure modes instead of making parse_trailer do it. In this function, also take in the list of separators, so that we can distinguish between command line arguments (which allow '=' as separator) and file input (which does not allow '=' as separator). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | trailer: make args have their own structJonathan Tan2016-10-201-50/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Improve type safety by making arguments (whether from configuration or from the command line) have their own "struct arg_item" type, separate from the "struct trailer_item" type used to represent the trailers in the buffer being manipulated. This change also prepares "struct trailer_item" to be further differentiated from "struct arg_item" in future patches. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | trailer: streamline trailer item create and addJonathan Tan2016-10-201-70/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, creation and addition (to a list) of trailer items are spread across multiple functions. Streamline this by only having 2 functions: one to parse the user-supplied string, and one to add the parsed information to a list. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | trailer: use list.h for doubly-linked listJonathan Tan2016-10-201-167/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace the existing handwritten implementation of a doubly-linked list in trailer.c with the functions and macros from list.h. This significantly simplifies the code. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | trailer: improve const correctnessJonathan Tan2016-10-141-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change "const char *" to "char *" in struct trailer_item and in the return value of apply_command (since those strings are owned strings). Change "struct conf_info *" to "const struct conf_info *" (since that struct is not modified). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | | | | Merge branch 'ls/filter-process'Junio C Hamano2016-10-3119-135/+1498
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The smudge/clean filter API expect an external process is spawned to filter the contents for each path that has a filter defined. A new type of "process" filter API has been added to allow the first request to run the filter for a path to spawn a single process, and all filtering need is served by this single process for multiple paths, reducing the process creation overhead. * ls/filter-process: contrib/long-running-filter: add long running filter example convert: add filter.<driver>.process option convert: prepare filter.<driver>.process option convert: make apply_filter() adhere to standard Git error handling pkt-line: add functions to read/write flush terminated packet streams pkt-line: add packet_write_gently() pkt-line: add packet_flush_gently() pkt-line: add packet_write_fmt_gently() pkt-line: extract set_packet_header() pkt-line: rename packet_write() to packet_write_fmt() run-command: add clean_on_exit_handler run-command: move check_pipe() from write_or_die to run_command convert: modernize tests convert: quote filter names in error messages
| * | | | | | | | | contrib/long-running-filter: add long running filter exampleLars Schneider2016-10-172-1/+131
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a simple pass-thru filter as example implementation for the Git filter protocol version 2. See Documentation/gitattributes.txt, section "Filter Protocol" for more info. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | convert: add filter.<driver>.process optionLars Schneider2016-10-174-10/+1082
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git's clean/smudge mechanism invokes an external filter process for every single blob that is affected by a filter. If Git filters a lot of blobs then the startup time of the external filter processes can become a significant part of the overall Git execution time. In a preliminary performance test this developer used a clean/smudge filter written in golang to filter 12,000 files. This process took 364s with the existing filter mechanism and 5s with the new mechanism. See details here: https://github.com/github/git-lfs/pull/1382 This patch adds the `filter.<driver>.process` string option which, if used, keeps the external filter process running and processes all blobs with the packet format (pkt-line) based protocol over standard input and standard output. The full protocol is explained in detail in `Documentation/gitattributes.txt`. A few key decisions: * The long running filter process is referred to as filter protocol version 2 because the existing single shot filter invocation is considered version 1. * Git sends a welcome message and expects a response right after the external filter process has started. This ensures that Git will not hang if a version 1 filter is incorrectly used with the filter.<driver>.process option for version 2 filters. In addition, Git can detect this kind of error and warn the user. * The status of a filter operation (e.g. "success" or "error) is set before the actual response and (if necessary!) re-set after the response. The advantage of this two step status response is that if the filter detects an error early, then the filter can communicate this and Git does not even need to create structures to read the response. * All status responses are pkt-line lists terminated with a flush packet. This allows us to send other status fields with the same protocol in the future. Helped-by: Martin-Louis Bright <mlbright@gmail.com> Reviewed-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | convert: prepare filter.<driver>.process optionLars Schneider2016-10-171-26/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactor the existing 'single shot filter mechanism' and prepare the new 'long running filter mechanism'. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | convert: make apply_filter() adhere to standard Git error handlingLars Schneider2016-10-171-9/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | apply_filter() returns a boolean that tells the caller if it "did convert or did not convert". The variable `ret` was used throughout the function to track errors whereas `1` denoted success and `0` failure. This is unusual for the Git source where `0` denotes success. Rename the variable and flip its value to make the function easier readable for Git developers. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | pkt-line: add functions to read/write flush terminated packet streamsLars Schneider2016-10-172-0/+80
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | write_packetized_from_fd() and write_packetized_from_buf() write a stream of packets. All content packets use the maximal packet size except for the last one. After the last content packet a `flush` control packet is written. read_packetized_to_strbuf() reads arbitrary sized packets until it detects a `flush` packet. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | pkt-line: add packet_write_gently()Lars Schneider2016-10-171-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | packet_write_fmt_gently() uses format_packet() which lets the caller only send string data via "%s". That means it cannot be used for arbitrary data that may contain NULs. Add packet_write_gently() which writes arbitrary data and does not die in case of an error. The function is used by other pkt-line functions in a subsequent patch. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | pkt-line: add packet_flush_gently()Lars Schneider2016-10-172-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | packet_flush() would die in case of a write error even though for some callers an error would be acceptable. Add packet_flush_gently() which writes a pkt-line flush packet like packet_flush() but does not die in case of an error. The function is used in a subsequent patch. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | pkt-line: add packet_write_fmt_gently()Lars Schneider2016-10-172-4/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | packet_write_fmt() would die in case of a write error even though for some callers an error would be acceptable. Add packet_write_fmt_gently() which writes a formatted pkt-line like packet_write_fmt() but does not die in case of an error. The function is used in a subsequent patch. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | pkt-line: extract set_packet_header()Lars Schneider2016-10-171-6/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extracted set_packet_header() function converts an integer to a 4 byte hex string. Make this function locally available so that other pkt-line functions could use it. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | pkt-line: rename packet_write() to packet_write_fmt()Lars Schneider2016-10-1711-29/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | packet_write() should be called packet_write_fmt() because it is a printf-like function that takes a format string as first parameter. packet_write_fmt() should be used for text strings only. Arbitrary binary data should use a new packet_write() function that is introduced in a subsequent patch. Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | run-command: add clean_on_exit_handlerLars Schneider2016-10-172-4/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some processes might want to perform cleanup tasks before Git kills them due to the 'clean_on_exit' flag. Let's give them an interface for doing this. The feature is used in a subsequent patch. Please note, that the cleanup callback is not executed if Git dies of a signal. The reason is that only "async-signal-safe" functions would be allowed to be call in that case. Since we cannot control what functions the callback will use, we will not support the case. See 507d7804 for more details. Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | run-command: move check_pipe() from write_or_die to run_commandLars Schneider2016-10-173-16/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move check_pipe() to run_command and make it public. This is necessary to call the function from pkt-line in a subsequent patch. While at it, make async_exit() static to run_command.c as it is no longer used from outside. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | convert: modernize testsLars Schneider2016-10-171-29/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use `test_config` to set the config, check that files are empty with `test_must_be_empty`, compare files with `test_cmp`, and remove spaces after ">" and "<". Please note that the "rot13" filter configured in "setup" keeps using `git config` instead of `test_config` because subsequent tests might depend on it. Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | | | | | | convert: quote filter names in error messagesLars Schneider2016-10-171-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git filter driver commands with spaces (e.g. `filter.sh foo`) are hard to read in error messages. Quote them to improve the readability. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | | | | | | | Merge branch 'ls/git-open-cloexec'Junio C Hamano2016-10-315-14/+26
|\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git generally does not explicitly close file descriptors that were open in the parent process when spawning a child process, but most of the time the child does not want to access them. As Windows does not allow removing or renaming a file that has a file descriptor open, a slow-to-exit child can even break the parent process by holding onto them. Use O_CLOEXEC flag to open files in various codepaths. * ls/git-open-cloexec: read-cache: make sure file handles are not inherited by child processes sha1_file: open window into packfiles with O_CLOEXEC sha1_file: rename git_open_noatime() to git_open()