summaryrefslogtreecommitdiff
path: root/remote.h
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'sg/parse-options-h-users'Junio C Hamano2023-03-301-1/+1
|\ | | | | | | | | | | | | | | | | Code clean-up to include and/or uninclude parse-options.h file as needed. * sg/parse-options-h-users: treewide: remove unnecessary inclusions of parse-options.h from headers treewide: include parse-options.h in source files
| * treewide: remove unnecessary inclusions of parse-options.h from headersSZEDER Gábor2023-03-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The headers 'diagnose.h', 'list-objects-filter-options.h', 'ref-filter.h' and 'remote.h' declare option parsing callback functions with a 'struct option*' parameter, and 'revision.h' declares an option parsing helper function taking 'struct parse_opt_ctx_t*' and 'struct option*' parameters. These headers all include 'parse-options.h', although they don't need any of the type definitions from that header file. Furthermore, 'list-objects-filter-options.h' and 'ref-filter.h' also define some OPT_* macros to initialize a 'struct option', but these don't necessitate the inclusion of parse-options.h in these headers either, because these macros are only expanded in source files. Remove these unnecessary inclusions of parse-options.h and use forward declarations to declare the necessary types. After this patch none of the header files include parse-options.h anymore. With these changes, the build time after modifying only parse-options.h is reduced by about 30%, and the number of targets built is almost 20% less: Before: $ touch parse-options.h && time make -j4 |wc -l 353 real 1m1.527s user 3m32.205s sys 0m15.903s After: 289 real 0m39.285s user 2m12.540s sys 0m11.164s Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | treewide: remove unnecessary cache.h includesElijah Newren2023-02-231-1/+0
|/ | | | | | | | | | | | | | | | | We had several header files include cache.h unnecessarily. Remove those. These have all been verified via both ensuring that gcc -E $HEADER | grep '"cache.h"' found no hits and that cat >temp.c <<EOF && #include "git-compat-util.h" #include "$HEADER" int main() {} EOF gcc -c temp.c successfully compiles without warnings. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* clone: request the 'bundle-uri' command when availableÆvar Arnfjörð Bjarmason2022-12-251-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | Set up all the needed client parts of the 'bundle-uri' protocol v2 command, without actually doing anything with the bundle URIs. If the server says it supports 'bundle-uri' teach Git to issue the 'bundle-uri' command after the 'ls-refs' during 'git clone'. The returned key=value pairs are passed to the bundle list code which is tested using a different ingest mechanism in t5750-bundle-uri-parse.sh. At this point, Git does nothing with that bundle list. It will not download any of the bundles. That will come in a later change after these protocol bits are finalized. The no-op client is initially used only by 'git clone' to test the basic functionality, and eventually will bootstrap the initial download of Git objects during a fresh clone. The bundle URI client will not be integrated into other fetches until a mechanism is created to select a subset of bundles for download. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'rs/mergesort'Junio C Hamano2022-08-031-2/+0
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make our mergesort implementation type-safe. * rs/mergesort: mergesort: remove llist_mergesort() packfile: use DEFINE_LIST_SORT fetch-pack: use DEFINE_LIST_SORT commit: use DEFINE_LIST_SORT blame: use DEFINE_LIST_SORT test-mergesort: use DEFINE_LIST_SORT test-mergesort: use DEFINE_LIST_SORT_DEBUG mergesort: add macros for typed sort of linked lists mergesort: tighten merge loop mergesort: unify ranks loops
| * fetch-pack: use DEFINE_LIST_SORTRené Scharfe2022-07-171-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Build a static typed ref sorting function using DEFINE_LIST_SORT along with a typed comparison function near its only two callers instead of having an exported version that calls llist_mergesort(). This gets rid of the next pointer accessor functions and their calling overhead at the cost of a slightly increased object text size. Before: __TEXT __DATA __OBJC others dec hex 23231 389 0 113689 137309 2185d fetch-pack.o 29158 80 0 146864 176102 2afe6 remote.o With this patch: __TEXT __DATA __OBJC others dec hex 23591 389 0 117759 141739 229ab fetch-pack.o 29070 80 0 145718 174868 2ab14 remote.o Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | remote: handle negative refspecs in git remote showJacob Keller2022-06-171-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, the git remote show command will query data from remotes to show data about what might be done on a future git fetch. This process currently does not handle negative refspecs. This can be confusing, because the show command will list refs as if they would be fetched. For example if the fetch refspec "^refs/heads/pr/*", it still displays the following: * remote jdk19 Fetch URL: git@github.com:openjdk/jdk19.git Push URL: git@github.com:openjdk/jdk19.git HEAD branch: master Remote branches: master tracked pr/1 new (next fetch will store in remotes/jdk19) pr/2 new (next fetch will store in remotes/jdk19) pr/3 new (next fetch will store in remotes/jdk19) Local ref configured for 'git push': master pushes to master (fast-forwardable) Fix this by adding an additional check inside of get_ref_states. If a ref matches one of the negative refspecs, mark it as skipped instead of marking it as new or tracked. With this change, we now report remote branches that are skipped due to negative refspecs properly: * remote jdk19 Fetch URL: git@github.com:openjdk/jdk19.git Push URL: git@github.com:openjdk/jdk19.git HEAD branch: master Remote branches: master tracked pr/1 skipped pr/2 skipped pr/3 skipped Local ref configured for 'git push': master pushes to master (fast-forwardable) By showing the refs as skipped, it helps clarify that these references won't actually be fetched. This does not properly handle refs going stale due to a newly added negative refspec. In addition, git remote prune doesn't handle that negative refspec case either. Fixing that requires digging into get_stale_heads and handling the case of a ref which exists on the remote but is omitted due to a negative refspec locally. Add a new test case which covers the functionality above, as well as a new expected failure indicating the poor overlap with stale refs. Reported-by: Pavel Rappo <pavel.rappo@gmail.com> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | remote: allow relative_url() to return an absolute urlDerrick Stolee2022-05-161-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | When the 'url' parameter was absolute, the previous implementation would concatenate 'remote_url' with 'url'. Instead, we want to return 'url' in this case. The documentation now discusses what happens when supplying two absolute URLs. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | remote: move relative_url()Derrick Stolee2022-05-161-0/+31
|/ | | | | | | | | | | | | | | | | | | | | This method was initially written in 63e95beb0 (submodule: port resolve_relative_url from shell to C, 2016-05-15). As we will need similar functionality in the bundle URI feature, extract this to be available in remote.h. The code is almost exactly the same, except for the following trivial differences: * Fix whitespace and wrapping issues with the prototype and argument lists. * Let's call starts_with_dot_{,dot_}slash_native() instead of the functionally identical "starts_with_dot_{,dot_}slash()" wrappers "builtin/submodule--helper.c". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* remote: die if branch is not found in repositoryGlen Choo2021-11-181-3/+2
| | | | | | | | | | | | | | | | | | | | | | | In a subsequent commit, we would like external-facing functions to be able to accept "struct repository" and "struct branch" as a pair. This is useful for functions like pushremote_for_branch(), which need to take values from the remote_state and branch, even if branch == NULL. However, a caller may supply an unrelated repository and branch, which is not supported behavior. To prevent misuse, add a die_on_missing_branch() helper function that dies if a given branch is not from a given repository. Speed up the existence check by replacing the branches list with a branches_hash hashmap. Like read_config(), die_on_missing_branch() is only called from non-static functions; static functions are less prone to misuse because they have strong conventions for keeping remote_state and branch in sync. Signed-off-by: Glen Choo <chooglen@google.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* remote: use remote_state parameter internallyGlen Choo2021-11-181-0/+2
| | | | | | | | | | | | | | | | Without changing external-facing functions, replace the_repository->remote_state internally by adding a struct remote_state parameter. As a result, external-facing functions are still tied to the_repository, but most static functions no longer reference the_repository->remote_state. The exceptions are those that are used in a way that depends on external-facing functions e.g. the callbacks to remote_get_1(). Signed-off-by: Glen Choo <chooglen@google.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* remote: move static variables into per-repository structGlen Choo2021-11-181-0/+34
| | | | | | | | | | | | | | | | remote.c does not works with non-the_repository because it stores its state as static variables. To support non-the_repository, we can use a per-repository struct for the remotes subsystem. Prepare for this change by defining a struct remote_state that holds the remotes subsystem state and move the static variables of remote.c into the_repository->remote_state. This introduces no behavioral or API changes. Signed-off-by: Glen Choo <chooglen@google.com> Reviewed-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'jt/clone-unborn-head'Junio C Hamano2021-02-171-1/+3
|\ | | | | | | | | | | | | | | | | | | | | | | "git clone" tries to locally check out the branch pointed at by HEAD of the remote repository after it is done, but the protocol did not convey the information necessary to do so when copying an empty repository. The protocol v2 learned how to do so. * jt/clone-unborn-head: clone: respect remote unborn HEAD connect, transport: encapsulate arg in struct ls-refs: report unborn targets of symrefs
| * connect, transport: encapsulate arg in structJonathan Tan2021-02-051-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In a future patch we plan to return the name of an unborn current branch from deep in the callchain to a caller via a new pointer parameter that points at a variable in the caller when the caller calls get_remote_refs() and transport_get_remote_refs(). In preparation for that, encapsulate the existing ref_prefixes parameter into a struct. The aforementioned unborn current branch will go into this new struct in the future patch. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | fetch: extract writing to FETCH_HEADPatrick Steinhardt2021-01-121-1/+1
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | When performing a fetch with the default `--write-fetch-head` option, we write all updated references to FETCH_HEAD while the updates are performed. Given that updates are not performed atomically, it means that we we write to FETCH_HEAD even if some or all of the reference updates fail. Given that we simply update FETCH_HEAD ad-hoc with each reference, the logic is completely contained in `store_update_refs` and thus quite hard to extend. This can already be seen by the way we skip writing to the FETCH_HEAD: instead of having a conditional which simply skips writing, we instead open "/dev/null" and needlessly write all updates there. We are about to extend git-fetch(1) to accept an `--atomic` flag which will make the fetch an all-or-nothing operation with regards to the reference updates. This will also require us to make the updates to FETCH_HEAD an all-or-nothing operation, but as explained doing so is not easy with the current layout. This commit thus refactors the wa we write to FETCH_HEAD and pulls out the logic to open, append to, commit and close the file. While this may seem rather over-the top at first, pulling out this logic will make it a lot easier to update the code in a subsequent commit. It also allows us to easily skip writing completely in case `--no-write-fetch-head` was passed. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'sk/force-if-includes'Junio C Hamano2020-10-271-1/+11
|\ | | | | | | | | | | | | | | | | | | | | | | | | "git push --force-with-lease[=<ref>]" can easily be misused to lose commits unless the user takes good care of their own "git fetch". A new option "--force-if-includes" attempts to ensure that what is being force-pushed was created after examining the commit at the tip of the remote ref that is about to be force-replaced. * sk/force-if-includes: t, doc: update tests, reference for "--force-if-includes" push: parse and set flag for "--force-if-includes" push: add reflog check for "--force-if-includes"
| * push: add reflog check for "--force-if-includes"Srinidhi Kaushik2020-10-031-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a check to verify if the remote-tracking ref of the local branch is reachable from one of its "reflog" entries. The check iterates through the local ref's reflog to see if there is an entry for the remote-tracking ref and collecting any commits that are seen, into a list; the iteration stops if an entry in the reflog matches the remote ref or if the entry timestamp is older the latest entry of the remote ref's "reflog". If there wasn't an entry found for the remote ref, "in_merge_bases_many()" is called to check if it is reachable from the list of collected commits. When a local branch that is based on a remote ref, has been rewound and is to be force pushed on the remote, "--force-if-includes" runs a check that ensures any updates to the remote-tracking ref that may have happened (by push from another repository) in-between the time of the last update to the local branch (via "git-pull", for instance) and right before the time of push, have been integrated locally before allowing a forced update. If the new option is passed without specifying "--force-with-lease", or specified along with "--force-with-lease=<refname>:<expect>" it is a "no-op". Signed-off-by: Srinidhi Kaushik <shrinidhi.kaushik@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'jk/refspecs-negative'Junio C Hamano2020-10-051-1/+8
|\ \ | |/ |/| | | | | | | | | "git fetch" and "git push" support negative refspecs. * jk/refspecs-negative: refspec: add support for negative refspecs
| * refspec: add support for negative refspecsJacob Keller2020-09-301-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Both fetch and push support pattern refspecs which allow fetching or pushing references that match a specific pattern. Because these patterns are globs, they have somewhat limited ability to express more complex situations. For example, suppose you wish to fetch all branches from a remote except for a specific one. To allow this, you must setup a set of refspecs which match only the branches you want. Because refspecs are either explicit name matches, or simple globs, many patterns cannot be expressed. Add support for a new type of refspec, referred to as "negative" refspecs. These are prefixed with a '^' and mean "exclude any ref matching this refspec". They can only have one "side" which always refers to the source. During a fetch, this refers to the name of the ref on the remote. During a push, this refers to the name of the ref on the local side. With negative refspecs, users can express more complex patterns. For example: git fetch origin refs/heads/*:refs/remotes/origin/* ^refs/heads/dontwant will fetch all branches on origin into remotes/origin, but will exclude fetching the branch named dontwant. Refspecs today are commutative, meaning that order doesn't expressly matter. Rather than forcing an implied order, negative refspecs will always be applied last. That is, in order to match, a ref must match at least one positive refspec, and match none of the negative refspecs. This is similar to how negative pathspecs work. Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'jx/proc-receive-hook'Junio C Hamano2020-09-251-0/+9
|\ \ | |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git receive-pack" that accepts requests by "git push" learned to outsource most of the ref updates to the new "proc-receive" hook. * jx/proc-receive-hook: doc: add documentation for the proc-receive hook transport: parse report options for tracking refs t5411: test updates of remote-tracking branches receive-pack: new config receive.procReceiveRefs doc: add document for capability report-status-v2 New capability "report-status-v2" for git-push receive-pack: feed report options to post-receive receive-pack: add new proc-receive hook t5411: add basic test cases for proc-receive hook transport: not report a non-head push as a branch
| * New capability "report-status-v2" for git-pushJiang Xin2020-08-271-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The new introduced "proc-receive" hook may handle a command for a pseudo-reference with a zero-old as its old-oid, while the hook may create or update a reference with different name, different new-oid, and different old-oid (the reference may exist already with a non-zero old-oid). Current "report-status" protocol cannot report the status for such reference rewrite. Add new capability "report-status-v2" and new report protocol which is not backward compatible for report of git-push. If a user pushes to a pseudo-reference "refs/for/master/topic", and "receive-pack" creates two new references "refs/changes/23/123/1" and "refs/changes/24/124/1", for client without the knowledge of "report-status-v2", "receive-pack" will only send "ok/ng" directives in the report, such as: ok ref/for/master/topic But for client which has the knowledge of "report-status-v2", "receive-pack" will use "option" directives to report more attributes for the reference given by the above "ok/ng" directive. ok refs/for/master/topic option refname refs/changes/23/123/1 option new-oid <new-oid> ok refs/for/master/topic option refname refs/changes/24/124/1 option new-oid <new-oid> The client will report two new created references to the end user. Suggested-by: Junio C Hamano <gitster@pobox.com> Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * receive-pack: add new proc-receive hookJiang Xin2020-08-271-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git calls an internal `execute_commands` function to handle commands sent from client to `git-receive-pack`. Regardless of what references the user pushes, git creates or updates the corresponding references if the user has write-permission. A contributor who has no write-permission, cannot push to the repository directly. So, the contributor has to write commits to an alternate location, and sends pull request by emails or by other ways. We call this workflow as a distributed workflow. It would be more convenient to work in a centralized workflow like what Gerrit provided for some cases. For example, a read-only user who cannot push to a branch directly can run the following `git push` command to push commits to a pseudo reference (has a prefix "refs/for/", not "refs/heads/") to create a code review. git push origin \ HEAD:refs/for/<branch-name>/<session> The `<branch-name>` in the above example can be as simple as "master", or a more complicated branch name like "foo/bar". The `<session>` in the above example command can be the local branch name of the client side, such as "my/topic". We cannot implement a centralized workflow elegantly by using "pre-receive" + "post-receive", because Git will call the internal function "execute_commands" to create references (even the special pseudo reference) between these two hooks. Even though we can delete the temporarily created pseudo reference via the "post-receive" hook, having a temporary reference is not safe for concurrent pushes. So, add a filter and a new handler to support this kind of workflow. The filter will check the prefix of the reference name, and if the command has a special reference name, the filter will turn a specific field (`run_proc_receive`) on for the command. Commands with this filed turned on will be executed by a new handler (a hook named "proc-receive") instead of the internal `execute_commands` function. We can use this "proc-receive" command to create pull requests or send emails for code review. Suggested by Junio, this "proc-receive" hook reads the commands, push-options (optional), and send result using a protocol in pkt-line format. In the following example, the letter "S" stands for "receive-pack" and letter "H" stands for the hook. # Version and features negotiation. S: PKT-LINE(version=1\0push-options atomic...) S: flush-pkt H: PKT-LINE(version=1\0push-options...) H: flush-pkt # Send commands from server to the hook. S: PKT-LINE(<old-oid> <new-oid> <ref>) S: ... ... S: flush-pkt # Send push-options only if the 'push-options' feature is enabled. S: PKT-LINE(push-option) S: ... ... S: flush-pkt # Receive result from the hook. # OK, run this command successfully. H: PKT-LINE(ok <ref>) # NO, I reject it. H: PKT-LINE(ng <ref> <reason>) # Fall through, let 'receive-pack' to execute it. H: PKT-LINE(ok <ref>) H: PKT-LINE(option fall-through) # OK, but has an alternate reference. The alternate reference name # and other status can be given in options H: PKT-LINE(ok <ref>) H: PKT-LINE(option refname <refname>) H: PKT-LINE(option old-oid <old-oid>) H: PKT-LINE(option new-oid <new-oid>) H: PKT-LINE(option forced-update) H: ... ... H: flush-pkt After receiving a command, the hook will execute the command, and may create/update different reference. For example, a command for a pseudo reference "refs/for/master/topic" may create/update different reference such as "refs/pull/123/head". The alternate reference name and other status are given in option lines. The list of commands returned from "proc-receive" will replace the relevant commands that are sent from user to "receive-pack", and "receive-pack" will continue to run the "execute_commands" function and other routines. Finally, the result of the execution of these commands will be reported to end user. The reporting function from "receive-pack" to "send-pack" will be extended in latter commit just like what the "proc-receive" hook reports to "receive-pack". Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | argv-array: rename to strvecJeff King2020-07-281-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The name "argv-array" isn't very good, because it describes what the data type can be used for (program argument arrays), not what it actually is (a dynamically-growing string array that maintains a NULL-terminator invariant). This leads to people being hesitant to use it for other cases where it would actually be a good fit. The existing name is also clunky to use. It's overly long, and the name often leads to saying things like "argv.argv" (i.e., the field names overlap with variable names, since they're describing the use, not the type). Let's give it a more neutral name. I settled on "strvec" because "vector" is the name for a dynamic array type in many programming languages. "strarray" would work, too, but it's longer and a bit more awkward to say (and don't we all say these things in our mind as we type them?). A more extreme direction would be a generic data structure which stores a NULL-terminated of _any_ type. That would be easy to do with void pointers, but we'd lose some type safety for the existing cases. Plus it raises questions about memory allocation and ownership. So I limited myself here to changing names only, and not semantics. If we do find a use for that more generic data type, we could perhaps implement it at a lower level and then provide type-safe wrappers around it for strings. But that can come later. This patch does the minimum to convert the struct and function names in the header and implementation, leaving a few things for follow-on patches: - files retain their original names for now - struct field names are retained for now - there's a preprocessor compat layer that lets most users remain the same for now. The exception is headers which made a manual forward declaration of the struct. I've converted them (and their dependent function declarations) here. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | stateless-connect: send response end packetDenton Liu2020-05-241-1/+2
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, remote-curl acts as a proxy and blindly forwards packets between an HTTP server and fetch-pack. In the case of a stateless RPC connection where the connection is terminated before the transaction is complete, remote-curl will blindly forward the packets before waiting on more input from fetch-pack. Meanwhile, fetch-pack will read the transaction and continue reading, expecting more input to continue the transaction. This results in a deadlock between the two processes. This can be seen in the following command which does not terminate: $ git -c protocol.version=2 clone https://github.com/git/git.git --shallow-since=20151012 Cloning into 'git'... whereas the v1 version does terminate as expected: $ git -c protocol.version=1 clone https://github.com/git/git.git --shallow-since=20151012 Cloning into 'git'... fatal: the remote end hung up unexpectedly Instead of blindly forwarding packets, make remote-curl insert a response end packet after proxying the responses from the remote server when using stateless_connect(). On the RPC client side, ensure that each response ends as described. A separate control packet is chosen because we need to be able to differentiate between what the remote server sends and remote-curl's control packets. By ensuring in the remote-curl code that a server cannot send response end packets, we prevent a malicious server from being able to perform a denial of service attack in which they spoof a response end packet and cause the described deadlock to happen. Reported-by: Force Charlie <charlieio@outlook.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* remote: drop "explicit" parameter from remote_ref_for_branch()Jeff King2020-03-031-2/+1
| | | | | | | | | | | | | | | | | | | | | | | Commit 9700fae5ee (for-each-ref: let upstream/push report the remote ref name, 2017-11-07) added a remote_ref_for_branch() helper, which is modeled after remote_for_branch(). This includes providing an "explicit" out-parameter that tells the caller whether the remote was configured by the user, or whether we picked a default name like "origin". But unlike remote names, there is no default name when the user didn't configure one. The only way the "explicit" parameter is used by the caller is to use the value returned from the helper when it is set, and use an empty string otherwise, ignoring the returned value from the helper. Let's drop the "explicit" out-parameter, and return NULL when the returned value from the helper should be ignored, to simplify the function interface. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Damien Robert <damien.olivier.robert+git@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* remote: move doc to remote.h and refspec.hHeba Waly2019-11-181-3/+54
| | | | | | | | | | | | | | | | | Move the documentation from Documentation/technical/api-remote.txt to remote.h and refspec.h as it's easier for the developers to find the usage information beside the code instead of looking for it in another doc file. N.B. The doc for both push and fetch members of the remote struct aren't moved because they are out of date, as the members were changed from arrays of rspecs to struct refspec 2 years ago. Also documentation/technical/api-remote.txt is removed because the information it has is now redundant and it'll be hard to keep it up to date and synchronized with the documentation in the header file. Signed-off-by: Heba Waly <heba.waly@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* hashmap_entry: remove first member requirement from docsEric Wong2019-10-071-1/+1
| | | | | | | | | Comments stating that "struct hashmap_entry" must be the first member in a struct are no longer valid. Suggested-by: Phillip Wood <phillip.wood123@gmail.com> Signed-off-by: Eric Wong <e@80x24.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'dl/no-extern-in-func-decl'Junio C Hamano2019-05-131-12/+12
|\ | | | | | | | | | | | | | | | | | | Mechanically and systematically drop "extern" from function declarlation. * dl/no-extern-in-func-decl: *.[ch]: manually align parameter lists *.[ch]: remove extern from function declarations using sed *.[ch]: remove extern from function declarations using spatch
| * *.[ch]: manually align parameter listsDenton Liu2019-05-051-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | In previous patches, extern was mechanically removed from function declarations without care to formatting, causing parameter lists to be misaligned. Manually format changed sections such that the parameter lists should be realigned. Viewing this patch with 'git diff -w' should produce no output. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * *.[ch]: remove extern from function declarations using spatchDenton Liu2019-05-051-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There has been a push to remove extern from function declarations. Remove some instances of "extern" for function declarations which are caught by Coccinelle. Note that Coccinelle has some difficulty with processing functions with `__attribute__` or varargs so some `extern` declarations are left behind to be dealt with in a future patch. This was the Coccinelle patch used: @@ type T; identifier f; @@ - extern T f(...); and it was run with: $ git ls-files \*.{c,h} | grep -v ^compat/ | xargs spatch --sp-file contrib/coccinelle/noextern.cocci --in-place Files under `compat/` are intentionally excluded as some are directly copied from external sources and we should avoid churning them as much as possible. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'dr/ref-filter-push-track-fix'Junio C Hamano2019-05-091-1/+2
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | %(push:track) token used in the --format option to "git for-each-ref" and friends was not showing the right branch, which has been fixed. * dr/ref-filter-push-track-fix: ref-filter: use correct branch for %(push:track)
| * | ref-filter: use correct branch for %(push:track)Damien Robert2019-04-181-1/+2
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In ref-filter.c, when processing the atom %(push:track), the ahead/behind values are computed using `stat_tracking_info` which refers to the upstream branch. Fix that by introducing a new flag `for_push` in `stat_tracking_info` in remote.c, which does the same thing but for the push branch. Update the few callers of `stat_tracking_info` to handle this flag. This ensure that whenever we use this function in the future, we are careful to specify is this should apply to the upstream or the push branch. This bug was not detected in t/t6300-for-each-ref.sh because in the test for push:track, both the upstream and the push branches were behind by 1 from the local branch. Change the test so that the upstream branch is behind by 1 while the push branch is ahead by 1. This allows us to test that %(push:track) refers to the correct branch. This changes the expected value of some following tests (by introducing new references), so update them too. Signed-off-by: Damien Robert <damien.olivier.robert+git@gmail.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | remote.c: make singular free_ref() publicJeff King2019-04-151-1/+3
|/ | | | | | | | | | | | | We provide a free_refs() function to free a list, but there's no easy way for a caller to free a single ref. Let's make our singular free_ref() function public. Since its name is so similar to the list-freeing free_refs(), and because both of those functions have the same signature, it might be easy to accidentally use the wrong one. Let's call the singular version the more verbose "free_one_ref()" to distinguish it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'ds/reachable'Junio C Hamano2018-09-171-1/+0
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code for computing history reachability has been shuffled, obtained a bunch of new tests to cover them, and then being improved. * ds/reachable: commit-reach: correct accidental #include of C file commit-reach: use can_all_from_reach commit-reach: make can_all_from_reach... linear commit-reach: replace ref_newer logic test-reach: test commit_contains test-reach: test can_all_from_reach_with_flags test-reach: test reduce_heads test-reach: test get_merge_bases_many test-reach: test is_descendant_of test-reach: test in_merge_bases test-reach: create new test tool for ref_newer commit-reach: move can_all_from_reach_with_flags upload-pack: generalize commit date cutoff upload-pack: refactor ok_to_give_up() upload-pack: make reachable() more generic commit-reach: move commit_contains from ref-filter commit-reach: move ref_newer from remote.c commit.h: remove method declarations commit-reach: move walk methods from commit.c
| * commit-reach: move ref_newer from remote.cDerrick Stolee2018-07-201-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are several commit walks in the codebase. Group them together into a new commit-reach.c file and corresponding header. After we group these walks into one place, we can reduce duplicate logic by calling equivalent methods. The ref_newer() method is used by 'git push -f' to check if a force-push is necessary. By making the method public, we make it possible to test the method directly without setting up an envieronment where a 'git push' call makes sense. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Add missing includes and forward declarationsElijah Newren2018-08-151-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | I looped over the toplevel header files, creating a temporary two-line C program for each consisting of #include "git-compat-util.h" #include $HEADER This patch is the result of manually fixing errors in compiling those tiny programs. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | fetch-pack: implement ref-in-wantBrandon Williams2018-06-281-0/+1
|/ | | | | | | | | | | | | | | | | | | | Implement ref-in-want on the client side so that when a server supports the "ref-in-want" feature, a client will send "want-ref" lines for each reference the client wants to fetch. This feature allows clients to tolerate inconsistencies that exist when a remote repository's refs change during the course of negotiation. This allows a client to request to request a particular ref without specifying the OID of the ref. This means that instead of hitting an error when a ref no longer points at the OID it did at the beginning of negotiation, negotiation can continue and the value of that ref will be sent at the termination of negotiation, just before a packfile is sent. More information on the ref-in-want feature can be found in Documentation/technical/protocol-v2.txt. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'bw/ref-prefix-for-configured-refspec'Junio C Hamano2018-05-301-39/+11
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "git fetch $there $refspec" that talks over protocol v2 can take advantage of server-side ref filtering; the code has been extended so that this mechanism triggers also when fetching with configured refspec. * bw/ref-prefix-for-configured-refspec: (38 commits) fetch: generate ref-prefixes when using a configured refspec refspec: consolidate ref-prefix generation logic submodule: convert push_unpushed_submodules to take a struct refspec remote: convert check_push_refs to take a struct refspec remote: convert match_push_refs to take a struct refspec http-push: store refspecs in a struct refspec transport: remove transport_verify_remote_names send-pack: store refspecs in a struct refspec transport: convert transport_push to take a struct refspec push: convert to use struct refspec push: check for errors earlier remote: convert match_explicit_refs to take a struct refspec remote: convert get_ref_match to take a struct refspec remote: convert query_refspecs to take a struct refspec remote: convert apply_refspecs to take a struct refspec remote: convert get_stale_heads to take a struct refspec fetch: convert prune_refs to take a struct refspec fetch: convert get_ref_map to take a struct refspec fetch: convert do_fetch to take a struct refspec refspec: remove the deprecated functions ...
| * remote: convert check_push_refs to take a struct refspecBrandon Williams2018-05-181-1/+1
| | | | | | | | | | | | | | | | Convert 'check_push_refs()' to take a 'struct refspec' as a parameter instead of an array of 'const char *'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * remote: convert match_push_refs to take a struct refspecBrandon Williams2018-05-181-1/+1
| | | | | | | | | | | | | | | | Convert 'match_push_refs()' to take a 'struct refspec' as a parameter instead of an array of 'const char *'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * remote: convert query_refspecs to take a struct refspecBrandon Williams2018-05-181-1/+1
| | | | | | | | | | | | | | | | Convert 'query_refspecs()' to take a 'struct refspec' as a parameter instead of a list of 'struct refspec_item'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * remote: convert apply_refspecs to take a struct refspecBrandon Williams2018-05-181-2/+1
| | | | | | | | | | | | | | | | Convert 'apply_refspecs()' to take a 'struct refspec' as a parameter instead of a list of 'struct refspec_item'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * remote: convert get_stale_heads to take a struct refspecBrandon Williams2018-05-181-1/+1
| | | | | | | | | | | | | | | | Convert 'get_stale_heads()' to take a 'struct refspec' as a parameter instead of a list of 'struct refspec_item'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * remote: remove add_prune_tags_to_fetch_refspecBrandon Williams2018-05-181-2/+0
| | | | | | | | | | | | | | | | Remove 'add_prune_tags_to_fetch_refspec()' function and instead have the only caller directly add the tag refspec using 'refspec_append()'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * remote: convert fetch refspecs to struct refspecBrandon Williams2018-05-181-4/+1
| | | | | | | | | | | | | | | | Convert the set of fetch refspecs stored in 'struct remote' to use 'struct refspec'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * remote: convert push refspecs to struct refspecBrandon Williams2018-05-181-4/+2
| | | | | | | | | | | | | | | | Convert the set of push refspecs stored in 'struct remote' to use 'struct refspec'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * refspec: rename struct refspec to struct refspec_itemBrandon Williams2018-05-181-8/+8
| | | | | | | | | | | | | | | | | | | | In preparation for introducing an abstraction around a collection of refspecs (much like how a 'struct pathspec' is a collection of 'struct pathspec_item's) rename the existing 'struct refspec' to 'struct refspec_item'. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * refspec: move refspec parsing logic into its own fileBrandon Williams2018-05-181-20/+0
| | | | | | | | | | | | | | | | In preparation for performing a refactor on refspec related code, move the refspec parsing logic into its own file. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * Merge branch 'bw/protocol-v2'Junio C Hamano2018-05-081-2/+9
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The beginning of the next-gen transfer protocol. * bw/protocol-v2: (35 commits) remote-curl: don't request v2 when pushing remote-curl: implement stateless-connect command http: eliminate "# service" line when using protocol v2 http: don't always add Git-Protocol header http: allow providing extra headers for http requests remote-curl: store the protocol version the server responded with remote-curl: create copy of the service name pkt-line: add packet_buf_write_len function transport-helper: introduce stateless-connect transport-helper: refactor process_connect_service transport-helper: remove name parameter connect: don't request v2 when pushing connect: refactor git_connect to only get the protocol version once fetch-pack: support shallow requests fetch-pack: perform a fetch using v2 upload-pack: introduce fetch server command push: pass ref prefixes when pushing fetch: pass ref prefixes when fetching ls-remote: pass ref prefixes when requesting a remote's refs transport: convert transport_get_remote_refs to take a list of ref prefixes ...
* | | ls-remote: send server options when using protocol v2Brandon Williams2018-04-241-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Teach ls-remote to optionally accept server options by specifying them on the cmdline via '-o' or '--server-option'. These server options are sent to the remote end when querying for the remote end's refs using protocol version 2. If communicating using a protocol other than v2 the provided options are ignored and not sent to the remote end. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>