summaryrefslogtreecommitdiff
path: root/spec
Commit message (Collapse)AuthorAgeFilesLines
* feat: make retryable http default clientSteve Azzopardi2023-01-301-6/+1
| | | | | | | | | | | | | | | | | | | | | | What --- Make the retryableHTTP client introduced in https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/703 the default HTTP client. Why --- In https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979#note_1254964426 we've seen a 99% error reduction on `git` commands from `gitlab-shell` when the retryableHTTP client is used. This has been running in production for over 2 weeks in `us-east1-b` and 5 days fleet-wide so we should be confident that this client works as expected. Reference: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979 Signed-off-by: Steve Azzopardi <sazzopardi@gitlab.com>
* feat: put retryablehttp.Client behind feature flagSteve Azzopardi2023-01-121-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | What --- - Update the `client.HttpClient` fields to have `http.Client` and `retryablehttp.Client`, one of them will be `nil` depending on the feature flag toggle. - Create new method `newRetryableRequest` which will create a `retryablehttp.Request` and use that if the `FF_GITLAB_SHELL_RETRYABLE_HTTP` feature flag is turned on. - Add checks for `FF_GITLAB_SHELL_RETRYABLE_HTTP` everywhere we use the http client to use the `retryablehttp.Client` or the default `http.Client` - New job `tests-integration-retryableHttp` to run the integraiton tests with the new retryablehttp client. We didn't update go tests because some assertions are different and will break table driven tests. Why --- As discussed in https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/703#note_1229645097 we want to put the client behind a feature flag, not just the retry logic. This does bring extra risk for accessing a `nil` field but there should be checks everytime we access `RetryableHTTP` and `HTTPClient`. Reference: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979 Signed-off-by: Steve Azzopardi <sazzopardi@gitlab.com>
* feat: retry on errorSteve Azzopardi2023-01-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | What --- Change the default `HTTP.Client` to `github.com/hashicorp/go-retryablehttp.Client` to get automatic retries and exponential backoff. We retry the request 2 times resulting in 3 attempts of sending the request, the min retry wait is 1 second, and the maximum is 15 seconds. Hide the retry logic behind a temporary feature flag `FF_GITLAB_SHELL_RETRYABLE_HTTP` to easily roll this out in GitLab.com. When we verify that this works as expected we will remove `FF_GITLAB_SHELL_RETRYABLE_HTTP` and have the retry logic as the default logic. Why --- In https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979 users end up seeing the following errors when trying to `git-clone(1)` a repository locally on in CI. ```shell remote: =============================== remote: remote: ERROR: Internal API unreachable remote: remote: ================================ ``` When we look at the application logs we see the following error: ```json { "err": "http://gitlab-webservice-git.gitlab.svc:8181/api/v4/internal/allowed": dial tcp 10.69.184.120:8181: connect: connection refused", "msg": "Internal API unreachable"} ``` In https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979#note_1222670120 we've correlated these `connection refused` errors with infrastructure events that remove the git pods that are hosting `gitlab-webservice-git` service. We could try to make the underlying infrastructure more reactive to these changes as suggested in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979#note_1225164944 but we can still end up serving bad requests. Implementing retry logic for 5xx or other errors would allow users to still be able to `git-clone(1)` reposirories, although it being slower. This is espically important during CI runs so users don't have to retry jobs themselves. Reference: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979 Closes: https://gitlab.com/gitlab-org/gitlab-shell/-/issues/604 Signed-off-by: Steve Azzopardi <sazzopardi@gitlab.com>
* Simplify 2FA Push auth processingIgor Drozdov2022-07-203-163/+125
| | | | Use a single channel to handle both Push Auth and OTP results
* Implement Push Auth support for 2FA verificationkmcknight2022-07-182-2/+84
| | | | | | | | | | | | | | | | | When `2fa_verify` command is executed: - A user is asked to enter OTP - A blocking call for push auth is performed Then: - If the push auth request fails, the user is still able to enter OTP - If OTP is invalid, the `2fa_verify` command ends the execution - If OTP is valid or push auth request succeeded, then the user is successfully authenticated - If 30 seconds passed while no OTP or Push have been provided, then the `2fa_verify` command ends the execution
* Fix the Geo SSH push proxy hangingValery Sizov2021-07-011-2/+2
| | | | | | | | | | Geo SSH proxy push currently impossible when the only action that happens is branch removal. This fix works in a way that it waits for flush packet from git and then checks pkt lines to determine is pack data is expected. The thing is that git doesnt send pack data when only branch removal happens. Explanation is in https://gitlab.com/gitlab-org/gitlab/-/issues/330494
* Fix a failing specNick Thomas2021-06-291-1/+1
| | | | | | | When the shell environment includes SSH_CONNECTION, one spec fails as the way we're stubbing the environment to the subprocess doesn't wipe out the pre-existing variable. This commit changes how we do it so the spec passes even in this environment.
* Remove session duration information from output of 2fa_verify command504-remove-the-session-duration-information-from-the-output-of-2fa_verify-commandManoj M J2021-01-291-1/+1
| | | | | This change removes session duration information from output of 2fa_verify command
* Add 2fa_verify commandImre Farkas2020-12-101-0/+81
|
* Add support obtaining personal access tokens via SSHTaylan Develioglu2020-08-171-0/+119
| | | | | | | | | | | | | | | | | | | | | | | | Implements the feature requested in gitlab-org/gitlab#19672 This requires the internal api counterpart in gitlab-org/gitlab!36302 to be merged first. It can be used as follows: ``` censored@censored-VirtualBox:~/git/gitlab$ ssh git@gitlab-2004 personal_access_token remote: remote: ======================================================================== remote: remote: Usage: personal_access_token <name> <scope1[,scope2,...]> [ttl_days] remote: remote: ======================================================================== remote: censored@censored-VirtualBox:~/git/gitlab$ ssh git@gitlab-2004 personal_access_token newtoken read_api,read_repository 30 Token: aAY1G3YPeemECgUvxuXY Scopes: read_api,read_repository Expires: 2020-08-07 ```
* Geo Pull custom action supportAsh McKenzie2020-04-171-0/+118
|
* Rename Geo proxy routes to be accurateAsh McKenzie2020-04-171-3/+3
|
* Log internal HTTP requestsStan Hu2020-03-101-0/+3
| | | | | | | This restores the previous behavior of logging the success and failures of internal HTTP requests. Part of https://gitlab.com/gitlab-org/gitlab/issues/207916
* Remove support for Custom data.info_message210-remove-action-custom-inform_client-once-12-3-has-been-releasedAsh McKenzie2019-12-241-7/+1
|
* More consistent console messages (Ruby)Ash McKenzie2019-10-232-9/+20
|
* Remove VCR, webmock, other unnecessary gems and supportNick Thomas2019-10-1546-2035/+2
|
* Simplify integration specsNick Thomas2019-10-155-130/+32
|
* Remove dead Ruby codeNick Thomas2019-10-1517-1795/+3
|
* Remove gitlab-keys scriptPatrick Bajao2019-09-041-312/+0
|
* Implement AuthorizedPrincipals command181-authorized-principals-check-goPatrick Bajao2019-08-091-1/+1
| | | | | | Build this command when `Executable` name is `gitlab-shell-authorized-principals-check`. Feature flag is the same name.
* Implement AuthorizedKeys command181-authorized-keys-check-goPatrick Bajao2019-08-081-1/+1
| | | | | | Build this command when `Executable` name is `gitlab-shell-authorized-keys-check`. Feature flag is the same name.
* Add Executable struct181-migrate-gitlab-shell-checks-fallbackPatrick Bajao2019-08-021-2/+2
| | | | | | | | | | This struct is responsible for determining the name and root dir of the executable. The `RootDir` property will be used to find the config. The `Name` property will be used to determine what `Command` and `CommandArgs` to be built.
* Support falling back to ruby version of checkersPatrick Bajao2019-07-292-32/+144
| | | | | | | | | | | | | | Rename the ruby scripts to have `-ruby` suffix and add a symlink for both to `./gitlab-shell`. The executable name will be used to determine how args will be parsed. For now, we only parse the arguments for gitlab-shell commands. If the executable is `gitlab-shell-authorized-keys-check` or `gitlab-shell-authorized-principals-check`, it'll always fallback to the ruby version. Ruby specs test the ruby script, the fallback from go to ruby and go implementation of both (still pending).
* Go implementation for LFS authenticateIgor2019-06-061-0/+146
|
* Go implementation for git-receive-packIgor2019-05-311-0/+134
|
* Add go tag to two-factor-recovery specsIgor Drozdov2019-05-221-1/+1
|
* Upgrade to Ruby 2.6.3sh-upgrade-to-ruby-2.6.3Stan Hu2019-05-061-1/+1
| | | | | | Restores some of the changes in https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/257, but still targets Rubocop for 2.3.
* Actually print matched keys in list_keysChristian Ratzenhofer2019-04-301-8/+12
|
* Print console messages to $stderr if present9217-warn-on-git-fetch-over-ssh-if-the-secondary-is-lagging-the-primaryAsh McKenzie2019-04-041-1/+16
| | | | Using ConsoleHelper::write_stderr
* New ConsoleHelper moduleAsh McKenzie2019-04-041-0/+38
| | | | | .write_stderr .format_for_stderr
* Remove hooks, they belong to Gitaly nowJacob Vosmaer2019-04-033-656/+0
|
* Merge branch 'display-post-receive-warnings' into 'master'Nick Thomas2019-04-011-6/+20
|\ | | | | | | | | Display post receive warnings from Rails See merge request gitlab-org/gitlab-shell!288
| * Rename print_broadbast method to be genericdisplay-post-receive-warningsLuke Duncalfe2019-04-011-1/+1
| | | | | | | | | | | | print_warnings is now using that method in order to print a large formatted warning message, so renaming the method to be less tied to the broadcast message functionality.
| * Fixing whitespace issues in specLuke Duncalfe2019-03-291-8/+8
| |
| * Allow GitlabPostReceive to print warningsLuke Duncalfe2019-03-281-0/+14
| | | | | | | | | | | | Warnings are printed using the same method that prints broadcast messages. This ensures that whatever warnings are returned will be displayed correctly.
* | Merge branch 'add-pry' into 'master'Rémy Coutable2019-04-011-0/+1
|\ \ | | | | | | | | | | | | Add pry gem to allow easy debugging See merge request gitlab-org/gitlab-shell!289
| * | Add pry gem for easy debuggingadd-pryLuke Duncalfe2019-03-291-0/+1
| |/
* | Provide go implementation for 2fa_recovery_codes commandIgor2019-03-214-94/+203
| |
* | Detect user based on key, username or idBob Van Landuyt2019-03-141-8/+25
|/ | | | | | | | | | | | | | | | | | This allows gitlab-shell to be called with an argument of the format `key-123` or `username-name`. When called in this way, `gitlab-shell` will call the GitLab internal API. If the API responds with user information, it will print a welcome message including the username. If the API responds with a successful but empty response, gitlab-shell will print a welcome message for an anonymous user. If the API response includes an error message in JSON, this message will be printed to stderr. If the API call fails, an error message including the status code will be printed to stderr.
* Add support for using gl_project_pathsh-use-gl-project-pathStan Hu2019-02-165-2/+13
| | | | | | This will enable us to track the human-readable project path with SSH access in gitlab-shell. Currently the Gitaly logs will only show the hashed storage and gl_repository path (e.g. project-1234).
* Provide expires_in for LFS if available57353-git-push-fails-on-large-lfs-files-where-the-push-take-a-long-timeAsh McKenzie2019-02-141-14/+34
|
* Don't fall back to ruby for non SSH connectionsbvl-feature-flag-commandsBob Van Landuyt2019-01-151-20/+24
| | | | | When SSH_CONNECTION is not set, we don't fall back to ruby, but instead fail directly in go writing the error to stderr.
* Allow enabling gitlab-shell "discover"-featureBob Van Landuyt2019-01-151-52/+82
| | | | | | | | | | | | | | | | | | | This adds the possibility to enable features for GitLab shell. The first feature being recognized is "Discover": It's the command that is executed when running `ssh git@gitlab.example.com` and is called without a command. The gitlab key id or username is already parsed from the command line arguments. Currently we only support communicating with GitLab-rails using unix sockets. So features will not be enabled if the GitLab-url is using a different protocol. The url for this read from the config yaml. Pending ruby-specs have been added for the gitlab-shell command. Refactor to have separate command packages
* Merge branch 'fix-rspec-warnings' into 'master'Douwe Maan2018-12-201-9/+15
|\ | | | | | | | | Fix rspec deprecation warning See merge request gitlab-org/gitlab-shell!267
| * Fix rspec deprecation warningsfix-rspec-warningsJacob Vosmaer2018-12-201-9/+15
| |
* | Fix spec failure with Ruby 2.5Jacob Vosmaer2018-12-201-1/+1
|/
* Use 'push_options' everywhere instead of 'push_opts'Jonathon Reinhart2018-12-132-5/+5
|
* Pass push options to GitLab /internal/post_receive APIJonathon Reinhart2018-12-132-3/+5
| | | | | | | | | | push_opts[] is named as such so that it becomes an array in the GitLab internal Grape::API params hash. If it were named push_opts, only the final (scalar) push_opts parameter would be present. Also update specs to include push_opts parameter: - gitlab_post_receive_spec - gitlab_net_spec for the internal API change
* Add lib/hooks_utils.rbJonathon Reinhart2018-12-131-0/+22
| | | | | | | This module is responsible for converting the GIT_PUSH_OPTION_* environment variables into an array. See https://gitlab.com/gitlab-org/gitlab-ce/issues/18667
* Remove unused lib/names_helper.rbJonathon Reinhart2018-12-131-12/+0
| | | | As of 7eb45672b7, NamesHelper is no longer used.
View Lorry Controller Status