| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
| |
Currently, we perform a request to Gitlab Rails that proxies
the request to primary
However, it causes timeouts on big pushes and consumes large
amount of memory. We can perform an HTTP request directly
from Gitlab Shell instead and stream the response to the user
|
| |
|
|
|
|
|
|
|
|
| |
All the implementations of DNS discovery were done in this epic:
https://gitlab.com/groups/gitlab-org/-/epics/8971. Gitaly allows clients
to configure DNS discovery via dial option. This MR adds the exposed
dial options to client connection creation in Gitlab-shell.
Issue: https://gitlab.com/gitlab-org/gitaly/-/issues/4722
Changelog: added
|
| |
|
|
|
|
|
|
|
|
|
| |
This commit also excludes gitlab-shell from dependencies:
Gitaly specifies Gitlab Shell as a dependency as well in order
to use gitlabnet client to perform API endpoints to Gitlab Rails.
As a result, Gitlab Shell requires Gitaly -> Gitaly requires an
older version of Gitlab Shell -> that version requires an older
version of Gitlab Shell, etc. Let's use exclude to break the
chain earlier
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
While gitlab-shell currently has a major version of v14, the module path
it exposes is not using that major version like it is required by the Go
standard. This makes it impossible for dependents to import gitlab-shell
as a dependency without using a commit as version.
Fix this by changing the module path of gitlab-shell to instead be
`gitlab.com/gitlab-org/gitlab-shell/v14` and adjust all imports
accordingly.
Changelog: fixed
|
| |
|
|
|
|
|
| |
Before this change, the GitLab internal API could use a boolean
response field to indicate whether gitlab-shell should make
sidechannel connections go Gitaly. We now ignore that response field
and always use sidechannel connections.
|
| |
|
|
|
|
|
|
| |
When gitlab-sshd has been introduced we've started running our
own SSH server. In this case we're able to cache and reuse
Gitaly connections and Registry.
It helps to reduce memory usage.
|
| |
|
|
|
|
|
| |
If the GitLab API returns an allowed response with use_sidechannel set
to true, gitlab-shell will establish a sidechannel connection and use
SSHUploadPackWithSidechannel instead of SSHUploadPack. This is an
efficiency improvement.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Logrus buffers its output internally, which makes these tests fail
intermittently. They're also not a good example to follow generally.
We now have acceptance tests that exercise this functionality so I'm
pretty relaxed about losing the expectations. However, we can test
them by inspecting the server-received metadata too, so there's no loss
of coverage here.
The move from logrus to labkit for logging also makes these tests hard
to justify keeping.
|
| |
|
|
|
|
|
|
|
|
|
| |
Gitaly project now properly respects module release flow
and includes a module suffix in the package name. It requires
to re-write all non-suffixed imports with suffixed of a specific
version of tha module. With proper module versioning we don't
need to use a 'replace' directive to point to specific commit
and can use semantic versioning for the gitaly dependency.
Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/3177
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, opentracing (if configured) was initialized late in the
gitlab-shell process's lifespan, coming just before making a gRPC
call to Gitaly.
By moving the opentracing initialization to be at process startup, we
make it available for the whole process lifecycle, which is very useful
to gitlab-sshd, as it means we'll only call tracing.Initialize() once
on process startup, rather than once per SSH connection.
To get this working, we need to introduce a context to gitlab-sshd.
This carries the client/service name, but also carries an initial
correlation ID. The main outcome of this is that all calls to the
authorized_keys endpoint from a given gitlab-sshd process will now
share a correlation ID. I don't have a strong opinion about this either
way.
Changelog: fixed
|
| |
|
|
|
|
|
|
|
| |
Without these changes, Gitaly calls would not be linked to a parent
context. This means that they would have an unassociated correlationID,
and Gitaly RPC calls would not be cancel()ed by parent context
cancellation.
Changelog: fixed
|
| |
|
|
|
| |
In this case we don't need to propagate cleanup
function. It simplifies the code.
|
| |
|
|
|
|
|
| |
Refactors introspection of execution environment to rely on
per-connection state (`gitlab-shell`) or per request (`gitlab-sshd`)
Relates to https://gitlab.com/gitlab-org/gitlab-shell/-/issues/496
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During a SSH receive-pack request (e.g. `git push`), gitlab-shell was
incorrectly using the user returned by the `/internal/allowed` API
endpoint to make an SSHReceivePack RPC call. This caused a number of
problems with deploy keys with write access:
1. Keys that were generated by a blocked user would be denied the
ability to write.
2. Keys that were generated by user that did not have write access to
the project would also be denied.
GitLab 12.4 removed the Ruby implementation of gitlab-shell in favor of
the Golang implementation, and these implementations worked slightly
differently. In
https://gitlab.com/gitlab-org/gitlab-shell/blob/v10.1.0/lib/gitlab_shell.rb,
the Ruby implementation would always use `@who` (e.g. `key-123`), but in
gitlab-shell v10.2.0 the Go implementation would always use the user
from the API response.
Reads did not have this issue because the user/deploy key is never
passed to Gitaly for additional permission checks. Writes need this
information for the pre-receive to check access to protected branches,
push rules, etc.
Relates to https://gitlab.com/gitlab-org/gitlab-shell/-/issues/479
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, gitlab-shell did not pass a context through the application.
Correlation IDs were generated down the call stack instead of passed
around from the start execution.
This has several potential downsides:
1. It's easier for programming mistakes to be made in future that lead
to multiple correlation IDs being generated for a single request.
2. Correlation IDs cannot be passed in from upstream requests
3. Other advantages of context passing, such as distributed tracing is
not possible.
This commit changes the behavior:
1. Extract the correlation ID from the environment at the start of
the application.
2. If no correlation ID exists, generate a random one.
3. Pass the correlation ID to the GitLabNet API requests.
This change also enables other clients of GitLabNet (e.g. Gitaly) to
pass along the correlation ID in the internal API requests
(https://gitlab.com/gitlab-org/gitaly/-/issues/2725).
Fixes https://gitlab.com/gitlab-org/gitlab-shell/-/issues/474
|
| |
|
|
|
| |
This will make it easier to tie an SSH access request to Rails API and
Gitaly requests.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Right now when a client such as gitlab-shell calls the
`/api/v4/internal/allowed` API, the response only tells the client what
user has been granted access, and it's impossible to tell which deploy
key/token was used in the authentication request.
This commit adds logs for the following when available:
1. `gl_key_type` (e.g. `deploy_key` or `key`)
2. `gl_key_id`
These fields make it possible for admins to identify the exact record
that was used to authenticate the user.
API changes in the `/internal/allowed` endpoint in
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37289 are needed
to support this.
Relates to https://gitlab.com/gitlab-org/gitlab-shell/-/issues/203
|
| |
|
|
|
|
|
|
| |
Admins may want to know what client IP originated the request. This
commit adds a `remote_ip` field to the log that extracts the IP address
from the `SSH_CONNECTION` environment variable.
Closes https://gitlab.com/gitlab-org/gitlab-shell/-/issues/199
|
| |
|
|
|
|
|
|
|
| |
logrus fires a Goroutine to write logs, so the tests could fail if they
checked the event queue before the logrus have fired. Since there isn't
an easy way to flush all outstanding hooks, we just retry every 100 ms
for up to a second for log to arrive in the queue.
Closes https://gitlab.com/gitlab-org/gitlab-shell/-/issues/450
|
| |
|
|
|
|
|
|
| |
Calling logrus hook.LastEntry() can lead to race conditions. Use
AllEntries instead:
https://github.com/sirupsen/logrus/blob/60c74ad9be0d874af0ab0daef6ab07c5c5911f0d/hooks/test/test.go#L77
Closes https://gitlab.com/gitlab-org/gitlab-shell/-/issues/450
|
| | |
|
| | |
|
| |
|
|
| |
We'll reuse this module for uploadpack in the future
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
This restores the previous Ruby gitlab-shell behavior.
|
| |
|
|
|
|
|
|
|
|
|
| |
Both git-upload-pack and git-receive-pack services inspect the
GIT_PROTOCOL environment transferred via SSH in order to decide which
protocols are supported by a given client. Currently, we don't use the
environment variable at all, though, but instead forward the GitProtocol
field of the access verification response.
Improve this by passing on the GIT_PROTOCOL environment variable
provided by the client as-is.
|
| | |
|
| |
|
|
|
| |
Add support for Gitaly feature flags
See merge request gitlab-org/gitlab-shell!351
|
| | |
|
| | |
|
| |
|
