diff options
author | Igor Drozdov <idrozdov@gitlab.com> | 2023-01-17 16:52:13 +0000 |
---|---|---|
committer | Igor Drozdov <idrozdov@gitlab.com> | 2023-01-17 16:52:13 +0000 |
commit | ad172bf0c9688238bc014d2ded2b181ae0b6e45a (patch) | |
tree | 82b9d983f48e3e97596a4405753f50d6ce9a1a2a | |
parent | 1e791e09f760064c41216535f210d935ee6435d7 (diff) | |
parent | ffab3310ced5284183a6c2f8b05e786fc4e8c3c1 (diff) | |
download | gitlab-shell-ad172bf0c9688238bc014d2ded2b181ae0b6e45a.tar.gz |
Merge branch '596-aqualls-truncate-and-redirect' into 'main'
docs: Truncate pages, point users to GitLab repo
See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/705
Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Torsten Linz <tlinz@gitlab.com>
Approved-by: Jerry Seto <jseto@gitlab.com>
Approved-by: Sean Carroll <scarroll@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Co-authored-by: Amy Qualls <aqualls@gitlab.com>
-rw-r--r-- | PROCESS.md | 45 | ||||
-rw-r--r-- | README.md | 108 | ||||
-rw-r--r-- | doc/architecture.md | 37 | ||||
-rw-r--r-- | doc/beginners_guide.md | 94 | ||||
-rw-r--r-- | doc/features.md | 87 | ||||
-rw-r--r-- | doc/gitlab-sshd.md | 36 |
6 files changed, 12 insertions, 395 deletions
@@ -1,44 +1,3 @@ -## Releasing a new version +# GitLab Shell process -GitLab Shell is versioned by git tags, and the version used by the Rails -application is stored in -[`GITLAB_SHELL_VERSION`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/GITLAB_SHELL_VERSION). - -For each version, there is a raw version and a tag version: - -- The **raw version** is the version number. For instance, `15.2.8`. -- The **tag version** is the raw version prefixed with `v`. For instance, `v15.2.8`. - -To release a new version of GitLab Shell and have that version available to the -Rails application: - -1. Create a merge request to update the [`CHANGELOG`](CHANGELOG) with the - **tag version** and the [`VERSION`](VERSION) file with the **raw version**. -2. Ask a maintainer to review and merge the merge request. If you're already a - maintainer, second maintainer review is not required. -3. Add a new git tag with the **tag version**. -4. Update `GITLAB_SHELL_VERSION` in the Rails application to the **raw - version**. (Note: this can be done as a separate MR to that, or in and MR - that will make use of the latest GitLab Shell changes.) - -## Security releases - -GitLab Shell is included in the packages we create for GitLab, and each version of GitLab specifies the version of GitLab Shell it uses in the `GITLAB_SHELL_VERSION` file, so security fixes in GitLab Shell are tightly coupled to the [GitLab security release](https://about.gitlab.com/handbook/engineering/workflow/#security-issues) workflow. - -For a security fix in GitLab Shell, two sets of merge requests are required: - -* The fix itself, in the `gitlab-org/security/gitlab-shell` repository and its backports to the previous versions of GitLab Shell -* Merge requests to change the versions of GitLab Shell included in the GitLab security release, in the `gitlab-org/security/gitlab` repository - -The first step could be to create a merge request with a fix targeting `main` in `gitlab-org/security/gitlab-shell`. When the merge request is approved by maintainers, backports targeting previous 3 versions of GitLab Shell must be created. The stable branches for those versions may not exist, feel free to ask a maintainer to create ones. The stable branches must be created out of the GitLab Shell tags/version used by the 3 previous GitLab releases. In order to find out the GitLab Shell version that is used on a particular GitLab stable release, the following steps may be helpful: - -```shell -git fetch security 13-9-stable-ee -git show refs/remotes/security/13-9-stable-ee:GITLAB_SHELL_VERSION -``` - -These steps display the version that is used by `13.9` version of GitLab. - -Close to the GitLab security release, a maintainer should merge the fix and backports and cut all the necessary GitLab Shell versions. This allows bumping the `GITLAB_SHELL_VERSION` for `gitlab-org/security/gitlab`. The GitLab merge request will be handled by the general GitLab security release process. - -Once the security release is done, a GitLab Shell maintainer is responsible for syncing tags and `main` to the `gitlab-org/gitlab-shell` repository. +This page [has moved into the `gitlab` repository](https://docs.gitlab.com/ee/development/gitlab_shell/process.html). @@ -4,114 +4,10 @@ group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# GitLab Shell +# GitLab Shell documentation has moved [![pipeline status](https://gitlab.com/gitlab-org/gitlab-shell/badges/main/pipeline.svg)](https://gitlab.com/gitlab-org/gitlab-shell/-/pipelines?ref=main) [![coverage report](https://gitlab.com/gitlab-org/gitlab-shell/badges/main/coverage.svg)](https://gitlab.com/gitlab-org/gitlab-shell/-/pipelines?ref=main) [![Code Climate](https://codeclimate.com/github/gitlabhq/gitlab-shell.svg)](https://codeclimate.com/github/gitlabhq/gitlab-shell) -GitLab Shell handles git SSH sessions for GitLab and modifies the list of authorized keys. -GitLab Shell is not a Unix shell nor a replacement for Bash or Zsh. - -GitLab supports Git LFS authentication through SSH. - -## Requirements - -GitLab Shell is written in Go, and needs a Go compiler to build. It still requires -Ruby to build and test, but not to run. - -GitLab Shell runs on `port 22` on an Omnibus installation. To use a regular SSH -service, configure it on an alternative port. - -Download and install the current version of Go from [golang.org](https://golang.org/dl/) -We follow the [Golang Release Policy](https://golang.org/doc/devel/release.html#policy) -of supporting the current stable version and the previous two major versions. - -## How GitLab Shell works - -When you access the GitLab server over SSH then GitLab Shell will: - -1. Limit you to predefined git commands (`git push`, `git pull`, `git fetch`). -1. Call the GitLab Rails API to check if you are authorized, and what Gitaly server your repository is on -1. Copy data back and forth between the SSH client and the Gitaly server - -If you access a GitLab server over HTTP(S) you end up in [gitlab-workhorse](https://gitlab.com/gitlab-org/gitlab/tree/master/workhorse). - -### `git pull` over SSH - -1. git pull over SSH -> gitlab-shell -> API call to gitlab-rails (Authorization) -> accept or decline -> establish Gitaly session - -### `git push` over SSH - -1. git push over SSH -> gitlab-shell (git command is not executed yet) -> establish Gitaly session -> (in Gitaly) gitlab-shell pre-receive hook -> API call to gitlab-rails (authorization) -> accept or decline push - -[Full feature list](doc/features.md) - -### Modifies `authorized_keys` - -GitLab Shell modifies the `authorized_keys` file on the client machine. - -## Rate Limiting - -GitLab Shell performs rate-limiting by user account and project for git operations. GitLab Shell accepts git operation requests and then makes a call to the Rails rate-limiter (backed by Redis). If the `user + project` exceeds the rate limit then GitLab Shell will then drop further connection requests for that `user + project`. - -The rate-limiter is applied at the git command (plumbing) level. Each command has a rate limit of 600/minute. For example, `git push` has 600/minute and `git pull` has another 600/minute. - -Because they are using the same plumbing command `git-upload-pack`, `git pull` and `git clone` are in effect the same command for the purposes of rate-limiting. - -There is also a rate-limiter in place in Gitaly, but the calls will never be made to Gitaly if the rate limit is exceeded in Gitlab Shell (Rails). - -## GitLab SaaS - -A diagram of the flow of `gitlab-shell` on GitLab.com: - -```mermaid -graph LR - a2 --> b2 - a2 --> b3 - a2 --> b4 - b2 --> c1 - b3 --> c1 - b4 --> c1 - c2 --> d1 - c2 --> d2 - c2 --> d3 - d1 --> e1 - d2 --> e1 - d3 --> e1 - a1[Cloudflare] --> a2[TCP<br/> load balancer] - e1[Git] - - subgraph HAProxy Fleet - b2[HAProxy] - b3[HAProxy] - b4[HAProxy] - end - - subgraph GKE - c1[Internal TCP<br/> load balancer<br/>port 2222] --> c2[GitLab-shell<br/> pods] - end - - subgraph Gitaly - d1[Gitaly] - d2[Gitaly] - d3[Gitaly] - end -``` - -## Releasing - -See [PROCESS.md](./PROCESS.md) - -## Contributing - -- See [CONTRIBUTING.md](./CONTRIBUTING.md). -- See the [beginner's guide](doc/beginners_guide.md). - -## License - -See [LICENSE](./LICENSE). - -## Related topics - -- [Using the GitLab Shell chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/#using-the-gitlab-shell-chart) +The documentation for GitLab Shell [has moved into the `gitlab` repository](https://docs.gitlab.com/ee/development/gitlab_shell/). diff --git a/doc/architecture.md b/doc/architecture.md index ff60273..d413e7d 100644 --- a/doc/architecture.md +++ b/doc/architecture.md @@ -1,37 +1,4 @@ ---- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - # GitLab Shell architecture -```mermaid -sequenceDiagram - participant Git on client - participant SSH server - participant AuthorizedKeysCommand - participant GitLab Shell - participant Rails - participant Gitaly - participant Git on server - - Note left of Git on client: git fetch - Git on client->>+SSH server: ssh git fetch-pack request - SSH server->>+AuthorizedKeysCommand: gitlab-shell-authorized-keys-check git AAAA... - AuthorizedKeysCommand->>+Rails: GET /internal/api/authorized_keys?key=AAAA... - Note right of Rails: Lookup key ID - Rails-->>-AuthorizedKeysCommand: 200 OK, command="gitlab-shell upload-pack key_id=1" - AuthorizedKeysCommand-->>-SSH server: command="gitlab-shell upload-pack key_id=1" - SSH server->>+GitLab Shell: gitlab-shell upload-pack key_id=1 - GitLab Shell->>+Rails: GET /internal/api/allowed?action=upload_pack&key_id=1 - Note right of Rails: Auth check - Rails-->>-GitLab Shell: 200 OK, { gitaly: ... } - GitLab Shell->>+Gitaly: SSHService.SSHUploadPack request - Gitaly->>+Git on server: git upload-pack request - Note over Git on client,Git on server: Bidirectional communication between Git client and server - Git on server-->>-Gitaly: git upload-pack response - Gitaly -->>-GitLab Shell: SSHService.SSHUploadPack response - GitLab Shell-->>-SSH server: gitlab-shell upload-pack response - SSH server-->>-Git on client: ssh git fetch-pack response -``` +The diagram on this page +[has moved into the `gitlab` repository](https://docs.gitlab.com/ee/development/gitlab_shell/#gitlab-shell-architecture). diff --git a/doc/beginners_guide.md b/doc/beginners_guide.md index f75ce73..1eb4862 100644 --- a/doc/beginners_guide.md +++ b/doc/beginners_guide.md @@ -1,94 +1,4 @@ ---- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments ---- - # Beginner's guide to GitLab Shell contributions -## Check - -Checks if GitLab API access and Redis via internal API can be reached: - -```shell -make check -``` - -## Compile - -Builds the `gitlab-shell` binaries, placing them into `bin/`. - -```shell -make compile -``` - -## Install - -Builds the `gitlab-shell` binaries and installs them onto the file system. The -default location is `/usr/local`, but you can change the location by setting the `PREFIX` -and `DESTDIR` environment variables. - -```shell -make install -``` - -## Setup - -This command is intended for use when installing GitLab from source on a single -machine. It compiles the GitLab Shell binaries, and ensures that -various paths on the file system exist with the correct permissions. Do not run -this command unless your installation method documentation instructs you to. - -```shell -make setup -``` - -## Testing - -Run tests: - -```shell -bundle install -make test -``` - -Run Gofmt: - -```shell -make verify -``` - -Run both test and verify (the default Makefile target): - -```shell -bundle install -make validate -``` - -## Gitaly - -Some tests need a Gitaly server. The -[`docker-compose.yml`](../docker-compose.yml) file runs Gitaly on port 8075. -To tell the tests where Gitaly is, set `GITALY_CONNECTION_INFO`: - -```plaintext -export GITALY_CONNECTION_INFO='{"address": "tcp://localhost:8075", "storage": "default"}' -make test -``` - -If no `GITALY_CONNECTION_INFO` is set, the test suite still runs, but any -tests requiring Gitaly are skipped. The tests always run in the CI environment. - -## Logging Guidelines - -In general, you can determine the structure, but not content, of a GitLab Shell -or `gitlab-sshd` session by inspecting the logs. Some guidelines: - -- We use [`gitlab.com/gitlab-org/labkit/log`](https://pkg.go.dev/gitlab.com/gitlab-org/labkit/log) - for logging. -- Always include a correlation ID. -- Log messages should be invariant and unique. Include accessory information in - fields, using `log.WithField`, `log.WithFields`, or `log.WithError`. -- Log both success cases and error cases. -- Logging too much is better than not logging enough. If a message seems too - verbose, consider reducing the log level before removing the message. +The information on this page +[has moved into the `gitlab` repository](https://docs.gitlab.com/ee/development/gitlab_shell/#contribute-to-gitlab-shell). diff --git a/doc/features.md b/doc/features.md index ae2a7eb..cdd0ac4 100644 --- a/doc/features.md +++ b/doc/features.md @@ -1,87 +1,4 @@ ---- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments ---- - # Feature list -## Discover - -Allows users to identify themselves on an instance via SSH. The command helps to -confirm quickly whether a user has SSH access to the instance: - -```shell -ssh git@<hostname> - -PTY allocation request failed on channel 0 -Welcome to GitLab, @username! -Connection to staging.gitlab.com closed. -``` - -When permission is denied, it returns: - -```shell -ssh git@<hostname> -git@<hostname>: Permission denied (publickey). -``` - -## Git operations - -GitLab Shell provides support for Git operations over SSH by processing -`git-upload-pack`, `git-receive-pack` and `git-upload-archive` SSH commands. -It limits the set of commands to predefined Git commands: - -- `git archive` -- `git clone` -- `git pull` -- `git push` - -## Generate new 2FA recovery codes - -Enables users to -[generate new 2FA recovery codes](https://docs.gitlab.com/ee/user/profile/account/two_factor_authentication.html#generate-new-recovery-codes-using-ssh): - -```plaintext -ssh git@<hostname> 2fa_recovery_codes -Are you sure you want to generate new two-factor recovery codes? -Any existing recovery codes you saved will be invalidated. (yes/no) -yes - -Your two-factor authentication recovery codes are: -... -``` - -## Verify 2FA OTP - -Allows users to verify their -[2FA one-time password (OTP)](https://docs.gitlab.com/ee/security/two_factor_authentication.html#2fa-for-git-over-ssh-operations): - -```shell -ssh git@<hostname> 2fa_verify -OTP: 347419 - -OTP validation failed. -``` - -## LFS authentication - -Enables users to generate credentials for LFS authentication: - -```shell -ssh git@<hostname> git-lfs-authenticate <project-path> <upload/download> - -{"header":{"Authorization":"Basic ..."},"href":"https://gitlab.com/user/project.git/info/lfs","expires_in":7200} -``` - -## Personal access token - -Enables users to use personal access tokens via SSH: - -```shell -ssh git@<hostname> personal_access_token <name> <scope1[,scope2,...]> [ttl_days] - -Token: glpat-... -Scopes: api -Expires: 2022-02-05 -``` +This page +[has moved into the `gitlab` repository](https://docs.gitlab.com/ee/development/gitlab_shell/features.html). diff --git a/doc/gitlab-sshd.md b/doc/gitlab-sshd.md index be97468..423237e 100644 --- a/doc/gitlab-sshd.md +++ b/doc/gitlab-sshd.md @@ -1,36 +1,4 @@ ---- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - # gitlab-sshd -`gitlab-sshd` is a binary in [`gitlab-shell`](https://gitlab.com/gitlab-org/gitlab-shell) -which runs as a persistent SSH daemon. It will replace `OpenSSH` on GitLab SaaS, -and eventually other cloud-native environments. Instead of running an `sshd` process, -we run a `gitlab-sshd` process that does the same job, in a more focused manner: - -```mermaid -sequenceDiagram - participant Git on client - participant GitLab SSHD - participant Rails - participant Gitaly - participant Git on server - - Note left of Git on client: git fetch - Git on client->>+GitLab SSHD: ssh git fetch-pack request - GitLab SSHD->>+Rails: GET /internal/api/authorized_keys?key=AAAA... - Note right of Rails: Lookup key ID - Rails-->>-GitLab SSHD: 200 OK, command="gitlab-shell upload-pack key_id=1" - GitLab SSHD->>+Rails: GET /internal/api/allowed?action=upload_pack&key_id=1 - Note right of Rails: Auth check - Rails-->>-GitLab SSHD: 200 OK, { gitaly: ... } - GitLab SSHD->>+Gitaly: SSHService.SSHUploadPack request - Gitaly->>+Git on server: git upload-pack request - Note over Git on client,Git on server: Bidirectional communication between Git client and server - Git on server-->>-Gitaly: git upload-pack response - Gitaly -->>-GitLab SSHD: SSHService.SSHUploadPack response - GitLab SSHD-->>-Git on client: ssh git fetch-pack response -``` +This page +[has moved into the `gitlab` repository](https://docs.gitlab.com/ee/development/gitlab_shell/gitlab_sshd.html). |