1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
|
---
stage: Create
group: Source Code
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# GitLab Shell
[](https://gitlab.com/gitlab-org/gitlab-shell/-/pipelines?ref=main) [](https://gitlab.com/gitlab-org/gitlab-shell/-/pipelines?ref=main) [](https://codeclimate.com/github/gitlabhq/gitlab-shell)
GitLab Shell handles Git SSH sessions for GitLab and modifies the list of authorized keys.
GitLab Shell is not a Unix shell nor a replacement for Bash or Zsh.
GitLab supports Git LFS authentication through SSH.
## Requirements
GitLab Shell is written in Go, and needs a Go compiler to build. It still requires
Ruby to build and test, but not to run.
GitLab Shell runs on `port 22` on an Omnibus installation. To use a regular SSH
service, configure it on an alternative port.
Download and install the current version of Go from [golang.org](https://golang.org/dl/).
We follow the [Golang Release Policy](https://golang.org/doc/devel/release.html#policy)
and support:
- The current stable version.
- The previous two major versions.
## How GitLab Shell works
When you access the GitLab server over SSH, GitLab Shell then:
1. Limits you to predefined Git commands (`git push`, `git pull`, `git fetch`).
1. Calls the GitLab Rails API to check if you are authorized, and what Gitaly server your repository is on.
1. Copies data back and forth between the SSH client and the Gitaly server.
If you access a GitLab server over HTTP(S) you end up in [`gitlab-workhorse`](../workhorse/index.md).
### `git pull` over SSH
```mermaid
graph LR
A[Git pull] --> |via SSH| B[gitlab-shell]
B -->|API call| C[gitlab-rails<br>authorization]
C -->|accept or decline| D[Gitaly session]
```
### `git push` over SSH
The `git push` command is not performed until after `gitlab-rails` accepts the push:
```mermaid
graph LR
subgraph User initiates
A[Git push] -->|via SSH| B[gitlab-shell]
end
subgraph Gitaly
B -->|establish Gitaly session| C[gitlab-shell pre-receive hook]
C -->|API auth call| D[Gitlab-rails]
D --> E[accept or decline push]
end
```
[Full feature list](features.md)
### Modifies `authorized_keys`
GitLab Shell modifies the `authorized_keys` file on the client machine.
## Contribute to GitLab Shell
To contribute to GitLab Shell:
1. Check if GitLab API access, and Redis via the internal API, can be reached: `make check`
1. Compile the `gitlab-shell` binaries, placing them into `bin/`: `make compile`
1. Run `make install` to build the `gitlab-shell` binaries and install. them onto the file system.
The default location is `/usr/local`. To change it, set the `PREFIX` and `DESTDIR` environment variables.
1. To install GitLab from source on a single machine, run `make setup`.
It compiles the GitLab Shell binaries, and ensures that various paths on the file system
exist with the correct permissions. Do not run this command unless your installation method
documentation instructs you to.
For more information, see
[CONTRIBUTING.md](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/CONTRIBUTING.md).
### Run tests
When contributing, run tests:
1. Run tests with `bundle install` and `make test`.
1. Run Gofmt: `make verify`
1. Run both test and verify (the default Makefile target):
```shell
bundle install
make validate
```
1. If needed, configure Gitaly.
### Configure Gitaly for local testing
Some tests need a Gitaly server. The
[`docker-compose.yml`](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/docker-compose.yml) file runs Gitaly on port 8075.
To tell the tests where Gitaly is, set `GITALY_CONNECTION_INFO`:
```plaintext
export GITALY_CONNECTION_INFO='{"address": "tcp://localhost:8075", "storage": "default"}'
make test
```
If no `GITALY_CONNECTION_INFO` is set, the test suite still runs, but any
tests requiring Gitaly are skipped. The tests always run in the CI environment.
## Rate limiting
GitLab Shell performs rate-limiting by user account and project for Git operations.
GitLab Shell accepts Git operation requests and then makes a call to the Rails
rate-limiter, backed by Redis. If the `user + project` exceeds the rate limit,
then GitLab Shell then drop further connection requests for that `user + project`.
The rate-limiter is applied at the Git command (plumbing) level. Each command has
a rate limit of 600 per minute. For example, `git push` has 600 per minute, and
`git pull` has another 600 per minute.
Because they are using the same plumbing command, `git-upload-pack`, `git pull`,
and `git clone` are in effect the same command for the purposes of rate-limiting.
Gitaly also has a rate-limiter in place, but calls are never made to Gitaly if
the rate limit is exceeded in GitLab Shell (Rails).
## Logs in GitLab Shell
In general, you can determine the structure, but not content, of a GitLab Shell
or `gitlab-sshd` session by inspecting the logs. Some guidelines:
- We use [`gitlab.com/gitlab-org/labkit/log`](https://pkg.go.dev/gitlab.com/gitlab-org/labkit/log)
for logging.
- Always include a correlation ID.
- Log messages should be invariant and unique. Include accessory information in
fields, using `log.WithField`, `log.WithFields`, or `log.WithError`.
- Log both success cases and error cases.
- Logging too much is better than not logging enough. If a message seems too
verbose, consider reducing the log level before removing the message.
## GitLab SaaS
A diagram of the flow of `gitlab-shell` on GitLab.com:
```mermaid
graph LR
a2 --> b2
a2 --> b3
a2 --> b4
b2 --> c1
b3 --> c1
b4 --> c1
c2 --> d1
c2 --> d2
c2 --> d3
d1 --> e1
d2 --> e1
d3 --> e1
a1[Cloudflare] --> a2[TCP<br/> load balancer]
e1[Git]
subgraph HAProxy Fleet
b2[HAProxy]
b3[HAProxy]
b4[HAProxy]
end
subgraph GKE
c1[Internal TCP<br/> load balancer<br/>port 2222] --> c2[GitLab-shell<br/> pods]
end
subgraph Gitaly
d1[Gitaly]
d2[Gitaly]
d3[Gitaly]
end
```
## GitLab Shell architecture
```mermaid
sequenceDiagram
participant Git on client
participant SSH server
participant AuthorizedKeysCommand
participant GitLab Shell
participant Rails
participant Gitaly
participant Git on server
Note left of Git on client: git fetch
Git on client->>+SSH server: ssh git fetch-pack request
SSH server->>+AuthorizedKeysCommand: gitlab-shell-authorized-keys-check git AAAA...
AuthorizedKeysCommand->>+Rails: GET /internal/api/authorized_keys?key=AAAA...
Note right of Rails: Lookup key ID
Rails-->>-AuthorizedKeysCommand: 200 OK, command="gitlab-shell upload-pack key_id=1"
AuthorizedKeysCommand-->>-SSH server: command="gitlab-shell upload-pack key_id=1"
SSH server->>+GitLab Shell: gitlab-shell upload-pack key_id=1
GitLab Shell->>+Rails: GET /internal/api/allowed?action=upload_pack&key_id=1
Note right of Rails: Auth check
Rails-->>-GitLab Shell: 200 OK, { gitaly: ... }
GitLab Shell->>+Gitaly: SSHService.SSHUploadPack request
Gitaly->>+Git on server: git upload-pack request
Note over Git on client,Git on server: Bidirectional communication between Git client and server
Git on server-->>-Gitaly: git upload-pack response
Gitaly -->>-GitLab Shell: SSHService.SSHUploadPack response
GitLab Shell-->>-SSH server: gitlab-shell upload-pack response
SSH server-->>-Git on client: ssh git fetch-pack response
```
## Related topics
- [LICENSE](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/LICENSE).
- [PROCESS.md](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/PROCESS.md)
- [Using the GitLab Shell chart](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/)
|
