diff options
Diffstat (limited to 'doc/administration/operations')
4 files changed, 83 insertions, 32 deletions
diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md index fd04efe8473..75f2ad5ed26 100644 --- a/doc/administration/operations/extra_sidekiq_processes.md +++ b/doc/administration/operations/extra_sidekiq_processes.md @@ -354,7 +354,7 @@ file is written, but this can be changed by passing the `--pidfile` option to ``` Keep in mind that the PID file contains the PID of the `sidekiq-cluster` -command and not the PID(s) of the started Sidekiq processes. +command and not the PIDs of the started Sidekiq processes. ### Environment diff --git a/doc/administration/operations/extra_sidekiq_routing.md b/doc/administration/operations/extra_sidekiq_routing.md index bb8eb184302..cd3a53b7c63 100644 --- a/doc/administration/operations/extra_sidekiq_routing.md +++ b/doc/administration/operations/extra_sidekiq_routing.md @@ -88,8 +88,8 @@ components: > [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/261) in GitLab 13.1 (`tags`). -Queue matching query works upon the worker attributes, described in [Sidekiq -style guide](../../development/sidekiq_style_guide.md). We support querying +Queue matching query works upon the worker attributes, described in +[Sidekiq style guide](../../development/sidekiq/index.md). We support querying based on a subset of worker attributes: - `feature_category` - the [GitLab feature diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index 477a11dd899..a3240a6041b 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -1,6 +1,6 @@ --- -stage: Enablement -group: Distribution +stage: Create +group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -14,7 +14,7 @@ drop-in replacement. Regular SSH operations become slow as the number of users grows because OpenSSH searches for a key to authorize a user via a linear search. In the worst case, -such as when the user is not authorized to access GitLab, OpenSSH will scan the +such as when the user is not authorized to access GitLab, OpenSSH scans the entire file to search for a key. This can take significant time and disk I/O, which delays users attempting to push or pull to a repository. Making matters worse, if users add or remove keys frequently, the operating system may @@ -33,21 +33,22 @@ able to accept a fingerprint. Check the version of OpenSSH on your server with ` Unlike [Cloud Native GitLab](https://docs.gitlab.com/charts/), Omnibus GitLab by default manages an `authorized_keys` file that is located in the -`git` user's home directory. For most installations, this will be located under -`/var/opt/gitlab/.ssh/authorized_keys`, but you can use the following command to locate the `authorized_keys` on your system: +`git` user's home directory. For most installations, this file is located under +`/var/opt/gitlab/.ssh/authorized_keys`, but you can use the following command to +locate the `authorized_keys` on your system: ```shell getent passwd git | cut -d: -f6 | awk '{print $1"/.ssh/authorized_keys"}' ``` The `authorized_keys` file contains all the public SSH keys for users allowed to access GitLab. However, to maintain a -single source of truth, [Geo](../geo/index.md) needs to be configured to perform SSH fingerprint +single source of truth, [Geo](../geo/index.md) must be configured to perform SSH fingerprint lookups via database lookup. As part of [setting up Geo](../geo/index.md#setup-instructions), you are required to follow the steps outlined below for both the primary and -secondary nodes, but the `Write to "authorized keys" file` checkbox -only needs to be unchecked on the primary node since it is reflected +secondary nodes, but **Write to "authorized keys" file** +must be unchecked only on the primary node, because it is reflected automatically on the secondary if database replication is working. ## Setting up fast lookup via GitLab Shell @@ -56,8 +57,8 @@ GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to check whether the user is authorized to access GitLab. -Add the following to your `sshd_config` file. This is usually located at -`/etc/ssh/sshd_config`, but it will be `/assets/sshd_config` if you're using +Add the following to your `sshd_config` file. This file is usually located at +`/etc/ssh/sshd_config`, but it is at `/assets/sshd_config` if you're using Omnibus Docker: ```plaintext @@ -84,17 +85,14 @@ file (start the line with a `#` to comment it), and from your local machine, att ssh -T git@gitlab.example.com ``` -A successful pull or [welcome message](../../user/ssh.md#verify-that-you-can-connect) would mean that GitLab was able to find the key in the database, -since it is not present in the file anymore. - -NOTE: -For Omnibus Docker, `AuthorizedKeysCommand` is setup by default in -GitLab 11.11 and later. +A successful pull or [welcome message](../../user/ssh.md#verify-that-you-can-connect) +means that GitLab was able to find the key in the database, +as it is not present in the file. NOTE: For Installations from source, the command would be located at `/home/git/gitlab-shell/bin/gitlab-shell-authorized-keys-check` if [the install from source](../../install/installation.md#install-gitlab-shell) instructions were followed. -You might want to consider creating a wrapper script somewhere else since this command needs to be +You might want to consider creating a wrapper script somewhere else, as this command must be owned by `root` and not be writable by group or others. You could also consider changing the ownership of this command as required, but that might require temporary ownership changes during `gitlab-shell` upgrades. @@ -123,7 +121,7 @@ or for users to re-add their keys. ## How to go back to using the `authorized_keys` file -This is a brief overview. Please refer to the above instructions for more context. +This overview is brief. Refer to the above instructions for more context. 1. [Rebuild the `authorized_keys` file](../raketasks/maintenance.md#rebuild-authorized_keys-file). 1. Enable writes to the `authorized_keys` file. @@ -136,27 +134,35 @@ This is a brief overview. Please refer to the above instructions for more contex ## Use `gitlab-sshd` instead of OpenSSH -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299109) in GitLab 14.5. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299109) in GitLab 14.5 as an **Alpha** release for self-managed customers. WARNING: `gitlab-sshd` is in [**Alpha**](../../policy/alpha-beta-support.md#alpha-features). It is not ready for production use. `gitlab-sshd` is [a standalone SSH server](https://gitlab.com/gitlab-org/gitlab-shell/-/tree/main/internal/sshd) - written in Go. It is provided as a part of `gitlab-shell` package. It has a lower memory - use as a OpenSSH alternative and supports - [group access restriction by IP address](../../user/group/index.md) for applications - running behind the proxy. +written in Go. It is provided as a part of the `gitlab-shell` package. It has a lower memory +use as a OpenSSH alternative, and supports +[group access restriction by IP address](../../user/group/index.md) for applications +running behind the proxy. + +`gitlab-sshd` is a lightweight alternative to OpenSSH for providing +[SSH operations](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/71a7f34a476f778e62f8fe7a453d632d395eaf8f/doc/features.md). +While OpenSSH uses a restricted shell approach, `gitlab-sshd` behaves more like a +modern multi-threaded server application, responding to incoming requests. The major +difference is that OpenSSH uses SSH as a transport protocol while `gitlab-sshd` uses Remote Procedure Calls (RPCs). + +The capabilities of GitLab Shell are not limited to Git operations. If you are considering switching from OpenSSH to `gitlab-sshd`, consider these concerns: - The `gitlab-sshd` component is only available for [Cloud Native Helm Charts](https://docs.gitlab.com/charts/) deployments. - `gitlab-sshd` supports the PROXY protocol. It can run behind proxy servers that rely - on it, such as HAProxy. -- `gitlab-sshd` does not share a SSH port with the system administrator's OpenSSH, - and requires a bind to port 22. -- `gitlab-sshd` **does not** support SSH certificates. + on it, such as HAProxy. The PROXY protocol not enabled by default, but can be enabled with a Helm chart setting. +- By default, `gitlab-sshd` binds to port 22, but you can configure a different port in the Helm chart. +- `gitlab-sshd` **does not** support SSH certificates. For more details, read + [issue #495](https://gitlab.com/gitlab-org/gitlab-shell/-/issues/495). To switch from OpenSSH to `gitlab-sshd`: @@ -178,7 +184,11 @@ GitLab supports `authorized_keys` database lookups with [SELinux](https://en.wik Because the SELinux policy is static, GitLab doesn't support the ability to change internal webserver ports at the moment. Administrators would have to create a special `.te` -file for the environment, since it isn't generated dynamically. +file for the environment, as it isn't generated dynamically. + +### Additional documentation + +Additional technical documentation for `gitlab-sshd` may be found on the [GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/README.md) documentation page. ## Troubleshooting @@ -187,7 +197,8 @@ or causing high CPU load, be sure to check the size of `/var/log/btmp`, and ensu If this file is very large, GitLab SSH fast lookup can cause the bottleneck to be hit more frequently, thus decreasing performance even further. If you are able to, you may consider disabling [`UsePAM` in your `sshd_config`](https://linux.die.net/man/5/sshd_config) to avoid reading `/var/log/btmp` altogether. -Running `strace` and `lsof` on a running `sshd: git` process can return useful debugging information. To get an `strace` on an in-progress Git over SSH connection for IP `x.x.x.x`, run: +Running `strace` and `lsof` on a running `sshd: git` process returns debugging information. +To get an `strace` on an in-progress Git over SSH connection for IP `x.x.x.x`, run: ```plaintext sudo strace -s 10000 -p $(sudo netstat -tp | grep x.x.x.x | egrep 'ssh.*: git' | sed -e 's/.*ESTABLISHED *//' -e 's#/.*##') diff --git a/doc/administration/operations/puma.md b/doc/administration/operations/puma.md index c12f75989c3..12a8b2faadc 100644 --- a/doc/administration/operations/puma.md +++ b/doc/administration/operations/puma.md @@ -179,6 +179,46 @@ optimal configuration: - To force Rugged to be used with multi-threaded Puma, you can use a [feature flag](../../development/gitaly.md#legacy-rugged-code). +## Configuring Puma to listen over SSL + +Puma, when deployed with Omnibus GitLab, listens over a Unix socket by +default. To configure Puma to listen over an HTTPS port instead, follow the +steps below: + +1. Generate an SSL certificate key-pair for the address where Puma will + listen. For the example below, this is `127.0.0.1`. + + NOTE: + If using a self-signed certificate from a custom Certificate Authority (CA), + follow [the documentation](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates) + to make them trusted by other GitLab components. + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + puma['ssl_listen'] = '127.0.0.1' + puma['ssl_port'] = 9111 + puma['ssl_certificate'] = '<path_to_certificate>' + puma['ssl_certificate_key'] = '<path_to_key>' + + # Disable UNIX socket + puma['socket'] = "" + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +NOTE: +In addition to the Unix socket, Puma also listens over HTTP on port 8080 for +providing metrics to be scraped by Prometheus. It is not currently possible to +make Prometheus scrape them over HTTPS, and support for it is being discussed +[in this issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6811). +Hence, it is not technically possible to turn off this HTTP listener without +losing Prometheus metrics. + ## Switch from Unicorn to Puma NOTE: |