From 10198448cbec2723d95704d30b89166eea5a2afa Mon Sep 17 00:00:00 2001 From: Igor Drozdov Date: Tue, 20 Sep 2022 14:37:56 +0200 Subject: Add developer documentation to sshd package --- internal/sshd/README.md | 44 ++++++++++++++++++++++++++++++++++++++++++++ internal/sshd/session.go | 4 ++++ 2 files changed, 48 insertions(+) create mode 100644 internal/sshd/README.md diff --git a/internal/sshd/README.md b/internal/sshd/README.md new file mode 100644 index 0000000..dccdc00 --- /dev/null +++ b/internal/sshd/README.md @@ -0,0 +1,44 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Overview + +This package is responsible for handling SSH connections. + +Since GitLab [13.9](https://gitlab.com/groups/gitlab-org/-/milestones/56), `gitlab-shell` provides an option to either use [OpenSSH](https://github.com/openssh/openssh-portable) to handle connections or utilize an internal implementation. For more information on the internal implementation, see the [Why we implemented our own SSHD solution](https://about.gitlab.com/blog/2022/08/17/why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass/) blog post. + +The package contains multiple public functions: + +- [func NewServer](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L43) for initializing [Server](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L33). +- [func (s \*Server) ListenAndServe](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L52) for listening to new connections and serving them. +- [func (s \*Server) Shutdown](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L63) for shutting down the server. +- [func (s \*Server) MonitoringServeMux](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L73) that provides monitoring endpoints. + +All these functions are used by the [`main`](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/822e49b34afbc2092ae189091d693ae7867a8e5a/cmd/gitlab-sshd/main.go) function to initialize and shutdown the server. When a client tries to connect to the server, the following sequence of steps happens: + +- When a new connection is [accepted](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L118) by the listener, it's [asynchronously handled](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L129). +- A new connection is [created](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L190) with [a customized server config](https://gitlab.com/gitlab-org/gitlab-shell/blob/1177e2675c15ed55d46528c17408fd98d797cdec/internal/sshd/server_config.go#L148) that allows the server to perform [the handshake](https://gitlab.com/gitlab-org/gitlab-shell/blob/1177e2675c15ed55d46528c17408fd98d797cdec/internal/sshd/connection.go#L77) that [verifies](https://gitlab.com/gitlab-org/gitlab-shell/blob/1177e2675c15ed55d46528c17408fd98d797cdec/internal/sshd/server_config.go#L151) the authorized keys via [a request](https://gitlab.com/gitlab-org/gitlab-shell/blob/1177e2675c15ed55d46528c17408fd98d797cdec/internal/sshd/server_config.go#L140) to Gitlab Rails. The server is ready to [handle sessions](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L65) via this connection. The reason why we [handle the sessions in a loop](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L98) is the [persisting connections](https://man.openbsd.org/ssh_config.5#ControlPersist): a connection remains open and reused for the future sessions. +- A new session is [handled](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L135) by the [channelHandler](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L39) function passed as a [parameter](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L199). This function [processes](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/session.go#L56) requests from the client. +- The client may send [env requests](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/session.go#L67) in order to set an environment variable but we [restrict it](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/session.go#L113) to only setting Git protocol version. The client eventually [executes a command](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/session.go#L68-74) either via `shell` or `exec` requests. +- When the command is [executed](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/session.go#L143), we [create a command](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/session.go#L165) of a [particular type](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/cmd/gitlab-shell/command/command.go#L57). More information on how the commands work can be found [here](https://gitlab.com/gitlab-org/gitlab-shell/-/tree/main/internal/command). +- When a command is executed, the [session is closed](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/session.go#L204) and the error about unsuccessful execution, if such took place, is [tracked](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L137). +- If a connection has been prematurely closed we [block execution](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L146) until all concurrent sessions are released. + +## Proxy protocol + +The package supports creating a server with PROXY protocol. The [`go-proxyproto`](https://github.com/pires/go-proxyproto) package is used to [wrap](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L98) the basic listener into the one that supports PROXY protocol. PROXY protocol enables us to implement [Group IP address restriction via SSH](https://gitlab.com/gitlab-org/gitlab/-/issues/271673). The policies are [configurable](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L203). + +## Configurable OpenSSH alternatives + +- [LoginGraceTime](https://man7.org/linux/man-pages/man5/sshd_config.5.html) is [implemented](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L73) via TCP deadlines. +- [ClientAliveInterval](https://man7.org/linux/man-pages/man5/sshd_config.5.html) is [implemented](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L151) via sending [keep-alive message](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/connection.go#L25) periodically. + +## State machine + +The server [maintains a state machine](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L26-31) to implement: + +- **Graceful shutdown.** When a termination signal [has been detected](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/cmd/gitlab-sshd/main.go#L96), then a service [is being shut down](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/cmd/gitlab-sshd/main.go#L105). The status is [changed](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L68) accordingly and no new connections [are accepted](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L120). A configurable [grace period is given](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/cmd/gitlab-sshd/main.go#L107) in order to allow the ongoing connections to complete. When the period expires, then the top-level context is [canceled](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/cmd/gitlab-sshd/main.go#L109). That means that all the ongoing HTTP and SSH connections are [closed](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L173). +- **Liveness and readiness probes** that help Kubernetes to evaluate the state of the server. If a state [is any other than ready](https://gitlab.com/gitlab-org/gitlab-shell/blob/2094323308f8c6cb1d935af1b3331df29edcc9b6/internal/sshd/sshd.go#L80) (for example, during graceful shutdown), then 502 is returned. diff --git a/internal/sshd/session.go b/internal/sshd/session.go index 1d9ae42..48cd86a 100644 --- a/internal/sshd/session.go +++ b/internal/sshd/session.go @@ -66,8 +66,12 @@ func (s *session) handle(ctx context.Context, requests <-chan *ssh.Request) erro case "env": shouldContinue, err = s.handleEnv(ctx, req) case "exec": + // The command has been executed as `ssh user@host command` or `exec` channel has been used + // in the app implementation shouldContinue, err = s.handleExec(ctx, req) case "shell": + // The command has been entered into the shell or `shell` channel has been used + // in the app implementation shouldContinue = false var status uint32 status, err = s.handleShell(ctx, req) -- cgit v1.2.1