summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJoshua Lambert <joshua@gitlab.com>2019-05-17 11:35:42 -0400
committerJoshua Lambert <joshua@gitlab.com>2019-05-17 11:35:42 -0400
commite73695513d454070201db7b61a05c038a6ae2fed (patch)
treed05ad2c07c9f08604e6b240d58779574507fafd1
parent2e7ef8a2afe461f46698927578ba1928c046c30f (diff)
downloadgitlab-ce-jl-reorganize-architecture-page-docs.tar.gz
Streamline component detail listjl-reorganize-architecture-page-docs
-rw-r--r--doc/development/architecture.md69
1 files changed, 22 insertions, 47 deletions
diff --git a/doc/development/architecture.md b/doc/development/architecture.md
index 8340a571809..aebfb2a1c70 100644
--- a/doc/development/architecture.md
+++ b/doc/development/architecture.md
@@ -26,7 +26,7 @@ Gitaly executes git operations from gitlab-shell and the GitLab web app, and pro
You may also be interested in the [production architecture of GitLab.com](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/).
-### Architecture diagram
+### Component diagram
```mermaid
graph TB
@@ -150,7 +150,7 @@ This document is designed to be consumed by systems administrators and GitLab Su
When deployed, GitLab should be considered the amalgamation of the below processes. When troubleshooting or debugging, be as specific as possible as to which component you are referencing. That should increase clarity and reduce confusion.
-#### Layers
+**Layers**
GitLab can be considered to have two layers from a process perspective:
@@ -159,28 +159,7 @@ GitLab can be considered to have two layers from a process perspective:
- **Processors**: These processes are responsible for actually performing operations and presenting the service.
- **Data**: These services store/expose structured data for the GitLab service.
-#### GitLab process descriptions
-
-As of this writing, a fresh GitLab 11.3.0 install with default settings will show the following processes with `gitlab-ctl status`:
-
-```
-run: alertmanager: (pid 30829) 14207s; run: log: (pid 13906) 2432044s
-run: gitaly: (pid 30771) 14210s; run: log: (pid 13843) 2432046s
-run: gitlab-monitor: (pid 30788) 14209s; run: log: (pid 13868) 2432045s
-run: gitlab-workhorse: (pid 30758) 14210s; run: log: (pid 13855) 2432046s
-run: logrotate: (pid 30246) 3407s; run: log: (pid 13825) 2432047s
-run: nginx: (pid 30849) 14207s; run: log: (pid 13856) 2432046s
-run: node-exporter: (pid 30929) 14206s; run: log: (pid 13877) 2432045s
-run: postgres-exporter: (pid 30935) 14206s; run: log: (pid 13931) 2432044s
-run: postgresql: (pid 13133) 2432214s; run: log: (pid 13848) 2432046s
-run: prometheus: (pid 30807) 14209s; run: log: (pid 13884) 2432045s
-run: redis: (pid 30560) 14274s; run: log: (pid 13807) 2432047s
-run: redis-exporter: (pid 30946) 14205s; run: log: (pid 13869) 2432045s
-run: sidekiq: (pid 30953) 14205s; run: log: (pid 13810) 2432047s
-run: unicorn: (pid 30960) 14204s; run: log: (pid 13809) 2432047s
-```
-
-#### alertmanager
+#### Alertmanager
- [Project page](https://github.com/prometheus/alertmanager/blob/master/README.md)
- Configuration: [Omnibus][alertmanager-omnibus], [Charts][alertmanager-charts]
@@ -188,7 +167,7 @@ run: unicorn: (pid 30960) 14204s; run: log: (pid 13809) 2432047s
[Alert manager](https://prometheus.io/docs/alerting/alertmanager/) is a tool provided by Prometheus that _"handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts."_ You can read more in [issue gitlab-ce#45740](https://gitlab.com/gitlab-org/gitlab-ce/issues/45740) about what we will be alerting on.
-#### gitaly
+#### Gitaly
- [Project page](https://gitlab.com/gitlab-org/gitaly/blob/master/README.md)
- Configuration: [Omnibus][gitaly-omnibus], [Charts][gitaly-charts]
@@ -196,7 +175,7 @@ run: unicorn: (pid 30960) 14204s; run: log: (pid 13809) 2432047s
Gitaly is a service designed by GitLab to remove our need for NFS for Git storage in distributed deployments of GitLab (think GitLab.com or High Availability Deployments). As of 11.3.0, this service handles all Git level access in GitLab. You can read more about the project [in the project's readme](https://gitlab.com/gitlab-org/gitaly).
-#### gitlab-monitor
+#### Gitlab Monitor
- [Project page](https://gitlab.com/gitlab-org/gitlab-monitor)
- Configuration: [Omnibus][gitlab-monitor-omnibus], [Charts][gitlab-monitor-charts]
@@ -204,7 +183,7 @@ Gitaly is a service designed by GitLab to remove our need for NFS for Git storag
GitLab Monitor is a process designed in house that allows us to export metrics about GitLab application internals to Prometheus. You can read more [in the project's readme](https://gitlab.com/gitlab-org/gitlab-monitor).
-#### gitlab-workhorse
+#### Gitlab Workhorse
- [Project page](https://gitlab.com/gitlab-org/gitlab-workhorse/blob/master/README.md)
- Configuration: [Omnibus][gitlab-workhorse-omnibus], [Charts][gitlab-workhorse-charts]
@@ -212,7 +191,7 @@ GitLab Monitor is a process designed in house that allows us to export metrics a
[GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) is a program designed at GitLab to help alleviate pressure from Unicorn. You can read more about the [historical reasons for developing](https://about.gitlab.com/2016/04/12/a-brief-history-of-gitlab-workhorse/). It's designed to act as a smart reverse proxy to help speed up GitLab as a whole.
-#### logrotate
+#### Logrotate
- [Project page](https://github.com/logrotate/logrotate/blob/master/README.md)
- Configuration: [Omnibus](https://docs.gitlab.com/omnibus/settings/logs.html#logrotate)
@@ -220,7 +199,7 @@ GitLab Monitor is a process designed in house that allows us to export metrics a
GitLab is comprised of a large number of services that all log. We started bundling our own logrotate as of 7.4 to make sure we were logging responsibly. This is just a packaged version of the common open source offering.
-#### nginx
+#### NGINX
- Project page: [Omnibus](https://github.com/nginx/nginx), [Charts](https://github.com/kubernetes/ingress-nginx/blob/master/README.md)
- Configuration: [Omnibus][nginx-omnibus], [Charts][nginx-charts]
@@ -228,7 +207,7 @@ GitLab is comprised of a large number of services that all log. We started bundl
Nginx as an ingress port for all HTTP requests and routes them to the approriate sub-systems within GitLab. We are bundling an unmodified version of the popular open source webserver.
-#### node-exporter
+#### Node Exporter
- [Project page](https://github.com/prometheus/node_exporter/blob/master/README.md)
- Configuration: [Omnibus][node-exporter-omnibus], [Charts][node-exporter-charts]
@@ -236,7 +215,7 @@ Nginx as an ingress port for all HTTP requests and routes them to the approriate
[Node Exporter](https://github.com/prometheus/node_exporter) is a Prometheus tool that gives us metrics on the underlying machine (think CPU/Disk/Load). It's just a packaged version of the common open source offering from the Prometheus project.
-#### postgres-exporter
+#### Postgres Exporter
- [Project page](https://github.com/wrouesnel/postgres_exporter/blob/master/README.md)
- Configuration: [Omnibus][postgres-exporter-omnibus], [Charts][postgres-exporter-charts]
@@ -244,7 +223,7 @@ Nginx as an ingress port for all HTTP requests and routes them to the approriate
[Postgres-exporter](https://github.com/wrouesnel/postgres_exporter) is the community provided Prometheus exporter that will deliver data about Postgres to Prometheus for use in Grafana Dashboards.
-#### postgresql
+#### Postgresql
- [Project page](https://github.com/postgres/postgres/blob/master/README)
- Configuration: [Omnibus][postgres-omnibus], [Charts][postgres-charts]
@@ -252,7 +231,7 @@ Nginx as an ingress port for all HTTP requests and routes them to the approriate
GitLab packages the popular Database to provide storage for Application meta data and user information.
-#### prometheus
+#### Prometheus
- [Project page](https://github.com/prometheus/prometheus/blob/master/README.md)
- Configuration: [Omnibus][prometheus-omnibus], [Charts][prometheus-charts]
@@ -260,7 +239,7 @@ GitLab packages the popular Database to provide storage for Application meta dat
Prometheus is a time-series tool that helps GitLab administrators expose metrics about the individual processes used to provide GitLab the service.
-#### redis
+#### Redis
- [Project page](https://github.com/antirez/redis/blob/unstable/README.md)
- Configuration: [Omnibus][redis-omnibus], [Charts][redis-charts]
@@ -272,7 +251,7 @@ Redis is packaged to provide a place to store:
- temporary cache information
- background job queues
-#### redis-exporter
+#### Redis Exporter
- [Project page](https://github.com/oliver006/redis_exporter/blob/master/README.md)
- Configuration: [Omnibus][redis-exporter-omnibus], [Charts][redis-exporter-charts]
@@ -280,7 +259,7 @@ Redis is packaged to provide a place to store:
[Redis Exporter](https://github.com/oliver006/redis_exporter) is designed to give specific metrics about the Redis process to Prometheus so that we can graph these metrics in Grafana.
-#### sidekiq
+#### Sidekiq
- [Project page](https://github.com/mperham/sidekiq/blob/master/README.md)
- Configuration: [Omnibus][sidekiq-omnibus], [Charts][sidekiq-charts]
@@ -288,7 +267,7 @@ Redis is packaged to provide a place to store:
Sidekiq is a Ruby background job processor that pulls jobs from the redis queue and processes them. Background jobs allow GitLab to provide a faster request/response cycle by moving work into the background.
-#### unicorn
+#### Unicorn
- [Project page](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/README.md)
- Configuration: [Omnibus][unicorn-omnibus], [Charts][unicorn-charts]
@@ -296,11 +275,7 @@ Sidekiq is a Ruby background job processor that pulls jobs from the redis queue
[Unicorn](https://bogomips.org/unicorn/) is a Ruby application server that is used to run the core Rails Application that provides the user facing features in GitLab. Often process output you will see this as `bundle` or `config.ru` depending on the GitLab version.
-#### Additional processes and components
-
-The following processes will be running if corresponding feature is enabled.
-
-##### gitlab-pages
+#### Gitlab Pages
- Configuration: [Omnibus][pages-omnibus], [Charts][pages-charts]
- Layer: Core Service (Processor)
@@ -309,7 +284,7 @@ GitLab Pages is a feature that allows you to publish static websites directly fr
You can use it either for personal or business websites, such as portfolios, documentation, manifestos, and business presentations. You can also attribute any license to your content.
-#### gitlab-shell
+#### Gitlab Shell
- [Project page](https://gitlab.com/gitlab-org/gitlab-shell/blob/master/README.md)
- Configuration: [Omnibus][shell-omnibus], [Charts][shell-charts]
@@ -325,7 +300,7 @@ You can use it either for personal or business websites, such as portfolios, doc
Mattermost is an open source, private cloud, Slack-alternative from https://mattermost.com.
-#### gitlab-runner
+#### Gitlab Runner
- [Project page](https://gitlab.com/gitlab-org/gitlab-runner/blob/master/README.md)
- Configuration: [Omnibus][runner-omnibus], [Charts][runner-charts]
@@ -335,7 +310,7 @@ GitLab Runner runs tests and sends the results to GitLab.
GitLab CI is the open-source continuous integration service included with GitLab that coordinates the testing. The old name of this project was GitLab CI Multi Runner but please use "GitLab Runner" (without CI) from now on.
-#### registry
+#### Registry
- [Project page](https://github.com/docker/distribution/blob/master/README.md)
- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts]
@@ -351,12 +326,12 @@ to the registry. Learn more about [token authentication](https://docs.docker.com
An external registry can also be configured to use GitLab as an auth endpoint.
-##### Database migrations
+#### Database migrations
- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts]
- Layer: Core Service (Data)
-##### Certificate management
+#### Certificate management
- Project page: [Omnibus](https://github.com/certbot/certbot/blob/master/README.rst), [Charts](https://github.com/jetstack/cert-manager/blob/master/README.md)
- Configuration: [Omnibus][certificate-management-omnibus], [Charts][certificate-management-charts]