diff options
Diffstat (limited to 'doc')
33 files changed, 440 insertions, 82 deletions
diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md index d43b3718bf9..f5d58db0133 100644 --- a/doc/administration/container_registry.md +++ b/doc/administration/container_registry.md @@ -471,7 +471,7 @@ You can use GitLab as an auth endpoint and use a non-bundled Container Registry. ``` 1. A certificate keypair is required for GitLab and the Container Registry to - communicate securely. By default omnibus-gitlab will generate one keypair, + communicate securely. By default Omnibus GitLab will generate one keypair, which is saved to `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key`. When using a non-bundled Container Registry, you will need to supply a custom certificate key. To do that, add the following to @@ -487,7 +487,7 @@ You can use GitLab as an auth endpoint and use a non-bundled Container Registry. **Note:** The file specified at `registry_key_path` gets populated with the content specified by `internal_key`, each time reconfigure is executed. If - no file is specified, omnibus-gitlab will default it to + no file is specified, Omnibus GitLab will default it to `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key` and will populate it. diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 32462a95a1a..7238d08ab09 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -12,17 +12,17 @@ NOTE: **Note:** Custom Git hooks won't be replicated to secondary nodes if you use [GitLab Geo](geo/replication/index.md) Git natively supports hooks that are executed on different actions. -Examples of server-side git hooks include pre-receive, post-receive, and update. +Examples of server-side Git hooks include pre-receive, post-receive, and update. See [Git SCM Server-Side Hooks][hooks] for more information about each hook type. -As of gitlab-shell version 2.2.0 (which requires GitLab 7.5+), GitLab -administrators can add custom git hooks to any GitLab project. +As of GitLab Shell version 2.2.0 (which requires GitLab 7.5+), GitLab +administrators can add custom Git hooks to any GitLab project. ## Create a custom Git hook for a repository Server-side Git hooks are typically placed in the repository's `hooks` -subdirectory. In GitLab, hook directories are symlinked to the gitlab-shell -`hooks` directory for ease of maintenance between gitlab-shell upgrades. +subdirectory. In GitLab, hook directories are symlinked to the GitLab Shell +`hooks` directory for ease of maintenance between GitLab Shell upgrades. Custom hooks are implemented differently, but the behavior is exactly the same once the hook is created. Follow the steps below to set up a custom hook for a repository: @@ -36,7 +36,7 @@ repository: 1. Inside the new `custom_hooks` directory, create a file with a name matching the hook type. For a pre-receive hook the file name should be `pre-receive` with no extension. -1. Make the hook file executable and make sure it's owned by git. +1. Make the hook file executable and make sure it's owned by Git. 1. Write the code to make the Git hook function as expected. Hooks can be in any language. Ensure the 'shebang' at the top properly reflects the language type. For example, if the script is in Ruby the shebang will probably be @@ -49,17 +49,17 @@ as appropriate. To create a Git hook that applies to all of your repositories in your instance, set a global Git hook. Since all the repositories' `hooks` -directories are symlinked to gitlab-shell's `hooks` directory, adding any hook -to the gitlab-shell `hooks` directory will also apply it to all repositories. Follow +directories are symlinked to GitLab Shell's `hooks` directory, adding any hook +to the GitLab Shell `hooks` directory will also apply it to all repositories. Follow the steps below to properly set up a custom hook for all repositories: 1. On the GitLab server, navigate to the configured custom hook directory. The - default is in the gitlab-shell directory. The gitlab-shell `hook` directory + default is in the GitLab Shell directory. The GitLab Shell `hook` directory for an installation from source the path is usually `/home/git/gitlab-shell/hooks`. For Omnibus installs the path is usually `/opt/gitlab/embedded/service/gitlab-shell/hooks`. To look in a different directory for the global custom hooks, - set `custom_hooks_dir` in the gitlab-shell config. For + set `custom_hooks_dir` in the GitLab Shell config. For Omnibus installations, this can be set in `gitlab.rb`; and in source installations, this can be set in `gitlab-shell/config.yml`. 1. Create a new directory in this location. Depending on your hook, it will be diff --git a/doc/administration/dependency_proxy.md b/doc/administration/dependency_proxy.md index d2c52b67e67..5153666705f 100644 --- a/doc/administration/dependency_proxy.md +++ b/doc/administration/dependency_proxy.md @@ -48,7 +48,7 @@ local location or even use object storage. The dependency proxy files for Omnibus GitLab installations are stored under `/var/opt/gitlab/gitlab-rails/shared/dependency_proxy/` and for source -installations under `shared/dependency_proxy/` (relative to the git home directory). +installations under `shared/dependency_proxy/` (relative to the Git home directory). To change the local storage path: **Omnibus GitLab installations** diff --git a/doc/administration/git_protocol.md b/doc/administration/git_protocol.md index 1780e1babe9..9d653d4e09e 100644 --- a/doc/administration/git_protocol.md +++ b/doc/administration/git_protocol.md @@ -53,7 +53,7 @@ sudo service ssh restart ## Instructions In order to use the new protocol, clients need to either pass the configuration -`-c protocol.version=2` to the git command, or set it globally: +`-c protocol.version=2` to the Git command, or set it globally: ```sh git config --global protocol.version 2 diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index 73a39a6dd35..29915cb3a99 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -151,7 +151,7 @@ Reply by email should now be working. #### Postfix -Example configuration for Postfix mail server. Assumes mailbox incoming@gitlab.example.com. +Example configuration for Postfix mail server. Assumes mailbox `incoming@gitlab.example.com`. Example for Omnibus installs: @@ -218,7 +218,7 @@ incoming_email: #### Gmail -Example configuration for Gmail/G Suite. Assumes mailbox gitlab-incoming@gmail.com. +Example configuration for Gmail/G Suite. Assumes mailbox `gitlab-incoming@gmail.com`. Example for Omnibus installs: diff --git a/doc/administration/index.md b/doc/administration/index.md index 9dbd90ecea3..650cb10a64a 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -193,6 +193,6 @@ Learn how to install, configure, update, and maintain your GitLab instance. for troubleshooting Kubernetes-related issues. - Useful links from the Support Team: - [GitLab Developer Docs](https://docs.gitlab.com/ee/development/README.html). - - [Repairing and recovering broken git repositories](https://git.seveas.net/repairing-and-recovering-broken-git-repositories.html). + - [Repairing and recovering broken Git repositories](https://git.seveas.net/repairing-and-recovering-broken-git-repositories.html). - [Testing with OpenSSL](https://www.feistyduck.com/library/openssl-cookbook/online/ch-testing-with-openssl.html). - [Strace zine](https://wizardzines.com/zines/strace/). diff --git a/doc/administration/issue_closing_pattern.md b/doc/administration/issue_closing_pattern.md index e1bbabb2878..66ab3ef8d81 100644 --- a/doc/administration/issue_closing_pattern.md +++ b/doc/administration/issue_closing_pattern.md @@ -13,7 +13,7 @@ in the project's default branch. In order to change the pattern you need to have access to the server that GitLab is installed on. -The default pattern can be located in [gitlab.yml.example] under the +The default pattern can be located in [`gitlab.yml.example`] under the "Automatic issue closing" section. > **Tip:** diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 60921ca791f..350cd5b7992 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -94,7 +94,7 @@ If you're enabling S3 in [GitLab HA](high_availability/README.md), you will need ### Object Storage Settings -For source installations the following settings are nested under `artifacts:` and then `object_store:`. On omnibus installs they are prefixed by `artifacts_object_store_`. +For source installations the following settings are nested under `artifacts:` and then `object_store:`. On Omnibus GitLab installs they are prefixed by `artifacts_object_store_`. | Setting | Description | Default | |---------|-------------|---------| diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index ec26c0b2e7e..0605fb76e2f 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -19,38 +19,106 @@ it, the client IP needs to be [included in a whitelist][whitelist]. For Omnibus and Chart installations, these metrics are automatically enabled and collected as of [GitLab 9.4](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1702). For source installations or earlier versions, these metrics will need to be enabled manually and collected by a Prometheus server. -## Unicorn Metrics available +## Metrics available The following metrics are available: -| Metric | Type | Since | Description | -|:--------------------------------- |:--------- |:----- |:----------- | -| db_ping_timeout | Gauge | 9.4 | Whether or not the last database ping timed out | -| db_ping_success | Gauge | 9.4 | Whether or not the last database ping succeeded | -| db_ping_latency_seconds | Gauge | 9.4 | Round trip time of the database ping | -| filesystem_access_latency_seconds | Gauge | 9.4 | Latency in accessing a specific filesystem | -| filesystem_accessible | Gauge | 9.4 | Whether or not a specific filesystem is accessible | -| filesystem_write_latency_seconds | Gauge | 9.4 | Write latency of a specific filesystem | -| filesystem_writable | Gauge | 9.4 | Whether or not the filesystem is writable | -| filesystem_read_latency_seconds | Gauge | 9.4 | Read latency of a specific filesystem | -| filesystem_readable | Gauge | 9.4 | Whether or not the filesystem is readable | -| gitlab_cache_misses_total | Counter | 10.2 | Cache read miss | -| gitlab_cache_operation_duration_seconds | Histogram | 10.2 | Cache access time | -| gitlab_cache_operations_total | Counter | 12.2 | Cache operations by controller/action | -| http_requests_total | Counter | 9.4 | Rack request count | -| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware | -| pipelines_created_total | Counter | 9.4 | Counter of pipelines created | -| rack_uncaught_errors_total | Counter | 9.4 | Rack connections handling uncaught errors count | -| redis_ping_timeout | Gauge | 9.4 | Whether or not the last redis ping timed out | -| redis_ping_success | Gauge | 9.4 | Whether or not the last redis ping succeeded | -| redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | -| user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | -| upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | -| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | -| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | -| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | -| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | -| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers | +| Metric | Type | Since | Description | Labels | +|:-------------------------------------------------------------|:----------|-----------------------:|:----------------------------------------------------------------------------------------------------|:----------------------------------------------------| +| gitlab_banzai_cached_render_real_duration_seconds | Histogram | 9.4 | Duration of rendering markdown into HTML when cached output exists | controller, action | +| gitlab_banzai_cacheless_render_real_duration_seconds | Histogram | 9.4 | Duration of rendering markdown into HTML when cached outupt does not exist | controller, action | +| gitlab_cache_misses_total | Counter | 10.2 | Cache read miss | controller, action | +| gitlab_cache_operation_duration_seconds | Histogram | 10.2 | Cache access time | | +| gitlab_cache_operations_total | Counter | 12.2 | Cache operations by controller/action | controller, action, operation | +| gitlab_database_transaction_seconds | Histogram | 12.1 | Time spent in database transactions, in seconds | | +| gitlab_method_call_duration_seconds | Histogram | 10.2 | Method calls real duration | controller, action, module, method | +| gitlab_rails_queue_duration_seconds | Histogram | 9.4 | Measures latency between gitlab-workhorse forwarding a request to Rails | | +| gitlab_sql_duration_seconds | Histogram | 10.2 | SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT | | +| gitlab_transaction_allocated_memory_bytes | Histogram | 10.2 | Allocated memory for all transactions (gitlab_transaction_* metrics) | | +| gitlab_transaction_cache_<key>_count_total | Counter | 10.2 | Counter for total Rails cache calls (per key) | | +| gitlab_transaction_cache_<key>_duration_total | Counter | 10.2 | Counter for total time (seconds) spent in Rails cache calls (per key) | | +| gitlab_transaction_cache_count_total | Counter | 10.2 | Counter for total Rails cache calls (aggregate) | | +| gitlab_transaction_cache_duration_total | Counter | 10.2 | Counter for total time (seconds) spent in Rails cache calls (aggregate) | | +| gitlab_transaction_cache_read_hit_count_total | Counter | 10.2 | Counter for cache hits for Rails cache calls | controller, action | +| gitlab_transaction_cache_read_miss_count_total | Counter | 10.2 | Counter for cache misses for Rails cache calls | controller, action | +| gitlab_transaction_duration_seconds | Histogram | 10.2 | Duration for all transactions (gitlab_transaction_* metrics) | controller, action | +| gitlab_transaction_event_build_found_total | Counter | 9.4 | Counter for build found for api /jobs/request | | +| gitlab_transaction_event_build_invalid_total | Counter | 9.4 | Counter for build invalid due to concurrency conflict for api /jobs/request | | +| gitlab_transaction_event_build_not_found_cached_total | Counter | 9.4 | Counter for cached response of build not found for api /jobs/request | | +| gitlab_transaction_event_build_not_found_total | Counter | 9.4 | Counter for build not found for api /jobs/request | | +| gitlab_transaction_event_change_default_branch_total | Counter | 9.4 | Counter when default branch is changed for any repository | | +| gitlab_transaction_event_create_repository_total | Counter | 9.4 | Counter when any repository is created | | +| gitlab_transaction_event_etag_caching_cache_hit_total | Counter | 9.4 | Counter for etag cache hit. | endpoint | +| gitlab_transaction_event_etag_caching_header_missing_total | Counter | 9.4 | Counter for etag cache miss - header missing | endpoint | +| gitlab_transaction_event_etag_caching_key_not_found_total | Counter | 9.4 | Counter for etag cache miss - key not found | endpoint | +| gitlab_transaction_event_etag_caching_middleware_used_total | Counter | 9.4 | Counter for etag middleware accessed | endpoint | +| gitlab_transaction_event_etag_caching_resource_changed_total | Counter | 9.4 | Counter for etag cache miss - resource changed | endpoint | +| gitlab_transaction_event_fork_repository_total | Counter | 9.4 | Counter for repository forks (RepositoryForkWorker). Only incremented when source repository exists | | +| gitlab_transaction_event_import_repository_total | Counter | 9.4 | Counter for repository imports (RepositoryImportWorker) | | +| gitlab_transaction_event_push_branch_total | Counter | 9.4 | Counter for all branch pushes | | +| gitlab_transaction_event_push_commit_total | Counter | 9.4 | Counter for commits | branch | +| gitlab_transaction_event_push_tag_total | Counter | 9.4 | Counter for tag pushes | | +| gitlab_transaction_event_rails_exception_total | Counter | 9.4 | Counter for number of rails exceptions | | +| gitlab_transaction_event_receive_email_total | Counter | 9.4 | Counter for recieved emails | handler | +| gitlab_transaction_event_remote_mirrors_failed_total | Counter | 10.8 | Counter for failed remote mirrors | | +| gitlab_transaction_event_remote_mirrors_finished_total | Counter | 10.8 | Counter for finished remote mirrors | | +| gitlab_transaction_event_remote_mirrors_running_total | Counter | 10.8 | Counter for running remote mirrors | | +| gitlab_transaction_event_remove_branch_total | Counter | 9.4 | Counter when a branch is removed for any repository | | +| gitlab_transaction_event_remove_repository_total | Counter | 9.4 | Counter when a repository is removed | | +| gitlab_transaction_event_remove_tag_total | Counter | 9.4 | Counter when a tag is remove for any repository | | +| gitlab_transaction_event_sidekiq_exception_total | Counter | 9.4 | Counter of sidekiq exceptions | | +| gitlab_transaction_event_stuck_import_jobs_total | Counter | 9.4 | Count of stuck import jobs | projects_without_jid_count, projects_with_jid_count | +| gitlab_transaction_event_update_build_total | Counter | 9.4 | Counter for update build for api /jobs/request/:id | | +| gitlab_transaction_new_redis_connections_total | Counter | 9.4 | Counter for new redis connections | | +| gitlab_transaction_queue_duration_total | Counter | 9.4 | Duration jobs were enqueued before processing | | +| gitlab_transaction_rails_queue_duration_total | Counter | 9.4 | Measures latency between gitlab-workhorse forwarding a request to Rails | controller, action | +| gitlab_transaction_view_duration_total | Counter | 9.4 | Duration for views | controller, action, view | +| gitlab_view_rendering_duration_seconds | Histogram | 10.2 | Duration for views (histogram) | controller, action, view | +| http_requests_total | Counter | 9.4 | Rack request count | method | +| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware | method, status | +| pipelines_created_total | Counter | 9.4 | Counter of pipelines created | | +| rack_uncaught_errors_total | Counter | 9.4 | Rack connections handling uncaught errors count | | +| user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | | +| upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | | +| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | | +| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | | + +## Metrics controlled by a feature flag + +The following metrics can be controlled by feature flags: + +| Metric | Feature Flag | +|:-------------------------------------------------------------|:-----------------------------------------------------------------| +| gitlab_method_call_duration_seconds | prometheus_metrics_method_instrumentation | +| gitlab_transaction_allocated_memory_bytes | prometheus_metrics_transaction_allocated_memory | +| gitlab_transaction_event_build_found_total | prometheus_transaction_event_build_found_total | +| gitlab_transaction_event_build_invalid_total | prometheus_transaction_event_build_invalid_total | +| gitlab_transaction_event_build_not_found_cached_total | prometheus_transaction_event_build_not_found_cached_total | +| gitlab_transaction_event_build_not_found_total | prometheus_transaction_event_build_not_found_total | +| gitlab_transaction_event_change_default_branch_total | prometheus_transaction_event_change_default_branch_total | +| gitlab_transaction_event_create_repository_total | prometheus_transaction_event_create_repository_total | +| gitlab_transaction_event_etag_caching_cache_hit_total | prometheus_transaction_event_etag_caching_cache_hit_total | +| gitlab_transaction_event_etag_caching_header_missing_total | prometheus_transaction_event_etag_caching_header_missing_total | +| gitlab_transaction_event_etag_caching_key_not_found_total | prometheus_transaction_event_etag_caching_key_not_found_total | +| gitlab_transaction_event_etag_caching_middleware_used_total | prometheus_transaction_event_etag_caching_middleware_used_total | +| gitlab_transaction_event_etag_caching_resource_changed_total | prometheus_transaction_event_etag_caching_resource_changed_total | +| gitlab_transaction_event_fork_repository_total | prometheus_transaction_event_fork_repository_total | +| gitlab_transaction_event_import_repository_total | prometheus_transaction_event_import_repository_total | +| gitlab_transaction_event_push_branch_total | prometheus_transaction_event_push_branch_total | +| gitlab_transaction_event_push_commit_total | prometheus_transaction_event_push_commit_total | +| gitlab_transaction_event_push_tag_total | prometheus_transaction_event_push_tag_total | +| gitlab_transaction_event_rails_exception_total | prometheus_transaction_event_rails_exception_total | +| gitlab_transaction_event_receive_email_total | prometheus_transaction_event_receive_email_total | +| gitlab_transaction_event_remote_mirrors_failed_total | prometheus_transaction_event_remote_mirrors_failed_total | +| gitlab_transaction_event_remote_mirrors_finished_total | prometheus_transaction_event_remote_mirrors_finished_total | +| gitlab_transaction_event_remote_mirrors_running_total | prometheus_transaction_event_remote_mirrors_running_total | +| gitlab_transaction_event_remove_branch_total | prometheus_transaction_event_remove_branch_total | +| gitlab_transaction_event_remove_repository_total | prometheus_transaction_event_remove_repository_total | +| gitlab_transaction_event_remove_tag_total | prometheus_transaction_event_remove_tag_total | +| gitlab_transaction_event_sidekiq_exception_total | prometheus_transaction_event_sidekiq_exception_total | +| gitlab_transaction_event_stuck_import_jobs_total | prometheus_transaction_event_stuck_import_jobs_total | +| gitlab_transaction_event_update_build_total | prometheus_transaction_event_update_build_total | +| gitlab_view_rendering_duration_seconds | prometheus_metrics_view_instrumentation | ## Sidekiq Metrics available for Geo **(PREMIUM)** @@ -99,17 +167,27 @@ Some basic Ruby runtime metrics are available: | Metric | Type | Since | Description | |:-------------------------------------- |:--------- |:----- |:----------- | -| ruby_gc_duration_seconds_total | Counter | 11.1 | Time spent by Ruby in GC | +| ruby_gc_duration_seconds | Counter | 11.1 | Time spent by Ruby in GC | | ruby_gc_stat_... | Gauge | 11.1 | Various metrics from [GC.stat] | | ruby_file_descriptors | Gauge | 11.1 | File descriptors per process | | ruby_memory_bytes | Gauge | 11.1 | Memory usage by process | -| ruby_sampler_duration_seconds_total | Counter | 11.1 | Time spent collecting stats | +| ruby_sampler_duration_seconds | Counter | 11.1 | Time spent collecting stats | | ruby_process_cpu_seconds_total | Gauge | 12.0 | Total amount of CPU time per process | | ruby_process_max_fds | Gauge | 12.0 | Maximum number of open file descriptors per process | | ruby_process_resident_memory_bytes | Gauge | 12.0 | Memory usage by process, measured in bytes | | ruby_process_start_time_seconds | Gauge | 12.0 | UNIX timestamp of process start time | -[GC.stat]: https://ruby-doc.org/core-2.3.0/GC.html#method-c-stat +[GC.stat]: https://ruby-doc.org/core-2.6.3/GC.html#method-c-stat + +## Unicorn Metrics + +Unicorn specific metrics, when Unicorn is used. + +| Metric | Type | Since | Description | +|:---------------------------|:------|:------|:---------------------------------------------------| +| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | +| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | +| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers | ## Puma Metrics **(EXPERIMENTAL)** @@ -126,7 +204,6 @@ When Puma is used instead of Unicorn, following metrics are available: | puma_pool_capacity | Gauge | 12.0 | Number of requests the worker is capable of taking right now | | puma_max_threads | Gauge | 12.0 | Maximum number of worker threads | | puma_idle_threads | Gauge | 12.0 | Number of spawned threads which are not processing a request | -| rack_state_total | Gauge | 12.0 | Number of requests in a given rack state | | puma_killer_terminations_total | Gauge | 12.0 | Number of workers terminated by PumaWorkerKiller | ## Metrics shared directory diff --git a/doc/administration/packages.md b/doc/administration/packages.md index c0f8777a8c0..1628b6d6f91 100644 --- a/doc/administration/packages.md +++ b/doc/administration/packages.md @@ -55,7 +55,7 @@ local location or even use object storage. The packages for Omnibus GitLab installations are stored under `/var/opt/gitlab/gitlab-rails/shared/packages/` and for source -installations under `shared/packages/` (relative to the git homedir). +installations under `shared/packages/` (relative to the Git homedir). To change the local storage path: **Omnibus GitLab installations** diff --git a/doc/administration/plugins.md b/doc/administration/plugins.md index 52f1e7cde28..92a4d56ca63 100644 --- a/doc/administration/plugins.md +++ b/doc/administration/plugins.md @@ -39,7 +39,7 @@ Follow the steps below to set up a custom hook: 1. Inside the `plugins` directory, create a file with a name of your choice, without spaces or special characters. -1. Make the hook file executable and make sure it's owned by the git user. +1. Make the hook file executable and make sure it's owned by the Git user. 1. Write the code to make the plugin function as expected. That can be in any language, and ensure the 'shebang' at the top properly reflects the language type. For example, if the script is in Ruby the shebang will diff --git a/doc/administration/reply_by_email_postfix_setup.md b/doc/administration/reply_by_email_postfix_setup.md index 6d1ffbffefb..56cd23b2eb8 100644 --- a/doc/administration/reply_by_email_postfix_setup.md +++ b/doc/administration/reply_by_email_postfix_setup.md @@ -18,7 +18,7 @@ The instructions make the assumption that you will be using the email address `i sudo apt-get install postfix ``` - When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches gitlab.example.com`. + When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches `gitlab.example.com`. 1. Install the `mailutils` package. diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md index 05a7cb006a3..ab911c1cf0e 100644 --- a/doc/administration/repository_checks.md +++ b/doc/administration/repository_checks.md @@ -3,7 +3,7 @@ > [Introduced][ce-3232] in GitLab 8.7. It is OFF by default because it still causes too many false alarms. -Git has a built-in mechanism, [git fsck][git-fsck], to verify the +Git has a built-in mechanism, [`git fsck`][git-fsck], to verify the integrity of all data committed to a repository. GitLab administrators can trigger such a check for a project via the project page under the admin panel. The checks run asynchronously so it may take a few minutes diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index 1669f8b128c..142bcc65561 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -118,7 +118,7 @@ If you do have any existing integration, you may want to do a small rollout firs to validate. You can do so by specifying a range with the operation. This is an example of how to limit the rollout to Project IDs 50 to 100, running in -an Omnibus Gitlab installation: +an Omnibus GitLab installation: ```bash sudo gitlab-rake gitlab:storage:migrate_to_hashed ID_FROM=50 ID_TO=100 @@ -139,7 +139,7 @@ To schedule a complete rollback, see the [rake task documentation for storage rollback](raketasks/storage.md#rollback-from-hashed-storage-to-legacy-storage) for instructions. The rollback task also supports specifying a range of Project IDs. Here is an example -of limiting the rollout to Project IDs 50 to 100, in an Omnibus Gitlab installation: +of limiting the rollout to Project IDs 50 to 100, in an Omnibus GitLab installation: ```bash sudo gitlab-rake gitlab:storage:rollback_to_legacy ID_FROM=50 ID_TO=100 @@ -185,7 +185,7 @@ CI Artifacts are S3 compatible since **9.4** (GitLab Premium), and available in ##### LFS Objects -LFS Objects implements a similar storage pattern using 2 chars, 2 level folders, following git own implementation: +LFS Objects implements a similar storage pattern using 2 chars, 2 level folders, following Git own implementation: ```ruby "shared/lfs-objects/#{oid[0..1}/#{oid[2..3]}/#{oid[4..-1]}" diff --git a/doc/administration/uploads.md b/doc/administration/uploads.md index 9e9d12f23bb..a4bed72b965 100644 --- a/doc/administration/uploads.md +++ b/doc/administration/uploads.md @@ -57,7 +57,7 @@ This configuration relies on valid AWS credentials to be configured already. ## Object Storage Settings -For source installations the following settings are nested under `uploads:` and then `object_store:`. On omnibus installs they are prefixed by `uploads_object_store_`. +For source installations the following settings are nested under `uploads:` and then `object_store:`. On Omnibus GitLab installs they are prefixed by `uploads_object_store_`. | Setting | Description | Default | |---------|-------------|---------| diff --git a/doc/api/dependencies.md b/doc/api/dependencies.md index 5767d3572dd..5296d4e316f 100644 --- a/doc/api/dependencies.md +++ b/doc/api/dependencies.md @@ -5,7 +5,8 @@ This API is in an alpha stage and considered unstable. The response payload may be subject to change or breakage across GitLab releases. -Every call to this endpoint requires authentication. To perform this call, user should be authorized to read +Every call to this endpoint requires authentication. To perform this call, user should be authorized to read repository. +To see vulnerabilities in response, user should be authorized to read [Project Security Dashboard](../user/application_security/security_dashboard/index.md#project-security-dashboard). ## List project dependencies @@ -17,8 +18,8 @@ supported by Gemnasium. ``` GET /projects/:id/dependencies -GET /projects/:id/vulnerabilities?package_manager=maven -GET /projects/:id/vulnerabilities?package_manager=yarn,bundler +GET /projects/:id/dependencies?package_manager=maven +GET /projects/:id/dependencies?package_manager=yarn,bundler ``` | Attribute | Type | Required | Description | @@ -38,13 +39,18 @@ Example response: "name": "rails", "version": "5.0.1", "package_manager": "bundler", - "dependency_file_path": "Gemfile.lock" + "dependency_file_path": "Gemfile.lock", + "vulnerabilities": [{ + "name": "DDoS", + "severity": "unknown" + }] }, { "name": "hanami", "version": "1.3.1", "package_manager": "bundler", - "dependency_file_path": "Gemfile.lock" + "dependency_file_path": "Gemfile.lock", + "vulnerabilities": [] } ] ``` diff --git a/doc/api/features.md b/doc/api/features.md index 6ecd4ec14b9..e8d0c7c942b 100644 --- a/doc/api/features.md +++ b/doc/api/features.md @@ -60,8 +60,8 @@ POST /features/:name | `value` | integer/string | yes | `true` or `false` to enable/disable, or an integer for percentage of time | | `feature_group` | string | no | A Feature group name | | `user` | string | no | A GitLab username | -| `group` | string | no | A GitLab group's path, for example 'gitlab-org' | -| `project` | string | no | A projects path, for example 'gitlab-org/gitlab-ce' | +| `group` | string | no | A GitLab group's path, for example `gitlab-org` | +| `project` | string | no | A projects path, for example `gitlab-org/gitlab-ce` | Note that you can enable or disable a feature for a `feature_group`, a `user`, a `group`, and a `project` in a single API call. diff --git a/doc/api/geo_nodes.md b/doc/api/geo_nodes.md index d0b33ab467f..5eba7f038ed 100644 --- a/doc/api/geo_nodes.md +++ b/doc/api/geo_nodes.md @@ -111,7 +111,7 @@ PUT /geo_nodes/:id |-----------------------------|---------|-----------|---------------------------------------------------------------------------| | `id` | integer | yes | The ID of the Geo node. | | `enabled` | boolean | no | Flag indicating if the Geo node is enabled. | -| `name` | string | yes | The unique identifier for the Geo node. Must match `geo_node_name` if it is set in gitlab.rb, otherwise it must match `external_url`. | +| `name` | string | yes | The unique identifier for the Geo node. Must match `geo_node_name` if it is set in `gitlab.rb`, otherwise it must match `external_url`. | | `url` | string | yes | The user-facing URL of the Geo node. | | `internal_url` | string | no | The URL defined on the primary node that secondary nodes should use to contact it. Returns `url` if not set.| | `files_max_capacity` | integer | no | Control the maximum concurrency of LFS/attachment backfill for this secondary node. | diff --git a/doc/api/lint.md b/doc/api/lint.md index 79f5e629c7f..dacd3f4c493 100644 --- a/doc/api/lint.md +++ b/doc/api/lint.md @@ -1,4 +1,4 @@ -# Validate the .gitlab-ci.yml (API) +# Validate the `.gitlab-ci.yml` (API) > [Introduced][ce-5953] in GitLab 8.12. @@ -10,7 +10,7 @@ POST /ci/lint | Attribute | Type | Required | Description | | ---------- | ------- | -------- | -------- | -| `content` | string | yes | the .gitlab-ci.yaml content| +| `content` | string | yes | the `.gitlab-ci.yaml` content| ```bash curl --header "Content-Type: application/json" https://gitlab.example.com/api/v4/ci/lint --data '{"content": "{ \"image\": \"ruby:2.6\", \"services\": [\"postgres\"], \"before_script\": [\"bundle install\", \"bundle exec rake db:create\"], \"variables\": {\"DB_NAME\": \"postgres\"}, \"types\": [\"test\", \"deploy\", \"notify\"], \"rspec\": { \"script\": \"rake spec\", \"tags\": [\"ruby\", \"postgres\"], \"only\": [\"branches\"]}}"}' diff --git a/doc/api/projects.md b/doc/api/projects.md index 373607f8f4b..9f392418153 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -2037,13 +2037,13 @@ Read more in the [Project Badges](project_badges.md) documentation. The non-default [issue and merge request description templates](../user/project/description_templates.md) are managed inside the project's repository. So you can manage them via the API through the [Repositories API](repositories.md) and the [Repository Files API](repository_files.md). -## Download snapshot of a git repository +## Download snapshot of a Git repository > Introduced in GitLab 10.7 This endpoint may only be accessed by an administrative user. -Download a snapshot of the project (or wiki, if requested) git repository. This +Download a snapshot of the project (or wiki, if requested) Git repository. This snapshot is always in uncompressed [tar](https://en.wikipedia.org/wiki/Tar_(computing)) format. diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index b292c9dd7de..513dc996c91 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -246,7 +246,7 @@ error message. Possible causes for a failed commit include: user tried to make an empty commit; - the branch was updated by a Git push while the file edit was in progress. -Currently gitlab-shell has a boolean return code, preventing GitLab from specifying the error. +Currently GitLab Shell has a boolean return code, preventing GitLab from specifying the error. ## Delete existing file in repository diff --git a/doc/api/settings.md b/doc/api/settings.md index 4b5b2b924d7..710b63c9a2f 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -232,7 +232,7 @@ are listed in the descriptions of the relevant settings. | `file_template_project_id` | integer | no | **(PREMIUM)** The ID of a project to load custom file templates from | | `first_day_of_week` | integer | no | Start day of the week for calendar views and date pickers. Valid values are `0` (default) for Sunday, `1` for Monday, and `6` for Saturday. | | `geo_status_timeout` | integer | no | **(PREMIUM)** The amount of seconds after which a request to get a secondary node status will time out. | -| `gitaly_timeout_default` | integer | no | Default Gitaly timeout, in seconds. This timeout is not enforced for git fetch/push operations or Sidekiq jobs. Set to `0` to disable timeouts. | +| `gitaly_timeout_default` | integer | no | Default Gitaly timeout, in seconds. This timeout is not enforced for Git fetch/push operations or Sidekiq jobs. Set to `0` to disable timeouts. | | `gitaly_timeout_fast` | integer | no | Gitaly fast operation timeout, in seconds. Some Gitaly operations are expected to be fast. If they exceed this threshold, there may be a problem with a storage shard and 'failing fast' can help maintain the stability of the GitLab instance. Set to `0` to disable timeouts. | | `gitaly_timeout_medium` | integer | no | Medium Gitaly timeout, in seconds. This should be a value between the Fast and the Default timeout. Set to `0` to disable timeouts. | | `gravatar_enabled` | boolean | no | Enable Gravatar. | @@ -244,7 +244,7 @@ are listed in the descriptions of the relevant settings. | `hide_third_party_offers` | boolean | no | Do not display offers from third parties within GitLab. | | `home_page_url` | string | no | Redirect to this URL when not logged in. | | `housekeeping_bitmaps_enabled` | boolean | required by: `housekeeping_enabled` | Enable Git pack file bitmap creation. | -| `housekeeping_enabled` | boolean | no | (**If enabled, requires:** `housekeeping_bitmaps_enabled`, `housekeeping_full_repack_period`, `housekeeping_gc_period`, and `housekeeping_incremental_repack_period`) Enable or disable git housekeeping. | +| `housekeeping_enabled` | boolean | no | (**If enabled, requires:** `housekeeping_bitmaps_enabled`, `housekeeping_full_repack_period`, `housekeeping_gc_period`, and `housekeeping_incremental_repack_period`) Enable or disable Git housekeeping. | | `housekeeping_full_repack_period` | integer | required by: `housekeeping_enabled` | Number of Git pushes after which an incremental `git repack` is run. | | `housekeeping_gc_period` | integer | required by: `housekeeping_enabled` | Number of Git pushes after which `git gc` is run. | | `housekeeping_incremental_repack_period` | integer | required by: `housekeeping_enabled` | Number of Git pushes after which an incremental `git repack` is run. | diff --git a/doc/api/tags.md b/doc/api/tags.md index af86ba961f4..1d874fea1f8 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -112,7 +112,7 @@ Parameters: - `tag_name` (required) - The name of a tag - `ref` (required) - Create tag using commit SHA, another tag name, or branch name. - `message` (optional) - Creates annotated tag. -- `release_description` (optional) - Add release notes to the git tag and store it in the GitLab database. +- `release_description` (optional) - Add release notes to the Git tag and store it in the GitLab database. ```json { @@ -166,7 +166,7 @@ Parameters: ## Create a new release -Add release notes to the existing git tag. If there +Add release notes to the existing Git tag. If there already exists a release for the given tag, status code `409` is returned. ``` diff --git a/doc/ci/README.md b/doc/ci/README.md index 94da8354f0b..4be13204227 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -174,7 +174,7 @@ been necessary. These are: #### 12.0 -- [Use refspec to clone/fetch git +- [Use refspec to clone/fetch Git repository](https://gitlab.com/gitlab-org/gitlab-runner/issues/4069). - [Old cache configuration](https://gitlab.com/gitlab-org/gitlab-runner/issues/4070). diff --git a/doc/customization/libravatar.md b/doc/customization/libravatar.md index 1c3bf877fa1..6d96528f760 100644 --- a/doc/customization/libravatar.md +++ b/doc/customization/libravatar.md @@ -14,7 +14,7 @@ server. ## Configuration -In the [gitlab.yml gravatar section](https://gitlab.com/gitlab-org/gitlab-ce/blob/672bd3902d86b78d730cea809fce312ec49d39d7/config/gitlab.yml.example#L122), set +In the [`gitlab.yml` gravatar section](https://gitlab.com/gitlab-org/gitlab-ce/blob/672bd3902d86b78d730cea809fce312ec49d39d7/config/gitlab.yml.example#L122), set the configuration options as follows: ### For HTTP @@ -46,7 +46,7 @@ For example, you host a service on `http://libravatar.example.com` and the `http://libravatar.example.com/avatar/%{hash}?s=%{size}&d=identicon` -### Omnibus-gitlab example +### Omnibus GitLab example In `/etc/gitlab/gitlab.rb`: diff --git a/doc/customization/system_header_and_footer_messages.md b/doc/customization/system_header_and_footer_messages.md index 15830be4e8a..bd2de3e201c 100644 --- a/doc/customization/system_header_and_footer_messages.md +++ b/doc/customization/system_header_and_footer_messages.md @@ -8,7 +8,7 @@ Navigate to the **Admin** area and go to the **Appearance** page. Under **System header and footer** insert your header message and/or footer message. Both background and font color of the header and footer are customizable. -You can also apply the header and footer messages to gitlab emails, +You can also apply the header and footer messages to GitLab emails, by checking the **Enable header and footer in emails** checkbox. Note that color settings will only be applied within the app interface and not to emails diff --git a/doc/development/README.md b/doc/development/README.md index 5e35d4c7437..3912a828dec 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -65,6 +65,7 @@ description: 'Learn how to contribute to GitLab.' - [Repository mirroring](repository_mirroring.md) - [Git LFS](lfs.md) - [Developing against interacting components or features](interacting_components.md) +- [File uploads](uploads.md) ## Performance guides diff --git a/doc/development/file_storage.md b/doc/development/file_storage.md index 475d1c1611e..44af2b020a4 100644 --- a/doc/development/file_storage.md +++ b/doc/development/file_storage.md @@ -2,6 +2,8 @@ We use the [CarrierWave] gem to handle file upload, store and retrieval. +File uploads should be accelerated by workhorse, for details please refer to [uploads development documentation](uploads.md). + There are many places where file uploading is used, according to contexts: - System diff --git a/doc/development/uploads.md b/doc/development/uploads.md new file mode 100644 index 00000000000..234539bb673 --- /dev/null +++ b/doc/development/uploads.md @@ -0,0 +1,271 @@ +# Uploads development documentation + +[GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) has special rules for handling uploads. +To prevent occupying a ruby process on I/O operations, we process the upload in workhorse, where is cheaper. +This process can also directly upload to object storage. + +## The problem description + +The following graph explains machine boundaries in a scalable GitLab installation. Without any workhorse optimization in place, we can expect incoming requests to follow the numbers on the arrows. + +```mermaid +graph TB + subgraph "load balancers" + LB(HA Proxy) + end + + subgraph "Shared storage" + nfs(NFS) + end + + subgraph "redis cluster" + r(persisted redis) + end + LB-- 1 -->workhorse + + subgraph "web or API fleet" + workhorse-- 2 -->rails + end + rails-- "3 (write files)" -->nfs + rails-- "4 (schedule a job)" -->r + + subgraph sidekiq + s(sidekiq) + end + s-- "5 (fetch a job)" -->r + s-- "6 (read files)" -->nfs +``` + +We have three challenges here: performance, availability, and scalability. + +### Performance + +Rails process are expensive in terms of both CPU and memory. Ruby [global interpreter lock](https://en.wikipedia.org/wiki/Global_interpreter_lock) adds to cost too because the ruby process will spend time on I/O operations on step 3 causing incoming requests to pile up. + +In order to improve this, [workhorse disk acceleration](#workhorse-disk-acceleration) was implemented. With this, Rails no longer deals with writing uploaded files to disk. + +```mermaid +graph TB + subgraph "load balancers" + LB(HA Proxy) + end + + subgraph "Shared storage" + nfs(NFS) + end + + subgraph "redis cluster" + r(persisted redis) + end + LB-- 1 -->workhorse + + subgraph "web or API fleet" + workhorse-- "3 (without files)" -->rails + end + workhorse -- "2 (write files)" -->nfs + rails-- "4 (schedule a job)" -->r + + subgraph sidekiq + s(sidekiq) + end + s-- "5 (fetch a job)" -->r + s-- "6 (read files)" -->nfs +``` + +### Availability + +There's also an availability problem in this setup, NFS is a [single point of failure](https://en.wikipedia.org/wiki/Single_point_of_failure). + +To address this problem an HA object storage can be used and it's supported by [workhorse object storage acceleration](#workhorse-object-storage-acceleration) + +### Scalability + +Scaling NFS is outside of our support scope, and NFS is not a part of cloud native installations. + +All features that require sidekiq and do not use object storage acceleration won't work without NFS. In Kubernetes, machine boundaries translate to PODs, and in this case the uploaded file will be written into the POD private disk. Since sidekiq POD cannot reach into other pods, the operation will fail to read it. + +## How to select the proper level of acceleration? + +Selecting the proper acceleration is a tradeoff between speed of development and operational costs. + +We can identify three major use-cases for an upload: + +1. **storage:** if we are uploading for storing a file (i.e. artifacts, packages, discussion attachments). In this case [object storage acceleration](#workhorse-object-storage-acceleration) is the proper level as it's the less resource-intensive operation. Additional information can be found on [File Storage in GitLab](file_storage.md). +1. **in-controller/synchronous processing:** if we allow processing **small files** synchronously, using [disk acceleration](#workhorse-disk-acceleration) may speed up development. +1. **sidekiq/asynchronous processing:** Async processing must implement [object storage acceleration](#workhorse-object-storage-acceleration), the reason being that it's the only way to support Cloud Native deployments without a shared NFS. + +For more details about currently broken feature see [epic &1802](https://gitlab.com/groups/gitlab-org/-/epics/1802). + +### Handling repository uploads + +Some features involves git repository uploads without using a regular git client. +Some examples are uploading a repository file from the web interface and [design management](../user/project/issues/design_management.md). + +Those uploads requires the rails controller to act as a git client in lieu of the user. +Those operation falls into _in-controller/synchronous processing_ category, but we have no warranties on the file size. + +In case of a LFS upload, the file pointer is committed synchronously, but file upload to object storage is performed asynchronously with sidekiq. + +## Upload encodings + +By upload encoding we mean how the file is included within the incoming request. + +We have three kinds of file encoding in our uploads: + +1. <i class="fa fa-check-circle"></i> **multipart**: `multipart/form-data` is the most common, a file is encoded as a part of a multipart encoded request. +1. <i class="fa fa-check-circle"></i> **body**: some APIs uploads files as the whole request body. +1. <i class="fa fa-times-circle"></i> **JSON**: some JSON API uploads files as base64 encoded strings. This requires [gitlab-workhorse#226](https://gitlab.com/gitlab-org/gitlab-workhorse/issues/226) to be implemented. + +## Uploading technologies + +By uploading technologies we mean how all the involved services interact with each other. + +GitLab supports 3 kinds of uploading technologies, here follows a brief description with a sequence diagram for each one. Diagrams are not meant to be exhaustive. + +### Regular rails upload + +This is the default kind of upload, and it's most expensive in terms of resources. + +In this case, workhorse is unaware of files being uploaded and acts as a regular proxy. + +When a multipart request reaches the rails application, `Rack::Multipart` leaves behind tempfiles in `/tmp` and uses valuable Ruby process time to copy files around. + +```mermaid +sequenceDiagram + participant c as Client + participant w as Workhorse + participant r as Rails + + activate c + c ->>+w: POST /some/url/upload + w->>+r: POST /some/url/upload + + r->>r: save the incoming file on /tmp + r->>r: read the file for processing + + r-->>-c: request result + deactivate c + deactivate w +``` + +### Workhorse disk acceleration + +This kind of upload avoids wasting resources caused by handling upload writes to `/tmp` in rails. + +This optimization is not active by default on REST API requests. + +When enabled, Workhorse looks for files in multipart MIME requests, uploading +any it finds to a temporary file on shared storage. The MIME data in the request +is replaced with the path to the corresponding file before it is forwarded to +Rails. + +To prevent abuse of this feature, Workhorse signs the modified request with a +special header, stating which entries it modified. Rails will ignore any +unsigned path entries. + +```mermaid +sequenceDiagram + participant c as Client + participant w as Workhorse + participant r as Rails + participant s as NFS + + activate c + c ->>+w: POST /some/url/upload + + w->>+s: save the incoming file on a temporary location + s-->>-w: + + w->>+r: POST /some/url/upload + Note over w,r: file was replaced with its location<br>and other metadata + + opt requires async processing + r->>+redis: schedule a job + redis-->>-r: + end + + r-->>-c: request result + deactivate c + w->>-w: cleanup + + opt requires async processing + activate sidekiq + sidekiq->>+redis: fetch a job + redis-->>-sidekiq: job + + sidekiq->>+s: read file + s-->>-sidekiq: file + + sidekiq->>sidekiq: process file + + deactivate sidekiq + end +``` + +### Workhorse object storage acceleration + +This is the more advanced acceleration technique we have in place. + +Workhorse asks rails for temporary pre-signed object storage URLs and directly uploads to object storage. + +In this setup an extra rails route needs to be implemented in order to handle authorization, +you can see an example of this in [`Projects::LfsStorageController`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/controllers/projects/lfs_storage_controller.rb) +and [its routes](https://gitlab.com/gitlab-org/gitlab-ce/blob/v12.2.0/config/routes/git_http.rb#L31-32). + +**note:** this will fallback to _Workhorse disk acceleration_ when object storage is not enabled in the gitlab instance. The answer to the `/authorize` call will only contain a file system path. + +```mermaid +sequenceDiagram + participant c as Client + participant w as Workhorse + participant r as Rails + participant os as Object Storage + + activate c + c ->>+w: POST /some/url/upload + + w ->>+r: POST /some/url/upload/authorize + Note over w,r: this request has an empty body + r-->>-w: presigned OS URL + + w->>+os: PUT file + Note over w,os: file is stored on a temporary location. Rails select the destination + os-->>-w: + + w->>+r: POST /some/url/upload + Note over w,r: file was replaced with its location<br>and other metadata + + r->>+os: move object to final destination + os-->>-r: + + opt requires async processing + r->>+redis: schedule a job + redis-->>-r: + end + + r-->>-c: request result + deactivate c + w->>-w: cleanup + + opt requires async processing + activate sidekiq + sidekiq->>+redis: fetch a job + redis-->>-sidekiq: job + + sidekiq->>+os: get object + os-->>-sidekiq: file + + sidekiq->>sidekiq: process file + + deactivate sidekiq + end +``` + +## What does the `direct_upload` setting mean? + +[Object storage setting](../administration/uploads.md#object-storage-settings) allows instance administators to enable `direct_upload`, this in an option that only affects the behavior of [workhorse object storage acceleration](#workhorse-object-storage-acceleration). + +This option affect the response to the `/authorize` call. When not enabled, the API response will not contain presigned URLs and workhorse will write the file the shared disk, on the path is provided by rails, acting like object storage was disabled. + +Once the request reachs rails, it will schedule an object storage upload as a sidekiq job. + diff --git a/doc/policy/maintenance.md b/doc/policy/maintenance.md index 018c273c51a..f7ba7c16a9e 100644 --- a/doc/policy/maintenance.md +++ b/doc/policy/maintenance.md @@ -86,6 +86,7 @@ Please see the table below for some examples: | 9.4.5 | 8.13.4 | `8.13.4` -> `8.17.7` -> `9.4.5` | `8.17.7` is the last version in version `8` | | 10.1.4 | 8.13.4 | `8.13.4 -> 8.17.7 -> 9.5.10 -> 10.1.4` | `8.17.7` is the last version in version `8`, `9.5.10` is the last version in version `9` | | 11.3.4 | 8.13.4 | `8.13.4` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> `11.3.4` | `8.17.7` is the last version in version `8`, `9.5.10` is the last version in version `9`, `10.8.7` is the last version in version `10` | +| 12.0.2 | 11.3.4 | `11.3.4` -> `11.11.x` -> `12.0.2` | `11.11.x` is the last version in version `11` More information about the release procedures can be found in our [release documentation](https://gitlab.com/gitlab-org/release/docs). You may also want to read our diff --git a/doc/user/project/merge_requests/img/approvals_premium_project_edit.png b/doc/user/project/merge_requests/img/approvals_premium_project_edit.png Binary files differdeleted file mode 100644 index 6a09412533f..00000000000 --- a/doc/user/project/merge_requests/img/approvals_premium_project_edit.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/approvals_premium_project_edit_v12_3.png b/doc/user/project/merge_requests/img/approvals_premium_project_edit_v12_3.png Binary files differnew file mode 100644 index 00000000000..32b9a3b9ce4 --- /dev/null +++ b/doc/user/project/merge_requests/img/approvals_premium_project_edit_v12_3.png diff --git a/doc/user/project/merge_requests/merge_request_approvals.md b/doc/user/project/merge_requests/merge_request_approvals.md index 5161b25de99..a0432414bcf 100644 --- a/doc/user/project/merge_requests/merge_request_approvals.md +++ b/doc/user/project/merge_requests/merge_request_approvals.md @@ -83,7 +83,7 @@ request approval rules: 1. Give the approval rule a name that describes the set of approvers selected. 1. Click **Add approvers** to submit the new rule. -  +  ## Multiple approval rules **(PREMIUM)** |
