diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-12-14 18:08:42 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-12-14 18:08:42 +0000 |
commit | 870dfaa9127e114a6ea2066220760815063fb3de (patch) | |
tree | 687cdcdc75e56796a8711511d9d0e4a56ff4822f /doc | |
parent | 8c4225a66b12683bcf1bba9bb9328fcf65395b6d (diff) | |
download | gitlab-ce-870dfaa9127e114a6ea2066220760815063fb3de.tar.gz |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r-- | doc/administration/auth/ldap/index.md | 47 | ||||
-rw-r--r-- | doc/administration/geo/replication/troubleshooting.md | 18 | ||||
-rw-r--r-- | doc/administration/gitaly/index.md | 41 | ||||
-rw-r--r-- | doc/administration/nfs.md | 49 | ||||
-rw-r--r-- | doc/administration/reference_architectures/10k_users.md | 40 | ||||
-rw-r--r-- | doc/administration/reference_architectures/25k_users.md | 40 | ||||
-rw-r--r-- | doc/administration/reference_architectures/2k_users.md | 30 | ||||
-rw-r--r-- | doc/administration/reference_architectures/3k_users.md | 41 | ||||
-rw-r--r-- | doc/administration/reference_architectures/50k_users.md | 40 | ||||
-rw-r--r-- | doc/administration/reference_architectures/5k_users.md | 41 | ||||
-rw-r--r-- | doc/api/graphql/reference/index.md | 2 | ||||
-rw-r--r-- | doc/topics/autodevops/troubleshooting.md | 2 | ||||
-rw-r--r-- | doc/update/index.md | 75 | ||||
-rw-r--r-- | doc/update/removals.md | 7 | ||||
-rw-r--r-- | doc/user/application_security/index.md | 4 |
15 files changed, 187 insertions, 290 deletions
diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md index 0f41cbae923..2cb9bac7af9 100644 --- a/doc/administration/auth/ldap/index.md +++ b/doc/administration/auth/ldap/index.md @@ -364,7 +364,9 @@ This can lead to several confusing issues such as creating links or namespaces w GitLab can automatically lowercase usernames provided by the LDAP server by enabling the configuration option `lowercase_usernames`. By default, this configuration option is `false`. -**Omnibus configuration** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -379,7 +381,7 @@ the configuration option `lowercase_usernames`. By default, this configuration o 1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -**Source configuration** +:::TabTitle Self-compiled (source) 1. Edit `config/gitlab.yaml`: @@ -394,6 +396,8 @@ the configuration option `lowercase_usernames`. By default, this configuration o 1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ### Disable LDAP web sign in It can be useful to prevent using LDAP credentials through the web UI when @@ -404,7 +408,9 @@ checks like custom 2FA. When LDAP web sign in is disabled, users don't see an **LDAP** tab on the sign-in page. This does not disable using LDAP credentials for Git access. -**Omnibus configuration** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -414,7 +420,30 @@ This does not disable using LDAP credentials for Git access. 1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -**Source configuration** +:::TabTitle Helm chart (Kubernetes) + +1. Export the Helm values: + + ```shell + helm get values gitlab > gitlab_values.yaml + ``` + +1. Edit `gitlab_values.yaml`: + + ```yaml + global: + appConfig: + ldap: + preventSignin: true + ``` + +1. Save the file and apply the new values: + + ```shell + helm upgrade -f gitlab_values.yaml gitlab gitlab/gitlab + ``` + +:::TabTitle Self-compiled (source) 1. Edit `config/gitlab.yaml`: @@ -426,6 +455,8 @@ This does not disable using LDAP credentials for Git access. 1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ### Use encrypted credentials Instead of having the LDAP integration credentials stored in plaintext in the configuration files, you can optionally @@ -445,7 +476,9 @@ The supported configuration items for the encrypted file are: The encrypted contents can be configured with the [LDAP secret edit Rake command](../../raketasks/ldap.md#edit-secret). -**Omnibus configuration** +::Tabs + +:::TabTitle Linux package (Omnibus) If initially your LDAP configuration looked like: @@ -479,7 +512,7 @@ If initially your LDAP configuration looked like: 1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -**Source configuration** +:::TabTitle Self-compiled (source) If initially your LDAP configuration looked like: @@ -513,6 +546,8 @@ If initially your LDAP configuration looked like: 1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ## Updating LDAP DN and email When an LDAP server creates a user in GitLab, the user's LDAP distinguished name (DN) is linked to their GitLab account diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 31477bf2919..1dcced781ce 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -49,6 +49,8 @@ health check manually to get this information and a few more details. #### Health check Rake task +> The use of a custom NTP server was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/105514) in GitLab 15.7. + This Rake task can be run on a **Rails** node in the **primary** or **secondary** Geo sites: @@ -80,6 +82,22 @@ All projects are in hashed storage? ... yes Checking Geo ... Finished ``` +You can also specify a custom NTP server using environment variables. For example: + +```shell +export NTP_HOST="ntp.ubuntu.com" +export NTP_TIMEOUT="30" +sudo gitlab-rake gitlab:geo:check +``` + +The following environment variables are supported. + +| Variable | Description | Default value | +| ----------- | ----------- | ------------- | +|`NTP_HOST` | The NTP host. | `pool.ntp.org` | +|`NTP_PORT` | The NTP port the host listens on. |`ntp`| +|`NTP_TIMEOUT`| The NTP timeout in seconds. | The value defined in the `net-ntp` Ruby library ([60 seconds](https://github.com/zencoder/net-ntp/blob/3d0990214f439a5127782e0f50faeaf2c8ca7023/lib/net/ntp/ntp.rb#L6)). | + #### Sync status Rake task Current sync information can be found manually by running this Rake task on any diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index c24664d87f0..32b44552b1a 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -54,9 +54,8 @@ Before deploying Gitaly Cluster, review: If you have: -- Not yet migrated to Gitaly Cluster and want to continue using NFS, remain on the service you are using. NFS is - supported in 14.x releases but is [deprecated](../../update/deprecations.md#nfs-for-git-repository-storage). - Support for storing Git repository data on NFS is scheduled to end for all versions of GitLab on November 22, 2022. +- Not yet migrated to Gitaly Cluster and want to continue using NFS, remain on the service you are using. However, NFS + is [no longer supported](../../update/removals.md#nfs-as-git-repository-storage-is-no-longer-supported). - Not yet migrated to Gitaly Cluster but want to migrate away from NFS, you have two options: - A sharded Gitaly instance. - Gitaly Cluster. @@ -104,35 +103,8 @@ These IOPS values are initial recommendations, and may be adjusted to greater or depending on the scale of your environment's workload. If you’re running the environment on a cloud provider, refer to their documentation about how to configure IOPS correctly. -For repository data, only local storage is supported for Gitaly and Gitaly Cluster for performance and consistency reasons. Alternatives such as -[NFS](#moving-beyond-nfs) or [cloud-based systems](../nfs.md#avoid-using-cloud-based-file-systems) are not supported. - -### Moving beyond NFS - -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be unavailable starting -November 22, 2022. See our [statement of support](https://about.gitlab.com/support/statement-of-support/#gitaly-and-nfs) -for more details. - -[Network File System (NFS)](https://en.wikipedia.org/wiki/Network_File_System) -is not well suited to Git workloads which are CPU and IOPS sensitive. -Specifically: - -- Git is sensitive to file system latency. Some operations require many - read operations. Operations that are fast on block storage can become an order of - magnitude slower. This significantly impacts GitLab application performance. -- NFS performance optimizations that prevent the performance gap between - block storage and NFS being even wider are vulnerable to race conditions. We have observed - [data inconsistencies](https://gitlab.com/gitlab-org/gitaly/-/issues/2589) - in production environments caused by simultaneous writes to different NFS - clients. Data corruption is not an acceptable risk. - -Gitaly Cluster is purpose built to provide reliable, high performance, fault -tolerant Git storage. - -Further reading: - -- Blog post: [The road to Gitaly v1.0 (aka, why GitLab doesn't require NFS for storing Git data anymore)](https://about.gitlab.com/blog/2018/09/12/the-road-to-gitaly-1-0/) -- Blog post: [How we spent two weeks hunting an NFS bug in the Linux kernel](https://about.gitlab.com/blog/2018/11/14/how-we-spent-two-weeks-hunting-an-nfs-bug/) +For repository data, only local storage is supported for Gitaly and Gitaly Cluster for performance and consistency reasons. +Alternatives such as [NFS](../nfs.md) or [cloud-based file systems](../nfs.md#avoid-using-cloud-based-file-systems) are not supported. ## Directly accessing repositories @@ -713,8 +685,3 @@ There are two facets to our efforts to remove direct Git access in GitLab: The second facet presents the only real solution. For this, we developed [Gitaly Cluster](#gitaly-cluster). - -## NFS deprecation notice - -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be -unavailable beginning November 22, 2022. For further information, see our [NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation) documentation. diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md index 85f35a1b188..972db315268 100644 --- a/doc/administration/nfs.md +++ b/doc/administration/nfs.md @@ -20,47 +20,14 @@ actions that read or write to Git repositories. For steps you can use to test file system performance, see [File System Performance Benchmarking](operations/filesystem_benchmarking.md). -## Gitaly and NFS deprecation +## Gitaly with NFS not supported -Starting with GitLab version 14.0, support for NFS to store Git repository data is deprecated. Technical customer support and engineering support is available for the 14.x releases. Engineering is fixing bugs and security vulnerabilities consistent with our [release and maintenance policy](../policy/maintenance.md#security-releases). +Technical and engineering support for using NFS to store Git repository data is officially at end-of-life. No product +changes or troubleshooting is provided through engineering, security or paid support channels. -Upon the release of GitLab 15.6 technical and engineering support for using NFS to store Git repository data is officially at end-of-life. There are no product changes or troubleshooting provided via Engineering, Security or Paid Support channels after the release date of 15.6, regardless of your GitLab version. - -Until the release of 15.6, for customers running 14.x releases, we continue to help with Git related tickets from customers running one or more Gitaly servers with its data stored on NFS. Examples may include: - -- Performance issues or timeouts accessing Git data -- Commits or branches vanish -- GitLab intermittently returns the wrong Git data (such as reporting that a repository has no branches) - -Assistance is limited to activities like: - -- Verifying developers' workflow uses features like protected branches -- Reviewing GitLab event data from the database to advise if it looks like a force push over-wrote branches -- Verifying that NFS client mount options match our [documented recommendations](#mount-options) -- Analyzing the GitLab Workhorse and Rails logs, and determining that `500` errors being seen in the environment are caused by slow responses from Gitaly - -GitLab support is unable to continue with the investigation if both: - -- The date of the request is on or after the release of GitLab version 15.6. -- Support Engineers and Management determine that all reasonable non-NFS root causes have been exhausted. - -If the issue is reproducible, or if it happens intermittently but regularly, GitLab Support can investigate providing the issue reproduces without the use of NFS. To reproduce without NFS, the affected repositories should be migrated to a different Gitaly shard, such as Gitaly cluster or a standalone Gitaly VM, backed with block storage. - -### Why remove NFS for Git repository data - -{:.no-toc} - -NFS is not well-suited to a workload consisting of many small files, like Git repositories. NFS does provide a number of configuration options designed to improve performance. However, over time, a number of these mount options have proven to result in inconsistencies across multiple nodes mounting the NFS volume, up to and including data loss. Addressing these inconsistencies consume extraordinary development and support engineer time that hamper our ability to develop [Gitaly Cluster](gitaly/praefect.md), our purpose-built solution to addressing the deficiencies of NFS in this environment. - -Gitaly Cluster provides highly-available Git repository storage. If this is not a requirement, single-node Gitaly backed by block storage is a suitable substitute. - -Engineering support for NFS for Git repositories is deprecated. Technical support is planned to be -unavailable from GitLab 15.0. No further enhancements are planned for this feature. - -Read: - -- [Moving beyond NFS](gitaly/index.md#moving-beyond-nfs). -- About the [correct mount options to use](#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). +If an issue is reproducible, or if it happens intermittently but regularly, GitLab Support can investigate providing the +issue can be reproduced without NFS. To reproduce without NFS, migrate the affected repositories to a different Gitaly +shard. For example, a Gitaly Cluster or a standalone Gitaly VM, backed with block storage. ## Known kernel version incompatibilities @@ -397,8 +364,8 @@ sudo ufw allow from <client_ip_address> to any port nfs ### Upgrade to Gitaly Cluster or disable caching if experiencing data loss WARNING: -Engineering support for NFS for Git repositories is deprecated. Read about -[moving beyond NFS](gitaly/index.md#moving-beyond-nfs). +Engineering support for NFS for Git repositories +[is unavailable](../update/removals.md#nfs-as-git-repository-storage-is-no-longer-supported). Customers and users have reported data loss on high-traffic repositories when using NFS for Git repositories. For example, we have seen: diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index f7667ec6783..88913eb1f7f 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -35,7 +35,6 @@ full list of reference architectures, see | GitLab Rails | 3 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | | Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Object storage<sup>4</sup> | - | - | - | - | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> @@ -228,9 +227,6 @@ To set up GitLab and its components to accommodate up to 10,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) - to have shared disk storage service as an alternative to Gitaly or object - storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. @@ -1760,9 +1756,8 @@ To configure Praefect with TLS: Sidekiq requires connection to the [Redis](#configure-redis), [PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. - `10.6.0.101`: Sidekiq 1 - `10.6.0.102`: Sidekiq 2 @@ -1930,9 +1925,8 @@ run [multiple Sidekiq processes](../sidekiq/extra_sidekiq_processes.md). ## Configure GitLab Rails This section describes how to configure the GitLab application (Rails) component. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. The following IPs will be used as an example: @@ -2071,7 +2065,6 @@ On each node perform the following: 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from the first Omnibus node you configured and add or replace the file of the same name on this server. If this is the first Omnibus node you are configuring then you can skip this step. - 1. To ensure database migrations are only run during reconfigure and not automatically on upgrade, run: ```shell @@ -2082,9 +2075,7 @@ On each node perform the following: [GitLab Rails post-configuration](#gitlab-rails-post-configuration) section. 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -1. [Enable incremental logging](#enable-incremental-logging), unless you are using [NFS](#configure-nfs-optional). - +1. [Enable incremental logging](#enable-incremental-logging). 1. Confirm the node can connect to Gitaly: ```shell @@ -2210,9 +2201,6 @@ To configure the Monitoring node: ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs-optional) and in general it's better -in larger setups as object storage is typically much more performant, reliable, -and scalable. GitLab has been tested on a number of object storage providers: @@ -2236,7 +2224,7 @@ NOTE: When using the [storage-specific form](../object_storage.md#storage-specific-configuration) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, -which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). +which was deprecated in 14.9, requires shared storage such as NFS. Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2255,22 +2243,6 @@ GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. -## Configure NFS (optional) - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) -after the release of GitLab 15.6. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 25280d3a06d..02739904f5e 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -35,7 +35,6 @@ full list of reference architectures, see | GitLab Rails | 5 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | | Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Object storage<sup>4</sup> | - | - | - | - | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> @@ -228,9 +227,6 @@ To set up GitLab and its components to accommodate up to 25,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) - to have shared disk storage service as an alternative to Gitaly or object - storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. @@ -1778,9 +1774,8 @@ To configure Praefect with TLS: Sidekiq requires connection to the [Redis](#configure-redis), [PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. - `10.6.0.101`: Sidekiq 1 - `10.6.0.102`: Sidekiq 2 @@ -1948,9 +1943,8 @@ run [multiple Sidekiq processes](../sidekiq/extra_sidekiq_processes.md). ## Configure GitLab Rails This section describes how to configure the GitLab application (Rails) component. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. The following IPs will be used as an example: @@ -2091,7 +2085,6 @@ On each node perform the following: 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from the first Omnibus node you configured and add or replace the file of the same name on this server. If this is the first Omnibus node you are configuring then you can skip this step. - 1. To ensure database migrations are only run during reconfigure and not automatically on upgrade, run: ```shell @@ -2102,9 +2095,7 @@ On each node perform the following: [GitLab Rails post-configuration](#gitlab-rails-post-configuration) section. 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -1. [Enable incremental logging](#enable-incremental-logging), unless you are using [NFS](#configure-nfs-optional). - +1. [Enable incremental logging](#enable-incremental-logging). 1. Confirm the node can connect to Gitaly: ```shell @@ -2229,9 +2220,6 @@ To configure the Monitoring node: ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs-optional) and in general it's better -in larger setups as object storage is typically much more performant, reliable, -and scalable. GitLab has been tested on a number of object storage providers: @@ -2255,7 +2243,7 @@ NOTE: When using the [storage-specific form](../object_storage.md#storage-specific-configuration) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, -which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). +which was deprecated in 14.9, requires shared storage such as NFS. Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2274,22 +2262,6 @@ GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. -## Configure NFS (optional) - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) -after the release of GitLab 15.6. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 6814356074d..f41c8e9cb24 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -29,7 +29,6 @@ For a full list of reference architectures, see | GitLab Rails | 2 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | `F8s v2` | | Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | `F2s v2` | | Object storage<sup>4</sup> | - | - | - | - | - | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | <!-- markdownlint-disable MD029 --> 1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. See [Recommended cloud providers and services](index.md#recommended-cloud-providers-and-services) for more information. @@ -152,9 +151,6 @@ To set up GitLab and its components to accommodate up to 2,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) - to have shared disk storage service as an alternative to Gitaly or object - storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. @@ -781,9 +777,7 @@ On each node perform the following: [GitLab Rails post-configuration](#gitlab-rails-post-configuration) section. 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -1. [Enable incremental logging](#enable-incremental-logging), unless you are using [NFS](#configure-nfs-optional). - +1. [Enable incremental logging](#enable-incremental-logging). 1. Run `sudo gitlab-rake gitlab:gitaly:check` to confirm the node can connect to Gitaly. 1. Tail the logs to see the requests: @@ -931,9 +925,6 @@ running [Prometheus](../monitoring/prometheus/index.md) and ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs-optional) and in general it's better -in larger setups as object storage is typically much more performant, reliable, -and scalable. GitLab has been tested on a number of object storage providers: @@ -958,7 +949,7 @@ NOTE: When using the [storage-specific form](../object_storage.md#storage-specific-configuration) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, -which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). +which was deprecated in 14.9, requires shared storage such as NFS. Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -977,23 +968,6 @@ GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. -## Configure NFS (optional) - -For improved performance, [object storage](#configure-the-object-storage), -along with [Gitaly](#configure-gitaly), are recommended over using NFS whenever -possible. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) -after the release of GitLab 15.6. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Configure Advanced Search **(PREMIUM SELF)** You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 6c2b97fc33c..008b5ffcc0e 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -44,7 +44,6 @@ For a full list of reference architectures, see | GitLab Rails | 3 | 8 vCPU, 7.2 GB memory | `n1-highcpu-8` | `c5.2xlarge` | | Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | | Object storage<sup>4</sup> | - | - | - | - | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> @@ -234,9 +233,6 @@ To set up GitLab and its components to accommodate up to 3,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) - to have shared disk storage service as an alternative to Gitaly or object - storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. @@ -1712,9 +1708,8 @@ To configure Praefect with TLS: Sidekiq requires connection to the [Redis](#configure-redis), [PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. The following IPs will be used as an example: @@ -1881,9 +1876,8 @@ run [multiple Sidekiq processes](../sidekiq/extra_sidekiq_processes.md). ## Configure GitLab Rails This section describes how to configure the GitLab application (Rails) component. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. On each node perform the following: @@ -2022,7 +2016,6 @@ On each node perform the following: 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from the first Omnibus node you configured and add or replace the file of the same name on this server. If this is the first Omnibus node you are configuring then you can skip this step. - 1. To ensure database migrations are only run during reconfigure and not automatically on upgrade, run: ```shell @@ -2033,11 +2026,8 @@ On each node perform the following: [GitLab Rails post-configuration](#gitlab-rails-post-configuration) section. 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -1. [Enable incremental logging](#enable-incremental-logging), unless you are using [NFS](#configure-nfs-optional). - +1. [Enable incremental logging](#enable-incremental-logging). 1. Run `sudo gitlab-rake gitlab:gitaly:check` to confirm the node can connect to Gitaly. - 1. Tail the logs to see the requests: ```shell @@ -2176,9 +2166,6 @@ running [Prometheus](../monitoring/prometheus/index.md) and ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs-optional) and in general it's better -in larger setups as object storage is typically much more performant, reliable, -and scalable. GitLab has been tested on a number of object storage providers: @@ -2202,7 +2189,7 @@ NOTE: When using the [storage-specific form](../object_storage.md#storage-specific-configuration) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, -which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). +which was deprecated in 14.9, requires shared storage such as NFS. Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2221,22 +2208,6 @@ GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. -## Configure NFS (optional) - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) -after the release of GitLab 15.6. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 60c0bf024cc..87d1408b568 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -35,7 +35,6 @@ full list of reference architectures, see | GitLab Rails | 12 | 32 vCPU, 28.8 GB memory | `n1-highcpu-32` | `c5.9xlarge` | | Monitoring node | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | | Object storage<sup>4</sup> | - | - | - | - | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> @@ -228,9 +227,6 @@ To set up GitLab and its components to accommodate up to 50,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) - to have shared disk storage service as an alternative to Gitaly or object - storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. @@ -1773,9 +1769,8 @@ To configure Praefect with TLS: Sidekiq requires connection to the [Redis](#configure-redis), [PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. - `10.6.0.101`: Sidekiq 1 - `10.6.0.102`: Sidekiq 2 @@ -1943,9 +1938,8 @@ run [multiple Sidekiq processes](../sidekiq/extra_sidekiq_processes.md). ## Configure GitLab Rails This section describes how to configure the GitLab application (Rails) component. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. The following IPs will be used as an example: @@ -2093,7 +2087,6 @@ On each node perform the following: 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from the first Omnibus node you configured and add or replace the file of the same name on this server. If this is the first Omnibus node you are configuring then you can skip this step. - 1. To ensure database migrations are only run during reconfigure and not automatically on upgrade, run: ```shell @@ -2104,9 +2097,7 @@ On each node perform the following: [GitLab Rails post-configuration](#gitlab-rails-post-configuration) section. 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -1. [Enable incremental logging](#enable-incremental-logging), unless you are using [NFS](#configure-nfs-optional). - +1. [Enable incremental logging](#enable-incremental-logging). 1. Confirm the node can connect to Gitaly: ```shell @@ -2231,9 +2222,6 @@ To configure the Monitoring node: ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs-optional) and in general it's better -in larger setups as object storage is typically much more performant, reliable, -and scalable. GitLab has been tested on a number of object storage providers: @@ -2257,7 +2245,7 @@ NOTE: When using the [storage-specific form](../object_storage.md#storage-specific-configuration) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, -which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). +which was deprecated in 14.9, requires shared storage such as NFS. Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2276,22 +2264,6 @@ GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. -## Configure NFS (optional) - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) -after the release of GitLab 15.6. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index ddc9575efb4..182edb82b5f 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -41,7 +41,6 @@ costly-to-operate environment by using the | GitLab Rails | 3 | 16 vCPU, 14.4 GB memory | `n1-highcpu-16` | `c5.4xlarge` | | Monitoring node | 1 | 2 vCPU, 1.8 GB memory | `n1-highcpu-2` | `c5.large` | | Object storage<sup>4</sup> | - | - | - | - | -| NFS server (non-Gitaly) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> @@ -231,9 +230,6 @@ To set up GitLab and its components to accommodate up to 5,000 users: environment. 1. [Configure the object storage](#configure-the-object-storage) used for shared data objects. -1. [Configure NFS](#configure-nfs-optional) (optional, and not recommended) - to have shared disk storage service as an alternative to Gitaly or object - storage. 1. [Configure Advanced Search](#configure-advanced-search) (optional) for faster, more advanced code search across your entire GitLab instance. @@ -1709,9 +1705,8 @@ To configure Praefect with TLS: Sidekiq requires connection to the [Redis](#configure-redis), [PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. - `10.6.0.71`: Sidekiq 1 - `10.6.0.72`: Sidekiq 2 @@ -1877,9 +1872,8 @@ run [multiple Sidekiq processes](../sidekiq/extra_sidekiq_processes.md). ## Configure GitLab Rails This section describes how to configure the GitLab application (Rails) component. -Since it's recommended to use [Object storage](#configure-the-object-storage) -over [NFS](#configure-nfs-optional) for data objects, the following examples -include the Object storage configuration. +Because you must use [Object storage](#configure-the-object-storage) instead of NFS for data objects, the following +examples include the Object storage configuration. On each node perform the following: @@ -2021,7 +2015,6 @@ On each node perform the following: 1. Copy the `/etc/gitlab/gitlab-secrets.json` file from the first Omnibus node you configured and add or replace the file of the same name on this server. If this is the first Omnibus node you are configuring then you can skip this step. - 1. To ensure database migrations are only run during reconfigure and not automatically on upgrade, run: ```shell @@ -2032,11 +2025,8 @@ On each node perform the following: [GitLab Rails post-configuration](#gitlab-rails-post-configuration) section. 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -1. [Enable incremental logging](#enable-incremental-logging), unless you are using [NFS](#configure-nfs-optional). - +1. [Enable incremental logging](#enable-incremental-logging). 1. Run `sudo gitlab-rake gitlab:gitaly:check` to confirm the node can connect to Gitaly. - 1. Tail the logs to see the requests: ```shell @@ -2175,9 +2165,6 @@ running [Prometheus](../monitoring/prometheus/index.md) and ## Configure the object storage GitLab supports using an object storage service for holding numerous types of data. -It's recommended over [NFS](#configure-nfs-optional) and in general it's better -in larger setups as object storage is typically much more performant, reliable, -and scalable. GitLab has been tested on a number of object storage providers: @@ -2201,7 +2188,7 @@ NOTE: When using the [storage-specific form](../object_storage.md#storage-specific-configuration) in GitLab 14.x and earlier, you should enable [direct upload mode](../../development/uploads/index.md#direct-upload). The previous [background upload](../../development/uploads/index.md#direct-upload) mode, -which was deprecated in 14.9, requires shared storage such as [NFS](#configure-nfs-optional). +which was deprecated in 14.9, requires shared storage such as NFS. Using separate buckets for each data type is the recommended approach for GitLab. This ensures there are no collisions across the various types of data GitLab stores. @@ -2220,22 +2207,6 @@ GitLab Runner returns job logs in chunks which Omnibus GitLab caches temporarily While sharing the job logs through NFS is supported, it's recommended to avoid the need to use NFS by enabling [incremental logging](../job_logs.md#incremental-logging-architecture) (required when no NFS node has been deployed). Incremental logging uses Redis instead of disk space for temporary caching of job logs. -## Configure NFS (optional) - -[Object storage](#configure-the-object-storage), along with [Gitaly](#configure-gitaly) -are recommended over NFS wherever possible for improved performance. - -See how to [configure NFS](../nfs.md). - -WARNING: -Engineering support for NFS for Git repositories is deprecated, and [technical support is scheduled to be unavailable](../nfs.md#gitaly-and-nfs-deprecation) -after the release of GitLab 15.6. No further enhancements are planned for this feature. - -Read: - -- [Gitaly and NFS Deprecation](../nfs.md#gitaly-and-nfs-deprecation). -- About the [correct mount options to use](../nfs.md#upgrade-to-gitaly-cluster-or-disable-caching-if-experiencing-data-loss). - ## Configure Advanced Search You can leverage Elasticsearch and [enable Advanced Search](../../integration/advanced_search/elasticsearch.md) diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index 87a8deaddd3..ad53715b57c 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -11994,6 +11994,7 @@ Approval summary of the deployment. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="deploymentpermissionsapprovedeployment"></a>`approveDeployment` | [`Boolean!`](#boolean) | Indicates the user can perform `approve_deployment` on this resource. This field can only be resolved for one environment in any single request. | | <a id="deploymentpermissionsdestroydeployment"></a>`destroyDeployment` | [`Boolean!`](#boolean) | Indicates the user can perform `destroy_deployment` on this resource. | | <a id="deploymentpermissionsupdatedeployment"></a>`updateDeployment` | [`Boolean!`](#boolean) | Indicates the user can perform `update_deployment` on this resource. | @@ -22798,6 +22799,7 @@ Name of the feature that the callout is for. | <a id="usercalloutfeaturenameenumuser_reached_limit_free_plan_alert"></a>`USER_REACHED_LIMIT_FREE_PLAN_ALERT` | Callout feature name for user_reached_limit_free_plan_alert. | | <a id="usercalloutfeaturenameenumverification_reminder"></a>`VERIFICATION_REMINDER` | Callout feature name for verification_reminder. | | <a id="usercalloutfeaturenameenumvscode_web_ide"></a>`VSCODE_WEB_IDE` | Callout feature name for vscode_web_ide. | +| <a id="usercalloutfeaturenameenumvscode_web_ide_callout"></a>`VSCODE_WEB_IDE_CALLOUT` | Callout feature name for vscode_web_ide_callout. | | <a id="usercalloutfeaturenameenumweb_ide_alert_dismissed"></a>`WEB_IDE_ALERT_DISMISSED` | Callout feature name for web_ide_alert_dismissed. | | <a id="usercalloutfeaturenameenumweb_ide_ci_environments_guidance"></a>`WEB_IDE_CI_ENVIRONMENTS_GUIDANCE` | Callout feature name for web_ide_ci_environments_guidance. | diff --git a/doc/topics/autodevops/troubleshooting.md b/doc/topics/autodevops/troubleshooting.md index bd8b40dc500..ef420323b32 100644 --- a/doc/topics/autodevops/troubleshooting.md +++ b/doc/topics/autodevops/troubleshooting.md @@ -40,7 +40,7 @@ The following are possible reasons: If your pipeline fails with the following message: ```plaintext -Found errors in your .gitlab-ci.yml: +Unable to create pipeline jobs:test config key may not be used with `rules`: only ``` diff --git a/doc/update/index.md b/doc/update/index.md index 8dda96c56c5..d838f8dda34 100644 --- a/doc/update/index.md +++ b/doc/update/index.md @@ -764,9 +764,82 @@ for how to proceed. gitlab-psql`): ```sql - select count(*) from background_migration_jobs where class_name = 'MigrateMergeRequestDiffCommitUsers' and status = 0; + select status, count(*) from background_migration_jobs + where class_name = 'MigrateMergeRequestDiffCommitUsers' group by status; ``` + As jobs are completed, the database records change from `0` (pending) to `1`. If the number of + pending jobs doesn't decrease after a while, it's possible that the + `MigrateMergeRequestDiffCommitUsers` background migration jobs have failed. You + can check for errors in the Sidekiq logs: + + ```shell + sudo grep MigrateMergeRequestDiffCommitUsers /var/log/gitlab/sidekiq/current | grep -i error + ``` + + If needed, you can attempt to run the `MigrateMergeRequestDiffCommitUsers` background + migration jobs manually in the [GitLab Rails Console](../administration/operations/rails_console.md). + This can be done using Sidekiq asynchronously, or by using a Rails process directly: + + - Using Sidekiq to schedule jobs asynchronously: + + ```ruby + # For the first run, only attempt to execute 1 migration. If successful, increase + # the limit for subsequent runs + limit = 1 + + jobs = Gitlab::Database::BackgroundMigrationJob.for_migration_class('MigrateMergeRequestDiffCommitUsers').pending.to_a + + pp "#{jobs.length} jobs remaining" + + jobs.first(limit).each do |job| + BackgroundMigrationWorker.perform_in(5.minutes, 'MigrateMergeRequestDiffCommitUsers', job.arguments) + end + ``` + + NOTE: + The queued jobs can be monitored using Sidekiq's admin panel, which can be accessed at the `/admin/sidekiq` endpoint URI. + + - Using a Rails process to run jobs synchronously: + + ```ruby + def process(concurrency: 1) + queue = Queue.new + + Gitlab::Database::BackgroundMigrationJob + .where(class_name: 'MigrateMergeRequestDiffCommitUsers', status: 0) + .each { |job| queue << job } + + concurrency + .times + .map do + Thread.new do + Thread.abort_on_exception = true + + loop do + job = queue.pop(true) + time = Benchmark.measure do + Gitlab::BackgroundMigration::MigrateMergeRequestDiffCommitUsers + .new + .perform(*job.arguments) + end + + puts "#{job.id} finished in #{time.real.round(2)} seconds" + rescue ThreadError + break + end + end + end + .each(&:join) + end + + ActiveRecord::Base.logger.level = Logger::ERROR + process + ``` + + NOTE: + When using Rails to execute these background migrations synchronously, make sure that the machine running the process has sufficient resources to handle the task. If the process gets terminated, it's likely due to insufficient memory available. If your SSH session times out after a while, it might be necessary to run the previous code by using a terminal multiplexer like `screen` or `tmux`. + - See [Maintenance mode issue in GitLab 13.9 to 14.4](#maintenance-mode-issue-in-gitlab-139-to-144). - You may see the following error when setting up two factor authentication (2FA) for accounts diff --git a/doc/update/removals.md b/doc/update/removals.md index e4338bae1fc..10d937f0f16 100644 --- a/doc/update/removals.md +++ b/doc/update/removals.md @@ -65,9 +65,12 @@ As of December 22, 2022, we are removing the Flowdock integration because the se ## Removed in 15.6 -### NFS as Git repository storage is no longer supported. Migrate to Gitaly Cluster as soon as possible +### NFS as Git repository storage is no longer supported -As of November 22, 2022, we are removing support for customers utilizing NFS for Git repository storage. This was originally planned for May 22, 2022, but in an effort to allow continued maturity of Gitaly Cluster, we chose to extend our removal of support date until now. Please see our official [Statement of Support](https://about.gitlab.com/support/statement-of-support/#gitaly-and-nfs) for further information. +As of November 22, 2022, we have removed support for customers using NFS for Git repository storage. This was +originally planned for May 22, 2022, but in an effort to allow continued maturity of Gitaly Cluster, we delayed +our removal of support date until now. Please see our official [Statement of Support](https://about.gitlab.com/support/statement-of-support/#gitaly-and-nfs) +for further information. This change in support follows the development deprecation for NFS for Git repository storage that occurred in GitLab 14.0. diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 2bb481d9ecf..6629c798cfa 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -339,7 +339,7 @@ custom job: The above `.gitlab-ci.yml` causes a linting error: ```plaintext -Found errors in your .gitlab-ci.yml: +Unable to create pipeline - dependency_scanning job: chosen stage does not exist; available stages are .pre - unit-tests - .post @@ -590,7 +590,7 @@ like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/l the following error may occur, depending on your GitLab CI/CD configuration: ```plaintext -Found errors in your .gitlab-ci.yml: +Unable to create pipeline jobs:sast config key may not be used with `rules`: only/except ``` |