diff options
Diffstat (limited to 'doc')
98 files changed, 700 insertions, 220 deletions
diff --git a/doc/README.md b/doc/README.md index 4248f62c08c..7548240bfef 100644 --- a/doc/README.md +++ b/doc/README.md @@ -191,7 +191,7 @@ instant how code changes impact your production environment. ### User account - [User account](user/profile/index.md): Manage your account - - [Authentication](topics/authentication/index.md): Account security with two-factor authentication, setup your ssh keys and deploy keys for secure access to your projects. + - [Authentication](topics/authentication/index.md): Account security with two-factor authentication, set up your ssh keys and deploy keys for secure access to your projects. - [Profile settings](user/profile/index.md#profile-settings): Manage your profile settings, two factor authentication and more. - [User permissions](user/permissions.md): Learn what each role in a project (external/guest/reporter/developer/maintainer/owner) can do. diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index 5c7392b99d0..54ded25291a 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -111,7 +111,7 @@ main: uid: 'sAMAccountName' # This should be the attribute, not the value that maps to uid. ## - ## Examples: 'america\\momo' or 'CN=Gitlab Git,CN=Users,DC=mydomain,DC=com' + ## Examples: 'america\momo' or 'CN=Gitlab Git,CN=Users,DC=mydomain,DC=com' ## bind_dn: '_the_full_dn_of_the_user_you_will_bind_with' password: '_the_password_of_the_bind_user' @@ -132,7 +132,7 @@ main: ## Enables SSL certificate verification if encryption method is ## "start_tls" or "simple_tls". Defaults to true since GitLab 10.0 for ## security. This may break installations upon upgrade to 10.0, that did - ## not know their LDAP SSL certificates were not setup properly. + ## not know their LDAP SSL certificates were not set up properly. ## verify_certificates: true diff --git a/doc/administration/external_database.md b/doc/administration/external_database.md index 31199f2cdc7..ec2d30c82d1 100644 --- a/doc/administration/external_database.md +++ b/doc/administration/external_database.md @@ -9,7 +9,7 @@ separate from the GitLab Omnibus package. If you use a cloud-managed service, or provide your own PostgreSQL instance: -1. Setup PostgreSQL according to the +1. Set up PostgreSQL according to the [database requirements document](../install/requirements.md#database). 1. Set up a `gitlab` username with a password of your choice. The `gitlab` user needs privileges to create the `gitlabhq_production` database. diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 964758837e5..49c6902bc42 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -101,6 +101,12 @@ documentation on configuring Gitaly authentication](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/configuration/README.md#authentication) . +Gitaly must trigger some callbacks to GitLab via GitLab Shell. As a result, +the GitLab Shell secret must be the same between the other GitLab servers and +the Gitaly server. The easiest way to accomplish this is to copy `/etc/gitlab/gitlab-secrets.json` +from an existing GitLab server to the Gitaly server. Without this shared secret, +Git operations in GitLab will result in an API error. + > **NOTE:** In most or all cases the storage paths below end in `/repositories` which is different than `path` in `git_data_dirs` of Omnibus installations. Check the directory layout on your Gitaly server to be sure. diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index b5124b1d540..c1eeb40b98f 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -13,7 +13,7 @@ Database Service (RDS) that runs PostgreSQL. If you use a cloud-managed service, or provide your own PostgreSQL: -1. Setup PostgreSQL according to the +1. Set up PostgreSQL according to the [database requirements document](../../install/requirements.md#database). 1. Set up a `gitlab` username with a password of your choice. The `gitlab` user needs privileges to create the `gitlabhq_production` database. diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index e05bebbef18..b5d1ff698c6 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -81,7 +81,7 @@ When a **Master** fails to respond, it's the application's responsibility (in our case GitLab) to handle timeout and reconnect (querying a **Sentinel** for a new **Master**). -To get a better understanding on how to correctly setup Sentinel, please read +To get a better understanding on how to correctly set up Sentinel, please read the [Redis Sentinel documentation](http://redis.io/topics/sentinel) first, as failing to configure it correctly can lead to data loss or can bring your whole cluster down, invalidating the failover effort. @@ -217,7 +217,7 @@ Pick the one that suits your needs. and configure Sentinel, jump directly to the Sentinel section in the [Redis HA installation from source](redis_source.md#step-3-configuring-the-redis-sentinel-instances) documentation. - [Omnibus GitLab **Enterprise Edition** (EE) package][ee]: Both Redis and Sentinel - are bundled in the package, so you can use the EE package to setup the whole + are bundled in the package, so you can use the EE package to set up the whole Redis HA infrastructure (master, slave and Sentinel) which is described in this document. - If you have installed GitLab using the Omnibus GitLab packages (CE or EE), @@ -228,7 +228,7 @@ Pick the one that suits your needs. ## Configuring Redis HA -This is the section where we install and setup the new Redis instances. +This is the section where we install and set up the new Redis instances. > **Notes:** > - We assume that you have installed GitLab and all HA components from scratch. If you @@ -370,7 +370,7 @@ You must have at least `3` Redis Sentinel servers, and they need to be each in an independent machine. You can configure them in the same machines where you've configured the other Redis servers. -With GitLab Enterprise Edition, you can use the Omnibus package to setup +With GitLab Enterprise Edition, you can use the Omnibus package to set up multiple machines with the Sentinel daemon. --- @@ -535,7 +535,7 @@ In this example we consider that all servers have an internal network interface with IPs in the `10.0.0.x` range, and that they can connect to each other using these IPs. -In a real world usage, you would also setup firewall rules to prevent +In a real world usage, you would also set up firewall rules to prevent unauthorized access from other machines and block traffic from the outside (Internet). diff --git a/doc/administration/high_availability/redis_source.md b/doc/administration/high_availability/redis_source.md index 8b7a515a076..5823c575251 100644 --- a/doc/administration/high_availability/redis_source.md +++ b/doc/administration/high_availability/redis_source.md @@ -24,7 +24,7 @@ the Omnibus Redis HA documentation. ## Configuring your own Redis server -This is the section where we install and setup the new Redis instances. +This is the section where we install and set up the new Redis instances. ### Prerequisites @@ -204,7 +204,7 @@ In this example we consider that all servers have an internal network interface with IPs in the `10.0.0.x` range, and that they can connect to each other using these IPs. -In a real world usage, you would also setup firewall rules to prevent +In a real world usage, you would also set up firewall rules to prevent unauthorized access from other machines, and block traffic from the outside ([Internet][it]). diff --git a/doc/administration/index.md b/doc/administration/index.md index 8b6a42b0ca5..702d8e554a8 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -60,7 +60,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Raketasks](../raketasks/README.md): Perform various tasks for maintenance, backups, automatic webhooks setup, etc. - [Backup and restore](../raketasks/backup_restore.md): Backup and restore your GitLab instance. -- [Operations](operations/index.md): Keeping GitLab up and running (clean up Redis sessions, moving repositories, Sidekiq Job throttling, Sidekiq MemoryKiller, Unicorn). +- [Operations](operations/index.md): Keeping GitLab up and running (clean up Redis sessions, moving repositories, Sidekiq MemoryKiller, Unicorn). - [Restart GitLab](restart_gitlab.md): Learn how to restart GitLab and its components. #### Updating GitLab diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 1f3bc611cdf..757865ea2c5 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -127,6 +127,7 @@ The connection settings match those provided by [Fog](https://github.com/fog), a | `host` | S3 compatible host for when not using AWS, e.g. `localhost` or `storage.example.com` | s3.amazonaws.com | | `endpoint` | Can be used when configuring an S3 compatible service such as [Minio](https://www.minio.io), by entering a URL such as `http://127.0.0.1:9000` | (optional) | | `path_style` | Set to true to use `host/bucket_name/object` style paths instead of `bucket_name.host/object`. Leave as false for AWS S3 | false | +| `use_iam_profile` | Set to true to use IAM profile instead of access keys | false **In Omnibus installations:** diff --git a/doc/administration/operations/img/sidekiq_job_throttling.png b/doc/administration/operations/img/sidekiq_job_throttling.png Binary files differdeleted file mode 100644 index abd09f3b115..00000000000 --- a/doc/administration/operations/img/sidekiq_job_throttling.png +++ /dev/null diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index e9cad99c4b0..dea98cb8197 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -9,8 +9,6 @@ GitLab 7.3 we recommend cleaning up stale sessions to compact the Redis database after you upgrade to GitLab 7.3. - [Moving repositories](moving_repositories.md): Moving all repositories managed by GitLab to another file system or another server. -- [Sidekiq job throttling](sidekiq_job_throttling.md): Throttle Sidekiq queues -that to prioritize important jobs. - [Sidekiq MemoryKiller](sidekiq_memory_killer.md): Configure Sidekiq MemoryKiller to restart Sidekiq. - [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer. diff --git a/doc/administration/operations/sidekiq_job_throttling.md b/doc/administration/operations/sidekiq_job_throttling.md deleted file mode 100644 index ddeaa22e288..00000000000 --- a/doc/administration/operations/sidekiq_job_throttling.md +++ /dev/null @@ -1,33 +0,0 @@ -# Sidekiq Job throttling - -> Note: Introduced with GitLab 8.14 - -When your GitLab installation needs to handle tens of thousands of background -jobs, it can be convenient to throttle queues that do not need to be executed -immediately, e.g. long running jobs like Pipelines, thus allowing jobs that do -need to be executed immediately to have access to more resources. - -In order to accomplish this, you can limit the amount of workers that certain -slow running queues can have available. This is what we call Sidekiq Job -Throttling. Depending on your infrastructure, you might have different slow -running queues, which is why you can choose which queues you want to throttle -and by how much you want to throttle them. - -These settings are available in the Application Settings of your GitLab -installation. - - - -The throttle factor determines the maximum number of workers a queue can run on. -This value gets multiplied by `:concurrency` value set in the Sidekiq settings -and rounded up to the closest full integer. - -So, for example, you set the `:concurrency` to 25 and the `Throttling factor` to -0.1, the maximum workers assigned to the selected queues would be 3. - -```ruby -queue_limit = (factor * Sidekiq.options[:concurrency]).ceil -``` - -After enabling the job throttling, you will need to restart your GitLab -instance, in order for the changes to take effect.
\ No newline at end of file diff --git a/doc/administration/operations/ssh_certificates.md b/doc/administration/operations/ssh_certificates.md index 9edccd25ced..b00301fec1c 100644 --- a/doc/administration/operations/ssh_certificates.md +++ b/doc/administration/operations/ssh_certificates.md @@ -31,7 +31,7 @@ uploading user SSH keys to GitLab entirely. ## Setting up SSH certificate lookup via GitLab Shell -How to fully setup SSH certificates is outside the scope of this +How to fully set up SSH certificates is outside the scope of this document. See [OpenSSH's PROTOCOL.certkeys](https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/PROTOCOL.certkeys?annotate=HEAD) for how it works, and e.g. [RedHat's documentation about @@ -132,7 +132,7 @@ message about this being an invalid user. ## Interaction with the `authorized_keys` file SSH certificates can be used in conjunction with the `authorized_keys` -file, and if setup as configured above the `authorized_keys` file will +file, and if set up as configured above the `authorized_keys` file will still serve as a fallback. This is because if the `AuthorizedPrincipalsCommand` can't diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index 2b4252a7b1d..d3ce7b6f2df 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -56,7 +56,7 @@ Runs the following rake tasks: - `gitlab:sidekiq:check` - `gitlab:app:check` -It will check that each component was setup according to the installation guide and suggest fixes for issues found. +It will check that each component was set up according to the installation guide and suggest fixes for issues found. You may also have a look at our Trouble Shooting Guides: - [Trouble Shooting Guide (GitLab)](http://docs.gitlab.com/ee/README.html#troubleshooting) diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md index 0cd33ffc122..b5c40478ea5 100644 --- a/doc/administration/raketasks/uploads/migrate.md +++ b/doc/administration/raketasks/uploads/migrate.md @@ -7,10 +7,32 @@ After [configuring the object storage](../../uploads.md#using-object-storage) fo >**Note:** All of the processing will be done in a background worker and requires **no downtime**. -This tasks uses 3 parameters to find uploads to migrate. +### All-in-one rake task + +GitLab provides a wrapper rake task that migrates all uploaded files - avatars, +logos, attachments, favicon, etc. - to object storage in one go. Under the hood, +it invokes individual rake tasks to migrate files falling under each of this +category one by one. The specifications of these individual rake tasks are +described in the next section. + +**Omnibus Installation** + +```bash +gitlab-rake "gitlab:uploads:migrate:all" +``` + +**Source Installation** + +```bash +sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate:all +``` + +### Individual rake tasks >**Note:** -These parameters are mainly internal to GitLab's structure, you may want to refer to the task list instead below. +If you already ran the rake task mentioned above, no need to run these individual rake tasks as that has been done automatically. + +The rake task uses 3 parameters to find uploads to migrate. Parameter | Type | Description --------- | ---- | ----------- @@ -18,6 +40,9 @@ Parameter | Type | Description `model_class` | string | Type of the model to migrate from `mount_point` | string/symbol | Name of the model's column on which the uploader is mounted on. +>**Note:** +These parameters are mainly internal to GitLab's structure, you may want to refer to the task list instead below. + This task also accepts some environment variables which you can use to override certain values: @@ -25,7 +50,7 @@ Variable | Type | Description -------- | ---- | ----------- `BATCH` | integer | Specifies the size of the batch. Defaults to 200. -** Omnibus Installation** +**Omnibus Installation** ```bash # gitlab-rake gitlab:uploads:migrate[uploader_class, model_class, mount_point] @@ -40,6 +65,9 @@ gitlab-rake "gitlab:uploads:migrate[AttachmentUploader, Note, :attachment]" gitlab-rake "gitlab:uploads:migrate[AttachmentUploader, Appearance, :logo]" gitlab-rake "gitlab:uploads:migrate[AttachmentUploader, Appearance, :header_logo]" +# Favicon +gitlab-rake "gitlab:uploads:migrate[FaviconUploader, Appearance, :favicon]" + # Markdown gitlab-rake "gitlab:uploads:migrate[FileUploader, Project]" gitlab-rake "gitlab:uploads:migrate[PersonalFileUploader, Snippet]" @@ -65,6 +93,9 @@ sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AttachmentUploader, Note sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AttachmentUploader, Appearance, :logo]" sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AttachmentUploader, Appearance, :header_logo]" +# Favicon +sudo -u git -H bundle exec rake "gitlab:uploads:migrate[FaviconUploader, Appearance, :favicon]" + # Markdown sudo -u git -H bundle exec rake "gitlab:uploads:migrate[FileUploader, Project]" sudo -u git -H bundle exec rake "gitlab:uploads:migrate[PersonalFileUploader, Snippet]" diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md index 426245c7aca..6d7069dd461 100644 --- a/doc/administration/reply_by_email.md +++ b/doc/administration/reply_by_email.md @@ -5,7 +5,7 @@ replying to notification emails. ## Requirement -Make sure [incoming email](incoming_email.md) is setup. +Make sure [incoming email](incoming_email.md) is set up. ## How it works? diff --git a/doc/administration/reply_by_email_postfix_setup.md b/doc/administration/reply_by_email_postfix_setup.md index 3e8b78e56d5..d1a03219542 100644 --- a/doc/administration/reply_by_email_postfix_setup.md +++ b/doc/administration/reply_by_email_postfix_setup.md @@ -245,7 +245,7 @@ Courier, which we will install later to add IMAP authentication, requires mailbo 220 gitlab.example.com ESMTP Postfix (Ubuntu) ``` - If you get a `Connection refused` error instead, make sure your firewall is setup to allow inbound traffic on port 25. + If you get a `Connection refused` error instead, make sure your firewall is set up to allow inbound traffic on port 25. 1. Send the `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt: diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md index efeec9db517..715bc0cd08c 100644 --- a/doc/administration/repository_checks.md +++ b/doc/administration/repository_checks.md @@ -18,7 +18,8 @@ repositories and wiki repositories in order to detect data corruption. A project will be checked no more than once per month. If any projects fail their repository checks all GitLab administrators will receive an email notification of the situation. This notification is sent out once a week, -by default, midnight at the start of Sunday. +by default, midnight at the start of Sunday. Repositories with known check +failures can be found at `/admin/projects?last_repository_check_failed=1`. ## Disabling periodic checks diff --git a/doc/administration/uploads.md b/doc/administration/uploads.md index 467deb43644..aec9a359ada 100644 --- a/doc/administration/uploads.md +++ b/doc/administration/uploads.md @@ -18,7 +18,7 @@ below. >**Notes:** For historical reasons, uploads are stored into a base directory, which by default is `uploads/-/system`. It is strongly discouraged to change this configuration option on an existing GitLab installation. -_The uploads are stored by default in `/var/opt/gitlab/gitlab-rails/public/uploads/-/system`._ +_The uploads are stored by default in `/var/opt/gitlab/gitlab-rails/uploads/-/system`._ 1. To change the storage path for example to `/mnt/storage/uploads`, edit `/etc/gitlab/gitlab.rb` and add the following line: @@ -86,6 +86,7 @@ The connection settings match those provided by [Fog](https://github.com/fog), a | `host` | S3 compatible host for when not using AWS, e.g. `localhost` or `storage.example.com` | s3.amazonaws.com | | `endpoint` | Can be used when configuring an S3 compatible service such as [Minio](https://www.minio.io), by entering a URL such as `http://127.0.0.1:9000` | (optional) | | `path_style` | Set to true to use `host/bucket_name/object` style paths instead of `bucket_name.host/object`. Leave as false for AWS S3 | false | +| `use_iam_profile` | Set to true to use IAM profile instead of access keys | false **In Omnibus installations:** diff --git a/doc/api/protected_branches.md b/doc/api/protected_branches.md index f6813f27dc0..ed8837574a0 100644 --- a/doc/api/protected_branches.md +++ b/doc/api/protected_branches.md @@ -4,7 +4,7 @@ **Valid access levels** -The access levels are defined in the `ProtectedRefAccess::ALLOWED_ACCESS_LEVELS` constant. Currently, these levels are recognized: +The access levels are defined in the `ProtectedRefAccess.allowed_access_levels` method. Currently, these levels are recognized: ``` 0 => No access 30 => Developer access diff --git a/doc/api/settings.md b/doc/api/settings.md index 83fa9b055d1..1c41b3345ad 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -193,7 +193,7 @@ are listed in the descriptions of the relevant settings. | `metrics_port` | integer | required by: `metrics_enabled` | The UDP port to use for connecting to InfluxDB. | | `metrics_sample_interval` | integer | required by: `metrics_enabled` | The sampling interval in seconds. | | `metrics_timeout` | integer | required by: `metrics_enabled` | The amount of seconds after which InfluxDB will time out. | -| `mirror_available` | boolean | no | Allow mirrors to be setup for projects. If disabled, only admins will be able to setup mirrors in projects. | +| `mirror_available` | boolean | no | Allow mirrors to be set up for projects. If disabled, only admins will be able to set up mirrors in projects. | | `pages_domain_verification_enabled` | boolean | no | Require users to prove ownership of custom domains. Domain verification is an essential security measure for public GitLab sites. Users are required to demonstrate they control a domain before it is enabled. | | `password_authentication_enabled_for_git` | boolean | no | Enable authentication for Git over HTTP(S) via a GitLab account password. Default is `true`. | | `password_authentication_enabled_for_web` | boolean | no | Enable authentication for the web interface via a GitLab account password. Default is `true`. | @@ -219,9 +219,6 @@ are listed in the descriptions of the relevant settings. | `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes | | `shared_runners_enabled` | boolean | no | (**If enabled, requires:** `shared_runners_text`) Enable shared runners for new projects. | | `shared_runners_text` | string | required by: `shared_runners_enabled` | Shared runners text. | -| `sidekiq_throttling_enabled` | boolean | no | (**If enabled, requires:** `sidekiq_throttling_factor` and `sidekiq_throttling_queues`) Enable Sidekiq Job Throttling. | -| `sidekiq_throttling_factor` | decimal | required by: `sidekiq_throttling_enabled` | The factor by which the queues should be throttled. A value between `0.0` and `1.0`, exclusive. | -| `sidekiq_throttling_queues` | array of strings | required by: `sidekiq_throttling_enabled` | Choose which queues you wish to throttle. | | `sign_in_text` | string | no | Text on the login page. | | `signin_enabled` | string | no | (Deprecated: Use `password_authentication_enabled_for_web` instead) Flag indicating if password authentication is enabled for the web interface. | | `signup_enabled` | boolean | no | Enable registration. Default is `true`. | diff --git a/doc/ci/README.md b/doc/ci/README.md index d782d64e971..dba1f38abe2 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -132,5 +132,3 @@ your whole GitLab instance as well as in each project. - [New CI job permissions model](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your jobs. There's a new way to access your Git submodules and LFS objects in jobs. - -[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml diff --git a/doc/ci/autodeploy/quick_start_guide.md b/doc/ci/autodeploy/quick_start_guide.md index cc6c9ec0e0a..3836216e951 100644 --- a/doc/ci/autodeploy/quick_start_guide.md +++ b/doc/ci/autodeploy/quick_start_guide.md @@ -11,7 +11,7 @@ We made a minimal [Ruby application](https://gitlab.com/gitlab-examples/minimal- Let’s start by forking our sample application. Go to [the project page](https://gitlab.com/gitlab-examples/minimal-ruby-app) and press the `Fork` button. Soon you should have a project under your namespace with the necessary files. -## Setup your own cluster on Google Kubernetes Engine +## Set up your own cluster on Google Kubernetes Engine If you do not already have a Google Cloud account, create one at https://console.cloud.google.com. @@ -71,7 +71,7 @@ Use this IP address to configure your DNS. This part heavily depends on your pre Use `nslookup minimal-ruby-app-staging.<yourdomain>` to confirm that domain is assigned to the cluster IP. -## Setup Auto Deploy +## Set up Auto Deploy Visit the home page of your GitLab.com project and press "Set up Auto Deploy" button. diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index b41101695f6..f479dc74d1f 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -178,8 +178,8 @@ runs of jobs for things like dependencies and commonly used libraries so they don't have to be re-fetched from the public internet. NOTE: **Note:** -For more examples, check the [GitLab CI Yml](https://gitlab.com/gitlab-org/gitlab-ci-yml) -project. +For more examples, check out our [GitLab CI/CD +templates](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates). ### Caching Nodejs dependencies @@ -190,7 +190,7 @@ Nodejs modules are installed in `node_modules/` and are cached per-branch: ```yaml # -# https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Nodejs.gitlab-ci.yml +# https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates/Nodejs.gitlab-ci.yml # image: node:latest @@ -217,7 +217,7 @@ are cached per-branch: ```yaml # -# https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/PHP.gitlab-ci.yml +# https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates/PHP.gitlab-ci.yml # image: php:7.2 @@ -246,7 +246,7 @@ pip's cache is defined under `.cache/pip/` and both are cached per-branch: ```yaml # -# https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Python.gitlab-ci.yml +# https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates/Python.gitlab-ci.yml # image: python:latest @@ -286,7 +286,7 @@ jobs inherit it. Gems are installed in `vendor/ruby/` and are cached per-branch: ```yaml # -# https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Ruby.gitlab-ci.yml +# https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates/Ruby.gitlab-ci.yml # image: ruby:2.5 diff --git a/doc/ci/environments.md b/doc/ci/environments.md index f1e5b00e927..4d740c32fd6 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -370,7 +370,7 @@ review_app: url: https://$CI_COMMIT_REF_SLUG.example.com ``` -It is assumed that the user has already setup NGINX and GitLab Runner in the +It is assumed that the user has already set up NGINX and GitLab Runner in the server this job will run on. >**Note:** diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 8eb96ae10b2..fdf09d332a5 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -4,8 +4,8 @@ comments: false # GitLab CI/CD Examples -A collection of `.gitlab-ci.yml` template files is maintained at the [GitLab CI/CD YAML project][gitlab-ci-templates]. When you create a new file via the UI, -GitLab will give you the option to choose one of the templates existent on this project. +A collection of [`.gitlab-ci.yml` template files][gitlab-ci-templates] is maintained in GitLab. When you create a new file via the UI, +GitLab will give you the option to choose one of these templates. If your favorite programming language or framework are missing we would love your help by sending a merge request with a new `.gitlab-ci.yml` to this project. @@ -87,4 +87,4 @@ language users and GitLab by sending a merge request with a guide for that langu You may want to apply for the [GitLab Community Writers Program](https://about.gitlab.com/community-writers/) to get paid for writing complete articles for GitLab. -[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml +[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/gitlab/ci/templates diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md index 0dab07a7f80..d36e97ebfd3 100644 --- a/doc/ci/examples/browser_performance.md +++ b/doc/ci/examples/browser_performance.md @@ -110,4 +110,4 @@ performance: - sitespeed-results/ ``` -A complete example can be found in our [Auto DevOps CI YML](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml). +A complete example can be found in our [Auto DevOps CI YML](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml). diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index c226b5bfb71..b75ed87bc91 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -520,7 +520,7 @@ a lot of breathing room in quickly getting changes to players. Here are some ideas to further investigate that can speed up or improve your pipeline: - [Yarn](https://yarnpkg.com) instead of npm -- Setup a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ci-yml) image that can preload dependencies and tools (like AWS CLI) +- Set up a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ci-yml) image that can preload dependencies and tools (like AWS CLI) - Forward a [custom domain](http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) to your game's S3 static website - Combine jobs if you find it unnecessary for a small project - Avoid the queues and set up your own [custom GitLab CI/CD runner](https://about.gitlab.com/2016/03/01/gitlab-runner-with-docker/) diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 39c65399332..ab429e0ded3 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -13,7 +13,7 @@ date: 2017-08-31 GitLab features our applications with Continuous Integration, and it is possible to easily deploy the new code changes to the production server whenever we want. -In this tutorial, we'll show you how to initialize a [Laravel](http://laravel.com/) application and setup our [Envoy](https://laravel.com/docs/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/). +In this tutorial, we'll show you how to initialize a [Laravel](http://laravel.com/) application and set up our [Envoy](https://laravel.com/docs/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/). We assume you have a basic experience with Laravel, Linux servers, and you know how to use GitLab. @@ -23,7 +23,7 @@ It has a great community with a [fantastic documentation](https://laravel.com/do Aside from the usual routing, controllers, requests, responses, views, and (blade) templates, out of the box Laravel provides plenty of additional services such as cache, events, localization, authentication and many others. We will use [Envoy](https://laravel.com/docs/master/envoy) as an SSH task runner based on PHP. -It uses a clean, minimal [Blade syntax](https://laravel.com/docs/blade) to setup tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/artisan). +It uses a clean, minimal [Blade syntax](https://laravel.com/docs/blade) to set up tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/artisan). ## Initialize our Laravel app on GitLab @@ -372,7 +372,7 @@ At the end, our `Envoy.blade.php` file will look like this: One more thing we should do before any deployment is to manually copy our application `storage` folder to the `/var/www/app` directory on the server for the first time. You might want to create another Envoy task to do that for you. -We also create the `.env` file in the same path to setup our production environment variables for Laravel. +We also create the `.env` file in the same path to set up our production environment variables for Laravel. These are persistent data and will be shared to every new release. Now, we would need to deploy our app by running `envoy run deploy`, but it won't be necessary since GitLab can handle that for us with CI's [environments](../../environments.md), which will be described [later](#setting-up-gitlab-ci-cd) in this tutorial. @@ -587,7 +587,7 @@ unit_test: script: # Install app dependencies - composer install - # Setup .env + # Set up .env - cp .env.example .env # Generate an environment key - php artisan key:generate diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index a2ba29a4ee2..df4805ea7ac 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -199,7 +199,7 @@ pecl install <extension> ``` It's not advised to add this to `.gitlab-ci.yml`. You should execute this -command once, only to setup the build environment. +command once, only to set up the build environment. ## Extend your tests diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index 8ce4fe55cec..7990917f809 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -3,7 +3,7 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/50144) in GitLab 11.3. Interactive web terminals give the user access to a terminal in GitLab for -running one-of commands for their CI pipeline. +running one-off commands for their CI pipeline. NOTE: **Note:** This is not available for the shared Runners on GitLab.com. diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index cf22450914c..3fd54647abb 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -140,6 +140,27 @@ java: - target/failsafe-reports/TEST-*.xml ``` +### C/C++ example + +There are a few tools that can produce JUnit reports in C/C++. + +#### GoogleTest + +In the following example, `gtest` is used to generate the test reports. +If there are multiple gtest executables created for different architectures (`x86`, `x64` or `arm`), +you will be required to run each test providing a unique filename. The results +will then be aggregated together. + +```yaml +cpp: + stage: test + script: + - gtest.exe --gtest_output="xml:report.xml" + artifacts: + reports: + junit: report.xml +``` + ## Limitations Currently, the following tools might not work because their XML formats are unsupported in GitLab. diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 7859d2ec631..83e0fa34ad6 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -29,7 +29,7 @@ are: - **Specific Runners** are useful for jobs that have special requirements or for projects with a specific demand. If a job has certain requirements, you can set up the specific Runner with this in mind, while not having to do this for all - Runners. For example, if you want to deploy a certain project, you can setup + Runners. For example, if you want to deploy a certain project, you can set up a specific Runner to have the right credentials for this. The [usage of tags](#using-tags) may be useful in this case. Specific Runners process jobs using a [FIFO] queue. - **Group Runners** are useful when you have multiple projects under one group @@ -222,7 +222,7 @@ should keep in mind. ### Using tags -You must setup a Runner to be able to run all the different types of jobs +You must set up a Runner to be able to run all the different types of jobs that it may encounter on the projects it's shared over. This would be problematic for large amounts of projects, if it wasn't for tags. @@ -298,7 +298,7 @@ and using more secure [Runner Executors](https://docs.gitlab.com/runner/executor ### Forks Whenever a project is forked, it copies the settings of the jobs that relate -to it. This means that if you have shared Runners setup for a project and +to it. This means that if you have shared Runners set up for a project and someone forks that project, the shared Runners will also serve jobs of this project. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 41de9a50efc..873a5c4301e 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -492,6 +492,7 @@ osx job: `allow_failure` is used when you want to allow a job to fail without impacting the rest of the CI suite. Failed jobs don't contribute to the commit status. +The default value is `false`. When enabled and the job fails, the pipeline will be successful/green for all intents and purposes, but a "CI build passed with warnings" message will be @@ -1231,13 +1232,16 @@ rspec: ``` The collected JUnit reports will be uploaded to GitLab as an artifact and will -be automatically [shown in merge requests](../junit_test_reports.md). +be automatically shown in merge requests. + +For more examples, see [JUnit test reports](../junit_test_reports.md). NOTE: **Note:** In case the JUnit tool you use exports to multiple XML files, you can specify -multiple test report paths within a single job -(`junit: [rspec-1.xml, rspec-2.xml, rspec-3.xml]`) and they will be automatically -concatenated into a single file. +multiple test report paths within a single job and they will be automatically +concatenated into a single file. Use a filename pattern (`junit: rspec-*.xml`), +an array of filenames (`junit: [rspec-1.xml, rspec-2.xml, rspec-3.xml]`), or a +combination thereof (`junit: [rspec.xml, test-results/TEST-*.xml]`). ## `dependencies` diff --git a/doc/development/README.md b/doc/development/README.md index efe37b8ba0b..43d3865da0e 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -7,7 +7,7 @@ description: 'Learn how to contribute to GitLab.' ## Get started! -- Setup GitLab's development environment with [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md) +- Set up GitLab's development environment with [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md) - [GitLab contributing guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md) - [Architecture](architecture.md) of GitLab - [Rake tasks](rake_tasks.md) for development @@ -53,11 +53,14 @@ description: 'Learn how to contribute to GitLab.' ## Performance guides -- [Instrumentation](instrumentation.md) -- [Performance guidelines](performance.md) +- [Instrumentation](instrumentation.md) for Ruby code running in production + environments +- [Performance guidelines](performance.md) for writing code, benchmarks, and + certain patterns to avoid - [Merge request performance guidelines](merge_request_performance_guidelines.md) for ensuring merge requests do not negatively impact GitLab performance -- [Profiling](profiling.md) for profiling a URL +- [Profiling](profiling.md) a URL, measuring performance using Sherlock, or + tracking down N+1 queries using Bullet ## Database guides @@ -91,6 +94,7 @@ description: 'Learn how to contribute to GitLab.' - [Verifying database capabilities](verifying_database_capabilities.md) - [Database Debugging and Troubleshooting](database_debugging.md) - [Query Count Limits](query_count_limits.md) +- [Database helper modules](database_helpers.md) ## Testing guides diff --git a/doc/development/automatic_ce_ee_merge.md b/doc/development/automatic_ce_ee_merge.md index f6509b5c1dd..9dd78806a12 100644 --- a/doc/development/automatic_ce_ee_merge.md +++ b/doc/development/automatic_ce_ee_merge.md @@ -63,7 +63,7 @@ EE version of your CE merge request. For each commit (except on `master`), the `ee_compat_check` CI job tries to detect if the current branch's changes will conflict during the CE->EE merge. -The job reports what files are conflicting and how to setup a merge request +The job reports what files are conflicting and how to set up a merge request against EE. #### How the job works diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 08042fa2aec..edd2d063458 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -224,7 +224,7 @@ on those issues. Please select someone with relevant experience from the the commit history for the affected files to find someone. We also use [GitLab Triage] to automate some triaging policies. This is -currently setup as a [scheduled pipeline] running on [quality/triage-ops] +currently set up as a [scheduled pipeline] running on [quality/triage-ops] project. [described in our handbook]: https://about.gitlab.com/handbook/engineering/issue-triage/ diff --git a/doc/development/database_debugging.md b/doc/development/database_debugging.md index 9c31265e417..b2c804b2ff0 100644 --- a/doc/development/database_debugging.md +++ b/doc/development/database_debugging.md @@ -33,7 +33,7 @@ If your test DB is giving you problems, it is safe to nuke it because it doesn't - `bundle exec rake db:migrate RAILS_ENV=development`: Execute any pending migrations that you may have picked up from a MR - `bundle exec rake db:migrate:status RAILS_ENV=development`: Check if all migrations are `up` or `down` - `bundle exec rake db:migrate:down VERSION=20170926203418 RAILS_ENV=development`: Tear down a migration - - `bundle exec rake db:migrate:up VERSION=20170926203418 RAILS_ENV=development`: Setup a migration + - `bundle exec rake db:migrate:up VERSION=20170926203418 RAILS_ENV=development`: Set up a migration - `bundle exec rake db:migrate:redo VERSION=20170926203418 RAILS_ENV=development`: Re-run a specific migration diff --git a/doc/development/database_helpers.md b/doc/development/database_helpers.md new file mode 100644 index 00000000000..21e4e725de6 --- /dev/null +++ b/doc/development/database_helpers.md @@ -0,0 +1,63 @@ +# Database helpers + +There are a number of useful helper modules defined in `/lib/gitlab/database/`. + +## Subquery + +In some cases it is not possible to perform an operation on a query. +For example: + +```ruby +Geo::EventLog.where('id < 100').limit(10).delete_all +``` + +Will give this error: + +> ActiveRecord::ActiveRecordError: delete_all doesn't support limit + +One solution would be to wrap it in another `where`: + +```ruby +Geo::EventLog.where(id: Geo::EventLog.where('id < 100').limit(10)).delete_all +``` + +This works with PostgreSQL, but with MySQL it gives this error: + +> ActiveRecord::StatementInvalid: Mysql2::Error: This version of MySQL +> doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery' + +Also, that query doesn't have very good performance. Using a +`INNER JOIN` with itself is better. + +So instead of this query: + +```sql +SELECT geo_event_log.* +FROM geo_event_log +WHERE geo_event_log.id IN + (SELECT geo_event_log.id + FROM geo_event_log + WHERE (id < 100) + LIMIT 10) +``` + +It's better to write: + +```sql +SELECT geo_event_log.* +FROM geo_event_log +INNER JOIN + (SELECT geo_event_log.* + FROM geo_event_log + WHERE (id < 100) + LIMIT 10) t2 ON geo_event_log.id = t2.id +``` + +And this is where `Gitlab::Database::Subquery.self_join` can help +you. So you can rewrite the above statement as: + +```ruby +Gitlab::Database::Subquery.self_join(Geo::EventLog.where('id < 100').limit(10)).delete_all +``` + +And this also works with MySQL, so you don't need to worry about that. diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 7ac211ed550..d6ae4cb39f0 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -411,6 +411,22 @@ The following GitLab features are used among others: Every GitLab instance includes the documentation, which is available from `/help` (`http://my-instance.com/help`), e.g., <https://gitlab.com/help>. +The documentation available online on docs.gitlab.com is continuously +deployed every hour from the `master` branch of CE, EE, Omnibus, and Runner. Therefore, +once a merge request gets merged, it will be available online on the same day, +but they will be shipped (and available on `/help`) within the milestone assigned +to the MR. + +For instance, let's say your merge request has a milestone set to 11.3, which +will be released on 2018-09-22. If it gets merged on 2018-09-15, it will be +available online on 2018-09-15, but, as the feature freeze date has passed, if +the MR does not have a "pick into 11.3" label, the milestone has to be changed +to 11.4 and it will be shipped with all GitLab packages only on 2018-10-22, +with GitLab 11.4. Meaning, it will only be available under `/help` from GitLab +11.4 onwards, but available on docs.gitlab.com on the same day it was merged. + +### Linking to `/help` + When you're building a new feature, you may need to link the documentation from GitLab, the application. This is normally done in files inside the `app/views/` directory with the help of the `help_page_path` helper method. diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md index 8083f219d4a..d4f7bb7ae74 100644 --- a/doc/development/documentation/styleguide.md +++ b/doc/development/documentation/styleguide.md @@ -251,7 +251,7 @@ below. (in that order) that introduced it. The above quote would be then transformed to: ```md - > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/1242) in GitLab 8.3. + > [Introduced](<link-to-issue>) in GitLab 8.3. ``` - If the feature is only available in GitLab Enterprise Edition, don't forget to mention @@ -259,10 +259,22 @@ below. the feature is available in: ```md - > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/1242) - in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. + > [Introduced](<link-to-issue>) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3. ``` +#### Early versions of EE + +If the feature was created before GitLab 9.2 (before [different EE tiers were introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1851)): + +- Declare it as "Introduced in GitLab Enterprise Edition X.Y". +- Note which tier the feature is available in. + +For example: + +```md +> [Introduced](<link-to-issue>) in GitLab Enterprise Edition 9.0. Available in [GitLab Premium](https://about.gitlab.com/pricing/). +``` + ### Product badges When a feature is available in EE-only tiers, add the corresponding tier according to the diff --git a/doc/development/fe_guide/components.md b/doc/development/fe_guide/components.md index 66a8abe42f7..ee0c2d534ff 100644 --- a/doc/development/fe_guide/components.md +++ b/doc/development/fe_guide/components.md @@ -42,7 +42,7 @@ See also the [corresponding UX guide](../ux_guide/components.md#dropdowns). See also the [corresponding UX guide](../ux_guide/components.md#modals). -We have a reusable Vue component for modals: [vue_shared/components/gl-modal.vue](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/vue_shared/components/gl-modal.vue) +We have a reusable Vue component for modals: [vue_shared/components/gl_modal.vue](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/vue_shared/components/gl_modal.vue) Here is an example of how to use it: diff --git a/doc/development/feature_flags.md b/doc/development/feature_flags.md index 6f757f1ce7b..417298205f5 100644 --- a/doc/development/feature_flags.md +++ b/doc/development/feature_flags.md @@ -65,13 +65,18 @@ In the rare case that you need the feature flag to be on automatically, use Feature.enabled?(:feature_flag, project, default_enabled: true) ``` +For more information about rolling out changes using feature flags, refer to the +[Rolling out changes using feature flags](rolling_out_changes_using_feature_flags.md) +guide. + ### Specs In the test environment `Feature.enabled?` is stubbed to always respond to `true`, so we make sure behavior under feature flag doesn't go untested in some non-specific contexts. -If you need to test the feature flag in a different state, you need to stub it with: +Whenever a feature flag is present, make sure to test _both_ states of the +feature flag. You can stub a feature flag as follows: ```ruby stub_feature_flags(my_feature_flag: false) diff --git a/doc/development/file_storage.md b/doc/development/file_storage.md index fdbd7f1fa37..6e014e8c751 100644 --- a/doc/development/file_storage.md +++ b/doc/development/file_storage.md @@ -45,6 +45,11 @@ In the case of Issues/MR/Notes Markdown attachments, there is a different approa instead of basing the path into a mutable variable `:project_path_with_namespace`, it's possible to use the hash of the project ID instead, if project migrates to the new approach (introduced in 10.2). +> Note: We provide an [all-in-one rake task] to migrate all uploads to object +> storage in one go. If a new Uploader class or model type is introduced, make +> sure you add a rake task invocation corresponding to it to the [category +> list]. + ### Path segments Files are stored at multiple locations and use different path schemes. @@ -137,3 +142,5 @@ end [CarrierWave]: https://github.com/carrierwaveuploader/carrierwave [Hashed Storage]: ../administration/repository_storage_types.md +[all-in-one rake task]: ../administration/raketasks/uploads/migrate.md +[category list]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/tasks/gitlab/uploads/migrate.rake diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index 7054ff39da0..5e13c725e15 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -30,6 +30,7 @@ are very appreciative of the work done by translators and proofreaders! - Korean - Chang-Ho Cha - [GitLab](https://gitlab.com/changho-cha), [Crowdin](https://crowdin.com/profile/zzazang) - Huang Tao - [GitLab](https://gitlab.com/htve), [Crowdin](https://crowdin.com/profile/htve) + - Ji Hun Oh - [GitLab](https://gitlab.com/Baw-Appie), [Crowdin](https://crowdin.com/profile/BawAppie) - Polish - Filip Mech - [GitLab](https://gitlab.com/mehenz), [Crowdin](https://crowdin.com/profile/mehenz) - Portuguese, Brazilian diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md index 12badbe39b2..ee01c89e0ed 100644 --- a/doc/development/merge_request_performance_guidelines.md +++ b/doc/development/merge_request_performance_guidelines.md @@ -168,6 +168,7 @@ user objects for every username we can remove the need for running the same query for every mention of `@alice`. Caching data per transaction can be done using -[RequestStore](https://github.com/steveklabnik/request_store). Caching data in -Redis can be done using [Rails' caching +[RequestStore](https://github.com/steveklabnik/request_store) (use +`Gitlab::SafeRequestStore` to avoid having to remember to check +`RequestStore.active?`). Caching data in Redis can be done using [Rails' caching system](http://guides.rubyonrails.org/caching_with_rails.html). diff --git a/doc/development/module_with_instance_variables.md b/doc/development/module_with_instance_variables.md index 48a1b7f847e..7bdfa04fc57 100644 --- a/doc/development/module_with_instance_variables.md +++ b/doc/development/module_with_instance_variables.md @@ -60,7 +60,7 @@ as long as it's contained in the same module; that is, no other modules or objects are touching them, then it would be an acceptable use. We especially allow the case where a single instance variable is used with -`||=` to setup the value. This would look like: +`||=` to set up the value. This would look like: ``` ruby module M diff --git a/doc/development/ordering_table_columns.md b/doc/development/ordering_table_columns.md index 5d00e1f7a0c..e9c6481635b 100644 --- a/doc/development/ordering_table_columns.md +++ b/doc/development/ordering_table_columns.md @@ -1,32 +1,49 @@ -# Ordering Table Columns +# Ordering Table Columns in PostgreSQL Similar to C structures the space of a table is influenced by the order of columns. This is because the size of columns is aligned depending on the type of -the column. Take the following column order for example: +the following column. Let's consider an example: -* id (integer, 4 bytes) -* name (text, variable) -* user_id (integer, 4 bytes) +- `id` (integer, 4 bytes) +- `name` (text, variable) +- `user_id` (integer, 4 bytes) -Integers are aligned to the word size. This means that on a 64 bit platform the -actual size of each column would be: 8 bytes, variable, 8 bytes. This means that -each row will require at least 16 bytes for the two integers, and a variable -amount for the text field. If a table has a few rows this is not an issue, but -once you start storing millions of rows you can save space by using a different -order. For the above example a more ideal column order would be the following: +The first column is a 4-byte integer. The next is text of variable length. The +`text` data type requires 1-word alignment, and on 64-bit platform, 1 word is 8 +bytes. To meet the alignment requirements, four zeros are to be added right +after the first column, so `id` occupies 4 bytes, then 4 bytes of alignment +padding, and only next `name` is being stored. Therefore, in this case, 8 bytes +will be spent for storing a 4-byte integer. -* id (integer, 4 bytes) -* user_id (integer, 4 bytes) -* name (text, variable) +The space between rows is also subject to alignment padding. The `user_id` +column takes only 4 bytes, and on 64-bit platform, 4 zeroes will be added for +alignment padding, to allow storing the next row beginning with the "clear" word. -In this setup the `id` and `user_id` columns can be packed together, which means -we only need 8 bytes to store _both_ of them. This in turn each row will require -8 bytes less of space. +As a result, the actual size of each column would be (ommiting variable length +data and 24-byte tuple header): 8 bytes, variable, 8 bytes. This means that +each row will require at least 16 bytes for the two 4-byte integers. If a table +has a few rows this is not an issue. However, once you start storing millions of +rows you can save space by using a different order. For the above example, the +ideal column order would be the following: + +- `id` (integer, 4 bytes) +- `user_id` (integer, 4 bytes) +- `name` (text, variable) + +or + +- `name` (text, variable) +- `id` (integer, 4 bytes) +- `user_id` (integer, 4 bytes) + +In these examples, the `id` and `user_id` columns are packed together, which +means we only need 8 bytes to store _both_ of them. This in turn means each row +will require 8 bytes less space. For GitLab we require that columns of new tables are ordered based to use the least amount of space. An easy way of doing this is to order them based on the -type size in descending order with variable sizes (string and text columns for -example) at the end. +type size in descending order with variable sizes (`text`, `varchar`, arrays, +`json`, `jsonb`, and so on) at the end. ## Type Sizes @@ -36,7 +53,7 @@ of information we will list the sizes of common types here so it's easier to look them up. Here "word" refers to the word size, which is 4 bytes for a 32 bits platform and 8 bytes for a 64 bits platform. -| Type | Size | Aligned To | +| Type | Size | Alignment needed | |:-----------------|:-------------------------------------|:-----------| | smallint | 2 bytes | 1 word | | integer | 4 bytes | 1 word | @@ -58,7 +75,7 @@ always be at the end of a table. ## Real Example -Let's use the "events" table as an example, which currently has the following +Let's use the `events` table as an example, which currently has the following layout: | Column | Type | Size | @@ -89,8 +106,8 @@ divided into fixed size chunks as follows: | 8 bytes | updated_at | | 8 bytes | action, author_id | -This means that excluding the variable sized data we need at least 48 bytes per -row. +This means that excluding the variable sized data and tuple header, we need at +least 8 * 6 = 48 bytes per row. We can optimise this by using the following column order instead: @@ -120,8 +137,8 @@ This would produce the following chunks: | variable | title | | variable | data | -Here we only need 40 bytes per row excluding the variable sized data. 8 bytes -being saved may not sound like much, but for tables as large as the "events" -table it does begin to matter. For example, when storing 80 000 000 rows this -translates to a space saving of at least 610 MB: all by just changing the order -of a few columns. +Here we only need 40 bytes per row excluding the variable sized data and 24-byte +tuple header. 8 bytes being saved may not sound like much, but for tables as +large as the `events` table it does begin to matter. For example, when storing +80 000 000 rows this translates to a space saving of at least 610 MB, all by +just changing the order of a few columns. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index fc51b74da1d..2ad748d4802 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -1,6 +1,6 @@ # Rake tasks for developers -## Setup db with developer seeds +## Set up db with developer seeds Note that if your db user does not have advanced privileges you must create the db manually before running this command. diff --git a/doc/development/rolling_out_changes_using_feature_flags.md b/doc/development/rolling_out_changes_using_feature_flags.md new file mode 100644 index 00000000000..905aa26a40b --- /dev/null +++ b/doc/development/rolling_out_changes_using_feature_flags.md @@ -0,0 +1,153 @@ +# Rolling out changes using feature flags + +[Feature flags](feature_flags.md) can be used to gradually roll out changes, be +it a new feature, or a performance improvement. By using feature flags, we can +comfortably measure the impact of our changes, while still being able to easily +disable those changes, without having to revert an entire release. + +## When to use feature flags + +Starting with GitLab 11.4, developers are required to use feature flags for +non-trivial changes. Such changes include: + +* New features (e.g. a new merge request widget, epics, etc). +* Complex performance improvements that may require additional testing in + production, such as rewriting complex queries. +* Invasive changes to the user interface, such as a new navigation bar or the + removal of a sidebar. +* Adding support for importing projects from a third-party service. + +In all cases, those working on the changes can best decide if a feature flag is +necessary. For example, changing the color of a button doesn't need a feature +flag, while changing the navigation bar definitely needs one. In case you are +uncertain if a feature flag is necessary, simply ask about this in the merge +request, and those reviewing the changes will likely provide you with an answer. + +When using a feature flag for UI elements, make sure to _also_ use a feature +flag for the underlying backend code, if there is any. This ensures there is +absolutely no way to use the feature until it is enabled. + +## The cost of feature flags + +When reading the above, one might be tempted to think this procedure is going to +add a lot of work. Fortunately, this is not the case, and we'll show why. For +this example we'll specify the cost of the work to do as a number, ranging from +0 to infinity. The greater the number, the more expensive the work is. The cost +does _not_ translate to time, it's just a way of measuring complexity of one +change relative to another. + +Let's say we are building a new feature, and we have determined that the cost of +this is 10. We have also determined that the cost of adding a feature flag check +in a variety of places is 1. If we do not use feature flags, and our feature +works as intended, our total cost is 10. This however is the best case scenario. +Optimising for the best case scenario is guaranteed to lead to trouble, whereas +optimising for the worst case scenario is almost always better. + +To illustrate this, let's say our feature causes an outage, and there's no +immediate way to resolve it. This means we'd have to take the following steps to +resolve the outage: + +1. Revert the release. +1. Perform any cleanups that might be necessary, depending on the changes that + were made. +1. Revert the commit, ensuring the "master" branch remains stable. This is + especially necessary if solving the problem can take days or even weeks. +1. Pick the revert commit into the appropriate stable branches, ensuring we + don't block any future releases until the problem is resolved. + +As history has shown, these steps are time consuming, complex, often involve +many developers, and worst of all: our users will have a bad experience using +GitLab.com until the problem is resolved. + +Now let's say that all of this has an associated cost of 10. This means that in +the worst case scenario, which we should optimise for, our total cost is now 20. + +If we had used a feature flag, things would have been very different. We don't +need to revert a release, and because feature flags are disabled by default we +don't need to revert and pick any Git commits. In fact, all we have to do is +disable the feature, and _maybe_ perform some cleanup. Let's say that the cost +of this is 1. In this case, our best case cost is 11: 10 to build the feature, +and 1 to add the feature flag. The worst case cost is now 12: 10 to build the +feature, 1 to add the feature flag, and 1 to disable it. + +Here we can see that in the best case scenario the work necessary is only a tiny +bit more compared to not using a feature flag. Meanwhile, the process of +reverting our changes has been made significantly cheaper, to the point of being +trivial. + +In other words, feature flags do not slow down the development process. Instead, +they speed up the process as managing incidents now becomes _much_ easier. Once +continuous deployments are easier to perform, the time to iterate on a feature +is reduced even further, as you no longer need to wait weeks before your changes +are available on GitLab.com. + +## Rolling out changes + +The procedure of using feature flags is straightforward, and similar to not +using them. You add the necessary tests (make sure to test both the on and off +states of your feature flag(s)), make sure they all pass, have the code +reviewed, etc. You then submit your merge request, and add the ~"feature flag" +label. This label is used to signal to release managers that your changes are +hidden behind a feature flag and that it is safe to pick the MR into a stable +branch, without the need for an exception request. + +When the changes are deployed it is time to start rolling out the feature to our +users. The exact procedure of rolling out a change is unspecified, as this can +vary from change to change. However, in general we recommend rolling out changes +incrementally, instead of enabling them for everybody right away. We also +recommend you to _not_ enable a feature _before_ the code is being deployed. +This allows you to separate rolling out a feature from a deploy, making it +easier to measure the impact of both separately. + +GitLab's feature library (using +[Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature +Flags](feature_flags.md) guide) supports rolling out changes to a percentage of +users. This in turn can be controlled using [GitLab +chatops](https://docs.gitlab.com/ee/ci/chatops/). + +For example, to enable a feature for 25% of all users, run the following in +Slack: + +``` +/chatops run feature set new_navigation_bar 25 +``` + +This will enable the feature for GitLab.com, with `new_navigation_bar` being the +name of the feature. We can also enable the feature for <https://dev.gitlab.org> +or <https://staging.gitlab.com>: + +``` +/chatops run feature set new_navigation_bar 25 --dev +/chatops run feature set new_navigation_bar 25 --staging +``` + +If you are not certain what percentages to use, simply use the following steps: + +1. 25% +1. 50% +1. 75% +1. 100% + +Between every step you'll want to wait a little while and monitor the +appropriate graphs on <https://dashboards.gitlab.net>. The exact time to wait +may differ. For some features a few minutes is enough, while for others you may +want to wait several hours or even days. This is entirely up to you, just make +sure it is clearly communicated to your team, and the Production team if you +anticipate any potential problems. + +Once a change is deemed stable, submit a new merge request to remove the +feature flag. This ensures the change is available to all users and self-hosted +instances. Make sure to add the ~"feature flag" label to this merge request so +release managers are aware the changes are hidden behind a feature flag. If the +merge request has to be picked into a stable branch (e.g. after the 7th), make +sure to also add the appropriate "Pick into X" label (e.g. "Pick into 11.4"). + +One might be tempted to think this will delay the release of a feature by at +least one month (= one release). This is not the case. A feature flag does not +have to stick around for a specific amount of time (e.g. at least one release), +instead they should stick around until the feature is deemed stable. Stable +means it works on GitLab.com without causing any problems, such as outages. In +most cases this will translate to a feature (with a feature flag) being shipped +in RC1, followed by the feature flag being removed in RC2. This in turn means +the feature will be stable by the time we publish a stable package around the +22nd of the month. diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 0cd63a54b55..67e4cfeda0e 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -59,6 +59,12 @@ parallelization, monitoring. --- +## [Review apps](review_apps.md) + +How review apps are set up for GitLab CE/EE and how to use them. + +--- + ## [Testing Rake tasks](testing_rake_tasks.md) Everything you should know about how to test Rake tasks. diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md new file mode 100644 index 00000000000..25c6371f3d7 --- /dev/null +++ b/doc/development/testing_guide/review_apps.md @@ -0,0 +1,82 @@ +# Review apps + +We currently have review apps available as a manual job in EE pipelines. Here is +[the first implementation](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6259). + +That said, [the Quality team is working](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6665) +on making Review Apps automatically deployed by each pipeline, both in CE and EE. + +## How does it work? + +1. On every EE [pipeline][gitlab-pipeline] during the `test` stage, you can + start the [`review` job][review-job] +1. The `review` job [triggers a pipeline][cng-pipeline] in the + [`CNG-mirror`][cng-mirror] [^1] project +1. The `CNG-mirror` pipeline creates the Docker images of each component (e.g. `gitlab-rails-ee`, + `gitlab-shell`, `gitaly` etc.) based on the commit from the + [GitLab pipeline][gitlab-pipeline] and store them in its + [registry][cng-mirror-registry] +1. Once all images are built, the review app is deployed using + [the official GitLab Helm chart][helm-chart] [^2] to the + [`review-apps-ee` Kubernetes cluster on GCP][review-apps-ee] + - The actual scripts used to deploy the review app can be found at + [`scripts/review_apps/review-apps.sh`][review-apps.sh] + - These scripts are basically + [our official Auto DevOps scripts][Auto-DevOps.gitlab-ci.yml] where the + default CNG images are overriden with the images built and stored in the + [`CNG-mirror` project's registry][cng-mirror-registry] +1. Once the `review` job succeeds, you should be able to use your review app + thanks to the direct link to it from the MR widget. The default username is + `root` and its password can be found in the 1Password secure note named + **gitlab-{ce,ee} review app's root password**. + +**Additional notes:** + +- The Kubernetes cluster is connected to the `gitlab-ee` project using [GitLab's + Kubernetes integration][gitlab-k8s-integration]. This basically allows to have + a link to the review app directly from the merge request widget. +- The manual `stop_review` in the `post-cleanup` stage can be used to stop a + review app manually, and is also started by GitLab once a branch is deleted +- [TBD] Review apps are cleaned up regularly using a pipeline schedule that runs + the [`scripts/review_apps/automated_cleanup.rb`][automated_cleanup.rb] script + +[^1]: We use the `CNG-mirror` project so that the `CNG`, (**C**loud **N**ative **G**itLab), project's registry is + not overloaded with a lot of transient Docker images. +[^2]: Since we're using [the official GitLab Helm chart][helm-chart], this means + you get the a dedicated environment for your branch that's very close to what it + would look in production + +## Frequently Asked Questions + +**Will it be too much to trigger CNG image builds on every test run? This could create thousands of unused docker images.** + + > We have to start somewhere and improve later. If we see this getting out of hand, we will revisit. + +**How big is the Kubernetes cluster?** + + > The cluster is currently setup with a single pool of preemptible + nodes, with a minimum of 1 node and a maximum of 30 nodes. + +**What are the machine running on the cluster?** + + > We're currently using `n1-standard-4` (4 vCPUs, 15 GB memory) machines. + +**How do we secure this from abuse? Apps are open to the world so we need to find a way to limit it to only us.** + + > This won't work for forks. We will add a root password to 1password shared vault. + +[gitlab-pipeline]: https://gitlab.com/gitlab-org/gitlab-ee/pipelines/29302122 +[review-job]: https://gitlab.com/gitlab-org/gitlab-ee/-/jobs/94294136 +[cng-mirror]: https://gitlab.com/gitlab-org/build/CNG-mirror +[cng-pipeline]: https://gitlab.com/gitlab-org/build/CNG-mirror/pipelines/29307727 +[cng-mirror-registry]: https://gitlab.com/gitlab-org/build/CNG-mirror/container_registry +[helm-chart]: https://gitlab.com/charts/gitlab/ +[review-apps-ee]: https://console.cloud.google.com/kubernetes/clusters/details/us-central1-b/review-apps-ee?project=gitlab-review-apps +[review-apps.sh]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/review-apps.sh +[automated_cleanup.rb]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/automated_cleanup.rb +[Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml +[gitlab-k8s-integration]: https://docs.gitlab.com/ee/user/project/clusters/index.html + +--- + +[Return to Testing documentation](index.md) diff --git a/doc/development/ux_guide/users.md b/doc/development/ux_guide/users.md index 6afb33cfc36..08f393132e8 100644 --- a/doc/development/ux_guide/users.md +++ b/doc/development/ux_guide/users.md @@ -154,7 +154,7 @@ He credits himself as being entirely responsible for moving his company to GitLa #### Updating to the latest release Matthieu introduced his company to GitLab. He is responsible for maintaining and managing the company's installation in addition to his day job. He feels updates are too frequent and he doesn't always have sufficient time to update GitLab. As a result, he's not up to date with releases. -Matthieu tried to set up automatic updates, however, as he isn't a Systems Administrator, he wasn't confident in his set-up. He feels he should be able to "upgrade without users even noticing" but hasn't figured out how to do this yet. Matthieu would like the "update process to be triggered from the Admin Panel, perhaps accompanied with a changelog and the option to skip updates." +Matthieu tried to set up automatic updates, however, as he isn't a Systems Administrator, he wasn't confident in his setup. He feels he should be able to "upgrade without users even noticing" but hasn't figured out how to do this yet. Matthieu would like the "update process to be triggered from the Admin Panel, perhaps accompanied with a changelog and the option to skip updates." Matthieu is looking for confirmation that his update procedure is "secure and efficient" so more tutorials related to this topic would be useful to him. diff --git a/doc/install/azure/index.md b/doc/install/azure/index.md index 570bd18c172..7835401cc0b 100644 --- a/doc/install/azure/index.md +++ b/doc/install/azure/index.md @@ -74,7 +74,7 @@ The first items we need to configure are the basic settings of the underlying vi > **Note:** if you're unsure which authentication type to use, select **Password** 1. If you chose **SSH public key** - enter your `SSH public key` into the field provided - _(read the [SSH documentation][GitLab-Docs-SSH] to learn more about how to setup SSH + _(read the [SSH documentation][GitLab-Docs-SSH] to learn more about how to set up SSH public keys)_ 1. If you chose **Password** - enter the password you wish to use _(this is the password that you will use later in this tutorial to [SSH] into the VM, so make sure it's a strong password/passphrase)_ @@ -154,7 +154,7 @@ on the Azure Dashboard (you may need to refresh the page): The new VM can also be accessed by clicking the `All resources` or `Virtual machines` icons in the Azure Portal sidebar navigation menu. -## Setup a domain name +## Set up a domain name The VM will have a public IP address (static by default), but Azure allows us to assign a friendly DNS name to the VM, so let's go ahead and do that. @@ -296,7 +296,7 @@ homepage for the project:  If you scroll further down the project's home page, you'll see some basic instructions on how to -setup a local clone of your new repository and push and pull from it: +set up a local clone of your new repository and push and pull from it:  @@ -347,7 +347,7 @@ If you're running [SSH] from the command-line (terminal), then type in the follo connect to your VM, substituting `username` and `your-azure-domain-name.com` for the correct values. Again, remember that your Azure VM domain name will be the one you -[setup previously in the tutorial](#set-up-a-domain-name). If you didn't setup a domain name for +[set up previously in the tutorial](#set-up-a-domain-name). If you didn't set up a domain name for your VM, you can use the IP address in its place in the following command: ```bash @@ -401,7 +401,7 @@ is now showing **"up-to-date"**: Naturally, we believe that GitLab is a great git repository tool. However, GitLab is a whole lot more than that too. GitLab unifies issues, code review, CI and CD into a single UI, helping you to move faster from idea to production, and in this tutorial we showed you how quick and easy it is to -setup and run your own instance of GitLab on Azure, Microsoft's cloud service. +set up and run your own instance of GitLab on Azure, Microsoft's cloud service. Azure is a great way to experiment with GitLab, and if you decide (as we hope) that GitLab is for you, you can continue to use Azure as your secure, scalable cloud provider or of course run GitLab @@ -424,7 +424,7 @@ Check out our other [Technical Articles][GitLab-Technical-Articles] or browse th - [Azure - Properly Shutdown an Azure VM][Azure-Properly-Shutdown-VM] - [SSH], [PuTTY] and [Using SSH in PuTTY][Using-SSH-In-Putty] -[Original-Blog-Post]: https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/ "How to Setup a GitLab Instance on Microsoft Azure" +[Original-Blog-Post]: https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/ "How to Set up a GitLab Instance on Microsoft Azure" [GitLab-Docs]: https://docs.gitlab.com/ce/README.html "GitLab Documentation" [GitLab-Technical-Articles]: https://docs.gitlab.com/ce/articles/index.html "GitLab Technical Articles" [GitLab-Docs-SSH]: https://docs.gitlab.com/ce/ssh/README.html "GitLab Documentation: SSH" diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index 4cf18f53239..acaed53e052 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -79,7 +79,7 @@ After installation or upgrade, remember to [convert any new tables](#tables-and- --- -GitLab 8.14 has introduced [a feature](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7420) requiring `utf8mb4` encoding to be supported in your GitLab MySQL Database, which is not the case if you have setup your database before GitLab 8.16. +GitLab 8.14 has introduced [a feature](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7420) requiring `utf8mb4` encoding to be supported in your GitLab MySQL Database, which is not the case if you have set up your database before GitLab 8.16. Follow the below instructions to ensure you use the most up to date requirements for your GitLab MySQL Database. diff --git a/doc/install/digitaloceandocker.md b/doc/install/digitaloceandocker.md index 8efc0530b8a..676392eacf2 100644 --- a/doc/install/digitaloceandocker.md +++ b/doc/install/digitaloceandocker.md @@ -87,7 +87,7 @@ You can add this to your `~/.bash_profile` file to ensure the `docker` client us + Container name: `gitlab-test-8.10` + GitLab version: **EE** `8.10.8-ee.0` -##### Setup container settings +##### Set up container settings ``` export SSH_PORT=2222 diff --git a/doc/install/installation.md b/doc/install/installation.md index 85431a80a81..7df81fbc46f 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -515,7 +515,7 @@ Make GitLab start on boot: sudo update-rc.d gitlab defaults 21 -### Setup Logrotate +### Set up Logrotate sudo cp lib/support/logrotate/gitlab /etc/logrotate.d/gitlab diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md index 70bc3ff770f..5c8a830ac8f 100644 --- a/doc/install/openshift_and_gitlab/index.md +++ b/doc/install/openshift_and_gitlab/index.md @@ -80,7 +80,7 @@ This will download the VirtualBox image and fire up the VM with some preconfigur values as you can see in the Vagrantfile. As you may have noticed, you need plenty of RAM (5GB in our example), so make sure you have enough. -Now that OpenShift is setup, let's see how the web console looks like. +Now that OpenShift is set up, let's see how the web console looks like. ### Explore the OpenShift web console diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md index 2f5d4142d04..5f129fd3bd1 100644 --- a/doc/install/relative_url.md +++ b/doc/install/relative_url.md @@ -124,5 +124,5 @@ To disable the relative URL: 1. Follow the same as above starting from 2. and set up the GitLab URL to one that doesn't contain a relative path. -[omnibus-rel]: http://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab "How to setup relative URL in Omnibus GitLab" +[omnibus-rel]: http://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab "How to set up relative URL in Omnibus GitLab" [restart gitlab]: ../administration/restart_gitlab.md#installations-from-source "How to restart GitLab" diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 5531dcde4e9..13a6a1c68ad 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -27,7 +27,7 @@ Please see the [installation from source guide](installation.md) and the [instal ### Non-Unix operating systems such as Windows -GitLab is developed for Unix operating systems. +GitLab is developed for Unix operating systems. It does **not** run on Windows, and we have no plans to support it in the near future. For the latest development status view this [issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/46567). Please consider using a virtual machine to run GitLab. @@ -103,19 +103,21 @@ features of GitLab work with MySQL/MariaDB: 1. MySQL support for subgroups was [dropped with GitLab 9.3][post]. See [issue #30472][30472] for more information. -1. GitLab Geo does [not support MySQL](https://docs.gitlab.com/ee/gitlab-geo/database.html#mysql-replication). -1. [Zero downtime migrations][zero] do not work with MySQL +1. Geo does [not support MySQL](https://docs.gitlab.com/ee/administration/geo/replication/database.html#mysql-replication). This means no supported Disaster Recovery solution if using MySQL. **[PREMIUM ONLY]** +1. [Zero downtime migrations][../update/README.md#upgrading-without-downtime] do not work with MySQL. 1. GitLab [optimizes the loading of dashboard events](https://gitlab.com/gitlab-org/gitlab-ce/issues/31806) using [PostgreSQL LATERAL JOINs](https://blog.heapanalytics.com/postgresqls-powerful-new-join-type-lateral/). 1. In general, SQL optimized for PostgreSQL may run much slower in MySQL due to differences in query planners. For example, subqueries that work well in PostgreSQL - may not be [performant in MySQL](https://dev.mysql.com/doc/refman/5.7/en/optimizing-subqueries.html) + may not be [performant in MySQL](https://dev.mysql.com/doc/refman/5.7/en/optimizing-subqueries.html). +1. Binary column index length is limited to 20 bytes. This is accomplished with [a hack](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/initializers/mysql_set_length_for_binary_indexes.rb). +1. MySQL requires a variety of hacks to increase limits on various columns, [for example](https://gitlab.com/gitlab-org/gitlab-ce/issues/49583). +1. [The milestone filter runs slower queries on MySQL](https://gitlab.com/gitlab-org/gitlab-ce/issues/51173#note_99391731). 1. We expect this list to grow over time. Existing users using GitLab with MySQL/MariaDB are advised to [migrate to PostgreSQL](../update/mysql_to_postgresql.md) instead. [30472]: https://gitlab.com/gitlab-org/gitlab-ce/issues/30472 -[zero]: ../update/README.md#upgrading-without-downtime [post]: https://about.gitlab.com/2017/06/22/gitlab-9-3-released/#dropping-support-for-subgroups-in-mysql ### PostgreSQL Requirements diff --git a/doc/integration/gmail_action_buttons_for_gitlab.md b/doc/integration/gmail_action_buttons_for_gitlab.md index 05a91d9bef9..c19320471e3 100644 --- a/doc/integration/gmail_action_buttons_for_gitlab.md +++ b/doc/integration/gmail_action_buttons_for_gitlab.md @@ -2,7 +2,7 @@ GitLab supports [Google actions in email](https://developers.google.com/gmail/markup/actions/actions-overview). -If correctly setup, emails that require an action will be marked in Gmail. +If correctly set up, emails that require an action will be marked in Gmail.  diff --git a/doc/policy/maintenance.md b/doc/policy/maintenance.md index 7f028565412..fc7b97b3cc2 100644 --- a/doc/policy/maintenance.md +++ b/doc/policy/maintenance.md @@ -44,7 +44,7 @@ This decision is made on a case-by-case basis. ## Upgrade recommendations -We encourage everyone to run the [latest stable release](https://about.gitlab.com/blog/categories/release/) to ensure that you can +We encourage everyone to run the [latest stable release](https://about.gitlab.com/blog/categories/releases/) to ensure that you can easily upgrade to the most secure and feature-rich GitLab experience. In order to make sure you can easily run the most recent stable release, we are working hard to keep the update process simple and reliable. diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 1d29f6d4e43..98fce7efb0b 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -523,7 +523,7 @@ more of the following options: - `BACKUP=timestamp_of_backup` - Required if more than one backup exists. Read what the [backup timestamp is about](#backup-timestamp). -- `force=yes` - Does not ask if the authorized_keys file should get regenerated and assumes 'yes' for warning that database tables will be removed. +- `force=yes` - Does not ask if the authorized_keys file should get regenerated and assumes 'yes' for warning that database tables will be removed, enabling the "Write to authorized_keys file" setting, and updating LDAP providers. If you are restoring into directories that are mountpoints you will need to make sure these directories are empty before attempting a restore. Otherwise GitLab diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md index f02f7b807cf..cd290a80314 100644 --- a/doc/security/two_factor_authentication.md +++ b/doc/security/two_factor_authentication.md @@ -11,7 +11,7 @@ You can read more about it here: ## Enforcing 2FA for all users Users on GitLab, can enable it without any admin's intervention. If you want to -enforce everyone to setup 2FA, you can choose from two different ways: +enforce everyone to set up 2FA, you can choose from two different ways: 1. Enforce on next login 2. Suggest on next login, but allow a grace period before enforcing. diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index a8aa11265d0..a645f65938f 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -44,6 +44,6 @@ This page gathers all the resources for the topic **Authentication** within GitL - [Kanboard Plugin GitLab Authentication](https://kanboard.net/plugin/gitlab-auth) - [Jenkins GitLab OAuth Plugin](https://wiki.jenkins-ci.org/display/JENKINS/GitLab+OAuth+Plugin) -- [Setup Gitlab CE with Active Directory authentication](https://www.caseylabs.com/setup-gitlab-ce-with-active-directory-authentication/) +- [Set up Gitlab CE with Active Directory authentication](https://www.caseylabs.com/setup-gitlab-ce-with-active-directory-authentication/) - [How to customize GitLab to support OpenID authentication](http://eric.van-der-vlist.com/blog/2013/11/23/how-to-customize-gitlab-to-support-openid-authentication/) - [Openshift - Configuring Authentication and User Agent](https://docs.openshift.org/latest/install_config/configuring_authentication.html#GitLab) diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index e778f1d83df..681dc8ff20d 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -71,11 +71,6 @@ For an overview on the creation of Auto DevOps, read the blog post [From 2/3 of ## Requirements -TIP: **Tip:** -For self-hosted installations, the easiest way to make use of Auto DevOps is to -install GitLab inside a Kubernetes cluster using the [GitLab Omnibus Helm Chart] -which automatically installs and configures everything you need! - To make full use of Auto DevOps, you will need: 1. **GitLab Runner** (needed for all stages) - Your Runner needs to be @@ -101,10 +96,6 @@ To make full use of Auto DevOps, you will need: Kubernetes cluster using the [`nginx-ingress`](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) Helm chart. - 1. **Wildcard TLS termination** - You can deploy the - [`kube-lego`](https://github.com/kubernetes/charts/tree/master/stable/kube-lego) - Helm chart to your Kubernetes cluster to automatically issue certificates - for your domains using Let's Encrypt. 1. **Prometheus** (needed for Auto Monitoring) - To enable Auto Monitoring, you will need Prometheus installed somewhere (inside or outside your cluster) and configured to scrape your Kubernetes cluster. To get response metrics @@ -148,18 +139,13 @@ Auto DevOps base domain to `1.2.3.4.nip.io`. Once set up, all requests will hit the load balancer, which in turn will route them to the Kubernetes pods that run your application(s). -NOTE: **Note:** -If GitLab is installed using the [GitLab Omnibus Helm Chart], there are two -options: provide a static IP, or have one assigned. For more information see the -relevant docs on the [network prerequisites](../../install/kubernetes/gitlab_omnibus.md#networking-prerequisites). - ## Using multiple Kubernetes clusters **[PREMIUM]** When using Auto DevOps, you may want to deploy different environments to different Kubernetes clusters. This is possible due to the 1:1 connection that [exists between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters). -In the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml) +In the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) (used behind the scenes by Auto DevOps), there are currently 3 defined environment names that you need to know: - `review/` (every environment starting with `review/`) @@ -482,10 +468,7 @@ The metrics include: - **Response Metrics:** latency, throughput, error rate - **System Metrics:** CPU utilization, memory utilization -If GitLab has been deployed using the [GitLab Omnibus Helm Chart], no -configuration is required. - -If you have installed GitLab using a different method, you need to: +In order to make use of monitoring you need to: 1. [Deploy Prometheus](../../user/project/integrations/prometheus.md#configuring-your-own-prometheus-server-within-kubernetes) into your Kubernetes cluster 1. If you would like response metrics, ensure you are running at least version @@ -849,7 +832,6 @@ curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https:/ [review-app]: ../../ci/review_apps/index.md [container-registry]: ../../user/project/container_registry.md [postgresql]: https://www.postgresql.org/ -[Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml -[GitLab Omnibus Helm Chart]: ../../install/kubernetes/gitlab_omnibus.md +[Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml [ee]: https://about.gitlab.com/pricing/ [ce-19507]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19507 diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index 89516dba60b..6ff27e495fb 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -395,7 +395,7 @@ Allow you to [organize issues](../../user/project/milestones/index.md) and merge ### Mirror Repositories -A project that is setup to automatically have its branches, tags, and commits [updated from an upstream repository](https://docs.gitlab.com/ee/workflow/repository_mirroring.html). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. +A project that is set up to automatically have its branches, tags, and commits [updated from an upstream repository](https://docs.gitlab.com/ee/workflow/repository_mirroring.html). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. ### MIT License @@ -673,7 +673,7 @@ Version control is a system that records changes to a file or set of files over ### Virtual Private Cloud (VPC) -A [VPC](https://docs.gitlab.com/ce/university/glossary/README.html#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [setup High Availability](https://docs.gitlab.com/ce/university/high-availability/aws/). +A [VPC](https://docs.gitlab.com/ce/university/glossary/README.html#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [set up High Availability](https://docs.gitlab.com/ce/university/high-availability/aws/). ### Virtual private server (VPS) diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 1bff1488746..0a7ce922de1 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -286,7 +286,7 @@ to make the EFS integration easier to manage. gitlab_rails['redis_port'] = 6379 Finally run reconfigure, you might find it useful to run a check and -a service status to make sure everything has been setup correctly. +a service status to make sure everything has been set up correctly. sudo gitlab-ctl reconfigure sudo gitlab-rake gitlab:check diff --git a/doc/university/support/README.md b/doc/university/support/README.md index 0cbae71d1f5..805af253367 100644 --- a/doc/university/support/README.md +++ b/doc/university/support/README.md @@ -37,7 +37,7 @@ Continue to look over remaining portions of the [University Overview](../README. Get your development machine ready to familiarize yourself with the codebase, the components, and to be prepared to reproduce issues that our users encounter - Install the [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) - - [Setup OpenLDAP as part of this](https://gitlab.com/gitlab-org/gitlab-development-kit#openldap) + - [Set up OpenLDAP as part of this](https://gitlab.com/gitlab-org/gitlab-development-kit#openldap) #### Become comfortable with the Installation processes that we support diff --git a/doc/university/training/end-user/README.md b/doc/university/training/end-user/README.md index 9b8a8db58e2..e5eb5d97e3b 100644 --- a/doc/university/training/end-user/README.md +++ b/doc/university/training/end-user/README.md @@ -80,7 +80,7 @@ git config --global user.name "Your Name" git config --global user.email you@example.com ``` -- If you don't use the global flag you can setup a different author for +- If you don't use the global flag you can set up a different author for each project - Check settings with: diff --git a/doc/update/4.2-to-5.0.md b/doc/update/4.2-to-5.0.md index 311664b2bc1..d292327efbd 100644 --- a/doc/update/4.2-to-5.0.md +++ b/doc/update/4.2-to-5.0.md @@ -32,7 +32,7 @@ cd /home/git/ sudo -u git -H git clone https://github.com/gitlabhq/gitlab-shell.git /home/git/gitlab-shell ``` -## 3. setup gitlab-shell +## 3. set up gitlab-shell ```bash # chmod all repos and files under git diff --git a/doc/update/7.5-to-7.6.md b/doc/update/7.5-to-7.6.md index f0dfb177b79..0d45a9528b9 100644 --- a/doc/update/7.5-to-7.6.md +++ b/doc/update/7.5-to-7.6.md @@ -82,7 +82,7 @@ git diff origin/7-5-stable:config/gitlab.yml.example origin/7-6-stable:config/gi * HTTP setups: Make `/etc/nginx/sites-available/gitlab` the same as [`lib/support/nginx/gitlab`][nginx] but with your settings * HTTPS setups: Make `/etc/nginx/sites-available/gitlab-ssl` the same as [`lib/support/nginx/gitlab-ssl`][nginx-ssl] but with your setting -#### Setup time zone (optional) +#### Set up time zone (optional) Consider setting the time zone in `gitlab.yml` otherwise GitLab will default to UTC. If you set a time zone previously in [`application.rb`][app] (unlikely), unset it. diff --git a/doc/update/7.6-to-7.7.md b/doc/update/7.6-to-7.7.md index 85de6b0c546..5e0b2ca7bcd 100644 --- a/doc/update/7.6-to-7.7.md +++ b/doc/update/7.6-to-7.7.md @@ -82,7 +82,7 @@ git diff origin/7-6-stable:config/gitlab.yml.example origin/7-7-stable:config/gi * HTTP setups: Make `/etc/nginx/sites-available/gitlab` the same as [`lib/support/nginx/gitlab`][nginx] but with your settings * HTTPS setups: Make `/etc/nginx/sites-available/gitlab-ssl` the same as [`lib/support/nginx/gitlab-ssl`][nginx-ssl] but with your setting -#### Setup time zone (optional) +#### Set up time zone (optional) Consider setting the time zone in `gitlab.yml` otherwise GitLab will default to UTC. If you set a time zone previously in [`application.rb`][app] (unlikely), unset it. diff --git a/doc/update/7.7-to-7.8.md b/doc/update/7.7-to-7.8.md index 7cee5f79a13..f5b1ebf0a9c 100644 --- a/doc/update/7.7-to-7.8.md +++ b/doc/update/7.7-to-7.8.md @@ -83,7 +83,7 @@ git diff origin/7-7-stable:config/gitlab.yml.example origin/7-8-stable:config/gi * HTTPS setups: Make `/etc/nginx/sites-available/gitlab-ssl` the same as [`lib/support/nginx/gitlab-ssl`][nginx-ssl] but with your settings. * A new `location /uploads/` section has been added that needs to have the same content as the existing `location @gitlab` section. -#### Setup time zone (optional) +#### Set up time zone (optional) Consider setting the time zone in `gitlab.yml` otherwise GitLab will default to UTC. If you set a time zone previously in [`application.rb`][app] (unlikely), unset it. diff --git a/doc/update/7.8-to-7.9.md b/doc/update/7.8-to-7.9.md index 5a8b689dbc1..0db7698936b 100644 --- a/doc/update/7.8-to-7.9.md +++ b/doc/update/7.8-to-7.9.md @@ -85,7 +85,7 @@ git diff origin/7-8-stable:config/gitlab.yml.example origin/7-9-stable:config/gi * HTTPS setups: Make `/etc/nginx/sites-available/gitlab-ssl` the same as [`lib/support/nginx/gitlab-ssl`][nginx-ssl] but with your settings. * A new `location /uploads/` section has been added that needs to have the same content as the existing `location @gitlab` section. -#### Setup time zone (optional) +#### Set up time zone (optional) Consider setting the time zone in `gitlab.yml` otherwise GitLab will default to UTC. If you set a time zone previously in [`application.rb`][app] (unlikely), unset it. diff --git a/doc/update/7.9-to-7.10.md b/doc/update/7.9-to-7.10.md index 99df51dbb99..782fb0736e6 100644 --- a/doc/update/7.9-to-7.10.md +++ b/doc/update/7.9-to-7.10.md @@ -81,7 +81,7 @@ git diff origin/7-9-stable:config/gitlab.yml.example origin/7-10-stable:config/g * HTTPS setups: Make `/etc/nginx/sites-available/gitlab-ssl` the same as [`lib/support/nginx/gitlab-ssl`][nginx-ssl] but with your settings. * A new `location /uploads/` section has been added that needs to have the same content as the existing `location @gitlab` section. -#### Setup time zone (optional) +#### Set up time zone (optional) Consider setting the time zone in `gitlab.yml` otherwise GitLab will default to UTC. If you set a time zone previously in [`application.rb`][app] (unlikely), unset it. diff --git a/doc/update/9.4-to-9.5.md b/doc/update/9.4-to-9.5.md index 1bfc1167c36..6a655f77a55 100644 --- a/doc/update/9.4-to-9.5.md +++ b/doc/update/9.4-to-9.5.md @@ -154,7 +154,7 @@ sudo -u git -H make #### New Gitaly configuration options required -In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell'. +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. ```shell echo ' diff --git a/doc/update/9.5-to-10.0.md b/doc/update/9.5-to-10.0.md index 8d1cf0f737b..7790d192a82 100644 --- a/doc/update/9.5-to-10.0.md +++ b/doc/update/9.5-to-10.0.md @@ -154,7 +154,7 @@ sudo -u git -H make #### New Gitaly configuration options required -In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell'. +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. ```shell echo ' diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index 910bd20f882..49f0ce2cd79 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -1,5 +1,8 @@ # Deleting a User Account +NOTE: **Note:** +Deleting a user will delete all projects in that user namespace. + - As a user, you can delete your own account by navigating to **Settings** > **Account** and selecting **Delete account** - As an admin, you can delete a user account by navigating to the **Admin Area**, selecting the **Users** tab, selecting a user, and clicking on **Delete user** diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index 8838efb18fe..e5411662511 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -69,7 +69,7 @@ of recovery codes. 1. Go to **Account**. 1. Click **Enable Two-Factor Authentication**. 1. Plug in your U2F device. -1. Click on **Setup New U2F Device**. +1. Click on **Set up New U2F Device**. 1. A light will start blinking on your device. Activate it by pressing its button. You will see a message indicating that your device was successfully set up. diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 1edc82ee9ef..41768998a59 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -127,8 +127,81 @@ applications running on the cluster. When GitLab creates the cluster, it enables and uses the legacy [Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/). The newer [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/) -authorization will be supported in a -[future release](https://gitlab.com/gitlab-org/gitlab-ce/issues/29398). +authorization is [experimental](#role-based-access-control-rbac). + +### Role-based access control (RBAC) **[CORE ONLY]** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21401) in GitLab 11.4. + +CAUTION: **Warning:** +The RBAC authorization is experimental. To enable it you need access to the +server where GitLab is installed. + +The support for RBAC-enabled clusters is hidden behind a feature flag. Once +the feature flag is enabled, GitLab will create the necessary service accounts +and privileges in order to install and run [GitLab managed applications](#installing-applications). + +To enable the feature flag: + +1. SSH into the server where GitLab is installed. +1. Enter the Rails console: + + **For Omnibus GitLab** + + ```sh + sudo gitlab-rails console + ``` + + **For installations from source** + + ```sh + sudo -u git -H bundle exec rails console + ``` + +1. Enable the RBAC authorization: + + ```ruby + Feature.enable('rbac_clusters') + ``` + +If you are creating a [new GKE cluster via +GitLab](#adding-and-creating-a-new-gke-cluster-via-gitlab), you will be +asked if you would like to create an RBAC-enabled cluster. Enabling this +setting will create a `gitlab` service account which will be used by +GitLab to manage the newly created cluster. To enable this, this service +account will have the `cluster-admin` privilege. + +If you are [adding an existing Kubernetes +cluster](#adding-an-existing-kubernetes-cluster), you will be asked if +the cluster you are adding is a RBAC-enabled cluster. Ensure the +token of the account has administrator privileges for the cluster. + +In both cases above, when you install Helm Tiller into your cluster, an +RBAC-enabled cluster will create a `tiller` service account, with `cluster-admin` +privileges in the `gitlab-managed-apps` namespace. This service account will be +added to the installed Helm Tiller and will be used by Helm to install and run +[GitLab managed applications](#installing-applications). + +The table below summarizes which resources will be created in a +RBAC-enabled cluster : + +| Name | Kind | Details | Created when | +| --- | --- | --- | --- | +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | +| `gitlab-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Creating a new GKE Cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | + + +Helm Tiller will also create additional service accounts and other RBAC +resources for each installed application. Consult the documentation for the +Helm charts for each application for details. + +NOTE: **Note:** +Auto DevOps will not successfully complete in a cluster that only has RBAC +authorization enabled. RBAC support for Auto DevOps is planned in a +[future release](https://gitlab.com/gitlab-org/gitlab-ce/issues/44597). ### Security of GitLab Runners @@ -161,13 +234,13 @@ with Tiller already installed, you should be careful as GitLab cannot detect it. By installing it via the applications will result into having it twice, which can lead to confusion during deployments. -| Application | GitLab version | Description | -| ----------- | :------------: | ----------- | -| [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps] or deploy your own web apps. | -| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | -| [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](#security-implications) before doing so. | -| [JupyterHub](http://jupyter.org/) | 11.0+ | [JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a multi-user service for managing notebooks across a team. [Jupyter Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a web-based interactive programming environment used for data analysis, visualization, and machine learning. We use [this](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) custom Jupyter image that installs additional useful packages on top of the base Jupyter. **Note**: Authentication will be enabled for any user of the GitLab server via OAuth2. HTTPS will be supported in a future release. | +| Application | GitLab version | Description | Helm Chart | +| ----------- | :------------: | ----------- | --------------- | +| [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | n/a | +| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps] or deploy your own web apps. | [stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) | +| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | [stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) | +| [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](#security-implications) before doing so. | [runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) | +| [JupyterHub](http://jupyter.org/) | 11.0+ | [JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a multi-user service for managing notebooks across a team. [Jupyter Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a web-based interactive programming environment used for data analysis, visualization, and machine learning. We use [this](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) custom Jupyter image that installs additional useful packages on top of the base Jupyter. **Note**: Authentication will be enabled for any user of the GitLab server via OAuth2. HTTPS will be supported in a future release. | [jupyter/jupyterhub](https://jupyterhub.github.io/helm-chart/) | ## Getting the external IP address diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index df850d4f68d..82cafcf432a 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -245,7 +245,7 @@ This will run mitmproxy on port `9000`. In another window, run: curl --proxy http://localhost:9000 https://httpbin.org/status/200 ``` -If everything is setup correctly, you will see information on the mitmproxy window and +If everything is set up correctly, you will see information on the mitmproxy window and no errors from the curl commands. #### Running the Docker daemon with a proxy diff --git a/doc/user/project/cycle_analytics.md b/doc/user/project/cycle_analytics.md index 7e788ae6220..ea843054f8e 100644 --- a/doc/user/project/cycle_analytics.md +++ b/doc/user/project/cycle_analytics.md @@ -154,7 +154,7 @@ You can [read more about permissions][permissions] in general. Learn more about Cycle Analytics in the following resources: -- [Cycle Analytics feature page](https://about.gitlab.com/solutions/cycle-analytics/) +- [Cycle Analytics feature page](https://about.gitlab.com/features/cycle-analytics/) - [Cycle Analytics feature preview](https://about.gitlab.com/2016/09/16/feature-preview-introducing-cycle-analytics/) - [Cycle Analytics feature highlight](https://about.gitlab.com/2016/09/21/cycle-analytics-feature-highlight/) diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 0b9b49f326f..ff647b2f0a2 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -51,7 +51,7 @@ To download a repository using a Deploy Token, you just need to: ```bash -git clone http://<username>:<deploy_token>@gitlab.example.com/tanuki/awesome_project.git +git clone https://<username>:<deploy_token>@gitlab.example.com/tanuki/awesome_project.git ``` Just replace `<username>` and `<deploy_token>` with the proper values diff --git a/doc/user/project/integrations/hangouts_chat.md b/doc/user/project/integrations/hangouts_chat.md index 47525617d95..20a71da927c 100644 --- a/doc/user/project/integrations/hangouts_chat.md +++ b/doc/user/project/integrations/hangouts_chat.md @@ -15,7 +15,7 @@ See also [the Hangouts Chat documentation for configuring incoming webhooks](htt ## On GitLab -When you have the **Webhook URL** for your Hangouts Chat room webhook, you can setup the GitLab service. +When you have the **Webhook URL** for your Hangouts Chat room webhook, you can set up the GitLab service. 1. Navigate to the [Integrations page](project_services.md#accessing-the-project-services) in your project's settings, i.e. **Project > Settings > Integrations**. 1. Select the **Hangouts Chat** project service to configure it. diff --git a/doc/user/project/integrations/img/webhooks_ssl.png b/doc/user/project/integrations/img/webhooks_ssl.png Binary files differindex f023e9665f2..e5777a2e99b 100644 --- a/doc/user/project/integrations/img/webhooks_ssl.png +++ b/doc/user/project/integrations/img/webhooks_ssl.png diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index b3821cf8391..ba8b79b911b 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -49,7 +49,7 @@ We have split this stage in steps so it is easier to follow. 1. The next step is to create a new user (e.g., `gitlab`) who has write access to projects in JIRA. Enter the user's name and a _valid_ e-mail address - since JIRA sends a verification e-mail to set-up the password. + since JIRA sends a verification e-mail to set up the password. _**Note:** JIRA creates the username automatically by using the e-mail prefix. You can change it later if you want._ diff --git a/doc/user/project/integrations/mattermost.md b/doc/user/project/integrations/mattermost.md index 3e77823a6aa..89de1fe4dcb 100644 --- a/doc/user/project/integrations/mattermost.md +++ b/doc/user/project/integrations/mattermost.md @@ -38,7 +38,7 @@ At the end, fill in your Mattermost details: | Field | Description | | ----- | ----------- | -| **Webhook** | The incoming webhook URL which you have to setup on Mattermost, it will be something like: http://mattermost.example/hooks/5xo… | +| **Webhook** | The incoming webhook URL which you have to set up on Mattermost, it will be something like: http://mattermost.example/hooks/5xo… | | **Username** | Optional username which can be on messages sent to Mattermost. Fill this in if you want to change the username of the bot. | | **Notify only broken pipelines** | If you choose to enable the **Pipeline** event and you want to be only notified about failed pipelines. | diff --git a/doc/user/project/integrations/mattermost_slash_commands.md b/doc/user/project/integrations/mattermost_slash_commands.md index 488f61c77a3..e031dcad2c3 100644 --- a/doc/user/project/integrations/mattermost_slash_commands.md +++ b/doc/user/project/integrations/mattermost_slash_commands.md @@ -102,7 +102,7 @@ in a new slash command.  -1. After you setup all the values, copy the token (we will use it below) and +1. After you set up all the values, copy the token (we will use it below) and click **Done**.  diff --git a/doc/user/project/integrations/microsoft_teams.md b/doc/user/project/integrations/microsoft_teams.md index 140c6738a49..ca32689910c 100644 --- a/doc/user/project/integrations/microsoft_teams.md +++ b/doc/user/project/integrations/microsoft_teams.md @@ -25,7 +25,7 @@ At the end fill in your Microsoft Teams details: | Field | Description | | ----- | ----------- | -| **Webhook** | The incoming webhook URL which you have to setup on Microsoft Teams. | +| **Webhook** | The incoming webhook URL which you have to set up on Microsoft Teams. | | **Notify only broken pipelines** | If you choose to enable the **Pipeline** event and you want to be only notified about failed pipelines. | After you are all done, click **Save changes** for the changes to take effect. diff --git a/doc/user/project/integrations/mock_ci.md b/doc/user/project/integrations/mock_ci.md index 6aefe5dbded..8b1908c46fe 100644 --- a/doc/user/project/integrations/mock_ci.md +++ b/doc/user/project/integrations/mock_ci.md @@ -2,7 +2,7 @@ **NB: This service is only listed if you are in a development environment!** -To setup the mock CI service server, respond to the following endpoints +To set up the mock CI service server, respond to the following endpoints - `commit_status`: `#{project.namespace.path}/#{project.path}/status/#{sha}.json` - Have your service return `200 { status: ['failed'|'canceled'|'running'|'pending'|'success'|'success_with_warnings'|'skipped'|'not_found'] }` diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index f687027e8c8..0b61a41aab0 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -8,7 +8,7 @@ within the GitLab interface.  -There are two ways to setup Prometheus integration, depending on where your apps are running: +There are two ways to set up Prometheus integration, depending on where your apps are running: * For deployments on Kubernetes, GitLab can automatically [deploy and manage Prometheus](#managed-prometheus-on-kubernetes) * For other deployment targets, simply [specify the Prometheus server](#manual-configuration-of-prometheus). diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 6104eadde35..e22f8e976be 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -57,6 +57,14 @@ You can turn this off in the webhook settings in your GitLab projects.  +## Branch filtering + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/20338) in GitLab 11.3. + +Push events can be filtered by branch using a branch name or wildcard pattern +to limit which push events are sent to your webhook endpoint. By default the +field is blank causing all push events to be sent to your webhook endpoint. + ## Events Below are described the supported events. diff --git a/doc/user/project/issues/issues_functionalities.md b/doc/user/project/issues/issues_functionalities.md index 46f25417fde..631f511b5fa 100644 --- a/doc/user/project/issues/issues_functionalities.md +++ b/doc/user/project/issues/issues_functionalities.md @@ -69,7 +69,7 @@ Learn more on the [Time Tracking documentation](../../../workflow/time_tracking. #### 6. Due date When you work on a tight schedule, and it's important to -have a way to setup a deadline for implementations and for solving +have a way to set up a deadline for implementations and for solving problems. This can be facilitated by the [due date](due_dates.md)). Due dates can be changed as many times as needed. diff --git a/doc/user/project/pipelines/job_artifacts.md b/doc/user/project/pipelines/job_artifacts.md index fc3970e2014..a8b47558c99 100644 --- a/doc/user/project/pipelines/job_artifacts.md +++ b/doc/user/project/pipelines/job_artifacts.md @@ -151,6 +151,20 @@ For example: https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/artifacts/master/browse?job=coverage ``` +There is also a URL to specific files, including html files that +are shown in [GitLab Pages](../../../administration/pages/index.md): + +``` +https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/file/<path>?job=<job_name> +``` + +For example, when a job `coverage` creates the artifact `htmlcov/index.html`, +you can access it at: + +``` +https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/artifacts/master/file/htmlcov/index.html?job=coverage +``` + The latest builds are also exposed in the UI in various places. Specifically, look for the download button in: diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 1c3915a5fdd..4d016277824 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -158,7 +158,9 @@ Find it under your project's **Repository > Graph**. ## Repository Languages For the default branch of each repository, GitLab will determine what programming languages -were used and display this on the projects pages. +were used and display this on the projects pages. If this information is missing, it will +be added after updating the default branch on the project. This process can take up to 5 +minutes.  diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index ad0ef60373c..127a30d6669 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -40,11 +40,6 @@ support Markdown, RDoc and AsciiDoc. For Markdown based pages, all the [Markdown features](../../markdown.md) are supported and for links there is some [wiki specific](../../markdown.md#wiki-specific-markdown) behavior. ->**Note:** -The wiki is based on a Git repository and contains only text files. Uploading -files via the web interface will upload them in GitLab itself, and they will -not be available if you clone the wiki repo locally. - In the web interface the commit message is optional, but the GitLab Wiki is based on Git and needs a commit message, so one will be created for you if you do not enter one. @@ -53,6 +48,14 @@ When you're ready, click the **Create page** and the new page will be created.  +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/33475) in GitLab 11.3. + +Starting with GitLab 11.3, any file that is uploaded to the wiki via GitLab's +interface will be stored in the wiki Git repository, and it will be available +if you clone the wiki repository locally. All uploaded files prior to GitLab +11.3 are stored in GitLab itself. If you want them to be part of the wiki's Git +repository, you will have to upload them again. + ## Editing a wiki page To edit a page, simply click on the **Edit** button. From there on, you can diff --git a/doc/workflow/lfs/lfs_administration.md b/doc/workflow/lfs/lfs_administration.md index 3f9ffedd61a..ec5943fd51b 100644 --- a/doc/workflow/lfs/lfs_administration.md +++ b/doc/workflow/lfs/lfs_administration.md @@ -54,7 +54,7 @@ to offload local hard disk R/W operations, and free up disk space significantly. GitLab is tightly integrated with `Fog`, so you can refer to its [documentation](http://fog.io/about/provider_documentation.html) to check which storage services can be integrated with GitLab. You can also use external object storage in a private local network. For example, -[Minio](https://www.minio.io/) is a standalone object storage service, is easy to setup, and works well with GitLab instances. +[Minio](https://www.minio.io/) is a standalone object storage service, is easy to set up, and works well with GitLab instances. GitLab provides two different options for the uploading mechanism: "Direct upload" and "Background upload". @@ -95,6 +95,7 @@ Here is a configuration example with S3. | `host` | S3 compatible host for when not using AWS, e.g. `localhost` or `storage.example.com` | s3.amazonaws.com | | `endpoint` | Can be used when configuring an S3 compatible service such as [Minio](https://www.minio.io), by entering a URL such as `http://127.0.0.1:9000` | (optional) | | `path_style` | Set to true to use `host/bucket_name/object` style paths instead of `bucket_name.host/object`. Leave as false for AWS S3 | false | +| `use_iam_profile` | Set to true to use IAM profile instead of access keys | false Here is a configuration example with GCS. |
