diff options
| author | Achilleas Pipinellis <axil@gitlab.com> | 2017-09-18 05:38:54 +0000 |
|---|---|---|
| committer | Achilleas Pipinellis <axil@gitlab.com> | 2017-09-18 05:38:54 +0000 |
| commit | 99b95ea97cd016e07e40cc361292be6a155325f8 (patch) | |
| tree | cf5316258cab80df747d119c8b397dc3a4ad8cd9 /doc | |
| parent | 8211ce1906217b4562d52c6d4c2192bba1f9764d (diff) | |
| parent | 2defc7b931ac46603780944907d7e19233ad1e97 (diff) | |
| download | gitlab-ce-99b95ea97cd016e07e40cc361292be6a155325f8.tar.gz | |
Merge branch 'master' into 'docs-replace-pipelines-cicd'
# Conflicts:
# doc/ci/variables/README.md
Diffstat (limited to 'doc')
35 files changed, 672 insertions, 111 deletions
diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 5732b6a1ca4..40099dcc967 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -145,8 +145,8 @@ Omnibus installations: ```ruby # /etc/gitlab/gitlab.rb git_data_dirs({ - { 'default' => { 'path' => '/mnt/gitlab/default', 'gitaly_address' => 'tcp://gitlab.internal:9999' } }, - { 'storage1' => { 'path' => '/mnt/gitlab/storage1', 'gitaly_address' => 'tcp://gitlab.internal:9999' } }, + 'default' => { 'path' => '/mnt/gitlab/default', 'gitaly_address' => 'tcp://gitlab.internal:9999' }, + 'storage1' => { 'path' => '/mnt/gitlab/storage1', 'gitaly_address' => 'tcp://gitlab.internal:9999' }, }) gitlab_rails['gitaly_token'] = 'abc123secret' diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md index e99a7ee29cc..1304476e678 100644 --- a/doc/administration/reply_by_email.md +++ b/doc/administration/reply_by_email.md @@ -77,6 +77,33 @@ and use [an application password](https://support.google.com/mail/answer/185833) To set up a basic Postfix mail server with IMAP access on Ubuntu, follow the [Postfix setup documentation](reply_by_email_postfix_setup.md). +### Security Concerns + +**WARNING:** Be careful when choosing the domain used for receiving incoming +email. + +For the sake of example, suppose your top-level company domain is `hooli.com`. +All employees in your company have an email address at that domain via Google +Apps, and your company's private Slack instance requires a valid `@hooli.com` +email address in order to sign up. + +If you also host a public-facing GitLab instance at `hooli.com` and set your +incoming email domain to `hooli.com`, an attacker could abuse the "Create new +issue by email" feature by using a project's unique address as the email when +signing up for Slack, which would send a confirmation email, which would create +a new issue on the project owned by the attacker, allowing them to click the +confirmation link and validate their account on your company's private Slack +instance. + +We recommend receiving incoming email on a subdomain, such as +`incoming.hooli.com`, and ensuring that you do not employ any services that +authenticate solely based on access to an email domain such as `*.hooli.com.` +Alternatively, use a dedicated domain for GitLab email communications such as +`hooli-gitlab.com`. + +See GitLab issue [#30366](https://gitlab.com/gitlab-org/gitlab-ce/issues/30366) +for a real-world example of this exploit. + ### Omnibus package installations 1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the @@ -141,7 +168,7 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow the # The IDLE command timeout. gitlab_rails['incoming_email_idle_timeout'] = 60 ``` - + ```ruby # Configuration for Microsoft Exchange mail server w/ IMAP enabled, assumes mailbox incoming@exchange.example.com gitlab_rails['incoming_email_enabled'] = true @@ -253,7 +280,7 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow the # The IDLE command timeout. idle_timeout: 60 ``` - + ```yaml # Configuration for Microsoft Exchange mail server w/ IMAP enabled, assumes mailbox incoming@exchange.example.com incoming_email: diff --git a/doc/api/keys.md b/doc/api/keys.md index 376ac27df3a..ddcf7830621 100644 --- a/doc/api/keys.md +++ b/doc/api/keys.md @@ -32,6 +32,7 @@ Parameters: "twitter": "", "website_url": "", "email": "john@example.com", + "theme_id": 2, "color_scheme_id": 1, "projects_limit": 10, "current_sign_in_at": null, diff --git a/doc/api/session.md b/doc/api/session.md index f79eac11689..b97e26f34a2 100644 --- a/doc/api/session.md +++ b/doc/api/session.md @@ -39,6 +39,7 @@ Example response: "twitter": "", "website_url": "", "email": "john@example.com", + "theme_id": 1, "color_scheme_id": 1, "projects_limit": 10, "current_sign_in_at": "2015-07-07T07:10:58.392Z", diff --git a/doc/api/users.md b/doc/api/users.md index 9f3e4caf2f4..6d5db16b36a 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -72,6 +72,7 @@ GET /users "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 2, "projects_limit": 100, @@ -105,6 +106,7 @@ GET /users "organization": "", "last_sign_in_at": null, "confirmed_at": "2012-05-30T16:53:06.148Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 3, "projects_limit": 100, @@ -215,6 +217,7 @@ Parameters: "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 2, "projects_limit": 100, @@ -341,6 +344,7 @@ GET /user "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 2, "projects_limit": 100, @@ -387,6 +391,7 @@ GET /user "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 2, "projects_limit": 100, diff --git a/doc/ci/environments.md b/doc/ci/environments.md index c1362b7bd5b..acd5682841a 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -240,55 +240,18 @@ Remember that if your environment's name is `production` (all lowercase), then it will get recorded in [Cycle Analytics](../user/project/cycle_analytics.md). Double the benefit! -## Web terminals - ->**Note:** -Web terminals were added in GitLab 8.15 and are only available to project -masters and owners. - -If you deploy to your environments with the help of a deployment service (e.g., -the [Kubernetes service][kubernetes-service], GitLab can open -a terminal session to your environment! This is a very powerful feature that -allows you to debug issues without leaving the comfort of your web browser. To -enable it, just follow the instructions given in the service documentation. - -Once enabled, your environments will gain a "terminal" button: - - - -You can also access the terminal button from the page for a specific environment: - - - -Wherever you find it, clicking the button will take you to a separate page to -establish the terminal session: - - - -This works just like any other terminal - you'll be in the container created -by your deployment, so you can run shell commands and get responses in real -time, check the logs, try out configuration or code tweaks, etc. You can open -multiple terminals to the same environment - they each get their own shell -session - and even a multiplexer like `screen` or `tmux`! - ->**Note:** -Container-based deployments often lack basic tools (like an editor), and may -be stopped or restarted at any time. If this happens, you will lose all your -changes! Treat this as a debugging tool, not a comprehensive online IDE. - ---- - -While this is fine for deploying to some stable environments like staging or -production, what happens for branches? So far we haven't defined anything -regarding deployments for branches other than `master`. Dynamic environments -will help us achieve that. - ## Dynamic environments As the name suggests, it is possible to create environments on the fly by just declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is the basis of [Review apps](review_apps/index.md). +>**Note:** +The `name` and `url` parameters can use any of the defined CI variables, +including predefined, secure variables and `.gitlab-ci.yml` +[`variables`](yaml/README.md#variables). +You however cannot use variables defined under `script` or on the Runner's side. + GitLab Runner exposes various [environment variables][variables] when a job runs, and as such, you can use them as environment names. Let's add another job in our example which will deploy to all branches except `master`: @@ -434,7 +397,8 @@ Let's briefly see where URL that's defined in the environments is exposed. ## Making use of the environment URL -The environment URL is exposed in a few places within GitLab. +The [environment URL](yaml/README.md#environments-url) is exposed in a few +places within GitLab. | In a merge request widget as a link | In the Environments view as a button | In the Deployments view as a button | | -------------------- | ------------ | ----------- | @@ -598,7 +562,7 @@ exist, you should see something like: >**Notes:** > -- For the monitor dashboard to appear, you need to: +- For the monitoring dashboard to appear, you need to: - Have enabled the [Prometheus integration][prom] - Configured Prometheus to collect at least one [supported metric](../user/project/integrations/prometheus_library/metrics.md) - With GitLab 9.2, all deployments to an environment are shown directly on the @@ -608,8 +572,7 @@ If you have enabled [Prometheus for monitoring system and response metrics](http Once configured, GitLab will attempt to retrieve [supported performance metrics](https://docs.gitlab.com/ee/user/project/integrations/prometheus_library/metrics.html) for any environment which has had a successful deployment. If monitoring data was -successfully retrieved, a Monitoring button will appear on the environment's -detail page. +successfully retrieved, a Monitoring button will appear for each environment.  @@ -623,6 +586,49 @@ version of the app, all without leaving GitLab.  +## Web terminals + +>**Note:** +Web terminals were added in GitLab 8.15 and are only available to project +masters and owners. + +If you deploy to your environments with the help of a deployment service (e.g., +the [Kubernetes service][kubernetes-service], GitLab can open +a terminal session to your environment! This is a very powerful feature that +allows you to debug issues without leaving the comfort of your web browser. To +enable it, just follow the instructions given in the service documentation. + +Once enabled, your environments will gain a "terminal" button: + + + +You can also access the terminal button from the page for a specific environment: + + + +Wherever you find it, clicking the button will take you to a separate page to +establish the terminal session: + + + +This works just like any other terminal - you'll be in the container created +by your deployment, so you can run shell commands and get responses in real +time, check the logs, try out configuration or code tweaks, etc. You can open +multiple terminals to the same environment - they each get their own shell +session - and even a multiplexer like `screen` or `tmux`! + +>**Note:** +Container-based deployments often lack basic tools (like an editor), and may +be stopped or restarted at any time. If this happens, you will lose all your +changes! Treat this as a debugging tool, not a comprehensive online IDE. + +--- + +While this is fine for deploying to some stable environments like staging or +production, what happens for branches? So far we haven't defined anything +regarding deployments for branches other than `master`. Dynamic environments +will help us achieve that. + ## Checkout deployments locally Since 8.13, a reference in the git repository is saved for each deployment, so diff --git a/doc/ci/img/environments_monitoring.png b/doc/ci/img/environments_monitoring.png Binary files differindex d9c46ea4c95..dcffdd1fdb8 100644 --- a/doc/ci/img/environments_monitoring.png +++ b/doc/ci/img/environments_monitoring.png diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 0737803206e..ebcb92b5db1 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -158,17 +158,17 @@ script: settings. Follow the discussion in issue [#13784][ce-13784] for masking the secret variables. -GitLab CI allows you to define per-project or per-group **secret variables** -that are set in the build environment. The secret variables are stored out of -the repository (`.gitlab-ci.yml`) and are securely passed to GitLab Runner -making them available in the build environment. It's the recommended method to -use for storing things like passwords, secret keys and credentials. +GitLab CI allows you to define per-project or per-group secret variables +that are set in the pipeline environment. The secret variables are stored out of +the repository (not in `.gitlab-ci.yml`) and are securely passed to GitLab Runner +making them available during a pipeline run. It's the recommended method to +use for storing things like passwords, SSH keys and credentials. Project-level secret variables can be added by going to your project's -**Settings ➔ CI/CD**, then finding the section called **Secret variables**. +**Settings > CI/CD**, then finding the section called **Secret variables**. Likewise, group-level secret variables can be added by going to your group's -**Settings ➔ CI/CD**, then finding the section called **Secret variables**. +**Settings > CI/CD**, then finding the section called **Secret variables**. Any variables of [subgroups] will be inherited recursively. Once you set them, they will be available for all subsequent pipelines. You can also @@ -185,8 +185,8 @@ protected, it would only be securely passed to pipelines running on the protected variables. Protected variables can be added by going to your project's -**Settings ➔ CI/CD**, then finding the section called -**Secret variables**, and check *Protected*. +**Settings > CI/CD**, then finding the section called +**Secret variables**, and check "Protected". Once you set them, they will be available for all subsequent pipelines. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 78733b9cc4b..f69d71a5c39 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -727,6 +727,9 @@ deployment to the `production` environment. - Before GitLab 8.11, the name of an environment could be defined as a string like `environment: production`. The recommended way now is to define it under the `name` keyword. +- The `name` parameter can use any of the defined CI variables, + including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). + You however cannot use variables defined under `script`. The `environment` name can contain: @@ -762,6 +765,9 @@ deploy to production: - Introduced in GitLab 8.11. - Before GitLab 8.11, the URL could be added only in GitLab's UI. The recommended way now is to define it in `.gitlab-ci.yml`. +- The `url` parameter can use any of the defined CI variables, + including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). + You however cannot use variables defined under `script`. This is an optional value that when set, it exposes buttons in various places in GitLab which when clicked take you to the defined URL. @@ -841,10 +847,9 @@ The `stop_review_app` job is **required** to have the following keywords defined **Notes:** - [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. - The `$CI_ENVIRONMENT_SLUG` was [introduced][ce-7983] in GitLab 8.15. - -`environment` can also represent a configuration hash with `name` and `url`. -These parameters can use any of the defined [CI variables](#variables) -(including predefined, secure variables and `.gitlab-ci.yml` variables). +- The `name` and `url` parameters can use any of the defined CI variables, + including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). + You however cannot use variables defined under `script`. For example: diff --git a/doc/development/README.md b/doc/development/README.md index dd150421b65..3096d9f25f0 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -43,6 +43,7 @@ - [Object state models](object_state_models.md) - [Building a package for testing purposes](build_test_package.md) - [Manage feature flags](feature_flags.md) +- [View sent emails or preview mailers](emails.md) ## Databases @@ -60,6 +61,7 @@ - [Ordering Table Columns](ordering_table_columns.md) - [Verifying Database Capabilities](verifying_database_capabilities.md) - [Hash Indexes](hash_indexes.md) +- [Swapping Tables](swapping_tables.md) ## i18n diff --git a/doc/development/emails.md b/doc/development/emails.md new file mode 100644 index 00000000000..18f47f44cb5 --- /dev/null +++ b/doc/development/emails.md @@ -0,0 +1,23 @@ +# Dealing with email in development + +## Sent emails + +To view rendered emails "sent" in your development instance, visit +[`/rails/letter_opener`](http://localhost:3000/rails/letter_opener). + +## Mailer previews + +Rails provides a way to preview our mailer templates in HTML and plaintext using +dummy data. + +The previews live in [`spec/mailers/previews`][previews] and can be viewed at +[`/rails/mailers`](http://localhost:3000/rails/mailers). + +See the [Rails guides] for more info. + +[previews]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/spec/mailers/previews +[Rails guides]: http://guides.rubyonrails.org/action_mailer_basics.html#previewing-emails + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/swapping_tables.md b/doc/development/swapping_tables.md new file mode 100644 index 00000000000..6b990ece72c --- /dev/null +++ b/doc/development/swapping_tables.md @@ -0,0 +1,53 @@ +# Swapping Tables + +Sometimes you need to replace one table with another. For example, when +migrating data in a very large table it's often better to create a copy of the +table and insert & migrate the data into this new table in the background. + +Let's say you want to swap the table "events" with "events_for_migration". In +this case you need to follow 3 steps: + +1. Rename "events" to "events_temporary" +2. Rename "events_for_migration" to "events" +3. Rename "events_temporary" to "events_for_migration" + +Rails allows you to do this using the `rename_table` method: + +```ruby +rename_table :events, :events_temporary +rename_table :events_for_migration, :events +rename_table :events_temporary, :events_for_migration +``` + +This does not require any downtime as long as the 3 `rename_table` calls are +executed in the _same_ database transaction. Rails by default uses database +transactions for migrations, but if it doesn't you'll need to start one +manually: + +```ruby +Event.transaction do + rename_table :events, :events_temporary + rename_table :events_for_migration, :events + rename_table :events_temporary, :events_for_migration +end +``` + +Once swapped you _have to_ reset the primary key of the new table. For +PostgreSQL you can use the `reset_pk_sequence!` method like so: + +```ruby +reset_pk_sequence!('events') +``` + +For MySQL however you need to do run the following: + +```ruby +amount = Event.pluck('COALESCE(MAX(id), 1)').first + +execute "ALTER TABLE events AUTO_INCREMENT = #{amount}" +``` + +Failure to reset the primary keys will result in newly created rows starting +with an ID value of 1. Depending on the existing data this can then lead to +duplicate key constraints from popping up, preventing users from creating new +data. diff --git a/doc/install/installation.md b/doc/install/installation.md index 66eb7675896..200cd94f43c 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -299,9 +299,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 9-5-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-0-stable gitlab -**Note:** You can change `9-5-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `10-0-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index c799f88ad74..467d5b92e0c 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -19,7 +19,7 @@ should be deployed, upgraded, and configured. ## GitLab-Omnibus Chart (Recommended) > **Note**: This chart is in beta while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being added. -This chart is the best available way to operate GitLab on Kubernetes. It deploys and configures nearly all features of GitLab, including: a [Runner](https://docs.gitlab.com/runner/), [Container Registry](https://docs.gitlab.com/ee/user/project/container_registry.html#gitlab-container-registry), [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/), [automatic SSL](https://github.com/kubernetes/charts/tree/master/stable/kube-lego), and a [load balancer](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). It is based on our [GitLab Omnibus Docker Images](https://docs.gitlab.com/omnibus/docker/README.html). +This chart is the best available way to operate GitLab on Kubernetes. It deploys and configures nearly all features of GitLab, including: a [Runner](https://docs.gitlab.com/runner/), [Container Registry](../../user/project/container_registry.html#gitlab-container-registry), [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/), [automatic SSL](https://github.com/kubernetes/charts/tree/master/stable/kube-lego), and a [load balancer](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). It is based on our [GitLab Omnibus Docker Images](https://docs.gitlab.com/omnibus/docker/README.html). Once the [cloud native charts](#upcoming-cloud-native-helm-charts) are ready for production use, this chart will be deprecated. Due to the difficulty in supporting upgrades to the new architecture, migrating will require exporting data out of this instance and importing it into the new deployment. @@ -41,7 +41,7 @@ This is a large project and will be worked on over the span of multiple releases ### GitLab Runner Chart -If you already have a GitLab instance running, inside or outside of Kubernetes, and you'd like to leverage the Runner's [Kubernetes capabilities](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/executors/kubernetes.md), it can be deployed with the GitLab Runner chart. +If you already have a GitLab instance running, inside or outside of Kubernetes, and you'd like to leverage the Runner's [Kubernetes capabilities](https://docs.gitlab.com/runner/executors/kubernetes.html), it can be deployed with the GitLab Runner chart. Learn more about [gitlab-runner chart.](gitlab_runner_chart.md) diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index babf44d2665..b31b8eaaca0 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -323,6 +323,23 @@ container registry. **Restarting a pod, scaling a service, or other actions whic require on-going access to the registry will fail**. On-going secure access is planned for a subsequent release. +## Disable the banner instance wide + +If an administrater would like to disable the banners on an instance level, this +feature can be disabled either through the console: + +```basb +$ gitlab-rails console +[1] pry(main)> Feature.get(:auto_devops_banner_disabled).disable +=> true +``` + +Or through the HTTP API with the admin access token: + +``` +curl --data "value=true" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled +``` + ## Troubleshooting - Auto Build and Auto Test may fail in detecting your language/framework. There diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index f23c9d794b4..564dd3222ac 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -18,45 +18,66 @@ example for this guide. It contains two files: ## Fork sample project on GitLab.com Let’s start by forking our sample application. Go to [the project -page](https://gitlab.com/gitlab-examples/minimal-ruby-app) and press the `Fork` -button. Soon you should have a project under your namespace with the necessary -files. +page](https://gitlab.com/auto-devops-examples/minimal-ruby-app) and press the +**Fork** button. Soon you should have a project under your namespace with the +necessary files. ## Setup your own cluster on Google Container Engine -If you do not already have a Google Cloud account, create one at https://console.cloud.google.com. +If you do not already have a Google Cloud account, create one at +https://console.cloud.google.com. -Visit the [`Container Engine`](https://console.cloud.google.com/kubernetes/list) tab and create a new cluster. You can change the name and leave the rest of the default settings. Once you have your cluster running, you need to connect to the cluster by following the Google interface. +Visit the [**Container Engine**](https://console.cloud.google.com/kubernetes/list) +tab and create a new cluster. You can change the name and leave the rest of the +default settings. Once you have your cluster running, you need to connect to the +cluster by following the Google interface. ## Connect to Kubernetes cluster You need to have the Google Cloud SDK installed. e.g. -On OSX, install [homebrew](https://brew.sh): +On macOS, install [homebrew](https://brew.sh): 1. Install Brew Caskroom: `brew install caskroom/cask/brew-cask` 2. Install Google Cloud SDK: `brew cask install google-cloud-sdk` -3. Add `kubectl`: `gcloud components install kubectl` +3. Add `kubectl` with: `gcloud components install kubectl` 4. Log in: `gcloud auth login` -Now go back to the Google interface, find your cluster, and follow the instructions under `Connect to the cluster` and open the Kubernetes Dashboard. It will look something like `gcloud container clusters get-credentials ruby-autodeploy \ --zone europe-west2-c --project api-project-XXXXXXX` and then `kubectl proxy`. +Now go back to the Google interface, find your cluster, follow the instructions +under "Connect to the cluster" and open the Kubernetes Dashboard. It will look +something like: + +```sh +gcloud container clusters get-credentials ruby-autodeploy \ --zone europe-west2-c --project api-project-XXXXXXX +``` + +Finally, run `kubectl proxy`.  ## Copy credentials to GitLab.com project -Once you have the Kubernetes Dashboard interface running, you should visit `Secrets` under the `Config` section. There you should find the settings we need for GitLab integration: ca.crt and token. +Once you have the Kubernetes Dashboard interface running, you should visit +**Secrets** under the "Config" section. There, you should find the settings we +need for GitLab integration: `ca.crt` and token.  -You need to copy-paste the ca.crt and token into your project on GitLab.com in the Kubernetes integration page under project **Settings > Integrations > Project services > Kubernetes**. Don't actually copy the namespace though. Each project should have a unique namespace, and by leaving it blank, GitLab will create one for you. +You need to copy-paste the `ca.crt` and token into your project on GitLab.com in +the Kubernetes integration page under project +**Settings > Integrations > Project services > Kubernetes**. Don't actually copy +the namespace though. Each project should have a unique namespace, and by leaving +it blank, GitLab will create one for you.  -For API URL, you should use the `Endpoint` IP from your cluster page on Google Cloud Platform. +For the API URL, you should use the "Endpoint" IP from your cluster page on +Google Cloud Platform. ## Expose application to the world -In order to be able to visit your application, you need to install an NGINX ingress controller and point your domain name to its external IP address. +In order to be able to visit your application, you need to install an NGINX +ingress controller and point your domain name to its external IP address. Let's +see how that's done. ### Set up Ingress controller @@ -68,28 +89,49 @@ helm init helm install --name ruby-app stable/nginx-ingress ``` -This should create several services including `ruby-app-nginx-ingress-controller`. You can list your services by running `kubectl get svc` to confirm that. +This should create several services including `ruby-app-nginx-ingress-controller`. +You can list your services by running `kubectl get svc` to confirm that. ### Point DNS at Cluster IP -Find out the external IP address of the `ruby-app-nginx-ingress-controller` by running: +Find out the external IP address of the `ruby-app-nginx-ingress-controller` by +running: ```sh kubectl get svc ruby-app-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' ``` -Use this IP address to configure your DNS. This part heavily depends on your preferences and domain provider. But in case you are not sure, just create an A record with a wildcard host like `*.<your-domain>`. +Use this IP address to configure your DNS. This part heavily depends on your +preferences and domain provider. But in case you are not sure, just create an +A record with a wildcard host like `*.<your-domain>`. -Use `nslookup minimal-ruby-app-staging.<yourdomain>` to confirm that domain is assigned to the cluster IP. +Use `nslookup minimal-ruby-app-staging.<yourdomain>` to confirm that domain is +assigned to the cluster IP. ## Set up Auto DevOps -In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps section. Select "Enable Auto DevOps", add in your base domain, and save. +In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps +section. Select "Enable Auto DevOps", add in your base domain, and save.  -Then trigger your first pipeline run. This will create a new pipeline with several jobs: `build`, `test`, `codequality`, and `production`. The `build` job will create a docker image with your new change and push it to the GitLab Container Registry. The `test` job will test your change. The `codequality` job will run static analysis on your change. The `production` job will deploy your change to a production application. Once the deploy job succeeds you should be able to see your application by visiting the Kubernetes dashboard. Select the namespace of your project, which will look like `minimal-ruby-app-23`, but with a unique ID for your project, and your app will be listed as "production" under the Deployment tab. +Next, a pipeline needs to be triggered. Since the test project doesn't have a +`.gitlab-ci.yml`, you need to either push a change to the repository or +manually visit `https://gitlab.com/<username>/minimal-ruby-app/pipelines/run`, +where `<username>` is your username. + +This will create a new pipeline with several jobs: `build`, `test`, `codequality`, +and `production`. The `build` job will create a Docker image with your new +change and push it to the Container Registry. The `test` job will test your +changes, whereas the `codequality` job will run static analysis on your changes. +Finally, the `production` job will deploy your changes to a production application. + +Once the deploy job succeeds you should be able to see your application by +visiting the Kubernetes dashboard. Select the namespace of your project, which +will look like `minimal-ruby-app-23`, but with a unique ID for your project, +and your app will be listed as "production" under the Deployment tab. -Once its ready - just visit http://minimal-ruby-app.example.com to see “Hello, world!” +Once its ready, just visit `http://minimal-ruby-app.example.com` to see the +famous "Hello, world!"! [ce-37115]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37115 diff --git a/doc/update/8.17-to-9.0.md b/doc/update/8.17-to-9.0.md index 2abc57da1a0..baab217b6b7 100644 --- a/doc/update/8.17-to-9.0.md +++ b/doc/update/8.17-to-9.0.md @@ -236,7 +236,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-17-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-0-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.0-to-9.1.md b/doc/update/9.0-to-9.1.md index 3fd1d023d2a..6f1870a1366 100644 --- a/doc/update/9.0-to-9.1.md +++ b/doc/update/9.0-to-9.1.md @@ -236,7 +236,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-0-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-1-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.1-to-9.2.md b/doc/update/9.1-to-9.2.md index 5f7a616cc7d..ce72b313031 100644 --- a/doc/update/9.1-to-9.2.md +++ b/doc/update/9.1-to-9.2.md @@ -194,7 +194,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-1-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-2-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md index 9d0b0da7edb..779ced0cf75 100644 --- a/doc/update/9.2-to-9.3.md +++ b/doc/update/9.2-to-9.3.md @@ -230,7 +230,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-2-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-3-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.3-to-9.4.md b/doc/update/9.3-to-9.4.md index 9ee01bc9c51..78d8a6c7de5 100644 --- a/doc/update/9.3-to-9.4.md +++ b/doc/update/9.3-to-9.4.md @@ -243,7 +243,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-3-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-4-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.4-to-9.5.md b/doc/update/9.4-to-9.5.md index 1b5a15589af..a7255142ef5 100644 --- a/doc/update/9.4-to-9.5.md +++ b/doc/update/9.4-to-9.5.md @@ -252,7 +252,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-4-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-5-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.5-to-10.0.md b/doc/update/9.5-to-10.0.md new file mode 100644 index 00000000000..8581e6511f2 --- /dev/null +++ b/doc/update/9.5-to-10.0.md @@ -0,0 +1,356 @@ +# From 9.5 to 10.0 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.3.tar.gz +echo '1014ee699071aa2ddd501907d18cbe15399c997d ruby-2.3.3.tar.gz' | shasum -c - && tar xzf ruby-2.3.3.tar.gz +cd ruby-2.3.3 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets and +it has a minimum requirement of node v4.3.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v4.3.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-0-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-0-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell'. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.0 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/9-5-stable:config/gitlab.yml.example origin/10-0-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/9-5-stable:lib/support/nginx/gitlab-ssl origin/10-0-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/9-5-stable:lib/support/nginx/gitlab origin/10-0-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-0-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-0-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/9-5-stable:lib/support/init.d/gitlab.default.example origin/10-0-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (9.5) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 9.4 to 9.5](9.4-to-9.5.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-0-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-0-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/update/mysql_to_postgresql.md b/doc/update/mysql_to_postgresql.md index a7de5648c0e..5dc8e6f65f8 100644 --- a/doc/update/mysql_to_postgresql.md +++ b/doc/update/mysql_to_postgresql.md @@ -1,3 +1,5 @@ +*** NOTE: These instructions should be considered deprecated. In GitLab 10.0 we will be releasing new migration instructions using [pgloader](http://pgloader.io/). + # Migrating GitLab from MySQL to Postgres *Make sure you view this [guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/update/mysql_to_postgresql.md#migrating-gitlab-from-mysql-to-postgres) for the most up to date instructions.* diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 30107360446..b2679d1ff22 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -74,7 +74,15 @@ cd /home/git/gitlab sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production ``` -### 5. Update gitlab-shell to the corresponding version +### 5. Update gitaly to the corresponding version + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake "gitlab:gitaly:install[/home/git/gitaly]" RAILS_ENV=production +``` + +### 6. Update gitlab-shell to the corresponding version ```bash cd /home/git/gitlab-shell @@ -84,14 +92,14 @@ sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` -b v`ca sudo -u git -H sh -c 'if [ -x bin/compile ]; then bin/compile; fi' ``` -### 6. Start application +### 7. Start application ```bash sudo service gitlab start sudo service nginx restart ``` -### 7. Check application status +### 8. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/user/admin_area/monitoring/convdev.md b/doc/user/admin_area/monitoring/convdev.md index 3d93c7557a4..a98602c4d70 100644 --- a/doc/user/admin_area/monitoring/convdev.md +++ b/doc/user/admin_area/monitoring/convdev.md @@ -23,7 +23,7 @@ If you have just started using GitLab, it may take a few weeks for data to be collected before this feature is available. This feature is accessible only to a system admin, at -**Admin area > Monitoring > ConvDev Index**. +**Admin area > Overview > ConvDev Index**. [ce-30469]: https://gitlab.com/gitlab-org/gitlab-ce/issues/30469 [ping]: ../settings/usage_statistics.md#usage-ping diff --git a/doc/user/admin_area/monitoring/img/convdev_index.png b/doc/user/admin_area/monitoring/img/convdev_index.png Binary files differindex 4e47ff2228d..ffe18d76c96 100644 --- a/doc/user/admin_area/monitoring/img/convdev_index.png +++ b/doc/user/admin_area/monitoring/img/convdev_index.png diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md index f2939f047a3..d4b5911a91c 100644 --- a/doc/user/project/integrations/prometheus_library/haproxy.md +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -7,7 +7,7 @@ GitLab has support for automatically detecting and monitoring HAProxy. This is p | Name | Query | | ---- | ----- | -| Throughput (req/sec) | sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | +| Throughput (req/sec) | sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) by (code) | | HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_requests_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | ## Configuring Prometheus to monitor for HAProxy metrics diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 9f0308d8111..4d39ae0c4fa 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -8,7 +8,7 @@ GitLab has support for automatically detecting and monitoring Kubernetes metrics | Name | Query | | ---- | ----- | | Average Memory Usage (MB) | (sum(container_memory_usage_bytes{container_name!="POD",%{environment_filter}}) / count(container_memory_usage_bytes{container_name!="POD",%{environment_filter}})) /1024/1024 | -| Average CPU Utilization (%) | sum(rate(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}[2m])) / count(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}) * 100 | +| Average CPU Utilization (%) | sum(rate(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}[2m])) by (cpu) * 100 | ## Configuring Prometheus to monitor for Kubernetes node metrics diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index 12e3321f5f3..bab22f9a384 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -7,7 +7,7 @@ GitLab has support for automatically detecting and monitoring NGINX. This is pro | Name | Query | | ---- | ----- | -| Throughput (req/sec) | sum(rate(nginx_requests_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) | +| Throughput (req/sec) | sum(rate(nginx_responses_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) by (status_code) | | Latency (ms) | avg(nginx_upstream_response_msecs_avg{%{environment_filter}}) | | HTTP Error Rate (HTTP Errors / sec) | rate(nginx_responses_total{status_code="5xx", %{environment_filter}}[2m])) | diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 84ee8bc45e5..17a47cfa646 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -7,19 +7,33 @@ GitLab has support for automatically detecting and monitoring the Kubernetes NGI | Name | Query | | ---- | ----- | -| Throughput (req/sec) | sum(rate(nginx_upstream_requests_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) | +| Throughput (req/sec) | sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code) | | Latency (ms) | avg(nginx_upstream_response_msecs_avg{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}) | | HTTP Error Rate (HTTP Errors / sec) | sum(rate(nginx_upstream_responses_total{status_code="5xx", upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) | -## Configuring Prometheus to monitor for NGINX ingress metrics +## Configuring NGINX ingress monitoring -The easiest way to get started is to use at least version 0.9.0 of [NGINX ingress](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). If you are using NGINX as your Kubernetes ingress, there is [direct support](https://github.com/kubernetes/ingress/pull/423) for enabling Prometheus monitoring in the 0.9.0 release. +If you have deployed with the [gitlab-omnibus](https://docs.gitlab.com/ee/install/kubernetes/gitlab_omnibus.md) Helm chart, and your application is running in the same cluster, no further action is required. The ingress metrics will be automatically enabled and annotated for Prometheus monitoring. Simply ensure Prometheus monitoring is [enabled for your project](../prometheus.md), which is on by default. -If you have deployed with the [gitlab-omnibus](https://docs.gitlab.com/ee/install/kubernetes/gitlab_omnibus.md) Helm chart, these metrics will be automatically enabled and annotated for Prometheus monitoring. +For other deployments, there is some configuration required depending on your installation: +* NGINX Ingress should be version 0.9.0 or above +* NGINX Ingress should be annotated for Prometheus monitoring +* Prometheus should be configured to monitor annotated pods + +### Setting up NGINX Ingress for Prometheus monitoring + +Version 0.9.0 and above of [NGINX ingress](https://github.com/kubernetes/ingress/tree/master/controllers/nginx) have built-in support for exporting Prometheus metrics. To enable, a ConfigMap setting must be passed: `enable-vts-status: "true"`. Once enabled, a Prometheus metrics endpoint will start running on port 10254. + +With metric data now available, Prometheus needs to be configured to collect it. The easiest way to do this is to leverage Prometheus' [built-in Kubernetes service discovery](https://prometheus.io/docs/operating/configuration/#kubernetes_sd_config), which automatically detects a variety of Kubernetes components and makes them available for monitoring. Since NGINX ingress metrics are exposed per pod, a scrape job for Kubernetes pods is required. A sample pod scraping configuration [is available](https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus-kubernetes.yml#L248). This configuration will detect pods and enable collection of metrics **only if** they have been specifically annotated for monitoring. + +Depending on how NGINX ingress was deployed, typically a DaemonSet or Deployment, edit the corresponding YML spec. Two new annotations need to be added: +* `prometheus.io/scrape: "true"` +* `prometheus.io/port: "10254"` + +Prometheus should now be collecting NGINX ingress metrics. To validate view the Prometheus Targets, available under `Status > Targets` on the Prometheus dashboard. New entries for NGINX should be listed in the kubernetes pod monitoring job, `kubernetes-pods`. ## Specifying the Environment label -In order to isolate and only display relevant metrics for a given environment -however, GitLab needs a method to detect which labels are associated. To do this, GitLab will search metrics with appropriate labels. In this case, the `upstream` label must be of the form `<Kubernetes Namespace>-<CI_ENVIRONMENT_SLUG>-*`. +In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do this, GitLab will search for metrics with appropriate labels. In this case, the `upstream` label must be of the form `<KUBE_NAMESPACE>-<CI_ENVIRONMENT_SLUG>-*`. If you have used [Auto Deploy](https://docs.gitlab.com/ee/ci/autodeploy/index.html) to deploy your app, this format will be used automatically and metrics will be detected with no action on your part. diff --git a/doc/user/project/issues/issues_functionalities.md b/doc/user/project/issues/issues_functionalities.md index 074b2c19c43..66140f389af 100644 --- a/doc/user/project/issues/issues_functionalities.md +++ b/doc/user/project/issues/issues_functionalities.md @@ -167,6 +167,7 @@ Once you wrote your comment, you can either: #### 18. New Merge Request - Create a new merge request (with a new source branch named after the issue) in one action. -The merge request will automatically close that issue as soon as merged. +The merge request will automatically inherit the milestone and labels of the issue. The merge +request will automatically close that issue as soon as merged. - Optionally, you can just create a [new branch](../repository/web_editor.md#create-a-new-branch-from-an-issue) named after that issue. diff --git a/doc/user/project/merge_requests/img/group_merge_requests_list_view.png b/doc/user/project/merge_requests/img/group_merge_requests_list_view.png Binary files differindex 02a88d0112f..7d0756505db 100644 --- a/doc/user/project/merge_requests/img/group_merge_requests_list_view.png +++ b/doc/user/project/merge_requests/img/group_merge_requests_list_view.png diff --git a/doc/user/project/repository/img/compare_branches.png b/doc/user/project/repository/img/compare_branches.png Binary files differindex 353bd72ef4e..d7ab587f030 100755..100644 --- a/doc/user/project/repository/img/compare_branches.png +++ b/doc/user/project/repository/img/compare_branches.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 21e96d8b11c..2b23c494dc4 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -31,8 +31,8 @@ on the search field on the top-right of your screen: If you want to search for issues present in a specific project, navigate to a project's **Issues** tab, and click on the field **Search or filter results...**. It will -display a dropdown menu, from which you can add filters per author, assignee, milestone, label, -and weight. When done, press **Enter** on your keyboard to filter the issues. +display a dropdown menu, from which you can add filters per author, assignee, milestone, +label, weight, and 'my-reaction' (based on your emoji votes). When done, press **Enter** on your keyboard to filter the issues.  @@ -63,8 +63,6 @@ the same way as you do for projects.  The same process is valid for merge requests. Navigate to your project's **Merge Requests** tab. -The search and filter UI currently uses dropdowns. In a future release, the same -dynamic UI as above will be carried over here. ## Search history |
