diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/README.md | 1 | ||||
| -rw-r--r-- | doc/administration/auth/ldap.md | 2 | ||||
| -rw-r--r-- | doc/administration/monitoring/prometheus/gitlab_metrics.md | 16 | ||||
| -rw-r--r-- | doc/ci/yaml/README.md | 26 | ||||
| -rw-r--r-- | doc/development/background_migrations.md | 36 | ||||
| -rw-r--r-- | doc/development/sidekiq_style_guide.md | 7 | ||||
| -rw-r--r-- | doc/install/installation.md | 2 | ||||
| -rw-r--r-- | doc/integration/img/enable_trello_powerup.png | bin | 0 -> 17905 bytes | |||
| -rw-r--r-- | doc/integration/img/trello_card_with_gitlab_powerup.png | bin | 0 -> 18667 bytes | |||
| -rw-r--r-- | doc/integration/trello_power_up.md | 42 | ||||
| -rw-r--r-- | doc/user/project/integrations/img/webhook_testing.png | bin | 0 -> 191267 bytes | |||
| -rw-r--r-- | doc/user/project/integrations/prometheus_library/haproxy.md | 20 | ||||
| -rw-r--r-- | doc/user/project/integrations/prometheus_library/metrics.md | 1 | ||||
| -rw-r--r-- | doc/user/project/integrations/prometheus_library/nginx.md | 2 | ||||
| -rw-r--r-- | doc/user/project/integrations/webhooks.md | 9 |
15 files changed, 159 insertions, 5 deletions
diff --git a/doc/README.md b/doc/README.md index 1a7638b3d7e..2b3b0998fcc 100644 --- a/doc/README.md +++ b/doc/README.md @@ -115,6 +115,7 @@ Manage files and branches from the UI (user interface): - [Project Services](user/project/integrations/project_services.md): Integrate a project with external services, such as CI and chat. - [GitLab Integration](integration/README.md): Integrate with multiple third-party services with GitLab to allow external issue trackers and external authentication. +- [Trello Power-Up](integration/trello_power_up.md): Integrate with GitLab's Trello Power-Up ---- diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index c8987dea5e2..3449f9e15ce 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -228,7 +228,7 @@ Tip: If you want to limit access to the nested members of an Active Directory group you can use the following syntax: ``` -(memberOf:1.2.840.113556.1.4.1941=CN=My Group,DC=Example,DC=com) +(memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com) ``` Find more information about this "LDAP_MATCHING_RULE_IN_CHAIN" filter at diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 7072ab5d02a..6baae20d16a 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -20,7 +20,7 @@ it, the client IP needs to be [included in a whitelist][whitelist]. Currently the embedded Prometheus server is not automatically configured to collect metrics from this endpoint. We recommend setting up another Prometheus server, because the embedded server configuration is overwritten once every -[reconfigure of GitLab][reconfigure]. In the future this will not be required. +[reconfigure of GitLab][reconfigure]. In the future this will not be required. ## Metrics available @@ -46,6 +46,20 @@ In this experimental phase, only a few metrics are available: | redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | | user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | +## Metrics shared directory + +GitLab's Prometheus client requires a directory to store metrics data shared between multi-process services. +Those files are shared among all instances running under Unicorn server. +The directory needs to be accessible to all running Unicorn's processes otherwise +metrics will not function correctly. + +For best performance its advisable that this directory will be located in `tmpfs`. + +Its location is configured using environment variable `prometheus_multiproc_dir`. + +If GitLab is installed using Omnibus and `tmpfs` is available then metrics +directory will be automatically configured. + [← Back to the main Prometheus page](index.md) [29118]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29118 diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 724843a4d56..e12ef6e2685 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -395,6 +395,7 @@ job_name: | after_script | no | Override a set of commands that are executed after job | | environment | no | Defines a name of environment to which deployment is done by this job | | coverage | no | Define code coverage settings for a given job | +| retry | no | Define how many times a job can be auto-retried in case of a failure | ### script @@ -1129,9 +1130,33 @@ A simple example: ```yaml job1: + script: rspec coverage: '/Code coverage: \d+\.\d+/' ``` +### retry + +**Notes:** +- [Introduced][ce-3442] in GitLab 9.5. + +`retry` allows you to configure how many times a job is going to be retried in +case of a failure. + +When a job fails, and has `retry` configured it is going to be processed again +up to the amount of times specified by the `retry` keyword. + +If `retry` is set to 2, and a job succeeds in a second run (first retry), it won't be retried +again. `retry` value has to be a positive integer, equal or larger than 0, but +lower or equal to 2 (two retries maximum, three runs in total). + +A simple example: + +```yaml +test: + script: rspec + retry: 2 +``` + ## Git Strategy > Introduced in GitLab 8.9 as an experimental feature. May change or be removed @@ -1506,3 +1531,4 @@ CI with various languages. [variables]: ../variables/README.md [ce-7983]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7983 [ce-7447]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7447 +[ce-3442]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3442 diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index 72a34aa7de9..e67db9ff142 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -31,6 +31,18 @@ It's also possible for different migrations to be executed at the same time. This means that different background migrations should not migrate data in a way that would cause conflicts. +## Idempotence + +Background migrations are executed in a context of a Sidekiq process. +Usual Sidekiq rules apply, especially the rule that jobs should be small +and idempotent. + +See [Sidekiq best practices guidelines](https://github.com/mperham/sidekiq/wiki/Best-Practices) +for more details. + +Make sure that in case that your migration job is going to be retried data +integrity is guarateed. + ## How It Works Background migrations are simple classes that define a `perform` method. A @@ -212,3 +224,27 @@ end This migration will then process any jobs for the ExtractServicesUrl migration and continue once all jobs have been processed. Once done you can safely remove the `services.properties` column. + +## Testing + +It is required to write tests for background migrations' scheduling migration +(either a regular migration or a post deployment migration), background +migration itself and a cleanup migration. You can use the `:migration` RSpec +tag when testing a regular / post deployment migration. +See [README][migrations-readme]. + +When you do that, keep in mind that `before` and `after` RSpec hooks are going +to migrate you database down and up, which can result in other background +migrations being called. That means that using `spy` test doubles with +`have_received` is encouraged, instead of using regular test doubles, because +your expectations defined in a `it` block can conflict with what is being +called in RSpec hooks. See [gitlab-org/gitlab-ce#35351][issue-rspec-hooks] +for more details. + +## Best practices + +1. Make sure that background migration jobs are idempotent. +1. Make sure that tests you write are not false positives. + +[migrations-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md +[issue-rspec-hooks]: https://gitlab.com/gitlab-org/gitlab-ce/issues/35351 diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index e3a20f29a09..1e9fdbc65e2 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -36,3 +36,10 @@ slow jobs blocking work (even for different jobs) on the shared queue. Each Sidekiq worker must be tested using RSpec, just like any other class. These tests should be placed in `spec/workers`. + +## Removing or renaming queues + +Try to avoid renaming or removing queues in minor and patch releases. +During online update instance can have pending jobs and removing the queue can +lead to those jobs being stuck forever. If you can't write migration for those +Sidekiq jobs, please consider doing rename or remove queue in major release only.
\ No newline at end of file diff --git a/doc/install/installation.md b/doc/install/installation.md index 5e981b0b3e7..8ded607bcab 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -64,7 +64,7 @@ up-to-date and install it. Install the required packages (needed to compile Ruby and native extensions to Ruby gems): - sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake + sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake If you want to use Kerberos for user authentication, then install libkrb5-dev: diff --git a/doc/integration/img/enable_trello_powerup.png b/doc/integration/img/enable_trello_powerup.png Binary files differnew file mode 100644 index 00000000000..65d01f1c38c --- /dev/null +++ b/doc/integration/img/enable_trello_powerup.png diff --git a/doc/integration/img/trello_card_with_gitlab_powerup.png b/doc/integration/img/trello_card_with_gitlab_powerup.png Binary files differnew file mode 100644 index 00000000000..2965dc35855 --- /dev/null +++ b/doc/integration/img/trello_card_with_gitlab_powerup.png diff --git a/doc/integration/trello_power_up.md b/doc/integration/trello_power_up.md new file mode 100644 index 00000000000..d264486a872 --- /dev/null +++ b/doc/integration/trello_power_up.md @@ -0,0 +1,42 @@ +# Trello Power-Up + +GitLab's Trello Power-Up enables you to seamlessly attach +GitLab **merge requests** to Trello cards. + + + +## Configuring the Power-Up + +In order to get started, you will need to configure your Power-Up. + +In Trello: + +1. Go to your Trello board +1. Select `Power-Ups` to see a listing of all the available Power-Ups +1. Look for a row that says `GitLab` and select the `Enable` button +1. Select the `Settings` (gear) icon +1. In the popup menu, select `Authorize Account` + +In this popup, fill in your `API URL` and `Personal Access Token`. After that, you will be able to attach any merge request to any Trello card on your selected Trello board. + +## What is my API URL? + +Your API URL should be your GitLab instance URL with `/api/v4` appended in the end of the URL. +For example, if your GitLab instance URL is `https://gitlab.com`, your API URL would be `https://gitlab.com/api/v4`. +If your instance's URL is `https://example.com`, your API URL will be `https://example.com/api/v4`. + + + +## What is my Personal Access Token? + +Your GitLab's personal access token will enable your GitLab account to be accessed +from Trello. + +> Find it in GitLab by clicking on your avatar (upright corner), from which you access +your user **Settings** > **Access Tokens**. + +Learn more about generating a personal access token in the +[Personal Access Token Documentation][personal-access-token-documentation]. +Don't forget to check the API scope checkbox! + +[personal-access-token-documentation]: ../user/profile/personal_access_tokens.html diff --git a/doc/user/project/integrations/img/webhook_testing.png b/doc/user/project/integrations/img/webhook_testing.png Binary files differnew file mode 100644 index 00000000000..176dcec9d8a --- /dev/null +++ b/doc/user/project/integrations/img/webhook_testing.png diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md new file mode 100644 index 00000000000..309da610cc0 --- /dev/null +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -0,0 +1,20 @@ +# Monitoring HA Proxy +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 + +GitLab has support for automatically detecting and monitoring HA Proxy. This is provided by leveraging the [HA Proxy Exporter](https://github.com/hnlq715/nginx-vts-exporter), which translates HA Proxy statistics into a Prometheus readable form. + +## Metrics supported + +| Name | Query | +| ---- | ----- | +| Throughput (req/sec) | sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | +| HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_requests_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | + +## Configuring Prometheus to monitor for HA Proxy metrics + +To get started with NGINX monitoring, you should install and configure the [HA Proxy exporter](https://github.com/prometheus/haproxy_exporter) which parses these statistics and translates them into a Prometheus monitoring endpoint. + +## Specifying the Environment label + +In order to isolate and only display relevant metrics for a given environment +however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). diff --git a/doc/user/project/integrations/prometheus_library/metrics.md b/doc/user/project/integrations/prometheus_library/metrics.md index 55146e57370..546e1f51df5 100644 --- a/doc/user/project/integrations/prometheus_library/metrics.md +++ b/doc/user/project/integrations/prometheus_library/metrics.md @@ -4,6 +4,7 @@ GitLab offers automatic detection of select [Prometheus exporters](https://prometheus.io/docs/instrumenting/exporters/). Currently supported exporters are: * [Kubernetes](kubernetes.md) * [NGINX](nginx.md) +* [HA Proxy](haproxy.md) * [Amazon Cloud Watch](cloudwatch.md) We have tried to surface the most important metrics for each exporter, and will be continuing to add support for additional exporters in future releases. If you would like to add support for other official exporters, [contributions](#adding-to-the-library) are welcome. diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index fe238e74e36..b3470773996 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -9,7 +9,7 @@ GitLab has support for automatically detecting and monitoring NGINX. This is pro | ---- | ----- | | Throughput (req/sec) | sum(rate(nginx_requests_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) | | Latency (ms) | avg(nginx_upstream_response_msecs_avg{%{environment_filter}}) * 1000 | -| HTTP Error Rate (%) | sum(nginx_responses_total{status_code="5xx", %{environment_filter}}) / sum(nginx_responses_total{server_zone!="*", server_zone!="_", %{environment_filter}}) | +| HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_responses_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_responses_total{%{environment_filter}}[2m])) | ## Configuring Prometheus to monitor for NGINX metrics diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 023c6932e41..c03a2df9a72 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1014,6 +1014,13 @@ X-Gitlab-Event: Build Hook } ``` +## Testing webhooks + +You can trigger the webhook manually. Sample data from the project will be used.Sample data will take from the project. +> For example: for triggering `Push Events` your project should have at least one commit. + + + ## Troubleshoot webhooks Gitlab stores each perform of the webhook. @@ -1056,7 +1063,7 @@ Pick an unused port (e.g. 8000) and start the script: `ruby print_http_body.rb 8000`. Then add your server as a webhook receiver in GitLab as `http://my.host:8000/`. -When you press 'Test Hook' in GitLab, you should see something like this in the +When you press 'Test' in GitLab, you should see something like this in the console: ``` |
