diff options
| author | Lin Jen-Shin <godfat@godfat.org> | 2016-12-20 20:54:40 +0800 |
|---|---|---|
| committer | Lin Jen-Shin <godfat@godfat.org> | 2016-12-20 20:54:40 +0800 |
| commit | 0f0738e78867f6822dd15cb26da1f17628acde77 (patch) | |
| tree | 0813c71b448d149afad6802e25c56a219efb0523 /doc | |
| parent | eb839b9af51d411a6a35786a1c1c58954da1a650 (diff) | |
| parent | ad1a1d976c877eca16858368db0c5b3ef800db8b (diff) | |
| download | gitlab-ce-0f0738e78867f6822dd15cb26da1f17628acde77.tar.gz | |
Merge remote-tracking branch 'upstream/master' into feature/1376-allow-write-access-deploy-keys
* upstream/master: (538 commits)
Reject blank environment vcariables in Gitlab::Git::RevList
Add online terminal documentation
Add changelog entry
Add terminal UI and controller actions
Fix specs
Even out padding on plus button in breadcrumb menu
Update font size of detail page header to 14px
Update CHANGELOG.md for 8.13.10
Update CHANGELOG.md for 8.14.5
Fix Route#rename_children behavior
Remove inline-block styling from status
Add terminals to the Kubernetes deployment service
Add a ReactiveCaching concern for use in the KubernetesService
Add xterm.js 2.1.0 and a wrapper class to the asset pipeline
Remove unnecessary hidden svg elements for icons.
Fix consistent typo in environment.js
Use a block to insert extra check for authenticate_build!
Align milestone column header with count number
Add Wiki import to BB importer
Make CI badge hitboxes better match container
...
Diffstat (limited to 'doc')
63 files changed, 1129 insertions, 259 deletions
diff --git a/doc/README.md b/doc/README.md index eba1e9845b1..8bf33cad5e4 100644 --- a/doc/README.md +++ b/doc/README.md @@ -8,7 +8,7 @@ - [GitLab as OAuth2 authentication service provider](integration/oauth_provider.md). It allows you to login to other applications from GitLab. - [Container Registry](user/project/container_registry.md) Learn how to use GitLab Container Registry. - [GitLab basics](gitlab-basics/README.md) Find step by step how to start working on your commandline and on GitLab. -- [Importing to GitLab](workflow/importing/README.md). +- [Importing to GitLab](workflow/importing/README.md) Import your projects from GitHub, Bitbucket, GitLab.com, FogBugz and SVN into GitLab. - [Importing and exporting projects between instances](user/project/settings/import_export.md). - [Markdown](user/markdown.md) GitLab's advanced formatting system. - [Migrating from SVN](workflow/importing/migrating_from_svn.md) Convert a SVN repository to Git and GitLab. @@ -34,6 +34,7 @@ - [Integration](integration/README.md) How to integrate with systems such as JIRA, Redmine, Twitter. - [Issue closing pattern](administration/issue_closing_pattern.md) Customize how to close an issue from commit messages. - [Koding](administration/integration/koding.md) Set up Koding to use with GitLab. +- [Online terminals](administration/integration/terminal.md) Provide terminal access to environments from within GitLab. - [Libravatar](customization/libravatar.md) Use Libravatar instead of Gravatar for user avatars. - [Log system](administration/logs.md) Log system. - [Environment Variables](administration/environment_variables.md) to configure GitLab. diff --git a/doc/administration/auth/README.md b/doc/administration/auth/README.md index 07e548aaabe..2fc5d0355b5 100644 --- a/doc/administration/auth/README.md +++ b/doc/administration/auth/README.md @@ -7,5 +7,6 @@ providers. and 389 Server - [OmniAuth](../../integration/omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google, Bitbucket, Facebook, Shibboleth, Crowd and Azure -- [SAML](../../integration/saml.md) Configure GitLab as a SAML 2.0 Service Provider - [CAS](../../integration/cas.md) Configure GitLab to sign in using CAS +- [SAML](../../integration/saml.md) Configure GitLab as a SAML 2.0 Service Provider +- [Okta](okta.md) Configure GitLab to sign in using Okta diff --git a/doc/administration/auth/img/okta_admin_panel.png b/doc/administration/auth/img/okta_admin_panel.png Binary files differnew file mode 100644 index 00000000000..12e21956715 --- /dev/null +++ b/doc/administration/auth/img/okta_admin_panel.png diff --git a/doc/administration/auth/img/okta_saml_settings.png b/doc/administration/auth/img/okta_saml_settings.png Binary files differnew file mode 100644 index 00000000000..ee275ece369 --- /dev/null +++ b/doc/administration/auth/img/okta_saml_settings.png diff --git a/doc/administration/auth/okta.md b/doc/administration/auth/okta.md new file mode 100644 index 00000000000..cb42b7743c5 --- /dev/null +++ b/doc/administration/auth/okta.md @@ -0,0 +1,160 @@ +# Okta SSO provider + +Okta is a [Single Sign-on provider][okta-sso] that can be used to authenticate +with GitLab. + +The following documentation enables Okta as a SAML provider. + +## Configure the Okta application + +1. On Okta go to the admin section and choose to **Add an App**. +1. When the app screen comes up you see another button to **Create an App** and + choose SAML 2.0 on the next screen. +1. Now, very important, add a logo + (you can choose it from https://about.gitlab.com/press/). You'll have to + crop and resize it. +1. Next, you'll need the to fill in the SAML general config. Here's an example + image. + +  + +1. The last part of the configuration is the feedback section where you can + just say you're a customer and creating an app for internal use. +1. When you have your app you'll have a few tabs on the top of the app's + profile. Click on the SAML 2.0 config instructions button which should + look like the following: + +  + +1. On the screen that comes up take note of the + **Identity Provider Single Sign-On URL** which you'll use for the + `idp_sso_target_url` on your GitLab config file. + +1. **Before you leave Okta make sure you add your user and groups if any.** + +--- + +Now that the Okta app is configured, it's time to enable it in GitLab. + +## Configure GitLab + +1. On your GitLab server, open the configuration file: + + **For Omnibus GitLab installations** + + ```sh + sudo editor /etc/gitlab/gitlab.rb + ``` + + **For installations from source** + + ```sh + cd /home/git/gitlab + sudo -u git -H editor config/gitlab.yml + ``` + +1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) + for initial settings. + +1. To allow your users to use Okta to sign up without having to manually create + an account first, don't forget to add the following values to your + configuration: + + **For Omnibus GitLab installations** + + ```ruby + gitlab_rails['omniauth_allow_single_sign_on'] = ['saml'] + gitlab_rails['omniauth_block_auto_created_users'] = false + ``` + + **For installations from source** + + ```yaml + allow_single_sign_on: ["saml"] + block_auto_created_users: false + ``` + +1. You can also automatically link Okta users with existing GitLab users if + their email addresses match by adding the following setting: + + **For Omnibus GitLab installations** + + ```ruby + gitlab_rails['omniauth_auto_link_saml_user'] = true + ``` + + **For installations from source** + + ```yaml + auto_link_saml_user: true + ``` + +1. Add the provider configuration. + + >**Notes:** + >- Change the value for `assertion_consumer_service_url` to match the HTTPS endpoint + of GitLab (append `users/auth/saml/callback` to the HTTPS URL of your GitLab + installation to generate the correct value). + >- To get the `idp_cert_fingerprint` fingerprint, first download the + certificate from the Okta app you registered and then run: + `openssl x509 -in okta.cert -noout -fingerprint`. Substitute `okta.cert` + with the location of your certificate. + >- Change the value of `idp_sso_target_url`, with the value of the + **Identity Provider Single Sign-On URL** from the step when you + configured the Okta app. + >- Change the value of `issuer` to a unique name, which will identify the application + to the IdP. + >- Leave `name_identifier_format` as-is. + + **For Omnibus GitLab installations** + + ```ruby + gitlab_rails['omniauth_providers'] = [ + { + name: 'saml', + args: { + assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback', + idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8', + idp_sso_target_url: 'https://gitlab.oktapreview.com/app/gitlabdev773716_gitlabsaml_1/exk8odl81tBrjpD4B0h7/sso/saml', + issuer: 'https://gitlab.example.com', + name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient' + }, + label: 'Okta' # optional label for SAML login button, defaults to "Saml" + } + ] + ``` + + **For installations from source** + + ```yaml + - { + name: 'saml', + args: { + assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback', + idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8', + idp_sso_target_url: 'https://gitlab.oktapreview.com/app/gitlabdev773716_gitlabsaml_1/exk8odl81tBrjpD4B0h7/sso/saml', + issuer: 'https://gitlab.example.com', + name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient' + }, + label: 'Okta' # optional label for SAML login button, defaults to "Saml" + } + ``` + + +1. [Reconfigure][reconf] or [restart] GitLab for Omnibus and installations + from source respectively for the changes to take effect. + +You might want to try this out on a incognito browser window. + +## Configuring groups + +>**Note:** +Make sure the groups exist and are assigned to the Okta app. + +You can take a look of the [SAML documentation][saml] on external groups since +it works the same. + +[okta-sso]: https://www.okta.com/products/single-sign-on/ +[saml]: ../../integration/saml.md#external-groups +[reconf]: ../restart_gitlab.md#omnibus-gitlab-reconfigure +[restart]: ../restart_gitlab.md#installations-from-source diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 06291705702..80e5d80aa41 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -44,22 +44,30 @@ as appropriate. ## Chained hooks support -> [Introduced][93] in GitLab Shell 4.1.0. +> [Introduced][93] in GitLab Shell 4.1.0 and GitLab 8.15. -The hooks could be also placed in `hooks/<hook_name>.d` (global) or `custom_hooks/<hook_name>.d` (per project) -directories supporting chained execution of the hooks. +Hooks can be also placed in `hooks/<hook_name>.d` (global) or +`custom_hooks/<hook_name>.d` (per project) directories supporting chained +execution of the hooks. + +To look in a different directory for the global custom hooks (those in +`hooks/<hook_name.d>`), set `custom_hooks_dir` in gitlab-shell config. For +Omnibus installations, this can be set in `gitlab.rb`; and in source +installations, this can be set in `gitlab-shell/config.yml`. The hooks are searched and executed in this order: + 1. `<project>.git/hooks/` - symlink to `gitlab-shell/hooks` global dir 1. `<project>.git/hooks/<hook_name>` - executed by `git` itself, this is `gitlab-shell/hooks/<hook_name>` 1. `<project>.git/custom_hooks/<hook_name>` - per project hook (this is already existing behavior) 1. `<project>.git/custom_hooks/<hook_name>.d/*` - per project hooks -1. `<project>.git/hooks/<hook_name>.d/*` - global hooks: all executable files (minus editor backup files) +1. `<project>.git/hooks/<hook_name>.d/*` OR `<custom_hooks_dir>/<hook_name.d>/*` - global hooks: all executable files (minus editor backup files) -Files in `.d` directories need to be executable and not match the backup file pattern (`*~`). +Files in `.d` directories need to be executable and not match the backup file +pattern (`*~`). -The hooks of the same type are executed in order and execution stops on the first -script exiting with non-zero value. +The hooks of the same type are executed in order and execution stops on the +first script exiting with a non-zero value. ## Custom error messages diff --git a/doc/administration/environment_variables.md b/doc/administration/environment_variables.md index b4a953d1ccc..76029b30dd8 100644 --- a/doc/administration/environment_variables.md +++ b/doc/administration/environment_variables.md @@ -13,17 +13,18 @@ override certain values. Variable | Type | Description -------- | ---- | ----------- -`GITLAB_ROOT_PASSWORD` | string | Sets the password for the `root` user on installation -`GITLAB_HOST` | string | The full URL of the GitLab server (including `http://` or `https://`) -`RAILS_ENV` | string | The Rails environment; can be one of `production`, `development`, `staging` or `test` -`DATABASE_URL` | string | The database URL; is of the form: `postgresql://localhost/blog_development` -`GITLAB_EMAIL_FROM` | string | The e-mail address used in the "From" field in e-mails sent by GitLab -`GITLAB_EMAIL_DISPLAY_NAME` | string | The name used in the "From" field in e-mails sent by GitLab -`GITLAB_EMAIL_REPLY_TO` | string | The e-mail address used in the "Reply-To" field in e-mails sent by GitLab -`GITLAB_EMAIL_REPLY_TO` | string | The e-mail address used in the "Reply-To" field in e-mails sent by GitLab -`GITLAB_EMAIL_SUBJECT_SUFFIX` | string | The e-mail subject suffix used in e-mails sent by GitLab -`GITLAB_UNICORN_MEMORY_MIN` | integer | The minimum memory threshold (in bytes) for the Unicorn worker killer -`GITLAB_UNICORN_MEMORY_MAX` | integer | The maximum memory threshold (in bytes) for the Unicorn worker killer +`GITLAB_ROOT_PASSWORD` | string | Sets the password for the `root` user on installation +`GITLAB_HOST` | string | The full URL of the GitLab server (including `http://` or `https://`) +`RAILS_ENV` | string | The Rails environment; can be one of `production`, `development`, `staging` or `test` +`DATABASE_URL` | string | The database URL; is of the form: `postgresql://localhost/blog_development` +`GITLAB_EMAIL_FROM` | string | The e-mail address used in the "From" field in e-mails sent by GitLab +`GITLAB_EMAIL_DISPLAY_NAME` | string | The name used in the "From" field in e-mails sent by GitLab +`GITLAB_EMAIL_REPLY_TO` | string | The e-mail address used in the "Reply-To" field in e-mails sent by GitLab +`GITLAB_EMAIL_REPLY_TO` | string | The e-mail address used in the "Reply-To" field in e-mails sent by GitLab +`GITLAB_EMAIL_SUBJECT_SUFFIX` | string | The e-mail subject suffix used in e-mails sent by GitLab +`GITLAB_UNICORN_MEMORY_MIN` | integer | The minimum memory threshold (in bytes) for the Unicorn worker killer +`GITLAB_UNICORN_MEMORY_MAX` | integer | The maximum memory threshold (in bytes) for the Unicorn worker killer +`GITLAB_SHARED_RUNNERS_REGISTRATION_TOKEN` | string | Sets the initial registration token used for GitLab Runners ## Complete database variables diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index b36cf18d459..e4f94eb7cb6 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -44,6 +44,9 @@ If you use a cloud-managed service, or provide your own PostgreSQL: gitlab_rails['db_password'] = 'DB password' postgresql['md5_auth_cidr_addresses'] = ['0.0.0.0/0'] postgresql['listen_address'] = '0.0.0.0' + + # Disable automatic database migrations + gitlab_rails['auto_migrate'] = false ``` 1. Run `sudo gitlab-ctl reconfigure` to install and configure PostgreSQL. @@ -102,9 +105,6 @@ If you use a cloud-managed service, or provide your own PostgreSQL: 1. Exit the database prompt by typing `\q` and Enter. 1. Exit the `gitlab-psql` user by running `exit` twice. 1. Run `sudo gitlab-ctl reconfigure` a final time. -1. Run `sudo touch /etc/gitlab/skip-auto-migrations` to prevent database migrations - from running on upgrade. Only the primary GitLab application server should - handle migrations. --- diff --git a/doc/administration/high_availability/load_balancer.md b/doc/administration/high_availability/load_balancer.md index 136f570ac27..e61ea359a6a 100644 --- a/doc/administration/high_availability/load_balancer.md +++ b/doc/administration/high_availability/load_balancer.md @@ -10,11 +10,11 @@ you need to use with GitLab. ## Basic ports -| LB Port | Backend Port | Protocol | -| ------- | ------------ | -------- | -| 80 | 80 | HTTP | -| 443 | 443 | HTTPS [^1] | -| 22 | 22 | TCP | +| LB Port | Backend Port | Protocol | +| ------- | ------------ | --------------- | +| 80 | 80 | HTTP [^1] | +| 443 | 443 | HTTPS [^1] [^2] | +| 22 | 22 | TCP | ## GitLab Pages Ports @@ -25,8 +25,8 @@ GitLab Pages requires a separate VIP. Configure DNS to point the | LB Port | Backend Port | Protocol | | ------- | ------------ | -------- | -| 80 | Varies [^2] | HTTP | -| 443 | Varies [^2] | TCP [^3] | +| 80 | Varies [^3] | HTTP | +| 443 | Varies [^3] | TCP [^4] | ## Alternate SSH Port @@ -50,13 +50,19 @@ Read more on high-availability configuration: 1. [Configure NFS](nfs.md) 1. [Configure the GitLab application servers](gitlab.md) -[^1]: When using HTTPS protocol for port 443, you will need to add an SSL +[^1]: [Terminal support](../../ci/environments.md#terminal-support) requires + your load balancer to correctly handle WebSocket connections. When using + HTTP or HTTPS proxying, this means your load balancer must be configured + to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the + [online terminal](../integration/terminal.md) integration guide for + more details. +[^2]: When using HTTPS protocol for port 443, you will need to add an SSL certificate to the load balancers. If you wish to terminate SSL at the GitLab application server instead, use TCP protocol. -[^2]: The backend port for GitLab Pages depends on the +[^3]: The backend port for GitLab Pages depends on the `gitlab_pages['external_http']` and `gitlab_pages['external_https']` setting. See [GitLab Pages documentation][gitlab-pages] for more details. -[^3]: Port 443 for GitLab Pages should always use the TCP protocol. Users can +[^4]: Port 443 for GitLab Pages should always use the TCP protocol. Users can configure custom domains with custom SSL, which would not be possible if SSL was terminated at the load balancer. diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index f532a106bc6..b4e7bf21e35 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -287,14 +287,14 @@ The prerequisites for a HA Redis setup are the following: redis['password'] = 'redis-password-goes-here' ``` -1. To prevent database migrations from running on upgrade, run: +1. Only the primary GitLab application server should handle migrations. To + prevent database migrations from running on upgrade, add the following + configuration to your `/etc/gitlab/gitlab.rb` file: ``` - sudo touch /etc/gitlab/skip-auto-migrations + gitlab_rails['auto_migrate'] = false ``` - Only the primary GitLab application server should handle migrations. - 1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. ### Step 2. Configuring the slave Redis instances diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md new file mode 100644 index 00000000000..05d0a97e554 --- /dev/null +++ b/doc/administration/integration/terminal.md @@ -0,0 +1,73 @@ +# Online terminals + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7690) +in GitLab 8.15. Only project masters and owners can access online terminals. + +With the introduction of the [Kubernetes](../../project_services/kubernetes.md) +project service, GitLab gained the ability to store and use credentials for a +Kubernetes cluster. One of the things it uses these credentials for is providing +access to [online terminals](../../ci/environments.html#online-terminals) +for environments. + +## How it works + +A detailed overview of the architecture of online terminals and how they work +can be found in [this document](https://gitlab.com/gitlab-org/gitlab-workhorse/blob/master/doc/terminal.md). +In brief: + +* GitLab relies on the user to provide their own Kubernetes credentials, and to + appropriately label the pods they create when deploying. +* When a user navigates to the terminal page for an environment, they are served + a JavaScript application that opens a WebSocket connection back to GitLab. +* The WebSocket is handled in [Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse), + rather than the Rails application server. +* Workhorse queries Rails for connection details and user permissions; Rails + queries Kubernetes for them in the background, using [Sidekiq](../troubleshooting/sidekiq.md) +* Workhorse acts as a proxy server between the user's browser and the Kubernetes + API, passing WebSocket frames between the two. +* Workhorse regularly polls Rails, terminating the WebSocket connection if the + user no longer has permission to access the terminal, or if the connection + details have changed. + +## Enabling and disabling terminal support + +As online terminals use WebSockets, every HTTP/HTTPS reverse proxy in front of +Workhorse needs to be configured to pass the `Connection` and `Upgrade` headers +through to the next one in the chain. If you installed Gitlab using Omnibus, or +from source, starting with GitLab 8.15, this should be done by the default +configuration, so there's no need for you to do anything. + +However, if you run a [load balancer](../high_availability/load_balancer.md) in +front of GitLab, you may need to make some changes to your configuration. These +guides document the necessary steps for a selection of popular reverse proxies: + +* [Apache](https://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html) +* [NGINX](https://www.nginx.com/blog/websocket-nginx/) +* [HAProxy](http://blog.haproxy.com/2012/11/07/websockets-load-balancing-with-haproxy/) +* [Varnish](https://www.varnish-cache.org/docs/4.1/users-guide/vcl-example-websockets.html) + +Workhorse won't let WebSocket requests through to non-WebSocket endpoints, so +it's safe to enable support for these headers globally. If you'd rather had a +narrower set of rules, you can restrict it to URLs ending with `/terminal.ws` +(although this may still have a few false positives). + +If you installed from source, or have made any configuration changes to your +Omnibus installation before upgrading to 8.15, you may need to make some +changes to your configuration. See the [8.14 to 8.15 upgrade](../../update/8.14-to-8.15.md#nginx-configuration) +document for more details. + +If you'd like to disable online terminal support in GitLab, just stop passing +the `Connection` and `Upgrade` hop-by-hop headers in the *first* HTTP reverse +proxy in the chain. For most users, this will be the NGINX server bundled with +Omnibus Gitlab, in which case, you need to: + +* Find the `nginx['proxy_set_headers']` section of your `gitlab.rb` file +* Ensure the whole block is uncommented, and then comment out or remove the + `Connection` and `Upgrade` lines. + +For your own load balancer, just reverse the configuration changes recommended +by the above guides. + +When these headers are not passed through, Workhorse will return a +`400 Bad Request` response to users attempting to use an online terminal. In +turn, they will receive a `Connection failed` message. diff --git a/doc/administration/logs.md b/doc/administration/logs.md index d757a3c2a66..4b8d5c5cc87 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -1,4 +1,4 @@ -## Log system +# Log system GitLab has an advanced log system where everything is logged so that you can analyze your instance using various system log files. In addition to @@ -9,10 +9,10 @@ documentation](http://docs.gitlab.com/ee/administration/audit_events.html) System log files are typically plain text in a standard log file format. This guide talks about how to read and use these system log files. -### production.log +## `production.log` This file lives in `/var/log/gitlab/gitlab-rails/production.log` for -omnibus package or in `/home/git/gitlab/log/production.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/production.log` for installations from source. (When Gitlab is running in an environment other than production, the corresponding logfile is shown here.) @@ -46,10 +46,10 @@ In this example we can see that server processed an HTTP request with URL 19:34:53 +0200. Also we can see that request was processed by `Projects::TreeController`. -### application.log +## `application.log` This file lives in `/var/log/gitlab/gitlab-rails/application.log` for -omnibus package or in `/home/git/gitlab/log/application.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/application.log` for installations from source. It helps you discover events happening in your instance such as user creation, @@ -63,10 +63,10 @@ October 07, 2014 11:25: User "Claudie Hodkiewicz" (nasir_stehr@olson.co.uk) was October 07, 2014 11:25: Project "project133" was removed ``` -### githost.log +## `githost.log` This file lives in `/var/log/gitlab/gitlab-rails/githost.log` for -omnibus package or in `/home/git/gitlab/log/githost.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/githost.log` for installations from source. GitLab has to interact with Git repositories but in some rare cases @@ -81,10 +81,10 @@ December 03, 2014 13:20 -> ERROR -> Command failed [1]: /usr/bin/git --git-dir=/ error: failed to push some refs to '/Users/vsizov/gitlab-development-kit/repositories/gitlabhq/gitlab_git.git' ``` -### sidekiq.log +## `sidekiq.log` This file lives in `/var/log/gitlab/gitlab-rails/sidekiq.log` for -omnibus package or in `/home/git/gitlab/log/sidekiq.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/sidekiq.log` for installations from source. GitLab uses background jobs for processing tasks which can take a long @@ -96,10 +96,10 @@ this file. For example: 2014-06-10T18:18:26Z 14299 TID-55uqo INFO: Booting Sidekiq 3.0.0 with redis options {:url=>"redis://localhost:6379/0", :namespace=>"sidekiq"} ``` -### gitlab-shell.log +## `gitlab-shell.log` This file lives in `/var/log/gitlab/gitlab-shell/gitlab-shell.log` for -omnibus package or in `/home/git/gitlab-shell/gitlab-shell.log` for +Omnibus GitLab packages or in `/home/git/gitlab-shell/gitlab-shell.log` for installations from source. GitLab shell is used by Gitlab for executing Git commands and provide @@ -110,10 +110,10 @@ I, [2015-02-13T06:17:00.671315 #9291] INFO -- : Adding project root/example.git I, [2015-02-13T06:17:00.679433 #9291] INFO -- : Moving existing hooks directory and symlinking global hooks directory for /var/opt/gitlab/git-data/repositories/root/example.git. ``` -### unicorn\_stderr.log +## `unicorn\_stderr.log` This file lives in `/var/log/gitlab/unicorn/unicorn_stderr.log` for -omnibus package or in `/home/git/gitlab/log/unicorn_stderr.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/unicorn_stderr.log` for installations from source. Unicorn is a high-performance forking Web server which is used for @@ -136,3 +136,13 @@ I, [2015-02-13T07:16:01.530733 #9047] INFO -- : reaped #<Process::Status: pid 9 I, [2015-02-13T07:16:01.534501 #13379] INFO -- : worker=1 spawned pid=13379 I, [2015-02-13T07:16:01.534848 #13379] INFO -- : worker=1 ready ``` + +## `repocheck.log` + +This file lives in `/var/log/gitlab/gitlab-rails/repocheck.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/repocheck.log` for +installations from source. + +It logs information whenever a [repository check is run][repocheck] on a project. + +[repocheck]: repository_checks.md diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index d1d2fed4861..c8b5434c068 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -74,24 +74,5 @@ Example output: The LDAP check Rake task will test the bind_dn and password credentials (if configured) and will list a sample of LDAP users. This task is also -executed as part of the `gitlab:check` task, but can run independently -using the command below. - -**Omnibus Installation** - -``` -sudo gitlab-rake gitlab:ldap:check -``` - -**Source Installation** - -```bash -sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production -``` - -By default, the task will return a sample of 100 LDAP users. Change this -limit by passing a number to the check task: - -```bash -rake gitlab:ldap:check[50] -``` +executed as part of the `gitlab:check` task, but can run independently. +See [LDAP Rake Tasks - LDAP Check](ldap.md#check) for details. diff --git a/doc/administration/raketasks/ldap.md b/doc/administration/raketasks/ldap.md new file mode 100644 index 00000000000..91fc0133d56 --- /dev/null +++ b/doc/administration/raketasks/ldap.md @@ -0,0 +1,120 @@ +# LDAP Rake Tasks + +## Check + +The LDAP check Rake task will test the `bind_dn` and `password` credentials +(if configured) and will list a sample of LDAP users. This task is also +executed as part of the `gitlab:check` task, but can run independently +using the command below. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:ldap:check +``` + +**Source Installation** + +```bash +sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production +``` + +------ + +By default, the task will return a sample of 100 LDAP users. Change this +limit by passing a number to the check task: + +```bash +rake gitlab:ldap:check[50] +``` + +## Rename a provider + +If you change the LDAP server ID in `gitlab.yml` or `gitlab.rb` you will need +to update all user identities or users will be unable to sign in. Input the +old and new provider and this task will update all matching identities in the +database. + +`old_provider` and `new_provider` are derived from the prefix `ldap` plus the +LDAP server ID from the configuration file. For example, in `gitlab.yml` or +`gitlab.rb` you may see LDAP configuration like this: + +```yaml +main: + label: 'LDAP' + host: '_your_ldap_server' + port: 389 + uid: 'sAMAccountName' + ... +``` + +`main` is the LDAP server ID. Together, the unique provider is `ldapmain`. + +> **Warning**: If you input an incorrect new provider users will be unable +to sign in. If this happens, run the task again with the incorrect provider +as the `old_provider` and the correct provider as the `new_provider`. + +**Omnibus Installation** + +```bash +sudo gitlab-rake gitlab:ldap:rename_provider[old_provider,new_provider] +``` + +**Source Installation** + +```bash +bundle exec rake gitlab:ldap:rename_provider[old_provider,new_provider] RAILS_ENV=production +``` + +### Example + +Consider beginning with the default server ID `main` (full provider `ldapmain`). +If we change `main` to `mycompany`, the `new_provider` is `ldapmycompany`. +To rename all user identities run the following command: + +```bash +sudo gitlab-rake gitlab:ldap:rename_provider[ldapmain,ldapmycompany] +``` + +Example output: + +``` +100 users with provider 'ldapmain' will be updated to 'ldapmycompany'. +If the new provider is incorrect, users will be unable to sign in. +Do you want to continue (yes/no)? yes + +User identities were successfully updated +``` + +### Other options + +If you do not specify an `old_provider` and `new_provider` you will be prompted +for them: + +**Omnibus Installation** + +```bash +sudo gitlab-rake gitlab:ldap:rename_provider +``` + +**Source Installation** + +```bash +bundle exec rake gitlab:ldap:rename_provider RAILS_ENV=production +``` + +**Example output:** + +``` +What is the old provider? Ex. 'ldapmain': ldapmain +What is the new provider? Ex. 'ldapcustom': ldapmycompany +``` + +------ + +This tasks also accepts the `force` environment variable which will skip the +confirmation dialog: + +```bash +sudo gitlab-rake gitlab:ldap:rename_provider[old_provider,new_provider] force=yes +``` diff --git a/doc/api/commits.md b/doc/api/commits.md index 0170af00e0e..5c11d0f83bb 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -183,6 +183,44 @@ Example response: } ``` +## Cherry pick a commit + +> [Introduced][ce-8047] in GitLab 8.15. + +Cherry picks a commit to a given branch. + +``` +POST /projects/:id/repository/commits/:sha/cherry_pick +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME owned by the authenticated user +| `sha` | string | yes | The commit hash | +| `branch` | string | yes | The name of the branch | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "branch=master" "https://gitlab.example.com/api/v3/projects/5/repository/commits/master/cherry_pick" +``` + +Example response: + +```json +{ + "id": "8b090c1b79a14f2bd9e8a738f717824ff53aebad", + "short_id": "8b090c1b", + "title": "Feature added", + "author_name": "Dmitriy Zaporozhets", + "author_email": "dmitriy.zaporozhets@gmail.com", + "created_at": "2016-12-12T20:10:39.000+01:00", + "committer_name": "Administrator", + "committer_email": "admin@example.com", + "message": "Feature added\n\nSigned-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>\n" +} +``` + ## Get the diff of a commit Get the diff of a commit in a project. @@ -438,3 +476,4 @@ Example response: ``` [ce-6096]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6096 "Multi-file commit" +[ce-8047]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8047 diff --git a/doc/api/enviroments.md b/doc/api/enviroments.md index 87a5fa67124..1299aca8c45 100644 --- a/doc/api/enviroments.md +++ b/doc/api/enviroments.md @@ -22,8 +22,9 @@ Example response: [ { "id": 1, - "name": "Env1", - "external_url": "https://env1.example.gitlab.com" + "name": "review/fix-foo", + "slug": "review-fix-foo-dfjre3", + "external_url": "https://review-fix-foo-dfjre3.example.gitlab.com" } ] ``` @@ -54,6 +55,7 @@ Example response: { "id": 1, "name": "deploy", + "slug": "deploy", "external_url": "https://deploy.example.gitlab.com" } ``` @@ -85,6 +87,7 @@ Example response: { "id": 1, "name": "staging", + "slug": "staging", "external_url": "https://staging.example.gitlab.com" } ``` @@ -112,6 +115,7 @@ Example response: { "id": 1, "name": "deploy", + "slug": "deploy", "external_url": "https://deploy.example.gitlab.com" } ``` diff --git a/doc/api/groups.md b/doc/api/groups.md index 5e6f498c365..134d7bda22f 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -50,12 +50,17 @@ GET /groups/:id/projects Parameters: -- `archived` (optional) - if passed, limit by archived status -- `visibility` (optional) - if passed, limit by visibility `public`, `internal`, `private` -- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` -- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` -- `search` (optional) - Return list of authorized projects according to a search criteria -- `ci_enabled_first` - Return projects ordered by ci_enabled flag. Projects with enabled GitLab CI go first +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or path of a group | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of authorized projects matching the search criteria | +| `simple` | boolean | no | Return only the ID, URL, name, and path of each project | + +Example response: ```json [ diff --git a/doc/api/projects.md b/doc/api/projects.md index 0bc2a5210aa..edffad555a5 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -664,7 +664,6 @@ Parameters: | `path` | string | no | Custom repository name for new project. By default generated based on name | | `default_branch` | string | no | `master` by default | | `namespace_id` | integer | no | Namespace for the new project (defaults to the current user's namespace) | -| `default_branch` | string | no | `master` by default | | `description` | string | no | Short project description | | `issues_enabled` | boolean | no | Enable issues for this project | | `merge_requests_enabled` | boolean | no | Enable merge requests for this project | diff --git a/doc/api/services.md b/doc/api/services.md index a5d733fe6c7..1466b8189b0 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -139,6 +139,43 @@ Get Buildkite service settings for a project. GET /projects/:id/services/buildkite ``` +## Build-Emails + +Get emails for GitLab CI builds. + +### Create/Edit Build-Emails service + +Set Build-Emails service for a project. + +``` +PUT /projects/:id/services/builds-email +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recipients` | string | yes | Comma-separated list of recipient email addresses | +| `add_pusher` | boolean | no | Add pusher to recipients list | +| `notify_only_broken_builds` | boolean | no | Notify only broken builds | + + +### Delete Build-Emails service + +Delete Build-Emails service for a project. + +``` +DELETE /projects/:id/services/builds-email +``` + +### Get Build-Emails service settings + +Get Build-Emails service settings for a project. + +``` +GET /projects/:id/services/builds-email +``` + ## Campfire Simple web-based real-time group chat @@ -476,12 +513,11 @@ PUT /projects/:id/services/jira | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `active` | boolean| no | Enable/disable the JIRA service. | | `url` | string | yes | The URL to the JIRA project which is being linked to this GitLab project, e.g., `https://jira.example.com`. | | `project_key` | string | yes | The short identifier for your JIRA project, all uppercase, e.g., `PROJ`. | | `username` | string | no | The username of the user created to be used with GitLab/JIRA. | | `password` | string | no | The password of the user created to be used with GitLab/JIRA. | -| `jira_issue_transition_id` | string | no | The ID of a transition that moves issues to a closed state. You can find this number under the JIRA workflow administration (**Administration > Issues > Workflows**) by selecting **View** under **Operations** of the desired workflow of your project. The ID of each state can be found inside the parenthesis of each transition name under the **Transitions (id)** column ([see screenshot][trans]). By default, this ID is set to `2`. | +| `jira_issue_transition_id` | integer | no | The ID of a transition that moves issues to a closed state. You can find this number under the JIRA workflow administration (**Administration > Issues > Workflows**) by selecting **View** under **Operations** of the desired workflow of your project. The ID of each state can be found inside the parenthesis of each transition name under the **Transitions (id)** column ([see screenshot][trans]). By default, this ID is set to `2`. | ### Delete JIRA service @@ -491,6 +527,78 @@ Remove all previously JIRA settings from a project. DELETE /projects/:id/services/jira ``` +## Mattermost Slash Commands + +Ability to receive slash commands from a Mattermost chat instance. + +### Create/Edit Mattermost Slash Command service + +Set Mattermost Slash Command for a project. + +``` +PUT /projects/:id/services/mattermost-slash-commands +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `token` | string | yes | The Mattermost token | + + +### Delete Mattermost Slash Command service + +Delete Mattermost Slash Command service for a project. + +``` +DELETE /projects/:id/services/mattermost-slash-commands +``` + +### Get Mattermost Slash Command service settings + +Get Mattermost Slash Command service settings for a project. + +``` +GET /projects/:id/services/mattermost-slash-commands +``` + +## Pipeline-Emails + +Get emails for GitLab CI pipelines. + +### Create/Edit Pipeline-Emails service + +Set Pipeline-Emails service for a project. + +``` +PUT /projects/:id/services/pipelines-email +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recipients` | string | yes | Comma-separated list of recipient email addresses | +| `add_pusher` | boolean | no | Add pusher to recipients list | +| `notify_only_broken_builds` | boolean | no | Notify only broken pipelines | + + +### Delete Pipeline-Emails service + +Delete Pipeline-Emails service for a project. + +``` +DELETE /projects/:id/services/pipelines-email +``` + +### Get Pipeline-Emails service settings + +Get Pipeline-Emails service settings for a project. + +``` +GET /projects/:id/services/pipelines-email +``` + ## PivotalTracker Project Management Software (Source Commits Endpoint) @@ -595,9 +703,9 @@ Get Redmine service settings for a project. GET /projects/:id/services/redmine ``` -## Slack +## Slack notifications -A team communication tool for the 21st century +Receive event notifications in Slack ### Create/Edit Slack service @@ -629,6 +737,40 @@ Get Slack service settings for a project. GET /projects/:id/services/slack ``` +## Mattermost notifications + +Receive event notifications in Mattermost + +### Create/Edit Mattermost notifications service + +Set Mattermost service for a project. + +``` +PUT /projects/:id/services/mattermost +``` + +Parameters: + +- `webhook` (**required**) - https://mattermost.example/hooks/1298aff... +- `username` (optional) - username +- `channel` (optional) - #channel + +### Delete Mattermost notifications service + +Delete Mattermost Notifications service for a project. + +``` +DELETE /projects/:id/services/mattermost +``` + +### Get Mattermost notifications service settings + +Get Mattermost notifications service settings for a project. + +``` +GET /projects/:id/services/mattermost +``` + ## JetBrains TeamCity CI A continuous integration and build server diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 9dd84a5ff81..07d92bb746c 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -25,7 +25,9 @@ Environments are like tags for your CI jobs, describing where code gets deployed Deployments are created when [jobs] deploy versions of code to environments, so every environment can have one or more deployments. GitLab keeps track of your deployments, so you always know what is currently being deployed on your -servers. +servers. If you have a deployment service such as [Kubernetes][kubernetes-service] +enabled for your project, you can use it to assist with your deployments, and +can even access a terminal for your environment from within GitLab! To better understand how environments and deployments work, let's consider an example. We assume that you have already created a project in GitLab and set up @@ -86,6 +88,13 @@ will later see, is exposed in various places within GitLab. Each time a job that has an environment specified and succeeds, a deployment is recorded, remembering the Git SHA and environment name. +>**Note:** +Starting with GitLab 8.15, the environment name is exposed to the Runner in +two forms: `$CI_ENVIRONMENT_NAME`, and `$CI_ENVIRONMENT_SLUG`. The first is +the name given in `.gitlab-ci.yml` (with any variables expanded), while the +second is a "cleaned-up" version of the name, suitable for use in URLs, DNS, +etc. + To sum up, with the above `.gitlab-ci.yml` we have achieved that: - All branches will run the `test` and `build` jobs. @@ -157,7 +166,7 @@ that can be found in the deployments page job with the commit associated with it. >**Note:** -Bare in mind that your mileage will vary and it's entirely up to how you define +Bear in mind that your mileage will vary and it's entirely up to how you define the deployment process in the job's `script` whether the rollback succeeds or not. GitLab CI is just following orders. @@ -226,6 +235,46 @@ Remember that if your environment's name is `production` (all lowercase), then it will get recorded in [Cycle Analytics](../user/project/cycle_analytics.md). Double the benefit! +## Terminal support + +>**Note:** +Terminal support was added in GitLab 8.15 and is only available to project +masters and owners. + +If you deploy to your environments with the help of a deployment service (e.g., +the [Kubernetes](../project_services/kubernetes.md) service), GitLab can open +a terminal session to your environment! This is a very powerful feature that +allows you to debug issues without leaving the comfort of your web browser. To +enable it, just follow the instructions given in the service documentation. + +Once enabled, your environments will gain a "terminal" button: + + + +You can also access the terminal button from the page for a specific environment: + + + +Wherever you find it, clicking the button will take you to a separate page to +establish the terminal session: + + + +This works just like any other terminal - you'll be in the container created +by your deployment, so you can run shell commands and get responses in real +time, check the logs, try out configuration or code tweaks, etc. You can open +multiple terminals to the same environment - they each get their own shell +session - and even a multiplexer like `screen` or `tmux`! + +>**Note:** +Container-based deployments often lack basic tools (like an editor), and may +be stopped or restarted at any time. If this happens, you will lose all your +changes! Treat this as a debugging tool, not a comprehensive online IDE. You +can use [Koding](../administration/integration/koding.md) for online +development. + +--- + While this is fine for deploying to some stable environments like staging or production, what happens for branches? So far we haven't defined anything regarding deployments for branches other than `master`. Dynamic environments @@ -248,7 +297,7 @@ deploy_review: - echo "Deploy a review app" environment: name: review/$CI_BUILD_REF_NAME - url: https://$CI_BUILD_REF_NAME.example.com + url: https://$CI_BUILD_REF_SLUG.review.example.com only: - branches except: @@ -259,15 +308,25 @@ Let's break it down in pieces. The job's name is `deploy_review` and it runs on the `deploy` stage. The `script` at this point is fictional, you'd have to use your own based on your deployment. Then, we set the `environment` with the `environment:name` being `review/$CI_BUILD_REF_NAME`. Now that's an interesting -one. Since the [environment name][env-name] can contain also slashes (`/`), we -can use this pattern to distinguish between dynamic environments and the regular +one. Since the [environment name][env-name] can contain slashes (`/`), we can +use this pattern to distinguish between dynamic environments and the regular ones. So, the first part is `review`, followed by a `/` and then `$CI_BUILD_REF_NAME` -which takes the value of the branch name. We also use the same -`$CI_BUILD_REF_NAME` value in the `environment:url` so that the environment -can get a specific and distinct URL for each branch. Again, the way you set up -the webserver to serve these requests is based on your setup. +which takes the value of the branch name. Since `$CI_BUILD_REF_NAME` itself may +also contain `/`, or other characters that would be invalid in a domain name or +URL, we use `$CI_ENVIRONMENT_SLUG` in the `environment:url` so that the +environment can get a specific and distinct URL for each branch. In this case, +given a `$CI_BUILD_REF_NAME` of `100-Do-The-Thing`, the URL will be something +like `https://review-100-do-the-4f99a2.example.com`. Again, the way you set up +the web server to serve these requests is based on your setup. + +You could also use `$CI_BUILD_REF_SLUG` in `environment:url`, e.g.: +`https://$CI_BUILD_REF_SLUG.review.example.com`. We use `$CI_ENVIRONMENT_SLUG` +here because it is guaranteed to be unique, but if you're using a workflow like +[GitLab Flow][gitlab-flow], collisions are very unlikely, and you may prefer +environment names to be more closely based on the branch name - the example +above would give you an URL like `https://100-do-the-thing.review.example.com` Last but not least, we tell the job to run [`only`][only] on branches [`except`][only] master. @@ -299,7 +358,7 @@ deploy_review: - echo "Deploy a review app" environment: name: review/$CI_BUILD_REF_NAME - url: https://$CI_BUILD_REF_NAME.example.com + url: https://$CI_ENVIRONMENT_SLUG.example.com only: - branches except: @@ -329,16 +388,16 @@ deploy_prod: A more realistic example would include copying files to a location where a webserver (NGINX) could then read and serve. The example below will copy the -`public` directory to `/srv/nginx/$CI_BUILD_REF_NAME/public`: +`public` directory to `/srv/nginx/$CI_BUILD_REF_SLUG/public`: ```yaml review_app: stage: deploy script: - - rsync -av --delete public /srv/nginx/$CI_BUILD_REF_NAME + - rsync -av --delete public /srv/nginx/$CI_BUILD_REF_SLUG environment: name: review/$CI_BUILD_REF_NAME - url: https://$CI_BUILD_REF_NAME.example.com + url: https://$CI_BUILD_REF_SLUG.example.com ``` It is assumed that the user has already setup NGINX and GitLab Runner in the @@ -346,7 +405,7 @@ server this job will run on. >**Note:** Be sure to check out the [limitations](#limitations) section for some edge -cases regarding naming of you branches and Review Apps. +cases regarding naming of your branches and Review Apps. --- @@ -418,7 +477,7 @@ deploy_review: - echo "Deploy a review app" environment: name: review/$CI_BUILD_REF_NAME - url: https://$CI_BUILD_REF_NAME.example.com + url: https://$CI_ENVIRONMENT_SLUG.example.com on_stop: stop_review only: - branches @@ -480,9 +539,8 @@ exist, you should see something like: ## Checkout deployments locally -Since 8.13, a reference in the git repository is saved for each deployment. So -knowing what the state is of your current environments is only a `git fetch` -away. +Since 8.13, a reference in the git repository is saved for each deployment, so +knowing the state of your current environments is only a `git fetch` away. In your git config, append the `[remote "<your-remote>"]` block with an extra fetch line: @@ -493,10 +551,6 @@ fetch = +refs/environments/*:refs/remotes/origin/environments/* ## Limitations -1. If the branch name contains special characters (`/`), and you use the - `$CI_BUILD_REF_NAME` variable to dynamically create environments, there might - be complications during your Review Apps deployment. Follow the - [issue 22849][ce-22849] for more information. 1. You are limited to use only the [CI predefined variables][variables] in the `environment: name`. If you try to re-use variables defined inside `script` as part of the environment name, it will not work. @@ -512,6 +566,7 @@ Below are some links you may find interesting: [Pipelines]: pipelines.md [jobs]: yaml/README.md#jobs [yaml]: yaml/README.md +[kubernetes-service]: ../project_services/kubernetes.md] [environments]: #environments [deployments]: #deployments [permissions]: ../user/permissions.md @@ -520,6 +575,6 @@ Below are some links you may find interesting: [only]: yaml/README.md#only-and-except [onstop]: yaml/README.md#environment-on_stop [ce-7015]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7015 -[ce-22849]: https://gitlab.com/gitlab-org/gitlab-ce/issues/22849 +[gitlab-flow]: ../workflow/gitlab_flow.md [gitlab runner]: https://docs.gitlab.com/runner/ [git-strategy]: yaml/README.md#git-strategy diff --git a/doc/ci/img/environments_terminal_button_on_index.png b/doc/ci/img/environments_terminal_button_on_index.png Binary files differnew file mode 100644 index 00000000000..6f05b2aa343 --- /dev/null +++ b/doc/ci/img/environments_terminal_button_on_index.png diff --git a/doc/ci/img/environments_terminal_button_on_show.png b/doc/ci/img/environments_terminal_button_on_show.png Binary files differnew file mode 100644 index 00000000000..9469fab99ab --- /dev/null +++ b/doc/ci/img/environments_terminal_button_on_show.png diff --git a/doc/ci/img/environments_terminal_page.png b/doc/ci/img/environments_terminal_page.png Binary files differnew file mode 100644 index 00000000000..fde1bf325a6 --- /dev/null +++ b/doc/ci/img/environments_terminal_page.png diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 03b9c4bb444..f91b9d350f7 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -36,6 +36,37 @@ Clicking on a pipeline will show the builds that were run for that pipeline. Clicking on an individual build will show you its build trace, and allow you to cancel the build, retry it, or erase the build trace. +## How the pipeline duration is calculated + +Total running time for a given pipeline would exclude retries and pending +(queue) time. We could reduce this problem down to finding the union of +periods. + +So each job would be represented as a `Period`, which consists of +`Period#first` as when the job started and `Period#last` as when the +job was finished. A simple example here would be: + +* A (1, 3) +* B (2, 4) +* C (6, 7) + +Here A begins from 1, and ends to 3. B begins from 2, and ends to 4. +C begins from 6, and ends to 7. Visually it could be viewed as: + +``` +0 1 2 3 4 5 6 7 + AAAAAAA + BBBBBBB + CCCC +``` + +The union of A, B, and C would be (1, 4) and (6, 7), therefore the +total running time should be: + +``` +(4 - 1) + (7 - 6) => 4 +``` + ## Badges Build status and test coverage report badges are available. You can find their diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index d142fe266a2..d3b9611b02e 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -13,6 +13,7 @@ this order: 1. [Secret variables](#secret-variables) 1. YAML-defined [job-level variables](../yaml/README.md#job-variables) 1. YAML-defined [global variables](../yaml/README.md#variables) +1. [Deployment variables](#deployment-variables) 1. [Predefined variables](#predefined-variables-environment-variables) (are the lowest in the chain) @@ -40,6 +41,7 @@ version of Runner required. | **CI_BUILD_NAME** | all | 0.5 | The name of the build as defined in `.gitlab-ci.yml` | | **CI_BUILD_STAGE** | all | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | | **CI_BUILD_REF_NAME** | all | all | The branch or tag name for which project is built | +| **CI_BUILD_REF_SLUG** | 8.15 | all | `$CI_BUILD_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | | **CI_BUILD_REPO** | all | all | The URL to clone the Git repository | | **CI_BUILD_TRIGGERED** | all | 0.5 | The flag to indicate that build was [triggered] | | **CI_BUILD_MANUAL** | 8.12 | all | The flag to indicate that build was manually started | @@ -51,12 +53,17 @@ version of Runner required. | **CI_PROJECT_PATH** | 8.10 | 0.5 | The namespace with project name | | **CI_PROJECT_URL** | 8.10 | 0.5 | The HTTP address to access project | | **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the build is run | +| **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this build | +| **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. | | **CI_REGISTRY** | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry | | **CI_REGISTRY_IMAGE** | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | | **CI_RUNNER_ID** | 8.10 | 0.5 | The unique id of runner being used | | **CI_RUNNER_DESCRIPTION** | 8.10 | 0.5 | The description of the runner as saved in GitLab | | **CI_RUNNER_TAGS** | 8.10 | 0.5 | The defined runner tags | | **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | +| **GET_SOURCES_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to fetch sources running a build | +| **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a build | +| **RESTORE_CACHE_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to restore the cache running a build | | **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the build | | **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the build | @@ -145,6 +152,20 @@ Secret variables can be added by going to your project's Once you set them, they will be available for all subsequent builds. +## Deployment variables + +>**Note:** +This feature requires GitLab CI 8.15 or higher. + +[Project services](../../project_services/project_services.md) that are +responsible for deployment configuration may define their own variables that +are set in the build environment. These variables are only defined for +[deployment builds](../environments.md). Please consult the documentation of +the project services that you are using to learn which variables they define. + +An example project service that defines deployment variables is +[Kubernetes Service](../../project_services/kubernetes.md). + ## Debug tracing > Introduced in GitLab Runner 1.7. diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 5f88974d360..7158b2e7895 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -690,18 +690,12 @@ The `stop_review_app` job is **required** to have the following keywords defined #### dynamic environments > [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. + `$CI_ENVIRONMENT_SLUG` was [introduced][ce-7983] in GitLab 8.15 `environment` can also represent a configuration hash with `name` and `url`. These parameters can use any of the defined [CI variables](#variables) (including predefined, secure variables and `.gitlab-ci.yml` variables). ->**Note:** -Be aware than if the branch name contains special characters and you use the -`$CI_BUILD_REF_NAME` variable to dynamically create environments, there might -be complications during deployment. Follow the -[issue 22849](https://gitlab.com/gitlab-org/gitlab-ce/issues/22849) for more -information. - For example: ``` @@ -709,15 +703,17 @@ deploy as review app: stage: deploy script: make deploy environment: - name: review-apps/$CI_BUILD_REF_NAME - url: https://$CI_BUILD_REF_NAME.review.example.com/ + name: review/$CI_BUILD_REF_NAME + url: https://$CI_ENVIRONMENT_SLUG.example.com/ ``` The `deploy as review app` job will be marked as deployment to dynamically -create the `review-apps/$CI_BUILD_REF_NAME` environment, which `$CI_BUILD_REF_NAME` -is an [environment variable][variables] set by the Runner. If for example the -`deploy as review app` job was run in a branch named `pow`, this environment -should be accessible under `https://pow.review.example.com/`. +create the `review/$CI_BUILD_REF_NAME` environment, where `$CI_BUILD_REF_NAME` +is an [environment variable][variables] set by the Runner. The +`$CI_ENVIRONMENT_SLUG` variable is based on the environment name, but suitable +for inclusion in URLs. In this case, if the `deploy as review app` job was run +in a branch named `pow`, this environment would be accessible with an URL like +`https://review-pow-aaaaaa.example.com/`. This of course implies that the underlying server which hosts the application is properly configured. @@ -1038,6 +1034,31 @@ variables: GIT_STRATEGY: none ``` +## Build stages attempts + +> Introduced in GitLab, it requires GitLab Runner v1.9+. + +You can set the number for attempts the running build will try to execute each +of the following stages: + +| Variable | Description | +|-------------------------|-------------| +| **GET_SOURCES_ATTEMPTS** | Number of attempts to fetch sources running a build | +| **ARTIFACT_DOWNLOAD_ATTEMPTS** | Number of attempts to download artifacts running a build | +| **RESTORE_CACHE_ATTEMPTS** | Number of attempts to restore the cache running a build | + +The default is one single attempt. + +Example: + +``` +variables: + GET_SOURCES_ATTEMPTS: "3" +``` + +You can set them in the global [`variables`](#variables) section or the [`variables`](#job-variables) +section for individual jobs. + ## Shallow cloning > Introduced in GitLab 8.9 as an experimental feature. May change in future @@ -1246,3 +1267,4 @@ CI with various languages. [environment]: ../environments.md [ce-6669]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6669 [variables]: ../variables/README.md +[ce-7983]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7983 diff --git a/doc/development/performance.md b/doc/development/performance.md index 5c43ae7b79a..f936a49a2aa 100644 --- a/doc/development/performance.md +++ b/doc/development/performance.md @@ -37,7 +37,7 @@ graphs/dashboards. GitLab provides built-in tools to aid the process of improving performance: * [Sherlock](profiling.md#sherlock) -* [GitLab Performance Monitoring](../administration/monitoring/performance/monitoring.md) +* [GitLab Performance Monitoring](../administration/monitoring/performance/introduction.md) * [Request Profiling](../administration/monitoring/performance/request_profiling.md) GitLab employees can use GitLab.com's performance monitoring systems located at diff --git a/doc/development/ux_guide/basics.md b/doc/development/ux_guide/basics.md index 76b739386a5..e81729556d8 100644 --- a/doc/development/ux_guide/basics.md +++ b/doc/development/ux_guide/basics.md @@ -5,6 +5,7 @@ * [Typography](#typography) * [Icons](#icons) * [Color](#color) +* [Cursors](#cursors) --- @@ -59,3 +60,18 @@ GitLab uses Font Awesome icons throughout our interface. > TODO: Establish a perspective for color in terms of our personality and rationalize with Marketing usage. +--- + +## Cursors +The mouse cursor is key in helping users understand how to interact with elements on the screen. + +| | | +| :------: | :------- | +|  | Default cursor | +|  | Pointer cursor: used to indicate that you can click on an element to invoke a command or navigate, such as links and buttons | +|  | Move cursor: used to indicate that you can move an element around on the screen | +|  | Pan cursor (opened): indicates that you can grab and move the entire canvas, affecting what is seen in the view port. | +|  | Pan cursor (closed): indicates that you are actively panning the canvas. | +|  | I-beam cursor: indicates that this is either text that you can select and copy, or a text field that you can click into to enter text. | + + diff --git a/doc/development/ux_guide/img/cursors-default.png b/doc/development/ux_guide/img/cursors-default.png Binary files differnew file mode 100644 index 00000000000..c188ec4e351 --- /dev/null +++ b/doc/development/ux_guide/img/cursors-default.png diff --git a/doc/development/ux_guide/img/cursors-ibeam.png b/doc/development/ux_guide/img/cursors-ibeam.png Binary files differnew file mode 100644 index 00000000000..86f28639982 --- /dev/null +++ b/doc/development/ux_guide/img/cursors-ibeam.png diff --git a/doc/development/ux_guide/img/cursors-move.png b/doc/development/ux_guide/img/cursors-move.png Binary files differnew file mode 100644 index 00000000000..a9c610eaa88 --- /dev/null +++ b/doc/development/ux_guide/img/cursors-move.png diff --git a/doc/development/ux_guide/img/cursors-panclosed.png b/doc/development/ux_guide/img/cursors-panclosed.png Binary files differnew file mode 100644 index 00000000000..6d247a765ac --- /dev/null +++ b/doc/development/ux_guide/img/cursors-panclosed.png diff --git a/doc/development/ux_guide/img/cursors-panopened.png b/doc/development/ux_guide/img/cursors-panopened.png Binary files differnew file mode 100644 index 00000000000..76f2eeda831 --- /dev/null +++ b/doc/development/ux_guide/img/cursors-panopened.png diff --git a/doc/development/ux_guide/img/cursors-pointer.png b/doc/development/ux_guide/img/cursors-pointer.png Binary files differnew file mode 100644 index 00000000000..d86dd955fa7 --- /dev/null +++ b/doc/development/ux_guide/img/cursors-pointer.png diff --git a/doc/integration/bitbucket.md b/doc/integration/bitbucket.md index 9122dc62e39..1dfc985eaea 100644 --- a/doc/integration/bitbucket.md +++ b/doc/integration/bitbucket.md @@ -5,7 +5,7 @@ Bitbucket.org account. ## Overview -You can set up Bitbucket.org as an OAuth provider so that you can use your +You can set up Bitbucket.org as an OAuth2 provider so that you can use your credentials to authenticate into GitLab or import your projects from Bitbucket.org. @@ -18,8 +18,10 @@ Bitbucket.org. ## Bitbucket OmniAuth provider > **Note:** -Make sure to first follow the [Initial OmniAuth configuration][init-oauth] -before proceeding with setting up the Bitbucket integration. +GitLab 8.15 significantly simplified the way to integrate Bitbucket.org with +GitLab. You are encouraged to upgrade your GitLab instance if you haven't done +already. If you're using GitLab 8.14 and below, [use the previous integration +docs][bb-old]. To enable the Bitbucket OmniAuth provider you must register your application with Bitbucket.org. Bitbucket will generate an application ID and secret key for @@ -44,14 +46,13 @@ you to use. And grant at least the following permissions: ``` - Account: Email - Repositories: Read, Admin + Account: Email, Read + Repositories: Read + Pull Requests: Read + Issues: Read + Wiki: Read and Write ``` - >**Note:** - It may seem a little odd to giving GitLab admin permissions to repositories, - but this is needed in order for GitLab to be able to clone the repositories. -  1. Select **Save**. @@ -93,7 +94,8 @@ you to use. ```yaml - { name: 'bitbucket', app_id: 'BITBUCKET_APP_KEY', - app_secret: 'BITBUCKET_APP_SECRET' } + app_secret: 'BITBUCKET_APP_SECRET', + url: 'https://bitbucket.org/' } ``` --- @@ -112,100 +114,12 @@ well, the user will be returned to GitLab and will be signed in. ## Bitbucket project import -To allow projects to be imported directly into GitLab, Bitbucket requires two -extra setup steps compared to [GitHub](github.md) and [GitLab.com](gitlab.md). - -Bitbucket doesn't allow OAuth applications to clone repositories over HTTPS, and -instead requires GitLab to use SSH and identify itself using your GitLab -server's SSH key. - -To be able to access repositories on Bitbucket, GitLab will automatically -register your public key with Bitbucket as a deploy key for the repositories to -be imported. Your public key needs to be at `~/.ssh/bitbucket_rsa` which -translates to `/var/opt/gitlab/.ssh/bitbucket_rsa` for Omnibus packages and to -`/home/git/.ssh/bitbucket_rsa` for installations from source. - ---- - -Below are the steps that will allow GitLab to be able to import your projects -from Bitbucket. - -1. Make sure you [have enabled the Bitbucket OAuth support](#bitbucket-omniauth-provider). -1. Create a new SSH key with an **empty passphrase**: - - ```sh - sudo -u git -H ssh-keygen - ``` - - When asked to 'Enter file in which to save the key' enter: - `/var/opt/gitlab/.ssh/bitbucket_rsa` for Omnibus packages or - `/home/git/.ssh/bitbucket_rsa` for installations from source. The name is - important so make sure to get it right. - - > **Warning:** - This key must NOT be associated with ANY existing Bitbucket accounts. If it - is, the import will fail with an `Access denied! Please verify you can add - deploy keys to this repository.` error. - -1. Next, you need to to configure the SSH client to use your new key. Open the - SSH configuration file of the `git` user: - - ``` - # For Omnibus packages - sudo editor /var/opt/gitlab/.ssh/config - - # For installations from source - sudo editor /home/git/.ssh/config - ``` - -1. Add a host configuration for `bitbucket.org`: - - ```sh - Host bitbucket.org - IdentityFile ~/.ssh/bitbucket_rsa - User git - ``` - -1. Save the file and exit. -1. Manually connect to `bitbucket.org` over SSH, while logged in as the `git` - user that GitLab will use: - - ```sh - sudo -u git -H ssh bitbucket.org - ``` - - That step is performed because GitLab needs to connect to Bitbucket over SSH, - in order to add `bitbucket.org` to your GitLab server's known SSH hosts. - -1. Verify the RSA key fingerprint you'll see in the response matches the one - in the [Bitbucket documentation][bitbucket-docs] (the specific IP address - doesn't matter): - - ```sh - The authenticity of host 'bitbucket.org (104.192.143.1)' can't be established. - RSA key fingerprint is SHA256:zzXQOXSRBEiUtuE8AikJYKwbHaxvSc0ojez9YXaGp1A. - Are you sure you want to continue connecting (yes/no)? - ``` - -1. If the fingerprint matches, type `yes` to continue connecting and have - `bitbucket.org` be added to your known SSH hosts. After confirming you should - see a permission denied message. If you see an authentication successful - message you have done something wrong. The key you are using has already been - added to a Bitbucket account and will cause the import script to fail. Ensure - the key you are using CANNOT authenticate with Bitbucket. -1. Restart GitLab to allow it to find the new public key. - -Your GitLab server is now able to connect to Bitbucket over SSH. You should be -able to see the "Import projects from Bitbucket" option on the New Project page -enabled. - -## Acknowledgements - -Special thanks to the writer behind the following article: - -- http://stratus3d.com/blog/2015/09/06/migrating-from-bitbucket-to-local-gitlab-server/ +Once the above configuration is set up, you can use Bitbucket to sign into +GitLab and [start importing your projects][bb-import]. [init-oauth]: omniauth.md#initial-omniauth-configuration +[bb-import]: ../workflow/importing/import_projects_from_bitbucket.md +[bb-old]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-14-stable/doc/integration/bitbucket.md [bitbucket-docs]: https://confluence.atlassian.com/bitbucket/use-the-ssh-protocol-with-bitbucket-cloud-221449711.html#UsetheSSHprotocolwithBitbucketCloud-KnownhostorBitbucket%27spublickeyfingerprints [reconfigure]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure [restart GitLab]: ../administration/restart_gitlab.md#installations-from-source diff --git a/doc/integration/img/bitbucket_oauth_settings_page.png b/doc/integration/img/bitbucket_oauth_settings_page.png Binary files differindex 8dbee9762d7..3e6dea6cfe9 100644 --- a/doc/integration/img/bitbucket_oauth_settings_page.png +++ b/doc/integration/img/bitbucket_oauth_settings_page.png diff --git a/doc/project_services/img/kubernetes_configuration.png b/doc/project_services/img/kubernetes_configuration.png Binary files differnew file mode 100644 index 00000000000..349a2dc8456 --- /dev/null +++ b/doc/project_services/img/kubernetes_configuration.png diff --git a/doc/project_services/img/mattermost_configuration.png b/doc/project_services/img/mattermost_configuration.png Binary files differnew file mode 100644 index 00000000000..3c5ff5ee317 --- /dev/null +++ b/doc/project_services/img/mattermost_configuration.png diff --git a/doc/project_services/kubernetes.md b/doc/project_services/kubernetes.md new file mode 100644 index 00000000000..0c5c88dd983 --- /dev/null +++ b/doc/project_services/kubernetes.md @@ -0,0 +1,63 @@ +# GitLab Kubernetes / OpenShift integration + +GitLab can be configured to interact with Kubernetes, or other systems using the +Kubernetes API (such as OpenShift). + +Each project can be configured to connect to a different Kubernetes cluster, see +the [configuration](#configuration) section. + +If you have a single cluster that you want to use for all your projects, +you can pre-fill the settings page with a default template. To configure the +template, see the [Services Templates](services-templates.md) document. + +## Configuration + + + +The Kubernetes service takes the following arguments: + +1. Kubernetes namespace +1. API URL +1. Service token +1. Custom CA bundle + +The API URL is the URL that GitLab uses to access the Kubernetes API. Kubernetes +exposes several APIs - we want the "base" URL that is common to all of them, +e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. + +GitLab authenticates against Kubernetes using service tokens, which are +scoped to a particular `namespace`. If you don't have a service token yet, +you can follow the +[Kubernetes documentation](http://kubernetes.io/docs/user-guide/service-accounts/) +to create one. You can also view or create service tokens in the +[Kubernetes dashboard](http://kubernetes.io/docs/user-guide/ui/) - visit +`Config -> Secrets`. + +Fill in the service token and namespace according to the values you just got. +If the API is using a self-signed TLS certificate, you'll also need to include +the `ca.crt` contents as the `Custom CA bundle`. + +## Deployment variables + +The Kubernetes service exposes following +[deployment variables](../ci/variables/README.md#deployment-variables) in the +GitLab CI build environment: + +- `KUBE_URL` - equal to the API URL +- `KUBE_TOKEN` +- `KUBE_NAMESPACE` +- `KUBE_CA_PEM` - only if a custom CA bundle was specified + +## Terminal support + +>**NOTE:** +Added in GitLab 8.15. You must be the project owner or have `master` permissions +to use terminals. Support is currently limited to the first container in the +first pod of your environment. + +When enabled, the Kubernetes service adds online [terminal support](../ci/environments.md#terminal-support) +to your environments. This is based on the `exec` functionality found in +Docker and Kubernetes, so you get a new shell session within your existing +containers. To use this integration, you should deploy to Kubernetes using +the deployment variables above, ensuring any pods you create are labelled with +`app=$CI_ENVIRONMENT_SLUG`. diff --git a/doc/project_services/mattermost.md b/doc/project_services/mattermost.md new file mode 100644 index 00000000000..fbc7dfeee6d --- /dev/null +++ b/doc/project_services/mattermost.md @@ -0,0 +1,45 @@ +# Mattermost Notifications Service + +## On Mattermost + +To enable Mattermost integration you must create an incoming webhook integration: + +1. Sign in to your Mattermost instance +1. Visit incoming webhooks, that will be something like: https://mattermost.example/your_team_name/integrations/incoming_webhooks/add +1. Choose a display name, description and channel, those can be overridden on GitLab +1. Save it, copy the **Webhook URL**, we'll need this later for GitLab. + +There might be some cases that Incoming Webhooks are blocked by admin, ask your mattermost admin to enable +it on https://mattermost.example/admin_console/integrations/custom. + +Display name override is not enabled by default, you need to ask your admin to enable it on that same section. + +## On GitLab + +After you set up Mattermost, it's time to set up GitLab. + +Go to your project's **Settings > Services > Mattermost Notifications** and you will see a +checkbox with the following events that can be triggered: + +- Push +- Issue +- Merge request +- Note +- Tag push +- Build +- Wiki page + +Bellow each of these event checkboxes, you will have an input field to insert +which Mattermost channel you want to send that event message, with `#town-square` +being the default. The hash sign is optional. + +At the end, fill in your Mattermost details: + +| Field | Description | +| ----- | ----------- | +| **Webhook** | The incoming webhooks which you have to setup on Mattermost, it will be something like: http://mattermost.example/hooks/5xo... | +| **Username** | Optional username which can be on messages sent to Mattermost. Fill this in if you want to change the username of the bot. | +| **Notify only broken builds** | If you choose to enable the **Build** event and you want to be only notified about failed builds. | + + + diff --git a/doc/project_services/mattermost_slash_commands.md b/doc/project_services/mattermost_slash_commands.md index 6fcbf6f1642..1a7c13a29b4 100644 --- a/doc/project_services/mattermost_slash_commands.md +++ b/doc/project_services/mattermost_slash_commands.md @@ -65,7 +65,7 @@ the administrator console. ### Step 3. Create a new custom slash command in Mattermost -Now that you have enabled the custom slash commands in Mattermost and opened +Now that you have enabled custom slash commands in Mattermost and opened the Mattermost slash commands service in GitLab, it's time to copy these values in a new slash command. @@ -128,20 +128,16 @@ GitLab using the Mattermost commands. ## Available slash commands -The available slash commands so far are: +The available slash commands are: | Command | Description | Example | | ------- | ----------- | ------- | -| `/<trigger> issue create <title>\n<description>` | Create a new issue in the project that `<trigger>` is tied to. `<description>` is optional. | `/trigger issue create We need to change the homepage` | -| `/<trigger> issue show <issue-number>` | Show the issue with ID `<issue-number>` from the project that `<trigger>` is tied to. | `/trigger issue show 42` | -| `/<trigger> deploy <environment> to <environment>` | Start the CI job that deploys from one environment to another, for example `staging` to `production`. CI/CD must be [properly configured][ciyaml]. | `/trigger deploy staging to production` | +| <kbd>/<trigger> issue new <title> <kbd>⇧ Shift</kbd>+<kbd>↵ Enter</kbd> <description></kbd> | Create a new issue in the project that `<trigger>` is tied to. `<description>` is optional. | <samp>/gitlab issue new We need to change the homepage</samp> | +| <kbd>/<trigger> issue show <issue-number></kbd> | Show the issue with ID `<issue-number>` from the project that `<trigger>` is tied to. | <samp>/gitlab issue show 42</samp> | +| <kbd>/<trigger> deploy <environment> to <environment></kbd> | Start the CI job that deploys from one environment to another, for example `staging` to `production`. CI/CD must be [properly configured][ciyaml]. | <samp>/gitlab deploy staging to production</samp> | -To see a list of available commands that can interact with GitLab, type the -trigger word followed by `help`: - -``` -/my-project help -``` +To see a list of available commands to interact with GitLab, type the +trigger word followed by <kbd>help</kbd>. Example: <samp>/gitlab help</samp>  diff --git a/doc/project_services/project_services.md b/doc/project_services/project_services.md index 890f7525b0e..0f398874b8f 100644 --- a/doc/project_services/project_services.md +++ b/doc/project_services/project_services.md @@ -42,11 +42,13 @@ further configuration instructions and details. Contributions are welcome. | [Irker (IRC gateway)](irker.md) | Send IRC messages, on update, to a list of recipients through an Irker gateway | | [JIRA](jira.md) | JIRA issue tracker | | JetBrains TeamCity CI | A continuous integration and build server | +| [Kubernetes](kubernetes.md) | A containerized deployment service | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | +| [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | +| [Slack Notifications](slack.md) | Receive event notifications in Slack | | PivotalTracker | Project Management Software (Source Commits Endpoint) | | Pushover | Pushover makes it easy to get real-time notifications on your Android device, iPhone, iPad, and Desktop | | [Redmine](redmine.md) | Redmine issue tracker | -| [Slack](slack.md) | A team communication tool for the 21st century | ## Services Templates diff --git a/doc/project_services/slack.md b/doc/project_services/slack.md index 3cfe77c9f85..0b682b43810 100644 --- a/doc/project_services/slack.md +++ b/doc/project_services/slack.md @@ -1,4 +1,4 @@ -# Slack Service +# Slack Notifications Service ## On Slack @@ -15,7 +15,7 @@ Slack: After you set up Slack, it's time to set up GitLab. -Go to your project's **Settings > Services > Slack** and you will see a +Go to your project's **Settings > Services > Slack Notifications** and you will see a checkbox with the following events that can be triggered: - Push diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md index a49c43b8ef2..2b81ebc9c59 100644 --- a/doc/raketasks/README.md +++ b/doc/raketasks/README.md @@ -4,7 +4,8 @@ - [Check](check.md) - [Cleanup](cleanup.md) - [Features](features.md) -- [Maintenance](maintenance.md) and self-checks +- [LDAP Maintenance](../administration/raketasks/ldap.md) +- [General Maintenance](maintenance.md) and self-checks - [User management](user_management.md) - [Webhooks](web_hooks.md) - [Import](import.md) of git repositories in bulk diff --git a/doc/update/8.13-to-8.14.md b/doc/update/8.13-to-8.14.md index a0e895773ce..c64d3407461 100644 --- a/doc/update/8.13-to-8.14.md +++ b/doc/update/8.13-to-8.14.md @@ -72,7 +72,7 @@ sudo -u git -H git checkout 8-14-stable-ee ```bash cd /home/git/gitlab-shell sudo -u git -H git fetch --all --tags -sudo -u git -H git checkout v4.0.0 +sudo -u git -H git checkout v4.0.3 ``` ### 6. Update gitlab-workhorse diff --git a/doc/update/8.14-to-8.15.md b/doc/update/8.14-to-8.15.md index 5556dae2551..8d4bfd913bd 100644 --- a/doc/update/8.14-to-8.15.md +++ b/doc/update/8.14-to-8.15.md @@ -72,7 +72,7 @@ sudo -u git -H git checkout 8-15-stable-ee ```bash cd /home/git/gitlab-shell sudo -u git -H git fetch --all --tags -sudo -u git -H git checkout v4.0.3 +sudo -u git -H git checkout v4.1.1 ``` ### 6. Update gitlab-workhorse diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 4d24eb21976..f6484688721 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -319,6 +319,40 @@ Here's a sample video:  +### Math + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#math + +It is possible to have math written with the LaTeX syntax rendered using [KaTeX][katex]. + +Math written inside ```$``$``` will be rendered inline with the text. + +Math written inside triple back quotes, with the language declared as `math`, will be rendered on a separate line. + +Example: + + This math is inline $`a^2+b^2=c^2`$. + + This is on a separate line + ```math + a^2+b^2=c^2 + ``` + +Becomes: + +This math is inline $`a^2+b^2=c^2`$. + +This is on a separate line +```math +a^2+b^2=c^2 +``` + +_Be advised that KaTeX only supports a [subset][katex-subset] of LaTeX._ + +>**Note:** +This also works for the asciidoctor `:stem: latexmath`. For details see the [asciidoctor user manual][asciidoctor-manual]. + ## Standard Markdown ### Headers @@ -764,3 +798,6 @@ A link starting with a `/` is relative to the wiki root. [markdown.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md [rouge]: http://rouge.jneen.net/ "Rouge website" [redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" +[katex]: https://github.com/Khan/KaTeX "KaTeX website" +[katex-subset]: https://github.com/Khan/KaTeX/wiki/Function-Support-in-KaTeX "Macros supported by KaTeX" +[asciidoctor-manual]: http://asciidoctor.org/docs/user-manual/#activating-stem-support "Asciidoctor user manual"
\ No newline at end of file diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 39fe2409a29..5ada8748d85 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -33,6 +33,7 @@ The following table depicts the various user permission levels in a project. | See a container registry | | ✓ | ✓ | ✓ | ✓ | | See environments | | ✓ | ✓ | ✓ | ✓ | | Create new environments | | | ✓ | ✓ | ✓ | +| Use environment terminals | | | | ✓ | ✓ | | Stop environments | | | ✓ | ✓ | ✓ | | See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | | Manage/Accept merge requests | | | ✓ | ✓ | ✓ | diff --git a/doc/workflow/importing/README.md b/doc/workflow/importing/README.md index 18e5d950866..2d91bee0e94 100644 --- a/doc/workflow/importing/README.md +++ b/doc/workflow/importing/README.md @@ -4,6 +4,7 @@ 1. [GitHub](import_projects_from_github.md)
1. [GitLab.com](import_projects_from_gitlab_com.md)
1. [FogBugz](import_projects_from_fogbugz.md)
+1. [Gitea](import_projects_from_gitea.md)
1. [SVN](migrating_from_svn.md)
In addition to the specific migration documentation above, you can import any
@@ -14,4 +15,3 @@ repository is too large the import can timeout. You can copy your repos by changing the remote and pushing to the new server;
but issues and merge requests can't be imported.
-
diff --git a/doc/workflow/importing/bitbucket_importer/bitbucket_import_grant_access.png b/doc/workflow/importing/bitbucket_importer/bitbucket_import_grant_access.png Binary files differdeleted file mode 100644 index df55a081803..00000000000 --- a/doc/workflow/importing/bitbucket_importer/bitbucket_import_grant_access.png +++ /dev/null diff --git a/doc/workflow/importing/bitbucket_importer/bitbucket_import_new_project.png b/doc/workflow/importing/bitbucket_importer/bitbucket_import_new_project.png Binary files differdeleted file mode 100644 index 5253889d251..00000000000 --- a/doc/workflow/importing/bitbucket_importer/bitbucket_import_new_project.png +++ /dev/null diff --git a/doc/workflow/importing/bitbucket_importer/bitbucket_import_select_bitbucket.png b/doc/workflow/importing/bitbucket_importer/bitbucket_import_select_bitbucket.png Binary files differdeleted file mode 100644 index ffa87ce5b2e..00000000000 --- a/doc/workflow/importing/bitbucket_importer/bitbucket_import_select_bitbucket.png +++ /dev/null diff --git a/doc/workflow/importing/bitbucket_importer/bitbucket_import_select_project.png b/doc/workflow/importing/bitbucket_importer/bitbucket_import_select_project.png Binary files differdeleted file mode 100644 index 1a5661de75d..00000000000 --- a/doc/workflow/importing/bitbucket_importer/bitbucket_import_select_project.png +++ /dev/null diff --git a/doc/workflow/importing/img/bitbucket_import_grant_access.png b/doc/workflow/importing/img/bitbucket_import_grant_access.png Binary files differnew file mode 100644 index 00000000000..429904e621d --- /dev/null +++ b/doc/workflow/importing/img/bitbucket_import_grant_access.png diff --git a/doc/workflow/importing/img/bitbucket_import_new_project.png b/doc/workflow/importing/img/bitbucket_import_new_project.png Binary files differnew file mode 100644 index 00000000000..8ed528c2f09 --- /dev/null +++ b/doc/workflow/importing/img/bitbucket_import_new_project.png diff --git a/doc/workflow/importing/img/bitbucket_import_select_project.png b/doc/workflow/importing/img/bitbucket_import_select_project.png Binary files differnew file mode 100644 index 00000000000..1bca6166ec8 --- /dev/null +++ b/doc/workflow/importing/img/bitbucket_import_select_project.png diff --git a/doc/workflow/importing/img/import_projects_from_gitea_new_import.png b/doc/workflow/importing/img/import_projects_from_gitea_new_import.png Binary files differnew file mode 100644 index 00000000000..a3f603cbd0a --- /dev/null +++ b/doc/workflow/importing/img/import_projects_from_gitea_new_import.png diff --git a/doc/workflow/importing/img/import_projects_from_github_new_project_page.png b/doc/workflow/importing/img/import_projects_from_github_new_project_page.png Binary files differdeleted file mode 100644 index b23ade4480c..00000000000 --- a/doc/workflow/importing/img/import_projects_from_github_new_project_page.png +++ /dev/null diff --git a/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png b/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png Binary files differindex f50d9266991..1ccb38a815e 100644 --- a/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png +++ b/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png diff --git a/doc/workflow/importing/img/import_projects_from_new_project_page.png b/doc/workflow/importing/img/import_projects_from_new_project_page.png Binary files differnew file mode 100644 index 00000000000..97ca30b2087 --- /dev/null +++ b/doc/workflow/importing/img/import_projects_from_new_project_page.png diff --git a/doc/workflow/importing/import_projects_from_bitbucket.md b/doc/workflow/importing/import_projects_from_bitbucket.md index 520c4216295..97380bce172 100644 --- a/doc/workflow/importing/import_projects_from_bitbucket.md +++ b/doc/workflow/importing/import_projects_from_bitbucket.md @@ -1,26 +1,62 @@ # Import your project from Bitbucket to GitLab
-It takes just a few steps to import your existing Bitbucket projects to GitLab. But keep in mind that it is possible only if Bitbucket support is enabled on your GitLab instance. You can read more about Bitbucket support [here](../../integration/bitbucket.md).
+Import your projects from Bitbucket to GitLab with minimal effort.
-* Sign in to GitLab.com and go to your dashboard
+## Overview
-* Click on "New project"
+>**Note:**
+The [Bitbucket integration][bb-import] must be first enabled in order to be
+able to import your projects from Bitbucket. Ask your GitLab administrator
+to enable this if not already.
-
+- At its current state, the Bitbucket importer can import:
+ - the repository description (GitLab 7.7+)
+ - the Git repository data (GitLab 7.7+)
+ - the issues (GitLab 7.7+)
+ - the issue comments (GitLab 8.15+)
+ - the pull requests (GitLab 8.4+)
+ - the pull request comments (GitLab 8.15+)
+ - the milestones (GitLab 8.15+)
+ - the wiki (GitLab 8.15+)
+- References to pull requests and issues are preserved (GitLab 8.7+)
+- Repository public access is retained. If a repository is private in Bitbucket
+ it will be created as private in GitLab as well.
-* Click on the "Bitbucket" button
-
+## How it works
-* Grant GitLab access to your Bitbucket account
+When issues/pull requests are being imported, the Bitbucket importer tries to find
+the Bitbucket author/assignee in GitLab's database using the Bitbucket ID. For this
+to work, the Bitbucket author/assignee should have signed in beforehand in GitLab
+and [**associated their Bitbucket account**][social sign-in]. If the user is not
+found in GitLab's database, the project creator (most of the times the current
+user that started the import process) is set as the author, but a reference on
+the issue about the original Bitbucket author is kept.
-
+The importer will create any new namespaces (groups) if they don't exist or in
+the case the namespace is taken, the repository will be imported under the user's
+namespace that started the import process.
-* Click on the projects that you'd like to import or "Import all projects"
+## Importing your Bitbucket repositories
-
+1. Sign in to GitLab and go to your dashboard.
+1. Click on **New project**.
-A new GitLab project will be created with your imported data.
+ 
-### Note
-Milestones and wiki pages are not imported from Bitbucket.
+1. Click on the "Bitbucket" button
+
+ 
+
+1. Grant GitLab access to your Bitbucket account
+
+ 
+
+1. Click on the projects that you'd like to import or **Import all projects**.
+ You can also select the namespace under which each project will be
+ imported.
+
+ 
+
+[bb-import]: ../../integration/bitbucket.md
+[social sign-in]: ../../user/profile/account/social_sign_in.md
diff --git a/doc/workflow/importing/import_projects_from_gitea.md b/doc/workflow/importing/import_projects_from_gitea.md new file mode 100644 index 00000000000..936cee89f45 --- /dev/null +++ b/doc/workflow/importing/import_projects_from_gitea.md @@ -0,0 +1,80 @@ +# Import your project from Gitea to GitLab + +Import your projects from Gitea to GitLab with minimal effort. + +## Overview + +>**Note:** +As of Gitea `v1.0.0`, issue & pull-request comments cannot be imported! This is +a [known issue][issue-401] that should be fixed in a near-future. + +- At its current state, Gitea importer can import: + - the repository description (GitLab 8.15+) + - the Git repository data (GitLab 8.15+) + - the issues (GitLab 8.15+) + - the pull requests (GitLab 8.15+) + - the milestones (GitLab 8.15+) + - the labels (GitLab 8.15+) +- Repository public access is retained. If a repository is private in Gitea + it will be created as private in GitLab as well. + +## How it works + +Since Gitea is currently not an OAuth provider, author/assignee cannot be mapped +to users in your GitLab's instance. This means that the project creator (most of +the times the current user that started the import process) is set as the author, +but a reference on the issue about the original Gitea author is kept. + +The importer will create any new namespaces (groups) if they don't exist or in +the case the namespace is taken, the repository will be imported under the user's +namespace that started the import process. + +## Importing your Gitea repositories + +The importer page is visible when you create a new project. + + + +Click on the **Gitea** link and the import authorization process will start. + + + +### Authorize access to your repositories using a personal access token + +With this method, you will perform a one-off authorization with Gitea to grant +GitLab access your repositories: + +1. Go to <https://you-gitea-instance/user/settings/applications> (replace + `you-gitea-instance` with the host of your Gitea instance). +1. Click **Generate New Token**. +1. Enter a token description. +1. Click **Generate Token**. +1. Copy the token hash. +1. Go back to GitLab and provide the token to the Gitea importer. +1. Hit the **List Your Gitea Repositories** button and wait while GitLab reads + your repositories' information. Once done, you'll be taken to the importer + page to select the repositories to import. + +### Select which repositories to import + +After you've authorized access to your Gitea repositories, you will be +redirected to the Gitea importer page. + +From there, you can see the import statuses of your Gitea repositories. + +- Those that are being imported will show a _started_ status, +- those already successfully imported will be green with a _done_ status, +- whereas those that are not yet imported will have an **Import** button on the + right side of the table. + +If you want, you can import all your Gitea projects in one go by hitting +**Import all projects** in the upper left corner. + + + +--- + +You can also choose a different name for the project and a different namespace, +if you have the privileges to do so. + +[issue-401]: https://github.com/go-gitea/gitea/issues/401 diff --git a/doc/workflow/importing/import_projects_from_github.md b/doc/workflow/importing/import_projects_from_github.md index c36dfdb78ec..86a016fc6d6 100644 --- a/doc/workflow/importing/import_projects_from_github.md +++ b/doc/workflow/importing/import_projects_from_github.md @@ -6,8 +6,9 @@ Import your projects from GitHub to GitLab with minimal effort. >**Note:**
If you are an administrator you can enable the [GitHub integration][gh-import]
-in your GitLab instance sitewide. This configuration is optional, users will be
-able import their GitHub repositories with a [personal access token][gh-token].
+in your GitLab instance sitewide. This configuration is optional, users will
+still be able to import their GitHub repositories with a
+[personal access token][gh-token].
- At its current state, GitHub importer can import:
- the repository description (GitLab 7.7+)
@@ -40,7 +41,7 @@ namespace that started the import process. The importer page is visible when you create a new project.
-
+
Click on the **GitHub** link and the import authorization process will start.
There are two ways to authorize access to your GitHub repositories:
@@ -85,7 +86,7 @@ authorization with GitHub to grant GitLab access your repositories: 1. Click **Generate token**.
1. Copy the token hash.
1. Go back to GitLab and provide the token to the GitHub importer.
-1. Hit the **List your GitHub repositories** button and wait while GitLab reads
+1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads
your repositories' information. Once done, you'll be taken to the importer
page to select the repositories to import.
@@ -112,7 +113,6 @@ You can also choose a different name for the project and a different namespace, if you have the privileges to do so.
[gh-import]: ../../integration/github.md "GitHub integration"
-[new-project]: ../../gitlab-basics/create-project.md "How to create a new project in GitLab"
[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration
[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token
[social sign-in]: ../../profile/account/social_sign_in.md
|
