diff options
Diffstat (limited to 'doc')
162 files changed, 3808 insertions, 801 deletions
diff --git a/doc/README.md b/doc/README.md index ab8ea192a26..8bb8e147cd1 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,12 +1,20 @@ -# GitLab Community Edition +# GitLab Documentation -[GitLab](https://about.gitlab.com/) is a Git-based fully featured platform -for software development. +Welcome to [GitLab](https://about.gitlab.com/), a Git-based fully featured +platform for software development! -**GitLab Community Edition (CE)** is an opensource product, self-hosted, free to use. -All [GitLab products](https://about.gitlab.com/products/) contain the features -available in GitLab CE. Premium features are available in -[GitLab Enterprise Edition (EE)](https://about.gitlab.com/gitlab-ee/). +We offer four different products for you and your company: + +- **GitLab Community Edition (CE)** is an [opensource product](https://gitlab.com/gitlab-org/gitlab-ce/), +self-hosted, free to use. Every feature available in GitLab CE is also available on GitLab Enterprise Edition (Starter and Premium) and GitLab.com. +- **GitLab Enterprise Edition (EE)** is an [opencore product](https://gitlab.com/gitlab-org/gitlab-ee/), +self-hosted, fully featured solution of GitLab, available under distinct [subscriptions](https://about.gitlab.com/products/): **GitLab Enterprise Edition Starter (EES)** and **GitLab Enterprise Edition Premium (EEP)**. +- **GitLab.com**: SaaS GitLab solution, with [free and paid subscriptions](https://about.gitlab.com/gitlab-com/). GitLab.com is hosted by GitLab, Inc., and administrated by GitLab (users don't have access to admin settings). + +> **GitLab EE** contains all features available in **GitLab CE**, +plus premium features available in each version: **Enterprise Edition Starter** +(**EES**) and **Enterprise Edition Premium** (**EEP**). Everything available in +**EES** is also available in **EEP**. ---- @@ -28,27 +36,29 @@ Shortcuts to GitLab's most visited docs: ### User account -- [Authentication](topics/authentication/index.md): Account security with two-factor authentication, setup your ssh keys and deploy keys for secure access to your projects. -- [Profile settings](profile/README.md): Manage your profile settings, two factor authentication and more. +- [User documentation](user/index.md): Learn how to use GitLab and explore its features +- [User account](user/profile/index.md): Manage your account + - [Authentication](topics/authentication/index.md): Account security with two-factor authentication, setup your ssh keys and deploy keys for secure access to your projects. + - [Profile settings](user/profile/index.md#profile-settings): Manage your profile settings, two factor authentication and more. - [User permissions](user/permissions.md): Learn what each role in a project (external/guest/reporter/developer/master/owner) can do. ### Projects and groups -- [Create a project](gitlab-basics/create-project.md) -- [Fork a project](gitlab-basics/fork-project.md) -- [Importing and exporting projects between instances](user/project/settings/import_export.md). -- [Project access](public_access/public_access.md): Setting up your project's visibility to public, internal, or private. -- [Groups](workflow/groups.md): Organize your projects in groups. - - [Create a group](gitlab-basics/create-group.md) - - [GitLab Subgroups](user/group/subgroups/index.md) +- [Projects](user/project/index.md): + - [Create a project](gitlab-basics/create-project.md) + - [Fork a project](gitlab-basics/fork-project.md) + - [Importing and exporting projects between instances](user/project/settings/import_export.md). + - [Project access](public_access/public_access.md): Setting up your project's visibility to public, internal, or private. + - [GitLab Pages](user/project/pages/index.md): Build, test, and deploy your static website with GitLab Pages. +- [Groups](user/group/index.md): Organize your projects in groups. + - [Subgroups](user/group/subgroups/index.md) - [Search through GitLab](user/search/index.md): Search for issues, merge requests, projects, groups, todos, and issues in Issue Boards. - [Snippets](user/snippets.md): Snippets allow you to create little bits of code. - [Wikis](user/project/wiki/index.md): Enhance your repository documentation with built-in wikis. -- [GitLab Pages](user/project/pages/index.md): Build, test, and deploy your static website with GitLab Pages. ### Repository -Manage files and branches from the UI (user interface): +Manage your [repositories](user/project/repository/index.md) from the UI (user interface): - Files - [Create a file](user/project/repository/web_editor.md#create-a-file) @@ -81,6 +91,7 @@ Manage files and branches from the UI (user interface): - [Git](topics/git/index.md): Getting started with Git, branching strategies, Git LFS, advanced use. - [Git cheatsheet](https://gitlab.com/gitlab-com/marketing/raw/master/design/print/git-cheatsheet/print-pdf/git-cheatsheet.pdf): Download a PDF describing the most used Git operations. - [GitLab Flow](workflow/gitlab_flow.md): explore the best of Git with the GitLab Flow strategy. +- [Signing commits](workflow/gpg_signed_commits/index.md): use GPG to sign your commits. ### Migrate and import your projects from other platforms @@ -104,6 +115,7 @@ Manage files and branches from the UI (user interface): - [Project Services](user/project/integrations/project_services.md): Integrate a project with external services, such as CI and chat. - [GitLab Integration](integration/README.md): Integrate with multiple third-party services with GitLab to allow external issue trackers and external authentication. +- [Trello Power-Up](integration/trello_power_up.md): Integrate with GitLab's Trello Power-Up ---- @@ -125,7 +137,7 @@ have access to GitLab administration tools and settings. - [Access restrictions](user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab - [Authentication/Authorization](topics/authentication/index.md#gitlab-administrators): Enforce 2FA, configure external authentication with LDAP, SAML, CAS and additional Omniauth providers. -### GitLab admins' superpowers +### Features - [Container Registry](administration/container_registry.md): Configure Docker Registry with GitLab. - [Custom Git hooks](administration/custom_hooks.md): Custom Git hooks (on the filesystem) for when webhooks aren't enough. @@ -155,6 +167,7 @@ have access to GitLab administration tools and settings. - [Operations](administration/operations.md): Keeping GitLab up and running. - [Polling](administration/polling.md): Configure how often the GitLab UI polls for updates. - [Request Profiling](administration/monitoring/performance/request_profiling.md): Get a detailed profile on slow requests. +- [Performance Bar](administration/monitoring/performance/performance_bar.md): Get performance information for the current page. ### Customization @@ -167,6 +180,7 @@ have access to GitLab administration tools and settings. ### Admin tools +- [Gitaly](administration/gitaly/index.md): Configuring Gitaly, GitLab's Git repository storage service - [Raketasks](raketasks/README.md): Backups, maintenance, automatic webhook setup and the importing of projects. - [Backup and restore](raketasks/backup_restore.md): Backup and restore your GitLab instance. - [Reply by email](administration/reply_by_email.md): Allow users to comment on issues and merge requests by replying to notification emails. diff --git a/doc/administration/auth/authentiq.md b/doc/administration/auth/authentiq.md index fb1a16b0f96..1528f1d2b17 100644 --- a/doc/administration/auth/authentiq.md +++ b/doc/administration/auth/authentiq.md @@ -32,7 +32,7 @@ Authentiq will generate a Client ID and the accompanying Client Secret for you t "app_id" => "YOUR_CLIENT_ID", "app_secret" => "YOUR_CLIENT_SECRET", "args" => { - scope: 'aq:name email~rs aq:push' + "scope": 'aq:name email~rs address aq:push' } } ] @@ -45,21 +45,20 @@ Authentiq will generate a Client ID and the accompanying Client Secret for you t app_id: 'YOUR_CLIENT_ID', app_secret: 'YOUR_CLIENT_SECRET', args: { - scope: 'aq:name email~rs aq:push' + scope: 'aq:name email~rs address aq:push' } } ``` 5. The `scope` is set to request the user's name, email (required and signed), and permission to send push notifications to sign in on subsequent visits. -See [OmniAuth Authentiq strategy](https://github.com/AuthentiqID/omniauth-authentiq#scopes-and-redirect-uri-configuration) for more information on scopes and modifiers. +See [OmniAuth Authentiq strategy](https://github.com/AuthentiqID/omniauth-authentiq/wiki/Scopes,-callback-url-configuration-and-responses) for more information on scopes and modifiers. 6. Change `YOUR_CLIENT_ID` and `YOUR_CLIENT_SECRET` to the Client credentials you received in step 1. 7. Save the configuration file. -8. [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../restart_gitlab.md#installations-from-source) - for the changes to take effect if you installed GitLab via Omnibus or from source respectively. +8. [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect if you installed GitLab via Omnibus or from source respectively. On the sign in page there should now be an Authentiq icon below the regular sign in form. Click the icon to begin the authentication process. diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index 725fc1f6076..a7395e03d1c 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -69,14 +69,42 @@ main: # 'main' is the GitLab 'provider ID' of this LDAP server # Example: 'ldap.mydomain.com' host: '_your_ldap_server' # This port is an example, it is sometimes different but it is always an integer and not a string - port: 389 + port: 389 # usually 636 for SSL uid: 'sAMAccountName' # This should be the attribute, not the value that maps to uid. - method: 'plain' # "tls" or "ssl" or "plain" # Examples: 'america\\momo' or 'CN=Gitlab Git,CN=Users,DC=mydomain,DC=com' bind_dn: '_the_full_dn_of_the_user_you_will_bind_with' password: '_the_password_of_the_bind_user' + # Encryption method. The "method" key is deprecated in favor of + # "encryption". + # + # Examples: "start_tls" or "simple_tls" or "plain" + # + # Deprecated values: "tls" was replaced with "start_tls" and "ssl" was + # replaced with "simple_tls". + # + encryption: 'plain' + + # Enables SSL certificate verification if encryption method is + # "start_tls" or "simple_tls". (Defaults to false for backward- + # compatibility) + verify_certificates: false + + # Specifies the path to a file containing a PEM-format CA certificate, + # e.g. if you need to use an internal CA. + # + # Example: '/etc/ca.pem' + # + ca_cert: '' + + # Specifies the SSL version for OpenSSL to use, if the OpenSSL default + # is not appropriate. + # + # Example: 'TLSv1_1' + # + ssl_version: '' + # Set a timeout, in seconds, for LDAP queries. This helps avoid blocking # a request if the LDAP server becomes unresponsive. # A value of 0 means there is no timeout. @@ -116,8 +144,8 @@ main: # 'main' is the GitLab 'provider ID' of this LDAP server # # Note: GitLab does not support omniauth-ldap's custom filter syntax. # - # Below an example for get only specific users - # Example: '(&(objectclass=user)(|(samaccountname=momo)(samaccountname=toto)))' + # Example for getting only specific users: + # '(&(objectclass=user)(|(samaccountname=momo)(samaccountname=toto)))' # user_filter: '' @@ -228,9 +256,14 @@ Tip: If you want to limit access to the nested members of an Active Directory group you can use the following syntax: ``` -(memberOf=CN=My Group,DC=Example,DC=com) +(memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com) ``` +Find more information about this "LDAP_MATCHING_RULE_IN_CHAIN" filter at +https://msdn.microsoft.com/en-us/library/aa746475(v=vs.85).aspx. Support for +nested members in the user filter should not be confused with +[group sync nested groups support (EE only)](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#supported-ldap-group-types-attributes). + Please note that GitLab does not support the custom filter syntax used by omniauth-ldap. @@ -245,6 +278,19 @@ In other words, if an existing GitLab user wants to enable LDAP sign-in for themselves, they should check that their GitLab email address matches their LDAP email address, and then sign into GitLab via their LDAP credentials. +## Encryption + +### TLS Server Authentication + +There are two encryption methods, `simple_tls` and `start_tls`. + +For either encryption method, if setting `validate_certificates: false`, TLS +encryption is established with the LDAP server before any LDAP-protocol data is +exchanged but no validation of the LDAP server's SSL certificate is performed. + +>**Note**: Before GitLab 9.5, `validate_certificates: false` is the default if +unspecified. + ## Limitations ### TLS Client Authentication @@ -254,14 +300,6 @@ You should disable anonymous LDAP authentication and enable simple or SASL authentication. The TLS client authentication setting in your LDAP server cannot be mandatory and clients cannot be authenticated with the TLS protocol. -### TLS Server Authentication - -Not supported by GitLab's configuration options. -When setting `method: ssl`, the underlying authentication method used by -`omniauth-ldap` is `simple_tls`. This method establishes TLS encryption with -the LDAP server before any LDAP-protocol data is exchanged but no validation of -the LDAP server's SSL certificate is performed. - ## Troubleshooting ### Debug LDAP user filter with ldapsearch @@ -301,9 +339,9 @@ tree and traverse it. ### Connection Refused If you are getting 'Connection Refused' errors when trying to connect to the -LDAP server please double-check the LDAP `port` and `method` settings used by -GitLab. Common combinations are `method: 'plain'` and `port: 389`, OR -`method: 'ssl'` and `port: 636`. +LDAP server please double-check the LDAP `port` and `encryption` settings used by +GitLab. Common combinations are `encryption: 'plain'` and `port: 389`, OR +`encryption: 'simple_tls'` and `port: 636`. ### Troubleshooting diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md index afafb6bf1f5..57e54815b68 100644 --- a/doc/administration/container_registry.md +++ b/doc/administration/container_registry.md @@ -112,7 +112,7 @@ GitLab from source respectively. >**Note:** Be careful to choose a port different than the one that Registry listens to (`5000` by default), -otherwise you will run into conflicts . +otherwise you will run into conflicts. --- @@ -465,23 +465,42 @@ on how to achieve that. ## Disable Container Registry but use GitLab as an auth endpoint -You can disable the embedded Container Registry to use an external one, but -still use GitLab as an auth endpoint. - **Omnibus GitLab** + +You can use GitLab as an auth endpoint and use a non-bundled Container Registry. + 1. Open `/etc/gitlab/gitlab.rb` and set necessary configurations: ```ruby - registry['enable'] = false gitlab_rails['registry_enabled'] = true gitlab_rails['registry_host'] = "registry.gitlab.example.com" gitlab_rails['registry_port'] = "5005" gitlab_rails['registry_api_url'] = "http://localhost:5000" - gitlab_rails['registry_key_path'] = "/var/opt/gitlab/gitlab-rails/certificate.key" gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry" gitlab_rails['registry_issuer'] = "omnibus-gitlab-issuer" ``` +1. A certificate keypair is required for GitLab and the Container Registry to + communicate securely. By default omnibus-gitlab will generate one keypair, + which is saved to `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key`. + When using an non-bundled Container Registry, you will need to supply a + custom certificate key. To do that, add the following to + `/etc/gitlab/gitlab.rb` + + ```ruby + gitlab_rails['registry_key_path'] = "/custom/path/to/registry-key.key" + # registry['internal_key'] should contain the contents of the custom key + # file. Line breaks in the key file should be marked using `\n` character + # Example: + registry['internal_key'] = "---BEGIN RSA PRIVATE KEY---\nMIIEpQIBAA\n" + ``` + + **Note:** The file specified at `registry_key_path` gets populated with the + content specified by `internal_key`, each time reconfigure is executed. If + no file is specified, omnibus-gitlab will default it to + `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key` and will populate + it. + 1. Save the file and [reconfigure GitLab][] for the changes to take effect. **Installations from source** diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 48929910a9c..5732b6a1ca4 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -2,8 +2,7 @@ [Gitaly](https://gitlab.com/gitlab-org/gitaly) (introduced in GitLab 9.0) is a service that provides high-level RPC access to Git -repositories. As of GitLab 9.3 it is still an optional component with -limited scope. +repositories. Gitaly is a mandatory component in GitLab 9.4 and newer. GitLab components that access Git repositories (gitlab-rails, gitlab-shell, gitlab-workhorse) act as clients to Gitaly. End users do @@ -33,36 +32,155 @@ prometheus_listen_addr = "localhost:9236" Changes to `/home/git/gitaly/config.toml` are applied when you run `service gitlab restart`. -## Configuring GitLab to not use Gitaly +## Running Gitaly on its own server -Gitaly is still an optional component in GitLab 9.3. This means you -can choose to not use it. +> This is an optional way to deploy Gitaly which can benefit GitLab +installations that are larger than a single machine. Most +installations will be better served with the default configuration +used by Omnibus and the GitLab source installation guide. -In Omnibus you can make the following change in -`/etc/gitlab/gitlab.rb` and reconfigure. This will both disable the -Gitaly service and configure the rest of GitLab not to use it. +Starting with GitLab 9.4 it is possible to run Gitaly on a different +server from the rest of the application. This can improve performance +when running GitLab with its repositories stored on an NFS server. + +At the moment (GitLab 9.4) Gitaly is not yet a replacement for NFS +because some parts of GitLab still bypass Gitaly when accessing Git +repositories. If you choose to deploy Gitaly on your NFS server you +must still also mount your Git shares on your GitLab application +servers. + +Gitaly network traffic is unencrypted so you should use a firewall to +restrict access to your Gitaly server. + +Below we describe how to configure a Gitaly server at address +`gitaly.internal:9999` with secret token `abc123secret`. We assume +your GitLab installation has two repository storages, `default` and +`storage1`. + +### Client side token configuration + +Start by configuring a token on the client side. + +Omnibus installations: ```ruby -gitaly['enable'] = false +# /etc/gitlab/gitlab.rb +gitlab_rails['gitaly_token'] = 'abc123secret' +``` + +Source installations: + +```yaml +# /home/git/gitlab/config/gitlab.yml +gitlab: + gitaly: + token: 'abc123secret' +``` + +You need to reconfigure (Omnibus) or restart (source) for these +changes to be picked up. + +### Gitaly server configuration + +Next, on the Gitaly server, we need to configure storage paths, enable +the network listener and configure the token. + +Note: if you want to reduce the risk of downtime when you enable +authentication you can temporarily disable enforcement, see [the +documentation on configuring Gitaly +authentication](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/configuration/README.md#authentication) +. + +In most or all cases the storage paths below end in `/repositories`. Check the +directory layout on your Gitaly server to be sure. + +Omnibus installations: + +```ruby +# /etc/gitlab/gitlab.rb +gitaly['listen_addr'] = '0.0.0.0:9999' +gitaly['auth_token'] = 'abc123secret' +gitaly['storage'] = [ + { 'name' => 'default', 'path' => '/path/to/default/repositories' }, + { 'name' => 'storage1', 'path' => '/path/to/storage1/repositories' }, +] +``` + +Source installations: + +```toml +# /home/git/gitaly/config.toml +listen_addr = '0.0.0.0:9999' + +[auth] +token = 'abc123secret' + +[[storage] +name = 'default' +path = '/path/to/default/repositories' + +[[storage]] +name = 'storage1' +path = '/path/to/storage1/repositories' ``` -In source installations, edit `/home/git/gitlab/config/gitlab.yml` and -make sure `enabled` in the `gitaly` section is set to 'false'. This -does not disable the Gitaly service in your init script; it only -prevents it from being used. +Again, reconfigure (Omnibus) or restart (source). + +### Converting clients to use the Gitaly server -Apply the change with `service gitlab restart`. +Now as the final step update the client machines to switch from using +their local Gitaly service to the new Gitaly server you just +configured. This is a risky step because if there is any sort of +network, firewall, or name resolution problem preventing your GitLab +server from reaching the Gitaly server then all Gitaly requests will +fail. + +We assume that your Gitaly server can be reached at +`gitaly.internal:9999` from your GitLab server, and that your GitLab +NFS shares are mounted at `/mnt/gitlab/default` and +`/mnt/gitlab/storage1` respectively. + +Omnibus installations: + +```ruby +# /etc/gitlab/gitlab.rb +git_data_dirs({ + { 'default' => { 'path' => '/mnt/gitlab/default', 'gitaly_address' => 'tcp://gitlab.internal:9999' } }, + { 'storage1' => { 'path' => '/mnt/gitlab/storage1', 'gitaly_address' => 'tcp://gitlab.internal:9999' } }, +}) + +gitlab_rails['gitaly_token'] = 'abc123secret' +``` + +Source installations: ```yaml +# /home/git/gitlab/config/gitlab.yml +gitlab: + repositories: + storages: + default: + path: /mnt/gitlab/default/repositories + gitaly_address: tcp://gitlab.internal:9999 + storage1: + path: /mnt/gitlab/storage1/repositories + gitaly_address: tcp://gitlab.internal:9999 + gitaly: - enabled: false + token: 'abc123secret' ``` +Now reconfigure (Omnibus) or restart (source). When you tail the +Gitaly logs on your Gitaly server (`sudo gitlab-ctl tail gitaly` or +`tail -f /home/git/gitlab/log/gitaly.log`) you should see requests +coming in. One sure way to trigger a Gitaly request is to clone a +repository from your GitLab server over HTTP. + ## Disabling or enabling the Gitaly service -Be careful: if you disable Gitaly without instructing the rest of your -GitLab installation not to use Gitaly, you may end up with errors -because GitLab tries to access a service that is not running. +If you are running Gitaly [as a remote +service](#running-gitaly-on-its-own-server) you may want to disable +the local Gitaly service that runs on your Gitlab server by default. To disable the Gitaly service in your Omnibus installation, add the following line to `/etc/gitlab/gitlab.rb`: @@ -81,4 +199,4 @@ following to `/etc/default/gitlab`: gitaly_enabled=false ``` -When you run `service gitlab restart` Gitaly will be disabled.
\ No newline at end of file +When you run `service gitlab restart` Gitaly will be disabled. diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index da9687aa849..ca6d8d2de67 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -97,9 +97,12 @@ If you use a cloud-managed service, or provide your own PostgreSQL: Enter new password: Enter it again: ``` - -1. Enable the `pg_trgm` extension: +1. Exit from editing `template1` prompt by typing `\q` and Enter. +1. Enable the `pg_trgm` extension within the `gitlabhq_production` database: + ``` + gitlab-psql -d gitlabhq_production + CREATE EXTENSION pg_trgm; # Output: diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md index 137fed35a73..42666357faf 100644 --- a/doc/administration/high_availability/gitlab.md +++ b/doc/administration/high_availability/gitlab.md @@ -5,7 +5,7 @@ configure the GitLab application server(s) now. Complete the steps below for each GitLab application server in your environment. > **Note:** There is some additional configuration near the bottom for - secondary GitLab application servers. It's important to read and understand + additional GitLab application servers. It's important to read and understand these additional steps before proceeding with GitLab installation. 1. If necessary, install the NFS client utility packages using the following @@ -70,10 +70,16 @@ for each GitLab application server in your environment. gitlab_rails['redis_host'] = '10.1.0.6' # IP/hostname of Redis server gitlab_rails['redis_password'] = 'Redis Password' ``` + + > **Note:** To maintain uniformity of links across HA clusters, the `external_url` + on the first application server as well as the additional application + servers should point to the external url that users will use to access GitLab. + In a typical HA setup, this will be the url of the load balancer which will + route traffic to all GitLab application servers in the HA cluster. 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. -## Primary GitLab application server +## First GitLab application server As a final step, run the setup rake task on the first GitLab application server. It is not necessary to run this on additional application servers. @@ -89,10 +95,10 @@ It is not necessary to run this on additional application servers. [Nginx documentation](http://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) for more information. -## Additional configuration for secondary GitLab application servers +## Extra configuration for additional GitLab application servers -Secondary GitLab servers (servers configured **after** the first GitLab server) -need some additional configuration. +Additional GitLab servers (servers configured **after** the first GitLab server) +need some extra configuration. 1. Configure shared secrets. These values can be obtained from the primary GitLab server in `/etc/gitlab/gitlab-secrets.json`. Add these to diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index d8e76d6ab94..90a2e9298bf 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -1,12 +1,35 @@ # NFS -## Required NFS Server features +You can view information and options set for each of the mounted NFS file +systems by running `sudo nfsstat -m`. + +## NFS Server features + +### Required features **File locking**: GitLab **requires** advisory file locking, which is only supported natively in NFS version 4. NFSv3 also supports locking as long as Linux Kernel 2.6.5+ is used. We recommend using version 4 and do not specifically test NFSv3. +### Recommended options + +When you define your NFS exports, we recommend you also add the following +options: + +- `no_root_squash` - NFS normally changes the `root` user to `nobody`. This is + a good security measure when NFS shares will be accessed by many different + users. However, in this case only GitLab will use the NFS share so it + is safe. GitLab recommends the `no_root_squash` setting because we need to + manage file permissions automatically. Without the setting you may receive + errors when the Omnibus package tries to alter permissions. Note that GitLab + and other bundled components do **not** run as `root` but as non-privileged + users. The recommendation for `no_root_squash` is to allow the Omnibus package + to set ownership and permissions on files, as needed. +- `sync` - Force synchronous behavior. Default is asynchronous and under certain + circumstances it could lead to data loss if a failure occurs before data has + synced. + ## AWS Elastic File System GitLab does not recommend using AWS Elastic File System (EFS). @@ -23,30 +46,17 @@ GitLab does not recommend using EFS with GitLab. many small files are written in a serialized manner are not well-suited for EFS. EBS with an NFS server on top will perform much better. +In addition, avoid storing GitLab log files (e.g. those in `/var/log/gitlab`) +because this will also affect performance. We recommend that the log files be +stored on a local volume. + For more details on another person's experience with EFS, see [Amazon's Elastic File System: Burst Credits](https://www.rawkode.io/2017/04/amazons-elastic-file-system-burst-credits/) -### Recommended options - -When you define your NFS exports, we recommend you also add the following -options: - -- `no_root_squash` - NFS normally changes the `root` user to `nobody`. This is - a good security measure when NFS shares will be accessed by many different - users. However, in this case only GitLab will use the NFS share so it - is safe. GitLab recommends the `no_root_squash` setting because we need to - manage file permissions automatically. Without the setting you may receive - errors when the Omnibus package tries to alter permissions. Note that GitLab - and other bundled components do **not** run as `root` but as non-privileged - users. The recommendation for `no_root_squash` is to allow the Omnibus package - to set ownership and permissions on files, as needed. -- `sync` - Force synchronous behavior. Default is asynchronous and under certain - circumstances it could lead to data loss if a failure occurs before data has - synced. - ## NFS Client mount options -Below is an example of an NFS mount point we use on GitLab.com: +Below is an example of an NFS mount point defined in `/etc/fstab` we use on +GitLab.com: ``` 10.1.1.1:/var/opt/gitlab/git-data /var/opt/gitlab/git-data nfs4 defaults,soft,rsize=1048576,wsize=1048576,noatime,nobootwait,lookupcache=positive 0 2 diff --git a/doc/administration/high_availability/redis_source.md b/doc/administration/high_availability/redis_source.md index 3629772b8af..8b7a515a076 100644 --- a/doc/administration/high_availability/redis_source.md +++ b/doc/administration/high_availability/redis_source.md @@ -4,6 +4,11 @@ This is the documentation for configuring a Highly Available Redis setup when you have installed Redis all by yourself and not using the bundled one that comes with the Omnibus packages. +Note also that you may elect to override all references to +`/home/git/gitlab/config/resque.yml` in accordance with the advanced Redis +settings outlined in +[Configuration Files Documentation](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/README.md). + We cannot stress enough the importance of reading the [Overview section](redis.md#overview) of the Omnibus Redis HA as it provides some invaluable information to the configuration of Redis. Please proceed to @@ -364,3 +369,4 @@ When in doubt, please read [Redis Sentinel documentation](http://redis.io/topics [downloads]: https://about.gitlab.com/downloads [restart]: ../restart_gitlab.md#installations-from-source [it]: https://gitlab.com/gitlab-org/gitlab-ce/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png +[resque]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/resque.yml.example diff --git a/doc/administration/monitoring/ip_whitelist.md b/doc/administration/monitoring/ip_whitelist.md new file mode 100644 index 00000000000..ad2773de132 --- /dev/null +++ b/doc/administration/monitoring/ip_whitelist.md @@ -0,0 +1,39 @@ +# IP whitelist + +> Introduced in GitLab 9.4. + +GitLab provides some [monitoring endpoints] that provide health check information +when probed. + +To control access to those endpoints via IP whitelisting, you can add single +hosts or use IP ranges: + +**For Omnibus installations** + +1. Open `/etc/gitlab/gitlab.rb` and add or uncomment the following: + + ```ruby + gitlab_rails['monitoring_whitelist'] = ['127.0.0.0/8', '192.168.0.1'] + ``` + +1. Save the file and [reconfigure] GitLab for the changes to take effect. + +--- + +**For installations from source** + +1. Edit `config/gitlab.yml`: + + ```yaml + monitoring: + # by default only local IPs are allowed to access monitoring resources + ip_whitelist: + - 127.0.0.0/8 + - 192.168.0.1 + ``` + +1. Save the file and [restart] GitLab for the changes to take effect. + +[reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure +[restart]: ../restart_gitlab.md#installations-from-source +[monitoring endpoints]: ../../user/admin_area/monitoring/health_check.md diff --git a/doc/administration/monitoring/performance/img/performance_bar.png b/doc/administration/monitoring/performance/img/performance_bar.png Binary files differnew file mode 100644 index 00000000000..b3c6bc474e3 --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar.png diff --git a/doc/administration/monitoring/performance/img/performance_bar_configuration_settings.png b/doc/administration/monitoring/performance/img/performance_bar_configuration_settings.png Binary files differnew file mode 100644 index 00000000000..2d64ef8c5fc --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar_configuration_settings.png diff --git a/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png b/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png Binary files differnew file mode 100644 index 00000000000..7868e2c46d1 --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png diff --git a/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png b/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png Binary files differnew file mode 100644 index 00000000000..372ae021f6b --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md new file mode 100644 index 00000000000..ee680c7b258 --- /dev/null +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -0,0 +1,35 @@ +# Performance Bar + +A Performance Bar can be displayed, to dig into the performance of a page. When +activated, it looks as follows: + + + +It allows you to: + +- see the current host serving the page +- see the timing of the page (backend, frontend) +- the number of DB queries, the time it took, and the detail of these queries + +- the number of calls to Redis, and the time it took +- the number of background jobs created by Sidekiq, and the time it took +- the number of Ruby GC calls, and the time it took +- profile the code used to generate the page, line by line + + +## Enable the Performance Bar via the Admin panel + +GitLab Performance Bar is disabled by default. To enable it for a given group, +navigate to the Admin area in **Settings > Profiling - Performance Bar** +(`/admin/application_settings`). + +The only required setting you need to set is the full path of the group that +will be allowed to display the Performance Bar. +Make sure _Enable the Performance Bar_ is checked and hit +**Save** to save the changes. + +--- + + + +--- diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 07c05b5a6fb..6baae20d16a 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -1,10 +1,8 @@ # GitLab Prometheus metrics >**Note:** -Available since [Omnibus GitLab 9.3][29118]. Currently experimental. For installations from source -you'll have to configure it yourself. - -GitLab monitors its own internal service metrics, and makes them available at the `/-/metrics` endpoint. Unlike other [Prometheus] exporters, this endpoint requires authentication as it is available on the same URL and port as user traffic. +Available since [Omnibus GitLab 9.3][29118]. Currently experimental. For +installations from source you'll have to configure it yourself. To enable the GitLab Prometheus metrics: @@ -15,33 +13,57 @@ To enable the GitLab Prometheus metrics: ## Collecting the metrics -Since the metrics endpoint is available on the same host and port as other traffic, it requires authentication. The token and URL to access is displayed on the [Health Check][health-check] page. +GitLab monitors its own internal service metrics, and makes them available at the +`/-/metrics` endpoint. Unlike other [Prometheus] exporters, in order to access +it, the client IP needs to be [included in a whitelist][whitelist]. -Currently the embedded Prometheus server is not automatically configured to collect metrics from this endpoint. We recommend setting up another Prometheus server, because the embedded server configuration is overwritten one every reconfigure of GitLab. In the future this will not be required. +Currently the embedded Prometheus server is not automatically configured to +collect metrics from this endpoint. We recommend setting up another Prometheus +server, because the embedded server configuration is overwritten once every +[reconfigure of GitLab][reconfigure]. In the future this will not be required. ## Metrics available In this experimental phase, only a few metrics are available: -| Metric | Type | Description | -| ------ | ---- | ----------- | -| db_ping_timeout | Gauge | Whether or not the last database ping timed out | -| db_ping_success | Gauge | Whether or not the last database ping succeeded | -| db_ping_latency | Gauge | Round trip time of the database ping | -| redis_ping_timeout | Gauge | Whether or not the last redis ping timed out | -| redis_ping_success | Gauge | Whether or not the last redis ping succeeded | -| redis_ping_latency | Gauge | Round trip time of the redis ping | -| filesystem_access_latency | gauge | Latency in accessing a specific filesystem | -| filesystem_accessible | gauge | Whether or not a specific filesystem is accessible | -| filesystem_write_latency | gauge | Write latency of a specific filesystem | -| filesystem_writable | gauge | Whether or not the filesystem is writable | -| filesystem_read_latency | gauge | Read latency of a specific filesystem | -| filesystem_readable | gauge | Whether or not the filesystem is readable | -| user_sessions_logins | Counter | Counter of how many users have logged in | +| Metric | Type | Since | Description | +|:--------------------------------- |:--------- |:----- |:----------- | +| db_ping_timeout | Gauge | 9.4 | Whether or not the last database ping timed out | +| db_ping_success | Gauge | 9.4 | Whether or not the last database ping succeeded | +| db_ping_latency_seconds | Gauge | 9.4 | Round trip time of the database ping | +| filesystem_access_latency_seconds | Gauge | 9.4 | Latency in accessing a specific filesystem | +| filesystem_accessible | Gauge | 9.4 | Whether or not a specific filesystem is accessible | +| filesystem_write_latency_seconds | Gauge | 9.4 | Write latency of a specific filesystem | +| filesystem_writable | Gauge | 9.4 | Whether or not the filesystem is writable | +| filesystem_read_latency_seconds | Gauge | 9.4 | Read latency of a specific filesystem | +| filesystem_readable | Gauge | 9.4 | Whether or not the filesystem is readable | +| http_requests_total | Counter | 9.4 | Rack request count | +| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware | +| pipelines_created_total | Counter | 9.4 | Counter of pipelines created | +| rack_uncaught_errors_total | Counter | 9.4 | Rack connections handling uncaught errors count | +| redis_ping_timeout | Gauge | 9.4 | Whether or not the last redis ping timed out | +| redis_ping_success | Gauge | 9.4 | Whether or not the last redis ping succeeded | +| redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | +| user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | + +## Metrics shared directory + +GitLab's Prometheus client requires a directory to store metrics data shared between multi-process services. +Those files are shared among all instances running under Unicorn server. +The directory needs to be accessible to all running Unicorn's processes otherwise +metrics will not function correctly. + +For best performance its advisable that this directory will be located in `tmpfs`. + +Its location is configured using environment variable `prometheus_multiproc_dir`. + +If GitLab is installed using Omnibus and `tmpfs` is available then metrics +directory will be automatically configured. [← Back to the main Prometheus page](index.md) [29118]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29118 [Prometheus]: https://prometheus.io [restart]: ../../restart_gitlab.md#omnibus-gitlab-restart -[health-check]: ../../../user/admin_area/monitoring/health_check.md +[whitelist]: ../ip_whitelist.md +[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index 695fdf09a87..f43c89dad87 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -95,8 +95,9 @@ Sample Prometheus queries: ## Configuring Prometheus to monitor Kubernetes > Introduced in GitLab 9.0. +> Pod monitoring introduced in GitLab 9.4. -If your GitLab server is running within Kubernetes, Prometheus will collect metrics from the Nodes in the cluster including performance data on each container. This is particularly helpful if your CI/CD environments run in the same cluster, as you can use the [Prometheus project integration][] to monitor them. +If your GitLab server is running within Kubernetes, Prometheus will collect metrics from the Nodes and [annotated Pods](https://prometheus.io/docs/operating/configuration/#<kubernetes_sd_config>) in the cluster, including performance data on each container. This is particularly helpful if your CI/CD environments run in the same cluster, as you can use the [Prometheus project integration][] to monitor them. To disable the monitoring of Kubernetes: diff --git a/doc/administration/operations/cleaning_up_redis_sessions.md b/doc/administration/operations/cleaning_up_redis_sessions.md index 93521e976d5..3a35aff8366 100644 --- a/doc/administration/operations/cleaning_up_redis_sessions.md +++ b/doc/administration/operations/cleaning_up_redis_sessions.md @@ -15,6 +15,12 @@ prefixed with 'session:gitlab:', so they would look like 'session:gitlab:976aa289e2189b17d7ef525a6702ace9'. Below we describe how to remove the keys in the old format. +**Note:** the instructions below must be modified in accordance with your +configuration settings if you have used the advanced Redis +settings outlined in +[Configuration Files Documentation](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/README.md). + + First we define a shell function with the proper Redis connection details. ``` diff --git a/doc/api/README.md b/doc/api/README.md index b7f6ee69193..fe29563eaca 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -17,6 +17,7 @@ following locations: - [Deploy Keys](deploy_keys.md) - [Environments](environments.md) - [Events](events.md) +- [Feature flags](features.md) - [Gitignores templates](templates/gitignores.md) - [GitLab CI Config templates](templates/gitlab_ci_ymls.md) - [Groups](groups.md) @@ -28,7 +29,8 @@ following locations: - [Keys](keys.md) - [Labels](labels.md) - [Merge Requests](merge_requests.md) -- [Milestones](milestones.md) +- [Project milestones](milestones.md) +- [Group milestones](group_milestones.md) - [Namespaces](namespaces.md) - [Notes](notes.md) (comments) - [Notification settings](notification_settings.md) @@ -338,7 +340,18 @@ URL-encoded. For example, `/` is represented by `%2F`: ``` -/api/v4/projects/diaspora%2Fdiaspora +GET /api/v4/projects/diaspora%2Fdiaspora +``` + +## Branches & tags name encoding + +If your branch or tag contains a `/`, make sure the branch/tag name is +URL-encoded. + +For example, `/` is represented by `%2F`: + +``` +GET /api/v4/projects/1/branches/my%2Fbranch/commits ``` ## `id` vs `iid` diff --git a/doc/api/ci/lint.md b/doc/api/ci/lint.md index 6a4dca92cfe..e4a6dc809b1 100644 --- a/doc/api/ci/lint.md +++ b/doc/api/ci/lint.md @@ -47,3 +47,5 @@ Example responses: "error": "content is missing" } ``` + +[ce-5953]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5953 diff --git a/doc/api/features.md b/doc/api/features.md index 89b8d3ac948..6861dbf00a2 100644 --- a/doc/api/features.md +++ b/doc/api/features.md @@ -1,4 +1,4 @@ -# Features API +# Features flags API All methods require administrator authorization. @@ -58,6 +58,11 @@ POST /features/:name | --------- | ---- | -------- | ----------- | | `name` | string | yes | Name of the feature to create or update | | `value` | integer/string | yes | `true` or `false` to enable/disable, or an integer for percentage of time | +| `feature_group` | string | no | A Feature group name | +| `user` | string | no | A GitLab username | + +Note that you can enable or disable a feature for both a `feature_group` and a +`user` with a single API call. ```bash curl --data "value=30" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/features/new_library diff --git a/doc/api/group_milestones.md b/doc/api/group_milestones.md new file mode 100644 index 00000000000..086fba7e91d --- /dev/null +++ b/doc/api/group_milestones.md @@ -0,0 +1,120 @@ +# Group milestones API + +## List group milestones + +Returns a list of group milestones. + +``` +GET /groups/:id/milestones +GET /groups/:id/milestones?iids=42 +GET /groups/:id/milestones?iids[]=42&iids[]=43 +GET /groups/:id/milestones?state=active +GET /groups/:id/milestones?state=closed +GET /groups/:id/milestones?search=version +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `iids` | Array[integer] | optional | Return only the milestones having the given `iids` | +| `state` | string | optional | Return only `active` or `closed` milestones` | +| `search` | string | optional | Return only milestones with a title or description matching the provided string | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/5/milestones +``` + +Example Response: + +```json +[ + { + "id": 12, + "iid": 3, + "group_id": 16, + "title": "10.0", + "description": "Version", + "due_date": "2013-11-29", + "start_date": "2013-11-10", + "state": "active", + "updated_at": "2013-10-02T09:24:18Z", + "created_at": "2013-10-02T09:24:18Z" + } +] +``` + + +## Get single milestone + +Gets a single group milestone. + +``` +GET /groups/:id/milestones/:milestone_id +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `milestone_id` (required) - The ID of the group milestone + +## Create new milestone + +Creates a new group milestone. + +``` +POST /groups/:id/milestones +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `title` (required) - The title of an milestone +- `description` (optional) - The description of the milestone +- `due_date` (optional) - The due date of the milestone +- `start_date` (optional) - The start date of the milestone + +## Edit milestone + +Updates an existing group milestone. + +``` +PUT /groups/:id/milestones/:milestone_id +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `milestone_id` (required) - The ID of a group milestone +- `title` (optional) - The title of a milestone +- `description` (optional) - The description of a milestone +- `due_date` (optional) - The due date of the milestone +- `start_date` (optional) - The start date of the milestone +- `state_event` (optional) - The state event of the milestone (close|activate) + +## Get all issues assigned to a single milestone + +Gets all issues assigned to a single group milestone. + +``` +GET /groups/:id/milestones/:milestone_id/issues +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `milestone_id` (required) - The ID of a group milestone + +## Get all merge requests assigned to a single milestone + +Gets all merge requests assigned to a single group milestone. + +``` +GET /groups/:id/milestones/:milestone_id/merge_requests +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user +- `milestone_id` (required) - The ID of a group milestone diff --git a/doc/api/issues.md b/doc/api/issues.md index df5666bb7b6..6bac2927339 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -14,7 +14,9 @@ Read more on [pagination](README.md#pagination). ## List issues -Get all issues created by the authenticated user. +Get all issues the authenticated user has access to. By default it +returns only issues created by the current user. To get all issues, +use parameter `scope=all`. ``` GET /issues @@ -26,7 +28,8 @@ GET /issues?labels=foo,bar&state=opened GET /issues?milestone=1.0.0 GET /issues?milestone=1.0.0&state=opened GET /issues?iids[]=42&iids[]=43 -GET /issues?search=issue+title+or+description +GET /issues?author_id=5 +GET /issues?assignee_id=5 ``` | Attribute | Type | Required | Description | @@ -34,9 +37,12 @@ GET /issues?search=issue+title+or+description | `state` | string | no | Return all issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | | `milestone` | string | no | The milestone title | +| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me`. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | | `iids` | Array[integer] | no | Return only the issues having the given `iid` | -| `order_by` | string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | -| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | +| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | | `search` | string | no | Search issues against their `title` and `description` | ```bash @@ -117,6 +123,8 @@ GET /groups/:id/issues?milestone=1.0.0 GET /groups/:id/issues?milestone=1.0.0&state=opened GET /groups/:id/issues?iids[]=42&iids[]=43 GET /groups/:id/issues?search=issue+title+or+description +GET /groups/:id/issues?author_id=5 +GET /groups/:id/issues?assignee_id=5 ``` | Attribute | Type | Required | Description | @@ -126,9 +134,12 @@ GET /groups/:id/issues?search=issue+title+or+description | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | | `iids` | Array[integer] | no | Return only the issues having the given `iid` | | `milestone` | string | no | The milestone title | -| `order_by` | string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | -| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | -| `search` | string | no | Search group issues against their `title` and `description` | +| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Search group issues against their `title` and `description` | ```bash @@ -209,6 +220,8 @@ GET /projects/:id/issues?milestone=1.0.0 GET /projects/:id/issues?milestone=1.0.0&state=opened GET /projects/:id/issues?iids[]=42&iids[]=43 GET /projects/:id/issues?search=issue+title+or+description +GET /projects/:id/issues?author_id=5 +GET /projects/:id/issues?assignee_id=5 ``` | Attribute | Type | Required | Description | @@ -218,9 +231,12 @@ GET /projects/:id/issues?search=issue+title+or+description | `state` | string | no | Return all issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | | `milestone` | string | no | The milestone title | -| `order_by` | string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | -| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | -| `search` | string | no | Search project issues against their `title` and `description` | +| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Search project issues against their `title` and `description` | | `created_after` | datetime | no | Return issues created after the given time (inclusive) | | `created_before` | datetime | no | Return issues created before the given time (inclusive) | @@ -356,7 +372,13 @@ Example response: "user_notes_count": 1, "due_date": null, "web_url": "http://example.com/example/example/issues/1", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -418,7 +440,13 @@ Example response: "user_notes_count": 0, "due_date": null, "web_url": "http://example.com/example/example/issues/14", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -481,7 +509,13 @@ Example response: "user_notes_count": 0, "due_date": "2016-07-22", "web_url": "http://example.com/example/example/issues/15", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -567,7 +601,13 @@ Example response: }, "due_date": null, "web_url": "http://example.com/example/example/issues/11", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -632,7 +672,13 @@ Example response: }, "due_date": null, "web_url": "http://example.com/example/example/issues/11", - "confidential": false + "confidential": false, + "_links": { + "self": "http://example.com/api/v4/projects/1/issues/2", + "notes": "http://example.com/api/v4/projects/1/issues/2/notes", + "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", + "project": "http://example.com/api/v4/projects/1" + } } ``` @@ -964,3 +1010,32 @@ Example response: ## Comments on issues Comments are done via the [notes](notes.md) resource. + +## Get user agent details + +Available only for admins. + +``` +GET /projects/:id/issues/:issue_iid/user_agent_detail +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `issue_iid` | integer | yes | The internal ID of a project's issue | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/93/user_agent_detail +``` + +Example response: + +```json +{ + "user_agent": "AppleWebKit/537.36", + "ip_address": "127.0.0.1", + "akismet_submitted": false +} +``` + +[ce-13004]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13004 diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 3dc808c196d..d0725b5e06e 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -1,7 +1,104 @@ # Merge requests API +Every API call to merge requests must be authenticated. + ## List merge requests +> [Introduced][ce-13060] in GitLab 9.5. + +Get all merge requests the authenticated user has access to. By +default it returns only merge requests created by the current user. To +get all merge requests, use parameter `scope=all`. + +The `state` parameter can be used to get only merge requests with a +given state (`opened`, `closed`, or `merged`) or all of them (`all`). +The pagination parameters `page` and `per_page` can be used to +restrict the list of merge requests. + +``` +GET /merge_requests +GET /merge_requests?state=opened +GET /merge_requests?state=all +GET /merge_requests?milestone=release +GET /merge_requests?labels=bug,reproduced +GET /merge_requests?author_id=5 +GET /merge_requests?scope=assigned-to-me +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, or `merged`| +| `order_by`| string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | +| `milestone` | string | no | Return merge requests for a specific milestone | +| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | +| `labels` | string | no | Return merge requests matching a comma separated list of labels | +| `created_after` | datetime | no | Return merge requests created after the given time (inclusive) | +| `created_before` | datetime | no | Return merge requests created before the given time (inclusive) | +| `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` | +| `author_id` | integer | no | Returns merge requests created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me` | +| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` | + +```json +[ + { + "id": 1, + "iid": 1, + "target_branch": "master", + "source_branch": "test1", + "project_id": 3, + "title": "test1", + "state": "opened", + "upvotes": 0, + "downvotes": 0, + "author": { + "id": 1, + "username": "admin", + "email": "admin@example.com", + "name": "Administrator", + "state": "active", + "created_at": "2012-04-29T08:46:00Z" + }, + "assignee": { + "id": 1, + "username": "admin", + "email": "admin@example.com", + "name": "Administrator", + "state": "active", + "created_at": "2012-04-29T08:46:00Z" + }, + "source_project_id": 2, + "target_project_id": 3, + "labels": [ ], + "description": "fixed login page css paddings", + "work_in_progress": false, + "milestone": { + "id": 5, + "iid": 1, + "project_id": 3, + "title": "v2.0", + "description": "Assumenda aut placeat expedita exercitationem labore sunt enim earum.", + "state": "closed", + "created_at": "2015-02-02T19:49:26.013Z", + "updated_at": "2015-02-02T19:49:26.013Z", + "due_date": null + }, + "merge_when_pipeline_succeeds": true, + "merge_status": "can_be_merged", + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null, + "user_notes_count": 1, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1" + } +] +``` + +## List project merge requests + Get all merge requests for this project. The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. @@ -25,9 +122,13 @@ Parameters: | `order_by`| string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | | `milestone` | string | no | Return merge requests for a specific milestone | +| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | | `labels` | string | no | Return merge requests matching a comma separated list of labels | | `created_after` | datetime | no | Return merge requests created after the given time (inclusive) | | `created_before` | datetime | no | Return merge requests created before the given time (inclusive) | +| `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13060] in GitLab 9.5)_ | +| `author_id` | integer | no | Returns merge requests created by the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | ```json [ @@ -1164,3 +1265,5 @@ Example response: "total_time_spent": 3600 } ``` + +[ce-13060]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13060 diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index 4ad6071a0ed..8133251dffe 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -29,22 +29,30 @@ Example response: { "id": 1, "path": "user1", - "kind": "user" + "kind": "user", + "full_path": "user1" }, { "id": 2, "path": "group1", - "kind": "group" + "kind": "group", + "full_path": "group1", + "parent_id": "null", + "members_count_with_descendants": 2 }, { "id": 3, "path": "bar", "kind": "group", "full_path": "foo/bar", + "parent_id": "9", + "members_count_with_descendants": 5 } ] ``` +**Note**: `members_count_with_descendants` are presented only for group masters/owners. + ## Search for namespace Get all namespaces that match a string in their name or path. @@ -72,6 +80,8 @@ Example response: "path": "twitter", "kind": "group", "full_path": "twitter", + "parent_id": "null", + "members_count_with_descendants": 2 } ] ``` diff --git a/doc/api/oauth2.md b/doc/api/oauth2.md index 07cb64cb373..77bb7a55d8c 100644 --- a/doc/api/oauth2.md +++ b/doc/api/oauth2.md @@ -1,59 +1,55 @@ # GitLab as an OAuth2 provider -This document covers using the OAuth2 protocol to access GitLab. +This document covers using the [OAuth2](https://oauth.net/2/) protocol to allow other services access Gitlab resources on user's behalf. -If you want GitLab to be an OAuth authentication service provider to sign into other services please see the [Oauth2 provider documentation](../integration/oauth_provider.md). +If you want GitLab to be an OAuth authentication service provider to sign into other services please see the [OAuth2 provider](../integration/oauth_provider.md) +documentation. -OAuth2 is a protocol that enables us to authenticate a user without requiring them to give their password to a third-party. +This functionality is based on [doorkeeper gem](https://github.com/doorkeeper-gem/doorkeeper). -This functionality is based on [doorkeeper gem](https://github.com/doorkeeper-gem/doorkeeper) +## Supported OAuth2 Flows -## Web Application Flow +Gitlab currently supports following authorization flows: -This is the most common type of flow and is used by server-side clients that wish to access GitLab on a user's behalf. +* *Web Application Flow* - Most secure and common type of flow, designed for the applications with secure server-side. +* *Implicit Flow* - This flow is designed for user-agent only apps (e.g. single page web application running on GitLab Pages). +* *Resource Owner Password Credentials Flow* - To be used **only** for securely hosted, first-party services. ->**Note:** -This flow **should not** be used for client-side clients as you would leak your `client_secret`. Client-side clients should use the Implicit Grant (which is currently unsupported). +Please refer to [OAuth RFC](https://tools.ietf.org/html/rfc6749) to find out in details how all those flows work and pick the right one for your use case. -For more detailed information, check out the [RFC spec](http://tools.ietf.org/html/rfc6749#section-4.1) +Both *web application* and *implicit* flows require `application` to be registered first via `/profile/applications` page +in your user's account. During registration, by enabling proper scopes you can limit the range of resources which the `application` can access. Upon creation +you'll obtain `application` credentials: _Application ID_ and _Client Secret_ - **keep them secure**. -In the following sections you will be introduced to the three steps needed for this flow. +>**Important:** OAuth specification advises sending `state` parameter with each request to `/oauth/authorize`. We highly recommended to send a unique +value with each request and validate it against the one in redirect request. This is important to prevent [CSRF attacks]. The `state` param really should +have been a requirement in the standard! -### 1. Registering the client +In the following sections you will find detailed instructions on how to obtain authorization with each flow. -First, you should create an application (`/profile/applications`) in your user's account. -Each application gets a unique App ID and App Secret parameters. +### Web Application Flow ->**Note:** -**You should not share/leak your App ID or App Secret.** +Check [RFC spec](http://tools.ietf.org/html/rfc6749#section-4.1) for a detailed flow description -### 2. Requesting authorization +#### 1. Requesting authorization code -To request the authorization code, you should redirect the user to the `/oauth/authorize` endpoint: +To request the authorization code, you should redirect the user to the `/oauth/authorize` endpoint with following GET parameters: ``` -https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=code&state=your_unique_state_hash +https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=code&state=YOUR_UNIQUE_STATE_HASH ``` -This will ask the user to approve the applications access to their account and then redirect back to the `REDIRECT_URI` you provided. +This will ask the user to approve the applications access to their account and then redirect back to the `REDIRECT_URI` you provided. The redirect will +include the GET `code` parameter, for example: -The redirect will include the GET `code` parameter, for example: - -``` -http://myapp.com/oauth/redirect?code=1234567890&state=your_unique_state_hash -``` +`http://myapp.com/oauth/redirect?code=1234567890&state=YOUR_UNIQUE_STATE_HASH` You should then use the `code` to request an access token. ->**Important:** -It is highly recommended that you send a `state` value with the request to `/oauth/authorize` and -validate that value is returned and matches in the redirect request. -This is important to prevent [CSRF attacks](http://www.oauthsecurity.com/#user-content-authorization-code-flow), -`state` really should have been a requirement in the standard! - -### 3. Requesting the access token +#### 2. Requesting access token -Once you have the authorization code you can request an `access_token` using the code, to do that you can use any HTTP client. In the following example, we are using Ruby's `rest-client`: +Once you have the authorization code you can request an `access_token` using the code, to do that you can use any HTTP client. In the following example, +we are using Ruby's `rest-client`: ``` parameters = 'client_id=APP_ID&client_secret=APP_SECRET&code=RETURNED_CODE&grant_type=authorization_code&redirect_uri=REDIRECT_URI' @@ -72,28 +68,40 @@ The `redirect_uri` must match the `redirect_uri` used in the original authorizat You can now make requests to the API with the access token returned. -### Use the access token to access the API -The access token allows you to make requests to the API on a behalf of a user. +### Implicit Grant + +Check [RFC spec](http://tools.ietf.org/html/rfc6749#section-4.2) for a detailed flow description. + +Unlike the web flow, the client receives an `access token` immediately as a result of the authorization request. The flow does not use client secret +or authorization code because all of the application code and storage is easily accessible, therefore __secrets__ can leak easily. + +>**Important:** Avoid using this flow for applications that store data outside of the Gitlab instance. If you do, make sure to verify `application id` +associated with access token before granting access to the data +(see [/oauth/token/info](https://github.com/doorkeeper-gem/doorkeeper/wiki/API-endpoint-descriptions-and-examples#get----oauthtokeninfo)). + + +#### 1. Requesting access token + +To request the access token, you should redirect the user to the `/oauth/authorize` endpoint using `token` response type: ``` -GET https://gitlab.example.com/api/v4/user?access_token=OAUTH-TOKEN +https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=token&state=YOUR_UNIQUE_STATE_HASH ``` -Or you can put the token to the Authorization header: +This will ask the user to approve the applications access to their account and then redirect back to the `REDIRECT_URI` you provided. The redirect +will include a fragment with `access_token` as well as token details in GET parameters, for example: ``` -curl --header "Authorization: Bearer OAUTH-TOKEN" https://gitlab.example.com/api/v4/user +http://myapp.com/oauth/redirect#access_token=ABCDExyz123&state=YOUR_UNIQUE_STATE_HASH&token_type=bearer&expires_in=3600 ``` -## Resource Owner Password Credentials +### Resource Owner Password Credentials -## Deprecation Notice +Check [RFC spec](http://tools.ietf.org/html/rfc6749#section-4.3) for a detailed flow description. -1. Starting in GitLab 8.11, the Resource Owner Password Credentials has been *disabled* for users with two-factor authentication turned on. -2. These users can access the API using [personal access tokens] instead. - ---- +> **Deprecation notice:** Starting in GitLab 8.11, the Resource Owner Password Credentials has been *disabled* for users with two-factor authentication +turned on. These users can access the API using [personal access tokens] instead. In this flow, a token is requested in exchange for the resource owner credentials (username and password). The credentials should only be used when there is a high degree of trust between the resource owner and the client (e.g. the @@ -101,12 +109,16 @@ client is part of the device operating system or a highly privileged application available (such as an authorization code). >**Important:** -Never store the users credentials and only use this grant type when your client is deployed to a trusted environment, in 99% of cases [personal access tokens] are a better choice. +Never store the users credentials and only use this grant type when your client is deployed to a trusted environment, in 99% of cases [personal access tokens] +are a better choice. Even though this grant type requires direct client access to the resource owner credentials, the resource owner credentials are used for a single request and are exchanged for an access token. This grant type can eliminate the need for the client to store the resource owner credentials for future use, by exchanging the credentials with a long-lived access token or refresh token. -You can do POST request to `/oauth/token` with parameters: + +#### 1. Requesting access token + +POST request to `/oauth/token` with parameters: ``` { @@ -134,4 +146,18 @@ access_token = client.password.get_token('user@example.com', 'secret') puts access_token.token ``` +## Access Gitlab API with `access token` + +The `access token` allows you to make requests to the API on a behalf of a user. You can pass the token either as GET parameter +``` +GET https://gitlab.example.com/api/v4/user?access_token=OAUTH-TOKEN +``` + +or you can put the token to the Authorization header: + +``` +curl --header "Authorization: Bearer OAUTH-TOKEN" https://gitlab.example.com/api/v4/user +``` + [personal access tokens]: ../user/profile/personal_access_tokens.md +[CSRF attacks]: http://www.oauthsecurity.com/#user-content-authorization-code-flow
\ No newline at end of file diff --git a/doc/api/project_snippets.md b/doc/api/project_snippets.md index 92491de4daa..d74398c6e65 100644 --- a/doc/api/project_snippets.md +++ b/doc/api/project_snippets.md @@ -119,3 +119,35 @@ Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user - `snippet_id` (required) - The ID of a project's snippet + +## Get user agent details + +> **Notes:** +> [Introduced][ce-29508] in GitLab 9.4. + + +Available only for admins. + +``` +GET /projects/:id/snippets/:snippet_id/user_agent_detail +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | Integer | yes | The ID of a snippet | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/1/snippets/1/user_agent_detail +``` + +Example response: + +```json +{ + "user_agent": "AppleWebKit/537.36", + "ip_address": "127.0.0.1", + "akismet_submitted": false +} +``` + +[ce-[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508 diff --git a/doc/api/projects.md b/doc/api/projects.md index cc1bb3911c8..d3f8e509612 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -99,6 +99,191 @@ Parameters: "repository_size": 1038090, "lfs_objects_size": 0, "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + }, + }, + { + "id": 6, + "description": null, + "default_branch": "master", + "visibility": "private", + "ssh_url_to_repo": "git@example.com:brightbox/puppet.git", + "http_url_to_repo": "http://example.com/brightbox/puppet.git", + "web_url": "http://example.com/brightbox/puppet", + "tag_list": [ + "example", + "puppet" + ], + "owner": { + "id": 4, + "name": "Brightbox", + "created_at": "2013-09-30T13:46:02Z" + }, + "name": "Puppet", + "name_with_namespace": "Brightbox / Puppet", + "path": "puppet", + "path_with_namespace": "brightbox/puppet", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "jobs_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": false, + "created_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "id": 4, + "name": "Brightbox", + "path": "brightbox", + "kind": "group", + "full_path": "brightbox" + }, + "import_status": "none", + "import_error": null, + "permissions": { + "project_access": { + "access_level": 10, + "notification_level": 3 + }, + "group_access": { + "access_level": 50, + "notification_level": 3 + } + }, + "archived": false, + "avatar_url": null, + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8547b1dc37721d05889db52fa2f02", + "public_jobs": true, + "shared_with_groups": [], + "only_allow_merge_if_pipeline_succeeds": false, + "only_allow_merge_if_all_discussions_are_resolved": false, + "request_access_enabled": false, + "statistics": { + "commit_count": 12, + "storage_size": 2066080, + "repository_size": 2066080, + "lfs_objects_size": 0, + "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } + } +] +``` + +### List a user's projects + +Get a list of visible projects for the given user. When accessed without authentication, only public projects are returned. + +``` +GET /users/:user_id/projects +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `user_id` | string | yes | The ID or username of the user | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of projects matching the search criteria | +| `simple` | boolean | no | Return only the ID, URL, name, and path of each project | +| `owned` | boolean | no | Limit by projects owned by the current user | +| `membership` | boolean | no | Limit by projects that the current user is a member of | +| `starred` | boolean | no | Limit by projects starred by the current user | +| `statistics` | boolean | no | Include project statistics | +| `with_issues_enabled` | boolean | no | Limit by enabled issues feature | +| `with_merge_requests_enabled` | boolean | no | Limit by enabled merge requests feature | + +```json +[ + { + "id": 4, + "description": null, + "default_branch": "master", + "visibility": "private", + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git", + "web_url": "http://example.com/diaspora/diaspora-client", + "tag_list": [ + "example", + "disapora client" + ], + "owner": { + "id": 3, + "name": "Diaspora", + "created_at": "2013-09-30T13:46:02Z" + }, + "name": "Diaspora Client", + "name_with_namespace": "Diaspora / Diaspora Client", + "path": "diaspora-client", + "path_with_namespace": "diaspora/diaspora-client", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "jobs_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": false, + "created_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "id": 3, + "name": "Diaspora", + "path": "diaspora", + "kind": "group", + "full_path": "diaspora" + }, + "import_status": "none", + "archived": false, + "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png", + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8547b1dc37721d05889db52fa2f02", + "public_jobs": true, + "shared_with_groups": [], + "only_allow_merge_if_pipeline_succeeds": false, + "only_allow_merge_if_all_discussions_are_resolved": false, + "request_access_enabled": false, + "statistics": { + "commit_count": 37, + "storage_size": 1038090, + "repository_size": 1038090, + "lfs_objects_size": 0, + "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" } }, { @@ -168,6 +353,15 @@ Parameters: "repository_size": 2066080, "lfs_objects_size": 0, "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" } } ] @@ -269,6 +463,15 @@ Parameters: "repository_size": 1038090, "lfs_objects_size": 0, "job_artifacts_size": 0 + }, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" } } ``` @@ -336,7 +539,7 @@ Parameters: | `snippets_enabled` | boolean | no | Enable snippets for this project | | `container_registry_enabled` | boolean | no | Enable container registry for this project | | `shared_runners_enabled` | boolean | no | Enable shared runners for this project | -| `visibility` | String | no | See [project visibility level](#project-visibility-level) | +| `visibility` | string | no | See [project visibility level](#project-visibility-level) | | `import_url` | string | no | URL to import repository from | | `public_jobs` | boolean | no | If `true`, jobs can be viewed by non-project-members | | `only_allow_merge_if_pipeline_succeeds` | boolean | no | Set whether merge requests can only be merged with successful jobs | @@ -346,6 +549,7 @@ Parameters: | `tag_list` | array | no | The list of tags for a project; put array of tags, that should be finally assigned to a project | | `avatar` | mixed | no | Image file for avatar of the project | | `printing_merge_request_link_enabled` | boolean | no | Show link to create/view merge request when pushing from the command line | +| `ci_config_path` | string | no | The path to CI config file | ### Create project for user @@ -382,6 +586,7 @@ Parameters: | `tag_list` | array | no | The list of tags for a project; put array of tags, that should be finally assigned to a project | | `avatar` | mixed | no | Image file for avatar of the project | | `printing_merge_request_link_enabled` | boolean | no | Show link to create/view merge request when pushing from the command line | +| `ci_config_path` | string | no | The path to CI config file | ### Edit project @@ -416,6 +621,7 @@ Parameters: | `request_access_enabled` | boolean | no | Allow users to request member access | | `tag_list` | array | no | The list of tags for a project; put array of tags, that should be finally assigned to a project | | `avatar` | mixed | no | Image file for avatar of the project | +| `ci_config_path` | string | no | The path to CI config file | ### Fork project @@ -498,7 +704,16 @@ Example response: "shared_with_groups": [], "only_allow_merge_if_pipeline_succeeds": false, "only_allow_merge_if_all_discussions_are_resolved": false, - "request_access_enabled": false + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } } ``` @@ -564,7 +779,16 @@ Example response: "shared_with_groups": [], "only_allow_merge_if_pipeline_succeeds": false, "only_allow_merge_if_all_discussions_are_resolved": false, - "request_access_enabled": false + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } } ``` @@ -648,7 +872,16 @@ Example response: "shared_with_groups": [], "only_allow_merge_if_pipeline_succeeds": false, "only_allow_merge_if_all_discussions_are_resolved": false, - "request_access_enabled": false + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } } ``` @@ -732,7 +965,16 @@ Example response: "shared_with_groups": [], "only_allow_merge_if_pipeline_succeeds": false, "only_allow_merge_if_all_discussions_are_resolved": false, - "request_access_enabled": false + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } } ``` @@ -1096,17 +1338,21 @@ endpoint can be accessed without authentication if the project is publicly accessible. ``` -GET /projects/search/:query +GET /projects ``` Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `query` | string | yes | A string contained in the project name | +| `search` | string | yes | A string contained in the project name | | `order_by` | string | no | Return requests ordered by `id`, `name`, `created_at` or `last_activity_at` fields | | `sort` | string | no | Return requests sorted in `asc` or `desc` order | +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects?search=test +``` + ## Start the Housekeeping task for a Project >**Note:** This feature was introduced in GitLab 9.0 diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index 18ceb8f779e..1fc577561a0 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -61,7 +61,7 @@ POST /projects/:id/repository/files/:file_path ``` ```bash -curl --request POST --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/app%2Fprojectrb%2E?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20content&commit_message=create%20a%20new%20file' +curl --request POST --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fprojectrb%2E?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20content&commit_message=create%20a%20new%20file' ``` Example response: @@ -90,7 +90,7 @@ PUT /projects/:id/repository/files/:file_path ``` ```bash -curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/app%2Fproject%2Erb?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20other%20content&commit_message=update%20file' +curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fproject%2Erb?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20other%20content&commit_message=update%20file' ``` Example response: @@ -129,7 +129,7 @@ DELETE /projects/:id/repository/files/:file_path ``` ```bash -curl --request DELETE --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/app%2Fproject%2Erb?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&commit_message=delete%20file' +curl --request DELETE --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fproject%2Erb?branch=master&author_email=author%40example.com&author_name=Firstname%20Lastname&commit_message=delete%20file' ``` Example response: diff --git a/doc/api/settings.md b/doc/api/settings.md index eefbdda42ce..94a9f8265fb 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -25,7 +25,7 @@ Example response: "id" : 1, "default_branch_protection" : 2, "restricted_visibility_levels" : [], - "signin_enabled" : true, + "password_authentication_enabled" : true, "after_sign_out_path" : null, "max_attachment_size" : 10, "user_oauth_applications" : true, @@ -42,7 +42,6 @@ Example response: "gravatar_enabled" : true, "sign_in_text" : null, "container_registry_token_expire_delay": 5, - "repository_storage": "default", "repository_storages": ["default"], "koding_enabled": false, "koding_url": null, @@ -63,7 +62,7 @@ PUT /application/settings | --------- | ---- | :------: | ----------- | | `default_projects_limit` | integer | no | Project limit per user. Default is `100000` | | `signup_enabled` | boolean | no | Enable registration. Default is `true`. | -| `signin_enabled` | boolean | no | Enable login via a GitLab account. Default is `true`. | +| `password_authentication_enabled` | boolean | no | Enable authentication via a GitLab account password. Default is `true`. | | `gravatar_enabled` | boolean | no | Enable Gravatar | | `sign_in_text` | string | no | Text on login page | | `home_page_url` | string | no | Redirect to this URL when not logged in | @@ -81,7 +80,6 @@ PUT /application/settings | `after_sign_out_path` | string | no | Where to redirect users after logout | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | | `repository_storages` | array of strings | no | A list of names of enabled storage paths, taken from `gitlab.yml`. New projects will be created in one of these stores, chosen at random. | -| `repository_storage` | string | no | The first entry in `repository_storages`. Deprecated, but retained for compatibility reasons | | `enabled_git_access_protocol` | string | no | Enabled protocols for Git access. Allowed values are: `ssh`, `http`, and `nil` to allow both protocols. | | `koding_enabled` | boolean | no | Enable Koding integration. Default is `false`. | | `koding_url` | string | yes (if `koding_enabled` is `true`) | The Koding instance URL for integration. | @@ -102,7 +100,7 @@ Example response: "id": 1, "default_projects_limit": 100000, "signup_enabled": true, - "signin_enabled": true, + "password_authentication_enabled": true, "gravatar_enabled": true, "sign_in_text": "", "created_at": "2015-06-12T15:51:55.432Z", @@ -121,7 +119,7 @@ Example response: "user_oauth_applications": true, "after_sign_out_path": "", "container_registry_token_expire_delay": 5, - "repository_storage": "default", + "repository_storages": ["default"], "koding_enabled": false, "koding_url": null, "plantuml_enabled": false, diff --git a/doc/api/snippets.md b/doc/api/snippets.md index efaab712367..fdafbfb5b9e 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -234,3 +234,35 @@ Example response: } ] ``` + +## Get user agent details + +> **Notes:** +> [Introduced][ce-29508] in GitLab 9.4. + + +Available only for admins. + +``` +GET /snippets/:id/user_agent_detail +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | Integer | yes | The ID of a snippet | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/snippets/1/user_agent_detail +``` + +Example response: + +```json +{ + "user_agent": "AppleWebKit/537.36", + "ip_address": "127.0.0.1", + "akismet_submitted": false +} +``` + +[ce-[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508 diff --git a/doc/api/users.md b/doc/api/users.md index cf09b8f44aa..57a13eb477d 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -146,6 +146,12 @@ GET /users?extern_uid=1234567&provider=github You can search for users who are external with: `/users?external=true` +You can search users by creation date time range with: + +``` +GET /users?created_before=2001-01-02T00:00:00.060Z&created_after=1999-01-02T00:00:00.060 +``` + ## Single user Get a single user. @@ -235,26 +241,26 @@ POST /users Parameters: -- `email` (required) - Email -- `password` (optional) - Password -- `reset_password` (optional) - Send user password reset link - true or false(default) -- `username` (required) - Username -- `name` (required) - Name -- `skype` (optional) - Skype ID -- `linkedin` (optional) - LinkedIn -- `twitter` (optional) - Twitter account -- `website_url` (optional) - Website URL -- `organization` (optional) - Organization name -- `projects_limit` (optional) - Number of projects user can create -- `extern_uid` (optional) - External UID -- `provider` (optional) - External provider name -- `bio` (optional) - User's biography -- `location` (optional) - User's location -- `admin` (optional) - User is admin - true or false (default) -- `can_create_group` (optional) - User can create groups - true or false -- `confirm` (optional) - Require confirmation - true (default) or false -- `external` (optional) - Flags the user as external - true or false(default) -- `avatar` (optional) - Image file for user's avatar +- `email` (required) - Email +- `password` (optional) - Password +- `reset_password` (optional) - Send user password reset link - true or false(default) +- `username` (required) - Username +- `name` (required) - Name +- `skype` (optional) - Skype ID +- `linkedin` (optional) - LinkedIn +- `twitter` (optional) - Twitter account +- `website_url` (optional) - Website URL +- `organization` (optional) - Organization name +- `projects_limit` (optional) - Number of projects user can create +- `extern_uid` (optional) - External UID +- `provider` (optional) - External provider name +- `bio` (optional) - User's biography +- `location` (optional) - User's location +- `admin` (optional) - User is admin - true or false (default) +- `can_create_group` (optional) - User can create groups - true or false +- `skip_confirmation` (optional) - Skip confirmation - true or false (default) +- `external` (optional) - Flags the user as external - true or false(default) +- `avatar` (optional) - Image file for user's avatar ## User modification @@ -358,7 +364,7 @@ GET /user Parameters: -- `sudo` (required) - the ID of a user +- `sudo` (optional) - the ID of a user to make the call in their place ``` GET /user diff --git a/doc/api/v3_to_v4.md b/doc/api/v3_to_v4.md index 9db8e0351cf..9835fab7c98 100644 --- a/doc/api/v3_to_v4.md +++ b/doc/api/v3_to_v4.md @@ -2,9 +2,11 @@ Since GitLab 9.0, API V4 is the preferred version to be used. -API V3 will be removed in GitLab 9.5, to be released on August 22, 2017. In the -meantime, we advise you to make any necessary changes to applications that use -V3. The V3 API documentation is still [available](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-16-stable/doc/api/README.md). +API V3 will be unsupported from GitLab 9.5, to be released on August +22, 2017. It will be removed in GitLab 9.5 or later. In the meantime, we advise +you to make any necessary changes to applications that use V3. The V3 API +documentation is still +[available](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-16-stable/doc/api/README.md). Below are the changes made between V3 and V4. diff --git a/doc/articles/how_to_configure_ldap_gitlab_ce/index.md b/doc/articles/how_to_configure_ldap_gitlab_ce/index.md index 6892905dd94..130e8f542b4 100644 --- a/doc/articles/how_to_configure_ldap_gitlab_ce/index.md +++ b/doc/articles/how_to_configure_ldap_gitlab_ce/index.md @@ -120,7 +120,8 @@ gitlab_rails['ldap_servers'] = { 'host' => 'ad.example.org', 'port' => 636, 'uid' => 'sAMAccountName', - 'method' => 'ssl', + 'encryption' => 'simple_tls', + 'verify_certificates' => true, 'bind_dn' => 'CN=GitLabSRV,CN=Users,DC=GitLab,DC=org', 'password' => 'Password1', 'active_directory' => true, @@ -255,7 +256,7 @@ If `allow_username_or_email_login` is enabled in the LDAP configuration, GitLab ## LDAP extended features on GitLab EE -With [GitLab Enterprise Edition (EE)](https://about.gitlab.com/giltab-ee/), besides everything we just described, you'll +With [GitLab Enterprise Edition (EE)](https://about.gitlab.com/gitlab-ee/), besides everything we just described, you'll have extended functionalities with LDAP, such as: - Group sync diff --git a/doc/articles/index.md b/doc/articles/index.md index 342fa88e80f..a4e41517d83 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -11,6 +11,7 @@ They are written by members of the GitLab Team and by - **LDAP** - [How to configure LDAP with GitLab CE](how_to_configure_ldap_gitlab_ce/index.md) + - [How to configure LDAP with GitLab EE](https://docs.gitlab.com/ee/articles/how_to_configure_ldap_gitlab_ee/) ## Git @@ -23,3 +24,23 @@ They are written by members of the GitLab Team and by - [Part 2: Quick start guide - Setting up GitLab Pages](../user/project/pages/getting_started_part_two.md) - [Part 3: Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](../user/project/pages/getting_started_part_three.md) - [Part 4: Creating and tweaking `.gitlab-ci.yml` for GitLab Pages](../user/project/pages/getting_started_part_four.md) +- [Building a new GitLab Docs site with Nanoc, GitLab CI, and GitLab Pages](https://about.gitlab.com/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) +- [GitLab CI: Deployment & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) + +## Sofware development + +- [In 13 minutes from Kubernetes to a complete application development tool](https://about.gitlab.com/2016/11/14/idea-to-production/) +- [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) +- [Fast and Natural Continuous Integration with GitLab CI](https://about.gitlab.com/2017/05/22/fast-and-natural-continuous-integration-with-gitlab-ci/) +- [GitLab Workflow, an Overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) +- [Continuous Integration, Delivery, and Deployment with GitLab](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) + +## Build, test, and deploy with GitLab CI/CD + +**Build, test, and deploy** the software you develop with **[GitLab CI/CD](../ci/README.md)** + +- [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) +- [Automated Debian Package Build with GitLab CI](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) +- [Building an Elixir Release into a Docker image using GitLab CI](https://about.gitlab.com/2016/08/11/building-an-elixir-release-into-docker-image-using-gitlab-ci-part-1/) +- [Setting up GitLab CI for Android projects](https://about.gitlab.com/2016/11/30/setting-up-gitlab-ci-for-android-projects/) +- [How to use GitLab CI and MacStadium to build your macOS or iOS projects](https://about.gitlab.com/2017/05/15/how-to-use-macstadium-and-gitlab-ci-to-build-your-macos-or-ios-projects/) diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index be4dea55c20..d3433594eb7 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,4 +1,4 @@ -# Using Docker Images +# Using Docker images GitLab CI in conjunction with [GitLab Runner](../runners/README.md) can use [Docker Engine](https://www.docker.com/) to test and build any application. @@ -17,14 +17,16 @@ can also run on your workstation. The added benefit is that you can test all the commands that we will explore later from your shell, rather than having to test them on a dedicated CI server. -## Register docker runner +## Register Docker Runner -To use GitLab Runner with docker you need to register a new runner to use the -`docker` executor: +To use GitLab Runner with Docker you need to [register a new Runner][register] +to use the `docker` executor. + +A one-line example can be seen below: ```bash -gitlab-ci-multi-runner register \ - --url "https://gitlab.com/" \ +sudo gitlab-runner register \ + --url "https://gitlab.example.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ --description "docker-ruby-2.1" \ --executor "docker" \ @@ -33,26 +35,26 @@ gitlab-ci-multi-runner register \ --docker-mysql latest ``` -The registered runner will use the `ruby:2.1` docker image and will run two +The registered runner will use the `ruby:2.1` Docker image and will run two services, `postgres:latest` and `mysql:latest`, both of which will be accessible during the build process. ## What is an image -The `image` keyword is the name of the docker image the docker executor -will run to perform the CI tasks. +The `image` keyword is the name of the Docker image the Docker executor +will run to perform the CI tasks. -By default the executor will only pull images from [Docker Hub][hub], +By default, the executor will only pull images from [Docker Hub][hub], but this can be configured in the `gitlab-runner/config.toml` by setting -the [docker pull policy][] to allow using local images. +the [Docker pull policy][] to allow using local images. For more information about images and Docker Hub please read the [Docker Fundamentals][] documentation. ## What is a service -The `services` keyword defines just another docker image that is run during -your job and is linked to the docker image that the `image` keyword defines. +The `services` keyword defines just another Docker image that is run during +your job and is linked to the Docker image that the `image` keyword defines. This allows you to access the service image during build time. The service image can run any application, but the most common use case is to @@ -60,6 +62,11 @@ run a database container, eg. `mysql`. It's easier and faster to use an existing image and run it as an additional container than install `mysql` every time the project is built. +You are not limited to have only database services. You can add as many +services you need to `.gitlab-ci.yml` or manually modify `config.toml`. +Any image found at [Docker Hub][hub] or your private Container Registry can be +used as a service. + You can see some widely used services examples in the relevant documentation of [CI services examples](../services/README.md). @@ -73,22 +80,49 @@ then be used to create a container that is linked to the job container. The service container for MySQL will be accessible under the hostname `mysql`. So, in order to access your database service you have to connect to the host -named `mysql` instead of a socket or `localhost`. +named `mysql` instead of a socket or `localhost`. Read more in [accessing the +services](#accessing-the-services). -## Overwrite image and services +### Accessing the services -See [How to use other images as services](#how-to-use-other-images-as-services). +Let's say that you need a Wordpress instance to test some API integration with +your application. -## How to use other images as services +You can then use for example the [tutum/wordpress][] image in your +`.gitlab-ci.yml`: -You are not limited to have only database services. You can add as many -services you need to `.gitlab-ci.yml` or manually modify `config.toml`. -Any image found at [Docker Hub][hub] can be used as a service. +```yaml +services: +- tutum/wordpress:latest +``` + +If you don't [specify a service alias](#available-settings-for-services-entry), +when the job is run, `tutum/wordpress` will be started and you will have +access to it from your build container under two hostnames to choose from: -## Define image and services from `.gitlab-ci.yml` +- `tutum-wordpress` +- `tutum__wordpress` + +>**Note:** +Hostnames with underscores are not RFC valid and may cause problems in 3rd party +applications. + +The default aliases for the service's hostname are created from its image name +following these rules: + +- Everything after the colon (`:`) is stripped +- Slash (`/`) is replaced with double underscores (`__`) and the primary alias + is created +- Slash (`/`) is replaced with a single dash (`-`) and the secondary alias is + created (requires GitLab Runner v1.1.0 or higher) + +To override the default behavior, you can +[specify a service alias](#available-settings-for-services-entry). + +## Define `image` and `services` from `.gitlab-ci.yml` You can simply define an image that will be used for all jobs and a list of -services that you want to use during build time. +services that you want to use during build time: ```yaml image: ruby:2.2 @@ -125,6 +159,203 @@ test:2.2: - bundle exec rake spec ``` +Or you can pass some [extended configuration options](#extended-docker-configuration-options) +for `image` and `services`: + +```yaml +image: + name: ruby:2.2 + entrypoint: ["/bin/bash"] + +services: +- name: my-postgres:9.4 + alias: db-postgres + entrypoint: ["/usr/local/bin/db-postgres"] + command: ["start"] + +before_script: +- bundle install + +test: + script: + - bundle exec rake spec +``` + +## Extended Docker configuration options + +> **Note:** +This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. + +When configuring the `image` or `services` entries, you can use a string or a map as +options: + +- when using a string as an option, it must be the full name of the image to use + (including the Registry part if you want to download the image from a Registry + other than Docker Hub) +- when using a map as an option, then it must contain at least the `name` + option, which is the same name of the image as used for the string setting + +For example, the following two definitions are equal: + +1. Using a string as an option to `image` and `services`: + + ```yaml + image: "registry.example.com/my/image:latest" + + services: + - postgresql:9.4 + - redis:latest + ``` + +1. Using a map as an option to `image` and `services`. The use of `image:name` is + required: + + ```yaml + image: + name: "registry.example.com/my/image:latest" + + services: + - name: postgresql:9.4 + - name: redis:latest + ``` + +### Available settings for `image` + +> **Note:** +This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. + +| Setting | Required | Description | +|------------|----------|-------------| +| `name` | yes, when used with any other option | Full name of the image that should be used. It should contain the Registry part if needed. | +| `entrypoint` | no | Command or script that should be executed as the container's entrypoint. It will be translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`][entrypoint] directive, where each shell token is a separate string in the array. | + +### Available settings for `services` + +> **Note:** +This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. + +| Setting | Required | Description | +|------------|----------|-------------| +| `name` | yes, when used with any other option | Full name of the image that should be used. It should contain the Registry part if needed. | +| `entrypoint` | no | Command or script that should be executed as the container's entrypoint. It will be translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`][entrypoint] directive, where each shell token is a separate string in the array. | +| `command` | no | Command or script that should be used as the container's command. It will be translated to arguments passed to Docker after the image's name. The syntax is similar to [`Dockerfile`'s `CMD`][cmd] directive, where each shell token is a separate string in the array. | +| `alias` | no | Additional alias that can be used to access the service from the job's container. Read [Accessing the services](#accessing-the-services) for more information. | + +### Starting multiple services from the same image + +Before the new extended Docker configuration options, the following configuration +would not work properly: + +```yaml +services: +- mysql:latest +- mysql:latest +``` + +The Runner would start two containers using the `mysql:latest` image, but both +of them would be added to the job's container with the `mysql` alias based on +the [default hostname naming](#accessing-the-services). This would end with one +of the services not being accessible. + +After the new extended Docker configuration options, the above example would +look like: + +```yaml +services: +- name: mysql:latest + alias: mysql-1 +- name: mysql:latest + alias: mysql-2 +``` + +The Runner will still start two containers using the `mysql:latest` image, +but now each of them will also be accessible with the alias configured +in `.gitlab-ci.yml` file. + +### Setting a command for the service + +Let's assume you have a `super/sql:latest` image with some SQL database +inside it and you would like to use it as a service for your job. Let's also +assume that this image doesn't start the database process while starting +the container and the user needs to manually use `/usr/bin/super-sql run` as +a command to start the database. + +Before the new extended Docker configuration options, you would need to create +your own image based on the `super/sql:latest` image, add the default command, +and then use it in job's configuration, like: + +```Dockerfile +# my-super-sql:latest image's Dockerfile + +FROM super/sql:latest +CMD ["/usr/bin/super-sql", "run"] +``` + +```yaml +# .gitlab-ci.yml + +services: +- my-super-sql:latest +``` + +After the new extended Docker configuration options, you can now simply +set a `command` in `.gitlab-ci.yml`, like: + +```yaml +# .gitlab-ci.yml + +services: +- name: super/sql:latest + command: ["/usr/bin/super-sql", "run"] +``` + +As you can see, the syntax of `command` is similar to [Dockerfile's `CMD`][cmd]. + +### Overriding the entrypoint of an image + +Let's assume you have a `super/sql:experimental` image with some SQL database +inside it and you would like to use it as a base image for your job because you +want to execute some tests with this database binary. Let's also assume that +this image is configured with `/usr/bin/super-sql run` as an entrypoint. That +means, that when starting the container without additional options, it will run +the database's process, while Runner expects that the image will have no +entrypoint or at least will start with a shell as its entrypoint. + +Previously we would need to create our own image based on the +`super/sql:experimental` image, set the entrypoint to a shell, and then use +it in job's configuration, e.g.: + +Before the new extended Docker configuration options, you would need to create +your own image based on the `super/sql:experimental` image, set the entrypoint +to a shell and then use it in job's configuration, like: + +```Dockerfile +# my-super-sql:experimental image's Dockerfile + +FROM super/sql:experimental +ENTRYPOINT ["/bin/sh"] +``` + +```yaml +# .gitlab-ci.yml + +image: my-super-sql:experimental +``` + +After the new extended Docker configuration options, you can now simply +set an `entrypoint` in `.gitlab-ci.yml`, like: + +```yaml +# .gitlab-ci.yml + +image: + name: super/sql:experimental + entrypoint: ["/bin/sh"] +``` + +As you can see the syntax of `entrypoint` is similar to +[Dockerfile's `ENTRYPOINT`][entrypoint]. + ## Define image and services in `config.toml` Look for the `[runners.docker]` section: @@ -138,7 +369,7 @@ Look for the `[runners.docker]` section: The image and services defined this way will be added to all job run by that runner. -## Define an image from a private Docker registry +## Define an image from a private Container Registry > **Notes:** - This feature requires GitLab Runner **1.8** or higher @@ -193,44 +424,6 @@ To configure access for `registry.example.com`, follow these steps: You can add configuration for as many registries as you want, adding more registries to the `"auths"` hash as described above. -## Accessing the services - -Let's say that you need a Wordpress instance to test some API integration with -your application. - -You can then use for example the [tutum/wordpress][] image in your -`.gitlab-ci.yml`: - -```yaml -services: -- tutum/wordpress:latest -``` - -When the job is run, `tutum/wordpress` will be started and you will have -access to it from your build container under the hostnames `tutum-wordpress` -(requires GitLab Runner v1.1.0 or newer) and `tutum__wordpress`. - -When using a private registry, the image name also includes a hostname and port -of the registry. - -```yaml -services: -- docker.example.com:5000/wordpress:latest -``` - -The service hostname will also include the registry hostname. Service will be -available under hostnames `docker.example.com-wordpress` (requires GitLab Runner v1.1.0 or newer) -and `docker.example.com__wordpress`. - -*Note: hostname with underscores is not RFC valid and may cause problems in 3rd party applications.* - -The alias hostnames for the service are made from the image name following these -rules: - -1. Everything after `:` is stripped -2. Slash (`/`) is replaced with double underscores (`__`) - primary alias -3. Slash (`/`) is replaced with dash (`-`) - secondary alias, requires GitLab Runner v1.1.0 or newer - ## Configuring services Many services accept environment variables which allow you to easily change @@ -257,7 +450,7 @@ See the specific documentation for ## How Docker integration works -Below is a high level overview of the steps performed by docker during job +Below is a high level overview of the steps performed by Docker during job time. 1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. @@ -274,7 +467,7 @@ time. ## How to debug a job locally *Note: The following commands are run without root privileges. You should be -able to run docker with your regular user account.* +able to run Docker with your regular user account.* First start with creating a file named `build_script`: @@ -334,3 +527,6 @@ creation. [mysql-hub]: https://hub.docker.com/r/_/mysql/ [runner-priv-reg]: http://docs.gitlab.com/runner/configuration/advanced-configuration.html#using-a-private-container-registry [secret variable]: ../variables/README.md#secret-variables +[entrypoint]: https://docs.docker.com/engine/reference/builder/#entrypoint +[cmd]: https://docs.docker.com/engine/reference/builder/#cmd +[register]: https://docs.gitlab.com/runner/register/ diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 3393030210e..6a7f694d705 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -602,14 +602,12 @@ exist, you should see something like: >**Notes:** > - For the monitor dashboard to appear, you need to: - - Have enabled the [Kubernetes integration][kube] - - Have your app deployed on Kubernetes - Have enabled the [Prometheus integration][prom] + - Configured Prometheus to collect at least one [supported metric](../user/project/integrations/prometheus_library/metrics.md) - With GitLab 9.2, all deployments to an environment are shown directly on the monitoring dashboard -If your application is deployed on Kubernetes and you have enabled Prometheus -collecting metrics, you can monitor the performance behavior of your app +If you have enabled Prometheus for collecting metrics, you can monitor the performance behavior of your app through the environments. Once configured, GitLab will attempt to retrieve performance metrics for any diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 8b0d8a003fd..b9f0485290e 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -20,12 +20,12 @@ before_script: - php -r "unlink('composer-setup.php');" ``` -This will make sure we have all requirements ready. Next, we want to run `composer update` to fetch all PHP dependencies and `npm install` to load node packages, then run the `npm` script. We need to append them into `before_script` section: +This will make sure we have all requirements ready. Next, we want to run `composer install` to fetch all PHP dependencies and `npm install` to load node packages, then run the `npm` script. We need to append them into `before_script` section: ```yaml before_script: # ... - - php composer.phar update + - php composer.phar install - npm install - npm run deploy ``` @@ -133,7 +133,7 @@ before_script: - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" - php composer-setup.php - php -r "unlink('composer-setup.php');" - - php composer.phar update + - php composer.phar install - npm install - npm run deploy - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' diff --git a/doc/ci/img/environments_monitoring.png b/doc/ci/img/environments_monitoring.png Binary files differindex 387b6c54b61..d9c46ea4c95 100644 --- a/doc/ci/img/environments_monitoring.png +++ b/doc/ci/img/environments_monitoring.png diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 41cae58782d..88e53ff40e8 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -155,7 +155,7 @@ Find more information about different Runners in the [Runners](../runners/README.md) documentation. You can find whether any Runners are assigned to your project by going to -**Settings ➔ CI/CD Pipelines**. Setting up a Runner is easy and straightforward. The +**Settings ➔ Pipelines**. Setting up a Runner is easy and straightforward. The official Runner supported by GitLab is written in Go and its documentation can be found at <https://docs.gitlab.com/runner/>. @@ -168,7 +168,7 @@ Follow the links above to set up your own Runner or use a Shared Runner as described in the next section. Once the Runner has been set up, you should see it on the Runners page of your -project, following **Settings ➔ CI/CD Pipelines**. +project, following **Settings ➔ Pipelines**.  @@ -181,7 +181,7 @@ These are special virtual machines that run on GitLab's infrastructure and can build any project. To enable the **Shared Runners** you have to go to your project's -**Settings ➔ CI/CD Pipelines** and click **Enable shared runners**. +**Settings ➔ Pipelines** and click **Enable shared runners**. [Read more on Shared Runners](../runners/README.md). diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index befaa06e918..cf25a8b618f 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -34,9 +34,9 @@ instructions to [generate an SSH key](../../ssh/README.md). Do not add a passphrase to the SSH key, or the `before_script` will prompt for it. Then, create a new **Secret Variable** in your project settings on GitLab -following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY` -and in the **Value** field paste the content of your _private_ key that you -created earlier. +following **Settings > Pipelines** and look for the "Secret Variables" section. +As **Key** add the name `SSH_PRIVATE_KEY` and in the **Value** field paste the +content of your _private_ key that you created earlier. It is also good practice to check the server's own public key to make sure you are not being targeted by a man-in-the-middle attack. To do this, add another diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index d1f9881e51b..22e7f6879ed 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -9,8 +9,9 @@ and a list of **user-defined variables**. The variables can be overwritten and they take precedence over each other in this order: -1. [Trigger variables][triggers] (take precedence over all) -1. [Secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) +1. [Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables) (take precedence over all) +1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) +1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) 1. YAML-defined [job-level variables](../yaml/README.md#job-variables) 1. YAML-defined [global variables](../yaml/README.md#variables) 1. [Deployment variables](#deployment-variables) @@ -37,9 +38,10 @@ future GitLab releases.** |-------------------------------- |--------|--------|-------------| | **CI** | all | 0.4 | Mark that job is executed in CI environment | | **CI_COMMIT_REF_NAME** | 9.0 | all | The branch or tag name for which project is built | -| **CI_COMMIT_REF_SLUG** | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | +| **CI_COMMIT_REF_SLUG** | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | | **CI_COMMIT_SHA** | 9.0 | all | The commit revision for which project is built | | **CI_COMMIT_TAG** | 9.0 | 0.5 | The commit tag name. Present only when building tags. | +| **CI_CONFIG_PATH** | 9.4 | 0.5 | The path to CI config file. Defaults to `.gitlab-ci.yml` | | **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | | **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this job | | **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. | @@ -141,25 +143,30 @@ script: >**Notes:** - This feature requires GitLab Runner 0.4.0 or higher. +- Group-level secret variables added in GitLab 9.4. - Be aware that secret variables are not masked, and their values can be shown in the job logs if explicitly asked to do so. If your project is public or internal, you can set the pipelines private from your project's Pipelines settings. Follow the discussion in issue [#13784][ce-13784] for masking the secret variables. -GitLab CI allows you to define per-project **secret variables** that are set in -the build environment. The secret variables are stored out of the repository -(`.gitlab-ci.yml`) and are securely passed to GitLab Runner making them -available in the build environment. It's the recommended method to use for -storing things like passwords, secret keys and credentials. +GitLab CI allows you to define per-project or per-group **secret variables** +that are set in the build environment. The secret variables are stored out of +the repository (`.gitlab-ci.yml`) and are securely passed to GitLab Runner +making them available in the build environment. It's the recommended method to +use for storing things like passwords, secret keys and credentials. -Secret variables can be added by going to your project's -**Settings ➔ Pipelines**, then finding the section called -**Secret variables**. +Project-level secret variables can be added by going to your project's +**Settings ➔ Pipelines**, then finding the section called **Secret variables**. -Once you set them, they will be available for all subsequent pipelines. +Likewise, group-level secret variables can be added by going to your group's +**Settings ➔ Pipelines**, then finding the section called **Secret variables**. +Any variables of [subgroups] will be inherited recursively. + +Once you set them, they will be available for all subsequent pipelines. You can also +[protect your variables](#protected-secret-variables). -## Protected secret variables +### Protected secret variables >**Notes:** This feature requires GitLab 9.3 or higher. @@ -425,10 +432,12 @@ export CI_REGISTRY_PASSWORD="longalfanumstring" ``` [ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 -[runner]: https://docs.gitlab.com/runner/ -[triggered]: ../triggers/README.md -[triggers]: ../triggers/README.md#pass-job-variables-to-a-trigger +[eep]: https://about.gitlab.com/gitlab-ee/ "Available only in GitLab Enterprise Edition Premium" +[envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md [protected tags]: ../../user/project/protected_tags.md +[runner]: https://docs.gitlab.com/runner/ [shellexecutors]: https://docs.gitlab.com/runner/executors/ -[eep]: https://about.gitlab.com/gitlab-ee/ "Available only in GitLab Enterprise Edition Premium" +[triggered]: ../triggers/README.md +[triggers]: ../triggers/README.md#pass-job-variables-to-a-trigger +[subgroups]: ../../user/group/subgroups/index.md diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 8a0662db6fd..1869782fe6e 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -306,6 +306,53 @@ cache: untracked: true ``` +### cache:policy + +> Introduced in GitLab 9.4. + +The default behaviour of a caching job is to download the files at the start of +execution, and to re-upload them at the end. This allows any changes made by the +job to be persisted for future runs, and is known as the `pull-push` cache +policy. + +If you know the job doesn't alter the cached files, you can skip the upload step +by setting `policy: pull` in the job specification. Typically, this would be +twinned with an ordinary cache job at an earlier stage to ensure the cache +is updated from time to time: + +```yaml +stages: + - setup + - test + +prepare: + stage: setup + cache: + key: gems + paths: + - vendor/bundle + script: + - bundle install --deployment + +rspec: + stage: test + cache: + key: gems + paths: + - vendor/bundle + policy: pull + script: + - bundle exec rspec ... +``` + +This helps to speed up job execution and reduce load on the cache server, +especially when you have a large number of cache-using jobs executing in +parallel. + +Additionally, if you have a job that unconditionally recreates the cache without +reference to its previous contents, you can use `policy: push` in that job to +skip the download step. + ## Jobs `.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job @@ -348,6 +395,7 @@ job_name: | after_script | no | Override a set of commands that are executed after job | | environment | no | Defines a name of environment to which deployment is done by this job | | coverage | no | Define code coverage settings for a given job | +| retry | no | Define how many times a job can be auto-retried in case of a failure | ### script @@ -393,13 +441,25 @@ There are a few rules that apply to the usage of refs policy: * `only` and `except` are inclusive. If both `only` and `except` are defined in a job specification, the ref is filtered by `only` and `except`. * `only` and `except` allow the use of regular expressions. -* `only` and `except` allow the use of special keywords: -`api`, `branches`, `external`, `tags`, `pushes`, `schedules`, `triggers`, and `web` * `only` and `except` allow to specify a repository path to filter jobs for forks. +In addition, `only` and `except` allow the use of special keywords: + +| **Value** | **Description** | +| --------- | ---------------- | +| `branches` | When a branch is pushed. | +| `tags` | When a tag is pushed. | +| `api` | When pipeline has been triggered by a second pipelines API (not triggers API). | +| `external` | When using CI services other than GitLab. | +| `pipelines` | For multi-project triggers, created using the API with `CI_JOB_TOKEN`. | +| `pushes` | Pipeline is triggered by a `git push` by the user. | +| `schedules` | For [scheduled pipelines][schedules]. | +| `triggers` | For pipelines created using a trigger token. | +| `web` | For pipelines created using **Run pipeline** button in GitLab UI (under your project's **Pipelines**). | + In the example below, `job` will run only for refs that start with `issue-`, -whereas all branches will be skipped. +whereas all branches will be skipped: ```yaml job: @@ -412,7 +472,7 @@ job: ``` In this example, `job` will run only for refs that are tagged, or if a build is -explicitly requested via an API trigger or a [Pipeline Schedule](../../user/project/pipelines/schedules.md). +explicitly requested via an API trigger or a [Pipeline Schedule][schedules]: ```yaml job: @@ -1082,9 +1142,33 @@ A simple example: ```yaml job1: + script: rspec coverage: '/Code coverage: \d+\.\d+/' ``` +### retry + +**Notes:** +- [Introduced][ce-3442] in GitLab 9.5. + +`retry` allows you to configure how many times a job is going to be retried in +case of a failure. + +When a job fails, and has `retry` configured it is going to be processed again +up to the amount of times specified by the `retry` keyword. + +If `retry` is set to 2, and a job succeeds in a second run (first retry), it won't be retried +again. `retry` value has to be a positive integer, equal or larger than 0, but +lower or equal to 2 (two retries maximum, three runs in total). + +A simple example: + +```yaml +test: + script: rspec + retry: 2 +``` + ## Git Strategy > Introduced in GitLab 8.9 as an experimental feature. May change or be removed @@ -1459,3 +1543,5 @@ CI with various languages. [variables]: ../variables/README.md [ce-7983]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7983 [ce-7447]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7447 +[ce-3442]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3442 +[schedules]: ../../user/project/pipelines/schedules.md diff --git a/doc/development/README.md b/doc/development/README.md index 9496a87d84d..58993c52dcd 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -54,6 +54,8 @@ - [Polymorphic Associations](polymorphic_associations.md) - [Single Table Inheritance](single_table_inheritance.md) - [Background Migrations](background_migrations.md) +- [Storing SHA1 Hashes As Binary](sha1_as_binary.md) +- [Iterating Tables In Batches](iterating_tables_in_batches.md) ## i18n diff --git a/doc/development/architecture.md b/doc/development/architecture.md index acd5e3c2093..54029e00507 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -194,3 +194,7 @@ bundle exec rake gitlab:check RAILS_ENV=production ``` Note: It is recommended to log into the `git` user using `sudo -i -u git` or `sudo su - git`. While the sudo commands provided by gitlabhq work in Ubuntu they do not always work in RHEL. + +## GitLab.com + +We've also detailed [our architecture of GitLab.com](https://about.gitlab.com/handbook/infrastructure/production-architecture/) but this is probably over the top unless you have millions of users. diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index 0239e6b3163..e67db9ff142 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -31,6 +31,18 @@ It's also possible for different migrations to be executed at the same time. This means that different background migrations should not migrate data in a way that would cause conflicts. +## Idempotence + +Background migrations are executed in a context of a Sidekiq process. +Usual Sidekiq rules apply, especially the rule that jobs should be small +and idempotent. + +See [Sidekiq best practices guidelines](https://github.com/mperham/sidekiq/wiki/Best-Practices) +for more details. + +Make sure that in case that your migration job is going to be retried data +integrity is guarateed. + ## How It Works Background migrations are simple classes that define a `perform` method. A @@ -50,14 +62,13 @@ your migration: BackgroundMigrationWorker.perform_async('BackgroundMigrationClassName', [arg1, arg2, ...]) ``` -Usually it's better to schedule jobs in bulk, for this you can use +Usually it's better to enqueue jobs in bulk, for this you can use `BackgroundMigrationWorker.perform_bulk`: ```ruby BackgroundMigrationWorker.perform_bulk( - ['BackgroundMigrationClassName', [1]], - ['BackgroundMigrationClassName', [2]], - ... + [['BackgroundMigrationClassName', [1]], + ['BackgroundMigrationClassName', [2]]] ) ``` @@ -68,6 +79,16 @@ consuming migrations it's best to schedule a background job using an updates. Removals in turn can be handled by simply defining foreign keys with cascading deletes. +If you would like to schedule jobs in bulk with a delay, you can use +`BackgroundMigrationWorker.perform_bulk_in`: + +```ruby +jobs = [['BackgroundMigrationClassName', [1]], + ['BackgroundMigrationClassName', [2]]] + +BackgroundMigrationWorker.perform_bulk_in(5.minutes, jobs) +``` + ## Cleaning Up Because background migrations can take a long time you can't immediately clean @@ -203,3 +224,27 @@ end This migration will then process any jobs for the ExtractServicesUrl migration and continue once all jobs have been processed. Once done you can safely remove the `services.properties` column. + +## Testing + +It is required to write tests for background migrations' scheduling migration +(either a regular migration or a post deployment migration), background +migration itself and a cleanup migration. You can use the `:migration` RSpec +tag when testing a regular / post deployment migration. +See [README][migrations-readme]. + +When you do that, keep in mind that `before` and `after` RSpec hooks are going +to migrate you database down and up, which can result in other background +migrations being called. That means that using `spy` test doubles with +`have_received` is encouraged, instead of using regular test doubles, because +your expectations defined in a `it` block can conflict with what is being +called in RSpec hooks. See [gitlab-org/gitlab-ce#35351][issue-rspec-hooks] +for more details. + +## Best practices + +1. Make sure that background migration jobs are idempotent. +1. Make sure that tests you write are not false positives. + +[migrations-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md +[issue-rspec-hooks]: https://gitlab.com/gitlab-org/gitlab-ce/issues/35351 diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 4ed89146072..e3f37616757 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -133,6 +133,55 @@ reviewee. tomorrow. When you are not able to find the right balance, ask other people about their opinion. +### GitLab-specific concerns + +GitLab is used in a lot of places. Many users use +our [Omnibus packages](https://about.gitlab.com/installation/), but some use +the [Docker images](https://docs.gitlab.com/omnibus/docker/), some are +[installed from source](https://docs.gitlab.com/ce/install/installation.html), +and there are other installation methods available. GitLab.com itself is a large +Enterprise Edition instance. This has some implications: + +1. **Query changes** should be tested to ensure that they don't result in worse + performance at the scale of GitLab.com: + 1. Generating large quantities of data locally can help. + 2. Asking for query plans from GitLab.com is the most reliable way to validate + these. +2. **Database migrations** must be: + 1. Reversible. + 2. Performant at the scale of GitLab.com - ask a maintainer to test the + migration on the staging environment if you aren't sure. + 3. Categorised correctly: + - Regular migrations run before the new code is running on the instance. + - [Post-deployment migrations](post_deployment_migrations.md) run _after_ + the new code is deployed, when the instance is configured to do that. + - [Background migrations](background_migrations.md) run in Sidekiq, and + should only be done for migrations that would take an extreme amount of + time at GitLab.com scale. +3. **Sidekiq workers** + [cannot change in a backwards-incompatible way](sidekiq_style_guide.md#removing-or-renaming-queues): + 1. Sidekiq queues are not drained before a deploy happens, so there will be + workers in the queue from the previous version of GitLab. + 2. If you need to change a method signature, try to do so across two releases, + and accept both the old and new arguments in the first of those. + 3. Similarly, if you need to remove a worker, stop it from being scheduled in + one release, then remove it in the next. This will allow existing jobs to + execute. + 4. Don't forget, not every instance will upgrade to every intermediate version + (some people may go from X.1.0 to X.10.0, or even try bigger upgrades!), so + try to be liberal in accepting the old format if it is cheap to do so. +4. **Cached values** may persist across releases. If you are changing the type a + cached value returns (say, from a string or nil to an array), change the + cache key at the same time. +5. **Settings** should be added as a + [last resort](https://about.gitlab.com/handbook/product/#convention-over-configuration). + If you're adding a new setting in `gitlab.yml`: + 1. Try to avoid that, and add to `ApplicationSetting` instead. + 2. Ensure that it is also + [added to Omnibus](https://docs.gitlab.com/omnibus/settings/gitlab.yml.html#adding-a-new-setting-to-gitlab-yml). +6. **Filesystem access** can be slow, so try to avoid + [shared files](shared_files.md) when an alternative solution is available. + ### Credits Largely based on the [thoughtbot code review guide]. diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 5b09f79f143..90d1d9657b9 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -105,8 +105,8 @@ merge request. considered beta or experimental, put this info in a note, not in the heading. - When introducing a new document, be careful for the headings to be grammatically and syntactically correct. It is advised to mention one or all - of the following GitLab members for a review: `@axil`, `@rspeicher`, `@marcia`, - `@SeanPackham`. This is to ensure that no document with wrong heading is going + of the following GitLab members for a review: `@axil`, `@rspeicher`, `@marcia`. + This is to ensure that no document with wrong heading is going live without an audit, thus preventing dead links and redirection issues when corrected - Leave exactly one newline after a heading @@ -388,8 +388,8 @@ the style below as a guide: 1. Save the file and [restart] GitLab for the changes to take effect. -[reconfigure]: path/to/administration/gitlab_restart.md#omnibus-gitlab-reconfigure -[restart]: path/to/administration/gitlab_restart.md#installations-from-source +[reconfigure]: path/to/administration/restart_gitlab.md#omnibus-gitlab-reconfigure +[restart]: path/to/administration/restart_gitlab.md#installations-from-source ```` In this case: diff --git a/doc/development/fe_guide/droplab/plugins/input_setter.md b/doc/development/fe_guide/droplab/plugins/input_setter.md index a549603c20d..94f3c8254a8 100644 --- a/doc/development/fe_guide/droplab/plugins/input_setter.md +++ b/doc/development/fe_guide/droplab/plugins/input_setter.md @@ -1,6 +1,6 @@ # InputSetter -`InputSetter` is a plugin that allows for udating DOM out of the scope of droplab when a list item is clicked. +`InputSetter` is a plugin that allows for updating DOM out of the scope of droplab when a list item is clicked. ## Usage diff --git a/doc/development/fe_guide/performance.md b/doc/development/fe_guide/performance.md index 2ddcbe13afa..14ac1133cc0 100644 --- a/doc/development/fe_guide/performance.md +++ b/doc/development/fe_guide/performance.md @@ -23,6 +23,19 @@ controlled by the server. 1. The backend code will most likely be using etags. You do not and should not check for status `304 Not Modified`. The browser will transform it for you. +### Lazy Loading + +To improve the time to first render we are using lazy loading for images. This works by setting +the actual image source on the `data-src` attribute. After the HTML is rendered and JavaScript is loaded, +the value of `data-src` will be moved to `src` automatically if the image is in the current viewport. + +* Prepare images in HTML for lazy loading by renaming the `src` attribute to `data-src` AND adding the class `lazy` +* If you are using the Rails `image_tag` helper, all images will be lazy-loaded by default unless `lazy: false` is provided. + +If you are asynchronously adding content which contains lazy images then you need to call the function +`gl.lazyLoader.searchLazyImages()` which will search for lazy images and load them if needed. +But in general it should be handled automatically through a `MutationObserver` in the lazy loading function. + ## Reducing Asset Footprint ### Page-specific JavaScript diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index d2d89517241..149a0159680 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -447,6 +447,7 @@ A forEach will cause side effects, it will be mutating the array being iterated. 1. `name` 1. `props` 1. `mixins` + 1. `directives` 1. `data` 1. `components` 1. `computedProps` @@ -463,20 +464,24 @@ A forEach will cause side effects, it will be mutating the array being iterated. 1. `destroyed` #### Vue and Boostrap -1. Tooltips: Do not rely on `has-tooltip` class name for vue components +1. Tooltips: Do not rely on `has-tooltip` class name for Vue components ``` javascript // bad - <span class="has-tooltip"> + <span + class="has-tooltip" + title="Some tooltip text"> Text </span> // good - <span data-toggle="tooltip"> + <span + v-tooltip + title="Some tooltip text"> Text </span> ``` -1. Tooltips: When using a tooltip, include the tooltip mixin +1. Tooltips: When using a tooltip, include the tooltip directive, `./app/assets/javascripts/vue_shared/directives/tooltip.js` 1. Don't change `data-original-title`. ```javascript diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index a984bb6c94c..0742b202807 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -112,7 +112,50 @@ Vue Resource should only be imported in the service file. Vue.use(VueResource); ``` -### CSRF token +#### Vue-resource gotchas +#### Headers +Headers are being parsed into a plain object in an interceptor. +In Vue-resource 1.x `headers` object was changed into an `Headers` object. In order to not change all old code, an interceptor was added. + +If you need to write a unit test that takes the headers in consideration, you need to include an interceptor to parse the headers after your test interceptor. +You can see an example in `spec/javascripts/environments/environment_spec.js`: + ```javascript + import { headersInterceptor } from './helpers/vue_resource_helper'; + + beforeEach(() => { + Vue.http.interceptors.push(myInterceptor); + Vue.http.interceptors.push(headersInterceptor); + }); + + afterEach(() => { + Vue.http.interceptors = _.without(Vue.http.interceptors, myInterceptor); + Vue.http.interceptors = _.without(Vue.http.interceptors, headersInterceptor); + }); + ``` + +#### `.json()` +When making a request to the server, you will most likely need to access the body of the response. +Use `.json()` to convert. Because `.json()` returns a Promise the follwoing structure should be used: + + ```javascript + service.get('url') + .then(resp => resp.json()) + .then((data) => { + this.store.storeData(data); + }) + .catch(() => new Flash('Something went wrong')); + ``` + +When using `Poll` (`app/assets/javascripts/lib/utils/poll.js`), the `successCallback` needs to handle `.json()` as a Promise: + ```javascript + successCallback: (response) => { + return response.json().then((data) => { + // handle the response + }); + } + ``` + +#### CSRF token We use a Vue Resource interceptor to manage the CSRF token. `app/assets/javascripts/vue_shared/vue_resource_interceptor.js` holds all our common interceptors. Note: You don't need to load `app/assets/javascripts/vue_shared/vue_resource_interceptor.js` @@ -126,13 +169,13 @@ The following example shows an application: // store.js export default class Store { - /** + /** * This is where we will iniatialize the state of our data. * Usually in a small SPA you don't need any options when starting the store. In the case you do * need guarantee it's an Object and it's documented. - * - * @param {Object} options - */ + * + * @param {Object} options + */ constructor(options) { this.options = options; @@ -205,14 +248,14 @@ import Store from 'store'; import Service from 'service'; import TodoComponent from 'todoComponent'; export default { - /** + /** * Although most data belongs in the store, each component it's own state. * We want to show a loading spinner while we are fetching the todos, this state belong * in the component. * * We need to access the store methods through all methods of our component. * We need to access the state of our store. - */ + */ data() { const store = new Store(); @@ -396,42 +439,46 @@ need to test the rendered output. [Vue][vue-test] guide's to unit test show us e [Vue Resource Interceptors][vue-resource-interceptor] allow us to add a interceptor with the response we need: -```javascript - // Mock the service to return data - const interceptor = (request, next) => { - next(request.respondWith(JSON.stringify([{ - title: 'This is a todo', - body: 'This is the text' - }]), { - status: 200, - })); - }; + ```javascript + // Mock the service to return data + const interceptor = (request, next) => { + next(request.respondWith(JSON.stringify([{ + title: 'This is a todo', + body: 'This is the text' + }]), { + status: 200, + })); + }; - beforeEach(() => { - Vue.http.interceptors.push(interceptor); - }); + beforeEach(() => { + Vue.http.interceptors.push(interceptor); + }); - afterEach(() => { - Vue.http.interceptors = _.without(Vue.http.interceptors, interceptor); - }); + afterEach(() => { + Vue.http.interceptors = _.without(Vue.http.interceptors, interceptor); + }); - it('should do something', (done) => { - setTimeout(() => { - // Test received data - done(); - }, 0); - }); -``` + it('should do something', (done) => { + setTimeout(() => { + // Test received data + done(); + }, 0); + }); + ``` + +1. Headers interceptor +Refer to [this section](vue.md#headers) 1. Use `$.mount()` to mount the component + ```javascript - // bad - new Component({ - el: document.createElement('div') - }); +// bad +new Component({ + el: document.createElement('div') +}); - // good - new Component().$mount(); +// good +new Component().$mount(); ``` [vue-docs]: http://vuejs.org/guide/index.html diff --git a/doc/development/feature_flags.md b/doc/development/feature_flags.md index 5c6316b9ac6..59e8a087e02 100644 --- a/doc/development/feature_flags.md +++ b/doc/development/feature_flags.md @@ -3,5 +3,19 @@ Starting from GitLab 9.3 we support feature flags via [Flipper](https://github.com/jnunemaker/flipper/). You should use the `Feature` class (defined in `lib/feature.rb`) in your code to get, set and list feature -flags. During runtime you can set the values for the gates via the -[admin API](../api/features.md). +flags. + +During runtime you can set the values for the gates via the +[features API](../api/features.md) (accessible to admins only). + +## Feature groups + +Starting from GitLab 9.4 we support feature groups via +[Flipper groups](https://github.com/jnunemaker/flipper/blob/v0.10.2/docs/Gates.md#2-group). + +Feature groups must be defined statically in `lib/feature.rb` (in the +`.register_feature_groups` method), but their implementation can obviously be +dynamic (querying the DB etc.). + +Once defined in `lib/feature.rb`, you will be able to activate a +feature for a given feature group via the [`feature_group` param of the features API](../api/features.md#set-or-create-a-feature) diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index 565d4b33457..c2ca8966a3f 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -3,35 +3,6 @@ The purpose of this guide is to document potential "gotchas" that contributors might encounter or should avoid during development of GitLab CE and EE. -## Do not `describe` symbols - -Consider the following model spec: - -```ruby -require 'rails_helper' - -describe User do - describe :to_param do - it 'converts the username to a param' do - user = described_class.new(username: 'John Smith') - - expect(user.to_param).to eq 'john-smith' - end - end -end -``` - -When run, this spec doesn't do what we might expect: - -```sh -spec/models/user_spec.rb|6 error| Failure/Error: u = described_class.new NoMethodError: undefined method `new' for :to_param:Symbol -``` - -### Solution - -Except for the top-level `describe` block, always provide a String argument to -`describe`. - ## Do not assert against the absolute value of a sequence-generated attribute Consider the following factory: diff --git a/doc/development/iterating_tables_in_batches.md b/doc/development/iterating_tables_in_batches.md new file mode 100644 index 00000000000..590c8cbba2d --- /dev/null +++ b/doc/development/iterating_tables_in_batches.md @@ -0,0 +1,37 @@ +# Iterating Tables In Batches + +Rails provides a method called `in_batches` that can be used to iterate over +rows in batches. For example: + +```ruby +User.in_batches(of: 10) do |relation| + relation.update_all(updated_at: Time.now) +end +``` + +Unfortunately this method is implemented in a way that is not very efficient, +both query and memory usage wise. + +To work around this you can include the `EachBatch` module into your models, +then use the `each_batch` class method. For example: + +```ruby +class User < ActiveRecord::Base + include EachBatch +end + +User.each_batch(of: 10) do |relation| + relation.update_all(updated_at: Time.now) +end +``` + +This will end up producing queries such as: + +``` +User Load (0.7ms) SELECT "users"."id" FROM "users" WHERE ("users"."id" >= 41654) ORDER BY "users"."id" ASC LIMIT 1 OFFSET 1000 + (0.7ms) SELECT COUNT(*) FROM "users" WHERE ("users"."id" >= 41654) AND ("users"."id" < 42687) +``` + +The API of this method is similar to `in_batches`, though it doesn't support +all of the arguments that `in_batches` supports. You should always use +`each_batch` _unless_ you have a specific need for `in_batches`. diff --git a/doc/development/policies.md b/doc/development/policies.md new file mode 100644 index 00000000000..62141356f59 --- /dev/null +++ b/doc/development/policies.md @@ -0,0 +1,116 @@ +# `DeclarativePolicy` framework + +The DeclarativePolicy framework is designed to assist in performance of policy checks, and to enable ease of extension for EE. The DSL code in `app/policies` is what `Ability.allowed?` uses to check whether a particular action is allowed on a subject. + +The policy used is based on the subject's class name - so `Ability.allowed?(user, :some_ability, project)` will create a `ProjectPolicy` and check permissions on that. + +## Managing Permission Rules + +Permissions are broken into two parts: `conditions` and `rules`. Conditions are boolean expressions that can access the database and the environment, while rules are statically configured combinations of expressions and other rules that enable or prevent certain abilities. For an ability to be allowed, it must be enabled by at least one rule, and not prevented by any. + + +### Conditions + +Conditions are defined by the `condition` method, and are given a name and a block. The block will be executed in the context of the policy object - so it can access `@user` and `@subject`, as well as call any methods defined on the policy. Note that `@user` may be nil (in the anonymous case), but `@subject` is guaranteed to be a real instance of the subject class. + +``` ruby +class FooPolicy < BasePolicy + condition(:is_public) do + # @subject guaranteed to be an instance of Foo + @subject.public? + end + + # instance methods can be called from the condition as well + condition(:thing) { check_thing } + + def check_thing + # ... + end +end +``` + +When you define a condition, a predicate method is defined on the policy to check whether that condition passes - so in the above example, an instance of `FooPolicy` will also respond to `#is_public?` and `#thing?`. + +Conditions are cached according to their scope. Scope and ordering will be covered later. + +### Rules + +A `rule` is a logical combination of conditions and other rules, that are configured to enable or prevent certain abilities. It is important to note that the rule configuration is static - a rule's logic cannot touch the database or know about `@user` or `@subject`. This allows us to cache only at the condition level. Rules are specified through the `rule` method, which takes a block of DSL configuration, and returns an object that responds to `#enable` or `#prevent`: + +``` ruby +class FooPolicy < BasePolicy + # ... + + rule { is_public }.enable :read + rule { thing }.prevent :read + + # equivalently, + rule { is_public }.policy do + enable :read + end + + rule { ~thing }.policy do + prevent :read + end +end +``` + +Within the rule DSL, you can use: + +* A regular word mentions a condition by name - a rule that is in effect when that condition is truthy. +* `~` indicates negation +* `&` and `|` are logical combinations, also available as `all?(...)` and `any?(...)` +* `can?(:other_ability)` delegates to the rules that apply to `:other_ability`. Note that this is distinct from the instance method `can?`, which can check dynamically - this only configures a delegation to another ability. + +## Scores, Order, Performance + +To see how the rules get evaluated into a judgment, it is useful in a console to use `policy.debug(:some_ability)`. This will print the rules in the order they are evaluated. + +When a policy is asked whether a particular ability is allowed (`policy.allowed?(:some_ability)`), it does not necessarily have to compute all the conditions on the policy. First, only the rules relevant to that particular ability are selected. Then, the execution model takes advantage of short-circuiting, and attempts to sort rules based on a heuristic of how expensive they will be to calculate. The sorting is dynamic and cache-aware, so that previously calculated conditions will be considered first, before computing other conditions. + +## Scope + +Sometimes, a condition will only use data from `@user` or only from `@subject`. In this case, we want to change the scope of the caching, so that we don't recalculate conditions unnecessarily. For example, given: + +``` ruby +class FooPolicy < BasePolicy + condition(:expensive_condition) { @subject.expensive_query? } + + rule { expensive_condition }.enable :some_ability +end +``` + +Naively, if we call `Ability.can?(user1, :some_ability, foo)` and `Ability.can?(user2, :some_ability, foo)`, we would have to calculate the condition twice - since they are for different users. But if we use the `scope: :subject` option: + +``` ruby + condition(:expensive_condition, scope: :subject) { @subject.expensive_query? } +``` + +then the result of the condition will be cached globally only based on the subject - so it will not be calculated repeatedly for different users. Similarly, `scope: :user` will cache only based on the user. + +**DANGER**: If you use a `:scope` option when the condition actually uses data from +both user and subject (including a simple anonymous check!) your result will be cached at too global of a scope and will result in cache bugs. + +Sometimes we are checking permissions for a lot of users for one subject, or a lot of subjects for one user. In this case, we want to set a *preferred scope* - i.e. tell the system that we prefer rules that can be cached on the repeated parameter. For example, in `Ability.users_that_can_read_project`: + +``` ruby +def users_that_can_read_project(users, project) + DeclarativePolicy.subject_scope do + users.select { |u| allowed?(u, :read_project, project) } + end +end +``` + +This will, for example, prefer checking `project.public?` to checking `user.admin?`. + +## Delegation + +Delegation is the inclusion of rules from another policy, on a different subject. For example, + +``` ruby +class FooPolicy < BasePolicy + delegate { @subject.project } +end +``` + +will include all rules from `ProjectPolicy`. The delegated conditions will be evaluated with the correct delegated subject, and will be sorted along with the regular rules in the policy. Note that only the relevant rules for a particular ability will actually be considered. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index fdaaa65fa28..42bb5e8619c 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -12,6 +12,56 @@ The `setup` task is a alias for `gitlab:setup`. This tasks calls `db:reset` to create the database, calls `add_limits_mysql` that adds limits to the database schema in case of a MySQL database and finally it calls `db:seed_fu` to seed the database. Note: `db:setup` calls `db:seed` but this does nothing. +### Automation + +If you're very sure that you want to **wipe the current database** and refill +seeds, you could: + +``` shell +echo 'yes' | bundle exec rake setup +``` + +To save you from answering `yes` manually. + +### Discard stdout + +Since the script would print a lot of information, it could be slowing down +your terminal, and it would generate more than 20G logs if you just redirect +it to a file. If we don't care about the output, we could just redirect it to +`/dev/null`: + +``` shell +echo 'yes' | bundle exec rake setup > /dev/null +``` + +Note that since you can't see the questions from stdout, you might just want +to `echo 'yes'` to keep it running. It would still print the errors on stderr +so no worries about missing errors. + +### Notes for MySQL + +Since the seeds would contain various UTF-8 characters, such as emojis or so, +we'll need to make sure that we're using `utf8mb4` for all the encoding +settings and `utf8mb4_unicode_ci` for collation. Please check +[MySQL utf8mb4 support](../install/database_mysql.md#mysql-utf8mb4-support) + +Make sure that `config/database.yml` has `encoding: utf8mb4`, too. + +Next, we'll need to update the schema to make the indices fit: + +``` shell +sed -i 's/limit: 255/limit: 191/g' db/schema.rb +``` + +Then run the setup script: + +``` shell +bundle exec rake setup +``` + +To make sure that indices still fit. You could find great details in: +[How to support full Unicode in MySQL databases](https://mathiasbynens.be/notes/mysql-utf8mb4) + ## Run tests In order to run the test you can use the following commands: diff --git a/doc/development/sha1_as_binary.md b/doc/development/sha1_as_binary.md new file mode 100644 index 00000000000..3151cc29bbc --- /dev/null +++ b/doc/development/sha1_as_binary.md @@ -0,0 +1,36 @@ +# Storing SHA1 Hashes As Binary + +Storing SHA1 hashes as strings is not very space efficient. A SHA1 as a string +requires at least 40 bytes, an additional byte to store the encoding, and +perhaps more space depending on the internals of PostgreSQL and MySQL. + +On the other hand, if one were to store a SHA1 as binary one would only need 20 +bytes for the actual SHA1, and 1 or 4 bytes of additional space (again depending +on database internals). This means that in the best case scenario we can reduce +the space usage by 50%. + +To make this easier to work with you can include the concern `ShaAttribute` into +a model and define a SHA attribute using the `sha_attribute` class method. For +example: + +```ruby +class Commit < ActiveRecord::Base + include ShaAttribute + + sha_attribute :sha +end +``` + +This allows you to use the value of the `sha` attribute as if it were a string, +while storing it as binary. This means that you can do something like this, +without having to worry about converting data to the right binary format: + +```ruby +commit = Commit.find_by(sha: '88c60307bd1f215095834f09a1a5cb18701ac8ad') +commit.sha = '971604de4cfa324d91c41650fabc129420c8d1cc' +commit.save +``` + +There is however one requirement: the column used to store the SHA has _must_ be +a binary type. For Rails this means you need to use the `:binary` type instead +of `:text` or `:string`. diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index e3a20f29a09..1e9fdbc65e2 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -36,3 +36,10 @@ slow jobs blocking work (even for different jobs) on the shared queue. Each Sidekiq worker must be tested using RSpec, just like any other class. These tests should be placed in `spec/workers`. + +## Removing or renaming queues + +Try to avoid renaming or removing queues in minor and patch releases. +During online update instance can have pending jobs and removing the queue can +lead to those jobs being stuck forever. If you can't write migration for those +Sidekiq jobs, please consider doing rename or remove queue in major release only.
\ No newline at end of file diff --git a/doc/development/testing.md b/doc/development/testing.md index cf3ea2ccfc2..e6aa4ae8f2f 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -195,7 +195,6 @@ Please consult the [dedicated "Frontend testing" guide](./fe_guide/testing.md). - Use `context` to test branching logic. - Use multi-line `do...end` blocks for `before` and `after`, even when it would fit on a single line. -- Don't `describe` symbols (see [Gotchas](gotchas.md#dont-describe-symbols)). - Don't assert against the absolute value of a sequence-generated attribute (see [Gotchas](gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). - Don't supply the `:each` argument to hooks since it's the default. - Prefer `not_to` to `to_not` (_this is enforced by RuboCop_). @@ -479,6 +478,11 @@ slowest test files and try to improve them. run the suite against MySQL for tags, `master`, and any branch that includes `mysql` in the name. - On EE, the test suite always runs both PostgreSQL and MySQL. +- Rails logging to `log/test.log` is disabled by default in CI [for + performance reasons][logging]. To override this setting, provide the + `RAILS_ENABLE_TEST_LOG` environment variable. + +[logging]: https://jtway.co/speed-up-your-rails-test-suite-by-6-in-1-line-13fedb869ec4 ## Spinach (feature) tests diff --git a/doc/development/ux_guide/copy.md b/doc/development/ux_guide/copy.md index 794c8eb6bfe..12e8d0a31bb 100644 --- a/doc/development/ux_guide/copy.md +++ b/doc/development/ux_guide/copy.md @@ -106,6 +106,14 @@ When using verbs or adjectives: * If the context clearly refers to the object, use them alone. Example: `Edit` or `Closed`
* If the context isn’t clear enough, use them with the object. Example: `Edit issue` or `Closed issues`
+### Search
+
+| Term | Use |
+| ---- | --- |
+| Search | When using all metadata to add criteria that match/don't match. Search can also affect ordering, by ranking best results. |
+| Filter | When taking a single criteria that removes items within a list that match/don't match. Filters do not affect ordering. |
+| Sort | Orders a list based on a single or grouped criteria |
+
### Projects and Groups
| Term | Use | :no_entry_sign: Don't |
diff --git a/doc/downgrade_ee_to_ce/README.md b/doc/downgrade_ee_to_ce/README.md index fe4b6d73771..75bae324585 100644 --- a/doc/downgrade_ee_to_ce/README.md +++ b/doc/downgrade_ee_to_ce/README.md @@ -46,6 +46,19 @@ $ sudo gitlab-rails runner "Service.where(type: ['JenkinsService', 'JenkinsDepre $ bundle exec rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService']).delete_all" production ``` +### Secret variables environment scopes + +If you're using this feature and there are variables sharing the same +key, but they have different scopes in a project, then you might want to +revisit the environment scope setting for those variables. + +In CE, environment scopes are completely ignored, therefore you could +accidentally get a variable which you're not expecting for a particular +environment. Make sure that you have the right variables in this case. + +Data is completely preserved, so you could always upgrade back to EE and +restore the behavior if you leave it alone. + ## Downgrade to CE After performing the above mentioned steps, you are now ready to downgrade your diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index 12466437edc..3d893ba53dd 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -6,7 +6,7 @@ Step-by-step guides on the basics of working with Git and GitLab. - [Start using Git on the command line](start-using-git.md) - [Create and add your SSH Keys](create-your-ssh-keys.md) - [Create a project](create-project.md) -- [Create a group](create-group.md) +- [Create a group](../user/group/index.md#create-a-new-group) - [Create a branch](create-branch.md) - [Fork a project](fork-project.md) - [Add a file](add-file.md) diff --git a/doc/gitlab-basics/create-group.md b/doc/gitlab-basics/create-group.md index b4889bb8818..985a52d88f5 100644 --- a/doc/gitlab-basics/create-group.md +++ b/doc/gitlab-basics/create-group.md @@ -1,50 +1,2 @@ -# How to create a group in GitLab -Your projects in GitLab can be organized in 2 different ways: -under your own namespace for single projects, such as `your-name/project-1` or -under groups. - -If you organize your projects under a group, it works like a folder. You can -manage your group members' permissions and access to the projects. - ---- - -To create a group: - -1. Expand the left sidebar by clicking the three bars at the upper left corner - and then navigate to **Groups**. - -  - -1. Once in your groups dashboard, click on **New group**. - -  - -1. Fill out the needed information: - - 1. Set the "Group path" which will be the namespace under which your projects - will be hosted (path can contain only letters, digits, underscores, dashes - and dots; it cannot start with dashes or end in dot). - 1. The "Group name" will populate with the path. Optionally, you can change - it. This is the name that will display in the group views. - 1. Optionally, you can add a description so that others can briefly understand - what this group is about. - 1. Optionally, choose and avatar for your project. - 1. Choose the [visibility level](../public_access/public_access.md). - -1. Finally, click the **Create group** button. - -## Add a new project to a group - -There are 2 different ways to add a new project to a group: - -- Select a group and then click on the **New project** button. - -  - - You can then continue on [creating a project](create-project.md). - -- While you are [creating a project](create-project.md), select a group namespace - you've already created from the dropdown menu. - -  +This document was moved to [another location](../user/group/index.md#create-a-new-group). diff --git a/doc/gitlab-basics/img/create_new_group_sidebar.png b/doc/gitlab-basics/img/create_new_group_sidebar.png Binary files differdeleted file mode 100644 index fa88d1d51c0..00000000000 --- a/doc/gitlab-basics/img/create_new_group_sidebar.png +++ /dev/null diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index 9a171d34671..bc75dc1447e 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -39,11 +39,14 @@ mysql> SET storage_engine=INNODB; # If you have MySQL < 5.7.7 and want to enable utf8mb4 character set support with your GitLab install, you must set the following NOW: mysql> SET GLOBAL innodb_file_per_table=1, innodb_file_format=Barracuda, innodb_large_prefix=1; +# If you use MySQL with replication, or just have MySQL configured with binary logging, you need to run the following to allow the use of `TRIGGER`: +mysql> SET GLOBAL log_bin_trust_function_creators = 1; + # Create the GitLab production database mysql> CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_general_ci`; # Grant the GitLab user necessary permissions on the database -mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES, REFERENCES ON `gitlabhq_production`.* TO 'git'@'localhost'; +mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES, REFERENCES, TRIGGER ON `gitlabhq_production`.* TO 'git'@'localhost'; # Quit the database session mysql> \q @@ -60,7 +63,15 @@ mysql> \q ``` You are done installing the database for now and can go back to the rest of the installation. -Please proceed to the rest of the installation before running through the utf8mb4 support section. +Please proceed to the rest of the installation **before** running through the steps below. + +### `log_bin_trust_function_creators` + +If you use MySQL with replication, or just have MySQL configured with binary logging, all of your MySQL servers will need to have `log_bin_trust_function_creators` enabled to allow the use of `TRIGGER` in migrations. You have already set this global variable in the steps above, but to make it persistent, add the following to your `my.cnf` file: + +``` +log_bin_trust_function_creators=1 +``` ### MySQL utf8mb4 support diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md index 35220119e9b..c6b767fff02 100644 --- a/doc/install/google_cloud_platform/index.md +++ b/doc/install/google_cloud_platform/index.md @@ -2,13 +2,13 @@  ->**Important note:** -GitLab has no official images in Google Cloud Platform yet. This guide serves -as a template for when the GitLab VM will be available. - The fastest way to get started on [Google Cloud Platform (GCP)][gcp] is through the [Google Cloud Launcher][launcher] program. +GitLab's official Google Launcher apps: +1. [GitLab Community Edition](https://console.cloud.google.com/launcher/details/gitlab-public/gitlab-community-edition?project=gitlab-public) +2. [GitLab Enterprise Edition](https://console.cloud.google.com/launcher/details/gitlab-public/gitlab-enterprise-edition?project=gitlab-public) + ## Prerequisites There are only two prerequisites in order to install GitLab on GCP: diff --git a/doc/install/installation.md b/doc/install/installation.md index 84af6432889..8ded607bcab 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -64,7 +64,7 @@ up-to-date and install it. Install the required packages (needed to compile Ruby and native extensions to Ruby gems): - sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake + sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake If you want to use Kerberos for user authentication, then install libkrb5-dev: @@ -294,9 +294,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 9-2-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 9-4-stable gitlab -**Note:** You can change `9-2-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `9-4-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It @@ -420,6 +420,12 @@ GitLab Shell is an SSH access and repository management software developed speci **Note:** Make sure your hostname can be resolved on the machine itself by either a proper DNS record or an additional line in /etc/hosts ("127.0.0.1 hostname"). This might be necessary for example if you set up GitLab behind a reverse proxy. If the hostname cannot be resolved, the final installation check will fail with "Check GitLab API access: FAILED. code: 401" and pushing commits will be rejected with "[remote rejected] master -> master (hook declined)". +**Note:** GitLab Shell application startup time can be greatly reduced by disabling RubyGems. This can be done in several manners: + +* Export `RUBYOPT=--disable-gems` environment variable for the processes +* Compile Ruby with `configure --disable-rubygems` to disable RubyGems by default. Not recommened for system-wide Ruby. +* Omnibus GitLab [replaces the *shebang* line of the `gitlab-shell/bin/*` scripts](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1707) + ### Install gitlab-workhorse GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/). The diff --git a/doc/install/kubernetes/gitlab_runner_chart.md b/doc/install/kubernetes/gitlab_runner_chart.md index b8bc0795f2e..515b2841d08 100644 --- a/doc/install/kubernetes/gitlab_runner_chart.md +++ b/doc/install/kubernetes/gitlab_runner_chart.md @@ -54,6 +54,13 @@ gitlabURL: http://gitlab.your-domain.com/ ## runnerRegistrationToken: "" +## Set the certsSecretName in order to pass custom certficates for GitLab Runner to use +## Provide resource name for a Kubernetes Secret Object in the same namespace, +## this is used to populate the /etc/gitlab-runner/certs directory +## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates +## +#certsSecretName: + ## Configure the maximum number of concurrent jobs ## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section ## @@ -135,6 +142,52 @@ runners: privileged: true ``` +### Providing a custom certificate for accessing GitLab + +You can provide a [Kubernetes Secret](https://kubernetes.io/docs/concepts/configuration/secret/) +to the GitLab Runner Helm Chart, which will be used to populate the container's +`/etc/gitlab-runner/certs` directory. + +Each key name in the Secret will be used as a filename in the directory, with the +file content being the value associated with the key. + +More information on how GitLab Runner uses these certificates can be found in the +[Runner Documentation](https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates). + + - The key/file name used should be in the format `<gitlab-hostname>.crt`. For example: `gitlab.your-domain.com.crt`. + - Any intermediate certificates need to be concatenated to your server certificate in the same file. + - The hostname used should be the one the certificate is registered for. + +The GitLab Runner Helm Chart does not create a secret for you. In order to create +the secret, you can prepare your certificate on you local machine, and then run +the `kubectl create secret` command from the directory with the certificate + +```bash +kubectl + --namespace <NAMESPACE> + create secret generic <SECRET_NAME> + --from-file=<CERTFICATE_FILENAME> +``` + +- `<NAMESPACE>` is the Kubernetes namespace where you want to install the GitLab Runner. +- `<SECRET_NAME>` is the Kubernetes Secret resource name. For example: `gitlab-domain-cert` +- `<CERTFICATE_FILENAME>` is the filename for the certificate in your current directory that will be imported into the secret + +You then need to provide the secret's name to the GitLab Runner chart. + +Add the following to your `values.yaml` + +```yaml +## Set the certsSecretName in order to pass custom certficates for GitLab Runner to use +## Provide resource name for a Kubernetes Secret Object in the same namespace, +## this is used to populate the /etc/gitlab-runner/certs directory +## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates +## +certsSecretName: <SECRET NAME> +``` + +- `<SECRET_NAME>` is the Kubernetes Secret resource name. For example: `gitlab-domain-cert` + ## Installing GitLab Runner using the Helm Chart Once you [have configured](#configuration) GitLab Runner in your `values.yml` file, diff --git a/doc/install/requirements.md b/doc/install/requirements.md index e5aaccdeadf..141df55f6bc 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -69,7 +69,7 @@ so keep in mind that you need at least 4GB available before running GitLab. With less memory GitLab will give strange errors during the reconfigure run and 500 errors during usage. -- 1GB RAM + 3GB of swap is the absolute minimum but we strongly **advise against** this amount of memory. See the unicorn worker section below for more advice. +- 1GB RAM + 3GB of swap is the absolute minimum but we strongly **advise against** this amount of memory. See the [unicorn worker section below](#unicorn-workers) for more advice. - 2GB RAM + 2GB swap supports up to 100 users but it will be very slow - **4GB RAM** is the **recommended** memory size for all installations and supports up to 100 users - 8GB RAM supports up to 1,000 users @@ -94,16 +94,17 @@ installation (e.g. the number of users, projects, etc). We currently support the following databases: -- PostgreSQL -- MySQL/MariaDB +- PostgreSQL (highly recommended) +- MySQL/MariaDB (strongly discouraged, not all GitLab features are supported, no support for [MySQL/MariaDB GTID](https://mariadb.com/kb/en/mariadb/gtid/)) -We **highly recommend** the use of PostgreSQL instead of MySQL/MariaDB as not all -features of GitLab may work with MySQL/MariaDB: +We highly recommend the use of PostgreSQL instead of MySQL/MariaDB as not all +features of GitLab work with MySQL/MariaDB: 1. MySQL support for subgroups was [dropped with GitLab 9.3][post]. See [issue #30472][30472] for more information. 1. GitLab Geo does [not support MySQL](https://docs.gitlab.com/ee/gitlab-geo/database.html#mysql-replication). 1. [Zero downtime migrations][zero] do not work with MySQL +1. We expect this list to grow over time. Existing users using GitLab with MySQL/MariaDB are advised to [migrate to PostgreSQL](../update/mysql_to_postgresql.md) instead. diff --git a/doc/integration/README.md b/doc/integration/README.md index e56e58498a6..d70b9a7f54b 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -5,19 +5,23 @@ trackers and external authentication. See the documentation below for details on how to configure these services. -- [JIRA](../user/project/integrations/jira.md) Integrate with the JIRA issue tracker +- [Akismet](akismet.md) Configure Akismet to stop spam +- [Auth0 OmniAuth](auth0.md) Enable the Auth0 OmniAuth provider +- [Bitbucket](bitbucket.md) Import projects from Bitbucket.org and login to your GitLab instance with your +Bitbucket.org account +- [CAS](cas.md) Configure GitLab to sign in using CAS - [External issue tracker](external-issue-tracker.md) Redmine, JIRA, etc. +- [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages +- [JIRA](../user/project/integrations/jira.md) Integrate with the JIRA issue tracker +- [Koding](../administration/integration/koding.md) Configure Koding to use IDE integration - [LDAP](ldap.md) Set up sign in via LDAP -- [OmniAuth](omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google, Bitbucket, Facebook, Shibboleth, SAML, Crowd, Azure and Authentiq ID -- [SAML](saml.md) Configure GitLab as a SAML 2.0 Service Provider -- [CAS](cas.md) Configure GitLab to sign in using CAS - [OAuth2 provider](oauth_provider.md) OAuth2 application creation +- [OmniAuth](omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google, Bitbucket, Facebook, Shibboleth, SAML, Crowd, Azure and Authentiq ID - [OpenID Connect](openid_connect_provider.md) Use GitLab as an identity provider -- [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages -- [reCAPTCHA](recaptcha.md) Configure GitLab to use Google reCAPTCHA for new users -- [Akismet](akismet.md) Configure Akismet to stop spam -- [Koding](../administration/integration/koding.md) Configure Koding to use IDE integration - [PlantUML](../administration/integration/plantuml.md) Configure PlantUML to use diagrams in AsciiDoc documents. +- [reCAPTCHA](recaptcha.md) Configure GitLab to use Google reCAPTCHA for new users +- [SAML](saml.md) Configure GitLab as a SAML 2.0 Service Provider +- [Trello](trello_power_up.md) Integrate Trello with GitLab > GitLab Enterprise Edition contains [advanced Jenkins support][jenkins]. diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index 265c891cf83..372e1909330 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -4,10 +4,11 @@ GitLab has a great issue tracker but you can also use an external one such as Jira, Redmine, or Bugzilla. Issue trackers are configurable per GitLab project and allow you to do the following: -- the **Issues** link on the GitLab project pages takes you to the appropriate - issue index of the external tracker -- clicking **New issue** on the project dashboard creates a new issue on the - external tracker +- you can reference these external issues inside GitLab interface + (merge requests, commits, comments) and they will be automatically converted + into links + +You can have enabled both external and internal GitLab issue trackers in parallel. The **Issues** link always opens the internal issue tracker and in case the internal issue tracker is disabled the link is not visible in the menu. ## Configuration diff --git a/doc/integration/img/enable_trello_powerup.png b/doc/integration/img/enable_trello_powerup.png Binary files differnew file mode 100644 index 00000000000..65d01f1c38c --- /dev/null +++ b/doc/integration/img/enable_trello_powerup.png diff --git a/doc/integration/img/trello_card_with_gitlab_powerup.png b/doc/integration/img/trello_card_with_gitlab_powerup.png Binary files differnew file mode 100644 index 00000000000..2965dc35855 --- /dev/null +++ b/doc/integration/img/trello_card_with_gitlab_powerup.png diff --git a/doc/integration/oauth_provider.md b/doc/integration/oauth_provider.md index af8a1c4e5ed..8ba2e8731c8 100644 --- a/doc/integration/oauth_provider.md +++ b/doc/integration/oauth_provider.md @@ -63,6 +63,9 @@ it from the admin area.  +You're also able to mark an application as _trusted_ when creating it through the admin area. By doing that, +the user authorization step is automatically skipped for this application. + --- ## Authorized applications diff --git a/doc/integration/slash_commands.md b/doc/integration/slash_commands.md index 5d880ba785c..aa52b5415cf 100644 --- a/doc/integration/slash_commands.md +++ b/doc/integration/slash_commands.md @@ -2,7 +2,11 @@ Slash commands in Mattermost and Slack allow you to control GitLab and view GitLab content right inside your chat client, without having to leave it. For Slack, this requires a [project service configuration](../user/project/integrations/slack_slash_commands.md). Simply type the command as a message in your chat client to activate it. -Commands are scoped to a project, with a trigger term that is specified during configuration. (We suggest you use the project name as the trigger term for simplicty and clarity.) Taking the trigger term as `project-name`, the commands are: +Commands are scoped to a project, with a trigger term that is specified during configuration. + +We suggest you use the project name as the trigger term for simplicity and clarity. + +Taking the trigger term as `project-name`, the commands are: | Command | Effect | @@ -12,3 +16,18 @@ Commands are scoped to a project, with a trigger term that is specified during c | `/project-name issue show <id>` | Shows the issue with id `<id>` | | `/project-name issue search <query>` | Shows up to 5 issues matching `<query>` | | `/project-name deploy <from> to <to>` | Deploy from the `<from>` environment to the `<to>` environment | + +## Issue commands + +It is possible to create new issue, display issue details and search up to 5 issues. + +## Deploy command + +In order to deploy to an environment, GitLab will try to find a deployment +manual action in the pipeline. + +If there is only one action for a given environment, it is going to be triggered. +If there is more than one action defined, GitLab will try to find an action +which name equals the environment name we want to deploy to. + +Command will return an error when no matching action has been found. diff --git a/doc/integration/trello_power_up.md b/doc/integration/trello_power_up.md new file mode 100644 index 00000000000..d264486a872 --- /dev/null +++ b/doc/integration/trello_power_up.md @@ -0,0 +1,42 @@ +# Trello Power-Up + +GitLab's Trello Power-Up enables you to seamlessly attach +GitLab **merge requests** to Trello cards. + + + +## Configuring the Power-Up + +In order to get started, you will need to configure your Power-Up. + +In Trello: + +1. Go to your Trello board +1. Select `Power-Ups` to see a listing of all the available Power-Ups +1. Look for a row that says `GitLab` and select the `Enable` button +1. Select the `Settings` (gear) icon +1. In the popup menu, select `Authorize Account` + +In this popup, fill in your `API URL` and `Personal Access Token`. After that, you will be able to attach any merge request to any Trello card on your selected Trello board. + +## What is my API URL? + +Your API URL should be your GitLab instance URL with `/api/v4` appended in the end of the URL. +For example, if your GitLab instance URL is `https://gitlab.com`, your API URL would be `https://gitlab.com/api/v4`. +If your instance's URL is `https://example.com`, your API URL will be `https://example.com/api/v4`. + + + +## What is my Personal Access Token? + +Your GitLab's personal access token will enable your GitLab account to be accessed +from Trello. + +> Find it in GitLab by clicking on your avatar (upright corner), from which you access +your user **Settings** > **Access Tokens**. + +Learn more about generating a personal access token in the +[Personal Access Token Documentation][personal-access-token-documentation]. +Don't forget to check the API scope checkbox! + +[personal-access-token-documentation]: ../user/profile/personal_access_tokens.html diff --git a/doc/profile/README.md b/doc/profile/README.md index aed64ac1228..fda6d85a84c 100644 --- a/doc/profile/README.md +++ b/doc/profile/README.md @@ -1,5 +1 @@ -# Profile settings - -- [Preferences](../user/profile/preferences.md) -- [Two-factor Authentication (2FA)](../user/profile/account/two_factor_authentication.md) -- [Deleting your account](../user/profile/account/delete_account.md) +This document was moved to [user/profile/account](../user/profile/index.md). diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 855f437cd73..10f5ab3370d 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -5,9 +5,9 @@ An application data backup creates an archive file that contains the database, all repositories and all attachments. -You can only restore a backup to **exactly the same version** of GitLab on which -it was created. The best way to migrate your repositories from one server to -another is through backup restore. +You can only restore a backup to **exactly the same version and type (CE/EE)** +of GitLab on which it was created. The best way to migrate your repositories +from one server to another is through backup restore. ## Backup @@ -270,6 +270,15 @@ For installations from source: remote_directory: 'gitlab_backups' ``` +### Specifying a custom directory for backups + +If you want to group your backups you can pass a `DIRECTORY` environment variable: + +``` +sudo gitlab-rake gitlab:backup:create DIRECTORY=daily +sudo gitlab-rake gitlab:backup:create DIRECTORY=weekly +``` + ### Backup archive permissions The backup archives created by GitLab (`1393513186_2014_02_27_gitlab_backup.tar`) @@ -369,8 +378,8 @@ The [restore prerequisites section](#restore-prerequisites) includes crucial information. Make sure to read and test the whole restore process at least once before attempting to perform it in a production environment. -You can only restore a backup to **exactly the same version** of GitLab that -you created it on, for example 9.1.0. +You can only restore a backup to **exactly the same version and type (CE/EE)** of +GitLab that you created it on, for example CE 9.1.0. ### Restore prerequisites @@ -441,8 +450,8 @@ Deleting tmp directories...[DONE] This procedure assumes that: -- You have installed the **exact same version** of GitLab Omnibus with which the - backup was created. +- You have installed the **exact same version and type (CE/EE)** of GitLab + Omnibus with which the backup was created. - You have run `sudo gitlab-ctl reconfigure` at least once. - GitLab is running. If not, start it using `sudo gitlab-ctl start`. diff --git a/doc/university/README.md b/doc/university/README.md index 399d54bcf23..170582bcd0c 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -122,6 +122,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project 1. [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) 1. [IBM: Continuous Delivery vs Continuous Deployment - Video](https://www.youtube.com/watch?v=igwFj8PPSnw) 1. [Amazon: Transition to Continuous Delivery - Video](https://www.youtube.com/watch?v=esEFaY0FDKc) +2. [TechBeacon: Doing continuous delivery? Focus first on reducing release cycle times](https://techbeacon.com/doing-continuous-delivery-focus-first-reducing-release-cycle-times) 1. See **[Integrations](#integrations)** for integrations with other CI services. #### 2.4. Workflow diff --git a/doc/university/process/README.md b/doc/university/process/README.md index 7ff53c2cc3f..04f2d52514f 100644 --- a/doc/university/process/README.md +++ b/doc/university/process/README.md @@ -27,4 +27,4 @@ please submit a merge request to add an upcoming class, assign to 1. Please upload any video recordings to our Youtube channel. We prefer them to be public, if needed they can be unlisted but if so they should be linked from this page. -1. Please create a merge request and assign to [SeanPackham](https://gitlab.com/u/SeanPackham). +1. Please create a merge request and assign to [Erica](https://gitlab.com/u/Erica). diff --git a/doc/update/8.9-to-8.10.md b/doc/update/8.9-to-8.10.md index d6b2f11d49a..42132f690d8 100644 --- a/doc/update/8.9-to-8.10.md +++ b/doc/update/8.9-to-8.10.md @@ -156,7 +156,7 @@ See [smtp_settings.rb.sample] as an example. Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab - + For Ubuntu 16.04.1 LTS: sudo systemctl daemon-reload diff --git a/doc/update/9.1-to-9.2.md b/doc/update/9.1-to-9.2.md index e7d97fde14e..225a4dcc924 100644 --- a/doc/update/9.1-to-9.2.md +++ b/doc/update/9.1-to-9.2.md @@ -70,7 +70,27 @@ curl --location https://yarnpkg.com/install.sh | bash - More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). -### 5. Get latest code +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code ```bash cd /home/git/gitlab @@ -97,7 +117,7 @@ cd /home/git/gitlab sudo -u git -H git checkout 9-2-stable-ee ``` -### 6. Update gitlab-shell +### 7. Update gitlab-shell ```bash cd /home/git/gitlab-shell @@ -107,11 +127,10 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) sudo -u git -H bin/compile ``` -### 7. Update gitlab-workhorse +### 8. Update gitlab-workhorse -Install and compile gitlab-workhorse. This requires -[Go 1.8](https://golang.org/dl). Go (at least 1.5) should already be on your system from -GitLab 8.1 and shall be upgraded if necessary. Please note that starting in Gitlab 9.3, only Go 1.8.3 and above will be supported. GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/). +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). If you are not using Linux you may have to run `gmake` instead of `make` below. @@ -123,7 +142,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) sudo -u git -H make ``` -### 8. Update configuration files +### 9. Update configuration files #### New configuration options for `gitlab.yml` @@ -197,7 +216,7 @@ For Ubuntu 16.04.1 LTS: sudo systemctl daemon-reload ``` -### 9. Install libs, migrations, etc. +### 10. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -223,7 +242,7 @@ sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production **MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). -### 10. Optional: install Gitaly +### 11. Optional: install Gitaly Gitaly is still an optional component of GitLab. If you want to save time during your 9.2 upgrade **you can skip this step**. @@ -240,14 +259,14 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) sudo -u git -H make ``` -### 11. Start application +### 12. Start application ```bash sudo service gitlab start sudo service nginx restart ``` -### 12. Check application status +### 13. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md index 0c32e4db53f..910539acc70 100644 --- a/doc/update/9.2-to-9.3.md +++ b/doc/update/9.2-to-9.3.md @@ -72,8 +72,8 @@ More information can be found on the [yarn website](https://yarnpkg.com/en/docs/ ### 5. Update Go -NOTE: GitLab 9.3 and higher only supports Go 1.8.3 and dropped support for Go 1.5.x through 1.7.x. Be -sure to upgrade your installation if necessary +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. You can check which version you are running with `go version`. @@ -117,7 +117,7 @@ cd /home/git/gitlab sudo -u git -H git checkout 9-3-stable-ee ``` -### 5. Update gitlab-shell +### 7. Update gitlab-shell ```bash cd /home/git/gitlab-shell @@ -127,11 +127,10 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) sudo -u git -H bin/compile ``` -### 6. Update gitlab-workhorse +### 8. Update gitlab-workhorse -Install and compile gitlab-workhorse. This requires -[Go 1.5](https://golang.org/dl) which should already be on your system from -GitLab 8.1. GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/). +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). If you are not using Linux you may have to run `gmake` instead of `make` below. @@ -143,7 +142,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) sudo -u git -H make ``` -### 7. Update Gitaly +### 9. Update Gitaly If you have not yet set up Gitaly then follow [Gitaly section of the installation guide](../install/installation.md#install-gitaly). @@ -157,7 +156,29 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) sudo -u git -H make ``` -### 10. Update configuration files +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files #### New configuration options for `gitlab.yml` @@ -231,7 +252,7 @@ For Ubuntu 16.04.1 LTS: sudo systemctl daemon-reload ``` -### 11. Install libs, migrations, etc. +### 12. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -257,14 +278,14 @@ sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production **MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). -### 12. Start application +### 13. Start application ```bash sudo service gitlab start sudo service nginx restart ``` -### 13. Check application status +### 14. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/update/9.3-to-9.4.md b/doc/update/9.3-to-9.4.md index a712ce5a8b1..9540c36e7d0 100644 --- a/doc/update/9.3-to-9.4.md +++ b/doc/update/9.3-to-9.4.md @@ -72,8 +72,8 @@ More information can be found on the [yarn website](https://yarnpkg.com/en/docs/ ### 5. Update Go -NOTE: GitLab 9.4 and higher only supports Go 1.8.3 and dropped support for Go 1.5.x through 1.7.x. Be -sure to upgrade your installation if necessary +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. You can check which version you are running with `go version`. @@ -117,7 +117,7 @@ cd /home/git/gitlab sudo -u git -H git checkout 9-4-stable-ee ``` -### 5. Update gitlab-shell +### 7. Update gitlab-shell ```bash cd /home/git/gitlab-shell @@ -127,11 +127,10 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) sudo -u git -H bin/compile ``` -### 6. Update gitlab-workhorse +### 8. Update gitlab-workhorse -Install and compile gitlab-workhorse. This requires -[Go 1.8](https://golang.org/dl) which should already be on your system from -GitLab 8.1. GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/). +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). If you are not using Linux you may have to run `gmake` instead of `make` below. @@ -143,11 +142,13 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) sudo -u git -H make ``` -### 7. Update Gitaly +### 9. Update Gitaly If you have not yet set up Gitaly then follow [Gitaly section of the installation guide](../install/installation.md#install-gitaly). +As of GitLab 9.4, Gitaly is a mandatory component of GitLab. + #### Check Gitaly configuration Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly @@ -156,8 +157,7 @@ configuration file may contain syntax errors. The block name file, should be `[[storage]]` instead. ```shell -cd /home/git/gitaly -sudo -u git -H editor config.toml +sudo -u git -H sed -i.pre-9.4 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml ``` #### Compile Gitaly @@ -169,7 +169,29 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) sudo -u git -H make ``` -### 10. Update configuration files +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files #### New configuration options for `gitlab.yml` @@ -243,7 +265,7 @@ For Ubuntu 16.04.1 LTS: sudo systemctl daemon-reload ``` -### 11. Install libs, migrations, etc. +### 12. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -269,14 +291,14 @@ sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production **MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). -### 12. Start application +### 13. Start application ```bash sudo service gitlab start sudo service nginx restart ``` -### 13. Check application status +### 14. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index a954840b8a6..70934f9960a 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -5,6 +5,8 @@ - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and will be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior) section. + - [Access token](#access-token) has been deprecated in GitLab 9.4 + in favor of [IP whitelist](#ip-whitelist) GitLab provides liveness and readiness probes to indicate service health and reachability to required services. These probes report on the status of the @@ -12,97 +14,101 @@ database connection, Redis connection, and access to the filesystem. These endpoints [can be provided to schedulers like Kubernetes][kubernetes] to hold traffic until the system is ready or restart the container as needed. -## Access Token +## IP whitelist -An access token needs to be provided while accessing the probe endpoints. The current -accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check** -(`admin/health_check`) page of your GitLab instance. +To access monitoring resources, the client IP needs to be included in a whitelist. - +[Read how to add IPs to a whitelist for the monitoring endpoints.][admin]. -The access token can be passed as a URL parameter: +## Using the endpoint -``` -https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN -``` +With default whitelist settings, the probes can be accessed from localhost: + +- `http://localhost/-/readiness` +- `http://localhost/-/liveness` -which will then provide a report of system health in JSON format: +which will then provide a report of system health in JSON format. + +Readiness example output: ``` { - "db_check": { - "status": "ok" - }, - "redis_check": { - "status": "ok" - }, - "fs_shards_check": { - "status": "ok", - "labels": { - "shard": "default" - } - } + "queues_check" : { + "status" : "ok" + }, + "redis_check" : { + "status" : "ok" + }, + "shared_state_check" : { + "status" : "ok" + }, + "fs_shards_check" : { + "labels" : { + "shard" : "default" + }, + "status" : "ok" + }, + "db_check" : { + "status" : "ok" + }, + "cache_check" : { + "status" : "ok" + } } ``` -## Using the Endpoint - -Once you have the access token, the probes can be accessed: +Liveness example output: -- `https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN` -- `https://gitlab.example.com/-/liveness?token=ACCESS_TOKEN` +``` +{ + "fs_shards_check" : { + "status" : "ok" + }, + "cache_check" : { + "status" : "ok" + }, + "db_check" : { + "status" : "ok" + }, + "redis_check" : { + "status" : "ok" + }, + "queues_check" : { + "status" : "ok" + }, + "shared_state_check" : { + "status" : "ok" + } +} +``` ## Status On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint will return a valid successful HTTP status code, and a `success` message. -## Old behavior - ->**Notes:** - - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1. - - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and will - be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior) - section. - -GitLab provides a health check endpoint for uptime monitoring on the `health_check` web -endpoint. The health check reports on the overall system status based on the status of -the database connection, the state of the database migrations, and the ability to write -and access the cache. This endpoint can be provided to uptime monitoring services like -[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health]. +## Access token (Deprecated) -Once you have the [access token](#access-token), health information can be -retrieved as plain text, JSON, or XML using the `health_check` endpoint: +>**Note:** +Access token has been deprecated in GitLab 9.4 +in favor of [IP whitelist](#ip-whitelist) -- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN` -- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN` -- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN` - -You can also ask for the status of specific services: - -- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN` -- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN` -- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN` - -For example, the JSON output of the following health check: +An access token needs to be provided while accessing the probe endpoints. The current +accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check** +(`admin/health_check`) page of your GitLab instance. -```bash -curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json -``` + -would be like: +The access token can be passed as a URL parameter: ``` -{"healthy":true,"message":"success"} +https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN ``` -On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint -will return a valid successful HTTP status code, and a `success` message. Ideally your -uptime monitoring should look for the success message. - [ce-10416]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10416 [ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888 [pingdom]: https://www.pingdom.com [nagios-health]: https://nagios-plugins.org/doc/man/check_http.html [newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring [kubernetes]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/ +[admin]: ../../../administration/monitoring/ip_whitelist.md diff --git a/doc/workflow/groups/access_requests_management.png b/doc/user/group/img/access_requests_management.png Binary files differindex 36deaa89a70..36deaa89a70 100644 --- a/doc/workflow/groups/access_requests_management.png +++ b/doc/user/group/img/access_requests_management.png diff --git a/doc/user/group/img/add_new_members.png b/doc/user/group/img/add_new_members.png Binary files differnew file mode 100644 index 00000000000..53f5596de23 --- /dev/null +++ b/doc/user/group/img/add_new_members.png diff --git a/doc/gitlab-basics/img/create_new_group_info.png b/doc/user/group/img/create_new_group_info.png Binary files differindex 8d2501d9f7a..8d2501d9f7a 100644 --- a/doc/gitlab-basics/img/create_new_group_info.png +++ b/doc/user/group/img/create_new_group_info.png diff --git a/doc/gitlab-basics/img/create_new_project_from_group.png b/doc/user/group/img/create_new_project_from_group.png Binary files differindex c35234660db..c35234660db 100644 --- a/doc/gitlab-basics/img/create_new_project_from_group.png +++ b/doc/user/group/img/create_new_project_from_group.png diff --git a/doc/user/group/img/group_settings.png b/doc/user/group/img/group_settings.png Binary files differnew file mode 100644 index 00000000000..629cd0729aa --- /dev/null +++ b/doc/user/group/img/group_settings.png diff --git a/doc/user/group/img/groups.png b/doc/user/group/img/groups.png Binary files differnew file mode 100644 index 00000000000..6211f999d5e --- /dev/null +++ b/doc/user/group/img/groups.png diff --git a/doc/user/group/img/membership_lock.png b/doc/user/group/img/membership_lock.png Binary files differnew file mode 100644 index 00000000000..d31fbb43375 --- /dev/null +++ b/doc/user/group/img/membership_lock.png diff --git a/doc/workflow/groups/new_group_form.png b/doc/user/group/img/new_group_form.png Binary files differindex 91727ab5336..91727ab5336 100644 --- a/doc/workflow/groups/new_group_form.png +++ b/doc/user/group/img/new_group_form.png diff --git a/doc/user/group/img/new_group_from_groups.png b/doc/user/group/img/new_group_from_groups.png Binary files differnew file mode 100644 index 00000000000..baf34244cb2 --- /dev/null +++ b/doc/user/group/img/new_group_from_groups.png diff --git a/doc/user/group/img/new_group_from_other_pages.png b/doc/user/group/img/new_group_from_other_pages.png Binary files differnew file mode 100644 index 00000000000..014a7088af2 --- /dev/null +++ b/doc/user/group/img/new_group_from_other_pages.png diff --git a/doc/workflow/groups/request_access_button.png b/doc/user/group/img/request_access_button.png Binary files differindex f1aae6afed7..f1aae6afed7 100644 --- a/doc/workflow/groups/request_access_button.png +++ b/doc/user/group/img/request_access_button.png diff --git a/doc/gitlab-basics/img/select_group_dropdown.png b/doc/user/group/img/select_group_dropdown.png Binary files differindex 68fc950304c..68fc950304c 100644 --- a/doc/gitlab-basics/img/select_group_dropdown.png +++ b/doc/user/group/img/select_group_dropdown.png diff --git a/doc/user/group/img/share_with_group_lock.png b/doc/user/group/img/share_with_group_lock.png Binary files differnew file mode 100644 index 00000000000..8df41bf9465 --- /dev/null +++ b/doc/user/group/img/share_with_group_lock.png diff --git a/doc/user/group/img/transfer_project_to_other_group.png b/doc/user/group/img/transfer_project_to_other_group.png Binary files differnew file mode 100644 index 00000000000..042c002f83f --- /dev/null +++ b/doc/user/group/img/transfer_project_to_other_group.png diff --git a/doc/workflow/groups/withdraw_access_request_button.png b/doc/user/group/img/withdraw_access_request_button.png Binary files differindex c5d8ef6c04f..c5d8ef6c04f 100644 --- a/doc/workflow/groups/withdraw_access_request_button.png +++ b/doc/user/group/img/withdraw_access_request_button.png diff --git a/doc/user/group/index.md b/doc/user/group/index.md new file mode 100644 index 00000000000..2691cf7d671 --- /dev/null +++ b/doc/user/group/index.md @@ -0,0 +1,208 @@ +# Groups + +With GitLab Groups you can assemble related projects together +and grant members access to several projects at once. + +Groups can also be nested in [subgroups](subgroups/index.md). + +Find your groups by expanding the left menu and clicking **Groups**: + + + +The Groups page displays all groups you are a member of, how many projects it holds, +how many members it has, the group visibility, and, if you have enough permissions, +a link to the group settings. By clicking the last button you can leave that group. + +## Use cases + +You can create groups for numerous reasons. To name a few: + +- Organize related projects under the same [namespace](#namespaces), add members to that +group and grant access to all their projects at once +- Create a group, include members of your team, and make it easier to +`@mention` all the team at once in issues and merge requests + - Create a group for your company members, and create [subgroups](subgroups/index.md) + for each individual team. Let's say you create a group called `company-team`, and among others, + you created subgroups in this group for each individual team `backend-team`, + `frontend-team`, and `production-team`: + 1. When you start a new implementation from an issue, you add a comment: + _"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_ + 1. When your backend team needs help from frontend, they add a comment: + _"`@company-team/frontend-team` could you help us here please?"_ + 1. When the frontend team completes their implementation, they comment: + _"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_ + +## Namespaces + +In GitLab, a namespace is a unique name to be used as a user name, a group name, or a subgroup name. + +- `http://gitlab.example.com/username` +- `http://gitlab.example.com/groupname` +- `http://gitlab.example.com/groupname/subgroup_name` + +For example, consider a user called John: + +1. John creates his account on GitLab.com with the username `jonh`; +his profile will be accessed under `https://gitlab.example.com/john` +1. John creates a group for his team with the groupname `john-team`; +his group and its projects will be accessed under `https://gitlab.example.com/john-team` +1. John creates a subgroup of `john-team` with the subgroup name `marketing`; +his subgroup and its projects will be accessed under `https://gitlab.example.com/john-team/marketing` + +By doing so: + +- Any team member mentions John with `@john` +- John mentions everyone from his team with `@john-team` +- John mentions only his marketing team with `@john-team/marketing` + +## Create a new group + +You can create a group in GitLab from: + +1. The Groups page: expand the left menu, click **Groups**, and click the green button **New group**: + +  + +1. Elsewhere: expand the `plus` sign button on the top navbar and choose **New group**: + +  + +Add the following information: + + + +1. Set the **Group path** which will be the **namespace** under which your projects + will be hosted (path can contain only letters, digits, underscores, dashes + and dots; it cannot start with dashes or end in dot). +1. The **Group name** will populate with the path. Optionally, you can change + it. This is the name that will display in the group views. +1. Optionally, you can add a description so that others can briefly understand + what this group is about. +1. Optionally, choose an avatar for your project. +1. Choose the [visibility level](../../public_access/public_access.md). + +## Add users to a group + +Add members to a group by navigating to the group's dashboard, and clicking **Members**: + + + +Select the [permission level][permissions] and add the new member. You can also set the expiring +date for that user, from which they will no longer have access to your group. + +One of the benefits of putting multiple projects in one group is that you can +give a user to access to all projects in the group with one action. + +Consider we have a group with two projects: + +- On the **Group Members** page we can now add a new user to the group. +- Now because this user is a **Developer** member of the group, he automatically +gets **Developer** access to **all projects** within that group. + +If necessary, you can increase the access level of an individual user for a specific project, +by adding them again as a new member to the project with the new permission levels. + +## Request access to a group + +As a group owner you can enable or disable non members to request access to +your group. Go to the group settings and click on **Allow users to request access**. + +As a user, you can request to be a member of a group. Go to the group you'd +like to be a member of, and click the **Request Access** button on the right +side of your screen. + + + +--- + +Group owners and masters will be notified of your request and will be able to approve or +decline it on the members page. + + + +--- + +If you change your mind before your request is approved, just click the +**Withdraw Access Request** button. + + + +## Add projects to a group + +There are two different ways to add a new project to a group: + +- Select a group and then click on the **New project** button. + +  + + You can then continue on [creating a project](../../gitlab-basics/create-project.md). + +- While you are creating a project, select a group namespace + you've already created from the dropdown menu. + +  + +## Transfer an existing project into a group + +You can transfer an existing project into a group as long as you have at least **Master** [permissions][permissions] to that group +and if you are an **Owner** of the project. + + + +Find this option under your project's settings. + +GitLab administrators can use the admin interface to move any project to any namespace if needed. + +## Manage group memberships via LDAP + +In GitLab Enterprise Edition it is possible to manage GitLab group memberships using LDAP groups. +See [the GitLab Enterprise Edition documentation](../../integration/ldap.md) for more information. + +## Group settings + +Once you have created a group, you can manage its settings by navigating to +the group's dashboard, and clicking **Settings**. + + + +### General settings + +Besides giving you the option to edit any settings you've previously +set when [creating the group](#create-a-new-group), you can also +access further configurations for your group. + +#### Enforce 2FA to group members + +Add a secury layer to your group by +[enforcing two-factor authentication (2FA)](../../security/two_factor_authentication.md#enforcing-2fa-for-all-users-in-a-group) +to all group members. + +#### Member Lock (EES/EEP) + +Available in [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/), +with **Member Lock** it is possible to lock membership in project to the +level of members in group. + +Learn more about [Member Lock](https://docs.gitlab.com/ee/user/group/index.html#member-lock-ees-eep). + +#### Share with group lock (EES/EEP) + +In [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/) +it is possible to prevent projects in a group from [sharing +a project with another group](../../workflow/share_projects_with_other_groups.md). +This allows for tighter control over project access. + +Learn more about [Share with group lock](https://docs.gitlab.com/ee/user/group/index.html#share-with-group-lock-ees-eep). + +### Advanced settings + +- **Projects**: view all projects within that group, add members to each project, +access each project's settings, and remove any project from the same screen. +- **Webhooks**: configure [webhooks](../project/integrations/webhooks.md) +and [push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html#push-rules) to your group (Push Rules is available in [GitLab Enteprise Edition Starter][ee].) +- **Audit Events**: view [Audit Events](https://docs.gitlab.com/ee/administration/audit_events.html#audit-events) +for the group (GitLab admins only, available in [GitLab Enterprise Edition Starter][ee]). +- **Pipelines quota**: keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group + +[permissions]: ../permissions.md#permissions +[ee]: https://about.gitlab.com/products/
\ No newline at end of file diff --git a/doc/user/index.md b/doc/user/index.md new file mode 100644 index 00000000000..1281cc6e4f0 --- /dev/null +++ b/doc/user/index.md @@ -0,0 +1,180 @@ +# User documentation + +Welcome to GitLab! We're glad to have you here! + +As a GitLab user you'll have access to all the features +your [subscription](https://about.gitlab.com/products/) +includes, except [GitLab administrator](../README.md#administrator-documentation) +settings, unless you have admin privileges to install, configure, +and upgrade your GitLab instance. + +For GitLab.com, admin privileges are restricted to the GitLab team. + +If you run your own GitLab instance and are looking for the administration settings, +please refer to the [administration](../README.md#administrator-documentation) +documentation. + +## Overview + +GitLab is a fully integrated software development platform that enables you +and your team to work cohesively, faster, transparently, and effectively, +since the discussion of a new idea until taking that idea to production all +all the way through, from within the same platform. + +Please check this page for an overview on [GitLab's features](https://about.gitlab.com/features/). + +## Use cases + +GitLab is a git-based platforms that integrates a great number of essential tools for software development and deployment, and project management: + +- Code hosting in repositories with version control +- Track proposals for new implementations, bug reports, and feedback with a +fully featured [Issue Tracker](project/issues/index.md#issue-tracker) +- Organize and prioritize with [Issue Boards](project/issues/index.md#issue-boards) +- Code review in [Merge Requests](project/merge_requests/index.md) with live-preview changes per +branch with [Review Apps](../ci/review_apps/index.md) +- Build, test and deploy with built-in [Continuous Integration](../ci/README.md) +- Deploy your personal and professional static websites with [GitLab Pages](project/pages/index.md) +- Integrate with Docker with [GitLab Container Registry](project/container_registry.md) +- Track the development lifecycle with [GitLab Cycle Analytics](project/cycle_analytics.md) + +With GitLab Enterprise Edition, you can also: + +- Provide support with [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html) +- Improve collaboration with +[Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/index.html#merge-request-approvals), +[Multiple Assignees for Issues](https://docs.gitlab.com/ee/user/project/issues/multiple_assignees_for_issues.html), +and [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards) +- Create formal relashionships between issues with [Related Issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html) +- Use [Burndown Charts](https://docs.gitlab.com/ee/user/project/milestones/burndown_charts.html) to track progress during a sprint or while working on a new version of their software. +- Leverage [Elasticsearch](https://docs.gitlab.com/ee/integration/elasticsearch.html) with [Advanced Global Search](https://docs.gitlab.com/ee/user/search/advanced_global_search.html) and [Advanced Syntax Search](https://docs.gitlab.com/ee/user/search/advanced_search_syntax.html) for faster, more advanced code search across your entire GitLab instance +- [Authenticate users with Kerberos](https://docs.gitlab.com/ee/integration/kerberos.html) +- [Mirror a repository](https://docs.gitlab.com/ee/workflow/repository_mirroring.html) from elsewhere on your local server. +- [Export issues as CSV](https://docs.gitlab.com/ee/user/project/issues/csv_export.html) +- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipeline Graphs](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Lock files](https://docs.gitlab.com/ee/user/project/file_lock.html) to prevent conflicts +- View of the current health and status of each CI environment running on Kubernetes with [Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) +- Leverage your continuous delivery method with [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) + +You can also [integrate](project/integrations/project_services.md) GitLab with numerous third-party applications, such as Mattermost, Microsoft Teams, HipChat, Trello, Slack, Bamboo CI, JIRA, and a lot more. + +### Articles + +For a complete workflow use case please check [GitLab Workflow, an Overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/#gitlab-workflow-use-case-scenario). + +For more use cases please check our [Technical Articles](../articles/index.md). + +## Projects + +In GitLab, you can create [projects](project/index.md) for numerous reasons, such as, host +your code, use it as an issue tracker, collaborate on code, and continuously +build, test, and deploy your app with built-in GitLab CI/CD. Or, you can do +it all at once, from one single project. + +### Repository + +Host your codebase in [GitLab repositories](project/repository/index.md) with version control +and as part of a fully integrated platform. + +### Issues + +Explore the best of GitLab [Issues](project/issues/index.md). + +### Merge Requests + +Collanorate on code, gather reviews, live preview changes per branch, and +request approvals with [Merge Requests](project/merge_requests/index.md). + +### Milestones + +Work on multiple issues and merge requests towards the same target date +with [Milestones](project/milestones/index.md). + +### GitLab Pages + +Publish your static site directly from GitLab with [GitLab Pages](project/pages/index.md). You +can [build, test, and deploy any Static Site Generator](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) with Pages. + +### Container Registry + +Build and deploy Docker images with [GitLab Container Registry](project/container_registry.md). + +## GitLab CI/CD + +Use built-in [GitLab CI/CD](../ci/README.md) to test, build, and deploy your applications +directly from GitLab. No third-party integrations needed. + +### Auto Deploy + +Deploy your application out-of-the-box with [GitLab Auto Deploy](../ci/autodeploy/index.md). + +### Review Apps + +Live-preview the changes introduced by a merge request with [Review Apps](../ci/review_apps/index.md). + +## Groups + +With GitLab [Groups](group/index.md) you can assemble related projects together +and grant members access to several projects at once. + +### Subgroups + +Groups can also be nested in [subgroups](group/subgroups/index.md). + +## Account + +There is a lot you can customize and configure +to enjoy the best of GitLab. + +[Manage your user settings](profile/index.md) to change your personal info, +personal access tokens, authorized applications, etc. + +### Authentication + +Read through the [authentication](../topics/authentication/index.md) methods available in GitLab. + +### Permissions + +Learn the different set of [permissions](permissions.md) for user type (guest, reporter, developer, master, owner). + +## Integrations + +[Integrate GitLab](../integration/README.md) with your preferred tool, +such as Trello, JIRA, etc. + +## Git and GitLab + +Learn what is [Git](../topics/git/index.md) and its best practices. + +## Discussions + +In GitLab, you can comment and mention collaborators in issues, +merge requests, code snippets, and commits. + +When performing inline reviews to implementations +to your codebase through merge requests you can +gather feedback through [resolvable discussions](discussions/index.md#resolvable-discussions). + +## Todos + +Never forget to reply to your collaborators. [GitLab Todos](../workflow/todos.md) +are a tool for working faster and more effectively with your team, +by listing all user or group mentions, as well as issues and merge +requests you're assigned to. + +## Snippets + +[Snippets](snippets.md) are code blocks that you want to store in GitLab, from which +you have quick access to. You can also gather feedback on them through +[discussions](#discussions). + +## Webhooks + +Configure [webhooks](project/integrations/webhooks.html) to listen for +specific events like pushes, issues or merge requests. GitLab will send a +POST request with data to the webhook URL. + +## API + +Automate GitLab via [API](../api/README.html). + diff --git a/doc/user/profile/account/index.md b/doc/user/profile/account/index.md index 764354e1e96..06667bfc5f1 100644 --- a/doc/user/profile/account/index.md +++ b/doc/user/profile/account/index.md @@ -1,5 +1,2 @@ -# Profile settings -## Account - -Set up [two-factor authentication](two_factor_authentication.md). +This document was moved to [../index.md#profile-settings](../index.md#profile-settings). diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md new file mode 100644 index 00000000000..7d25970fcb1 --- /dev/null +++ b/doc/user/profile/index.md @@ -0,0 +1,47 @@ +# User account + +When logged into their GitLab account, users can customize their +experience according to the best approach to their cases. + +## Username + +Your `username` is a unique [`namespace`](../group/index.md#namespaces) +related to your user ID. + +You can change your `username` from your +[profile settings](#profile-settings). To avoid breaking +paths when you change your `username`, we suggest you follow +[this procedure from the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#how-to-change-your-username-at-gitlabcom). + +## User profile + +Your profile is available from the up-right corner menu bar (user's avatar) > **Profile**, +or from `https://example.gitlab.com/username`. + +On your profile page, you will see the following information: + +- Personal information +- Activity stream: see your activity streamline and the history of your contributions +- Groups: [groups](../group/index.md) you're a member of +- Contributed projects: [projects](../project/index.md) you contributed to +- Personal projects: your personal projects (respecting the project's visibility level) +- Snippets: your personal code [snippets](../snippets.md#personal-snippets) + +## Profile settings + +You can edit your account settings by navigating from the up-right corner menu bar +(user's avatar) > **Settings**, or visiting `https://example.gitlab.com/profile`. + +From there, you can: + +- Update your personal information +- Manage [private tokens](../../api/README.md#private-tokens), email tokens, [2FA](account/two_factor_authentication.md) +- Change your username and [delete your account](account/delete_account.md) +- Manage applications that can +[use GitLab as an OAuth provider](../../integration/oauth_provider.md#introduction-to-oauth) +- Manage [personal access tokens](personal_access_tokens.md) to access your account via API and authorized applications +- Add and delete emails linked to your account +- Manage [SSH keys](../../ssh/README.md#ssh) to access your account via SSH +- Manage your [preferences](preferences.md#syntax-highlighting-theme) +to customize your own GitLab experience +- Acess your audit log, a security log of important events involving your account diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 9488ce1ef30..f28c034e74c 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -14,6 +14,9 @@ accepted method of authentication when you have Once you have your token, [pass it to the API][usage] using either the `private_token` parameter or the `PRIVATE-TOKEN` header. +The expiration of personal access tokens happens on the date you define, +at midnight UTC. + ## Creating a personal access token You can create as many personal access tokens as you like from your GitLab diff --git a/doc/user/project/img/issue_board.png b/doc/user/project/img/issue_board.png Binary files differindex b636cb294b8..cf7f519f783 100644 --- a/doc/user/project/img/issue_board.png +++ b/doc/user/project/img/issue_board.png diff --git a/doc/user/project/img/issue_board_add_list.png b/doc/user/project/img/issue_board_add_list.png Binary files differindex cdfc466d23f..973d9f7cde4 100644 --- a/doc/user/project/img/issue_board_add_list.png +++ b/doc/user/project/img/issue_board_add_list.png diff --git a/doc/user/project/img/issue_board_move_issue_card_list.png b/doc/user/project/img/issue_board_move_issue_card_list.png Binary files differnew file mode 100644 index 00000000000..c6b17ada40e --- /dev/null +++ b/doc/user/project/img/issue_board_move_issue_card_list.png diff --git a/doc/user/project/img/issue_board_welcome_message.png b/doc/user/project/img/issue_board_welcome_message.png Binary files differindex 5318e6ea4a9..127b9b08cc7 100644 --- a/doc/user/project/img/issue_board_welcome_message.png +++ b/doc/user/project/img/issue_board_welcome_message.png diff --git a/doc/user/project/img/issue_boards_add_issues_modal.png b/doc/user/project/img/issue_boards_add_issues_modal.png Binary files differindex 33049dce74f..bedaf724a15 100644 --- a/doc/user/project/img/issue_boards_add_issues_modal.png +++ b/doc/user/project/img/issue_boards_add_issues_modal.png diff --git a/doc/user/project/index.md b/doc/user/project/index.md new file mode 100644 index 00000000000..91a19600951 --- /dev/null +++ b/doc/user/project/index.md @@ -0,0 +1,107 @@ +# Projects + +In GitLab, you can create projects for hosting +your codebase, use it as an issue tracker, collaborate on code, and continuously +build, test, and deploy your app with built-in GitLab CI/CD. + +Your projects can be [available](../../public_access/public_access.md) +publicly, internally, or privately, at your choice. GitLab does not limit +the number of private projects you create. + +## Project's features + +When you create a project in GitLab, you'll have access to a large number of +[features](https://about.gitlab.com/features/): + +**Issues and merge requests:** + +- [Issue tracker](issues/index.md): Discuss implementations with your team within issues + - [Issue Boards](issue_board.md): Organize and prioritize your workflow + - [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards) (**EES/EEP**): Allow your teams to create their own workflows (Issue Boards) for the same project +- [Repositories](repository/index.md): Host your code in a fully +integrated platform + - [Protected branches](protected_branches.md): Prevent collaborators + from messing with history or pushing code without review + - [Protected tags](protected_tags.md): Control over who has + permission to create tags, and prevent accidental update or deletion +- [Merge Requests](merge_requests/index.md): Apply your branching +strategy and get reviewed by your team + - [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) (**EES/EEP**): Ask for approval before + implementing a change + - [Fix merge conflicts from the UI](merge_requests/resolve_conflicts.md): + Your Git diff tool right from GitLab's UI + - [Review Apps](../../ci/review_apps/index.md): Live preview the results + of the changes proposed in a merge request in a per-branch basis +- [Labels](labels.md): Organize issues and merge requests by labels +- [Time Tracking](../../workflow/time_tracking.md): Track estimate time +and time spent on + the conclusion of an issue or merge request +- [Milestones](milestones/index.md): Work towards a target date +- [Description templates](description_templates.md): Define context-specific +templates for issue and merge request description fields for your project +- [Slash commands (quick actions)](quick_actions.md): Textual shortcuts for +common actions on issues or merge requests + +**GitLab CI/CD:** + +- [GitLab CI/CD](../../ci/README.md): GitLab's built-in [Continuous Integration, Delivery, and Deployment](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) tool + - [Container Registry](container_registry.md): Build and push Docker + images out-of-the-box + - [Auto Deploy](../../ci/autodeploy/index.md): Configure GitLab CI/CD + to automatically set up your app's deployment + - [Enable and disable GitLab CI](../../ci/enable_or_disable_ci.md) + - [Pipelines](../../ci/pipelines.md#pipelines): Configure and visualize + your GitLab CI/CD pipelines from the UI + - [Scheduled Pipelines](pipelines/schedules.md): Schedule a pipeline + to start at a chosen time + - [Pipeline Graphs](../../ci/pipelines.md#pipeline-graphs): View your + entire pipeline from the UI + - [Job artifacts](pipelines/job_artifacts.md): Define, + browse, and download job artifacts + - [Pipeline settings](pipelines/settings.md): Set up Git strategy (choose the default way your repository is fetched from GitLab in a job), + timeout (defines the maximum amount of time in minutes that a job is able run), custom path for `.gitlab-ci.yml`, test coverage parsing, pipeline's visibility, and much more +- [GitLab Pages](pages/index.md): Build, test, and deploy your static +website with GitLab Pages + +**Other features:** + +- [Cycle Analytics](cycle_analytics.md): Review your development lifecycle +- [Koding integration](koding.md) (not available on GitLab.com): Integrate +with Koding to have access to a web terminal right from the GitLab UI +- [Syntax highlighting](highlighting.md): An alternative to customize +your code blocks, overriding GitLab's default choice of language + +### Project's integrations + +[Integrate your project](integrations/index.md) with Jira, Mattermost, +Kubernetes, Slack, and a lot more. + +## New project + +Learn how to [create a new project](../../gitlab-basics/create-project.md) in GitLab. + +### Fork a project + +You can [fork a project](../../gitlab-basics/fork-project.md) in order to: + +- Collaborate on code by forking a project and creating a merge request +from your fork to the upstream project +- Fork a sample project to work on the top of that + +## Import or export a project + +- Import a project from: + - [GitHub to GitLab](../../workflow/importing/import_projects_from_github.md) + - [BitBucket to GitLab](../../workflow/importing/import_projects_from_bitbucket.md) + - [Gitea to GitLab](../../workflow/importing/import_projects_from_gitea.md) + - [FogBugz to GitLab](../../workflow/importing/import_projects_from_fogbugz.md) +- [Export a project from GitLab](settings/import_export.md#exporting-a-project-and-its-data) +- [Importing and exporting projects between GitLab instances](settings/import_export.md) + +## Leave a project + +**Leave project** will only display on the project's dashboard +when a project is part of a group (under a +[group namespace](../group/index.md#namespaces)). +If you choose to leave a project you will no longer be a project +member, therefore, unable to contribute. diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index 0b219e84478..ba2adc1afda 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -16,3 +16,16 @@ Once you have configured and enabled Bugzilla: - the **Issues** link on the GitLab project pages takes you to the appropriate Bugzilla product page - clicking **New issue** on the project dashboard takes you to Bugzilla for entering a new issue + +## Referencing issues in Bugzilla + +Issues in Bugzilla can be referenced in two alternative ways: +1. `#<ID>` where `<ID>` is a number (example `#143`). +2. `<PROJECT>-<ID>` where `<PROJECT>` starts with a capital letter which is + then followed by capital letters, numbers or underscores, and `<ID>` is + a number (example `API_32-143`). + +We suggest using the longer format if you have both internal and external issue trackers enabled. If you use the shorter format and an issue with the same ID exists in the internal issue tracker the internal issue will be linked. + +Please note that `<PROJECT>` part is ignored and links always point to the +address specified in `issues_url`. diff --git a/doc/user/project/integrations/img/webhook_testing.png b/doc/user/project/integrations/img/webhook_testing.png Binary files differnew file mode 100644 index 00000000000..176dcec9d8a --- /dev/null +++ b/doc/user/project/integrations/img/webhook_testing.png diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index cf03f2a9033..4f583879a4e 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -1,18 +1,22 @@ # GitLab JIRA integration -GitLab can be configured to interact with JIRA. Configuration happens via -user name and password. Connecting to a JIRA server via CAS is not possible. +GitLab can be configured to interact with [JIRA], a project management platform. -Each project can be configured to connect to a different JIRA instance, see the -[configuration](#configuration) section. If you have one JIRA instance you can -pre-fill the settings page with a default template. To configure the template -see the [Services Templates][services-templates] document. +Once your GitLab project is connected to JIRA, you can reference and close the +issues in JIRA directly from GitLab. -Once the project is connected to JIRA, you can reference and close the issues -in JIRA directly from GitLab. +For a use case, check out this article of [How and why to integrate GitLab with +JIRA](https://www.programmableweb.com/news/how-and-why-to-integrate-gitlab-jira/how-to/2017/04/25). ## Configuration +Each GitLab project can be configured to connect to a different JIRA instance. +If you have one JIRA instance you can pre-fill the settings page with a default +template, see the [Services Templates][services-templates] docs. + +Configuration happens via user name and password. Connecting to a JIRA server +via CAS is not possible. + In order to enable the JIRA service in GitLab, you need to first configure the project in JIRA and then enter the correct values in GitLab. @@ -98,11 +102,11 @@ in the table below. | Field | Description | | ----- | ----------- | | `Web URL` | The base URL to the JIRA instance web interface which is being linked to this GitLab project. E.g., `https://jira.example.com`. | -| `JIRA API URL` | The base URL to the JIRA instance API. E.g., `https://jira-api.example.com`. This is optional. If not entered, the Web URL value be used. | +| `JIRA API URL` | The base URL to the JIRA instance API. Web URL value will be used if not set. E.g., `https://jira-api.example.com`. | | `Project key` | Put a JIRA project key (in uppercase), e.g. `MARS` in this field. This is only for testing the configuration settings. JIRA integration in GitLab works with _all_ JIRA projects in your JIRA instance. This field will be removed in a future release. | | `Username` | The user name created in [configuring JIRA step](#configuring-jira). | | `Password` |The password of the user created in [configuring JIRA step](#configuring-jira). | -| `JIRA issue transition` | This is the ID of a transition that moves issues to a closed state. You can find this number under JIRA workflow administration ([see screenshot](img/jira_workflow_screenshot.png)). **Closing JIRA issues via commits or Merge Requests won't work if you don't set the ID correctly.** | +| `Transition ID` | This is the ID of a transition that moves issues to a closed state. You can find this number under JIRA workflow administration ([see screenshot](img/jira_workflow_screenshot.png)). **Closing JIRA issues via commits or Merge Requests won't work if you don't set the ID correctly.** | After saving the configuration, your GitLab project will be able to interact with all JIRA projects in your JIRA instance. @@ -213,3 +217,4 @@ your project needs to close a ticket. [services-templates]: services_templates.md [jira-repo-old-docs]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-13-stable/doc/project_services/jira.md +[jira]: https://www.atlassian.com/software/jira diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index 73fa83d72a8..f4000523938 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -19,10 +19,10 @@ of your project and select the **Kubernetes** service to configure it. The Kubernetes service takes the following arguments: -1. Kubernetes namespace 1. API URL -1. Service token 1. Custom CA bundle +1. Kubernetes namespace +1. Service token The API URL is the URL that GitLab uses to access the Kubernetes API. Kubernetes exposes several APIs - we want the "base" URL that is common to all of them, @@ -55,6 +55,7 @@ GitLab CI build environment: - `KUBE_CA_PEM_FILE` - only present if a custom CA bundle was specified. Path to a file containing PEM data. - `KUBE_CA_PEM` (deprecated)- only if a custom CA bundle was specified. Raw PEM data. +- `KUBECONFIG` - Path to a file containing kubeconfig for this deployment. CA bundle would be embedded if specified. ## Web terminals diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 86ceb14b965..6f15765751c 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -17,35 +17,30 @@ the settings page with a default template. To configure the template, see the Integration with Prometheus requires the following: 1. GitLab 9.0 or higher -1. The [Kubernetes integration must be enabled][kube] on your project -1. Your app must be deployed on [Kubernetes][] -1. Prometheus must be configured to collect Kubernetes metrics +1. Prometheus must be configured to collect one of the [supported metrics](prometheus_library/metrics.md) 1. Each metric must be have a label to indicate the environment -1. GitLab must have network connectivity to the Prometheus sever +1. GitLab must have network connectivity to the Prometheus server -There are a few steps necessary to set up integration between Prometheus and -GitLab. +## Getting started with Prometheus monitoring -## Configuring Prometheus to collect Kubernetes metrics +Depending on your deployment and where you have located your GitLab server, there are a few options to get started with Prometheus monitoring. -In order for Prometheus to collect Kubernetes metrics, you first must have a -Prometheus server up and running. You have two options here: +* If both GitLab and your applications are installed in the same Kubernetes cluster, you can leverage the [bundled Prometheus server within GitLab](#configuring-omnibus-gitlab-prometheus-to-monitor-kubernetes). +* If your applications are deployed on Kubernetes, but GitLab is not in the same cluster, then you can [configure a Prometheus server in your Kubernetes cluster](#configuring-your-own-prometheus-server-within-kubernetes). +* If your applications are not running in Kubernetes, [get started with Prometheus](#getting-started-with-prometheus-outside-of-kubernetes). -- If you installed Omnibus GitLab inside of Kubernetes, you can simply use the - [bundled version of Prometheus][promgldocs]. In that case, follow the info in the - [Omnibus GitLab section](#configuring-omnibus-gitlab-prometheus-to-monitor-kubernetes) - below. -- If you are using GitLab.com or installed GitLab outside of Kubernetes, you - will likely need to run a Prometheus server within the Kubernetes cluster. - Once installed, the easiest way to monitor Kubernetes is to simply use - Prometheus' support for [Kubernetes Service Discovery][prometheus-k8s-sd]. - In that case, follow the instructions on - [configuring your own Prometheus server within Kubernetes](#configuring-your-own-prometheus-server-within-kubernetes). +### Getting started with Prometheus outside of Kubernetes -### Configuring Omnibus GitLab Prometheus to monitor Kubernetes +Installing and configuring Prometheus to monitor applications is fairly straight forward. + +1. [Install Prometheus](https://prometheus.io/docs/introduction/install/) +1. Set up one of the [supported monitoring targets](prometheus_library/metrics.md) +1. Configure the Prometheus server to [collect their metrics](https://prometheus.io/docs/operating/configuration/#scrape_config) + +### Configuring Omnibus GitLab Prometheus to monitor Kubernetes deployments With Omnibus GitLab running inside of Kubernetes, you can leverage the bundled -version of Prometheus to collect the required metrics. +version of Prometheus to collect the supported metrics. Once enabled, Prometheus will automatically begin monitoring Kubernetes Nodes and any [annotated Pods](https://prometheus.io/docs/operating/configuration/#<kubernetes_sd_config>). 1. Read how to configure the bundled Prometheus server in the [Administration guide][gitlab-prometheus-k8s-monitor]. @@ -74,7 +69,7 @@ kubectl apply -f path/to/prometheus.yml Once deployed, you should see the Prometheus service, deployment, and pod start within the `prometheus` namespace. The server will begin to collect metrics from each Kubernetes Node in the cluster, based on the configuration -provided in the template. +provided in the template. It will also attempt to collect metrics from any Kubernetes Pods that have been [annotated for Prometheus](https://prometheus.io/docs/operating/configuration/#pod). Since GitLab is not running within Kubernetes, the template provides external network access via a `NodePort` running on `30090`. This method allows access @@ -133,30 +128,6 @@ to integrate with.  -## Metrics and Labels - -GitLab retrieves performance data from two metrics, `container_cpu_usage_seconds_total` -and `container_memory_usage_bytes`. These metrics are collected from the -Kubernetes pods via Prometheus, and report CPU and Memory utilization of each -container or Pod running in the cluster. - -In order to isolate and only display relevant metrics for a given environment -however, GitLab needs a method to detect which pods are associated. To do that, -GitLab will specifically request metrics that have an `environment` tag that -matches the [$CI_ENVIRONMENT_SLUG][ci-environment-slug]. - -If you are using [GitLab Auto-Deploy][autodeploy] and one of the methods of -configuring Prometheus above, the `environment` will be automatically added. - -### GitLab Prometheus queries - -The queries utilized by GitLab are shown in the following table. - -| Metric | Query | -| ------ | ----- | -| Average Memory (MB) | `(sum(container_memory_usage_bytes{container_name!="POD",environment="$CI_ENVIRONMENT_SLUG"}) / count(container_memory_usage_bytes{container_name!="POD",environment="$CI_ENVIRONMENT_SLUG"})) /1024/1024` | -| Average CPU Utilization (%) | `sum(rate(container_cpu_usage_seconds_total{container_name!="POD",environment="$CI_ENVIRONMENT_SLUG"}[2m])) / count(container_cpu_usage_seconds_total{container_name!="POD",environment="$CI_ENVIRONMENT_SLUG"}) * 100` | - ## Monitoring CI/CD Environments Once configured, GitLab will attempt to retrieve performance metrics for any @@ -168,8 +139,9 @@ environment which has had a successful deployment. > [Introduced][ce-10408] in GitLab 9.2. > GitLab 9.3 added the [numeric comparison](https://gitlab.com/gitlab-org/gitlab-ce/issues/27439) of the 30 minute averages. +> Requires [Kubernetes](prometheus_library/kubernetes.md) metrics -Developers can view the performance impact of their changes within the merge +Developers can view theperformance impact of their changes within the merge request workflow. When a source branch has been deployed to an environment, a sparkline and numeric comparison of the average memory consumption will appear. On the sparkline, a dot indicates when the current changes were deployed, with up to 30 minutes of performance data displayed before and after. The comparison shows the difference between the 30 minute average before and after the deployment. This information is updated after diff --git a/doc/user/project/integrations/prometheus_library/cloudwatch.md b/doc/user/project/integrations/prometheus_library/cloudwatch.md new file mode 100644 index 00000000000..cc5cee36d28 --- /dev/null +++ b/doc/user/project/integrations/prometheus_library/cloudwatch.md @@ -0,0 +1,25 @@ +# Monitoring AWS Resources +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 + +GitLab has support for automatically detecting and monitoring AWS resources, starting with the [Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/). This is provided by leveraging the official [Cloudwatch exporter](https://github.com/prometheus/cloudwatch_exporter), which translates [Cloudwatch metrics](https://aws.amazon.com/cloudwatch/) into a Prometheus readable form. + +## Metrics supported + +| Name | Query | +| ---- | ----- | +| Throughput (req/sec) | sum(aws_elb_request_count_sum{%{environment_filter}}) / 60 | +| Latency (ms) | avg(aws_elb_latency_average{%{environment_filter}}) * 1000 | +| HTTP Error Rate (%) | sum(aws_elb_httpcode_backend_5_xx_sum{%{environment_filter}}) / sum(aws_elb_request_count_sum{%{environment_filter}}) | + +## Configuring Prometheus to monitor for Cloudwatch metrics + +To get started with Cloudwatch monitoring, you should install and configure the [Cloudwatch exporter](https://github.com/hnlq715/nginx-vts-exporter) which retrieves and parses the specified Cloudwatch metrics and translates them into a Prometheus monitoring endpoint. + +Right now, the only AWS resource supported is the Elastic Load Balancer, whose Cloudwatch metrics can be found [here](http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-cloudwatch-metrics.html). + +A sample Cloudwatch Exporter configuration file, configured for basic AWS ELB monitoring, is [available for download](../samples/cloudwatch.yml). + +## Specifying the Environment label + +In order to isolate and only display relevant metrics for a given environment +however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md new file mode 100644 index 00000000000..f2939f047a3 --- /dev/null +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -0,0 +1,20 @@ +# Monitoring HAProxy +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 + +GitLab has support for automatically detecting and monitoring HAProxy. This is provided by leveraging the [HAProxy Exporter](https://github.com/prometheus/haproxy_exporter), which translates HAProxy statistics into a Prometheus readable form. + +## Metrics supported + +| Name | Query | +| ---- | ----- | +| Throughput (req/sec) | sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | +| HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_requests_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | + +## Configuring Prometheus to monitor for HAProxy metrics + +To get started with NGINX monitoring, you should install and configure the [HAProxy exporter](https://github.com/prometheus/haproxy_exporter) which parses these statistics and translates them into a Prometheus monitoring endpoint. + +## Specifying the Environment label + +In order to isolate and only display relevant metrics for a given environment +however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md new file mode 100644 index 00000000000..eb8cd821ddc --- /dev/null +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -0,0 +1,26 @@ +# Monitoring Kubernetes +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8935) in GitLab 9.0 + +GitLab has support for automatically detecting and monitoring Kubernetes metrics. Kubernetes exposes Node level metrics out of the box via the built-in [Prometheus metrics support in cAdvisor](https://github.com/google/cadvisor). No additional services or exporters are needed. + +## Metrics supported + +| Name | Query | +| ---- | ----- | +| Average Memory Usage (MB) | (sum(container_memory_usage_bytes{container_name!="POD",%{environment_filter}}) / count(container_memory_usage_bytes{container_name!="POD",%{environment_filter}})) /1024/1024 | +| Average CPU Utilization (%) | sum(rate(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}[2m])) / count(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}) * 100 | + +## Configuring Prometheus to monitor for Kubernetes node metrics + +In order for Prometheus to collect Kubernetes metrics, you first must have a +Prometheus server up and running. You have two options here: + +- If you have an Omnibus based GitLab installation within your Kubernetes cluster, you can leverage the bundled Prometheus server to [monitor Kubernetes](../../../../administration/monitoring/prometheus/index.md#configuring-prometheus-to-monitor-kubernetes). +- To configure your own Prometheus server, you can follow the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/) or [our guide](../../../../administration/monitoring/prometheus/index.md#configuring-your-own-prometheus-server-within-kubernetes). + +## Specifying the Environment label + +In order to isolate and only display relevant metrics for a given environment +however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). + +If you are using [GitLab Auto-Deploy][autodeploy] and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. diff --git a/doc/user/project/integrations/prometheus_library/metrics.md b/doc/user/project/integrations/prometheus_library/metrics.md new file mode 100644 index 00000000000..6bdffce9c55 --- /dev/null +++ b/doc/user/project/integrations/prometheus_library/metrics.md @@ -0,0 +1,26 @@ +# Prometheus Metrics library +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8935) in GitLab 9.0 + +GitLab offers automatic detection of select [Prometheus exporters](https://prometheus.io/docs/instrumenting/exporters/). Currently supported exporters are: +* [Kubernetes](kubernetes.md) +* [NGINX](nginx.md) +* [HAProxy](haproxy.md) +* [Amazon Cloud Watch](cloudwatch.md) + +We have tried to surface the most important metrics for each exporter, and will be continuing to add support for additional exporters in future releases. If you would like to add support for other official exporters, [contributions](#adding-to-the-library) are welcome. + +## Identifying Environments + +GitLab retrieves performance data from the configured Prometheus server, and attempts to identifying the presence of known metrics. Once identified, GitLab then needs to be able to map the data to a particular environment. + +In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do that, +GitLab will look for the required metrics which have a label that +matches the [$CI_ENVIRONMENT_SLUG][ci-environment-slug]. + +For example if you are deploying to an environment named `production`, there must be a label for the metric with the value of `production`. + +## Adding to the library + +We strive to support the 2-4 most important metrics for each common system service that supports Prometheus. If you are looking for support for a particular exporter which has not yet been added to the library, additions can be made [to the `additional_metrics.yml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/prometheus/additional_metrics.yml) file. + +> Note: The library is only for monitoring public, common, system services which all customers can benefit from. Support for monitoring [customer proprietary metrics](https://gitlab.com/gitlab-org/gitlab-ee/issues/2273) will be added in a subsequent release. diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md new file mode 100644 index 00000000000..b3470773996 --- /dev/null +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -0,0 +1,23 @@ +# Monitoring NGINX +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 + +GitLab has support for automatically detecting and monitoring NGINX. This is provided by leveraging the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter), which translates [VTS statistics](https://github.com/vozlt/nginx-module-vts) into a Prometheus readable form. + +## Metrics supported + +| Name | Query | +| ---- | ----- | +| Throughput (req/sec) | sum(rate(nginx_requests_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) | +| Latency (ms) | avg(nginx_upstream_response_msecs_avg{%{environment_filter}}) * 1000 | +| HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_responses_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_responses_total{%{environment_filter}}[2m])) | + +## Configuring Prometheus to monitor for NGINX metrics + +To get started with NGINX monitoring, you should first enable the [VTS statistics](https://github.com/vozlt/nginx-module-vts)) module for your NGINX server. This will capture and display statistics in an HTML readable form. Next, you should install and configure the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter) which parses these statistics and translates them into a Prometheus monitoring endpoint. + +If you are using NGINX as your Kubernetes ingress, there is [upcoming direct support](https://github.com/kubernetes/ingress/pull/423) for enabling Prometheus monitoring in the 0.9.0 release. + +## Specifying the Environment label + +In order to isolate and only display relevant metrics for a given environment +however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index 89c0312d3c2..cf92465da53 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -21,3 +21,16 @@ Once you have configured and enabled Redmine: As an example, below is a configuration for a project named gitlab-ci.  + +## Referencing issues in Redmine + +Issues in Redmine can be referenced in two alternative ways: +1. `#<ID>` where `<ID>` is a number (example `#143`) +2. `<PROJECT>-<ID>` where `<PROJECT>` starts with a capital letter which is + then followed by capital letters, numbers or underscores, and `<ID>` is + a number (example `API_32-143`). + +We suggest using the longer format if you have both internal and external issue trackers enabled. If you use the shorter format and an issue with the same ID exists in the internal issue tracker the internal issue will be linked. + +Please note that `<PROJECT>` part is ignored and links always point to the +address specified in `issues_url`. diff --git a/doc/user/project/integrations/samples/cloudwatch.yml b/doc/user/project/integrations/samples/cloudwatch.yml new file mode 100644 index 00000000000..d9b58f52c32 --- /dev/null +++ b/doc/user/project/integrations/samples/cloudwatch.yml @@ -0,0 +1,26 @@ +region: us-east-1 + metrics: + - aws_namespace: AWS/ELB + aws_metric_name: RequestCount + aws_dimensions: [AvailabilityZone, LoadBalancerName] + aws_dimension_select: + LoadBalancerName: [gitlab-ha-lb] + aws_statistics: [Sum] + - aws_namespace: AWS/ELB + aws_metric_name: Latency + aws_dimensions: [AvailabilityZone, LoadBalancerName] + aws_dimension_select: + LoadBalancerName: [gitlab-ha-lb] + aws_statistics: [Average] + - aws_namespace: AWS/ELB + aws_metric_name: HTTPCode_Backend_2XX + aws_dimensions: [AvailabilityZone, LoadBalancerName] + aws_dimension_select: + LoadBalancerName: [gitlab-ha-lb] + aws_statistics: [Sum] + - aws_namespace: AWS/ELB + aws_metric_name: HTTPCode_Backend_5XX + aws_dimensions: [AvailabilityZone, LoadBalancerName] + aws_dimension_select: + LoadBalancerName: [gitlab-ha-lb] + aws_statistics: [Sum] diff --git a/doc/user/project/integrations/samples/prometheus.yml b/doc/user/project/integrations/samples/prometheus.yml index 01bbcaffe1e..30b59e172a1 100644 --- a/doc/user/project/integrations/samples/prometheus.yml +++ b/doc/user/project/integrations/samples/prometheus.yml @@ -24,6 +24,44 @@ data: target_label: environment regex: (.+)-.+-.+ replacement: $1 + - job_name: kubernetes-pods + tls_config: + ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" + insecure_skip_verify: true + bearer_token_file: "/var/run/secrets/kubernetes.io/serviceaccount/token" + kubernetes_sd_configs: + - role: pod + api_server: https://kubernetes.default.svc:443 + tls_config: + ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" + bearer_token_file: "/var/run/secrets/kubernetes.io/serviceaccount/token" + relabel_configs: + - source_labels: + - __meta_kubernetes_pod_annotation_prometheus_io_scrape + action: keep + regex: 'true' + - source_labels: + - __meta_kubernetes_pod_annotation_prometheus_io_path + action: replace + target_label: __metrics_path__ + regex: "(.+)" + - source_labels: + - __address__ + - __meta_kubernetes_pod_annotation_prometheus_io_port + action: replace + regex: "([^:]+)(?::[0-9]+)?;([0-9]+)" + replacement: "$1:$2" + target_label: __address__ + - action: labelmap + regex: __meta_kubernetes_pod_label_(.+) + - source_labels: + - __meta_kubernetes_namespace + action: replace + target_label: kubernetes_namespace + - source_labels: + - __meta_kubernetes_pod_name + action: replace + target_label: kubernetes_pod_name --- apiVersion: v1 kind: Service diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 0517ed3ec18..c03a2df9a72 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -736,7 +736,7 @@ X-Gitlab-Event: Merge Request Hook ### Wiki Page events -Triggered when a wiki page is created, edited or deleted. +Triggered when a wiki page is created, updated or deleted. **Request Header**: @@ -1014,6 +1014,13 @@ X-Gitlab-Event: Build Hook } ``` +## Testing webhooks + +You can trigger the webhook manually. Sample data from the project will be used.Sample data will take from the project. +> For example: for triggering `Push Events` your project should have at least one commit. + + + ## Troubleshoot webhooks Gitlab stores each perform of the webhook. @@ -1056,7 +1063,7 @@ Pick an unused port (e.g. 8000) and start the script: `ruby print_http_body.rb 8000`. Then add your server as a webhook receiver in GitLab as `http://my.host:8000/`. -When you press 'Test Hook' in GitLab, you should see something like this in the +When you press 'Test' in GitLab, you should see something like this in the console: ``` diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index ebea7062ecb..e2cc67726e0 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -1,8 +1,7 @@ -# Issue board +# Issue Board ->**Notes:** -- [Introduced][ce-5554] in GitLab 8.11. -- The Backlog column was replaced by the **Add issues** button in GitLab 8.17. +>**Note:** +[Introduced][ce-5554] in [GitLab 8.11](https://about.gitlab.com/2016/08/22/gitlab-8-11-released/#issue-board). The GitLab Issue Board is a software project management tool used to plan, organize, and visualize a workflow for a feature or product release. @@ -15,12 +14,65 @@ Other interesting links: ## Overview -The Issue Board builds on GitLab's existing issue tracking functionality and +The Issue Board builds on GitLab's existing +[issue tracking functionality](issues/index.md#issue-tracker) and leverages the power of [labels] by utilizing them as lists of the scrum board. -With the Issue Board you can have a different view of your issues while also +With the Issue Board you can have a different view of your issues while maintaining the same filtering and sorting abilities you see across the -issue tracker. +issue tracker. An Issue Board is based on its project's label structure, therefore, it +applies the same descriptive labels to indicate placement on the board, keeping +consistency throughout the entire development lifecycle. + +An Issue Board shows you what issues your team is working on, who is assigned to each, +and where in the workflow those issues are. + +You create issues, host code, perform reviews, build, test, +and deploy from one single platform. Issue Boards help you to visualize +and manage the entire process _in_ GitLab. + +With [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards), available +only in [GitLab Enterprise Edition](https://about.gitlab.com/gitlab-ee/), +you go even further, as you can not only keep yourself and your project +organized from a broader perspective with one Issue Board per project, +but also allow your team members to organize their own workflow by creating +multiple Issue Boards within the same project. + +## Use cases + +GitLab Workflow allows you to discuss proposals in issues, categorize them +with labels, and from there organize and prioritize them with Issue Boards. + +For example, let's consider this simplified development workflow: + +1. You have a repository hosting your app's codebase +and your team actively contributing to code +1. Your **backend** team starts working a new +implementation, gathers feedback and approval, and pass it over to **frontend** +1. When frontend is complete, the new feature is deployed to **staging** to be tested +1. When successful, it is deployed to **production** + +If we have the labels "**backend**", "**frontend**", "**staging**", and +"**production**", and an Issue Board with a list for each, we can: + +- Visualize the entire flow of implementations since the +beginning of the development lifecycle until deployed to production +- Prioritize the issues in a list by moving them vertically +- Move issues between lists to organize them according to the labels you've set +- Add multiple issues to lists in the board by selecting one or more existing issues + + + +> **Notes:** +> +>- For a broader use case, please check the blog post +[GitLab Workflow, an Overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/#gitlab-workflow-use-case-scenario). +> +>- For a real use case, please check why +[Codepen decided to adopt Issue Boards](https://about.gitlab.com/2017/01/27/codepen-welcome-to-gitlab/#project-management-everything-in-one-place) +to improve their workflow with [multiple boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards). + +## Issue Board terminology Below is a table of the definitions used for GitLab's Issue Board. @@ -57,7 +109,7 @@ In short, here's a list of actions you can take in an Issue Board: If you are not able to perform one or more of the things above, make sure you have the right [permissions](#permissions). -## First time using the issue board +## First time using the Issue Board The first time you navigate to your Issue Board, you will be presented with a default list (**Done**) and a welcoming message that gives @@ -98,7 +150,7 @@ list view that is removed. You can always add it back later if you need. ## Adding issues to a list You can add issues to a list by clicking the **Add issues** button that is -present in the upper right corner of the issue board. This will open up a modal +present in the upper right corner of the Issue Board. This will open up a modal window where you can see all the issues that do not belong to any list. Select one or more issues by clicking on the cards and then click **Add issues** diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index fe87e6f9495..1f78849a92c 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -1,17 +1,17 @@ -# Issues documentation +# Issues The GitLab Issue Tracker is an advanced and complete tool for tracking the evolution of a new idea or the process of solving a problem. It allows you, your team, and your collaborators to share -and discuss proposals, before and while implementing them. +and discuss proposals before and while implementing them. Issues and the GitLab Issue Tracker are available in all [GitLab Products](https://about.gitlab.com/products/) as part of the [GitLab Workflow](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/). -## Use-Cases +## Use cases Issues can have endless applications. Just to exemplify, these are some cases for which creating issues are most used: @@ -23,7 +23,28 @@ some cases for which creating issues are most used: - Obtaining support - Elaborating new code implementations -See also the blog post [Always start a discussion with an issue](https://about.gitlab.com/2016/03/03/start-with-an-issue/). +See also the blog post "[Always start a discussion with an issue](https://about.gitlab.com/2016/03/03/start-with-an-issue/)". + +### Keep private things private + +For instance, let's assume you have a public project but want to start a discussion on something +you don't want to be public. With [Confidential Issues](#confidential-issues), +you can discuss private matters among the project members, and still keep +your project public, open to collaboration. + +### Streamline collaboration + +With [Multiple Assignees for Issues](https://docs.gitlab.com/ee/user/project/issues/multiple_assignees_for_issues.html), +available in [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/) +you can streamline collaboration and allow shared responsibilities to be clearly displayed. +All assignees are shown across your workflows and receive notifications (as they +would as single assignees), simplifying communication and ownership. + +### Consistent collaboration + +Create [issue templates](#issue-templates) to make collaboration consistent and +containing all information you need. For example, you can create a template +for feature proposals and another one for bug reports. ## Issue Tracker @@ -96,8 +117,8 @@ Find GitLab Issue Boards by navigating to your **Project's Dashboard** > **Issue Read through the documentation for [Issue Boards](../issue_board.md) to find out more about this feature. -[Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards) -are available only in [GitLab Enterprise Edition](https://about.gitlab.com/gitlab-ee/). +With [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/), you can also +create various boards per project with [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards). ### Issue's API diff --git a/doc/user/project/issues/issues_functionalities.md b/doc/user/project/issues/issues_functionalities.md index 294176e61f9..074b2c19c43 100644 --- a/doc/user/project/issues/issues_functionalities.md +++ b/doc/user/project/issues/issues_functionalities.md @@ -43,7 +43,7 @@ assigned to them if they created the issue themselves. ##### 3.1. Multiple Assignees (EES/EEP) -Issue Weights are only available in [GitLab Enterprise Edition](https://about.gitlab.com/gitlab-ee/). +Multiple Assignees are only available in [GitLab Enterprise Edition](https://about.gitlab.com/gitlab-ee/). Often multiple people likely work on the same issue together, which can especially be difficult to track in large teams @@ -52,25 +52,21 @@ where there is shared ownership of an issue. In GitLab Enterprise Edition, you can also select multiple assignees to an issue. -> **Note:** -Multiple Assignees was [introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/1904) -in [GitLab Enterprise Edition 9.2](https://about.gitlab.com/2017/05/22/gitlab-9-2-released/#multiple-assignees-for-issues). +Learn more on the [Multiple Assignees documentation](https://docs.gitlab.com/ee/user/project/issues/multiple_assignees_for_issues.html). #### 4. Milestone - Select a [milestone](../milestones/index.md) to attribute that issue to. -#### 5. Time Tracking (EES/EEP) - -This feature is available only in [GitLab Enterprise Edition](https://about.gitlab.com/gitlab-ee/). +#### 5. Time Tracking - Estimate time: add an estimate time in which the issue will be implemented - Spend: add the time spent on the implementation of that issue > **Note:** -both estimate and spend times are set via [GitLab Quick Actions](../quick_actions.md). +Both estimate and spend times are set via [GitLab Quick Actions](../quick_actions.md). -Learn more on the [Time Tracking documentation](https://docs.gitlab.com/ee/workflow/time_tracking.html). +Learn more on the [Time Tracking documentation](../../../workflow/time_tracking.md). #### 6. Due date diff --git a/doc/user/project/koding.md b/doc/user/project/koding.md index c56a1efe3c2..455e2ee47b4 100644 --- a/doc/user/project/koding.md +++ b/doc/user/project/koding.md @@ -1,4 +1,4 @@ -# Koding & GitLab +# Koding integration > [Introduced][ce-5909] in GitLab 8.11. diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 954454f7e7a..9bdf2a998d3 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -3,6 +3,59 @@ Merge requests allow you to exchange changes you made to source code and collaborate with other people on the same project. +## Overview + +A Merge Request (**MR**) is the basis of GitLab as a code collaboration +and version control platform. +Is it simple as the name implies: a _request_ to _merge_ one branch into another. + +With GitLab merge requests, you can: + +- Compare the changes between two [branches](https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell#_git_branching) +- [Review and discuss](../../discussions/index.md#discussions) the proposed modifications inline +- Live preview the changes when [Review Apps](../../../ci/review_apps/index.md) is configured for your project +- Build, test, and deploy your code in a per-branch basis with built-in [GitLab CI/CD](../../../ci/README.md) +- Prevent the merge request from being merged before it's ready with [WIP MRs](#work-in-progress-merge-requests) +- View the deployment process through [Pipeline Graphs](../../../ci/pipelines.md#pipeline-graphs) +- [Automatically close the issue(s)](../../project/issues/closing_issues.md#via-merge-request) that originated the implementation proposed in the merge request +- Assign it to any registered user, and change the assignee how many times you need +- Assign a [milestone](../../project/milestones/index.md) and track the development of a broader implementation +- Organize your issues and merge requests consistently throughout the project with [labels](../../project/labels.md) +- Add a time estimation and the time spent with that merge request with [Time Tracking](../../../workflow/time_tracking.html#time-tracking) +- [Resolve merge conflicts from the UI](#resolve-conflicts) + +With **[GitLab Enterprise Edition][ee]**, you can also: + +- View the deployment process across projects with [Multi-Project Pipeline Graphs](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html#multi-project-pipeline-graphs) (available only in GitLab Enterprise Edition Premium) +- Request [approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your managers (available in GitLab Enterprise Edition Starter) +- Enable [fast-forward merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/fast_forward_merge.html) (available in GitLab Enterprise Edition Starter) +- [Squash and merge](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html) for a cleaner commit history (available in GitLab Enterprise Edition Starter) +- Enable [semi-linear history merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/index.html#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch (available in GitLab Enterprise Edition Starter) +- Analise the impact of your changes with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) (available in GitLab Enterprise Edition Starter) + +## Use cases + +A. Consider you are a software developer working in a team: + +1. You checkout a new branch, and submit your changes through a merge request +1. You gather feedback from your team +1. You work on the implementation optimizing code with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) (available in GitLab Enterprise Edition Starter) +1. You build and test your changes with GitLab CI/CD +1. You request the approval from your manager +1. Your manager pushes a commit with his final review, [approves the merge request](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html), and set it to [merge when pipeline succeeds](#merge-when-pipeline-succeeds) (Merge Request Approvals are available in GitLab Enterprise Edition Starter) +1. Your changes get deployed to production with [manual actions](../../../ci/yaml/README.md#manual-actions) for GitLab CI/CD +1. Your implementations were successfully shipped to your customer + +B. Consider you're a web developer writing a webpage for your company's: + +1. You checkout a new branch, and submit a new page through a merge request +1. You gather feedback from your reviewers +1. Your changes are previewed with [Review Apps](../../../ci/review_apps/index.md) +1. You request your web designers for their implementation +1. You request the [approval](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your manager (available in GitLab Enterprise Edition Starter) +1. Once approved, your merge request is [squashed and merged](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html), and [deployed to staging with GitLab Pages](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) (Squash and Merge is available in GitLab Enterprise Edition Starter) +1. Your production team [cherry picks](#cherry-pick-changes) the merge commit into production + ## Authorization for merge requests There are two main ways to have a merge request flow with GitLab: @@ -79,6 +132,16 @@ specific commit page. You can append `?w=1` while on the diffs page of a merge request to ignore any whitespace changes. +## Live preview with Review Apps + +If you configured [Review Apps](https://about.gitlab.com/features/review-apps/) for your project, +you can preview the changes submitted to a feature-branch through a merge request +in a per-branch basis. No need to checkout the branch, install and preview locally; +all your changes will be available to preview by anyone with the Review Apps link. + +[Read more about Review Apps.](../../../ci/review_apps/index.md) + + ## Tips Here are some tips that will help you be more efficient with merge requests in @@ -167,3 +230,4 @@ git checkout origin/merge-requests/1 ``` [protected branches]: ../protected_branches.md +[ee]: https://about.gitlab.com/gitlab-ee/ "GitLab Enterprise Edition" diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 99233ed5ae2..1848514e2dd 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -21,14 +21,11 @@ Once you fill in all the details, hit the **Create milestone** button. >**Note:** You need [Master permissions](../../permissions.md) in order to create a milestone. -You can create a milestone for several projects in the same group simultaneously. -On the group's **Issues ➔ Milestones** page, you will be able to see the status -of that milestone across all of the selected projects. To create a new milestone -for selected projects in the group, click the **New milestone** button. The -form is the same as when creating a milestone for a specific project with the -addition of the selection of the projects you want to inherit this milestone. - - +You can create a milestone for a group that will be shared across group projects. +On the group's **Issues ➔ Milestones** page, you will be able to see the state +of that milestone and the issues/merge requests count that it shares across the group projects. To create a new milestone click the **New milestone** button. The form is the same as when creating a milestone for a specific project which you can find in the previous item. + +In addition to that you will be able to filter issues or merge requests by group milestones in all projects that belongs to the milestone group. ## Special milestone filters diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 2f104c7becc..46fa4378fe7 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -41,7 +41,7 @@ server up and running for your GitLab instance. Before we begin, let's understand a few concepts first. -### Static sites +## Static sites GitLab Pages only supports static websites, meaning, your output files must be HTML, CSS, and JavaScript only. @@ -51,14 +51,14 @@ CSS, and JS, or use a [Static Site Generator (SSG)](https://www.staticgen.com/) to simplify your code and build the static site for you, which is highly recommendable and much faster than hardcoding. -#### Further Reading +### Further reading - Read through this technical overview on [Static versus Dynamic Websites](https://about.gitlab.com/2016/06/03/ssg-overview-gitlab-pages-part-1-dynamic-x-static/) - Understand [how modern Static Site Generators work](https://about.gitlab.com/2016/06/10/ssg-overview-gitlab-pages-part-2/) and what you can add to your static site - You can use [any SSG with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) - Fork an [example project](https://gitlab.com/pages) to build your website based upon -### GitLab Pages domain +## GitLab Pages domain If you set up a GitLab Pages project on GitLab.com, it will automatically be accessible under a @@ -73,9 +73,9 @@ Pages wildcard domain. This guide is valid for any GitLab instance, you just need to replace Pages wildcard domain on GitLab.com (`*.gitlab.io`) with your own. -#### Practical examples +### Practical examples -**Project Websites:** +#### Project Websites - You created a project called `blog` under your username `john`, therefore your project URL is `https://gitlab.com/john/blog/`. @@ -87,16 +87,21 @@ URL is `https://gitlab.com/websites/blog/`. Once you enable GitLab Pages for this project, the site will live under `https://websites.gitlab.io/blog/`. -**User and Group Websites:** +#### User and Group Websites - Under your username, `john`, you created a project called `john.gitlab.io`. Your project URL will be `https://gitlab.com/john/john.gitlab.io`. Once you enable GitLab Pages for your project, your website will be published under `https://john.gitlab.io`. - Under your group `websites`, you created a project called -`websites.gitlab.io`. your project's URL will be `https://gitlab.com/websites/websites.gitlab.io`. Once you enable GitLab Pages for your project, +`websites.gitlab.io`. your project's URL will be `https://gitlab.com/websites/websites.gitlab.io`. +Once you enable GitLab Pages for your project, your website will be published under `https://websites.gitlab.io`. +>**Note:** +GitLab Pages [does **not** support subgroups](../../group/subgroups/index.md#limitations). +You can only create the highest level group website. + **General example:** - On GitLab.com, a project site will always be available under diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index deaceabb7c5..9ecf7a3a8e7 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -398,6 +398,9 @@ don't redirect HTTP to HTTPS. [rfc]: https://tools.ietf.org/html/rfc2818#section-3.1 "HTTP Over TLS RFC" +GitLab Pages [does **not** support subgroups](../../group/subgroups/index.md#limitations). +You can only create the highest level group website. + ## Redirects in GitLab Pages Since you cannot use any custom server configuration files, like `.htaccess` or diff --git a/doc/user/project/pipelines/img/pipeline_schedule_variables.png b/doc/user/project/pipelines/img/pipeline_schedule_variables.png Binary files differnew file mode 100644 index 00000000000..47a0c6f3697 --- /dev/null +++ b/doc/user/project/pipelines/img/pipeline_schedule_variables.png diff --git a/doc/user/project/pipelines/img/pipeline_schedules_new_form.png b/doc/user/project/pipelines/img/pipeline_schedules_new_form.png Binary files differindex ea5394fa8a6..5a0e5965992 100644 --- a/doc/user/project/pipelines/img/pipeline_schedules_new_form.png +++ b/doc/user/project/pipelines/img/pipeline_schedules_new_form.png diff --git a/doc/user/project/pipelines/schedules.md b/doc/user/project/pipelines/schedules.md index 17cc21238ff..9ad15a12c3c 100644 --- a/doc/user/project/pipelines/schedules.md +++ b/doc/user/project/pipelines/schedules.md @@ -31,6 +31,15 @@ is installed on.  +### Making use of scheduled pipeline variables + +> [Introduced][ce-12328] in GitLab 9.4. + +You can pass any number of arbitrary variables and they will be available in +GitLab CI so that they can be used in your `.gitlab-ci.yml` file. + + + ## Using only and except To configure that a job can be executed only when the pipeline has been @@ -62,9 +71,10 @@ The next time a pipeline is scheduled, your credentials will be used. >**Note:** When the owner of the schedule doesn't have the ability to create pipelines -anymore, due to e.g., being blocked or removed from the project, the schedule -is deactivated. Another user can take ownership and activate it, so the -schedule can be run again. +anymore, due to e.g., being blocked or removed from the project, or lacking +the permission to run on protected branches or tags. When this happened, the +schedule is deactivated. Another user can take ownership and activate it, so +the schedule can be run again. ## Advanced admin configuration @@ -79,4 +89,5 @@ don't have admin access to the server, ask your administrator. [ce-10533]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10533 [ce-10853]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10853 +[ce-12328]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12328 [settings]: https://about.gitlab.com/gitlab-com/settings/#cron-jobs diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index 1d2eba4f74b..3ff5a08d72c 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -1,7 +1,7 @@ # Pipelines settings To reach the pipelines settings navigate to your project's -**Settings ➔ CI/CD Pipelines**. +**Settings ➔ Pipelines**. The following settings can be configured per project. @@ -27,6 +27,22 @@ The default value is 60 minutes. Decrease the time limit if you want to impose a hard limit on your jobs' running time or increase it otherwise. In any case, if the job surpasses the threshold, it is marked as failed. +## Custom CI config path + +> - [Introduced][ce-12509] in GitLab 9.4. + +By default we look for the `.gitlab-ci.yml` file in the project's root +directory. If you require a different location **within** the repository, +you can set a custom filepath that will be used to lookup the config file, +this filepath should be **relative** to the root. + +Here are some valid examples: + +> * .gitlab-ci.yml +> * .my-custom-file.yml +> * my/path/.gitlab-ci.yml +> * my/path/.my-custom-file.yml + ## Test coverage parsing If you use test coverage in your code, GitLab can capture its output in the @@ -59,8 +75,8 @@ pipelines** checkbox and save the changes. > [Introduced][ce-9362] in GitLab 9.1. -If you want to auto-cancel all pending non-HEAD pipelines on branch, when -new pipeline will be created (after your git push or manually from UI), +If you want to auto-cancel all pending non-HEAD pipelines on branch, when +new pipeline will be created (after your git push or manually from UI), check **Auto-cancel pending pipelines** checkbox and save the changes. ## Badges @@ -115,3 +131,4 @@ into your `README.md`: [var]: ../../../ci/yaml/README.md#git-strategy [coverage report]: #test-coverage-parsing [ce-9362]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9362 +[ce-12509]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12509 diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 19b51c83222..ce4dd4e99d5 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -37,3 +37,4 @@ do. | `/target_branch <Branch Name>` | Set target branch for current merge request | | `/award :emoji:` | Toggle award for :emoji: | | `/board_move ~column` | Move issue to column on the board | +| `/duplicate #issue` | Closes this issue and marks it as a duplicate of another issue | diff --git a/doc/user/project/repository/img/compare_branches.png b/doc/user/project/repository/img/compare_branches.png Binary files differnew file mode 100755 index 00000000000..353bd72ef4e --- /dev/null +++ b/doc/user/project/repository/img/compare_branches.png diff --git a/doc/user/project/repository/img/contributors_graph.png b/doc/user/project/repository/img/contributors_graph.png Binary files differnew file mode 100755 index 00000000000..c31da7aa1ff --- /dev/null +++ b/doc/user/project/repository/img/contributors_graph.png diff --git a/doc/user/project/repository/img/repo_graph.png b/doc/user/project/repository/img/repo_graph.png Binary files differnew file mode 100755 index 00000000000..28da8ad9589 --- /dev/null +++ b/doc/user/project/repository/img/repo_graph.png diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md new file mode 100644 index 00000000000..4b2c435a120 --- /dev/null +++ b/doc/user/project/repository/index.md @@ -0,0 +1,150 @@ +# Repository + +A [repository](https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository) +is what you use to store your codebase in GitLab and change it with version control. +A repository is part of a [project](../index.md), which has a lot of other features. + +## Create a repository + +To create a new repository, all you need to do is +[create a new project](../../../gitlab-basics/create-project.md). + +Once you create a new project, you can add new files via UI +(read the section below) or via command line. +To add files from the command line, follow the instructions that will +be presented on the screen when you create a new project, or read +through them in the [command line basics](../../../gitlab-basics/start-using-git.md) +documentation. + +> **Important:** +For security reasons, when using the command line, we strongly recommend +you to [connect with GitLab via SSH](../../../ssh/README.md). + +## Create and edit files + +Host your codebase in GitLab repositories by pushing your files to GitLab. +You can either use the user interface (UI), or connect your local computer +with GitLab [through the command line](../../../gitlab-basics/command-line-commands.md#start-working-on-your-project). + +To configure [GitLab CI/CD](../../../ci/README.md) to build, test, and deploy +you code, add a file called [.`gitlab-ci.yml`](../../../ci/quick_start/README.md) +to your repository's root. + +**From the user interface:** + +GitLab's UI allows you to perform lots of Git commands without having to +touch the command line. Even if you use the command line regularly, sometimes +it's easier to do so [via GitLab UI](web_editor.md): + +- [Create a file](web_editor.md#create-a-file) +- [Upload a file](web_editor.md#upload-a-file) +- [File templates](web_editor.md#template-dropdowns) +- [Create a directory](web_editor.md#create-a-directory) +- [Start a merge request](web_editor.md#tips) + +**From the command line:** + +To get started with the command line, please read through the +[command line basics documentation](../../../gitlab-basics/command-line-commands.md). + +## Branches + +When you submit changes in a new branch, you create a new version +of that project's file tree. Your branch contains all the changes +you are presenting, which are detected by Git line by line. + +To continue your workflow, once you pushed your changes to a new branch, +you can create a [merge request](../merge_requests/index.md), perform +inline code review, and [discuss](../../discussions/index.md) +your implementation with your team. +You can live preview changes submitted to a new branch with +[Review Apps](../../../ci/review_apps/index.md). + +With [GitLab Enterprise Edition](https://about.gitlab.com/gitlab-ee/) +subscriptions, you can also request +[approval](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#merge-request-approvals) from your managers. + +To create, delete, and branches via GitLab's UI: + +- [Create a branch](web_editor.md#create-a-new-branch) +- [Protected branches](../protected_branches.md#protected-branches) +- [Delete merged branches](branches/index.md#delete-merged-branches) + +Alternatively, you can use the +[command line](../../../gitlab-basics/start-using-git.md#create-a-branch). + +To learn more about branching strategies read through the +[GitLab Flow](../../../university/training/gitlab_flow.md) documentation. + +## Commits + +When you [commit your changes](https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository), +you are introducing those changes to your branch. +Via command line, you can commit multiple times before pushing. + +- **Commit message:** +A commit message is important to identity what is being changed and, +more importantly, why. In GitLab, you can add keywords to the commit +message that will perform one of the actions below: + - **Trigger a GitLab CI/CD pipeline:** + If you have your project configured with [GitLab CI/CD](../../../ci/README.md), + you will trigger a pipeline per push, not per commit. + - **Skip pipelines:** + You can add to you commit message the keyword + [`[ci skip]`](../../../ci/yaml/README.html#skipping-jobs) + and GitLab CI will skip that pipeline. + - **Cross-link issues and merge requests:** + [Cross-linking](../issues/crosslinking_issues.md#from-commit-messages) + is great to keep track of what's is somehow related in your workflow. + If you mention an issue or a merge request in a commit message, they will be shown + on their respective thread. +- **Cherry-pick a commit:** +In GitLab, you can +[cherry-pick a commit](../merge_requests/cherry_pick_changes.md#cherry-picking-a-commit) +right from the UI. +- **Revert a commit:** +Easily [revert a commit](../merge_requests/revert_changes.md#reverting-a-commit) +from the UI to a selected branch. + +## Repository size + +In GitLab.com, your repository size limit it 10GB. For other instances, +the repository size is limited by your system administrators. + +You can also [reduce a repository size using Git](reducing_the_repo_size_using_git.md). + +## Contributors + +All the contributors to your codebase are displayed under your project's **Settings > Contributors**. + +They are ordered from the collaborator with the greatest number +of commits to the fewest, and displayed on a nice graph: + + + +## Repository graph + +The repository graph displays visually the Git flow strategy used in that repository: + + + +Find it under your project's **Repository > Graph**. + +## Compare + +Select branches to compare and view the changes inline: + + + +Find it under your project's **Repository > Compare**. + +## Locked files (EEP) + +Lock your files to prevent any conflicting changes. + +[File Locking](https://docs.gitlab.com/ee/user/project/file_lock.html) is available only in +[GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/). + +## Repository's API + +You can access your repos via [repository API](../../../api/repositories.md). diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md new file mode 100644 index 00000000000..08805a4dc99 --- /dev/null +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -0,0 +1,81 @@ +# Reducing the repository size using Git + +A GitLab Entrerprise Edition administrator can set a [repository size limit][admin-repo-size] +which will prevent you to exceed it. + +When a project has reached its size limit, you will not be able to push to it, +create a new merge request, or merge existing ones. You will still be able to +create new issues, and clone the project though. Uploading LFS objects will +also be denied. + +In order to lift these restrictions, the administrator of the GitLab instance +needs to increase the limit on the particular project that exceeded it or you +need to instruct Git to rewrite changes. + +If you exceed the repository size limit, your first thought might be to remove +some data, make a new commit and push back to the repository. Unfortunately, +it's not so easy and that workflow won't work. Deleting files in a commit doesn't +actually reduce the size of the repo since the earlier commits and blobs are +still around. What you need to do is rewrite history with Git's +[`filter-branch` option][gitscm]. + +Note that even with that method, until `git gc` runs on the GitLab side, the +"removed" commits and blobs will still be around. And if a commit was ever +included in an MR, or if a build was run for a commit, or if a user commented +on it, it will be kept around too. So, in these cases the size will not decrease. + +The only fool proof way to actually decrease the repository size is to prune all +the unneeded stuff locally, and then create a new project on GitLab and start +using that instead. + +With that being said, you can try reducing your repository size with the +following method. + +## Using `git filter-branch` to purge files + +> +**Warning:** +Make sure to first make a copy of your repository since rewriting history will +purge the files and information you are about to delete. Also make sure to +inform any collaborators to not use `pull` after your changes, but use `rebase`. + +1. Navigate to your repository: + + ``` + cd my_repository/ + ``` + +1. Change to the branch you want to remove the big file from: + + ``` + git checkout master + ``` + +1. Use `filter-branch` to remove the big file: + + ``` + git filter-branch --force --tree-filter 'rm -f path/to/big_file.mpg' HEAD + ``` + +1. Instruct Git to purge the unwanted data: + + ``` + git reflog expire --expire=now --all && git gc --prune=now --aggressive + ``` + +1. Lastly, force push to the repository: + + ``` + git push --force origin master + ``` + +Your repository should now be below the size limit. + +>**Note:** +As an alternative to `filter-branch`, you can use the `bfg` tool with a +command like: `bfg --delete-files path/to/big_file.mpg`. Read the +[BFG Repo-Cleaner][bfg] documentation for more information. + +[admin-repo-size]: https://docs.gitlab.com/ee/user/admin_area/settings/account_and_limit_settings.html#repository-size-limit +[bfg]: https://rtyley.github.io/bfg-repo-cleaner/ +[gitscm]: https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#The-Nuclear-Option:-filter-branch diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 54d4028a50a..925bbf76d49 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -6,7 +6,7 @@ - [Description templates](../user/project/description_templates.md) - [Feature branch workflow](workflow.md) - [GitLab Flow](gitlab_flow.md) -- [Groups](groups.md) +- [Groups](../user/group/index.md) - Issues - The GitLab Issue Tracker is an advanced and complete tool for tracking the evolution of a new idea or the process of solving a problem. - [Confidential issues](../user/project/issues/confidential_issues.md) diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index e10ccc4fc46..9d466ae1971 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -91,7 +91,6 @@ This workflow where commits only flow downstream ensures that everything has bee If you need to cherry-pick a commit with a hotfix it is common to develop it on a feature branch and merge it into master with a merge request, do not delete the feature branch. If master is good to go (it should be if you are practicing [continuous delivery](http://martinfowler.com/bliki/ContinuousDelivery.html)) you then merge it to the other branches. If this is not possible because more manual testing is required you can send merge requests from the feature branch to the downstream branches. -An 'extreme' version of environment branches are setting up an environment for each feature branch as done by [Teatro](https://teatro.io/). ## Release branches with GitLab flow @@ -300,7 +299,7 @@ If there are no merge conflicts and the feature branches are short lived the ris If there are merge conflicts you merge the master branch into the feature branch and the CI server will rerun the tests. If you have long lived feature branches that last for more than a few days you should make your issues smaller. -## Working wih feature branches +## Working with feature branches  diff --git a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys.png b/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys.png Binary files differnew file mode 100644 index 00000000000..e525083918b --- /dev/null +++ b/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys.png diff --git a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png b/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png Binary files differnew file mode 100644 index 00000000000..8e26d98f1b0 --- /dev/null +++ b/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png diff --git a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png b/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png Binary files differnew file mode 100644 index 00000000000..f715c46adc3 --- /dev/null +++ b/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png diff --git a/doc/workflow/gpg_signed_commits/img/project_signed_and_unsigned_commits.png b/doc/workflow/gpg_signed_commits/img/project_signed_and_unsigned_commits.png Binary files differnew file mode 100644 index 00000000000..16ec2d031ae --- /dev/null +++ b/doc/workflow/gpg_signed_commits/img/project_signed_and_unsigned_commits.png diff --git a/doc/workflow/gpg_signed_commits/img/project_signed_commit_unverified_signature.png b/doc/workflow/gpg_signed_commits/img/project_signed_commit_unverified_signature.png Binary files differnew file mode 100644 index 00000000000..22565cf7c7e --- /dev/null +++ b/doc/workflow/gpg_signed_commits/img/project_signed_commit_unverified_signature.png diff --git a/doc/workflow/gpg_signed_commits/img/project_signed_commit_verified_signature.png b/doc/workflow/gpg_signed_commits/img/project_signed_commit_verified_signature.png Binary files differnew file mode 100644 index 00000000000..1778b2ddf2b --- /dev/null +++ b/doc/workflow/gpg_signed_commits/img/project_signed_commit_verified_signature.png diff --git a/doc/workflow/gpg_signed_commits/index.md b/doc/workflow/gpg_signed_commits/index.md new file mode 100644 index 00000000000..7d5762d2b9d --- /dev/null +++ b/doc/workflow/gpg_signed_commits/index.md @@ -0,0 +1,84 @@ +# Signing commits with GPG + +## Getting started + +- [Git Tools - Signing Your Work](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) +- [Git Tools - Signing Your Work: GPG introduction](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work#_gpg_introduction) +- [Git Tools - Signing Your Work: Signing commits](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work#_signing_commits) + +## How GitLab handles GPG + +GitLab uses its own keyring to verify the GPG signature. It does not access any +public key server. + +In order to have a commit verified on GitLab the corresponding public key needs +to be uploaded to GitLab. + +For a signature to be verified two prerequisites need to be met: + +1. The public key needs to be added to GitLab +1. One of the emails in the GPG key matches your **primary** email + +## Add a GPG key + +1. On the upper right corner, click on your avatar and go to your **Settings**. + +  + +1. Navigate to the **GPG keys** tab. + +  + +1. Paste your **public** key in the 'Key' box. + +  + +1. Finally, click on **Add key** to add it to GitLab. You will be able to see + its fingerprint, the corresponding email address and creation date. + +  + +>**Note:** +Once you add a key, you cannot edit it, only remove it. In case the paste +didn't work, you will have to remove the offending key and re-add it. + +## Remove a GPG key + +1. On the upper right corner, click on your avatar and go to your **Settings**. + +1. Navigate to the **GPG keys** tab. + +1. Click on the trash icon besides the GPG key you want to delete. + +>**Note:** +Removing a key **does not unverify** already signed commits. Commits that were +verified by using this key will stay verified. Only unpushed commits will stay +unverified once you remove this key. + +## Revoke a GPG key + +1. On the upper right corner, click on your avatar and go to your **Settings**. + +1. Navigate to the **GPG keys** tab. + +1. Click on **Revoke** besides the GPG key you want to delete. + +>**Note:** +Revoking a key **unverifies** already signed commits. Commits that were +verified by using this key will change to an unverified state. Future commits +will also stay unverified once you revoke this key. This action should be used +in case your key has been compromised. + +## Verifying commits + +1. Within a project navigate to the **Commits** tag. Signed commits will show a + badge containing either "Verified" or "Unverified", depending on the + verification status of the GPG signature. + +  + +1. By clicking on the GPG badge details of the signature are displayed. + +  + +  diff --git a/doc/workflow/groups.md b/doc/workflow/groups.md index 1645e7e8d65..06eec1ed928 100644 --- a/doc/workflow/groups.md +++ b/doc/workflow/groups.md @@ -1,97 +1,2 @@ -# GitLab Groups -GitLab groups allow you to group projects into directories and give users access to several projects at once. - -When you create a new project in GitLab, the default namespace for the project is the personal namespace associated with your GitLab user. -In this document we will see how to create groups, put projects in groups and manage who can access the projects in a group. - -## Creating groups - -You can create a group by going to the 'Groups' tab of the GitLab dashboard and clicking the 'New group' button. - - - -Next, enter the path and name (required) and the optional description and group avatar. - - - -When your group has been created you are presented with the group dashboard feed, which will be empty. - - - -You can use the 'New project' button to add a project to the new group. - -## Transferring an existing project into a group - -You can transfer an existing project into a group you have at least Master access in from the project settings page. -The option to transfer a project is only available if you are the Owner of the project. -First scroll down to the 'Dangerous settings' and click 'Show them to me'. -Now you can pick any of the groups you have at least Master access in as the new namespace for the group. - - - -GitLab administrators can use the admin interface to move any project to any namespace if needed. - -## Adding users to a group - -One of the benefits of putting multiple projects in one group is that you can give a user to access to all projects in the group with one action. - -Suppose we have a group with two projects. - - - -On the 'Group Members' page we can now add a new user Barry to the group. - - - -Now because Barry is a 'Developer' member of the 'Open Source' group, he automatically gets 'Developer' access to all projects in the 'Open Source' group. - - - -If necessary, you can increase the access level of an individual user for a specific project, by adding them as a Member to the project. - - - -## Requesting access to a group - -As a group owner you can enable or disable non members to request access to -your group. Go to the group settings and click on **Allow users to request access**. - -As a user, you can request to be a member of a group. Go to the group you'd -like to be a member of, and click the **Request Access** button on the right -side of your screen. - - - ---- - -Group owners & masters will be notified of your request and will be able to approve or -decline it on the members page. - - - ---- - -If you change your mind before your request is approved, just click the -**Withdraw Access Request** button. - - - -## Managing group memberships via LDAP - -In GitLab Enterprise Edition it is possible to manage GitLab group memberships using LDAP groups. -See [the GitLab Enterprise Edition documentation](http://docs.gitlab.com/ee/integration/ldap.html) for more information. - -## Allowing only admins to create groups - -By default, any GitLab user can create new groups. -This ability can be disabled for individual users from the admin panel. -It is also possible to configure GitLab so that new users default to not being able to create groups: - -``` -# For omnibus-gitlab, put the following in /etc/gitlab/gitlab.rb -gitlab_rails['gitlab_default_can_create_group'] = false - -# For installations from source, uncomment the 'default_can_create_group' -# line in /home/git/gitlab/config/gitlab.yml -``` +This document was moved to [another location](../user/group/index.md). diff --git a/doc/workflow/groups/add_member_to_group.png b/doc/workflow/groups/add_member_to_group.png Binary files differdeleted file mode 100644 index a10d5032bb0..00000000000 --- a/doc/workflow/groups/add_member_to_group.png +++ /dev/null diff --git a/doc/workflow/groups/group_dashboard.png b/doc/workflow/groups/group_dashboard.png Binary files differdeleted file mode 100644 index a5829f25808..00000000000 --- a/doc/workflow/groups/group_dashboard.png +++ /dev/null diff --git a/doc/workflow/groups/group_with_two_projects.png b/doc/workflow/groups/group_with_two_projects.png Binary files differdeleted file mode 100644 index 76d0a1b8ab2..00000000000 --- a/doc/workflow/groups/group_with_two_projects.png +++ /dev/null diff --git a/doc/workflow/groups/new_group_button.png b/doc/workflow/groups/new_group_button.png Binary files differdeleted file mode 100644 index 7155d6280bd..00000000000 --- a/doc/workflow/groups/new_group_button.png +++ /dev/null diff --git a/doc/workflow/groups/override_access_level.png b/doc/workflow/groups/override_access_level.png Binary files differdeleted file mode 100644 index 2b3e9a49842..00000000000 --- a/doc/workflow/groups/override_access_level.png +++ /dev/null diff --git a/doc/workflow/groups/project_members_via_group.png b/doc/workflow/groups/project_members_via_group.png Binary files differdeleted file mode 100644 index 878c9a03ac9..00000000000 --- a/doc/workflow/groups/project_members_via_group.png +++ /dev/null diff --git a/doc/workflow/groups/transfer_project.png b/doc/workflow/groups/transfer_project.png Binary files differdeleted file mode 100644 index 52161817f11..00000000000 --- a/doc/workflow/groups/transfer_project.png +++ /dev/null diff --git a/doc/workflow/share_projects_with_other_groups.md b/doc/workflow/share_projects_with_other_groups.md index 8e50cb03e63..40d756bc199 100644 --- a/doc/workflow/share_projects_with_other_groups.md +++ b/doc/workflow/share_projects_with_other_groups.md @@ -5,7 +5,7 @@ to a project with a single action. ## Groups as collections of users -Groups are used primarily to [create collections of projects](groups.md), but you can also +Groups are used primarily to [create collections of projects](../user/group/index.md), but you can also take advantage of the fact that groups define collections of _users_, namely the group members. |
