diff options
Diffstat (limited to 'doc')
241 files changed, 5917 insertions, 2715 deletions
diff --git a/doc/README.md b/doc/README.md index a2e152ce383..fee920f2012 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,5 +1,6 @@ --- comments: false +description: 'Learn how to use and administer GitLab, the most scalable Git-based fully integrated platform for software development.' --- # GitLab Documentation @@ -161,7 +162,7 @@ configuration. Then customize everything from buildpacks to CI/CD. - [Auto DevOps](topics/autodevops/index.md) - [Deployment of Helm, Ingress, and Prometheus on Kubernetes](user/project/clusters/index.md#installing-applications) -- [Protected secret variables](ci/variables/README.md#protected-secret-variables) +- [Protected variables](ci/variables/README.md#protected-variables) - [Easy creation of Kubernetes clusters on GKE](user/project/clusters/index.md#adding-and-creating-a-new-gke-cluster-via-gitlab) ### Monitor @@ -190,7 +191,7 @@ instant how code changes impact your production environment. - [User account](user/profile/index.md): Manage your account - [Authentication](topics/authentication/index.md): Account security with two-factor authentication, setup your ssh keys and deploy keys for secure access to your projects. - [Profile settings](user/profile/index.md#profile-settings): Manage your profile settings, two factor authentication and more. -- [User permissions](user/permissions.md): Learn what each role in a project (external/guest/reporter/developer/master/owner) can do. +- [User permissions](user/permissions.md): Learn what each role in a project (external/guest/reporter/developer/maintainer/owner) can do. ### Git and GitLab @@ -214,7 +215,7 @@ Learn how to contribute to GitLab: - [Development](development/README.md): All styleguides and explanations how to contribute. - [Legal](legal/README.md): Contributor license agreements. -- [Writing documentation](development/writing_documentation.md): Contributing to GitLab Docs. +- [Writing documentation](development/documentation/index.md): Contributing to GitLab Docs. ## GitLab subscriptions @@ -240,7 +241,7 @@ GitLab.com is hosted, managed, and administered by GitLab, Inc., with and teams: Free, Bronze, Silver, and Gold. GitLab.com subscriptions grants access -to the same features available in GitLab self-hosted, **expect +to the same features available in GitLab self-hosted, **except [administration](administration/index.md) tools and settings**: - GitLab.com Free includes the same features available in Core diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index 63fbb24bac1..3c98d683924 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -1,10 +1,20 @@ +[//]: # (Do *NOT* modify this file in EE documentation. All changes in this) +[//]: # (file should happen in CE, too. If the change is EE-specific, put) +[//]: # (it in `ldap-ee.md`.) + # LDAP GitLab integrates with LDAP to support user authentication. This integration works with most LDAP-compliant directory servers, including Microsoft Active Directory, Apple Open Directory, Open LDAP, -and 389 Server. GitLab EE includes enhanced integration, including group -membership syncing. +and 389 Server. GitLab Enterprise Editions include enhanced integration, +including group membership syncing as well as multiple LDAP servers support. + +## GitLab EE + +The information on this page is relevant for both GitLab CE and EE. For more +details about EE-specific LDAP features, see the +[LDAP Enterprise Edition documentation](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html). ## Security @@ -27,8 +37,10 @@ are already logged in or are using Git over SSH will still be able to access GitLab for up to one hour. Manually block the user in the GitLab Admin area to immediately block all access. ->**Note**: GitLab EE supports a configurable sync time, with a default -of one hour. +NOTE: **Note**: +GitLab Enterprise Edition Starter supports a +[configurable sync time](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#adjusting-ldap-user-and-group-sync-schedules), +with a default of one hour. ## Git password authentication @@ -38,19 +50,21 @@ in the application settings. ## Configuration +NOTE: **Note**: +In GitLab Enterprise Edition Starter, you can configure multiple LDAP servers +to connect to one GitLab server. + For a complete guide on configuring LDAP with GitLab Community Edition, please check the admin guide [How to configure LDAP with GitLab CE](how_to_configure_ldap_gitlab_ce/index.md). To enable LDAP integration you need to add your LDAP server settings in -`/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml`. +`/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml` for Omnibus +GitLab and installations from source respectively. There is a Rake task to check LDAP configuration. After configuring LDAP using the documentation below, see [LDAP check Rake task](../raketasks/check.md#ldap-check) for information on the LDAP check Rake task. ->**Note**: In GitLab EE, you can configure multiple LDAP servers to connect to -one GitLab server. - Prior to version 7.4, GitLab used a different syntax for configuring LDAP integration. The old LDAP integration syntax still works but may be removed in a future version. If your `gitlab.rb` or `gitlab.yml` file contains @@ -61,159 +75,202 @@ The configuration inside `gitlab_rails['ldap_servers']` below is sensitive to incorrect indentation. Be sure to retain the indentation given in the example. Copy/paste can sometimes cause problems. +NOTE: **Note:** +The `encryption` value `ssl` corresponds to 'Simple TLS' in the LDAP +library. `tls` corresponds to StartTLS, not to be confused with regular TLS. +Normally, if you specify `ssl` it will be on port 636, while `tls` (StartTLS) +would be on port 389. `plain` also operates on port 389. + **Omnibus configuration** ```ruby gitlab_rails['ldap_enabled'] = true gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below -main: # 'main' is the GitLab 'provider ID' of this LDAP server - ## label - # - # A human-friendly name for your LDAP server. It is OK to change the label later, - # for instance if you find out it is too large to fit on the web page. - # - # Example: 'Paris' or 'Acme, Ltd.' +## +## 'main' is the GitLab 'provider ID' of this LDAP server +## +main: + ## + ## A human-friendly name for your LDAP server. It is OK to change the label later, + ## for instance if you find out it is too large to fit on the web page. + ## + ## Example: 'Paris' or 'Acme, Ltd.' + ## label: 'LDAP' - # Example: 'ldap.mydomain.com' + ## + ## Example: 'ldap.mydomain.com' + ## host: '_your_ldap_server' - # This port is an example, it is sometimes different but it is always an integer and not a string + + ## + ## This port is an example, it is sometimes different but it is always an + ## integer and not a string. + ## port: 389 # usually 636 for SSL uid: 'sAMAccountName' # This should be the attribute, not the value that maps to uid. - # Examples: 'america\\momo' or 'CN=Gitlab Git,CN=Users,DC=mydomain,DC=com' + ## + ## Examples: 'america\\momo' or 'CN=Gitlab Git,CN=Users,DC=mydomain,DC=com' + ## bind_dn: '_the_full_dn_of_the_user_you_will_bind_with' password: '_the_password_of_the_bind_user' - # Encryption method. The "method" key is deprecated in favor of - # "encryption". - # - # Examples: "start_tls" or "simple_tls" or "plain" - # - # Deprecated values: "tls" was replaced with "start_tls" and "ssl" was - # replaced with "simple_tls". - # + ## + ## Encryption method. The "method" key is deprecated in favor of + ## "encryption". + ## + ## Examples: "start_tls" or "simple_tls" or "plain" + ## + ## Deprecated values: "tls" was replaced with "start_tls" and "ssl" was + ## replaced with "simple_tls". + ## + ## encryption: 'plain' - # Enables SSL certificate verification if encryption method is - # "start_tls" or "simple_tls". Defaults to true since GitLab 10.0 for - # security. This may break installations upon upgrade to 10.0, that did - # not know their LDAP SSL certificates were not setup properly. For - # example, when using self-signed certificates, the ca_file path may - # need to be specified. + ## + ## Enables SSL certificate verification if encryption method is + ## "start_tls" or "simple_tls". Defaults to true since GitLab 10.0 for + ## security. This may break installations upon upgrade to 10.0, that did + ## not know their LDAP SSL certificates were not setup properly. + ## verify_certificates: true - # Specifies the path to a file containing a PEM-format CA certificate, - # e.g. if you need to use an internal CA. - # - # Example: '/etc/ca.pem' - # - ca_file: '' - - # Specifies the SSL version for OpenSSL to use, if the OpenSSL default - # is not appropriate. - # - # Example: 'TLSv1_1' - # + ## + ## Specifies the SSL version for OpenSSL to use, if the OpenSSL default + ## is not appropriate. + ## + ## Example: 'TLSv1_1' + ## + ## ssl_version: '' - # Set a timeout, in seconds, for LDAP queries. This helps avoid blocking - # a request if the LDAP server becomes unresponsive. - # A value of 0 means there is no timeout. + ## + ## Set a timeout, in seconds, for LDAP queries. This helps avoid blocking + ## a request if the LDAP server becomes unresponsive. + ## A value of 0 means there is no timeout. + ## timeout: 10 - # This setting specifies if LDAP server is Active Directory LDAP server. - # For non AD servers it skips the AD specific queries. - # If your LDAP server is not AD, set this to false. + ## + ## This setting specifies if LDAP server is Active Directory LDAP server. + ## For non AD servers it skips the AD specific queries. + ## If your LDAP server is not AD, set this to false. + ## active_directory: true - # If allow_username_or_email_login is enabled, GitLab will ignore everything - # after the first '@' in the LDAP username submitted by the user on login. - # - # Example: - # - the user enters 'jane.doe@example.com' and 'p@ssw0rd' as LDAP credentials; - # - GitLab queries the LDAP server with 'jane.doe' and 'p@ssw0rd'. - # - # If you are using "uid: 'userPrincipalName'" on ActiveDirectory you need to - # disable this setting, because the userPrincipalName contains an '@'. + ## + ## If allow_username_or_email_login is enabled, GitLab will ignore everything + ## after the first '@' in the LDAP username submitted by the user on login. + ## + ## Example: + ## - the user enters 'jane.doe@example.com' and 'p@ssw0rd' as LDAP credentials; + ## - GitLab queries the LDAP server with 'jane.doe' and 'p@ssw0rd'. + ## + ## If you are using "uid: 'userPrincipalName'" on ActiveDirectory you need to + ## disable this setting, because the userPrincipalName contains an '@'. + ## allow_username_or_email_login: false - # To maintain tight control over the number of active users on your GitLab installation, - # enable this setting to keep new users blocked until they have been cleared by the admin - # (default: false). + ## + ## To maintain tight control over the number of active users on your GitLab installation, + ## enable this setting to keep new users blocked until they have been cleared by the admin + ## (default: false). + ## block_auto_created_users: false - # Base where we can search for users - # - # Ex. 'ou=People,dc=gitlab,dc=example' or 'DC=mydomain,DC=com' - # + ## + ## Base where we can search for users + ## + ## Ex. 'ou=People,dc=gitlab,dc=example' or 'DC=mydomain,DC=com' + ## + ## base: '' - # Filter LDAP users - # - # Format: RFC 4515 https://tools.ietf.org/search/rfc4515 - # Ex. (employeeType=developer) - # - # Note: GitLab does not support omniauth-ldap's custom filter syntax. - # - # Example for getting only specific users: - # '(&(objectclass=user)(|(samaccountname=momo)(samaccountname=toto)))' - # + ## + ## Filter LDAP users + ## + ## Format: RFC 4515 https://tools.ietf.org/search/rfc4515 + ## Ex. (employeeType=developer) + ## + ## Note: GitLab does not support omniauth-ldap's custom filter syntax. + ## + ## Example for getting only specific users: + ## '(&(objectclass=user)(|(samaccountname=momo)(samaccountname=toto)))' + ## user_filter: '' - # LDAP attributes that GitLab will use to create an account for the LDAP user. - # The specified attribute can either be the attribute name as a string (e.g. 'mail'), - # or an array of attribute names to try in order (e.g. ['mail', 'email']). - # Note that the user's LDAP login will always be the attribute specified as `uid` above. + ## + ## LDAP attributes that GitLab will use to create an account for the LDAP user. + ## The specified attribute can either be the attribute name as a string (e.g. 'mail'), + ## or an array of attribute names to try in order (e.g. ['mail', 'email']). + ## Note that the user's LDAP login will always be the attribute specified as `uid` above. + ## attributes: - # The username will be used in paths for the user's own projects - # (like `gitlab.example.com/username/project`) and when mentioning - # them in issues, merge request and comments (like `@username`). - # If the attribute specified for `username` contains an email address, - # the GitLab username will be the part of the email address before the '@'. + ## + ## The username will be used in paths for the user's own projects + ## (like `gitlab.example.com/username/project`) and when mentioning + ## them in issues, merge request and comments (like `@username`). + ## If the attribute specified for `username` contains an email address, + ## the GitLab username will be the part of the email address before the '@'. + ## username: ['uid', 'userid', 'sAMAccountName'] email: ['mail', 'email', 'userPrincipalName'] - # If no full name could be found at the attribute specified for `name`, - # the full name is determined using the attributes specified for - # `first_name` and `last_name`. + ## + ## If no full name could be found at the attribute specified for `name`, + ## the full name is determined using the attributes specified for + ## `first_name` and `last_name`. + ## name: 'cn' first_name: 'givenName' last_name: 'sn' - # If lowercase_usernames is enabled, GitLab will lower case the username. + ## + ## If lowercase_usernames is enabled, GitLab will lower case the username. + ## lowercase_usernames: false - + ## ## EE only + ## - # Base where we can search for groups - # - # Ex. ou=groups,dc=gitlab,dc=example - # + ## Base where we can search for groups + ## + ## Ex. ou=groups,dc=gitlab,dc=example + ## group_base: '' - # The CN of a group containing GitLab administrators - # - # Ex. administrators - # - # Note: Not `cn=administrators` or the full DN - # + ## The CN of a group containing GitLab administrators + ## + ## Ex. administrators + ## + ## Note: Not `cn=administrators` or the full DN + ## admin_group: '' - # The LDAP attribute containing a user's public SSH key - # - # Ex. ssh_public_key - # + ## An array of CNs of groups containing users that should be considered external + ## + ## Ex. ['interns', 'contractors'] + ## + ## Note: Not `cn=interns` or the full DN + ## + external_groups: [] + + ## + ## The LDAP attribute containing a user's public SSH key + ## + ## Example: sshPublicKey + ## sync_ssh_keys: false -# GitLab EE only: add more LDAP servers -# Choose an ID made of a-z and 0-9 . This ID will be stored in the database -# so that GitLab can remember which LDAP server a user belongs to. -# uswest2: -# label: -# host: -# .... +## GitLab EE only: add more LDAP servers +## Choose an ID made of a-z and 0-9 . This ID will be stored in the database +## so that GitLab can remember which LDAP server a user belongs to. +#uswest2: +# label: +# host: +# .... EOS ``` @@ -222,21 +279,23 @@ EOS Use the same format as `gitlab_rails['ldap_servers']` for the contents under `servers:` in the example below: -``` +```yaml production: # snip... ldap: enabled: false servers: - main: # 'main' is the GitLab 'provider ID' of this LDAP server - ## label - # - # A human-friendly name for your LDAP server. It is OK to change the label later, - # for instance if you find out it is too large to fit on the web page. - # - # Example: 'Paris' or 'Acme, Ltd.' + ## + ## 'main' is the GitLab 'provider ID' of this LDAP server + ## + main: + ## + ## A human-friendly name for your LDAP server. It is OK to change the label later, + ## for instance if you find out it is too large to fit on the web page. + ## + ## Example: 'Paris' or 'Acme, Ltd.' label: 'LDAP' - # snip... + ## snip... ``` ## Using an LDAP filter to limit access to your GitLab server @@ -283,6 +342,24 @@ nested members in the user filter should not be confused with Please note that GitLab does not support the custom filter syntax used by omniauth-ldap. +### Escaping special characters + +If the `user_filter` DN contains special characters. For example, a comma: + +``` +OU=GitLab, Inc,DC=gitlab,DC=com +``` + +This character needs to be escaped as documented in [RFC 4515](https://tools.ietf.org/search/rfc4515). + +Due to the way the string is parsed, the special character needs to be converted +to hex and `\\5C\\` (`5C` = `\` in hex) added before it. +As an example the above DN would look like + +``` +OU=GitLab\\5C\\2C Inc,DC=gitlab,DC=com +``` + ## Enabling LDAP sign-in for existing GitLab users When a user signs in to GitLab with LDAP for the first time, and their LDAP diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 960970aea30..1c508c77ffa 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -3,7 +3,7 @@ > **Note:** Custom Git hooks must be configured on the filesystem of the GitLab server. Only GitLab server administrators will be able to complete these tasks. -Please explore [webhooks] as an option if you do not +Please explore [webhooks] and [CI] as an option if you do not have filesystem access. For a user configurable Git hook interface, see [Push Rules](https://docs.gitlab.com/ee/push_rules/push_rules.html), available in GitLab Enterprise Edition. @@ -80,6 +80,7 @@ STDERR takes precedence over STDOUT.  +[CI]: ../ci/README.md [hooks]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#Server-Side-Hooks [webhooks]: ../user/project/integrations/webhooks.md [5073]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5073 diff --git a/doc/administration/external_database.md b/doc/administration/external_database.md new file mode 100644 index 00000000000..31199f2cdc7 --- /dev/null +++ b/doc/administration/external_database.md @@ -0,0 +1,17 @@ +# Configure GitLab using an external PostgreSQL service + +If you're hosting GitLab on a cloud provider, you can optionally use a +managed service for PostgreSQL. For example, AWS offers a managed Relational +Database Service (RDS) that runs PostgreSQL. + +Alternatively, you may opt to manage your own PostgreSQL instance or cluster +separate from the GitLab Omnibus package. + +If you use a cloud-managed service, or provide your own PostgreSQL instance: + +1. Setup PostgreSQL according to the + [database requirements document](../install/requirements.md#database). +1. Set up a `gitlab` username with a password of your choice. The `gitlab` user + needs privileges to create the `gitlabhq_production` database. +1. Configure the GitLab application servers with the appropriate details. + This step is covered in [Configuring GitLab for HA](high_availability/gitlab.md). diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index ca6d8d2de67..b5124b1d540 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -33,16 +33,7 @@ If you use a cloud-managed service, or provide your own PostgreSQL: external_url 'https://gitlab.example.com' # Disable all components except PostgreSQL - postgresql['enable'] = true - bootstrap['enable'] = false - nginx['enable'] = false - unicorn['enable'] = false - sidekiq['enable'] = false - redis['enable'] = false - prometheus['enable'] = false - gitaly['enable'] = false - gitlab_workhorse['enable'] = false - mailroom['enable'] = false + roles ['postgres_role'] # PostgreSQL configuration gitlab_rails['db_password'] = 'DB password' diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md index e201848791c..0d9c10687f2 100644 --- a/doc/administration/high_availability/gitlab.md +++ b/doc/administration/high_availability/gitlab.md @@ -47,7 +47,8 @@ for each GitLab application server in your environment. URL. Depending your the NFS configuration, you may need to change some GitLab data locations. See [NFS documentation](nfs.md) for `/etc/gitlab/gitlab.rb` configuration values for various scenarios. The example below assumes you've - added NFS mounts in the default data locations. + added NFS mounts in the default data locations. Additionally the UID and GIDs + given are just examples and you should configure with your preferred values. ```ruby external_url 'https://gitlab.example.com' @@ -68,6 +69,14 @@ for each GitLab application server in your environment. gitlab_rails['redis_port'] = '6379' gitlab_rails['redis_host'] = '10.1.0.6' # IP/hostname of Redis server gitlab_rails['redis_password'] = 'Redis Password' + + # Ensure UIDs and GIDs match between servers for permissions via NFS + user['uid'] = 9000 + user['gid'] = 9000 + web_server['uid'] = 9001 + web_server['gid'] = 9001 + registry['uid'] = 9002 + registry['gid'] = 9002 ``` > **Note:** To maintain uniformity of links across HA clusters, the `external_url` @@ -75,25 +84,24 @@ for each GitLab application server in your environment. servers should point to the external url that users will use to access GitLab. In a typical HA setup, this will be the url of the load balancer which will route traffic to all GitLab application servers in the HA cluster. - -1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + + > **Note:** When you specify `https` in the `external_url`, as in the example + above, GitLab assumes you have SSL certificates in `/etc/gitlab/ssl/`. If + certificates are not present, Nginx will fail to start. See + [Nginx documentation](http://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) + for more information. ## First GitLab application server -As a final step, run the setup rake task on the first GitLab application server. -It is not necessary to run this on additional application servers. +As a final step, run the setup rake task **only on** the first GitLab application server. +Do not run this on additional application servers. 1. Initialize the database by running `sudo gitlab-rake gitlab:setup`. +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. > **WARNING:** Only run this setup task on **NEW** GitLab instances because it will wipe any existing data. -> **Note:** When you specify `https` in the `external_url`, as in the example - above, GitLab assumes you have SSL certificates in `/etc/gitlab/ssl/`. If - certificates are not present, Nginx will fail to start. See - [Nginx documentation](http://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) - for more information. - ## Extra configuration for additional GitLab application servers Additional GitLab servers (servers configured **after** the first GitLab server) @@ -101,8 +109,7 @@ need some extra configuration. 1. Configure shared secrets. These values can be obtained from the primary GitLab server in `/etc/gitlab/gitlab-secrets.json`. Add these to - `/etc/gitlab/gitlab.rb` **prior to** running the first `reconfigure` in - the steps above. + `/etc/gitlab/gitlab.rb` **prior to** running the first `reconfigure`. ```ruby gitlab_shell['secret_token'] = 'fbfb19c355066a9afb030992231c4a363357f77345edd0f2e772359e5be59b02538e1fa6cae8f93f7d23355341cea2b93600dab6d6c3edcdced558fc6d739860' @@ -115,6 +122,8 @@ need some extra configuration. from running on upgrade. Only the primary GitLab application server should handle migrations. +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + ## Troubleshooting - `mount: wrong fs type, bad option, bad superblock on` diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index ad8ffc46559..87e96b71dd4 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -1,7 +1,7 @@ # NFS You can view information and options set for each of the mounted NFS file -systems by running `sudo nfsstat -m`. +systems by running `nfsstat -m` and `cat /etc/fstab`. ## NFS Server features @@ -25,7 +25,9 @@ options: errors when the Omnibus package tries to alter permissions. Note that GitLab and other bundled components do **not** run as `root` but as non-privileged users. The recommendation for `no_root_squash` is to allow the Omnibus package - to set ownership and permissions on files, as needed. + to set ownership and permissions on files, as needed. In some cases where the + `no_root_squash` option is not available, the `root` flag can achieve the same + result. - `sync` - Force synchronous behavior. Default is asynchronous and under certain circumstances it could lead to data loss if a failure occurs before data has synced. diff --git a/doc/administration/index.md b/doc/administration/index.md index b472ca5b4d8..0e65f9a9963 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -1,3 +1,7 @@ +--- +description: 'Learn how to install, configure, update, and maintain your GitLab instance.' +--- + # Administrator documentation **[CORE ONLY]** Learn how to administer your GitLab instance (Community Edition and @@ -40,10 +44,12 @@ Learn how to install, configure, update, and maintain your GitLab instance. [source installations](../install/installation.md#installation-from-source). - [Environment variables](environment_variables.md): Supported environment variables that can be used to override their defaults values in order to configure GitLab. - [Plugins](plugins.md): With custom plugins, GitLab administrators can introduce custom integrations without modifying GitLab's source code. +- [Enforcing Terms of Service](../user/admin_area/settings/terms.md) #### Customizing GitLab's appearance - [Header logo](../customization/branded_page_and_email_header.md): Change the logo on all pages and email headers. +- [Favicon](../customization/favicon.md): Change the default favicon to your own logo. - [Branded login page](../customization/branded_login_page.md): Customize the login page with your own logo, title, and description. - [Welcome message](../customization/welcome_message.md): Add a custom welcome message to the sign-in page. - ["New Project" page](../customization/new_project_page.md): Customize the text to be displayed on the page that opens whenever your users create a new project. diff --git a/doc/administration/integration/koding.md b/doc/administration/integration/koding.md index 67f9f01efb8..6c1ec3028cc 100644 --- a/doc/administration/integration/koding.md +++ b/doc/administration/integration/koding.md @@ -100,14 +100,14 @@ As it's pointed out before, you will need public access to this machine that you've installed Koding and GitLab on. Better to use a domain but a static IP is also fine. -For IP based installation you can use [xip.io](https://xip.io) service which is +For IP based installation you can use [nip.io](https://nip.io) service which is free and provides DNS resolution to IP based requests like following; - - 127.0.0.1.xip.io -> resolves to 127.0.0.1 - - foo.bar.baz.127.0.0.1.xip.io -> resolves to 127.0.0.1 + - 127.0.0.1.nip.io -> resolves to 127.0.0.1 + - foo.bar.baz.127.0.0.1.nip.io -> resolves to 127.0.0.1 - and so on... -As Koding needs subdomains for team names; `foo.127.0.0.1.xip.io` requests for +As Koding needs subdomains for team names; `foo.127.0.0.1.nip.io` requests for a running koding instance on `127.0.0.1` server will be handled as `foo` team requests. @@ -127,8 +127,8 @@ your Koding installation. Team called `gitlab` has integration on Koding out of the box, so if you didn't change anything your team on Koding should be `gitlab`. -So, if your Koding is running on `http://1.2.3.4.xip.io:8090` your URL needs -to be `http://gitlab.1.2.3.4.xip.io:8090`. You need to provide the same host +So, if your Koding is running on `http://1.2.3.4.nip.io:8090` your URL needs +to be `http://gitlab.1.2.3.4.nip.io:8090`. You need to provide the same host with your Koding installation here. @@ -192,7 +192,7 @@ cd koding docker-compose run \ --service-ports backend \ /opt/koding/scripts/bootstrap-container build \ - --host=**YOUR_IP**.xip.io \ + --host=**YOUR_IP**.nip.io \ --gitlabHost=**GITLAB_IP** \ --gitlabPort=**GITLAB_PORT** \ --gitlabToken=**SECRET_TOKEN** \ @@ -224,7 +224,7 @@ cd koding docker-compose run \ --service-ports backend \ /opt/koding/scripts/bootstrap-container build \ - --host=**YOUR_IP**.xip.io \ + --host=**YOUR_IP**.nip.io \ ``` #### Enable Single Sign On @@ -233,7 +233,7 @@ Once you restarted your Koding and logged in with your username and password you need to activate oauth authentication for your user. To do that - Navigate to Dashboard on Koding from; - `http://gitlab.**YOUR_IP**.xip.io:8090/Home/my-account` + `http://gitlab.**YOUR_IP**.nip.io:8090/Home/my-account` - Scroll down to Integrations section - Click on toggle to turn On integration in GitLab integration section diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 91e844c7b42..e11ed58eb91 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -1,12 +1,13 @@ # Web terminals -> [Introduced][ce-7690] in GitLab 8.15. Only project masters and owners can - access web terminals. +> +[Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7690) +in GitLab 8.15. Only project maintainers and owners can access web terminals. -With the introduction of the [Kubernetes project service][kubservice], GitLab -gained the ability to store and use credentials for a Kubernetes cluster. One -of the things it uses these credentials for is providing access to -[web terminals](../../ci/environments.html#web-terminals) for environments. +With the introduction of the [Kubernetes integration](../../user/project/clusters/index.md), +GitLab gained the ability to store and use credentials for a Kubernetes cluster. +One of the things it uses these credentials for is providing access to +[web terminals](../../ci/environments.md#web-terminals) for environments. ## How it works @@ -80,6 +81,3 @@ Terminal sessions use long-lived connections; by default, these may last forever. You can configure a maximum session time in the Admin area of your GitLab instance if you find this undesirable from a scalability or security point of view. - -[ce-7690]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7690 -[kubservice]: ../../user/project/integrations/kubernetes.md diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 77fe4d561a1..e59ab5a72e1 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -94,6 +94,7 @@ _The artifacts are stored by default in > Available in [GitLab Premium](https://about.gitlab.com/products/) and [GitLab.com Silver](https://about.gitlab.com/gitlab-com/). > Since version 10.6, available in [GitLab CE](https://about.gitlab.com/products/) +> Since version 11.0, we support direct_upload to S3. If you don't want to use the local disk where GitLab is installed to store the artifacts, you can use an object storage like AWS S3 instead. @@ -108,7 +109,7 @@ For source installations the following settings are nested under `artifacts:` an |---------|-------------|---------| | `enabled` | Enable/disable object storage | `false` | | `remote_directory` | The bucket name where Artifacts will be stored| | -| `direct_upload` | Set to true to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. Currently only `Google` provider is supported | `false` | +| `direct_upload` | Set to true to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. | `false` | | `background_upload` | Set to false to disable automatic upload. Option may be removed once upload is direct to S3 | `true` | | `proxy_download` | Set to true to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` | | `connection` | Various connection options described below | | diff --git a/doc/administration/job_traces.md b/doc/administration/job_traces.md index 84a1ffeec98..f0b2054a7f3 100644 --- a/doc/administration/job_traces.md +++ b/doc/administration/job_traces.md @@ -40,3 +40,98 @@ To change the location where the job logs will be stored, follow the steps below [reconfigure gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure "How to reconfigure Omnibus GitLab" [restart gitlab]: restart_gitlab.md#installations-from-source "How to restart GitLab" + +## New live trace architecture + +> [Introduced][ce-18169] in GitLab 10.4. + +> **Notes**: +- This feature is still Beta, which could impact GitLab.com/on-premises instances, and in the worst case scenario, traces will be lost. +- This feature is still being discussed in [an issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/46097) for the performance improvements. +- This feature is off by default. Please check below how to enable/disable this featrue. + +**What is "live trace"?** + +Job trace that is sent by runner while jobs are running. You can see live trace in job pages UI. +The live traces are archived once job finishes. + +**What is new architecture?** + +So far, when GitLab Runner sends a job trace to GitLab-Rails, traces have been saved to file storage as text files. +This was a problem for [Cloud Native-compatible GitLab application](https://gitlab.com/gitlab-com/migration/issues/23) where GitLab had to rely on File Storage. + +This new live trace architecture stores chunks of traces in Redis and database instead of file storage. +Redis is used as first-class storage, and it stores up-to 128kB. Once the full chunk is sent it will be flushed to database. Afterwhile, the data in Redis and database will be archived to ObjectStorage. + +Here is the detailed data flow. + +1. GitLab Runner picks a job from GitLab-Rails +1. GitLab Runner sends a piece of trace to GitLab-Rails +1. GitLab-Rails appends the data to Redis +1. If the data in Redis is fulfilled 128kB, the data is flushed to Database. +1. 2.~4. is continued until the job is finished +1. Once the job is finished, GitLab-Rails schedules a sidekiq worker to archive the trace +1. The sidekiq worker archives the trace to Object Storage, and cleanup the trace in Redis and Database + +**How to check if it's on or off?** + +```ruby +Feature.enabled?('ci_enable_live_trace') +``` + +**How to enable?** + +```ruby +Feature.enable('ci_enable_live_trace') +``` + +>**Note:** +The transition period will be handled gracefully. Upcoming traces will be generated with the new architecture, and on-going live traces will stay with the legacy architecture (i.e. on-going live traces won't be re-generated forcibly with the new architecture). + +**How to disable?** + +```ruby +Feature.disable('ci_enable_live_trace') +``` + +>**Note:** +The transition period will be handled gracefully. Upcoming traces will be generated with the legacy architecture, and on-going live traces will stay with the new architecture (i.e. on-going live traces won't be re-generated forcibly with the legacy architecture). + +**Redis namespace:** + +`Gitlab::Redis::SharedState` + +**Potential impact:** + +- This feature could incur data loss: + - Case 1: When all data in Redis are accidentally flushed. + - On-going live traces could be recovered by re-sending traces (This is supported by all versions of GitLab Runner) + - Finished jobs which has not archived live traces will lose the last part (~128kB) of trace data. + - Case 2: When sidekiq workers failed to archive (e.g. There was a bug that prevents archiving process, Sidekiq inconsistancy, etc): + - Currently all trace data in Redis will be deleted after one week. If the sidekiq workers can't finish by the expiry date, the part of trace data will be lost. +- This feature could consume all memory on Redis instance. If the number of jobs is 1000, 128MB (128kB * 1000) is consumed. +- This feature could pressure Database replication lag. `INSERT` are generated to indicate that we have trace chunk. `UPDATE` with 128kB of data is issued once we receive multiple chunks. +- and so on + +**How to test?** + +We're currently evaluating this feature on dev.gitalb.org or staging.gitlab.com to verify this features. Here is the list of tests/measurements. + +- Features: + - Live traces should be visible on job pages + - Archived traces should be visible on job pages + - Live traces should be archived to Object storage + - Live traces should be cleaned up after archived + - etc +- Performance: + - Schedule 1000~10000 jobs and let GitLab-runners process concurrently. Measure memoery presssure, IO load, etc. + - etc +- Failover: + - Simulate Redis outage + - etc + +**How to verify the correctnesss?** + +- TBD + +[ce-44935]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18169 diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 69600cad25c..411a0fae93f 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -1,7 +1,7 @@ # GitLab Prometheus metrics >**Note:** -Available since [Omnibus GitLab 9.3][29118]. Currently experimental. For +Available since [Omnibus GitLab 9.3][29118]. For installations from source you'll have to configure it yourself. To enable the GitLab Prometheus metrics: @@ -24,7 +24,7 @@ server, because the embedded server configuration is overwritten once every ## Metrics available -In this experimental phase, only a few metrics are available: +The following metrics are available: | Metric | Type | Since | Description | |:--------------------------------- |:--------- |:----- |:----------- | diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index 3d24812c66a..1c79e86dcb4 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -29,7 +29,8 @@ For installations from source you'll have to install and configure it yourself. Prometheus and it's exporters are on by default, starting with GitLab 9.0. Prometheus will run as the `gitlab-prometheus` user and listen on -`http://localhost:9090`. Each exporter will be automatically be set up as a +`http://localhost:9090`. By default Prometheus is only accessible from the GitLab server itself. +Each exporter will be automatically set up as a monitoring target for Prometheus, unless individually disabled. To disable Prometheus and all of its exporters, as well as any added in the future: @@ -44,14 +45,16 @@ To disable Prometheus and all of its exporters, as well as any added in the futu 1. Save the file and [reconfigure GitLab][reconfigure] for the changes to take effect -## Changing the port Prometheus listens on +## Changing the port and address Prometheus listens on >**Note:** The following change was added in [GitLab Omnibus 8.17][1261]. Although possible, -it's not recommended to change the default address and port Prometheus listens +it's not recommended to change the port Prometheus listens on as this might affect or conflict with other services running on the GitLab server. Proceed at your own risk. +In order to access Prometheus from outside the GitLab server you will need to +set a FQDN or IP in `prometheus['listen_address']`. To change the address/port that Prometheus listens on: 1. Edit `/etc/gitlab/gitlab.rb` @@ -80,9 +83,9 @@ You can visit `http://localhost:9090` for the dashboard that Prometheus offers b >**Note:** If SSL has been enabled on your GitLab instance, you may not be able to access -Prometheus on the same browser as GitLab due to [HSTS][hsts]. We plan to +Prometheus on the same browser as GitLab if using the same FQDN due to [HSTS][hsts]. We plan to [provide access via GitLab][multi-user-prometheus], but in the interim there are -some workarounds: using a separate browser for Prometheus, resetting HSTS, or +some workarounds: using a separate FQDN, using server IP, using a separate browser for Prometheus, resetting HSTS, or having [Nginx proxy it][nginx-custom-config]. The performance data collected by Prometheus can be viewed directly in the @@ -120,7 +123,7 @@ To disable the monitoring of Kubernetes: ## GitLab Prometheus metrics -> Introduced as an experimental feature in GitLab 9.3. +> Introduced in GitLab 9.3. GitLab monitors its own internal service metrics, and makes them available at the `/-/metrics` endpoint. Unlike other exporters, this endpoint requires authentication as it is available on the same URL and port as user traffic. diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 9b3b1e48efd..9b1297ca4ba 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -1,3 +1,7 @@ +--- +description: 'Learn how to administer GitLab Pages.' +--- + # GitLab Pages administration > **Notes:** @@ -8,8 +12,6 @@ GitLab from source, follow the [Pages source installation document](source.md). - To learn how to use GitLab Pages, read the [user documentation][pages-userguide]. ---- - This document describes how to set up the _latest_ GitLab Pages feature. Make sure to read the [changelog](#changelog) if you are upgrading to a new GitLab version as it may include new features and changes needed to be made in your @@ -24,8 +26,6 @@ SNI and exposes pages using HTTP2 by default. You are encouraged to read its [README][pages-readme] to fully understand how it works. ---- - In the case of [custom domains](#custom-domains) (but not [wildcard domains](#wildcard-domains)), the Pages daemon needs to listen on ports `80` and/or `443`. For that reason, there is some flexibility in the way @@ -83,12 +83,12 @@ you need to add a [wildcard DNS A record][wiki-wildcard-dns] pointing to the host that GitLab runs. For example, an entry would look like this: ``` -*.example.io. 1800 IN A 1.1.1.1 +*.example.io. 1800 IN A 192.0.2.1 *.example.io. 1800 IN AAAA 2001::1 ``` where `example.io` is the domain under which GitLab Pages will be served -and `1.1.1.1` is the IPv4 address of your GitLab instance and `2001::1` is the +and `192.0.2.1` is the IPv4 address of your GitLab instance and `2001::1` is the IPv6 address. If you don't have IPv6, you can omit the AAAA record. > **Note:** @@ -193,13 +193,13 @@ world. Custom domains are supported, but no TLS. ```shell pages_external_url "http://example.io" - nginx['listen_addresses'] = ['1.1.1.1'] + nginx['listen_addresses'] = ['192.0.2.1'] pages_nginx['enable'] = false - gitlab_pages['external_http'] = ['1.1.1.2:80', '[2001::2]:80'] + gitlab_pages['external_http'] = ['192.0.2.2:80', '[2001::2]:80'] ``` - where `1.1.1.1` is the primary IP address that GitLab is listening to and - `1.1.1.2` and `2001::2` are the secondary IPs the GitLab Pages daemon + where `192.0.2.1` is the primary IP address that GitLab is listening to and + `192.0.2.2` and `2001::2` are the secondary IPs the GitLab Pages daemon listens on. If you don't have IPv6, you can omit the IPv6 address. 1. [Reconfigure GitLab][reconfigure] @@ -228,16 +228,16 @@ world. Custom domains and TLS are supported. ```shell pages_external_url "https://example.io" - nginx['listen_addresses'] = ['1.1.1.1'] + nginx['listen_addresses'] = ['192.0.2.1'] pages_nginx['enable'] = false gitlab_pages['cert'] = "/etc/gitlab/ssl/example.io.crt" gitlab_pages['cert_key'] = "/etc/gitlab/ssl/example.io.key" - gitlab_pages['external_http'] = ['1.1.1.2:80', '[2001::2]:80'] - gitlab_pages['external_https'] = ['1.1.1.2:443', '[2001::2]:443'] + gitlab_pages['external_http'] = ['192.0.2.2:80', '[2001::2]:80'] + gitlab_pages['external_https'] = ['192.0.2.2:443', '[2001::2]:443'] ``` - where `1.1.1.1` is the primary IP address that GitLab is listening to and - `1.1.1.2` and `2001::2` are the secondary IPs where the GitLab Pages daemon + where `192.0.2.1` is the primary IP address that GitLab is listening to and + `192.0.2.2` and `2001::2` are the secondary IPs where the GitLab Pages daemon listens on. If you don't have IPv6, you can omit the IPv6 address. 1. [Reconfigure GitLab][reconfigure] diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md index a45c3306457..4e40a7cb18d 100644 --- a/doc/administration/pages/source.md +++ b/doc/administration/pages/source.md @@ -67,11 +67,11 @@ you need to add a [wildcard DNS A record][wiki-wildcard-dns] pointing to the host that GitLab runs. For example, an entry would look like this: ``` -*.example.io. 1800 IN A 1.1.1.1 +*.example.io. 1800 IN A 192.0.2.1 ``` where `example.io` is the domain under which GitLab Pages will be served -and `1.1.1.1` is the IP address of your GitLab instance. +and `192.0.2.1` is the IP address of your GitLab instance. > **Note:** You should not use the GitLab domain to serve user pages. For more information @@ -253,7 +253,7 @@ world. Custom domains are supported, but no TLS. port: 80 https: false - external_http: 1.1.1.2:80 + external_http: 192.0.2.2:80 ``` 1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in @@ -263,7 +263,7 @@ world. Custom domains are supported, but no TLS. ``` gitlab_pages_enabled=true - gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -listen-http 1.1.1.2:80" + gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -listen-http 192.0.2.2:80" ``` 1. Copy the `gitlab-pages-ssl` Nginx configuration file: @@ -274,7 +274,7 @@ world. Custom domains are supported, but no TLS. ``` 1. Edit all GitLab related configs in `/etc/nginx/site-available/` and replace - `0.0.0.0` with `1.1.1.1`, where `1.1.1.1` the primary IP where GitLab + `0.0.0.0` with `192.0.2.1`, where `192.0.2.1` the primary IP where GitLab listens to. 1. Restart NGINX 1. [Restart GitLab][restart] @@ -320,8 +320,8 @@ world. Custom domains and TLS are supported. port: 443 https: true - external_http: 1.1.1.2:80 - external_https: 1.1.1.2:443 + external_http: 192.0.2.2:80 + external_https: 192.0.2.2:443 ``` 1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in @@ -333,7 +333,7 @@ world. Custom domains and TLS are supported. ``` gitlab_pages_enabled=true - gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -listen-http 1.1.1.2:80 -listen-https 1.1.1.2:443 -root-cert /path/to/example.io.crt -root-key /path/to/example.io.key + gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -listen-http 192.0.2.2:80 -listen-https 192.0.2.2:443 -root-cert /path/to/example.io.crt -root-key /path/to/example.io.key ``` 1. Copy the `gitlab-pages-ssl` Nginx configuration file: @@ -344,7 +344,7 @@ world. Custom domains and TLS are supported. ``` 1. Edit all GitLab related configs in `/etc/nginx/site-available/` and replace - `0.0.0.0` with `1.1.1.1`, where `1.1.1.1` the primary IP where GitLab + `0.0.0.0` with `192.0.2.1`, where `192.0.2.1` the primary IP where GitLab listens to. 1. Restart NGINX 1. [Restart GitLab][restart] diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index 7d34d35e7d1..2649bf61d74 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -78,9 +78,10 @@ Example output: ## Uploaded Files Integrity -Various types of file can be uploaded to a GitLab installation by users. -Checksums are generated and stored in the database upon upload, and integrity -checks using those checksums can be run. These checks also detect missing files. +Various types of files can be uploaded to a GitLab installation by users. +These integrity checks can detect missing files. Additionally, for locally +stored files, checksums are generated and stored in the database upon upload, +and these checks will verify them against current files. Currently, integrity checks are supported for the following types of file: diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index 39b1883375e..ecc4ac6b29b 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -1,4 +1,4 @@ -# Project import/export +# Project import/export administration **[CORE ONLY]** >**Note:** > diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md index 6ec5baeb6e3..7ad38abe4f5 100644 --- a/doc/administration/raketasks/storage.md +++ b/doc/administration/raketasks/storage.md @@ -17,14 +17,21 @@ This task will schedule all your existing projects and attachments associated wi **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:migrate_to_hashed +sudo gitlab-rake gitlab:storage:migrate_to_hashed ``` **Source Installation** ```bash -rake gitlab:storage:migrate_to_hashed +sudo -u git -H bundle exec rake gitlab:storage:migrate_to_hashed RAILS_ENV=production +``` + +They both also accept a range as environment variable: +```bash +# to migrate any non migrated project from ID 20 to 50. +export ID_FROM=20 +export ID_TO=50 ``` You can monitor the progress in the _Admin > Monitoring > Background jobs_ screen. @@ -45,14 +52,13 @@ To have a simple summary of projects using **Legacy** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:legacy_projects +sudo gitlab-rake gitlab:storage:legacy_projects ``` **Source Installation** ```bash -rake gitlab:storage:legacy_projects - +sudo -u git -H bundle exec rake gitlab:storage:legacy_projects RAILS_ENV=production ``` ------ @@ -62,13 +68,13 @@ To list projects using **Legacy** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:list_legacy_projects +sudo gitlab-rake gitlab:storage:list_legacy_projects ``` **Source Installation** ```bash -rake gitlab:storage:list_legacy_projects +sudo -u git -H bundle exec rake gitlab:storage:list_legacy_projects RAILS_ENV=production ``` @@ -79,14 +85,13 @@ To have a simple summary of projects using **Hashed** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:hashed_projects +sudo gitlab-rake gitlab:storage:hashed_projects ``` **Source Installation** ```bash -rake gitlab:storage:hashed_projects - +sudo -u git -H bundle exec rake gitlab:storage:hashed_projects RAILS_ENV=production ``` ------ @@ -96,14 +101,13 @@ To list projects using **Hashed** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:list_hashed_projects +sudo gitlab-rake gitlab:storage:list_hashed_projects ``` **Source Installation** ```bash -rake gitlab:storage:list_hashed_projects - +sudo -u git -H bundle exec rake gitlab:storage:list_hashed_projects RAILS_ENV=production ``` ## List attachments on Legacy storage @@ -113,14 +117,13 @@ To have a simple summary of project attachments using **Legacy** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:legacy_attachments +sudo gitlab-rake gitlab:storage:legacy_attachments ``` **Source Installation** ```bash -rake gitlab:storage:legacy_attachments - +sudo -u git -H bundle exec rake gitlab:storage:legacy_attachments RAILS_ENV=production ``` ------ @@ -130,14 +133,13 @@ To list project attachments using **Legacy** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:list_legacy_attachments +sudo gitlab-rake gitlab:storage:list_legacy_attachments ``` **Source Installation** ```bash -rake gitlab:storage:list_legacy_attachments - +sudo -u git -H bundle exec rake gitlab:storage:list_legacy_attachments RAILS_ENV=production ``` ## List attachments on Hashed storage @@ -147,14 +149,13 @@ To have a simple summary of project attachments using **Hashed** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:hashed_attachments +sudo gitlab-rake gitlab:storage:hashed_attachments ``` **Source Installation** ```bash -rake gitlab:storage:hashed_attachments - +sudo -u git -H bundle exec rake gitlab:storage:hashed_attachments RAILS_ENV=production ``` ------ @@ -164,14 +165,13 @@ To list project attachments using **Hashed** storage: **Omnibus Installation** ```bash -gitlab-rake gitlab:storage:list_hashed_attachments +sudo gitlab-rake gitlab:storage:list_hashed_attachments ``` **Source Installation** ```bash -rake gitlab:storage:list_hashed_attachments - +sudo -u git -H bundle exec rake gitlab:storage:list_hashed_attachments RAILS_ENV=production ``` [storage-types]: ../repository_storage_types.md diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md index ee37ea49874..efeec9db517 100644 --- a/doc/administration/repository_checks.md +++ b/doc/administration/repository_checks.md @@ -13,12 +13,12 @@ checks failed you can see their output on the admin log page under ## Periodic checks -When enabled, GitLab periodically runs a repository check on all project -repositories and wiki repositories in order to detect data corruption problems. +When enabled, GitLab periodically runs a repository check on all project +repositories and wiki repositories in order to detect data corruption. A project will be checked no more than once per month. If any projects fail their repository checks all GitLab administrators will receive an email -notification of the situation. This notification is sent out once a week on -Sunday, by default. +notification of the situation. This notification is sent out once a week, +by default, midnight at the start of Sunday. ## Disabling periodic checks @@ -28,16 +28,18 @@ panel. ## What to do if a check failed If the repository check fails for some repository you should look up the error -in repocheck.log (in the admin panel or on disk; see -`/var/log/gitlab/gitlab-rails` for Omnibus installations or -`/home/git/gitlab/log` for installations from source). Once you have -resolved the issue use the admin panel to trigger a new repository check on -the project. This will clear the 'check failed' state. +in `repocheck.log`: + +- in the [admin panel](logs.md#repocheck.log) +- or on disk, see: + - `/var/log/gitlab/gitlab-rails` for Omnibus installations + - `/home/git/gitlab/log` for installations from source If for some reason the periodic repository check caused a lot of false -alarms you can choose to clear ALL repository check states from the -'Settings' page of the admin panel. +alarms you can choose to clear *all* repository check states by +clicking "Clear all repository checks" on the **Settings** page of the +admin panel (`/admin/application_settings`). --- [ce-3232]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3232 "Auto git fsck" -[git-fsck]: https://www.kernel.org/pub/software/scm/git/docs/git-fsck.html "git fsck documentation" +[git-fsck]: https://git-scm.com/docs/git-fsck "git fsck documentation" diff --git a/doc/api/README.md b/doc/api/README.md index e777fc63d2b..6267618d3bc 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -33,6 +33,7 @@ following locations: - [Jobs](jobs.md) - [Keys](keys.md) - [Labels](labels.md) +- [Markdown](markdown.md) - [Merge Requests](merge_requests.md) - [Project milestones](milestones.md) - [Group milestones](group_milestones.md) @@ -89,24 +90,23 @@ specification. ## Compatibility Guidelines The HTTP API is versioned using a single number, the current one being 4. This -number symbolises the same as the major version number as described by +number symbolises the same as the major version number as described by [SemVer](https://semver.org/). This mean that backward incompatible changes will require this version number to change. However, the minor version is -not explicit. This allows for a stable API endpoint, but also means new +not explicit. This allows for a stable API endpoint, but also means new features can be added to the API in the same version number. New features and bug fixes are released in tandem with a new GitLab, and apart from incidental patch and security releases, are released on the 22nd each -month. Backward incompatible changes (e.g. endpoints removal, parameters -removal etc.), as well as removal of entire API versions are done in tandem -with a major point release of GitLab itself. All deprecations and changes -between two versions should be listed in the documentation. For the changes +month. Backward incompatible changes (e.g. endpoints removal, parameters +removal etc.), as well as removal of entire API versions are done in tandem +with a major point release of GitLab itself. All deprecations and changes +between two versions should be listed in the documentation. For the changes between v3 and v4; please read the [v3 to v4 documentation](v3_to_v4.md) -#### Current status +### Current status -Currently two API versions are available, v3 and v4. v3 is deprecated and -will soon be removed. Deletion is scheduled for +Currently only API version v4 is available. Version v3 was removed in [GitLab 11.0](https://gitlab.com/gitlab-org/gitlab-ce/issues/36819). ## Basic usage diff --git a/doc/api/access_requests.md b/doc/api/access_requests.md index 603fa4a8194..4b2014ca843 100644 --- a/doc/api/access_requests.md +++ b/doc/api/access_requests.md @@ -10,7 +10,7 @@ 10 => Guest access 20 => Reporter access 30 => Developer access -40 => Master access +40 => Maintainer access 50 => Owner access # Only valid for groups ``` diff --git a/doc/api/applications.md b/doc/api/applications.md index 933867ed0bb..6d244594b71 100644 --- a/doc/api/applications.md +++ b/doc/api/applications.md @@ -23,7 +23,7 @@ POST /applications | `scopes` | string | yes | The scopes of the application | ```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "name=MyApplication&redirect_uri=http://redirect.uri&scopes=" https://gitlab.example.com/api/v3/applications +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "name=MyApplication&redirect_uri=http://redirect.uri&scopes=" https://gitlab.example.com/api/v4/applications ``` Example response: diff --git a/doc/api/avatar.md b/doc/api/avatar.md new file mode 100644 index 00000000000..7faed893066 --- /dev/null +++ b/doc/api/avatar.md @@ -0,0 +1,33 @@ +# Avatar API + +> [Introduced][ce-19121] in GitLab 11.0 + +## Get a single avatar URL + +Get a single avatar URL for a given email addres. If user with matching public +email address is not found, results from external avatar services are returned. +This endpoint can be accessed without authentication. In case public visibility +is restricted, response will be `403 Forbidden` when unauthenticated. + +``` +GET /avatar?email=admin@example.com +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `email` | string | yes | Public email address of the user | +| `size` | integer | no | Single pixel dimension (since images are squares). Only used for avatar lookups at `Gravatar` or at the configured `Libravatar` server | + +```bash +curl https://gitlab.example.com/api/v4/avatar?email=admin@example.com +``` + +Example response: + +```json +{ + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon" +} +``` + +[ce-19121]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19121 diff --git a/doc/api/commits.md b/doc/api/commits.md index d1584cf64de..d07b9d5614a 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -16,6 +16,7 @@ GET /projects/:id/repository/commits | `until` | string | no | Only commits before or on this date will be returned in ISO 8601 format YYYY-MM-DDTHH:MM:SSZ | | `path` | string | no | The file path | | `all` | boolean | no | Retrieve every commit from the repository | +| `with_stats` | boolean | no | Stats about each commit will be added to the response | ```bash diff --git a/doc/api/environments.md b/doc/api/environments.md index 6e20781f51a..29da4590a59 100644 --- a/doc/api/environments.md +++ b/doc/api/environments.md @@ -123,7 +123,7 @@ POST /projects/:id/environments/:environment_id/stop | `environment_id` | integer | yes | The ID of the environment | ```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/environments/1/stop" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/environments/1/stop" ``` Example response: diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md new file mode 100644 index 00000000000..59e27922f64 --- /dev/null +++ b/doc/api/graphql/index.md @@ -0,0 +1,40 @@ +# GraphQL API (Beta) + +> [Introduced][ce-19008] in GitLab 11.0. + +[GraphQL](https://graphql.org/) is a query language for APIs that +allows clients to request exactly the data they need, making it +possible to get all required data in a limited number of requests. + +The GraphQL data (fields) can be described in the form of types, +allowing clients to use [clientside GraphQL +libraries](https://graphql.org/code/#graphql-clients) to consume the +API and avoid manual parsing. + +Since there's no fixed endpoints and datamodel, new abilities can be +added to the API without creating breaking changes. This allows us to +have a versionless API as described in [the GraphQL +documentation](https://graphql.org/learn/best-practices/#versioning). + +## Enabling the GraphQL feature + +The GraphQL API itself is currently in Alpha, and therefore hidden behind a +feature flag. You can enable the feature using the [features api][features-api] on a self-hosted instance. + +For example: + +```shell +curl --data "value=100" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/features/graphql +``` + +## Available queries + +A first iteration of a GraphQL API includes a query for: `project`. Within a project it is also possible to fetch a `mergeRequest` by IID. + +## GraphiQL + +The API can be explored by using the GraphiQL IDE, it is available on your +instance on `gitlab.example.com/-/graphql-explorer`. + +[ce-19008]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19008 +[features-api]: ../features.md diff --git a/doc/api/group_milestones.md b/doc/api/group_milestones.md index 21d3ac73000..152929b7614 100644 --- a/doc/api/group_milestones.md +++ b/doc/api/group_milestones.md @@ -22,7 +22,7 @@ Parameters: | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | | `iids[]` | Array[integer] | optional | Return only the milestones having the given `iid` | -| `state` | string | optional | Return only `active` or `closed` milestones` | +| `state` | string | optional | Return only `active` or `closed` milestones | | `search` | string | optional | Return only milestones with a title or description matching the provided string | ```bash diff --git a/doc/api/groups.md b/doc/api/groups.md index 923fd662a5b..96842ef330f 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -487,6 +487,9 @@ Parameters: - `id` (required) - The ID or path of a user group +This will queue a background job to delete all projects in the group. The +response will be a 202 Accepted if the user has authorization. + ## Search for group Get all groups that match your string in their name or path. diff --git a/doc/api/issues.md b/doc/api/issues.md index 7479c1d2f93..5613cb6d915 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -38,8 +38,8 @@ GET /issues?my_reaction_emoji=star | `state` | string | no | Return all issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | | `milestone` | string | no | The milestone title | -| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `author_id` | integer | no | Return issues created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me`. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`. Defaults to `created_by_me`<br> For versions before 11.0, use the now deprecated `created-by-me` or `assigned-to-me` scopes instead.<br> _([Introduced][ce-13004] in GitLab 9.5. [Changed to snake_case][ce-18935] in GitLab 11.0)_ | +| `author_id` | integer | no | Return issues created by the given user `id`. Combine with `scope=all` or `scope=assigned_to_me`. _([Introduced][ce-13004] in GitLab 9.5)_ | | `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | | `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | | `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | @@ -152,7 +152,7 @@ GET /groups/:id/issues?my_reaction_emoji=star | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | | `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | | `milestone` | string | no | The milestone title | -| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`.<br> For versions before 11.0, use the now deprecated `created-by-me` or `assigned-to-me` scopes instead.<br> _([Introduced][ce-13004] in GitLab 9.5. [Changed to snake_case][ce-18935] in GitLab 11.0)_ | | `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | | `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | | `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | @@ -266,7 +266,7 @@ GET /projects/:id/issues?my_reaction_emoji=star | `state` | string | no | Return all issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | | `milestone` | string | no | The milestone title | -| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`.<br> For versions before 11.0, use the now deprecated `created-by-me` or `assigned-to-me` scopes instead.<br> _([Introduced][ce-13004] in GitLab 9.5. [Changed to snake_case][ce-18935] in GitLab 11.0)_ | | `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | | `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | | `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | @@ -467,7 +467,7 @@ POST /projects/:id/issues | `description` | string | no | The description of an issue | | `confidential` | boolean | no | Set an issue to be confidential. Default is `false`. | | `assignee_ids` | Array[integer] | no | The ID of the users to assign issue | -| `milestone_id` | integer | no | The ID of a milestone to assign issue | +| `milestone_id` | integer | no | The global ID of a milestone to assign issue | | `labels` | string | no | Comma-separated label names for an issue | | `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` (requires admin or project owner rights) | | `due_date` | string | no | Date time string in the format YEAR-MONTH-DAY, e.g. `2016-03-11` | @@ -548,7 +548,7 @@ PUT /projects/:id/issues/:issue_iid | `description` | string | no | The description of an issue | | `confidential` | boolean | no | Updates an issue to be confidential | | `assignee_ids` | Array[integer] | no | The ID of the user(s) to assign the issue to. Set to `0` or provide an empty value to unassign all assignees. | -| `milestone_id` | integer | no | The ID of a milestone to assign the issue to. Set to `0` or provide an empty value to unassign a milestone.| +| `milestone_id` | integer | no | The global ID of a milestone to assign the issue to. Set to `0` or provide an empty value to unassign a milestone.| | `labels` | string | no | Comma-separated label names for an issue. Set to an empty string to unassign all labels. | | `state_event` | string | no | The state event of an issue. Set `close` to close the issue and `reopen` to reopen it | | `updated_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` (requires admin or project owner rights) | @@ -1254,3 +1254,4 @@ Example response: [ce-13004]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13004 [ce-14016]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14016 [ce-17042]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17042 +[ce-18935]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18935 diff --git a/doc/api/jobs.md b/doc/api/jobs.md index db4fe2f6880..0fbfc7cf0fd 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -38,6 +38,7 @@ Example of response "size": 1000 }, "finished_at": "2015-12-24T17:54:27.895Z", + "artifacts_expire_at": "2016-01-23T17:54:27.895Z" "id": 7, "name": "teaspoon", "pipeline": { @@ -81,8 +82,9 @@ Example of response "created_at": "2015-12-24T15:51:21.727Z", "artifacts_file": null, "finished_at": "2015-12-24T17:54:24.921Z", + "artifacts_expire_at": "2016-01-23T17:54:24.921Z", "id": 6, - "name": "spinach:other", + "name": "rspec:other", "pipeline": { "id": 6, "ref": "master", @@ -152,6 +154,7 @@ Example of response "size": 1000 }, "finished_at": "2015-12-24T17:54:27.895Z", + "artifacts_expire_at": "2016-01-23T17:54:27.895Z" "id": 7, "name": "teaspoon", "pipeline": { @@ -195,8 +198,9 @@ Example of response "created_at": "2015-12-24T15:51:21.727Z", "artifacts_file": null, "finished_at": "2015-12-24T17:54:24.921Z", + "artifacts_expire_at": "2016-01-23T17:54:24.921Z" "id": 6, - "name": "spinach:other", + "name": "rspec:other", "pipeline": { "id": 6, "ref": "master", @@ -261,6 +265,7 @@ Example of response "created_at": "2015-12-24T15:51:21.880Z", "artifacts_file": null, "finished_at": "2015-12-24T17:54:31.198Z", + "artifacts_expire_at": "2016-01-23T17:54:31.198Z", "id": 8, "name": "rubocop", "pipeline": { diff --git a/doc/api/markdown.md b/doc/api/markdown.md new file mode 100644 index 00000000000..f406838e887 --- /dev/null +++ b/doc/api/markdown.md @@ -0,0 +1,29 @@ +# Markdown API + +> [Introduced][ce-18926] in GitLab 11.0. + +Available only in APIv4. + +## Render an arbitrary Markdown document + +``` +POST /api/v4/markdown +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | ------------- | ------------------------------------------ | +| `text` | string | yes | The markdown text to render | +| `gfm` | boolean | no (optional) | Render text using GitLab Flavored Markdown. Default is `false` | +| `project` | string | no (optional) | Use `project` as a context when creating references using GitLab Flavored Markdown. [Authentication](README.html#authentication) is required if a project is not public. | + +```bash +curl --header Content-Type:application/json --data '{"text":"Hello world! :tada:", "gfm":true, "project":"group_example/project_example"}' https://gitlab.example.com/api/v4/markdown +``` + +Response example: + +```json +{ "html": "<p dir=\"auto\">Hello world! <gl-emoji title=\"party popper\" data-name=\"tada\" data-unicode-version=\"6.0\">🎉</gl-emoji></p>" } +``` + +[ce-18926]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18926 diff --git a/doc/api/members.md b/doc/api/members.md index 3234f833eae..1a10aa75ac0 100644 --- a/doc/api/members.md +++ b/doc/api/members.md @@ -8,7 +8,7 @@ The access levels are defined in the `Gitlab::Access` module. Currently, these l 10 => Guest access 20 => Reporter access 30 => Developer access -40 => Master access +40 => Maintainer access 50 => Owner access # Only valid for groups ``` diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index b9a4f661777..da74045b702 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -28,7 +28,7 @@ GET /merge_requests?milestone=release GET /merge_requests?labels=bug,reproduced GET /merge_requests?author_id=5 GET /merge_requests?my_reaction_emoji=star -GET /merge_requests?scope=assigned-to-me +GET /merge_requests?scope=assigned_to_me ``` Parameters: @@ -45,8 +45,8 @@ Parameters: | `created_before` | datetime | no | Return merge requests created on or before the given time | | `updated_after` | datetime | no | Return merge requests updated on or after the given time | | `updated_before` | datetime | no | Return merge requests updated on or before the given time | -| `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` | -| `author_id` | integer | no | Returns merge requests created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me` | +| `scope` | string | no | Return merge requests for the given scope: `created_by_me`, `assigned_to_me` or `all`. Defaults to `created_by_me`<br> For versions before 11.0, use the now deprecated `created-by-me` or `assigned-to-me` scopes instead. | +| `author_id` | integer | no | Returns merge requests created by the given user `id`. Combine with `scope=all` or `scope=assigned_to_me` | | `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` | | `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | | `source_branch` | string | no | Return merge requests with the given source branch | @@ -70,18 +70,18 @@ Parameters: "author": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "assignee": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "source_project_id": 2, "target_project_id": 3, @@ -107,6 +107,7 @@ Parameters: "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "time_stats": { "time_estimate": 0, @@ -164,7 +165,7 @@ Parameters: | `created_before` | datetime | no | Return merge requests created on or before the given time | | `updated_after` | datetime | no | Return merge requests updated on or after the given time | | `updated_before` | datetime | no | Return merge requests updated on or before the given time | -| `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13060] in GitLab 9.5)_ | +| `scope` | string | no | Return merge requests for the given scope: `created_by_me`, `assigned_to_me` or `all`.<br> For versions before 11.0, use the now deprecated `created-by-me` or `assigned-to-me` scopes instead.<br> _([Introduced][ce-13060] in GitLab 9.5. [Changed to snake_case][ce-18935] in GitLab 11.0)_ | | `author_id` | integer | no | Returns merge requests created by the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | | `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | | `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | @@ -189,18 +190,125 @@ Parameters: "author": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "assignee": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" + }, + "source_project_id": 2, + "target_project_id": 3, + "labels": [ ], + "description": "fixed login page css paddings", + "work_in_progress": false, + "milestone": { + "id": 5, + "iid": 1, + "project_id": 3, + "title": "v2.0", + "description": "Assumenda aut placeat expedita exercitationem labore sunt enim earum.", + "state": "closed", + "created_at": "2015-02-02T19:49:26.013Z", + "updated_at": "2015-02-02T19:49:26.013Z", + "due_date": null + }, + "merge_when_pipeline_succeeds": true, + "merge_status": "can_be_merged", + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null, + "user_notes_count": 1, + "changes_count": "1", + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "squash": false, + "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } + } +] +``` + +## List group merge requests + +Get all merge requests for this group and its subgroups. +The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). +The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. + +``` +GET /groups/:id/merge_requests +GET /groups/:id/merge_requests?state=opened +GET /groups/:id/merge_requests?state=all +GET /groups/:id/merge_requests?milestone=release +GET /groups/:id/merge_requests?labels=bug,reproduced +GET /groups/:id/merge_requests?my_reaction_emoji=star +``` + +`group_id` represents the ID of the group which contains the project where the MR resides. + +Parameters: + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ | +| `id` | integer | yes | The ID of a group | +| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, or `merged` | +| `order_by` | string | no | Return merge requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return merge requests sorted in `asc` or `desc` order. Default is `desc` | +| `milestone` | string | no | Return merge requests for a specific milestone | +| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | +| `labels` | string | no | Return merge requests matching a comma separated list of labels | +| `created_after` | datetime | no | Return merge requests created on or after the given time | +| `created_before` | datetime | no | Return merge requests created on or before the given time | +| `updated_after` | datetime | no | Return merge requests updated on or after the given time | +| `updated_before` | datetime | no | Return merge requests updated on or before the given time | +| `scope` | string | no | Return merge requests for the given scope: `created_by_me`, `assigned_to_me` or `all`.<br> | +| `author_id` | integer | no | Returns merge requests created by the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | +| `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | +| `source_branch` | string | no | Return merge requests with the given source branch | +| `target_branch` | string | no | Return merge requests with the given target branch | +| `search` | string | no | Search merge requests against their `title` and `description` | + +```json +[ + { + "id": 1, + "iid": 1, + "target_branch": "master", + "source_branch": "test1", + "project_id": 3, + "title": "test1", + "state": "opened", + "created_at": "2017-04-29T08:46:00Z", + "updated_at": "2017-04-29T08:46:00Z", + "upvotes": 0, + "downvotes": 0, + "author": { + "id": 1, + "username": "admin", + "name": "Administrator", + "state": "active", + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" + }, + "assignee": { + "id": 1, + "username": "admin", + "name": "Administrator", + "state": "active", + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "source_project_id": 2, "target_project_id": 3, @@ -305,6 +413,7 @@ Parameters: "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, "time_stats": { @@ -439,14 +548,16 @@ Parameters: "username": "jarrett", "id": 5, "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/b95567800f828948baf5f4160ebb2473?s=40&d=identicon" + "avatar_url": "http://www.gravatar.com/avatar/b95567800f828948baf5f4160ebb2473?s=40&d=identicon", + "web_url" : "https://gitlab.example.com/jarrett" }, "assignee": { "name": "Administrator", "username": "root", "id": 1, "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40&d=identicon" + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40&d=identicon", + "web_url" : "https://gitlab.example.com/root" }, "source_project_id": 4, "target_project_id": 4, @@ -473,6 +584,7 @@ Parameters: "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, "time_stats": { @@ -539,9 +651,11 @@ POST /projects/:id/merge_requests | `description` | string | no | Description of MR | | `target_project_id` | integer | no | The target project (numeric id) | | `labels` | string | no | Labels for MR as a comma-separated list | -| `milestone_id` | integer | no | The ID of a milestone | +| `milestone_id` | integer | no | The global ID of a milestone | | `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | -| `allow_maintainer_to_push` | boolean | no | Whether or not a maintainer of the target project can push to the source branch | +| `allow_collaboration` | boolean | no | Allow commits from members who can merge to the target branch | +| `allow_maintainer_to_push` | boolean | no | Deprecated, see allow_collaboration | +| `squash` | boolean | no | Squash commits into a single commit when merging | ```json { @@ -557,18 +671,18 @@ POST /projects/:id/merge_requests "author": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "assignee": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "source_project_id": 3, "target_project_id": 4, @@ -595,8 +709,10 @@ POST /projects/:id/merge_requests "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, + "allow_collaboration": false, "allow_maintainer_to_push": false, "time_stats": { "time_estimate": 0, @@ -622,13 +738,15 @@ PUT /projects/:id/merge_requests/:merge_request_iid | `target_branch` | string | no | The target branch | | `title` | string | no | Title of MR | | `assignee_id` | integer | no | The ID of the user to assign the merge request to. Set to `0` or provide an empty value to unassign all assignees. | -| `milestone_id` | integer | no | The ID of a milestone to assign the merge request to. Set to `0` or provide an empty value to unassign a milestone.| +| `milestone_id` | integer | no | The global ID of a milestone to assign the merge request to. Set to `0` or provide an empty value to unassign a milestone.| | `labels` | string | no | Comma-separated label names for a merge request. Set to an empty string to unassign all labels. | | `description` | string | no | Description of MR | | `state_event` | string | no | New state (close/reopen) | | `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | +| `squash` | boolean | no | Squash commits into a single commit when merging | | `discussion_locked` | boolean | no | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | -| `allow_maintainer_to_push` | boolean | no | Whether or not a maintainer of the target project can push to the source branch | +| `allow_collaboration` | boolean | no | Allow commits from members who can merge to the target branch | +| `allow_maintainer_to_push` | boolean | no | Deprecated, see allow_collaboration | Must include at least one non-required attribute from above. @@ -645,18 +763,18 @@ Must include at least one non-required attribute from above. "author": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "assignee": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "source_project_id": 3, "target_project_id": 4, @@ -683,8 +801,10 @@ Must include at least one non-required attribute from above. "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, + "allow_collaboration": false, "allow_maintainer_to_push": false, "time_stats": { "time_estimate": 0, @@ -752,18 +872,18 @@ Parameters: "author": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "assignee": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "source_project_id": 4, "target_project_id": 4, @@ -790,6 +910,7 @@ Parameters: "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, "time_stats": { @@ -830,18 +951,18 @@ Parameters: "author": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "assignee": { "id": 1, "username": "admin", - "email": "admin@example.com", "name": "Administrator", "state": "active", - "created_at": "2012-04-29T08:46:00Z" + "avatar_url": null, + "web_url" : "https://gitlab.example.com/admin" }, "source_project_id": 4, "target_project_id": 4, @@ -868,6 +989,7 @@ Parameters: "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, "time_stats": { @@ -1200,6 +1322,7 @@ Example response: "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1" }, "target_url": "https://gitlab.example.com/gitlab-org/gitlab-ci/merge_requests/7", @@ -1460,3 +1583,4 @@ Example response: [ce-13060]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13060 [ce-14016]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14016 [ce-15454]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15454 +[ce-18935]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18935 diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index 1f0dc700640..656bf07bb55 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -54,7 +54,7 @@ Example response: ] ``` -**Note**: `members_count_with_descendants` are presented only for group masters/owners. +**Note**: `members_count_with_descendants` are presented only for group maintainers/owners. ## Search for namespace diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index 899f5da6647..ebae68fe389 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -102,6 +102,7 @@ POST /projects/:id/pipeline |------------|---------|----------|---------------------| | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `ref` | string | yes | Reference to commit | +| `variables` | array | no | An array containing the variables available in the pipeline, matching the structure [{ 'key' => 'UPLOAD_TO_S3', 'value' => 'true' }] | ``` curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/pipeline?ref=master" diff --git a/doc/api/projects.md b/doc/api/projects.md index fe21dfe23f9..d3e95926322 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -66,6 +66,7 @@ GET /projects "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git", "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git", "web_url": "http://example.com/diaspora/diaspora-client", + "readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md", "tag_list": [ "example", "disapora client" @@ -135,6 +136,7 @@ GET /projects "ssh_url_to_repo": "git@example.com:brightbox/puppet.git", "http_url_to_repo": "http://example.com/brightbox/puppet.git", "web_url": "http://example.com/brightbox/puppet", + "readme_url": "http://example.com/brightbox/puppet/blob/master/README.md", "tag_list": [ "example", "puppet" @@ -252,6 +254,7 @@ GET /users/:user_id/projects "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git", "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git", "web_url": "http://example.com/diaspora/diaspora-client", + "readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md", "tag_list": [ "example", "disapora client" @@ -321,6 +324,7 @@ GET /users/:user_id/projects "ssh_url_to_repo": "git@example.com:brightbox/puppet.git", "http_url_to_repo": "http://example.com/brightbox/puppet.git", "web_url": "http://example.com/brightbox/puppet", + "readme_url": "http://example.com/brightbox/puppet/blob/master/README.md", "tag_list": [ "example", "puppet" @@ -420,6 +424,7 @@ GET /projects/:id "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", "web_url": "http://example.com/diaspora/diaspora-project-site", + "readme_url": "http://example.com/diaspora/diaspora-project-site/blob/master/README.md", "tag_list": [ "example", "disapora project" @@ -710,6 +715,7 @@ Example responses: "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", "web_url": "http://example.com/diaspora/diaspora-project-site", + "readme_url": "http://example.com/diaspora/diaspora-project-site/blob/master/README.md", "tag_list": [ "example", "disapora project" @@ -788,6 +794,7 @@ Example response: "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", "web_url": "http://example.com/diaspora/diaspora-project-site", + "readme_url": "http://example.com/diaspora/diaspora-project-site/blob/master/README.md", "tag_list": [ "example", "disapora project" @@ -865,6 +872,7 @@ Example response: "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", "web_url": "http://example.com/diaspora/diaspora-project-site", + "readme_url": "http://example.com/diaspora/diaspora-project-site/blob/master/README.md", "tag_list": [ "example", "disapora project" @@ -966,6 +974,7 @@ Example response: "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", "web_url": "http://example.com/diaspora/diaspora-project-site", + "readme_url": "http://example.com/diaspora/diaspora-project-site/blob/master/README.md", "tag_list": [ "example", "disapora project" @@ -1061,6 +1070,7 @@ Example response: "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", "web_url": "http://example.com/diaspora/diaspora-project-site", + "readme_url": "http://example.com/diaspora/diaspora-project-site/blob/master/README.md", "tag_list": [ "example", "disapora project" @@ -1159,7 +1169,7 @@ The `file=` parameter must point to a file on your filesystem and be preceded by `@`. For example: ```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "file=@dk.png" https://gitlab.example.com/api/v3/projects/5/uploads +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "file=@dk.png" https://gitlab.example.com/api/v4/projects/5/uploads ``` Returned object: @@ -1421,4 +1431,3 @@ GET /projects/:id/snapshot | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | | `wiki` | boolean | no | Whether to download the wiki, rather than project, repository | - diff --git a/doc/api/protected_branches.md b/doc/api/protected_branches.md index 950ead52560..f6813f27dc0 100644 --- a/doc/api/protected_branches.md +++ b/doc/api/protected_branches.md @@ -8,7 +8,7 @@ The access levels are defined in the `ProtectedRefAccess::ALLOWED_ACCESS_LEVELS` ``` 0 => No access 30 => Developer access -40 => Master access +40 => Maintainer access ``` ## List protected branches @@ -36,13 +36,13 @@ Example response: "push_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ], "merge_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ] }, @@ -75,13 +75,13 @@ Example response: "push_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ], "merge_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ] } @@ -104,8 +104,8 @@ curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" 'https://gitl | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `name` | string | yes | The name of the branch or wildcard | -| `push_access_level` | string | no | Access levels allowed to push (defaults: `40`, master access level) | -| `merge_access_level` | string | no | Access levels allowed to merge (defaults: `40`, master access level) | +| `push_access_level` | string | no | Access levels allowed to push (defaults: `40`, maintainer access level) | +| `merge_access_level` | string | no | Access levels allowed to merge (defaults: `40`, maintainer access level) | Example response: @@ -115,13 +115,13 @@ Example response: "push_access_levels": [ { "access_level": 30, - "access_level_description": "Developers + Masters" + "access_level_description": "Developers + Maintainers" } ], "merge_access_levels": [ { "access_level": 30, - "access_level_description": "Developers + Masters" + "access_level_description": "Developers + Maintainers" } ] } diff --git a/doc/api/runners.md b/doc/api/runners.md index f384ac57bfe..3ca07ce9795 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -411,3 +411,86 @@ DELETE /projects/:id/runners/:runner_id ``` curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/9/runners/9" ``` + +## Register a new Runner + +Register a new Runner for the instance. + +``` +POST /runners +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|---------------------| +| `token` | string | yes | Registration token ([Read how to obtain a token](../ci/runners/README.md)) | +| `description`| string | no | Runner's description| +| `info` | hash | no | Runner's metadata | +| `active` | boolean| no | Whether the Runner is active | +| `locked` | boolean| no | Whether the Runner should be locked for current project | +| `run_untagged` | boolean | no | Whether the Runner should handle untagged jobs | +| `tag_list` | Array[String] | no | List of Runner's tags | +| `maximum_timeout` | integer | no | Maximum timeout set when this Runner will handle the job | + +``` +curl --request POST "https://gitlab.example.com/api/v4/runners" --form "token=ipzXrMhuyyJPifUt6ANz" --form "description=test-1-20150125-test" --form "tag_list=ruby,mysql,tag1,tag2" +``` + +Response: + +| Status | Description | +|-----------|---------------------------------| +| 201 | Runner was created | + +Example response: + +```json +{ + "id": "12345", + "token": "6337ff461c94fd3fa32ba3b1ff4125" +} +``` + +## Delete a registered Runner + +Deletes a registed Runner. + +``` +DELETE /runners +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|---------------------| +| `token` | string | yes | Runner's authentication token | + +``` +curl --request DELETE "https://gitlab.example.com/api/v4/runners" --form "token=ebb6fc00521627750c8bb750f2490e" +``` + +Response: + +| Status | Description | +|-----------|---------------------------------| +| 204 | Runner was deleted | + +## Verify authentication for a registered Runner + +Validates authentication credentials for a registered Runner. + +``` +POST /runners/verify +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|---------------------| +| `token` | string | yes | Runner's authentication token | + +``` +curl --request POST "https://gitlab.example.com/api/v4/runners/verify" --form "token=ebb6fc00521627750c8bb750f2490e" +``` + +Response: + +| Status | Description | +|-----------|---------------------------------| +| 200 | Credentials are valid | +| 403 | Credentials are invalid | diff --git a/doc/api/services.md b/doc/api/services.md index 92f12acbc73..aeb48ccc36c 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -1,6 +1,6 @@ # Services API ->**Note:** This API requires an access token with Master or Owner permissions +>**Note:** This API requires an access token with Maintainer or Owner permissions ## Asana @@ -405,6 +405,13 @@ GET /projects/:id/services/flowdock Gemnasium monitors your project dependencies and alerts you about updates and security vulnerabilities. +CAUTION: **Warning:** +Gemnasium service integration has been deprecated in GitLab 11.0. Gemnasium has been +[acquired by GitLab](https://about.gitlab.com/press/releases/2018-01-30-gemnasium-acquisition.html) +in January 2018 and since May 15, 2018, the service provided by Gemnasium is no longer available. +You can [migrate from Gemnasium to GitLab](https://docs.gitlab.com/ee/user/project/import/gemnasium.html) +to keep monitoring your dependencies. + ### Create/Edit Gemnasium service Set Gemnasium service for a project. @@ -968,7 +975,7 @@ Group Chat Software Set Microsoft Teams service for a project. ``` -PUT /projects/:id/services/microsoft_teams +PUT /projects/:id/services/microsoft-teams ``` Parameters: @@ -982,7 +989,7 @@ Parameters: Delete Microsoft Teams service for a project. ``` -DELETE /projects/:id/services/microsoft_teams +DELETE /projects/:id/services/microsoft-teams ``` ### Get Microsoft Teams service settings @@ -990,7 +997,7 @@ DELETE /projects/:id/services/microsoft_teams Get Microsoft Teams service settings for a project. ``` -GET /projects/:id/services/microsoft_teams +GET /projects/:id/services/microsoft-teams ``` ## Mattermost notifications diff --git a/doc/api/settings.md b/doc/api/settings.md index 0b5b1f0c134..e6b207d8746 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -53,6 +53,9 @@ Example response: "dsa_key_restriction": 0, "ecdsa_key_restriction": 0, "ed25519_key_restriction": 0, + "enforce_terms": true, + "terms": "Hello world!", + "performance_bar_allowed_group_id": 42 } ``` @@ -78,7 +81,7 @@ PUT /application/settings | `clientside_sentry_enabled` | boolean | no | Enable Sentry error reporting for the client side | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts | -| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and masters can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but masters can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | +| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and maintainers can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and maintainers can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but maintainers can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | | `default_group_visibility` | string | no | What visibility level new groups receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_project_visibility` | string | no | What visibility level new projects receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_projects_limit` | integer | no | Project limit per user. Default is `100000` | @@ -118,8 +121,9 @@ PUT /application/settings | `metrics_timeout` | integer | yes (if `metrics_enabled` is `true`) | The amount of seconds after which InfluxDB will time out. | | `password_authentication_enabled_for_web` | boolean | no | Enable authentication for the web interface via a GitLab account password. Default is `true`. | | `password_authentication_enabled_for_git` | boolean | no | Enable authentication for Git over HTTP(S) via a GitLab account password. Default is `true`. | -| `performance_bar_allowed_group_id` | string | no | The group that is allowed to enable the performance bar | -| `performance_bar_enabled` | boolean | no | Allow enabling the performance bar | +| `performance_bar_allowed_group_path` | string | no | Path of the group that is allowed to toggle the performance bar | +| `performance_bar_allowed_group_id` | string | no | Deprecated: Use `performance_bar_allowed_group_path` instead. Path of the group that is allowed to toggle the performance bar | +| `performance_bar_enabled` | boolean | no | Deprecated: Pass `performance_bar_allowed_group_path: nil` instead. Allow enabling the performance bar | | `plantuml_enabled` | boolean | no | Enable PlantUML integration. Default is `false`. | | `plantuml_url` | string | yes (if `plantuml_enabled` is `true`) | The PlantUML instance URL for integration. | | `polling_interval_multiplier` | decimal | no | Interval multiplier used by endpoints that perform polling. Set to 0 to disable polling. | @@ -131,7 +135,7 @@ PUT /application/settings | `repository_checks_enabled` | boolean | no | GitLab will periodically run 'git fsck' in all project and wiki repositories to look for silent disk corruption issues. | | `repository_storages` | array of strings | no | A list of names of enabled storage paths, taken from `gitlab.yml`. New projects will be created in one of these stores, chosen at random. | | `require_two_factor_authentication` | boolean | no | Require all users to setup Two-factor authentication | -| `restricted_visibility_levels` | array of strings | no | Selected levels cannot be used by non-admin users for projects or snippets. Can take `private`, `internal` and `public` as a parameter. Default is null which means there is no restriction. | +| `restricted_visibility_levels` | array of strings | no | Selected levels cannot be used by non-admin users for groups, projects or snippets. Can take `private`, `internal` and `public` as a parameter. Default is null which means there is no restriction. | | `rsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded RSA key. Default is `0` (no restriction). `-1` disables RSA keys. | | `send_user_confirmation_email` | boolean | no | Send confirmation email on sign-up | | `sentry_dsn` | string | yes (if `sentry_enabled` is true) | Sentry Data Source Name | @@ -153,6 +157,8 @@ PUT /application/settings | `user_default_external` | boolean | no | Newly registered users will by default be external | | `user_oauth_applications` | boolean | no | Allow users to register any application to use GitLab as an OAuth provider | | `version_check_enabled` | boolean | no | Let GitLab inform you when an update is available. | +| `enforce_terms` | boolean | no | Enforce application ToS to all users | +| `terms` | text | yes (if `enforce_terms` is true) | Markdown content for the ToS | ```bash curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/application/settings?signup_enabled=false&default_project_visibility=internal @@ -195,5 +201,8 @@ Example response: "dsa_key_restriction": 0, "ecdsa_key_restriction": 0, "ed25519_key_restriction": 0, + "enforce_terms": true, + "terms": "Hello world!", + "performance_bar_allowed_group_id": 42 } ``` diff --git a/doc/api/snippets.md b/doc/api/snippets.md index 42b760c107d..7892866cd8e 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -49,6 +49,7 @@ Example response: "title": "test", "file_name": "add.rb", "description": "Ruby test snippet", + "visibility": "private", "author": { "id": 1, "username": "john_smith", @@ -99,6 +100,7 @@ Example response: "title": "This is a snippet", "file_name": "test.txt", "description": "Hello World snippet", + "visibility": "internal", "author": { "id": 1, "username": "john_smith", @@ -150,6 +152,7 @@ Example response: "title": "test", "file_name": "add.rb", "description": "description of snippet", + "visibility": "internal", "author": { "id": 1, "username": "john_smith", @@ -238,7 +241,8 @@ Example response: "raw_url": "http://localhost:3000/snippets/48/raw", "title": "Minus similique nesciunt vel fugiat qui ullam sunt.", "updated_at": "2016-11-25T16:53:34.479Z", - "web_url": "http://localhost:3000/snippets/48" + "web_url": "http://localhost:3000/snippets/48", + "visibility": "public" } ] ``` diff --git a/doc/api/v3_to_v4.md b/doc/api/v3_to_v4.md index 9835fab7c98..98eae66469f 100644 --- a/doc/api/v3_to_v4.md +++ b/doc/api/v3_to_v4.md @@ -2,10 +2,9 @@ Since GitLab 9.0, API V4 is the preferred version to be used. -API V3 will be unsupported from GitLab 9.5, to be released on August -22, 2017. It will be removed in GitLab 9.5 or later. In the meantime, we advise -you to make any necessary changes to applications that use V3. The V3 API -documentation is still +API V3 was unsupported from GitLab 9.5, released on August +22, 2017. API v3 was removed in [GitLab 11.0](https://gitlab.com/gitlab-org/gitlab-ce/issues/36819). +The V3 API documentation is still [available](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-16-stable/doc/api/README.md). Below are the changes made between V3 and V4. diff --git a/doc/articles/index.md b/doc/articles/index.md index 9f85533ea94..87ee17bb6de 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -4,7 +4,7 @@ comments: false # Technical articles list (deprecated) -[Technical articles](../development/writing_documentation.md#technical-articles) are +[Technical articles](../development/documentation/index.md#technical-articles) are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. diff --git a/doc/articles/openshift_and_gitlab/index.md b/doc/articles/openshift_and_gitlab/index.md index b7594cfef7f..76fdb2eb00a 100644 --- a/doc/articles/openshift_and_gitlab/index.md +++ b/doc/articles/openshift_and_gitlab/index.md @@ -1 +1,4 @@ -This document was moved to [another location](../../install/openshift_and_gitlab/index.html). +--- +redirect_to: '../../install/openshift_and_gitlab/index.html' +--- + diff --git a/doc/ci/README.md b/doc/ci/README.md index 6aa0e5885db..7666219acb0 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,5 +1,6 @@ --- comments: false +description: "Learn how to use GitLab CI/CD, the GitLab built-in Continuous Integration, Continuous Deployment, and Continuous Delivery toolset to build, test, and deploy your application." --- # GitLab Continuous Integration (GitLab CI/CD) @@ -18,7 +19,7 @@ Here's some info we've gathered to get you started. The first steps towards your GitLab CI/CD journey. - [Getting started with GitLab CI/CD](quick_start/README.md): understand how GitLab CI/CD works. -- GitLab CI/CD configuration file: [`.gitlab-ci.yml`](yaml/README.md) - Learn all about the ins and outs of `.gitlab-ci.yml`. +- [GitLab CI/CD configuration file: `.gitlab-ci.yml`](yaml/README.md) - Learn all about the ins and outs of `.gitlab-ci.yml`. - [Pipelines and jobs](pipelines.md): configure your GitLab CI/CD pipelines to build, test, and deploy your application. - Runners: The [GitLab Runner](https://docs.gitlab.com/runner/) is responsible by running the jobs in your CI/CD pipeline. On GitLab.com, Shared Runners are enabled by default, so you don't need to set up anything to start to use them with GitLab CI/CD. @@ -45,7 +46,9 @@ you don't need to set up anything to start to use them with GitLab CI/CD. ## Exploring GitLab CI/CD - [CI/CD Variables](variables/README.md) - Learn how to use variables defined in - your `.gitlab-ci.yml` or secured ones defined in your project's settings + your `.gitlab-ci.yml` or the ones defined in your project's settings + - [Where variables can be used](variables/where_variables_can_be_used.md) - A + deeper look on where and how the CI/CD variables can be used - **The permissions model** - Learn about the access levels a user can have for performing certain CI actions - [User permissions](../user/permissions.md#gitlab-ci) diff --git a/doc/ci/autodeploy/index.md b/doc/ci/autodeploy/index.md index 7102af5c529..985ec4b972c 100644 --- a/doc/ci/autodeploy/index.md +++ b/doc/ci/autodeploy/index.md @@ -1,129 +1 @@ -# Auto Deploy - -> [Introduced][mr-8135] in GitLab 8.15. -> Auto deploy is an experimental feature and is **not recommended for Production use** at this time. - -> As of GitLab 9.1, access to the container registry is only available while the -Pipeline is running. Restarting a pod, scaling a service, or other actions which -require on-going access **will fail**. On-going secure access is planned for a -subsequent release. - -> As of GitLab 10.0, Auto Deploy templates are **deprecated** and the -functionality has been included in [Auto -DevOps](../../topics/autodevops/index.md). - -Auto deploy is an easy way to configure GitLab CI for the deployment of your -application. GitLab Community maintains a list of `.gitlab-ci.yml` -templates for various infrastructure providers and deployment scripts -powering them. These scripts are responsible for packaging your application, -setting up the infrastructure and spinning up necessary services (for -example a database). - -## How it works - -The Autodeploy templates are based on the [kubernetes-deploy][kube-deploy] -project which is used to simplify the deployment process to Kubernetes by -providing intelligent `build`, `deploy`, and `destroy` commands which you can -use in your `.gitlab-ci.yml` as is. It uses [Herokuish](https://github.com/gliderlabs/herokuish), -which uses [Heroku buildpacks](https://devcenter.heroku.com/articles/buildpacks) -to do some of the work, plus some of GitLab's own tools to package it all up. For -your convenience, a [Docker image][kube-image] is also provided. - -You can use the [Kubernetes project service](../../user/project/integrations/kubernetes.md) -to store credentials to your infrastructure provider and they will be available -during the deployment. - -## Quick start - -We made a [simple guide](quick_start_guide.md) to using Auto Deploy with GitLab.com. - -For a demonstration of GitLab Auto Deploy, read the blog post [Auto Deploy from GitLab to an OpenShift Container Cluster](https://about.gitlab.com/2017/05/16/devops-containers-gitlab-openshift/) - -## Supported templates - -The list of supported auto deploy templates is available in the -[gitlab-ci-yml project][auto-deploy-templates]. - -## Configuration - ->**Note:** -In order to understand why the following steps are required, read the -[how it works](#how-it-works) section. - -To configure Autodeploy, you will need to: - -1. Enable a deployment [project service][project-services] to store your - credentials. For example, if you want to deploy to OpenShift you have to - enable [Kubernetes service][kubernetes-service]. -1. Configure GitLab Runner to use the - [Docker or Kubernetes executor](https://docs.gitlab.com/runner/executors/) with - [privileged mode enabled][docker-in-docker]. -1. Navigate to the "Project" tab and click "Set up auto deploy" button. -  -1. Select a template. -  -1. Commit your changes and create a merge request. -1. Test your deployment configuration using a [Review App][review-app] that was - created automatically for you. - -## Private project support - -> Experimental support [introduced][mr-2] in GitLab 9.1. - -When a project has been marked as private, GitLab's [Container Registry][container-registry] requires authentication when downloading containers. Auto deploy will automatically provide the required authentication information to Kubernetes, allowing temporary access to the registry. Authentication credentials will be valid while the pipeline is running, allowing for a successful initial deployment. - -After the pipeline completes, Kubernetes will no longer be able to access the container registry. Restarting a pod, scaling a service, or other actions which require on-going access to the registry will fail. On-going secure access is planned for a subsequent release. - -## PostgreSQL database support - -> Experimental support [introduced][mr-8] in GitLab 9.1. - -In order to support applications that require a database, [PostgreSQL][postgresql] is provisioned by default. Credentials to access the database are preconfigured, but can be customized by setting the associated [variables](#postgresql-variables). These credentials can be used for defining a `DATABASE_URL` of the format: `postgres://user:password@postgres-host:postgres-port/postgres-database`. It is important to note that the database itself is temporary, and contents will be not be saved. - -PostgreSQL provisioning can be disabled by setting the variable `DISABLE_POSTGRES` to `"yes"`. - -The following PostgreSQL variables are supported: - -1. `DISABLE_POSTGRES: "yes"`: disable automatic deployment of PostgreSQL -1. `POSTGRES_USER: "my-user"`: use custom username for PostgreSQL -1. `POSTGRES_PASSWORD: "password"`: use custom password for PostgreSQL -1. `POSTGRES_DB: "my database"`: use custom database name for PostgreSQL - -## Auto Monitoring - -> Introduced in [GitLab 9.5](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13438). - -Apps auto-deployed using one the [Kubernetes templates](#supported-templates) can also be automatically monitored for: - -* Response Metrics: latency, throughput, error rate -* System Metrics: CPU utilization, memory utilization - -Metrics are gathered from [nginx-ingress](../../user/project/integrations/prometheus_library/nginx_ingress.md) and [Kubernetes](../../user/project/integrations/prometheus_library/kubernetes.md). - -To view the metrics, open the [Monitoring dashboard for a deployed environment](../environments.md#monitoring-environments). - - - -### Configuring Auto Monitoring - -If GitLab has been deployed using the [omnibus-gitlab](../../install/kubernetes/gitlab_omnibus.md) Helm chart, no configuration is required. - -If you have installed GitLab using a different method: - -1. [Deploy Prometheus](../../user/project/integrations/prometheus.md#configuring-your-own-prometheus-server-within-kubernetes) into your Kubernetes cluster -1. If you would like response metrics, ensure you are running at least version 0.9.0 of NGINX Ingress and [enable Prometheus metrics](https://github.com/kubernetes/ingress/blob/master/examples/customization/custom-vts-metrics/nginx/nginx-vts-metrics-conf.yaml). -1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) the NGINX Ingress deployment to be scraped by Prometheus using `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. - -[mr-8135]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8135 -[mr-2]: https://gitlab.com/gitlab-examples/kubernetes-deploy/merge_requests/2 -[mr-8]: https://gitlab.com/gitlab-examples/kubernetes-deploy/merge_requests/8 -[project-settings]: https://docs.gitlab.com/ce/public_access/public_access.html -[project-services]: ../../user/project/integrations/project_services.md -[auto-deploy-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml/tree/master/autodeploy -[kubernetes-service]: ../../user/project/integrations/kubernetes.md -[docker-in-docker]: ../docker/using_docker_build.md#use-docker-in-docker-executor -[review-app]: ../review_apps/index.md -[kube-image]: https://gitlab.com/gitlab-examples/kubernetes-deploy/container_registry "Kubernetes deploy Container Registry" -[kube-deploy]: https://gitlab.com/gitlab-examples/kubernetes-deploy "Kubernetes deploy example project" -[container-registry]: https://docs.gitlab.com/ce/user/project/container_registry.html -[postgresql]: https://www.postgresql.org/ +This document was moved to [another location](../../topics/autodevops/index.md#auto-deploy). diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 7c0f837ea9c..71f1d69cdf4 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -496,7 +496,7 @@ To configure access for `registry.example.com`, follow these steps: bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= ``` -1. Create a [secret variable] `DOCKER_AUTH_CONFIG` with the content of the +1. Create a [variable] `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: ```json @@ -632,7 +632,7 @@ creation. [postgres-hub]: https://hub.docker.com/r/_/postgres/ [mysql-hub]: https://hub.docker.com/r/_/mysql/ [runner-priv-reg]: http://docs.gitlab.com/runner/configuration/advanced-configuration.html#using-a-private-container-registry -[secret variable]: ../variables/README.md#secret-variables +[variable]: ../variables/README.md#variables [entrypoint]: https://docs.docker.com/engine/reference/builder/#entrypoint [cmd]: https://docs.docker.com/engine/reference/builder/#cmd [register]: https://docs.gitlab.com/runner/register/ diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 517e25f00f7..8ea2e0a81dc 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -24,7 +24,7 @@ Environments are like tags for your CI jobs, describing where code gets deployed Deployments are created when [jobs] deploy versions of code to environments, so every environment can have one or more deployments. GitLab keeps track of your deployments, so you always know what is currently being deployed on your -servers. If you have a deployment service such as [Kubernetes][kubernetes-service] +servers. If you have a deployment service such as [Kubernetes][kube] enabled for your project, you can use it to assist with your deployments, and can even access a [web terminal](#web-terminals) for your environment from within GitLab! @@ -114,7 +114,7 @@ Let's now see how that information is exposed within GitLab. ## Viewing the current status of an environment -The environment list under your project's **Pipelines ➔ Environments**, is +The environment list under your project's **Operations > Environments**, is where you can find information of the last deployment status of an environment. Here's how the Environments page looks so far. @@ -167,7 +167,7 @@ that works. You can't control everything, so sometimes things go wrong. When that unfortunate time comes GitLab has you covered. Simply by clicking the **Rollback** button that can be found in the deployments page -(**Pipelines ➔ Environments ➔ `environment name`**) you can relaunch the +(**Operations > Environments > `environment name`**) you can relaunch the job with the commit associated with it. >**Note:** @@ -246,22 +246,14 @@ As the name suggests, it is possible to create environments on the fly by just declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is the basis of [Review apps](review_apps/index.md). ->**Note:** -The `name` and `url` parameters can use most of the defined CI variables, -including predefined, secure variables and `.gitlab-ci.yml` -[`variables`](yaml/README.md#variables). You however cannot use variables -defined under `script` or on the Runner's side. There are other variables that -are unsupported in environment name context: -- `CI_JOB_ID` -- `CI_JOB_TOKEN` -- `CI_BUILD_ID` -- `CI_BUILD_TOKEN` -- `CI_REGISTRY_USER` -- `CI_REGISTRY_PASSWORD` -- `CI_REPOSITORY_URL` -- `CI_ENVIRONMENT_URL` -- `CI_DEPLOY_USER` -- `CI_DEPLOY_PASSWORD` +NOTE: **Note:** +The `name` and `url` parameters can use most of the CI/CD variables, +including [predefined](variables/README.md#predefined-variables-environment-variables), +[project/group ones](variables/README.md#variables) and +[`.gitlab-ci.yml` variables](yaml/README.md#variables). You however cannot use variables +defined under `script` or on the Runner's side. There are also other variables that +are unsupported in the context of `environment:name`. You can read more about +[where variables can be used](variables/where_variables_can_be_used.md). GitLab Runner exposes various [environment variables][variables] when a job runs, and as such, you can use them as environment names. Let's add another job in @@ -601,10 +593,10 @@ version of the app, all without leaving GitLab. >**Note:** Web terminals were added in GitLab 8.15 and are only available to project -masters and owners. +maintainers and owners. If you deploy to your environments with the help of a deployment service (e.g., -the [Kubernetes service][kubernetes-service]), GitLab can open +the [Kubernetes integration][kube]), GitLab can open a terminal session to your environment! This is a very powerful feature that allows you to debug issues without leaving the comfort of your web browser. To enable it, just follow the instructions given in the service integration @@ -670,7 +662,6 @@ Below are some links you may find interesting: [Pipelines]: pipelines.md [jobs]: yaml/README.md#jobs [yaml]: yaml/README.md -[kubernetes-service]: ../user/project/integrations/kubernetes.md [environments]: #environments [deployments]: #deployments [permissions]: ../user/permissions.md @@ -682,5 +673,5 @@ Below are some links you may find interesting: [gitlab-flow]: ../workflow/gitlab_flow.md [gitlab runner]: https://docs.gitlab.com/runner/ [git-strategy]: yaml/README.md#git-strategy -[kube]: ../user/project/integrations/kubernetes.md +[kube]: ../user/project/clusters/index.md [prom]: ../user/project/integrations/prometheus.md diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index de60cd27cd1..aa31e172641 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -19,7 +19,9 @@ There's also a collection of repositories with [example projects](https://gitlab - [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) - **Ruby**: [Test and deploy a Ruby application to Heroku](test-and-deploy-ruby-application-to-heroku.md) - **Python**: [Test and deploy a Python application to Heroku](test-and-deploy-python-application-to-heroku.md) -- **Java**: [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) +- **Java**: + - [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md) + - [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) - **Scala**: [Test a Scala application](test-scala-application.md) - **Clojure**: [Test a Clojure application](test-clojure-application.md) - **Elixir**: diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index d931c9a77f4..9657f52159e 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -58,7 +58,7 @@ The application is ready to use, but you need some additional steps to deploy it 1. Log in to Artifactory with your user's credentials. 1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel. 1. Copy to clipboard the configuration snippet under the **Deploy** paragraph. -1. Change the `url` value in order to have it configurable via secret variables. +1. Change the `url` value in order to have it configurable via variables. 1. Copy the snippet in the `pom.xml` file for your project, just after the `dependencies` section. The snippet should look like this: @@ -98,7 +98,7 @@ parameter in `.gitlab-ci.yml` to use the custom location instead of the default </settings> ``` - Username and password will be replaced by the correct values using secret variables. + Username and password will be replaced by the correct values using variables. ### Configure GitLab CI/CD for `simple-maven-dep` @@ -107,8 +107,8 @@ Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/features/gitlab- GitLab CI/CD uses a file in the root of the repo, named `.gitlab-ci.yml`, to read the definitions for jobs that will be executed by the configured GitLab Runners. You can read more about this file in the [GitLab Documentation](https://docs.gitlab.com/ee/ci/yaml/). -First of all, remember to set up secret variables for your deployment. Navigate to your project's **Settings > CI/CD** page -and add the following secret variables (replace them with your current values, of course): +First of all, remember to set up variables for your deployment. Navigate to your project's **Settings > CI/CD > Variables** page +and add the following ones (replace them with your current values, of course): - **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL) - **MAVEN_REPO_USER**: `gitlab` (your Artifactory username) @@ -156,7 +156,7 @@ by running all Maven phases in a sequential order, therefore, executing `mvn tes Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application. -Deploy to Artifactory is done as defined by the secret variables we have just set up. +Deploy to Artifactory is done as defined by the variables we have just set up. The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published. Done! Now you have all the changes in the GitLab repo, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening. diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md index d1aa783cc9c..cc19e090964 100644 --- a/doc/ci/examples/code_climate.md +++ b/doc/ci/examples/code_climate.md @@ -5,10 +5,10 @@ GitLab CI and Docker. First, you need GitLab Runner with [docker-in-docker executor][dind]. -Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called `codequality`: +Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called `code_quality`: ```yaml -codequality: +code_quality: image: docker:stable variables: DOCKER_DRIVER: overlay2 @@ -23,20 +23,27 @@ codequality: --volume /var/run/docker.sock:/var/run/docker.sock "registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code artifacts: - paths: [codeclimate.json] + paths: [gl-code-quality-report.json] ``` -The above example will create a `codequality` job in your CI/CD pipeline which +The above example will create a `code_quality` job in your CI/CD pipeline which will scan your source code for code quality issues. The report will be saved as an artifact that you can later download and analyze. TIP: **Tip:** Starting with [GitLab Starter][ee] 9.3, this information will be automatically extracted and shown right in the merge request widget. To do -so, the CI/CD job must be named `codequality` and the artifact path must be -`codeclimate.json`. +so, the CI/CD job must be named `code_quality` and the artifact path must be +`gl-code-quality-report.json`. [Learn more on code quality diffs in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html). +CAUTION: **Caution:** +Code Quality was previously using `codeclimate` and `codequality` for job name and +`codeclimate.json` for the artifact name. While these old names +are still maintained they have been deprecated with GitLab 11.0 and may be removed +in next major release, GitLab 12.0. You are advised to update your current `.gitlab-ci.yml` +configuration to reflect that change. + [cli]: https://github.com/codeclimate/codeclimate [dind]: ../docker/using_docker_build.md#use-docker-in-docker-executor [ee]: https://about.gitlab.com/products/ diff --git a/doc/ci/examples/container_scanning.md b/doc/ci/examples/container_scanning.md index eb76521cc02..92ff90507ee 100644 --- a/doc/ci/examples/container_scanning.md +++ b/doc/ci/examples/container_scanning.md @@ -7,10 +7,10 @@ for Vulnerability Static Analysis for containers. All you need is a GitLab Runner with the Docker executor (the shared Runners on GitLab.com will work fine). You can then add a new job to `.gitlab-ci.yml`, -called `sast:container`: +called `container_scanning`: ```yaml -sast:container: +container_scanning: image: docker:stable variables: DOCKER_DRIVER: overlay2 @@ -23,7 +23,7 @@ sast:container: - docker:stable-dind script: - docker run -d --name db arminc/clair-db:latest - - docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.0.1 + - docker run -p 6060:6060 --link db:postgres -d --name clair --restart on-failure arminc/clair-local-scan:v2.0.1 - apk add -U wget ca-certificates - docker pull ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} - wget https://github.com/arminc/clair-scanner/releases/download/v8/clair-scanner_linux_amd64 @@ -34,12 +34,12 @@ sast:container: - retries=0 - echo "Waiting for clair daemon to start" - while( ! wget -T 10 -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; echo -n "." ; if [ $retries -eq 10 ] ; then echo " Timeout, aborting." ; exit 1 ; fi ; retries=$(($retries+1)) ; done - - ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-sast-container-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true + - ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-container-scanning-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true artifacts: - paths: [gl-sast-container-report.json] + paths: [gl-container-scanning-report.json] ``` -The above example will create a `sast:container` job in your CI/CD pipeline, pull +The above example will create a `container_scanning` job in your CI/CD pipeline, pull the image from the [Container Registry](../../user/project/container_registry.md) (whose name is defined from the two `CI_APPLICATION_` variables) and scan it for possible vulnerabilities. The report will be saved as an artifact that you @@ -52,8 +52,15 @@ in our case its named `clair-whitelist.yml`. TIP: **Tip:** Starting with [GitLab Ultimate][ee] 10.4, this information will be automatically extracted and shown right in the merge request widget. To do -so, the CI/CD job must be named `sast:container` and the artifact path must be -`gl-sast-container-report.json`. +so, the CI/CD job must be named `container_scanning` and the artifact path must be +`gl-container-scanning-report.json`. [Learn more on container scanning results shown in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html). +CAUTION: **Caution:** +Container Scanning was previously using `sast:container` for job name and +`gl-sast-container-report.json` for the artifact name. While these old names +are still maintained they have been deprecated with GitLab 11.0 and may be removed +in next major release, GitLab 12.0. You are advised to update your current `.gitlab-ci.yml` +configuration to reflect that change. + [ee]: https://about.gitlab.com/products/ diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_secret_variables.png b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_secret_variables.png Binary files differnew file mode 100644 index 00000000000..5b5d91ec07a --- /dev/null +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/cloud_foundry_secret_variables.png diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png Binary files differnew file mode 100644 index 00000000000..f3761632556 --- /dev/null +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/img/create_from_template.png diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md new file mode 100644 index 00000000000..b88761be56b --- /dev/null +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md @@ -0,0 +1,142 @@ +--- +author: Dylan Griffith +author_gitlab: DylanGriffith +level: intermediary +article_type: tutorial +date: 2018-06-07 +description: "Continuous Deployment of a Spring Boot application to Cloud Foundry with GitLab CI/CD" +--- + +# Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD + +## Introduction + +In this article, we'll demonstrate how to deploy a [Spring +Boot](https://projects.spring.io/spring-boot/) application to [Cloud +Foundry (CF)](https://www.cloudfoundry.org/) with GitLab CI/CD using the [Continuous +Deployment](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-deployment) +method. + +All the code for this project can be found in this [GitLab +repo](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). + +In case you're interested in deploying Spring Boot applications to Kubernetes +using GitLab CI/CD, read through the blog post [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/). + +## Requirements + +_We assume you are familiar with Java, GitLab, Cloud Foundry, and GitLab CI/CD._ + +To follow along with this tutorial you will need the following: + +- An account on [Pivotal Web Services (PWS)](https://run.pivotal.io/) or any + other Cloud Foundry instance +- An account on GitLab + +NOTE: **Note:** +You will need to replace the `api.run.pivotal.io` URL in the all below +commands with the [API +URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html) of your CF +instance if you're not deploying to PWS. + +## Create your project + +To create your Spring Boot application you can use the Spring template in +GitLab when creating a new project: + + + +## Configure the deployment to Cloud Foundry + +To deploy to Cloud Foundry we need to add a `manifest.yml` file. This +is the configuration for the CF CLI we will use to deploy the application. We +will create this in the root directory of our project with the following +content: + +```yaml +--- +applications: +- name: gitlab-hello-world + random-route: true + memory: 1G + path: target/demo-0.0.1-SNAPSHOT.jar +``` + +## Configure GitLab CI/CD to deploy your application + +Now we need to add the the GitLab CI/CD configuration file +([`.gitlab-ci.yml`](../../yaml/README.md)) to our +project's root. This is how GitLab figures out what commands need to be run whenever +code is pushed to our repository. We will add the following `.gitlab-ci.yml` +file to the root directory of the repository, GitLab will detect it +automatically and run the steps defined once we push our code: + +```yaml +image: java:8 + +stages: + - build + - deploy + +build: + stage: build + script: ./mvnw package + artifacts: + paths: + - target/demo-0.0.1-SNAPSHOT.jar + +production: + stage: deploy + script: + - curl --location "https://cli.run.pivotal.io/stable?release=linux64-binary&source=github" | tar zx + - ./cf login -u $CF_USERNAME -p $CF_PASSWORD -a api.run.pivotal.io + - ./cf push + only: + - master +``` + +We've used the `java:8` [docker +image](../../docker/using_docker_images.md) to build +our application as it provides the up-to-date Java 8 JDK on [Docker +Hub](https://hub.docker.com/). We've also added the [`only` +clause](../../yaml/README.md#only-and-except-simplified) +to ensure our deployments only happen when we push to the master branch. + +Now, since the steps defined in `.gitlab-ci.yml` require credentials to login +to CF, you'll need to add your CF credentials as [environment +variables](../../variables/README.md#predefined-variables-environment-variables) +on GitLab CI/CD. To set the environment variables, navigate to your project's +**Settings > CI/CD** and expand **Secret Variables**. Name the variables +`CF_USERNAME` and `CF_PASSWORD` and set them to the correct values. + + + +Once set up, GitLab CI/CD will deploy your app to CF at every push to your +repository's deafult branch. To see the build logs or watch your builds running +live, navigate to **CI/CD > Pipelines**. + +CAUTION: **Caution:** +It is considered best practice for security to create a separate deploy +user for your application and add its credentials to GitLab instead of using +a developer's credentials. + +To start a manual deployment in GitLab go to **CI/CD > Pipelines** then click +on **Run Pipeline**. Once the app is finished deploying it will display the URL +of your application in the logs for the `production` job like: + +```shell +requested state: started +instances: 1/1 +usage: 1G x 1 instances +urls: gitlab-hello-world-undissembling-hotchpot.cfapps.io +last uploaded: Mon Nov 6 10:02:25 UTC 2017 +stack: cflinuxfs2 +buildpack: client-certificate-mapper=1.2.0_RELEASE container-security-provider=1.8.0_RELEASE java-buildpack=v4.5-offline-https://github.com/cloudfoundry/java-buildpack.git#ffeefb9 java-main java-opts jvmkill-agent=1.10.0_RELEASE open-jdk-like-jre=1.8.0_1... + + state since cpu memory disk details +#0 running 2017-11-06 09:03:22 PM 120.4% 291.9M of 1G 137.6M of 1G +``` + +You can then visit your deployed application (for this example, +https://gitlab-hello-world-undissembling-hotchpot.cfapps.io/) and you should +see the "Spring is here!" message. diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 2dcdc2d41ec..bd60d641493 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -111,7 +111,7 @@ We also use two secure variables: ## Storing API keys Secure Variables can added by going to your project's -**Settings ➔ CI / CD ➔ Secret variables**. The variables that are defined +**Settings ➔ CI / CD ➔ Variables**. The variables that are defined in the project settings are sent along with the build script to the Runner. The secure variables are stored out of the repository. Never store secrets in your project's `.gitlab-ci.yml`. It is also important that the secret's value diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 3d21c0cc306..c226b5bfb71 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -406,7 +406,7 @@ and further delves into the principles of GitLab CI/CD than discussed in this ar We need to be able to deploy to AWS with our AWS account credentials, but we certainly don't want to put secrets into source code. Luckily GitLab provides a solution for this -with [Secret Variables](../../../ci/variables/README.md). This can get complicated +with [Variables](../../../ci/variables/README.md). This can get complicated due to [IAM](https://aws.amazon.com/iam/) management. As a best practice, you shouldn't use root security credentials. Proper IAM credential management is beyond the scope of this article, but AWS will remind you that using root credentials is unadvised and against their @@ -428,7 +428,7 @@ fully understand [IAM Best Practices in AWS](http://docs.aws.amazon.com/IAM/late To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on the Shared Runner. The Shared Runner also needs to be able to authenticate with your AWS account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID` -and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the secret variables we +and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the variables we set up in the prior section using the `variables` portion of the `deploy` job. At the end, we add directives to ensure deployment `only` happens on pushes to `master`. This way, every single branch still runs through CI, and only merging (or committing directly) to master will diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 1f9b9d53fc1..39c65399332 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -116,11 +116,11 @@ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_rsa ``` -Now, let's add it to your GitLab project as a [secret variable](../../variables/README.md#secret-variables). -Secret variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. +Now, let's add it to your GitLab project as a [variable](../../variables/README.md#variables). +Variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. They can be added per project by navigating to the project's **Settings** > **CI/CD**. - + To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our remote server as the deployer user without entering its password. diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index b16cbc61d14..4e964af97f5 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -258,7 +258,7 @@ on that specific branch: - trigger **manual actions** on existing pipelines - **retry/cancel** existing jobs (using Web UI or Pipelines API) -**Secret variables** marked as **protected** are accessible only to jobs that +**Variables** marked as **protected** are accessible only to jobs that run on protected branches, avoiding untrusted users to get unintended access to sensitive information like deployment credentials and tokens. diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index fec0ff87326..47e658f610e 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -151,7 +151,8 @@ The next step is to configure a Runner so that it picks the pending jobs. In GitLab, Runners run the jobs that you define in `.gitlab-ci.yml`. A Runner can be a virtual machine, a VPS, a bare-metal machine, a docker container or even a cluster of containers. GitLab and the Runners communicate through an API, -so the only requirement is that the Runner's machine has [Internet] access. +so the only requirement is that the Runner's machine has network access to the +GitLab server. A Runner can be specific to a certain project or serve multiple projects in GitLab. If it serves all projects it's called a _Shared Runner_. @@ -226,4 +227,3 @@ CI with various languages. [enabled]: ../enable_or_disable_ci.md [stages]: ../yaml/README.md#stages [pipeline]: ../pipelines.md -[internet]: https://about.gitlab.com/images/theinternet.png diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 821413900fd..8f1ff190804 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -11,7 +11,7 @@ Ideally, the GitLab Runner should not be installed on the same machine as GitLab Read the [requirements documentation](../../install/requirements.md#gitlab-runner) for more information. -## Shared vs specific Runners +## Shared, specific and group Runners After [installing the Runner][install], you can either register it as shared or specific. You can only register a shared Runner if you have admin access to @@ -32,6 +32,9 @@ are: Runners. For example, if you want to deploy a certain project, you can setup a specific Runner to have the right credentials for this. The [usage of tags](#using-tags) may be useful in this case. Specific Runners process jobs using a [FIFO] queue. +- **Group Runners** are useful when you have multiple projects under one group + and would like all projects to have access to a set of Runners. Group Runners + process jobs using a [FIFO] queue. A Runner that is specific only runs for the specified project(s). A shared Runner can run jobs for every project that has enabled the option **Allow shared Runners** @@ -66,7 +69,7 @@ Runners to disabled. ## Registering a specific Runner -Registering a specific can be done in two ways: +Registering a specific Runner can be done in two ways: 1. Creating a Runner with the project registration token 1. Converting a shared Runner into a specific Runner (one-way, admin only) @@ -79,6 +82,14 @@ visit the project you want to make the Runner work for in GitLab: 1. Go to **Settings > CI/CD** to obtain the token 1. [Register the Runner][register] +## Registering a group Runner + +Creating a group Runner requires Maintainer permissions for the group. To create a +group Runner visit the group you want to make the Runner work for in GitLab: + +1. Go to **Settings > CI/CD** to obtain the token +1. [Register the Runner][register] + ### Making an existing shared Runner specific If you are an admin on your GitLab instance, you can turn any shared Runner into @@ -109,9 +120,9 @@ To lock/unlock a Runner: ## Assigning a Runner to another project -If you are Master on a project where a specific Runner is assigned to, and the +If you are Maintainer on a project where a specific Runner is assigned to, and the Runner is not [locked only to that project](#locking-a-specific-runner-from-being-enabled-for-other-projects), -you can enable the Runner also on any other project where you have Master permissions. +you can enable the Runner also on any other project where you have Maintainer permissions. To enable/disable a Runner in your project: @@ -121,7 +132,7 @@ To enable/disable a Runner in your project: > **Note**: Consider that if you don't lock your specific Runner to a specific project, any -user with Master role in you project can assign your runner to another arbitrary +user with Maintainer role in you project can assign your Runner to another arbitrary project without requiring your authorization, so use it with caution. An admin can enable/disable a specific Runner for projects: diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md index 787c5e462e4..e5fc7a3c85f 100644 --- a/doc/ci/services/docker-services.md +++ b/doc/ci/services/docker-services.md @@ -1,9 +1,3 @@ --- -comments: false +redirect_to: 'README.md' --- - -# GitLab CI Services - -- [Using MySQL](mysql.md) -- [Using PostgreSQL](postgres.md) -- [Using Redis](redis.md) diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 693c8e9ef18..4cb05509e7b 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -25,7 +25,7 @@ with any type of [executor](https://docs.gitlab.com/runner/executors/) ## How it works 1. Create a new SSH key pair locally with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen) -1. Add the private key as a [secret variable](../variables/README.md) to +1. Add the private key as a [variable](../variables/README.md) to your project 1. Run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during job to load the private key. @@ -49,7 +49,7 @@ to access it. This is where an SSH key pair comes in handy. **Do not** add a passphrase to the SSH key, or the `before_script` will\ prompt for it. -1. Create a new [secret variable](../variables/README.md#secret-variables). +1. Create a new [variable](../variables/README.md#variables). As **Key** enter the name `SSH_PRIVATE_KEY` and in the **Value** field paste the content of your _private_ key that you created earlier. @@ -157,7 +157,7 @@ ssh-keyscan example.com ssh-keyscan 1.2.3.4 ``` -Create a new [secret variable](../variables/README.md#secret-variables) with +Create a new [variable](../variables/README.md#variables) with `SSH_KNOWN_HOSTS` as "Key", and as a "Value" add the output of `ssh-keyscan`. NOTE: **Note:** @@ -165,7 +165,7 @@ If you need to connect to multiple servers, all the server host keys need to be collected in the **Value** of the variable, one key per line. TIP: **Tip:** -By using a secret variable instead of `ssh-keyscan` directly inside +By using a variable instead of `ssh-keyscan` directly inside `.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml` if the host domain name changes for some reason. Also, the values are predefined by you, meaning that if the host keys suddenly change, the CI/CD job will fail, diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 47a576fdf5f..c507036aa6a 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -53,7 +53,7 @@ The action is irreversible. it will not trigger a job. - If your project is public, passing the token in plain text is probably not the wisest idea, so you might want to use a - [secret variable](../variables/README.md#secret-variables) for that purpose. + [variable](../variables/README.md#variables) for that purpose. To trigger a job you need to send a `POST` request to GitLab's API endpoint: diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 38a988f4507..a3da6515a19 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -10,17 +10,23 @@ The variables can be overwritten and they take precedence over each other in this order: 1. [Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables) (take precedence over all) -1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) -1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) +1. Project-level [variables](#variables) or [protected variables](#protected-variables) +1. Group-level [variables](#variables) or [protected variables](#protected-variables) 1. YAML-defined [job-level variables](../yaml/README.md#variables) 1. YAML-defined [global variables](../yaml/README.md#variables) 1. [Deployment variables](#deployment-variables) 1. [Predefined variables](#predefined-variables-environment-variables) (are the lowest in the chain) -For example, if you define `API_TOKEN=secure` as a secret variable and +For example, if you define `API_TOKEN=secure` as a project variable and `API_TOKEN=yaml` in your `.gitlab-ci.yml`, the `API_TOKEN` will take the value -`secure` as the secret variables are higher in the chain. +`secure` as the project variables are higher in the chain. + +## Unsupported variables + +There are cases where some variables cannot be used in the context of a +`.gitlab-ci.yml` definition (for example under `script`). Read more +about which variables are [not supported](where_variables_can_be_used.md). ## Predefined variables (Environment variables) @@ -36,13 +42,19 @@ future GitLab releases.** | Variable | GitLab | Runner | Description | |-------------------------------- |--------|--------|-------------| +| **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a job | | **CI** | all | 0.4 | Mark that job is executed in CI environment | | **CI_COMMIT_REF_NAME** | 9.0 | all | The branch or tag name for which project is built | | **CI_COMMIT_REF_SLUG** | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | | **CI_COMMIT_SHA** | 9.0 | all | The commit revision for which project is built | | **CI_COMMIT_TAG** | 9.0 | 0.5 | The commit tag name. Present only when building tags. | +| **CI_COMMIT_MESSAGE** | 10.8 | all | The full commit message. | +| **CI_COMMIT_TITLE** | 10.8 | all | The title of the commit - the full first line of the message | +| **CI_COMMIT_DESCRIPTION** | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. | | **CI_CONFIG_PATH** | 9.4 | 0.5 | The path to CI config file. Defaults to `.gitlab-ci.yml` | | **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | +| **CI_DEPLOY_USER** | 10.8 | all | Authentication username of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.| +| **CI_DEPLOY_PASSWORD** | 10.8 | all | Authentication password of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.| | **CI_DISPOSABLE_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. | | **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this job | | **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. | @@ -52,6 +64,7 @@ future GitLab releases.** | **CI_JOB_NAME** | 9.0 | 0.5 | The name of the job as defined in `.gitlab-ci.yml` | | **CI_JOB_STAGE** | 9.0 | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | | **CI_JOB_TOKEN** | 9.0 | 1.2 | Token used for authenticating with the GitLab Container Registry | +| **CI_JOB_URL** | 11.0 | 0.5 | Job details URL | | **CI_REPOSITORY_URL** | 9.0 | all | The URL to clone the Git repository | | **CI_RUNNER_DESCRIPTION** | 8.10 | 0.5 | The description of the runner as saved in GitLab | | **CI_RUNNER_ID** | 8.10 | 0.5 | The unique id of runner being used | @@ -60,6 +73,7 @@ future GitLab releases.** | **CI_RUNNER_REVISION** | all | 10.6 | GitLab Runner revision that is executing the current job | | **CI_RUNNER_EXECUTABLE_ARCH** | all | 10.6 | The OS/architecture of the GitLab Runner executable (note that this is not necessarily the same as the environment of the executor) | | **CI_PIPELINE_ID** | 8.10 | 0.5 | The unique id of the current pipeline that GitLab CI uses internally | +| **CI_PIPELINE_IID** | 11.0 | all | The unique id of the current pipeline scoped to project | | **CI_PIPELINE_TRIGGERED** | all | all | The flag to indicate that job was [triggered] | | **CI_PIPELINE_SOURCE** | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `trigger`, `schedule`, `api`, and `pipeline`. For pipelines created before GitLab 9.5, this will show as `unknown` | | **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the job is run | @@ -68,6 +82,7 @@ future GitLab releases.** | **CI_PROJECT_NAMESPACE** | 8.10 | 0.5 | The project namespace (username or groupname) that is currently being built | | **CI_PROJECT_PATH** | 8.10 | 0.5 | The namespace with project name | | **CI_PROJECT_PATH_SLUG** | 9.3 | all | `$CI_PROJECT_PATH` lowercased and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | +| **CI_PIPELINE_URL** | 11.0 | 0.5 | Pipeline details URL | | **CI_PROJECT_URL** | 8.10 | 0.5 | The HTTP address to access project | | **CI_PROJECT_VISIBILITY** | 10.3 | all | The project visibility (internal, private, public) | | **CI_REGISTRY** | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry | @@ -79,16 +94,13 @@ future GitLab releases.** | **CI_SERVER_REVISION** | all | all | GitLab revision that is used to schedule jobs | | **CI_SERVER_VERSION** | all | all | GitLab version that is used to schedule jobs | | **CI_SHARED_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. | -| **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a job | | **GET_SOURCES_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to fetch sources running a job | | **GITLAB_CI** | all | all | Mark that job is executed in GitLab CI environment | -| **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the job | | **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the job | +| **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the job | | **GITLAB_USER_LOGIN** | 10.0 | all | The login username of the user who started the job | | **GITLAB_USER_NAME** | 10.0 | all | The real name of the user who started the job | | **RESTORE_CACHE_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to restore the cache running a job | -| **CI_DEPLOY_USER** | 10.8 | all | Authentication username of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.| -| **CI_DEPLOY_PASSWORD** | 10.8 | all | Authentication password of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.| ## 9.0 Renaming @@ -155,49 +167,49 @@ script: - 'eval $LS_CMD' # will execute 'ls -al $TMP_DIR' ``` -## Secret variables +## Variables NOTE: **Note:** -Group-level secret variables were added in GitLab 9.4. +Group-level variables were added in GitLab 9.4. CAUTION: **Important:** -Be aware that secret variables are not masked, and their values can be shown +Be aware that variables are not masked, and their values can be shown in the job logs if explicitly asked to do so. If your project is public or internal, you can set the pipelines private from your [project's Pipelines settings](../../user/project/pipelines/settings.md#visibility-of-pipelines). -Follow the discussion in issue [#13784][ce-13784] for masking the secret variables. +Follow the discussion in issue [#13784][ce-13784] for masking the variables. -GitLab CI allows you to define per-project or per-group secret variables -that are set in the pipeline environment. The secret variables are stored out of +GitLab CI allows you to define per-project or per-group variables +that are set in the pipeline environment. The variables are stored out of the repository (not in `.gitlab-ci.yml`) and are securely passed to GitLab Runner making them available during a pipeline run. It's the recommended method to use for storing things like passwords, SSH keys and credentials. -Project-level secret variables can be added by going to your project's -**Settings > CI/CD**, then finding the section called **Secret variables**. +Project-level variables can be added by going to your project's +**Settings > CI/CD**, then finding the section called **Variables**. -Likewise, group-level secret variables can be added by going to your group's -**Settings > CI/CD**, then finding the section called **Secret variables**. +Likewise, group-level variables can be added by going to your group's +**Settings > CI/CD**, then finding the section called **Variables**. Any variables of [subgroups] will be inherited recursively. - + Once you set them, they will be available for all subsequent pipelines. You can also -[protect your variables](#protected-secret-variables). +[protect your variables](#protected-variables). -### Protected secret variables +### Protected variables >**Notes:** This feature requires GitLab 9.3 or higher. -Secret variables could be protected. Whenever a secret variable is +Variables could be protected. Whenever a variable is protected, it would only be securely passed to pipelines running on the [protected branches] or [protected tags]. The other pipelines would not get any protected variables. Protected variables can be added by going to your project's **Settings > CI/CD**, then finding the section called -**Secret variables**, and check "Protected". +**Variables**, and check "Protected". Once you set them, they will be available for all subsequent pipelines. @@ -212,8 +224,8 @@ are set in the build environment. These variables are only defined for [deployment jobs](../environments.md). Please consult the documentation of the project services that you are using to learn which variables they define. -An example project service that defines deployment variables is -[Kubernetes Service](../../user/project/integrations/kubernetes.md#deployment-variables). +An example project service that defines deployment variables is the +[Kubernetes integration](../../user/project/clusters/index.md#deployment-variables). ## Debug tracing @@ -221,7 +233,7 @@ An example project service that defines deployment variables is CAUTION: **Warning:** Enabling debug tracing can have severe security implications. The -output **will** contain the content of all your secret variables and any other +output **will** contain the content of all your variables and any other secrets! The output **will** be uploaded to the GitLab server and made visible in job traces! @@ -343,6 +355,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ CI_PROJECT_URL=https://example.com/gitlab-examples/ci-debug-trace ++ export CI_PIPELINE_ID=52666 ++ CI_PIPELINE_ID=52666 +++ export CI_PIPELINE_IID=123 +++ CI_PIPELINE_IID=123 ++ export CI_RUNNER_ID=1337 ++ CI_RUNNER_ID=1337 ++ export CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com @@ -407,7 +421,7 @@ job_name: ``` You can also list all environment variables with the `export` command, -but be aware that this will also expose the values of all the secret variables +but be aware that this will also expose the values of all the variables you set, in the job log: ```yaml @@ -430,6 +444,7 @@ export CI_JOB_MANUAL="true" export CI_JOB_TRIGGERED="true" export CI_JOB_TOKEN="abcde-1234ABCD5678ef" export CI_PIPELINE_ID="1000" +export CI_PIPELINE_IID="10" export CI_PROJECT_ID="34" export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" export CI_PROJECT_NAME="gitlab-ce" @@ -459,7 +474,7 @@ It is possible to use variables expressions with only / except policies in `.gitlab-ci.yml`. By using this approach you can limit what jobs are going to be created within a pipeline after pushing a code to GitLab. -This is particularly useful in combination with secret variables and triggered +This is particularly useful in combination with variables and triggered pipeline variables. ```yaml @@ -527,34 +542,17 @@ Below you can find supported syntax reference: `$STAGING` value needs to a string, with length higher than zero. Variable that contains only whitespace characters is not an empty variable. -### Unsupported predefined variables - -Because GitLab evaluates variables before creating jobs, we do not support a -few variables that depend on persistence layer, like `$CI_JOB_ID`. - -Environments (like `production` or `staging`) are also being created based on -what jobs pipeline consists of, thus some environment-specific variables are -not supported as well. - -We do not support variables containing tokens because of security reasons. +1. Pattern matching _(added in 11.0)_ -You can find a full list of unsupported variables below: + > Example: `$VARIABLE =~ /^content.*/` -- `CI_JOB_ID` -- `CI_JOB_TOKEN` -- `CI_BUILD_ID` -- `CI_BUILD_TOKEN` -- `CI_REGISTRY_USER` -- `CI_REGISTRY_PASSWORD` -- `CI_REPOSITORY_URL` -- `CI_ENVIRONMENT_URL` -- `CI_DEPLOY_USER` -- `CI_DEPLOY_PASSWORD` + It is possible perform pattern matching against a variable and regular + expression. Expression like this evaluates to truth if matches are found. -These variables are also not supported in a context of a -[dynamic environment name][dynamic-environments]. + Pattern matching is case-sensitive by default. Use `i` flag modifier, like + `/pattern/i` to make a pattern case-insensitive. -[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI secret variables" +[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI variables" [eep]: https://about.gitlab.com/products/ "Available only in GitLab Premium" [envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md @@ -565,5 +563,4 @@ These variables are also not supported in a context of a [triggers]: ../triggers/README.md#pass-job-variables-to-a-trigger [subgroups]: ../../user/group/subgroups/index.md [builds-policies]: ../yaml/README.md#only-and-except-complex -[dynamic-environments]: ../environments.md#dynamic-environments [gitlab-deploy-token]: ../../user/project/deploy_tokens/index.md#gitlab-deploy-token diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md new file mode 100644 index 00000000000..b2b4a26bdda --- /dev/null +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -0,0 +1,113 @@ +# Where variables can be used + +As it's described in the [CI/CD variables](README.md) docs, you can +define many different variables. Some of them can be used for all GitLab CI/CD +features, but some of them are more or less limited. + +This document describes where and how the different types of variables can be used. + +## Variables usage + +There are basically two places where you can use any defined variables: + +1. On GitLab's side there's `.gitlab-ci.yml` +1. On the Runner's side there's `config.toml` + +### `.gitlab-ci.yml` file + +| Definition | Can be expanded? | Expansion place | Description | +|--------------------------------------|-------------------|-----------------|--------------| +| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>**Supported:** all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>**Not suported:** variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> | +| `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion **doesn't support**: <ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> | +| `variables` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | +| `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | +| `services:[]` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | +| `services:[]:name` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | +| `cache:key` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | +| `artifacts:name` | yes | Runner | The variable expansion is made by GitLab Runner's shell environment | +| `script`, `before_script`, `after_script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) | +| `only:variables:[]`, `except:variables:[]` | no | n/a | The variable must be in the form of `$variable`.<br/>**Not supported:**<ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> | + +### `config.toml` file + +NOTE: **Note:** +You can read more about `config.toml` in the [Runner's docs](https://docs.gitlab.com/runner/configuration/advanced-configuration.html). + +| Definition | Can be expanded? | Description | +|--------------------------------------|------------------|-------------| +| `runners.environment` | yes | The variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | +| `runners.kubernetes.pod_labels` | yes | The Variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | +| `runners.kubernetes.pod_annotations` | yes | The Variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | + +## Expansion mechanisms + +There are three expansion mechanisms: + +- GitLab +- GitLab Runner +- Execution shell environment + +### GitLab internal variable expansion mechanism + +The expanded part needs to be in a form of `$variable`, or `${variable}` or `%variable%`. +Each form is handled in the same way, no matter which OS/shell will finally handle the job, +since the expansion is done in GitLab before any Runner will get the job. + +### GitLab Runner internal variable expansion mechanism + +- **Supported:** project/group variables, `.gitlab-ci.yml` variables, `config.toml` variables, and + variables from triggers and pipeline schedules +- **Not supported:** variables defined inside of scripts (e.g., `export MY_VARIABLE="test"`) + +The Runner uses Go's `os.Expand()` method for variable expansion. It means that it will handle +only variables defined as `$variable` and `${variable}`. What's also important, is that +the expansion is done only once, so nested variables may or may not work, depending on the +ordering of variables definitions. + +### Execution shell environment + +This is an expansion that takes place during the `script` execution. +How it works depends on the used shell (bash/sh/cmd/PowerShell). For example, if the job's +`script` contains a line `echo $MY_VARIABLE-${MY_VARIABLE_2}`, it should be properly handled +by bash/sh (leaving empty strings or some values depending whether the variables were +defined or not), but will not work with Windows' cmd/PowerShell, since these shells +are using a different variables syntax. + +**Supported:** + +- The `script` may use all available variables that are default for the shell (e.g., `$PATH` which + should be present in all bash/sh shells) and all variables defined by GitLab CI/CD (project/group variables, + `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules). +- The `script` may also use all variables defined in the lines before. So, for example, if you define + a variable `export MY_VARIABLE="test"`: + + - in `before_script`, it will work in the following lines of `before_script` and + all lines of the related `script` + - in `script`, it will work in the following lines of `script` + - in `after_script`, it will work in following lines of `after_script` + +## Persisted variables + +NOTE: **Note:** +Some of the persisted variables contain tokens and cannot be used by some definitions +due to security reasons. + +The following variables are known as "persisted": + +- `CI_PIPELINE_ID` +- `CI_JOB_ID` +- `CI_JOB_TOKEN` +- `CI_BUILD_ID` +- `CI_BUILD_TOKEN` +- `CI_REGISTRY_USER` +- `CI_REGISTRY_PASSWORD` +- `CI_REPOSITORY_URL` +- `CI_DEPLOY_USER` +- `CI_DEPLOY_PASSWORD` + +They are: + +- **supported** for all definitions as [described in the table](#gitlab-ci-yml-file) where the "Expansion place" is "Runner" +- **not supported:** + - by the definitions [described in the table](#gitlab-ci-yml-file) where the "Expansion place" is "GitLab" + - in the `only` and `except` [variables expressions](README.md#variables-expressions) diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index fb6d9826d08..8e13ceb9257 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -327,7 +327,7 @@ Refs strategy equals to simplified only/except configuration, whereas kubernetes strategy accepts only `active` keyword. `variables` keyword is used to define variables expressions. In other words -you can use predefined variables / secret variables / project / group or +you can use predefined variables / project / group or environment-scoped variables to define an expression GitLab is going to evaluate in order to decide whether a job should be created or not. @@ -344,10 +344,11 @@ job: kubernetes: active ``` -Example of using variables expressions: +Examples of using variables expressions: ```yaml deploy: + script: cap staging deploy only: refs: - branches @@ -356,6 +357,16 @@ deploy: - $STAGING ``` +Another use case is exluding jobs depending on a commit message _(added in 11.0)_: + +```yaml +end-to-end: + script: rake test:end-to-end + except: + variables: + - $CI_COMMIT_MESSAGE =~ /skip-end-to-end-tests/ +``` + Learn more about variables expressions on [a separate page][variables-expressions]. ## `tags` @@ -738,10 +749,15 @@ cache: rspec: script: test cache: + key: rspec paths: - binaries/ ``` +Note that since cache is shared between jobs, if you're using different +paths for different jobs, you should also set a different **cache:key** +otherwise cache content can be overwritten. + ### `cache:key` > Introduced in GitLab Runner v1.0.0. @@ -756,10 +772,9 @@ or any other way that fits your workflow. This way, you can fine tune caching, allowing you to cache data between different jobs or even different branches. The `cache:key` variable can use any of the -[predefined variables](../variables/README.md), and the default key, if not set, -is `$CI_JOB_NAME-$CI_COMMIT_REF_NAME` which translates as "per-job and -per-branch". It is the default across the project, therefore everything is -shared between pipelines and jobs running on the same branch by default. +[predefined variables](../variables/README.md), and the default key, if not +set, is just literal `default` which means everything is shared between each +pipelines and jobs by default, starting from GitLab 9.0. NOTE: **Note:** The `cache:key` variable cannot contain the `/` character, or the equivalent @@ -779,17 +794,7 @@ If you use **Windows Batch** to run your shell scripts you need to replace ```yaml cache: - key: "%CI_JOB_STAGE%-%CI_COMMIT_REF_SLUG%" - paths: - - binaries/ -``` - -If you use **Windows PowerShell** to run your shell scripts you need to replace -`$` with `$env:`: - -```yaml -cache: - key: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_SLUG" + key: "%CI_COMMIT_REF_SLUG%" paths: - binaries/ ``` @@ -1234,7 +1239,7 @@ Runner itself](../variables/README.md#predefined-variables-environment-variables One example would be `CI_COMMIT_REF_NAME` which has the value of the branch or tag name for which project is built. Apart from the variables you can set in `.gitlab-ci.yml`, there are also the so called -[secret variables](../variables/README.md#secret-variables) +[Variables](../variables/README.md#variables) which can be set in GitLab's UI. [Learn more about variables and their priority.][variables] @@ -1572,7 +1577,7 @@ capitalization, the commit will be created but the pipeline will be skipped. ## Validate the .gitlab-ci.yml Each instance of GitLab CI has an embedded debug tool called Lint, which validates the -content of your `.gitlab-ci.yml` files. You can find the Lint under the page `ci/lint` of your +content of your `.gitlab-ci.yml` files. You can find the Lint under the page `ci/lint` of your project namespace (e.g, `http://gitlab-example.com/gitlab-org/project-123/-/ci/lint`) ## Using reserved keywords diff --git a/doc/container_registry/README.md b/doc/container_registry/README.md index fe3e4681ba7..5d2f5edcb18 100644 --- a/doc/container_registry/README.md +++ b/doc/container_registry/README.md @@ -1 +1 @@ -This document was moved in [user/project/container_registry](../user/project/container_registry.md). +This document was moved to [another location](../user/project/container_registry.md). diff --git a/doc/customization/favicon.md b/doc/customization/favicon.md new file mode 100644 index 00000000000..45a18159b5e --- /dev/null +++ b/doc/customization/favicon.md @@ -0,0 +1,16 @@ +# Changing the favicon + +> [Introduced][ce-14497] in GitLab 11.0. + +[ce-14497]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14497 + +Navigate to the **Admin** area and go to the **Appearance** page. + +Upload the custom favicon (**Favicon**) in the section **Favicon**. + + + +After saving the page, the new favicon will be shown in the browser. The main +favicon as well as the CI status icons will show the custom icon: + + diff --git a/doc/customization/favicon/appearance.png b/doc/customization/favicon/appearance.png Binary files differnew file mode 100644 index 00000000000..6c41a05fc1f --- /dev/null +++ b/doc/customization/favicon/appearance.png diff --git a/doc/customization/favicon/custom_favicon.png b/doc/customization/favicon/custom_favicon.png Binary files differnew file mode 100644 index 00000000000..fa1b8827a36 --- /dev/null +++ b/doc/customization/favicon/custom_favicon.png diff --git a/doc/customization/issue_closing.md b/doc/customization/issue_closing.md index d14ba6ad522..680c51e7524 100644 --- a/doc/customization/issue_closing.md +++ b/doc/customization/issue_closing.md @@ -1,8 +1,3 @@ --- -comments: false +redirect_to: '../user/project/issues/automatic_issue_closing.md' --- - -This document was split into: - -- [administration/issue_closing_pattern.md](../administration/issue_closing_pattern.md). -- [user/project/issues/automatic_issue_closing](../user/project/issues/automatic_issue_closing.md). diff --git a/doc/development/README.md b/doc/development/README.md index 3c77e99b8cf..5d6fed5bc72 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -1,5 +1,6 @@ --- comments: false +description: 'Learn how to contribute to GitLab.' --- # GitLab development guides @@ -18,6 +19,7 @@ comments: false - [Code review guidelines](code_review.md) for reviewing code and having code reviewed. - [Automatic CE->EE merge](automatic_ce_ee_merge.md) - [Guidelines for implementing Enterprise Edition features](ee_features.md) +- [Security process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) ## UX and frontend guides @@ -30,6 +32,8 @@ comments: false - [GitLab utilities](utilities.md) - [API styleguide](api_styleguide.md) Use this styleguide if you are contributing to the API. +- [GraphQL API styleguide](api_graphql_styleguide.md) Use this + styleguide if you are contribution to the [GraphQL API](../api/graphql/index.md) - [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers - [Working with Gitaly](gitaly.md) - [Manage feature flags](feature_flags.md) @@ -84,8 +88,8 @@ comments: false ## Documentation guides -- [Writing documentation](writing_documentation.md) -- [Documentation styleguide](doc_styleguide.md) +- [Writing documentation](documentation/index.md) +- [Documentation styleguide](documentation/styleguide.md) - [Markdown](../user/markdown.md) ## Internationalization (i18n) guides diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md new file mode 100644 index 00000000000..f74e4f0bd7e --- /dev/null +++ b/doc/development/api_graphql_styleguide.md @@ -0,0 +1,81 @@ +# GraphQL API + +## Authentication + +Authentication happens through the `GraphqlController`, right now this +uses the same authentication as the Rails application. So the session +can be shared. + +It is also possible to add a `private_token` to the querystring, or +add a `HTTP_PRIVATE_TOKEN` header. + +### Authorization + +Fields can be authorized using the same abilities used in the Rails +app. This can be done using the `authorize` helper: + +```ruby +module Types + class QueryType < BaseObject + graphql_name 'Query' + + field :project, Types::ProjectType, null: true, resolver: Resolvers::ProjectResolver do + authorize :read_project + end + end +``` + +The object found by the resolve call is used for authorization. + +This works for authorizing a single record, for authorizing +collections, we should only load what the currently authenticated user +is allowed to view. Preferably we use our existing finders for that. + +## Types + +When exposing a model through the GraphQL API, we do so by creating a +new type in `app/graphql/types`. + +When exposing properties in a type, make sure to keep the logic inside +the definition as minimal as possible. Instead, consider moving any +logic into a presenter: + +```ruby +class Types::MergeRequestType < BaseObject + present_using MergeRequestPresenter + + name 'MergeRequest' +end +``` + +An existing presenter could be used, but it is also possible to create +a new presenter specifically for GraphQL. + +The presenter is initialized using the object resolved by a field, and +the context. + +## Resolvers + +To find objects to display in a field, we can add resolvers to +`app/graphql/resolvers`. + +Arguments can be defined within the resolver, those arguments will be +made available to the fields using the resolver. + +We already have a `FullPathLoader` that can be included in other +resolvers to quickly find Projects and Namespaces which will have a +lot of dependant objects. + +To limit the amount of queries performed, we can use `BatchLoader`. + +## Testing + +_full stack_ tests for a graphql query or mutation live in +`spec/requests/api/graphql`. + +When adding a query, the `a working graphql query` shared example can +be used to test if the query renders valid results. + +Using the `GraphqlHelpers#all_graphql_fields_for`-helper, a query +including all available fields can be constructed. This makes it easy +to add a test rendering all possible fields for a query. diff --git a/doc/development/changelog.md b/doc/development/changelog.md index a9fa5ae834f..9e0c81b3d60 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -45,6 +45,8 @@ the `author` field. GitLab team members **should not**. a changelog entry regardless of these guidelines if the contributor wants one. Example: "Fixed a typo on the search results page. (Jane Smith)" - Performance improvements **should** have a changelog entry. +- Any change that introduces a database migration **must** have a + changelog entry. ## Writing good changelog entries diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 7165b8062a7..23c80799235 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -22,13 +22,17 @@ There are a few rules to get your merge request accepted: 1. If your merge request includes UX, frontend and backend changes [^1], it must be **approved by a [UX team member, a frontend and a backend maintainer][team]**. 1. If your merge request includes a new dependency or a filesystem change, it must - be **approved by a [Build team member][team]**. See [how to work with the Build team][build handbook] for more details. + be *approved by a [Distribution team member][team]*. See how to work with the [Distribution team for more details.](https://about.gitlab.com/handbook/engineering/dev-backend/distribution/) 1. To lower the amount of merge requests maintainers need to review, you can ask or assign any [reviewers][projects] for a first review. 1. If you need some guidance (e.g. it's your first merge request), feel free to ask one of the [Merge request coaches][team]. 1. The reviewer will assign the merge request to a maintainer once the reviewer is satisfied with the state of the merge request. +1. Keep in mind that maintainers are also going to perform a final code review. + The ideal scenario is that the reviewer has already addressed any concerns + the maintainer would have found, and the maintainer only has to perform the + merge, but be prepared for further review comments. For more guidance, see [CONTRIBUTING.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md). @@ -207,3 +211,4 @@ Largely based on the [thoughtbot code review guide]. [projects]: https://about.gitlab.com/handbook/engineering/projects/ [team]: https://about.gitlab.com/team/ [build handbook]: https://about.gitlab.com/handbook/build/handbook/build#how-to-work-with-build +[^1]: Please note that specs other than JavaScript specs are considered backend code. diff --git a/doc/development/database_debugging.md b/doc/development/database_debugging.md index 32f392f1303..9c31265e417 100644 --- a/doc/development/database_debugging.md +++ b/doc/development/database_debugging.md @@ -11,7 +11,7 @@ Available `RAILS_ENV` - `production` (generally not for your main GDK db, but you may need this for e.g. omnibus) - `development` (this is your main GDK db) - - `test` (used for tests like rspec and spinach) + - `test` (used for tests like rspec) ## Nuke everything and start over diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 5da015ca557..7d84f8ca86a 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -1,427 +1,3 @@ -# Documentation style guidelines - -The documentation style guide defines the markup structure used in -GitLab documentation. Check the -[documentation guidelines](writing_documentation.md) for general development instructions. - -Check the GitLab handbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). - -## Text - -- Split up long lines (wrap text), this makes it much easier to review and edit. Only - double line breaks are shown as a full line break in [GitLab markdown][gfm]. - 80-100 characters is a good line length -- Make sure that the documentation is added in the correct - [directory](writing_documentation.md#documentation-directory-structure) and that - there's a link to it somewhere useful -- Do not duplicate information -- Be brief and clear -- Unless there's a logical reason not to, add documents in alphabetical order -- Write in US English -- Use [single spaces][] instead of double spaces -- Jump a line between different markups (e.g., after every paragraph, header, list, etc) -- Capitalize "G" and "L" in GitLab -- Capitalize feature, products, and methods names. E.g.: GitLab Runner, Geo, -Issue Boards, Git, Prometheus, Continuous Integration. - -## Formatting - -- Use dashes (`-`) for unordered lists instead of asterisks (`*`) -- Use the number one (`1`) for ordered lists -- Use underscores (`_`) to mark a word or text in italics -- Use double asterisks (`**`) to mark a word or text in bold -- When using lists, prefer not to end each item with a period. You can use - them if there are multiple sentences, just keep the last sentence without - a period - -## Headings - -- Add only one H1 title in each document, by adding `#` at the beginning of - it (when using markdown). For subheadings, use `##`, `###` and so on -- Avoid putting numbers in headings. Numbers shift, hence documentation anchor - links shift too, which eventually leads to dead links. If you think it is - compelling to add numbers in headings, make sure to at least discuss it with - someone in the Merge Request -- [Avoid using symbols and special chars](https://gitlab.com/gitlab-com/gitlab-docs/issues/84) - in headers. Whenever possible, they should be plain and short text. -- Avoid adding things that show ephemeral statuses. For example, if a feature is - considered beta or experimental, put this info in a note, not in the heading. -- When introducing a new document, be careful for the headings to be - grammatically and syntactically correct. Mention one or all - of the following GitLab members for a review: `@axil` or `@marcia`. - This is to ensure that no document with wrong heading is going - live without an audit, thus preventing dead links and redirection issues when - corrected -- Leave exactly one newline after a heading - -## Links - -- Use the regular inline link markdown markup `[Text](https://example.com)`. - It's easier to read, review, and maintain. -- If there's a link that repeats several times through the same document, - you can use `[Text][identifier]` and at the bottom of the section or the - document add: `[identifier]: https://example.com`, in which case, we do - encourage you to also add an alternative text: `[identifier]: https://example.com "Alternative text"` that appears when hovering your mouse on a link. -- To link to internal documentation, use relative links, not full URLs. Use `../` to - navigate tp high-level directories, and always add the file name `file.md` at the - end of the link with the `.md` extension, not `.html`. - Example: instead of `[text](../../merge_requests/)`, use - `[text](../../merge_requests/index.md)` or, `[text](../../ci/README.md)`, or, - for anchor links, `[text](../../ci/README.md#examples)`. - Using the markdown extension is necessary for the [`/help`](writing_documentation.md#gitlab-help) - section of GitLab. -- To link from CE to EE-only documentation, use the EE-only doc full URL. -- Use [meaningful anchor texts](https://www.futurehosting.com/blog/links-should-have-meaningful-anchor-text-heres-why/). - E.g., instead of writing something like `Read more about GitLab Issue Boards [here](LINK)`, - write `Read more about [GitLab Issue Boards](LINK)`. - -## Images - -- Place images in a separate directory named `img/` in the same directory where - the `.md` document that you're working on is located. Always prepend their - names with the name of the document that they will be included in. For - example, if there is a document called `twitter.md`, then a valid image name - could be `twitter_login_screen.png`. [**Exception**: images for - [articles](writing_documentation.md#technical-articles) should be - put in a directory called `img` underneath `/articles/article_title/img/`, therefore, - there's no need to prepend the document name to their filenames.] -- Images should have a specific, non-generic name that will differentiate them. -- Keep all file names in lower case. -- Consider using PNG images instead of JPEG. -- Compress all images with <https://tinypng.com/> or similar tool. -- Compress gifs with <https://ezgif.com/optimize> or similar tool. -- Images should be used (only when necessary) to _illustrate_ the description -of a process, not to _replace_ it. - -Inside the document: - -- The Markdown way of using an image inside a document is: - `` -- Always use a proper description for what the image is about. That way, when a - browser fails to show the image, this text will be used as an alternative - description -- If there are consecutive images with little text between them, always add - three dashes (`---`) between the image and the text to create a horizontal - line for better clarity -- If a heading is placed right after an image, always add three dashes (`---`) - between the image and the heading - -## Notes - -- Notes should be quoted with the word `Note:` being bold. Use this form: - - ```md - >**Note:** - This is something to note. - ``` - - which renders to: - - >**Note:** - This is something to note. - - If the note spans across multiple lines it's OK to split the line. - -## Specific sections and terms - -To mention and/or reference specific terms in GitLab, please follow the styles -below. - -### GitLab versions and tiers - -- Every piece of documentation that comes with a new feature should declare the - GitLab version that feature got introduced. Right below the heading add a - note: - - ```md - > Introduced in GitLab 8.3. - ``` - -- If possible every feature should have a link to the MR, issue, or epic that introduced it. - The above note would be then transformed to: - - ```md - > [Introduced][ce-1242] in GitLab 8.3. - ``` - - , where the [link identifier](#links) is named after the repository (CE) and - the MR number. - -- If the feature is only available in GitLab Enterprise Edition, don't forget to mention - the [paid tier](https://about.gitlab.com/handbook/marketing/product-marketing/#tiers) - the feature is available in: - - ```md - > [Introduced][ee-1234] in [GitLab Starter](https://about.gitlab.com/products/) 8.3. - ``` - - Otherwise, leave this mention out. - -### Product badges - -When a feature is available in EE-only tiers, add the corresponding tier according to the -feature availability: - -- For GitLab Starter and GitLab.com Bronze: `**[STARTER]**` -- For GitLab Premium and GitLab.com Silver: `**[PREMIUM]**` -- For GitLab Ultimate and GitLab.com Gold: `**[ULTIMATE]**` -- For GitLab Core and GitLab.com Free: `**[CORE]**` - -To exclude GitLab.com tiers (when the feature is not available in GitLab.com), add the -keyword "only": - -- For GitLab Starter: `**[STARTER ONLY]**` -- For GitLab Premium: `**[PREMIUM ONLY]**` -- For GitLab Ultimate: `**[ULTIMATE ONLY]**` -- For GitLab Core: `**[CORE ONLY]**` - -The tier should be ideally added to headers, so that the full badge will be displayed. -But it can be also mentioned from paragraphs, list items, and table cells. For these cases, -the tier mention will be represented by an orange question mark. -E.g., `**[STARTER]**` renders **[STARTER]**, `**[STARTER ONLY]**` renders **[STARTER ONLY]**. - -The absence of tiers' mentions mean that the feature is available in GitLab Core, -GitLab.com Free, and higher tiers. - -#### How it works - -Introduced by [!244](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/244), -the special markup `**[STARTER]**` will generate a `span` element to trigger the -badges and tooltips (`<span class="badge-trigger starter">`). When the keyword -"only" is added, the corresponding GitLab.com badge will not be displayed. - -### GitLab Restart - -There are many cases that a restart/reconfigure of GitLab is required. To -avoid duplication, link to the special document that can be found in -[`doc/administration/restart_gitlab.md`][doc-restart]. Usually the text will -read like: - - ``` - Save the file and [reconfigure GitLab](../administration/restart_gitlab.md) - for the changes to take effect. - ``` - -If the document you are editing resides in a place other than the GitLab CE/EE -`doc/` directory, instead of the relative link, use the full path: -`http://docs.gitlab.com/ce/administration/restart_gitlab.html`. -Replace `reconfigure` with `restart` where appropriate. - -### Installation guide - -**Ruby:** -In [step 2 of the installation guide](../install/installation.md#2-ruby), -we install Ruby from source. Whenever there is a new version that needs to -be updated, remember to change it throughout the codeblock and also replace -the sha256sum (it can be found in the [downloads page][ruby-dl] of the Ruby -website). - -[ruby-dl]: https://www.ruby-lang.org/en/downloads/ "Ruby download website" - -### Configuration documentation for source and Omnibus installations - -GitLab currently officially supports two installation methods: installations -from source and Omnibus packages installations. - -Whenever there is a setting that is configurable for both installation methods, -prefer to document it in the CE docs to avoid duplication. - -Configuration settings include: - -- settings that touch configuration files in `config/` -- NGINX settings and settings in `lib/support/` in general - -When there is a list of steps to perform, usually that entails editing the -configuration file and reconfiguring/restarting GitLab. In such case, follow -the style below as a guide: - -```md -**For Omnibus installations** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - external_url "https://gitlab.example.com" - ``` - -1. Save the file and [reconfigure] GitLab for the changes to take effect. - --- - -**For installations from source** - -1. Edit `config/gitlab.yml`: - - ```yaml - gitlab: - host: "gitlab.example.com" - ``` - -1. Save the file and [restart] GitLab for the changes to take effect. - - -[reconfigure]: path/to/administration/restart_gitlab.md#omnibus-gitlab-reconfigure -[restart]: path/to/administration/restart_gitlab.md#installations-from-source -``` - -In this case: - -- before each step list the installation method is declared in bold -- three dashes (`---`) are used to create a horizontal line and separate the - two methods -- the code blocks are indented one or more spaces under the list item to render - correctly -- different highlighting languages are used for each config in the code block -- the [references](#references) guide is used for reconfigure/restart - -### Fake tokens - -There may be times where a token is needed to demonstrate an API call using -cURL or a secret variable used in CI. It is strongly advised not to use real -tokens in documentation even if the probability of a token being exploited is -low. - -You can use the following fake tokens as examples. - -| **Token type** | **Token value** | -| --------------------- | --------------------------------- | -| Private user token | `9koXpg98eAheJpvBs5tK` | -| Personal access token | `n671WNGecHugsdEDPsyo` | -| Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` | -| Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` | -| Secret CI variable | `Li8j-mLUVA3eZYjPfd_H` | -| Specific Runner token | `yrnZW46BrtBFqM7xDzE7dddd` | -| Shared Runner token | `6Vk7ZsosqQyfreAxXTZr` | -| Trigger token | `be20d8dcc028677c931e04f3871a9b` | -| Webhook secret token | `6XhDroRcYPM5by_h-HLY` | -| Health check token | `Tu7BgjR9qeZTEyRzGG2P` | -| Request profile token | `7VgpS4Ax5utVD2esNstz` | - -### API - -Here is a list of must-have items. Use them in the exact order that appears -on this document. Further explanation is given below. - -- Every method must have the REST API request. For example: - - ``` - GET /projects/:id/repository/branches - ``` - -- Every method must have a detailed - [description of the parameters](#method-description). -- Every method must have a cURL example. -- Every method must have a response body (in JSON format). - -#### Method description - -Use the following table headers to describe the methods. Attributes should -always be in code blocks using backticks (``` ` ```). - -``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -``` - -Rendered example: - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `user` | string | yes | The GitLab username | - -#### cURL commands - -- Use `https://gitlab.example.com/api/v4/` as an endpoint. -- Wherever needed use this personal access token: `9koXpg98eAheJpvBs5tK`. -- Always put the request first. `GET` is the default so you don't have to - include it. -- Use double quotes to the URL when it includes additional parameters. -- Prefer to use examples using the personal access token and don't pass data of - username and password. - -| Methods | Description | -| ------- | ----------- | -| `-H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK"` | Use this method as is, whenever authentication needed | -| `-X POST` | Use this method when creating new objects | -| `-X PUT` | Use this method when updating existing objects | -| `-X DELETE` | Use this method when removing existing objects | - -#### cURL Examples - -Below is a set of [cURL][] examples that you can use in the API documentation. - -##### Simple cURL command - -Get the details of a group: - -```bash -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/gitlab-org -``` - -##### cURL example with parameters passed in the URL - -Create a new project under the authenticated user's namespace: - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects?name=foo" -``` - -##### Post data using cURL's --data - -Instead of using `-X POST` and appending the parameters to the URI, you can use -cURL's `--data` option. The example below will create a new project `foo` under -the authenticated user's namespace. - -```bash -curl --data "name=foo" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects" -``` - -##### Post data using JSON content - -> **Note:** In this example we create a new group. Watch carefully the single -and double quotes. - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v4/groups -``` - -##### Post data using form-data - -Instead of using JSON or urlencode you can use multipart/form-data which -properly handles data encoding: - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "title=ssh-key" --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." https://gitlab.example.com/api/v4/users/25/keys -``` - -The above example is run by and administrator and will add an SSH public key -titled ssh-key to user's account which has an id of 25. - -##### Escape special characters - -Spaces or slashes (`/`) may sometimes result to errors, thus it is recommended -to escape them when possible. In the example below we create a new issue which -contains spaces in its title. Observe how spaces are escaped using the `%20` -ASCII code. - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20Dude" -``` - -Use `%2F` for slashes (`/`). - -##### Pass arrays to API calls - -The GitLab API sometimes accepts arrays of strings or integers. For example, to -restrict the sign-up e-mail domains of a GitLab instance to `*.example.com` and -`example.net`, you would do something like this: - -```bash -curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "domain_whitelist[]=*.example.com" --data "domain_whitelist[]=example.net" https://gitlab.example.com/api/v4/application/settings -``` - -[cURL]: http://curl.haxx.se/ "cURL website" -[single spaces]: http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html -[gfm]: http://docs.gitlab.com/ce/user/markdown.html#newlines "GitLab flavored markdown documentation" -[ce-1242]: https://gitlab.com/gitlab-org/gitlab-ce/issues/1242 -[doc-restart]: ../administration/restart_gitlab.md "GitLab restart documentation" +redirect_to: 'documentation/styleguide.md' +--- diff --git a/doc/development/img/manual_build_docs.png b/doc/development/documentation/img/manual_build_docs.png Binary files differindex 615facabb5f..615facabb5f 100644 --- a/doc/development/img/manual_build_docs.png +++ b/doc/development/documentation/img/manual_build_docs.png diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md new file mode 100644 index 00000000000..48e1685082a --- /dev/null +++ b/doc/development/documentation/index.md @@ -0,0 +1,558 @@ +--- +description: Learn how to contribute to GitLab Documentation. +--- + +# GitLab Documentation guidelines + + - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. + - **[Technical Articles](#technical-articles)**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). + - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). + +## Contributing to docs + +Whenever a feature is changed, updated, introduced, or deprecated, the merge +request introducing these changes must be accompanied by the documentation +(either updating existing ones or creating new ones). This is also valid when +changes are introduced to the UI. + +The one responsible for writing the first piece of documentation is the developer who +wrote the code. It's the job of the Product Manager to ensure all features are +shipped with its docs, whether is a small or big change. At the pace GitLab evolves, +this is the only way to keep the docs up-to-date. If you have any questions about it, +ask a Technical Writer. Otherwise, when your content is ready, assign one of +them to review it for you. + +We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything +is documented. + +Whenever you submit a merge request for the documentation, use the documentation MR description template. + +Please check the [documentation workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) before getting started. + +## Documentation structure + +- Overview and use cases: what it is, why it is necessary, why one would use it +- Requirements: what do we need to get started +- Tutorial: how to set it up, how to use it + +Always link a new document from its topic-related index, otherwise, it will +not be included it in the documentation site search. + +_Note: to be extended._ + +### Feature overview and use cases + +Every major feature (regardless if present in GitLab Community or Enterprise editions) +should present, at the beginning of the document, two main sections: **overview** and +**use cases**. Every GitLab EE-only feature should also contain these sections. + +**Overview**: as the name suggests, the goal here is to provide an overview of the feature. +Describe what is it, what it does, why it is important/cool/nice-to-have, +what problem it solves, and what you can do with this feature that you couldn't +do before. + +**Use cases**: provide at least two, ideally three, use cases for every major feature. +You should answer this question: what can you do with this feature/change? Use cases +are examples of how this feature or change can be used in real life. + +Examples: +- CE and EE: [Issues](../user/project/issues/index.md#use-cases) +- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) +- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) +- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) + +Note that if you don't have anything to add between the doc title (`<h1>`) and +the header `## Overview`, you can omit the header, but keep the content of the +overview there. + +> **Overview** and **use cases** are required to **every** Enterprise Edition feature, +and for every **major** feature present in Community Edition. + +## Markdown and styles + +Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. + +All the docs follow the [documentation style guidelines](styleguide.md). + +## Documentation directory structure + +The documentation is structured based on the GitLab UI structure itself, +separated by [`user`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/user), +[`administrator`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/administration), and [`contributor`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/development). + +In order to have a [solid site structure](https://searchengineland.com/seo-benefits-developing-solid-site-structure-277456) for our documentation, +all docs should be linked. Every new document should be cross-linked to its related documentation, and linked from its topic-related index, when existent. + +The directories `/workflow/`, `/gitlab-basics/`, `/university/`, and `/articles/` have +been deprecated and the majority their docs have been moved to their correct location +in small iterations. Please don't create new docs in these folders. + +### Location and naming documents + +The documentation hierarchy can be vastly improved by providing a better layout +and organization of directories. + +Having a structured document layout, we will be able to have meaningful URLs +like `docs.gitlab.com/user/project/merge_requests/index.html`. With this pattern, +you can immediately tell that you are navigating a user related documentation +and is about the project and its merge requests. + +Do not create summaries of similar types of content (e.g. an index of all articles, videos, etc.), +rather organize content by its subject (e.g. everything related to CI goes together) +and cross-link between any related content. + +The table below shows what kind of documentation goes where. + +| Directory | What belongs here | +| --------- | -------------- | +| `doc/user/` | User related documentation. Anything that can be done within the GitLab UI goes here including `/admin`. | +| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. The admin settings that can be accessed via GitLab's interface go under `doc/user/admin_area/`. | +| `doc/api/` | API related documentation. | +| `doc/development/` | Documentation related to the development of GitLab. Any styleguides should go here. | +| `doc/legal/` | Legal documents about contributing to GitLab. | +| `doc/install/`| Probably the most visited directory, since `installation.md` is there. Ideally this should go under `doc/administration/`, but it's best to leave it as-is in order to avoid confusion (still debated though). | +| `doc/update/` | Same with `doc/install/`. Should be under `administration/`, but this is a well known location, better leave as-is, at least for now. | +| `doc/topics/` | Indexes per Topic (`doc/topics/topic-name/index.md`): all resources for that topic (user and admin documentation, articles, and third-party docs) | + +--- + +**General rules:** + +1. The correct naming and location of a new document, is a combination + of the relative URL of the document in question and the GitLab Map design + that is used for UX purposes ([source][graffle], [image][gitlab-map]). +1. When creating a new document and it has more than one word in its name, + make sure to use underscores instead of spaces or dashes (`-`). For example, + a proper naming would be `import_projects_from_github.md`. The same rule + applies to images. +1. Start a new directory with an `index.md` file. +1. There are four main directories, `user`, `administration`, `api` and `development`. +1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, + `profile/`, `dashboard/` and `admin_area/`. + 1. `doc/user/project/` should contain all project related documentation. + 1. `doc/user/group/` should contain all group related documentation. + 1. `doc/user/profile/` should contain all profile related documentation. + Every page you would navigate under `/profile` should have its own document, + i.e. `account.md`, `applications.md`, `emails.md`, etc. + 1. `doc/user/dashboard/` should contain all dashboard related documentation. + 1. `doc/user/admin_area/` should contain all admin related documentation + describing what can be achieved by accessing GitLab's admin interface + (_not to be confused with `doc/administration` where server access is + required_). + 1. Every category under `/admin/application_settings` should have its + own document located at `doc/user/admin_area/settings/`. For example, + the **Visibility and Access Controls** category should have a document + located at `doc/user/admin_area/settings/visibility_and_access_controls.md`. +1. The `doc/topics/` directory holds topic-related technical content. Create + `doc/topics/topic-name/subtopic-name/index.md` when subtopics become necessary. + General user- and admin- related documentation, should be placed accordingly. + +If you are unsure where a document should live, you can ping `@axil` or `@marcia` in your +merge request. + +### Changing document location + +Changing a document's location is not to be taken lightly. Remember that the +documentation is available to all installations under `help/` and not only to +GitLab.com or http://docs.gitlab.com. Make sure this is discussed with the +Documentation team beforehand. + +If you indeed need to change a document's location, do NOT remove the old +document, but rather replace all of its contents with a new line: + +``` +This document was moved to [another location](path/to/new_doc.md). +``` + +where `path/to/new_doc.md` is the relative path to the root directory `doc/`. + +--- + +For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to +`doc/administration/lfs.md`, then the steps would be: + +1. Copy `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md` +1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with: + + ``` + This document was moved to [another location](../../administration/lfs.md). + ``` + +1. Find and replace any occurrences of the old location with the new one. + A quick way to find them is to use `git grep`. First go to the root directory + where you cloned the `gitlab-ce` repository and then do: + + ``` + git grep -n "workflow/lfs/lfs_administration" + git grep -n "lfs/lfs_administration" + ``` + +NOTE: **Note:** +If the document being moved has any Disqus comments on it, there are extra steps +to follow documented just [below](#redirections-for-pages-with-disqus-comments). + +Things to note: + +- Since we also use inline documentation, except for the documentation itself, + the document might also be referenced in the views of GitLab (`app/`) which will + render when visiting `/help`, and sometimes in the testing suite (`spec/`). +- The above `git grep` command will search recursively in the directory you run + it in for `workflow/lfs/lfs_administration` and `lfs/lfs_administration` + and will print the file and the line where this file is mentioned. + You may ask why the two greps. Since we use relative paths to link to + documentation, sometimes it might be useful to search a path deeper. +- The `*.md` extension is not used when a document is linked to GitLab's + built-in help page, that's why we omit it in `git grep`. +- Use the checklist on the documentation MR description template. + +#### Alternative redirection method + +Alternatively to the method described above, you can simply replace the content +of the old file with a frontmatter containing a redirect link: + +```yaml +--- +redirect_to: '../path/to/file/README.md' +--- +``` + +It supports both full and relative URLs, e.g. `https://docs.gitlab.com/ee/path/to/file.html`, `../path/to/file.html`, `path/to/file.md`. Note that any `*.md` paths will be compiled to `*.html`. + +### Redirections for pages with Disqus comments + +If the documentation page being relocated already has any Disqus comments, +we need to preserve the Disqus thread. + +Disqus uses an identifier per page, and for docs.gitlab.com, the page identifier +is configured to be the page URL. Therefore, when we change the document location, +we need to preserve the old URL as the same Disqus identifier. + +To do that, add to the frontmatter the variable `redirect_from`, +using the old URL as value. For example, let's say I moved the document +available under `https://docs.gitlab.com/my-old-location/README.html` to a new location, +`https://docs.gitlab.com/my-new-location/index.html`. + +Into the **new document** frontmatter add the following: + +```yaml +--- +redirect_from: 'https://docs.gitlab.com/my-old-location/README.html' +--- +``` + +Note: it is necessary to include the file name in the `redirect_from` URL, +even if it's `index.html` or `README.html`. + +## Testing + +We treat documentation as code, thus have implemented some testing. +Currently, the following tests are in place: + +1. `docs lint`: Check that all internal (relative) links work correctly and + that all cURL examples in API docs use the full switches. It's recommended + to [check locally](#previewing-locally) before pushing to GitLab by executing the command + `bundle exec nanoc check internal_links` on your local + [`gitlab-docs`](https://gitlab.com/gitlab-com/gitlab-docs) directory. +1. [`ee_compat_check`](../automatic_ce_ee_merge.md#avoiding-ce-gt-ee-merge-conflicts-beforehand) (runs on CE only): + When you submit a merge request to GitLab Community Edition (CE), + there is this additional job that runs against Enterprise Edition (EE) + and checks if your changes can apply cleanly to the EE codebase. + If that job fails, read the instructions in the job log for what to do next. + As CE is merged into EE once a day, it's important to avoid merge conflicts. + Submitting an EE-equivalent merge request cherry-picking all commits from CE to EE is + essential to avoid them. + +## Branch naming + +If your contribution contains **only** documentation changes, you can speed up +the CI process by following some branch naming conventions. You have three +choices: + +| Branch name | Valid example | +| ----------- | ------------- | +| Starting with `docs/` | `docs/update-api-issues` | +| Starting with `docs-` | `docs-update-api-issues` | +| Ending in `-docs` | `123-update-api-issues-docs` | + +If your branch name matches any of the above, it will run only the docs +tests. If it doesn't, the whole test suite will run (including docs). + +## Merge requests for GitLab documentation + +Before getting started, make sure you read the introductory section +"[contributing to docs](#contributing-to-docs)" above and the +[tech writing workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) +for GitLab Team members. + +- Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/merge_request_templates/Documentation.md) +- Use the correct [branch name](#branch-naming) +- Label the MR `Documentation` +- Assign the correct milestone (see note below) + + +NOTE: **Note:** +If the release version you want to add the documentation to has already been +frozen or released, use the label `Pick into X.Y` to get it merged into +the correct release. Avoid picking into a past release as much as you can, as +it increases the work of the release managers. + +### Cherry-picking from CE to EE + +As we have the `master` branch of CE merged into EE once a day, it's common to +run into merge conflicts. To avoid them, we [test for merge conflicts against EE](#testing) +with the `ee-compat-check` job, and use the following method of creating equivalent +branches for CE and EE. + +Follow this [method for cherry-picking from CE to EE](../automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee), with a few adjustments: + +- Create the [CE branch](#branch-naming) starting with `docs-`, + e.g.: `git checkout -b docs-example` +- Create the EE-equivalent branch ending with `-ee`, e.g., + `git checkout -b docs-example-ee` +- Once all the jobs are passing in CE and EE, and you've addressed the +feedback from your own team, assign the CE MR to a technical writer for review +- When both MRs are ready, the EE merge request will be merged first, and the +CE-equivalent will be merged next. +- Note that the review will occur only in the CE MR, as the EE MR +contains the same commits as the CE MR. +- If you have a few more changes that apply to the EE-version only, you can submit +a couple more commits to the EE branch, but ask the reviewer to review the EE merge request +additionally to the CE MR. If there are many EE-only changes though, start a new MR +to EE only. + +## Previewing the changes live + +To preview your changes to documentation locally, please follow +this [development guide](https://gitlab.com/gitlab-com/gitlab-docs/blob/master/README.md#development). + +If you want to preview the doc changes of your merge request live, you can use +the manual `review-docs-deploy` job in your merge request. You will need at +least Maintainer permissions to be able to run it and is currently enabled for the +following projects: + +- https://gitlab.com/gitlab-org/gitlab-ce +- https://gitlab.com/gitlab-org/gitlab-ee + +NOTE: **Note:** +You will need to push a branch to those repositories, it doesn't work for forks. + +TIP: **Tip:** +If your branch contains only documentation changes, you can use +[special branch names](#branch-naming) to avoid long running pipelines. + +In the mini pipeline graph, you should see an `>>` icon. Clicking on it will +reveal the `review-docs-deploy` job. Hit the play button for the job to start. + + + +This job will: + +1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) + project named after the scheme: `preview-<branch-slug>` +1. Trigger a cross project pipeline and build the docs site with your changes + +After a few minutes, the Review App will be deployed and you will be able to +preview the changes. The docs URL can be found in two places: + +- In the merge request widget +- In the output of the `review-docs-deploy` job, which also includes the + triggered pipeline so that you can investigate whether something went wrong + +In case the Review App URL returns 404, follow these steps to debug: + +1. **Did you follow the URL from the merge request widget?** If yes, then check if + the link is the same as the one in the job output. It can happen that if the + branch name slug is longer than 35 characters, it is automatically + truncated. That means that the merge request widget will not show the proper + URL due to a limitation of how `environment: url` works, but you can find the + real URL from the output of the `review-docs-deploy` job. +1. **Did you follow the URL from the job output?** If yes, then it means that + either the site is not yet deployed or something went wrong with the remote + pipeline. Give it a few minutes and it should appear online, otherwise you + can check the status of the remote pipeline from the link in the job output. + If the pipeline failed or got stuck, drop a line in the `#docs` chat channel. + +TIP: **Tip:** +Someone that has no merge rights to the CE/EE projects (think of forks from +contributors) will not be able to run the manual job. In that case, you can +ask someone from the GitLab team who has the permissions to do that for you. + +NOTE: **Note:** +Make sure that you always delete the branch of the merge request you were +working on. If you don't, the remote docs branch won't be removed either, +and the server where the Review Apps are hosted will eventually be out of +disk space. + +### Technical aspects + +If you want to know the hot details, here's what's really happening: + +1. You manually run the `review-docs-deploy` job in a CE/EE merge request. +1. The job runs the [`scripts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) + script with the `deploy` flag, which in turn: + 1. Takes your branch name and applies the following: + - The slug of the branch name is used to avoid special characters since + ultimately this will be used by NGINX. + - The `preview-` prefix is added to avoid conflicts if there's a remote branch + with the same name that you created in the merge request. + - The final branch name is truncated to 42 characters to avoid filesystem + limitations with long branch names (> 63 chars). + 1. The remote branch is then created if it doesn't exist (meaning you can + re-run the manual job as many times as you want and this step will be skipped). + 1. A new cross-project pipeline is triggered in the docs project. + 1. The preview URL is shown both at the job output and in the merge request + widget. You also get the link to the remote pipeline. +1. In the docs project, the pipeline is created and it + [skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55) + to lower the build time. +1. Once the docs site is built, the HTML files are uploaded as artifacts. +1. A specific Runner tied only to the docs project, runs the Review App job + that downloads the artifacts and uses `rsync` to transfer the files over + to a location where NGINX serves them. + +The following GitLab features are used among others: + +- [Manual actions](../../ci/yaml/README.md#manual-actions) +- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Review Apps](../../ci/review_apps/index.md) +- [Artifacts](../../ci/yaml/README.md#artifacts) +- [Specific Runner](../../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) + +## GitLab `/help` + +Every GitLab instance includes the documentation, which is available from `/help` +(`http://my-instance.com/help`), e.g., <https://gitlab.com/help>. + +When you're building a new feature, you may need to link the documentation +from GitLab, the application. This is normally done in files inside the +`app/views/` directory with the help of the `help_page_path` helper method. + +In its simplest form, the HAML code to generate a link to the `/help` page is: + +```haml += link_to 'Help page', help_page_path('user/permissions') +``` + +The `help_page_path` contains the path to the document you want to link to with +the following conventions: + +- it is relative to the `doc/` directory in the GitLab repository +- the `.md` extension must be omitted +- it must not end with a slash (`/`) + +Below are some special cases where should be used depending on the context. +You can combine one or more of the following: + +1. **Linking to an anchor link.** Use `anchor` as part of the `help_page_path` + method: + + ```haml + = link_to 'Help page', help_page_path('user/permissions', anchor: 'anchor-link') + ``` + +1. **Opening links in a new tab.** This should be the default behavior: + + ```haml + = link_to 'Help page', help_page_path('user/permissions'), target: '_blank' + ``` + +1. **Linking to a circle icon.** Usually used in settings where a long + description cannot be used, like near checkboxes. You can basically use + any font awesome icon, but prefer the `question-circle`: + + ```haml + = link_to icon('question-circle'), help_page_path('user/permissions') + ``` + +1. **Using a button link.** Useful in places where text would be out of context + with the rest of the page layout: + + ```haml + = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' + ``` + +1. **Using links inline of some text.** + + ```haml + Description to #{link_to 'Help page', help_page_path('user/permissions')}. + ``` + +1. **Adding a period at the end of the sentence.** Useful when you don't want + the period to be part of the link: + + ```haml + = succeed '.' do + Learn more in the + = link_to 'Help page', help_page_path('user/permissions') + ``` + +## General Documentation vs Technical Articles + +### General documentation + +General documentation is categorized by _User_, _Admin_, and _Contributor_, and describe what that feature is, what it does, and its available settings. + +### Technical Articles + +Technical articles replace technical content that once lived in the [GitLab Blog](https://about.gitlab.com/blog/), where they got out-of-date and weren't easily found. + +They are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. + +A technical article guides users and/or admins to achieve certain objectives (within guides and tutorials), or provide an overview of that particular topic or feature (within technical overviews). It can also describe the use, implementation, or integration of third-party tools with GitLab. + +They should be placed in a new directory named `/article-title/index.md` under a topic-related folder, and their images should be placed in `/article-title/img/`. For example, a new article on GitLab Pages should be placed in `doc/user/project/pages/article-title/` and a new article on GitLab CI/CD should be placed in `doc/ci/examples/article-title/`. + +#### Types of Technical Articles + +- **User guides**: technical content to guide regular users from point A to point B +- **Admin guides**: technical content to guide administrators of GitLab instances from point A to point B +- **Technical Overviews**: technical content describing features, solutions, and third-party integrations +- **Tutorials**: technical content provided step-by-step on how to do things, or how to reach very specific objectives + +#### Understanding guides, tutorials, and technical overviews + +Suppose there's a process to go from point A to point B in 5 steps: `(A) 1 > 2 > 3 > 4 > 5 (B)`. + +A **guide** can be understood as a description of certain processes to achieve a particular objective. A guide brings you from A to B describing the characteristics of that process, but not necessarily going over each step. It can mention, for example, steps 2 and 3, but does not necessarily explain how to accomplish them. + +- Live example: "[Static sites and GitLab Pages domains (Part 1)](../user/project/pages/getting_started_part_one.md) to [Creating and Tweaking GitLab CI/CD for GitLab Pages (Part 4)](../../user/project/pages/getting_started_part_four.md)" + +A **tutorial** requires a clear **step-by-step** guidance to achieve a singular objective. It brings you from A to B, describing precisely all the necessary steps involved in that process, showing each of the 5 steps to go from A to B. +It does not only describes steps 2 and 3, but also shows you how to accomplish them. + +- Live example (on the blog): [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) + +A **technical overview** is a description of what a certain feature is, and what it does, but does not walk +through the process of how to use it systematically. + +- Live example (on the blog): [GitLab Workflow, an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) + +#### Special format + +Every **Technical Article** contains a frontmatter at the beginning of the doc +with the following information: + +- **Type of article** (user guide, admin guide, technical overview, tutorial) +- **Knowledge level** expected from the reader to be able to follow through (beginner, intermediate, advanced) +- **Author's name** and **GitLab.com handle** +- **Publication date** (ISO format YYYY-MM-DD) + +For example: + + +```yaml +--- +author: John Doe +author_gitlab: johnDoe +level: beginner +article_type: user guide +date: 2017-02-01 +--- +``` + +#### Technical Articles - Writing Method + +Use the [writing method](https://about.gitlab.com/handbook/product/technical-writing/#writing-method) defined by the Technical Writing team. + +[gitlab-map]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png +[graffle]: https://gitlab.com/gitlab-org/gitlab-design/blob/d8d39f4a87b90fb9ae89ca12dc565347b4900d5e/production/resources/gitlab-map.graffle diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md new file mode 100644 index 00000000000..e7ffba635c9 --- /dev/null +++ b/doc/development/documentation/styleguide.md @@ -0,0 +1,499 @@ +--- +description: 'Writing styles, markup, formatting, and reusing regular expressions throughout the GitLab Documentation.' +--- + +# Documentation style guidelines + +The documentation style guide defines the markup structure used in +GitLab documentation. Check the +[documentation guidelines](index.md) for general development instructions. + +Check the GitLab handbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). + +## Text + +- Split up long lines (wrap text), this makes it much easier to review and edit. Only + double line breaks are shown as a full line break in [GitLab markdown][gfm]. + 80-100 characters is a good line length +- Make sure that the documentation is added in the correct + [directory](index.md#documentation-directory-structure) and that + there's a link to it somewhere useful +- Do not duplicate information +- Be brief and clear +- Unless there's a logical reason not to, add documents in alphabetical order +- Write in US English +- Use [single spaces][] instead of double spaces +- Jump a line between different markups (e.g., after every paragraph, header, list, etc) +- Capitalize "G" and "L" in GitLab +- Use sentence case for titles, headings, labels, menu items, and buttons. +- Use title case when referring to [features](https://about.gitlab.com/features/) or [products](https://about.gitlab.com/pricing/), and methods. Note that some features are also objects (e.g. "Merge Requests" and "merge requests"). E.g.: GitLab Runner, Geo, Issue Boards, Git, Prometheus, Continuous Integration. + +## Formatting + +- Use double asterisks (`**`) to mark a word or text in bold (`**bold**`) +- Use undescore (`_`) for text in italics (`_italic_`) +- Jump a line between different markups, for example: + + ```md + ## Header + + Paragraph. + + - List item + - List item + ``` + +### Punctuation + +For punctuation rules, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/). + +### Ordered and unordered lists + +- Use dashes (`-`) for unordered lists instead of asterisks (`*`) +- Use the number one (`1`) for ordered lists +- For punctuation in bullet lists, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/) + +## Headings + +- Add **only one H1** in each document, by adding `#` at the beginning of + it (when using markdown). The `h1` will be the document `<title>`. +- For subheadings, use `##`, `###` and so on +- Avoid putting numbers in headings. Numbers shift, hence documentation anchor + links shift too, which eventually leads to dead links. If you think it is + compelling to add numbers in headings, make sure to at least discuss it with + someone in the Merge Request +- [Avoid using symbols and special chars](https://gitlab.com/gitlab-com/gitlab-docs/issues/84) + in headers. Whenever possible, they should be plain and short text. +- Avoid adding things that show ephemeral statuses. For example, if a feature is + considered beta or experimental, put this info in a note, not in the heading. +- When introducing a new document, be careful for the headings to be + grammatically and syntactically correct. Mention one or all + of the following GitLab members for a review: `@axil` or `@marcia`. + This is to ensure that no document with wrong heading is going + live without an audit, thus preventing dead links and redirection issues when + corrected +- Leave exactly one newline after a heading + +## Links + +- Use the regular inline link markdown markup `[Text](https://example.com)`. + It's easier to read, review, and maintain. +- If there's a link that repeats several times through the same document, + you can use `[Text][identifier]` and at the bottom of the section or the + document add: `[identifier]: https://example.com`, in which case, we do + encourage you to also add an alternative text: `[identifier]: https://example.com "Alternative text"` that appears when hovering your mouse on a link. +- To link to internal documentation, use relative links, not full URLs. Use `../` to + navigate tp high-level directories, and always add the file name `file.md` at the + end of the link with the `.md` extension, not `.html`. + Example: instead of `[text](../../merge_requests/)`, use + `[text](../../merge_requests/index.md)` or, `[text](../../ci/README.md)`, or, + for anchor links, `[text](../../ci/README.md#examples)`. + Using the markdown extension is necessary for the [`/help`](index.md#gitlab-help) + section of GitLab. +- To link from CE to EE-only documentation, use the EE-only doc full URL. +- Use [meaningful anchor texts](https://www.futurehosting.com/blog/links-should-have-meaningful-anchor-text-heres-why/). + E.g., instead of writing something like `Read more about GitLab Issue Boards [here](LINK)`, + write `Read more about [GitLab Issue Boards](LINK)`. + +## Images + +- Place images in a separate directory named `img/` in the same directory where + the `.md` document that you're working on is located. Always prepend their + names with the name of the document that they will be included in. For + example, if there is a document called `twitter.md`, then a valid image name + could be `twitter_login_screen.png`. [**Exception**: images for + [articles](index.md#technical-articles) should be + put in a directory called `img` underneath `/articles/article_title/img/`, therefore, + there's no need to prepend the document name to their filenames.] +- Images should have a specific, non-generic name that will differentiate them. +- Keep all file names in lower case. +- Consider using PNG images instead of JPEG. +- Compress all images with <https://tinypng.com/> or similar tool. +- Compress gifs with <https://ezgif.com/optimize> or similar tool. +- Images should be used (only when necessary) to _illustrate_ the description +of a process, not to _replace_ it. + +Inside the document: + +- The Markdown way of using an image inside a document is: + `` +- Always use a proper description for what the image is about. That way, when a + browser fails to show the image, this text will be used as an alternative + description +- If there are consecutive images with little text between them, always add + three dashes (`---`) between the image and the text to create a horizontal + line for better clarity +- If a heading is placed right after an image, always add three dashes (`---`) + between the image and the heading + +## Alert boxes + +Whenever you want to call the attention to a particular sentence, +use the following markup for highlighting. + +_Note that the alert boxes only work for one paragraph only. Multiple paragraphs, +lists, headers, etc will not render correctly._ + +### Note + +```md +NOTE: **Note:** +This is something to note. +``` + +How it renders in docs.gitlab.com: + +NOTE: **Note:** +This is something to note. + +### Tip + +```md +TIP: **Tip:** +This is a tip. +``` + +How it renders in docs.gitlab.com: + +TIP: **Tip:** +This is a tip. + +### Caution + +```md +CAUTION: **Caution:** +This is something to be cautious about. +``` + +How it renders in docs.gitlab.com: + +CAUTION: **Caution:** +This is something to be cautious about. + +### Danger + +```md +DANGER: **Danger:** +This is a breaking change, a bug, or something very important to note. +``` + +How it renders in docs.gitlab.com: + +DANGER: **Danger:** +This is a breaking change, a bug, or something very important to note. + +## Blockquotes + +For highlighting a text within a blue blockquote, use this format: + +```md +> This is a blockquote. +``` + +which renders in docs.gitlab.com to: + +> This is a blockquote. + +If the text spans across multiple lines it's OK to split the line. + +## Specific sections and terms + +To mention and/or reference specific terms in GitLab, please follow the styles +below. + +### GitLab versions and tiers + +- Every piece of documentation that comes with a new feature should declare the + GitLab version that feature got introduced. Right below the heading add a + note: + + ```md + > Introduced in GitLab 8.3. + ``` + +- Whenever possible, every feature should have a link to the MR, issue, or epic that introduced it. + The above note would be then transformed to: + + ```md + > [Introduced][ce-1242] in GitLab 8.3. + ``` + + , where the [link identifier](#links) is named after the repository (CE) and + the MR number. + +- If the feature is only available in GitLab Enterprise Edition, don't forget to mention + the [paid tier](https://about.gitlab.com/handbook/marketing/product-marketing/#tiers) + the feature is available in: + + ```md + > [Introduced][ee-1234] in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. + ``` + +### Product badges + +When a feature is available in EE-only tiers, add the corresponding tier according to the +feature availability: + +- For GitLab Starter and GitLab.com Bronze: `**[STARTER]**` +- For GitLab Premium and GitLab.com Silver: `**[PREMIUM]**` +- For GitLab Ultimate and GitLab.com Gold: `**[ULTIMATE]**` +- For GitLab Core and GitLab.com Free: `**[CORE]**` + +To exclude GitLab.com tiers (when the feature is not available in GitLab.com), add the +keyword "only": + +- For GitLab Starter: `**[STARTER ONLY]**` +- For GitLab Premium: `**[PREMIUM ONLY]**` +- For GitLab Ultimate: `**[ULTIMATE ONLY]**` +- For GitLab Core: `**[CORE ONLY]**` + +The tier should be ideally added to headers, so that the full badge will be displayed. +But it can be also mentioned from paragraphs, list items, and table cells. For these cases, +the tier mention will be represented by an orange question mark. +E.g., `**[STARTER]**` renders **[STARTER]**, `**[STARTER ONLY]**` renders **[STARTER ONLY]**. + +The absence of tiers' mentions mean that the feature is available in GitLab Core, +GitLab.com Free, and higher tiers. + +#### How it works + +Introduced by [!244](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/244), +the special markup `**[STARTER]**` will generate a `span` element to trigger the +badges and tooltips (`<span class="badge-trigger starter">`). When the keyword +"only" is added, the corresponding GitLab.com badge will not be displayed. + +### GitLab Restart + +There are many cases that a restart/reconfigure of GitLab is required. To +avoid duplication, link to the special document that can be found in +[`doc/administration/restart_gitlab.md`][doc-restart]. Usually the text will +read like: + + ``` + Save the file and [reconfigure GitLab](../../administration/restart_gitlab.md) + for the changes to take effect. + ``` + +If the document you are editing resides in a place other than the GitLab CE/EE +`doc/` directory, instead of the relative link, use the full path: +`http://docs.gitlab.com/ce/administration/restart_gitlab.html`. +Replace `reconfigure` with `restart` where appropriate. + +### Installation guide + +**Ruby:** +In [step 2 of the installation guide](../../install/installation.md#2-ruby), +we install Ruby from source. Whenever there is a new version that needs to +be updated, remember to change it throughout the codeblock and also replace +the sha256sum (it can be found in the [downloads page][ruby-dl] of the Ruby +website). + +[ruby-dl]: https://www.ruby-lang.org/en/downloads/ "Ruby download website" + +### Configuration documentation for source and Omnibus installations + +GitLab currently officially supports two installation methods: installations +from source and Omnibus packages installations. + +Whenever there is a setting that is configurable for both installation methods, +prefer to document it in the CE docs to avoid duplication. + +Configuration settings include: + +- settings that touch configuration files in `config/` +- NGINX settings and settings in `lib/support/` in general + +When there is a list of steps to perform, usually that entails editing the +configuration file and reconfiguring/restarting GitLab. In such case, follow +the style below as a guide: + +```md +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + external_url "https://gitlab.example.com" + ``` + +1. Save the file and [reconfigure] GitLab for the changes to take effect. + +--- + +**For installations from source** + +1. Edit `config/gitlab.yml`: + + ```yaml + gitlab: + host: "gitlab.example.com" + ``` + +1. Save the file and [restart] GitLab for the changes to take effect. + + +[reconfigure]: path/to/administration/restart_gitlab.md#omnibus-gitlab-reconfigure +[restart]: path/to/administration/restart_gitlab.md#installations-from-source +``` + +In this case: + +- before each step list the installation method is declared in bold +- three dashes (`---`) are used to create a horizontal line and separate the + two methods +- the code blocks are indented one or more spaces under the list item to render + correctly +- different highlighting languages are used for each config in the code block +- the [references](#references) guide is used for reconfigure/restart + +### Fake tokens + +There may be times where a token is needed to demonstrate an API call using +cURL or a variable used in CI. It is strongly advised not to use real +tokens in documentation even if the probability of a token being exploited is +low. + +You can use the following fake tokens as examples. + +| **Token type** | **Token value** | +| --------------------- | --------------------------------- | +| Private user token | `9koXpg98eAheJpvBs5tK` | +| Personal access token | `n671WNGecHugsdEDPsyo` | +| Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` | +| Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` | +| Secret CI variable | `Li8j-mLUVA3eZYjPfd_H` | +| Specific Runner token | `yrnZW46BrtBFqM7xDzE7dddd` | +| Shared Runner token | `6Vk7ZsosqQyfreAxXTZr` | +| Trigger token | `be20d8dcc028677c931e04f3871a9b` | +| Webhook secret token | `6XhDroRcYPM5by_h-HLY` | +| Health check token | `Tu7BgjR9qeZTEyRzGG2P` | +| Request profile token | `7VgpS4Ax5utVD2esNstz` | + +### API + +Here is a list of must-have items. Use them in the exact order that appears +on this document. Further explanation is given below. + +- Every method must have the REST API request. For example: + + ``` + GET /projects/:id/repository/branches + ``` + +- Every method must have a detailed + [description of the parameters](#method-description). +- Every method must have a cURL example. +- Every method must have a response body (in JSON format). + +#### Method description + +Use the following table headers to describe the methods. Attributes should +always be in code blocks using backticks (``` ` ```). + +``` +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +``` + +Rendered example: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `user` | string | yes | The GitLab username | + +#### cURL commands + +- Use `https://gitlab.example.com/api/v4/` as an endpoint. +- Wherever needed use this personal access token: `9koXpg98eAheJpvBs5tK`. +- Always put the request first. `GET` is the default so you don't have to + include it. +- Use double quotes to the URL when it includes additional parameters. +- Prefer to use examples using the personal access token and don't pass data of + username and password. + +| Methods | Description | +| ------- | ----------- | +| `-H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK"` | Use this method as is, whenever authentication needed | +| `-X POST` | Use this method when creating new objects | +| `-X PUT` | Use this method when updating existing objects | +| `-X DELETE` | Use this method when removing existing objects | + +#### cURL Examples + +Below is a set of [cURL][] examples that you can use in the API documentation. + +##### Simple cURL command + +Get the details of a group: + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/gitlab-org +``` + +##### cURL example with parameters passed in the URL + +Create a new project under the authenticated user's namespace: + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects?name=foo" +``` + +##### Post data using cURL's --data + +Instead of using `-X POST` and appending the parameters to the URI, you can use +cURL's `--data` option. The example below will create a new project `foo` under +the authenticated user's namespace. + +```bash +curl --data "name=foo" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects" +``` + +##### Post data using JSON content + +> **Note:** In this example we create a new group. Watch carefully the single +and double quotes. + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v4/groups +``` + +##### Post data using form-data + +Instead of using JSON or urlencode you can use multipart/form-data which +properly handles data encoding: + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "title=ssh-key" --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." https://gitlab.example.com/api/v4/users/25/keys +``` + +The above example is run by and administrator and will add an SSH public key +titled ssh-key to user's account which has an id of 25. + +##### Escape special characters + +Spaces or slashes (`/`) may sometimes result to errors, thus it is recommended +to escape them when possible. In the example below we create a new issue which +contains spaces in its title. Observe how spaces are escaped using the `%20` +ASCII code. + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20Dude" +``` + +Use `%2F` for slashes (`/`). + +##### Pass arrays to API calls + +The GitLab API sometimes accepts arrays of strings or integers. For example, to +restrict the sign-up e-mail domains of a GitLab instance to `*.example.com` and +`example.net`, you would do something like this: + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "domain_whitelist[]=*.example.com" --data "domain_whitelist[]=example.net" https://gitlab.example.com/api/v4/application/settings +``` + +[cURL]: http://curl.haxx.se/ "cURL website" +[single spaces]: http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html +[gfm]: http://docs.gitlab.com/ce/user/markdown.html#newlines "GitLab flavored markdown documentation" +[ce-1242]: https://gitlab.com/gitlab-org/gitlab-ce/issues/1242 +[doc-restart]: ../../administration/restart_gitlab.md "GitLab restart documentation" diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index 4873090a2d4..7f061d06da8 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -53,6 +53,9 @@ stub_licensed_features(variable_environment_scope: true) EE-specific comments should not be backported to CE. +**Note:** This is only meant as a workaround, we should follow up and +resolve this soon. + ### Detection of EE-only files For each commit (except on `master`), the `ee-files-location-check` CI job tries @@ -105,11 +108,14 @@ is applied not only to models. Here's a list of other examples: - `ee/app/services/foo/create_service.rb` - `ee/app/validators/foo_attr_validator.rb` - `ee/app/workers/foo_worker.rb` +- `ee/app/views/foo.html.haml` +- `ee/app/views/foo/_bar.html.haml` This works because for every path that are present in CE's eager-load/auto-load paths, we add the same `ee/`-prepended path in [`config/application.rb`]. +This also applies to views. -[`config/application.rb`]: https://gitlab.com/gitlab-org/gitlab-ee/blob/d278b76d6600a0e27d8019a0be27971ba23ab640/config/application.rb#L41-51 +[`config/application.rb`]: https://gitlab.com/gitlab-org/gitlab-ee/blob/925d3d4ebc7a2c72964ce97623ae41b8af12538d/config/application.rb#L42-52 ### EE features based on CE features @@ -359,9 +365,63 @@ Blocks of code that are EE-specific should be moved to partials. This avoids conflicts with big chunks of HAML code that that are not fun to resolve when you add the indentation to the equation. -EE-specific views should be placed in `ee/app/views/ee/`, using extra +EE-specific views should be placed in `ee/app/views/`, using extra sub-directories if appropriate. +#### Using `render_if_exists` + +Instead of using regular `render`, we should use `render_if_exists`, which +will not render anything if it cannot find the specific partial. We use this +so that we could put `render_if_exists` in CE, keeping code the same between +CE and EE. + +The advantages of this: + +- Minimal code difference between CE and EE. +- Very clear hints about where we're extending EE views while reading CE codes. + +The disadvantage of this: + +- Slightly more work while developing EE features, because now we need to + port `render_if_exists` to CE. +- If we have typos in the partial name, it would be silently ignored. + +#### Using `render_ce` + +For `render` and `render_if_exists`, they search for the EE partial first, +and then CE partial. They would only render a particular partial, not all +partials with the same name. We could take the advantage of this, so that +the same partial path (e.g. `shared/issuable/form/default_templates`) could +be referring to the CE partial in CE (i.e. +`app/views/shared/issuable/form/_default_templates.html.haml`), while EE +partial in EE (i.e. +`ee/app/views/shared/issuable/form/_default_templates.html.haml`). This way, +we could show different things between CE and EE. + +However sometimes we would also want to reuse the CE partial in EE partial +because we might just want to add something to the existing CE partial. We +could workaround this by adding another partial with a different name, but it +would be tedious to do so. + +In this case, we could as well just use `render_ce` which would ignore any EE +partials. One example would be +`ee/app/views/shared/issuable/form/_default_templates.html.haml`: + +``` haml +- if @project.feature_available?(:issuable_default_templates) + = render_ce 'shared/issuable/form/default_templates' +- elsif show_promotions? + = render 'shared/promotions/promote_issue_templates' +``` + +In the above example, we can't use +`render 'shared/issuable/form/default_templates'` because it would find the +same EE partial, causing infinite recursion. Instead, we could use `render_ce` +so it ignores any partials in `ee/` and then it would render the CE partial +(i.e. `app/views/shared/issuable/form/_default_templates.html.haml`) +for the same path (i.e. `shared/issuable/form/default_templates`). This way +we could easily wrap around the CE partial. + ### Code in `lib/` Place EE-specific logic in the top-level `EE` module namespace. Namespace the diff --git a/doc/development/fe_guide/development_process.md b/doc/development/fe_guide/development_process.md index 5504a6421d5..d240dbe8b02 100644 --- a/doc/development/fe_guide/development_process.md +++ b/doc/development/fe_guide/development_process.md @@ -22,9 +22,9 @@ Please use your best judgement when to use it and please contribute new points t - [ ] Are all [departments](https://about.gitlab.com/handbook/engineering/#engineering-teams) that are needed from your perspective already involved in the issue? (For example is UX missing?) - [ ] Is the specification complete? Are you missing decisions? How about error handling/defaults/edge cases? Take your time to understand the needed implementation and go through its flow. - [ ] Are all necessary UX specifications available that you will need in order to implement? Are there new UX components/patterns in the designs? Then contact the UI component team early on. How should error messages or validation be handled? -- [ ] **Library usage** Use Vuex as soon as you have even a medium state to manage, use Vue router if you need to have different views internally and want to link from the outside. Check what libraries we already have for which occassions. +- [ ] **Library usage** Use Vuex as soon as you have even a medium state to manage, use Vue router if you need to have different views internally and want to link from the outside. Check what libraries we already have for which occasions. - [ ] **Plan your implementation:** - - [ ] **Architecture plan:** Create a plan aligned with GitLab's architecture, how you are going to do the implementation, for example Vue application setup and its components (through [onion skinning](https://gitlab.com/gitlab-org/gitlab-ce/issues/35873#note_39994091)), Store structure and data flow, which existing Vue components can you reuse. Its a good idea to go through your plan with another engineer to refine it. + - [ ] **Architecture plan:** Create a plan aligned with GitLab's architecture, how you are going to do the implementation, for example Vue application setup and its components (through [onion skinning](https://gitlab.com/gitlab-org/gitlab-ce/issues/35873#note_39994091)), Store structure and data flow, which existing Vue components can you reuse. It's a good idea to go through your plan with another engineer to refine it. - [ ] **Backend:** The best way is to kickoff the implementation in a call and discuss with the assigned Backend engineer what you will need from the backend and also when. Can you reuse existing API's? How is the performance with the planned architecture? Maybe create together a JSON mock object to already start with development. - [ ] **Communication:** It also makes sense to have for bigger features an own slack channel (normally called #f_{feature_name}) and even weekly demo calls with all people involved. - [ ] **Dependency Plan:** Are there big dependencies in the plan between you and others, then maybe create an execution diagram to show what is blocking which part and the order of the different parts. @@ -56,7 +56,7 @@ Please use your best judgement when to use it and please contribute new points t - [ ] If you have multiple MR's then also smoke test against the final merge. - [ ] Are there any big changes on how and especially how frequently we use the API then let production know about it - [ ] Smoke test of the RC on dev., staging., canary deployments and .com -- [ ] Follow up on issues that came out of the review. Create isssues for discovered edge cases that should be covered in future iterations. +- [ ] Follow up on issues that came out of the review. Create issues for discovered edge cases that should be covered in future iterations. --- diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md index b469a9c6aef..3d8da6accc1 100644 --- a/doc/development/fe_guide/icons.md +++ b/doc/development/fe_guide/icons.md @@ -1,26 +1,44 @@ -# Icons +# Icons and SVG Illustrations -We are using SVG Icons in GitLab with a SVG Sprite, due to this the icons are only loaded once and then referenced through an ID. The sprite SVG is located under `/assets/icons.svg`. Our goal is to replace one by one all inline SVG Icons (as those currently bloat the HTML) and also all Font Awesome usages. +We manage our own Icon and Illustration library in the [gitlab-svgs][gitlab-svgs] repository. +This repository is published on [npm][npm] and managed as a dependency via yarn. +You can browse all available Icons and Illustrations [here][svg-preview]. +To upgrade to a new version run `yarn upgrade @gitlab-org/gitlab-svgs`. -### Usage in HAML/Rails +## Icons -To use a sprite Icon in HAML or Rails we use a specific helper function : +We are using SVG Icons in GitLab with a SVG Sprite. +This means the icons are only loaded once, and are referenced through an ID. +The sprite SVG is located under `/assets/icons.svg`. + +Our goal is to replace one by one all inline SVG Icons (as those currently bloat the HTML) and also all Font Awesome icons. -`sprite_icon(icon_name, size: nil, css_class: '')` +### Usage in HAML/Rails -**icon_name** Use the icon_name that you can find in the SVG Sprite ([Overview is available here](http://gitlab-org.gitlab.io/gitlab-svgs/)`). +To use a sprite Icon in HAML or Rails we use a specific helper function : -**size (optional)** Use one of the following sizes : 16,24,32,48,72 (this will be translated into a `s16` class) +```ruby +sprite_icon(icon_name, size: nil, css_class: '') +``` -**css_class (optional)** If you want to add additional css classes +- **icon_name** Use the icon_name that you can find in the SVG Sprite + ([Overview is available here][svg-preview]). +- **size (optional)** Use one of the following sizes : 16, 24, 32, 48, 72 (this will be translated into a `s16` class) +- **css_class (optional)** If you want to add additional css classes **Example** -`= sprite_icon('issues', size: 72, css_class: 'icon-danger')` +```haml += sprite_icon('issues', size: 72, css_class: 'icon-danger') +``` **Output from example above** -`<svg class="s72 icon-danger"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="/assets/icons.svg#issues"></use></svg>` +```html +<svg class="s72 icon-danger"> + <use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="/assets/icons.svg#issues"></use> +</svg> +``` ### Usage in Vue @@ -28,33 +46,71 @@ We have a special Vue component for our sprite icons in `\vue_shared\components\ Sample usage : -`<icon - name="retry" - :size="32" - css-classes="top" - />` - -**name** Name of the Icon in the SVG Sprite ([Overview is available here](http://gitlab-org.gitlab.io/gitlab-svgs/)`). - -**size (optional)** Number value for the size which is then mapped to a specific CSS class (Available Sizes: 8,12,16,18,24,32,48,72 are mapped to `sXX` css classes) - -**css-classes (optional)** Additional CSS Classes to add to the svg tag. +```javascript +<script> +import Icon from "~/vue_shared/components/icon.vue" + +export default { + components: { + Icon, + }, +}; +<script> +<template> + <icon + name="issues" + :size="72" + css-classes="icon-danger" + /> +</template> +``` + +- **name** Name of the Icon in the SVG Sprite ([Overview is available here][svg-preview]). +- **size (optional)** Number value for the size which is then mapped to a specific CSS class + (Available Sizes: 8, 12, 16, 18, 24, 32, 48, 72 are mapped to `sXX` css classes) +- **css-classes (optional)** Additional CSS Classes to add to the svg tag. ### Usage in HTML/JS -Please use the following function inside JS to render an icon : +Please use the following function inside JS to render an icon: `gl.utils.spriteIcon(iconName)` -## Adding a new icon to the sprite +## SVG Illustrations -All Icons and Illustrations are managed in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository which is added as a dev-dependency. +Please use from now on for any SVG based illustrations simple `img` tags to show an illustration by simply using either `image_tag` or `image_path` helpers. +Please use the class `svg-content` around it to ensure nice rendering. -To upgrade to a new SVG Sprite version run `yarn upgrade @gitlab-org/gitlab-svgs`. +### Usage in HAML/Rails -# SVG Illustrations +**Example** -Please use from now on for any SVG based illustrations simple `img` tags to show an illustration by simply using either `image_tag` or `image_path` helpers. Please use the class `svg-content` around it to ensure nice rendering. The illustrations are also organised in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository (as they are then automatically optimised). +```haml +.svg-content + = image_tag 'illustrations/merge_requests.svg' +``` -**Example** +### Usage in Vue -`= image_tag 'illustrations/merge_requests.svg'` +To use an SVG illustrations in a template provide the path as a property and display it through a standard img tag. + +Component: + +```js +<script> +export default { + props: { + svgIllustrationPath: { + type: String, + required: true, + }, + }, +}; +<script> +<template> + <img :src="svgIllustrationPath" /> +</template> +``` + +[npm]: https://www.npmjs.com/package/@gitlab-org/gitlab-svgs +[gitlab-svgs]: https://gitlab.com/gitlab-org/gitlab-svgs +[svg-preview]: https://gitlab-org.gitlab.io/gitlab-svgs diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 3b4dfd50761..11b9a2e6a64 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -1,5 +1,8 @@ # Frontend Development Guidelines +> **Notice:** +We are currently in the process of re-writing our development guide to make it easier to find information. The new guide is still WIP but viewable in [development/new_fe_guide](../new_fe_guide/index.md) + This document describes various guidelines to ensure consistency and quality across GitLab's frontend team. @@ -45,11 +48,14 @@ Common JavaScript design patterns in GitLab's codebase. ## [Vue.js Best Practices](vue.md) Vue specific design patterns and practices. +## [Vuex](vuex.md) +Vuex specific design patterns and practices. + ## [Axios](axios.md) Axios specific practices and gotchas. -## [Icons](icons.md) -How we use SVG for our Icons. +## [Icons and Illustrations](icons.md) +How we use SVG for our Icons and Illustrations. ## [Components](components.md) diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 677168937c7..284b4b53334 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -310,7 +310,7 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. })); ``` -1. Don not use a singleton for the service or the store +1. Do not use a singleton for the service or the store ```javascript // bad class Store { @@ -328,9 +328,11 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. } } ``` +1. Use `.vue` for Vue templates. Do not use `%template` in HAML. #### Naming -1. **Extensions**: Use `.vue` extension for Vue components. + +1. **Extensions**: Use `.vue` extension for Vue components. Do not use `.js` as file extension ([#34371]). 1. **Reference Naming**: Use PascalCase for their instances: ```javascript // bad @@ -364,6 +366,8 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. <component my-prop="prop" /> ``` +[#34371]: https://gitlab.com/gitlab-org/gitlab-ce/issues/34371 + #### Alignment 1. Follow these alignment styles for the template method: 1. With more than one attribute, all attributes should be on a new line: @@ -628,7 +632,7 @@ Useful links: // good <span title="tooltip text">Foo</span> - $('span').tooltip('fixTitle'); + $('span').tooltip('_fixTitle'); ``` ### The Javascript/Vue Accord diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md index 655d94793dd..48eb6d0a7d6 100644 --- a/doc/development/fe_guide/style_guide_scss.md +++ b/doc/development/fe_guide/style_guide_scss.md @@ -216,12 +216,12 @@ If you want a line or set of lines to be ignored by the linter, you can use `// scss-lint:disable RuleName` ([more info][disabling-linters]): ```scss -// This lint rule is disabled because the class name comes from a gem. -// scss-lint:disable SelectorFormat -.ui_indigo { - background-color: #333; +// This lint rule is disabled because it is supported only in Chrome/Safari +// scss-lint:disable PropertySpelling +body { + text-decoration-skip: ink; } -// scss-lint:enable SelectorFormat +// scss-lint:enable PropertySpelling ``` Make sure a comment is added on the line above the `disable` rule, otherwise the diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 9c4b0e86351..e31ee087358 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -1,36 +1,14 @@ # Vue -For more complex frontend features, we recommend using Vue.js. It shares -some ideas with React.js as well as Angular. - To get started with Vue, read through [their documentation][vue-docs]. -## When to use Vue.js - -We recommend using Vue for more complex features. Here are some guidelines for when to use Vue.js: - -- If you are starting a new feature or refactoring an old one that highly interacts with the DOM; -- For real time data updates; -- If you are creating a component that will be reused elsewhere; - -## When not to use Vue.js - -We don't want to refactor all GitLab frontend code into Vue.js, here are some guidelines for -when not to use Vue.js: - -- Adding or changing static information; -- Features that highly depend on jQuery will be hard to work with Vue.js; -- Features without reactive data; - -As always, the Frontend Architectural Experts are available to help with any Vue or JavaScript questions. - ## Vue architecture All new features built with Vue.js must follow a [Flux architecture][flux]. The main goal we are trying to achieve is to have only one data flow and only one data entry. In order to achieve this goal, you can either use [vuex](#vuex) or use the [store pattern][state-management], explained below: -Each Vue bundle needs a Store - where we keep all the data -,a Service - that we use to communicate with the server - and a main Vue component. +Each Vue bundle needs a Store - where we keep all the data -, a Service - that we use to communicate with the server - and a main Vue component. Think of the Main Vue Component as the entry point of your application. This is the only smart component that should exist in each Vue feature. @@ -39,7 +17,7 @@ This component is responsible for: 1. Calling the Store to store the data received 1. Mounting all the other components -  + You can also read about this architecture in vue docs about [state management][state-management] and about [one way data flow][one-way-data-flow]. @@ -57,15 +35,15 @@ new_feature │ └── ... ├── stores │ └── new_feature_store.js -├── services +├── services # only when not using vuex │ └── new_feature_service.js -├── new_feature_bundle.js +├── index.js ``` _For consistency purposes, we recommend you to follow the same structure._ Let's look into each of them: -### A `*_bundle.js` file +### A `index.js` file This is the index file of your new feature. This is where the root Vue instance of the new feature should be. @@ -73,14 +51,14 @@ of the new feature should be. The Store and the Service should be imported and initialized in this file and provided as a prop to the main component. -Don't forget to follow [these steps.][page_specific_javascript] +Don't forget to follow [these steps][page_specific_javascript]. ### Bootstrapping Gotchas -#### Providing data from Haml to JavaScript +#### Providing data from HAML to JavaScript While mounting a Vue application may be a need to provide data from Rails to JavaScript. To do that, provide the data through `data` attributes in the HTML element and query them while mounting the application. -_Note:_ You should only do this while initing the application, because the mounted element will be replaced with Vue-generated DOM. +_Note:_ You should only do this while initializing the application, because the mounted element will be replaced with Vue-generated DOM. The advantage of providing data from the DOM to the Vue instance through `props` in the `render` function instead of querying the DOM inside the main vue component is that makes tests easier by avoiding the need to @@ -90,6 +68,7 @@ create a fixture or an HTML element in the unit test. See the following example: // haml .js-vue-app{ data: { endpoint: 'foo' }} +// index.js document.addEventListener('DOMContentLoaded', () => new Vue({ el: '.js-vue-app', data() { @@ -109,13 +88,11 @@ document.addEventListener('DOMContentLoaded', () => new Vue({ ``` #### Accessing the `gl` object -When we need to query the `gl` object for data that won't change during the application's lyfecyle, we should do it in the same place where we query the DOM. +When we need to query the `gl` object for data that won't change during the application's life cyle, we should do it in the same place where we query the DOM. By following this practice, we can avoid the need to mock the `gl` object, which will make tests easier. It should be done while initializing our Vue instance, and the data should be provided as `props` to the main component: -##### example: ```javascript - document.addEventListener('DOMContentLoaded', () => new Vue({ el: '.js-vue-app', render(createElement) { @@ -143,31 +120,12 @@ in one table would not be a good use of this pattern. You can read more about components in Vue.js site, [Component System][component-system] -#### Components Gotchas -1. Using SVGs in components: To use an SVG in a template we need to make it a property we can access through the component. -A `prop` and a property returned by the `data` functions require `vue` to set a `getter` and a `setter` for each of them. -The SVG should be a computed property in order to improve performance, note that computed properties are cached based on their dependencies. - -```javascript -// bad -import svg from 'svg.svg'; -data() { - return { - myIcon: svg, - }; -}; - -// good -import svg from 'svg.svg'; -computed: { - myIcon() { - return svg; - } -} -``` - ### A folder for the Store +#### Vuex +Check this [page](vuex.md) for more details. + +#### Flux like state management The Store is a class that allows us to manage the state in a single source of truth. It is not aware of the service or the components. @@ -176,6 +134,8 @@ itself, please read this guide: [State Management][state-management] ### A folder for the Service +**If you are using Vuex you won't need this step** + The Service is a class used only to communicate with the server. It does not store or manipulate any data. It is not aware of the store or the components. We use [axios][axios] to communicate with the server. @@ -183,13 +143,13 @@ Refer to [axios](axios.md) for more details. Axios instance should only be imported in the service file. - ```javascript - import axios from 'javascripts/lib/utils/axios_utils'; - ``` +```javascript +import axios from '~/lib/utils/axios_utils'; +``` ### End Result -The following example shows an application: +The following example shows an application: ```javascript // store.js @@ -197,8 +157,8 @@ export default class Store { /** * This is where we will iniatialize the state of our data. - * Usually in a small SPA you don't need any options when starting the store. In the case you do - * need guarantee it's an Object and it's documented. + * Usually in a small SPA you don't need any options when starting the store. + * In that case you do need guarantee it's an Object and it's documented. * * @param {Object} options */ @@ -206,7 +166,7 @@ export default class Store { this.options = options; // Create a state object to handle all our data in the same place - this.todos = []: + this.todos = []; } setTodos(todos = []) { @@ -227,7 +187,7 @@ export default class Store { } // service.js -import axios from 'javascripts/lib/utils/axios_utils' +import axios from '~/lib/utils/axios_utils' export default class Service { constructor(options) { @@ -253,8 +213,8 @@ export default { type: Object, required: true, }, - } -} + }, +}; </script> <template> <div> @@ -273,6 +233,9 @@ import Store from 'store'; import Service from 'service'; import TodoComponent from 'todoComponent'; export default { + components: { + todo: TodoComponent, + }, /** * Although most data belongs in the store, each component it's own state. * We want to show a loading spinner while we are fetching the todos, this state belong @@ -291,12 +254,8 @@ export default { }; }, - components: { - todo: TodoComponent, - }, - created() { - this.service = new Service('todos'); + this.service = new Service('/todos'); this.getTodos(); }, @@ -305,9 +264,9 @@ export default { getTodos() { this.isLoading = true; - this.service.getTodos() - .then(response => response.json()) - .then((response) => { + this.service + .getTodos() + .then(response => { this.store.setTodos(response); this.isLoading = false; }) @@ -317,18 +276,21 @@ export default { }); }, - addTodo(todo) { - this.service.addTodo(todo) - then(response => response.json()) - .then((response) => { - this.store.addTodo(response); - }) - .catch(() => { - // Show an error - }); - } - } -} + addTodo(event) { + this.service + .addTodo({ + title: 'New entry', + text: `You clicked on ${event.target.tagName}`, + }) + .then(response => { + this.store.addTodo(response); + }) + .catch(() => { + // Show an error + }); + }, + }, +}; </script> <template> <div class="container"> @@ -354,7 +316,7 @@ export default { <div> </template> -// bundle.js +// index.js import todoComponent from 'todos_main_component.vue'; new Vue({ @@ -386,76 +348,79 @@ Each Vue component has a unique output. This output is always present in the ren Although we can test each method of a Vue component individually, our goal must be to test the output of the render/template function, which represents the state at all times. -Make use of Vue Resource Interceptors to mock data returned by the service. +Make use of the [axios mock adapter](axios.md#mock-axios-response-on-tests) to mock data returned. Here's how we would test the Todo App above: ```javascript -import component from 'todos_main_component'; +import Vue from 'vue'; +import axios from '~/lib/utils/axios_utils'; +import MockAdapter from 'axios-mock-adapter'; describe('Todos App', () => { - it('should render the loading state while the request is being made', () => { + let vm; + let mock; + + beforeEach(() => { + // Create a mock adapter for stubbing axios API requests + mock = new MockAdapter(axios); + const Component = Vue.extend(component); - const vm = new Component().$mount(); + // Mount the Component + vm = new Component().$mount(); + }); + afterEach(() => { + // Reset the mock adapter + mock.restore(); + // Destroy the mounted component + vm.$destroy(); + }); + + it('should render the loading state while the request is being made', () => { expect(vm.$el.querySelector('i.fa-spin')).toBeDefined(); }); - describe('with data', () => { - // Mock the service to return data - const interceptor = (request, next) => { - next(request.respondWith(JSON.stringify([{ + it('should render todos returned by the endpoint', done => { + // Mock the get request on the API endpoint to return data + mock.onGet('/todos').replyOnce(200, [ + { title: 'This is a todo', - body: 'This is the text' - }]), { - status: 200, - })); - }; - - let vm; - - beforeEach(() => { - Vue.http.interceptors.push(interceptor); - - const Component = Vue.extend(component); + text: 'This is the text', + }, + ]); - vm = new Component().$mount(); + Vue.nextTick(() => { + const items = vm.$el.querySelectorAll('.js-todo-list div') + expect(items.length).toBe(1); + expect(items[0].textContent).toContain('This is the text'); + done(); }); + }); - afterEach(() => { - Vue.http.interceptors = _.without(Vue.http.interceptors, interceptor); - }); + it('should add a todos on button click', (done) => { + // Mock the put request and check that the sent data object is correct + mock.onPut('/todos').replyOnce((req) => { + expect(req.data).toContain('text'); + expect(req.data).toContain('title'); - it('should render todos', (done) => { - setTimeout(() => { - expect(vm.$el.querySelectorAll('.js-todo-list div').length).toBe(1); - done(); - }, 0); + return [201, {}]; }); - }); - describe('add todo', () => { - let vm; - beforeEach(() => { - const Component = Vue.extend(component); - vm = new Component().$mount(); - }); - it('should add a todos', (done) => { - setTimeout(() => { - vm.$el.querySelector('.js-add-todo').click(); + vm.$el.querySelector('.js-add-todo').click(); - // Add a new interceptor to mock the add Todo request - Vue.nextTick(() => { - expect(vm.$el.querySelectorAll('.js-todo-list div').length).toBe(2); - }); - }, 0); + // Add a new interceptor to mock the add Todo request + Vue.nextTick(() => { + expect(vm.$el.querySelectorAll('.js-todo-list div').length).toBe(2); + done(); }); }); }); ``` -#### `mountComponent` helper + +### `mountComponent` helper There is a helper in `spec/javascripts/helpers/vue_mount_component_helper.js` that allows you to mount a component with the given props: ```javascript @@ -468,208 +433,10 @@ const data = {prop: 'foo'}; const vm = mountComponent(Component, data); ``` -#### Test the component's output +### Test the component's output The main return value of a Vue component is the rendered output. In order to test the component we need to test the rendered output. [Vue][vue-test] guide's to unit test show us exactly that: -### Stubbing API responses -Refer to [mock axios](axios.md#mock-axios-response-on-tests) - - -## Vuex -To manage the state of an application you may use [Vuex][vuex-docs]. - -_Note:_ All of the below is explained in more detail in the official [Vuex documentation][vuex-docs]. - -### Separation of concerns -Vuex is composed of State, Getters, Mutations, Actions and Modules. - -When a user clicks on an action, we need to `dispatch` it. This action will `commit` a mutation that will change the state. -_Note:_ The action itself will not update the state, only a mutation should update the state. - -#### File structure -When using Vuex at GitLab, separate this concerns into different files to improve readability. If you can, separate the Mutation Types as well: - -``` -└── store - ├── index.js # where we assemble modules and export the store - ├── actions.js # actions - ├── mutations.js # mutations - ├── getters.js # getters - └── mutation_types.js # mutation types -``` -The following examples show an application that lists and adds users to the state. - -##### `index.js` -This is the entry point for our store. You can use the following as a guide: - -```javascript -import Vue from 'vue'; -import Vuex from 'vuex'; -import * as actions from './actions'; -import * as getters from './getters'; -import mutations from './mutations'; - -Vue.use(Vuex); - -export default new Vuex.Store({ - actions, - getters, - mutations, - state: { - users: [], - }, -}); -``` -_Note:_ If the state of the application is too complex, an individual file for the state may be better. - -##### `actions.js` -An action commits a mutation. In this file, we will write the actions that will commit the respective mutation: - -```javascript - import * as types from './mutation_types'; - - export const addUser = ({ commit }, user) => { - commit(types.ADD_USER, user); - }; -``` - -To dispatch an action from a component, use the `mapActions` helper: -```javascript -import { mapActions } from 'vuex'; - -{ - methods: { - ...mapActions([ - 'addUser', - ]), - onClickUser(user) { - this.addUser(user); - }, - }, -}; -``` - -##### `getters.js` -Sometimes we may need to get derived state based on store state, like filtering for a specific prop. This can be done through the `getters`: - -```javascript -// get all the users with pets -export getUsersWithPets = (state, getters) => { - return state.users.filter(user => user.pet !== undefined); -}; -``` - -To access a getter from a component, use the `mapGetters` helper: -```javascript -import { mapGetters } from 'vuex'; - -{ - computed: { - ...mapGetters([ - 'getUsersWithPets', - ]), - }, -}; -``` - -##### `mutations.js` -The only way to actually change state in a Vuex store is by committing a mutation. - -```javascript - import * as types from './mutation_types'; - - export default { - [types.ADD_USER](state, user) { - state.users.push(user); - }, - }; -``` - -##### `mutations_types.js` -From [vuex mutations docs][vuex-mutations]: -> It is a commonly seen pattern to use constants for mutation types in various Flux implementations. This allows the code to take advantage of tooling like linters, and putting all constants in a single file allows your collaborators to get an at-a-glance view of what mutations are possible in the entire application. - -```javascript -export const ADD_USER = 'ADD_USER'; -``` - -### How to include the store in your application -The store should be included in the main component of your application: -```javascript - // app.vue - import store from 'store'; // it will include the index.js file - - export default { - name: 'application', - store, - ... - }; -``` - -### Vuex Gotchas -1. Avoid calling a mutation directly. Always use an action to commit a mutation. Doing so will keep consistency through out the application. From Vuex docs: - - > why don't we just call store.commit('action') directly? Well, remember that mutations must be synchronous? Actions aren't. We can perform asynchronous operations inside an action. - - ```javascript - // component.vue - - // bad - created() { - this.$store.commit('mutation'); - } - - // good - created() { - this.$store.dispatch('action'); - } - ``` -1. When possible, use mutation types instead of hardcoding strings. It will be less error prone. -1. The State will be accessible in all components descending from the use where the store is instantiated. - -### Testing Vuex -#### Testing Vuex concerns -Refer to [vuex docs][vuex-testing] regarding testing Actions, Getters and Mutations. - -#### Testing components that need a store -Smaller components might use `store` properties to access the data. -In order to write unit tests for those components, we need to include the store and provide the correct state: - -```javascript -//component_spec.js -import Vue from 'vue'; -import store from './store'; -import component from './component.vue' - -describe('component', () => { - let vm; - let Component; - - beforeEach(() => { - Component = Vue.extend(issueActions); - }); - - afterEach(() => { - vm.$destroy(); - }); - - it('should show a user', () => { - const user = { - name: 'Foo', - age: '30', - }; - - // populate the store - store.dispatch('addUser', user); - - vm = new Component({ - store, - propsData: props, - }).$mount(); - }); -}); -``` [vue-docs]: http://vuejs.org/guide/index.html [issue-boards]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/assets/javascripts/boards @@ -681,9 +448,4 @@ describe('component', () => { [vue-test]: https://vuejs.org/v2/guide/unit-testing.html [issue-boards-service]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/boards/services/board_service.js.es6 [flux]: https://facebook.github.io/flux -[vuex-docs]: https://vuex.vuejs.org -[vuex-structure]: https://vuex.vuejs.org/en/structure.html -[vuex-mutations]: https://vuex.vuejs.org/en/mutations.html -[vuex-testing]: https://vuex.vuejs.org/en/testing.html [axios]: https://github.com/axios/axios -[axios-interceptors]: https://github.com/axios/axios#interceptors diff --git a/doc/development/fe_guide/vuex.md b/doc/development/fe_guide/vuex.md new file mode 100644 index 00000000000..858b03c60bf --- /dev/null +++ b/doc/development/fe_guide/vuex.md @@ -0,0 +1,374 @@ +# Vuex +To manage the state of an application you should use [Vuex][vuex-docs]. + +_Note:_ All of the below is explained in more detail in the official [Vuex documentation][vuex-docs]. + +## Separation of concerns +Vuex is composed of State, Getters, Mutations, Actions and Modules. + +When a user clicks on an action, we need to `dispatch` it. This action will `commit` a mutation that will change the state. +_Note:_ The action itself will not update the state, only a mutation should update the state. + +## File structure +When using Vuex at GitLab, separate this concerns into different files to improve readability: + +``` +└── store + ├── index.js # where we assemble modules and export the store + ├── actions.js # actions + ├── mutations.js # mutations + ├── getters.js # getters + ├── state.js # state + └── mutation_types.js # mutation types +``` +The following example shows an application that lists and adds users to the state. +(For a more complex example implementation take a look at the security applications store in [here](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/ee/app/assets/javascripts/vue_shared/security_reports/store)) + +### `index.js` +This is the entry point for our store. You can use the following as a guide: + +```javascript +import Vue from 'vue'; +import Vuex from 'vuex'; +import * as actions from './actions'; +import * as getters from './getters'; +import mutations from './mutations'; +import state from './state'; + +Vue.use(Vuex); + +export const createStore = () => new Vuex.Store({ + actions, + getters, + mutations, + state, +}); +export default createStore(); +``` + +### `state.js` +The first thing you should do before writing any code is to design the state. + +Often we need to provide data from haml to our Vue application. Let's store it in the state for better access. + +```javascript + export default { + endpoint: null, + + isLoading: false, + error: null, + + isAddingUser: false, + errorAddingUser: false, + + users: [], + }; +``` + +#### Access `state` properties +You can use `mapState` to access state properties in the components. + +### `actions.js` +An action is a payload of information to send data from our application to our store. + +An action is usually composed by a `type` and a `payload` and they describe what happened. +Enforcing that every change is described as an action lets us have a clear understanding of what is going on in the app. + +In this file, we will write the actions that will call the respective mutations: + +```javascript + import * as types from './mutation_types'; + import axios from '~/lib/utils/axios-utils'; + import createFlash from '~/flash'; + + export const requestUsers = ({ commit }) => commit(types.REQUEST_USERS); + export const receiveUsersSuccess = ({ commit }, data) => commit(types.RECEIVE_USERS_SUCCESS, data); + export const receiveUsersError = ({ commit }, error) => commit(types.REQUEST_USERS_ERROR, error); + + export const fetchUsers = ({ state, dispatch }) => { + dispatch('requestUsers'); + + axios.get(state.endpoint) + .then(({ data }) => dispatch('receiveUsersSuccess', data)) + .catch((error) => { + dispatch('receiveUsersError', error) + createFlash('There was an error') + }); + } + + export const requestAddUser = ({ commit }) => commit(types.REQUEST_ADD_USER); + export const receiveAddUserSuccess = ({ commit }, data) => commit(types.RECEIVE_ADD_USER_SUCCESS, data); + export const receiveAddUserError = ({ commit }, error) => commit(types.REQUEST_ADD_USER_ERROR, error); + + export const addUser = ({ state, dispatch }, user) => { + dispatch('requestAddUser'); + + axios.post(state.endpoint, user) + .then(({ data }) => dispatch('receiveAddUserSuccess', data)) + .catch((error) => dispatch('receiveAddUserError', error)); + } +``` + +#### Actions Pattern: `request` and `receive` namespaces +When a request is made we often want to show a loading state to the user. + +Instead of creating an action to toggle the loading state and dispatch it in the component, +create: +1. An action `requestSomething`, to toggle the loading state +1. An action `receiveSomethingSuccess`, to handle the success callback +1. An action `receiveSomethingError`, to handle the error callback +1. An action `fetchSomething` to make the request. + 1. In case your application does more than a `GET` request you can use these as examples: + 1. `PUT`: `createSomething` + 2. `POST`: `updateSomething` + 3. `DELETE`: `deleteSomething` + +The component MUST only dispatch the `fetchNamespace` action. Actions namespaced with `request` or `receive` should not be called from the component +The `fetch` action will be responsible to dispatch `requestNamespace`, `receiveNamespaceSuccess` and `receiveNamespaceError` + +By following this pattern we guarantee: +1. All applications follow the same pattern, making it easier for anyone to maintain the code +1. All data in the application follows the same lifecycle pattern +1. Actions are contained and human friendly +1. Unit tests are easier +1. Actions are simple and straightforward + +#### Dispatching actions +To dispatch an action from a component, use the `mapActions` helper: +```javascript +import { mapActions } from 'vuex'; + +{ + methods: { + ...mapActions([ + 'addUser', + ]), + onClickUser(user) { + this.addUser(user); + }, + }, +}; +``` + +### `mutations.js` +The mutations specify how the application state changes in response to actions sent to the store. +The only way to change state in a Vuex store should be by committing a mutation. + +**It's a good idea to think of the state before writing any code.** + +Remember that actions only describe that something happened, they don't describe how the application state changes. + +**Never commit a mutation directly from a component** + +```javascript + import * as types from './mutation_types'; + + export default { + [types.REQUEST_USERS](state) { + state.isLoading = true; + }, + [types.RECEIVE_USERS_SUCCESS](state, data) { + // Do any needed data transformation to the received payload here + state.users = data; + state.isLoading = false; + }, + [types.REQUEST_USERS_ERROR](state, error) { + state.isLoading = false; + }, + [types.REQUEST_ADD_USER](state, user) { + state.isAddingUser = true; + }, + [types.RECEIVE_ADD_USER_SUCCESS](state, user) { + state.isAddingUser = false; + state.users.push(user); + }, + [types.REQUEST_ADD_USER_ERROR](state, error) { + state.isAddingUser = true; + state.errorAddingUser = error; + }, + }; +``` + +### `getters.js` +Sometimes we may need to get derived state based on store state, like filtering for a specific prop. +Using a getter will also cache the result based on dependencies due to [how computed props work](https://vuejs.org/v2/guide/computed.html#Computed-Caching-vs-Methods) +This can be done through the `getters`: + +```javascript +// get all the users with pets +export const getUsersWithPets = (state, getters) => { + return state.users.filter(user => user.pet !== undefined); +}; +``` + +To access a getter from a component, use the `mapGetters` helper: +```javascript +import { mapGetters } from 'vuex'; + +{ + computed: { + ...mapGetters([ + 'getUsersWithPets', + ]), + }, +}; +``` + +### `mutations_types.js` +From [vuex mutations docs][vuex-mutations]: +> It is a commonly seen pattern to use constants for mutation types in various Flux implementations. This allows the code to take advantage of tooling like linters, and putting all constants in a single file allows your collaborators to get an at-a-glance view of what mutations are possible in the entire application. + +```javascript +export const ADD_USER = 'ADD_USER'; +``` + +### How to include the store in your application +The store should be included in the main component of your application: +```javascript + // app.vue + import store from 'store'; // it will include the index.js file + + export default { + name: 'application', + store, + ... + }; +``` + +### Communicating with the Store +```javascript +<script> +import { mapActions, mapState, mapGetters } from 'vuex'; +import store from './store'; + +export default { + store, + computed: { + ...mapGetters([ + 'getUsersWithPets' + ]), + ...mapState([ + 'isLoading', + 'users', + 'error', + ]), + }, + methods: { + ...mapActions([ + 'fetchUsers', + 'addUser', + ]), + + onClickAddUser(data) { + this.addUser(data); + } + }, + + created() { + this.fetchUsers() + } +} +</script> +<template> + <ul> + <li v-if="isLoading"> + Loading... + </li> + <li v-else-if="error"> + {{ error }} + </li> + <template v-else> + <li + v-for="user in users" + :key="user.id" + > + {{ user }} + </li> + </template> + </ul> +</template> +``` + +### Vuex Gotchas +1. Do not call a mutation directly. Always use an action to commit a mutation. Doing so will keep consistency throughout the application. From Vuex docs: + + > why don't we just call store.commit('action') directly? Well, remember that mutations must be synchronous? Actions aren't. We can perform asynchronous operations inside an action. + + ```javascript + // component.vue + + // bad + created() { + this.$store.commit('mutation'); + } + + // good + created() { + this.$store.dispatch('action'); + } + ``` +1. Use mutation types instead of hardcoding strings. It will be less error prone. +1. The State will be accessible in all components descending from the use where the store is instantiated. + +### Testing Vuex +#### Testing Vuex concerns +Refer to [vuex docs][vuex-testing] regarding testing Actions, Getters and Mutations. + +#### Testing components that need a store +Smaller components might use `store` properties to access the data. +In order to write unit tests for those components, we need to include the store and provide the correct state: + +```javascript +//component_spec.js +import Vue from 'vue'; +import { createStore } from './store'; +import component from './component.vue' + +describe('component', () => { + let store; + let vm; + let Component; + + beforeEach(() => { + Component = Vue.extend(issueActions); + }); + + afterEach(() => { + vm.$destroy(); + }); + + it('should show a user', () => { + const user = { + name: 'Foo', + age: '30', + }; + + store = createStore(); + + // populate the store + store.dispatch('addUser', user); + + vm = new Component({ + store, + propsData: props, + }).$mount(); + }); +}); +``` + +#### Testing Vuex actions and getters +Because we're currently using [`babel-plugin-rewire`](https://github.com/speedskater/babel-plugin-rewire), you may encounter the following error when testing your Vuex actions and getters: +`[vuex] actions should be function or object with "handler" function` + +To prevent this error from happening, you need to export an empty function as `default`: +``` +// getters.js or actions.js + +// prevent babel-plugin-rewire from generating an invalid default during karma tests +export default () => {}; +``` + +[vuex-docs]: https://vuex.vuejs.org +[vuex-structure]: https://vuex.vuejs.org/en/structure.html +[vuex-mutations]: https://vuex.vuejs.org/en/mutations.html +[vuex-testing]: https://vuex.vuejs.org/en/testing.html diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 0edcb23c7c5..4ba9958e2c6 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -233,7 +233,7 @@ This makes use of [`Intl.DateTimeFormat`]. Please never split a sentence as that would assume the sentence grammar and structure is the same in all languages. -For instance, the following +For instance, the following: ```js {{ s__("mrWidget|Set by") }} @@ -247,6 +247,27 @@ should be externalized as follows: {{ sprintf(s__("mrWidget|Set by %{author} to be merged automatically when the pipeline succeeds"), { author: author.name }) }} ``` +#### Avoid splitting sentences when adding links + +This also applies when using links in between translated sentences, otherwise these texts are not translatable in certain languages. + +Instead of: + +```haml +- zones_link = link_to(s_('ClusterIntegration|zones'), 'https://cloud.google.com/compute/docs/regions-zones/regions-zones', target: '_blank', rel: 'noopener noreferrer') += s_('ClusterIntegration|Learn more about %{zones_link}').html_safe % { zones_link: zones_link } +``` + +Set the link starting and ending HTML fragments as variables like so: + +```haml +- zones_link_url = 'https://cloud.google.com/compute/docs/regions-zones/regions-zones' +- zones_link_start = '<a href="%{url}" target="_blank" rel="noopener noreferrer">'.html_safe % { url: zones_link_url } += s_('ClusterIntegration|Learn more about %{zones_link_start}zones%{zones_link_end}').html_safe % { zones_link_start: zones_link_start, zones_link_end: '</a>'.html_safe } +``` + +The reasoning behind this is that in some languages words change depending on context. For example in Japanese は is added to the subject of a sentence and を to the object. This is impossible to translate correctly if we extract individual words from the sentence. + When in doubt, try to follow the best practices described in this [Mozilla Developer documentation][mdn]. diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index 5185d843ccb..9a677bf09b2 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -16,7 +16,7 @@ are very appreciative of the work done by translators and proofreaders! - Dutch - Esperanto - French - - Rémy Coutable - [GitLab](https://gitlab.com/rymai), [Crowdin](https://crowdin.com/profile/rymai) + - Davy Defaud - [GitLab](https://gitlab.com/DevDef), [Crowdin](https://crowdin.com/profile/DevDef) - German - Indonesian - Ahmad Naufal Mukhtar - [GitLab](https://gitlab.com/anaufalm), [Crowdin](https://crowdin.com/profile/anaufalm) diff --git a/doc/development/new_fe_guide/dependencies.md b/doc/development/new_fe_guide/dependencies.md index 3417d77a06d..12a4f089d41 100644 --- a/doc/development/new_fe_guide/dependencies.md +++ b/doc/development/new_fe_guide/dependencies.md @@ -1,3 +1,20 @@ # Dependencies -> TODO: Add Dependencies
\ No newline at end of file +## Adding Dependencies. + +GitLab uses `yarn` to manage dependencies. These dependencies are defined in +two groups within `package.json`, `dependencies` and `devDependencies`. For +our purposes, we consider anything that is required to compile our production +assets a "production" dependency. That is, anything required to run the +`webpack` script with `NODE_ENV=production`. Tools like `eslint`, `karma`, and +various plugins and tools used in development are considered `devDependencies`. +This distinction is used by omnibus to determine which dependencies it requires +when building GitLab. + +Exceptions are made for some tools that we require in the +`gitlab:assets:compile` CI job such as `webpack-bundle-analyzer` to analyze our +production assets post-compile. + +--- + +> TODO: Add Dependencies diff --git a/doc/development/new_fe_guide/development/accessibility.md b/doc/development/new_fe_guide/development/accessibility.md index ed35f08432f..2a3a126ca5c 100644 --- a/doc/development/new_fe_guide/development/accessibility.md +++ b/doc/development/new_fe_guide/development/accessibility.md @@ -1,3 +1,48 @@ -# Accessibility +# Accessiblity +Using semantic HTML plays a key role when it comes to accessibility. -> TODO: Add content +## Accessible Rich Internet Applications - ARIA +WAI-ARIA, the Accessible Rich Internet Applications specification, defines a way to make Web content and Web applications more accessible to people with disabilities. + +> Note: It is [recommended][using-aria] to use semantic elements as the primary method to achieve accessibility rather than adding aria attributes. Adding aria attributes should be seen as a secondary method for creating accessible elements. + +### Role +The `role` attribute describes the role the element plays in the context of the document. + +Check the list of WAI-ARIA roles [here][roles] + +## Icons +When using icons or images that aren't absolutely needed to understand the context, we should use `aria-hidden="true"`. + +On the other hand, if an icon is crucial to understand the context we should do one of the following: +1. Use `aria-label` in the element with a meaningful description +1. Use `aria-labelledby` to point to an element that contains the explanation for that icon + +## Form inputs +In forms we should use the `for` attribute in the label statement: +``` +<div> + <label for="name">Fill in your name:</label> + <input type="text" id="name" name="name"> +</div> +``` + +## Testing + +1. On MacOS you can use [VoiceOver][voice-over] by pressing `cmd+F5`. +1. On Windows you can use [Narrator][narrator] by pressing Windows logo key + Ctrl + Enter. + +## Online resources + +- [Chrome Accessibility Developer Tools][dev-tools] for testing accessibility +- [Audit Rules Page][audit-rules] for best practices +- [Lighthouse Accessibility Score][lighthouse] for accessibility audits + +[using-aria]: https://www.w3.org/TR/using-aria/#notes2 +[dev-tools]: https://github.com/GoogleChrome/accessibility-developer-tools +[audit-rules]: https://github.com/GoogleChrome/accessibility-developer-tools/wiki/Audit-Rules +[aria-w3c]: https://www.w3.org/TR/wai-aria-1.1/ +[roles]: https://www.w3.org/TR/wai-aria-1.1/#landmark_roles +[voice-over]: https://www.apple.com/accessibility/mac/vision/ +[narrator]: https://www.microsoft.com/en-us/accessibility/windows +[lighthouse]: https://developers.google.com/web/tools/lighthouse/scoring#a11y diff --git a/doc/development/new_fe_guide/development/testing.md b/doc/development/new_fe_guide/development/testing.md index c359bd83ed1..e1e13474b75 100644 --- a/doc/development/new_fe_guide/development/testing.md +++ b/doc/development/new_fe_guide/development/testing.md @@ -1,3 +1,135 @@ -# Testing +# Overview of Frontend Testing -> TODO: Add content +## Types of tests in our codebase + +* **RSpec** + * **[Ruby unit tests](#ruby-unit-tests-spec-rb)** for models, controllers, helpers, etc. (`/spec/**/*.rb`) + * **[Full feature tests](#full-feature-tests-spec-features-rb)** (`/spec/features/**/*.rb`) +* **[Karma](#karma-tests-spec-javascripts-js)** (`/spec/javascripts/**/*.js`) +* ~~Spinach~~ — These have been removed from our codebase in May 2018. (`/features/`) + +## RSpec: Ruby unit tests `/spec/**/*.rb` + +These tests are meant to unit test the ruby models, controllers and helpers. + +### When do we write/update these tests? + +Whenever we create or modify any Ruby models, controllers or helpers we add/update corresponding tests. + +--- + +## RSpec: Full feature tests `/spec/features/**/*.rb` + +Full feature tests will load a full app environment and allow us to test things like rendering DOM, interacting with links and buttons, testing the outcome of those interactions through multiple pages if necessary. These are also called end-to-end tests but should not be confused with QA end-to-end tests (`package-and-qa` manual pipeline job). + +### When do we write/update these tests? + +When we add a new feature, we write at least two tests covering the success and the failure scenarios. + +### Relevant notes + +A `:js` flag is added to the test to make sure the full environment is loaded. + +``` +scenario 'successfully', :js do + sign_in(create(:admin)) +end +``` + +The steps of each test are written using capybara methods ([documentation](http://www.rubydoc.info/gems/capybara/2.15.1)). + +Bear in mind <abbr title="XMLHttpRequest">XHR</abbr> calls might require you to use `wait_for_requests` in between steps, like so: + +```rspec +find('.form-control').native.send_keys(:enter) + +wait_for_requests + +expect(page).not_to have_selector('.card') +``` + +--- + +## Karma tests `/spec/javascripts/**/*.js` + +These are the more frontend-focused, at the moment. They're **faster** than `rspec` and make for very quick testing of frontend components. + +### When do we write/update these tests? + +When we add/update a method/action/mutation to Vue or Vuex, we write karma tests to ensure the logic we wrote doesn't break. We should, however, refrain from writing tests that double-test Vue's internal features. + +### Relevant notes + +Karma tests are run against a virtual DOM. + +To populate the DOM, we can use fixtures to fake the generation of HTML instead of having Rails do that. + +Be sure to check the [best practices for karma tests](../../testing_guide/frontend_testing.html#best-practices). + +### Vue and Vuex + +Test as much as possible without double-testing Vue's internal features, as mentioned above. + +Make sure to test computedProperties, mutations, actions. Run the action and test that the proper mutations are committed. + +Also check these [notes on testing Vue components](../../fe_guide/vue.html#testing-vue-components). + +#### Vuex Helper: `testAction` + +We have a helper available to make testing actions easier, as per [official documentation](https://vuex.vuejs.org/en/testing.html): + +``` +testAction( + actions.actionName, // action + { }, // params to be passed to action + state, // state + [ + { type: types.MUTATION}, + { type: types.MUTATION_1, payload: {}}, + ], // mutations committed + [ + { type: 'actionName', payload: {}}, + { type: 'actionName1', payload: {}}, + ] // actions dispatched + done, +); +``` + +Check an example in [spec/javascripts/ide/stores/actions_spec.jsspec/javascripts/ide/stores/actions_spec.js](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/javascripts/ide/stores/actions_spec.js). + +#### Vue Helper: `mountComponent` + +To make mounting a Vue component easier and more readable, we have a few helpers available in `spec/helpers/vue_mount_component_helper`. + +* `createComponentWithStore` +* `mountComponentWithStore` + +Examples of usage: + +``` +beforeEach(() => { + vm = createComponentWithStore(Component, store); + + vm.$store.state.currentBranchId = 'master'; + + vm.$mount(); +}, +``` + +``` +beforeEach(() => { + vm = mountComponentWithStore(Component, { + el: '#dummy-element', + store, + props: { badge }, + }); +}, +``` + +Don't forget to clean up: + +``` +afterEach(() => { + vm.$destroy(); +}); +``` diff --git a/doc/development/new_fe_guide/tips.md b/doc/development/new_fe_guide/tips.md index f0cdf52d618..881ad1662ae 100644 --- a/doc/development/new_fe_guide/tips.md +++ b/doc/development/new_fe_guide/tips.md @@ -1,3 +1,9 @@ # Tips -> TODO: Add tips +## Clearing production compiled assets + +To clear production compiled assets created with `yarn webpack-prod` you can run: + +``` +yarn clean +``` diff --git a/doc/development/query_recorder.md b/doc/development/query_recorder.md index 12e90101139..61e5e1afede 100644 --- a/doc/development/query_recorder.md +++ b/doc/development/query_recorder.md @@ -2,7 +2,7 @@ QueryRecorder is a tool for detecting the [N+1 queries problem](http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations) from tests. -> Implemented in [spec/support/query_recorder.rb](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/support/query_recorder.rb) via [9c623e3e](https://gitlab.com/gitlab-org/gitlab-ce/commit/9c623e3e5d7434f2e30f7c389d13e5af4ede770a) +> Implemented in [spec/support/query_recorder.rb](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/support/helpers/query_recorder.rb) via [9c623e3e](https://gitlab.com/gitlab-org/gitlab-ce/commit/9c623e3e5d7434f2e30f7c389d13e5af4ede770a) As a rule, merge requests [should not increase query counts](merge_request_performance_guidelines.md#query-counts). If you find yourself adding something like `.includes(:author, :assignee)` to avoid having `N+1` queries, consider using QueryRecorder to enforce this with a test. Without this, a new feature which causes an additional model to be accessed will silently reintroduce the problem. @@ -22,6 +22,19 @@ As an example you might create 5 issues in between counts, which would cause the > **Note:** In some cases the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible. +## Cached queries + +By default, QueryRecorder will ignore cached queries in the count. However, it may be better to count +all queries to avoid introducing an N+1 query that may be masked by the statement cache. To do this, +pass the `skip_cached` variable to `QueryRecorder` and use the `exceed_all_query_limit` matcher: + +it "avoids N+1 database queries" do + control_count = ActiveRecord::QueryRecorder.new(skip_cached: false) { visit_some_page }.count + create_list(:issue, 5) + expect { visit_some_page }.not_to exceed_all_query_limit(control_count) +end +``` + ## Finding the source of the query It may be useful to identify the source of the queries by looking at the call backtrace. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index fdfa1f10402..fc51b74da1d 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -65,12 +65,11 @@ To make sure that indices still fit. You could find great details in: ## Run tests In order to run the test you can use the following commands: -- `rake spinach` to run the spinach suite - `rake spec` to run the rspec suite - `rake karma` to run the karma test suite - `rake gitlab:test` to run all the tests -Note: Both `rake spinach` and `rake spec` takes significant time to pass. +Note: `rake spec` takes significant time to pass. Instead of running full test suite locally you can save a lot of time by running a single test or directory related to your changes. After you submit merge request CI will run full test suite for you. Green CI status in the merge request means @@ -82,12 +81,10 @@ files it can find, also the ones in `/tmp` To run a single test file you can use: - `bin/rspec spec/controllers/commit_controller_spec.rb` for a rspec test -- `bin/spinach features/project/issues/milestones.feature` for a spinach test To run several tests inside one directory: - `bin/rspec spec/requests/api/` for the rspec tests if you want to test API only -- `bin/spinach features/profile/` for the spinach tests if you want to test only profile pages ### Speed-up tests, rake tasks, and migrations @@ -179,3 +176,20 @@ git push -u origin update-project-templates ``` Now create a merge request and merge that to master. + +## Generate route lists + +To see the full list of API routes, you can run: + +```shell +bundle exec rake grape:path_helpers +``` + +For the Rails controllers, run: + +```shell +bundle exec rake routes +``` + +Since these take some time to create, it's often helpful to save the output to +a file for quick reference. diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 7b32e0a47e1..1a926a660f1 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -12,8 +12,7 @@ Here are some things to keep in mind regarding test performance: - `FactoryBot.build(...)` and `.build_stubbed` are faster than `.create`. - Don't `create` an object when `build`, `build_stubbed`, `attributes_for`, `spy`, or `double` will do. Database persistence is slow! -- Don't mark a feature as requiring JavaScript (through `@javascript` in - Spinach or `:js` in RSpec) unless it's _actually_ required for the test +- Don't mark a feature as requiring JavaScript (through `:js` in RSpec) unless it's _actually_ required for the test to be valid. Headless browser testing is slow! [parallelization]: ci.md#test-suite-parallelization-on-the-ci @@ -121,6 +120,10 @@ Add `screenshot_and_save_page` in a `:js` spec to screenshot what Capybara Add `screenshot_and_open_image` in a `:js` spec to screenshot what Capybara "sees", and automatically open the image. +The HTML dumps created by this are missing CSS. +This results in them looking very different from the actual application. +There is a [small hack](https://gitlab.com/gitlab-org/gitlab-ce/snippets/1718469) to add CSS which makes debugging easier. + ### Fast unit tests Some classes are well-isolated from Rails and you should be able to test them @@ -134,11 +137,24 @@ really fast since: - gitlab-shell and Gitaly setup are skipped - Test repositories setup are skipped -Note that in some cases, you might have to add some `require_dependency 'foo'` -in your file under test since Rails autoloading is not available in these cases. +`fast_spec_helper` also support autoloading classes that are located inside the +`lib/` directory. It means that as long as your class / module is using only +code from the `lib/` directory you will not need to explicitly load any +dependencies. `fast_spec_helper` also loads all ActiveSupport extensions, +including core extensions that are commonly used in the Rails environment. + +Note that in some cases, you might still have to load some dependencies using +`require_dependency` when a code is using gems or a dependency is not located +in `lib/`. + +For example, if you want to test your code that is calling the +`Gitlab::UntrustedRegexp` class, which under the hood uses `re2` library, you +should either add `require_dependency 're2'` to files in your library that +need `re2` gem, to make this requirement explicit, or you can add it to the +spec itself, but the former is preferred. -This shouldn't be a problem since explicitely listing dependencies should be -considered a good practice anyway. +It takes around one second to load tests that are using `fast_spec_helper` +instead of 30+ seconds in case of a regular `spec_helper`. ### `let` variables @@ -230,6 +246,11 @@ describe "#==" do end ``` +### Prometheus tests + +Prometheus metrics may be preserved from one test run to another. To ensure that metrics are +reset before each example, add the `:prometheus` tag to the Rspec test. + ### Matchers Custom matchers should be created to clarify the intent and/or hide the diff --git a/doc/development/testing_guide/ci.md b/doc/development/testing_guide/ci.md index e90de55068d..0d8e150e090 100644 --- a/doc/development/testing_guide/ci.md +++ b/doc/development/testing_guide/ci.md @@ -24,8 +24,7 @@ Our current CI parallelization setup is as follows: uploaded to S3. After that, the next pipeline will use the up-to-date -`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. The same strategy -is used for Spinach tests as well. +`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. ### Monitoring diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 0d0d511582b..3b2b9c8c947 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -280,26 +280,6 @@ describe "Admin::AbuseReports", :js do end ``` -### Spinach errors due to missing JavaScript - -NOTE: **Note:** Since we are discouraging the use of Spinach when writing new -feature tests, you shouldn't ever need to use this. This information is kept -available for legacy purposes only. - -In Spinach, the JavaScript driver is enabled differently. In the `*.feature` -file for the failing spec, add the `@javascript` flag above the Scenario: - -``` -@javascript -Scenario: Developer can approve merge request - Given I am a "Shop" developer - And I visit project "Shop" merge requests page - And merge request 'Bug NS-04' must be approved - And I click link "Bug NS-04" - When I click link "Approve" - Then I should see approved merge request "Bug NS-04" -``` - [jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html [jasmine-jquery]: https://github.com/velesin/jasmine-jquery [karma]: http://karma-runner.github.io/ diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 74d09eb91ff..0cd63a54b55 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -72,21 +72,6 @@ Everything you should know about how to run end-to-end tests using --- -## Spinach (feature) tests - -GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426) -for its feature/integration tests in September 2012. - -As of March 2016, we are [trying to avoid adding new Spinach -tests](https://gitlab.com/gitlab-org/gitlab-ce/issues/14121) going forward, -opting for [RSpec feature](#features-integration) specs. - -Adding new Spinach scenarios is acceptable _only if_ the new scenario requires -no more than one new `step` definition. If more than that is required, the -test should be re-implemented using RSpec instead. - ---- - [Return to Development documentation](../README.md) [^1]: /ci/yaml/README.html#dependencies diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 51794f7f4df..07ced36f0c1 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -81,7 +81,6 @@ possible). | Tests path | Testing engine | Notes | | ---------- | -------------- | ----- | | `spec/features/` | [Capybara] + [RSpec] | If your spec has the `:js` metadata, the browser driver will be [Poltergeist], otherwise it's using [RackTest]. | -| `features/` | Spinach | Spinach tests are deprecated, [you shouldn't add new Spinach tests](#spinach-feature-tests). | ### Consider **not** writing a system test! diff --git a/doc/development/ux_guide/components.md b/doc/development/ux_guide/components.md index b57520a00e0..4a3b3125f59 100644 --- a/doc/development/ux_guide/components.md +++ b/doc/development/ux_guide/components.md @@ -193,7 +193,7 @@ List with avatar, title and description using .content-list  -List with hover effect .well-list +List with hover effect .content-list  diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 9bca4637830..038a4b1e6ea 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -1,541 +1,3 @@ -# GitLab Documentation guidelines - - - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. - - **[Technical Articles](#technical-articles)**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). - - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). - -## Contributing to docs - -Whenever a feature is changed, updated, introduced, or deprecated, the merge -request introducing these changes must be accompanied by the documentation -(either updating existing ones or creating new ones). This is also valid when -changes are introduced to the UI. - -The one responsible for writing the first piece of documentation is the developer who -wrote the code. It's the job of the Product Manager to ensure all features are -shipped with its docs, whether is a small or big change. At the pace GitLab evolves, -this is the only way to keep the docs up-to-date. If you have any questions about it, -ask a Technical Writer. Otherwise, when your content is ready, assign one of -them to review it for you. - -We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything -is documented. - -Whenever you submit a merge request for the documentation, use the documentation MR description template. - -Please check the [documentation workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) before getting started. - -## Documentation structure - -- Overview and use cases: what it is, why it is necessary, why one would use it -- Requirements: what do we need to get started -- Tutorial: how to set it up, how to use it - -Always link a new document from its topic-related index, otherwise, it will -not be included it in the documentation site search. - -_Note: to be extended._ - -### Feature overview and use cases - -Every major feature (regardless if present in GitLab Community or Enterprise editions) -should present, at the beginning of the document, two main sections: **overview** and -**use cases**. Every GitLab EE-only feature should also contain these sections. - -**Overview**: as the name suggests, the goal here is to provide an overview of the feature. -Describe what is it, what it does, why it is important/cool/nice-to-have, -what problem it solves, and what you can do with this feature that you couldn't -do before. - -**Use cases**: provide at least two, ideally three, use cases for every major feature. -You should answer this question: what can you do with this feature/change? Use cases -are examples of how this feature or change can be used in real life. - -Examples: -- CE and EE: [Issues](../user/project/issues/index.md#use-cases) -- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) -- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) -- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) - -Note that if you don't have anything to add between the doc title (`<h1>`) and -the header `## Overview`, you can omit the header, but keep the content of the -overview there. - -> **Overview** and **use cases** are required to **every** Enterprise Edition feature, -and for every **major** feature present in Community Edition. - -## Markdown and styles - -Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. - -All the docs follow the [documentation style guidelines](doc_styleguide.md). - -## Documentation directory structure - -The documentation is structured based on the GitLab UI structure itself, -separated by [`user`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/user), -[`administrator`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/administration), and [`contributor`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/development). - -In order to have a [solid site structure](https://searchengineland.com/seo-benefits-developing-solid-site-structure-277456) for our documentation, -all docs should be linked. Every new document should be cross-linked to its related documentation, and linked from its topic-related index, when existent. - -The directories `/workflow/`, `/gitlab-basics/`, `/university/`, and `/articles/` have -been deprecated and the majority their docs have been moved to their correct location -in small iterations. Please don't create new docs in these folders. - -### Location and naming documents - -The documentation hierarchy can be vastly improved by providing a better layout -and organization of directories. - -Having a structured document layout, we will be able to have meaningful URLs -like `docs.gitlab.com/user/project/merge_requests/index.html`. With this pattern, -you can immediately tell that you are navigating a user related documentation -and is about the project and its merge requests. - -Do not create summaries of similar types of content (e.g. an index of all articles, videos, etc.), -rather organize content by its subject (e.g. everything related to CI goes together) -and cross-link between any related content. - -The table below shows what kind of documentation goes where. - -| Directory | What belongs here | -| --------- | -------------- | -| `doc/user/` | User related documentation. Anything that can be done within the GitLab UI goes here including `/admin`. | -| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. The admin settings that can be accessed via GitLab's interface go under `doc/user/admin_area/`. | -| `doc/api/` | API related documentation. | -| `doc/development/` | Documentation related to the development of GitLab. Any styleguides should go here. | -| `doc/legal/` | Legal documents about contributing to GitLab. | -| `doc/install/`| Probably the most visited directory, since `installation.md` is there. Ideally this should go under `doc/administration/`, but it's best to leave it as-is in order to avoid confusion (still debated though). | -| `doc/update/` | Same with `doc/install/`. Should be under `administration/`, but this is a well known location, better leave as-is, at least for now. | -| `doc/topics/` | Indexes per Topic (`doc/topics/topic-name/index.md`): all resources for that topic (user and admin documentation, articles, and third-party docs) | - --- - -**General rules:** - -1. The correct naming and location of a new document, is a combination - of the relative URL of the document in question and the GitLab Map design - that is used for UX purposes ([source][graffle], [image][gitlab-map]). -1. When creating a new document and it has more than one word in its name, - make sure to use underscores instead of spaces or dashes (`-`). For example, - a proper naming would be `import_projects_from_github.md`. The same rule - applies to images. -1. Start a new directory with an `index.md` file. -1. There are four main directories, `user`, `administration`, `api` and `development`. -1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, - `profile/`, `dashboard/` and `admin_area/`. - 1. `doc/user/project/` should contain all project related documentation. - 1. `doc/user/group/` should contain all group related documentation. - 1. `doc/user/profile/` should contain all profile related documentation. - Every page you would navigate under `/profile` should have its own document, - i.e. `account.md`, `applications.md`, `emails.md`, etc. - 1. `doc/user/dashboard/` should contain all dashboard related documentation. - 1. `doc/user/admin_area/` should contain all admin related documentation - describing what can be achieved by accessing GitLab's admin interface - (_not to be confused with `doc/administration` where server access is - required_). - 1. Every category under `/admin/application_settings` should have its - own document located at `doc/user/admin_area/settings/`. For example, - the **Visibility and Access Controls** category should have a document - located at `doc/user/admin_area/settings/visibility_and_access_controls.md`. -1. The `doc/topics/` directory holds topic-related technical content. Create - `doc/topics/topic-name/subtopic-name/index.md` when subtopics become necessary. - General user- and admin- related documentation, should be placed accordingly. - -If you are unsure where a document should live, you can ping `@axil` or `@marcia` in your -merge request. - -### Changing document location - -Changing a document's location is not to be taken lightly. Remember that the -documentation is available to all installations under `help/` and not only to -GitLab.com or http://docs.gitlab.com. Make sure this is discussed with the -Documentation team beforehand. - -If you indeed need to change a document's location, do NOT remove the old -document, but rather replace all of its contents with a new line: - -``` -This document was moved to [another location](path/to/new_doc.md). -``` - -where `path/to/new_doc.md` is the relative path to the root directory `doc/`. - +redirect_to: 'documentation/index.md' --- - -For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to -`doc/administration/lfs.md`, then the steps would be: - -1. Copy `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md` -1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with: - - ``` - This document was moved to [another location](../../administration/lfs.md). - ``` - -1. Find and replace any occurrences of the old location with the new one. - A quick way to find them is to use `git grep`. First go to the root directory - where you cloned the `gitlab-ce` repository and then do: - - ``` - git grep -n "workflow/lfs/lfs_administration" - git grep -n "lfs/lfs_administration" - ``` - -NOTE: **Note:** -If the document being moved has any Disqus comments on it, there are extra steps -to follow documented just [below](#redirections-for-pages-with-disqus-comments). - -Things to note: - -- Since we also use inline documentation, except for the documentation itself, - the document might also be referenced in the views of GitLab (`app/`) which will - render when visiting `/help`, and sometimes in the testing suite (`spec/`). -- The above `git grep` command will search recursively in the directory you run - it in for `workflow/lfs/lfs_administration` and `lfs/lfs_administration` - and will print the file and the line where this file is mentioned. - You may ask why the two greps. Since we use relative paths to link to - documentation, sometimes it might be useful to search a path deeper. -- The `*.md` extension is not used when a document is linked to GitLab's - built-in help page, that's why we omit it in `git grep`. -- Use the checklist on the documentation MR description template. - -### Redirections for pages with Disqus comments - -If the documentation page being relocated already has any Disqus comments, -we need to preserve the Disqus thread. - -Disqus uses an identifier per page, and for docs.gitlab.com, the page identifier -is configured to be the page URL. Therefore, when we change the document location, -we need to preserve the old URL as the same Disqus identifier. - -To do that, add to the frontmatter the variable `redirect_from`, -using the old URL as value. For example, let's say I moved the document -available under `https://docs.gitlab.com/my-old-location/README.html` to a new location, -`https://docs.gitlab.com/my-new-location/index.html`. - -Into the **new document** frontmatter add the following: - -```yaml ---- -redirect_from: 'https://docs.gitlab.com/my-old-location/README.html' ---- -``` - -Note: it is necessary to include the file name in the `redirect_from` URL, -even if it's `index.html` or `README.html`. - -## Testing - -We treat documentation as code, thus have implemented some testing. -Currently, the following tests are in place: - -1. `docs lint`: Check that all internal (relative) links work correctly and - that all cURL examples in API docs use the full switches. It's recommended - to [check locally](#previewing-locally) before pushing to GitLab by executing the command - `bundle exec nanoc check internal_links` on your local - [`gitlab-docs`](https://gitlab.com/gitlab-com/gitlab-docs) directory. -1. [`ee_compat_check`](https://docs.gitlab.com/ee/development/automatic_ce_ee_merge.html#avoiding-ce-gt-ee-merge-conflicts-beforehand) (runs on CE only): - When you submit a merge request to GitLab Community Edition (CE), - there is this additional job that runs against Enterprise Edition (EE) - and checks if your changes can apply cleanly to the EE codebase. - If that job fails, read the instructions in the job log for what to do next. - As CE is merged into EE once a day, it's important to avoid merge conflicts. - Submitting an EE-equivalent merge request cherry-picking all commits from CE to EE is - essential to avoid them. - -## Branch naming - -If your contribution contains **only** documentation changes, you can speed up -the CI process by following some branch naming conventions. You have three -choices: - -| Branch name | Valid example | -| ----------- | ------------- | -| Starting with `docs/` | `docs/update-api-issues` | -| Starting with `docs-` | `docs-update-api-issues` | -| Ending in `-docs` | `123-update-api-issues-docs` | - -If your branch name matches any of the above, it will run only the docs -tests. If it doesn't, the whole test suite will run (including docs). - -## Merge requests for GitLab documentation - -Before getting started, make sure you read the introductory section -"[contributing to docs](#contributing-to-docs)" above and the -[tech writing workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) -for GitLab Team members. - -- Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/merge_request_templates/Documentation.md) -- Use the correct [branch name](#branch-naming) -- Label the MR `Documentation` -- Assign the correct milestone (see note below) - - -NOTE: **Note:** -If the release version you want to add the documentation to has already been -frozen or released, use the label `Pick into X.Y` to get it merged into -the correct release. Avoid picking into a past release as much as you can, as -it increases the work of the release managers. - -### Cherry-picking from CE to EE - -As we have the `master` branch of CE merged into EE once a day, it's common to -run into merge conflicts. To avoid them, we [test for merge conflicts against EE](#testing) -with the `ee-compat-check` job, and use the following method of creating equivalent -branches for CE and EE. - -Follow this [method for cherry-picking from CE to EE](automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee), with a few adjustments: - -- Create the [CE branch](#branch-naming) starting with `docs-`, - e.g.: `git checkout -b docs-example` -- Create the EE-equivalent branch ending with `-ee`, e.g., - `git checkout -b docs-example-ee` -- Once all the jobs are passing in CE and EE, and you've addressed the -feedback from your own team, assign the CE MR to a technical writer for review -- When both MRs are ready, the EE merge request will be merged first, and the -CE-equivalent will be merged next. -- Note that the review will occur only in the CE MR, as the EE MR -contains the same commits as the CE MR. -- If you have a few more changes that apply to the EE-version only, you can submit -a couple more commits to the EE branch, but ask the reviewer to review the EE merge request -additionally to the CE MR. If there are many EE-only changes though, start a new MR -to EE only. - -## Previewing the changes live - -To preview your changes to documentation locally, please follow -this [development guide](https://gitlab.com/gitlab-com/gitlab-docs/blob/master/README.md#development). - -If you want to preview the doc changes of your merge request live, you can use -the manual `review-docs-deploy` job in your merge request. You will need at -least Master permissions to be able to run it and is currently enabled for the -following projects: - -- https://gitlab.com/gitlab-org/gitlab-ce -- https://gitlab.com/gitlab-org/gitlab-ee - -NOTE: **Note:** -You will need to push a branch to those repositories, it doesn't work for forks. - -TIP: **Tip:** -If your branch contains only documentation changes, you can use -[special branch names](#branch-naming) to avoid long running pipelines. - -In the mini pipeline graph, you should see an `>>` icon. Clicking on it will -reveal the `review-docs-deploy` job. Hit the play button for the job to start. - - - -This job will: - -1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) - project named after the scheme: `preview-<branch-slug>` -1. Trigger a cross project pipeline and build the docs site with your changes - -After a few minutes, the Review App will be deployed and you will be able to -preview the changes. The docs URL can be found in two places: - -- In the merge request widget -- In the output of the `review-docs-deploy` job, which also includes the - triggered pipeline so that you can investigate whether something went wrong - -In case the Review App URL returns 404, follow these steps to debug: - -1. **Did you follow the URL from the merge request widget?** If yes, then check if - the link is the same as the one in the job output. It can happen that if the - branch name slug is longer than 35 characters, it is automatically - truncated. That means that the merge request widget will not show the proper - URL due to a limitation of how `environment: url` works, but you can find the - real URL from the output of the `review-docs-deploy` job. -1. **Did you follow the URL from the job output?** If yes, then it means that - either the site is not yet deployed or something went wrong with the remote - pipeline. Give it a few minutes and it should appear online, otherwise you - can check the status of the remote pipeline from the link in the job output. - If the pipeline failed or got stuck, drop a line in the `#docs` chat channel. - -TIP: **Tip:** -Someone that has no merge rights to the CE/EE projects (think of forks from -contributors) will not be able to run the manual job. In that case, you can -ask someone from the GitLab team who has the permissions to do that for you. - -NOTE: **Note:** -Make sure that you always delete the branch of the merge request you were -working on. If you don't, the remote docs branch won't be removed either, -and the server where the Review Apps are hosted will eventually be out of -disk space. - -### Technical aspects - -If you want to know the hot details, here's what's really happening: - -1. You manually run the `review-docs-deploy` job in a CE/EE merge request. -1. The job runs the [`scripts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) - script with the `deploy` flag, which in turn: - 1. Takes your branch name and applies the following: - - The slug of the branch name is used to avoid special characters since - ultimately this will be used by NGINX. - - The `preview-` prefix is added to avoid conflicts if there's a remote branch - with the same name that you created in the merge request. - - The final branch name is truncated to 42 characters to avoid filesystem - limitations with long branch names (> 63 chars). - 1. The remote branch is then created if it doesn't exist (meaning you can - re-run the manual job as many times as you want and this step will be skipped). - 1. A new cross-project pipeline is triggered in the docs project. - 1. The preview URL is shown both at the job output and in the merge request - widget. You also get the link to the remote pipeline. -1. In the docs project, the pipeline is created and it - [skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55) - to lower the build time. -1. Once the docs site is built, the HTML files are uploaded as artifacts. -1. A specific Runner tied only to the docs project, runs the Review App job - that downloads the artifacts and uses `rsync` to transfer the files over - to a location where NGINX serves them. - -The following GitLab features are used among others: - -- [Manual actions](../ci/yaml/README.md#manual-actions) -- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) -- [Review Apps](../ci/review_apps/index.md) -- [Artifacts](../ci/yaml/README.md#artifacts) -- [Specific Runner](../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) - -## GitLab `/help` - -Every GitLab instance includes the documentation, which is available from `/help` -(`http://my-instance.com/help`), e.g., <https://gitlab.com/help>. - -When you're building a new feature, you may need to link the documentation -from GitLab, the application. This is normally done in files inside the -`app/views/` directory with the help of the `help_page_path` helper method. - -In its simplest form, the HAML code to generate a link to the `/help` page is: - -```haml -= link_to 'Help page', help_page_path('user/permissions') -``` - -The `help_page_path` contains the path to the document you want to link to with -the following conventions: - -- it is relative to the `doc/` directory in the GitLab repository -- the `.md` extension must be omitted -- it must not end with a slash (`/`) - -Below are some special cases where should be used depending on the context. -You can combine one or more of the following: - -1. **Linking to an anchor link.** Use `anchor` as part of the `help_page_path` - method: - - ```haml - = link_to 'Help page', help_page_path('user/permissions', anchor: 'anchor-link') - ``` - -1. **Opening links in a new tab.** This should be the default behavior: - - ```haml - = link_to 'Help page', help_page_path('user/permissions'), target: '_blank' - ``` - -1. **Linking to a circle icon.** Usually used in settings where a long - description cannot be used, like near checkboxes. You can basically use - any font awesome icon, but prefer the `question-circle`: - - ```haml - = link_to icon('question-circle'), help_page_path('user/permissions') - ``` - -1. **Using a button link.** Useful in places where text would be out of context - with the rest of the page layout: - - ```haml - = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' - ``` - -1. **Using links inline of some text.** - - ```haml - Description to #{link_to 'Help page', help_page_path('user/permissions')}. - ``` - -1. **Adding a period at the end of the sentence.** Useful when you don't want - the period to be part of the link: - - ```haml - = succeed '.' do - Learn more in the - = link_to 'Help page', help_page_path('user/permissions') - ``` - -## General Documentation vs Technical Articles - -### General documentation - -General documentation is categorized by _User_, _Admin_, and _Contributor_, and describe what that feature is, what it does, and its available settings. - -### Technical Articles - -Technical articles replace technical content that once lived in the [GitLab Blog](https://about.gitlab.com/blog/), where they got out-of-date and weren't easily found. - -They are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. - -A technical article guides users and/or admins to achieve certain objectives (within guides and tutorials), or provide an overview of that particular topic or feature (within technical overviews). It can also describe the use, implementation, or integration of third-party tools with GitLab. - -They should be placed in a new directory named `/article-title/index.md` under a topic-related folder, and their images should be placed in `/article-title/img/`. For example, a new article on GitLab Pages should be placed in `doc/user/project/pages/article-title/` and a new article on GitLab CI/CD should be placed in `doc/ci/examples/article-title/`. - -#### Types of Technical Articles - -- **User guides**: technical content to guide regular users from point A to point B -- **Admin guides**: technical content to guide administrators of GitLab instances from point A to point B -- **Technical Overviews**: technical content describing features, solutions, and third-party integrations -- **Tutorials**: technical content provided step-by-step on how to do things, or how to reach very specific objectives - -#### Understanding guides, tutorials, and technical overviews - -Suppose there's a process to go from point A to point B in 5 steps: `(A) 1 > 2 > 3 > 4 > 5 (B)`. - -A **guide** can be understood as a description of certain processes to achieve a particular objective. A guide brings you from A to B describing the characteristics of that process, but not necessarily going over each step. It can mention, for example, steps 2 and 3, but does not necessarily explain how to accomplish them. - -- Live example: "[Static sites and GitLab Pages domains (Part 1)](../user/project/pages/getting_started_part_one.md) to [Creating and Tweaking GitLab CI/CD for GitLab Pages (Part 4)](../user/project/pages/getting_started_part_four.md)" - -A **tutorial** requires a clear **step-by-step** guidance to achieve a singular objective. It brings you from A to B, describing precisely all the necessary steps involved in that process, showing each of the 5 steps to go from A to B. -It does not only describes steps 2 and 3, but also shows you how to accomplish them. - -- Live example (on the blog): [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) - -A **technical overview** is a description of what a certain feature is, and what it does, but does not walk -through the process of how to use it systematically. - -- Live example (on the blog): [GitLab Workflow, an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) - -#### Special format - -Every **Technical Article** contains a frontmatter at the beginning of the doc -with the following information: - -- **Type of article** (user guide, admin guide, technical overview, tutorial) -- **Knowledge level** expected from the reader to be able to follow through (beginner, intermediate, advanced) -- **Author's name** and **GitLab.com handle** -- **Publication date** (ISO format YYYY-MM-DD) - -For example: - - -```yaml ---- -author: John Doe -author_gitlab: johnDoe -level: beginner -article_type: user guide -date: 2017-02-01 ---- -``` - -#### Technical Articles - Writing Method - -Use the [writing method](https://about.gitlab.com/handbook/product/technical-writing/#writing-method) defined by the Technical Writing team. - -[gitlab-map]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png -[graffle]: https://gitlab.com/gitlab-org/gitlab-design/blob/d8d39f4a87b90fb9ae89ca12dc565347b4900d5e/production/resources/gitlab-map.graffle diff --git a/doc/downgrade_ee_to_ce/README.md b/doc/downgrade_ee_to_ce/README.md index f656057e3da..a187b3cbb07 100644 --- a/doc/downgrade_ee_to_ce/README.md +++ b/doc/downgrade_ee_to_ce/README.md @@ -15,9 +15,9 @@ Kerberos and Atlassian Crowd are only available on the Enterprise Edition, so you should disable these mechanisms before downgrading and you should provide alternative authentication methods to your users. -### Remove Jenkins CI Service entries from the database +### Remove Service Integration entries from the database -The `JenkinsService` class is only available on the Enterprise Edition codebase, +The `JenkinsService` and `GithubService` classes are only available in the Enterprise Edition codebase, so if you downgrade to the Community Edition, you'll come across the following error: @@ -30,23 +30,34 @@ column if you didn't intend it to be used for storing the inheritance class or o use another column for that information.) ``` +or + +``` +Completed 500 Internal Server Error in 497ms (ActiveRecord: 32.2ms) + +ActionView::Template::Error (The single-table inheritance mechanism failed to locate the subclass: 'GithubService'. This +error is raised because the column 'type' is reserved for storing the class in case of inheritance. Please rename this +column if you didn't intend it to be used for storing the inheritance class or overwrite Service.inheritance_column to +use another column for that information.) +``` + All services are created automatically for every project you have, so in order to avoid getting this error, you need to remove all instances of the -`JenkinsService` from your database: +`JenkinsService` and `GithubService` from your database: **Omnibus Installation** ``` -$ sudo gitlab-rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService']).delete_all" +$ sudo gitlab-rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService', 'GithubService']).delete_all" ``` **Source Installation** ``` -$ bundle exec rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService']).delete_all" production +$ bundle exec rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService', 'GithubService']).delete_all" production ``` -### Secret variables environment scopes +### Variables environment scopes If you're using this feature and there are variables sharing the same key, but they have different scopes in a project, then you might want to diff --git a/doc/gitlab-basics/command-line-commands.md b/doc/gitlab-basics/command-line-commands.md index c9766040234..4666511d747 100644 --- a/doc/gitlab-basics/command-line-commands.md +++ b/doc/gitlab-basics/command-line-commands.md @@ -7,17 +7,19 @@ In Git, when you copy a project you say you "clone" it. To work on a git project When you are on your Dashboard, click on the project that you'd like to clone. To work in the project, you can copy a link to the Git repository through a SSH or a HTTPS protocol. SSH is easier to use after it's been -[setup](create-your-ssh-keys.md). While you are at the **Project** tab, select -HTTPS or SSH from the dropdown menu and copy the link using the 'Copy to clipboard' +[set up](create-your-ssh-keys.md). While you are at the **Project** tab, select +HTTPS or SSH from the dropdown menu and copy the link using the _Copy URL to clipboard_ button (you'll have to paste it on your shell in the next step).  ## On the command line +This section has examples of some basic shell commands that you might find useful. For more information, search the web for _bash commands_. + ### Clone your project -Go to your computer's shell and type the following command: +Go to your computer's shell and type the following command with your SSH or HTTPS URL: ``` git clone PASTE HTTPS OR SSH HERE @@ -25,33 +27,45 @@ git clone PASTE HTTPS OR SSH HERE A clone of the project will be created in your computer. ->**Note:** If you clone your project via an URL that contains special characters, make sure that they are URL-encoded. +>**Note:** If you clone your project via a URL that contains special characters, make sure that characters are URL-encoded. -### Go into a project, directory or file to work in it +### Go into a project directory to work in it ``` -cd NAME-OF-PROJECT-OR-FILE +cd NAME-OF-PROJECT ``` -### Go back one directory or file +### Go back one directory ``` -cd ../ +cd .. ``` -### View what’s in the directory that you are in +### List what’s in the current directory ``` ls ``` -### Create a directory +### List what’s in the current directory that starts with `a` + +``` +ls a* +``` + +### List what’s in the current directory that ends with `.md` + +``` +ls *.md +``` + +### Create a new directory ``` mkdir NAME-OF-YOUR-DIRECTORY ``` -### Create a README.md or file in directory +### Create a README.md file in the current directory ``` touch README.md @@ -62,6 +76,12 @@ nano README.md #### Press: enter ``` +### Show the contents of the README.md file + +``` +cat README.md +``` + ### Remove a file ``` @@ -74,12 +94,18 @@ rm NAME-OF-FILE rm -r NAME-OF-DIRECTORY ``` -### View history in the command line +### View command history ``` history ``` +### Execute command 123 from history + +``` +!123 +``` + ### Carry out commands for which the account you are using lacks authority You will be asked for an administrator’s password. @@ -88,8 +114,14 @@ You will be asked for an administrator’s password. sudo ``` -### Tell where you are +### Show which directory I am in ``` pwd ``` + +### Clear the shell window + +``` +clear +``` diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md index 42cd8bb3e48..0d9994c9925 100644 --- a/doc/gitlab-basics/start-using-git.md +++ b/doc/gitlab-basics/start-using-git.md @@ -17,112 +17,197 @@ Depending on your operating system, you will need to use a shell of your prefere Git is usually preinstalled on Mac and Linux. Type the following command and then press enter: -``` + +```bash git --version ``` -You should receive a message that will tell you which Git version you have on your computer. If you don’t receive a "Git version" message, it means that you need to [download Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). +You should receive a message that tells you which Git version you have on your computer. If you don’t receive a "Git version" message, it means that you need to [download Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). If Git doesn't automatically download, there's an option on the website to [download manually](https://git-scm.com/downloads). Then follow the steps on the installation window. -After you are finished installing, open a new shell and type "git --version" again to verify that it was correctly installed. +After you are finished installing Git, open a new shell and type `git --version` again to verify that it was correctly installed. ## Add your Git username and set your email -It is important to configure your Git username and email address as every Git commit will use this information to identify you as the author. +It is important to configure your Git username and email address, since every Git commit will use this information to identify you as the author. On your shell, type the following command to add your username: -``` + +```bash git config --global user.name "YOUR_USERNAME" ``` Then verify that you have the correct username: -``` + +```bash git config --global user.name ``` To set your email address, type the following command: -``` + +```bash git config --global user.email "your_email_address@example.com" ``` To verify that you entered your email correctly, type: -``` + +```bash git config --global user.email ``` -You'll need to do this only once as you are using the `--global` option. It tells Git to always use this information for anything you do on that system. If you want to override this with a different username or email address for specific projects, you can run the command without the `--global` option when you’re in that project. +You'll need to do this only once, since you are using the `--global` option. It tells Git to always use this information for anything you do on that system. If you want to override this with a different username or email address for specific projects, you can run the command without the `--global` option when you’re in that project. ## Check your information -To view the information that you entered, type: -``` +To view the information that you entered, along with other global options, type: + +```bash git config --global --list ``` + ## Basic Git commands ### Go to the master branch to pull the latest changes from there -``` +```bash git checkout master ``` ### Download the latest changes in the project -This is for you to work on an up-to-date copy (it is important to do every time you work on a project), while you setup tracking branches. + +This is for you to work on an up-to-date copy (it is important to do this every time you start working on a project), while you set up tracking branches. You pull from remote repositories to get all the changes made by users since the last time you cloned or pulled the project. Later, you can push your local commits to the remote repositories. + +```bash +git pull REMOTE NAME-OF-BRANCH ``` -git pull REMOTE NAME-OF-BRANCH -u + +When you first clone a repository, REMOTE is typically "origin". This is where the repository came from, and it indicates the SSH or HTTPS URL of the repository on the remote server. NAME-OF-BRANCH is usually "master", but it may be any existing branch. + +### View your remote repositories + +To view your remote repositories, type: + +```bash +git remote -v ``` -(REMOTE: origin) (NAME-OF-BRANCH: could be "master" or an existing branch) ### Create a branch -Spaces won't be recognized, so you will need to use a hyphen or underscore. -``` + +To create a branch, type the following (spaces won't be recognized in the branch name, so you will need to use a hyphen or underscore): + +```bash git checkout -b NAME-OF-BRANCH ``` -### Work on a branch that has already been created -``` +### Work on an existing branch + +To switch to an existing branch, so you can work on it: + +```bash git checkout NAME-OF-BRANCH ``` ### View the changes you've made -It's important to be aware of what's happening and what's the status of your changes. -``` + +It's important to be aware of what's happening and the status of your changes. When you add, change, or delete files/folders, Git knows about it. To check the status of your changes: + +```bash git status ``` -### Add changes to commit -You'll see your changes in red when you type "git status". +### View differences + +To view the differences between your local, unstaged changes and the repository versions that you cloned or pulled, type: + +```bash +git diff +``` + +### Add and commit local changes + +You'll see your local changes in red when you type `git status`. These changes may be new, modified, or deleted files/folders. Use `git add` to stage a local file/folder for committing. Then use `git commit` to commit the staged files: + +```bash +git add FILE OR FOLDER +git commit -m "COMMENT TO DESCRIBE THE INTENTION OF THE COMMIT" ``` -git add CHANGES IN RED -git commit -m "DESCRIBE THE INTENTION OF THE COMMIT" + +### Add all changes to commit + +To add and commit all local changes in one command: + +```bash +git add . +git commit -m "COMMENT TO DESCRIBE THE INTENTION OF THE COMMIT" ``` +NOTE: **Note:** +The `.` character typically means _all_ in Git. + ### Send changes to gitlab.com -``` + +To push all local commits to the remote repository: + +```bash git push REMOTE NAME-OF-BRANCH ``` -### Delete all changes in the Git repository, but leave unstaged things +For example, to push your local commits to the _master_ branch of the _origin_ remote: + +```bash +git push origin master ``` + +### Delete all changes in the Git repository + +To delete all local changes in the repository that have not been added to the staging area, and leave unstaged files/folders, type: + +```bash git checkout . ``` -### Delete all changes in the Git repository, including untracked files -``` +### Delete all untracked changes in the Git repository + +```bash git clean -f ``` +### Unstage all changes that have been added to the staging area + +To undo the most recent add, but not committed, files/folders: + +```bash +git reset . +``` + +### Undo most recent commit + +To undo the most recent commit, type: + +```bash +git reset HEAD~1 +``` + +This leaves the files and folders unstaged in your local repository. + +CAUTION: **Warning:** +A Git commit is mostly irreversible, particularly if you already pushed it to the remote repository. Although you can undo a commit, the best option is to avoid the situation altogether. + ### Merge created branch with master branch + You need to be in the created branch. -``` + +```bash git checkout NAME-OF-BRANCH git merge master ``` ### Merge master branch with created branch + You need to be in the master branch. -``` + +```bash git checkout master git merge NAME-OF-BRANCH ``` diff --git a/doc/install/README.md b/doc/install/README.md index 5dadf57ea9a..27df03c6ac6 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -1,5 +1,6 @@ --- comments: false +description: Read through the GitLab installation methods. --- # Installation diff --git a/doc/install/azure/index.md b/doc/install/azure/index.md index b0c3ad960bb..21694b02d18 100644 --- a/doc/install/azure/index.md +++ b/doc/install/azure/index.md @@ -1,3 +1,8 @@ +--- +description: 'Learn how to spin up a +pre-configured GitLab VM on Microsoft Azure and have your very own private GitLab instance up and running in around 30 minutes.' +--- + # Install GitLab on Microsoft Azure > _This article was originally written by Dave Wentzel and [published on the GitLab Blog][Original-Blog-Post]._ diff --git a/doc/install/docker.md b/doc/install/docker.md index 933756072ff..c7dc9db70c5 100644 --- a/doc/install/docker.md +++ b/doc/install/docker.md @@ -1,4 +1,4 @@ -# GitLab Docker images +# Install GitLab with Docker [Docker](https://www.docker.com) and container technology have been revolutionizing the software world for the past few years. They combine the performance and efficiency of native execution with the abstraction, security, and immutability of virtualization. diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md index 2691495e0d4..ab5f7507f24 100644 --- a/doc/install/google_cloud_platform/index.md +++ b/doc/install/google_cloud_platform/index.md @@ -1,3 +1,7 @@ +--- +description: 'Learn how to install a GitLab instance on Google Cloud Platform.' +--- + # Installing GitLab on Google Cloud Platform  diff --git a/doc/install/installation.md b/doc/install/installation.md index fa5bcfa6f07..6cd1fb4c2d7 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -133,9 +133,9 @@ Remove the old Ruby 1.8 if present: Download Ruby and compile it: mkdir /tmp/ruby && cd /tmp/ruby - curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.7.tar.gz - echo '540996fec64984ab6099e34d2f5820b14904f15a ruby-2.3.7.tar.gz' | shasum -c - && tar xzf ruby-2.3.7.tar.gz - cd ruby-2.3.7 + curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.4.tar.gz + echo 'ec82b0d53bd0adad9b19e6b45e44d54e9ec3f10c ruby-2.4.4.tar.gz' | shasum -c - && tar xzf ruby-2.4.4.tar.gz + cd ruby-2.4.4 ./configure --disable-install-rdoc make @@ -301,9 +301,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-7-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 11-0-stable gitlab -**Note:** You can change `10-6-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `11-0-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It diff --git a/doc/install/kubernetes/gitlab_omnibus.md b/doc/install/kubernetes/gitlab_omnibus.md index 9c5258c2cdf..852a58a9afc 100644 --- a/doc/install/kubernetes/gitlab_omnibus.md +++ b/doc/install/kubernetes/gitlab_omnibus.md @@ -120,7 +120,7 @@ gitlabConfigStorageSize: 1Gi Ingress routing and SSL are automatically configured within this Chart. An NGINX ingress is provisioned and configured, and will route traffic to any service. SSL certificates are automatically created and configured by [kube-lego](https://github.com/kubernetes/charts/tree/master/stable/kube-lego). > **Note:** -Let's Encrypt limits a single TLD to five certificate requests within a single week. This means that common DNS wildcard services like [xip.io](http://xip.io) and [nip.io](http://nip.io) are unlikely to work. +Let's Encrypt limits a single TLD to five certificate requests within a single week. This means that common DNS wildcard services like [nip.io](http://nip.io) and [nip.io](http://nip.io) are unlikely to work. ## Installing GitLab using the Helm Chart > **Note:** @@ -129,8 +129,8 @@ You may see a temporary error message `SchedulerPredicates failed due to Persist Add the GitLab Helm repository and initialize Helm: ```bash -helm repo add gitlab https://charts.gitlab.io helm init +helm repo add gitlab https://charts.gitlab.io ``` Once you have reviewed the [configuration settings](#configuring-and-installing-gitlab) you can install the chart. We recommending saving your configuration options in a `values.yaml` file for easier upgrades in the future. @@ -144,7 +144,7 @@ helm install --name gitlab -f values.yaml gitlab/gitlab-omnibus or passing them on the command line: ```bash -helm install --name gitlab --set baseDomain=gitlab.io,baseIP=1.1.1.1,gitlab=ee,gitlabEELicense=$LICENSE,legoEmail=email@gitlab.com gitlab/gitlab-omnibus +helm install --name gitlab --set baseDomain=gitlab.io,baseIP=192.0.2.1,gitlab=ee,gitlabEELicense=$LICENSE,legoEmail=email@gitlab.com gitlab/gitlab-omnibus ``` ## Updating GitLab using the Helm Chart diff --git a/doc/install/kubernetes/gitlab_runner_chart.md b/doc/install/kubernetes/gitlab_runner_chart.md index 0a093c9ec32..2aab225fcdb 100644 --- a/doc/install/kubernetes/gitlab_runner_chart.md +++ b/doc/install/kubernetes/gitlab_runner_chart.md @@ -1,6 +1,6 @@ # GitLab Runner Helm Chart > **Note:** -These charts have been tested on Google Kubernetes Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/charts.gitlab.io/issues). +These charts have been tested on Google Kubernetes Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/gitlab-runner/issues). The `gitlab-runner` Helm chart deploys a GitLab Runner instance into your Kubernetes cluster. @@ -25,7 +25,7 @@ For more information on available GitLab Helm Charts, please see our [overview]( Create a `values.yaml` file for your GitLab Runner configuration. See [Helm docs](https://github.com/kubernetes/helm/blob/master/docs/chart_template_guide/values_files.md) for information on how your values file will override the defaults. -The default configuration can always be found in the [values.yaml](https://gitlab.com/charts/charts.gitlab.io/blob/master/charts/gitlab-runner/values.yaml) in the chart repository. +The default configuration can always be found in the [values.yaml](https://gitlab.com/charts/gitlab-runner/blob/master/values.yaml) in the chart repository. ### Required configuration @@ -39,7 +39,7 @@ Unless you need to specify additional configuration, you are [ready to install]( ### Other configuration -The rest of the configuration is [documented in the `values.yaml`](https://gitlab.com/charts/charts.gitlab.io/blob/master/charts/gitlab-runner/values.yaml) in the chart repository. +The rest of the configuration is [documented in the `values.yaml`](https://gitlab.com/charts/gitlab-runner/blob/master/values.yaml) in the chart repository. Here is a snippet of the important settings: diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index 7d8b8fc1597..aeaa739edab 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -1,4 +1,9 @@ +--- +description: 'Read through the different methods to deploy GitLab on Kubernetes.' +--- + # Installing GitLab on Kubernetes + > **Note**: These charts have been tested on Google Kubernetes Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/charts.gitlab.io/issues). The easiest method to deploy GitLab on [Kubernetes](https://kubernetes.io/) is @@ -16,6 +21,7 @@ should be deployed, upgraded, and configured. * [Community Contributed Charts](#community-contributed-charts): Community contributed charts, deprecated by the official GitLab chart. ## GitLab-Omnibus Chart (Recommended) + > **Note**: This chart is in beta while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being added. This chart is the best available way to operate GitLab on Kubernetes. It deploys and configures nearly all features of GitLab, including: a [Runner](https://docs.gitlab.com/runner/), [Container Registry](../../user/project/container_registry.html#gitlab-container-registry), [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/), [automatic SSL](https://github.com/kubernetes/charts/tree/master/stable/kube-lego), and a [load balancer](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). It is based on our [GitLab Omnibus Docker Images](https://docs.gitlab.com/omnibus/docker/README.html). diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md index e6ccfccd33f..60e1e2b5f8a 100644 --- a/doc/install/openshift_and_gitlab/index.md +++ b/doc/install/openshift_and_gitlab/index.md @@ -6,7 +6,7 @@ article_type: tutorial date: 2016-06-28 --- -# Getting started with OpenShift Origin 3 and GitLab +# How to install GitLab on OpenShift Origin 3 ## Introduction @@ -307,10 +307,10 @@ hostname** and use greater values for the volume sizes. If you don't provide a password for PostgreSQL, it will be created automatically. >**Note:** -The `gitlab.apps.10.2.2.2.xip.io` hostname that is used by default will +The `gitlab.apps.10.2.2.2.nip.io` hostname that is used by default will resolve to the host with IP `10.2.2.2` which is the IP our VM uses. It is a trick to have distinct FQDNs pointing to services that are on our local network. -Read more on how this works in <http://xip.io>. +Read more on how this works in <http://nip.io>. Now that we configured this, let's see how to manage and scale GitLab. @@ -347,7 +347,7 @@ Navigate back to the **Overview** and hopefully all pods will be up and running.  Congratulations! You can now navigate to your new shinny GitLab instance by -visiting <http://gitlab.apps.10.2.2.2.xip.io> where you will be asked to +visiting <http://gitlab.apps.10.2.2.2.nip.io> where you will be asked to change the root user password. Login using `root` as username and providing the password you just set, and start using GitLab! @@ -521,4 +521,4 @@ PaaS and managing your applications with the ease of containers. [autoscaling]: https://docs.openshift.org/latest/dev_guide/pod_autoscaling.html "Documentation - Autoscale" [basic-cli]: https://docs.openshift.org/latest/cli_reference/basic_cli_operations.html "Documentation - Basic CLI operations" [openshift-docs]: https://docs.openshift.org "OpenShift documentation" -[scc]: https://docs.openshift.org/latest/admin_guide/manage_scc.html "Documentation - Managing Security Context Constraints"
\ No newline at end of file +[scc]: https://docs.openshift.org/latest/admin_guide/manage_scc.html "Documentation - Managing Security Context Constraints" diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 1f2b4d9d3d9..1f399a8a3f7 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -27,8 +27,8 @@ Please see the [installation from source guide](installation.md) and the [instal ### Non-Unix operating systems such as Windows -GitLab is developed for Unix operating systems. -GitLab does **not** run on Windows and we have no plans of supporting it in the near future. +GitLab is developed for Unix operating systems. +It does **not** run on Windows, and we have no plans to support it in the near future. For the latest development status view this [issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/46567). Please consider using a virtual machine to run GitLab. ## Ruby versions diff --git a/doc/integration/github.md b/doc/integration/github.md index 23bb8ef9303..680712f9e01 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -110,7 +110,7 @@ On the sign in page there should now be a GitHub icon below the regular sign in Click the icon to begin the authentication process. GitHub will ask the user to sign in and authorize the GitLab application. If everything goes well the user will be returned to GitLab and will be signed in. -### GitHub Enterprise with Self-Signed Certificate +## GitHub Enterprise with self-signed Certificate If you are attempting to import projects from GitHub Enterprise with a self-signed certificate and the imports are failing, you will need to disable SSL verification. diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index eec40a9b8f1..70087576678 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -7,13 +7,11 @@ GitLab.com will generate an application ID and secret key for you to use. 1. Sign in to GitLab.com -1. Navigate to your profile settings. +1. On the upper right corner, click on your avatar and go to your **Settings**. -1. Select "Applications" in the left menu. +1. Select **Applications** in the left menu. -1. Select "New application". - -1. Provide the required details. +1. Provide the required details for **Add new application**. - Name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive. - Redirect URI: @@ -24,9 +22,9 @@ GitLab.com will generate an application ID and secret key for you to use. The first link is required for the importer and second for the authorization. -1. Select "Submit". +1. Select **Save application**. -1. You should now see a Client ID and Client Secret near the top right of the page (see screenshot). +1. You should now see a **Application Id** and **Secret** near the top right of the page (see screenshot). Keep this page open as you continue configuration.  diff --git a/doc/integration/img/gitlab_app.png b/doc/integration/img/gitlab_app.png Binary files differindex b4958581a9b..8d6a4456fc4 100644 --- a/doc/integration/img/gitlab_app.png +++ b/doc/integration/img/gitlab_app.png diff --git a/doc/integration/shibboleth.md b/doc/integration/shibboleth.md index 8611d4f7315..0e43b4a39a4 100644 --- a/doc/integration/shibboleth.md +++ b/doc/integration/shibboleth.md @@ -107,7 +107,7 @@ you will not get a shibboleth session! RewriteEngine on #Don't escape encoded characters in api requests - RewriteCond %{REQUEST_URI} ^/api/v3/.* + RewriteCond %{REQUEST_URI} ^/api/v4/.* RewriteCond %{REQUEST_URI} !/Shibboleth.sso RewriteCond %{REQUEST_URI} !/shibboleth-sp RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA,NE] diff --git a/doc/legal/README.md b/doc/legal/README.md index 6413f1d645f..d991429a652 100644 --- a/doc/legal/README.md +++ b/doc/legal/README.md @@ -4,5 +4,4 @@ comments: false # Legal -- [Corporate contributor license agreement](corporate_contributor_license_agreement.md) -- [Individual contributor license agreement](individual_contributor_license_agreement.md) +Please read through the [GitLab License Agreement](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md). diff --git a/doc/legal/corporate_contributor_license_agreement.md b/doc/legal/corporate_contributor_license_agreement.md index ebb24ba0a7f..e5fc7a3c85f 100644 --- a/doc/legal/corporate_contributor_license_agreement.md +++ b/doc/legal/corporate_contributor_license_agreement.md @@ -1,2 +1,3 @@ -This document has been replaced by a Developer Certificate of Origin and License, -as described in [Contributing.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md).
\ No newline at end of file +--- +redirect_to: 'README.md' +--- diff --git a/doc/legal/individual_contributor_license_agreement.md b/doc/legal/individual_contributor_license_agreement.md index ebb24ba0a7f..e5fc7a3c85f 100644 --- a/doc/legal/individual_contributor_license_agreement.md +++ b/doc/legal/individual_contributor_license_agreement.md @@ -1,2 +1,3 @@ -This document has been replaced by a Developer Certificate of Origin and License, -as described in [Contributing.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md).
\ No newline at end of file +--- +redirect_to: 'README.md' +--- diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 785cc32d590..95221d8b6b1 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -31,8 +31,8 @@ sudo apt-get install -y rsync >**Note:** In GitLab 9.2 the timestamp format was changed from `EPOCH_YYYY_MM_DD` to -`EPOCH_YYYY_MM_DD_GitLab version`, for example `1493107454_2017_04_25` -would become `1493107454_2017_04_25_9.1.0`. +`EPOCH_YYYY_MM_DD_GitLab_version`, for example `1493107454_2018_04_25` +would become `1493107454_2018_04_25_10.6.4-ce`. The backup archive will be saved in `backup_path`, which is specified in the `config/gitlab.yml` file. @@ -41,8 +41,8 @@ identifies the time at which each backup was created, plus the GitLab version. The timestamp is needed if you need to restore GitLab and multiple backups are available. -For example, if the backup name is `1493107454_2017_04_25_9.1.0_gitlab_backup.tar`, -then the timestamp is `1493107454_2017_04_25_9.1.0`. +For example, if the backup name is `1493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar`, +then the timestamp is `1493107454_2018_04_25_10.6.4-ce`. ### Creating a backup of the GitLab system @@ -64,6 +64,13 @@ If you are running GitLab within a Docker container, you can run the backup from docker exec -t <container name> gitlab-rake gitlab:backup:create ``` +If you are using the gitlab-omnibus helm chart on a Kubernetes cluster, you can +run the backup task on the gitlab application pod using kubectl + +``` +kubectl exec -it <gitlab-gitlab pod> gitlab-rake gitlab:backup:create +``` + Example output: ``` @@ -485,8 +492,8 @@ directory (repositories, uploads). To restore a backup, you will also need to restore `/etc/gitlab/gitlab-secrets.json` (for Omnibus packages) or `/home/git/gitlab/.secret` (for installations from source). This file contains the database encryption key, -[CI secret variables](../ci/variables/README.md#secret-variables), and -secret variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). +[CI/CD variables](../ci/variables/README.md#variables), and +variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). If you fail to restore this encryption key file along with the application data backup, users with two-factor authentication enabled and GitLab Runners will lose access to your GitLab server. @@ -567,7 +574,7 @@ First make sure your backup tar file is in the backup directory described in the `/var/opt/gitlab/backups`. ```shell -sudo cp 1493107454_2017_04_25_9.1.0_gitlab_backup.tar /var/opt/gitlab/backups/ +sudo cp 11493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar /var/opt/gitlab/backups/ ``` Stop the processes that are connected to the database. Leave the rest of GitLab @@ -585,7 +592,7 @@ restore: ```shell # This command will overwrite the contents of your GitLab database! -sudo gitlab-rake gitlab:backup:restore BACKUP=1493107454_2017_04_25_9.1.0 +sudo gitlab-rake gitlab:backup:restore BACKUP=1493107454_2018_04_25_10.6.4-ce ``` Next, restore `/etc/gitlab/gitlab-secrets.json` if necessary as mentioned above. @@ -601,6 +608,34 @@ If there is a GitLab version mismatch between your backup tar file and the insta version of GitLab, the restore command will abort with an error. Install the [correct GitLab version](https://packages.gitlab.com/gitlab/) and try again. +### Restore for Docker image and gitlab-omnibus helm chart + +For GitLab installations using docker image or the gitlab-omnibus helm chart on +a Kubernetes cluster, restore task expects the restore directories to be empty. +However, with docker and Kubernetes volume mounts, some system level directories +may be created at the volume roots, like `lost+found` directory found in Linux +operating systems. These directories are usually owned by `root`, which can +cause access permission errors since the restore rake task runs as `git` user. +So, to restore a GitLab installation, users have to confirm the restore target +directories are empty. + +For both these installation types, the backup tarball has to be available in the +backup location (default location is `/var/opt/gitlab/backups`). + +For docker installations, the restore task can be run from host using the +command + +``` +docker exec -it <name of container> gitlab-rake gitlab:backup:restore +``` + +Similarly, for gitlab-omnibus helm chart, the restore task can be run on the +gitlab application pod using kubectl + +``` +kubectl exec -it <gitlab-gitlab pod> gitlab-rake gitlab:backup:restore +``` + ## Alternative backup strategies If your GitLab server contains a lot of Git repository data you may find the GitLab backup script to be too slow. diff --git a/doc/raketasks/features.md b/doc/raketasks/features.md index fee49cc27cc..57c16f110e4 100644 --- a/doc/raketasks/features.md +++ b/doc/raketasks/features.md @@ -1,4 +1,4 @@ -# Features +# Namespaces ## Enable usernames and namespaces for user projects diff --git a/doc/raketasks/web_hooks.md b/doc/raketasks/web_hooks.md index 2ebf7c48f4e..5f3143f76cd 100644 --- a/doc/raketasks/web_hooks.md +++ b/doc/raketasks/web_hooks.md @@ -1,4 +1,4 @@ -# Webhooks +# Webhooks administration **[CORE ONLY]** ## Add a webhook for **ALL** projects: diff --git a/doc/security/information_exclusivity.md b/doc/security/information_exclusivity.md index f8e7fc3fd0e..22756232025 100644 --- a/doc/security/information_exclusivity.md +++ b/doc/security/information_exclusivity.md @@ -2,7 +2,7 @@ Git is a distributed version control system (DVCS). This means that everyone that works with the source code has a local copy of the complete repository. -In GitLab every project member that is not a guest (so reporters, developers and masters) can clone the repository to get a local copy. +In GitLab every project member that is not a guest (so reporters, developers and maintainers) can clone the repository to get a local copy. After obtaining this local copy the user can upload the full repository anywhere, including another project under their control or another server. The consequence is that you can't build access controls that prevent the intentional sharing of source code by users that have access to the source code. This is an inherent feature of a DVCS and all git management systems have this limitation. diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md index a573445ab5b..b17b0a4bc4a 100644 --- a/doc/security/webhooks.md +++ b/doc/security/webhooks.md @@ -2,7 +2,7 @@ If you have non-GitLab web services running on your GitLab server or within its local network, these may be vulnerable to exploitation via Webhooks. -With [Webhooks](../user/project/integrations/webhooks.md), you and your project masters and owners can set up URLs to be triggered when specific things happen to projects. Normally, these requests are sent to external web services specifically set up for this purpose, that process the request and its attached data in some appropriate way. +With [Webhooks](../user/project/integrations/webhooks.md), you and your project maintainers and owners can set up URLs to be triggered when specific things happen to projects. Normally, these requests are sent to external web services specifically set up for this purpose, that process the request and its attached data in some appropriate way. Things get hairy, however, when a Webhook is set up with a URL that doesn't point to an external, but to an internal service, that may do something completely unintended when the webhook is triggered and the POST request is sent. diff --git a/doc/ssh/README.md b/doc/ssh/README.md index b71e9bf3000..bab196e7609 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -171,7 +171,7 @@ This is really useful for cloning repositories to your Continuous Integration (CI) server. By using deploy keys, you don't have to set up a dummy user account. -If you are a project master or owner, you can add a deploy key in the +If you are a project maintainer or owner, you can add a deploy key in the project settings under the section 'Repository'. Specify a title for the new deploy key and paste a public SSH key. After this, the machine that uses the corresponding private SSH key has read-only or read-write (if enabled) @@ -196,7 +196,7 @@ This is really useful for integrating repositories to secured, shared Continuous Integration (CI) services or other shared services. GitLab administrators can set up the Global Shared Deploy key in GitLab and add the private key to any shared systems. Individual repositories opt into -exposing their repository using these keys when a project masters (or higher) +exposing their repository using these keys when a project maintainers (or higher) authorizes a Global Shared Deploy key to be used with their project. Global Shared Keys can provide greater security compared to Per-Project Deploy @@ -205,7 +205,7 @@ who needs to know and configure the private key. GitLab administrators set up Global Deploy keys in the Admin area under the section **Deploy Keys**. Ensure keys have a meaningful title as that will be -the primary way for project masters and owners to identify the correct Global +the primary way for project maintainers and owners to identify the correct Global Deploy key to add. For instance, if the key gives access to a SaaS CI instance, use the name of that service in the key name if that is all it is used for. When creating Global Shared Deploy keys, give some thought to the granularity @@ -213,7 +213,7 @@ of keys - they could be of very narrow usage such as just a specific service or of broader usage for something like "Anywhere you need to give read access to your repository". -Once a GitLab administrator adds the Global Deployment key, project masters +Once a GitLab administrator adds the Global Deployment key, project maintainers and owners can add it in project's **Settings > Repository** section by expanding the **Deploy Key** section and clicking **Enable** next to the appropriate key listed under **Public deploy keys available to any project**. diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md index 9ba05c7815b..cce02d218c2 100644 --- a/doc/system_hooks/system_hooks.md +++ b/doc/system_hooks/system_hooks.md @@ -138,7 +138,7 @@ Please refer to `group_rename` and `user_rename` for that case. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_add_to_team", - "project_access": "Master", + "project_access": "Maintainer", "project_id": 74, "project_name": "StoreCloud", "project_path": "storecloud", @@ -158,7 +158,7 @@ Please refer to `group_rename` and `user_rename` for that case. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_remove_from_team", - "project_access": "Master", + "project_access": "Maintainer", "project_id": 74, "project_name": "StoreCloud", "project_path": "storecloud", @@ -318,7 +318,7 @@ If the user is blocked via LDAP, `state` will be `ldap_blocked`. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_add_to_group", - "group_access": "Master", + "group_access": "Maintainer", "group_id": 78, "group_name": "StoreCloud", "group_path": "storecloud", @@ -335,7 +335,7 @@ If the user is blocked via LDAP, `state` will be `ldap_blocked`. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_remove_from_group", - "group_access": "Master", + "group_access": "Maintainer", "group_id": 78, "group_name": "StoreCloud", "group_path": "storecloud", diff --git a/doc/topics/autodevops/img/autodevops_domain_variables.png b/doc/topics/autodevops/img/autodevops_domain_variables.png Binary files differnew file mode 100644 index 00000000000..b6f8864796f --- /dev/null +++ b/doc/topics/autodevops/img/autodevops_domain_variables.png diff --git a/doc/topics/autodevops/img/autodevops_multiple_clusters.png b/doc/topics/autodevops/img/autodevops_multiple_clusters.png Binary files differnew file mode 100644 index 00000000000..f4d101ca921 --- /dev/null +++ b/doc/topics/autodevops/img/autodevops_multiple_clusters.png diff --git a/doc/topics/autodevops/img/rollout_enabled.png b/doc/topics/autodevops/img/rollout_enabled.png Binary files differnew file mode 100644 index 00000000000..d6e7d790cf2 --- /dev/null +++ b/doc/topics/autodevops/img/rollout_enabled.png diff --git a/doc/topics/autodevops/img/rollout_staging_disabled.png b/doc/topics/autodevops/img/rollout_staging_disabled.png Binary files differnew file mode 100644 index 00000000000..71e36b440f0 --- /dev/null +++ b/doc/topics/autodevops/img/rollout_staging_disabled.png diff --git a/doc/topics/autodevops/img/rollout_staging_enabled.png b/doc/topics/autodevops/img/rollout_staging_enabled.png Binary files differnew file mode 100644 index 00000000000..d0d1d356627 --- /dev/null +++ b/doc/topics/autodevops/img/rollout_staging_enabled.png diff --git a/doc/topics/autodevops/img/staging_enabled.png b/doc/topics/autodevops/img/staging_enabled.png Binary files differnew file mode 100644 index 00000000000..0ef1a67d641 --- /dev/null +++ b/doc/topics/autodevops/img/staging_enabled.png diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 7c0cd2c40d2..103836e59d0 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -1,7 +1,5 @@ # Auto DevOps -DANGER: Auto DevOps is currently in **Beta** and _not recommended for production use_. - > [Introduced][ce-37115] in GitLab 10.0. Auto DevOps automatically detects, builds, tests, deploys, and monitors your @@ -43,6 +41,7 @@ project in an easy and automatic way: 1. [Auto Code Quality](#auto-code-quality) 1. [Auto SAST (Static Application Security Testing)](#auto-sast) 1. [Auto Dependency Scanning](#auto-dependency-scanning) +1. [Auto License Management](#auto-license-management) 1. [Auto Container Scanning](#auto-container-scanning) 1. [Auto Review Apps](#auto-review-apps) 1. [Auto DAST (Dynamic Application Security Testing)](#auto-dast) @@ -64,7 +63,7 @@ Auto DevOps provides great defaults for all the stages; you can, however, For an overview on the creation of Auto DevOps, read the blog post [From 2/3 of the Self-Hosted Git Market, to the Next-Generation CI System, to Auto DevOps](https://about.gitlab.com/2017/06/29/whats-next-for-gitlab-ci/). -## Prerequisites +## Requirements TIP: **Tip:** For self-hosted installations, the easiest way to make use of Auto DevOps is to @@ -114,30 +113,31 @@ NOTE: **Note:** If you do not have Kubernetes or Prometheus installed, then Auto Review Apps, Auto Deploy, and Auto Monitoring will be silently skipped. -### Auto DevOps base domain +## Auto DevOps base domain The Auto DevOps base domain is required if you want to make use of [Auto -Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It is defined -either under the project's CI/CD settings while -[enabling Auto DevOps](#enabling-auto-devops) or in instance-wide settings in -the CI/CD section. -It can also be set at the project or group level as a variable, `AUTO_DEVOPS_DOMAIN`. +Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It can be defined +in three places: + +- either under the project's CI/CD settings while [enabling Auto DevOps](#enabling-auto-devops) +- or in instance-wide settings in the **admin area > Settings** under the "Continuous Integration and Delivery" section +- or at the project or group level as a variable: `AUTO_DEVOPS_DOMAIN` (required if you want to use [multiple clusters](#using-multiple-kubernetes-clusters)) -A wildcard DNS A record matching the base domain is required, for example, +A wildcard DNS A record matching the base domain(s) is required, for example, given a base domain of `example.com`, you'd need a DNS entry like: ``` *.example.com 3600 A 1.2.3.4 ``` -where `example.com` is the domain name under which the deployed apps will be served, +In this case, `example.com` is the domain name under which the deployed apps will be served, and `1.2.3.4` is the IP address of your load balancer; generally NGINX -([see prerequisites](#prerequisites)). How to set up the DNS record is beyond +([see requirements](#requirements)). How to set up the DNS record is beyond the scope of this document; you should check with your DNS provider. -Alternatively you can use free public services like [xip.io](http://xip.io) or +Alternatively you can use free public services like [nip.io](http://nip.io) or [nip.io](http://nip.io) which provide automatic wildcard DNS without any -configuration. Just set the Auto DevOps base domain to `1.2.3.4.xip.io` or +configuration. Just set the Auto DevOps base domain to `1.2.3.4.nip.io` or `1.2.3.4.nip.io`. Once set up, all requests will hit the load balancer, which in turn will route @@ -148,6 +148,56 @@ If GitLab is installed using the [GitLab Omnibus Helm Chart], there are two options: provide a static IP, or have one assigned. For more information see the relevant docs on the [network prerequisites](../../install/kubernetes/gitlab_omnibus.md#networking-prerequisites). +## Using multiple Kubernetes clusters **[PREMIUM]** + +When using Auto DevOps, you may want to deploy different environments to +different Kubernetes clusters. This is possible due to the 1:1 connection that +[exists between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters). + +In the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml) +(used behind the scenes by Auto DevOps), there are currently 3 defined environment names that you need to know: + +- `review/` (every environment starting with `review/`) +- `staging` +- `production` + +Those environments are tied to jobs that use [Auto Deploy](#auto-deploy), so +except for the environment scope, they would also need to have a different +domain they would be deployed to. This is why you need to define a separate +`AUTO_DEVOPS_DOMAIN` variable for all the above +[based on the environment](../../ci/variables/README.md#limiting-environment-scopes-of-variables). + +The following table is an example of how the three different clusters would +be configured. + +| Cluster name | Cluster environment scope | `AUTO_DEVOPS_DOMAIN` variable value | Variable environment scope | Notes | +| ------------ | -------------- | ----------------------------- | ------------- | ------ | +| review | `review/*` | `review.example.com` | `review/*` | The review cluster which will run all [Review Apps](../../ci/review_apps/index.md). `*` is a wildcard, which means it will be used by every environment name starting with `review/`. | +| staging | `staging` | `staging.example.com` | `staging` | (Optional) The staging cluster which will run the deployments of the staging environments. You need to [enable it first](#deploy-policy-for-staging-and-production-environments). | +| production | `production` | `example.com` | `production` | The production cluster which will run the deployments of the production environment. You can use [incremental rollouts](#incremental-rollout-to-production). | + +To add a different cluster for each environment: + +1. Navigate to your project's **Operations > Kubernetes** and create the Kubernetes clusters + with their respective environment scope as described from the table above. + +  + +1. After the clusters are created, navigate to each one and install Helm Tiller + and Ingress. +1. Make sure you have [configured your DNS](#auto-devops-base-domain) with the + specified Auto DevOps domains. +1. Navigate to your project's **Settings > CI/CD > Variables** and add + the `AUTO_DEVOPS_DOMAIN` variables with their respective environment + scope. + +  + +Now that all is configured, you can test your setup by creating a merge request +and verifying that your app is deployed as a review app in the Kubernetes +cluster with the `review/*` environment scope. Similarly, you can check the +other environments. + ## Quick start If you are using GitLab.com, see our [quick start guide](quick_start_guide.md) @@ -156,18 +206,18 @@ Google Cloud. ## Enabling Auto DevOps -If you haven't done already, read the [prerequisites](#prerequisites) to make +If you haven't done already, read the [requirements](#requirements) to make full use of Auto DevOps. If this is your fist time, we recommend you follow the -[quick start guide](#quick-start). +[quick start guide](quick_start_guide.md). To enable Auto DevOps to your project: -1. Check that your project doesn't have a `.gitlab-ci.yml`, and remove it otherwise -1. Go to your project's **Settings > CI/CD > General pipelines settings** and - find the Auto DevOps section +1. Check that your project doesn't have a `.gitlab-ci.yml`, or remove it otherwise +1. Go to your project's **Settings > CI/CD > Auto DevOps** 1. Select "Enable Auto DevOps" 1. Optionally, but recommended, add in the [base domain](#auto-devops-base-domain) - that will be used by Kubernetes to deploy your application + that will be used by Kubernetes to [deploy your application](#auto-deploy) + and choose the [deployment strategy](#deployment-strategy) 1. Hit **Save changes** for the changes to take effect Once saved, an Auto DevOps pipeline will be triggered on the default branch. @@ -184,6 +234,24 @@ in **Admin Area > Settings > Continuous Integration and Deployment**. Doing that all the projects that haven't explicitly set an option will have Auto DevOps enabled by default. +### Deployment strategy + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/38542) in GitLab 11.0. + +You can change the deployment strategy used by Auto DevOps by going to your +project's **Settings > CI/CD > Auto DevOps**. + +The available options are: + +- **Continuous deployment to production** - enables [Auto Deploy](#auto-deploy) + by setting the [`STAGING_ENABLED`](#deploy-policy-for-staging-and-production-environments) and + [`INCREMENTAL_ROLLOUT_ENABLED`](#incremental-rollout-to-production) variables + to false. +- **Automatic deployment to staging, manual deployment to production** - sets the + [`STAGING_ENABLED`](#deploy-policy-for-staging-and-production-environments) and + [`INCREMENTAL_ROLLOUT_ENABLED`](#incremental-rollout-to-production) variables + to true, and the user is responsible for manually deploying to staging and production. + ## Stages of Auto DevOps The following sections describe the stages of Auto DevOps. Read them carefully @@ -222,8 +290,8 @@ tests, it's up to you to add them. ### Auto Code Quality -Auto Code Quality uses the open source -[`codeclimate` image](https://hub.docker.com/r/codeclimate/codeclimate/) to run +Auto Code Quality uses the +[Code Quality image](https://gitlab.com/gitlab-org/security-products/codequality) to run static analysis and other code checks on the current code. The report is created, and is uploaded as an artifact which you can later download and check out. @@ -232,7 +300,7 @@ In GitLab Starter, differences between the source and target branches are also [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html). -### Auto SAST +### Auto SAST **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.3. @@ -243,9 +311,9 @@ report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/sast.html). -### Auto Dependency Scanning +### Auto Dependency Scanning **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.7. @@ -256,7 +324,20 @@ report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/dependency_scanning.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/dependency_scanning.html). + +### Auto License Management **[ULTIMATE]** + +> Introduced in [GitLab Ultimate][ee] 11.0. + +License Management uses the +[License Management Docker image](https://gitlab.com/gitlab-org/security-products/license_management) +to search the project dependencies for their license. Once the +report is created, it's uploaded as an artifact which you can later download and +check out. + +In GitLab Ultimate, any licenses are also +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/license_management.html). ### Auto Container Scanning @@ -269,13 +350,13 @@ created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/container_scanning.html). ### Auto Review Apps NOTE: **Note:** This is an optional step, since many projects do not have a Kubernetes cluster -available. If the [prerequisites](#prerequisites) are not met, the job will +available. If the [requirements](#requirements) are not met, the job will silently be skipped. CAUTION: **Caution:** @@ -297,7 +378,7 @@ up in the merge request widget for easy discovery. When the branch is deleted, for example after the merge request is merged, the Review App will automatically be deleted. -### Auto DAST +### Auto DAST **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.4. @@ -308,9 +389,9 @@ issues. Once the report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/dast.html). -### Auto Browser Performance Testing +### Auto Browser Performance Testing **[PREMIUM]** > Introduced in [GitLab Premium][ee] 10.4. @@ -322,13 +403,14 @@ Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://h /direction ``` -In GitLab Premium, performance differences between the source and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). +In GitLab Premium, performance differences between the source +and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html). ### Auto Deploy NOTE: **Note:** This is an optional step, since many projects do not have a Kubernetes cluster -available. If the [prerequisites](#prerequisites) are not met, the job will +available. If the [requirements](#requirements) are not met, the job will silently be skipped. CAUTION: **Caution:** @@ -362,10 +444,19 @@ no longer be valid as soon as the deployment job finishes. This means that Kubernetes can run the application, but in case it should be restarted or executed somewhere else, it cannot be accessed again. +> [Introduced][ce-19507] in GitLab 11.0. + +For internal and private projects a [GitLab Deploy Token](../../user/project/deploy_tokens/index.md###gitlab-deploy-token) +will be automatically created, when Auto DevOps is enabled and the Auto DevOps settings are saved. This Deploy Token +can be used for permanent access to the registry. + +Note: **Note** +When the GitLab Deploy Token has been manually revoked, it won't be automatically created. + ### Auto Monitoring NOTE: **Note:** -Check the [prerequisites](#prerequisites) for Auto Monitoring to make this stage +Check the [requirements](#requirements) for Auto Monitoring to make this stage work. Once your application is deployed, Auto Monitoring makes it possible to monitor @@ -389,7 +480,7 @@ If you have installed GitLab using a different method, you need to: 1. [Deploy Prometheus](../../user/project/integrations/prometheus.md#configuring-your-own-prometheus-server-within-kubernetes) into your Kubernetes cluster 1. If you would like response metrics, ensure you are running at least version 0.9.0 of NGINX Ingress and - [enable Prometheus metrics](https://github.com/kubernetes/ingress/blob/master/examples/customization/custom-vts-metrics/nginx/nginx-vts-metrics-conf.yaml). + [enable Prometheus metrics](https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/customization/custom-vts-metrics-prometheus/nginx-vts-metrics-conf.yaml). 1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) the NGINX Ingress deployment to be scraped by Prometheus using `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. @@ -495,6 +586,19 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. | | `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-variables-environment-variables). Set it to use a custom database name. | | `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142` | +| `SAST_CONFIDENCE_LEVEL` | The minimum confidence level of security issues you want to be reported; `1` for Low, `2` for Medium, `3` for High; defaults to `3`.| +| `DEP_SCAN_DISABLE_REMOTE_CHECKS` | Whether remote Dependency Scanning checks are disabled; defaults to `"false"`. Set to `"true"` to disable checks that send data to GitLab central servers. [Read more about remote checks](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks).| +| `STAGING_ENABLED` | From GitLab 10.8, this variable can be used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). | +| `CANARY_ENABLED` | From GitLab 11.0, this variable can be used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). | +| `INCREMENTAL_ROLLOUT_ENABLED`| From GitLab 10.8, this variable can be used to enable an [incremental rollout](#incremental-rollout-to-production) of your application for the production environment. | +| `TEST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `test` job. If the variable is present, the job will not be created. | +| `CODEQUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `codequality` job. If the variable is present, the job will not be created. | +| `SAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast` job. If the variable is present, the job will not be created. | +| `DEPENDENCY_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dependency_scanning` job. If the variable is present, the job will not be created. | +| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast:container` job. If the variable is present, the job will not be created. | +| `REVIEW_DISABLED` | From GitLab 11.0, this variable can be used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs will not be created. | +| `DAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dast` job. If the variable is present, the job will not be created. | +| `PERFORMANCE_DISABLED` | From GitLab 11.0, this variable can be used to disable the `performance` job. If the variable is present, the job will not be created. | TIP: **Tip:** Set up the replica variables using a @@ -561,6 +665,94 @@ service: internalPort: 5000 ``` +#### Deploy policy for staging and production environments + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/160) +in GitLab 10.8. + +TIP: **Tip:** +You can also set this inside your [project's settings](#deployment-strategy). + +The normal behavior of Auto DevOps is to use Continuous Deployment, pushing +automatically to the `production` environment every time a new pipeline is run +on the default branch. However, there are cases where you might want to use a +staging environment and deploy to production manually. For this scenario, the +`STAGING_ENABLED` environment variable was introduced. + +If `STAGING_ENABLED` is defined in your project (e.g., set `STAGING_ENABLED` to +`1` as a secret variable), then the application will be automatically deployed +to a `staging` environment, and a `production_manual` job will be created for +you when you're ready to manually deploy to production. + +#### Deploy policy for canary environments **[PREMIUM]** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/171) +in GitLab 11.0. + +A [canary environment](https://docs.gitlab.com/ee/user/project/canary_deployments.html) can be used +before any changes are deployed to production. + +If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to +`1` as a secret variable) then two manual jobs will be created: + +- `canary` which will deploy the application to the canary environment +- `production_manual` which is to be used by you when you're ready to manually + deploy to production. + +#### Incremental rollout to production **[PREMIUM]** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5415) in GitLab 10.8. + +TIP: **Tip:** +You can also set this inside your [project's settings](#deployment-strategy). + +When you have a new version of your app to deploy in production, you may want +to use an incremental rollout to replace just a few pods with the latest code. +This will allow you to first check how the app is behaving, and later manually +increasing the rollout up to 100%. + +If `INCREMENTAL_ROLLOUT_ENABLED` is defined in your project (e.g., set +`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a secret variable), then instead of the +standard `production` job, 4 different +[manual jobs](../../ci/pipelines.md#manual-actions-from-the-pipeline-graph) +will be created: + +1. `rollout 10%` +1. `rollout 25%` +1. `rollout 50%` +1. `rollout 100%` + +The percentage is based on the `REPLICAS` variable and defines the number of +pods you want to have for your deployment. If you say `10`, and then you run +the `10%` rollout job, there will be `1` new pod + `9` old ones. + +To start a job, click on the play icon next to the job's name. You are not +required to go from `10%` to `100%`, you can jump to whatever job you want. +You can also scale down by running a lower percentage job, just before hitting +`100%`. Once you get to `100%`, you cannot scale down, and you'd have to roll +back by redeploying the old version using the +[rollback button](../../ci/environments.md#rolling-back-changes) in the +environment page. + +Below, you can see how the pipeline will look if the rollout or staging +variables are defined. + +- **Without `INCREMENTAL_ROLLOUT_ENABLED` and without `STAGING_ENABLED`** + +  + +- **Without `INCREMENTAL_ROLLOUT_ENABLED` and with `STAGING_ENABLED`** + +  + +- **With `INCREMENTAL_ROLLOUT_ENABLED` and without `STAGING_ENABLED`** + +  + +- **With `INCREMENTAL_ROLLOUT_ENABLED` and with `STAGING_ENABLED`** + +  + ## Currently supported languages NOTE: **Note:** @@ -650,3 +842,4 @@ curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https:/ [Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml [GitLab Omnibus Helm Chart]: ../../install/kubernetes/gitlab_omnibus.md [ee]: https://about.gitlab.com/products/ +[ce-19507]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19507 diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 15567715c98..61c04f3d9bb 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -1,7 +1,5 @@ # Auto DevOps: quick start guide -DANGER: Auto DevOps is currently in **Beta** and _not recommended for production use_. - > [Introduced][ce-37115] in GitLab 10.0. This is a step-by-step guide to deploying a project hosted on GitLab.com to @@ -23,6 +21,10 @@ page](https://gitlab.com/auto-devops-examples/minimal-ruby-app) and press the **Fork** button. Soon you should have a project under your namespace with the necessary files. +You can also start a new project from a +[GitLab project template](https://gitlab.com/gitlab-org/project-templates) if +you want to use a different language. + ## Setup your own cluster on Google Kubernetes Engine If you do not already have a Google Cloud account, create one at @@ -124,10 +126,10 @@ Next, a pipeline needs to be triggered. Since the test project doesn't have a manually visit `https://gitlab.com/<username>/minimal-ruby-app/pipelines/new`, where `<username>` is your username. -This will create a new pipeline with several jobs: `build`, `test`, `codequality`, +This will create a new pipeline with several jobs: `build`, `test`, `code_quality`, and `production`. The `build` job will create a Docker image with your new change and push it to the Container Registry. The `test` job will test your -changes, whereas the `codequality` job will run static analysis on your changes. +changes, whereas the `code_quality` job will run static analysis on your changes. Finally, the `production` job will deploy your changes to a production application. Once the deploy job succeeds you should be able to see your application by diff --git a/doc/topics/git/how_to_install_git/index.md b/doc/topics/git/how_to_install_git/index.md index 6c909a1ba86..58d86f7d387 100644 --- a/doc/topics/git/how_to_install_git/index.md +++ b/doc/topics/git/how_to_install_git/index.md @@ -4,6 +4,7 @@ author_gitlab: SeanPackham level: beginner article_type: user guide date: 2017-05-15 +description: 'This article describes how to install Git on macOS, Ubuntu Linux and Windows.' --- # Installing Git diff --git a/doc/university/README.md b/doc/university/README.md index 55865ac23e8..595fc480887 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -15,11 +15,11 @@ Would you like to contribute to GitLab University? Then please take a look at ou The curriculum is composed of GitLab videos, screencasts, presentations, projects and external GitLab content hosted on other services and has been organized into the following sections. -1. [GitLab Beginner](#beginner) -1. [GitLab Intermediate](#intermediate) -1. [GitLab Advanced](#advanced) -1. [External Articles](#external) -1. [Resources for GitLab Team Members](#team) +1. [GitLab Beginner](#1-gitlab-beginner) +1. [GitLab Intermediate](#2-gitlab-intermediate) +1. [GitLab Advanced](#3-gitlab-advanced) +1. [External Articles](#4-external-articles) +1. [Resources for GitLab Team Members](#5-resources-for-gitlab-team-members) --- @@ -28,7 +28,6 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 1.1. Version Control and Git 1. [Version Control Systems](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit#slide=id.g72f2e4906_2_29) -1. [Operating Systems and How Git Works](https://drive.google.com/a/gitlab.com/file/d/0B41DBToSSIG_OVYxVFJDOGI3Vzg/view?usp=sharing) 1. [Code School: An Introduction to Git](https://www.codeschool.com/account/courses/try-git) #### 1.2. GitLab Basics @@ -127,7 +126,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project 1. [IBM: Continuous Delivery vs Continuous Deployment - Video](https://www.youtube.com/watch?v=igwFj8PPSnw) 1. [Amazon: Transition to Continuous Delivery - Video](https://www.youtube.com/watch?v=esEFaY0FDKc) 2. [TechBeacon: Doing continuous delivery? Focus first on reducing release cycle times](https://techbeacon.com/doing-continuous-delivery-focus-first-reducing-release-cycle-times) -1. See **[Integrations](#integrations)** for integrations with other CI services. +1. See **[Integrations](#39-integrations)** for integrations with other CI services. #### 2.4. Workflow diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index 945d6a578b0..89516dba60b 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -557,10 +557,6 @@ Software that is hosted centrally and accessed on-demand (i.e. whenever you want This term is often used by people when they mean "Version Control." -#### SCLAU - -Abbreviation for SQO Count [Large And Up](https://about.gitlab.com/handbook/sales/#market-segmentation). This is the number of opportunities in large and strategic organizations passed from marketing to sales. - ### Scrum An Agile [framework](https://www.scrum.org/Resources/What-is-Scrum) designed to typically help complete complex software projects. It's made up of several parts: product requirements backlog, sprint planning, sprint (development), sprint review, and retrospec (analyzing the sprint). The goal is to end up with potentially shippable products. diff --git a/doc/university/training/gitlab_flow.md b/doc/university/training/gitlab_flow.md index 02a6ad48a38..d7bc7bda43f 100644 --- a/doc/university/training/gitlab_flow.md +++ b/doc/university/training/gitlab_flow.md @@ -2,39 +2,31 @@ comments: false --- -# GitLab Flow +# What is the GitLab Flow - A simplified branching strategy - All features and fixes first go to master - Allows for 'production' or 'stable' branches - Bug fixes/hot fix patches are cherry-picked from master ---- - -# Feature branches +## Feature branches - Create a feature/bugfix branch to do all work - Use merge requests to merge to master  ---- - -# Production branch +## Production branch - One, long-running production release branch as opposed to individual stable branches - Consider creating a tag for each version that gets deployed ---- - -# Production branch +## Production branch  ---- - -# Release branch +## Release branch - Useful if you release software to customers - When preparing a new release, create stable branch @@ -43,15 +35,11 @@ comments: false - Cherry-pick critical bug fixes to stable branch for patch release - Never commit bug fixes directly to stable branch ---- - -# Release branch +## Release branch  ---- - -# More details +## More details -Blog post on 'GitLab Flow' at -[http://doc.gitlab.com/ee/workflow/gitlab_flow.html](http://doc.gitlab.com/ee/workflow/gitlab_flow.html) +For more information read through the [GitLab Flow](../../workflow/gitlab_flow.md) +documentation. diff --git a/doc/university/training/topics/gitlab_flow.md b/doc/university/training/topics/gitlab_flow.md index b8049b5c80e..f6006ce0a60 100644 --- a/doc/university/training/topics/gitlab_flow.md +++ b/doc/university/training/topics/gitlab_flow.md @@ -1,57 +1,3 @@ --- -comments: false +redirect_to: '../gitlab_flow.md' --- - -# GitLab Flow - ----------- - -- A simplified branching strategy -- All features and fixes first go to master -- Allows for 'production' or 'stable' branches -- Bug fixes/hot fix patches are cherry-picked from master - ----------- - -### Feature branches - -- Create a feature/bugfix branch to do all work -- Use merge requests to merge to master - - - ----------- - -## Production branch - -- One, long-running production release branch - as opposed to individual stable branches -- Consider creating a tag for each version that gets deployed - ----------- - -## Production branch - - - ----------- - -## Release branch - -- Useful if you release software to customers -- When preparing a new release, create stable branch - from master -- Consider creating a tag for each version -- Cherry-pick critical bug fixes to stable branch for patch release -- Never commit bug fixes directly to stable branch - ----------- - - - ----------- - -## More details - -Blog post on 'GitLab Flow' at -[http://doc.gitlab.com/ee/workflow/gitlab_flow.html](http://doc.gitlab.com/ee/workflow/gitlab_flow.html) diff --git a/doc/university/training/topics/merge_requests.md b/doc/university/training/topics/merge_requests.md index 4e8c9de85a1..d7b771cd87b 100644 --- a/doc/university/training/topics/merge_requests.md +++ b/doc/university/training/topics/merge_requests.md @@ -2,7 +2,7 @@ comments: false --- -# Merge requests +# Code review and collaboration with Merge Requests ---------- diff --git a/doc/update/10.6-to-10.7.md b/doc/update/10.6-to-10.7.md index 4a76ae14d2e..4efbb8c65cf 100644 --- a/doc/update/10.6-to-10.7.md +++ b/doc/update/10.6-to-10.7.md @@ -80,8 +80,8 @@ More information can be found on the [yarn website](https://yarnpkg.com/en/docs/ ### 5. Update Go -NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go -1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. +NOTE: GitLab 9.2 and higher only supports Go 1.9 and dropped support for Go +1.5.x through 1.8.x. Be sure to upgrade your installation if necessary. You can check which version you are running with `go version`. @@ -91,11 +91,11 @@ Download and install Go: # Remove former Go installation folder sudo rm -rf /usr/local/go -curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz -echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ - sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +curl --remote-name --progress https://storage.googleapis.com/golang/go1.9.linux-amd64.tar.gz +echo 'd70eadefce8e160638a9a6db97f7192d8463069ab33138893ad3bf31b0650a79 go1.9.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.9.linux-amd64.tar.gz sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ -rm go1.8.3.linux-amd64.tar.gz +rm go1.9.linux-amd64.tar.gz ``` ### 6. Get latest code diff --git a/doc/update/10.7-to-10.8.md b/doc/update/10.7-to-10.8.md new file mode 100644 index 00000000000..13101a987f4 --- /dev/null +++ b/doc/update/10.7-to-10.8.md @@ -0,0 +1,362 @@ +--- +comments: false +--- + +# From 10.7 to 10.8 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +NOTE: If you installed GitLab from source, make sure `rsync` is installed. + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download Ruby and compile it: + + ```bash + mkdir /tmp/ruby && cd /tmp/ruby + curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.7.tar.gz + echo '540996fec64984ab6099e34d2f5820b14904f15a ruby-2.3.7.tar.gz' | shasum -c - && tar xzf ruby-2.3.7.tar.gz + cd ruby-2.3.7 + + ./configure --disable-install-rdoc + make + sudo make install + ``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab utilizes [webpack](http://webpack.js.org) to compile frontend assets. +This requires a minimum version of node v6.0.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v6.0.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + +GitLab also requires the use of yarn `>= v1.2.0` to manage JavaScript +dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all --prune +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-8-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-8-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-7-stable:config/gitlab.yml.example origin/10-8-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-7-stable:lib/support/nginx/gitlab-ssl origin/10-8-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-7-stable:lib/support/nginx/gitlab origin/10-8-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-8-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-8-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-7-stable:lib/support/init.d/gitlab.default.example origin/10-8-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (10.7) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 10.6 to 10.7](10.6-to-10.7.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-8-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-8-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/update/10.8-to-11.0.md b/doc/update/10.8-to-11.0.md new file mode 100644 index 00000000000..78a47ab939f --- /dev/null +++ b/doc/update/10.8-to-11.0.md @@ -0,0 +1,362 @@ +--- +comments: false +--- + +# From 10.8 to 11.0 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +NOTE: If you installed GitLab from source, make sure `rsync` is installed. + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 11.0 and higher only support Ruby 2.4.x and dropped support for Ruby 2.3.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download Ruby and compile it: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.4.tar.gz +echo 'ec82b0d53bd0adad9b19e6b45e44d54e9ec3f10c ruby-2.4.4.tar.gz' | shasum -c - && tar xzf ruby-2.4.4.tar.gz +cd ruby-2.4.4 + +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab utilizes [webpack](http://webpack.js.org) to compile frontend assets. +This requires a minimum version of node v6.0.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v6.0.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + +GitLab also requires the use of yarn `>= v1.2.0` to manage JavaScript +dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all --prune +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 11-0-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 11-0-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags --prune +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-8-stable:config/gitlab.yml.example origin/11-0-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-8-stable:lib/support/nginx/gitlab-ssl origin/11-0-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-8-stable:lib/support/nginx/gitlab origin/11-0-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-0-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-0-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-8-stable:lib/support/init.d/gitlab.default.example origin/11-0-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (10.8) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 10.7 to 10.8](10.7-to-10.8.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-0-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/11-0-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md index 9e2a89ebdf6..e383142c33e 100644 --- a/doc/user/admin_area/labels.md +++ b/doc/user/admin_area/labels.md @@ -1,4 +1,4 @@ -# Labels +# Labels administration **[CORE ONLY]** ## Default Labels diff --git a/doc/user/admin_area/settings/img/enforce_terms.png b/doc/user/admin_area/settings/img/enforce_terms.png Binary files differnew file mode 100644 index 00000000000..3d93e1cc891 --- /dev/null +++ b/doc/user/admin_area/settings/img/enforce_terms.png diff --git a/doc/user/admin_area/settings/img/respond_to_terms.png b/doc/user/admin_area/settings/img/respond_to_terms.png Binary files differnew file mode 100644 index 00000000000..d0d086c3498 --- /dev/null +++ b/doc/user/admin_area/settings/img/respond_to_terms.png diff --git a/doc/user/admin_area/settings/img/sign_up_terms.png b/doc/user/admin_area/settings/img/sign_up_terms.png Binary files differnew file mode 100644 index 00000000000..4cac9d426c9 --- /dev/null +++ b/doc/user/admin_area/settings/img/sign_up_terms.png diff --git a/doc/user/admin_area/settings/terms.md b/doc/user/admin_area/settings/terms.md new file mode 100644 index 00000000000..aa817c9a209 --- /dev/null +++ b/doc/user/admin_area/settings/terms.md @@ -0,0 +1,51 @@ +# Enforce accepting Terms of Service + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18570) +> in [GitLab Core](https://about.gitlab.com/pricing/) 10.8 + +## Configuration + +When it is required for all users of the GitLab instance to accept the +Terms of Service, this can be configured by an admin on the settings +page: + +. + +The terms itself can be entered using Markdown. For each update to the +terms, a new version is stored. When a user accepts or declines the +terms, GitLab will keep track of which version they accepted or +declined. + +When an admin enables this feature, they will automattically be +directed to the page to accept the terms themselves. After they +accept, they will be directed back to the settings page. + +## New registrations + +When this feature is enabled, a checkbox will be available in the +sign-up form. + + + +This checkbox will be required during sign up. + +Users can review the terms entered in the admin panel before +accepting. The page will be opened in a new window so they can +continue their registration afterwards. + +## Accepting terms + +When this feature was enabled, the users that have not accepted the +terms of service will be presented with a screen where they can either +accept or decline the terms. + + + +When the user accepts the terms, they will be directed to where they +were going. After a sign-in or sign-up this will most likely be the +dashboard. + +When the user was already logged in when the feature was turned on, +they will be asked to accept the terms on their next interaction. + +When a user declines the terms, they will be signed out. diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 159109e8954..9b0ff02f227 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -11,7 +11,7 @@ You can leave a comment in the following places: - commit diffs The comment area supports [Markdown] and [quick actions]. One can edit their -own comment at any time, and anyone with [Master access level][permissions] or +own comment at any time, and anyone with [Maintainer access level][permissions] or higher can also edit a comment made by someone else. You could also reply to the notification email in order to reply to a comment, @@ -253,7 +253,7 @@ to newer issues or merge requests. - The people participating in the discussion are trolling, abusive, or otherwise being unproductive. -In these cases, a user with Master permissions or higher in the project can lock (and unlock) +In these cases, a user with Maintainer permissions or higher in the project can lock (and unlock) an issue or a merge request, using the "Lock" section in the sidebar: | Unlock | Lock | diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 7baccb796c6..d054561d5f3 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -60,6 +60,7 @@ Below are the current settings regarding [GitLab CI/CD](../../ci/README.md). | Setting | GitLab.com | Default | | ----------- | ----------------- | ------------- | | Artifacts maximum size | 1G | 100M | +| Artifacts [expiry time](../../ci/yaml/README.md#artifacts-expire_in) | kept forever | deleted after 30 days unless otherwise specified | ## Repository size limit @@ -75,7 +76,6 @@ Shared Runners on GitLab.com run in [autoscale mode] and powered by Google Cloud Platform and DigitalOcean. Autoscaling means reduced waiting times to spin up CI/CD jobs, and isolated VMs for each project, thus maximizing security. - They're free to use for public open source projects and limited to 2000 CI minutes per month per group for private projects. Read about all [GitLab.com plans](https://about.gitlab.com/pricing/). @@ -90,6 +90,10 @@ ephemeral instances with 3.75GB of RAM, CoreOS and the latest Docker Engine installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default region of the VMs is US East1. +Jobs handled by the shared Runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), +**will be timed out after 3 hours**, regardless of the timeout configured in a +project. Check the issues [4010] and [4070] for the reference. + Below are the shared Runners settings. | Setting | GitLab.com | Default | @@ -340,3 +344,5 @@ High Performance TCP/HTTP Load Balancer: [mailgun]: https://www.mailgun.com/ "Mailgun website" [sidekiq]: http://sidekiq.org/ "Sidekiq website" [unicorn-worker-killer]: https://rubygems.org/gems/unicorn-worker-killer "unicorn-worker-killer" +[4010]: https://gitlab.com/gitlab-com/infrastructure/issues/4010 "Find a good value for maximum timeout for Shared Runners" +[4070]: https://gitlab.com/gitlab-com/infrastructure/issues/4070 "Configure per-runner timeout for shared-runners-manager-X on GitLab.com" diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 88f4bb2ee04..e6bf32a2dc5 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -40,20 +40,20 @@ In GitLab, a namespace is a unique name to be used as a user name, a group name, - `http://gitlab.example.com/groupname` - `http://gitlab.example.com/groupname/subgroup_name` -For example, consider a user called John: +For example, consider a user named Alex: -1. John creates his account on GitLab.com with the username `john`; -his profile will be accessed under `https://gitlab.example.com/john` -1. John creates a group for his team with the groupname `john-team`; -his group and its projects will be accessed under `https://gitlab.example.com/john-team` -1. John creates a subgroup of `john-team` with the subgroup name `marketing`; -his subgroup and its projects will be accessed under `https://gitlab.example.com/john-team/marketing` +1. Alex creates an account on GitLab.com with the username `alex`; +their profile will be accessed under `https://gitlab.example.com/alex` +1. Alex creates a group for their team with the groupname `alex-team`; +the group and its projects will be accessed under `https://gitlab.example.com/alex-team` +1. Alex creates a subgroup of `alex-team` with the subgroup name `marketing`; +this subgroup and its projects will be accessed under `https://gitlab.example.com/alex-team/marketing` By doing so: -- Any team member mentions John with `@john` -- John mentions everyone from his team with `@john-team` -- John mentions only his marketing team with `@john-team/marketing` +- Any team member mentions Alex with `@alex` +- Alex mentions everyone from their team with `@alex-team` +- Alex mentions only the marketing team with `@alex-team/marketing` ## Issues and merge requests within a group @@ -125,7 +125,7 @@ side of your screen. --- -Group owners and masters will be notified of your request and will be able to approve or +Group owners and maintainers will be notified of your request and will be able to approve or decline it on the members page.  diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 02f8ef08117..08849ac1df4 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -145,8 +145,8 @@ permissions. For example, if User0 was first added to group `group-1/group-1-1` with Developer permissions, then they will inherit those permissions in every other subgroup -of `group-1/group-1-1`. To give them Master access to `group-1/group-1-1/group1-1-1`, -you would add them again in that group as Master. Removing them from that group, +of `group-1/group-1-1`. To give them Maintainer access to `group-1/group-1-1/group1-1-1`, +you would add them again in that group as Maintainer. Removing them from that group, the permissions will fallback to those of the ancestor group. ## Mentioning subgroups diff --git a/doc/user/index.md b/doc/user/index.md index 2494df46f1c..edf50019c2f 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -1,3 +1,7 @@ +--- +description: 'Read through the GitLab User documentation to learn how to use, configure, and customize GitLab and GitLab.com to your own needs.' +--- + # User documentation Welcome to GitLab! We're glad to have you here! @@ -19,7 +23,7 @@ documentation. GitLab is a fully integrated software development platform that enables you and your team to work cohesively, faster, transparently, and effectively, since the discussion of a new idea until taking that idea to production all -all the way through, from within the same platform. +the way through, from within the same platform. Please check this page for an overview on [GitLab's features](https://about.gitlab.com/features/). @@ -106,7 +110,7 @@ personal access tokens, authorized applications, etc. - [Authentication](../topics/authentication/index.md): Read through the authentication methods available in GitLab. - [Permissions](permissions.md): Learn the different set of permissions levels for each -user type (guest, reporter, developer, master, owner). +user type (guest, reporter, developer, maintainer, owner). - [Feature highlight](feature_highlight.md): Learn more about the little blue dots around the app that explain certain features @@ -157,13 +161,13 @@ such as Trello, JIRA, etc. ## Webhooks -Configure [webhooks](project/integrations/webhooks.html) to listen for +Configure [webhooks](project/integrations/webhooks.md) to listen for specific events like pushes, issues or merge requests. GitLab will send a POST request with data to the webhook URL. ## API -Automate GitLab via [API](../api/README.html). +Automate GitLab via [API](../api/README.md). ## Git and GitLab diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 650d60f1585..8e87c896a72 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -3,13 +3,15 @@ ## GitLab Flavored Markdown (GFM) > **Note:** -Not all of the GitLab-specific extensions to Markdown that are described in -this document currently work on our documentation website. +> Not all of the GitLab-specific extensions to Markdown that are described in +> this document currently work on our documentation website. > -For the best result, we encourage you to check this document out as rendered +> For the best result, we encourage you to check this document out as rendered by GitLab: [markdown.md] -_GitLab uses the [Redcarpet Ruby library][redcarpet] for Markdown processing._ +_GitLab uses (as of 11.1) the [CommonMark Ruby Library][commonmarker] for Markdown processing of all new issues, merge requests, comments, and other Markdown content in the GitLab system. Previous content and Markdown files `.md` in the repositories are still processed using the [Redcarpet Ruby library][redcarpet]._ + +_Where there are significant differences, we will try to call them out in this document._ GitLab uses "GitLab Flavored Markdown" (GFM). It extends the standard Markdown in a few significant ways to add some useful functionality. It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/). @@ -21,7 +23,7 @@ You can use GFM in the following areas: - milestones - snippets (the snippet must be named with a `.md` extension) - wiki pages -- markdown documents inside the repository +- markdown documents inside the repository (currently only rendered by Redcarpet) You can also use other rich text files in GitLab. You might have to install a dependency to do so. Please see the [github-markup gem readme](https://github.com/gitlabhq/markup#markups) for more information. @@ -34,7 +36,7 @@ https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#newline GFM honors the markdown specification in how [paragraphs and line breaks are handled](https://daringfireball.net/projects/markdown/syntax#p). A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines. -Line-breaks, or softreturns, are rendered if you end a line with two or more spaces: +Line-breaks, or soft returns, are rendered if you end a line with two or more spaces: [//]: # (Do *NOT* remove the two ending whitespaces in the following line.) [//]: # (They are needed for the Markdown text to render correctly.) @@ -197,7 +199,7 @@ https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline- With inline diffs tags you can display {+ additions +} or [- deletions -]. -The wrapping tags can be either curly braces or square brackets [+ additions +] or {- deletions -}. +The wrapping tags can be either curly braces or square brackets: [+ additions +] or {- deletions -}. Examples: @@ -228,7 +230,7 @@ https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#emoji You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. - If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up on the supported codes. + If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes. Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup: @@ -238,7 +240,7 @@ Sometimes you want to :monkey: around a bit and add some :star2: to your :speech You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. -If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up on the supported codes. +If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes. Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup: @@ -394,17 +396,17 @@ Color written inside backticks will be followed by a color "chip". Examples: - `#F00` - `#F00A` - `#FF0000` - `#FF0000AA` - `RGB(0,255,0)` - `RGB(0%,100%,0%)` - `RGBA(0,255,0,0.7)` - `HSL(540,70%,50%)` + `#F00` + `#F00A` + `#FF0000` + `#FF0000AA` + `RGB(0,255,0)` + `RGB(0%,100%,0%)` + `RGBA(0,255,0,0.7)` + `HSL(540,70%,50%)` `HSLA(540,70%,50%,0.7)` -Becomes: +Become: `#F00` `#F00A` @@ -414,7 +416,7 @@ Becomes: `RGB(0%,100%,0%)` `RGBA(0,255,0,0.7)` `HSL(540,70%,50%)` -`HSLA(540,70%,50%,0.7)` +`HSLA(540,70%,50%,0.7)` #### Supported formats: @@ -481,14 +483,14 @@ Alt-H2 All Markdown-rendered headers automatically get IDs, except in comments. -On hover a link to those IDs becomes visible to make it easier to copy the link to the header to give it to someone else. +On hover, a link to those IDs becomes visible to make it easier to copy the link to the header to give it to someone else. The IDs are generated from the content of the header according to the following rules: -1. All text is converted to lowercase -1. All non-word text (e.g., punctuation, HTML) is removed -1. All spaces are converted to hyphens -1. Two or more hyphens in a row are converted to one +1. All text is converted to lowercase. +1. All non-word text (e.g., punctuation, HTML) is removed. +1. All spaces are converted to hyphens. +1. Two or more hyphens in a row are converted to one. 1. If a header with the same ID has already been generated, a unique incrementing number is appended, starting at 1. @@ -500,6 +502,7 @@ For example: # This header has Unicode in it: 한글 ## This header has spaces in it ### This header has spaces in it +## This header has 3.5 in it (and parentheses) ``` Would generate the following link IDs: @@ -509,11 +512,14 @@ Would generate the following link IDs: 1. `this-header-has-unicode-in-it-한글` 1. `this-header-has-spaces-in-it` 1. `this-header-has-spaces-in-it-1` +1. `this-header-has-3-5-in-it-and-parentheses` Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. ### Emphasis +Examples: + ```no-highlight Emphasis, aka italics, with *asterisks* or _underscores_. @@ -524,6 +530,8 @@ Combined emphasis with **asterisks and _underscores_**. Strikethrough uses two tildes. ~~Scratch this.~~ ``` +Become: + Emphasis, aka italics, with *asterisks* or _underscores_. Strong emphasis, aka bold, with **asterisks** or __underscores__. @@ -534,12 +542,14 @@ Strikethrough uses two tildes. ~~Scratch this.~~ ### Lists +Examples: + ```no-highlight 1. First ordered list item 2. Another item - * Unordered sub-list. + * Unordered sub-list. 1. Actual numbers don't matter, just that it's a number - 1. Ordered sub-list + 1. Ordered sub-list 4. And another item. * Unordered list can use asterisks @@ -547,11 +557,13 @@ Strikethrough uses two tildes. ~~Scratch this.~~ + Or pluses ``` +Become: + 1. First ordered list item 2. Another item - * Unordered sub-list. + * Unordered sub-list. 1. Actual numbers don't matter, just that it's a number - 1. Ordered sub-list + 1. Ordered sub-list 4. And another item. * Unordered list can use asterisks @@ -559,33 +571,45 @@ Strikethrough uses two tildes. ~~Scratch this.~~ + Or pluses If a list item contains multiple paragraphs, -each subsequent paragraph should be indented with four spaces. +each subsequent paragraph should be indented to the same level as the start of the list item text (_Redcarpet: paragraph should be indented with four spaces._) + +Example: ```no-highlight -1. First ordered list item +1. First ordered list item - Second paragraph of first item. -2. Another item + Second paragraph of first item. + +2. Another item ``` +Becomes: + 1. First ordered list item - Second paragraph of first item. + Paragraph of first item. + 2. Another item -If the second paragraph isn't indented with four spaces, -the second list item will be incorrectly labeled as `1`. +If the paragraph of the first item is not indented with the proper number of spaces, +the paragraph will appear outside the list, instead of properly indented under the list item. + +Example: ```no-highlight 1. First ordered list item - Second paragraph of first item. + Paragraph of first item. + 2. Another item ``` +Becomes: + 1. First ordered list item - Second paragraph of first item. + Paragraph of first item. + 2. Another item ### Links @@ -620,6 +644,8 @@ will point the link to `wikis/style` when the link is inside of a wiki markdown ### Images +Examples: + Here's our logo (hover to see the title text): Inline-style: @@ -630,6 +656,8 @@ will point the link to `wikis/style` when the link is inside of a wiki markdown [logo]: img/markdown_logo.png +Become: + Here's our logo: Inline-style: @@ -644,6 +672,8 @@ Reference-style: ### Blockquotes +Examples: + ```no-highlight > Blockquotes are very handy in email to emulate reply text. > This line is part of the same quote. @@ -653,6 +683,8 @@ Quote break. > This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. ``` +Become: + > Blockquotes are very handy in email to emulate reply text. > This line is part of the same quote. @@ -666,6 +698,8 @@ You can also use raw HTML in your Markdown, and it'll mostly work pretty well. See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/1.11.0/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span`, `abbr`, `details` and `summary` elements. +Examples: + ```no-highlight <dl> <dt>Definition list</dt> @@ -676,6 +710,8 @@ See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubyd </dl> ``` +Become: + <dl> <dt>Definition list</dt> <dd>Is something people use sometimes.</dd> @@ -691,25 +727,31 @@ Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.or <p> <details> <summary>Click me to collapse/fold.</summary> -These details will remain hidden until expanded. + +These details <em>will</em> remain <strong>hidden</strong> until expanded. <pre><code>PASTE LOGS HERE</code></pre> + </details> </p> -**Note:** Unfortunately Markdown is not supported inside these tags, as described by the [markdown specification](https://daringfireball.net/projects/markdown/syntax#html). You can work around this by using HTML, for example you can use `<pre><code>` tags instead of [code fences](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting). +**Note:** Markdown inside these tags is supported, as long as you have a blank link after the `</summary>` tag and before the `</details>` tag, as shown in the example. _Redcarpet does not support Markdown inside these tags. You can work around this by using HTML, for example you can use `<pre><code>` tags instead of [code fences](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting)._ ```html <details> <summary>Click me to collapse/fold.</summary> -These details will remain hidden until expanded. -<pre><code>PASTE LOGS HERE</code></pre> +These details _will_ remain **hidden** until expanded. + + PASTE LOGS HERE + </details> ``` ### Horizontal Rule +Examples: + ``` Three or more... @@ -726,6 +768,8 @@ ___ Underscores ``` +Become: + Three or more... --- @@ -742,10 +786,12 @@ Underscores ### Line Breaks -My basic recommendation for learning how line breaks work is to experiment and discover -- hit <Enter> once (i.e., insert one newline), then hit it twice (i.e., insert two newlines), see what happens. You'll soon learn to get what you want. "Markdown Toggle" is your friend. +A good way to learn how line breaks work is to experiment and discover -- hit <kbd>Enter</kbd> once (i.e., insert one newline), then hit it twice (i.e., insert two newlines), see what happens. You'll soon learn to get what you want. The "Preview" tab is your friend. Here are some things to try out: +Examples: + ``` Here's a line for us to start with. @@ -760,6 +806,8 @@ This line is *on its own line*, because the previous line ends with two spaces. spaces. ``` +Become: + Here's a line for us to start with. This line is separated from the one above by two newlines, so it will be a *separate paragraph*. @@ -774,7 +822,9 @@ spaces. ### Tables -Tables aren't part of the core Markdown spec, but they are part of GFM and Markdown Here supports them. +Tables aren't part of the core Markdown spec, but they are part of GFM. + +Example: ``` | header 1 | header 2 | @@ -783,18 +833,18 @@ Tables aren't part of the core Markdown spec, but they are part of GFM and Markd | cell 3 | cell 4 | ``` -Code above produces next output: +Becomes: | header 1 | header 2 | | -------- | -------- | | cell 1 | cell 2 | | cell 3 | cell 4 | -**Note** +**Note:** The row of dashes between the table header and body must have at least three dashes in each column. -The row of dashes between the table header and body must have at least three dashes in each column. +By including colons in the header row, you can align the text within that column. -By including colons in the header row, you can align the text within that column: +Example: ``` | Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | @@ -803,6 +853,8 @@ By including colons in the header row, you can align the text within that column | Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | ``` +Becomes: + | Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | | :----------- | :------: | ------------: | :----------- | :------: | ------------: | | Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | @@ -810,13 +862,29 @@ By including colons in the header row, you can align the text within that column ### Footnotes +Example: + ``` You can add footnotes to your text as follows.[^2] [^2]: This is my awesome footnote. ``` +Becomes: + You can add footnotes to your text as follows.[^2] +### Superscripts / Subscripts + +CommonMark and GFM currently do not support the superscript syntax ( `x^2` ) that Redcarpet does. You can use the standard HTML syntax for superscripts and subscripts. + +``` +The formula for water is H<sub>2</sub>O +while the equation for the theory of relativity is E = mc<sup>2</sup>. +``` + +The formula for water is H<sub>2</sub>O while the equation for the theory of relativity is E = mc<sup>2</sup>. + + ## Wiki-specific Markdown The following examples show how links inside wikis behave. @@ -908,3 +976,4 @@ A link starting with a `/` is relative to the wiki root. [katex]: https://github.com/Khan/KaTeX "KaTeX website" [katex-subset]: https://github.com/Khan/KaTeX/wiki/Function-Support-in-KaTeX "Macros supported by KaTeX" [asciidoctor-manual]: http://asciidoctor.org/docs/user-manual/#activating-stem-support "Asciidoctor user manual" +[commonmarker]: https://github.com/gjtorikian/commonmarker diff --git a/doc/user/permissions.md b/doc/user/permissions.md index a9ba2a51242..b36b0b4f757 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -1,3 +1,7 @@ +--- +description: 'Understand and explore the user permission levels in GitLab, and what features each of them grants you access to.' +--- + # Permissions Users have different abilities depending on the access level they have in a @@ -23,7 +27,7 @@ See our [product handbook on permissions](https://about.gitlab.com/handbook/prod The following table depicts the various user permission levels in a project. -| Action | Guest | Reporter | Developer | Master | Owner | +| Action | Guest | Reporter | Developer |Maintainer| Owner | |---------------------------------------|---------|------------|-------------|----------|--------| | Create new issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | | Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | @@ -37,7 +41,8 @@ The following table depicts the various user permission levels in a project. | View wiki pages | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | | Pull project code | [^1] | ✓ | ✓ | ✓ | ✓ | | Download project | [^1] | ✓ | ✓ | ✓ | ✓ | -| Assign issues and merge requests | | ✓ | ✓ | ✓ | ✓ | +| Assign issues | | ✓ | ✓ | ✓ | ✓ | +| Assign merge requests | | | ✓ | ✓ | ✓ | | Label issues and merge requests | | ✓ | ✓ | ✓ | ✓ | | Create code snippets | | ✓ | ✓ | ✓ | ✓ | | Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | @@ -46,6 +51,9 @@ The following table depicts the various user permission levels in a project. | See a container registry | | ✓ | ✓ | ✓ | ✓ | | See environments | | ✓ | ✓ | ✓ | ✓ | | See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | +| Manage related issues **[STARTER]** | | ✓ | ✓ | ✓ | ✓ | +| Lock issue discussions | | ✓ | ✓ | ✓ | ✓ | +| Lock merge request discussions | | | ✓ | ✓ | ✓ | | Create new environments | | | ✓ | ✓ | ✓ | | Stop environments | | | ✓ | ✓ | ✓ | | Manage/Accept merge requests | | | ✓ | ✓ | ✓ | @@ -71,11 +79,12 @@ The following table depicts the various user permission levels in a project. | Edit project | | | | ✓ | ✓ | | Add deploy keys to project | | | | ✓ | ✓ | | Configure project hooks | | | | ✓ | ✓ | -| Manage runners | | | | ✓ | ✓ | +| Manage Runners | | | | ✓ | ✓ | | Manage job triggers | | | | ✓ | ✓ | | Manage variables | | | | ✓ | ✓ | -| Manage pages | | | | ✓ | ✓ | -| Manage pages domains and certificates | | | | ✓ | ✓ | +| Manage GitLab Pages | | | | ✓ | ✓ | +| Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | +| Remove GitLab Pages | | | | | ✓ | | Manage clusters | | | | ✓ | ✓ | | Edit comments (posted by any user) | | | | ✓ | ✓ | | Switch visibility level | | | | | ✓ | @@ -85,6 +94,7 @@ The following table depicts the various user permission levels in a project. | Remove pages | | | | | ✓ | | Force push to protected branches [^4] | | | | | | | Remove protected branches [^4] | | | | | | +| View project Audit Events | | | | ✓ | ✓ | ## Project features permissions @@ -104,7 +114,7 @@ review, we've created protected branches. Read through the documentation on [protected branches](project/protected_branches.md) to learn more. -Additionally, you can allow or forbid users with Master and/or +Additionally, you can allow or forbid users with Maintainer and/or Developer permissions to push to a protected branch. Read through the documentation on [Allowed to Merge and Allowed to Push settings](project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings) to learn more. @@ -122,17 +132,12 @@ and drag issues around. Read though the [documentation on Issue Boards permissions](project/issue_board.md#permissions) to learn more. -### File Locking permissions - -> Available in [GitLab Premium](https://about.gitlab.com/products/). +### File Locking permissions **[PREMIUM]** The user that locks a file or directory is the only one that can edit and push their changes back to the repository where the locked objects are located. Read through the documentation on [permissions for File Locking](https://docs.gitlab.com/ee/user/project/file_lock.html#permissions-on-file-locking) to learn more. -File Locking is available in -[GitLab Premium](https://about.gitlab.com/products/) only. - ### Confidential Issues permissions Confidential issues can be accessed by reporters and higher permission levels, @@ -145,7 +150,7 @@ Any user can remove themselves from a group, unless they are the last Owner of the group. The following table depicts the various user permission levels in a group. -| Action | Guest | Reporter | Developer | Master | Owner | +| Action | Guest | Reporter | Developer | Maintainer | Owner | |-------------------------|-------|----------|-----------|--------|-------| | Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | | Edit group | | | | | ✓ | @@ -155,6 +160,12 @@ group. | Remove group | | | | | ✓ | | Manage group labels | | ✓ | ✓ | ✓ | ✓ | | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | +| View private group epic **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | +| View internal group epic **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View public group epic **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| Create/edit group epic **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | +| Delete group epic **[ULTIMATE]** | | | | | ✓ | +| View group Audit Events | | | | | ✓ | ### Subgroup permissions @@ -189,13 +200,34 @@ will find the option to flag the user as external. By default new users are not set as external users. This behavior can be changed by an administrator under **Admin > Application Settings**. +## Auditor users **[PREMIUM ONLY]** + +>[Introduced][ee-998] in [GitLab Premium][eep] 8.17. + +Auditor users are given read-only access to all projects, groups, and other +resources on the GitLab instance. + +An Auditor user should be able to access all projects and groups of a GitLab instance +with the permissions described on the documentation on [auditor users permissions](https://docs.gitlab.com/ee/administration/auditor_users.html#permissions-and-restrictions-of-an-auditor-user). + +[Read more about Auditor users.](https://docs.gitlab.com/ee/administration/auditor_users.html) + +## Project features + +Project features like wiki and issues can be hidden from users depending on +which visibility level you select on project settings. + +- Disabled: disabled for everyone +- Only team members: only team members will see even if your project is public or internal +- Everyone with access: everyone can see depending on your project visibility level + ## GitLab CI/CD permissions GitLab CI/CD permissions rely on the role the user has in GitLab. There are four permission levels in total: - admin -- master +- maintainer - developer - guest/reporter @@ -203,7 +235,7 @@ The admin user can perform any action on GitLab CI/CD in scope of the GitLab instance and project. In addition, all admins can use the admin interface under `/admin/runners`. -| Action | Guest, Reporter | Developer | Master | Admin | +| Action | Guest, Reporter | Developer |Maintainer| Admin | |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | @@ -225,7 +257,7 @@ Read all about the [new model and its implications][new-mod]. This table shows granted privileges for jobs triggered by specific types of users: -| Action | Guest, Reporter | Developer | Master | Admin | +| Action | Guest, Reporter | Developer |Maintainer| Admin | |---------------------------------------------|-----------------|-------------|----------|--------| | Run CI job | | ✓ | ✓ | ✓ | | Clone source and LFS from current project | | ✓ | ✓ | ✓ | @@ -258,23 +290,15 @@ for details about the pipelines security model. Since GitLab 8.15, LDAP user permissions can now be manually overridden by an admin user. Read through the documentation on [LDAP users permissions](https://docs.gitlab.com/ee/articles/how_to_configure_ldap_gitlab_ee/index.html#updating-user-permissions-new-feature) to learn more. -## Auditor users permissions - -> Available in [GitLab Premium](https://about.gitlab.com/products/). - -An Auditor user should be able to access all projects and groups of a GitLab instance -with the permissions described on the documentation on [auditor users permissions](https://docs.gitlab.com/ee/administration/auditor_users.html#permissions-and-restrictions-of-an-auditor-user). - -Auditor users are available in [GitLab Premium](https://about.gitlab.com/products/) -only. - [^1]: On public and internal projects, all users are able to perform this action [^2]: Guest users can only view the confidential issues they created themselves [^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** -[^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner +[^4]: Not allowed for Guest, Reporter, Developer, Maintainer, or Owner [^5]: Only if the job was triggered by the user [^6]: Only if user is not external one [^7]: Only if user is a member of the project [ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 [new-mod]: project/new_ci_build_permissions_model.md +[ee-998]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/998 +[eep]: https://about.gitlab.com/products/
\ No newline at end of file diff --git a/doc/user/profile/img/profil-preferences-navigation-theme.png b/doc/user/profile/img/profil-preferences-navigation-theme.png Binary files differnew file mode 100644 index 00000000000..7adaec33b60 --- /dev/null +++ b/doc/user/profile/img/profil-preferences-navigation-theme.png diff --git a/doc/user/profile/img/profile-preferences-syntax-themes.png b/doc/user/profile/img/profile-preferences-syntax-themes.png Binary files differnew file mode 100644 index 00000000000..719df9410fc --- /dev/null +++ b/doc/user/profile/img/profile-preferences-syntax-themes.png diff --git a/doc/user/profile/img/profile_settings_dropdown.png b/doc/user/profile/img/profile_settings_dropdown.png Binary files differindex a2c620642e2..99b06a1bf58 100644 --- a/doc/user/profile/img/profile_settings_dropdown.png +++ b/doc/user/profile/img/profile_settings_dropdown.png diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index 930e506802a..e028861a419 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -7,6 +7,34 @@ To navigate to your profile's preferences, click your avatar icon in the top right corner and select **Settings**. From there on, choose the **Preferences** tab. + + +## Navigation theme + +>**Note:** +Navigation themes have been re-introduced with [GitLab 10.0](https://about.gitlab.com/2017/09/22/gitlab-10-0-released/). + +The GitLab navigation theme setting allows you to personalize your GitLab experience. +You can choose from several color themes that add unique colors to the top navigation +and left side navigation. +Using individual color themes might help you differentiate between your different +GitLab instances. + +The default palette is Indigo. You can choose between 10 different themes: + +- Indigo +- Light Indigo +- Blue +- Light Blue +- Green +- Light Green +- Red +- Light Red +- Dark +- Light + + + ## Syntax highlighting theme >**Note:** @@ -16,7 +44,7 @@ list of supported languages visit the rouge website. Changing this setting allows you to customize the color theme when viewing any syntax highlighted code on GitLab. -The default one is **White**, and you can choose among 5 different colors: +The default syntax theme is White, and you can choose among 5 different colors: - White - Dark @@ -24,6 +52,8 @@ The default one is **White**, and you can choose among 5 different colors: - Solarized dark - Monokai + + ## Behavior The following settings allow you to customize the behavior of GitLab's layout @@ -52,16 +82,16 @@ You have 8 options here that you can use for your default dashboard view: - Assigned Issues - Assigned Merge Requests -### Project home page content +### Project overview content -The project home page content setting allows you to choose what content you want to +The project overview content setting allows you to choose what content you want to see on a project’s home page. You can choose between 3 options: -- Show the files and the readme (default) -- Show the readme -- Show the project’s activity +- Files and Readme (default) +- Readme +- Activity [rouge]: http://rouge.jneen.net/ "Rouge website" [todos]: ../../workflow/todos.md diff --git a/doc/user/project/bulk_editing.md b/doc/user/project/bulk_editing.md new file mode 100644 index 00000000000..324a7fa6603 --- /dev/null +++ b/doc/user/project/bulk_editing.md @@ -0,0 +1,17 @@ +# Bulk Editing + +>**Note:** +- A permission level of `Reporter` or higher is required in order to manage +issues. +- A permission level of `Developer` or higher is required in order to manage +merge requests. + +Fields across multiple issues or merge requests can be updated simutaneously by using the bulk edit feature. + +>**Note:** +- Bulk editing of issues and merge requests is only available at the project level. + +To access the feature, navigate to either the issue or merge request list for the project and click 'Edit Issues' or 'Edit Merge Requests'. This will cause a sidebar to be shown on the right-hand side of the screen, where the available, editable fields are displayed. Checkboxes will also appear to the left-hand side of each issue or merge request, ready to be selected. + +Once all items have been selected, choose the appropriate fields and their values from the sidebar and click 'Update All' to apply these changes. + diff --git a/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png b/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png Binary files differnew file mode 100644 index 00000000000..9a0559a19d4 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png b/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png Binary files differnew file mode 100644 index 00000000000..657ab0d9fa9 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/create_project.png b/doc/user/project/clusters/eks_and_gitlab/img/create_project.png Binary files differnew file mode 100644 index 00000000000..f3446131419 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/create_project.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png b/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png Binary files differnew file mode 100644 index 00000000000..d6c3b1b3a94 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/environment.png b/doc/user/project/clusters/eks_and_gitlab/img/environment.png Binary files differnew file mode 100644 index 00000000000..77d711ba8f6 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/environment.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/new_project.png b/doc/user/project/clusters/eks_and_gitlab/img/new_project.png Binary files differnew file mode 100644 index 00000000000..d401c4ac2bf --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/new_project.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png b/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png Binary files differnew file mode 100644 index 00000000000..5f9c9815c24 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png diff --git a/doc/user/project/clusters/eks_and_gitlab/index.md b/doc/user/project/clusters/eks_and_gitlab/index.md new file mode 100644 index 00000000000..2d8fdf0d1da --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/index.md @@ -0,0 +1,135 @@ +--- +author: Joshua Lambert +author_gitlab: joshlambert +level: intermediate +article_type: tutorial +date: 2018-06-05 +--- + +# Connecting and deploying to an Amazon EKS cluster + +## Introduction + +In this tutorial, we will show how easy it is to integrate an [Amazon EKS](https://aws.amazon.com/eks/) cluster with GitLab, and begin deploying applications. + +For an end-to-end walkthrough we will: +1. Start with a new project based on the sample Ruby on Rails template +1. Integrate an EKS cluster +1. Utilize [Auto DevOps](../../../../topics/autodevops/) to build, test, and deploy our application + +You will need: +1. An account on GitLab, like [GitLab.com](https://gitlab.com) +1. An Amazon EKS cluster +1. `kubectl` [installed and configured for access to the EKS cluster](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html#get-started-kubectl) + +If you don't have an Amazon EKS cluster, one can be created by following [the EKS getting started guide](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html). + +## Creating a new project + +On GitLab, create a new project by clicking on the `+` icon in the top navigation bar, and selecting `New project`. + + + +On the new project screen, click on the `Create from template` tab, and select `Use template` for the Ruby on Rails sample project. + +Give the project a name, and then select `Create project`. + + + +## Connecting the EKS cluster + +From the left side bar, hover over `Operations` and select `Kubernetes`, then click on `Add Kubernetes cluster`, and finally `Add an existing Kubernetes cluster`. + +A few details from the EKS cluster will be required to connect it to GitLab. + +1. A valid Kubernetes certificate and token are needed to authenticate to the EKS cluster. A pair is created by default, which can be used. Open a shell and use `kubectl` to retrieve them: + * List the secrets with `kubectl get secrets`, and one should named similar to `default-token-xxxxx`. Copy that token name for use below. + * Get the certificate with `kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 -D` + * Retrieve the token with `kubectl get secret <secret name> -o jsonpath="{['data']['token']}" | base64 -D`. +1. The API server endpoint is also required, so GitLab can connect to the cluster. This is displayed on the AWS EKS console, when viewing the EKS cluster details. + +You now have all the information needed to connect the EKS cluster: +* Kubernetes cluster name: Provide a name for the cluster to identify it within GitLab. +* Environment scope: Leave this as `*` for now, since we are only connecting a single cluster. +* API URL: Paste in the API server endpoint retrieved above. +* CA Certificate: Paste the certificate data from the earlier step, as-is. +* Paste the token value. Note on some versions of Kubernetes a trailing `%` is output, do not include it. +* Project namespace: This can be left blank to accept the default namespace, based on the project name. + + + +Click on `Add Kubernetes cluster`, the cluster is now connected to GitLab. At this point, [Kubernetes deployment variables](../#deployment-variables) will automatically be available during CI jobs, making it easy to interact with the cluster. + +If you would like to utilize your own CI/CD scripts to deploy to the cluster, you can stop here. + +## Disable Role Based-Access Control (RBAC) + +Presently, Auto DevOps and one-click app installs do not support [Kubernetes role-based access control](https://kubernetes.io/docs/reference/access-authn-authz/rbac/). Support is [being worked on](https://gitlab.com/groups/gitlab-org/-/epics/136), but in the interim RBAC must be disabled to utilize for these features. + +> **Note**: Disabling RBAC means that any application running in the cluster, or user who can authenticate to the cluster, has full API access. This is a [security concern](https://docs.gitlab.com/ee/user/project/clusters/#security-implications), and may not be desirable. + +To effectively disable RBAC, global permissions can be applied granting full access: + +```bash +kubectl create clusterrolebinding permissive-binding \ + --clusterrole=cluster-admin \ + --user=admin \ + --user=kubelet \ + --group=system:serviceaccounts +``` + +## Deploy services to the cluster + +GitLab supports one-click deployment of helpful services to the cluster, many of which support Auto DevOps. Back on the Kubernetes cluster screen in GitLab, a list of applications is now available to deploy. + +First install Helm Tiller, a package manager for Kubernetes. This enables deployment of the other applications. + + + +### Deploying NGINX Ingress (optional) + +Next, if you would like the deployed app to be reachable on the internet, deploy the Ingress. Note that this will also cause an [Elastic Load Balancer](https://aws.amazon.com/documentation/elastic-load-balancing/) to be created, which will incur additional AWS costs. + +Once installed, you may see a `?` for `Ingress IP Address`. This is because the created ELB is available at a DNS name, not an IP address. To get the DNS name, run: `kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -o jsonpath="{.status.loadBalancer.ingress[0].hostname}"`. Note, you may see a trailing `%` on some Kubernetes versions, do not include it. + +The Ingress is now available at this address, and will route incoming requests to the proper service based on the DNS name in the request. To support this, a wildcard DNS CNAME record should be created for the desired domain name. For example `*.myekscluster.com` would point to the Ingress hostname obtained earlier. + + + +### Deploying the GitLab Runner (optional) + +If the project is on GitLab.com, free shared runners are available and you do not have to deploy one. If a project specific runner is desired, or there are no shared runners, it is easy to deploy one. + +Simply click on the `Install` button for the GitLab Runner. It is important to note that the runner deployed is set as **privileged**, which means it essentially has root access to the underlying machine. This is required to build docker images, and so is on by default. + +### Deploying Prometheus (optional) + +GitLab is able to monitor applications automatically, utilizing [Prometheus](../../integrations/prometheus.html). Kubernetes container CPU and memory metrics are automatically collected, and response metrics are retrieved from NGINX Ingress as well. + +To enable monitoring, simply install Prometheus into the cluster with the `Install` button. + +## Create a default Storage Class + +Amazon EKS does not have a default Storage Class out of the box, which means requests for persistent volumes will not be automatically fulfilled. As part of Auto DevOps, the deployed Postgres instance requests persistent storage, and without a default storage class it will fail to start. + +If a default Storage Class does not already exist and is desired, follow Amazon's [short guide](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html) to create one. + +Alternatively, disable Postgres by setting the project variable [`POSTGRES_ENABLED`](../../../../topics/autodevops/#environment-variables) to `false`. + +## Deploy the app to EKS + +With RBAC disabled and services deployed, [Auto DevOps](https://docs.gitlab.com/ee/topics/autodevops/) can now be leveraged to build, test, and deploy the app. To enable, click on `Settings` in the left sidebar, then `CI/CD`. You will see a section for `Auto DevOps`, expand it. Click on the radio button to `Enable Auto DevOps`. + +If a wildcard DNS entry was created resolving to the Load Balancer, enter it in the `domain` field. Otherwise, the deployed app will not be externally available outside of the cluster. To save, click `Save changes`. + + + +A new pipeline will automatically be created, which will begin to build, test, and deploy the app. + +After the pipeline has finished, your app will be running in EKS and available to users. Click on `CI/CD` tab in the left navigation bar, and choose `Environments`. + + + +You will see a list of the environments and their deploy status, as well as options to browse to the app, view monitoring metrics, and even access a shell on the running pod. + +To learn more about Auto DevOps, review our [documentation](../../../../topics/autodevops/). diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index edb875bc7e6..48bb2e543c1 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -19,7 +19,7 @@ or provide the credentials to an [existing Kubernetes cluster](#adding-an-existi ## Adding and creating a new GKE cluster via GitLab NOTE: **Note:** -You need Master [permissions] and above to access the Kubernetes page. +You need Maintainer [permissions] and above to access the Kubernetes page. Before proceeding, make sure the following requirements are met: @@ -30,7 +30,7 @@ Before proceeding, make sure the following requirements are met: clusters on GKE. That would mean that a [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account) must be set up and that you have to have permissions to access it. -- You must have Master [permissions] in order to be able to access the +- You must have Maintainer [permissions] in order to be able to access the **Kubernetes** page. - You must have [Cloud Billing API](https://cloud.google.com/billing/) enabled - You must have [Resource Manager @@ -39,22 +39,22 @@ Before proceeding, make sure the following requirements are met: If all of the above requirements are met, you can proceed to create and add a new Kubernetes cluster that will be hosted on GKE to your project: -1. Navigate to your project's **CI/CD > Kubernetes** page. +1. Navigate to your project's **Operations > Kubernetes** page. 1. Click on **Add Kubernetes cluster**. -1. Click on **Create with GKE**. +1. Click on **Create with Google Kubernetes Engine**. 1. Connect your Google account if you haven't done already by clicking the **Sign in with Google** button. 1. Fill in the requested values: - - **Cluster name** (required) - The name you wish to give the cluster. - - **GCP project ID** (required) - The ID of the project you created in your GCP + - **Kubernetes cluster name** - The name you wish to give the cluster. + - **Environment scope** - The [associated environment](#setting-the-environment-scope) to this cluster. + - **Google Cloud Platform project** - The project you created in your GCP console that will host the Kubernetes cluster. This must **not** be confused - with the project name. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). + with the project ID. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). - **Zone** - The [zone](https://cloud.google.com/compute/docs/regions-zones/) under which the cluster will be created. - **Number of nodes** - The number of nodes you wish the cluster to have. - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types) of the Virtual Machine instance that the cluster will be based on. - - **Environment scope** - The [associated environment](#setting-the-environment-scope) to this cluster. 1. Finally, click the **Create Kubernetes cluster** button. After a few moments, your cluster should be created. If something goes wrong, @@ -66,11 +66,11 @@ enable the Cluster integration. ## Adding an existing Kubernetes cluster NOTE: **Note:** -You need Master [permissions] and above to access the Kubernetes page. +You need Maintainer [permissions] and above to access the Kubernetes page. To add an existing Kubernetes cluster to your project: -1. Navigate to your project's **CI/CD > Kubernetes** page. +1. Navigate to your project's **Operations > Kubernetes** page. 1. Click on **Add Kubernetes cluster**. 1. Click on **Add an existing Kubernetes cluster** and fill in the details: - **Kubernetes cluster name** (required) - The name you wish to give the cluster. @@ -156,6 +156,7 @@ added directly to your configured cluster. Those applications are needed for | [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps] or deploy your own web apps. | | [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications | | [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](#security-implications) before doing so. | +| [JupyterHub](http://jupyter.org/) | 11.0+ | The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. | ## Getting the external IP address @@ -200,6 +201,11 @@ Otherwise, you can list the IP addresses of all load balancers: kubectl get svc --all-namespaces -o jsonpath='{range.items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[*].ip} ' ``` +> **Note**: Some Kubernetes clusters return a hostname instead, like [Amazon EKS](https://aws.amazon.com/eks/). For these platforms, run: +> ```bash +> kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -o jsonpath="{.status.loadBalancer.ingress[0].hostname}"`. +> ``` + The output is the external IP address of your cluster. This information can then be used to set up DNS entries and forwarding rules that allow external access to your deployed applications. @@ -232,7 +238,7 @@ When adding more than one Kubernetes clusters to your project, you need to differentiate them with an environment scope. The environment scope associates clusters and [environments](../../../ci/environments.md) in an 1:1 relationship similar to how the -[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-secret-variables) +[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-variables) work. The default environment scope is `*`, which means all jobs, regardless of their @@ -324,7 +330,7 @@ To disable the Kubernetes cluster integration, follow the same procedure. ## Removing the Kubernetes cluster integration NOTE: **Note:** -You need Master [permissions] and above to remove a Kubernetes cluster integration. +You need Maintainer [permissions] and above to remove a Kubernetes cluster integration. NOTE: **Note:** When you remove a cluster, you only remove its relation to GitLab, not the @@ -381,7 +387,7 @@ you will need the Kubernetes project integration enabled. ### Web terminals NOTE: **Note:** -Introduced in GitLab 8.15. You must be the project owner or have `master` permissions +Introduced in GitLab 8.15. You must be the project owner or have `maintainer` permissions to use terminals. Support is limited to the first container in the first pod of your environment. @@ -392,6 +398,10 @@ containers. To use this integration, you should deploy to Kubernetes using the deployment variables above, ensuring any pods you create are labelled with `app=$CI_ENVIRONMENT_SLUG`. GitLab will do the rest! +## Read more + +- [Connecting and deploying to an Amazon EKS cluster](eks_and_gitlab/index.md) + [permissions]: ../../permissions.md [ee]: https://about.gitlab.com/products/ [Auto DevOps]: ../../../topics/autodevops/index.md diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 9c5e3509046..03302b3815d 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -143,6 +143,24 @@ docker login registry.example.com -u <your_username> -p <your_access_token> for errors (e.g. `/var/log/gitlab/gitlab-rails/production.log`). You may be able to find clues there. +#### Enable the registry debug server + +The optional debug server can be enabled by setting the registry debug address +in your `gitlab.rb` configuration. + +```ruby +registry['debug_addr'] = "localhost:5001" +``` + +After adding the setting, [reconfigure] GitLab to apply the change. + +Use curl to request debug output from the debug server: + +```bash +curl localhost:5001/debug/health +curl localhost:5001/debug/vars +``` + ### Advanced Troubleshooting >**NOTE:** The following section is only recommended for experts. @@ -275,3 +293,4 @@ Once the right permissions were set, the error will go away. [docker-docs]: https://docs.docker.com/engine/userguide/intro/ [pat]: ../profile/personal_access_tokens.md [pdt]: ../project/deploy_tokens/index.md +[reconfigure]: ../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
\ No newline at end of file diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 7a8b3c75690..0b9b49f326f 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -5,7 +5,7 @@ Deploy tokens allow to download (through `git clone`), or read the container registry images of a project without the need of having a user and a password. Please note, that the expiration of deploy tokens happens on the date you define, -at midnight UTC and that they can be only managed by [masters](https://docs.gitlab.com/ee/user/permissions.html). +at midnight UTC and that they can be only managed by [maintainers](https://docs.gitlab.com/ee/user/permissions.html). ## Creating a Deploy Token @@ -76,7 +76,7 @@ pull images from your Container Registry. > [Introduced][ce-18414] in GitLab 10.8. There's a special case when it comes to Deploy Tokens, if a user creates one -named `gitlab-deploy-token`, the name and token of the Deploy Token will be +named `gitlab-deploy-token`, the username and token of the Deploy Token will be automatically exposed to the CI/CD jobs as environment variables: `CI_DEPLOY_USER` and `CI_DEPLOY_PASSWORD`, respectively. diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 8c639bd5343..fcd6192e82f 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -1,154 +1,145 @@ # Import your project from GitHub to GitLab -Import your projects from GitHub to GitLab with minimal effort. +Using the importer, you can import your GitHub repositories to GitLab.com or to +your self-hosted GitLab instance. ## Overview ->**Note:** -If you are an administrator you can enable the [GitHub integration][gh-import] -in your GitLab instance sitewide. This configuration is optional, users will -still be able to import their GitHub repositories with a -[personal access token][gh-token]. - ->**Note:** -Administrators of a GitLab instance (Community or Enterprise Edition) can also -use the [GitHub rake task][gh-rake] to import projects from GitHub without the -constrains of a Sidekiq worker. - -- At its current state, GitHub importer can import: - - the repository description (GitLab 7.7+) - - the Git repository data (GitLab 7.7+) - - the issues (GitLab 7.7+) - - the pull requests (GitLab 8.4+) - - the wiki pages (GitLab 8.4+) - - the milestones (GitLab 8.7+) - - the labels (GitLab 8.7+) - - the release note descriptions (GitLab 8.12+) - - the pull request review comments (GitLab 10.2+) - - the regular issue and pull request comments -- References to pull requests and issues are preserved (GitLab 8.7+) -- Repository public access is retained. If a repository is private in GitHub - it will be created as private in GitLab as well. +NOTE: **Note:** +While these instructions will always work for users on GitLab.com, if you are an +administrator of a self-hosted GitLab instance, you will need to enable the +[GitHub integration][gh-import] in order for users to follow the preferred +import method described on this page. If this is not enabled, users can alternatively import their +GitHub repositories using a [personal access token](#using-a-github-token) from GitHub, +but this method will not be able to associate all user activity (such as issues and pull requests) +with matching GitLab users. As an administrator of a self-hosted GitLab instance, you can also use +the [GitHub rake task](../../../administration/raketasks/github_import.md) to import projects from +GitHub without the constraints of a Sidekiq worker. + +The following aspects of a project are imported: + * Repository description (GitLab.com & 7.7+) + * Git repository data (GitLab.com & 7.7+) + * Issues (GitLab.com & 7.7+) + * Pull requests (GitLab.com & 8.4+) + * Wiki pages (GitLab.com & 8.4+) + * Milestones (GitLab.com & 8.7+) + * Labels (GitLab.com & 8.7+) + * Release note descriptions (GitLab.com & 8.12+) + * Pull request review comments (GitLab.com & 10.2+) + * Regular issue and pull request comments + +References to pull requests and issues are preserved (GitLab.com & 8.7+), and +each imported repository maintains visibility level unless that [visibility +level is restricted](../../../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects), +in which case it defaults to the default project visibility. ## How it works -When issues/pull requests are being imported, the GitHub importer tries to find -the GitHub author/assignee in GitLab's database using the GitHub ID. For this -to work, the GitHub author/assignee should have signed in beforehand in GitLab -and **associated their GitHub account**. If the user is not -found in GitLab's database, the project creator (most of the times the current -user that started the import process) is set as the author, but a reference on -the issue about the original GitHub author is kept. +When issues and pull requests are being imported, the importer attempts to find their GitHub authors and +assignees in the database of the GitLab instance (note that pull requests are called "merge requests" in GitLab). -The importer will create any new namespaces (groups) if they don't exist or in -the case the namespace is taken, the repository will be imported under the user's -namespace that started the import process. +For this association to succeed, prior to the import, each GitHub author and assignee in the repository must +have either previously logged in to a GitLab account using the GitHub icon **or** have a GitHub account with +a [public email address](https://help.github.com/articles/setting-your-commit-email-address-on-github/) that +matches their GitLab account's email address. -The importer will also import branches on forks of projects related to open pull -requests. These branches will be imported with a naming scheme similar to -GH-SHA-Username/Pull-Request-number/fork-name/branch. This may lead to a discrepancy -in branches compared to the GitHub Repository. +If a user referenced in the project is not found in GitLab's database, the project creator (typically the user +that initiated the import process) is set as the author/assignee, but a note on the issue mentioning the original +GitHub author is added. -For a more technical description and an overview of the architecture you can -refer to [Working with the GitHub importer][gh-import-dev-docs]. +The importer creates any new namespaces (groups) if they do not exist, or, if the namespace is taken, the +repository is imported under the namespace of the user who initiated the import process. The namespace/repository +name can also be edited, with the proper permissions. -## Importing your GitHub repositories +The importer will also import branches on forks of projects related to open pull requests. These branches will be +imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to +a discrepancy in branches compared to those of the GitHub repository. -The importer page is visible when you create a new project. +For additional technical details, you can refer to the +[GitHub Importer](../../../development/github_importer.md "Working with the GitHub importer") +developer documentation. - +## Import your GitHub repository into GitLab -Click on the **GitHub** link and the import authorization process will start. -There are two ways to authorize access to your GitHub repositories: +### Using the GitHub integration -1. [Using the GitHub integration][gh-integration] (if it's enabled by your - GitLab administrator). This is the preferred way as it's possible to - preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works) - section. -1. [Using a personal access token][gh-token] provided by GitHub. +Before you begin, ensure that any GitHub users who you want to map to GitLab users have either: - +1. A GitLab account that has logged in using the GitHub icon +\- or - +2. A GitLab account with an email address that matches the [public email address](https://help.github.com/articles/setting-your-commit-email-address-on-github/) of the GitHub user -### Authorize access to your repositories using the GitHub integration +User-matching attempts occur in that order, and if a user is not identified either way, the activity is associated with +the user account that is performing the import. -If the [GitHub integration][gh-import] is enabled by your GitLab administrator, -you can use it instead of the personal access token. +NOTE: **Note:** +If you are using a self-hosted GitLab instance, this process requires that you have configured the +[GitHub integration][gh-import]. -1. First you may want to connect your GitHub account to GitLab in order for - the username mapping to be correct. -1. Once you connect GitHub, click the **List your GitHub repositories** button - and you will be redirected to GitHub for permission to access your projects. -1. After accepting, you'll be automatically redirected to the importer. +1. From the top navigation bar, click **+** and select **New project**. +2. Select the **Import project** tab and then select **GitHub**. +3. Select the first button to **List your GitHub repositories**. You are redirected to a page on github.com to authorize the GitLab application. +4. Click **Authorize gitlabhq**. You are redirected back to GitLab's Import page and all of your GitHub repositories are listed. +5. Continue on to [selecting which repositories to import](#selecting-which-repositories-to-import). -You can now go on and [select which repositories to import](#select-which-repositories-to-import). +### Using a GitHub token -### Authorize access to your repositories using a personal access token +NOTE: **Note:** +For a proper author/assignee mapping for issues and pull requests, the [GitHub integration method (above)](#using-the-github-integration) +should be used instead of the personal access token. If you are using GitLab.com or a self-hosted GitLab instance with the GitHub +integration enabled, that should be the preferred method to import your repositories. Read more in the [How it works](#how-it-works) section. ->**Note:** -For a proper author/assignee mapping for issues and pull requests, the -[GitHub integration][gh-integration] should be used instead of the -[personal access token][gh-token]. If the GitHub integration is enabled by your -GitLab administrator, it should be the preferred method to import your repositories. -Read more in the [How it works](#how-it-works) section. +If you are not using the GitHub integration, you can still perform an authorization with GitHub to grant GitLab access your repositories: -If you are not using the GitHub integration, you can still perform a one-off -authorization with GitHub to grant GitLab access your repositories: +1. Go to https://github.com/settings/tokens/new +2. Enter a token description. +3. Select the repo scope. +4. Click **Generate token**. +5. Copy the token hash. +6. Go back to GitLab and provide the token to the GitHub importer. +7. Hit the **List Your GitHub Repositories** button and wait while GitLab reads your repositories' information. + Once done, you'll be taken to the importer page to select the repositories to import. -1. Go to <https://github.com/settings/tokens/new>. -1. Enter a token description. -1. Check the `repo` scope. -1. Click **Generate token**. -1. Copy the token hash. -1. Go back to GitLab and provide the token to the GitHub importer. -1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads - your repositories' information. Once done, you'll be taken to the importer - page to select the repositories to import. +### Selecting which repositories to import -### Select which repositories to import +After you have authorized access to your GitHub repositories, you are redirected to the GitHub importer page and +your GitHub repositories are listed. -After you've authorized access to your GitHub repositories, you will be -redirected to the GitHub importer page. +1. By default, the proposed repository namespaces match the names as they exist in GitHub, but based on your permissions, + you can choose to edit these names before you proceed to import any of them. +2. Select the **Import** button next to any number of repositories, or select **Import all repositories**. +3. The **Status** column shows the import status of each repository. You can choose to leave the page open and it will + update in realtime or you can return to it later. +4. Once a repository has been imported, click its GitLab path to open its GitLab URL. -From there, you can see the import statuses of your GitHub repositories. +## Mirroring and pipeline status sharing -- Those that are being imported will show a _started_ status, -- those already successfully imported will be green with a _done_ status, -- whereas those that are not yet imported will have an **Import** button on the - right side of the table. +Depending your GitLab tier, [project mirroring](../../../workflow/repository_mirroring.md) can be set up to keep +your imported project in sync with its GitHub copy. -If you want, you can import all your GitHub projects in one go by hitting -**Import all projects** in the upper left corner. +Additionally, you can configure GitLab to send pipeline status updates back GitHub with the +[GitHub Project Integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). **[PREMIUM]** - +If you import your project using [CI/CD for external repo](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/), then both +of the above are automatically configured. **[PREMIUM]** ---- +## Improving the speed of imports on self-hosted instances -You can also choose a different name for the project and a different namespace, -if you have the privileges to do so. +NOTE: **Note:** +Admin access to the GitLab server is required. -## Making the import process go faster - -For large projects it may take a while to import all data. To reduce the time -necessary you can increase the number of Sidekiq workers that process the -following queues: +For large projects it may take a while to import all data. To reduce the time necessary, you can increase the number of +Sidekiq workers that process the following queues: * `github_importer` * `github_importer_advance_stage` -For an optimal experience we recommend having at least 4 Sidekiq processes (each -running a number of threads equal to the number of CPU cores) that _only_ -process these queues. We also recommend that these processes run on separate -servers. For 4 servers with 8 cores this means you can import up to 32 objects -(e.g. issues) in parallel. +For an optimal experience, it's recommended having at least 4 Sidekiq processes (each running a number of threads equal +to the number of CPU cores) that *only* process these queues. It's also recommended that these processes run on separate +servers. For 4 servers with 8 cores this means you can import up to 32 objects (e.g., issues) in parallel. -Reducing the time spent in cloning a repository can be done by increasing -network throughput, CPU capacity, and disk performance (e.g. by using high -performance SSDs) of the disks that store the Git repositories (for your GitLab -instance). Increasing the number of Sidekiq workers will _not_ reduce the time -spent cloning repositories. +Reducing the time spent in cloning a repository can be done by increasing network throughput, CPU capacity, and disk +performance (e.g., by using high performance SSDs) of the disks that store the Git repositories (for your GitLab instance). +Increasing the number of Sidekiq workers will *not* reduce the time spent cloning repositories. [gh-import]: ../../../integration/github.md "GitHub integration" -[gh-rake]: ../../../administration/raketasks/github_import.md "GitHub rake task" -[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration -[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token -[gh-import-dev-docs]: ../../../development/github_importer.md "Working with the GitHub importer" diff --git a/doc/user/project/import/img/import_projects_from_repo_url.png b/doc/user/project/import/img/import_projects_from_repo_url.png Binary files differindex ec867da1087..c453c7e558a 100644 --- a/doc/user/project/import/img/import_projects_from_repo_url.png +++ b/doc/user/project/import/img/import_projects_from_repo_url.png diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 557375a1da9..e63ed88249d 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -72,6 +72,8 @@ website with GitLab Pages **Other features:** +- [Wiki](wiki/index.md): Document your GitLab project in an integrated Wiki +- [Snippets](../snippets.md): Store, share and collaborate on code snippets - [Cycle Analytics](cycle_analytics.md): Review your development lifecycle - [Syntax highlighting](highlighting.md): An alternative to customize your code blocks, overriding GitLab's default choice of language diff --git a/doc/user/project/integrations/img/kubernetes_configuration.png b/doc/user/project/integrations/img/kubernetes_configuration.png Binary files differdeleted file mode 100644 index e535e2b8d46..00000000000 --- a/doc/user/project/integrations/img/kubernetes_configuration.png +++ /dev/null diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index e384ed57de9..363e994f36d 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -2,7 +2,7 @@ You can find the available integrations under your project's **Settings ➔ Integrations** page. You need to have at least -[master permission][permissions] on the project. +[maintainer permission][permissions] on the project. ## Project services diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 5933bcedc8b..4d5b2c97291 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -111,7 +111,7 @@ in the table below. | ----- | ----------- | | `Web URL` | The base URL to the JIRA instance web interface which is being linked to this GitLab project. E.g., `https://jira.example.com`. | | `JIRA API URL` | The base URL to the JIRA instance API. Web URL value will be used if not set. E.g., `https://jira-api.example.com`. | -| `Username` | The user name created in [configuring JIRA step](#configuring-jira). | +| `Username` | The user name created in [configuring JIRA step](#configuring-jira). Using the email address will cause `401 unauthorized`. | | `Password` |The password of the user created in [configuring JIRA step](#configuring-jira). | | `Transition ID` | This is the ID of a transition that moves issues to the desired state. **Closing JIRA issues via commits or Merge Requests won't work if you don't set the ID correctly.** | diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index f502d1c9821..9342a2cbb00 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -1,137 +1 @@ ---- -last_updated: 2017-12-28 ---- - -# GitLab Kubernetes / OpenShift integration - -CAUTION: **Warning:** -The Kubernetes service integration has been deprecated in GitLab 10.3. If the -service is active, the cluster information will still be editable, however we -advise to disable and reconfigure the clusters using the new -[Clusters](../clusters/index.md) page. If the service is inactive, the fields -will not be editable. Read [GitLab 10.3 release post](https://about.gitlab.com/2017/12/22/gitlab-10-3-released/#kubernetes-integration-service) for more information. - -GitLab can be configured to interact with Kubernetes, or other systems using the -Kubernetes API (such as OpenShift). - -Each project can be configured to connect to a different Kubernetes cluster, see -the [configuration](#configuration) section. - -## Configuration - -Navigate to the [Integrations page](project_services.md#accessing-the-project-services) -of your project and select the **Kubernetes** service to configure it. Fill in -all the needed parameters, check the "Active" checkbox and hit **Save changes** -for the changes to take effect. - - - -The Kubernetes service takes the following parameters: - -- **API URL** - - It's the URL that GitLab uses to access the Kubernetes API. Kubernetes - exposes several APIs, we want the "base" URL that is common to all of them, - e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. -- **CA certificate** (optional) - - If the API is using a self-signed TLS certificate, you'll also need to include - the `ca.crt` contents here. -- **Project namespace** (optional) - The following apply: - - By default you don't have to fill it in; by leaving it blank, GitLab will - create one for you. - - Each project should have a unique namespace. - - The project namespace is not necessarily the namespace of the secret, if - you're using a secret with broader permissions, like the secret from `default`. - - You should **not** use `default` as the project namespace. - - If you or someone created a secret specifically for the project, usually - with limited permissions, the secret's namespace and project namespace may - be the same. -- **Token** - - GitLab authenticates against Kubernetes using service tokens, which are - scoped to a particular `namespace`. If you don't have a service token yet, - you can follow the - [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) - to create one. You can also view or create service tokens in the - [Kubernetes dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/#config) - (under **Config > Secrets**). - -TIP: **Tip:** -If you have a single cluster that you want to use for all your projects, -you can pre-fill the settings page with a default template. To configure the -template, see [Services Templates](services_templates.md). - -## Deployment variables - -The Kubernetes service exposes the following -[deployment variables](../../../ci/variables/README.md#deployment-variables) in the -GitLab CI/CD build environment: - -- `KUBE_URL` - Equal to the API URL. -- `KUBE_TOKEN` - The Kubernetes token. -- `KUBE_NAMESPACE` - The Kubernetes namespace is auto-generated if not specified. - The default value is `<project_name>-<project_id>`. You can overwrite it to - use different one if needed, otherwise the `KUBE_NAMESPACE` variable will - receive the default value. -- `KUBE_CA_PEM_FILE` - Only present if a custom CA bundle was specified. Path - to a file containing PEM data. -- `KUBE_CA_PEM` (deprecated) - Only if a custom CA bundle was specified. Raw PEM data. -- `KUBECONFIG` - Path to a file containing `kubeconfig` for this deployment. - CA bundle would be embedded if specified. - -## What you can get with the Kubernetes integration - -Here's what you can do with GitLab if you enable the Kubernetes integration. - -### Deploy Boards - -> Available in [GitLab Premium][ee]. - -GitLab's Deploy Boards offer a consolidated view of the current health and -status of each CI [environment](../../../ci/environments.md) running on Kubernetes, -displaying the status of the pods in the deployment. Developers and other -teammates can view the progress and status of a rollout, pod by pod, in the -workflow they already use without any need to access Kubernetes. - -[> Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) - -### Canary Deployments - -> Available in [GitLab Premium][ee]. - -Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) -and visualize your canary deployments right inside the Deploy Board, without -the need to leave GitLab. - -[> Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) - -### Kubernetes monitoring - -Automatically detect and monitor Kubernetes metrics. Automatic monitoring of -[NGINX ingress](./prometheus_library/nginx.md) is also supported. - -[> Read more about Kubernetes monitoring](prometheus_library/kubernetes.md) - -### Auto DevOps - -Auto DevOps automatically detects, builds, tests, deploys, and monitors your -applications. - -To make full use of Auto DevOps(Auto Deploy, Auto Review Apps, and Auto Monitoring) -you will need the Kubernetes project integration enabled. - -[> Read more about Auto DevOps](../../../topics/autodevops/index.md) - -### Web terminals - -NOTE: **Note:** -Introduced in GitLab 8.15. You must be the project owner or have `master` permissions -to use terminals. Support is limited to the first container in the -first pod of your environment. - -When enabled, the Kubernetes service adds [web terminal](../../../ci/environments.md#web-terminals) -support to your [environments](../../../ci/environments.md). This is based on the `exec` functionality found in -Docker and Kubernetes, so you get a new shell session within your existing -containers. To use this integration, you should deploy to Kubernetes using -the deployment variables above, ensuring any pods you create are labelled with -`app=$CI_ENVIRONMENT_SLUG`. GitLab will do the rest! - -[ee]: https://about.gitlab.com/products/ +This document was moved to [another location](../clusters/index.md). diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index 9496d6f2ce0..8c51eb9915e 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -34,12 +34,11 @@ Click on the service links to see further configuration instructions and details | [Emails on push](emails_on_push.md) | Email the commits and diff of each push to a list of recipients | | External Wiki | Replaces the link to the internal wiki with a link to an external wiki | | Flowdock | Flowdock is a collaboration web app for technical teams | -| Gemnasium | Gemnasium monitors your project dependencies and alerts you about updates and security vulnerabilities | +| Gemnasium _(Has been deprecated in GitLab 11.0)_ | Gemnasium monitors your project dependencies and alerts you about updates and security vulnerabilities | | [HipChat](hipchat.md) | Private group chat and IM | | [Irker (IRC gateway)](irker.md) | Send IRC messages, on update, to a list of recipients through an Irker gateway | | [JIRA](jira.md) | JIRA issue tracker | | JetBrains TeamCity CI | A continuous integration and build server | -| [Kubernetes](kubernetes.md) _(Has been deprecated in GitLab 10.3)_ | A containerized deployment service | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | | [Microsoft teams](microsoft_teams.md) | Receive notifications for actions that happen on GitLab into a room on Microsoft Teams using Office 365 Connectors | diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index fa7e504c4aa..f687027e8c8 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -30,7 +30,7 @@ GitLab can seamlessly deploy and manage Prometheus on a [connected Kubernetes cl Once you have a connected Kubernetes cluster with Helm installed, deploying a managed Prometheus is as easy as a single click. -1. Go to the `CI/CD > Kubernetes` page, to view your connected clusters +1. Go to the `Operations > Kubernetes` page, to view your connected clusters 1. Select the cluster you would like to deploy Prometheus to 1. Click the **Install** button to deploy Prometheus to the cluster diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index fea3231006b..557487e1a75 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -10,17 +10,19 @@ The [Prometheus service](../prometheus.md) must be enabled. ## Metrics supported +NGINX server metrics are detected, which tracks the pages and content directly served by NGINX. + | Name | Query | | ---- | ----- | -| Throughput (req/sec) | sum(rate(nginx_responses_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) by (status_code) | -| Latency (ms) | avg(nginx_upstream_response_msecs_avg{%{environment_filter}}) | -| HTTP Error Rate (HTTP Errors / sec) | rate(nginx_responses_total{status_code="5xx", %{environment_filter}}[2m])) | +| Throughput (req/sec) | sum(rate(nginx_server_requests{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) by (code) | +| Latency (ms) | avg(nginx_server_requestMsec{%{environment_filter}}) | +| HTTP Error Rate (HTTP Errors / sec) | sum(rate(nginx_server_requests{code="5xx", %{environment_filter}}[2m])) | ## Configuring Prometheus to monitor for NGINX metrics To get started with NGINX monitoring, you should first enable the [VTS statistics](https://github.com/vozlt/nginx-module-vts)) module for your NGINX server. This will capture and display statistics in an HTML readable form. Next, you should install and configure the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter) which parses these statistics and translates them into a Prometheus monitoring endpoint. -If you are using NGINX as your Kubernetes ingress, there is [upcoming direct support](https://github.com/kubernetes/ingress/pull/423) for enabling Prometheus monitoring in the 0.9.0 release. +If you are using NGINX as your Kubernetes ingress, GitLab will [automatically detect](nginx_ingress.md) the metrics once enabled in 0.9.0 and later releases. ## Specifying the Environment label diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 590b1c4275a..a1db79538a4 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -14,7 +14,7 @@ GitLab has support for automatically detecting and monitoring the Kubernetes NGI | ---- | ----- | | Throughput (req/sec) | sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code) | | Latency (ms) | avg(nginx_upstream_response_msecs_avg{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}) | -| HTTP Error Rate (HTTP Errors / sec) | sum(rate(nginx_upstream_responses_total{status_code="5xx", upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) | +| HTTP Error Rate (%) | sum(rate(nginx_upstream_responses_total{status_code="5xx", upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) / sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) * 100 | ## Configuring NGINX ingress monitoring diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 7eab825fa32..9ca1e6226c5 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -237,23 +237,25 @@ Issue Board, that is create/delete lists and drag issues around. ## Group Issue Board ->Introduced in GitLab 10.6 +> Introduced in [GitLab 10.6](https://about.gitlab.com/2018/03/22/gitlab-10-6-released/#single-group-issue-board-in-core-and-free) Group issue board is analogous to project-level issue board and it is accessible at the group navigation level. A group-level issue board allows you to view all issues from all projects in that group or descendant subgroups. Similarly, you can only filter by group labels for these boards. When updating milestones and labels for an issue through the sidebar update mechanism, again only group-level objects are available. +One group issue board per group was made available in GitLab 10.6 Core after multiple group issue boards were originally introduced in [GitLab 10.0 Premium](https://about.gitlab.com/2017/09/22/gitlab-10-0-released/#group-issue-boards). + ## Features per tier Different issue board features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), as shown in the following table: -| Tier | Number of project issue boards | Board with configuration in project issue boards | Number of group issue boards | Board with configuration in group issue boards | -| --- | --- | --- | --- | --- | -| Core | 1 | No | 1 | No | -| Starter | Multiple | Yes | 1 | No | -| Premium | Multiple | Yes | Multiple | Yes | -| Ultimate | Multiple | Yes | Multiple | Yes | +| Tier | Number of Project Issue Boards | Number of Group Issue Boards | Configurable Project Issue Boards | Configurable Group Issue Boards | Assignee Lists +| --- | --- | --- | --- | --- | --- | +| Core | 1 | 1 | No | No | No | +| Starter | Multiple | 1 | Yes | No | No | +| Premium | Multiple | Multiple | Yes | Yes | Yes | +| Ultimate | Multiple | Multiple | Yes | Yes | Yes | ## Tips diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 0bf1f396f9d..8eada25234f 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -71,10 +71,10 @@ least [Reporter access][permissions]. However, a guest user can also create confidential issues, but can only view the ones that they created themselves. Confidential issues are also hidden in search results for unprivileged users. -For example, here's what a user with Master and Guest access sees in the +For example, here's what a user with Maintainer and Guest access sees in the project's search results respectively. -| Master access | Guest access | +| Maintainer access | Guest access | | :-----------: | :----------: | |  |  | diff --git a/doc/user/project/issues/due_dates.md b/doc/user/project/issues/due_dates.md index 1bf8b776c2e..93306437c6c 100644 --- a/doc/user/project/issues/due_dates.md +++ b/doc/user/project/issues/due_dates.md @@ -39,5 +39,13 @@ The day before an open issue is due, an email will be sent to all participants of the issue. Both the due date and the day before are calculated using the server's timezone. +Issues with due dates can also be exported as an iCalendar feed. The URL of the +feed can be added to calendar applications. The feed is accessible by clicking +on the _Subscribe to calendar_ button on the following pages: +- on the **Assigned Issues** page that is linked on the right-hand side of the + GitLab header +- on the **Project Issues** page +- on the **Group Issues** page + [ce-3614]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3614 [permissions]: ../../permissions.md#project diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index be4436749f9..bf17731c523 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -151,3 +151,7 @@ or Bugzilla. ### Issue's API Read through the [API documentation](../../../api/issues.md). + +### Bulk editing issues + +Find out about [bulk editing issues](../../project/bulk_editing.md). diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 43713855e26..2c2e8e2d556 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -4,7 +4,7 @@ You can manage the groups and users and their access levels in all of your projects. You can also personalize the access level you give each user, per-project. -You should have `master` or `owner` [permissions](../../permissions.md) to add +You should have Maintainer or Owner [permissions](../../permissions.md) to add or import a new user to your project. To view, edit, add, and remove project's members, go to your @@ -43,7 +43,7 @@ level to the project. You can import another project's users in your own project by hitting the **Import members** button on the upper right corner of the **Members** menu. -In the dropdown menu, you can see only the projects you are Master on. +In the dropdown menu, you can see only the projects you are Maintainer on.  @@ -99,7 +99,7 @@ side of your screen. --- -Project owners & masters will be notified of your request and will be able to approve or +Project owners & maintainers will be notified of your request and will be able to approve or decline it on the members page.  diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index 5d819998dd9..611ff0e6bfb 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -42,7 +42,7 @@ Admins are able to share projects with any group in the system. ## Maximum access level -In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Master' or 'Owner') will only have 'Developer' access to 'Project Acme'. +In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Maintainer' or 'Owner') will only have 'Developer' access to 'Project Acme'. ## Share project with group lock diff --git a/doc/user/project/merge_requests/allow_collaboration.md b/doc/user/project/merge_requests/allow_collaboration.md new file mode 100644 index 00000000000..859ac92ef89 --- /dev/null +++ b/doc/user/project/merge_requests/allow_collaboration.md @@ -0,0 +1,20 @@ +# Allow collaboration on merge requests across forks + +> [Introduced][ce-17395] in GitLab 10.6. + +This feature is available for merge requests across forked projects that are +publicly accessible. It makes it easier for members of projects to +collaborate on merge requests across forks. + +When enabled for a merge request, members with merge access to the target +branch of the project will be granted write permissions to the source branch +of the merge request. + +The feature can only be enabled by users who already have push access to the +source project, and only lasts while the merge request is open. + +Enable this functionality while creating or editing a merge request: + + + +[ce-17395]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17395 diff --git a/doc/user/project/merge_requests/authorization_for_merge_requests.md b/doc/user/project/merge_requests/authorization_for_merge_requests.md index 59b3fe7242c..79444ee5682 100644 --- a/doc/user/project/merge_requests/authorization_for_merge_requests.md +++ b/doc/user/project/merge_requests/authorization_for_merge_requests.md @@ -9,7 +9,7 @@ There are two main ways to have a merge request flow with GitLab: With the protected branch flow everybody works within the same GitLab project. -The project maintainers get Master access and the regular developers get +The project maintainers get Maintainer access and the regular developers get Developer access. The maintainers mark the authoritative branches as 'Protected'. @@ -18,7 +18,7 @@ The developers push feature branches to the project and create merge requests to have their feature branches reviewed and merged into one of the protected branches. -By default, only users with Master access can merge changes into a protected +By default, only users with Maintainer access can merge changes into a protected branch. **Advantages** @@ -32,7 +32,7 @@ branch. ## Forking workflow -With the forking workflow the maintainers get Master access and the regular +With the forking workflow the maintainers get Maintainer access and the regular developers get Reporter access to the authoritative repository, which prohibits them from pushing any changes to it. diff --git a/doc/user/project/merge_requests/img/allow_collaboration.png b/doc/user/project/merge_requests/img/allow_collaboration.png Binary files differnew file mode 100644 index 00000000000..75596e7d9ad --- /dev/null +++ b/doc/user/project/merge_requests/img/allow_collaboration.png diff --git a/doc/user/project/merge_requests/img/allow_maintainer_push.png b/doc/user/project/merge_requests/img/allow_maintainer_push.png Binary files differdeleted file mode 100644 index 91cc399f4ff..00000000000 --- a/doc/user/project/merge_requests/img/allow_maintainer_push.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/squash_edit_form.png b/doc/user/project/merge_requests/img/squash_edit_form.png Binary files differnew file mode 100644 index 00000000000..496c6f44ea7 --- /dev/null +++ b/doc/user/project/merge_requests/img/squash_edit_form.png diff --git a/doc/user/project/merge_requests/img/squash_mr_commits.png b/doc/user/project/merge_requests/img/squash_mr_commits.png Binary files differnew file mode 100644 index 00000000000..5fc6a8c48bb --- /dev/null +++ b/doc/user/project/merge_requests/img/squash_mr_commits.png diff --git a/doc/user/project/merge_requests/img/squash_mr_widget.png b/doc/user/project/merge_requests/img/squash_mr_widget.png Binary files differnew file mode 100644 index 00000000000..9cb458b2a35 --- /dev/null +++ b/doc/user/project/merge_requests/img/squash_mr_widget.png diff --git a/doc/user/project/merge_requests/img/squash_squashed_commit.png b/doc/user/project/merge_requests/img/squash_squashed_commit.png Binary files differnew file mode 100644 index 00000000000..0cf5875f82c --- /dev/null +++ b/doc/user/project/merge_requests/img/squash_squashed_commit.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index a6c0fd49c45..5e2e0c3d171 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -28,13 +28,13 @@ With GitLab merge requests, you can: - Enable [fast-forward merge requests](#fast-forward-merge-requests) - Enable [semi-linear history merge requests](#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch - [Create new merge requests by email](#create-new-merge-requests-by-email) -- Allow maintainers of the target project to push directly to the fork by [allowing edits from maintainers](maintainer_access.md) +- [Allow collaboration](allow_collaboration.md) so members of the target project can push directly to the fork +- [Squash and merge](squash_and_merge.md) for a cleaner commit history With **[GitLab Enterprise Edition][ee]**, you can also: - View the deployment process across projects with [Multi-Project Pipeline Graphs](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html#multi-project-pipeline-graphs) **[PREMIUM]** - Request [approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your managers **[STARTER]** -- [Squash and merge](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html) for a cleaner commit history **[STARTER]** - Analyze the impact of your changes with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) **[STARTER]** ## Use cases @@ -57,7 +57,7 @@ B. Consider you're a web developer writing a webpage for your company's: 1. Your changes are previewed with [Review Apps](../../../ci/review_apps/index.md) 1. You request your web designers for their implementation 1. You request the [approval](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your manager **[STARTER]** -1. Once approved, your merge request is [squashed and merged](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html), and [deployed to staging with GitLab Pages](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) (Squash and Merge is available in GitLab Starter) +1. Once approved, your merge request is [squashed and merged](squash_and_merge.md), and [deployed to staging with GitLab Pages](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) 1. Your production team [cherry picks](#cherry-pick-changes) the merge commit into production ## Merge requests per project @@ -85,7 +85,7 @@ request is merged. This option is also visible in an existing merge request next to the merge request button and can be selected/deselected before merging. It's only visible -to users with [Master permissions](../../permissions.md) in the source project. +to users with [Maintainer permissions](../../permissions.md) in the source project. If the user viewing the merge request does not have the correct permissions to remove the source branch and the source branch is set for removal, the merge @@ -233,6 +233,10 @@ all your changes will be available to preview by anyone with the Review Apps lin [Read more about Review Apps.](../../../ci/review_apps/index.md) +## Bulk editing merge requests + +Find out about [bulk editing merge requests](../../project/bulk_editing.md). + ## Tips Here are some tips that will help you be more efficient with merge requests in diff --git a/doc/user/project/merge_requests/maintainer_access.md b/doc/user/project/merge_requests/maintainer_access.md index c9763a3fe02..d59afecd375 100644 --- a/doc/user/project/merge_requests/maintainer_access.md +++ b/doc/user/project/merge_requests/maintainer_access.md @@ -1,18 +1 @@ -# Allow maintainer pushes for merge requests across forks - -> [Introduced][ce-17395] in GitLab 10.6. - -This feature is available for merge requests across forked projects that are -publicly accessible. It makes it easier for maintainers of projects to -collaborate on merge requests across forks. - -When enabled for a merge request, members with merge access to the target -branch of the project will be granted write permissions to the source branch -of the merge request. - -The feature can only be enabled by users who already have push access to the -source project, and only lasts while the merge request is open. - -Enable this functionality while creating a merge request: - - +This document was moved to [another location](allow_collaboration.md). diff --git a/doc/user/project/merge_requests/resolve_conflicts.md b/doc/user/project/merge_requests/resolve_conflicts.md index 68c49054e47..ecbc8534eea 100644 --- a/doc/user/project/merge_requests/resolve_conflicts.md +++ b/doc/user/project/merge_requests/resolve_conflicts.md @@ -29,7 +29,7 @@ The merge conflict resolution editor allows for more complex merge conflicts, which require the user to manually modify a file in order to resolve a conflict, to be solved right form the GitLab interface. Use the **Edit inline** button to open the editor. Once you're sure about your changes, hit the -**Commit conflict resolution** button. +**Commit to source branch** button.  diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md new file mode 100644 index 00000000000..a6efe893853 --- /dev/null +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -0,0 +1,80 @@ +# Squash and merge + +> [Introduced][ee-1024] in [GitLab Starter][ee] 8.17, and in [GitLab CE][ce] [11.0][ce-18956]. + +Combine all commits of your merge request into one and retain a clean history. + +## Overview + +Squashing lets you tidy up the commit history of a branch when accepting a merge +request. It applies all of the changes in the merge request as a single commit, +and then merges that commit using the merge method set for the project. + +In other words, squashing a merge request turns a long list of commits: + +![List of commits from a merge request][mr-commits] + +Into a single commit on merge: + +![A squashed commit followed by a merge commit][squashed-commit] + +The squashed commit's commit message is the merge request title. And note that +the squashed commit is still followed by a merge commit, as the merge +method for this example repository uses a merge commit. Squashing also works +with the fast-forward merge strategy, see +[squashing and fast-forward merge](#squash-and-fast-forward-merge) for more +details. + +## Use cases + +When working on a feature branch, you sometimes want to commit your current +progress, but don't really care about the commit messages. Those 'work in +progress commits' don't necessarily contain important information and as such +you'd rather not include them in your target branch. + +With squash and merge, when the merge request is ready to be merged, +all you have to do is enable squashing before you press merge to join +the commits include in the merge request into a single commit. + +This way, the history of your base branch remains clean with +meaningful commit messages and is simpler to [revert] if necessary. + +## Enabling squash for a merge request + +Anyone who can create or edit a merge request can choose for it to be squashed +on the merge request form: + +![Squash commits checkbox on edit form][squash-edit-form] + +--- + +This can then be overridden at the time of accepting the merge request: + +![Squash commits checkbox on accept merge request form][squash-mr-widget] + +## Commit metadata for squashed commits + +The squashed commit has the following metadata: + +* Message: the title of the merge request. +* Author: the author of the merge request. +* Committer: the user who initiated the squash. + +## Squash and fast-forward merge + +When a project has the [fast-forward merge setting enabled][ff-merge], the merge +request must be able to be fast-forwarded without squashing in order to squash +it. This is because squashing is only available when accepting a merge request, +so a merge request may need to be rebased before squashing, even though +squashing can itself be considered equivalent to rebasing. + +[ee-1024]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1024 +[ce-18956]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18956 +[mr-commits]: img/squash_mr_commits.png +[squashed-commit]: img/squash_squashed_commit.png +[squash-edit-form]: img/squash_edit_form.png +[squash-mr-widget]: img/squash_mr_widget.png +[ff-merge]: fast_forward_merge.md#enabling-fast-forward-merges +[ce]: https://about.gitlab.com/products/ +[ee]: https://about.gitlab.com/products/ +[revert]: revert_changes.md diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 64bb33be547..632253db94c 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -10,7 +10,6 @@ Milestones allow you to organize issues and merge requests into a cohesive group - **Project milestones** can be assigned to issues or merge requests in that project only. - **Group milestones** can be assigned to any issue or merge request of any project in that group. -- In the [future](https://gitlab.com/gitlab-org/gitlab-ce/issues/36862), you will be able to assign group milestones to issues and merge requests of projects in [subgroups](../../group/subgroups/index.md). ## Creating milestones diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index a97ce84b861..205f0283107 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -1,3 +1,7 @@ +--- +description: 'Learn how to use GitLab Pages to deploy a static website at no additional cost.' +--- + # GitLab Pages With GitLab Pages it's easy to publish your project website. GitLab Pages is a hosting service for static websites, at no additional cost. @@ -38,7 +42,7 @@ to secure them. Your files live in a project [repository](../repository/index.md) on GitLab. [GitLab CI](../../../ci/README.md) picks up those files and makes them available at, typically, -`http://<username>.gilab.io/<projectname>`. Please read through the docs on +`https://<username>.gitlab.io/<projectname>`. Please read through the docs on [GitLab Pages domains](getting_started_part_one.md#gitlab-pages-domain) for more info. ## Explore GitLab Pages diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 0b5076b8c5d..fe4d15adfa1 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -1,4 +1,4 @@ -# GitLab Pages +# Exploring GitLab Pages > **Notes:** > - This feature was [introduced][ee-80] in GitLab EE 8.3. @@ -14,9 +14,7 @@ deploy static pages for your individual projects, your user or your group. Read [GitLab Pages on GitLab.com](#gitlab-pages-on-gitlab-com) for specific information, if you are using GitLab.com to host your website. -Read through [All you Need to Know About GitLab Pages][pages-index-guide] for a list of all learning materials we have prepared for GitLab Pages (webpages, articles, guides, blog posts, video tutorials). - -## Getting started with GitLab Pages +## Getting started with GitLab Pages domains > **Note:** > In the rest of this document we will assume that the general domain name that diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 0cbb0c878c2..3bf63a22963 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -10,8 +10,8 @@ created protected branches. By default, a protected branch does four simple things: - it prevents its creation, if not already created, from everybody except users - with Master permission -- it prevents pushes from everybody except users with Master permission + with Maintainer permission +- it prevents pushes from everybody except users with Maintainer permission - it prevents **anyone** from force pushing to the branch - it prevents **anyone** from deleting the branch @@ -24,7 +24,7 @@ See the [Changelog](#changelog) section for changes over time. ## Configuring protected branches -To protect a branch, you need to have at least Master permission level. Note +To protect a branch, you need to have at least Maintainer permission level. Note that the `master` branch is protected by default. 1. Navigate to your project's **Settings ➔ Repository** @@ -45,19 +45,19 @@ that the `master` branch is protected by default. Since GitLab 8.11, we added another layer of branch protection which provides more granular management of protected branches. The "Developers can push" option was replaced by an "Allowed to push" setting which can be set to -allow/prohibit Masters and/or Developers to push to a protected branch. +allow/prohibit Maintainers and/or Developers to push to a protected branch. Using the "Allowed to push" and "Allowed to merge" settings, you can control the actions that different roles can perform with the protected branch. For example, you could set "Allowed to push" to "No one", and "Allowed to merge" -to "Developers + Masters", to require _everyone_ to submit a merge request for +to "Developers + Maintainers", to require _everyone_ to submit a merge request for changes going into the protected branch. This is compatible with workflows like the [GitLab workflow](../../workflow/gitlab_flow.md). However, there are workflows where that is not needed, and only protecting from force pushes and branch removal is useful. For those workflows, you can allow everyone with write access to push to a protected branch by setting -"Allowed to push" to "Developers + Masters". +"Allowed to push" to "Developers + Maintainers". You can set the "Allowed to push" and "Allowed to merge" options while creating a protected branch or afterwards by selecting the option you want from the @@ -66,7 +66,7 @@ dropdown list in the "Already protected" area.  If you don't choose any of those options while creating a protected branch, -they are set to "Masters" by default. +they are set to "Maintainers" by default. ## Wildcard protected branches @@ -101,7 +101,7 @@ all matching branches: From time to time, it may be required to delete or clean up branches that are protected. -User with [Master permissions][perm] and up can manually delete protected +User with [Maintainer permissions][perm] and up can manually delete protected branches via GitLab's web interface: 1. Visit **Repository > Branches** diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 0cb7aefdb2f..a5eaf2e9835 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -8,12 +8,12 @@ This feature evolved out of [Protected Branches](protected_branches.md) ## Overview -Protected tags will prevent anyone from updating or deleting the tag, as and will prevent creation of matching tags based on the permissions you have selected. By default, anyone without Master permission will be prevented from creating tags. +Protected tags will prevent anyone from updating or deleting the tag, as and will prevent creation of matching tags based on the permissions you have selected. By default, anyone without Maintainer permission will be prevented from creating tags. ## Configuring protected tags -To protect a tag, you need to have at least Master permission level. +To protect a tag, you need to have at least Maintainer permission level. 1. Navigate to the project's Settings -> Repository page diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 2f4ed3493c2..0ef8eddad20 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -42,3 +42,4 @@ do. | `/tableflip` | Append the comment with `(╯°□°)╯︵ ┻━┻` | | `/shrug` | Append the comment with `¯\＿(ツ)＿/¯` | | <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request | +| `/confidential` | Makes the issue confidential |
\ No newline at end of file diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 2c90f4b4413..9034a9b5179 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -5,6 +5,7 @@ > - [Introduced][ce-3050] in GitLab 8.9. > - Importing will not be possible if the import instance version differs from > that of the exporter. +> - For GitLab admins, please read through [Project import/export administration](../../../administration/raketasks/project_import_export.md). > - For existing installations, the project import option has to be enabled in > application settings (`/admin/application_settings`) under 'Import sources'. > Ask your administrator if you don't see the **GitLab export** button when @@ -18,7 +19,7 @@ > - The exports are stored in a temporary [shared directory][tmp] and are deleted > every 24 hours by a specific worker. > - Group members will get exported as project members, as long as the user has -> master or admin access to the group where the exported project lives. An admin +> maintainer or admin access to the group where the exported project lives. An admin > in the import side is required to map the users, based on email or username. > Otherwise, a supplementary comment is left to mention the original author and > the MRs, notes or issues will be owned by the importer. diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index c9d2f8dc32d..212e271ce6f 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -1,7 +1,7 @@ # Project settings NOTE: **Note:** -Only project Masters and Admin users have the [permissions] to access a project +Only project Maintainers and Admin users have the [permissions] to access a project settings. You can adjust your [project](../index.md) settings by navigating @@ -74,7 +74,7 @@ To archive a project: #### Renaming a repository NOTE: **Note:** -Only project Masters and Admin users have the [permissions] to rename a +Only project Maintainers and Admin users have the [permissions] to rename a repository. Not to be confused with a project's name where it can also be changed from the [general project settings](#general-project-settings). @@ -98,7 +98,7 @@ Only project Owners and Admin users have the [permissions] to transfer a project You can transfer an existing project into a [group](../../group/index.md) if: -1. you have at least **Master** [permissions] to that group +1. you have at least **Maintainer** [permissions] to that group 1. you are an **Owner** of the project. Similarly, if you are an owner of a group, you can transfer any of its projects diff --git a/doc/user/project/web_ide/img/commit_changes.png b/doc/user/project/web_ide/img/commit_changes.png Binary files differindex b6fcbf699aa..a5364c12760 100644 --- a/doc/user/project/web_ide/img/commit_changes.png +++ b/doc/user/project/web_ide/img/commit_changes.png diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index b7064b83c4e..b0143e45ab6 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -13,15 +13,27 @@ and from merge requests.  -## Commit changes +## File finder -Changed files are shown on the right in the commit panel. All changes are -automatically staged. To commit your changes, add a commit message and click -the 'Commit Button'. +> [Introduced in](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18323) [GitLab Core][ce] 10.8. + +The file finder allows you to quickly open files in the current branch by +searching. The file finder is launched using the keyboard shortcut `Command-p`, +`Control-p`, or `t` (when editor is not in focus). Type the filename or +file path fragments to start seeing results. + +## Stage and commit changes + +After making your changes, click the Commit button in the bottom left to +review the list of changed files. Click on each file to review the changes and +click the tick icon to stage the file. + +Once you have staged some changes, you can add a commit message and commit the +staged changes. Unstaged changes will not be commited.  -## Comparing changes +## Reviewing changes Before you commit your changes, you can compare them with the previous commit by switching to the review mode or selecting the file from the staged files @@ -30,4 +42,26 @@ list. An additional review mode is available when you open a merge request, which shows you a preview of the merge request diff if you commit your changes. +## View CI job logs + +> [Introduced in](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19279) [GitLab Core][ce] 11.0. + +The Web IDE can be used to quickly fix failing tests by opening the branch or +merge request in the Web IDE and opening the logs of the failed job. The status +of all jobs for the most recent pipeline and job traces for the current commit +can be accessed by clicking the **Pipelines** button in the top right. + +The pipeline status is also shown at all times in the status bar in the bottom +left. + +## Switching merge requests + +> [Introduced in](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19318) [GitLab Core][ce] 11.0. + +Switching between your authored and assigned merge requests can be done without +leaving the Web IDE. Click the project name in the top left to open a list of +merge requests. You will need to commit or discard all your changes before +switching to a different merge request. + +[ce]: https://about.gitlab.com/pricing/ [ee]: https://about.gitlab.com/pricing/ diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md index 50ec99be48b..6c1378560ef 100644 --- a/doc/user/reserved_names.md +++ b/doc/user/reserved_names.md @@ -58,7 +58,7 @@ Currently the following names are reserved as top level groups: - dashboard - deploy.html - explore -- favicon.ico +- favicon.png - groups - header_logo_dark.png - header_logo_light.png diff --git a/doc/user/snippets.md b/doc/user/snippets.md index 2170b079f62..7efb6bafee7 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -1,29 +1,79 @@ # Snippets -Snippets are little bits of code or text. +With GitLab Snippets you can store and share bits of code and text with other users.  -There are 2 types of snippets - project snippets and personal snippets. +There are 2 types of snippets, personal snippets and project snippets. -## Comments - -With GitLab Snippets you engage in a conversation about that piece of code, -facilitating the collaboration among users. +## Personal snippets -> **Note:** -Comments on snippets was [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/12910) in [GitLab Community Edition 9.2](https://about.gitlab.com/2017/05/22/gitlab-9-2-released/#comments-for-personal-snippets). +Personal snippets are not related to any project and can be created completely +independently. There are 3 visibility levels that can be set, public, internal +and private. See [Public access](../public_access/public_access.md) for more information. ## Project snippets -Project snippets are always related to a specific project - see [Project's features](project/index.md#project-39-s-features) for more information. +Project snippets are always related to a specific project. +See [Project's features](project/index.md#project-39-s-features) for more information. -## Personal snippets +## Discover snippets + +There are two main ways of how you can discover snippets in GitLab. -Personal snippets are not related to any project and can be created completely independently. There are 3 visibility levels that can be set (public, internal, private - see [Public Access](../public_access/public_access.md) for more information). +For exploring all snippets that are visible to you, you can go to the Snippets +dashboard of your GitLab instance via the top navigation. For GitLab.com you can +find it [here](https://gitlab.com/dashboard/snippets). This navigates you to an +overview that shows snippets you created and allows you to explore all snippets. + +If you want to discover snippets that belong to a specific project, you can navigate +to the Snippets page via the left side navigation on the project page. + +## Snippet comments + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/12910) in GitLab 9.2. + +With GitLab Snippets you engage in a conversation about that piece of code, +facilitating the collaboration among users. ## Downloading snippets You can download the raw content of a snippet. -By default snippets will be downloaded with Linux-style line endings (`LF`). If you want to preserve the original line endings you need to add a parameter `line_ending=raw` (eg. `https://gitlab.com/snippets/SNIPPET_ID/raw?line_ending=raw`). In case a snippet was created using the GitLab web interface the original line ending is Windows-like (`CRLF`). +By default snippets will be downloaded with Linux-style line endings (`LF`). If +you want to preserve the original line endings you need to add a parameter `line_ending=raw` +(e.g., `https://gitlab.com/snippets/SNIPPET_ID/raw?line_ending=raw`). In case a +snippet was created using the GitLab web interface the original line ending is Windows-like (`CRLF`). + +## Embedded snippets + +> Introduced in GitLab 10.8. + +Public snippets can not only be shared, but also embedded on any website. This +allows to reuse a GitLab snippet in multiple places and any change to the source +is automatically reflected in the embedded snippet. + +To embed a snippet, first make sure that: + +- The project is public (if it's a project snippet) +- The snippet is public +- In **Project > Settings > Permissions**, the snippets permissions are + set to **Everyone with access** + +Once the above conditions are met, the "Embed" section will appear in your snippet +where you can simply click on the "Copy to clipboard" button. This copies a one-line +script that you can add to any website or blog post. + +Here's how an example code looks like: + +```html +<script src="https://gitlab.com/namespace/project/snippets/SNIPPET_ID.js"></script> +``` + +Here's how an embedded snippet looks like: + +<script src="https://gitlab.com/gitlab-org/gitlab-ce/snippets/1717978.js"></script> + +Embedded snippets are displayed with a header that shows the file name if defined, +the snippet size, a link to GitLab, and the actual snippet content. Actions in +the header allow users to see the snippet in raw format and download it. diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 23b67310d25..a7313082fac 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -131,7 +131,7 @@ There is room for more feedback and after the assigned person feels comfortable If the assigned person does not feel comfortable they can close the merge request without merging. In GitLab it is common to protect the long-lived branches (e.g. the master branch) so that normal developers [can't modify these protected branches](http://docs.gitlab.com/ce/permissions/permissions.html). -So if you want to merge it into a protected branch you assign it to someone with master authorizations. +So if you want to merge it into a protected branch you assign it to someone with maintainer authorizations. ## Issue tracking with GitLab flow diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index 0d592a6d43e..ae161e43233 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -144,7 +144,7 @@ git lfs unlock --id=123 ``` If for some reason you need to unlock a file that was not locked by you, -you can use the `--force` flag as long as you have a `master` access on +you can use the `--force` flag as long as you have a `maintainer` access on the project: ```bash diff --git a/doc/workflow/merge_requests.md b/doc/workflow/merge_requests.md index a68bb8b27ca..dc6da1938f3 100644 --- a/doc/workflow/merge_requests.md +++ b/doc/workflow/merge_requests.md @@ -1 +1 @@ -This document was moved to [user/project/merge_requests](../user/project/merge_requests.md). +This document was moved to [user/project/merge_requests/index.md](../user/project/merge_requests/index.md). diff --git a/doc/workflow/notifications.md b/doc/workflow/notifications.md index f1501c81b27..edb0c6bdc30 100644 --- a/doc/workflow/notifications.md +++ b/doc/workflow/notifications.md @@ -34,9 +34,14 @@ anything that is set at Global Settings.  -Group Settings are taking precedence over Global Settings but are on a level below Project Settings. +Group Settings are taking precedence over Global Settings but are on a level below Project or Subgroup Settings: + +``` +Group < Subgroup < Project +``` + This means that you can set a different level of notifications per group while still being able -to have a finer level setting per project. +to have a finer level setting per project or subgroup. Organization like this is suitable for users that belong to different groups but don't have the same need for being notified for every group they are member of. These settings can be configured on group page under the name of the group. It will be the dropdown with the bell icon. They can also be configured on the user profile notifications dropdown. @@ -106,6 +111,10 @@ by yourself (except when an issue is due). You will only receive automatic notifications when somebody else comments or adds changes to the ones that you've created or mentions you. +If a merge request becomes unmergeable, its author will be notified about the cause. +If a user has also set the merge request to automatically merge once pipeline succeeds, +then that user will also be notified. + ### Email Headers Notification emails include headers that provide extra content about the notification received: diff --git a/doc/workflow/protected_branches.md b/doc/workflow/protected_branches.md index aa48b8f750e..ced7d391ace 100644 --- a/doc/workflow/protected_branches.md +++ b/doc/workflow/protected_branches.md @@ -1 +1 @@ -This document is moved to [user/project/protected_branches.md](../user/project/protected_branches.md) +This document was moved to [another location](../user/project/protected_branches.md). diff --git a/doc/workflow/repository_mirroring.md b/doc/workflow/repository_mirroring.md new file mode 100644 index 00000000000..8c4e6ea8eab --- /dev/null +++ b/doc/workflow/repository_mirroring.md @@ -0,0 +1,351 @@ +# Repository mirroring + +Repository mirroring is a way to mirror repositories from external sources. +It can be used to mirror all branches, tags, and commits that you have +in your repository. + +Your mirror at GitLab will be updated automatically. You can +also manually trigger an update at most once every 5 minutes. + +## Overview + +Repository mirroring is very useful when, for some reason, you must use a +project from another source. + +There are two kinds of repository mirroring features supported by GitLab: +**push** and **pull**, the latter being only available in GitLab Enterprise Edition. +The **push** method mirrors the repository in GitLab to another location. + +Once the mirror repository is updated, all new branches, +tags, and commits will be visible in the project's activity feed. +Users with at least [developer access][perms] to the project can also force an +immediate update with the click of a button. This button will not be available if +the mirror is already being updated or 5 minutes still haven't passed since its last update. + +A few things/limitations to consider: + +- The repository must be accessible over `http://`, `https://`, `ssh://` or `git://`. +- If your HTTP repository is not publicly accessible, add authentication + information to the URL, like: `https://username@gitlab.company.com/group/project.git`. + In some cases, you might need to use a personal access token instead of a + password, e.g., you want to mirror to GitHub and have 2FA enabled. +- The import will time out after 15 minutes. For repositories that take longer + use a clone/push combination. +- The Git LFS objects will not be synced. You'll need to push/pull them + manually. + +## Use cases + +- You migrated to GitLab but still need to keep your project in another source. + In that case, you can simply set it up to mirror to GitLab (pull) and all the + essential history of commits, tags and branches will be available in your + GitLab instance. +- You have old projects in another source that you don't use actively anymore, + but don't want to remove for archiving purposes. In that case, you can create + a push mirror so that your active GitLab repository can push its changes to the + old location. + +## Pulling from a remote repository **[STARTER]** + +>[Introduced][ee-51] in GitLab Enterprise Edition 8.2. + +You can set up a repository to automatically have its branches, tags, and commits +updated from an upstream repository. This is useful when a repository you're +interested in is located on a different server, and you want to be able to +browse its content and its activity using the familiar GitLab interface. + +When creating a new project, you can enable repository mirroring when you choose +to import the repository from "Any repo by URL". Enter the full URL of the Git +repository to pull from and click on the **Mirror repository** checkbox. + + + +For an existing project, you can set up mirror pulling by visiting your project's +**Settings ➔ Repository** and searching for the "Pull from a remote repository" +section. Check the "Mirror repository" box and hit **Save changes** at the bottom. +You have a few options to choose from one being the user who will be the author +of all events in the activity feed that are the result of an update. This user +needs to have at least [master access][perms] to the project. Another option is +whether you want to trigger builds for mirror updates. + + + +Since the repository on GitLab functions as a mirror of the upstream repository, +you are advised not to push commits directly to the repository on GitLab. +Instead, any commits should be pushed to the upstream repository, and will end +up in the GitLab repository automatically within a certain period of time +or when a [forced update](#forcing-an-update) is initiated. + +If you do manually update a branch in the GitLab repository, the branch will +become diverged from upstream, and GitLab will no longer automatically update +this branch to prevent any changes from being lost. + + + +### Trigger update using API **[STARTER]** + +>[Introduced][ee-3453] in GitLab Enterprise Edition 10.3. + +Pull mirroring uses polling to detect new branches and commits added upstream, +often many minutes afterwards. If you notify GitLab by [API][pull-api], updates +will be pulled immediately. + +Read the [Pull Mirror Trigger API docs][pull-api]. + +### Pull only protected branches **[STARTER]** + +>[Introduced][ee-3326] in GitLab Enterprise Edition 10.3. + +You can choose to only pull the protected branches from your remote repository to GitLab. + +To use this option go to your project's repository settings page under pull mirror. + +### Overwrite diverged branches **[STARTER]** + +>[Introduced][ee-4559] in GitLab Enterprise Edition 10.6. + +You can choose to always update your local branch with the remote version even +if your local version has diverged from the remote. + +To use this option go to your project's repository settings page under pull mirror. + +### Hard failure **[STARTER]** + +>[Introduced][ee-3117] in GitLab Enterprise Edition 10.2. + +Once a mirror gets retried 14 times in a row, it will get marked as hard failed, +this will become visible in either the project main dashboard or in the +pull mirror settings page. + + + + + +When a project is hard failed, it will no longer get picked up for mirroring. +A user can resume the project mirroring again by either [forcing an update](#forcing-an-update) +or by changing the import URL in repository settings. + +### SSH authentication **[STARTER]** + +> [Introduced][ee-2551] in GitLab Starter 9.5 + +If you're mirroring over SSH (i.e., an `ssh://` URL), you can authenticate using +password-based authentication, just as over HTTPS, but you can also use public +key authentication. This is often more secure than password authentication, +especially when the source repository supports [Deploy Keys][deploy-key]. + +To get started, navigate to **Settings ➔ Repository ➔ Pull from a remote repository**, +enable mirroring (if not already enabled) and enter an `ssh://` URL. + +> **NOTE**: SCP-style URLs, e.g., `git@example.com:group/project.git`, are not +supported at this time. + +Entering the URL adds two features to the page - `Fingerprints` and +`SSH public key authentication`: + + + +SSH authentication is mutual. You have to prove to the server that you're +allowed to access the repository, but the server also has to prove to *you* that +it's who it claims to be. You provide your credentials as a password or public +key. The server that the source repository resides on provides its credentials +as a "host key", the fingerprint of which needs to be verified manually. + +Press the `Detect host keys` button. GitLab will fetch the host keys from the +server, and display the fingerprints to you: + + + +You now need to verify that the fingerprints are those you expect. GitLab.com +and other code hosting sites publish their fingerprints in the open for you +to check: + +* [AWS CodeCommit](http://docs.aws.amazon.com/codecommit/latest/userguide/regions.html#regions-fingerprints) +* [Bitbucket](https://confluence.atlassian.com/bitbucket/use-the-ssh-protocol-with-bitbucket-cloud-221449711.html#UsetheSSHprotocolwithBitbucketCloud-KnownhostorBitbucket%27spublickeyfingerprints) +* [GitHub](https://help.github.com/articles/github-s-ssh-key-fingerprints/) +* [GitLab.com](https://about.gitlab.com/gitlab-com/settings/#ssh-host-keys-fingerprints) +* [Launchpad](https://help.launchpad.net/SSHFingerprints) +* [Savannah](http://savannah.gnu.org/maintenance/SshAccess/) +* [SourceForge](https://sourceforge.net/p/forge/documentation/SSH%20Key%20Fingerprints/) + +Other providers will vary. If you're running on-premises GitLab, or otherwise +have access to the source server, you can securely gather the key fingerprints: + +``` +$ cat /etc/ssh/ssh_host*pub | ssh-keygen -E md5 -l -f - +256 MD5:f4:28:9f:23:99:15:21:1b:bf:ed:1f:8e:a0:76:b2:9d root@example.com (ECDSA) +256 MD5:e6:eb:45:8a:3c:59:35:5f:e9:5b:80:12:be:7e:22:73 root@example.com (ED25519) +2048 MD5:3f:72:be:3d:62:03:5c:62:83:e8:6e:14:34:3a:85:1d root@example.com (RSA) +``` + +(You may need to exclude `-E md5` for some older versions of SSH). + +If you're an SSH expert and already have a `known_hosts` file you'd like to use +unaltered, then you can skip these steps. Just press the "Show advanced" button +and paste in the file contents: + + + +Once you've **carefully verified** that all the fingerprints match your trusted +source, you can press `Save changes`. This will record the host keys, along with +the person who verified them (you!) and the date: + + + +When pulling changes from the source repository, GitLab will now check that at +least one of the stored host keys matches before connecting. This can prevent +malicious code from being injected into your mirror, or your password being +stolen! + +To use SSH public key authentication, you'll also need to choose that option +from the authentication methods dropdown. GitLab will generate a 4096-bit RSA +key and display the public component of that key to you: + + + +You then need to add the public SSH key to the source repository configuration. +If the source is hosted on GitLab, you should add it as a [Deploy Key][deploy-key]. +Other sources may require you to add the key to your user's `authorized_keys` +file - just paste the entire `ssh-rsa AAA.... user@host` block into the file on +its own line and save it. + +Once the public key is set up on the source repository, press `Save changes` and your +mirror will begin working. + +If you need to change the key at any time, you can press the `Regenerate key` +button to do so. You'll have to update the source repository with the new key +to keep the mirror running. + +### How it works + +Once you activate the pull mirroring feature, the mirror will be inserted into +a queue. A scheduler will start every minute and schedule a fixed amount of +mirrors for update, based on the configured maximum capacity. + +If the mirror successfully updates it will be enqueued once again with a small +backoff period. + +If the mirror fails (eg: branch diverged from upstream), the project's backoff +period will be penalized each time it fails up to a maximum amount of time. + +## Pushing to a remote repository + +>[Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/249) in +GitLab Enterprise Edition 8.7. [Moved to GitLab Community Edition][ce-18715] in 10.8. + +For an existing project, you can set up push mirror from your project's +**Settings ➔ Repository** and searching for the "Push to a remote repository" +section. Check the "Remote mirror repository" box and fill in the Git URL of +the repository to push to. Click **Save changes** for the changes to take +effect. + + + +When push mirroring is enabled, you are advised not to push commits directly +to the mirrored repository to prevent the mirror diverging. +All changes will end up in the mirrored repository whenever commits +are pushed to GitLab, or when a [forced update](#forcing-an-update) is +initiated. + +Pushes into GitLab are automatically pushed to the remote mirror at least once +every 5 minutes after they are received or once every minute if **push only +protected branches** is enabled. + +In case of a diverged branch, you will see an error indicated at the **Mirror +repository** settings. + + + +### Push only protected branches + +>[Introduced][ee-3350] in GitLab Enterprise Edition 10.3. [Moved to GitLab Community Edition][ce-18715] in 10.8. + +You can choose to only push your protected branches from GitLab to your remote repository. + +To use this option go to your project's repository settings page under push mirror. + +## Setting up a push mirror from GitLab to GitHub + +To set up a mirror from GitLab to GitHub, you need to follow these steps: + +1. Create a [GitHub personal access token](https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/) with the "public_repo" box checked: + +  + +1. Fill in the "Git repository URL" with the personal access token replacing the password `https://GitHubUsername:GitHubPersonalAccessToken@github.com/group/project.git`: + +  + +1. Save +1. And either wait or trigger the "Update Now" button: + +  + +## Forcing an update + +While mirrors are scheduled to update automatically, you can always force an update +by using the **Update now** button which is exposed in various places: + +- in the commits page +- in the branches page +- in the tags page +- in the **Mirror repository** settings page + +## Bidirectional mirroring + +CAUTION: **Warning:** +There is no bidirectional support without conflicts. If you +configure a repository to pull and push to a second remote, there is no +guarantee that it will update correctly on both remotes. If you configure +a repository for bidirectional mirroring, you should consider when conflicts +occur who and how they will be resolved. + +Rewriting any mirrored commit on either remote will cause conflicts and +mirroring to fail. This can be prevented by [only pulling protected branches]( +#pull-only-protected-branches) and [only pushing protected branches]( +#push-only-protected-branches). You should protect the branches you wish to +mirror on both remotes to prevent conflicts caused by rewriting history. + +Bidirectional mirroring also creates a race condition where commits to the same +branch in close proximity will cause conflicts. The race condition can be +mitigated by reducing the mirroring delay by using a Push event webhook to +trigger an immediate pull to GitLab. Push mirroring from GitLab is rate limited +to once per minute when only push mirroring protected branches. + +It may be possible to implement a locking mechanism using the server-side +`pre-receive` hook to prevent the race condition. Read about [configuring +custom Git hooks][hooks] on the GitLab server. + +### Mirroring with Perforce via GitFusion + +CAUTION: **Warning:** +Bidirectional mirroring should not be used as a permanent +configuration. There is no bidirectional mirroring without conflicts. +Refer to [Migrating from Perforce Helix][perforce] for alternative migration +approaches. + +GitFusion provides a Git interface to Perforce which can be used by GitLab to +bidirectionally mirror projects with GitLab. This may be useful in some +situations when migrating from Perforce to GitLab where overlapping Perforce +workspaces cannot be migrated simultaneously to GitLab. + +If using mirroring with Perforce you should only mirror protected branches. +Perforce will reject any pushes that rewrite history. It is recommended that +only the fewest number of branches are mirrored due to the performance +limitations of GitFusion. + +[ee-51]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/51 +[ee-2551]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2551 +[ee-3117]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3117 +[ee-3326]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3326 +[ee-3350]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3350 +[ee-3453]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3453 +[ee-4559]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4559 +[ce-18715]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18715 +[perms]: ../user/permissions.md +[hooks]: ../administration/custom_hooks.md +[deploy-key]: ../ssh/README.md#deploy-keys +[webhook]: ../user/project/integrations/webhooks.md#push-events +[pull-api]: ../api/projects.md#start-the-pull-mirroring-process-for-a-project +[perforce]: ../user/project/import/perforce.md diff --git a/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png b/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png Binary files differnew file mode 100644 index 00000000000..333648942f8 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png b/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png Binary files differnew file mode 100644 index 00000000000..45c9bce0889 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch_push.png b/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch_push.png Binary files differnew file mode 100644 index 00000000000..038b05cb31d --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch_push.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_github_edit_personal_access_token.png b/doc/workflow/repository_mirroring/repository_mirroring_github_edit_personal_access_token.png Binary files differnew file mode 100644 index 00000000000..139de42d8db --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_github_edit_personal_access_token.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_gitlab_push_to_a_remote_repository.png b/doc/workflow/repository_mirroring/repository_mirroring_gitlab_push_to_a_remote_repository.png Binary files differnew file mode 100644 index 00000000000..ccbc1d92329 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_gitlab_push_to_a_remote_repository.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_gitlab_push_to_a_remote_repository_update_now.png b/doc/workflow/repository_mirroring/repository_mirroring_gitlab_push_to_a_remote_repository_update_now.png Binary files differnew file mode 100644 index 00000000000..b16b3d2828e --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_gitlab_push_to_a_remote_repository_update_now.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png Binary files differnew file mode 100644 index 00000000000..99d429a1802 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png Binary files differnew file mode 100644 index 00000000000..0ab07afa3cc --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_new_project.png b/doc/workflow/repository_mirroring/repository_mirroring_new_project.png Binary files differnew file mode 100644 index 00000000000..43bf304838f --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_new_project.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png Binary files differnew file mode 100644 index 00000000000..5da5a7436bb --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png Binary files differnew file mode 100644 index 00000000000..4b9085302a1 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png Binary files differnew file mode 100644 index 00000000000..8c2efdafa43 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_push_settings.png b/doc/workflow/repository_mirroring/repository_mirroring_push_settings.png Binary files differnew file mode 100644 index 00000000000..f8199aa7c0f --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_push_settings.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png b/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png Binary files differnew file mode 100644 index 00000000000..93f3a532a0e --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png b/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png Binary files differnew file mode 100644 index 00000000000..6997ad511d9 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png diff --git a/doc/workflow/shortcuts.md b/doc/workflow/shortcuts.md index 2e1bd6bfe5c..b2f1cbec204 100644 --- a/doc/workflow/shortcuts.md +++ b/doc/workflow/shortcuts.md @@ -11,7 +11,7 @@ You can see GitLab's keyboard shortcuts by using 'shift + ?' | <kbd>f</kbd> | Focus filter | | <kbd>p</kbd> + <kbd>b</kbd> | Show/hide the Performance Bar | | <kbd>?</kbd> | Show/hide this dialog | -| <kbd>⌘</kbd> + <kbd>shift</kbd> + <kbd>p</kbd> | Toggle markdown preview | +| <kbd>Cmd</kbd>/<kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>p</kbd> | Toggle markdown preview | | <kbd>↑</kbd> | Edit last comment (when focused on an empty textarea) | ## Project Files Browsing @@ -46,15 +46,19 @@ You can see GitLab's keyboard shortcuts by using 'shift + ?' | Keyboard Shortcut | Description | | ----------------- | ----------- | | <kbd>g</kbd> + <kbd>p</kbd> | Go to the project's home page | -| <kbd>g</kbd> + <kbd>e</kbd> | Go to the project's activity feed | +| <kbd>g</kbd> + <kbd>v</kbd> | Go to the project's activity feed | | <kbd>g</kbd> + <kbd>f</kbd> | Go to files | | <kbd>g</kbd> + <kbd>c</kbd> | Go to commits | -| <kbd>g</kbd> + <kbd>b</kbd> | Go to jobs | +| <kbd>g</kbd> + <kbd>j</kbd> | Go to jobs | | <kbd>g</kbd> + <kbd>n</kbd> | Go to network graph | -| <kbd>g</kbd> + <kbd>g</kbd> | Go to repository charts | +| <kbd>g</kbd> + <kbd>d</kbd> | Go to repository charts | | <kbd>g</kbd> + <kbd>i</kbd> | Go to issues | +| <kbd>g</kbd> + <kbd>b</kbd> | Go to issue boards | | <kbd>g</kbd> + <kbd>m</kbd> | Go to merge requests | +| <kbd>g</kbd> + <kbd>e</kbd> | Go to environments | +| <kbd>g</kbd> + <kbd>k</kbd> | Go to kubernetes | | <kbd>g</kbd> + <kbd>s</kbd> | Go to snippets | +| <kbd>g</kbd> + <kbd>w</kbd> | Go to wiki | | <kbd>t</kbd> | Go to finding file | | <kbd>i</kbd> | New issue | @@ -66,8 +70,8 @@ You can see GitLab's keyboard shortcuts by using 'shift + ?' | <kbd>→</kbd> or <kbd>l</kbd> | Scroll right | | <kbd>↑</kbd> or <kbd>k</kbd> | Scroll up | | <kbd>↓</kbd> or <kbd>j</kbd> | Scroll down | -| <kbd>shift</kbd> + <kbd>↑</kbd> or <kbd>shift</kbd> + <kbd>k</kbd> | Scroll to top | -| <kbd>shift</kbd> + <kbd>↓</kbd> or <kbd>shift</kbd> + <kbd>j</kbd> | Scroll to bottom | +| <kbd>Shift</kbd> + <kbd>↑</kbd> or <kbd>Shift</kbd> + <kbd>k</kbd> | Scroll to top | +| <kbd>Shift</kbd> + <kbd>↓</kbd> or <kbd>Shift</kbd> + <kbd>j</kbd> | Scroll to bottom | ## Issues and Merge Requests @@ -84,3 +88,9 @@ You can see GitLab's keyboard shortcuts by using 'shift + ?' | Keyboard Shortcut | Description | | ----------------- | ----------- | | <kbd>e</kbd> | Edit wiki page| + +## Web IDE + +| Keyboard Shortcut | Description | +| ----------------- | ----------- | +| <kbd>⌘</kbd> + <kbd>p</kbd> | Go to file | diff --git a/doc/workflow/todos.md b/doc/workflow/todos.md index f13d29884d4..762bf616268 100644 --- a/doc/workflow/todos.md +++ b/doc/workflow/todos.md @@ -31,6 +31,9 @@ A Todo appears in your Todos dashboard when: - you are `@mentioned` in a comment on a commit, - a job in the CI pipeline running for your merge request failed, but this job is not allowed to fail. +- a merge request becomes unmergeable, and you are either: + - the author, or + - have set it to automatically merge once pipeline succeeds. ### Directly addressed Todos |
