diff options
Diffstat (limited to 'doc')
96 files changed, 1966 insertions, 1373 deletions
diff --git a/doc/README.md b/doc/README.md index ff8dd3fab8a..fee920f2012 100644 --- a/doc/README.md +++ b/doc/README.md @@ -162,7 +162,7 @@ configuration. Then customize everything from buildpacks to CI/CD. - [Auto DevOps](topics/autodevops/index.md) - [Deployment of Helm, Ingress, and Prometheus on Kubernetes](user/project/clusters/index.md#installing-applications) -- [Protected secret variables](ci/variables/README.md#protected-secret-variables) +- [Protected variables](ci/variables/README.md#protected-variables) - [Easy creation of Kubernetes clusters on GKE](user/project/clusters/index.md#adding-and-creating-a-new-gke-cluster-via-gitlab) ### Monitor @@ -191,7 +191,7 @@ instant how code changes impact your production environment. - [User account](user/profile/index.md): Manage your account - [Authentication](topics/authentication/index.md): Account security with two-factor authentication, setup your ssh keys and deploy keys for secure access to your projects. - [Profile settings](user/profile/index.md#profile-settings): Manage your profile settings, two factor authentication and more. -- [User permissions](user/permissions.md): Learn what each role in a project (external/guest/reporter/developer/master/owner) can do. +- [User permissions](user/permissions.md): Learn what each role in a project (external/guest/reporter/developer/maintainer/owner) can do. ### Git and GitLab @@ -215,7 +215,7 @@ Learn how to contribute to GitLab: - [Development](development/README.md): All styleguides and explanations how to contribute. - [Legal](legal/README.md): Contributor license agreements. -- [Writing documentation](development/writing_documentation.md): Contributing to GitLab Docs. +- [Writing documentation](development/documentation/index.md): Contributing to GitLab Docs. ## GitLab subscriptions diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 32ad63c3706..e11ed58eb91 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -2,7 +2,7 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7690) -in GitLab 8.15. Only project masters and owners can access web terminals. +in GitLab 8.15. Only project maintainers and owners can access web terminals. With the introduction of the [Kubernetes integration](../../user/project/clusters/index.md), GitLab gained the ability to store and use credentials for a Kubernetes cluster. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 77fe4d561a1..e59ab5a72e1 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -94,6 +94,7 @@ _The artifacts are stored by default in > Available in [GitLab Premium](https://about.gitlab.com/products/) and [GitLab.com Silver](https://about.gitlab.com/gitlab-com/). > Since version 10.6, available in [GitLab CE](https://about.gitlab.com/products/) +> Since version 11.0, we support direct_upload to S3. If you don't want to use the local disk where GitLab is installed to store the artifacts, you can use an object storage like AWS S3 instead. @@ -108,7 +109,7 @@ For source installations the following settings are nested under `artifacts:` an |---------|-------------|---------| | `enabled` | Enable/disable object storage | `false` | | `remote_directory` | The bucket name where Artifacts will be stored| | -| `direct_upload` | Set to true to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. Currently only `Google` provider is supported | `false` | +| `direct_upload` | Set to true to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. | `false` | | `background_upload` | Set to false to disable automatic upload. Option may be removed once upload is direct to S3 | `true` | | `proxy_download` | Set to true to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` | | `connection` | Various connection options described below | | diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index c0221533f13..9b1297ca4ba 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -83,12 +83,12 @@ you need to add a [wildcard DNS A record][wiki-wildcard-dns] pointing to the host that GitLab runs. For example, an entry would look like this: ``` -*.example.io. 1800 IN A 1.1.1.1 +*.example.io. 1800 IN A 192.0.2.1 *.example.io. 1800 IN AAAA 2001::1 ``` where `example.io` is the domain under which GitLab Pages will be served -and `1.1.1.1` is the IPv4 address of your GitLab instance and `2001::1` is the +and `192.0.2.1` is the IPv4 address of your GitLab instance and `2001::1` is the IPv6 address. If you don't have IPv6, you can omit the AAAA record. > **Note:** @@ -193,13 +193,13 @@ world. Custom domains are supported, but no TLS. ```shell pages_external_url "http://example.io" - nginx['listen_addresses'] = ['1.1.1.1'] + nginx['listen_addresses'] = ['192.0.2.1'] pages_nginx['enable'] = false - gitlab_pages['external_http'] = ['1.1.1.2:80', '[2001::2]:80'] + gitlab_pages['external_http'] = ['192.0.2.2:80', '[2001::2]:80'] ``` - where `1.1.1.1` is the primary IP address that GitLab is listening to and - `1.1.1.2` and `2001::2` are the secondary IPs the GitLab Pages daemon + where `192.0.2.1` is the primary IP address that GitLab is listening to and + `192.0.2.2` and `2001::2` are the secondary IPs the GitLab Pages daemon listens on. If you don't have IPv6, you can omit the IPv6 address. 1. [Reconfigure GitLab][reconfigure] @@ -228,16 +228,16 @@ world. Custom domains and TLS are supported. ```shell pages_external_url "https://example.io" - nginx['listen_addresses'] = ['1.1.1.1'] + nginx['listen_addresses'] = ['192.0.2.1'] pages_nginx['enable'] = false gitlab_pages['cert'] = "/etc/gitlab/ssl/example.io.crt" gitlab_pages['cert_key'] = "/etc/gitlab/ssl/example.io.key" - gitlab_pages['external_http'] = ['1.1.1.2:80', '[2001::2]:80'] - gitlab_pages['external_https'] = ['1.1.1.2:443', '[2001::2]:443'] + gitlab_pages['external_http'] = ['192.0.2.2:80', '[2001::2]:80'] + gitlab_pages['external_https'] = ['192.0.2.2:443', '[2001::2]:443'] ``` - where `1.1.1.1` is the primary IP address that GitLab is listening to and - `1.1.1.2` and `2001::2` are the secondary IPs where the GitLab Pages daemon + where `192.0.2.1` is the primary IP address that GitLab is listening to and + `192.0.2.2` and `2001::2` are the secondary IPs where the GitLab Pages daemon listens on. If you don't have IPv6, you can omit the IPv6 address. 1. [Reconfigure GitLab][reconfigure] diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md index a45c3306457..4e40a7cb18d 100644 --- a/doc/administration/pages/source.md +++ b/doc/administration/pages/source.md @@ -67,11 +67,11 @@ you need to add a [wildcard DNS A record][wiki-wildcard-dns] pointing to the host that GitLab runs. For example, an entry would look like this: ``` -*.example.io. 1800 IN A 1.1.1.1 +*.example.io. 1800 IN A 192.0.2.1 ``` where `example.io` is the domain under which GitLab Pages will be served -and `1.1.1.1` is the IP address of your GitLab instance. +and `192.0.2.1` is the IP address of your GitLab instance. > **Note:** You should not use the GitLab domain to serve user pages. For more information @@ -253,7 +253,7 @@ world. Custom domains are supported, but no TLS. port: 80 https: false - external_http: 1.1.1.2:80 + external_http: 192.0.2.2:80 ``` 1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in @@ -263,7 +263,7 @@ world. Custom domains are supported, but no TLS. ``` gitlab_pages_enabled=true - gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -listen-http 1.1.1.2:80" + gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -listen-http 192.0.2.2:80" ``` 1. Copy the `gitlab-pages-ssl` Nginx configuration file: @@ -274,7 +274,7 @@ world. Custom domains are supported, but no TLS. ``` 1. Edit all GitLab related configs in `/etc/nginx/site-available/` and replace - `0.0.0.0` with `1.1.1.1`, where `1.1.1.1` the primary IP where GitLab + `0.0.0.0` with `192.0.2.1`, where `192.0.2.1` the primary IP where GitLab listens to. 1. Restart NGINX 1. [Restart GitLab][restart] @@ -320,8 +320,8 @@ world. Custom domains and TLS are supported. port: 443 https: true - external_http: 1.1.1.2:80 - external_https: 1.1.1.2:443 + external_http: 192.0.2.2:80 + external_https: 192.0.2.2:443 ``` 1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in @@ -333,7 +333,7 @@ world. Custom domains and TLS are supported. ``` gitlab_pages_enabled=true - gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -listen-http 1.1.1.2:80 -listen-https 1.1.1.2:443 -root-cert /path/to/example.io.crt -root-key /path/to/example.io.key + gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -listen-http 192.0.2.2:80 -listen-https 192.0.2.2:443 -root-cert /path/to/example.io.crt -root-key /path/to/example.io.key ``` 1. Copy the `gitlab-pages-ssl` Nginx configuration file: @@ -344,7 +344,7 @@ world. Custom domains and TLS are supported. ``` 1. Edit all GitLab related configs in `/etc/nginx/site-available/` and replace - `0.0.0.0` with `1.1.1.1`, where `1.1.1.1` the primary IP where GitLab + `0.0.0.0` with `192.0.2.1`, where `192.0.2.1` the primary IP where GitLab listens to. 1. Restart NGINX 1. [Restart GitLab][restart] diff --git a/doc/api/access_requests.md b/doc/api/access_requests.md index 603fa4a8194..4b2014ca843 100644 --- a/doc/api/access_requests.md +++ b/doc/api/access_requests.md @@ -10,7 +10,7 @@ 10 => Guest access 20 => Reporter access 30 => Developer access -40 => Master access +40 => Maintainer access 50 => Owner access # Only valid for groups ``` diff --git a/doc/api/avatar.md b/doc/api/avatar.md new file mode 100644 index 00000000000..7faed893066 --- /dev/null +++ b/doc/api/avatar.md @@ -0,0 +1,33 @@ +# Avatar API + +> [Introduced][ce-19121] in GitLab 11.0 + +## Get a single avatar URL + +Get a single avatar URL for a given email addres. If user with matching public +email address is not found, results from external avatar services are returned. +This endpoint can be accessed without authentication. In case public visibility +is restricted, response will be `403 Forbidden` when unauthenticated. + +``` +GET /avatar?email=admin@example.com +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `email` | string | yes | Public email address of the user | +| `size` | integer | no | Single pixel dimension (since images are squares). Only used for avatar lookups at `Gravatar` or at the configured `Libravatar` server | + +```bash +curl https://gitlab.example.com/api/v4/avatar?email=admin@example.com +``` + +Example response: + +```json +{ + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon" +} +``` + +[ce-19121]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19121 diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md new file mode 100644 index 00000000000..dcd5377284c --- /dev/null +++ b/doc/api/graphql/index.md @@ -0,0 +1,42 @@ +# GraphQL API (Beta) + +> [Introduced][ce-19008] in GitLab 11.0. + +[GraphQL](https://graphql.org/) is a query language for APIs that +allows clients to request exactly the data they need, making it +possible to get all required data in a limited number of requests. + +The GraphQL data (fields) can be described in the form of types, +allowing clients to use [clientside GraphQL +libraries](https://graphql.org/code/#graphql-clients) to consume the +API and avoid manual parsing. + +Since there's no fixed endpoints and datamodel, new abilities can be +added to the API without creating breaking changes. This allows us to +have a versionless API as described in [the GraphQL +documentation](https://graphql.org/learn/best-practices/#versioning). + +## Enabling the GraphQL feature + +The GraphQL API itself is currently in Alpha, and therefore hidden behind a +feature flag. You can enable the feature using the [features api][features-api] on a self-hosted instance. + +For example: + +```shell +curl --data "value=100" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/features/graphql +``` + +## Available queries + +A first iteration of a GraphQL API includes only 2 queries: `project` and +`merge_request` and only returns scalar fields, or fields of the type `Project` +or `MergeRequest`. + +## GraphiQL + +The API can be explored by using the GraphiQL IDE, it is available on your +instance on `gitlab.example.com/-/graphql-explorer`. + +[ce-19008]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19008 +[features-api]: ../features.md diff --git a/doc/api/members.md b/doc/api/members.md index 3234f833eae..1a10aa75ac0 100644 --- a/doc/api/members.md +++ b/doc/api/members.md @@ -8,7 +8,7 @@ The access levels are defined in the `Gitlab::Access` module. Currently, these l 10 => Guest access 20 => Reporter access 30 => Developer access -40 => Master access +40 => Maintainer access 50 => Owner access # Only valid for groups ``` diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 051d2a10bc6..9f06e20f803 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -582,6 +582,7 @@ Parameters: "changes_count": "1", "should_remove_source_branch": true, "force_remove_source_branch": false, + "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, "time_stats": { @@ -650,7 +651,8 @@ POST /projects/:id/merge_requests | `labels` | string | no | Labels for MR as a comma-separated list | | `milestone_id` | integer | no | The global ID of a milestone | | `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | -| `allow_maintainer_to_push` | boolean | no | Whether or not a maintainer of the target project can push to the source branch | +| `allow_collaboration` | boolean | no | Allow commits from members who can merge to the target branch | +| `allow_maintainer_to_push` | boolean | no | Deprecated, see allow_collaboration | | `squash` | boolean | no | Squash commits into a single commit when merging | ```json @@ -708,6 +710,7 @@ POST /projects/:id/merge_requests "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, + "allow_collaboration": false, "allow_maintainer_to_push": false, "time_stats": { "time_estimate": 0, @@ -740,7 +743,8 @@ PUT /projects/:id/merge_requests/:merge_request_iid | `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | | `squash` | boolean | no | Squash commits into a single commit when merging | | `discussion_locked` | boolean | no | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | -| `allow_maintainer_to_push` | boolean | no | Whether or not a maintainer of the target project can push to the source branch | +| `allow_collaboration` | boolean | no | Allow commits from members who can merge to the target branch | +| `allow_maintainer_to_push` | boolean | no | Deprecated, see allow_collaboration | Must include at least one non-required attribute from above. @@ -798,6 +802,7 @@ Must include at least one non-required attribute from above. "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, + "allow_collaboration": false, "allow_maintainer_to_push": false, "time_stats": { "time_estimate": 0, diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index 1f0dc700640..656bf07bb55 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -54,7 +54,7 @@ Example response: ] ``` -**Note**: `members_count_with_descendants` are presented only for group masters/owners. +**Note**: `members_count_with_descendants` are presented only for group maintainers/owners. ## Search for namespace diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index 899f5da6647..ebae68fe389 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -102,6 +102,7 @@ POST /projects/:id/pipeline |------------|---------|----------|---------------------| | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `ref` | string | yes | Reference to commit | +| `variables` | array | no | An array containing the variables available in the pipeline, matching the structure [{ 'key' => 'UPLOAD_TO_S3', 'value' => 'true' }] | ``` curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/pipeline?ref=master" diff --git a/doc/api/protected_branches.md b/doc/api/protected_branches.md index 950ead52560..f6813f27dc0 100644 --- a/doc/api/protected_branches.md +++ b/doc/api/protected_branches.md @@ -8,7 +8,7 @@ The access levels are defined in the `ProtectedRefAccess::ALLOWED_ACCESS_LEVELS` ``` 0 => No access 30 => Developer access -40 => Master access +40 => Maintainer access ``` ## List protected branches @@ -36,13 +36,13 @@ Example response: "push_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ], "merge_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ] }, @@ -75,13 +75,13 @@ Example response: "push_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ], "merge_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ] } @@ -104,8 +104,8 @@ curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" 'https://gitl | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `name` | string | yes | The name of the branch or wildcard | -| `push_access_level` | string | no | Access levels allowed to push (defaults: `40`, master access level) | -| `merge_access_level` | string | no | Access levels allowed to merge (defaults: `40`, master access level) | +| `push_access_level` | string | no | Access levels allowed to push (defaults: `40`, maintainer access level) | +| `merge_access_level` | string | no | Access levels allowed to merge (defaults: `40`, maintainer access level) | Example response: @@ -115,13 +115,13 @@ Example response: "push_access_levels": [ { "access_level": 30, - "access_level_description": "Developers + Masters" + "access_level_description": "Developers + Maintainers" } ], "merge_access_levels": [ { "access_level": 30, - "access_level_description": "Developers + Masters" + "access_level_description": "Developers + Maintainers" } ] } diff --git a/doc/api/services.md b/doc/api/services.md index f23303ef836..aeb48ccc36c 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -1,6 +1,6 @@ # Services API ->**Note:** This API requires an access token with Master or Owner permissions +>**Note:** This API requires an access token with Maintainer or Owner permissions ## Asana diff --git a/doc/api/settings.md b/doc/api/settings.md index 36a0782d8f2..e6b207d8746 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -81,7 +81,7 @@ PUT /application/settings | `clientside_sentry_enabled` | boolean | no | Enable Sentry error reporting for the client side | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts | -| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and masters can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but masters can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | +| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and maintainers can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and maintainers can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but maintainers can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | | `default_group_visibility` | string | no | What visibility level new groups receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_project_visibility` | string | no | What visibility level new projects receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_projects_limit` | integer | no | Project limit per user. Default is `100000` | diff --git a/doc/articles/index.md b/doc/articles/index.md index 9f85533ea94..87ee17bb6de 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -4,7 +4,7 @@ comments: false # Technical articles list (deprecated) -[Technical articles](../development/writing_documentation.md#technical-articles) are +[Technical articles](../development/documentation/index.md#technical-articles) are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 7c0f837ea9c..71f1d69cdf4 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -496,7 +496,7 @@ To configure access for `registry.example.com`, follow these steps: bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= ``` -1. Create a [secret variable] `DOCKER_AUTH_CONFIG` with the content of the +1. Create a [variable] `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: ```json @@ -632,7 +632,7 @@ creation. [postgres-hub]: https://hub.docker.com/r/_/postgres/ [mysql-hub]: https://hub.docker.com/r/_/mysql/ [runner-priv-reg]: http://docs.gitlab.com/runner/configuration/advanced-configuration.html#using-a-private-container-registry -[secret variable]: ../variables/README.md#secret-variables +[variable]: ../variables/README.md#variables [entrypoint]: https://docs.docker.com/engine/reference/builder/#entrypoint [cmd]: https://docs.docker.com/engine/reference/builder/#cmd [register]: https://docs.gitlab.com/runner/register/ diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 7f034409580..36fd8affa5b 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -249,7 +249,7 @@ the basis of [Review apps](review_apps/index.md). NOTE: **Note:** The `name` and `url` parameters can use most of the CI/CD variables, including [predefined](variables/README.md#predefined-variables-environment-variables), -[secret](variables/README.md#secret-variables) and +[project/group ones](variables/README.md#variables) and [`.gitlab-ci.yml` variables](yaml/README.md#variables). You however cannot use variables defined under `script` or on the Runner's side. There are also other variables that are unsupported in the context of `environment:name`. You can read more about @@ -593,7 +593,7 @@ version of the app, all without leaving GitLab. >**Note:** Web terminals were added in GitLab 8.15 and are only available to project -masters and owners. +maintainers and owners. If you deploy to your environments with the help of a deployment service (e.g., the [Kubernetes integration][kube]), GitLab can open diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index d931c9a77f4..9657f52159e 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -58,7 +58,7 @@ The application is ready to use, but you need some additional steps to deploy it 1. Log in to Artifactory with your user's credentials. 1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel. 1. Copy to clipboard the configuration snippet under the **Deploy** paragraph. -1. Change the `url` value in order to have it configurable via secret variables. +1. Change the `url` value in order to have it configurable via variables. 1. Copy the snippet in the `pom.xml` file for your project, just after the `dependencies` section. The snippet should look like this: @@ -98,7 +98,7 @@ parameter in `.gitlab-ci.yml` to use the custom location instead of the default </settings> ``` - Username and password will be replaced by the correct values using secret variables. + Username and password will be replaced by the correct values using variables. ### Configure GitLab CI/CD for `simple-maven-dep` @@ -107,8 +107,8 @@ Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/features/gitlab- GitLab CI/CD uses a file in the root of the repo, named `.gitlab-ci.yml`, to read the definitions for jobs that will be executed by the configured GitLab Runners. You can read more about this file in the [GitLab Documentation](https://docs.gitlab.com/ee/ci/yaml/). -First of all, remember to set up secret variables for your deployment. Navigate to your project's **Settings > CI/CD** page -and add the following secret variables (replace them with your current values, of course): +First of all, remember to set up variables for your deployment. Navigate to your project's **Settings > CI/CD > Variables** page +and add the following ones (replace them with your current values, of course): - **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL) - **MAVEN_REPO_USER**: `gitlab` (your Artifactory username) @@ -156,7 +156,7 @@ by running all Maven phases in a sequential order, therefore, executing `mvn tes Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application. -Deploy to Artifactory is done as defined by the secret variables we have just set up. +Deploy to Artifactory is done as defined by the variables we have just set up. The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published. Done! Now you have all the changes in the GitLab repo, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening. diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 2dcdc2d41ec..bd60d641493 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -111,7 +111,7 @@ We also use two secure variables: ## Storing API keys Secure Variables can added by going to your project's -**Settings ➔ CI / CD ➔ Secret variables**. The variables that are defined +**Settings ➔ CI / CD ➔ Variables**. The variables that are defined in the project settings are sent along with the build script to the Runner. The secure variables are stored out of the repository. Never store secrets in your project's `.gitlab-ci.yml`. It is also important that the secret's value diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 3d21c0cc306..c226b5bfb71 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -406,7 +406,7 @@ and further delves into the principles of GitLab CI/CD than discussed in this ar We need to be able to deploy to AWS with our AWS account credentials, but we certainly don't want to put secrets into source code. Luckily GitLab provides a solution for this -with [Secret Variables](../../../ci/variables/README.md). This can get complicated +with [Variables](../../../ci/variables/README.md). This can get complicated due to [IAM](https://aws.amazon.com/iam/) management. As a best practice, you shouldn't use root security credentials. Proper IAM credential management is beyond the scope of this article, but AWS will remind you that using root credentials is unadvised and against their @@ -428,7 +428,7 @@ fully understand [IAM Best Practices in AWS](http://docs.aws.amazon.com/IAM/late To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on the Shared Runner. The Shared Runner also needs to be able to authenticate with your AWS account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID` -and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the secret variables we +and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the variables we set up in the prior section using the `variables` portion of the `deploy` job. At the end, we add directives to ensure deployment `only` happens on pushes to `master`. This way, every single branch still runs through CI, and only merging (or committing directly) to master will diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 1f9b9d53fc1..39c65399332 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -116,11 +116,11 @@ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_rsa ``` -Now, let's add it to your GitLab project as a [secret variable](../../variables/README.md#secret-variables). -Secret variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. +Now, let's add it to your GitLab project as a [variable](../../variables/README.md#variables). +Variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. They can be added per project by navigating to the project's **Settings** > **CI/CD**. - + To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our remote server as the deployer user without entering its password. diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index b16cbc61d14..4e964af97f5 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -258,7 +258,7 @@ on that specific branch: - trigger **manual actions** on existing pipelines - **retry/cancel** existing jobs (using Web UI or Pipelines API) -**Secret variables** marked as **protected** are accessible only to jobs that +**Variables** marked as **protected** are accessible only to jobs that run on protected branches, avoiding untrusted users to get unintended access to sensitive information like deployment credentials and tokens. diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 703a7f030ed..8f1ff190804 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -84,7 +84,7 @@ visit the project you want to make the Runner work for in GitLab: ## Registering a group Runner -Creating a group Runner requires Master permissions for the group. To create a +Creating a group Runner requires Maintainer permissions for the group. To create a group Runner visit the group you want to make the Runner work for in GitLab: 1. Go to **Settings > CI/CD** to obtain the token @@ -120,9 +120,9 @@ To lock/unlock a Runner: ## Assigning a Runner to another project -If you are Master on a project where a specific Runner is assigned to, and the +If you are Maintainer on a project where a specific Runner is assigned to, and the Runner is not [locked only to that project](#locking-a-specific-runner-from-being-enabled-for-other-projects), -you can enable the Runner also on any other project where you have Master permissions. +you can enable the Runner also on any other project where you have Maintainer permissions. To enable/disable a Runner in your project: @@ -132,7 +132,7 @@ To enable/disable a Runner in your project: > **Note**: Consider that if you don't lock your specific Runner to a specific project, any -user with Master role in you project can assign your Runner to another arbitrary +user with Maintainer role in you project can assign your Runner to another arbitrary project without requiring your authorization, so use it with caution. An admin can enable/disable a specific Runner for projects: diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 693c8e9ef18..4cb05509e7b 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -25,7 +25,7 @@ with any type of [executor](https://docs.gitlab.com/runner/executors/) ## How it works 1. Create a new SSH key pair locally with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen) -1. Add the private key as a [secret variable](../variables/README.md) to +1. Add the private key as a [variable](../variables/README.md) to your project 1. Run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during job to load the private key. @@ -49,7 +49,7 @@ to access it. This is where an SSH key pair comes in handy. **Do not** add a passphrase to the SSH key, or the `before_script` will\ prompt for it. -1. Create a new [secret variable](../variables/README.md#secret-variables). +1. Create a new [variable](../variables/README.md#variables). As **Key** enter the name `SSH_PRIVATE_KEY` and in the **Value** field paste the content of your _private_ key that you created earlier. @@ -157,7 +157,7 @@ ssh-keyscan example.com ssh-keyscan 1.2.3.4 ``` -Create a new [secret variable](../variables/README.md#secret-variables) with +Create a new [variable](../variables/README.md#variables) with `SSH_KNOWN_HOSTS` as "Key", and as a "Value" add the output of `ssh-keyscan`. NOTE: **Note:** @@ -165,7 +165,7 @@ If you need to connect to multiple servers, all the server host keys need to be collected in the **Value** of the variable, one key per line. TIP: **Tip:** -By using a secret variable instead of `ssh-keyscan` directly inside +By using a variable instead of `ssh-keyscan` directly inside `.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml` if the host domain name changes for some reason. Also, the values are predefined by you, meaning that if the host keys suddenly change, the CI/CD job will fail, diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 47a576fdf5f..c507036aa6a 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -53,7 +53,7 @@ The action is irreversible. it will not trigger a job. - If your project is public, passing the token in plain text is probably not the wisest idea, so you might want to use a - [secret variable](../variables/README.md#secret-variables) for that purpose. + [variable](../variables/README.md#variables) for that purpose. To trigger a job you need to send a `POST` request to GitLab's API endpoint: diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index f10423b92cf..1b24bcdbf6f 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -10,17 +10,17 @@ The variables can be overwritten and they take precedence over each other in this order: 1. [Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables) (take precedence over all) -1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) -1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) +1. Project-level [variables](#variables) or [protected variables](#protected-variables) +1. Group-level [variables](#variables) or [protected variables](#protected-variables) 1. YAML-defined [job-level variables](../yaml/README.md#variables) 1. YAML-defined [global variables](../yaml/README.md#variables) 1. [Deployment variables](#deployment-variables) 1. [Predefined variables](#predefined-variables-environment-variables) (are the lowest in the chain) -For example, if you define `API_TOKEN=secure` as a secret variable and +For example, if you define `API_TOKEN=secure` as a project variable and `API_TOKEN=yaml` in your `.gitlab-ci.yml`, the `API_TOKEN` will take the value -`secure` as the secret variables are higher in the chain. +`secure` as the project variables are higher in the chain. ## Unsupported variables @@ -72,6 +72,7 @@ future GitLab releases.** | **CI_RUNNER_REVISION** | all | 10.6 | GitLab Runner revision that is executing the current job | | **CI_RUNNER_EXECUTABLE_ARCH** | all | 10.6 | The OS/architecture of the GitLab Runner executable (note that this is not necessarily the same as the environment of the executor) | | **CI_PIPELINE_ID** | 8.10 | 0.5 | The unique id of the current pipeline that GitLab CI uses internally | +| **CI_PIPELINE_IID** | 11.0 | all | The unique id of the current pipeline scoped to project | | **CI_PIPELINE_TRIGGERED** | all | all | The flag to indicate that job was [triggered] | | **CI_PIPELINE_SOURCE** | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `trigger`, `schedule`, `api`, and `pipeline`. For pipelines created before GitLab 9.5, this will show as `unknown` | | **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the job is run | @@ -164,49 +165,49 @@ script: - 'eval $LS_CMD' # will execute 'ls -al $TMP_DIR' ``` -## Secret variables +## Variables NOTE: **Note:** -Group-level secret variables were added in GitLab 9.4. +Group-level variables were added in GitLab 9.4. CAUTION: **Important:** -Be aware that secret variables are not masked, and their values can be shown +Be aware that variables are not masked, and their values can be shown in the job logs if explicitly asked to do so. If your project is public or internal, you can set the pipelines private from your [project's Pipelines settings](../../user/project/pipelines/settings.md#visibility-of-pipelines). -Follow the discussion in issue [#13784][ce-13784] for masking the secret variables. +Follow the discussion in issue [#13784][ce-13784] for masking the variables. -GitLab CI allows you to define per-project or per-group secret variables -that are set in the pipeline environment. The secret variables are stored out of +GitLab CI allows you to define per-project or per-group variables +that are set in the pipeline environment. The variables are stored out of the repository (not in `.gitlab-ci.yml`) and are securely passed to GitLab Runner making them available during a pipeline run. It's the recommended method to use for storing things like passwords, SSH keys and credentials. -Project-level secret variables can be added by going to your project's -**Settings > CI/CD**, then finding the section called **Secret variables**. +Project-level variables can be added by going to your project's +**Settings > CI/CD**, then finding the section called **Variables**. -Likewise, group-level secret variables can be added by going to your group's -**Settings > CI/CD**, then finding the section called **Secret variables**. +Likewise, group-level variables can be added by going to your group's +**Settings > CI/CD**, then finding the section called **Variables**. Any variables of [subgroups] will be inherited recursively. - + Once you set them, they will be available for all subsequent pipelines. You can also -[protect your variables](#protected-secret-variables). +[protect your variables](#protected-variables). -### Protected secret variables +### Protected variables >**Notes:** This feature requires GitLab 9.3 or higher. -Secret variables could be protected. Whenever a secret variable is +Variables could be protected. Whenever a variable is protected, it would only be securely passed to pipelines running on the [protected branches] or [protected tags]. The other pipelines would not get any protected variables. Protected variables can be added by going to your project's **Settings > CI/CD**, then finding the section called -**Secret variables**, and check "Protected". +**Variables**, and check "Protected". Once you set them, they will be available for all subsequent pipelines. @@ -230,7 +231,7 @@ An example project service that defines deployment variables is the CAUTION: **Warning:** Enabling debug tracing can have severe security implications. The -output **will** contain the content of all your secret variables and any other +output **will** contain the content of all your variables and any other secrets! The output **will** be uploaded to the GitLab server and made visible in job traces! @@ -352,6 +353,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ CI_PROJECT_URL=https://example.com/gitlab-examples/ci-debug-trace ++ export CI_PIPELINE_ID=52666 ++ CI_PIPELINE_ID=52666 +++ export CI_PIPELINE_IID=123 +++ CI_PIPELINE_IID=123 ++ export CI_RUNNER_ID=1337 ++ CI_RUNNER_ID=1337 ++ export CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com @@ -416,7 +419,7 @@ job_name: ``` You can also list all environment variables with the `export` command, -but be aware that this will also expose the values of all the secret variables +but be aware that this will also expose the values of all the variables you set, in the job log: ```yaml @@ -439,6 +442,7 @@ export CI_JOB_MANUAL="true" export CI_JOB_TRIGGERED="true" export CI_JOB_TOKEN="abcde-1234ABCD5678ef" export CI_PIPELINE_ID="1000" +export CI_PIPELINE_IID="10" export CI_PROJECT_ID="34" export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" export CI_PROJECT_NAME="gitlab-ce" @@ -468,7 +472,7 @@ It is possible to use variables expressions with only / except policies in `.gitlab-ci.yml`. By using this approach you can limit what jobs are going to be created within a pipeline after pushing a code to GitLab. -This is particularly useful in combination with secret variables and triggered +This is particularly useful in combination with variables and triggered pipeline variables. ```yaml @@ -546,7 +550,7 @@ Below you can find supported syntax reference: Pattern matching is case-sensitive by default. Use `i` flag modifier, like `/pattern/i` to make a pattern case-insensitive. -[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI secret variables" +[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI variables" [eep]: https://about.gitlab.com/products/ "Available only in GitLab Premium" [envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index 9800784d918..b2b4a26bdda 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -17,7 +17,7 @@ There are basically two places where you can use any defined variables: | Definition | Can be expanded? | Expansion place | Description | |--------------------------------------|-------------------|-----------------|--------------| -| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>**Supported:** all variables defined for a job (secret variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>**Not suported:** variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> | +| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>**Supported:** all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>**Not suported:** variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> | | `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion **doesn't support**: <ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> | | `variables` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | | `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | @@ -55,7 +55,7 @@ since the expansion is done in GitLab before any Runner will get the job. ### GitLab Runner internal variable expansion mechanism -- **Supported:** secret variables, `.gitlab-ci.yml` variables, `config.toml` variables, and +- **Supported:** project/group variables, `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules - **Not supported:** variables defined inside of scripts (e.g., `export MY_VARIABLE="test"`) @@ -76,7 +76,7 @@ are using a different variables syntax. **Supported:** - The `script` may use all available variables that are default for the shell (e.g., `$PATH` which - should be present in all bash/sh shells) and all variables defined by GitLab CI/CD (secret variables, + should be present in all bash/sh shells) and all variables defined by GitLab CI/CD (project/group variables, `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules). - The `script` may also use all variables defined in the lines before. So, for example, if you define a variable `export MY_VARIABLE="test"`: diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3e77a6f58b7..f946536701e 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -327,7 +327,7 @@ Refs strategy equals to simplified only/except configuration, whereas kubernetes strategy accepts only `active` keyword. `variables` keyword is used to define variables expressions. In other words -you can use predefined variables / secret variables / project / group or +you can use predefined variables / project / group or environment-scoped variables to define an expression GitLab is going to evaluate in order to decide whether a job should be created or not. @@ -1249,7 +1249,7 @@ Runner itself](../variables/README.md#predefined-variables-environment-variables One example would be `CI_COMMIT_REF_NAME` which has the value of the branch or tag name for which project is built. Apart from the variables you can set in `.gitlab-ci.yml`, there are also the so called -[secret variables](../variables/README.md#secret-variables) +[Variables](../variables/README.md#variables) which can be set in GitLab's UI. [Learn more about variables and their priority.][variables] diff --git a/doc/development/README.md b/doc/development/README.md index 898c60e96c0..92d9829192e 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -32,6 +32,8 @@ description: 'Learn how to contribute to GitLab.' - [GitLab utilities](utilities.md) - [API styleguide](api_styleguide.md) Use this styleguide if you are contributing to the API. +- [GrapQL API styleguide](api_graphql_styleguide.md) Use this + styleguide if you are contribution to the [GraphQL API](../api/graphql/index.md) - [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers - [Working with Gitaly](gitaly.md) - [Manage feature flags](feature_flags.md) @@ -86,8 +88,8 @@ description: 'Learn how to contribute to GitLab.' ## Documentation guides -- [Writing documentation](writing_documentation.md) -- [Documentation styleguide](doc_styleguide.md) +- [Writing documentation](documentation/index.md) +- [Documentation styleguide](documentation/styleguide.md) - [Markdown](../user/markdown.md) ## Internationalization (i18n) guides diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md new file mode 100644 index 00000000000..f74e4f0bd7e --- /dev/null +++ b/doc/development/api_graphql_styleguide.md @@ -0,0 +1,81 @@ +# GraphQL API + +## Authentication + +Authentication happens through the `GraphqlController`, right now this +uses the same authentication as the Rails application. So the session +can be shared. + +It is also possible to add a `private_token` to the querystring, or +add a `HTTP_PRIVATE_TOKEN` header. + +### Authorization + +Fields can be authorized using the same abilities used in the Rails +app. This can be done using the `authorize` helper: + +```ruby +module Types + class QueryType < BaseObject + graphql_name 'Query' + + field :project, Types::ProjectType, null: true, resolver: Resolvers::ProjectResolver do + authorize :read_project + end + end +``` + +The object found by the resolve call is used for authorization. + +This works for authorizing a single record, for authorizing +collections, we should only load what the currently authenticated user +is allowed to view. Preferably we use our existing finders for that. + +## Types + +When exposing a model through the GraphQL API, we do so by creating a +new type in `app/graphql/types`. + +When exposing properties in a type, make sure to keep the logic inside +the definition as minimal as possible. Instead, consider moving any +logic into a presenter: + +```ruby +class Types::MergeRequestType < BaseObject + present_using MergeRequestPresenter + + name 'MergeRequest' +end +``` + +An existing presenter could be used, but it is also possible to create +a new presenter specifically for GraphQL. + +The presenter is initialized using the object resolved by a field, and +the context. + +## Resolvers + +To find objects to display in a field, we can add resolvers to +`app/graphql/resolvers`. + +Arguments can be defined within the resolver, those arguments will be +made available to the fields using the resolver. + +We already have a `FullPathLoader` that can be included in other +resolvers to quickly find Projects and Namespaces which will have a +lot of dependant objects. + +To limit the amount of queries performed, we can use `BatchLoader`. + +## Testing + +_full stack_ tests for a graphql query or mutation live in +`spec/requests/api/graphql`. + +When adding a query, the `a working graphql query` shared example can +be used to test if the query renders valid results. + +Using the `GraphqlHelpers#all_graphql_fields_for`-helper, a query +including all available fields can be constructed. This makes it easy +to add a test rendering all possible fields for a query. diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 5d595c33915..7d84f8ca86a 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -1,499 +1,3 @@ --- -description: 'Writing styles, markup, formatting, and reusing regular expressions throughout the GitLab Documentation.' +redirect_to: 'documentation/styleguide.md' --- - -# Documentation style guidelines - -The documentation style guide defines the markup structure used in -GitLab documentation. Check the -[documentation guidelines](writing_documentation.md) for general development instructions. - -Check the GitLab handbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). - -## Text - -- Split up long lines (wrap text), this makes it much easier to review and edit. Only - double line breaks are shown as a full line break in [GitLab markdown][gfm]. - 80-100 characters is a good line length -- Make sure that the documentation is added in the correct - [directory](writing_documentation.md#documentation-directory-structure) and that - there's a link to it somewhere useful -- Do not duplicate information -- Be brief and clear -- Unless there's a logical reason not to, add documents in alphabetical order -- Write in US English -- Use [single spaces][] instead of double spaces -- Jump a line between different markups (e.g., after every paragraph, header, list, etc) -- Capitalize "G" and "L" in GitLab -- Use sentence case for titles, headings, labels, menu items, and buttons. -- Use title case when referring to [features](https://about.gitlab.com/features/) or [products](https://about.gitlab.com/pricing/), and methods. Note that some features are also objects (e.g. "Merge Requests" and "merge requests"). E.g.: GitLab Runner, Geo, Issue Boards, Git, Prometheus, Continuous Integration. - -## Formatting - -- Use double asterisks (`**`) to mark a word or text in bold (`**bold**`) -- Use undescore (`_`) for text in italics (`_italic_`) -- Jump a line between different markups, for example: - - ```md - ## Header - - Paragraph. - - - List item - - List item - ``` - -### Punctuation - -For punctuation rules, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/). - -### Ordered and unordered lists - -- Use dashes (`-`) for unordered lists instead of asterisks (`*`) -- Use the number one (`1`) for ordered lists -- For punctuation in bullet lists, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/) - -## Headings - -- Add **only one H1** in each document, by adding `#` at the beginning of - it (when using markdown). The `h1` will be the document `<title>`. -- For subheadings, use `##`, `###` and so on -- Avoid putting numbers in headings. Numbers shift, hence documentation anchor - links shift too, which eventually leads to dead links. If you think it is - compelling to add numbers in headings, make sure to at least discuss it with - someone in the Merge Request -- [Avoid using symbols and special chars](https://gitlab.com/gitlab-com/gitlab-docs/issues/84) - in headers. Whenever possible, they should be plain and short text. -- Avoid adding things that show ephemeral statuses. For example, if a feature is - considered beta or experimental, put this info in a note, not in the heading. -- When introducing a new document, be careful for the headings to be - grammatically and syntactically correct. Mention one or all - of the following GitLab members for a review: `@axil` or `@marcia`. - This is to ensure that no document with wrong heading is going - live without an audit, thus preventing dead links and redirection issues when - corrected -- Leave exactly one newline after a heading - -## Links - -- Use the regular inline link markdown markup `[Text](https://example.com)`. - It's easier to read, review, and maintain. -- If there's a link that repeats several times through the same document, - you can use `[Text][identifier]` and at the bottom of the section or the - document add: `[identifier]: https://example.com`, in which case, we do - encourage you to also add an alternative text: `[identifier]: https://example.com "Alternative text"` that appears when hovering your mouse on a link. -- To link to internal documentation, use relative links, not full URLs. Use `../` to - navigate tp high-level directories, and always add the file name `file.md` at the - end of the link with the `.md` extension, not `.html`. - Example: instead of `[text](../../merge_requests/)`, use - `[text](../../merge_requests/index.md)` or, `[text](../../ci/README.md)`, or, - for anchor links, `[text](../../ci/README.md#examples)`. - Using the markdown extension is necessary for the [`/help`](writing_documentation.md#gitlab-help) - section of GitLab. -- To link from CE to EE-only documentation, use the EE-only doc full URL. -- Use [meaningful anchor texts](https://www.futurehosting.com/blog/links-should-have-meaningful-anchor-text-heres-why/). - E.g., instead of writing something like `Read more about GitLab Issue Boards [here](LINK)`, - write `Read more about [GitLab Issue Boards](LINK)`. - -## Images - -- Place images in a separate directory named `img/` in the same directory where - the `.md` document that you're working on is located. Always prepend their - names with the name of the document that they will be included in. For - example, if there is a document called `twitter.md`, then a valid image name - could be `twitter_login_screen.png`. [**Exception**: images for - [articles](writing_documentation.md#technical-articles) should be - put in a directory called `img` underneath `/articles/article_title/img/`, therefore, - there's no need to prepend the document name to their filenames.] -- Images should have a specific, non-generic name that will differentiate them. -- Keep all file names in lower case. -- Consider using PNG images instead of JPEG. -- Compress all images with <https://tinypng.com/> or similar tool. -- Compress gifs with <https://ezgif.com/optimize> or similar tool. -- Images should be used (only when necessary) to _illustrate_ the description -of a process, not to _replace_ it. - -Inside the document: - -- The Markdown way of using an image inside a document is: - `` -- Always use a proper description for what the image is about. That way, when a - browser fails to show the image, this text will be used as an alternative - description -- If there are consecutive images with little text between them, always add - three dashes (`---`) between the image and the text to create a horizontal - line for better clarity -- If a heading is placed right after an image, always add three dashes (`---`) - between the image and the heading - -## Alert boxes - -Whenever you want to call the attention to a particular sentence, -use the following markup for highlighting. - -_Note that the alert boxes only work for one paragraph only. Multiple paragraphs, -lists, headers, etc will not render correctly._ - -### Note - -```md -NOTE: **Note:** -This is something to note. -``` - -How it renders in docs.gitlab.com: - -NOTE: **Note:** -This is something to note. - -### Tip - -```md -TIP: **Tip:** -This is a tip. -``` - -How it renders in docs.gitlab.com: - -TIP: **Tip:** -This is a tip. - -### Caution - -```md -CAUTION: **Caution:** -This is something to be cautious about. -``` - -How it renders in docs.gitlab.com: - -CAUTION: **Caution:** -This is something to be cautious about. - -### Danger - -```md -DANGER: **Danger:** -This is a breaking change, a bug, or something very important to note. -``` - -How it renders in docs.gitlab.com: - -DANGER: **Danger:** -This is a breaking change, a bug, or something very important to note. - -## Blockquotes - -For highlighting a text within a blue blockquote, use this format: - -```md -> This is a blockquote. -``` - -which renders in docs.gitlab.com to: - -> This is a blockquote. - -If the text spans across multiple lines it's OK to split the line. - -## Specific sections and terms - -To mention and/or reference specific terms in GitLab, please follow the styles -below. - -### GitLab versions and tiers - -- Every piece of documentation that comes with a new feature should declare the - GitLab version that feature got introduced. Right below the heading add a - note: - - ```md - > Introduced in GitLab 8.3. - ``` - -- Whenever possible, every feature should have a link to the MR, issue, or epic that introduced it. - The above note would be then transformed to: - - ```md - > [Introduced][ce-1242] in GitLab 8.3. - ``` - - , where the [link identifier](#links) is named after the repository (CE) and - the MR number. - -- If the feature is only available in GitLab Enterprise Edition, don't forget to mention - the [paid tier](https://about.gitlab.com/handbook/marketing/product-marketing/#tiers) - the feature is available in: - - ```md - > [Introduced][ee-1234] in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. - ``` - -### Product badges - -When a feature is available in EE-only tiers, add the corresponding tier according to the -feature availability: - -- For GitLab Starter and GitLab.com Bronze: `**[STARTER]**` -- For GitLab Premium and GitLab.com Silver: `**[PREMIUM]**` -- For GitLab Ultimate and GitLab.com Gold: `**[ULTIMATE]**` -- For GitLab Core and GitLab.com Free: `**[CORE]**` - -To exclude GitLab.com tiers (when the feature is not available in GitLab.com), add the -keyword "only": - -- For GitLab Starter: `**[STARTER ONLY]**` -- For GitLab Premium: `**[PREMIUM ONLY]**` -- For GitLab Ultimate: `**[ULTIMATE ONLY]**` -- For GitLab Core: `**[CORE ONLY]**` - -The tier should be ideally added to headers, so that the full badge will be displayed. -But it can be also mentioned from paragraphs, list items, and table cells. For these cases, -the tier mention will be represented by an orange question mark. -E.g., `**[STARTER]**` renders **[STARTER]**, `**[STARTER ONLY]**` renders **[STARTER ONLY]**. - -The absence of tiers' mentions mean that the feature is available in GitLab Core, -GitLab.com Free, and higher tiers. - -#### How it works - -Introduced by [!244](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/244), -the special markup `**[STARTER]**` will generate a `span` element to trigger the -badges and tooltips (`<span class="badge-trigger starter">`). When the keyword -"only" is added, the corresponding GitLab.com badge will not be displayed. - -### GitLab Restart - -There are many cases that a restart/reconfigure of GitLab is required. To -avoid duplication, link to the special document that can be found in -[`doc/administration/restart_gitlab.md`][doc-restart]. Usually the text will -read like: - - ``` - Save the file and [reconfigure GitLab](../administration/restart_gitlab.md) - for the changes to take effect. - ``` - -If the document you are editing resides in a place other than the GitLab CE/EE -`doc/` directory, instead of the relative link, use the full path: -`http://docs.gitlab.com/ce/administration/restart_gitlab.html`. -Replace `reconfigure` with `restart` where appropriate. - -### Installation guide - -**Ruby:** -In [step 2 of the installation guide](../install/installation.md#2-ruby), -we install Ruby from source. Whenever there is a new version that needs to -be updated, remember to change it throughout the codeblock and also replace -the sha256sum (it can be found in the [downloads page][ruby-dl] of the Ruby -website). - -[ruby-dl]: https://www.ruby-lang.org/en/downloads/ "Ruby download website" - -### Configuration documentation for source and Omnibus installations - -GitLab currently officially supports two installation methods: installations -from source and Omnibus packages installations. - -Whenever there is a setting that is configurable for both installation methods, -prefer to document it in the CE docs to avoid duplication. - -Configuration settings include: - -- settings that touch configuration files in `config/` -- NGINX settings and settings in `lib/support/` in general - -When there is a list of steps to perform, usually that entails editing the -configuration file and reconfiguring/restarting GitLab. In such case, follow -the style below as a guide: - -```md -**For Omnibus installations** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - external_url "https://gitlab.example.com" - ``` - -1. Save the file and [reconfigure] GitLab for the changes to take effect. - ---- - -**For installations from source** - -1. Edit `config/gitlab.yml`: - - ```yaml - gitlab: - host: "gitlab.example.com" - ``` - -1. Save the file and [restart] GitLab for the changes to take effect. - - -[reconfigure]: path/to/administration/restart_gitlab.md#omnibus-gitlab-reconfigure -[restart]: path/to/administration/restart_gitlab.md#installations-from-source -``` - -In this case: - -- before each step list the installation method is declared in bold -- three dashes (`---`) are used to create a horizontal line and separate the - two methods -- the code blocks are indented one or more spaces under the list item to render - correctly -- different highlighting languages are used for each config in the code block -- the [references](#references) guide is used for reconfigure/restart - -### Fake tokens - -There may be times where a token is needed to demonstrate an API call using -cURL or a secret variable used in CI. It is strongly advised not to use real -tokens in documentation even if the probability of a token being exploited is -low. - -You can use the following fake tokens as examples. - -| **Token type** | **Token value** | -| --------------------- | --------------------------------- | -| Private user token | `9koXpg98eAheJpvBs5tK` | -| Personal access token | `n671WNGecHugsdEDPsyo` | -| Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` | -| Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` | -| Secret CI variable | `Li8j-mLUVA3eZYjPfd_H` | -| Specific Runner token | `yrnZW46BrtBFqM7xDzE7dddd` | -| Shared Runner token | `6Vk7ZsosqQyfreAxXTZr` | -| Trigger token | `be20d8dcc028677c931e04f3871a9b` | -| Webhook secret token | `6XhDroRcYPM5by_h-HLY` | -| Health check token | `Tu7BgjR9qeZTEyRzGG2P` | -| Request profile token | `7VgpS4Ax5utVD2esNstz` | - -### API - -Here is a list of must-have items. Use them in the exact order that appears -on this document. Further explanation is given below. - -- Every method must have the REST API request. For example: - - ``` - GET /projects/:id/repository/branches - ``` - -- Every method must have a detailed - [description of the parameters](#method-description). -- Every method must have a cURL example. -- Every method must have a response body (in JSON format). - -#### Method description - -Use the following table headers to describe the methods. Attributes should -always be in code blocks using backticks (``` ` ```). - -``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -``` - -Rendered example: - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `user` | string | yes | The GitLab username | - -#### cURL commands - -- Use `https://gitlab.example.com/api/v4/` as an endpoint. -- Wherever needed use this personal access token: `9koXpg98eAheJpvBs5tK`. -- Always put the request first. `GET` is the default so you don't have to - include it. -- Use double quotes to the URL when it includes additional parameters. -- Prefer to use examples using the personal access token and don't pass data of - username and password. - -| Methods | Description | -| ------- | ----------- | -| `-H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK"` | Use this method as is, whenever authentication needed | -| `-X POST` | Use this method when creating new objects | -| `-X PUT` | Use this method when updating existing objects | -| `-X DELETE` | Use this method when removing existing objects | - -#### cURL Examples - -Below is a set of [cURL][] examples that you can use in the API documentation. - -##### Simple cURL command - -Get the details of a group: - -```bash -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/gitlab-org -``` - -##### cURL example with parameters passed in the URL - -Create a new project under the authenticated user's namespace: - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects?name=foo" -``` - -##### Post data using cURL's --data - -Instead of using `-X POST` and appending the parameters to the URI, you can use -cURL's `--data` option. The example below will create a new project `foo` under -the authenticated user's namespace. - -```bash -curl --data "name=foo" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects" -``` - -##### Post data using JSON content - -> **Note:** In this example we create a new group. Watch carefully the single -and double quotes. - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v4/groups -``` - -##### Post data using form-data - -Instead of using JSON or urlencode you can use multipart/form-data which -properly handles data encoding: - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "title=ssh-key" --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." https://gitlab.example.com/api/v4/users/25/keys -``` - -The above example is run by and administrator and will add an SSH public key -titled ssh-key to user's account which has an id of 25. - -##### Escape special characters - -Spaces or slashes (`/`) may sometimes result to errors, thus it is recommended -to escape them when possible. In the example below we create a new issue which -contains spaces in its title. Observe how spaces are escaped using the `%20` -ASCII code. - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20Dude" -``` - -Use `%2F` for slashes (`/`). - -##### Pass arrays to API calls - -The GitLab API sometimes accepts arrays of strings or integers. For example, to -restrict the sign-up e-mail domains of a GitLab instance to `*.example.com` and -`example.net`, you would do something like this: - -```bash -curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "domain_whitelist[]=*.example.com" --data "domain_whitelist[]=example.net" https://gitlab.example.com/api/v4/application/settings -``` - -[cURL]: http://curl.haxx.se/ "cURL website" -[single spaces]: http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html -[gfm]: http://docs.gitlab.com/ce/user/markdown.html#newlines "GitLab flavored markdown documentation" -[ce-1242]: https://gitlab.com/gitlab-org/gitlab-ce/issues/1242 -[doc-restart]: ../administration/restart_gitlab.md "GitLab restart documentation" diff --git a/doc/development/img/manual_build_docs.png b/doc/development/documentation/img/manual_build_docs.png Binary files differindex 615facabb5f..615facabb5f 100644 --- a/doc/development/img/manual_build_docs.png +++ b/doc/development/documentation/img/manual_build_docs.png diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md new file mode 100644 index 00000000000..48e1685082a --- /dev/null +++ b/doc/development/documentation/index.md @@ -0,0 +1,558 @@ +--- +description: Learn how to contribute to GitLab Documentation. +--- + +# GitLab Documentation guidelines + + - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. + - **[Technical Articles](#technical-articles)**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). + - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). + +## Contributing to docs + +Whenever a feature is changed, updated, introduced, or deprecated, the merge +request introducing these changes must be accompanied by the documentation +(either updating existing ones or creating new ones). This is also valid when +changes are introduced to the UI. + +The one responsible for writing the first piece of documentation is the developer who +wrote the code. It's the job of the Product Manager to ensure all features are +shipped with its docs, whether is a small or big change. At the pace GitLab evolves, +this is the only way to keep the docs up-to-date. If you have any questions about it, +ask a Technical Writer. Otherwise, when your content is ready, assign one of +them to review it for you. + +We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything +is documented. + +Whenever you submit a merge request for the documentation, use the documentation MR description template. + +Please check the [documentation workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) before getting started. + +## Documentation structure + +- Overview and use cases: what it is, why it is necessary, why one would use it +- Requirements: what do we need to get started +- Tutorial: how to set it up, how to use it + +Always link a new document from its topic-related index, otherwise, it will +not be included it in the documentation site search. + +_Note: to be extended._ + +### Feature overview and use cases + +Every major feature (regardless if present in GitLab Community or Enterprise editions) +should present, at the beginning of the document, two main sections: **overview** and +**use cases**. Every GitLab EE-only feature should also contain these sections. + +**Overview**: as the name suggests, the goal here is to provide an overview of the feature. +Describe what is it, what it does, why it is important/cool/nice-to-have, +what problem it solves, and what you can do with this feature that you couldn't +do before. + +**Use cases**: provide at least two, ideally three, use cases for every major feature. +You should answer this question: what can you do with this feature/change? Use cases +are examples of how this feature or change can be used in real life. + +Examples: +- CE and EE: [Issues](../user/project/issues/index.md#use-cases) +- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) +- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) +- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) + +Note that if you don't have anything to add between the doc title (`<h1>`) and +the header `## Overview`, you can omit the header, but keep the content of the +overview there. + +> **Overview** and **use cases** are required to **every** Enterprise Edition feature, +and for every **major** feature present in Community Edition. + +## Markdown and styles + +Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. + +All the docs follow the [documentation style guidelines](styleguide.md). + +## Documentation directory structure + +The documentation is structured based on the GitLab UI structure itself, +separated by [`user`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/user), +[`administrator`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/administration), and [`contributor`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/development). + +In order to have a [solid site structure](https://searchengineland.com/seo-benefits-developing-solid-site-structure-277456) for our documentation, +all docs should be linked. Every new document should be cross-linked to its related documentation, and linked from its topic-related index, when existent. + +The directories `/workflow/`, `/gitlab-basics/`, `/university/`, and `/articles/` have +been deprecated and the majority their docs have been moved to their correct location +in small iterations. Please don't create new docs in these folders. + +### Location and naming documents + +The documentation hierarchy can be vastly improved by providing a better layout +and organization of directories. + +Having a structured document layout, we will be able to have meaningful URLs +like `docs.gitlab.com/user/project/merge_requests/index.html`. With this pattern, +you can immediately tell that you are navigating a user related documentation +and is about the project and its merge requests. + +Do not create summaries of similar types of content (e.g. an index of all articles, videos, etc.), +rather organize content by its subject (e.g. everything related to CI goes together) +and cross-link between any related content. + +The table below shows what kind of documentation goes where. + +| Directory | What belongs here | +| --------- | -------------- | +| `doc/user/` | User related documentation. Anything that can be done within the GitLab UI goes here including `/admin`. | +| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. The admin settings that can be accessed via GitLab's interface go under `doc/user/admin_area/`. | +| `doc/api/` | API related documentation. | +| `doc/development/` | Documentation related to the development of GitLab. Any styleguides should go here. | +| `doc/legal/` | Legal documents about contributing to GitLab. | +| `doc/install/`| Probably the most visited directory, since `installation.md` is there. Ideally this should go under `doc/administration/`, but it's best to leave it as-is in order to avoid confusion (still debated though). | +| `doc/update/` | Same with `doc/install/`. Should be under `administration/`, but this is a well known location, better leave as-is, at least for now. | +| `doc/topics/` | Indexes per Topic (`doc/topics/topic-name/index.md`): all resources for that topic (user and admin documentation, articles, and third-party docs) | + +--- + +**General rules:** + +1. The correct naming and location of a new document, is a combination + of the relative URL of the document in question and the GitLab Map design + that is used for UX purposes ([source][graffle], [image][gitlab-map]). +1. When creating a new document and it has more than one word in its name, + make sure to use underscores instead of spaces or dashes (`-`). For example, + a proper naming would be `import_projects_from_github.md`. The same rule + applies to images. +1. Start a new directory with an `index.md` file. +1. There are four main directories, `user`, `administration`, `api` and `development`. +1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, + `profile/`, `dashboard/` and `admin_area/`. + 1. `doc/user/project/` should contain all project related documentation. + 1. `doc/user/group/` should contain all group related documentation. + 1. `doc/user/profile/` should contain all profile related documentation. + Every page you would navigate under `/profile` should have its own document, + i.e. `account.md`, `applications.md`, `emails.md`, etc. + 1. `doc/user/dashboard/` should contain all dashboard related documentation. + 1. `doc/user/admin_area/` should contain all admin related documentation + describing what can be achieved by accessing GitLab's admin interface + (_not to be confused with `doc/administration` where server access is + required_). + 1. Every category under `/admin/application_settings` should have its + own document located at `doc/user/admin_area/settings/`. For example, + the **Visibility and Access Controls** category should have a document + located at `doc/user/admin_area/settings/visibility_and_access_controls.md`. +1. The `doc/topics/` directory holds topic-related technical content. Create + `doc/topics/topic-name/subtopic-name/index.md` when subtopics become necessary. + General user- and admin- related documentation, should be placed accordingly. + +If you are unsure where a document should live, you can ping `@axil` or `@marcia` in your +merge request. + +### Changing document location + +Changing a document's location is not to be taken lightly. Remember that the +documentation is available to all installations under `help/` and not only to +GitLab.com or http://docs.gitlab.com. Make sure this is discussed with the +Documentation team beforehand. + +If you indeed need to change a document's location, do NOT remove the old +document, but rather replace all of its contents with a new line: + +``` +This document was moved to [another location](path/to/new_doc.md). +``` + +where `path/to/new_doc.md` is the relative path to the root directory `doc/`. + +--- + +For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to +`doc/administration/lfs.md`, then the steps would be: + +1. Copy `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md` +1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with: + + ``` + This document was moved to [another location](../../administration/lfs.md). + ``` + +1. Find and replace any occurrences of the old location with the new one. + A quick way to find them is to use `git grep`. First go to the root directory + where you cloned the `gitlab-ce` repository and then do: + + ``` + git grep -n "workflow/lfs/lfs_administration" + git grep -n "lfs/lfs_administration" + ``` + +NOTE: **Note:** +If the document being moved has any Disqus comments on it, there are extra steps +to follow documented just [below](#redirections-for-pages-with-disqus-comments). + +Things to note: + +- Since we also use inline documentation, except for the documentation itself, + the document might also be referenced in the views of GitLab (`app/`) which will + render when visiting `/help`, and sometimes in the testing suite (`spec/`). +- The above `git grep` command will search recursively in the directory you run + it in for `workflow/lfs/lfs_administration` and `lfs/lfs_administration` + and will print the file and the line where this file is mentioned. + You may ask why the two greps. Since we use relative paths to link to + documentation, sometimes it might be useful to search a path deeper. +- The `*.md` extension is not used when a document is linked to GitLab's + built-in help page, that's why we omit it in `git grep`. +- Use the checklist on the documentation MR description template. + +#### Alternative redirection method + +Alternatively to the method described above, you can simply replace the content +of the old file with a frontmatter containing a redirect link: + +```yaml +--- +redirect_to: '../path/to/file/README.md' +--- +``` + +It supports both full and relative URLs, e.g. `https://docs.gitlab.com/ee/path/to/file.html`, `../path/to/file.html`, `path/to/file.md`. Note that any `*.md` paths will be compiled to `*.html`. + +### Redirections for pages with Disqus comments + +If the documentation page being relocated already has any Disqus comments, +we need to preserve the Disqus thread. + +Disqus uses an identifier per page, and for docs.gitlab.com, the page identifier +is configured to be the page URL. Therefore, when we change the document location, +we need to preserve the old URL as the same Disqus identifier. + +To do that, add to the frontmatter the variable `redirect_from`, +using the old URL as value. For example, let's say I moved the document +available under `https://docs.gitlab.com/my-old-location/README.html` to a new location, +`https://docs.gitlab.com/my-new-location/index.html`. + +Into the **new document** frontmatter add the following: + +```yaml +--- +redirect_from: 'https://docs.gitlab.com/my-old-location/README.html' +--- +``` + +Note: it is necessary to include the file name in the `redirect_from` URL, +even if it's `index.html` or `README.html`. + +## Testing + +We treat documentation as code, thus have implemented some testing. +Currently, the following tests are in place: + +1. `docs lint`: Check that all internal (relative) links work correctly and + that all cURL examples in API docs use the full switches. It's recommended + to [check locally](#previewing-locally) before pushing to GitLab by executing the command + `bundle exec nanoc check internal_links` on your local + [`gitlab-docs`](https://gitlab.com/gitlab-com/gitlab-docs) directory. +1. [`ee_compat_check`](../automatic_ce_ee_merge.md#avoiding-ce-gt-ee-merge-conflicts-beforehand) (runs on CE only): + When you submit a merge request to GitLab Community Edition (CE), + there is this additional job that runs against Enterprise Edition (EE) + and checks if your changes can apply cleanly to the EE codebase. + If that job fails, read the instructions in the job log for what to do next. + As CE is merged into EE once a day, it's important to avoid merge conflicts. + Submitting an EE-equivalent merge request cherry-picking all commits from CE to EE is + essential to avoid them. + +## Branch naming + +If your contribution contains **only** documentation changes, you can speed up +the CI process by following some branch naming conventions. You have three +choices: + +| Branch name | Valid example | +| ----------- | ------------- | +| Starting with `docs/` | `docs/update-api-issues` | +| Starting with `docs-` | `docs-update-api-issues` | +| Ending in `-docs` | `123-update-api-issues-docs` | + +If your branch name matches any of the above, it will run only the docs +tests. If it doesn't, the whole test suite will run (including docs). + +## Merge requests for GitLab documentation + +Before getting started, make sure you read the introductory section +"[contributing to docs](#contributing-to-docs)" above and the +[tech writing workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) +for GitLab Team members. + +- Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/merge_request_templates/Documentation.md) +- Use the correct [branch name](#branch-naming) +- Label the MR `Documentation` +- Assign the correct milestone (see note below) + + +NOTE: **Note:** +If the release version you want to add the documentation to has already been +frozen or released, use the label `Pick into X.Y` to get it merged into +the correct release. Avoid picking into a past release as much as you can, as +it increases the work of the release managers. + +### Cherry-picking from CE to EE + +As we have the `master` branch of CE merged into EE once a day, it's common to +run into merge conflicts. To avoid them, we [test for merge conflicts against EE](#testing) +with the `ee-compat-check` job, and use the following method of creating equivalent +branches for CE and EE. + +Follow this [method for cherry-picking from CE to EE](../automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee), with a few adjustments: + +- Create the [CE branch](#branch-naming) starting with `docs-`, + e.g.: `git checkout -b docs-example` +- Create the EE-equivalent branch ending with `-ee`, e.g., + `git checkout -b docs-example-ee` +- Once all the jobs are passing in CE and EE, and you've addressed the +feedback from your own team, assign the CE MR to a technical writer for review +- When both MRs are ready, the EE merge request will be merged first, and the +CE-equivalent will be merged next. +- Note that the review will occur only in the CE MR, as the EE MR +contains the same commits as the CE MR. +- If you have a few more changes that apply to the EE-version only, you can submit +a couple more commits to the EE branch, but ask the reviewer to review the EE merge request +additionally to the CE MR. If there are many EE-only changes though, start a new MR +to EE only. + +## Previewing the changes live + +To preview your changes to documentation locally, please follow +this [development guide](https://gitlab.com/gitlab-com/gitlab-docs/blob/master/README.md#development). + +If you want to preview the doc changes of your merge request live, you can use +the manual `review-docs-deploy` job in your merge request. You will need at +least Maintainer permissions to be able to run it and is currently enabled for the +following projects: + +- https://gitlab.com/gitlab-org/gitlab-ce +- https://gitlab.com/gitlab-org/gitlab-ee + +NOTE: **Note:** +You will need to push a branch to those repositories, it doesn't work for forks. + +TIP: **Tip:** +If your branch contains only documentation changes, you can use +[special branch names](#branch-naming) to avoid long running pipelines. + +In the mini pipeline graph, you should see an `>>` icon. Clicking on it will +reveal the `review-docs-deploy` job. Hit the play button for the job to start. + + + +This job will: + +1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) + project named after the scheme: `preview-<branch-slug>` +1. Trigger a cross project pipeline and build the docs site with your changes + +After a few minutes, the Review App will be deployed and you will be able to +preview the changes. The docs URL can be found in two places: + +- In the merge request widget +- In the output of the `review-docs-deploy` job, which also includes the + triggered pipeline so that you can investigate whether something went wrong + +In case the Review App URL returns 404, follow these steps to debug: + +1. **Did you follow the URL from the merge request widget?** If yes, then check if + the link is the same as the one in the job output. It can happen that if the + branch name slug is longer than 35 characters, it is automatically + truncated. That means that the merge request widget will not show the proper + URL due to a limitation of how `environment: url` works, but you can find the + real URL from the output of the `review-docs-deploy` job. +1. **Did you follow the URL from the job output?** If yes, then it means that + either the site is not yet deployed or something went wrong with the remote + pipeline. Give it a few minutes and it should appear online, otherwise you + can check the status of the remote pipeline from the link in the job output. + If the pipeline failed or got stuck, drop a line in the `#docs` chat channel. + +TIP: **Tip:** +Someone that has no merge rights to the CE/EE projects (think of forks from +contributors) will not be able to run the manual job. In that case, you can +ask someone from the GitLab team who has the permissions to do that for you. + +NOTE: **Note:** +Make sure that you always delete the branch of the merge request you were +working on. If you don't, the remote docs branch won't be removed either, +and the server where the Review Apps are hosted will eventually be out of +disk space. + +### Technical aspects + +If you want to know the hot details, here's what's really happening: + +1. You manually run the `review-docs-deploy` job in a CE/EE merge request. +1. The job runs the [`scripts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) + script with the `deploy` flag, which in turn: + 1. Takes your branch name and applies the following: + - The slug of the branch name is used to avoid special characters since + ultimately this will be used by NGINX. + - The `preview-` prefix is added to avoid conflicts if there's a remote branch + with the same name that you created in the merge request. + - The final branch name is truncated to 42 characters to avoid filesystem + limitations with long branch names (> 63 chars). + 1. The remote branch is then created if it doesn't exist (meaning you can + re-run the manual job as many times as you want and this step will be skipped). + 1. A new cross-project pipeline is triggered in the docs project. + 1. The preview URL is shown both at the job output and in the merge request + widget. You also get the link to the remote pipeline. +1. In the docs project, the pipeline is created and it + [skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55) + to lower the build time. +1. Once the docs site is built, the HTML files are uploaded as artifacts. +1. A specific Runner tied only to the docs project, runs the Review App job + that downloads the artifacts and uses `rsync` to transfer the files over + to a location where NGINX serves them. + +The following GitLab features are used among others: + +- [Manual actions](../../ci/yaml/README.md#manual-actions) +- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Review Apps](../../ci/review_apps/index.md) +- [Artifacts](../../ci/yaml/README.md#artifacts) +- [Specific Runner](../../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) + +## GitLab `/help` + +Every GitLab instance includes the documentation, which is available from `/help` +(`http://my-instance.com/help`), e.g., <https://gitlab.com/help>. + +When you're building a new feature, you may need to link the documentation +from GitLab, the application. This is normally done in files inside the +`app/views/` directory with the help of the `help_page_path` helper method. + +In its simplest form, the HAML code to generate a link to the `/help` page is: + +```haml += link_to 'Help page', help_page_path('user/permissions') +``` + +The `help_page_path` contains the path to the document you want to link to with +the following conventions: + +- it is relative to the `doc/` directory in the GitLab repository +- the `.md` extension must be omitted +- it must not end with a slash (`/`) + +Below are some special cases where should be used depending on the context. +You can combine one or more of the following: + +1. **Linking to an anchor link.** Use `anchor` as part of the `help_page_path` + method: + + ```haml + = link_to 'Help page', help_page_path('user/permissions', anchor: 'anchor-link') + ``` + +1. **Opening links in a new tab.** This should be the default behavior: + + ```haml + = link_to 'Help page', help_page_path('user/permissions'), target: '_blank' + ``` + +1. **Linking to a circle icon.** Usually used in settings where a long + description cannot be used, like near checkboxes. You can basically use + any font awesome icon, but prefer the `question-circle`: + + ```haml + = link_to icon('question-circle'), help_page_path('user/permissions') + ``` + +1. **Using a button link.** Useful in places where text would be out of context + with the rest of the page layout: + + ```haml + = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' + ``` + +1. **Using links inline of some text.** + + ```haml + Description to #{link_to 'Help page', help_page_path('user/permissions')}. + ``` + +1. **Adding a period at the end of the sentence.** Useful when you don't want + the period to be part of the link: + + ```haml + = succeed '.' do + Learn more in the + = link_to 'Help page', help_page_path('user/permissions') + ``` + +## General Documentation vs Technical Articles + +### General documentation + +General documentation is categorized by _User_, _Admin_, and _Contributor_, and describe what that feature is, what it does, and its available settings. + +### Technical Articles + +Technical articles replace technical content that once lived in the [GitLab Blog](https://about.gitlab.com/blog/), where they got out-of-date and weren't easily found. + +They are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. + +A technical article guides users and/or admins to achieve certain objectives (within guides and tutorials), or provide an overview of that particular topic or feature (within technical overviews). It can also describe the use, implementation, or integration of third-party tools with GitLab. + +They should be placed in a new directory named `/article-title/index.md` under a topic-related folder, and their images should be placed in `/article-title/img/`. For example, a new article on GitLab Pages should be placed in `doc/user/project/pages/article-title/` and a new article on GitLab CI/CD should be placed in `doc/ci/examples/article-title/`. + +#### Types of Technical Articles + +- **User guides**: technical content to guide regular users from point A to point B +- **Admin guides**: technical content to guide administrators of GitLab instances from point A to point B +- **Technical Overviews**: technical content describing features, solutions, and third-party integrations +- **Tutorials**: technical content provided step-by-step on how to do things, or how to reach very specific objectives + +#### Understanding guides, tutorials, and technical overviews + +Suppose there's a process to go from point A to point B in 5 steps: `(A) 1 > 2 > 3 > 4 > 5 (B)`. + +A **guide** can be understood as a description of certain processes to achieve a particular objective. A guide brings you from A to B describing the characteristics of that process, but not necessarily going over each step. It can mention, for example, steps 2 and 3, but does not necessarily explain how to accomplish them. + +- Live example: "[Static sites and GitLab Pages domains (Part 1)](../user/project/pages/getting_started_part_one.md) to [Creating and Tweaking GitLab CI/CD for GitLab Pages (Part 4)](../../user/project/pages/getting_started_part_four.md)" + +A **tutorial** requires a clear **step-by-step** guidance to achieve a singular objective. It brings you from A to B, describing precisely all the necessary steps involved in that process, showing each of the 5 steps to go from A to B. +It does not only describes steps 2 and 3, but also shows you how to accomplish them. + +- Live example (on the blog): [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) + +A **technical overview** is a description of what a certain feature is, and what it does, but does not walk +through the process of how to use it systematically. + +- Live example (on the blog): [GitLab Workflow, an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) + +#### Special format + +Every **Technical Article** contains a frontmatter at the beginning of the doc +with the following information: + +- **Type of article** (user guide, admin guide, technical overview, tutorial) +- **Knowledge level** expected from the reader to be able to follow through (beginner, intermediate, advanced) +- **Author's name** and **GitLab.com handle** +- **Publication date** (ISO format YYYY-MM-DD) + +For example: + + +```yaml +--- +author: John Doe +author_gitlab: johnDoe +level: beginner +article_type: user guide +date: 2017-02-01 +--- +``` + +#### Technical Articles - Writing Method + +Use the [writing method](https://about.gitlab.com/handbook/product/technical-writing/#writing-method) defined by the Technical Writing team. + +[gitlab-map]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png +[graffle]: https://gitlab.com/gitlab-org/gitlab-design/blob/d8d39f4a87b90fb9ae89ca12dc565347b4900d5e/production/resources/gitlab-map.graffle diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md new file mode 100644 index 00000000000..e7ffba635c9 --- /dev/null +++ b/doc/development/documentation/styleguide.md @@ -0,0 +1,499 @@ +--- +description: 'Writing styles, markup, formatting, and reusing regular expressions throughout the GitLab Documentation.' +--- + +# Documentation style guidelines + +The documentation style guide defines the markup structure used in +GitLab documentation. Check the +[documentation guidelines](index.md) for general development instructions. + +Check the GitLab handbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). + +## Text + +- Split up long lines (wrap text), this makes it much easier to review and edit. Only + double line breaks are shown as a full line break in [GitLab markdown][gfm]. + 80-100 characters is a good line length +- Make sure that the documentation is added in the correct + [directory](index.md#documentation-directory-structure) and that + there's a link to it somewhere useful +- Do not duplicate information +- Be brief and clear +- Unless there's a logical reason not to, add documents in alphabetical order +- Write in US English +- Use [single spaces][] instead of double spaces +- Jump a line between different markups (e.g., after every paragraph, header, list, etc) +- Capitalize "G" and "L" in GitLab +- Use sentence case for titles, headings, labels, menu items, and buttons. +- Use title case when referring to [features](https://about.gitlab.com/features/) or [products](https://about.gitlab.com/pricing/), and methods. Note that some features are also objects (e.g. "Merge Requests" and "merge requests"). E.g.: GitLab Runner, Geo, Issue Boards, Git, Prometheus, Continuous Integration. + +## Formatting + +- Use double asterisks (`**`) to mark a word or text in bold (`**bold**`) +- Use undescore (`_`) for text in italics (`_italic_`) +- Jump a line between different markups, for example: + + ```md + ## Header + + Paragraph. + + - List item + - List item + ``` + +### Punctuation + +For punctuation rules, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/). + +### Ordered and unordered lists + +- Use dashes (`-`) for unordered lists instead of asterisks (`*`) +- Use the number one (`1`) for ordered lists +- For punctuation in bullet lists, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/) + +## Headings + +- Add **only one H1** in each document, by adding `#` at the beginning of + it (when using markdown). The `h1` will be the document `<title>`. +- For subheadings, use `##`, `###` and so on +- Avoid putting numbers in headings. Numbers shift, hence documentation anchor + links shift too, which eventually leads to dead links. If you think it is + compelling to add numbers in headings, make sure to at least discuss it with + someone in the Merge Request +- [Avoid using symbols and special chars](https://gitlab.com/gitlab-com/gitlab-docs/issues/84) + in headers. Whenever possible, they should be plain and short text. +- Avoid adding things that show ephemeral statuses. For example, if a feature is + considered beta or experimental, put this info in a note, not in the heading. +- When introducing a new document, be careful for the headings to be + grammatically and syntactically correct. Mention one or all + of the following GitLab members for a review: `@axil` or `@marcia`. + This is to ensure that no document with wrong heading is going + live without an audit, thus preventing dead links and redirection issues when + corrected +- Leave exactly one newline after a heading + +## Links + +- Use the regular inline link markdown markup `[Text](https://example.com)`. + It's easier to read, review, and maintain. +- If there's a link that repeats several times through the same document, + you can use `[Text][identifier]` and at the bottom of the section or the + document add: `[identifier]: https://example.com`, in which case, we do + encourage you to also add an alternative text: `[identifier]: https://example.com "Alternative text"` that appears when hovering your mouse on a link. +- To link to internal documentation, use relative links, not full URLs. Use `../` to + navigate tp high-level directories, and always add the file name `file.md` at the + end of the link with the `.md` extension, not `.html`. + Example: instead of `[text](../../merge_requests/)`, use + `[text](../../merge_requests/index.md)` or, `[text](../../ci/README.md)`, or, + for anchor links, `[text](../../ci/README.md#examples)`. + Using the markdown extension is necessary for the [`/help`](index.md#gitlab-help) + section of GitLab. +- To link from CE to EE-only documentation, use the EE-only doc full URL. +- Use [meaningful anchor texts](https://www.futurehosting.com/blog/links-should-have-meaningful-anchor-text-heres-why/). + E.g., instead of writing something like `Read more about GitLab Issue Boards [here](LINK)`, + write `Read more about [GitLab Issue Boards](LINK)`. + +## Images + +- Place images in a separate directory named `img/` in the same directory where + the `.md` document that you're working on is located. Always prepend their + names with the name of the document that they will be included in. For + example, if there is a document called `twitter.md`, then a valid image name + could be `twitter_login_screen.png`. [**Exception**: images for + [articles](index.md#technical-articles) should be + put in a directory called `img` underneath `/articles/article_title/img/`, therefore, + there's no need to prepend the document name to their filenames.] +- Images should have a specific, non-generic name that will differentiate them. +- Keep all file names in lower case. +- Consider using PNG images instead of JPEG. +- Compress all images with <https://tinypng.com/> or similar tool. +- Compress gifs with <https://ezgif.com/optimize> or similar tool. +- Images should be used (only when necessary) to _illustrate_ the description +of a process, not to _replace_ it. + +Inside the document: + +- The Markdown way of using an image inside a document is: + `` +- Always use a proper description for what the image is about. That way, when a + browser fails to show the image, this text will be used as an alternative + description +- If there are consecutive images with little text between them, always add + three dashes (`---`) between the image and the text to create a horizontal + line for better clarity +- If a heading is placed right after an image, always add three dashes (`---`) + between the image and the heading + +## Alert boxes + +Whenever you want to call the attention to a particular sentence, +use the following markup for highlighting. + +_Note that the alert boxes only work for one paragraph only. Multiple paragraphs, +lists, headers, etc will not render correctly._ + +### Note + +```md +NOTE: **Note:** +This is something to note. +``` + +How it renders in docs.gitlab.com: + +NOTE: **Note:** +This is something to note. + +### Tip + +```md +TIP: **Tip:** +This is a tip. +``` + +How it renders in docs.gitlab.com: + +TIP: **Tip:** +This is a tip. + +### Caution + +```md +CAUTION: **Caution:** +This is something to be cautious about. +``` + +How it renders in docs.gitlab.com: + +CAUTION: **Caution:** +This is something to be cautious about. + +### Danger + +```md +DANGER: **Danger:** +This is a breaking change, a bug, or something very important to note. +``` + +How it renders in docs.gitlab.com: + +DANGER: **Danger:** +This is a breaking change, a bug, or something very important to note. + +## Blockquotes + +For highlighting a text within a blue blockquote, use this format: + +```md +> This is a blockquote. +``` + +which renders in docs.gitlab.com to: + +> This is a blockquote. + +If the text spans across multiple lines it's OK to split the line. + +## Specific sections and terms + +To mention and/or reference specific terms in GitLab, please follow the styles +below. + +### GitLab versions and tiers + +- Every piece of documentation that comes with a new feature should declare the + GitLab version that feature got introduced. Right below the heading add a + note: + + ```md + > Introduced in GitLab 8.3. + ``` + +- Whenever possible, every feature should have a link to the MR, issue, or epic that introduced it. + The above note would be then transformed to: + + ```md + > [Introduced][ce-1242] in GitLab 8.3. + ``` + + , where the [link identifier](#links) is named after the repository (CE) and + the MR number. + +- If the feature is only available in GitLab Enterprise Edition, don't forget to mention + the [paid tier](https://about.gitlab.com/handbook/marketing/product-marketing/#tiers) + the feature is available in: + + ```md + > [Introduced][ee-1234] in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. + ``` + +### Product badges + +When a feature is available in EE-only tiers, add the corresponding tier according to the +feature availability: + +- For GitLab Starter and GitLab.com Bronze: `**[STARTER]**` +- For GitLab Premium and GitLab.com Silver: `**[PREMIUM]**` +- For GitLab Ultimate and GitLab.com Gold: `**[ULTIMATE]**` +- For GitLab Core and GitLab.com Free: `**[CORE]**` + +To exclude GitLab.com tiers (when the feature is not available in GitLab.com), add the +keyword "only": + +- For GitLab Starter: `**[STARTER ONLY]**` +- For GitLab Premium: `**[PREMIUM ONLY]**` +- For GitLab Ultimate: `**[ULTIMATE ONLY]**` +- For GitLab Core: `**[CORE ONLY]**` + +The tier should be ideally added to headers, so that the full badge will be displayed. +But it can be also mentioned from paragraphs, list items, and table cells. For these cases, +the tier mention will be represented by an orange question mark. +E.g., `**[STARTER]**` renders **[STARTER]**, `**[STARTER ONLY]**` renders **[STARTER ONLY]**. + +The absence of tiers' mentions mean that the feature is available in GitLab Core, +GitLab.com Free, and higher tiers. + +#### How it works + +Introduced by [!244](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/244), +the special markup `**[STARTER]**` will generate a `span` element to trigger the +badges and tooltips (`<span class="badge-trigger starter">`). When the keyword +"only" is added, the corresponding GitLab.com badge will not be displayed. + +### GitLab Restart + +There are many cases that a restart/reconfigure of GitLab is required. To +avoid duplication, link to the special document that can be found in +[`doc/administration/restart_gitlab.md`][doc-restart]. Usually the text will +read like: + + ``` + Save the file and [reconfigure GitLab](../../administration/restart_gitlab.md) + for the changes to take effect. + ``` + +If the document you are editing resides in a place other than the GitLab CE/EE +`doc/` directory, instead of the relative link, use the full path: +`http://docs.gitlab.com/ce/administration/restart_gitlab.html`. +Replace `reconfigure` with `restart` where appropriate. + +### Installation guide + +**Ruby:** +In [step 2 of the installation guide](../../install/installation.md#2-ruby), +we install Ruby from source. Whenever there is a new version that needs to +be updated, remember to change it throughout the codeblock and also replace +the sha256sum (it can be found in the [downloads page][ruby-dl] of the Ruby +website). + +[ruby-dl]: https://www.ruby-lang.org/en/downloads/ "Ruby download website" + +### Configuration documentation for source and Omnibus installations + +GitLab currently officially supports two installation methods: installations +from source and Omnibus packages installations. + +Whenever there is a setting that is configurable for both installation methods, +prefer to document it in the CE docs to avoid duplication. + +Configuration settings include: + +- settings that touch configuration files in `config/` +- NGINX settings and settings in `lib/support/` in general + +When there is a list of steps to perform, usually that entails editing the +configuration file and reconfiguring/restarting GitLab. In such case, follow +the style below as a guide: + +```md +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + external_url "https://gitlab.example.com" + ``` + +1. Save the file and [reconfigure] GitLab for the changes to take effect. + +--- + +**For installations from source** + +1. Edit `config/gitlab.yml`: + + ```yaml + gitlab: + host: "gitlab.example.com" + ``` + +1. Save the file and [restart] GitLab for the changes to take effect. + + +[reconfigure]: path/to/administration/restart_gitlab.md#omnibus-gitlab-reconfigure +[restart]: path/to/administration/restart_gitlab.md#installations-from-source +``` + +In this case: + +- before each step list the installation method is declared in bold +- three dashes (`---`) are used to create a horizontal line and separate the + two methods +- the code blocks are indented one or more spaces under the list item to render + correctly +- different highlighting languages are used for each config in the code block +- the [references](#references) guide is used for reconfigure/restart + +### Fake tokens + +There may be times where a token is needed to demonstrate an API call using +cURL or a variable used in CI. It is strongly advised not to use real +tokens in documentation even if the probability of a token being exploited is +low. + +You can use the following fake tokens as examples. + +| **Token type** | **Token value** | +| --------------------- | --------------------------------- | +| Private user token | `9koXpg98eAheJpvBs5tK` | +| Personal access token | `n671WNGecHugsdEDPsyo` | +| Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` | +| Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` | +| Secret CI variable | `Li8j-mLUVA3eZYjPfd_H` | +| Specific Runner token | `yrnZW46BrtBFqM7xDzE7dddd` | +| Shared Runner token | `6Vk7ZsosqQyfreAxXTZr` | +| Trigger token | `be20d8dcc028677c931e04f3871a9b` | +| Webhook secret token | `6XhDroRcYPM5by_h-HLY` | +| Health check token | `Tu7BgjR9qeZTEyRzGG2P` | +| Request profile token | `7VgpS4Ax5utVD2esNstz` | + +### API + +Here is a list of must-have items. Use them in the exact order that appears +on this document. Further explanation is given below. + +- Every method must have the REST API request. For example: + + ``` + GET /projects/:id/repository/branches + ``` + +- Every method must have a detailed + [description of the parameters](#method-description). +- Every method must have a cURL example. +- Every method must have a response body (in JSON format). + +#### Method description + +Use the following table headers to describe the methods. Attributes should +always be in code blocks using backticks (``` ` ```). + +``` +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +``` + +Rendered example: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `user` | string | yes | The GitLab username | + +#### cURL commands + +- Use `https://gitlab.example.com/api/v4/` as an endpoint. +- Wherever needed use this personal access token: `9koXpg98eAheJpvBs5tK`. +- Always put the request first. `GET` is the default so you don't have to + include it. +- Use double quotes to the URL when it includes additional parameters. +- Prefer to use examples using the personal access token and don't pass data of + username and password. + +| Methods | Description | +| ------- | ----------- | +| `-H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK"` | Use this method as is, whenever authentication needed | +| `-X POST` | Use this method when creating new objects | +| `-X PUT` | Use this method when updating existing objects | +| `-X DELETE` | Use this method when removing existing objects | + +#### cURL Examples + +Below is a set of [cURL][] examples that you can use in the API documentation. + +##### Simple cURL command + +Get the details of a group: + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/gitlab-org +``` + +##### cURL example with parameters passed in the URL + +Create a new project under the authenticated user's namespace: + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects?name=foo" +``` + +##### Post data using cURL's --data + +Instead of using `-X POST` and appending the parameters to the URI, you can use +cURL's `--data` option. The example below will create a new project `foo` under +the authenticated user's namespace. + +```bash +curl --data "name=foo" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects" +``` + +##### Post data using JSON content + +> **Note:** In this example we create a new group. Watch carefully the single +and double quotes. + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v4/groups +``` + +##### Post data using form-data + +Instead of using JSON or urlencode you can use multipart/form-data which +properly handles data encoding: + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "title=ssh-key" --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." https://gitlab.example.com/api/v4/users/25/keys +``` + +The above example is run by and administrator and will add an SSH public key +titled ssh-key to user's account which has an id of 25. + +##### Escape special characters + +Spaces or slashes (`/`) may sometimes result to errors, thus it is recommended +to escape them when possible. In the example below we create a new issue which +contains spaces in its title. Observe how spaces are escaped using the `%20` +ASCII code. + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20Dude" +``` + +Use `%2F` for slashes (`/`). + +##### Pass arrays to API calls + +The GitLab API sometimes accepts arrays of strings or integers. For example, to +restrict the sign-up e-mail domains of a GitLab instance to `*.example.com` and +`example.net`, you would do something like this: + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "domain_whitelist[]=*.example.com" --data "domain_whitelist[]=example.net" https://gitlab.example.com/api/v4/application/settings +``` + +[cURL]: http://curl.haxx.se/ "cURL website" +[single spaces]: http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html +[gfm]: http://docs.gitlab.com/ce/user/markdown.html#newlines "GitLab flavored markdown documentation" +[ce-1242]: https://gitlab.com/gitlab-org/gitlab-ce/issues/1242 +[doc-restart]: ../../administration/restart_gitlab.md "GitLab restart documentation" diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index 5185d843ccb..9a677bf09b2 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -16,7 +16,7 @@ are very appreciative of the work done by translators and proofreaders! - Dutch - Esperanto - French - - Rémy Coutable - [GitLab](https://gitlab.com/rymai), [Crowdin](https://crowdin.com/profile/rymai) + - Davy Defaud - [GitLab](https://gitlab.com/DevDef), [Crowdin](https://crowdin.com/profile/DevDef) - German - Indonesian - Ahmad Naufal Mukhtar - [GitLab](https://gitlab.com/anaufalm), [Crowdin](https://crowdin.com/profile/anaufalm) diff --git a/doc/development/query_recorder.md b/doc/development/query_recorder.md index 26d3355e94d..61e5e1afede 100644 --- a/doc/development/query_recorder.md +++ b/doc/development/query_recorder.md @@ -22,6 +22,19 @@ As an example you might create 5 issues in between counts, which would cause the > **Note:** In some cases the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible. +## Cached queries + +By default, QueryRecorder will ignore cached queries in the count. However, it may be better to count +all queries to avoid introducing an N+1 query that may be masked by the statement cache. To do this, +pass the `skip_cached` variable to `QueryRecorder` and use the `exceed_all_query_limit` matcher: + +it "avoids N+1 database queries" do + control_count = ActiveRecord::QueryRecorder.new(skip_cached: false) { visit_some_page }.count + create_list(:issue, 5) + expect { visit_some_page }.not_to exceed_all_query_limit(control_count) +end +``` + ## Finding the source of the query It may be useful to identify the source of the queries by looking at the call backtrace. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 31addcaf675..fc51b74da1d 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -176,3 +176,20 @@ git push -u origin update-project-templates ``` Now create a merge request and merge that to master. + +## Generate route lists + +To see the full list of API routes, you can run: + +```shell +bundle exec rake grape:path_helpers +``` + +For the Rails controllers, run: + +```shell +bundle exec rake routes +``` + +Since these take some time to create, it's often helpful to save the output to +a file for quick reference. diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 1c41fc7611f..038a4b1e6ea 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -1,558 +1,3 @@ --- -description: Learn how to contribute to GitLab Documentation. +redirect_to: 'documentation/index.md' --- - -# GitLab Documentation guidelines - - - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. - - **[Technical Articles](#technical-articles)**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). - - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). - -## Contributing to docs - -Whenever a feature is changed, updated, introduced, or deprecated, the merge -request introducing these changes must be accompanied by the documentation -(either updating existing ones or creating new ones). This is also valid when -changes are introduced to the UI. - -The one responsible for writing the first piece of documentation is the developer who -wrote the code. It's the job of the Product Manager to ensure all features are -shipped with its docs, whether is a small or big change. At the pace GitLab evolves, -this is the only way to keep the docs up-to-date. If you have any questions about it, -ask a Technical Writer. Otherwise, when your content is ready, assign one of -them to review it for you. - -We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything -is documented. - -Whenever you submit a merge request for the documentation, use the documentation MR description template. - -Please check the [documentation workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) before getting started. - -## Documentation structure - -- Overview and use cases: what it is, why it is necessary, why one would use it -- Requirements: what do we need to get started -- Tutorial: how to set it up, how to use it - -Always link a new document from its topic-related index, otherwise, it will -not be included it in the documentation site search. - -_Note: to be extended._ - -### Feature overview and use cases - -Every major feature (regardless if present in GitLab Community or Enterprise editions) -should present, at the beginning of the document, two main sections: **overview** and -**use cases**. Every GitLab EE-only feature should also contain these sections. - -**Overview**: as the name suggests, the goal here is to provide an overview of the feature. -Describe what is it, what it does, why it is important/cool/nice-to-have, -what problem it solves, and what you can do with this feature that you couldn't -do before. - -**Use cases**: provide at least two, ideally three, use cases for every major feature. -You should answer this question: what can you do with this feature/change? Use cases -are examples of how this feature or change can be used in real life. - -Examples: -- CE and EE: [Issues](../user/project/issues/index.md#use-cases) -- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) -- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) -- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) - -Note that if you don't have anything to add between the doc title (`<h1>`) and -the header `## Overview`, you can omit the header, but keep the content of the -overview there. - -> **Overview** and **use cases** are required to **every** Enterprise Edition feature, -and for every **major** feature present in Community Edition. - -## Markdown and styles - -Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. - -All the docs follow the [documentation style guidelines](doc_styleguide.md). - -## Documentation directory structure - -The documentation is structured based on the GitLab UI structure itself, -separated by [`user`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/user), -[`administrator`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/administration), and [`contributor`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/development). - -In order to have a [solid site structure](https://searchengineland.com/seo-benefits-developing-solid-site-structure-277456) for our documentation, -all docs should be linked. Every new document should be cross-linked to its related documentation, and linked from its topic-related index, when existent. - -The directories `/workflow/`, `/gitlab-basics/`, `/university/`, and `/articles/` have -been deprecated and the majority their docs have been moved to their correct location -in small iterations. Please don't create new docs in these folders. - -### Location and naming documents - -The documentation hierarchy can be vastly improved by providing a better layout -and organization of directories. - -Having a structured document layout, we will be able to have meaningful URLs -like `docs.gitlab.com/user/project/merge_requests/index.html`. With this pattern, -you can immediately tell that you are navigating a user related documentation -and is about the project and its merge requests. - -Do not create summaries of similar types of content (e.g. an index of all articles, videos, etc.), -rather organize content by its subject (e.g. everything related to CI goes together) -and cross-link between any related content. - -The table below shows what kind of documentation goes where. - -| Directory | What belongs here | -| --------- | -------------- | -| `doc/user/` | User related documentation. Anything that can be done within the GitLab UI goes here including `/admin`. | -| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. The admin settings that can be accessed via GitLab's interface go under `doc/user/admin_area/`. | -| `doc/api/` | API related documentation. | -| `doc/development/` | Documentation related to the development of GitLab. Any styleguides should go here. | -| `doc/legal/` | Legal documents about contributing to GitLab. | -| `doc/install/`| Probably the most visited directory, since `installation.md` is there. Ideally this should go under `doc/administration/`, but it's best to leave it as-is in order to avoid confusion (still debated though). | -| `doc/update/` | Same with `doc/install/`. Should be under `administration/`, but this is a well known location, better leave as-is, at least for now. | -| `doc/topics/` | Indexes per Topic (`doc/topics/topic-name/index.md`): all resources for that topic (user and admin documentation, articles, and third-party docs) | - ---- - -**General rules:** - -1. The correct naming and location of a new document, is a combination - of the relative URL of the document in question and the GitLab Map design - that is used for UX purposes ([source][graffle], [image][gitlab-map]). -1. When creating a new document and it has more than one word in its name, - make sure to use underscores instead of spaces or dashes (`-`). For example, - a proper naming would be `import_projects_from_github.md`. The same rule - applies to images. -1. Start a new directory with an `index.md` file. -1. There are four main directories, `user`, `administration`, `api` and `development`. -1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, - `profile/`, `dashboard/` and `admin_area/`. - 1. `doc/user/project/` should contain all project related documentation. - 1. `doc/user/group/` should contain all group related documentation. - 1. `doc/user/profile/` should contain all profile related documentation. - Every page you would navigate under `/profile` should have its own document, - i.e. `account.md`, `applications.md`, `emails.md`, etc. - 1. `doc/user/dashboard/` should contain all dashboard related documentation. - 1. `doc/user/admin_area/` should contain all admin related documentation - describing what can be achieved by accessing GitLab's admin interface - (_not to be confused with `doc/administration` where server access is - required_). - 1. Every category under `/admin/application_settings` should have its - own document located at `doc/user/admin_area/settings/`. For example, - the **Visibility and Access Controls** category should have a document - located at `doc/user/admin_area/settings/visibility_and_access_controls.md`. -1. The `doc/topics/` directory holds topic-related technical content. Create - `doc/topics/topic-name/subtopic-name/index.md` when subtopics become necessary. - General user- and admin- related documentation, should be placed accordingly. - -If you are unsure where a document should live, you can ping `@axil` or `@marcia` in your -merge request. - -### Changing document location - -Changing a document's location is not to be taken lightly. Remember that the -documentation is available to all installations under `help/` and not only to -GitLab.com or http://docs.gitlab.com. Make sure this is discussed with the -Documentation team beforehand. - -If you indeed need to change a document's location, do NOT remove the old -document, but rather replace all of its contents with a new line: - -``` -This document was moved to [another location](path/to/new_doc.md). -``` - -where `path/to/new_doc.md` is the relative path to the root directory `doc/`. - ---- - -For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to -`doc/administration/lfs.md`, then the steps would be: - -1. Copy `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md` -1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with: - - ``` - This document was moved to [another location](../../administration/lfs.md). - ``` - -1. Find and replace any occurrences of the old location with the new one. - A quick way to find them is to use `git grep`. First go to the root directory - where you cloned the `gitlab-ce` repository and then do: - - ``` - git grep -n "workflow/lfs/lfs_administration" - git grep -n "lfs/lfs_administration" - ``` - -NOTE: **Note:** -If the document being moved has any Disqus comments on it, there are extra steps -to follow documented just [below](#redirections-for-pages-with-disqus-comments). - -Things to note: - -- Since we also use inline documentation, except for the documentation itself, - the document might also be referenced in the views of GitLab (`app/`) which will - render when visiting `/help`, and sometimes in the testing suite (`spec/`). -- The above `git grep` command will search recursively in the directory you run - it in for `workflow/lfs/lfs_administration` and `lfs/lfs_administration` - and will print the file and the line where this file is mentioned. - You may ask why the two greps. Since we use relative paths to link to - documentation, sometimes it might be useful to search a path deeper. -- The `*.md` extension is not used when a document is linked to GitLab's - built-in help page, that's why we omit it in `git grep`. -- Use the checklist on the documentation MR description template. - -#### Alternative redirection method - -Alternatively to the method described above, you can simply replace the content -of the old file with a frontmatter containing a redirect link: - -```yaml ---- -redirect_to: '../path/to/file/README.md' ---- -``` - -It supports both full and relative URLs, e.g. `https://docs.gitlab.com/ee/path/to/file.html`, `../path/to/file.html`, `path/to/file.md`. Note that any `*.md` paths will be compiled to `*.html`. - -### Redirections for pages with Disqus comments - -If the documentation page being relocated already has any Disqus comments, -we need to preserve the Disqus thread. - -Disqus uses an identifier per page, and for docs.gitlab.com, the page identifier -is configured to be the page URL. Therefore, when we change the document location, -we need to preserve the old URL as the same Disqus identifier. - -To do that, add to the frontmatter the variable `redirect_from`, -using the old URL as value. For example, let's say I moved the document -available under `https://docs.gitlab.com/my-old-location/README.html` to a new location, -`https://docs.gitlab.com/my-new-location/index.html`. - -Into the **new document** frontmatter add the following: - -```yaml ---- -redirect_from: 'https://docs.gitlab.com/my-old-location/README.html' ---- -``` - -Note: it is necessary to include the file name in the `redirect_from` URL, -even if it's `index.html` or `README.html`. - -## Testing - -We treat documentation as code, thus have implemented some testing. -Currently, the following tests are in place: - -1. `docs lint`: Check that all internal (relative) links work correctly and - that all cURL examples in API docs use the full switches. It's recommended - to [check locally](#previewing-locally) before pushing to GitLab by executing the command - `bundle exec nanoc check internal_links` on your local - [`gitlab-docs`](https://gitlab.com/gitlab-com/gitlab-docs) directory. -1. [`ee_compat_check`](https://docs.gitlab.com/ee/development/automatic_ce_ee_merge.html#avoiding-ce-gt-ee-merge-conflicts-beforehand) (runs on CE only): - When you submit a merge request to GitLab Community Edition (CE), - there is this additional job that runs against Enterprise Edition (EE) - and checks if your changes can apply cleanly to the EE codebase. - If that job fails, read the instructions in the job log for what to do next. - As CE is merged into EE once a day, it's important to avoid merge conflicts. - Submitting an EE-equivalent merge request cherry-picking all commits from CE to EE is - essential to avoid them. - -## Branch naming - -If your contribution contains **only** documentation changes, you can speed up -the CI process by following some branch naming conventions. You have three -choices: - -| Branch name | Valid example | -| ----------- | ------------- | -| Starting with `docs/` | `docs/update-api-issues` | -| Starting with `docs-` | `docs-update-api-issues` | -| Ending in `-docs` | `123-update-api-issues-docs` | - -If your branch name matches any of the above, it will run only the docs -tests. If it doesn't, the whole test suite will run (including docs). - -## Merge requests for GitLab documentation - -Before getting started, make sure you read the introductory section -"[contributing to docs](#contributing-to-docs)" above and the -[tech writing workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) -for GitLab Team members. - -- Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/merge_request_templates/Documentation.md) -- Use the correct [branch name](#branch-naming) -- Label the MR `Documentation` -- Assign the correct milestone (see note below) - - -NOTE: **Note:** -If the release version you want to add the documentation to has already been -frozen or released, use the label `Pick into X.Y` to get it merged into -the correct release. Avoid picking into a past release as much as you can, as -it increases the work of the release managers. - -### Cherry-picking from CE to EE - -As we have the `master` branch of CE merged into EE once a day, it's common to -run into merge conflicts. To avoid them, we [test for merge conflicts against EE](#testing) -with the `ee-compat-check` job, and use the following method of creating equivalent -branches for CE and EE. - -Follow this [method for cherry-picking from CE to EE](automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee), with a few adjustments: - -- Create the [CE branch](#branch-naming) starting with `docs-`, - e.g.: `git checkout -b docs-example` -- Create the EE-equivalent branch ending with `-ee`, e.g., - `git checkout -b docs-example-ee` -- Once all the jobs are passing in CE and EE, and you've addressed the -feedback from your own team, assign the CE MR to a technical writer for review -- When both MRs are ready, the EE merge request will be merged first, and the -CE-equivalent will be merged next. -- Note that the review will occur only in the CE MR, as the EE MR -contains the same commits as the CE MR. -- If you have a few more changes that apply to the EE-version only, you can submit -a couple more commits to the EE branch, but ask the reviewer to review the EE merge request -additionally to the CE MR. If there are many EE-only changes though, start a new MR -to EE only. - -## Previewing the changes live - -To preview your changes to documentation locally, please follow -this [development guide](https://gitlab.com/gitlab-com/gitlab-docs/blob/master/README.md#development). - -If you want to preview the doc changes of your merge request live, you can use -the manual `review-docs-deploy` job in your merge request. You will need at -least Master permissions to be able to run it and is currently enabled for the -following projects: - -- https://gitlab.com/gitlab-org/gitlab-ce -- https://gitlab.com/gitlab-org/gitlab-ee - -NOTE: **Note:** -You will need to push a branch to those repositories, it doesn't work for forks. - -TIP: **Tip:** -If your branch contains only documentation changes, you can use -[special branch names](#branch-naming) to avoid long running pipelines. - -In the mini pipeline graph, you should see an `>>` icon. Clicking on it will -reveal the `review-docs-deploy` job. Hit the play button for the job to start. - - - -This job will: - -1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) - project named after the scheme: `preview-<branch-slug>` -1. Trigger a cross project pipeline and build the docs site with your changes - -After a few minutes, the Review App will be deployed and you will be able to -preview the changes. The docs URL can be found in two places: - -- In the merge request widget -- In the output of the `review-docs-deploy` job, which also includes the - triggered pipeline so that you can investigate whether something went wrong - -In case the Review App URL returns 404, follow these steps to debug: - -1. **Did you follow the URL from the merge request widget?** If yes, then check if - the link is the same as the one in the job output. It can happen that if the - branch name slug is longer than 35 characters, it is automatically - truncated. That means that the merge request widget will not show the proper - URL due to a limitation of how `environment: url` works, but you can find the - real URL from the output of the `review-docs-deploy` job. -1. **Did you follow the URL from the job output?** If yes, then it means that - either the site is not yet deployed or something went wrong with the remote - pipeline. Give it a few minutes and it should appear online, otherwise you - can check the status of the remote pipeline from the link in the job output. - If the pipeline failed or got stuck, drop a line in the `#docs` chat channel. - -TIP: **Tip:** -Someone that has no merge rights to the CE/EE projects (think of forks from -contributors) will not be able to run the manual job. In that case, you can -ask someone from the GitLab team who has the permissions to do that for you. - -NOTE: **Note:** -Make sure that you always delete the branch of the merge request you were -working on. If you don't, the remote docs branch won't be removed either, -and the server where the Review Apps are hosted will eventually be out of -disk space. - -### Technical aspects - -If you want to know the hot details, here's what's really happening: - -1. You manually run the `review-docs-deploy` job in a CE/EE merge request. -1. The job runs the [`scripts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) - script with the `deploy` flag, which in turn: - 1. Takes your branch name and applies the following: - - The slug of the branch name is used to avoid special characters since - ultimately this will be used by NGINX. - - The `preview-` prefix is added to avoid conflicts if there's a remote branch - with the same name that you created in the merge request. - - The final branch name is truncated to 42 characters to avoid filesystem - limitations with long branch names (> 63 chars). - 1. The remote branch is then created if it doesn't exist (meaning you can - re-run the manual job as many times as you want and this step will be skipped). - 1. A new cross-project pipeline is triggered in the docs project. - 1. The preview URL is shown both at the job output and in the merge request - widget. You also get the link to the remote pipeline. -1. In the docs project, the pipeline is created and it - [skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55) - to lower the build time. -1. Once the docs site is built, the HTML files are uploaded as artifacts. -1. A specific Runner tied only to the docs project, runs the Review App job - that downloads the artifacts and uses `rsync` to transfer the files over - to a location where NGINX serves them. - -The following GitLab features are used among others: - -- [Manual actions](../ci/yaml/README.md#manual-actions) -- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) -- [Review Apps](../ci/review_apps/index.md) -- [Artifacts](../ci/yaml/README.md#artifacts) -- [Specific Runner](../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) - -## GitLab `/help` - -Every GitLab instance includes the documentation, which is available from `/help` -(`http://my-instance.com/help`), e.g., <https://gitlab.com/help>. - -When you're building a new feature, you may need to link the documentation -from GitLab, the application. This is normally done in files inside the -`app/views/` directory with the help of the `help_page_path` helper method. - -In its simplest form, the HAML code to generate a link to the `/help` page is: - -```haml -= link_to 'Help page', help_page_path('user/permissions') -``` - -The `help_page_path` contains the path to the document you want to link to with -the following conventions: - -- it is relative to the `doc/` directory in the GitLab repository -- the `.md` extension must be omitted -- it must not end with a slash (`/`) - -Below are some special cases where should be used depending on the context. -You can combine one or more of the following: - -1. **Linking to an anchor link.** Use `anchor` as part of the `help_page_path` - method: - - ```haml - = link_to 'Help page', help_page_path('user/permissions', anchor: 'anchor-link') - ``` - -1. **Opening links in a new tab.** This should be the default behavior: - - ```haml - = link_to 'Help page', help_page_path('user/permissions'), target: '_blank' - ``` - -1. **Linking to a circle icon.** Usually used in settings where a long - description cannot be used, like near checkboxes. You can basically use - any font awesome icon, but prefer the `question-circle`: - - ```haml - = link_to icon('question-circle'), help_page_path('user/permissions') - ``` - -1. **Using a button link.** Useful in places where text would be out of context - with the rest of the page layout: - - ```haml - = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' - ``` - -1. **Using links inline of some text.** - - ```haml - Description to #{link_to 'Help page', help_page_path('user/permissions')}. - ``` - -1. **Adding a period at the end of the sentence.** Useful when you don't want - the period to be part of the link: - - ```haml - = succeed '.' do - Learn more in the - = link_to 'Help page', help_page_path('user/permissions') - ``` - -## General Documentation vs Technical Articles - -### General documentation - -General documentation is categorized by _User_, _Admin_, and _Contributor_, and describe what that feature is, what it does, and its available settings. - -### Technical Articles - -Technical articles replace technical content that once lived in the [GitLab Blog](https://about.gitlab.com/blog/), where they got out-of-date and weren't easily found. - -They are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. - -A technical article guides users and/or admins to achieve certain objectives (within guides and tutorials), or provide an overview of that particular topic or feature (within technical overviews). It can also describe the use, implementation, or integration of third-party tools with GitLab. - -They should be placed in a new directory named `/article-title/index.md` under a topic-related folder, and their images should be placed in `/article-title/img/`. For example, a new article on GitLab Pages should be placed in `doc/user/project/pages/article-title/` and a new article on GitLab CI/CD should be placed in `doc/ci/examples/article-title/`. - -#### Types of Technical Articles - -- **User guides**: technical content to guide regular users from point A to point B -- **Admin guides**: technical content to guide administrators of GitLab instances from point A to point B -- **Technical Overviews**: technical content describing features, solutions, and third-party integrations -- **Tutorials**: technical content provided step-by-step on how to do things, or how to reach very specific objectives - -#### Understanding guides, tutorials, and technical overviews - -Suppose there's a process to go from point A to point B in 5 steps: `(A) 1 > 2 > 3 > 4 > 5 (B)`. - -A **guide** can be understood as a description of certain processes to achieve a particular objective. A guide brings you from A to B describing the characteristics of that process, but not necessarily going over each step. It can mention, for example, steps 2 and 3, but does not necessarily explain how to accomplish them. - -- Live example: "[Static sites and GitLab Pages domains (Part 1)](../user/project/pages/getting_started_part_one.md) to [Creating and Tweaking GitLab CI/CD for GitLab Pages (Part 4)](../user/project/pages/getting_started_part_four.md)" - -A **tutorial** requires a clear **step-by-step** guidance to achieve a singular objective. It brings you from A to B, describing precisely all the necessary steps involved in that process, showing each of the 5 steps to go from A to B. -It does not only describes steps 2 and 3, but also shows you how to accomplish them. - -- Live example (on the blog): [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) - -A **technical overview** is a description of what a certain feature is, and what it does, but does not walk -through the process of how to use it systematically. - -- Live example (on the blog): [GitLab Workflow, an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) - -#### Special format - -Every **Technical Article** contains a frontmatter at the beginning of the doc -with the following information: - -- **Type of article** (user guide, admin guide, technical overview, tutorial) -- **Knowledge level** expected from the reader to be able to follow through (beginner, intermediate, advanced) -- **Author's name** and **GitLab.com handle** -- **Publication date** (ISO format YYYY-MM-DD) - -For example: - - -```yaml ---- -author: John Doe -author_gitlab: johnDoe -level: beginner -article_type: user guide -date: 2017-02-01 ---- -``` - -#### Technical Articles - Writing Method - -Use the [writing method](https://about.gitlab.com/handbook/product/technical-writing/#writing-method) defined by the Technical Writing team. - -[gitlab-map]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png -[graffle]: https://gitlab.com/gitlab-org/gitlab-design/blob/d8d39f4a87b90fb9ae89ca12dc565347b4900d5e/production/resources/gitlab-map.graffle diff --git a/doc/downgrade_ee_to_ce/README.md b/doc/downgrade_ee_to_ce/README.md index 236408762e3..a187b3cbb07 100644 --- a/doc/downgrade_ee_to_ce/README.md +++ b/doc/downgrade_ee_to_ce/README.md @@ -57,7 +57,7 @@ $ sudo gitlab-rails runner "Service.where(type: ['JenkinsService', 'JenkinsDepre $ bundle exec rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService', 'GithubService']).delete_all" production ``` -### Secret variables environment scopes +### Variables environment scopes If you're using this feature and there are variables sharing the same key, but they have different scopes in a project, then you might want to diff --git a/doc/install/kubernetes/gitlab_omnibus.md b/doc/install/kubernetes/gitlab_omnibus.md index 98af87455ec..e1d1969651e 100644 --- a/doc/install/kubernetes/gitlab_omnibus.md +++ b/doc/install/kubernetes/gitlab_omnibus.md @@ -144,7 +144,7 @@ helm install --name gitlab -f values.yaml gitlab/gitlab-omnibus or passing them on the command line: ```bash -helm install --name gitlab --set baseDomain=gitlab.io,baseIP=1.1.1.1,gitlab=ee,gitlabEELicense=$LICENSE,legoEmail=email@gitlab.com gitlab/gitlab-omnibus +helm install --name gitlab --set baseDomain=gitlab.io,baseIP=192.0.2.1,gitlab=ee,gitlabEELicense=$LICENSE,legoEmail=email@gitlab.com gitlab/gitlab-omnibus ``` ## Updating GitLab using the Helm Chart diff --git a/doc/integration/github.md b/doc/integration/github.md index 23bb8ef9303..680712f9e01 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -110,7 +110,7 @@ On the sign in page there should now be a GitHub icon below the regular sign in Click the icon to begin the authentication process. GitHub will ask the user to sign in and authorize the GitLab application. If everything goes well the user will be returned to GitLab and will be signed in. -### GitHub Enterprise with Self-Signed Certificate +## GitHub Enterprise with self-signed Certificate If you are attempting to import projects from GitHub Enterprise with a self-signed certificate and the imports are failing, you will need to disable SSL verification. diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index eec40a9b8f1..70087576678 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -7,13 +7,11 @@ GitLab.com will generate an application ID and secret key for you to use. 1. Sign in to GitLab.com -1. Navigate to your profile settings. +1. On the upper right corner, click on your avatar and go to your **Settings**. -1. Select "Applications" in the left menu. +1. Select **Applications** in the left menu. -1. Select "New application". - -1. Provide the required details. +1. Provide the required details for **Add new application**. - Name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive. - Redirect URI: @@ -24,9 +22,9 @@ GitLab.com will generate an application ID and secret key for you to use. The first link is required for the importer and second for the authorization. -1. Select "Submit". +1. Select **Save application**. -1. You should now see a Client ID and Client Secret near the top right of the page (see screenshot). +1. You should now see a **Application Id** and **Secret** near the top right of the page (see screenshot). Keep this page open as you continue configuration.  diff --git a/doc/integration/img/gitlab_app.png b/doc/integration/img/gitlab_app.png Binary files differindex b4958581a9b..8d6a4456fc4 100644 --- a/doc/integration/img/gitlab_app.png +++ b/doc/integration/img/gitlab_app.png diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 5faae7ca2d6..95221d8b6b1 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -492,8 +492,8 @@ directory (repositories, uploads). To restore a backup, you will also need to restore `/etc/gitlab/gitlab-secrets.json` (for Omnibus packages) or `/home/git/gitlab/.secret` (for installations from source). This file contains the database encryption key, -[CI secret variables](../ci/variables/README.md#secret-variables), and -secret variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). +[CI/CD variables](../ci/variables/README.md#variables), and +variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). If you fail to restore this encryption key file along with the application data backup, users with two-factor authentication enabled and GitLab Runners will lose access to your GitLab server. diff --git a/doc/security/information_exclusivity.md b/doc/security/information_exclusivity.md index f8e7fc3fd0e..22756232025 100644 --- a/doc/security/information_exclusivity.md +++ b/doc/security/information_exclusivity.md @@ -2,7 +2,7 @@ Git is a distributed version control system (DVCS). This means that everyone that works with the source code has a local copy of the complete repository. -In GitLab every project member that is not a guest (so reporters, developers and masters) can clone the repository to get a local copy. +In GitLab every project member that is not a guest (so reporters, developers and maintainers) can clone the repository to get a local copy. After obtaining this local copy the user can upload the full repository anywhere, including another project under their control or another server. The consequence is that you can't build access controls that prevent the intentional sharing of source code by users that have access to the source code. This is an inherent feature of a DVCS and all git management systems have this limitation. diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md index a573445ab5b..b17b0a4bc4a 100644 --- a/doc/security/webhooks.md +++ b/doc/security/webhooks.md @@ -2,7 +2,7 @@ If you have non-GitLab web services running on your GitLab server or within its local network, these may be vulnerable to exploitation via Webhooks. -With [Webhooks](../user/project/integrations/webhooks.md), you and your project masters and owners can set up URLs to be triggered when specific things happen to projects. Normally, these requests are sent to external web services specifically set up for this purpose, that process the request and its attached data in some appropriate way. +With [Webhooks](../user/project/integrations/webhooks.md), you and your project maintainers and owners can set up URLs to be triggered when specific things happen to projects. Normally, these requests are sent to external web services specifically set up for this purpose, that process the request and its attached data in some appropriate way. Things get hairy, however, when a Webhook is set up with a URL that doesn't point to an external, but to an internal service, that may do something completely unintended when the webhook is triggered and the POST request is sent. diff --git a/doc/ssh/README.md b/doc/ssh/README.md index b71e9bf3000..bab196e7609 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -171,7 +171,7 @@ This is really useful for cloning repositories to your Continuous Integration (CI) server. By using deploy keys, you don't have to set up a dummy user account. -If you are a project master or owner, you can add a deploy key in the +If you are a project maintainer or owner, you can add a deploy key in the project settings under the section 'Repository'. Specify a title for the new deploy key and paste a public SSH key. After this, the machine that uses the corresponding private SSH key has read-only or read-write (if enabled) @@ -196,7 +196,7 @@ This is really useful for integrating repositories to secured, shared Continuous Integration (CI) services or other shared services. GitLab administrators can set up the Global Shared Deploy key in GitLab and add the private key to any shared systems. Individual repositories opt into -exposing their repository using these keys when a project masters (or higher) +exposing their repository using these keys when a project maintainers (or higher) authorizes a Global Shared Deploy key to be used with their project. Global Shared Keys can provide greater security compared to Per-Project Deploy @@ -205,7 +205,7 @@ who needs to know and configure the private key. GitLab administrators set up Global Deploy keys in the Admin area under the section **Deploy Keys**. Ensure keys have a meaningful title as that will be -the primary way for project masters and owners to identify the correct Global +the primary way for project maintainers and owners to identify the correct Global Deploy key to add. For instance, if the key gives access to a SaaS CI instance, use the name of that service in the key name if that is all it is used for. When creating Global Shared Deploy keys, give some thought to the granularity @@ -213,7 +213,7 @@ of keys - they could be of very narrow usage such as just a specific service or of broader usage for something like "Anywhere you need to give read access to your repository". -Once a GitLab administrator adds the Global Deployment key, project masters +Once a GitLab administrator adds the Global Deployment key, project maintainers and owners can add it in project's **Settings > Repository** section by expanding the **Deploy Key** section and clicking **Enable** next to the appropriate key listed under **Public deploy keys available to any project**. diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md index 9ba05c7815b..cce02d218c2 100644 --- a/doc/system_hooks/system_hooks.md +++ b/doc/system_hooks/system_hooks.md @@ -138,7 +138,7 @@ Please refer to `group_rename` and `user_rename` for that case. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_add_to_team", - "project_access": "Master", + "project_access": "Maintainer", "project_id": 74, "project_name": "StoreCloud", "project_path": "storecloud", @@ -158,7 +158,7 @@ Please refer to `group_rename` and `user_rename` for that case. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_remove_from_team", - "project_access": "Master", + "project_access": "Maintainer", "project_id": 74, "project_name": "StoreCloud", "project_path": "storecloud", @@ -318,7 +318,7 @@ If the user is blocked via LDAP, `state` will be `ldap_blocked`. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_add_to_group", - "group_access": "Master", + "group_access": "Maintainer", "group_id": 78, "group_name": "StoreCloud", "group_path": "storecloud", @@ -335,7 +335,7 @@ If the user is blocked via LDAP, `state` will be `ldap_blocked`. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_remove_from_group", - "group_access": "Master", + "group_access": "Maintainer", "group_id": 78, "group_name": "StoreCloud", "group_path": "storecloud", diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index efec365042a..fec575f263f 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -437,7 +437,7 @@ repo or by specifying a project variable: file in it, Auto DevOps will detect the chart and use it instead of the [default one](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). This can be a great way to control exactly how your application is deployed. -- **Project variable** - Create a [project variable](../../ci/variables/README.md#secret-variables) +- **Project variable** - Create a [variable](../../ci/variables/README.md#variables) `AUTO_DEVOPS_CHART` with the URL of a custom chart to use. ### Customizing `.gitlab-ci.yml` @@ -497,17 +497,17 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `CANARY_ENABLED` | From GitLab 11.0, this variable can be used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). | | `INCREMENTAL_ROLLOUT_ENABLED`| From GitLab 10.8, this variable can be used to enable an [incremental rollout](#incremental-rollout-to-production) of your application for the production environment. | | `TEST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `test` job. If the variable is present, the job will not be created. | -| `CODEQUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `codequality` job. If the variable is present, the job will not be created. | +| `CODE_QUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `code_quality` job. If the variable is present, the job will not be created. | | `SAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast` job. If the variable is present, the job will not be created. | | `DEPENDENCY_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dependency_scanning` job. If the variable is present, the job will not be created. | -| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast:container` job. If the variable is present, the job will not be created. | +| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `container_scanning` job. If the variable is present, the job will not be created. | | `REVIEW_DISABLED` | From GitLab 11.0, this variable can be used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs will not be created. | | `DAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dast` job. If the variable is present, the job will not be created. | | `PERFORMANCE_DISABLED` | From GitLab 11.0, this variable can be used to disable the `performance` job. If the variable is present, the job will not be created. | TIP: **Tip:** Set up the replica variables using a -[project variable](../../ci/variables/README.md#secret-variables) +[project variable](../../ci/variables/README.md#variables) and scale your application by just redeploying it! CAUTION: **Caution:** @@ -582,7 +582,7 @@ staging environment and deploy to production manually. For this scenario, the `STAGING_ENABLED` environment variable was introduced. If `STAGING_ENABLED` is defined in your project (e.g., set `STAGING_ENABLED` to -`1` as a secret variable), then the application will be automatically deployed +`1` as a variable), then the application will be automatically deployed to a `staging` environment, and a `production_manual` job will be created for you when you're ready to manually deploy to production. @@ -595,7 +595,7 @@ A [canary environment](https://docs.gitlab.com/ee/user/project/canary_deployment before any changes are deployed to production. If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to -`1` as a secret variable) then two manual jobs will be created: +`1` as a variable) then two manual jobs will be created: - `canary` which will deploy the application to the canary environment - `production_manual` which is to be used by you when you're ready to manually @@ -611,7 +611,7 @@ This will allow you to first check how the app is behaving, and later manually increasing the rollout up to 100%. If `INCREMENTAL_ROLLOUT_ENABLED` is defined in your project (e.g., set -`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a secret variable), then instead of the +`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a variable), then instead of the standard `production` job, 4 different [manual jobs](../../ci/pipelines.md#manual-actions-from-the-pipeline-graph) will be created: diff --git a/doc/update/10.6-to-10.7.md b/doc/update/10.6-to-10.7.md index 4a76ae14d2e..4efbb8c65cf 100644 --- a/doc/update/10.6-to-10.7.md +++ b/doc/update/10.6-to-10.7.md @@ -80,8 +80,8 @@ More information can be found on the [yarn website](https://yarnpkg.com/en/docs/ ### 5. Update Go -NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go -1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. +NOTE: GitLab 9.2 and higher only supports Go 1.9 and dropped support for Go +1.5.x through 1.8.x. Be sure to upgrade your installation if necessary. You can check which version you are running with `go version`. @@ -91,11 +91,11 @@ Download and install Go: # Remove former Go installation folder sudo rm -rf /usr/local/go -curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz -echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ - sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +curl --remote-name --progress https://storage.googleapis.com/golang/go1.9.linux-amd64.tar.gz +echo 'd70eadefce8e160638a9a6db97f7192d8463069ab33138893ad3bf31b0650a79 go1.9.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.9.linux-amd64.tar.gz sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ -rm go1.8.3.linux-amd64.tar.gz +rm go1.9.linux-amd64.tar.gz ``` ### 6. Get latest code diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 159109e8954..9b0ff02f227 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -11,7 +11,7 @@ You can leave a comment in the following places: - commit diffs The comment area supports [Markdown] and [quick actions]. One can edit their -own comment at any time, and anyone with [Master access level][permissions] or +own comment at any time, and anyone with [Maintainer access level][permissions] or higher can also edit a comment made by someone else. You could also reply to the notification email in order to reply to a comment, @@ -253,7 +253,7 @@ to newer issues or merge requests. - The people participating in the discussion are trolling, abusive, or otherwise being unproductive. -In these cases, a user with Master permissions or higher in the project can lock (and unlock) +In these cases, a user with Maintainer permissions or higher in the project can lock (and unlock) an issue or a merge request, using the "Lock" section in the sidebar: | Unlock | Lock | diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 30761a66563..e6bf32a2dc5 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -125,7 +125,7 @@ side of your screen. --- -Group owners and masters will be notified of your request and will be able to approve or +Group owners and maintainers will be notified of your request and will be able to approve or decline it on the members page.  diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 02f8ef08117..08849ac1df4 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -145,8 +145,8 @@ permissions. For example, if User0 was first added to group `group-1/group-1-1` with Developer permissions, then they will inherit those permissions in every other subgroup -of `group-1/group-1-1`. To give them Master access to `group-1/group-1-1/group1-1-1`, -you would add them again in that group as Master. Removing them from that group, +of `group-1/group-1-1`. To give them Maintainer access to `group-1/group-1-1/group1-1-1`, +you would add them again in that group as Maintainer. Removing them from that group, the permissions will fallback to those of the ancestor group. ## Mentioning subgroups diff --git a/doc/user/index.md b/doc/user/index.md index a50e5e8fbf8..640edab6ea4 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -110,7 +110,7 @@ personal access tokens, authorized applications, etc. - [Authentication](../topics/authentication/index.md): Read through the authentication methods available in GitLab. - [Permissions](permissions.md): Learn the different set of permissions levels for each -user type (guest, reporter, developer, master, owner). +user type (guest, reporter, developer, maintainer, owner). - [Feature highlight](feature_highlight.md): Learn more about the little blue dots around the app that explain certain features diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 61dd0fbaed1..16c19855136 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -27,7 +27,7 @@ See our [product handbook on permissions](https://about.gitlab.com/handbook/prod The following table depicts the various user permission levels in a project. -| Action | Guest | Reporter | Developer | Master | Owner | +| Action | Guest | Reporter | Developer |Maintainer| Owner | |---------------------------------------|---------|------------|-------------|----------|--------| | Create new issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | | Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | @@ -109,7 +109,7 @@ review, we've created protected branches. Read through the documentation on [protected branches](project/protected_branches.md) to learn more. -Additionally, you can allow or forbid users with Master and/or +Additionally, you can allow or forbid users with Maintainer and/or Developer permissions to push to a protected branch. Read through the documentation on [Allowed to Merge and Allowed to Push settings](project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings) to learn more. @@ -150,7 +150,7 @@ Any user can remove themselves from a group, unless they are the last Owner of the group. The following table depicts the various user permission levels in a group. -| Action | Guest | Reporter | Developer | Master | Owner | +| Action | Guest | Reporter | Developer | Maintainer | Owner | |-------------------------|-------|----------|-----------|--------|-------| | Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | | Edit group | | | | | ✓ | @@ -200,7 +200,7 @@ GitLab CI/CD permissions rely on the role the user has in GitLab. There are four permission levels in total: - admin -- master +- maintainer - developer - guest/reporter @@ -208,7 +208,7 @@ The admin user can perform any action on GitLab CI/CD in scope of the GitLab instance and project. In addition, all admins can use the admin interface under `/admin/runners`. -| Action | Guest, Reporter | Developer | Master | Admin | +| Action | Guest, Reporter | Developer |Maintainer| Admin | |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | @@ -230,7 +230,7 @@ Read all about the [new model and its implications][new-mod]. This table shows granted privileges for jobs triggered by specific types of users: -| Action | Guest, Reporter | Developer | Master | Admin | +| Action | Guest, Reporter | Developer |Maintainer| Admin | |---------------------------------------------|-----------------|-------------|----------|--------| | Run CI job | | ✓ | ✓ | ✓ | | Clone source and LFS from current project | | ✓ | ✓ | ✓ | @@ -276,7 +276,7 @@ only. [^1]: On public and internal projects, all users are able to perform this action [^2]: Guest users can only view the confidential issues they created themselves [^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** -[^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner +[^4]: Not allowed for Guest, Reporter, Developer, Maintainer, or Owner [^5]: Only if the job was triggered by the user [^6]: Only if user is not external one [^7]: Only if user is a member of the project diff --git a/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png b/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png Binary files differnew file mode 100644 index 00000000000..9a0559a19d4 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png b/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png Binary files differnew file mode 100644 index 00000000000..657ab0d9fa9 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/create_project.png b/doc/user/project/clusters/eks_and_gitlab/img/create_project.png Binary files differnew file mode 100644 index 00000000000..f3446131419 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/create_project.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png b/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png Binary files differnew file mode 100644 index 00000000000..d6c3b1b3a94 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/environment.png b/doc/user/project/clusters/eks_and_gitlab/img/environment.png Binary files differnew file mode 100644 index 00000000000..77d711ba8f6 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/environment.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/new_project.png b/doc/user/project/clusters/eks_and_gitlab/img/new_project.png Binary files differnew file mode 100644 index 00000000000..d401c4ac2bf --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/new_project.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png b/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png Binary files differnew file mode 100644 index 00000000000..5f9c9815c24 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png diff --git a/doc/user/project/clusters/eks_and_gitlab/index.md b/doc/user/project/clusters/eks_and_gitlab/index.md new file mode 100644 index 00000000000..f50615729dd --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/index.md @@ -0,0 +1,136 @@ +--- +redirect_from: 'https://docs.gitlab.com/ee/user/project/clusters/eks_and_gitlab/index.html' +--- + +# Connecting and deploying to an Amazon EKS cluster + +> **[Article Type](../../../../development/writing_documentation.md#types-of-technical-articles):** tutorial || +> **Level:** intermediate || +> **Author:** [Joshua Lambert](https://gitlab.com/joshlambert) || +> **Publication date:** 2018-06-05 + +## Introduction + +In this tutorial, we will show how easy it is to integrate an [Amazon EKS](https://aws.amazon.com/eks/) cluster with GitLab, and begin deploying applications. + +For an end-to-end walkthrough we will: +1. Start with a new project based on the sample Ruby on Rails template +1. Integrate an EKS cluster +1. Utilize [Auto DevOps](../../../../topics/autodevops/) to build, test, and deploy our application + +You will need: +1. An account on GitLab, like [GitLab.com](https://gitlab.com) +1. An Amazon EKS cluster +1. `kubectl` [installed and configured for access to the EKS cluster](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html#get-started-kubectl) + +If you don't have an Amazon EKS cluster, one can be created by following [the EKS getting started guide](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html). + +## Creating a new project + +On GitLab, create a new project by clicking on the `+` icon in the top navigation bar, and selecting `New project`. + + + +On the new project screen, click on the `Create from template` tab, and select `Use template` for the Ruby on Rails sample project. + +Give the project a name, and then select `Create project`. + + + +## Connecting the EKS cluster + +From the left side bar, hover over `CI/CD` and select `Kubernetes`, then click on `Add Kubernetes cluster`, and finally `Add an existing Kubernetes cluster`. + +A few details from the EKS cluster will be required to connect it to GitLab. + +1. A valid Kubernetes certificate and token are needed to authenticate to the EKS cluster. A pair is created by default, which can be used. Open a shell and use `kubectl` to retrieve them: + * List the secrets with `kubectl get secrets`, and one should named similar to `default-token-xxxxx`. Copy that token name for use below. + * Get the certificate with `kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 -D` + * Retrieve the token with `kubectl get secret <secret name> -o jsonpath="{['data']['token']}" | base64 -D`. +1. The API server endpoint is also required, so GitLab can connect to the cluster. This is displayed on the AWS EKS console, when viewing the EKS cluster details. + +You now have all the information needed to connect the EKS cluster: +* Kubernetes cluster name: Provide a name for the cluster to identify it within GitLab. +* Environment scope: Leave this as `*` for now, since we are only connecting a single cluster. +* API URL: Paste in the API server endpoint retrieved above. +* CA Certificate: Paste the certificate data from the earlier step, as-is. +* Paste the token value. Note on some versions of Kubernetes a trailing `%` is output, do not include it. +* Project namespace: This can be left blank to accept the default namespace, based on the project name. + + + +Click on `Add Kubernetes cluster`, the cluster is now connected to GitLab. At this point, [Kubernetes deployment variables](../#deployment-variables) will automatically be available during CI jobs, making it easy to interact with the cluster. + +If you would like to utilize your own CI/CD scripts to deploy to the cluster, you can stop here. + +## Disable Role Based-Access Control (RBAC) + +Presently, Auto DevOps and one-click app installs do not support [Kubernetes role-based access control](https://kubernetes.io/docs/reference/access-authn-authz/rbac/). Support is [being worked on](https://gitlab.com/groups/gitlab-org/-/epics/136), but in the interim RBAC must be disabled to utilize for these features. + +> **Note**: Disabling RBAC means that any application running in the cluster, or user who can authenticate to the cluster, has full API access. This is a [security concern](https://docs.gitlab.com/ee/user/project/clusters/#security-implications), and may not be desirable. + +To effectively disable RBAC, global permissions can be applied granting full access: + +```bash +kubectl create clusterrolebinding permissive-binding \ + --clusterrole=cluster-admin \ + --user=admin \ + --user=kubelet \ + --group=system:serviceaccounts +``` + +## Deploy services to the cluster + +GitLab supports one-click deployment of helpful services to the cluster, many of which support Auto DevOps. Back on the Kubernetes cluster screen in GitLab, a list of applications is now available to deploy. + +First install Helm Tiller, a package manager for Kubernetes. This enables deployment of the other applications. + + + +### Deploying NGINX Ingress (optional) + +Next, if you would like the deployed app to be reachable on the internet, deploy the Ingress. Note that this will also cause an [Elastic Load Balancer](https://aws.amazon.com/documentation/elastic-load-balancing/) to be created, which will incur additional AWS costs. + +Once installed, you may see a `?` for `Ingress IP Address`. This is because the created ELB is available at a DNS name, not an IP address. To get the DNS name, run: `kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -o jsonpath="{.status.loadBalancer.ingress[0].hostname}"`. Note, you may see a trailing `%` on some Kubernetes versions, do not include it. + +The Ingress is now available at this address, and will route incoming requests to the proper service based on the DNS name in the request. To support this, a wildcard DNS CNAME record should be created for the desired domain name. For example `*.myekscluster.com` would point to the Ingress hostname obtained earlier. + + + +### Deploying the GitLab Runner (optional) + +If the project is on GitLab.com, free shared runners are available and you do not have to deploy one. If a project specific runner is desired, or there are no shared runners, it is easy to deploy one. + +Simply click on the `Install` button for the GitLab Runner. It is important to note that the runner deployed is set as **privileged**, which means it essentially has root access to the underlying machine. This is required to build docker images, and so is on by default. + +### Deploying Prometheus (optional) + +GitLab is able to monitor applications automatically, utilizing [Prometheus](../../integrations/prometheus.html). Kubernetes container CPU and memory metrics are automatically collected, and response metrics are retrieved from NGINX Ingress as well. + +To enable monitoring, simply install Prometheus into the cluster with the `Install` button. + +## Create a default Storage Class + +Amazon EKS does not have a default Storage Class out of the box, which means requests for persistent volumes will not be automatically fulfilled. As part of Auto DevOps, the deployed Postgres instance requests persistent storage, and without a default storage class it will fail to start. + +If a default Storage Class does not already exist and is desired, follow Amazon's [short guide](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html) to create one. + +Alternatively, disable Postgres by setting the project variable [`POSTGRES_ENABLED`](../../../../topics/autodevops/#environment-variables) to `false`. + +## Deploy the app to EKS + +With RBAC disabled and services deployed, [Auto DevOps](https://docs.gitlab.com/ee/topics/autodevops/) can now be leveraged to build, test, and deploy the app. To enable, click on `Settings` in the left sidebar, then `CI/CD`. You will see a section for `Auto DevOps`, expand it. Click on the radio button to `Enable Auto DevOps`. + +If a wildcard DNS entry was created resolving to the Load Balancer, enter it in the `domain` field. Otherwise, the deployed app will not be externally available outside of the cluster. To save, click `Save changes`. + + + +A new pipeline will automatically be created, which will begin to build, test, and deploy the app. + +After the pipeline has finished, your app will be running in EKS and available to users. Click on `CI/CD` tab in the left navigation bar, and choose `Environments`. + + + +You will see a list of the environments and their deploy status, as well as options to browse to the app, view monitoring metrics, and even access a shell on the running pod. + +To learn more about Auto DevOps, review our [documentation](../../../../topics/autodevops/). diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 65cdece8d3d..1efbe20b84f 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -19,7 +19,7 @@ or provide the credentials to an [existing Kubernetes cluster](#adding-an-existi ## Adding and creating a new GKE cluster via GitLab NOTE: **Note:** -You need Master [permissions] and above to access the Kubernetes page. +You need Maintainer [permissions] and above to access the Kubernetes page. Before proceeding, make sure the following requirements are met: @@ -30,7 +30,7 @@ Before proceeding, make sure the following requirements are met: clusters on GKE. That would mean that a [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account) must be set up and that you have to have permissions to access it. -- You must have Master [permissions] in order to be able to access the +- You must have Maintainer [permissions] in order to be able to access the **Kubernetes** page. - You must have [Cloud Billing API](https://cloud.google.com/billing/) enabled - You must have [Resource Manager @@ -66,7 +66,7 @@ enable the Cluster integration. ## Adding an existing Kubernetes cluster NOTE: **Note:** -You need Master [permissions] and above to access the Kubernetes page. +You need Maintainer [permissions] and above to access the Kubernetes page. To add an existing Kubernetes cluster to your project: @@ -201,6 +201,11 @@ Otherwise, you can list the IP addresses of all load balancers: kubectl get svc --all-namespaces -o jsonpath='{range.items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[*].ip} ' ``` +> **Note**: Some Kubernetes clusters return a hostname instead, like [Amazon EKS](https://aws.amazon.com/eks/). For these platforms, run: +> ```bash +> kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -o jsonpath="{.status.loadBalancer.ingress[0].hostname}"`. +> ``` + The output is the external IP address of your cluster. This information can then be used to set up DNS entries and forwarding rules that allow external access to your deployed applications. @@ -233,7 +238,7 @@ When adding more than one Kubernetes clusters to your project, you need to differentiate them with an environment scope. The environment scope associates clusters and [environments](../../../ci/environments.md) in an 1:1 relationship similar to how the -[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-secret-variables) +[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-variables) work. The default environment scope is `*`, which means all jobs, regardless of their @@ -325,7 +330,7 @@ To disable the Kubernetes cluster integration, follow the same procedure. ## Removing the Kubernetes cluster integration NOTE: **Note:** -You need Master [permissions] and above to remove a Kubernetes cluster integration. +You need Maintainer [permissions] and above to remove a Kubernetes cluster integration. NOTE: **Note:** When you remove a cluster, you only remove its relation to GitLab, not the @@ -382,7 +387,7 @@ you will need the Kubernetes project integration enabled. ### Web terminals NOTE: **Note:** -Introduced in GitLab 8.15. You must be the project owner or have `master` permissions +Introduced in GitLab 8.15. You must be the project owner or have `maintainer` permissions to use terminals. Support is limited to the first container in the first pod of your environment. diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 9c5e3509046..03302b3815d 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -143,6 +143,24 @@ docker login registry.example.com -u <your_username> -p <your_access_token> for errors (e.g. `/var/log/gitlab/gitlab-rails/production.log`). You may be able to find clues there. +#### Enable the registry debug server + +The optional debug server can be enabled by setting the registry debug address +in your `gitlab.rb` configuration. + +```ruby +registry['debug_addr'] = "localhost:5001" +``` + +After adding the setting, [reconfigure] GitLab to apply the change. + +Use curl to request debug output from the debug server: + +```bash +curl localhost:5001/debug/health +curl localhost:5001/debug/vars +``` + ### Advanced Troubleshooting >**NOTE:** The following section is only recommended for experts. @@ -275,3 +293,4 @@ Once the right permissions were set, the error will go away. [docker-docs]: https://docs.docker.com/engine/userguide/intro/ [pat]: ../profile/personal_access_tokens.md [pdt]: ../project/deploy_tokens/index.md +[reconfigure]: ../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
\ No newline at end of file diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index c09d5aeba8e..0b9b49f326f 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -5,7 +5,7 @@ Deploy tokens allow to download (through `git clone`), or read the container registry images of a project without the need of having a user and a password. Please note, that the expiration of deploy tokens happens on the date you define, -at midnight UTC and that they can be only managed by [masters](https://docs.gitlab.com/ee/user/permissions.html). +at midnight UTC and that they can be only managed by [maintainers](https://docs.gitlab.com/ee/user/permissions.html). ## Creating a Deploy Token diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 8c639bd5343..cad85881c4d 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -1,154 +1,143 @@ # Import your project from GitHub to GitLab -Import your projects from GitHub to GitLab with minimal effort. +Using the importer, you can import your GitHub repositories to GitLab.com or to +your self-hosted GitLab instance. ## Overview ->**Note:** -If you are an administrator you can enable the [GitHub integration][gh-import] -in your GitLab instance sitewide. This configuration is optional, users will -still be able to import their GitHub repositories with a -[personal access token][gh-token]. - ->**Note:** -Administrators of a GitLab instance (Community or Enterprise Edition) can also -use the [GitHub rake task][gh-rake] to import projects from GitHub without the -constrains of a Sidekiq worker. - -- At its current state, GitHub importer can import: - - the repository description (GitLab 7.7+) - - the Git repository data (GitLab 7.7+) - - the issues (GitLab 7.7+) - - the pull requests (GitLab 8.4+) - - the wiki pages (GitLab 8.4+) - - the milestones (GitLab 8.7+) - - the labels (GitLab 8.7+) - - the release note descriptions (GitLab 8.12+) - - the pull request review comments (GitLab 10.2+) - - the regular issue and pull request comments -- References to pull requests and issues are preserved (GitLab 8.7+) -- Repository public access is retained. If a repository is private in GitHub - it will be created as private in GitLab as well. +NOTE: **Note:** +While these instructions will always work for users on GitLab.com, if you are an +administrator of a self-hosted GitLab instance, you will need to enable the +[GitHub integration][gh-import] in order for users to follow the preferred +import method described on this page. If this is not enabled, users can alternatively import their +GitHub repositories using a [personal access token](#using-a-github-token) from GitHub, +but this method will not be able to associate all user activity (such as issues and pull requests) +with matching GitLab users. As an administrator of a self-hosted GitLab instance, you can also use +the [GitHub rake task](../../../administration/raketasks/github_import.md) to import projects from +GitHub without the constraints of a Sidekiq worker. + +The following aspects of a project are imported: + * Repository description (GitLab.com & 7.7+) + * Git repository data (GitLab.com & 7.7+) + * Issues (GitLab.com & 7.7+) + * Pull requests (GitLab.com & 8.4+) + * Wiki pages (GitLab.com & 8.4+) + * Milestones (GitLab.com & 8.7+) + * Labels (GitLab.com & 8.7+) + * Release note descriptions (GitLab.com & 8.12+) + * Pull request review comments (GitLab.com & 10.2+) + * Regular issue and pull request comments + +References to pull requests and issues are preserved (GitLab.com & 8.7+), and +each imported repository defaults to `private` but [can be made public](../settings/index.md#sharing-and-permissions), as needed. ## How it works -When issues/pull requests are being imported, the GitHub importer tries to find -the GitHub author/assignee in GitLab's database using the GitHub ID. For this -to work, the GitHub author/assignee should have signed in beforehand in GitLab -and **associated their GitHub account**. If the user is not -found in GitLab's database, the project creator (most of the times the current -user that started the import process) is set as the author, but a reference on -the issue about the original GitHub author is kept. +When issues and pull requests are being imported, the importer attempts to find their GitHub authors and +assignees in the database of the GitLab instance (note that pull requests are called "merge requests" in GitLab). -The importer will create any new namespaces (groups) if they don't exist or in -the case the namespace is taken, the repository will be imported under the user's -namespace that started the import process. +For this association to succeed, prior to the import, each GitHub author and assignee in the repository must +have either previously logged in to a GitLab account using the GitHub icon **or** have a GitHub account with +a [public email address](https://help.github.com/articles/setting-your-commit-email-address-on-github/) that +matches their GitLab account's email address. -The importer will also import branches on forks of projects related to open pull -requests. These branches will be imported with a naming scheme similar to -GH-SHA-Username/Pull-Request-number/fork-name/branch. This may lead to a discrepancy -in branches compared to the GitHub Repository. +If a user referenced in the project is not found in GitLab's database, the project creator (typically the user +that initiated the import process) is set as the author/assignee, but a note on the issue mentioning the original +GitHub author is added. -For a more technical description and an overview of the architecture you can -refer to [Working with the GitHub importer][gh-import-dev-docs]. +The importer creates any new namespaces (groups) if they do not exist, or, if the namespace is taken, the +repository is imported under the namespace of the user who initiated the import process. The namespace/repository +name can also be edited, with the proper permissions. -## Importing your GitHub repositories +The importer will also import branches on forks of projects related to open pull requests. These branches will be +imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to +a discrepancy in branches compared to those of the GitHub repository. -The importer page is visible when you create a new project. +For additional technical details, you can refer to the +[GitHub Importer](../../../development/github_importer.md "Working with the GitHub importer") +developer documentation. - +## Import your GitHub repository into GitLab -Click on the **GitHub** link and the import authorization process will start. -There are two ways to authorize access to your GitHub repositories: +### Using the GitHub integration -1. [Using the GitHub integration][gh-integration] (if it's enabled by your - GitLab administrator). This is the preferred way as it's possible to - preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works) - section. -1. [Using a personal access token][gh-token] provided by GitHub. +Before you begin, ensure that any GitHub users who you want to map to GitLab users have either: - +1. A GitLab account that has logged in using the GitHub icon +\- or - +2. A GitLab account with an email address that matches the [public email address](https://help.github.com/articles/setting-your-commit-email-address-on-github/) of the GitHub user -### Authorize access to your repositories using the GitHub integration +User-matching attempts occur in that order, and if a user is not identified either way, the activity is associated with +the user account that is performing the import. -If the [GitHub integration][gh-import] is enabled by your GitLab administrator, -you can use it instead of the personal access token. +NOTE: **Note:** +If you are using a self-hosted GitLab instance, this process requires that you have configured the +[GitHub integration][gh-import]. -1. First you may want to connect your GitHub account to GitLab in order for - the username mapping to be correct. -1. Once you connect GitHub, click the **List your GitHub repositories** button - and you will be redirected to GitHub for permission to access your projects. -1. After accepting, you'll be automatically redirected to the importer. +1. From the top navigation bar, click **+** and select **New project**. +2. Select the **Import project** tab and then select **GitHub**. +3. Select the first button to **List your GitHub repositories**. You are redirected to a page on github.com to authorize the GitLab application. +4. Click **Authorize gitlabhq**. You are redirected back to GitLab's Import page and all of your GitHub repositories are listed. +5. Continue on to [selecting which repositories to import](#selecting-which-repositories-to-import). -You can now go on and [select which repositories to import](#select-which-repositories-to-import). +### Using a GitHub token -### Authorize access to your repositories using a personal access token +NOTE: **Note:** +For a proper author/assignee mapping for issues and pull requests, the [GitHub integration method (above)](#using-the-github-integration) +should be used instead of the personal access token. If you are using GitLab.com or a self-hosted GitLab instance with the GitHub +integration enabled, that should be the preferred method to import your repositories. Read more in the [How it works](#how-it-works) section. ->**Note:** -For a proper author/assignee mapping for issues and pull requests, the -[GitHub integration][gh-integration] should be used instead of the -[personal access token][gh-token]. If the GitHub integration is enabled by your -GitLab administrator, it should be the preferred method to import your repositories. -Read more in the [How it works](#how-it-works) section. +If you are not using the GitHub integration, you can still perform an authorization with GitHub to grant GitLab access your repositories: -If you are not using the GitHub integration, you can still perform a one-off -authorization with GitHub to grant GitLab access your repositories: +1. Go to https://github.com/settings/tokens/new +2. Enter a token description. +3. Select the repo scope. +4. Click **Generate token**. +5. Copy the token hash. +6. Go back to GitLab and provide the token to the GitHub importer. +7. Hit the **List Your GitHub Repositories** button and wait while GitLab reads your repositories' information. + Once done, you'll be taken to the importer page to select the repositories to import. -1. Go to <https://github.com/settings/tokens/new>. -1. Enter a token description. -1. Check the `repo` scope. -1. Click **Generate token**. -1. Copy the token hash. -1. Go back to GitLab and provide the token to the GitHub importer. -1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads - your repositories' information. Once done, you'll be taken to the importer - page to select the repositories to import. +### Selecting which repositories to import -### Select which repositories to import +After you have authorized access to your GitHub repositories, you are redirected to the GitHub importer page and +your GitHub repositories are listed. -After you've authorized access to your GitHub repositories, you will be -redirected to the GitHub importer page. +1. By default, the proposed repository namespaces match the names as they exist in GitHub, but based on your permissions, + you can choose to edit these names before you proceed to import any of them. +2. Select the **Import** button next to any number of repositories, or select **Import all repositories**. +3. The **Status** column shows the import status of each repository. You can choose to leave the page open and it will + update in realtime or you can return to it later. +4. Once a repository has been imported, click its GitLab path to open its GitLab URL. -From there, you can see the import statuses of your GitHub repositories. +## Mirroring and pipeline status sharing -- Those that are being imported will show a _started_ status, -- those already successfully imported will be green with a _done_ status, -- whereas those that are not yet imported will have an **Import** button on the - right side of the table. +Depending your GitLab tier, [project mirroring](../../../workflow/repository_mirroring.md) can be set up to keep +your imported project in sync with its GitHub copy. -If you want, you can import all your GitHub projects in one go by hitting -**Import all projects** in the upper left corner. +Additionally, you can configure GitLab to send pipeline status updates back GitHub with the +[GitHub Project Integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). **[PREMIUM]** - +If you import your project using [CI/CD for external repo](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/), then both +of the above are automatically configured. **[PREMIUM]** ---- +## Improving the speed of imports on self-hosted instances -You can also choose a different name for the project and a different namespace, -if you have the privileges to do so. +NOTE: **Note:** +Admin access to the GitLab server is required. -## Making the import process go faster - -For large projects it may take a while to import all data. To reduce the time -necessary you can increase the number of Sidekiq workers that process the -following queues: +For large projects it may take a while to import all data. To reduce the time necessary, you can increase the number of +Sidekiq workers that process the following queues: * `github_importer` * `github_importer_advance_stage` -For an optimal experience we recommend having at least 4 Sidekiq processes (each -running a number of threads equal to the number of CPU cores) that _only_ -process these queues. We also recommend that these processes run on separate -servers. For 4 servers with 8 cores this means you can import up to 32 objects -(e.g. issues) in parallel. +For an optimal experience, it's recommended having at least 4 Sidekiq processes (each running a number of threads equal +to the number of CPU cores) that *only* process these queues. It's also recommended that these processes run on separate +servers. For 4 servers with 8 cores this means you can import up to 32 objects (e.g., issues) in parallel. -Reducing the time spent in cloning a repository can be done by increasing -network throughput, CPU capacity, and disk performance (e.g. by using high -performance SSDs) of the disks that store the Git repositories (for your GitLab -instance). Increasing the number of Sidekiq workers will _not_ reduce the time -spent cloning repositories. +Reducing the time spent in cloning a repository can be done by increasing network throughput, CPU capacity, and disk +performance (e.g., by using high performance SSDs) of the disks that store the Git repositories (for your GitLab instance). +Increasing the number of Sidekiq workers will *not* reduce the time spent cloning repositories. [gh-import]: ../../../integration/github.md "GitHub integration" -[gh-rake]: ../../../administration/raketasks/github_import.md "GitHub rake task" -[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration -[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token -[gh-import-dev-docs]: ../../../development/github_importer.md "Working with the GitHub importer" diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index e384ed57de9..363e994f36d 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -2,7 +2,7 @@ You can find the available integrations under your project's **Settings ➔ Integrations** page. You need to have at least -[master permission][permissions] on the project. +[maintainer permission][permissions] on the project. ## Project services diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 0bf1f396f9d..8eada25234f 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -71,10 +71,10 @@ least [Reporter access][permissions]. However, a guest user can also create confidential issues, but can only view the ones that they created themselves. Confidential issues are also hidden in search results for unprivileged users. -For example, here's what a user with Master and Guest access sees in the +For example, here's what a user with Maintainer and Guest access sees in the project's search results respectively. -| Master access | Guest access | +| Maintainer access | Guest access | | :-----------: | :----------: | |  |  | diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 43713855e26..2c2e8e2d556 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -4,7 +4,7 @@ You can manage the groups and users and their access levels in all of your projects. You can also personalize the access level you give each user, per-project. -You should have `master` or `owner` [permissions](../../permissions.md) to add +You should have Maintainer or Owner [permissions](../../permissions.md) to add or import a new user to your project. To view, edit, add, and remove project's members, go to your @@ -43,7 +43,7 @@ level to the project. You can import another project's users in your own project by hitting the **Import members** button on the upper right corner of the **Members** menu. -In the dropdown menu, you can see only the projects you are Master on. +In the dropdown menu, you can see only the projects you are Maintainer on.  @@ -99,7 +99,7 @@ side of your screen. --- -Project owners & masters will be notified of your request and will be able to approve or +Project owners & maintainers will be notified of your request and will be able to approve or decline it on the members page.  diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index 5d819998dd9..611ff0e6bfb 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -42,7 +42,7 @@ Admins are able to share projects with any group in the system. ## Maximum access level -In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Master' or 'Owner') will only have 'Developer' access to 'Project Acme'. +In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Maintainer' or 'Owner') will only have 'Developer' access to 'Project Acme'. ## Share project with group lock diff --git a/doc/user/project/merge_requests/allow_collaboration.md b/doc/user/project/merge_requests/allow_collaboration.md new file mode 100644 index 00000000000..859ac92ef89 --- /dev/null +++ b/doc/user/project/merge_requests/allow_collaboration.md @@ -0,0 +1,20 @@ +# Allow collaboration on merge requests across forks + +> [Introduced][ce-17395] in GitLab 10.6. + +This feature is available for merge requests across forked projects that are +publicly accessible. It makes it easier for members of projects to +collaborate on merge requests across forks. + +When enabled for a merge request, members with merge access to the target +branch of the project will be granted write permissions to the source branch +of the merge request. + +The feature can only be enabled by users who already have push access to the +source project, and only lasts while the merge request is open. + +Enable this functionality while creating or editing a merge request: + + + +[ce-17395]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17395 diff --git a/doc/user/project/merge_requests/authorization_for_merge_requests.md b/doc/user/project/merge_requests/authorization_for_merge_requests.md index 59b3fe7242c..79444ee5682 100644 --- a/doc/user/project/merge_requests/authorization_for_merge_requests.md +++ b/doc/user/project/merge_requests/authorization_for_merge_requests.md @@ -9,7 +9,7 @@ There are two main ways to have a merge request flow with GitLab: With the protected branch flow everybody works within the same GitLab project. -The project maintainers get Master access and the regular developers get +The project maintainers get Maintainer access and the regular developers get Developer access. The maintainers mark the authoritative branches as 'Protected'. @@ -18,7 +18,7 @@ The developers push feature branches to the project and create merge requests to have their feature branches reviewed and merged into one of the protected branches. -By default, only users with Master access can merge changes into a protected +By default, only users with Maintainer access can merge changes into a protected branch. **Advantages** @@ -32,7 +32,7 @@ branch. ## Forking workflow -With the forking workflow the maintainers get Master access and the regular +With the forking workflow the maintainers get Maintainer access and the regular developers get Reporter access to the authoritative repository, which prohibits them from pushing any changes to it. diff --git a/doc/user/project/merge_requests/img/allow_collaboration.png b/doc/user/project/merge_requests/img/allow_collaboration.png Binary files differnew file mode 100644 index 00000000000..75596e7d9ad --- /dev/null +++ b/doc/user/project/merge_requests/img/allow_collaboration.png diff --git a/doc/user/project/merge_requests/img/allow_maintainer_push.png b/doc/user/project/merge_requests/img/allow_maintainer_push.png Binary files differdeleted file mode 100644 index 91cc399f4ff..00000000000 --- a/doc/user/project/merge_requests/img/allow_maintainer_push.png +++ /dev/null diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index b75bcacc9d7..5e2e0c3d171 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -28,7 +28,7 @@ With GitLab merge requests, you can: - Enable [fast-forward merge requests](#fast-forward-merge-requests) - Enable [semi-linear history merge requests](#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch - [Create new merge requests by email](#create-new-merge-requests-by-email) -- Allow maintainers of the target project to push directly to the fork by [allowing edits from maintainers](maintainer_access.md) +- [Allow collaboration](allow_collaboration.md) so members of the target project can push directly to the fork - [Squash and merge](squash_and_merge.md) for a cleaner commit history With **[GitLab Enterprise Edition][ee]**, you can also: @@ -85,7 +85,7 @@ request is merged. This option is also visible in an existing merge request next to the merge request button and can be selected/deselected before merging. It's only visible -to users with [Master permissions](../../permissions.md) in the source project. +to users with [Maintainer permissions](../../permissions.md) in the source project. If the user viewing the merge request does not have the correct permissions to remove the source branch and the source branch is set for removal, the merge diff --git a/doc/user/project/merge_requests/maintainer_access.md b/doc/user/project/merge_requests/maintainer_access.md index 89f71e16a50..d59afecd375 100644 --- a/doc/user/project/merge_requests/maintainer_access.md +++ b/doc/user/project/merge_requests/maintainer_access.md @@ -1,20 +1 @@ -# Allow maintainer pushes for merge requests across forks - -> [Introduced][ce-17395] in GitLab 10.6. - -This feature is available for merge requests across forked projects that are -publicly accessible. It makes it easier for maintainers of projects to -collaborate on merge requests across forks. - -When enabled for a merge request, members with merge access to the target -branch of the project will be granted write permissions to the source branch -of the merge request. - -The feature can only be enabled by users who already have push access to the -source project, and only lasts while the merge request is open. - -Enable this functionality while creating a merge request: - - - -[ce-17395]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17395 +This document was moved to [another location](allow_collaboration.md). diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 862e4a3b466..205f0283107 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -42,7 +42,7 @@ to secure them. Your files live in a project [repository](../repository/index.md) on GitLab. [GitLab CI](../../../ci/README.md) picks up those files and makes them available at, typically, -`http://<username>.gitlab.io/<projectname>`. Please read through the docs on +`https://<username>.gitlab.io/<projectname>`. Please read through the docs on [GitLab Pages domains](getting_started_part_one.md#gitlab-pages-domain) for more info. ## Explore GitLab Pages diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 0cbb0c878c2..3bf63a22963 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -10,8 +10,8 @@ created protected branches. By default, a protected branch does four simple things: - it prevents its creation, if not already created, from everybody except users - with Master permission -- it prevents pushes from everybody except users with Master permission + with Maintainer permission +- it prevents pushes from everybody except users with Maintainer permission - it prevents **anyone** from force pushing to the branch - it prevents **anyone** from deleting the branch @@ -24,7 +24,7 @@ See the [Changelog](#changelog) section for changes over time. ## Configuring protected branches -To protect a branch, you need to have at least Master permission level. Note +To protect a branch, you need to have at least Maintainer permission level. Note that the `master` branch is protected by default. 1. Navigate to your project's **Settings ➔ Repository** @@ -45,19 +45,19 @@ that the `master` branch is protected by default. Since GitLab 8.11, we added another layer of branch protection which provides more granular management of protected branches. The "Developers can push" option was replaced by an "Allowed to push" setting which can be set to -allow/prohibit Masters and/or Developers to push to a protected branch. +allow/prohibit Maintainers and/or Developers to push to a protected branch. Using the "Allowed to push" and "Allowed to merge" settings, you can control the actions that different roles can perform with the protected branch. For example, you could set "Allowed to push" to "No one", and "Allowed to merge" -to "Developers + Masters", to require _everyone_ to submit a merge request for +to "Developers + Maintainers", to require _everyone_ to submit a merge request for changes going into the protected branch. This is compatible with workflows like the [GitLab workflow](../../workflow/gitlab_flow.md). However, there are workflows where that is not needed, and only protecting from force pushes and branch removal is useful. For those workflows, you can allow everyone with write access to push to a protected branch by setting -"Allowed to push" to "Developers + Masters". +"Allowed to push" to "Developers + Maintainers". You can set the "Allowed to push" and "Allowed to merge" options while creating a protected branch or afterwards by selecting the option you want from the @@ -66,7 +66,7 @@ dropdown list in the "Already protected" area.  If you don't choose any of those options while creating a protected branch, -they are set to "Masters" by default. +they are set to "Maintainers" by default. ## Wildcard protected branches @@ -101,7 +101,7 @@ all matching branches: From time to time, it may be required to delete or clean up branches that are protected. -User with [Master permissions][perm] and up can manually delete protected +User with [Maintainer permissions][perm] and up can manually delete protected branches via GitLab's web interface: 1. Visit **Repository > Branches** diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 0cb7aefdb2f..a5eaf2e9835 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -8,12 +8,12 @@ This feature evolved out of [Protected Branches](protected_branches.md) ## Overview -Protected tags will prevent anyone from updating or deleting the tag, as and will prevent creation of matching tags based on the permissions you have selected. By default, anyone without Master permission will be prevented from creating tags. +Protected tags will prevent anyone from updating or deleting the tag, as and will prevent creation of matching tags based on the permissions you have selected. By default, anyone without Maintainer permission will be prevented from creating tags. ## Configuring protected tags -To protect a tag, you need to have at least Master permission level. +To protect a tag, you need to have at least Maintainer permission level. 1. Navigate to the project's Settings -> Repository page diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index b8bac01959e..9034a9b5179 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -19,7 +19,7 @@ > - The exports are stored in a temporary [shared directory][tmp] and are deleted > every 24 hours by a specific worker. > - Group members will get exported as project members, as long as the user has -> master or admin access to the group where the exported project lives. An admin +> maintainer or admin access to the group where the exported project lives. An admin > in the import side is required to map the users, based on email or username. > Otherwise, a supplementary comment is left to mention the original author and > the MRs, notes or issues will be owned by the importer. diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index c9d2f8dc32d..212e271ce6f 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -1,7 +1,7 @@ # Project settings NOTE: **Note:** -Only project Masters and Admin users have the [permissions] to access a project +Only project Maintainers and Admin users have the [permissions] to access a project settings. You can adjust your [project](../index.md) settings by navigating @@ -74,7 +74,7 @@ To archive a project: #### Renaming a repository NOTE: **Note:** -Only project Masters and Admin users have the [permissions] to rename a +Only project Maintainers and Admin users have the [permissions] to rename a repository. Not to be confused with a project's name where it can also be changed from the [general project settings](#general-project-settings). @@ -98,7 +98,7 @@ Only project Owners and Admin users have the [permissions] to transfer a project You can transfer an existing project into a [group](../../group/index.md) if: -1. you have at least **Master** [permissions] to that group +1. you have at least **Maintainer** [permissions] to that group 1. you are an **Owner** of the project. Similarly, if you are an owner of a group, you can transfer any of its projects diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 23b67310d25..a7313082fac 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -131,7 +131,7 @@ There is room for more feedback and after the assigned person feels comfortable If the assigned person does not feel comfortable they can close the merge request without merging. In GitLab it is common to protect the long-lived branches (e.g. the master branch) so that normal developers [can't modify these protected branches](http://docs.gitlab.com/ce/permissions/permissions.html). -So if you want to merge it into a protected branch you assign it to someone with master authorizations. +So if you want to merge it into a protected branch you assign it to someone with maintainer authorizations. ## Issue tracking with GitLab flow diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index 0d592a6d43e..ae161e43233 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -144,7 +144,7 @@ git lfs unlock --id=123 ``` If for some reason you need to unlock a file that was not locked by you, -you can use the `--force` flag as long as you have a `master` access on +you can use the `--force` flag as long as you have a `maintainer` access on the project: ```bash diff --git a/doc/workflow/repository_mirroring.md b/doc/workflow/repository_mirroring.md index dbe63144e38..aaddbe4fbf5 100644 --- a/doc/workflow/repository_mirroring.md +++ b/doc/workflow/repository_mirroring.md @@ -1,6 +1,6 @@ # Repository mirroring -Repository Mirroring is a way to mirror repositories from external sources. +Repository mirroring is a way to mirror repositories from external sources. It can be used to mirror all branches, tags, and commits that you have in your repository. @@ -34,13 +34,200 @@ A few things/limitations to consider: - The Git LFS objects will not be synced. You'll need to push/pull them manually. -## Use-case +## Use cases +- You migrated to GitLab but still need to keep your project in another source. + In that case, you can simply set it up to mirror to GitLab (pull) and all the + essential history of commits, tags and branches will be available in your + GitLab instance. - You have old projects in another source that you don't use actively anymore, but don't want to remove for archiving purposes. In that case, you can create a push mirror so that your active GitLab repository can push its changes to the old location. +## Pulling from a remote repository **[STARTER]** + +>[Introduced][ee-51] in GitLab Enterprise Edition 8.2. + +You can set up a repository to automatically have its branches, tags, and commits +updated from an upstream repository. This is useful when a repository you're +interested in is located on a different server, and you want to be able to +browse its content and its activity using the familiar GitLab interface. + +When creating a new project, you can enable repository mirroring when you choose +to import the repository from "Any repo by URL". Enter the full URL of the Git +repository to pull from and click on the **Mirror repository** checkbox. + + + +For an existing project, you can set up mirror pulling by visiting your project's +**Settings ➔ Repository** and searching for the "Pull from a remote repository" +section. Check the "Mirror repository" box and hit **Save changes** at the bottom. +You have a few options to choose from one being the user who will be the author +of all events in the activity feed that are the result of an update. This user +needs to have at least [master access][perms] to the project. Another option is +whether you want to trigger builds for mirror updates. + + + +Since the repository on GitLab functions as a mirror of the upstream repository, +you are advised not to push commits directly to the repository on GitLab. +Instead, any commits should be pushed to the upstream repository, and will end +up in the GitLab repository automatically within a certain period of time +or when a [forced update](#forcing-an-update) is initiated. + +If you do manually update a branch in the GitLab repository, the branch will +become diverged from upstream, and GitLab will no longer automatically update +this branch to prevent any changes from being lost. + + + +### Trigger update using API **[STARTER]** + +>[Introduced][ee-3453] in GitLab Enterprise Edition 10.3. + +Pull mirroring uses polling to detect new branches and commits added upstream, +often many minutes afterwards. If you notify GitLab by [API][pull-api], updates +will be pulled immediately. + +Read the [Pull Mirror Trigger API docs][pull-api]. + +### Pull only protected branches **[STARTER]** + +>[Introduced][ee-3326] in GitLab Enterprise Edition 10.3. + +You can choose to only pull the protected branches from your remote repository to GitLab. + +To use this option go to your project's repository settings page under pull mirror. + +### Overwrite diverged branches **[STARTER]** + +>[Introduced][ee-4559] in GitLab Enterprise Edition 10.6. + +You can choose to always update your local branch with the remote version even +if your local version has diverged from the remote. + +To use this option go to your project's repository settings page under pull mirror. + +### Hard failure **[STARTER]** + +>[Introduced][ee-3117] in GitLab Enterprise Edition 10.2. + +Once a mirror gets retried 14 times in a row, it will get marked as hard failed, +this will become visible in either the project main dashboard or in the +pull mirror settings page. + + + + + +When a project is hard failed, it will no longer get picked up for mirroring. +A user can resume the project mirroring again by either [forcing an update](#forcing-an-update) +or by changing the import URL in repository settings. + +### SSH authentication **[STARTER]** + +> [Introduced][ee-2551] in GitLab Starter 9.5 + +If you're mirroring over SSH (i.e., an `ssh://` URL), you can authenticate using +password-based authentication, just as over HTTPS, but you can also use public +key authentication. This is often more secure than password authentication, +especially when the source repository supports [Deploy Keys][deploy-key]. + +To get started, navigate to **Settings ➔ Repository ➔ Pull from a remote repository**, +enable mirroring (if not already enabled) and enter an `ssh://` URL. + +> **NOTE**: SCP-style URLs, e.g., `git@example.com:group/project.git`, are not +supported at this time. + +Entering the URL adds two features to the page - `Fingerprints` and +`SSH public key authentication`: + + + +SSH authentication is mutual. You have to prove to the server that you're +allowed to access the repository, but the server also has to prove to *you* that +it's who it claims to be. You provide your credentials as a password or public +key. The server that the source repository resides on provides its credentials +as a "host key", the fingerprint of which needs to be verified manually. + +Press the `Detect host keys` button. GitLab will fetch the host keys from the +server, and display the fingerprints to you: + + + +You now need to verify that the fingerprints are those you expect. GitLab.com +and other code hosting sites publish their fingerprints in the open for you +to check: + +* [AWS CodeCommit](http://docs.aws.amazon.com/codecommit/latest/userguide/regions.html#regions-fingerprints) +* [Bitbucket](https://confluence.atlassian.com/bitbucket/use-the-ssh-protocol-with-bitbucket-cloud-221449711.html#UsetheSSHprotocolwithBitbucketCloud-KnownhostorBitbucket%27spublickeyfingerprints) +* [GitHub](https://help.github.com/articles/github-s-ssh-key-fingerprints/) +* [GitLab.com](https://about.gitlab.com/gitlab-com/settings/#ssh-host-keys-fingerprints) +* [Launchpad](https://help.launchpad.net/SSHFingerprints) +* [Savannah](http://savannah.gnu.org/maintenance/SshAccess/) +* [SourceForge](https://sourceforge.net/p/forge/documentation/SSH%20Key%20Fingerprints/) + +Other providers will vary. If you're running on-premises GitLab, or otherwise +have access to the source server, you can securely gather the key fingerprints: + +``` +$ cat /etc/ssh/ssh_host*pub | ssh-keygen -E md5 -l -f - +256 MD5:f4:28:9f:23:99:15:21:1b:bf:ed:1f:8e:a0:76:b2:9d root@example.com (ECDSA) +256 MD5:e6:eb:45:8a:3c:59:35:5f:e9:5b:80:12:be:7e:22:73 root@example.com (ED25519) +2048 MD5:3f:72:be:3d:62:03:5c:62:83:e8:6e:14:34:3a:85:1d root@example.com (RSA) +``` + +(You may need to exclude `-E md5` for some older versions of SSH). + +If you're an SSH expert and already have a `known_hosts` file you'd like to use +unaltered, then you can skip these steps. Just press the "Show advanced" button +and paste in the file contents: + + + +Once you've **carefully verified** that all the fingerprints match your trusted +source, you can press `Save changes`. This will record the host keys, along with +the person who verified them (you!) and the date: + + + +When pulling changes from the source repository, GitLab will now check that at +least one of the stored host keys matches before connecting. This can prevent +malicious code from being injected into your mirror, or your password being +stolen! + +To use SSH public key authentication, you'll also need to choose that option +from the authentication methods dropdown. GitLab will generate a 4096-bit RSA +key and display the public component of that key to you: + + + +You then need to add the public SSH key to the source repository configuration. +If the source is hosted on GitLab, you should add it as a [Deploy Key][deploy-key]. +Other sources may require you to add the key to your user's `authorized_keys` +file - just paste the entire `ssh-rsa AAA.... user@host` block into the file on +its own line and save it. + +Once the public key is set up on the source repository, press `Save changes` and your +mirror will begin working. + +If you need to change the key at any time, you can press the `Regenerate key` +button to do so. You'll have to update the source repository with the new key +to keep the mirror running. + +### How it works + +Once you activate the pull mirroring feature, the mirror will be inserted into +a queue. A scheduler will start every minute and schedule a fixed amount of +mirrors for update, based on the configured maximum capacity. + +If the mirror successfully updates it will be enqueued once again with a small +backoff period. + +If the mirror fails (eg: branch diverged from upstream), the project's backoff +period will be penalized each time it fails up to a maximum amount of time. + ## Pushing to a remote repository **[STARTER]** >[Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/249) in @@ -83,7 +270,7 @@ To use this option go to your project's repository settings page under push mirr To set up a mirror from GitLab to GitHub, you need to follow these steps: 1. Create a [GitHub personal access token](https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/) with the "public_repo" box checked: - +  1. Fill in the "Git repository URL" with the personal access token replacing the password `https://GitHubUsername:GitHubPersonalAccessToken@github.com/group/project.git`: @@ -94,7 +281,7 @@ To set up a mirror from GitLab to GitHub, you need to follow these steps: 1. And either wait or trigger the "Update Now" button:  - + ## Forcing an update While mirrors are scheduled to update automatically, you can always force an update @@ -105,7 +292,60 @@ by using the **Update now** button which is exposed in various places: - in the tags page - in the **Mirror repository** settings page +## Bidirectional mirroring + +CAUTION: **Warning:** +There is no bidirectional support without conflicts. If you +configure a repository to pull and push to a second remote, there is no +guarantee that it will update correctly on both remotes. If you configure +a repository for bidirectional mirroring, you should consider when conflicts +occur who and how they will be resolved. + +Rewriting any mirrored commit on either remote will cause conflicts and +mirroring to fail. This can be prevented by [only pulling protected branches]( +#pull-only-protected-branches) and [only pushing protected branches]( +#push-only-protected-branches). You should protect the branches you wish to +mirror on both remotes to prevent conflicts caused by rewriting history. + +Bidirectional mirroring also creates a race condition where commits to the same +branch in close proximity will cause conflicts. The race condition can be +mitigated by reducing the mirroring delay by using a Push event webhook to +trigger an immediate pull to GitLab. Push mirroring from GitLab is rate limited +to once per minute when only push mirroring protected branches. + +It may be possible to implement a locking mechanism using the server-side +`pre-receive` hook to prevent the race condition. Read about [configuring +custom Git hooks][hooks] on the GitLab server. + +### Mirroring with Perforce via GitFusion + +CAUTION: **Warning:** +Bidirectional mirroring should not be used as a permanent +configuration. There is no bidirectional mirroring without conflicts. +Refer to [Migrating from Perforce Helix][perforce] for alternative migration +approaches. + +GitFusion provides a Git interface to Perforce which can be used by GitLab to +bidirectionally mirror projects with GitLab. This may be useful in some +situations when migrating from Perforce to GitLab where overlapping Perforce +workspaces cannot be migrated simultaneously to GitLab. + +If using mirroring with Perforce you should only mirror protected branches. +Perforce will reject any pushes that rewrite history. It is recommended that +only the fewest number of branches are mirrored due to the performance +limitations of GitFusion. + +[ee-51]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/51 +[ee-2551]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2551 +[ee-3117]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3117 +[ee-3326]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3326 [ee-3350]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3350 +[ee-3453]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3453 +[ee-4559]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4559 [ce-18715]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18715 [perms]: ../user/permissions.md - +[hooks]: ../administration/custom_hooks.md +[deploy-key]: ../ssh/README.md#deploy-keys +[webhook]: ../user/project/integrations/webhooks.md#push-events +[pull-api]: ../api/projects.md#start-the-pull-mirroring-process-for-a-project +[perforce]: ../user/project/import/perforce.md diff --git a/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png b/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png Binary files differnew file mode 100644 index 00000000000..333648942f8 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png b/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png Binary files differnew file mode 100644 index 00000000000..45c9bce0889 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png Binary files differnew file mode 100644 index 00000000000..99d429a1802 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png Binary files differnew file mode 100644 index 00000000000..0ab07afa3cc --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_new_project.png b/doc/workflow/repository_mirroring/repository_mirroring_new_project.png Binary files differnew file mode 100644 index 00000000000..43bf304838f --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_new_project.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png Binary files differnew file mode 100644 index 00000000000..5da5a7436bb --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png Binary files differnew file mode 100644 index 00000000000..4b9085302a1 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png Binary files differnew file mode 100644 index 00000000000..8c2efdafa43 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png b/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png Binary files differnew file mode 100644 index 00000000000..93f3a532a0e --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png b/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png Binary files differnew file mode 100644 index 00000000000..6997ad511d9 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png |
