diff options
Diffstat (limited to 'doc')
318 files changed, 11745 insertions, 2843 deletions
diff --git a/doc/README.md b/doc/README.md index af573a3eb34..1cdb5bc7b47 100644 --- a/doc/README.md +++ b/doc/README.md @@ -23,7 +23,7 @@ No matter how you use GitLab, we have documentation for you. | Essential Documentation | Essential Documentation | |:-------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------| | [**User Documentation**](user/index.md)<br/>Discover features and concepts for GitLab users. | [**Administrator documentation**](administration/index.md)<br/>Everything GitLab self-managed administrators need to know. | -| [**Contributing to GitLab**](#contributing-to-gitlab)<br/>At GitLab, everyone can contribute! | [**New to Git and GitLab?**](#new-to-git-and-gitlab)<br/>We have resources to get you started. | +| [**Contributing to GitLab**](#contributing-to-gitlab)<br/>At GitLab, everyone can contribute! | [**New to Git and GitLab?**](#new-to-git-and-gitlab)<br/>We have the resources to get you started. | | [**Building an integration with GitLab?**](#building-an-integration-with-gitlab)<br/>Consult our automation and integration documentation. | [**Coming to GitLab from another platform?**](#coming-to-gitlab-from-another-platform)<br/>Consult our handy guides. | | [**Install GitLab**](https://about.gitlab.com/install/)<br/>Installation options for different platforms. | [**Customers**](subscriptions/index.md)<br/>Information for new and existing customers. | | [**Update GitLab**](update/README.md)<br/>Update your GitLab self-managed instance to the latest version. | [**GitLab Releases**](https://about.gitlab.com/releases/)<br/>What's new in GitLab. | @@ -42,6 +42,7 @@ Have a look at some of our most popular documentation resources: | [Kubernetes integration](user/project/clusters/index.md) | Use GitLab with Kubernetes. | | [SSH authentication](ssh/README.md) | Secure your network communications. | | [Using Docker images](ci/docker/using_docker_images.md) | Build and test your applications with Docker. | +| [GraphQL](api/graphql/index.md) | Explore GitLab's GraphQL API. | ## The entire DevOps Lifecycle @@ -87,7 +88,7 @@ The following documentation relates to the DevOps **Manage** stage: | Manage Topics | Description | |:--------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [Authentication and<br/>Authorization](administration/auth/README.md) **(CORE ONLY)** | Supported authentication and authorization providers. | -| [GitLab Cycle Analytics](user/project/cycle_analytics.md) | Measure the time it takes to go from an [idea to production](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab) for each project you have. | +| [GitLab Cycle Analytics](user/project/cycle_analytics.md) | Measure the time it takes to go from an [idea to production](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab) for each project you have. | | [Instance Statistics](user/instance_statistics/index.md) | Discover statistics on how many GitLab features you use and user activity. | <div align="right"> @@ -314,6 +315,7 @@ The following documentation relates to the DevOps **Configure** stage: | [Protected variables](ci/variables/README.md#protected-environment-variables) | Restrict variables to protected branches and tags. | | [Serverless](user/project/clusters/serverless/index.md) | Run serverless workloads on Kubernetes. | | [Slack slash commands](user/project/integrations/slack_slash_commands.md) | Enable and use slash commands from within Slack. | +| [Manage your infrastructure with Terraform](user/infrastructure/index.md) | Manage your infrastructure as you run your CI/CD pipeline. | <div align="right"> <a type="button" class="btn btn-default" href="#overview"> @@ -376,7 +378,7 @@ We have the following documentation to rapidly uplift your GitLab knowledge: | Topic | Description | |:-----------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------| | [GitLab basics guides](gitlab-basics/README.md) | Start working on the command line and with GitLab. | -| [GitLab workflow overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) | Enhance your workflow with the best of GitLab Workflow. | +| [GitLab workflow overview](https://about.gitlab.com/blog/2016/10/25/gitlab-workflow-an-overview/) | Enhance your workflow with the best of GitLab Workflow. | | [Get started with GitLab CI/CD](ci/quick_start/README.md) | Quickly implement GitLab CI/CD. | | [Auto DevOps](topics/autodevops/index.md) | Learn more about GitLab's Auto DevOps. | | [GitLab Markdown](user/markdown.md) | GitLab's advanced formatting system (GitLab Flavored Markdown) | @@ -411,7 +413,7 @@ Learn more about using Git, and using Git with GitLab: | Topic | Description | |:----------------------------------------------------------------------------|:---------------------------------------------------------------------------| | [Git](topics/git/index.md) | Getting started with Git, branching strategies, Git LFS, and advanced use. | -| [Git cheatsheet](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) | Download a PDF describing the most used Git operations. | +| [Git cheat sheet](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) | Download a PDF describing the most used Git operations. | | [GitLab Flow](topics/gitlab_flow.md) | Explore the best of Git with the GitLab Flow strategy. | <div align="right"> diff --git a/doc/administration/auth/ldap-ee.md b/doc/administration/auth/ldap-ee.md index e2894318fe5..34fd97a24ee 100644 --- a/doc/administration/auth/ldap-ee.md +++ b/doc/administration/auth/ldap-ee.md @@ -10,13 +10,16 @@ This section documents LDAP features specific to to GitLab Enterprise Edition For documentation relevant to both Community Edition and Enterprise Edition, see the main [LDAP documentation](ldap.md). +NOTE: **Note:** +[Microsoft Active Directory Trusts](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc771568(v=ws.10)) are not supported + ## Use cases - User sync: Once a day, GitLab will update users against LDAP. - Group sync: Once an hour, GitLab will update group membership based on LDAP group members. -## Multiple LDAP servers **(STARTER ONLY)** +## Multiple LDAP servers With GitLab Enterprise Edition Starter, you can configure multiple LDAP servers that your GitLab instance will connect to. @@ -107,12 +110,23 @@ following. 1. [Restart GitLab][restart] for the changes to take effect. -To take advantage of group sync, group owners or maintainers will need to create an -LDAP group link in their group **Settings > LDAP Groups** page. +To take advantage of group sync, group owners or maintainers will need to [create one +or more LDAP group links](#adding-group-links). + +### Adding group links + +Once [group sync has been configured](#group-sync) on the instance, one or more LDAP +groups can be linked to a GitLab group to grant their members access to its +contents. + +Group owners or maintainers can add and use LDAP group links by: -Multiple LDAP groups and [filters](#filters-premium-only) can be linked with -a single GitLab group. When the link is created, an access level/role is -specified (Guest, Reporter, Developer, Maintainer, or Owner). +1. Navigating to the group's **Settings > LDAP Synchronization** page. Here, one or more + LDAP groups and [filters](#filters-premium-only) can be linked to this GitLab group, + each one with a configured [permission level](../../user/permissions.md#group-members-permissions) + for its members. +1. Updating the group's membership by navigating to the group's **Settings > Members** + page and clicking **Sync now**. ### Filters **(PREMIUM ONLY)** diff --git a/doc/administration/auth/smartcard.md b/doc/administration/auth/smartcard.md index eb63df6b482..84ddd4278ab 100644 --- a/doc/administration/auth/smartcard.md +++ b/doc/administration/auth/smartcard.md @@ -6,6 +6,16 @@ type: reference GitLab supports authentication using smartcards. +## Existing password authentication + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/33669) in GitLab 12.6. + +By default, existing users can continue to log in with a username and password when smartcard +authentication is enabled. + +To force existing users to use only smartcard authentication, +[disable username and password authentication](../../user/admin_area/settings/sign_in_restrictions.md#password-authentication-enabled). + ## Authentication methods GitLab supports two authentication methods: @@ -51,10 +61,14 @@ This is an experimental feature. Smartcard authentication against local database change or be removed completely in future releases. To use a smartcard with an X.509 certificate to authenticate against a local -database with GitLab, at least one of the `subjectAltName` (SAN) extensions -need to define the user identity (`email`) within the GitLab instance (`URI`). +database with GitLab, in: -`URI`: needs to match `Gitlab.config.host.gitlab`. +- GitLab 12.4 and later, at least one of the `subjectAltName` (SAN) extensions + need to define the user identity (`email`) within the GitLab instance (`URI`). + `URI`: needs to match `Gitlab.config.host.gitlab`. +- From [GitLab 12.5](https://gitlab.com/gitlab-org/gitlab/issues/33907), + if your certificate contains only **one** SAN email entry, you don't need to + add or modify it to match the `email` with the `URI`. For example: @@ -80,10 +94,7 @@ Certificate: ### Authentication against an LDAP server -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/7693) in -[GitLab Premium](https://about.gitlab.com/pricing/) 11.8 as an experimental -feature. Smartcard authentication against an LDAP server may change or be -removed completely in future releases. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/7693) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8 as an experimental feature. Smartcard authentication against an LDAP server may change or be removed completely in future releases. GitLab implements a standard way of certificate matching following [RFC4523](https://tools.ietf.org/html/rfc4523). It uses the diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 246addb6dc9..44e1cc8059a 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -16,3 +16,4 @@ GitLab’s [security features](../security/README.md) may also help you meet rel |**[LDAP group sync filters](auth/ldap-ee.md#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions.|Premium+|| |**[Audit logs](audit_events.md)**<br>To maintain the integrity of your code, GitLab Enterprise Edition Premium gives admins the ability to view any modifications made within the GitLab server in an advanced audit log system, so you can control, analyze and track every change.|Premium+|| |**[Auditor users](auditor_users.md)**<br>Auditor users are users who are given read-only access to all projects, groups, and other resources on the GitLab instance.|Premium+|| +|**[Credentials inventory](../user/admin_area/credentials_inventory.md)**<br>With a credentials inventory, GitLab administrators can keep track of the credentials used by all of the users in their GitLab instance. |Ultimate|| diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 0702e0aa141..437c9db1630 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -1,4 +1,4 @@ -# Custom server-side Git hooks +# Custom server-side Git hooks **(CORE ONLY)** NOTE: **Note:** Custom Git hooks must be configured on the filesystem of the GitLab diff --git a/doc/administration/external_pipeline_validation.md b/doc/administration/external_pipeline_validation.md new file mode 100644 index 00000000000..19d4de3b705 --- /dev/null +++ b/doc/administration/external_pipeline_validation.md @@ -0,0 +1,103 @@ +# External Pipeline Validation + +You can use an external service for validating a pipeline before it's created. + +CAUTION: **Warning:** +This is an experimental feature and subject to change without notice. + +## Usage + +GitLab will send a POST request to the external service URL with the pipeline +data as payload. GitLab will then invalidate the pipeline based on the response +code. If there's an error or the request times out, the pipeline will not be +invalidated. + +Response Code Legend: + +- `200` - Accepted +- `4xx` - Not Accepted +- Other Codes - Accepted and Logged + +## Configuration + +Set the `EXTERNAL_VALIDATION_SERVICE_URL` to the external service url. + +## Payload Schema + +```json +{ + "type": "object", + "required" : [ + "project", + "user", + "pipeline", + "builds" + ], + "properties" : { + "project": { + "type": "object", + "required": [ + "id", + "path" + ], + "properties": { + "id": { "type": "integer" }, + "path": { "type": "string" } + } + }, + "user": { + "type": "object", + "required": [ + "id", + "username", + "email" + ], + "properties": { + "id": { "type": "integer" }, + "username": { "type": "string" }, + "email": { "type": "string" } + } + }, + "pipeline": { + "type": "object", + "required": [ + "sha", + "ref", + "type" + ], + "properties": { + "sha": { "type": "string" }, + "ref": { "type": "string" }, + "type": { "type": "string" } + } + }, + "builds": { + "type": "array", + "items": { + "type": "object", + "required": [ + "name", + "stage", + "image", + "services", + "script" + ], + "properties": { + "name": { "type": "string" }, + "stage": { "type": "string" }, + "image": { "type": ["string", "null"] }, + "services": { + "type": ["array", "null"], + "items": { "type": "string" } + }, + "script": { + "type": "array", + "items": { "type": "string" } + } + } + } + } + }, + "additionalProperties": false +} +``` diff --git a/doc/administration/geo/replication/database.md b/doc/administration/geo/replication/database.md index f7da4e14e9d..72c3692716b 100644 --- a/doc/administration/geo/replication/database.md +++ b/doc/administration/geo/replication/database.md @@ -35,7 +35,7 @@ The following guide assumes that: - You are using Omnibus and therefore you are using PostgreSQL 9.6 or later which includes the [`pg_basebackup` tool](https://www.postgresql.org/docs/9.6/app-pgbasebackup.html) and improved - [Foreign Data Wrapper][FDW](https://www.postgresql.org/docs/9.6/postgres-fdw.html) support. + [Foreign Data Wrapper](https://www.postgresql.org/docs/9.6/postgres-fdw.html) support. - You have a **primary** node already set up (the GitLab server you are replicating from), running Omnibus' PostgreSQL (or equivalent version), and you have a new **secondary** server set up with the same versions of the OS, @@ -155,8 +155,8 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o | `postgresql['md5_auth_cidr_addresses']` | **Secondary** node's public or VPC private addresses. | If you are using Google Cloud Platform, SoftLayer, or any other vendor that - provides a virtual private cloud (VPC) you can use the **secondary** node's private - address (corresponds to "internal address" for Google Cloud Platform) for + provides a virtual private cloud (VPC) you can use the **primary** and **secondary** nodes + private addresses (corresponds to "internal address" for Google Cloud Platform) for `postgresql['md5_auth_cidr_addresses']` and `postgresql['listen_address']`. The `listen_address` option opens PostgreSQL up to network connections with the interface diff --git a/doc/administration/geo/replication/docker_registry.md b/doc/administration/geo/replication/docker_registry.md index 12cd39c0213..95db766e482 100644 --- a/doc/administration/geo/replication/docker_registry.md +++ b/doc/administration/geo/replication/docker_registry.md @@ -51,7 +51,7 @@ We need to make Docker Registry send notification events to the 'threshold' => 5, 'backoff' => '1s', 'headers' => { - 'Authorization' => ['<replace_with_a_secret_token>'] + 'Authorization' => ['<replace_with_a_secret_token>'] # An alphanumeric string. Case sensitive and must start with a letter. } } ] @@ -59,7 +59,7 @@ We need to make Docker Registry send notification events to the NOTE: **Note:** If you use an external Registry (not the one integrated with GitLab), you must add - these settings to its configuration. In this case, you will also have to specify + these settings to its configuration yourself. In this case, you will also have to specify notification secret in `registry.notification_secret` section of `/etc/gitlab/gitlab.rb` file. @@ -100,7 +100,7 @@ generate a short-lived JWT that is pull-only-capable to access the ```ruby gitlab_rails['geo_registry_replication_enabled'] = true - gitlab_rails['geo_registry_replication_primary_api_url'] = 'http://primary.example.com:5000/' # internal address to the primary registry, will be used by GitLab to directly communicate with primary registry API + gitlab_rails['geo_registry_replication_primary_api_url'] = 'http://primary.example.com:4567/' # Primary registry address, it will be used by the secondary node to directly communicate to primary registry ``` 1. Reconfigure the **secondary** node for the change to take effect: diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index d2fe02abbab..0a2602261d1 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -115,10 +115,12 @@ Any **secondary** nodes should point only to read-only instances. #### Can Geo detect the current node correctly? -Geo finds the current machine's name in `/etc/gitlab/gitlab.rb` by first looking -for `gitlab_rails['geo_node_name']`. If it is not defined, then it defaults to -the external URL defined in e.g. `external_url "http://gitlab.example.com"`. To -get a machine's name, run: +Geo finds the current machine's name in `/etc/gitlab/gitlab.rb` by: + +- Using the `gitlab_rails['geo_node_name']` setting. +- If that is not defined, using the `external_url` setting. + +To get a machine's name, run: ```sh sudo gitlab-rails runner "puts GeoNode.current_node_name" @@ -401,6 +403,8 @@ to start again from scratch, there are a few steps that can help you: mv /var/opt/gitlab/gitlab-rails/uploads /var/opt/gitlab/gitlab-rails/uploads.old mkdir -p /var/opt/gitlab/gitlab-rails/uploads + + gitlab-ctl start geo-postgresql ``` Reconfigure in order to recreate the folders and make sure permissions and ownership @@ -699,3 +703,17 @@ See ["Foreign Data Wrapper (FDW) is not configured" error?](#foreign-data-wrappe This can be caused by orphaned records in the project registry. You can clear them [using a Rake task](../../../administration/raketasks/geo.md#remove-orphaned-project-registries). + +### Geo Admin Area returns 404 error for a secondary node + +Sometimes `sudo gitlab-rake gitlab:geo:check` indicates that the **secondary** node is +healthy, but a 404 error for the **secondary** node is returned in the Geo Admin Area on +the **primary** node. + +To resolve this issue: + +- Try restarting the **secondary** using `sudo gitlab-ctl restart`. +- Check `/var/log/gitlab/gitlab-rails/geo.log` to see if the **secondary** node is + using IPv6 to send its status to the **primary** node. If it is, add an entry to + the **primary** node using IPv4 in the `/etc/hosts` file. Alternatively, you should + [enable IPv6 on the primary node](https://docs.gitlab.com/omnibus/settings/nginx.html#setting-the-nginx-listen-address-or-addresses). diff --git a/doc/administration/gitaly/img/gitlab_gitaly_version_mismatch_v12_4.png b/doc/administration/gitaly/img/gitlab_gitaly_version_mismatch_v12_4.png Binary files differnew file mode 100644 index 00000000000..4d2c5cdb00c --- /dev/null +++ b/doc/administration/gitaly/img/gitlab_gitaly_version_mismatch_v12_4.png diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 82283650070..9218ffa4006 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -221,6 +221,8 @@ Git operations in GitLab will result in an API error. ```toml listen_addr = '0.0.0.0:8075' + internal_socket_dir = '/var/opt/gitlab/gitaly' + [auth] token = 'abc123secret' @@ -332,7 +334,10 @@ When you tail the Gitaly logs on your Gitaly server you should see requests coming in. One sure way to trigger a Gitaly request is to clone a repository from your GitLab server over HTTP. -DANGER: **Danger:** If you have [custom server-side Git hooks](../custom_hooks.md#custom-server-side-git-hooks) configured, either per repository or globally, you must move these to the Gitaly node. If you have multiple Gitaly nodes, copy your custom hook(s) to all nodes. +DANGER: **Danger:** +If you have [custom server-side Git hooks](../custom_hooks.md) configured, +either per repository or globally, you must move these to the Gitaly node. +If you have multiple Gitaly nodes, copy your custom hook(s) to all nodes. ### Disabling the Gitaly service in a cluster environment @@ -714,8 +719,115 @@ result as you did in the beginning: Note that `enforced="true"`, meaning that authentication is being enforced. +## Direct Git access in GitLab Rails + +Also known as "the Rugged patches". + +### History + +Before Gitaly existed, the things that are now Gitaly clients used to +access Git repositories directly. Either on a local disk in the case of +e.g. a single-machine Omnibus GitLab installation, or via NFS in the +case of a horizontally scaled GitLab installation. + +Besides running plain `git` commands, in GitLab Rails we also used to +use a Ruby gem (library) called +[Rugged](https://github.com/libgit2/rugged). Rugged is a wrapper around +[libgit2](https://libgit2.org/), a stand-alone implementation of Git in +the form of a C library. + +Over time it has become clear to use that Rugged, and particularly +Rugged in combination with the [Unicorn](https://bogomips.org/unicorn/) +web server, is extremely efficient. Because libgit2 is a *library* and +not an external process, there was very little overhead between GitLab +application code that tried to look up data in Git repositories, and the +Git implementation itself. + +Because Rugged+Unicorn was so efficient, GitLab's application code ended +up with lots of duplicate Git object lookups (like looking up the +`master` commmit a dozen times in one request). We could write +inefficient code without being punished for it. + +When we migrated these Git lookups to Gitaly calls, we were suddenly +getting a much higher fixed cost per Git lookup. Even when Gitaly is +able to re-use an already-running `git` process to look up e.g. a commit +you still have the cost of a network roundtrip to Gitaly, and within +Gitaly a write/read roundtrip on the Unix pipes that connect Gitaly to +the `git` process. + +Using GitLab.com performance as our yardstick, we pushed down the number +of Gitaly calls per request until the loss of Rugged's efficiency was no +longer felt. It also helped that we run Gitaly itself directly on the +Git file severs, rather than via NFS mounts: this gave us a speed boost +that counteracted the negative effect of not using Rugged anymore. + +Unfortunately, some *other* deployments of GitLab could not ditch NFS +like we did on GitLab.com and they got the worst of both worlds: the +slowness of NFS and the increased inherent overhead of Gitaly. + +As a performance band-aid for these stuck-on-NFS deployments, we +re-introduced some of the old Rugged code that got deleted from +GitLab Rails during the Gitaly migration project. These pieces of +re-introduced code are informally referred to as "the Rugged patches". + +### Activation of direct Git access in GitLab Rails + +The Ruby methods that perform direct Git access are hidden behind [feature +flags](../../development/gitaly.md#legacy-rugged-code). These feature +flags are off by default. It is not good if you need to know about +feature flags to get the best performance so in a second iteration, we +added an automatic mechanism that will enable direct Git access. + +When GitLab Rails calls a function that has a Rugged patch it performs +two checks. The result of both of these checks is cached. + +1. Is the feature flag for this patch set in the database? If so, do + what the feature flag says. +1. If the feature flag is not set (i.e. neither true nor false), try to + see if we can access filesystem underneath the Gitaly server + directly. If so, use the Rugged patch. + +To see if GitLab Rails can access the repo filesystem directly, we use +the following heuristic: + +- Gitaly ensures that the filesystem has a metadata file in its root + with a UUID in it. +- Gitaly reports this UUID to GitLab Rails via the `ServerInfo` RPC. +- GitLab Rails tries to read the metadata file directly. If it exists, + and if the UUID's match, assume we have direct access. + +Because of the way the UUID check works, and because Omnibus GitLab will +fill in the correct repository paths in the GitLab Rails config file +`config/gitlab.yml`, **direct Git access in GitLab Rails is on by default in +Omnibus**. + +### Plans to remove direct Git access in GitLab Rails + +For the sake of removing complexity it is desirable that we get rid of +direct Git access in GitLab Rails. For as long as some GitLab installations are stuck +with Git repositories on slow NFS, however, we cannot just remove them. + +There are two prongs to our efforts to remove direct Git access in GitLab Rails: + +1. Reduce the number of (inefficient) Gitaly queries made by + GitLab Rails. +1. Persuade everybody who runs a Highly Available / horizontally scaled + GitLab installation to move off of NFS. + +The second prong is the only real solution. For this we need [Gitaly +HA](https://gitlab.com/groups/gitlab-org/-/epics?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=Gitaly%20HA), +which is still under development as of December 2019. + ## Troubleshooting Gitaly +### Checking versions when using standalone Gitaly nodes + +When using standalone Gitaly nodes, you must make sure they are the same version +as GitLab to ensure full compatibility. Check **Admin Area > Gitaly Servers** on +your GitLab instance and confirm all Gitaly Servers are `Up to date`. + + + ### `gitaly-debug` The `gitaly-debug` command provides "production debugging" tools for Gitaly and Git diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index 83c9aa3f013..6193a40ac4f 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -20,21 +20,19 @@ for updates and roadmap. ### Architecture -For this document, the following network topology is assumed: +The most common architecture for Praefect is simplified in the diagram below: ```mermaid graph TB - GitLab --> Gitaly; GitLab --> Praefect; - Praefect --> Praefect-Gitaly-1; - Praefect --> Praefect-Gitaly-2; - Praefect --> Praefect-Gitaly-3; + Praefect --> Gitaly-1; + Praefect --> Gitaly-2; + Praefect --> Gitaly-3; ``` Where `GitLab` is the collection of clients that can request Git operations. -`Gitaly` is a Gitaly server before using Praefect. The Praefect node has three -storage nodes attached. Praefect itself doesn't store data, but connects to -three Gitaly nodes, `Praefect-Gitaly-1`, `Praefect-Gitaly-2`, and `Praefect-Gitaly-3`. +The Praefect node has threestorage nodes attached. Praefect itself doesn't +store data, but connects to three Gitaly nodes, `Gitaly-1`, `Gitaly-2`, and `Gitaly-3`. Praefect may be enabled on its own node or can be run on the GitLab server. In the example below we will use a separate server, but the optimal configuration @@ -49,41 +47,43 @@ purposes. In this setup guide we will start by configuring Praefect, then its child Gitaly nodes, and lastly the GitLab server configuration. +#### Secrets + +We need to manage the following secrets and make them match across hosts: + +1. `GITLAB_SHELL_SECRET_TOKEN`: this is used by Git hooks to make + callback HTTP API requests to GitLab when accepting a Git push. This + secret is shared with GitLab Shell for legacy reasons. +1. `PRAEFECT_EXTERNAL_TOKEN`: repositories hosted on your Praefect + cluster can only be accessed by Gitaly clients that carry this + token. +1. `PRAEFECT_INTERNAL_TOKEN`: this token is used for replication + traffic inside your Praefect cluster. This is distinct from + `PRAEFECT_EXTERNAL_TOKEN` because Gitaly clients must not be able to + access internal nodes of the Praefect cluster directly; that could + lead to data loss. + #### Praefect On the Praefect node we disable all other services, including Gitaly. We list each -Gitaly node that will be connected to Praefect under `praefect['storage_nodes']`. +Gitaly node that will be connected to Praefect as members of the `praefect` hash in `praefect['virtual_storages']`. -In the example below, the Gitaly nodes are named `praefect-gitaly-N`. Note that one +In the example below, the Gitaly nodes are named `gitaly-N`. Note that one node is designated as primary by setting the primary to `true`. -`praefect['auth_token']` is the token used to authenticate with the GitLab server, -just like `gitaly['auth_token']` is used for a standard Gitaly server. - -The `token` field under each storage listed in `praefect['storage_nodes']` is used -to authenticate each child Gitaly node with Praefect. - ```ruby -# /etc/gitlab/gitlab.rb +# /etc/gitlab/gitlab.rb on praefect server # Avoid running unnecessary services on the Gitaly server postgresql['enable'] = false redis['enable'] = false nginx['enable'] = false prometheus['enable'] = false +grafana['enable'] = false unicorn['enable'] = false sidekiq['enable'] = false gitlab_workhorse['enable'] = false gitaly['enable'] = false -``` - -##### Set up Praefect and its Gitaly nodes - -In the example below, the Gitaly nodes are named `praefect-git-X`. Note that one node is designated as -primary, by setting the primary to `true`: - -```ruby -# /etc/gitlab/gitlab.rb # Prevent database connections during 'gitlab-ctl reconfigure' gitlab_rails['rake_cache_clear'] = false @@ -95,25 +95,27 @@ praefect['enable'] = true # firewalls to restrict access to this address/port. praefect['listen_addr'] = '0.0.0.0:2305' -# virtual_storage_name must match the same storage name given to praefect in git_data_dirs -praefect['virtual_storage_name'] = 'praefect' - -# Authentication token to ensure only authorized servers can communicate with -# Praefect server -praefect['auth_token'] = 'praefect-token' -praefect['storage_nodes'] = { - 'praefect-gitaly-1' => { - 'address' => 'tcp://praefect-git-1.internal:8075', - 'token' => 'praefect-gitaly-token', - 'primary' => true - }, - 'praefect-gitaly-2' => { - 'address' => 'tcp://praefect-git-2.internal:8075', - 'token' => 'praefect-gitaly-token' - }, - 'praefect-gitaly-3' => { - 'address' => 'tcp://praefect-git-3.internal:8075', - 'token' => 'praefect-gitaly-token' +# Replace PRAEFECT_EXTERNAL_TOKEN with a real secret +praefect['auth_token'] = 'PRAEFECT_EXTERNAL_TOKEN' + +# Replace each instance of PRAEFECT_INTERNAL_TOKEN below with a real +# secret, distinct from PRAEFECT_EXTERNAL_TOKEN. +# Name of storage hash must match storage name in git_data_dirs on GitLab server. +praefect['virtual_storages'] = { + 'praefect' => { + 'gitaly-1' => { + 'address' => 'tcp://gitaly-1.internal:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN', + 'primary' => true + }, + 'gitaly-2' => { + 'address' => 'tcp://gitaly-2.internal:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN' + }, + 'gitaly-3' => { + 'address' => 'tcp://gitaly-3.internal:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN' + } } } ``` @@ -126,37 +128,40 @@ Next we will configure each Gitaly server assigned to Praefect. Configuration f is the same as a normal standalone Gitaly server, except that we use storage names and auth tokens from Praefect instead of GitLab. -Below is an example configuration for `praefect-gitaly-1`, the only difference for the +Below is an example configuration for `gitaly-1`, the only difference for the other Gitaly nodes is the storage name under `git_data_dirs`. -Note that `gitaly['auth_token']` matches the `token` value listed under `praefect['storage_nodes']` +Note that `gitaly['auth_token']` matches the `token` value listed under `praefect['virtual_storages']` on the Praefect node. ```ruby -# /etc/gitlab/gitlab.rb +# /etc/gitlab/gitlab.rb on gitaly node inside praefect cluster # Avoid running unnecessary services on the Gitaly server postgresql['enable'] = false redis['enable'] = false nginx['enable'] = false prometheus['enable'] = false +grafana['enable'] = false unicorn['enable'] = false sidekiq['enable'] = false gitlab_workhorse['enable'] = false +prometheus_monitoring['enable'] = false # Prevent database connections during 'gitlab-ctl reconfigure' gitlab_rails['rake_cache_clear'] = false gitlab_rails['auto_migrate'] = false +# Replace GITLAB_SHELL_SECRET_TOKEN below with real secret +gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' + # Configure the gitlab-shell API callback URL. Without this, `git push` will # fail. This can be your 'front door' GitLab URL or an internal load # balancer. -# Don't forget to copy `/etc/gitlab/gitlab-secrets.json` from web server to Gitaly server. gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' -# Authentication token to ensure only authorized servers can communicate with -# Gitaly server -gitaly['auth_token'] = 'praefect-gitaly-token' +# Replace PRAEFECT_INTERNAL_TOKEN below with a real secret. +gitaly['auth_token'] = 'PRAEFECT_INTERNAL_TOKEN' # Make Gitaly accept connections on all network interfaces. You must use # firewalls to restrict access to this address/port. @@ -164,16 +169,13 @@ gitaly['auth_token'] = 'praefect-gitaly-token' gitaly['listen_addr'] = "0.0.0.0:8075" git_data_dirs({ - "praefect-gitaly-1" => { + "gitaly-1" => { "path" => "/var/opt/gitlab/git-data" } }) ``` -Note that just as with a standard Gitaly server, `/etc/gitlab/gitlab-secrets.json` must -be copied from the GitLab server to the Gitaly node for authentication purposes. - -For more information on Gitaly server configuration, see our [gitaly documentation](index.md#3-gitaly-server-configuration). +For more information on Gitaly server configuration, see our [Gitaly documentation](index.md#3-gitaly-server-configuration). #### GitLab @@ -182,25 +184,29 @@ is done through setting the `git_data_dirs`. Assuming the default storage is present, there should be two storages available to GitLab: ```ruby +# /etc/gitlab/gitlab.rb on gitlab server + +# Replace PRAEFECT_EXTERNAL_TOKEN below with real secret. git_data_dirs({ "default" => { - "gitaly_address" => "tcp://gitaly.internal" + "path" => "/var/opt/gitlab/git-data" }, "praefect" => { - "gitaly_address" => "tcp://praefect.internal:2305" + "gitaly_address" => "tcp://praefect.internal:2305", + "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' } }) -gitlab_rails['gitaly_token'] = 'praefect-token' +# Replace GITLAB_SHELL_SECRET_TOKEN below with real secret +gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN' ``` Note that the storage name used is the same as the `praefect['virtual_storage_name']` set on the Praefect node. -Also, the `gitlab_rails['gitaly_token']` matches the value of `praefect['auth_token']` -on Praefect. +Save your changes and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). -Restart GitLab using `gitlab-ctl restart` on the GitLab node. +Run `gitlab-rake gitlab:gitaly:check` to confirm that GitLab can reach Praefect. ### Testing Praefect @@ -208,10 +214,18 @@ To test Praefect, first set it as the default storage node for new projects using **Admin Area > Settings > Repository > Repository storage**. Next, create a new project and check the "Initialize repository with a README" box. -If you receive a 503 error, check `/var/log/gitlab/gitlab-rails/production.log`. -A `GRPC::Unavailable (14:failed to connect to all addresses)` error indicates -that GitLab was unable to connect to Praefect. - -If the project is created but the README is not, then ensure that the -`/etc/gitlab/gitlab-secrets.json` file from the GitLab server has been copied -to the Gitaly servers. +If you receive an error, check `/var/log/gitlab/gitlab-rails/production.log`. + +Here are common errors and potential causes: + +- 500 response code + - **ActionView::Template::Error (7:permission denied)** + - `praefect['auth_token']` and `gitlab_rails['gitaly_token']` do not match on the GitLab server. + - **Unable to save project. Error: 7:permission denied** + - Secret token in `praefect['storage_nodes']` on GitLab server does not match the + value in `gitaly['auth_token']` on one or more Gitaly servers. +- 503 response code + - **GRPC::Unavailable (14:failed to connect to all addresses)** + - GitLab was unable to reach Praefect. + - **GRPC::Unavailable (14:all SubCons are in TransientFailure...)** + - Praefect cannot reach one or more of its child Gitaly nodes. diff --git a/doc/administration/gitaly/reference.md b/doc/administration/gitaly/reference.md index fe88ef13958..2c5e54743c3 100644 --- a/doc/administration/gitaly/reference.md +++ b/doc/administration/gitaly/reference.md @@ -134,7 +134,7 @@ A lot of Gitaly RPCs need to look up Git objects from repositories. Most of the time we use `git cat-file --batch` processes for that. For better performance, Gitaly can re-use these `git cat-file` processes across RPC calls. Previously used processes are kept around in a -["git cat-file cache"](https://about.gitlab.com/blog/2019/07/08/git-performance-on-nfs/#enter-cat-file-cache). +["Git cat-file cache"](https://about.gitlab.com/blog/2019/07/08/git-performance-on-nfs/#enter-cat-file-cache). In order to control how much system resources this uses, we have a maximum number of cat-file processes that can go into the cache. @@ -165,11 +165,11 @@ Gitaly restarts its `gitaly-ruby` helpers when their memory exceeds the | Name | Type | Required | Description | | ---- | ---- | -------- | ----------- | -| `dir` | string | yes | Path to where gitaly-ruby is installed (needed to boot the process).| -| `max_rss` | integer | no | Resident set size limit that triggers a gitaly-ruby restart, in bytes. Default is `200000000` (200MB). | -| `graceful_restart_timeout` | string | no | Grace period before a gitaly-ruby process is forcibly terminated after exceeding `max_rss`. Default is `10m` (10 minutes).| -| `restart_delay` | string | no |Time that gitaly-ruby memory must remain high before a restart. Default is `5m` (5 minutes).| -| `num_workers` | integer | no |Number of gitaly-ruby worker processes. Try increasing this number in case of `ResourceExhausted` errors. Default is `2`, minimum is `2`.| +| `dir` | string | yes | Path to where `gitaly-ruby` is installed (needed to boot the process).| +| `max_rss` | integer | no | Resident set size limit that triggers a `gitaly-ruby` restart, in bytes. Default is `200000000` (200MB). | +| `graceful_restart_timeout` | string | no | Grace period before a `gitaly-ruby` process is forcibly terminated after exceeding `max_rss`. Default is `10m` (10 minutes).| +| `restart_delay` | string | no |Time that `gitaly-ruby` memory must remain high before a restart. Default is `5m` (5 minutes).| +| `num_workers` | integer | no |Number of `gitaly-ruby` worker processes. Try increasing this number in case of `ResourceExhausted` errors. Default is `2`, minimum is `2`.| | `linguist_languages_path` | string | no | Override for dynamic `languages.json` discovery. Defaults to an empty string (use of dynamic discovery).| Example: @@ -231,11 +231,11 @@ The following values configure logging in Gitaly under the `[logging]` section. | `level` | string | no | Log level: `debug`, `info`, `warn`, `error`, `fatal`, or `panic`. Default: `info`. | | `sentry_dsn` | string | no | Sentry DSN for exception monitoring. | | `sentry_environment` | string | no | [Sentry Environment](https://docs.sentry.io/enriching-error-data/environments/) for exception monitoring. | -| `ruby_sentry_dsn` | string | no | Sentry DSN for gitaly-ruby exception monitoring. | +| `ruby_sentry_dsn` | string | no | Sentry DSN for `gitaly-ruby` exception monitoring. | While the main Gitaly application logs go to stdout, there are some extra log files that go to a configured directory, like the GitLab Shell logs. -Gitlab Shell does not support `panic` or `trace` level logs. `panic` will fall +GitLab Shell does not support `panic` or `trace` level logs. `panic` will fall back to `error`, while `trace` will fall back to `debug`. Any other invalid log levels will default to `info`. diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 7f0b4056acc..d411fb7f20f 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -4,18 +4,19 @@ type: reference, concepts # Scaling and High Availability -GitLab supports several different types of clustering and high-availability. +GitLab supports a number of options for scaling your self-managed instance and configuring high availability (HA). The solution you choose will be based on the level of scalability and availability you require. The easiest solutions are scalable, but not necessarily highly available. -GitLab provides a service that is usually essential to most organizations: it +GitLab provides a service that is essential to most organizations: it enables people to collaborate on code in a timely fashion. Any downtime should -therefore be short and planned. Luckily, GitLab provides a solid setup even on -a single server without special measures. Due to the distributed nature -of Git, developers can still commit code locally even when GitLab is not +therefore be short and planned. Due to the distributed nature +of Git, developers can continue to commit code locally even when GitLab is not available. However, some GitLab features such as the issue tracker and -Continuous Integration are not available when GitLab is down. +continuous integration are not available when GitLab is down. +If you require all GitLab functionality to be highly available, +consider the options outlined below. **Keep in mind that all highly-available solutions come with a trade-off between cost/complexity and uptime**. The more uptime you want, the more complex the @@ -25,8 +26,8 @@ solution should balance the costs against the benefits. There are many options when choosing a highly-available GitLab architecture. We recommend engaging with GitLab Support to choose the best architecture for your -use case. This page contains some various options and guidelines based on -experience with GitLab.com and Enterprise Edition on-premises customers. +use case. This page contains recommendations based on +experience with GitLab.com and internal scale testing. For detailed insight into how GitLab scales and configures GitLab.com, you can watch [this 1 hour Q&A](https://www.youtube.com/watch?v=uCU8jdYzpac) @@ -82,12 +83,13 @@ Complete the following installation steps in order. A link at the end of each section will bring you back to the Scalable Architecture Examples section so you can continue with the next step. -1. [PostgreSQL](database.md#postgresql-in-a-scaled-environment) with [PGBouncer](https://docs.gitlab.com/ee/administration/high_availability/pgbouncer.html) +1. [Load Balancer(s)](load_balancer.md)[^2] +1. [Consul](consul.md) +1. [PostgreSQL](database.md#postgresql-in-a-scaled-environment) with [PgBouncer](https://docs.gitlab.com/ee/administration/high_availability/pgbouncer.html) 1. [Redis](redis.md#redis-in-a-scaled-environment) 1. [Gitaly](gitaly.md) (recommended) and / or [NFS](nfs.md)[^4] 1. [GitLab application nodes](gitlab.md) - With [Object Storage service enabled](../gitaly/index.md#eliminating-nfs-altogether)[^3] -1. [Load Balancer(s)](load_balancer.md)[^2] 1. [Monitoring node (Prometheus and Grafana)](monitoring_node.md) ### Full Scaling @@ -201,13 +203,14 @@ with the added complexity of many more nodes to configure, manage, and monitor. ## Reference Architecture Examples The Support and Quality teams build, performance test, and validate Reference -Architectures that support set large numbers of users. The specifications below are a -representation of this work so far and may be adjusted in the future based on +Architectures that support large numbers of users. The specifications below are +a representation of this work so far and may be adjusted in the future based on additional testing and iteration. -The architectures have been tested with specific coded workloads. The throughputs -used for testing are calculated based on sample customer data. We test each endpoint -type with the following number of requests per second (RPS) per 1000 users: +The architectures have been tested with specific coded workloads, and the +throughputs used for testing were calculated based on sample customer data. We +test each endpoint type with the following number of requests per second (RPS) +per 1000 users: - API: 20 RPS - Web: 2 RPS @@ -217,13 +220,74 @@ Note that your exact needs may be more, depending on your workload. Your workload is influenced by factors such as - but not limited to - how active your users are, how much automation you use, mirroring, and repo/change size. +### 2,000 User Configuration + +- **Supported Users (approximate):** 2,000 +- **Test RPS Rates:** API: 40 RPS, Web: 4 RPS, Git: 4 RPS +- **Status:** Work-in-progress +- **Known Issues:** For the latest list of known performance issues head +[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). + +NOTE: **Note:** This architecture is a work-in-progress of the work so far. The +Quality team will be certifying this environment in late 2019 or early 2020. The specifications +may be adjusted prior to certification based on performance testing. + +| Service | Nodes | Configuration | GCP type | +| ----------------------------|-------|-----------------------|---------------| +| GitLab Rails <br> - Puma workers on each node set to 90% of available CPUs with 8 threads | 3 | 8 vCPU, 7.2GB Memory | n1-highcpu-8 | +| PostgreSQL | 3 | 2 vCPU, 7.5GB Memory | n1-standard-2 | +| PgBouncer | 3 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | +| Gitaly <br> - Gitaly Ruby workers on each node set to 20% of available CPUs | X[^1] . | 4 vCPU, 15GB Memory | n1-standard-4 | +| Redis Cache + Sentinel <br> - Cache maxmemory set to 90% of available memory | 3 | 2 vCPU, 7.5GB Memory | n1-standard-2 | +| Redis Persistent + Sentinel | 3 | 2 vCPU, 7.5GB Memory | n1-standard-2 | +| Sidekiq | 4 | 2 vCPU, 7.5GB Memory | n1-standard-2 | +| Consul | 3 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | +| NFS Server[^4] . | 1 | 4 vCPU, 3.6GB Memory | n1-highcpu-4 | +| S3 Object Storage[^3] . | - | - | - | +| Monitoring node | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | +| External load balancing node[^2] . | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | +| Internal load balancing node[^2] . | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | + +NOTE: **Note:** Memory values are given directly by GCP machine sizes. On different cloud +vendors a best effort like for like can be used. + +### 5,000 User Configuration + +- **Supported Users (approximate):** 5,000 +- **Test RPS Rates:** API: 100 RPS, Web: 10 RPS, Git: 10 RPS +- **Status:** Work-in-progress +- **Known Issues:** For the latest list of known performance issues head +[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). + +NOTE: **Note:** This architecture is a work-in-progress of the work so far. The +Quality team will be certifying this environment in late 2019 or early 2020. The specifications +may be adjusted prior to certification based on performance testing. + +| Service | Nodes | Configuration | GCP type | +| ----------------------------|-------|-----------------------|---------------| +| GitLab Rails <br> - Puma workers on each node set to 90% of available CPUs with 16 threads | 3 | 16 vCPU, 14.4GB Memory | n1-highcpu-16 | +| PostgreSQL | 3 | 2 vCPU, 7.5GB Memory | n1-standard-2 | +| PgBouncer | 3 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | +| Gitaly <br> - Gitaly Ruby workers on each node set to 20% of available CPUs | X[^1] . | 8 vCPU, 30GB Memory | n1-standard-8 | +| Redis Cache + Sentinel <br> - Cache maxmemory set to 90% of available memory | 3 | 2 vCPU, 7.5GB Memory | n1-standard-2 | +| Redis Persistent + Sentinel | 3 | 2 vCPU, 7.5GB Memory | n1-standard-2 | +| Sidekiq | 4 | 2 vCPU, 7.5GB Memory | n1-standard-2 | +| Consul | 3 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | +| NFS Server[^4] . | 1 | 4 vCPU, 3.6GB Memory | n1-highcpu-4 | +| S3 Object Storage[^3] . | - | - | - | +| Monitoring node | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | +| External load balancing node[^2] . | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | +| Internal load balancing node[^2] . | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | + +NOTE: **Note:** Memory values are given directly by GCP machine sizes. On different cloud +vendors a best effort like for like can be used. + ### 10,000 User Configuration - **Supported Users (approximate):** 10,000 - **Test RPS Rates:** API: 200 RPS, Web: 20 RPS, Git: 20 RPS -- **Known Issues:** While validating the reference architecture, slow API endpoints - were discovered. For details, see the related issues list in - [this issue](https://gitlab.com/gitlab-org/gitlab-foss/issues/64335). +- **Known Issues:** For the latest list of known performance issues head +[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). | Service | Nodes | Configuration | GCP type | | ----------------------------|-------|-----------------------|---------------| @@ -248,10 +312,8 @@ vendors a best effort like for like can be used. - **Supported Users (approximate):** 25,000 - **Test RPS Rates:** API: 500 RPS, Web: 50 RPS, Git: 50 RPS -- **Known Issues:** The slow API endpoints that were discovered during testing - the 10,000 user architecture also affect the 25,000 user architecture. For - details, see the related issues list in - [this issue](https://gitlab.com/gitlab-org/gitlab-foss/issues/64335). +- **Known Issues:** For the latest list of known performance issues head +[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). | Service | Nodes | Configuration | GCP type | | ----------------------------|-------|-----------------------|---------------| @@ -276,12 +338,8 @@ vendors a best effort like for like can be used. - **Supported Users (approximate):** 50,000 - **Test RPS Rates:** API: 1000 RPS, Web: 100 RPS, Git: 100 RPS -- **Status:** Work-in-progress -- **Related Issue:** See the [related issue](https://gitlab.com/gitlab-org/quality/performance/issues/66) for more information. - -NOTE: **Note:** This architecture is a work-in-progress of the work so far. The -Quality team will be certifying this environment in late 2019. The specifications -may be adjusted prior to certification based on performance testing. +- **Known Issues:** For the latest list of known performance issues head +[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). | Service | Nodes | Configuration | GCP type | | ----------------------------|-------|-----------------------|---------------| @@ -302,15 +360,16 @@ may be adjusted prior to certification based on performance testing. NOTE: **Note:** Memory values are given directly by GCP machine sizes. On different cloud vendors a best effort like for like can be used. -[^1]: Gitaly node requirements are dependent on customer data. We recommend 2 - nodes as an absolute minimum for performance at the 10,000 and 25,000 user - scale and 4 nodes as an absolute minimum at the 50,000 user scale, but - additional nodes should be considered in conjunction with a review of - project counts and sizes. +[^1]: Gitaly node requirements are dependent on customer data, specifically the number of + projects and their sizes. We recommend 2 nodes as an absolute minimum for HA environments + and at least 4 nodes should be used when supporting 50,000 or more users. + We recommend that each Gitaly node should store no more than 5TB of data. + Additional nodes should be considered in conjunction with a review of expected + data size and spread based on the recommendations above. [^2]: Our architectures have been tested and validated with [HAProxy](https://www.haproxy.org/) as the load balancer. However other reputable load balancers with similar feature sets - should also work here but be aware these aren't validated. + should also work instead but be aware these aren't validated. [^3]: For data objects such as LFS, Uploads, Artifacts, etc... We recommend a S3 Object Storage where possible over NFS due to better performance and availability. Several types of objects diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index 72968cfed56..9b733034f5b 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -71,7 +71,7 @@ Omnibus: redis['port'] = 6379 redis['password'] = 'SECRET_PASSWORD_HERE' - gitlab_rails['auto_migrate'] = false + gitlab_rails['enable'] = false ``` 1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. diff --git a/doc/administration/img/repository_storages_admin_ui.png b/doc/administration/img/repository_storages_admin_ui.png Binary files differindex 5f1b4936704..315b4b0144c 100644 --- a/doc/administration/img/repository_storages_admin_ui.png +++ b/doc/administration/img/repository_storages_admin_ui.png diff --git a/doc/administration/index.md b/doc/administration/index.md index bf21347fb99..2a9980cddb3 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -80,7 +80,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Backup and restore](../raketasks/backup_restore.md): Backup and restore your GitLab instance. - [Operations](operations/index.md): Keeping GitLab up and running (clean up Redis sessions, moving repositories, Sidekiq MemoryKiller, Unicorn). - [Restart GitLab](restart_gitlab.md): Learn how to restart GitLab and its components. -- [Invalidate markdown cache](invalidate_markdown_cache.md): Invalidate any cached markdown. +- [Invalidate Markdown cache](invalidate_markdown_cache.md): Invalidate any cached Markdown. #### Updating GitLab @@ -124,6 +124,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. basic Postfix mail server with IMAP authentication on Ubuntu for incoming emails. - [Abuse reports](../user/admin_area/abuse_reports.md): View and resolve abuse reports from your users. +- [Credentials Inventory](../user/admin_area/credentials_inventory.md): With Credentials inventory, GitLab administrators can keep track of the credentials used by their users in their GitLab self-managed instance. **(ULTIMATE ONLY)** ## Project settings @@ -152,12 +153,17 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Enable/disable GitLab CI/CD](../ci/enable_or_disable_ci.md#site-wide-admin-setting): Enable or disable GitLab CI/CD for your instance. - [GitLab CI/CD admin settings](../user/admin_area/settings/continuous_integration.md): Enable or disable Auto DevOps site-wide and define the artifacts' max size and expiration time. +- [External Pipeline Validation](external_pipeline_validation.md): Enable, disable and configure external pipeline validation. - [Job artifacts](job_artifacts.md): Enable, disable, and configure job artifacts (a set of files and directories which are outputted by a job when it completes successfully). - [Job logs](job_logs.md): Information about the job logs. - [Register Shared and specific Runners](../ci/runners/README.md#registering-a-shared-runner): Learn how to register and configure Shared and specific Runners to your own instance. - [Shared Runners pipelines quota](../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota-starter-only): Limit the usage of pipeline minutes for Shared Runners. **(STARTER ONLY)** - [Enable/disable Auto DevOps](../topics/autodevops/index.md#enablingdisabling-auto-devops): Enable or disable Auto DevOps for your instance. +## Snippet settings + +- [Setting snippet content size limit](snippets/index.md): Set a maximum size limit for snippets' content. + ## Git configuration options - [Custom Git hooks](custom_hooks.md): Custom Git hooks (on the filesystem) for when webhooks aren't enough. diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 23803b82543..33ac925748f 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -60,11 +60,11 @@ mvn package The above sequence of commands will generate a WAR file that can be deployed using Tomcat: -```sh -sudo apt-get install tomcat7 -sudo cp target/plantuml.war /var/lib/tomcat7/webapps/plantuml.war -sudo chown tomcat7:tomcat7 /var/lib/tomcat7/webapps/plantuml.war -sudo service tomcat7 restart +```shell +sudo apt-get install tomcat8 +sudo cp target/plantuml.war /var/lib/tomcat8/webapps/plantuml.war +sudo chown tomcat8:tomcat8 /var/lib/tomcat8/webapps/plantuml.war +sudo service tomcat8 restart ``` Once the Tomcat service restarts the PlantUML service will be ready and @@ -74,7 +74,7 @@ listening for requests on port 8080: http://localhost:8080/plantuml ``` -you can change these defaults by editing the `/etc/tomcat7/server.xml` file. +you can change these defaults by editing the `/etc/tomcat8/server.xml` file. Note that the default URL is different than when using the Docker-based image, where the service is available at the root of URL with no relative path. Adjust diff --git a/doc/administration/invalidate_markdown_cache.md b/doc/administration/invalidate_markdown_cache.md index ad64cb077c1..ebd8578e410 100644 --- a/doc/administration/invalidate_markdown_cache.md +++ b/doc/administration/invalidate_markdown_cache.md @@ -1,6 +1,6 @@ # Invalidate Markdown Cache -For performance reasons, GitLab caches the HTML version of markdown text +For performance reasons, GitLab caches the HTML version of Markdown text (e.g. issue and merge request descriptions, comments). It's possible that these cached versions become outdated, for example when the `external_url` configuration option is changed - causing links diff --git a/doc/administration/logs.md b/doc/administration/logs.md index e1910b0b3f3..f4a1c754252 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -360,6 +360,17 @@ Introduced in GitLab 12.3. This file lives in `/var/log/gitlab/gitlab-rails/migr Omnibus GitLab packages or in `/home/git/gitlab/log/migrations.log` for installations from source. +## `mail_room_json.log` (default) + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/19186) in GitLab 12.6. + +This file lives in `/var/log/gitlab/mail_room/mail_room_json.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/mail_room_json.log` for +installations from source. + +This structured log file records internal activity in the `mail_room` gem. +Its name and path are configurable, so the name and path may not match the above. + ## Reconfigure Logs Reconfigure log files live in `/var/log/gitlab/reconfigure` for Omnibus GitLab @@ -393,6 +404,23 @@ It is stored at: - `/var/log/gitlab/gitlab-rails/database_load_balancing.log` for Omnibus GitLab packages. - `/home/git/gitlab/log/database_load_balancing.log` for installations from source. +## `elasticsearch.log` + +Introduced in GitLab 12.6. This file lives in +`/var/log/gitlab/gitlab-rails/elasticsearch.log` for Omnibus GitLab +packages or in `/home/git/gitlab/log/elasticsearch.log` for installations +from source. + +It logs information related to the Elasticsearch Integration including +errors during indexing or searching Elasticsearch. + +Each line contains a JSON line that can be ingested by Elasticsearch, Splunk, +etc. For example: + +```json +{"severity":"DEBUG","time":"2019-10-17T06:23:13.227Z","correlation_id":null,"message":"redacted_search_result","class_name":"Milestone","id":2,"ability":"read_milestone","current_user_id":2,"query":"project"} +``` + [repocheck]: repository_checks.md [Rack Attack]: ../security/rack_attack.md [Rate Limit]: ../user/admin_area/settings/rate_limits_on_raw_endpoints.md diff --git a/doc/administration/monitoring/performance/img/performance_bar.png b/doc/administration/monitoring/performance/img/performance_bar.png Binary files differindex acad60f863e..d206d5a4268 100644 --- a/doc/administration/monitoring/performance/img/performance_bar.png +++ b/doc/administration/monitoring/performance/img/performance_bar.png diff --git a/doc/administration/monitoring/performance/img/performance_bar_frontend.png b/doc/administration/monitoring/performance/img/performance_bar_frontend.png Binary files differnew file mode 100644 index 00000000000..489f855fe33 --- /dev/null +++ b/doc/administration/monitoring/performance/img/performance_bar_frontend.png diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index a52b6227e14..98c611ea140 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -16,9 +16,16 @@ It allows you to see (from left to right):  - time taken and number of Redis calls; click through for details of these calls  +- total load timings of the page; click through for details of these calls. Values in the following order: + - Backend - Time that the actual base page took to load + - [First Contentful Paint](https://developers.google.com/web/tools/lighthouse/audits/first-contentful-paint) - Time until something was visible to the user + - [DomContentLoaded](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/measure-crp) Event + - Number of Requests that the page loaded +  - a link to add a request's details to the performance bar; the request can be added by its full URL (authenticated as the current user), or by the value of its `X-Request-Id` header +- a link to download the raw JSON used to generate the Performance Bar reports On the far right is a request selector that allows you to view the same metrics (excluding the page timing and line profiler) for any requests made while the diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 02920293daa..57048059476 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -25,8 +25,8 @@ The following metrics are available: | Metric | Type | Since | Description | Labels | |:---------------------------------------------------------------|:----------|-----------------------:|:----------------------------------------------------------------------------------------------------|:----------------------------------------------------| -| `gitlab_banzai_cached_render_real_duration_seconds` | Histogram | 9.4 | Duration of rendering markdown into HTML when cached output exists | controller, action | -| `gitlab_banzai_cacheless_render_real_duration_seconds` | Histogram | 9.4 | Duration of rendering markdown into HTML when cached outupt does not exist | controller, action | +| `gitlab_banzai_cached_render_real_duration_seconds` | Histogram | 9.4 | Duration of rendering Markdown into HTML when cached output exists | controller, action | +| `gitlab_banzai_cacheless_render_real_duration_seconds` | Histogram | 9.4 | Duration of rendering Markdown into HTML when cached outupt does not exist | controller, action | | `gitlab_cache_misses_total` | Counter | 10.2 | Cache read miss | controller, action | | `gitlab_cache_operation_duration_seconds` | Histogram | 10.2 | Cache access time | | | `gitlab_cache_operations_total` | Counter | 12.2 | Cache operations by controller/action | controller, action, operation | @@ -90,7 +90,6 @@ The following metrics can be controlled by feature flags: | Metric | Feature Flag | |:---------------------------------------------------------------|:-------------------------------------------------------------------| | `gitlab_method_call_duration_seconds` | `prometheus_metrics_method_instrumentation` | -| `gitlab_transaction_allocated_memory_bytes` | `prometheus_metrics_transaction_allocated_memory` | | `gitlab_view_rendering_duration_seconds` | `prometheus_metrics_view_instrumentation` | ## Sidekiq Metrics available for Geo **(PREMIUM)** diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index c0b563bd76e..eb7a2d791c1 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -288,6 +288,12 @@ The PgBouncer exporter allows you to measure various PgBouncer metrics. [➔ Read more about the PgBouncer exporter.](pgbouncer_exporter.md) +### Registry exporter + +The Registry exporter allows you to measure various Registry metrics. + +[➔ Read more about the Registry exporter.](registry_exporter.md) + ### GitLab exporter The GitLab exporter allows you to measure various GitLab metrics, pulled from Redis and the database. diff --git a/doc/administration/monitoring/prometheus/postgres_exporter.md b/doc/administration/monitoring/prometheus/postgres_exporter.md index 3ad15b65497..27771865e10 100644 --- a/doc/administration/monitoring/prometheus/postgres_exporter.md +++ b/doc/administration/monitoring/prometheus/postgres_exporter.md @@ -1,30 +1,58 @@ -# Postgres exporter +# PostgreSQL Server Exporter >**Note:** -Available since [Omnibus GitLab 8.17][1131]. For installations from source -you'll have to install and configure it yourself. +Available since [Omnibus GitLab 8.17](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1131). +For installations from source you will have to install and configure it yourself. -The [postgres exporter] allows you to measure various PostgreSQL metrics. +The [PostgreSQL Server Exporter](https://github.com/wrouesnel/postgres_exporter) allows you to export various PostgreSQL metrics. -To enable the postgres exporter: +To enable the PostgreSQL Server Exporter: -1. [Enable Prometheus](index.md#configuring-prometheus) -1. Edit `/etc/gitlab/gitlab.rb` -1. Add or find and uncomment the following line, making sure it's set to `true`: +1. [Enable Prometheus](index.md#configuring-prometheus). +1. Edit `/etc/gitlab/gitlab.rb` and enable `postgres_exporter`: ```ruby postgres_exporter['enable'] = true ``` -1. Save the file and [reconfigure GitLab][reconfigure] for the changes to - take effect +NOTE: **Note:** +If PostgreSQL Server Exporter is configured on a separate node, make sure that the local +address is [listed in `trust_auth_cidr_addresses`](../../high_availability/database.md#network-information) or the +exporter will not be able to connect to the database. + +1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to + take effect. Prometheus will now automatically begin collecting performance data from -the postgres exporter exposed under `localhost:9187`. +the PostgreSQL Server Exporter exposed under `localhost:9187`. -[← Back to the main Prometheus page](index.md) +## Advanced configuration + +In most cases, PostgreSQL Server Exporter will work with the defaults and you should not +need to change anything. + +To further customize the PostgreSQL Server Exporter, use the following configuration options: + +1. Edit `/etc/gitlab/gitlab.rb`: -[1131]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1131 -[postgres exporter]: https://github.com/wrouesnel/postgres_exporter -[prometheus]: https://prometheus.io -[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure + ```ruby + postgres_exporter['dbname'] = 'pgbouncer' # The name of the database to connect to. + postgres_exporter['user'] = 'gitlab-psql' # The user to sign in as. + postgres_exporter['password'] = '' # The user's password. + postgres_exporter['host'] = 'localhost' # The host to connect to. Values that start with '/' are for unix domain sockets (default is 'localhost'). + postgres_exporter['port'] = 5432 # The port to bind to (default is '5432'). + postgres_exporter['sslmode'] = 'require' # Whether or not to use SSL. Valid options are: + # 'disable' (no SSL), + # 'require' (always use SSL and skip verification, this is the default value), + # 'verify-ca' (always use SSL and verify that the certificate presented by the server was signed by a trusted CA), + # 'verify-full' (always use SSL and verify that the certification presented by the server was signed by a trusted CA and the server host name matches the one in the certificate). + postgres_exporter['fallback_application_name'] = '' # An application_name to fall back to if one isn't provided. + postgres_exporter['connect_timeout'] = '' # Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely. + postgres_exporter['sslcert'] = 'ssl.crt' # Cert file location. The file must contain PEM encoded data. + postgres_exporter['sslkey'] = 'ssl.key' # Key file location. The file must contain PEM encoded data. + postgres_exporter['sslrootcert'] = 'ssl-root.crt' # The location of the root certificate file. The file must contain PEM encoded data. + ``` + +1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + +[← Back to the main Prometheus page](index.md) diff --git a/doc/administration/monitoring/prometheus/registry_exporter.md b/doc/administration/monitoring/prometheus/registry_exporter.md new file mode 100644 index 00000000000..692e589185e --- /dev/null +++ b/doc/administration/monitoring/prometheus/registry_exporter.md @@ -0,0 +1,21 @@ +# Registry exporter + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2884) in GitLab 11.9. + +The Registry exporter allows you to measure various Registry metrics. +To enable it: + +1. [Enable Prometheus](index.md#configuring-prometheus). +1. Edit `/etc/gitlab/gitlab.rb` and enable [debug mode](https://docs.docker.com/registry/#debug) for the Registry: + + ```ruby + registry['debug_addr'] = "localhost:5001" # localhost:5001/metrics + ``` + +1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) + for the changes to take effect. + +Prometheus will now automatically begin collecting performance data from +the registry exporter exposed under `localhost:5001/metrics`. + +[← Back to the main Prometheus page](index.md) diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md index 0b5ddfd03ee..e15f91ebab2 100644 --- a/doc/administration/operations/extra_sidekiq_processes.md +++ b/doc/administration/operations/extra_sidekiq_processes.md @@ -126,12 +126,26 @@ queues will use three threads in total. ## Limiting concurrency -To limit the concurrency of the Sidekiq processes: +To limit the concurrency of the Sidekiq process: 1. Edit `/etc/gitlab/gitlab.rb` and add: ```ruby - sidekiq_cluster['concurrency'] = 25 + sidekiq['concurrency'] = 25 + ``` + +1. Save the file and reconfigure GitLab for the changes to take effect: + + ```sh + sudo gitlab-ctl reconfigure + ``` + +To limit the max concurrency of the Sidekiq cluster processes: + +1. Edit `/etc/gitlab/gitlab.rb` and add: + + ```ruby + sidekiq_cluster['max_concurrency'] = 25 ``` 1. Save the file and reconfigure GitLab for the changes to take effect: diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md index a62e3ab603d..e735d8dd97e 100644 --- a/doc/administration/packages/container_registry.md +++ b/doc/administration/packages/container_registry.md @@ -457,36 +457,40 @@ If Registry is enabled in your GitLab instance, but you don't need it for your project, you can disable it from your project's settings. Read the user guide on how to achieve that. -## Disable Container Registry but use GitLab as an auth endpoint +## Use an external container registry with GitLab as an auth endpoint **Omnibus GitLab** -You can use GitLab as an auth endpoint and use a non-bundled Container Registry. +You can use GitLab as an auth endpoint with an external container registry. 1. Open `/etc/gitlab/gitlab.rb` and set necessary configurations: ```ruby gitlab_rails['registry_enabled'] = true - gitlab_rails['registry_host'] = "registry.gitlab.example.com" - gitlab_rails['registry_port'] = "5005" gitlab_rails['registry_api_url'] = "http://localhost:5000" - gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry" gitlab_rails['registry_issuer'] = "omnibus-gitlab-issuer" ``` -1. A certificate keypair is required for GitLab and the Container Registry to - communicate securely. By default Omnibus GitLab will generate one keypair, - which is saved to `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key`. - When using a non-bundled Container Registry, you will need to supply a - custom certificate key. To do that, add the following to - `/etc/gitlab/gitlab.rb` + NOTE: **Note:** + `gitlab_rails['registry_enabled'] = true` is needed to enable GitLab's + Container Registry features and authentication endpoint. GitLab's bundled + Container Registry service will not be started even with this enabled. + +1. A certificate-key pair is required for GitLab and the external container + registry to communicate securely. You will need to create a certificate-key + pair, configuring the external container registry with the public + certificate and configuring GitLab with the private key. To do that, add + the following to `/etc/gitlab/gitlab.rb`: ```ruby - gitlab_rails['registry_key_path'] = "/custom/path/to/registry-key.key" # registry['internal_key'] should contain the contents of the custom key # file. Line breaks in the key file should be marked using `\n` character # Example: registry['internal_key'] = "---BEGIN RSA PRIVATE KEY---\nMIIEpQIBAA\n" + + # Optionally define a custom file for Omnibus GitLab to write the contents + # of registry['internal_key'] to. + gitlab_rails['registry_key_path'] = "/custom/path/to/registry-key.key" ``` NOTE: **Note:** @@ -496,7 +500,16 @@ You can use GitLab as an auth endpoint and use a non-bundled Container Registry. `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key` and will populate it. -1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. +1. To change the container registry URL displayed in the GitLab Container + Registry pages, set the following configurations: + + ```ruby + gitlab_rails['registry_host'] = "registry.gitlab.example.com" + gitlab_rails['registry_port'] = "5005" + ``` + +1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) + for the changes to take effect. **Installations from source** diff --git a/doc/administration/packages/index.md b/doc/administration/packages/index.md index d4afc65577e..58dd8201c15 100644 --- a/doc/administration/packages/index.md +++ b/doc/administration/packages/index.md @@ -14,7 +14,7 @@ The Packages feature allows GitLab to act as a repository for the following: Don't you see your package management system supported yet? Please consider contributing -to GitLab. This [development documentation](../../development/packages.md) will guide you through the process. +to GitLab. This [development documentation](../../development/packages.md) will guide you through the process, it includes a list of [suggested contributions](../../development/packages.md#suggested-contributions). ## Enabling the Packages feature diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index f51c375860b..d1b58f2ee18 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -305,7 +305,7 @@ Pages access control is disabled by default. To enable it: ``` 1. [Reconfigure GitLab][reconfigure]. -1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core). +1. Users can now configure it in their [projects' settings](../../user/project/pages/pages_access_control.md). ### Running behind a proxy @@ -321,6 +321,23 @@ pages: 1. [Reconfigure GitLab][reconfigure] for the changes to take effect. +### Using a custom Certificate Authority (CA) with Access Control + +When using certificates issued by a custom CA, Access Control on GitLab Pages may fail to work if the custom CA is not recognized. + +This usually results in this error: +`Post /oauth/token: x509: certificate signed by unknown authority`. + +For GitLab Pages Access Control with TLS/SSL certs issued by an internal or custom CA: + +1. Copy the certificate bundle to `/opt/gitlab/embedded/ssl/certs/` in `.pem` format. + +1. [Restart](../restart_gitlab.md) the GitLab Pages Daemon. For GitLab Omnibus instances: + + ```bash + sudo gitlab-ctl restart gitlab-pages + ``` + ## Activate verbose logging for daemon Verbose logging was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2533) in @@ -483,6 +500,7 @@ then you must use the following procedure to configure [access control](#access- ```ruby gitlab_pages['gitlab_server'] = "https://<your-gitlab-server-URL>" + gitlab_pages['access_control'] = true ``` 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index e63e0c40393..149b8d62309 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -270,3 +270,15 @@ database: gitlabhq_production -------------------------------------------------- up migration_id migration_name ``` + +## Import common metrics + +Sometimes you may need to re-import the common metrics that power the Metrics dashboards. + +This could be as a result of [updating existing metrics](../../development/prometheus_metrics.md#update-existing-metrics), or as a [troubleshooting measure](../../user/project/integrations/prometheus.md#troubleshooting). + +To re-import the metrics you can run: + +```sh +sudo gitlab-rake metrics:setup_common_metrics +``` diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md index 517d6b01438..26c811ca54d 100644 --- a/doc/administration/raketasks/uploads/migrate.md +++ b/doc/administration/raketasks/uploads/migrate.md @@ -122,10 +122,10 @@ your data out of Object Storage and back into your local storage. **Before proceeding, it is important to disable both `direct_upload` and `background_upload` under `uploads` settings in `gitlab.rb`** CAUTION: **Warning:** - **Extended downtime is required** so no new files are created in object storage during - the migration. A configuration setting will be added soon to allow migrating - from object storage to local files with only a brief moment of downtime for configuration changes. - See issue [gitlab-org/gitlab#30979](https://gitlab.com/gitlab-org/gitlab/issues/30979) +**Extended downtime is required** so no new files are created in object storage during +the migration. A configuration setting will be added soon to allow migrating +from object storage to local files with only a brief moment of downtime for configuration changes. +To follow progress, see the [relevant issue](https://gitlab.com/gitlab-org/gitlab/issues/30979). ### All-in-one rake task diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index 86998280b93..d31b1d7fcd6 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -107,7 +107,8 @@ working, you can remove the `repos_path` line. ## Choose where new project repositories will be stored Once you set the multiple storage paths, you can choose where new projects will -be stored via the **Application Settings** in the Admin area. +be stored under **Admin Area > Settings > Repository > Repository storage > Storage +nodes for new projects**.  diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md index 9f95080654f..6f3c6028f71 100644 --- a/doc/administration/restart_gitlab.md +++ b/doc/administration/restart_gitlab.md @@ -8,6 +8,7 @@ If you want the TL;DR versions, jump to: - [Omnibus GitLab restart](#omnibus-gitlab-restart) - [Omnibus GitLab reconfigure](#omnibus-gitlab-reconfigure) - [Source installation restart](#installations-from-source) +- [Helm chart installation restart](#helm-chart-installations) ## Omnibus installations @@ -143,3 +144,16 @@ If you are using other init systems, like systemd, you can check the [chef]: https://www.chef.io/products/chef-infra/ "Chef official website" [src-service]: https://gitlab.com/gitlab-org/gitlab/blob/master/lib/support/init.d/gitlab "GitLab init service file" [gl-recipes]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/init "GitLab Recipes repository" + +## Helm chart installations + +There is no single command to restart the entire GitLab application installed via +the [cloud native Helm Chart](https://docs.gitlab.com/charts/). Usually, it should be +enough to restart a specific component separately (`gitaly`, `unicorn`, +`workhorse`, `gitlab-shell`, etc.) by deleting all the pods related to it: + +```bash +kubectl delete pods -l release=<helm release name>,app=<component name> +``` + +The release name can be obtained from the output of the `helm list` command. diff --git a/doc/administration/snippets/index.md b/doc/administration/snippets/index.md new file mode 100644 index 00000000000..2e17db7b1f6 --- /dev/null +++ b/doc/administration/snippets/index.md @@ -0,0 +1,71 @@ +--- +type: reference, howto +--- + +# Snippets settings **(CORE ONLY)** + +Adjust the snippets' settings of your GitLab instance. + +## Snippets content size limit + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/31133) in GitLab 12.6. + +You can set a content size max limit in snippets. This limit can prevent +abuses of the feature. The default content size limit is **52428800 Bytes** (50MB). + +### How does it work? + +The content size limit will be applied when a snippet is created or +updated. Nevertheless, in order not to break any existing snippet, +the limit will only be enforced in stored snippets when the content +is updated. + +### Snippets size limit configuration + +This setting is not available through the [Admin Area settings](../../user/admin_area/settings/index.md). +In order to configure this setting, use either the Rails console +or the [Application settings API](../../api/settings.md). + +NOTE: **IMPORTANT:** +The value of the limit **must** be in Bytes. + +#### Through the Rails console + +The steps to configure this setting through the Rails console are: + +1. Start the Rails console: + + ```bash + # For Omnibus installations + sudo gitlab-rails console + + # For installations from source + sudo -u git -H bundle exec rails console production + ``` + +1. Update the snippets maximum file size: + + ```ruby + ApplicationSetting.first.update!(snippet_size_limit: 50.megabytes) + ``` + +To retrieve the current value, start the Rails console and run: + + ```ruby + Gitlab::CurrentSettings.snippet_size_limit + ``` + +#### Through the API + +The process to set the snippets size limit through the Application Settings API is +exactly the same as you would do to [update any other setting](../../api/settings.md#change-application-settings). + +```bash +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/application/settings?snippet_size_limit=52428800 +``` + +You can also use the API to [retrieve the current value](../../api/settings.md#get-current-application-settings). + +```bash +curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/application/settings +``` diff --git a/doc/administration/troubleshooting/elasticsearch.md b/doc/administration/troubleshooting/elasticsearch.md index 37ec32413f8..5846514c574 100644 --- a/doc/administration/troubleshooting/elasticsearch.md +++ b/doc/administration/troubleshooting/elasticsearch.md @@ -171,7 +171,7 @@ To do this: pp s.search_objects.class.name ``` -The ouput from the last command is the key here. If it shows: +The output from the last command is the key here. If it shows: - `ActiveRecord::Relation`, **it is not** using Elasticsearch. - `Kaminari::PaginatableArray`, **it is** using Elasticsearch. @@ -326,7 +326,7 @@ feel free to update that page with issues you encounter and solutions. Setting up Elasticsearch isn't too bad, but it can be a bit finnicky and time consuming. -The eastiest method is to spin up a docker container with the required version and +The easiest method is to spin up a docker container with the required version and bind ports 9200/9300 so it can be used. The following is an example of running a docker container of Elasticsearch v7.2.0: diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md index dd220d0871d..cb0b24ae026 100644 --- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md +++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md @@ -258,6 +258,22 @@ Project.find_each do |project| end ``` +## Wikis + +### Recreate + +A Projects Wiki can be recreated by + +**Note:** This is a destructive operation, the Wiki will be empty + +```ruby +p = Project.find_by_full_path('<username-or-group>/<project-name>') ### enter your projects path + +GitlabShellWorker.perform_in(0, :remove_repository, p.repository_storage, p.wiki.disk_path) ### deletes the wiki project from the filesystem + +p.create_wiki ### creates the wiki project on the filesystem +``` + ## Imports / Exports ```ruby @@ -907,7 +923,7 @@ queue = Sidekiq::Queue.new('update_merge_requests') queue.each { |job| job.delete if job.args[0]==125 and job.args[4]=='ref/heads/my_branch'} ``` -**Note:** Running jobs will not be killed. Stop sidekiq before doing this, to get all matching jobs. +**Note:** Running jobs will not be killed. Stop Sidekiq before doing this, to get all matching jobs. ### Enable debug logging of Sidekiq diff --git a/doc/administration/troubleshooting/postgresql.md b/doc/administration/troubleshooting/postgresql.md index f427cd88ce0..65c6952bf1c 100644 --- a/doc/administration/troubleshooting/postgresql.md +++ b/doc/administration/troubleshooting/postgresql.md @@ -46,7 +46,7 @@ This section is for links to information elsewhere in the GitLab documentation. - Managing Omnibus PostgreSQL versions [from the development docs](https://docs.gitlab.com/omnibus/development/managing-postgresql-versions.html) - [PostgreSQL scaling and HA](../high_availability/database.md) - - including [troubleshooting](../high_availability/database.md#troubleshooting) gitlab-ctl repmgr-check-master and pgbouncer errors + - including [troubleshooting](../high_availability/database.md#troubleshooting) `gitlab-ctl repmgr-check-master` and PgBouncer errors - [Developer database documentation](../../development/README.md#database-guides) - some of which is absolutely not for production use. Including: - understanding EXPLAIN plans @@ -58,30 +58,30 @@ This section is for links to information elsewhere in the GitLab documentation. - required extension pg_trgm - required extension postgres_fdw for Geo -- Errors like this in the production/sidekiq log; see: [Set default_transaction_isolation into read committed](https://docs.gitlab.com/omnibus/settings/database.html#set-default_transaction_isolation-into-read-committed) +- Errors like this in the `production/sidekiq` log; see: [Set default_transaction_isolation into read committed](https://docs.gitlab.com/omnibus/settings/database.html#set-default_transaction_isolation-into-read-committed): -``` -ActiveRecord::StatementInvalid PG::TRSerializationFailure: ERROR: could not serialize access due to concurrent update -``` + ```plaintext + ActiveRecord::StatementInvalid PG::TRSerializationFailure: ERROR: could not serialize access due to concurrent update + ``` -- PostgreSQL HA - [replication slot errors](https://docs.gitlab.com/omnibus/settings/database.html#troubleshooting-upgrades-in-an-ha-cluster) +- PostgreSQL HA - [replication slot errors](https://docs.gitlab.com/omnibus/settings/database.html#troubleshooting-upgrades-in-an-ha-cluster): -``` -pg_basebackup: could not create temporary replication slot "pg_basebackup_12345": ERROR: all replication slots are in use -HINT: Free one or increase max_replication_slots. -``` + ```plaintext + pg_basebackup: could not create temporary replication slot "pg_basebackup_12345": ERROR: all replication slots are in use + HINT: Free one or increase max_replication_slots. + ``` - GEO [replication errors](../geo/replication/troubleshooting.md#fixing-replication-errors) including: -``` -ERROR: replication slots can only be used if max_replication_slots > 0 + ```plaintext + ERROR: replication slots can only be used if max_replication_slots > 0 -FATAL: could not start WAL streaming: ERROR: replication slot “geo_secondary_my_domain_com” does not exist + FATAL: could not start WAL streaming: ERROR: replication slot “geo_secondary_my_domain_com” does not exist -Command exceeded allowed execution time + Command exceeded allowed execution time -PANIC: could not write to file ‘pg_xlog/xlogtemp.123’: No space left on device -``` + PANIC: could not write to file ‘pg_xlog/xlogtemp.123’: No space left on device + ``` - [Checking GEO configuration](../geo/replication/troubleshooting.md#checking-configuration) including - reconfiguring hosts/ports @@ -96,7 +96,7 @@ PANIC: could not write to file ‘pg_xlog/xlogtemp.123’: No space left on devi References: - [Issue #1 Deadlocks with GitLab 12.1, PostgreSQL 10.7](https://gitlab.com/gitlab-org/gitlab/issues/30528) -- [Customer ticket (internal) GitLab 12.1.6](https://gitlab.zendesk.com/agent/tickets/134307) and [google doc (internal)](https://docs.google.com/document/d/19xw2d_D1ChLiU-MO1QzWab-4-QXgsIUcN5e_04WTKy4) +- [Customer ticket (internal) GitLab 12.1.6](https://gitlab.zendesk.com/agent/tickets/134307) and [Google doc (internal)](https://docs.google.com/document/d/19xw2d_D1ChLiU-MO1QzWab-4-QXgsIUcN5e_04WTKy4) - [Issue #2 deadlocks can occur if an instance is flooded with pushes](https://gitlab.com/gitlab-org/gitlab/issues/33650). Provided for context about how GitLab code can have this sort of unanticipated effect in unusual situations. ``` @@ -117,7 +117,7 @@ Quoting from from issue [#1](https://gitlab.com/gitlab-org/gitlab/issues/30528): TIP: **Tip:** In support, our general approach to reconfiguring timeouts (applies also to the HTTP stack as well) is that it's acceptable to do it temporarily as a workaround. If it makes GitLab usable for the customer, then it buys time to understand the problem more completely, implement a hot fix, or make some other change that addresses the root cause. Generally, the timeouts should be put back to reasonable defaults once the root cause is resolved. -In this case, the guidance we had from development was to drop deadlock_timeout and/or statement_timeout but to leave the third setting at 60s. Setting idle_in_transaction protects the database from sessions potentially hanging for days. There's more discussion in [the issue relating to introducing this timeout on gitlab.com.](https://gitlab.com/gitlab-com/gl-infra/production/issues/1053) +In this case, the guidance we had from development was to drop deadlock_timeout and/or statement_timeout but to leave the third setting at 60s. Setting idle_in_transaction protects the database from sessions potentially hanging for days. There's more discussion in [the issue relating to introducing this timeout on GitLab.com](https://gitlab.com/gitlab-com/gl-infra/production/issues/1053). PostgresSQL defaults: diff --git a/doc/api/README.md b/doc/api/README.md index 6858e5b7d56..e756cd51997 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -82,7 +82,7 @@ There are four ways to authenticate with the GitLab API: 1. [OAuth2 tokens](#oauth2-tokens) 1. [Personal access tokens](#personal-access-tokens) 1. [Session cookie](#session-cookie) -1. [GitLab CI job token](#gitlab-ci-job-token-premium) **(PREMIUM)** +1. [GitLab CI job token](#gitlab-ci-job-token) **(Specific endpoints only)** For admins who want to authenticate with the API as a specific user, or who want to build applications or scripts that do so, two options are available: @@ -152,13 +152,14 @@ The primary user of this authentication method is the web frontend of GitLab its which can use the API as the authenticated user to get a list of their projects, for example, without needing to explicitly pass an access token. -### GitLab CI job token **(PREMIUM)** +### GitLab CI job token With a few API endpoints you can use a [GitLab CI job token](../user/project/new_ci_build_permissions_model.md#job-token) to authenticate with the API: - [Get job artifacts](jobs.md#get-job-artifacts) - [Pipeline triggers](pipeline_triggers.md) +- [Release creation](releases/index.md#create-a-release) ### Impersonation tokens @@ -313,6 +314,17 @@ The following table shows the possible return codes for API requests. ## Pagination +We support two kinds of pagination methods: + +- Offset-based pagination. This is the default method and available on all endpoints. +- Keyset-based pagination. Added to selected endpoints but being + [progressively rolled out](https://gitlab.com/groups/gitlab-org/-/epics/2039). + +For large collections, we recommend keyset pagination (when available) over offset +pagination for performance reasons. + +### Offset-based pagination + Sometimes the returned result will span across many pages. When listing resources you can pass the following parameters: @@ -324,10 +336,10 @@ resources you can pass the following parameters: In the example below, we list 50 [namespaces](namespaces.md) per page. ```bash -curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/namespaces?per_page=50 +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/namespaces?per_page=50" ``` -### Pagination Link header +#### Pagination Link header [Link headers](https://www.w3.org/wiki/LinkHeader) are sent back with each response. They have `rel` set to prev/next/first/last and contain the relevant @@ -362,7 +374,7 @@ X-Total: 8 X-Total-Pages: 3 ``` -### Other pagination headers +#### Other pagination headers Additional pagination headers are also sent back. @@ -383,6 +395,48 @@ and **behind the `api_kaminari_count_with_limit` more than 10,000, the `X-Total` and `X-Total-Pages` headers as well as the `rel="last"` `Link` are not present in the response headers. +### Keyset-based pagination + +Keyset-pagination allows for more efficient retrieval of pages and - in contrast to offset-based pagination - runtime +is independent of the size of the collection. + +This method is controlled by the following parameters: + +| Parameter | Description | +| ------------ | -------------------------------------- | +| `pagination` | `keyset` (to enable keyset pagination) | +| `per_page` | Number of items to list per page (default: `20`, max: `100`) | + +In the example below, we list 50 [projects](projects.md) per page, ordered by `id` ascending. + +```bash +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects?pagination=keyset&per_page=50&order_by=id&sort=asc" +``` + +The response header includes a link to the next page. For example: + +``` +HTTP/1.1 200 OK +... +Link: <https://gitlab.example.com/api/v4/projects?pagination=keyset&per_page=50&order_by=id&sort=asc&id_after=42>; rel="next" +Status: 200 OK +... +``` + +The link to the next page contains an additional filter `id_after=42` which excludes records we have retrieved already. +Note the type of filter depends on the `order_by` option used and we may have more than one additional filter. + +The `Link` header is absent when the end of the collection has been reached and there are no additional records to retrieve. + +We recommend using only the given link to retrieve the next page instead of building your own URL. Apart from the headers shown, +we don't expose additional pagination headers. + +Keyset-based pagination is only supported for selected resources and ordering options: + +| Resource | Order | +| ------------------------- | -------------------------- | +| [Projects](projects.md) | `order_by=id` only | + ## Namespaced path encoding If using namespaced API calls, make sure that the `NAMESPACE/PROJECT_PATH` is diff --git a/doc/api/boards.md b/doc/api/boards.md index b9c2a984dc5..9ebe1570a59 100644 --- a/doc/api/boards.md +++ b/doc/api/boards.md @@ -49,7 +49,8 @@ Example response: "description" : null }, "position" : 1, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 }, { "id" : 2, @@ -59,7 +60,8 @@ Example response: "description" : null }, "position" : 2, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 }, { "id" : 3, @@ -69,7 +71,8 @@ Example response: "description" : null }, "position" : 3, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 } ] } @@ -121,7 +124,8 @@ Example response: "description" : null }, "position" : 1, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 }, { "id" : 2, @@ -131,7 +135,8 @@ Example response: "description" : null }, "position" : 2, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 }, { "id" : 3, @@ -141,7 +146,8 @@ Example response: "description" : null }, "position" : 3, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 } ] } @@ -192,7 +198,8 @@ Example response: "description" : null }, "position" : 1, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 }, { "id" : 2, @@ -202,7 +209,8 @@ Example response: "description" : null }, "position" : 2, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 }, { "id" : 3, @@ -212,7 +220,8 @@ Example response: "description" : null }, "position" : 3, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 } ] } @@ -346,7 +355,8 @@ Example response: "description" : null }, "position" : 1, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 }, { "id" : 2, @@ -356,7 +366,8 @@ Example response: "description" : null }, "position" : 2, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 }, { "id" : 3, @@ -366,7 +377,8 @@ Example response: "description" : null }, "position" : 3, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 } ] ``` @@ -400,7 +412,8 @@ Example response: "description" : null }, "position" : 1, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 } ``` @@ -441,7 +454,8 @@ Example response: "description" : null }, "position" : 1, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 } ``` @@ -475,7 +489,8 @@ Example response: "description" : null }, "position" : 1, - "max_issue_count": 0 + "max_issue_count": 0, + "max_issue_weight": 0 } ``` diff --git a/doc/api/broadcast_messages.md b/doc/api/broadcast_messages.md index 357d9916ade..ce68eec87ff 100644 --- a/doc/api/broadcast_messages.md +++ b/doc/api/broadcast_messages.md @@ -34,7 +34,8 @@ Example response: "color":"#E75E40", "font":"#FFFFFF", "id":1, - "active": false + "active": false, + "target_path": "*/welcome" } ] ``` @@ -69,7 +70,8 @@ Example response: "color":"#cecece", "font":"#FFFFFF", "id":1, - "active":false + "active":false, + "target_path": "*/welcome" } ``` @@ -107,7 +109,8 @@ Example response: "color":"#cecece", "font":"#FFFFFF", "id":1, - "active": true + "active": true, + "target_path": "*/welcome" } ``` @@ -146,7 +149,8 @@ Example response: "color":"#000", "font":"#FFFFFF", "id":1, - "active": true + "active": true, + "target_path": "*/welcome" } ``` diff --git a/doc/api/deployments.md b/doc/api/deployments.md index 6fc6599a47d..916c99d5f89 100644 --- a/doc/api/deployments.md +++ b/doc/api/deployments.md @@ -13,6 +13,8 @@ GET /projects/:id/deployments | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `order_by`| string | no | Return deployments ordered by `id` or `iid` or `created_at` or `updated_at` or `ref` fields. Default is `id` | | `sort` | string | no | Return deployments sorted in `asc` or `desc` order. Default is `asc` | +| `updated_after` | datetime | no | Return deployments updated after the specified date | +| `updated_before` | datetime | no | Return deployments updated before the specified date | ```bash curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/deployments" diff --git a/doc/api/graphql/getting_started.md b/doc/api/graphql/getting_started.md new file mode 100644 index 00000000000..57c0fcc659c --- /dev/null +++ b/doc/api/graphql/getting_started.md @@ -0,0 +1,361 @@ +# Getting started with GitLab GraphQL API + +This guide demonstrates basic usage of GitLab's GraphQL API. + +See the [GraphQL API StyleGuide](../../development/api_graphql_styleguide.md) for implementation details +aimed at developers who wish to work on developing the API itself. + +## Running examples + +The examples documented here can be run using: + +- The command line. +- GraphiQL. + +### Command line + +You can run GraphQL queries in a `curl` request on the command line on your local machine. +A GraphQL request can be made as a `POST` request to `/api/graphql` with the query as the payload. +You can authorize your request by generating a [personal access token](../../user/profile/personal_access_tokens.md) +to use as a bearer token. + +Example: + +```sh +GRAPHQL_TOKEN=<your-token> +curl 'https://gitlab.com/api/graphql' --header "Authorization: Bearer $GRAPHQL_TOKEN" --header "Content-Type: application/json" --request POST --data "{\"query\": \"query {currentUser {name}}\"}" +``` + +### GraphiQL + +GraphiQL (pronounced “graphical”) allows you to run queries directly against the server endpoint +with syntax highlighting and autocomplete. It also allows you to explore the schema and types. + +The examples below: + +- Can be run directly against GitLab 11.0 or later, though some of the types and fields +may not be supported in older versions. +- Will work against GitLab.com without any further setup. Make sure you are signed in and +navigate to the [GraphiQL Explorer](https://www.gitlab.com/-/graphql-explorer). + +If you want to run the queries locally, or on a self-managed instance, +you will need to either: + +- Create the `gitlab-org` group with a project called `graphql-sandbox` under it. Create +several issues within the project. +- Edit the queries to replace `gitlab-org/graphql-sandbox` with your own group and project. + +Please refer to [running GraphiQL](index.md#graphiql) for more information. + +NOTE: **Note:** +If you are running GitLab 11.0 to 12.0, enable the `graphql` +[feature flag](../features.md#set-or-create-a-feature). + +## Queries and mutations + +The GitLab GraphQL API can be used to perform: + +- Queries for data retrieval. +- [Mutations](#mutations) for creating, updating, and deleting data. + +NOTE: **Note:** +In the GitLab GraphQL API, `id` generally refers to a global ID, +which is an object identifier in the format of `gid://gitlab/Issue/123`. + +[GitLab's GraphQL Schema](reference/index.md) outlines which objects and fields are +available for clients to query and their corresponding data types. + +Example: Get only the names of all the projects the currently logged in user can access (up to a limit, more on that later) +in the group `gitlab-org`. + +```graphql +query { + group(fullPath: "gitlab-org") { + id + name + projects { + nodes { + name + } + } + } +} +``` + +Example: Get a specific project and the title of Issue #2. + +```graphql +query { + project(fullPath: "gitlab-org/graphql-sandbox") { + name + issue(iid: "2") { + title + } + } +} +``` + +### Graph traversal + +When retrieving child nodes use: + +- the `edges { node { } }` syntax. +- the short form `nodes { }` syntax. + +Underneath it all is a graph we are traversing, hence the name GraphQL. + +Example: Get a project (only its name) and the titles of all its issues. + +```graphql +query { + project(fullPath: "gitlab-org/graphql-sandbox") { + name + issues { + nodes { + title + description + } + } + } +} +``` + +More about queries: +[GraphQL docs](https://graphql.org/learn/queries/) + +### Authorization + +Authorization uses the same engine as the GitLab application (and GitLab.com). So if you've signed in to GitLab +and use GraphiQL, all queries will be performed as you, the signed in user. For more information, see the +[GitLab API documentation](../README.md#authentication). + +### Mutations + +Mutations make changes to data. We can update, delete, or create new records. Mutations +generally use InputTypes and variables, neither of which appear here. + +Mutations have: + +- Inputs. For example, arguments, such as which emoji you'd like to award, +and to which object. +- Return statements. That is, what you'd like to get back when it's successful. +- Errors. Always ask for what went wrong, just in case. + +#### Creation mutations + +Example: Let's have some tea - add a `:tea:` reaction emoji to an issue. + +```graphql +mutation { + addAwardEmoji(input: { awardableId: "gid://gitlab/Issue/27039960", + name: "tea" + }) { + awardEmoji { + name + description + unicode + emoji + unicodeVersion + user { + name + } + } + errors + } +} +``` + +Example: Add a comment to the issue (we're using the ID of the `GitLab.com` issue - but +if you're using a local instance, you'll need to get the ID of an issue you can write to). + +```graphql +mutation { + createNote(input: { noteableId: "gid://gitlab/Issue/27039960", + body: "*sips tea*" + }) { + note { + id + body + discussion { + id + } + } + errors + } +} +``` + +#### Update mutations + +When you see the result `id` of the note you created - take a note of it. Now let's edit it to sip faster! + +```graphql +mutation { + updateNote(input: { id: "gid://gitlab/Note/<note id>", + body: "*SIPS TEA*" + }) { + note { + id + body + } + errors + } +} +``` + +#### Deletion mutations + +Let's delete the comment, since our tea is all gone. + +```graphql +mutation { + destroyNote(input: { id: "gid://gitlab/Note/<note id>" }) { + note { + id + body + } + errors + } +} +``` + +You should get something like the following output: + +```json +{ + "data": { + "destroyNote": { + "errors": [], + "note": null + } + } +} +``` + +We've asked for the note details, but it doesn't exist anymore, so we get `null`. + +More about mutations: +[GraphQL Docs](https://graphql.org/learn/queries/#mutations). + +### Introspective queries + +Clients can query the GraphQL endpoint for information about its own schema. +by making an [introspective query](https://graphql.org/learn/introspection/). + +It is through an introspection query that the [GraphiQL Query Explorer](https://gitlab.com/-/graphql-explorer) +gets all of its knowledge about our GraphQL schema to do autocompletion and provide +its interactive `Docs` tab. + +Example: Get all the type names in the schema. + +```graphql +{ + __schema { + types { + name + } + } +} +``` + +Example: Get all the fields associated with Issue. +`kind` tells us the enum value for the type, like `OBJECT`, `SCALAR` or `INTERFACE`. + +```graphql +query IssueTypes { + __type(name: "Issue") { + kind + name + fields { + name + description + type { + name + } + } + } +} +``` + +More about introspection: +[GraphQL docs](https://graphql.org/learn/introspection/) + +## Sorting + +Some of GitLab's GraphQL endpoints allow you to specify how you'd like a collection of +objects to be sorted. You can only sort by what the schema allows you to. + +Example: Issues can be sorted by creation date: + +```graphql +query { + project(fullPath: "gitlab-org/graphql-sandbox") { + name + issues(sort: created_asc) { + nodes { + title + createdAt + } + } + } +} +``` + +## Pagination + +Pagination is a way of only asking for a subset of the records (say, the first 10). +If we want more of them, we can make another request for the next 10 from the server +(in the form of something like "please give me the next 10 records"). + +By default, GitLab's GraphQL API will return only the first 100 records of any collection. +This can be changed by using `first` or `last` arguments. Both arguments take a value, +so `first: 10` will return the first 10 records, and `last: 10` the last 10 records. + +Example: Retrieve only the first 2 issues (slicing). The `cursor` field gives us a position from which +we can retrieve further records relative to that one. + +```graphql +query { + project(fullPath: "gitlab-org/graphql-sandbox") { + name + issues(first: 2) { + edges { + node { + title + } + } + pageInfo { + endCursor + hasNextPage + } + } + } +} +``` + +Example: Retrieve the next 3. (The cursor value +`eyJpZCI6IjI3MDM4OTMzIiwiY3JlYXRlZF9hdCI6IjIwMTktMTEtMTQgMDU6NTY6NDQgVVRDIn0` +could be different, but it's the `cursor` value returned for the second issue returned above.) + +```graphql +query { + project(fullPath: "gitlab-org/graphql-sandbox") { + name + issues(first: 3, after: "eyJpZCI6IjI3MDM4OTMzIiwiY3JlYXRlZF9hdCI6IjIwMTktMTEtMTQgMDU6NTY6NDQgVVRDIn0") { + edges { + node { + title + } + cursor + } + pageInfo { + endCursor + hasNextPage + } + } + } +} +``` + +More on pagination and cursors: +[GraphQL docs](https://graphql.org/learn/pagination/) diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md index 510b36eba8f..f86802d4463 100644 --- a/doc/api/graphql/index.md +++ b/doc/api/graphql/index.md @@ -4,6 +4,27 @@ > - [Always enabled](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/30444) in GitLab 12.1. +## Getting Started + +For those new to the GitLab GraphQL API, see +[Getting started with GitLab GraphQL API](getting_started.md). + +### Quick Reference + +- GitLab's GraphQL API endpoint is located at `/api/graphql`. +- Get an [introduction to GraphQL from graphql.org](https://graphql.org/). +- GitLab supports a wide range of resources, listed in the [GraphQL API Reference](reference/index.md). + +#### GraphiQL + +Explore the GraphQL API using the interactive [GraphiQL explorer](https://gitlab.com/-/graphql-explorer), +or on your self-managed GitLab instance on +`https://<your-gitlab-site.com>/-/graphql-explorer`. + +See the [GitLab GraphQL overview](getting_started.md#graphiql) for more information about the GraphiQL Explorer. + +## What is GraphQL? + [GraphQL](https://graphql.org/) is a query language for APIs that allows clients to request exactly the data they need, making it possible to get all required data in a limited number of requests. @@ -33,11 +54,17 @@ possible. ## Available queries -A first iteration of a GraphQL API includes the following queries +The GraphQL API includes the following queries at the root level: -1. `project` : Within a project it is also possible to fetch a `mergeRequest` by IID. +1. `project` : Project information, with many of its associations such as issues and merge requests also available. 1. `group` : Basic group information and epics **(ULTIMATE)** are currently supported. 1. `namespace` : Within a namespace it is also possible to fetch `projects`. +1. `currentUser`: Information about the currently logged in user. +1. `metaData`: Metadata about GitLab and the GraphQL API. +1. `snippets`: Snippets visible to the currently logged in user. + +Root-level queries are defined in +[`app/graphql/types/query_type.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/graphql/types/query_type.rb). ### Multiplex queries @@ -58,10 +85,5 @@ Machine-readable versions are also available: - [JSON format](reference/gitlab_schema.json) - [IDL format](reference/gitlab_schema.graphql) -## GraphiQL - -The API can be explored by using the GraphiQL IDE, it is available on your -instance on `gitlab.example.com/-/graphql-explorer`. - [ce-19008]: https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/19008 [features-api]: ../features.md diff --git a/doc/api/graphql/reference/gitlab_schema.graphql b/doc/api/graphql/reference/gitlab_schema.graphql index a357c93b020..4673356cf9d 100644 --- a/doc/api/graphql/reference/gitlab_schema.graphql +++ b/doc/api/graphql/reference/gitlab_schema.graphql @@ -237,7 +237,7 @@ Autogenerated input type of CreateDiffNote """ input CreateDiffNoteInput { """ - The content note itself + Content of the note """ body: String! @@ -357,7 +357,7 @@ Autogenerated input type of CreateImageDiffNote """ input CreateImageDiffNoteInput { """ - The content note itself + Content of the note """ body: String! @@ -402,7 +402,7 @@ Autogenerated input type of CreateNote """ input CreateNoteInput { """ - The content note itself + Content of the note """ body: String! @@ -442,6 +442,66 @@ type CreateNotePayload { note: Note } +""" +Autogenerated input type of CreateSnippet +""" +input CreateSnippetInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Content of the snippet + """ + content: String! + + """ + Description of the snippet + """ + description: String + + """ + File name of the snippet + """ + fileName: String + + """ + The project full path the snippet is associated with + """ + projectPath: ID + + """ + Title of the snippet + """ + title: String! + + """ + The visibility level of the snippet + """ + visibilityLevel: VisibilityLevelsEnum! +} + +""" +Autogenerated return type of CreateSnippet +""" +type CreateSnippetPayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + The snippet after mutation + """ + snippet: Snippet +} + type Design implements Noteable { diffRefs: DiffRefs! @@ -861,6 +921,41 @@ type DestroyNotePayload { note: Note } +""" +Autogenerated input type of DestroySnippet +""" +input DestroySnippetInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + The global id of the snippet to destroy + """ + id: ID! +} + +""" +Autogenerated return type of DestroySnippet +""" +type DestroySnippetPayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + The snippet after mutation + """ + snippet: Snippet +} + type DetailedStatus { detailsPath: String! favicon: String! @@ -874,17 +969,17 @@ type DetailedStatus { input DiffImagePositionInput { """ - The merge base of the branch the comment was made on + Merge base of the branch the comment was made on """ baseSha: String """ - The sha of the head at the time the comment was made + SHA of the HEAD at the time the comment was made """ headSha: String! """ - The total height of the image + Total height of the image """ height: Int! @@ -895,22 +990,22 @@ input DiffImagePositionInput { paths: DiffPathsInput! """ - The sha of the branch being compared against + SHA of the branch being compared against """ startSha: String! """ - The total width of the image + Total width of the image """ width: Int! """ - The X postion on which the comment was made + X position on which the comment was made """ x: Int! """ - The Y position on which the comment was made + Y position on which the comment was made """ y: Int! } @@ -928,73 +1023,80 @@ input DiffPathsInput { } type DiffPosition { + """ + Information about the branch, HEAD, and base at the time of commenting + """ diffRefs: DiffRefs! """ - The path of the file that was changed + Path of the file that was changed """ filePath: String! """ - The total height of the image + Total height of the image """ height: Int """ - The line on head sha that was changed + Line on HEAD SHA that was changed """ newLine: Int """ - The path of the file on the head sha. + Path of the file on the HEAD SHA """ newPath: String """ - The line on start sha that was changed + Line on start SHA that was changed """ oldLine: Int """ - The path of the file on the start sha. + Path of the file on the start SHA """ oldPath: String + + """ + Type of file the position refers to + """ positionType: DiffPositionType! """ - The total width of the image + Total width of the image """ width: Int """ - The X postion on which the comment was made + X position on which the comment was made """ x: Int """ - The Y position on which the comment was made + Y position on which the comment was made """ y: Int } input DiffPositionInput { """ - The merge base of the branch the comment was made on + Merge base of the branch the comment was made on """ baseSha: String """ - The sha of the head at the time the comment was made + SHA of the HEAD at the time the comment was made """ headSha: String! """ - The line on head sha that was changed + Line on HEAD SHA that was changed """ newLine: Int! """ - The line on start sha that was changed + Line on start SHA that was changed """ oldLine: Int @@ -1005,7 +1107,7 @@ input DiffPositionInput { paths: DiffPathsInput! """ - The sha of the branch being compared against + SHA of the branch being compared against """ startSha: String! } @@ -1020,23 +1122,30 @@ enum DiffPositionType { type DiffRefs { """ - The merge base of the branch the comment was made on + Merge base of the branch the comment was made on """ baseSha: String! """ - The sha of the head at the time the comment was made + SHA of the HEAD at the time the comment was made """ headSha: String! """ - The sha of the branch being compared against + SHA of the branch being compared against """ startSha: String! } type Discussion { + """ + Timestamp of the discussion's creation + """ createdAt: Time! + + """ + ID of this discussion + """ id: ID! """ @@ -1065,7 +1174,7 @@ type Discussion { ): NoteConnection! """ - The ID used to reply to this discussion + ID used to reply to this discussion """ replyId: ID! } @@ -1128,7 +1237,14 @@ enum EntryType { } type Epic implements Noteable { + """ + Author of the epic + """ author: User! + + """ + Children (sub-epics) of the epic + """ children( """ Returns the elements in the list that come after the specified cursor. @@ -1157,12 +1273,12 @@ type Epic implements Noteable { first: Int """ - The IID of the epic, e.g., "1" + IID of the epic, e.g., "1" """ iid: ID """ - The list of IIDs of epics, e.g., [1, 2] + List of IIDs of epics, e.g., [1, 2] """ iids: [ID!] @@ -1197,13 +1313,25 @@ type Epic implements Noteable { """ state: EpicState ): EpicConnection + + """ + Timestamp of the epic's closure + """ closedAt: Time + + """ + Timestamp of the epic's creation + """ createdAt: Time """ Number of open and closed descendant epics and issues """ descendantCounts: EpicDescendantCount + + """ + Description of the epic + """ description: String """ @@ -1230,14 +1358,55 @@ type Epic implements Noteable { """ last: Int ): DiscussionConnection! + + """ + Number of downvotes the epic has received + """ + downvotes: Int! + + """ + Due date of the epic + """ dueDate: Time + + """ + Fixed due date of the epic + """ dueDateFixed: Time + + """ + Inherited due date of the epic from milestones + """ dueDateFromMilestones: Time + + """ + Indicates if the due date has been manually set + """ dueDateIsFixed: Boolean + + """ + Group to which the epic belongs + """ group: Group! + + """ + Indicates if the epic has children + """ hasChildren: Boolean! + + """ + Indicates if the epic has direct issues + """ hasIssues: Boolean! + + """ + ID of the epic + """ id: ID! + + """ + Internal ID of the epic + """ iid: ID! """ @@ -1314,6 +1483,10 @@ type Epic implements Noteable { """ last: Int ): NoteConnection! + + """ + Parent epic of the epic + """ parent: Epic """ @@ -1344,23 +1517,56 @@ type Epic implements Noteable { relationPath: String """ - The relative position of the epic in the Epic tree + The relative position of the epic in the epic tree """ relativePosition: Int + + """ + Start date of the epic + """ startDate: Time + + """ + Fixed start date of the epic + """ startDateFixed: Time + + """ + Inherited start date of the epic from milestones + """ startDateFromMilestones: Time + + """ + Indicates if the start date has been manually set + """ startDateIsFixed: Boolean + + """ + State of the epic + """ state: EpicState! """ Boolean flag for whether the currently logged in user is subscribed to this epic """ subscribed: Boolean! + + """ + Title of the epic + """ title: String + + """ + Timestamp of the epic's last activity + """ updatedAt: Time """ + Number of upvotes the epic has received + """ + upvotes: Int! + + """ Permissions for the current user on the resource """ userPermissions: EpicPermissions! @@ -1480,7 +1686,15 @@ type EpicIssue implements Noteable { The GitLab Flavored Markdown rendering of `description` """ descriptionHtml: String + + """ + Collection of design images associated with this issue + """ designCollection: DesignCollection + + """ + Deprecated. Use `design_collection`. + """ designs: DesignCollection @deprecated(reason: "use design_collection") """ @@ -1524,13 +1738,17 @@ type EpicIssue implements Noteable { dueDate: Time """ - The epic to which issue belongs + Epic to which this issue belongs """ epic: Epic + + """ + ID of the epic-issue relation + """ epicIssueId: ID! """ - The global id of the epic-issue relation + Global ID of the epic-issue relation """ id: ID @@ -1628,6 +1846,10 @@ type EpicIssue implements Noteable { """ full: Boolean = false ): String! + + """ + URI path of the epic-issue relation + """ relationPath: String """ @@ -1699,6 +1921,10 @@ type EpicIssue implements Noteable { Web URL of the issue """ webUrl: String! + + """ + Weight of the issue + """ weight: Int } @@ -1943,6 +2169,10 @@ type Group { The GitLab Flavored Markdown rendering of `description` """ descriptionHtml: String + + """ + Find a single epic + """ epic( """ Filter epics by author @@ -1956,12 +2186,12 @@ type Group { endDate: Time """ - The IID of the epic, e.g., "1" + IID of the epic, e.g., "1" """ iid: ID """ - The list of IIDs of epics, e.g., [1, 2] + List of IIDs of epics, e.g., [1, 2] """ iids: [ID!] @@ -1991,6 +2221,10 @@ type Group { """ state: EpicState ): Epic + + """ + Find epics + """ epics( """ Returns the elements in the list that come after the specified cursor. @@ -2019,12 +2253,12 @@ type Group { first: Int """ - The IID of the epic, e.g., "1" + IID of the epic, e.g., "1" """ iid: ID """ - The list of IIDs of epics, e.g., [1, 2] + List of IIDs of epics, e.g., [1, 2] """ iids: [ID!] @@ -2059,6 +2293,10 @@ type Group { """ state: EpicState ): EpicConnection + + """ + Indicates if Epics are enabled for namespace + """ epicsEnabled: Boolean """ @@ -2072,6 +2310,11 @@ type Group { fullPath: ID! """ + Indicates if Group timelogs are enabled for namespace + """ + groupTimelogsEnabled: Boolean + + """ ID of the namespace """ id: ID! @@ -2137,6 +2380,41 @@ type Group { rootStorageStatistics: RootStorageStatistics """ + Time logged in issues by group members + """ + timelogs( + """ + Returns the elements in the list that come after the specified cursor. + """ + after: String + + """ + Returns the elements in the list that come before the specified cursor. + """ + before: String + + """ + List time logs within a time range where the logged date is before end_date parameter. + """ + endDate: Time! + + """ + Returns the first _n_ elements from the list. + """ + first: Int + + """ + Returns the last _n_ elements from the list. + """ + last: Int + + """ + List time logs within a time range where the logged date is after start_date parameter. + """ + startDate: Time! + ): TimelogConnection! + + """ Permissions for the current user on the resource """ userPermissions: GroupPermissions! @@ -2223,7 +2501,15 @@ type Issue implements Noteable { The GitLab Flavored Markdown rendering of `description` """ descriptionHtml: String + + """ + Collection of design images associated with this issue + """ designCollection: DesignCollection + + """ + Deprecated. Use `design_collection`. + """ designs: DesignCollection @deprecated(reason: "use design_collection") """ @@ -2267,7 +2553,7 @@ type Issue implements Noteable { dueDate: Time """ - The epic to which issue belongs + Epic to which this issue belongs """ epic: Epic @@ -2435,6 +2721,10 @@ type Issue implements Noteable { Web URL of the issue """ webUrl: String! + + """ + Weight of the issue + """ weight: Int } @@ -2519,6 +2809,141 @@ type IssuePermissions { } """ +Autogenerated input type of IssueSetConfidential +""" +input IssueSetConfidentialInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Whether or not to set the issue as a confidential. + """ + confidential: Boolean! + + """ + The iid of the issue to mutate + """ + iid: String! + + """ + The project the issue to mutate is in + """ + projectPath: ID! +} + +""" +Autogenerated return type of IssueSetConfidential +""" +type IssueSetConfidentialPayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + The issue after mutation + """ + issue: Issue +} + +""" +Autogenerated input type of IssueSetDueDate +""" +input IssueSetDueDateInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + The desired due date for the issue + """ + dueDate: Time! + + """ + The iid of the issue to mutate + """ + iid: String! + + """ + The project the issue to mutate is in + """ + projectPath: ID! +} + +""" +Autogenerated return type of IssueSetDueDate +""" +type IssueSetDueDatePayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + The issue after mutation + """ + issue: Issue +} + +""" +Autogenerated input type of IssueSetWeight +""" +input IssueSetWeightInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + The iid of the issue to mutate + """ + iid: String! + + """ + The project the issue to mutate is in + """ + projectPath: ID! + + """ + The desired weight for the issue + """ + weight: Int! +} + +""" +Autogenerated return type of IssueSetWeight +""" +type IssueSetWeightPayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + The issue after mutation + """ + issue: Issue +} + +""" Values for sorting issues """ enum IssueSort { @@ -2538,6 +2963,16 @@ enum IssueSort { RELATIVE_POSITION_ASC """ + Weight by ascending order + """ + WEIGHT_ASC + + """ + Weight by descending order + """ + WEIGHT_DESC + + """ Created at ascending order """ created_asc @@ -2574,7 +3009,7 @@ type Label { color: String! """ - Description of the label (markdown rendered as HTML for caching) + Description of the label (Markdown rendered as HTML for caching) """ description: String @@ -2634,6 +3069,41 @@ type LabelEdge { node: Label } +""" +Autogenerated input type of MarkAsSpamSnippet +""" +input MarkAsSpamSnippetInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + The global id of the snippet to update + """ + id: ID! +} + +""" +Autogenerated return type of MarkAsSpamSnippet +""" +type MarkAsSpamSnippetPayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + The snippet after mutation + """ + snippet: Snippet +} + type MergeRequest implements Noteable { """ Indicates if members of the target project can push to the fork @@ -2676,7 +3146,7 @@ type MergeRequest implements Noteable { defaultMergeCommitMessage: String """ - Description of the merge request (markdown rendered as HTML for caching) + Description of the merge request (Markdown rendered as HTML for caching) """ description: String @@ -3496,11 +3966,17 @@ type Mutation { createEpic(input: CreateEpicInput!): CreateEpicPayload createImageDiffNote(input: CreateImageDiffNoteInput!): CreateImageDiffNotePayload createNote(input: CreateNoteInput!): CreateNotePayload + createSnippet(input: CreateSnippetInput!): CreateSnippetPayload designManagementDelete(input: DesignManagementDeleteInput!): DesignManagementDeletePayload designManagementUpload(input: DesignManagementUploadInput!): DesignManagementUploadPayload destroyNote(input: DestroyNoteInput!): DestroyNotePayload + destroySnippet(input: DestroySnippetInput!): DestroySnippetPayload epicSetSubscription(input: EpicSetSubscriptionInput!): EpicSetSubscriptionPayload epicTreeReorder(input: EpicTreeReorderInput!): EpicTreeReorderPayload + issueSetConfidential(input: IssueSetConfidentialInput!): IssueSetConfidentialPayload + issueSetDueDate(input: IssueSetDueDateInput!): IssueSetDueDatePayload + issueSetWeight(input: IssueSetWeightInput!): IssueSetWeightPayload + markAsSpamSnippet(input: MarkAsSpamSnippetInput!): MarkAsSpamSnippetPayload mergeRequestSetAssignees(input: MergeRequestSetAssigneesInput!): MergeRequestSetAssigneesPayload mergeRequestSetLabels(input: MergeRequestSetLabelsInput!): MergeRequestSetLabelsPayload mergeRequestSetLocked(input: MergeRequestSetLockedInput!): MergeRequestSetLockedPayload @@ -3509,9 +3985,12 @@ type Mutation { mergeRequestSetWip(input: MergeRequestSetWipInput!): MergeRequestSetWipPayload removeAwardEmoji(input: RemoveAwardEmojiInput!): RemoveAwardEmojiPayload todoMarkDone(input: TodoMarkDoneInput!): TodoMarkDonePayload + todoRestore(input: TodoRestoreInput!): TodoRestorePayload + todosMarkAllDone(input: TodosMarkAllDoneInput!): TodosMarkAllDonePayload toggleAwardEmoji(input: ToggleAwardEmojiInput!): ToggleAwardEmojiPayload updateEpic(input: UpdateEpicInput!): UpdateEpicPayload updateNote(input: UpdateNoteInput!): UpdateNotePayload + updateSnippet(input: UpdateSnippetInput!): UpdateSnippetPayload } """ @@ -3623,12 +4102,12 @@ type Namespace { type Note { """ - The user who wrote this note + User who wrote this note """ author: User! """ - The content note itself + Content of the note """ body: String! @@ -3636,12 +4115,20 @@ type Note { The GitLab Flavored Markdown rendering of `note` """ bodyHtml: String + + """ + Timestamp of the note creation + """ createdAt: Time! """ The discussion this note is a part of """ discussion: Discussion + + """ + ID of the note + """ id: ID! """ @@ -3650,25 +4137,33 @@ type Note { position: DiffPosition """ - The project this note is associated to + Project associated with the note """ project: Project + + """ + Indicates if this note can be resolved. That is, if it is a resolvable discussion or simply a standalone note + """ resolvable: Boolean! """ - The time the discussion was resolved + Timestamp of the note's resolution """ resolvedAt: Time """ - The user that resolved the discussion + User that resolved the discussion """ resolvedBy: User """ - Whether or not this note was created by the system or by a user + Indicates whether this note was created by the system or by a user """ system: Boolean! + + """ + Timestamp of the note's last activity + """ updatedAt: Time! """ @@ -3996,19 +4491,23 @@ type Project { createdBefore: Time """ - The IID of the issue, e.g., "1" + IID of the issue. For example, "1" """ iid: String """ - The list of IIDs of issues, e.g., [1, 2] + List of IIDs of issues. For example, [1, 2] """ iids: [String!] """ - Labels applied to the Issue + Labels applied to this issue """ labelName: [String] + + """ + Search query for finding issues by title or description + """ search: String """ @@ -4017,7 +4516,7 @@ type Project { sort: IssueSort = created_desc """ - Current state of Issue + Current state of this issue """ state: IssuableState @@ -4072,17 +4571,17 @@ type Project { first: Int """ - The IID of the issue, e.g., "1" + IID of the issue. For example, "1" """ iid: String """ - The list of IIDs of issues, e.g., [1, 2] + List of IIDs of issues. For example, [1, 2] """ iids: [String!] """ - Labels applied to the Issue + Labels applied to this issue """ labelName: [String] @@ -4090,6 +4589,10 @@ type Project { Returns the last _n_ elements from the list. """ last: Int + + """ + Search query for finding issues by title or description + """ search: String """ @@ -4098,7 +4601,7 @@ type Project { sort: IssueSort = created_desc """ - Current state of Issue + Current state of this issue """ state: IssuableState @@ -4297,11 +4800,66 @@ type Project { requestAccessEnabled: Boolean """ + Detailed version of a Sentry error on the project + """ + sentryDetailedError( + """ + ID of the Sentry issue + """ + id: ID! + ): SentryDetailedError + + """ + E-mail address of the service desk. + """ + serviceDeskAddress: String + + """ + Indicates if the project has service desk enabled. + """ + serviceDeskEnabled: Boolean + + """ Indicates if shared runners are enabled on the project """ sharedRunnersEnabled: Boolean """ + Snippets of the project + """ + snippets( + """ + Returns the elements in the list that come after the specified cursor. + """ + after: String + + """ + Returns the elements in the list that come before the specified cursor. + """ + before: String + + """ + Returns the first _n_ elements from the list. + """ + first: Int + + """ + Array of global snippet ids, e.g., "gid://gitlab/ProjectSnippet/1" + """ + ids: [ID!] + + """ + Returns the last _n_ elements from the list. + """ + last: Int + + """ + The visibility of the snippet + """ + visibility: VisibilityScopesEnum + ): SnippetConnection + + """ (deprecated) Does this project have snippets enabled?. Use `snippets_access_level` instead """ snippetsEnabled: Boolean @@ -4464,9 +5022,9 @@ type ProjectPermissions { createPipelineSchedule: Boolean! """ - Whether or not a user can perform `create_project_snippet` on this resource + Whether or not a user can perform `create_snippet` on this resource """ - createProjectSnippet: Boolean! + createSnippet: Boolean! """ Whether or not a user can perform `create_wiki` on this resource @@ -4633,9 +5191,14 @@ type Query { currentUser: User """ - Testing endpoint to validate the API with + Text to echo back """ - echo(text: String!): String! + echo( + """ + Text to echo back + """ + text: String! + ): String! """ Find a group @@ -4671,6 +5234,61 @@ type Query { """ fullPath: ID! ): Project + + """ + Find Snippets visible to the current user + """ + snippets( + """ + Returns the elements in the list that come after the specified cursor. + """ + after: String + + """ + The ID of an author + """ + authorId: ID + + """ + Returns the elements in the list that come before the specified cursor. + """ + before: String + + """ + Explore personal snippets + """ + explore: Boolean + + """ + Returns the first _n_ elements from the list. + """ + first: Int + + """ + Array of global snippet ids, e.g., "gid://gitlab/ProjectSnippet/1" + """ + ids: [ID!] + + """ + Returns the last _n_ elements from the list. + """ + last: Int + + """ + The ID of a project + """ + projectId: ID + + """ + The type of snippet + """ + type: TypeEnum + + """ + The visibility of the snippet + """ + visibility: VisibilityScopesEnum + ): SnippetConnection } """ @@ -4767,7 +5385,7 @@ type RootStorageStatistics { packagesSize: Int! """ - The git repository size in bytes + The Git repository size in bytes """ repositorySize: Int! @@ -4782,6 +5400,342 @@ type RootStorageStatistics { wikiSize: Int! } +type SentryDetailedError { + """ + Count of occurrences + """ + count: Int! + + """ + Culprit of the error + """ + culprit: String! + + """ + External URL of the error + """ + externalUrl: String! + + """ + Commit the error was first seen + """ + firstReleaseLastCommit: String + + """ + Release version the error was first seen + """ + firstReleaseShortVersion: String + + """ + Timestamp when the error was first seen + """ + firstSeen: Time! + + """ + Last 24hr stats of the error + """ + frequency: [SentryErrorFrequency!]! + + """ + ID (global ID) of the error + """ + id: ID! + + """ + Commit the error was last seen + """ + lastReleaseLastCommit: String + + """ + Release version the error was last seen + """ + lastReleaseShortVersion: String + + """ + Timestamp when the error was last seen + """ + lastSeen: Time! + + """ + Sentry metadata message of the error + """ + message: String + + """ + ID (Sentry ID) of the error + """ + sentryId: String! + + """ + ID of the project (Sentry project) + """ + sentryProjectId: ID! + + """ + Name of the project affected by the error + """ + sentryProjectName: String! + + """ + Slug of the project affected by the error + """ + sentryProjectSlug: String! + + """ + Short ID (Sentry ID) of the error + """ + shortId: String! + + """ + Status of the error + """ + status: SentryErrorStatus! + + """ + Title of the error + """ + title: String! + + """ + Type of the error + """ + type: String! + + """ + Count of users affected by the error + """ + userCount: Int! +} + +type SentryErrorFrequency { + """ + Count of errors received since the previously recorded time + """ + count: Int! + + """ + Time the error frequency stats were recorded + """ + time: Time! +} + +""" +State of a Sentry error +""" +enum SentryErrorStatus { + """ + Error has been ignored + """ + IGNORED + + """ + Error has been resolved + """ + RESOLVED + + """ + Error has been ignored until next release + """ + RESOLVED_IN_NEXT_RELEASE + + """ + Error is unresolved + """ + UNRESOLVED +} + +""" +Represents a snippet entry +""" +type Snippet implements Noteable { + """ + The owner of the snippet + """ + author: User! + + """ + Content of the snippet + """ + content: String! + + """ + Timestamp this snippet was created + """ + createdAt: Time! + + """ + Description of the snippet + """ + description: String + + """ + The GitLab Flavored Markdown rendering of `description` + """ + descriptionHtml: String + + """ + All discussions on this noteable + """ + discussions( + """ + Returns the elements in the list that come after the specified cursor. + """ + after: String + + """ + Returns the elements in the list that come before the specified cursor. + """ + before: String + + """ + Returns the first _n_ elements from the list. + """ + first: Int + + """ + Returns the last _n_ elements from the list. + """ + last: Int + ): DiscussionConnection! + + """ + File Name of the snippet + """ + fileName: String + + """ + Id of the snippet + """ + id: ID! + + """ + All notes on this noteable + """ + notes( + """ + Returns the elements in the list that come after the specified cursor. + """ + after: String + + """ + Returns the elements in the list that come before the specified cursor. + """ + before: String + + """ + Returns the first _n_ elements from the list. + """ + first: Int + + """ + Returns the last _n_ elements from the list. + """ + last: Int + ): NoteConnection! + + """ + The project the snippet is associated with + """ + project: Project + + """ + Raw URL of the snippet + """ + rawUrl: String! + + """ + Title of the snippet + """ + title: String! + + """ + Timestamp this snippet was updated + """ + updatedAt: Time! + + """ + Permissions for the current user on the resource + """ + userPermissions: SnippetPermissions! + + """ + Visibility Level of the snippet + """ + visibilityLevel: VisibilityLevelsEnum! + + """ + Web URL of the snippet + """ + webUrl: String! +} + +""" +The connection type for Snippet. +""" +type SnippetConnection { + """ + A list of edges. + """ + edges: [SnippetEdge] + + """ + A list of nodes. + """ + nodes: [Snippet] + + """ + Information to aid in pagination. + """ + pageInfo: PageInfo! +} + +""" +An edge in a connection. +""" +type SnippetEdge { + """ + A cursor for use in pagination. + """ + cursor: String! + + """ + The item at the end of the edge. + """ + node: Snippet +} + +type SnippetPermissions { + """ + Whether or not a user can perform `admin_snippet` on this resource + """ + adminSnippet: Boolean! + + """ + Whether or not a user can perform `award_emoji` on this resource + """ + awardEmoji: Boolean! + + """ + Whether or not a user can perform `create_note` on this resource + """ + createNote: Boolean! + + """ + Whether or not a user can perform `read_snippet` on this resource + """ + readSnippet: Boolean! + + """ + Whether or not a user can perform `report_snippet` on this resource + """ + reportSnippet: Boolean! + + """ + Whether or not a user can perform `update_snippet` on this resource + """ + updateSnippet: Boolean! +} + type Submodule implements Entry { flatPath: String! id: ID! @@ -4852,6 +5806,63 @@ Time represented in ISO 8601 """ scalar Time +type Timelog { + """ + The date when the time tracked was spent at + """ + date: Time! + + """ + The issue that logged time was added to + """ + issue: Issue + + """ + The time spent displayed in seconds + """ + timeSpent: Int! + + """ + The user that logged the time + """ + user: User! +} + +""" +The connection type for Timelog. +""" +type TimelogConnection { + """ + A list of edges. + """ + edges: [TimelogEdge] + + """ + A list of nodes. + """ + nodes: [Timelog] + + """ + Information to aid in pagination. + """ + pageInfo: PageInfo! +} + +""" +An edge in a connection. +""" +type TimelogEdge { + """ + A cursor for use in pagination. + """ + cursor: String! + + """ + The item at the end of the edge. + """ + node: Timelog +} + """ Representing a todo entry """ @@ -4982,6 +5993,41 @@ type TodoMarkDonePayload { todo: Todo! } +""" +Autogenerated input type of TodoRestore +""" +input TodoRestoreInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + The global id of the todo to restore + """ + id: ID! +} + +""" +Autogenerated return type of TodoRestore +""" +type TodoRestorePayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + The requested todo + """ + todo: Todo! +} + enum TodoStateEnum { done pending @@ -5015,6 +6061,36 @@ enum TodoTargetEnum { } """ +Autogenerated input type of TodosMarkAllDone +""" +input TodosMarkAllDoneInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String +} + +""" +Autogenerated return type of TodosMarkAllDone +""" +type TodosMarkAllDonePayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + Ids of the updated todos + """ + updatedIds: [ID!]! +} + +""" Autogenerated input type of ToggleAwardEmoji """ input ToggleAwardEmojiInput { @@ -5182,6 +6258,11 @@ type TreeEntryEdge { node: TreeEntry } +enum TypeEnum { + personal + project +} + """ Autogenerated input type of UpdateEpic """ @@ -5272,7 +6353,7 @@ Autogenerated input type of UpdateNote """ input UpdateNoteInput { """ - The content note itself + Content of the note """ body: String! @@ -5307,6 +6388,66 @@ type UpdateNotePayload { note: Note } +""" +Autogenerated input type of UpdateSnippet +""" +input UpdateSnippetInput { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Content of the snippet + """ + content: String + + """ + Description of the snippet + """ + description: String + + """ + File name of the snippet + """ + fileName: String + + """ + The global id of the snippet to update + """ + id: ID! + + """ + Title of the snippet + """ + title: String + + """ + The visibility level of the snippet + """ + visibilityLevel: VisibilityLevelsEnum +} + +""" +Autogenerated return type of UpdateSnippet +""" +type UpdateSnippetPayload { + """ + A unique identifier for the client performing the mutation. + """ + clientMutationId: String + + """ + Reasons why the mutation failed. + """ + errors: [String!]! + + """ + The snippet after mutation + """ + snippet: Snippet +} + scalar Upload type User { @@ -5321,6 +6462,46 @@ type User { name: String! """ + Snippets authored by the user + """ + snippets( + """ + Returns the elements in the list that come after the specified cursor. + """ + after: String + + """ + Returns the elements in the list that come before the specified cursor. + """ + before: String + + """ + Returns the first _n_ elements from the list. + """ + first: Int + + """ + Array of global snippet ids, e.g., "gid://gitlab/ProjectSnippet/1" + """ + ids: [ID!] + + """ + Returns the last _n_ elements from the list. + """ + last: Int + + """ + The type of snippet + """ + type: TypeEnum + + """ + The visibility of the snippet + """ + visibility: VisibilityScopesEnum + ): SnippetConnection + + """ Todos of the user """ todos( @@ -5376,6 +6557,11 @@ type User { ): TodoConnection! """ + Permissions for the current user on the resource + """ + userPermissions: UserPermissions! + + """ Username of the user. Unique within this instance of GitLab """ username: String! @@ -5419,4 +6605,23 @@ type UserEdge { The item at the end of the edge. """ node: User +} + +type UserPermissions { + """ + Whether or not a user can perform `create_snippet` on this resource + """ + createSnippet: Boolean! +} + +enum VisibilityLevelsEnum { + internal + private + public +} + +enum VisibilityScopesEnum { + internal + private + public }
\ No newline at end of file diff --git a/doc/api/graphql/reference/gitlab_schema.json b/doc/api/graphql/reference/gitlab_schema.json index fea67f28d69..398ae52c130 100644 --- a/doc/api/graphql/reference/gitlab_schema.json +++ b/doc/api/graphql/reference/gitlab_schema.json @@ -50,11 +50,11 @@ }, { "name": "echo", - "description": "Testing endpoint to validate the API with", + "description": "Text to echo back", "args": [ { "name": "text", - "description": null, + "description": "Text to echo back", "type": { "kind": "NON_NULL", "name": null, @@ -173,6 +173,127 @@ }, "isDeprecated": false, "deprecationReason": null + }, + { + "name": "snippets", + "description": "Find Snippets visible to the current user", + "args": [ + { + "name": "ids", + "description": "Array of global snippet ids, e.g., \"gid://gitlab/ProjectSnippet/1\"", + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + } + }, + "defaultValue": null + }, + { + "name": "visibility", + "description": "The visibility of the snippet", + "type": { + "kind": "ENUM", + "name": "VisibilityScopesEnum", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "authorId", + "description": "The ID of an author", + "type": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "projectId", + "description": "The ID of a project", + "type": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "type", + "description": "The type of snippet", + "type": { + "kind": "ENUM", + "name": "TypeEnum", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "explore", + "description": "Explore personal snippets", + "type": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "after", + "description": "Returns the elements in the list that come after the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "before", + "description": "Returns the elements in the list that come before the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "first", + "description": "Returns the first _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "last", + "description": "Returns the last _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "SnippetConnection", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null } ], "inputFields": null, @@ -373,7 +494,7 @@ "args": [ { "name": "iid", - "description": "The IID of the issue, e.g., \"1\"", + "description": "IID of the issue. For example, \"1\"", "type": { "kind": "SCALAR", "name": "String", @@ -383,7 +504,7 @@ }, { "name": "iids", - "description": "The list of IIDs of issues, e.g., [1, 2]", + "description": "List of IIDs of issues. For example, [1, 2]", "type": { "kind": "LIST", "name": null, @@ -401,7 +522,7 @@ }, { "name": "state", - "description": "Current state of Issue", + "description": "Current state of this issue", "type": { "kind": "ENUM", "name": "IssuableState", @@ -411,7 +532,7 @@ }, { "name": "labelName", - "description": "Labels applied to the Issue", + "description": "Labels applied to this issue", "type": { "kind": "LIST", "name": null, @@ -485,7 +606,7 @@ }, { "name": "search", - "description": null, + "description": "Search query for finding issues by title or description", "type": { "kind": "SCALAR", "name": "String", @@ -518,7 +639,7 @@ "args": [ { "name": "iid", - "description": "The IID of the issue, e.g., \"1\"", + "description": "IID of the issue. For example, \"1\"", "type": { "kind": "SCALAR", "name": "String", @@ -528,7 +649,7 @@ }, { "name": "iids", - "description": "The list of IIDs of issues, e.g., [1, 2]", + "description": "List of IIDs of issues. For example, [1, 2]", "type": { "kind": "LIST", "name": null, @@ -546,7 +667,7 @@ }, { "name": "state", - "description": "Current state of Issue", + "description": "Current state of this issue", "type": { "kind": "ENUM", "name": "IssuableState", @@ -556,7 +677,7 @@ }, { "name": "labelName", - "description": "Labels applied to the Issue", + "description": "Labels applied to this issue", "type": { "kind": "LIST", "name": null, @@ -630,7 +751,7 @@ }, { "name": "search", - "description": null, + "description": "Search query for finding issues by title or description", "type": { "kind": "SCALAR", "name": "String", @@ -1167,6 +1288,61 @@ "deprecationReason": null }, { + "name": "sentryDetailedError", + "description": "Detailed version of a Sentry error on the project", + "args": [ + { + "name": "id", + "description": "ID of the Sentry issue", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "SentryDetailedError", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "serviceDeskAddress", + "description": "E-mail address of the service desk.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "serviceDeskEnabled", + "description": "Indicates if the project has service desk enabled.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "sharedRunnersEnabled", "description": "Indicates if shared runners are enabled on the project", "args": [ @@ -1181,6 +1357,87 @@ "deprecationReason": null }, { + "name": "snippets", + "description": "Snippets of the project", + "args": [ + { + "name": "ids", + "description": "Array of global snippet ids, e.g., \"gid://gitlab/ProjectSnippet/1\"", + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + } + }, + "defaultValue": null + }, + { + "name": "visibility", + "description": "The visibility of the snippet", + "type": { + "kind": "ENUM", + "name": "VisibilityScopesEnum", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "after", + "description": "Returns the elements in the list that come after the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "before", + "description": "Returns the elements in the list that come before the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "first", + "description": "Returns the first _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "last", + "description": "Returns the last _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "SnippetConnection", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "snippetsEnabled", "description": "(deprecated) Does this project have snippets enabled?. Use `snippets_access_level` instead", "args": [ @@ -1616,8 +1873,8 @@ "deprecationReason": null }, { - "name": "createProjectSnippet", - "description": "Whether or not a user can perform `create_project_snippet` on this resource", + "name": "createSnippet", + "description": "Whether or not a user can perform `create_snippet` on this resource", "args": [ ], @@ -2418,7 +2675,7 @@ }, { "name": "repositorySize", - "description": "The git repository size in bytes", + "description": "The Git repository size in bytes", "args": [ ], @@ -2716,11 +2973,11 @@ }, { "name": "epic", - "description": null, + "description": "Find a single epic", "args": [ { "name": "iid", - "description": "The IID of the epic, e.g., \"1\"", + "description": "IID of the epic, e.g., \"1\"", "type": { "kind": "SCALAR", "name": "ID", @@ -2730,7 +2987,7 @@ }, { "name": "iids", - "description": "The list of IIDs of epics, e.g., [1, 2]", + "description": "List of IIDs of epics, e.g., [1, 2]", "type": { "kind": "LIST", "name": null, @@ -2835,11 +3092,11 @@ }, { "name": "epics", - "description": null, + "description": "Find epics", "args": [ { "name": "iid", - "description": "The IID of the epic, e.g., \"1\"", + "description": "IID of the epic, e.g., \"1\"", "type": { "kind": "SCALAR", "name": "ID", @@ -2849,7 +3106,7 @@ }, { "name": "iids", - "description": "The list of IIDs of epics, e.g., [1, 2]", + "description": "List of IIDs of epics, e.g., [1, 2]", "type": { "kind": "LIST", "name": null, @@ -2994,7 +3251,7 @@ }, { "name": "epicsEnabled", - "description": null, + "description": "Indicates if Epics are enabled for namespace", "args": [ ], @@ -3043,6 +3300,20 @@ "deprecationReason": null }, { + "name": "groupTimelogsEnabled", + "description": "Indicates if Group timelogs are enabled for namespace", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "id", "description": "ID of the namespace", "args": [ @@ -3220,6 +3491,91 @@ "deprecationReason": null }, { + "name": "timelogs", + "description": "Time logged in issues by group members", + "args": [ + { + "name": "startDate", + "description": "List time logs within a time range where the logged date is after start_date parameter.", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "endDate", + "description": "List time logs within a time range where the logged date is before end_date parameter.", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "after", + "description": "Returns the elements in the list that come after the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "before", + "description": "Returns the elements in the list that come before the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "first", + "description": "Returns the first _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "last", + "description": "Returns the last _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "TimelogConnection", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "userPermissions", "description": "Permissions for the current user on the resource", "args": [ @@ -3315,7 +3671,7 @@ "fields": [ { "name": "author", - "description": null, + "description": "Author of the epic", "args": [ ], @@ -3333,11 +3689,11 @@ }, { "name": "children", - "description": null, + "description": "Children (sub-epics) of the epic", "args": [ { "name": "iid", - "description": "The IID of the epic, e.g., \"1\"", + "description": "IID of the epic, e.g., \"1\"", "type": { "kind": "SCALAR", "name": "ID", @@ -3347,7 +3703,7 @@ }, { "name": "iids", - "description": "The list of IIDs of epics, e.g., [1, 2]", + "description": "List of IIDs of epics, e.g., [1, 2]", "type": { "kind": "LIST", "name": null, @@ -3492,7 +3848,7 @@ }, { "name": "closedAt", - "description": null, + "description": "Timestamp of the epic's closure", "args": [ ], @@ -3506,7 +3862,7 @@ }, { "name": "createdAt", - "description": null, + "description": "Timestamp of the epic's creation", "args": [ ], @@ -3534,7 +3890,7 @@ }, { "name": "description", - "description": null, + "description": "Description of the epic", "args": [ ], @@ -3604,8 +3960,26 @@ "deprecationReason": null }, { + "name": "downvotes", + "description": "Number of downvotes the epic has received", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "dueDate", - "description": null, + "description": "Due date of the epic", "args": [ ], @@ -3619,7 +3993,7 @@ }, { "name": "dueDateFixed", - "description": null, + "description": "Fixed due date of the epic", "args": [ ], @@ -3633,7 +4007,7 @@ }, { "name": "dueDateFromMilestones", - "description": null, + "description": "Inherited due date of the epic from milestones", "args": [ ], @@ -3647,7 +4021,7 @@ }, { "name": "dueDateIsFixed", - "description": null, + "description": "Indicates if the due date has been manually set", "args": [ ], @@ -3661,7 +4035,7 @@ }, { "name": "group", - "description": null, + "description": "Group to which the epic belongs", "args": [ ], @@ -3679,7 +4053,7 @@ }, { "name": "hasChildren", - "description": null, + "description": "Indicates if the epic has children", "args": [ ], @@ -3697,7 +4071,7 @@ }, { "name": "hasIssues", - "description": null, + "description": "Indicates if the epic has direct issues", "args": [ ], @@ -3715,7 +4089,7 @@ }, { "name": "id", - "description": null, + "description": "ID of the epic", "args": [ ], @@ -3733,7 +4107,7 @@ }, { "name": "iid", - "description": null, + "description": "Internal ID of the epic", "args": [ ], @@ -3914,7 +4288,7 @@ }, { "name": "parent", - "description": null, + "description": "Parent epic of the epic", "args": [ ], @@ -4022,7 +4396,7 @@ }, { "name": "relativePosition", - "description": "The relative position of the epic in the Epic tree", + "description": "The relative position of the epic in the epic tree", "args": [ ], @@ -4036,7 +4410,7 @@ }, { "name": "startDate", - "description": null, + "description": "Start date of the epic", "args": [ ], @@ -4050,7 +4424,7 @@ }, { "name": "startDateFixed", - "description": null, + "description": "Fixed start date of the epic", "args": [ ], @@ -4064,7 +4438,7 @@ }, { "name": "startDateFromMilestones", - "description": null, + "description": "Inherited start date of the epic from milestones", "args": [ ], @@ -4078,7 +4452,7 @@ }, { "name": "startDateIsFixed", - "description": null, + "description": "Indicates if the start date has been manually set", "args": [ ], @@ -4092,7 +4466,7 @@ }, { "name": "state", - "description": null, + "description": "State of the epic", "args": [ ], @@ -4128,7 +4502,7 @@ }, { "name": "title", - "description": null, + "description": "Title of the epic", "args": [ ], @@ -4142,7 +4516,7 @@ }, { "name": "updatedAt", - "description": null, + "description": "Timestamp of the epic's last activity", "args": [ ], @@ -4155,6 +4529,24 @@ "deprecationReason": null }, { + "name": "upvotes", + "description": "Number of upvotes the epic has received", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "userPermissions", "description": "Permissions for the current user on the resource", "args": [ @@ -4368,6 +4760,11 @@ "kind": "OBJECT", "name": "MergeRequest", "ofType": null + }, + { + "kind": "OBJECT", + "name": "Snippet", + "ofType": null } ] }, @@ -4490,7 +4887,7 @@ "fields": [ { "name": "author", - "description": "The user who wrote this note", + "description": "User who wrote this note", "args": [ ], @@ -4508,7 +4905,7 @@ }, { "name": "body", - "description": "The content note itself", + "description": "Content of the note", "args": [ ], @@ -4540,7 +4937,7 @@ }, { "name": "createdAt", - "description": null, + "description": "Timestamp of the note creation", "args": [ ], @@ -4572,7 +4969,7 @@ }, { "name": "id", - "description": null, + "description": "ID of the note", "args": [ ], @@ -4604,7 +5001,7 @@ }, { "name": "project", - "description": "The project this note is associated to", + "description": "Project associated with the note", "args": [ ], @@ -4618,7 +5015,7 @@ }, { "name": "resolvable", - "description": null, + "description": "Indicates if this note can be resolved. That is, if it is a resolvable discussion or simply a standalone note", "args": [ ], @@ -4636,7 +5033,7 @@ }, { "name": "resolvedAt", - "description": "The time the discussion was resolved", + "description": "Timestamp of the note's resolution", "args": [ ], @@ -4650,7 +5047,7 @@ }, { "name": "resolvedBy", - "description": "The user that resolved the discussion", + "description": "User that resolved the discussion", "args": [ ], @@ -4664,7 +5061,7 @@ }, { "name": "system", - "description": "Whether or not this note was created by the system or by a user", + "description": "Indicates whether this note was created by the system or by a user", "args": [ ], @@ -4682,7 +5079,7 @@ }, { "name": "updatedAt", - "description": null, + "description": "Timestamp of the note's last activity", "args": [ ], @@ -4869,6 +5266,97 @@ "deprecationReason": null }, { + "name": "snippets", + "description": "Snippets authored by the user", + "args": [ + { + "name": "ids", + "description": "Array of global snippet ids, e.g., \"gid://gitlab/ProjectSnippet/1\"", + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + } + }, + "defaultValue": null + }, + { + "name": "visibility", + "description": "The visibility of the snippet", + "type": { + "kind": "ENUM", + "name": "VisibilityScopesEnum", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "type", + "description": "The type of snippet", + "type": { + "kind": "ENUM", + "name": "TypeEnum", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "after", + "description": "Returns the elements in the list that come after the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "before", + "description": "Returns the elements in the list that come before the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "first", + "description": "Returns the first _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "last", + "description": "Returns the last _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "SnippetConnection", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "todos", "description": "Todos of the user", "args": [ @@ -5034,6 +5522,24 @@ "deprecationReason": null }, { + "name": "userPermissions", + "description": "Permissions for the current user on the resource", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "UserPermissions", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "username", "description": "Username of the user. Unique within this instance of GitLab", "args": [ @@ -5079,6 +5585,37 @@ }, { "kind": "OBJECT", + "name": "UserPermissions", + "description": null, + "fields": [ + { + "name": "createSnippet", + "description": "Whether or not a user can perform `create_snippet` on this resource", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", "name": "TodoConnection", "description": "The connection type for Todo.", "fields": [ @@ -5475,12 +6012,160 @@ }, { "kind": "OBJECT", - "name": "Discussion", - "description": null, + "name": "SnippetConnection", + "description": "The connection type for Snippet.", + "fields": [ + { + "name": "edges", + "description": "A list of edges.", + "args": [ + + ], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "SnippetEdge", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "nodes", + "description": "A list of nodes.", + "args": [ + + ], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "Snippet", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "pageInfo", + "description": "Information to aid in pagination.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "PageInfo", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "SnippetEdge", + "description": "An edge in a connection.", "fields": [ { + "name": "cursor", + "description": "A cursor for use in pagination.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "node", + "description": "The item at the end of the edge.", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Snippet", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "Snippet", + "description": "Represents a snippet entry", + "fields": [ + { + "name": "author", + "description": "The owner of the snippet", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "User", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "content", + "description": "Content of the snippet", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "createdAt", - "description": null, + "description": "Timestamp this snippet was created", "args": [ ], @@ -5497,8 +6182,107 @@ "deprecationReason": null }, { + "name": "description", + "description": "Description of the snippet", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "descriptionHtml", + "description": "The GitLab Flavored Markdown rendering of `description`", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "discussions", + "description": "All discussions on this noteable", + "args": [ + { + "name": "after", + "description": "Returns the elements in the list that come after the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "before", + "description": "Returns the elements in the list that come before the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "first", + "description": "Returns the first _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "last", + "description": "Returns the last _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "DiscussionConnection", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "fileName", + "description": "File Name of the snippet", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "id", - "description": null, + "description": "Id of the snippet", "args": [ ], @@ -5516,7 +6300,7 @@ }, { "name": "notes", - "description": "All notes in the discussion", + "description": "All notes on this noteable", "args": [ { "name": "after", @@ -5572,8 +6356,22 @@ "deprecationReason": null }, { - "name": "replyId", - "description": "The ID used to reply to this discussion", + "name": "project", + "description": "The project the snippet is associated with", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Project", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "rawUrl", + "description": "Raw URL of the snippet", "args": [ ], @@ -5582,29 +6380,16 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "ID", + "name": "String", "ofType": null } }, "isDeprecated": false, "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [ - - ], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "OBJECT", - "name": "DiffPosition", - "description": null, - "fields": [ + }, { - "name": "diffRefs", - "description": null, + "name": "title", + "description": "Title of the snippet", "args": [ ], @@ -5612,8 +6397,8 @@ "kind": "NON_NULL", "name": null, "ofType": { - "kind": "OBJECT", - "name": "DiffRefs", + "kind": "SCALAR", + "name": "String", "ofType": null } }, @@ -5621,8 +6406,8 @@ "deprecationReason": null }, { - "name": "filePath", - "description": "The path of the file that was changed", + "name": "updatedAt", + "description": "Timestamp this snippet was updated", "args": [ ], @@ -5631,7 +6416,7 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "String", + "name": "Time", "ofType": null } }, @@ -5639,78 +6424,115 @@ "deprecationReason": null }, { - "name": "height", - "description": "The total height of the image", + "name": "userPermissions", + "description": "Permissions for the current user on the resource", "args": [ ], "type": { - "kind": "SCALAR", - "name": "Int", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "SnippetPermissions", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "newLine", - "description": "The line on head sha that was changed", + "name": "visibilityLevel", + "description": "Visibility Level of the snippet", "args": [ ], "type": { - "kind": "SCALAR", - "name": "Int", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "ENUM", + "name": "VisibilityLevelsEnum", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "newPath", - "description": "The path of the file on the head sha.", + "name": "webUrl", + "description": "Web URL of the snippet", "args": [ ], "type": { - "kind": "SCALAR", - "name": "String", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null - }, + } + ], + "inputFields": null, + "interfaces": [ { - "name": "oldLine", - "description": "The line on start sha that was changed", + "kind": "INTERFACE", + "name": "Noteable", + "ofType": null + } + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "DiscussionConnection", + "description": "The connection type for Discussion.", + "fields": [ + { + "name": "edges", + "description": "A list of edges.", "args": [ ], "type": { - "kind": "SCALAR", - "name": "Int", - "ofType": null + "kind": "LIST", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "DiscussionEdge", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "oldPath", - "description": "The path of the file on the start sha.", + "name": "nodes", + "description": "A list of nodes.", "args": [ ], "type": { - "kind": "SCALAR", - "name": "String", - "ofType": null + "kind": "LIST", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "Discussion", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "positionType", - "description": null, + "name": "pageInfo", + "description": "Information to aid in pagination.", "args": [ ], @@ -5718,8 +6540,39 @@ "kind": "NON_NULL", "name": null, "ofType": { - "kind": "ENUM", - "name": "DiffPositionType", + "kind": "OBJECT", + "name": "PageInfo", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "DiscussionEdge", + "description": "An edge in a connection.", + "fields": [ + { + "name": "cursor", + "description": "A cursor for use in pagination.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", "ofType": null } }, @@ -5727,43 +6580,139 @@ "deprecationReason": null }, { - "name": "width", - "description": "The total width of the image", + "name": "node", + "description": "The item at the end of the edge.", "args": [ ], "type": { - "kind": "SCALAR", - "name": "Int", + "kind": "OBJECT", + "name": "Discussion", "ofType": null }, "isDeprecated": false, "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "Discussion", + "description": null, + "fields": [ + { + "name": "createdAt", + "description": "Timestamp of the discussion's creation", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null }, { - "name": "x", - "description": "The X postion on which the comment was made", + "name": "id", + "description": "ID of this discussion", "args": [ ], "type": { - "kind": "SCALAR", - "name": "Int", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null }, { - "name": "y", - "description": "The Y position on which the comment was made", + "name": "notes", + "description": "All notes in the discussion", + "args": [ + { + "name": "after", + "description": "Returns the elements in the list that come after the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "before", + "description": "Returns the elements in the list that come before the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "first", + "description": "Returns the first _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "last", + "description": "Returns the last _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + } + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "NoteConnection", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "replyId", + "description": "ID used to reply to this discussion", "args": [ ], "type": { - "kind": "SCALAR", - "name": "Int", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null @@ -5778,12 +6727,12 @@ }, { "kind": "OBJECT", - "name": "DiffRefs", + "name": "SnippetPermissions", "description": null, "fields": [ { - "name": "baseSha", - "description": "The merge base of the branch the comment was made on", + "name": "adminSnippet", + "description": "Whether or not a user can perform `admin_snippet` on this resource", "args": [ ], @@ -5792,7 +6741,7 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "String", + "name": "Boolean", "ofType": null } }, @@ -5800,8 +6749,8 @@ "deprecationReason": null }, { - "name": "headSha", - "description": "The sha of the head at the time the comment was made", + "name": "awardEmoji", + "description": "Whether or not a user can perform `award_emoji` on this resource", "args": [ ], @@ -5810,7 +6759,7 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "String", + "name": "Boolean", "ofType": null } }, @@ -5818,8 +6767,8 @@ "deprecationReason": null }, { - "name": "startSha", - "description": "The sha of the branch being compared against", + "name": "createNote", + "description": "Whether or not a user can perform `create_note` on this resource", "args": [ ], @@ -5828,7 +6777,61 @@ "name": null, "ofType": { "kind": "SCALAR", - "name": "String", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "readSnippet", + "description": "Whether or not a user can perform `read_snippet` on this resource", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "reportSnippet", + "description": "Whether or not a user can perform `report_snippet` on this resource", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "updateSnippet", + "description": "Whether or not a user can perform `update_snippet` on this resource", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", "ofType": null } }, @@ -5845,20 +6848,78 @@ }, { "kind": "ENUM", - "name": "DiffPositionType", - "description": "Type of file the position refers to", + "name": "VisibilityLevelsEnum", + "description": null, "fields": null, "inputFields": null, "interfaces": null, "enumValues": [ { - "name": "text", + "name": "private", "description": null, "isDeprecated": false, "deprecationReason": null }, { - "name": "image", + "name": "internal", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "public", + "description": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "possibleTypes": null + }, + { + "kind": "ENUM", + "name": "VisibilityScopesEnum", + "description": null, + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ + { + "name": "private", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "internal", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "public", + "description": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "possibleTypes": null + }, + { + "kind": "ENUM", + "name": "TypeEnum", + "description": null, + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ + { + "name": "personal", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "project", "description": null, "isDeprecated": false, "deprecationReason": null @@ -5868,21 +6929,21 @@ }, { "kind": "OBJECT", - "name": "DiscussionConnection", - "description": "The connection type for Discussion.", + "name": "DiffPosition", + "description": null, "fields": [ { - "name": "edges", - "description": "A list of edges.", + "name": "diffRefs", + "description": "Information about the branch, HEAD, and base at the time of commenting", "args": [ ], "type": { - "kind": "LIST", + "kind": "NON_NULL", "name": null, "ofType": { "kind": "OBJECT", - "name": "DiscussionEdge", + "name": "DiffRefs", "ofType": null } }, @@ -5890,17 +6951,17 @@ "deprecationReason": null }, { - "name": "nodes", - "description": "A list of nodes.", + "name": "filePath", + "description": "Path of the file that was changed", "args": [ ], "type": { - "kind": "LIST", + "kind": "NON_NULL", "name": null, "ofType": { - "kind": "OBJECT", - "name": "Discussion", + "kind": "SCALAR", + "name": "String", "ofType": null } }, @@ -5908,8 +6969,78 @@ "deprecationReason": null }, { - "name": "pageInfo", - "description": "Information to aid in pagination.", + "name": "height", + "description": "Total height of the image", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "newLine", + "description": "Line on HEAD SHA that was changed", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "newPath", + "description": "Path of the file on the HEAD SHA", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "oldLine", + "description": "Line on start SHA that was changed", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "oldPath", + "description": "Path of the file on the start SHA", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "positionType", + "description": "Type of file the position refers to", "args": [ ], @@ -5917,13 +7048,55 @@ "kind": "NON_NULL", "name": null, "ofType": { - "kind": "OBJECT", - "name": "PageInfo", + "kind": "ENUM", + "name": "DiffPositionType", "ofType": null } }, "isDeprecated": false, "deprecationReason": null + }, + { + "name": "width", + "description": "Total width of the image", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "x", + "description": "X position on which the comment was made", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "y", + "description": "Y position on which the comment was made", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null } ], "inputFields": null, @@ -5935,12 +7108,12 @@ }, { "kind": "OBJECT", - "name": "DiscussionEdge", - "description": "An edge in a connection.", + "name": "DiffRefs", + "description": null, "fields": [ { - "name": "cursor", - "description": "A cursor for use in pagination.", + "name": "baseSha", + "description": "Merge base of the branch the comment was made on", "args": [ ], @@ -5957,15 +7130,37 @@ "deprecationReason": null }, { - "name": "node", - "description": "The item at the end of the edge.", + "name": "headSha", + "description": "SHA of the HEAD at the time the comment was made", "args": [ ], "type": { - "kind": "OBJECT", - "name": "Discussion", - "ofType": null + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "startSha", + "description": "SHA of the branch being compared against", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } }, "isDeprecated": false, "deprecationReason": null @@ -5979,6 +7174,29 @@ "possibleTypes": null }, { + "kind": "ENUM", + "name": "DiffPositionType", + "description": "Type of file the position refers to", + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ + { + "name": "text", + "description": null, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "image", + "description": null, + "isDeprecated": false, + "deprecationReason": null + } + ], + "possibleTypes": null + }, + { "kind": "OBJECT", "name": "EpicPermissions", "description": "Check permissions for the current user on an epic", @@ -6448,7 +7666,7 @@ }, { "name": "description", - "description": "Description of the label (markdown rendered as HTML for caching)", + "description": "Description of the label (Markdown rendered as HTML for caching)", "args": [ ], @@ -6916,7 +8134,7 @@ }, { "name": "designCollection", - "description": null, + "description": "Collection of design images associated with this issue", "args": [ ], @@ -6930,7 +8148,7 @@ }, { "name": "designs", - "description": null, + "description": "Deprecated. Use `design_collection`.", "args": [ ], @@ -7051,7 +8269,7 @@ }, { "name": "epic", - "description": "The epic to which issue belongs", + "description": "Epic to which this issue belongs", "args": [ ], @@ -7065,7 +8283,7 @@ }, { "name": "epicIssueId", - "description": null, + "description": "ID of the epic-issue relation", "args": [ ], @@ -7083,7 +8301,7 @@ }, { "name": "id", - "description": "The global id of the epic-issue relation", + "description": "Global ID of the epic-issue relation", "args": [ ], @@ -7319,7 +8537,7 @@ }, { "name": "relationPath", - "description": null, + "description": "URI path of the epic-issue relation", "args": [ ], @@ -7577,7 +8795,7 @@ }, { "name": "weight", - "description": null, + "description": "Weight of the issue", "args": [ ], @@ -8346,7 +9564,7 @@ }, { "name": "designCollection", - "description": null, + "description": "Collection of design images associated with this issue", "args": [ ], @@ -8360,7 +9578,7 @@ }, { "name": "designs", - "description": null, + "description": "Deprecated. Use `design_collection`.", "args": [ ], @@ -8481,7 +9699,7 @@ }, { "name": "epic", - "description": "The epic to which issue belongs", + "description": "Epic to which this issue belongs", "args": [ ], @@ -8961,7 +10179,7 @@ }, { "name": "weight", - "description": null, + "description": "Weight of the issue", "args": [ ], @@ -9771,6 +10989,199 @@ }, { "kind": "OBJECT", + "name": "TimelogConnection", + "description": "The connection type for Timelog.", + "fields": [ + { + "name": "edges", + "description": "A list of edges.", + "args": [ + + ], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "TimelogEdge", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "nodes", + "description": "A list of nodes.", + "args": [ + + ], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "Timelog", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "pageInfo", + "description": "Information to aid in pagination.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "PageInfo", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "TimelogEdge", + "description": "An edge in a connection.", + "fields": [ + { + "name": "cursor", + "description": "A cursor for use in pagination.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "node", + "description": "The item at the end of the edge.", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Timelog", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "Timelog", + "description": null, + "fields": [ + { + "name": "date", + "description": "The date when the time tracked was spent at", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "issue", + "description": "The issue that logged time was added to", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Issue", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "timeSpent", + "description": "The time spent displayed in seconds", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "user", + "description": "The user that logged the time", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "User", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", "name": "ProjectStatistics", "description": null, "fields": [ @@ -12320,7 +13731,7 @@ }, { "name": "description", - "description": "Description of the merge request (markdown rendered as HTML for caching)", + "description": "Description of the merge request (Markdown rendered as HTML for caching)", "args": [ ], @@ -13772,8 +15183,483 @@ "description": "Relative position by ascending order", "isDeprecated": false, "deprecationReason": null + }, + { + "name": "WEIGHT_ASC", + "description": "Weight by ascending order", + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "WEIGHT_DESC", + "description": "Weight by descending order", + "isDeprecated": false, + "deprecationReason": null + } + ], + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "SentryDetailedError", + "description": null, + "fields": [ + { + "name": "count", + "description": "Count of occurrences", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "culprit", + "description": "Culprit of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "externalUrl", + "description": "External URL of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "firstReleaseLastCommit", + "description": "Commit the error was first seen", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "firstReleaseShortVersion", + "description": "Release version the error was first seen", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "firstSeen", + "description": "Timestamp when the error was first seen", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "frequency", + "description": "Last 24hr stats of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "SentryErrorFrequency", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "id", + "description": "ID (global ID) of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "lastReleaseLastCommit", + "description": "Commit the error was last seen", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "lastReleaseShortVersion", + "description": "Release version the error was last seen", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "lastSeen", + "description": "Timestamp when the error was last seen", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "message", + "description": "Sentry metadata message of the error", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sentryId", + "description": "ID (Sentry ID) of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sentryProjectId", + "description": "ID of the project (Sentry project)", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sentryProjectName", + "description": "Name of the project affected by the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "sentryProjectSlug", + "description": "Slug of the project affected by the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "shortId", + "description": "Short ID (Sentry ID) of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "status", + "description": "Status of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "ENUM", + "name": "SentryErrorStatus", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "title", + "description": "Title of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "type", + "description": "Type of the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "userCount", + "description": "Count of users affected by the error", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "ENUM", + "name": "SentryErrorStatus", + "description": "State of a Sentry error", + "fields": null, + "inputFields": null, + "interfaces": null, + "enumValues": [ + { + "name": "RESOLVED", + "description": "Error has been resolved", + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "RESOLVED_IN_NEXT_RELEASE", + "description": "Error has been ignored until next release", + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "UNRESOLVED", + "description": "Error is unresolved", + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "IGNORED", + "description": "Error has been ignored", + "isDeprecated": false, + "deprecationReason": null + } + ], + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "SentryErrorFrequency", + "description": null, + "fields": [ + { + "name": "count", + "description": "Count of errors received since the previously recorded time", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "time", + "description": "Time the error frequency stats were recorded", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null } ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, "possibleTypes": null }, { @@ -13966,6 +15852,33 @@ "deprecationReason": null }, { + "name": "createSnippet", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "CreateSnippetInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "CreateSnippetPayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "designManagementDelete", "description": null, "args": [ @@ -14047,6 +15960,33 @@ "deprecationReason": null }, { + "name": "destroySnippet", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "DestroySnippetInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "DestroySnippetPayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "epicSetSubscription", "description": null, "args": [ @@ -14101,6 +16041,114 @@ "deprecationReason": null }, { + "name": "issueSetConfidential", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "IssueSetConfidentialInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "IssueSetConfidentialPayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "issueSetDueDate", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "IssueSetDueDateInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "IssueSetDueDatePayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "issueSetWeight", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "IssueSetWeightInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "IssueSetWeightPayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "markAsSpamSnippet", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "MarkAsSpamSnippetInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "MarkAsSpamSnippetPayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "mergeRequestSetAssignees", "description": null, "args": [ @@ -14317,6 +16365,60 @@ "deprecationReason": null }, { + "name": "todoRestore", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "TodoRestoreInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "TodoRestorePayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "todosMarkAllDone", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "TodosMarkAllDoneInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "TodosMarkAllDonePayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "toggleAwardEmoji", "description": null, "args": [ @@ -14396,6 +16498,33 @@ }, "isDeprecated": false, "deprecationReason": null + }, + { + "name": "updateSnippet", + "description": null, + "args": [ + { + "name": "input", + "description": null, + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "INPUT_OBJECT", + "name": "UpdateSnippetInput", + "ofType": null + } + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "UpdateSnippetPayload", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null } ], "inputFields": null, @@ -14894,6 +17023,266 @@ }, { "kind": "OBJECT", + "name": "IssueSetConfidentialPayload", + "description": "Autogenerated return type of IssueSetConfidential", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "issue", + "description": "The issue after mutation", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Issue", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "IssueSetConfidentialInput", + "description": "Autogenerated input type of IssueSetConfidential", + "fields": null, + "inputFields": [ + { + "name": "projectPath", + "description": "The project the issue to mutate is in", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "iid", + "description": "The iid of the issue to mutate", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "confidential", + "description": "Whether or not to set the issue as a confidential.", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Boolean", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "IssueSetDueDatePayload", + "description": "Autogenerated return type of IssueSetDueDate", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "issue", + "description": "The issue after mutation", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Issue", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "IssueSetDueDateInput", + "description": "Autogenerated input type of IssueSetDueDate", + "fields": null, + "inputFields": [ + { + "name": "projectPath", + "description": "The project the issue to mutate is in", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "iid", + "description": "The iid of the issue to mutate", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "dueDate", + "description": "The desired due date for the issue", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", "name": "MergeRequestSetLabelsPayload", "description": "Autogenerated return type of MergeRequestSetLabels", "fields": [ @@ -15822,7 +18211,7 @@ }, { "name": "body", - "description": "The content note itself", + "description": "Content of the note", "type": { "kind": "NON_NULL", "name": null, @@ -15948,7 +18337,7 @@ }, { "name": "body", - "description": "The content note itself", + "description": "Content of the note", "type": { "kind": "NON_NULL", "name": null, @@ -15997,7 +18386,7 @@ "inputFields": [ { "name": "headSha", - "description": "The sha of the head at the time the comment was made", + "description": "SHA of the HEAD at the time the comment was made", "type": { "kind": "NON_NULL", "name": null, @@ -16011,7 +18400,7 @@ }, { "name": "baseSha", - "description": "The merge base of the branch the comment was made on", + "description": "Merge base of the branch the comment was made on", "type": { "kind": "SCALAR", "name": "String", @@ -16021,7 +18410,7 @@ }, { "name": "startSha", - "description": "The sha of the branch being compared against", + "description": "SHA of the branch being compared against", "type": { "kind": "NON_NULL", "name": null, @@ -16049,7 +18438,7 @@ }, { "name": "oldLine", - "description": "The line on start sha that was changed", + "description": "Line on start SHA that was changed", "type": { "kind": "SCALAR", "name": "Int", @@ -16059,7 +18448,7 @@ }, { "name": "newLine", - "description": "The line on head sha that was changed", + "description": "Line on HEAD SHA that was changed", "type": { "kind": "NON_NULL", "name": null, @@ -16196,7 +18585,7 @@ }, { "name": "body", - "description": "The content note itself", + "description": "Content of the note", "type": { "kind": "NON_NULL", "name": null, @@ -16245,7 +18634,7 @@ "inputFields": [ { "name": "headSha", - "description": "The sha of the head at the time the comment was made", + "description": "SHA of the HEAD at the time the comment was made", "type": { "kind": "NON_NULL", "name": null, @@ -16259,7 +18648,7 @@ }, { "name": "baseSha", - "description": "The merge base of the branch the comment was made on", + "description": "Merge base of the branch the comment was made on", "type": { "kind": "SCALAR", "name": "String", @@ -16269,7 +18658,7 @@ }, { "name": "startSha", - "description": "The sha of the branch being compared against", + "description": "SHA of the branch being compared against", "type": { "kind": "NON_NULL", "name": null, @@ -16297,7 +18686,7 @@ }, { "name": "x", - "description": "The X postion on which the comment was made", + "description": "X position on which the comment was made", "type": { "kind": "NON_NULL", "name": null, @@ -16311,7 +18700,7 @@ }, { "name": "y", - "description": "The Y position on which the comment was made", + "description": "Y position on which the comment was made", "type": { "kind": "NON_NULL", "name": null, @@ -16325,7 +18714,7 @@ }, { "name": "width", - "description": "The total width of the image", + "description": "Total width of the image", "type": { "kind": "NON_NULL", "name": null, @@ -16339,7 +18728,7 @@ }, { "name": "height", - "description": "The total height of the image", + "description": "Total height of the image", "type": { "kind": "NON_NULL", "name": null, @@ -16445,7 +18834,7 @@ }, { "name": "body", - "description": "The content note itself", + "description": "Content of the note", "type": { "kind": "NON_NULL", "name": null, @@ -16682,6 +19071,728 @@ }, { "kind": "OBJECT", + "name": "TodoRestorePayload", + "description": "Autogenerated return type of TodoRestore", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "todo", + "description": "The requested todo", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "Todo", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "TodoRestoreInput", + "description": "Autogenerated input type of TodoRestore", + "fields": null, + "inputFields": [ + { + "name": "id", + "description": "The global id of the todo to restore", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "TodosMarkAllDonePayload", + "description": "Autogenerated return type of TodosMarkAllDone", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "updatedIds", + "description": "Ids of the updated todos", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "TodosMarkAllDoneInput", + "description": "Autogenerated input type of TodosMarkAllDone", + "fields": null, + "inputFields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "DestroySnippetPayload", + "description": "Autogenerated return type of DestroySnippet", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "snippet", + "description": "The snippet after mutation", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Snippet", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "DestroySnippetInput", + "description": "Autogenerated input type of DestroySnippet", + "fields": null, + "inputFields": [ + { + "name": "id", + "description": "The global id of the snippet to destroy", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "UpdateSnippetPayload", + "description": "Autogenerated return type of UpdateSnippet", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "snippet", + "description": "The snippet after mutation", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Snippet", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "UpdateSnippetInput", + "description": "Autogenerated input type of UpdateSnippet", + "fields": null, + "inputFields": [ + { + "name": "id", + "description": "The global id of the snippet to update", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "title", + "description": "Title of the snippet", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "fileName", + "description": "File name of the snippet", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "content", + "description": "Content of the snippet", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "description", + "description": "Description of the snippet", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "visibilityLevel", + "description": "The visibility level of the snippet", + "type": { + "kind": "ENUM", + "name": "VisibilityLevelsEnum", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "CreateSnippetPayload", + "description": "Autogenerated return type of CreateSnippet", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "snippet", + "description": "The snippet after mutation", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Snippet", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "CreateSnippetInput", + "description": "Autogenerated input type of CreateSnippet", + "fields": null, + "inputFields": [ + { + "name": "title", + "description": "Title of the snippet", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "fileName", + "description": "File name of the snippet", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "content", + "description": "Content of the snippet", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "description", + "description": "Description of the snippet", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "visibilityLevel", + "description": "The visibility level of the snippet", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "ENUM", + "name": "VisibilityLevelsEnum", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "projectPath", + "description": "The project full path the snippet is associated with", + "type": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "MarkAsSpamSnippetPayload", + "description": "Autogenerated return type of MarkAsSpamSnippet", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "snippet", + "description": "The snippet after mutation", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Snippet", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "MarkAsSpamSnippetInput", + "description": "Autogenerated input type of MarkAsSpamSnippet", + "fields": null, + "inputFields": [ + { + "name": "id", + "description": "The global id of the snippet to update", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", "name": "DesignManagementUploadPayload", "description": "Autogenerated return type of DesignManagementUpload", "fields": [ @@ -17006,6 +20117,136 @@ }, { "kind": "OBJECT", + "name": "IssueSetWeightPayload", + "description": "Autogenerated return type of IssueSetWeight", + "fields": [ + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "errors", + "description": "Reasons why the mutation failed.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + } + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "issue", + "description": "The issue after mutation", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Issue", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "INPUT_OBJECT", + "name": "IssueSetWeightInput", + "description": "Autogenerated input type of IssueSetWeight", + "fields": null, + "inputFields": [ + { + "name": "projectPath", + "description": "The project the issue to mutate is in", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "iid", + "description": "The iid of the issue to mutate", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "weight", + "description": "The desired weight for the issue", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "clientMutationId", + "description": "A unique identifier for the client performing the mutation.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + } + ], + "interfaces": null, + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", "name": "EpicTreeReorderPayload", "description": "Autogenerated return type of EpicTreeReorder", "fields": [ diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index 151e43f4cff..9fb39322f5c 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -92,6 +92,14 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `errors` | String! => Array | Reasons why the mutation failed. | | `note` | Note | The note after mutation | +### CreateSnippetPayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `snippet` | Snippet | The snippet after mutation | + ### Design | Name | Type | Description | @@ -145,6 +153,14 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `errors` | String! => Array | Reasons why the mutation failed. | | `note` | Note | The note after mutation | +### DestroySnippetPayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `snippet` | Snippet | The snippet after mutation | + ### DetailedStatus | Name | Type | Description | @@ -162,63 +178,65 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | Name | Type | Description | | --- | ---- | ---------- | -| `diffRefs` | DiffRefs! | | -| `filePath` | String! | The path of the file that was changed | -| `oldPath` | String | The path of the file on the start sha. | -| `newPath` | String | The path of the file on the head sha. | -| `positionType` | DiffPositionType! | | -| `oldLine` | Int | The line on start sha that was changed | -| `newLine` | Int | The line on head sha that was changed | -| `x` | Int | The X postion on which the comment was made | -| `y` | Int | The Y position on which the comment was made | -| `width` | Int | The total width of the image | -| `height` | Int | The total height of the image | +| `diffRefs` | DiffRefs! | Information about the branch, HEAD, and base at the time of commenting | +| `filePath` | String! | Path of the file that was changed | +| `oldPath` | String | Path of the file on the start SHA | +| `newPath` | String | Path of the file on the HEAD SHA | +| `positionType` | DiffPositionType! | Type of file the position refers to | +| `oldLine` | Int | Line on start SHA that was changed | +| `newLine` | Int | Line on HEAD SHA that was changed | +| `x` | Int | X position on which the comment was made | +| `y` | Int | Y position on which the comment was made | +| `width` | Int | Total width of the image | +| `height` | Int | Total height of the image | ### DiffRefs | Name | Type | Description | | --- | ---- | ---------- | -| `headSha` | String! | The sha of the head at the time the comment was made | -| `baseSha` | String! | The merge base of the branch the comment was made on | -| `startSha` | String! | The sha of the branch being compared against | +| `headSha` | String! | SHA of the HEAD at the time the comment was made | +| `baseSha` | String! | Merge base of the branch the comment was made on | +| `startSha` | String! | SHA of the branch being compared against | ### Discussion | Name | Type | Description | | --- | ---- | ---------- | -| `id` | ID! | | -| `replyId` | ID! | The ID used to reply to this discussion | -| `createdAt` | Time! | | +| `id` | ID! | ID of this discussion | +| `replyId` | ID! | ID used to reply to this discussion | +| `createdAt` | Time! | Timestamp of the discussion's creation | ### Epic | Name | Type | Description | | --- | ---- | ---------- | | `userPermissions` | EpicPermissions! | Permissions for the current user on the resource | -| `id` | ID! | | -| `iid` | ID! | | -| `title` | String | | -| `description` | String | | -| `state` | EpicState! | | -| `group` | Group! | | -| `parent` | Epic | | -| `author` | User! | | -| `startDate` | Time | | -| `startDateIsFixed` | Boolean | | -| `startDateFixed` | Time | | -| `startDateFromMilestones` | Time | | -| `dueDate` | Time | | -| `dueDateIsFixed` | Boolean | | -| `dueDateFixed` | Time | | -| `dueDateFromMilestones` | Time | | -| `closedAt` | Time | | -| `createdAt` | Time | | -| `updatedAt` | Time | | -| `hasChildren` | Boolean! | | -| `hasIssues` | Boolean! | | +| `id` | ID! | ID of the epic | +| `iid` | ID! | Internal ID of the epic | +| `title` | String | Title of the epic | +| `description` | String | Description of the epic | +| `state` | EpicState! | State of the epic | +| `group` | Group! | Group to which the epic belongs | +| `parent` | Epic | Parent epic of the epic | +| `author` | User! | Author of the epic | +| `startDate` | Time | Start date of the epic | +| `startDateIsFixed` | Boolean | Indicates if the start date has been manually set | +| `startDateFixed` | Time | Fixed start date of the epic | +| `startDateFromMilestones` | Time | Inherited start date of the epic from milestones | +| `dueDate` | Time | Due date of the epic | +| `dueDateIsFixed` | Boolean | Indicates if the due date has been manually set | +| `dueDateFixed` | Time | Fixed due date of the epic | +| `dueDateFromMilestones` | Time | Inherited due date of the epic from milestones | +| `upvotes` | Int! | Number of upvotes the epic has received | +| `downvotes` | Int! | Number of downvotes the epic has received | +| `closedAt` | Time | Timestamp of the epic's closure | +| `createdAt` | Time | Timestamp of the epic's creation | +| `updatedAt` | Time | Timestamp of the epic's last activity | +| `hasChildren` | Boolean! | Indicates if the epic has children | +| `hasIssues` | Boolean! | Indicates if the epic has direct issues | | `webPath` | String! | | | `webUrl` | String! | | -| `relativePosition` | Int | The relative position of the epic in the Epic tree | +| `relativePosition` | Int | The relative position of the epic in the epic tree | | `relationPath` | String | | | `reference` | String! | | | `subscribed` | Boolean! | Boolean flag for whether the currently logged in user is subscribed to this epic | @@ -263,13 +281,13 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `createdAt` | Time! | Timestamp of when the issue was created | | `updatedAt` | Time! | Timestamp of when the issue was last updated | | `taskCompletionStatus` | TaskCompletionStatus! | Task completion status of the issue | -| `epic` | Epic | The epic to which issue belongs | -| `weight` | Int | | -| `designs` | DesignCollection | | -| `designCollection` | DesignCollection | | -| `epicIssueId` | ID! | | -| `relationPath` | String | | -| `id` | ID | The global id of the epic-issue relation | +| `epic` | Epic | Epic to which this issue belongs | +| `weight` | Int | Weight of the issue | +| `designs` | DesignCollection | Deprecated. Use `design_collection`. | +| `designCollection` | DesignCollection | Collection of design images associated with this issue | +| `epicIssueId` | ID! | ID of the epic-issue relation | +| `relationPath` | String | URI path of the epic-issue relation | +| `id` | ID | Global ID of the epic-issue relation | ### EpicPermissions @@ -318,8 +336,9 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `webUrl` | String! | Web URL of the group | | `avatarUrl` | String | Avatar URL of the group | | `parent` | Group | Parent group | -| `epicsEnabled` | Boolean | | -| `epic` | Epic | | +| `epicsEnabled` | Boolean | Indicates if Epics are enabled for namespace | +| `groupTimelogsEnabled` | Boolean | Indicates if Group timelogs are enabled for namespace | +| `epic` | Epic | Find a single epic | ### GroupPermissions @@ -357,10 +376,10 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `createdAt` | Time! | Timestamp of when the issue was created | | `updatedAt` | Time! | Timestamp of when the issue was last updated | | `taskCompletionStatus` | TaskCompletionStatus! | Task completion status of the issue | -| `epic` | Epic | The epic to which issue belongs | -| `weight` | Int | | -| `designs` | DesignCollection | | -| `designCollection` | DesignCollection | | +| `epic` | Epic | Epic to which this issue belongs | +| `weight` | Int | Weight of the issue | +| `designs` | DesignCollection | Deprecated. Use `design_collection`. | +| `designCollection` | DesignCollection | Collection of design images associated with this issue | ### IssuePermissions @@ -375,17 +394,49 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `createDesign` | Boolean! | Whether or not a user can perform `create_design` on this resource | | `destroyDesign` | Boolean! | Whether or not a user can perform `destroy_design` on this resource | +### IssueSetConfidentialPayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `issue` | Issue | The issue after mutation | + +### IssueSetDueDatePayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `issue` | Issue | The issue after mutation | + +### IssueSetWeightPayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `issue` | Issue | The issue after mutation | + ### Label | Name | Type | Description | | --- | ---- | ---------- | | `id` | ID! | Label ID | -| `description` | String | Description of the label (markdown rendered as HTML for caching) | +| `description` | String | Description of the label (Markdown rendered as HTML for caching) | | `descriptionHtml` | String | The GitLab Flavored Markdown rendering of `description` | | `title` | String! | Content of the label | | `color` | String! | Background color of the label | | `textColor` | String! | Text color of the label | +### MarkAsSpamSnippetPayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `snippet` | Snippet | The snippet after mutation | + ### MergeRequest | Name | Type | Description | @@ -395,7 +446,7 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `iid` | String! | Internal ID of the merge request | | `title` | String! | Title of the merge request | | `titleHtml` | String | The GitLab Flavored Markdown rendering of `title` | -| `description` | String | Description of the merge request (markdown rendered as HTML for caching) | +| `description` | String | Description of the merge request (Markdown rendered as HTML for caching) | | `descriptionHtml` | String | The GitLab Flavored Markdown rendering of `description` | | `state` | MergeRequestState! | State of the merge request | | `createdAt` | Time! | Timestamp of when the merge request was created | @@ -542,18 +593,18 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | Name | Type | Description | | --- | ---- | ---------- | | `userPermissions` | NotePermissions! | Permissions for the current user on the resource | -| `id` | ID! | | -| `project` | Project | The project this note is associated to | -| `author` | User! | The user who wrote this note | -| `resolvedBy` | User | The user that resolved the discussion | -| `system` | Boolean! | Whether or not this note was created by the system or by a user | -| `body` | String! | The content note itself | +| `id` | ID! | ID of the note | +| `project` | Project | Project associated with the note | +| `author` | User! | User who wrote this note | +| `resolvedBy` | User | User that resolved the discussion | +| `system` | Boolean! | Indicates whether this note was created by the system or by a user | +| `body` | String! | Content of the note | | `bodyHtml` | String | The GitLab Flavored Markdown rendering of `note` | -| `createdAt` | Time! | | -| `updatedAt` | Time! | | +| `createdAt` | Time! | Timestamp of the note creation | +| `updatedAt` | Time! | Timestamp of the note's last activity | | `discussion` | Discussion | The discussion this note is a part of | -| `resolvable` | Boolean! | | -| `resolvedAt` | Time | The time the discussion was resolved | +| `resolvable` | Boolean! | Indicates if this note can be resolved. That is, if it is a resolvable discussion or simply a standalone note | +| `resolvedAt` | Time | Timestamp of the note's resolution | | `position` | DiffPosition | The position of this note on a diff | ### NotePermissions @@ -648,6 +699,9 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `repository` | Repository | Git repository of the project | | `mergeRequest` | MergeRequest | A single merge request of the project | | `issue` | Issue | A single issue of the project | +| `sentryDetailedError` | SentryDetailedError | Detailed version of a Sentry error on the project | +| `serviceDeskEnabled` | Boolean | Indicates if the project has service desk enabled. | +| `serviceDeskAddress` | String | E-mail address of the service desk. | ### ProjectPermissions @@ -670,7 +724,6 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `downloadCode` | Boolean! | Whether or not a user can perform `download_code` on this resource | | `downloadWikiCode` | Boolean! | Whether or not a user can perform `download_wiki_code` on this resource | | `forkProject` | Boolean! | Whether or not a user can perform `fork_project` on this resource | -| `createProjectSnippet` | Boolean! | Whether or not a user can perform `create_project_snippet` on this resource | | `readCommitStatus` | Boolean! | Whether or not a user can perform `read_commit_status` on this resource | | `requestAccess` | Boolean! | Whether or not a user can perform `request_access` on this resource | | `createPipeline` | Boolean! | Whether or not a user can perform `create_pipeline` on this resource | @@ -691,6 +744,7 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `destroyPages` | Boolean! | Whether or not a user can perform `destroy_pages` on this resource | | `readPagesContent` | Boolean! | Whether or not a user can perform `read_pages_content` on this resource | | `adminOperations` | Boolean! | Whether or not a user can perform `admin_operations` on this resource | +| `createSnippet` | Boolean! | Whether or not a user can perform `create_snippet` on this resource | | `readDesign` | Boolean! | Whether or not a user can perform `read_design` on this resource | | `createDesign` | Boolean! | Whether or not a user can perform `create_design` on this resource | | `destroyDesign` | Boolean! | Whether or not a user can perform `destroy_design` on this resource | @@ -729,12 +783,75 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | Name | Type | Description | | --- | ---- | ---------- | | `storageSize` | Int! | The total storage in bytes | -| `repositorySize` | Int! | The git repository size in bytes | +| `repositorySize` | Int! | The Git repository size in bytes | | `lfsObjectsSize` | Int! | The LFS objects size in bytes | | `buildArtifactsSize` | Int! | The CI artifacts size in bytes | | `packagesSize` | Int! | The packages size in bytes | | `wikiSize` | Int! | The wiki size in bytes | +### SentryDetailedError + +| Name | Type | Description | +| --- | ---- | ---------- | +| `id` | ID! | ID (global ID) of the error | +| `sentryId` | String! | ID (Sentry ID) of the error | +| `title` | String! | Title of the error | +| `type` | String! | Type of the error | +| `userCount` | Int! | Count of users affected by the error | +| `count` | Int! | Count of occurrences | +| `firstSeen` | Time! | Timestamp when the error was first seen | +| `lastSeen` | Time! | Timestamp when the error was last seen | +| `message` | String | Sentry metadata message of the error | +| `culprit` | String! | Culprit of the error | +| `externalUrl` | String! | External URL of the error | +| `sentryProjectId` | ID! | ID of the project (Sentry project) | +| `sentryProjectName` | String! | Name of the project affected by the error | +| `sentryProjectSlug` | String! | Slug of the project affected by the error | +| `shortId` | String! | Short ID (Sentry ID) of the error | +| `status` | SentryErrorStatus! | Status of the error | +| `frequency` | SentryErrorFrequency! => Array | Last 24hr stats of the error | +| `firstReleaseLastCommit` | String | Commit the error was first seen | +| `lastReleaseLastCommit` | String | Commit the error was last seen | +| `firstReleaseShortVersion` | String | Release version the error was first seen | +| `lastReleaseShortVersion` | String | Release version the error was last seen | + +### SentryErrorFrequency + +| Name | Type | Description | +| --- | ---- | ---------- | +| `time` | Time! | Time the error frequency stats were recorded | +| `count` | Int! | Count of errors received since the previously recorded time | + +### Snippet + +| Name | Type | Description | +| --- | ---- | ---------- | +| `userPermissions` | SnippetPermissions! | Permissions for the current user on the resource | +| `id` | ID! | Id of the snippet | +| `title` | String! | Title of the snippet | +| `project` | Project | The project the snippet is associated with | +| `author` | User! | The owner of the snippet | +| `fileName` | String | File Name of the snippet | +| `content` | String! | Content of the snippet | +| `description` | String | Description of the snippet | +| `visibilityLevel` | VisibilityLevelsEnum! | Visibility Level of the snippet | +| `createdAt` | Time! | Timestamp this snippet was created | +| `updatedAt` | Time! | Timestamp this snippet was updated | +| `webUrl` | String! | Web URL of the snippet | +| `rawUrl` | String! | Raw URL of the snippet | +| `descriptionHtml` | String | The GitLab Flavored Markdown rendering of `description` | + +### SnippetPermissions + +| Name | Type | Description | +| --- | ---- | ---------- | +| `createNote` | Boolean! | Whether or not a user can perform `create_note` on this resource | +| `awardEmoji` | Boolean! | Whether or not a user can perform `award_emoji` on this resource | +| `readSnippet` | Boolean! | Whether or not a user can perform `read_snippet` on this resource | +| `updateSnippet` | Boolean! | Whether or not a user can perform `update_snippet` on this resource | +| `adminSnippet` | Boolean! | Whether or not a user can perform `admin_snippet` on this resource | +| `reportSnippet` | Boolean! | Whether or not a user can perform `report_snippet` on this resource | + ### Submodule | Name | Type | Description | @@ -755,6 +872,15 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `count` | Int! | Number of total tasks | | `completedCount` | Int! | Number of completed tasks | +### Timelog + +| Name | Type | Description | +| --- | ---- | ---------- | +| `date` | Time! | The date when the time tracked was spent at | +| `timeSpent` | Int! | The time spent displayed in seconds | +| `user` | User! | The user that logged the time | +| `issue` | Issue | The issue that logged time was added to | + ### Todo | Name | Type | Description | @@ -777,6 +903,22 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `errors` | String! => Array | Reasons why the mutation failed. | | `todo` | Todo! | The requested todo | +### TodoRestorePayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `todo` | Todo! | The requested todo | + +### TodosMarkAllDonePayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `updatedIds` | ID! => Array | Ids of the updated todos | + ### ToggleAwardEmojiPayload | Name | Type | Description | @@ -820,11 +962,26 @@ The API can be explored interactively using the [GraphiQL IDE](../index.md#graph | `errors` | String! => Array | Reasons why the mutation failed. | | `note` | Note | The note after mutation | +### UpdateSnippetPayload + +| Name | Type | Description | +| --- | ---- | ---------- | +| `clientMutationId` | String | A unique identifier for the client performing the mutation. | +| `errors` | String! => Array | Reasons why the mutation failed. | +| `snippet` | Snippet | The snippet after mutation | + ### User | Name | Type | Description | | --- | ---- | ---------- | +| `userPermissions` | UserPermissions! | Permissions for the current user on the resource | | `name` | String! | Human-readable name of the user | | `username` | String! | Username of the user. Unique within this instance of GitLab | | `avatarUrl` | String! | URL of the user's avatar | | `webUrl` | String! | Web URL of the user | + +### UserPermissions + +| Name | Type | Description | +| --- | ---- | ---------- | +| `createSnippet` | Boolean! | Whether or not a user can perform `create_snippet` on this resource | diff --git a/doc/api/group_badges.md b/doc/api/group_badges.md index afefc3925cd..70179ecde29 100644 --- a/doc/api/group_badges.md +++ b/doc/api/group_badges.md @@ -26,9 +26,10 @@ GET /groups/:id/badges | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `name` | string | no | Name of the badges to return (case-sensitive). | ```bash -curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/:id/badges +curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/:id/badges?name=Coverage ``` Example response: @@ -36,21 +37,14 @@ Example response: ```json [ { + "name": "Coverage", "id": 1, "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", "image_url": "https://shields.io/my/badge", "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", "rendered_image_url": "https://shields.io/my/badge", "kind": "group" - }, - { - "id": 2, - "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", - "image_url": "https://shields.io/my/badge", - "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", - "rendered_image_url": "https://shields.io/my/badge", - "kind": "group" - }, + } ] ``` diff --git a/doc/api/groups.md b/doc/api/groups.md index 94f46b11a0f..32e2a88f25b 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -5,6 +5,8 @@ Get a list of visible groups for the authenticated user. When accessed without authentication, only public groups are returned. +By default, this request returns 20 results at a time because the API results [are paginated](README.md#pagination). + Parameters: | Attribute | Type | Required | Description | @@ -106,6 +108,8 @@ GET /groups?custom_attributes[key]=value&custom_attributes[other_key]=other_valu Get a list of visible direct subgroups in this group. When accessed without authentication, only public groups are returned. +By default, this request returns 20 results at a time because the API results [are paginated](README.md#pagination). + Parameters: | Attribute | Type | Required | Description | @@ -154,8 +158,9 @@ GET /groups/:id/subgroups ## List a group's projects -Get a list of projects in this group. When accessed without authentication, only -public projects are returned. +Get a list of projects in this group. When accessed without authentication, only public projects are returned. + +By default, this request returns 20 results at a time because the API results [are paginated](README.md#pagination). ``` GET /groups/:id/projects @@ -247,6 +252,13 @@ Parameters: curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/4 ``` +This endpoint returns: + +- All projects and shared projects in GitLab 12.5 and earlier. +- A maximum of 100 projects and shared projects [in GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/issues/31031) + and later. To get the details of all projects within a group, use the + [list a group's projects endpoint](#list-a-groups-projects) instead. + Example response: ```json @@ -439,6 +451,18 @@ Example response: } ``` +### Disabling the results limit + +The 100 results limit can be disabled if it breaks integrations developed using GitLab +12.4 and earlier. + +To disable the limit while migrating to using the [list a group's projects](#list-a-groups-projects) endpoint, ask a GitLab administrator +with Rails console access to run the following command: + +```ruby +Feature.disable(:limit_projects_in_groups_api) +``` + ## New group Creates a new project group. Available only for users who can create groups. @@ -518,6 +542,13 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab ``` +This endpoint returns: + +- All projects and shared projects in GitLab 12.5 and earlier. +- A maximum of 100 projects and shared projects [in GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/issues/31031) + and later. To get the details of all projects within a group, use the + [list a group's projects endpoint](#list-a-groups-projects) instead. + Example response: ```json @@ -577,6 +608,19 @@ Example response: } ``` +### Disabling the results limit + +The 100 results limit can be disabled if it breaks integrations developed using GitLab +12.4 and earlier. + +To disable the limit while migrating to using the +[list a group's projects](#list-a-groups-projects) endpoint, ask a GitLab administrator +with Rails console access to run the following command: + +```ruby +Feature.disable(:limit_projects_in_groups_api) +``` + ## Remove group Removes group with all projects inside. Only available to group owners and administrators. diff --git a/doc/api/issue_links.md b/doc/api/issue_links.md index 280431fa87c..b7e21310a19 100644 --- a/doc/api/issue_links.md +++ b/doc/api/issue_links.md @@ -57,7 +57,7 @@ Parameters: Creates a two-way relation between two issues. User must be allowed to update both issues in order to succeed. ``` -POST /projects/:id/issues/:issue_iid/links +POST /projects/:id/issues/:issue_iid/links/:target_project_id/:target_issue_iid ``` | Attribute | Type | Required | Description | diff --git a/doc/api/issues.md b/doc/api/issues.md index 54b27370741..fe551cfb397 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -113,6 +113,7 @@ Example response: "id" : 76, "title" : "Consequatur vero maxime deserunt laboriosam est voluptas dolorem.", "created_at" : "2016-01-04T15:31:51.081Z", + "moved_to_id" : null, "iid" : 6, "labels" : ["foo", "bar"], "upvotes": 4, @@ -626,7 +627,8 @@ POST /projects/:id/issues | `merge_request_to_resolve_discussions_of` | integer | no | The IID of a merge request in which to resolve all issues. This will fill the issue with a default description and mark all discussions as resolved. When passing a description or title, these values will take precedence over the default values.| | `discussion_to_resolve` | string | no | The ID of a discussion to resolve. This will fill in the issue with a default description and mark the discussion as resolved. Use in combination with `merge_request_to_resolve_discussions_of`. | | `weight` **(STARTER)** | integer | no | The weight of the issue. Valid values are greater than or equal to 0. | -| `epic_iid` **(ULTIMATE)** | integer | no | IID of the epic to add the issue to. Valid values are greater than or equal to 0. | +| `epic_id` **(ULTIMATE)** | integer | no | ID of the epic to add the issue to. Valid values are greater than or equal to 0. | +| `epic_iid` **(ULTIMATE)** | integer | no | IID of the epic to add the issue to. Valid values are greater than or equal to 0. (deprecated, [will be removed in 13.0](https://gitlab.com/gitlab-org/gitlab/issues/35157)) | ```bash curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/4/issues?title=Issues%20with%20auth&labels=bug @@ -728,7 +730,8 @@ PUT /projects/:id/issues/:issue_iid | `due_date` | string | no | Date time string in the format YEAR-MONTH-DAY, e.g. `2016-03-11` | | `weight` **(STARTER)** | integer | no | The weight of the issue. Valid values are greater than or equal to 0. 0 | | `discussion_locked` | boolean | no | Flag indicating if the issue's discussion is locked. If the discussion is locked only project members can add or edit comments. | -| `epic_iid` **(ULTIMATE)** | integer | no | IID of the epic to add the issue to. Valid values are greater than or equal to 0. | +| `epic_id` **(ULTIMATE)** | integer | no | ID of the epic to add the issue to. Valid values are greater than or equal to 0. | +| `epic_iid` **(ULTIMATE)** | integer | no | IID of the epic to add the issue to. Valid values are greater than or equal to 0. (deprecated, [will be removed in 13.0](https://gitlab.com/gitlab-org/gitlab/issues/35157)) | ```bash curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/4/issues/85?state_event=close diff --git a/doc/api/jobs.md b/doc/api/jobs.md index bafcfd110d3..5f661ac4c76 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -2,7 +2,7 @@ ## List project jobs -Get a list of jobs in a project. +Get a list of jobs in a project. Jobs are sorted in descending order of their IDs. ``` GET /projects/:id/jobs @@ -33,13 +33,23 @@ Example of response }, "coverage": null, "allow_failure": false, - "created_at": "2015-12-24T15:51:21.727Z", - "started_at": "2015-12-24T17:54:24.729Z", - "finished_at": "2015-12-24T17:54:24.921Z", - "duration": 0.192, - "artifacts_expire_at": "2016-01-23T17:54:24.921Z", - "id": 6, - "name": "rspec:other", + "created_at": "2015-12-24T15:51:21.802Z", + "started_at": "2015-12-24T17:54:27.722Z", + "finished_at": "2015-12-24T17:54:27.895Z", + "duration": 0.173, + "artifacts_file": { + "filename": "artifacts.zip", + "size": 1000 + }, + "artifacts": [ + {"file_type": "archive", "size": 1000, "filename": "artifacts.zip", "file_format": "zip"}, + {"file_type": "metadata", "size": 186, "filename": "metadata.gz", "file_format": "gzip"}, + {"file_type": "trace", "size": 1500, "filename": "job.log", "file_format": "raw"}, + {"file_type": "junit", "size": 750, "filename": "junit.xml.gz", "file_format": "gzip"} + ], + "artifacts_expire_at": "2016-01-23T17:54:27.895Z", + "id": 7, + "name": "teaspoon", "pipeline": { "id": 6, "ref": "master", @@ -52,7 +62,7 @@ Example of response "stage": "test", "status": "failed", "tag": false, - "web_url": "https://example.com/foo/bar/-/jobs/6", + "web_url": "https://example.com/foo/bar/-/jobs/7", "user": { "id": 1, "name": "Administrator", @@ -83,23 +93,13 @@ Example of response }, "coverage": null, "allow_failure": false, - "created_at": "2015-12-24T15:51:21.802Z", - "started_at": "2015-12-24T17:54:27.722Z", - "finished_at": "2015-12-24T17:54:27.895Z", - "duration": 0.173, - "artifacts_file": { - "filename": "artifacts.zip", - "size": 1000 - }, - "artifacts": [ - {"file_type": "archive", "size": 1000, "filename": "artifacts.zip", "file_format": "zip"}, - {"file_type": "metadata", "size": 186, "filename": "metadata.gz", "file_format": "gzip"}, - {"file_type": "trace", "size": 1500, "filename": "job.log", "file_format": "raw"}, - {"file_type": "junit", "size": 750, "filename": "junit.xml.gz", "file_format": "gzip"} - ], - "artifacts_expire_at": "2016-01-23T17:54:27.895Z", - "id": 7, - "name": "teaspoon", + "created_at": "2015-12-24T15:51:21.727Z", + "started_at": "2015-12-24T17:54:24.729Z", + "finished_at": "2015-12-24T17:54:24.921Z", + "duration": 0.192, + "artifacts_expire_at": "2016-01-23T17:54:24.921Z", + "id": 6, + "name": "rspec:other", "pipeline": { "id": 6, "ref": "master", @@ -112,7 +112,7 @@ Example of response "stage": "test", "status": "failed", "tag": false, - "web_url": "https://example.com/foo/bar/-/jobs/7", + "web_url": "https://example.com/foo/bar/-/jobs/6", "user": { "id": 1, "name": "Administrator", diff --git a/doc/api/keys.md b/doc/api/keys.md index 06b31a67d6a..5dedb630a27 100644 --- a/doc/api/keys.md +++ b/doc/api/keys.md @@ -4,13 +4,19 @@ Get SSH key with user by ID of an SSH key. Note only administrators can lookup SSH key with user by ID of an SSH key. -``` +```text GET /keys/:id ``` -Parameters: +| Attribute | Type | Required | Description | +|:----------|:--------|:---------|:---------------------| +| `id` | integer | yes | The ID of an SSH key | + +Example request: -- `id` (required) - The ID of an SSH key +```sh +curl --header "PRIVATE-TOKEN: <your_access_token>" 'https://gitlab.example.com/api/v4/keys/1 +``` ```json { @@ -51,3 +57,74 @@ Parameters: } } ``` + +## Get user by fingerprint of SSH key + +You can search for a user that owns a specific SSH key. Note only administrators can lookup SSH key with the fingerprint of an SSH key. + +```text +GET /keys +``` + +| Attribute | Type | Required | Description | +|:--------------|:-------|:---------|:------------------------------| +| `fingerprint` | string | yes | The fingerprint of an SSH key | + +Example request: + +```sh +curl --header "PRIVATE-TOKEN: <your_access_token>" 'https://gitlab.example.com/api/v4/keys?fingerprint=ba:81:59:68:d7:6c:cd:02:02:bf:6a:9b:55:4e:af:d1' +``` + +If using sha256 fingerprint API calls, make sure that the fingerprint is URL-encoded. + +For example, `/` is represented by `%2F` and `:` is represented by`%3A`: + +```sh +curl --header "PRIVATE-TOKEN: <your_access_token>" 'https://gitlab.example.com/api/v4/keys?fingerprint=SHA256%3AnUhzNyftwADy8AH3wFY31tAKs7HufskYTte2aXo%2FlCg +``` + +Example response: + +```json +{ + "id": 1, + "title": "Sample key 1", + "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt1016k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=", + "created_at": "2019-11-14T15:11:13.222Z", + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://0.0.0.0:3000/root", + "created_at": "2019-11-14T15:09:34.831Z", + "bio": null, + "location": null, + "public_email": "", + "skype": "", + "linkedin": "", + "twitter": "", + "website_url": "", + "organization": null, + "last_sign_in_at": "2019-11-16T22:41:26.663Z", + "confirmed_at": "2019-11-14T15:09:34.575Z", + "last_activity_on": "2019-11-20", + "email": "admin@example.com", + "theme_id": 1, + "color_scheme_id": 1, + "projects_limit": 100000, + "current_sign_in_at": "2019-11-19T14:42:18.078Z", + "identities": [ + ], + "can_create_group": true, + "can_create_project": true, + "two_factor_enabled": false, + "external": false, + "private_profile": false, + "shared_runners_minutes_limit": null, + "extra_shared_runners_minutes_limit": null + } +} +``` diff --git a/doc/api/markdown.md b/doc/api/markdown.md index f5aee725c6a..f4ad1de9ad8 100644 --- a/doc/api/markdown.md +++ b/doc/api/markdown.md @@ -12,7 +12,7 @@ POST /api/v4/markdown | Attribute | Type | Required | Description | | --------- | ------- | ------------- | ------------------------------------------ | -| `text` | string | yes | The markdown text to render | +| `text` | string | yes | The Markdown text to render | | `gfm` | boolean | no (optional) | Render text using GitLab Flavored Markdown. Default is `false` | | `project` | string | no (optional) | Use `project` as a context when creating references using GitLab Flavored Markdown. [Authentication](README.html#authentication) is required if a project is not public. | diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 7074d0249ef..541aa03450f 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -1407,7 +1407,7 @@ If the merge request is already merged or closed - you get `405` and error messa In case the merge request is not set to be merged when the pipeline succeeds, you'll also get a `406` error. ``` -PUT /projects/:id/merge_requests/:merge_request_iid/cancel_merge_when_pipeline_succeeds +POST /projects/:id/merge_requests/:merge_request_iid/cancel_merge_when_pipeline_succeeds ``` Parameters: diff --git a/doc/api/packages.md b/doc/api/packages.md index bab3f91bc40..5b490b872da 100644 --- a/doc/api/packages.md +++ b/doc/api/packages.md @@ -72,19 +72,32 @@ Example response: "id": 1, "name": "com/mycompany/my-app", "version": "1.0-SNAPSHOT", - "package_type": "maven" + "package_type": "maven", + "_links": { + "web_path": "/namespace1/project1/-/packages/1", + "delete_api_path": "/namespace1/project1/-/packages/1" + } }, { "id": 2, "name": "@foo/bar", "version": "1.0.3", - "package_type": "npm" + "package_type": "npm", + "_links": { + "web_path": "/namespace1/project1/-/packages/1", + "delete_api_path": "/namespace1/project1/-/packages/1" + } } ] ``` By default, the `GET` request will return 20 results, since the API is [paginated](README.md#pagination). +The `_links` object contains the following properties: + +- `web_path`: The path which you can visit in GitLab and see the details of the package. +- `delete_api_path`: The API path to delete the package. Only available if the request user has permission to do so. + ## Get a project package > [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/9667) in GitLab 11.9. @@ -111,10 +124,19 @@ Example response: "id": 1, "name": "com/mycompany/my-app", "version": "1.0-SNAPSHOT", - "package_type": "maven" + "package_type": "maven", + "_links": { + "web_path": "/namespace1/project1/-/packages/1", + "delete_api_path": "/namespace1/project1/-/packages/1" + } } ``` +The `_links` object contains the following properties: + +- `web_path`: The path which you can visit in GitLab and see the details of the package. +- `delete_api_path`: The API path to delete the package. Only available if the request user has permission to do so. + ## List package files > [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/9305) in GitLab 11.8. diff --git a/doc/api/pages.md b/doc/api/pages.md new file mode 100644 index 00000000000..0babca61650 --- /dev/null +++ b/doc/api/pages.md @@ -0,0 +1,21 @@ +# Pages API + +Endpoints for managing [GitLab Pages](https://about.gitlab.com/product/pages/). + +The GitLab Pages feature must be enabled to use these endpoints. Find out more about [administering](../administration/pages/index.md) and [using](../user/project/pages/index.md) the feature. + +## Unpublish pages + +Remove pages. The user must have admin priviledges. + +```text +DELETE /projects/:id/pages +``` + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | + +```bash +curl --request 'DELETE' --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/2/pages +``` diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index 90a4f8d6e26..e1b2c12dd00 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -18,7 +18,9 @@ GET /projects/:id/pipelines | `yaml_errors`| boolean | no | Returns pipelines with invalid configurations | | `name`| string | no | The name of the user who triggered pipelines | | `username`| string | no | The username of the user who triggered pipelines | -| `order_by`| string | no | Order pipelines by `id`, `status`, `ref`, or `user_id` (default: `id`) | +| `updated_after` | datetime | no | Return pipelines updated after the specified date. Format: ISO 8601 YYYY-MM-DDTHH:MM:SSZ | +| `updated_before` | datetime | no | Return pipelines updated before the specified date. Format: ISO 8601 YYYY-MM-DDTHH:MM:SSZ | +| `order_by`| string | no | Order pipelines by `id`, `status`, `ref`, `updated_at` or `user_id` (default: `id`) | | `sort` | string | no | Sort pipelines in `asc` or `desc` order (default: `desc`) | ``` diff --git a/doc/api/project_badges.md b/doc/api/project_badges.md index 527db478a50..0cf22808036 100644 --- a/doc/api/project_badges.md +++ b/doc/api/project_badges.md @@ -23,6 +23,7 @@ GET /projects/:id/badges | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `name` | string | no | Name of the badges to return (case-sensitive). | ```bash curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/:id/badges @@ -33,6 +34,7 @@ Example response: ```json [ { + "name": "Coverage", "id": 1, "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", "image_url": "https://shields.io/my/badge", @@ -41,6 +43,7 @@ Example response: "kind": "project" }, { + "name": "Pipeline", "id": 2, "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", "image_url": "https://shields.io/my/badge", diff --git a/doc/api/projects.md b/doc/api/projects.md index 222ab729810..209d41d62cd 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -61,6 +61,9 @@ GET /projects | `id_after` | integer | no | Limit results to projects with IDs greater than the specified ID | | `id_before` | integer | no | Limit results to projects with IDs less than the specified ID | +NOTE: **Note:** +This endpoint supports [keyset pagination](README.md#keyset-based-pagination) for selected `order_by` options. + When `simple=true` or the user is unauthenticated this returns something like: ```json @@ -238,6 +241,19 @@ When the user is authenticated and `simple` is not set this returns something li "remove_source_branch_after_merge": false, "request_access_enabled": false, "merge_method": "merge", + "auto_devops_enabled": true, + "auto_devops_deploy_strategy": "continuous", + "repository_storage": "default", + "approvals_before_merge": 0, + "mirror": false, + "mirror_user_id": 45, + "mirror_trigger_builds": false, + "only_mirror_protected_branches": false, + "mirror_overwrites_diverged_branches": false, + "external_authorization_classification_label": null, + "packages_enabled": true, + "service_desk_enabled": false, + "service_desk_address": null, "statistics": { "commit_count": 12, "storage_size": 2066080, @@ -309,6 +325,9 @@ GET /users/:user_id/projects | `id_after` | integer | no | Limit results to projects with IDs greater than the specified ID | | `id_before` | integer | no | Limit results to projects with IDs less than the specified ID | +NOTE: **Note:** +This endpoint supports [keyset pagination](README.md#keyset-based-pagination) for selected `order_by` options. + ```json [ { @@ -451,6 +470,19 @@ GET /users/:user_id/projects "remove_source_branch_after_merge": false, "request_access_enabled": false, "merge_method": "merge", + "auto_devops_enabled": true, + "auto_devops_deploy_strategy": "continuous", + "repository_storage": "default", + "approvals_before_merge": 0, + "mirror": false, + "mirror_user_id": 45, + "mirror_trigger_builds": false, + "only_mirror_protected_branches": false, + "mirror_overwrites_diverged_branches": false, + "external_authorization_classification_label": null, + "packages_enabled": true, + "service_desk_enabled": false, + "service_desk_address": null, "statistics": { "commit_count": 12, "storage_size": 2066080, @@ -643,6 +675,19 @@ Example response: "remove_source_branch_after_merge": false, "request_access_enabled": false, "merge_method": "merge", + "auto_devops_enabled": true, + "auto_devops_deploy_strategy": "continuous", + "repository_storage": "default", + "approvals_before_merge": 0, + "mirror": false, + "mirror_user_id": 45, + "mirror_trigger_builds": false, + "only_mirror_protected_branches": false, + "mirror_overwrites_diverged_branches": false, + "external_authorization_classification_label": null, + "packages_enabled": true, + "service_desk_enabled": false, + "service_desk_address": null, "statistics": { "commit_count": 12, "storage_size": 2066080, @@ -771,6 +816,19 @@ GET /projects/:id "printing_merge_requests_link_enabled": true, "request_access_enabled": false, "merge_method": "merge", + "auto_devops_enabled": true, + "auto_devops_deploy_strategy": "continuous", + "repository_storage": "default", + "approvals_before_merge": 0, + "mirror": false, + "mirror_user_id": 45, + "mirror_trigger_builds": false, + "only_mirror_protected_branches": false, + "mirror_overwrites_diverged_branches": false, + "external_authorization_classification_label": null, + "packages_enabled": true, + "service_desk_enabled": false, + "service_desk_address": null, "statistics": { "commit_count": 37, "storage_size": 1038090, @@ -951,6 +1009,7 @@ POST /projects | `template_project_id` | integer | no | **(PREMIUM)** When used with `use_custom_template`, project ID of a custom project template. This is preferable to using `template_name` since `template_name` may be ambiguous. | | `use_custom_template` | boolean | no | **(PREMIUM)** Use either custom [instance](../user/admin_area/custom_project_templates.md) or [group](../user/group/custom_project_templates.md) (with `group_with_project_templates_id`) project template | | `group_with_project_templates_id` | integer | no | **(PREMIUM)** For group-level custom templates, specifies ID of group from which all the custom project templates are sourced. Leave empty for instance-level templates. Requires `use_custom_template` to be true | +| `packages_enabled` | boolean | no | **(PREMIUM ONLY)** Enable or disable packages repository feature | NOTE: **Note:** If your HTTP repository is not publicly accessible, add authentication information to the URL: `https://username:password@gitlab.company.com/group/project.git` @@ -1013,6 +1072,7 @@ POST /projects/user/:user_id | `template_name` | string | no | When used without `use_custom_template`, name of a [built-in project template](../gitlab-basics/create-project.md#built-in-templates). When used with `use_custom_template`, name of a custom project template | | `use_custom_template` | boolean | no | **(PREMIUM)** Use either custom [instance](../user/admin_area/custom_project_templates.md) or [group](../user/group/custom_project_templates.md) (with `group_with_project_templates_id`) project template | | `group_with_project_templates_id` | integer | no | **(PREMIUM)** For group-level custom templates, specifies ID of group from which all the custom project templates are sourced. Leave empty for instance-level templates. Requires `use_custom_template` to be true | +| `packages_enabled` | boolean | no | **(PREMIUM ONLY)** Enable or disable packages repository feature | NOTE: **Note:** If your HTTP repository is not publicly accessible, add authentication information to the URL: `https://username:password@gitlab.company.com/group/project.git` @@ -1075,6 +1135,7 @@ PUT /projects/:id | `only_mirror_protected_branches` | boolean | no | **(STARTER)** Only mirror protected branches | | `mirror_overwrites_diverged_branches` | boolean | no | **(STARTER)** Pull mirror overwrites diverged branches | | `packages_enabled` | boolean | no | **(PREMIUM ONLY)** Enable or disable packages repository feature | +| `service_desk_enabled` | boolean | no | **(PREMIUM ONLY)** Enable or disable service desk feature | NOTE: **Note:** If your HTTP repository is not publicly accessible, add authentication information to the URL: `https://username:password@gitlab.company.com/group/project.git` @@ -1652,7 +1713,12 @@ Example response: ## Remove project -Removes a project including all associated resources (issues, merge requests etc). +This endpoint either: + +- Removes a project including all associated resources (issues, merge requests etc). +- From GitLab 12.6 on Premium or higher tiers, marks a project for deletion. Actual + deletion happens after number of days specified in + [instance settings](../user/admin_area/settings/visibility_and_access_controls.md#project-deletion-adjourned-period-premium-only). ``` DELETE /projects/:id @@ -1662,6 +1728,18 @@ DELETE /projects/:id | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +## Restore project marked for deletion **(PREMIUM)** + +Restores project marked for deletion. + +``` +POST /projects/:id/restore +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | + ## Upload a file Uploads a file to the specified project to be used in an issue or merge request description, or a comment. diff --git a/doc/api/releases/index.md b/doc/api/releases/index.md index 7f41e237401..b5e24188043 100644 --- a/doc/api/releases/index.md +++ b/doc/api/releases/index.md @@ -3,6 +3,7 @@ > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/41766) in GitLab 11.7. > - Using this API you can manipulate GitLab's [Release](../../user/project/releases/index.md) entries. > - For manipulating links as a release asset, see [Release Links API](links.md). +> - Release Evidences were [introduced](https://gitlab.com/gitlab-org/gitlab/issues/26019) in GitLab 12.5. ## List Releases @@ -87,6 +88,7 @@ Example response: ], "commit_path":"/root/awesome-app/commit/588440f66559714280628a4f9799f0c4eb880a4a", "tag_path":"/root/awesome-app/-/tags/v0.11.1", + "evidence_sha":"760d6cdfb0879c3ffedec13af470e0f71cf52c6cde4d", "assets":{ "count":6, "sources":[ @@ -120,7 +122,8 @@ Example response: "url":"http://192.168.10.15:3000", "external":true } - ] + ], + "evidence_file_path":"https://gitlab.example.com/root/awesome-app/-/releases/v0.2/evidence.json" }, }, { @@ -154,6 +157,7 @@ Example response: "committer_email":"admin@example.com", "committed_date":"2019-01-03T01:53:28.000Z" }, + "evidence_sha":"760d6cdfb0879c3ffedec13af470e0f71cf52c6cde4d", "assets":{ "count":4, "sources":[ @@ -176,7 +180,8 @@ Example response: ], "links":[ - ] + ], + "evidence_file_path":"https://gitlab.example.com/root/awesome-app/-/releases/v0.1/evidence.json" }, } ] @@ -265,8 +270,9 @@ Example response: ], "commit_path":"/root/awesome-app/commit/588440f66559714280628a4f9799f0c4eb880a4a", "tag_path":"/root/awesome-app/-/tags/v0.11.1", + "evidence_sha":"760d6cdfb0879c3ffedec13af470e0f71cf52c6cde4d", "assets":{ - "count":4, + "count":5, "sources":[ { "format":"zip", @@ -287,7 +293,8 @@ Example response: ], "links":[ - ] + ], + "evidence_url":"https://gitlab.example.com/root/awesome-app/-/releases/v0.1/evidence.json" }, } ``` @@ -305,7 +312,7 @@ POST /projects/:id/releases | `id` | integer/string | yes | The ID or [URL-encoded path of the project](../README.md#namespaced-path-encoding). | | `name` | string | no | The release name. | | `tag_name` | string | yes | The tag where the release will be created from. | -| `description` | string | yes | The description of the release. You can use [markdown](../../user/markdown.md). | +| `description` | string | yes | The description of the release. You can use [Markdown](../../user/markdown.md). | | `ref` | string | yes, if `tag_name` doesn't exist | If `tag_name` doesn't exist, the release will be created from `ref`. It can be a commit SHA, another tag name, or a branch name. | | `milestones` | array of string | no | The title of each milestone the release is associated with. | | `assets:links` | array of hash | no | An array of assets links. | @@ -385,6 +392,7 @@ Example response: ], "commit_path":"/root/awesome-app/commit/588440f66559714280628a4f9799f0c4eb880a4a", "tag_path":"/root/awesome-app/-/tags/v0.11.1", + "evidence_sha":"760d6cdfb0879c3ffedec13af470e0f71cf52c6cde4d", "assets":{ "count":5, "sources":[ @@ -412,7 +420,8 @@ Example response: "url":"https://google.com", "external":true } - ] + ], + "evidence_file_path":"https://gitlab.example.com/root/awesome-app/-/releases/v0.3/evidence.json" }, } ``` @@ -430,7 +439,7 @@ PUT /projects/:id/releases/:tag_name | `id` | integer/string | yes | The ID or [URL-encoded path of the project](../README.md#namespaced-path-encoding). | | `tag_name` | string | yes | The tag where the release will be created from. | | `name` | string | no | The release name. | -| `description` | string | no | The description of the release. You can use [markdown](../../user/markdown.md). | +| `description` | string | no | The description of the release. You can use [Markdown](../../user/markdown.md). | | `milestones` | array of string | no | The title of each milestone to associate with the release (`[]` to remove all milestones from the release). | | `released_at` | datetime | no | The date when the release will be/was ready. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | @@ -491,6 +500,7 @@ Example response: ], "commit_path":"/root/awesome-app/commit/588440f66559714280628a4f9799f0c4eb880a4a", "tag_path":"/root/awesome-app/-/tags/v0.11.1", + "evidence_sha":"760d6cdfb0879c3ffedec13af470e0f71cf52c6cde4d", "assets":{ "count":4, "sources":[ @@ -513,7 +523,8 @@ Example response: ], "links":[ - ] + ], + "evidence_file_path":"https://gitlab.example.com/root/awesome-app/-/releases/v0.1/evidence.json" }, } ``` @@ -573,6 +584,7 @@ Example response: }, "commit_path":"/root/awesome-app/commit/588440f66559714280628a4f9799f0c4eb880a4a", "tag_path":"/root/awesome-app/-/tags/v0.11.1", + "evidence_sha":"760d6cdfb0879c3ffedec13af470e0f71cf52c6cde4d", "assets":{ "count":4, "sources":[ @@ -595,7 +607,8 @@ Example response: ], "links":[ - ] + ], + "evidence_file_path":"https://gitlab.example.com/root/awesome-app/-/releases/v0.1/evidence.json" }, } ``` diff --git a/doc/api/search.md b/doc/api/search.md index 8e20722052e..46209fb753f 100644 --- a/doc/api/search.md +++ b/doc/api/search.md @@ -760,6 +760,7 @@ GET /projects/:id/search | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `scope` | string | yes | The scope to search in | | `search` | string | yes | The search query | +| `ref` | string | no | The name of a repository branch or tag to search on. The project's default branch is used by default. This is only applicable for scopes: commits, blobs, and wiki_blobs. | Search the expression within the specified scope. Currently these scopes are supported: issues, merge_requests, milestones, notes, wiki_blobs, commits, blobs, users. @@ -1058,7 +1059,7 @@ Blobs searches are performed on both filenames and contents. Search results: times in the content. ```bash -curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/6/search?scope=blobs&search=installation +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/6/search?scope=blobs&search=installation&ref=feature ``` Example response: @@ -1072,7 +1073,7 @@ Example response: "path": "README.md", "filename": "README.md", "id": null, - "ref": "master", + "ref": "feature", "startline": 46, "project_id": 6 } diff --git a/doc/api/services.md b/doc/api/services.md index 609c7e62e36..02a31ba9d38 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -229,6 +229,51 @@ Get Campfire service settings for a project. GET /projects/:id/services/campfire ``` +## Unify Circuit + +Unify Circuit RTC and collaboration tool. + +### Create/Edit Unify Circuit service + +Set Unify Circuit service for a project. + +``` +PUT /projects/:id/services/unify-circuit +``` + +Parameters: + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `webhook` | string | true | The Unify Circuit webhook. For example, `https://circuit.com/rest/v2/webhooks/incoming/...`. | +| `notify_only_broken_pipelines` | boolean | false | Send notifications for broken pipelines | +| `branches_to_be_notified` | string | all | Branches to send notifications for. Valid options are "all", "default", "protected", and "default_and_protected" | +| `push_events` | boolean | false | Enable notifications for push events | +| `issues_events` | boolean | false | Enable notifications for issue events | +| `confidential_issues_events` | boolean | false | Enable notifications for confidential issue events | +| `merge_requests_events` | boolean | false | Enable notifications for merge request events | +| `tag_push_events` | boolean | false | Enable notifications for tag push events | +| `note_events` | boolean | false | Enable notifications for note events | +| `confidential_note_events` | boolean | false | Enable notifications for confidential note events | +| `pipeline_events` | boolean | false | Enable notifications for pipeline events | +| `wiki_page_events` | boolean | false | Enable notifications for wiki page events | + +### Delete Unify Circuit service + +Delete Unify Circuit service for a project. + +``` +DELETE /projects/:id/services/unify-circuit +``` + +### Get Unify Circuit service settings + +Get Unify Circuit service settings for a project. + +``` +GET /projects/:id/services/unify-circuit +``` + ## Custom Issue Tracker Custom issue tracker @@ -480,6 +525,7 @@ Parameters: | `merge_requests_events` | boolean | false | Enable notifications for merge request events | | `tag_push_events` | boolean | false | Enable notifications for tag push events | | `note_events` | boolean | false | Enable notifications for note events | +| `confidential_note_events` | boolean | false | Enable notifications for confidential note events | | `pipeline_events` | boolean | false | Enable notifications for pipeline events | | `wiki_page_events` | boolean | false | Enable notifications for wiki page events | @@ -1088,6 +1134,7 @@ Parameters: | `merge_requests_events` | boolean | false | Enable notifications for merge request events | | `tag_push_events` | boolean | false | Enable notifications for tag push events | | `note_events` | boolean | false | Enable notifications for note events | +| `confidential_note_events` | boolean | false | Enable notifications for confidential note events | | `pipeline_events` | boolean | false | Enable notifications for pipeline events | | `wiki_page_events` | boolean | false | Enable notifications for wiki page events | | `push_channel` | string | false | The name of the channel to receive push events notifications | @@ -1095,6 +1142,7 @@ Parameters: | `confidential_issue_channel` | string | false | The name of the channel to receive confidential issues events notifications | | `merge_request_channel` | string | false | The name of the channel to receive merge request events notifications | | `note_channel` | string | false | The name of the channel to receive note events notifications | +| `confidential_note_channel` | boolean | The name of the channel to receive confidential note events notifications | | `tag_push_channel` | string | false | The name of the channel to receive tag push events notifications | | `pipeline_channel` | string | false | The name of the channel to receive pipeline events notifications | | `wiki_page_channel` | string | false | The name of the channel to receive wiki page events notifications | diff --git a/doc/api/settings.md b/doc/api/settings.md index 51d5e5f35d7..fa0efcaa5f0 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -72,14 +72,15 @@ Example response: ``` Users on GitLab [Premium or Ultimate](https://about.gitlab.com/pricing/) may also see -the `file_template_project_id` or the `geo_node_allowed_ips` parameters: +the `file_template_project_id`, `deletion_adjourned_period`, or the `geo_node_allowed_ips` parameters: ```json { "id" : 1, "signup_enabled" : true, "file_template_project_id": 1, - "geo_node_allowed_ips": "0.0.0.0/0, ::/0" + "geo_node_allowed_ips": "0.0.0.0/0, ::/0", + "deletion_adjourned_period": 7, ... } ``` @@ -162,6 +163,7 @@ these parameters: - `file_template_project_id` - `geo_node_allowed_ips` - `geo_status_timeout` +- `deletion_adjourned_period` Example responses: **(PREMIUM ONLY)** @@ -265,10 +267,11 @@ are listed in the descriptions of the relevant settings. | `html_emails_enabled` | boolean | no | Enable HTML emails. | | `import_sources` | array of strings | no | Sources to allow project import from, possible values: `github`, `bitbucket`, `bitbucket_server`, `gitlab`, `google_code`, `fogbugz`, `git`, `gitlab_project`, `gitea`, `manifest`, and `phabricator`. | | `instance_statistics_visibility_private` | boolean | no | When set to `true` Instance statistics will only be available to admins. | -| `local_markdown_version` | integer | no | Increase this value when any cached markdown should be invalidated. | +| `local_markdown_version` | integer | no | Increase this value when any cached Markdown should be invalidated. | | `max_artifacts_size` | integer | no | Maximum artifacts size in MB | | `max_attachment_size` | integer | no | Limit attachment size in MB | | `max_pages_size` | integer | no | Maximum size of pages repositories in MB | +| `max_personal_access_token_lifetime` | integer | no | **(ULTIMATE ONLY)** Maximum allowable lifetime for personal access tokens in days | | `metrics_enabled` | boolean | no | (**If enabled, requires:** `metrics_host`, `metrics_method_call_threshold`, `metrics_packet_size`, `metrics_pool_size`, `metrics_port`, `metrics_sample_interval` and `metrics_timeout`) Enable influxDB metrics. | | `metrics_host` | string | required by: `metrics_enabled` | InfluxDB host. | | `metrics_method_call_threshold` | integer | required by: `metrics_enabled` | A method call is only tracked when it takes longer than the given amount of milliseconds. | @@ -291,6 +294,7 @@ are listed in the descriptions of the relevant settings. | `plantuml_enabled` | boolean | no | (**If enabled, requires:** `plantuml_url`) Enable PlantUML integration. Default is `false`. | | `plantuml_url` | string | required by: `plantuml_enabled` | The PlantUML instance URL for integration. | | `polling_interval_multiplier` | decimal | no | Interval multiplier used by endpoints that perform polling. Set to `0` to disable polling. | +| `deletion_adjourned_period` | integer | no | **(PREMIUM ONLY)** How many days after marking project for deletion it is actually removed. Value between 0 and 90. | `project_export_enabled` | boolean | no | Enable project export. | | `prometheus_metrics_enabled` | boolean | no | Enable Prometheus metrics. | | `protected_ci_variables` | boolean | no | Environment variables are protected by default. | @@ -350,3 +354,4 @@ are listed in the descriptions of the relevant settings. | `user_show_add_ssh_key_message` | boolean | no | When set to `false` disable the "You won't be able to pull or push project code via SSH" warning shown to users with no uploaded SSH key. | | `version_check_enabled` | boolean | no | Let GitLab inform you when an update is available. | | `web_ide_clientside_preview_enabled` | boolean | no | Client side evaluation (allow live previews of JavaScript projects in the Web IDE using CodeSandbox client side evaluation). | +| `snippet_size_limit` | integer | no | Max snippet content size in **bytes**. Default: 52428800 Bytes (50MB).| diff --git a/doc/api/tags.md b/doc/api/tags.md index 13c4b83dda8..11291065edc 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -189,7 +189,7 @@ Parameters: Request body: -- `description` (required) - Release notes with markdown support +- `description` (required) - Release notes with Markdown support ```json { @@ -221,7 +221,7 @@ Parameters: Request body: -- `description` (required) - Release notes with markdown support +- `description` (required) - Release notes with Markdown support ```json { diff --git a/doc/api/users.md b/doc/api/users.md index c82a5e23c8e..4491a339d25 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -430,7 +430,7 @@ e.g. when renaming the email address to some existing one. ## User deletion Deletes a user. Available only for administrators. -This returns a `204 No Content` status code if the operation was successfully or `404` if the resource was not found. +This returns a `204 No Content` status code if the operation was successfully, `404` if the resource was not found or `409` if the user cannot be soft deleted. ``` DELETE /users/:id diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 6b8e7fa2ad5..b6518c87e13 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -23,61 +23,55 @@ how it is defined in `.gitlab-ci.yml`. NOTE: **Note:** Be careful if you use cache and artifacts to store the same path in your jobs -as **caches are restored before artifacts** and the content would be overwritten. - -Don't mix the caching with passing artifacts between stages. Caching is not -designed to pass artifacts between stages. Cache is for runtime dependencies -needed to compile the project: - -- `cache`: **Use for temporary storage for project dependencies.** Not useful - for keeping intermediate build results, like `jar` or `apk` files. - Cache was designed to be used to speed up invocations of subsequent runs of a - given job, by keeping things like dependencies (e.g., npm packages, Go vendor - packages, etc.) so they don't have to be re-fetched from the public internet. - While the cache can be abused to pass intermediate build results between - stages, there may be cases where artifacts are a better fit. +as **caches are restored before artifacts** and the content could be overwritten. + +Don't use caching for passing artifacts between stages, as it is designed to store +runtime dependencies needed to compile the project: + +- `cache`: **For storing project dependencies** + + Caches are used to speed up runs of a given job in **subsequent pipelines**, by + storing downloaded dependencies so that they don't have to be fetched from the + internet again (like npm packages, Go vendor packages, etc.) While the cache could + be configured to pass intermediate build results between stages, this should be + done with artifacts instead. + - `artifacts`: **Use for stage results that will be passed between stages.** - Artifacts were designed to upload some compiled/generated bits of the build, - and they can be fetched by any number of concurrent Runners. They are - guaranteed to be available and are there to pass data between jobs. They are - also exposed to be downloaded from the UI. **Artifacts can only exist in - directories relative to the build directory** and specifying paths which don't - comply to this rule trigger an unintuitive and illogical error message (an - enhancement is discussed at - [https://gitlab.com/gitlab-org/gitlab-foss/issues/15530](https://gitlab.com/gitlab-org/gitlab-foss/issues/15530) - ). Artifacts need to be uploaded to the GitLab instance (not only the GitLab - runner) before the next stage job(s) can start, so you need to evaluate - carefully whether your bandwidth allows you to profit from parallelization - with stages and shared artifacts before investing time in changes to the - setup. - -It's sometimes confusing because the name artifact sounds like something that -is only useful outside of the job, like for downloading a final image. But -artifacts are also available in between stages within a pipeline. So if you -build your application by downloading all the required modules, you might want -to declare them as artifacts so that each subsequent stage can depend on them -being there. There are some optimizations like declaring an -[expiry time](../yaml/README.md#artifactsexpire_in) so you don't keep artifacts -around too long, and using [dependencies](../yaml/README.md#dependencies) to -control exactly where artifacts are passed around. - -In summary: - -- Caches are disabled if not defined globally or per job (using `cache:`). -- Caches are available for all jobs in your `.gitlab-ci.yml` if enabled globally. -- Caches can be used by subsequent pipelines of that same job (a script in - a stage) in which the cache was created (if not defined globally). -- Caches are stored where the Runner is installed **and** uploaded to S3 if - [distributed cache is enabled](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching). -- Caches defined per job are only used, either: - - For the next pipeline of that job. - - If that same cache is also defined in a subsequent job of the same pipeline. -- Artifacts are disabled if not defined per job (using `artifacts:`). -- Artifacts can only be enabled per job, not globally. -- Artifacts are created during a pipeline and can be used by the subsequent - jobs of that currently active pipeline. -- Artifacts are always uploaded to GitLab (known as coordinator). -- Artifacts can have an expiration value for controlling disk usage (30 days by default). + + Artifacts are files generated by a job which are stored and uploaded, and can then + be fetched and used by jobs in later stages of the **same pipeline**. This data + will not be available in different pipelines, but is available to be downloaded + from the UI. + +The name `artifacts` sounds like it's only useful outside of the job, like for downloading +a final image, but artifacts are also available in later stages within a pipeline. +So if you build your application by downloading all the required modules, you might +want to declare them as artifacts so that subsequent stages can use them. There are +some optimizations like declaring an [expiry time](../yaml/README.md#artifactsexpire_in) +so you don't keep artifacts around too long, or using [dependencies](../yaml/README.md#dependencies) +to control which jobs fetch the artifacts. + +Caches: + +- Are disabled if not defined globally or per job (using `cache:`). +- Are available for all jobs in your `.gitlab-ci.yml` if enabled globally. +- Can be used in subsequent pipelines by the same job in which the cache was created (if not defined globally). +- Are stored where the Runner is installed **and** uploaded to S3 if [distributed cache is enabled](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching). +- If defined per job, are used: + - By the same job in a subsequent pipeline. + - By subsequent jobs in the same pipeline, if the they have identical dependencies. + +Artifacts: + +- Are disabled if not defined per job (using `artifacts:`). +- Can only be enabled per job, not globally. +- Are created during a pipeline and can be used by the subsequent jobs of that currently active pipeline. +- Are always uploaded to GitLab (known as coordinator). +- Can have an expiration value for controlling disk usage (30 days by default). + +NOTE: **Note:** +Both artifacts and caches define their paths relative to the project directory, and +can't link to files outside it. ## Good caching practices diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 489791141ed..e58fe5e4604 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -576,8 +576,8 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: will use the available system keystore to store the result of `docker login`. In that case, it's impossible to read `~/.docker/config.json`, so you will need to prepare the required base64-encoded version of - `${username}:${password}` manually. Open a terminal and execute the - following command: + `${username}:${password}` and create the Docker configuration JSON manually. + Open a terminal and execute the following command: ```bash echo -n "my_username:my_password" | base64 @@ -585,6 +585,18 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: # Example output to copy bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= ``` + + Create the Docker JSON configuration content as follows: + + ```json + { + "auths": { + "registry.example.com:5000": { + "auth": "(Base64 content from above)" + } + } + } + ``` #### Configuring a job diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index dcf4d8dde2d..ff104d4e177 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -34,13 +34,17 @@ pipelines that are run from an [external integration](../user/project/integratio ## Per-project user setting -The setting to enable or disable GitLab CI/CD Pipelines can be found in your project in -**Settings > General > Visibility, project features, permissions**. If the project -visibility is set to: +To enable or disable GitLab CI/CD Pipelines in your project: -- **Private**, only project members can access pipelines. -- **Internal** or **Public**, pipelines can be made accessible to either - project members only or everyone with access. +1. Navigate to **Settings > General > Visibility, project features, permissions**. +1. Expand the **Repository** section +1. Enable or disable the **Pipelines** checkbox as required. + +**Project visibility** will also affect pipeline visibility. If set to: + +- **Private**: Only project members can access pipelines. +- **Internal** or **Public**: Pipelines can be set to either **Only Project Members** + or **Everyone With Access** via the drop-down box. Press **Save changes** for the settings to take effect. diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 0f978a8f3a9..245a4d20e2d 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -303,6 +303,41 @@ You are not required to use the same prefix or only slashes (`/`) in the dynamic names. However, using this format will enable the [grouping similar environments](#grouping-similar-environments) feature. +### Configuring Kubernetes deployments + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/27630) in GitLab 12.6. + +If you are deploying to a [Kubernetes cluster](../user/project/clusters/index.md) +associated with your project, you can configure these deployments from your +`gitlab-ci.yml` file. + +The following configuration options are supported: + +- [`namespace`](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) + +In the following example, the job will deploy your application to the +`production` Kubernetes namespace. + +```yaml +deploy: + stage: deploy + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + kubernetes: + namespace: production + only: + - master +``` + +NOTE: **Note:** +Kubernetes configuration is not supported for Kubernetes clusters +that are [managed by GitLab](../user/project/clusters/index.md#gitlab-managed-clusters). +To follow progress on support for GitLab-managed clusters, see the +[relevant issue](https://gitlab.com/gitlab-org/gitlab/issues/38054). + ### Complete example The configuration in this section provides a full development workflow where your app is: @@ -485,6 +520,13 @@ To retry or rollback a deployment: - **Retry** button next to the last deployment, to retry that deployment. - **Rollback** button next to a previously successful deployment, to roll back to that deployment. +#### What to expect with a rollback + +Pressing the **Rollback** button on a specific commit will trigger a _new_ deployment with its +own unique job ID. + +This means that you will see a new deployment that points to the commit you are rolling back to. + NOTE: **Note:** The defined deployment process in the job's `script` determines whether the rollback succeeds or not. diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 34d53f67adc..03381dc16ff 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -83,7 +83,7 @@ gitlab-runner register \ --description "python-3.5" \ --executor "docker" \ --docker-image python:3.5 \ - --docker-postgres latest + --docker-services postgres:latest ``` With the command above, you create a runner that uses the [`python:3.5`](https://hub.docker.com/_/python) image and uses a [PostgreSQL](https://hub.docker.com/_/postgres) database. diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md index a81568d6cd4..541578db294 100644 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md @@ -367,7 +367,7 @@ If we take a look at the project's main page on the GitLab UI, we can see the st build made by GitLab CI/CD. Time to show the world our green build badge! Navigate to your project's **Settings > CI/CD** and -expand **General pipelines settings**. Scroll down to **Pipeline status** and copy the markdown code +expand **General pipelines settings**. Scroll down to **Pipeline status** and copy the Markdown code for your badge. Paste it on the top of your `README.md` file, to let people outside of our project see if our latest code is running without errors. diff --git a/doc/ci/img/collapsible_log.png b/doc/ci/img/collapsible_log.png Binary files differdeleted file mode 100644 index d2a570e246e..00000000000 --- a/doc/ci/img/collapsible_log.png +++ /dev/null diff --git a/doc/ci/img/collapsible_log_v12_6.png b/doc/ci/img/collapsible_log_v12_6.png Binary files differnew file mode 100644 index 00000000000..294421c645d --- /dev/null +++ b/doc/ci/img/collapsible_log_v12_6.png diff --git a/doc/ci/jenkins/index.md b/doc/ci/jenkins/index.md index 321d0d2778f..6e9e723feb5 100644 --- a/doc/ci/jenkins/index.md +++ b/doc/ci/jenkins/index.md @@ -228,5 +228,5 @@ our very powerful [`only/except` rules system](../yaml/README.md#onlyexcept-basi ```yaml my_job: - only: branches + only: [branches] ``` diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index c03d1798ae1..f0c3da4358a 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -171,6 +171,32 @@ cpp: junit: report.xml ``` +### .Net example + +The [JunitXML.TestLogger](https://www.nuget.org/packages/JunitXml.TestLogger/) NuGet +package can generate test reports for .Net Framework and .Net Core applications. The following +example expects a solution in the root folder of the repository, with one or more +project files in sub-folders. One result file is produced per test project, and each file +is placed in a new artifacts folder. This example includes optional formatting arguments, which +improve the readability of test data in the test widget. A full .Net Core +[example is available](https://gitlab.com/Siphonophora/dot-net-cicd-test-logging-demo). + +```yaml +## Source code and documentation are here: https://github.com/spekt/junit.testlogger/ + +Test: + stage: test + script: + - 'dotnet test --test-adapter-path:. --logger:"junit;LogFilePath=..\artifacts\{assembly}-test-result.xml;MethodFormat=Class;FailureBodyFormat=Verbose"' + artifacts: + when: always + paths: + - ./**/*test-result.xml + reports: + junit: + - ./**/*test-result.xml +``` + ## Limitations Currently, the following tools might not work because their XML formats are unsupported in GitLab. diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index a49279f1932..9ac41f88ff6 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -30,7 +30,7 @@ Pipelines for merge requests have the following requirements and limitations: ## Configuring pipelines for merge requests -To configure pipelines for merge requests, add the `only: merge_requests` parameter to +To configure pipelines for merge requests, add the `only: [merge_requests]` parameter to the jobs that you want to run only for merge requests. Then, when developers create or update merge requests, a pipeline runs @@ -68,7 +68,7 @@ After the merge request is updated with new commits: - The pipeline fetches the latest code from the source branch and run tests against it. In the above example, the pipeline contains only a `test` job. -Since the `build` and `deploy` jobs don't have the `only: merge_requests` parameter, +Since the `build` and `deploy` jobs don't have the `only: [merge_requests]` parameter, they will not run in the merge request. Pipelines tagged with the **detached** badge indicate that they were triggered @@ -86,7 +86,7 @@ Read the [documentation on Merge Trains](pipelines_for_merged_results/merge_trai ## Excluding certain jobs -The behavior of the `only: merge_requests` parameter is such that _only_ jobs with +The behavior of the `only: [merge_requests]` parameter is such that _only_ jobs with that parameter are run in the context of a merge request; no other jobs will be run. However, you may want to reverse this behavior, having all of your jobs to run _except_ @@ -134,6 +134,36 @@ to add that `only:` rule to all of your jobs in order to make them always run. Y can use this for scenarios like having only pipelines with merge requests get a Review App set up, helping to save resources. +## Excluding certain branches + +Pipelines for merge requests require special treatement when +using [`only`/`except`](../yaml/README.md#onlyexcept-basic). Unlike ordinary +branch refs (for example `refs/heads/my-feature-branch`), merge request refs +use a special Git reference that looks like `refs/merge-requests/:iid/head`. Because +of this, the following configuration will **not** work as expected: + +```yaml +# Does not exclude a branch named "docs-my-fix"! +test: + only: [merge_requests] + except: [/^docs-/] +``` + +Instead, you can use the +[`$CI_COMMIT_REF_NAME` predefined environment +variable](../variables/predefined_variables.md#variables-reference) in +combination with +[`only:variables`](../yaml/README.md#onlyvariablesexceptvariables) to +accomplish this behavior: + +```yaml +test: + only: [merge_requests] + except: + variables: + $CI_COMMIT_REF_NAME =~ /^docs-/ +``` + ## Important notes about merge requests from forked projects Note that the current behavior is subject to change. In the usual contribution diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md index 3a0848fcd08..379644beacd 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/index.md @@ -96,7 +96,7 @@ To check these feature flag values, please ask administrator to execute the foll ### Intermittently pipelines fail by `fatal: reference is not a tree:` error Since pipelines for merged results are a run on a merge ref of a merge request -(`refs/merge-requests/<iid>/merge`), the git-reference could be overwritten at an +(`refs/merge-requests/<iid>/merge`), the Git reference could be overwritten at an unexpected timing, for example, when a source or target branch is advanced. In this case, the pipeline fails because of `fatal: reference is not a tree:` error, which indicates that the checkout-SHA is not found in the merge ref. diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_confirmation_dialog_v12_6.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_confirmation_dialog_v12_6.png Binary files differnew file mode 100644 index 00000000000..241f837a748 --- /dev/null +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_confirmation_dialog_v12_6.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png Binary files differnew file mode 100644 index 00000000000..b46522395e0 --- /dev/null +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index b8976ffae7f..fca6916bee4 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -6,6 +6,7 @@ last_update: 2019-07-03 # Merge Trains **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9186) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.0. +> [Squash and merge](../../../../user/project/merge_requests/squash_and_merge.md) support [introduced](https://gitlab.com/gitlab-org/gitlab/issues/13001) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.6. [Pipelines for merged results](../index.md#pipelines-for-merged-results-premium) introduces running a build on the result of the merged code prior to merging, as a way to keep master green. @@ -36,7 +37,6 @@ Merge trains have the following requirements and limitations: If more than twenty merge requests are added to the merge train, the merge requests will be queued until a slot in the merge train is free. There is no limit to the number of merge requests that can be queued. -- This feature does not support [squash and merge](../../../../user/project/merge_requests/squash_and_merge.md). <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch this video for a demonstration on [how parallel execution @@ -86,12 +86,15 @@ In case, you have a high-priority merge request (e.g. critical patch) to be merg you can use **Merge Immediately** option for bypassing the merge train. This is the fastest option to get the change merged into the target branch. - + However, every time you merge a merge request immediately, it could affect the existing merge train to be reconstructed, specifically, it regenerates expected merge commits and pipelines. This means, merging immediately essentially wastes -CI resources. +CI resources. Because of these downsides, you will be asked to confirm before +the merge is initiated: + + ## Troubleshooting diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 590a02b306c..d1e50039417 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -149,14 +149,15 @@ The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/14664) in GitLab > 12.0. -Job logs are divided into sections that can be collapsed or expanded. +Job logs are divided into sections that can be collapsed or expanded. Each section will display +the duration. In the following example: -- Two sections are expanded and can be collapsed. -- One section is collapsed and can be expanded. +- Two sections are collapsed and can be expanded. +- Three sections are expanded and can be collapsed. - + ## Configuring pipelines @@ -468,3 +469,15 @@ To illustrate its life cycle: even if the commit history of the `example` branch has been overwritten by force-push. 1. GitLab Runner fetches the persistent pipeline ref and gets source code from the checkout-SHA. 1. When the pipeline finished, its persistent ref is cleaned up in a background process. + +NOTE: **NOTE**: At this moment, this feature is on by default and can be manually disabled +by disabling `depend_on_persistent_pipeline_ref` feature flag. If you're interested in +manually disabling this behavior, please ask the administrator +to execute the following commands in rails console. + +```shell +> sudo gitlab-rails console # Login to Rails console of GitLab instance. +> project = Project.find_by_full_path('namespace/project-name') # Get the project instance. +> Feature.disable(:depend_on_persistent_pipeline_ref, project) # Disable the feature flag for specific project +> Feature.disable(:depend_on_persistent_pipeline_ref) # Disable the feature flag system-wide +``` diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index da92fadafc4..0da1228aa53 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -85,7 +85,7 @@ in your repository map to paths of pages on your website using a Route Map. Once set, GitLab will display **View on ...** buttons, which will take you to the pages changed directly from merge requests. -To set up a route map, add a a file inside the repository at `.gitlab/route-map.yml`, +To set up a route map, add a file inside the repository at `.gitlab/route-map.yml`, which contains a YAML array that maps `source` paths (in the repository) to `public` paths (on the website). @@ -163,6 +163,13 @@ that spawned the Review App. ### Configuring Visual Reviews +Ensure that the `anonymous_visual_review_feedback` feature flag is enabled. +Administrators can enable with a Rails console as follows: + +```ruby +Feature.enabled(:anonymous_visual_review_feedback) +``` + The feedback form is served through a script you add to pages in your Review App. If you have [Developer permissions](../../user/permissions.md) to the project, you can access it by clicking the **Review** button in the **Pipeline** section @@ -221,6 +228,19 @@ NOTE: **Note:** Future enhancements [are planned](https://gitlab.com/gitlab-org/gitlab/issues/11322) to make this process even easier. +### Determining merge request ID + +The visual review tools retrieve the merge request ID from the `data-merge-request-id` +data attribute included in the `script` HTML tag used to add the visual review tools +to your review app. + +After determining the ID for the merge request to link to a visual review app, you +can supply the ID by either: + +- Hardcoding it in the script tag via the data attribute `data-merge-request-id` of the app. +- Dynamically adding the `data-merge-request-id` value during the build of the app. +- Supplying it manually through the visual review form in the app. + ### Using Visual Reviews After Visual Reviews has been [enabled](#configuring-visual-reviews) for the @@ -231,25 +251,15 @@ the bottom-right corner. To use the feedback form: -1. Create a [personal access token](../../user/profile/personal_access_tokens.md) - with the API scope selected. -1. Paste the token into the feedback box when prompted. If you select **Remember me**, - your browser stores the token so that future visits to Review Apps at the same URL - will not require you to re-enter the token. To clear the token, click **Log out**. 1. Make a comment on the visual review. You can make use of all the [Markdown annotations](../../user/markdown.md) that are also available in merge request comments. +1. Submit your feedback anonymously or add your name. 1. Finally, click **Send feedback**. After you make and submit a comment in the visual review box, it will appear automatically in the respective merge request. -TIP: **Tip:** -Because tokens must be entered on a per-domain basis and they can only be accessed -once, different review apps will not remember your token. You can save the token -to your password manager specifically for the purpose of Visual Reviews. This way, -you will not need to create additional tokens for each merge request. - ## Limitations Review App limitations are the same as [environments limitations](../environments.md#limitations). diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index cff797549ba..384216fed2c 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -284,6 +284,8 @@ export CI_PROJECT_PATH="gitlab-org/gitlab-foss" export CI_PROJECT_URL="https://example.com/gitlab-org/gitlab-foss" export CI_REGISTRY="registry.example.com" export CI_REGISTRY_IMAGE="registry.example.com/gitlab-org/gitlab-foss" +export CI_REGISTRY_USER="gitlab-ci-token" +export CI_REGISTRY_PASSWORD="longalfanumstring" export CI_RUNNER_ID="10" export CI_RUNNER_DESCRIPTION="my runner" export CI_RUNNER_TAGS="docker, linux" @@ -295,10 +297,8 @@ export CI_SERVER_VERSION="8.9.0" export CI_SERVER_VERSION_MAJOR="8" export CI_SERVER_VERSION_MINOR="9" export CI_SERVER_VERSION_PATCH="0" -export GITLAB_USER_ID="42" export GITLAB_USER_EMAIL="user@example.com" -export CI_REGISTRY_USER="gitlab-ci-token" -export CI_REGISTRY_PASSWORD="longalfanumstring" +export GITLAB_USER_ID="42" ``` ### `.gitlab-ci.yml` defined variables @@ -699,8 +699,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ CI_JOB_ID=7046507 ++ export CI_JOB_TOKEN=xxxxxxxxxxxxxxxxxxxx ++ CI_JOB_TOKEN=xxxxxxxxxxxxxxxxxxxx -++ export CI_COMMIT_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045 -++ CI_COMMIT_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ export CI_COMMIT_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ CI_COMMIT_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 ++ export CI_COMMIT_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 ++ CI_COMMIT_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 ++ export CI_COMMIT_REF_NAME=master diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index b93ff62cc21..e25425b01ae 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -37,6 +37,7 @@ future GitLab releases.** | `CI_COMMIT_REF_SLUG` | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. | | `CI_COMMIT_SHA` | 9.0 | all | The commit revision for which project is built | | `CI_COMMIT_SHORT_SHA` | 11.7 | all | The first eight characters of `CI_COMMIT_SHA` | +| `CI_COMMIT_BRANCH` | 12.6 | 0.5 | The commit branch name. Present only when building branches. | | `CI_COMMIT_TAG` | 9.0 | 0.5 | The commit tag name. Present only when building tags. | | `CI_COMMIT_TITLE` | 10.8 | all | The title of the commit - the full first line of the message | | `CI_CONCURRENT_ID` | all | 11.10 | Unique ID of build execution within a single executor. | @@ -84,7 +85,7 @@ future GitLab releases.** | `CI_PAGES_URL` | 11.8 | all | URL to GitLab Pages-built pages. Always belongs to a subdomain of `CI_PAGES_DOMAIN`. | | `CI_PIPELINE_ID` | 8.10 | all | The unique id of the current pipeline that GitLab CI uses internally | | `CI_PIPELINE_IID` | 11.0 | all | The unique id of the current pipeline scoped to project | -| `CI_PIPELINE_SOURCE` | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `trigger`, `schedule`, `api`, and `pipeline`. For pipelines created before GitLab 9.5, this will show as `unknown` | +| `CI_PIPELINE_SOURCE` | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `trigger`, `schedule`, `api`, `pipeline` and `merge_request_event`. For pipelines created before GitLab 9.5, this will show as `unknown` | | `CI_PIPELINE_TRIGGERED` | all | all | The flag to indicate that job was [triggered](../triggers/README.md) | | `CI_PIPELINE_URL` | 11.1 | 0.5 | Pipeline details URL | | `CI_PROJECT_DIR` | all | all | The full path where the repository is cloned and where the job is run. If the GitLab Runner `builds_dir` parameter is set, this variable is set relative to the value of `builds_dir`. For more information, see [Advanced configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) for GitLab Runner. | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 73e976a6145..84e0bb873a7 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -75,6 +75,7 @@ cannot be used as job names**: - `after_script` - `variables` - `cache` +- `include` ### Using reserved keywords @@ -134,7 +135,11 @@ The following job parameters can be defined inside a `default:` block: - [`services`](#services) - [`before_script`](#before_script-and-after_script) - [`after_script`](#before_script-and-after_script) +- [`tags`](#tags) - [`cache`](#cache) +- [`artifacts`](#artifacts) +- [`retry`](#retry) +- [`timeout`](#timeout) - [`interruptible`](#interruptible) In the following example, the `ruby:2.5` image is set as the default for all @@ -182,6 +187,17 @@ that the YAML parser knows to interpret the whole thing as a string rather than a "key: value" pair. Be careful when using special characters: `:`, `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``. +If any of the script commands return an exit code different from zero, the job +will fail and further commands will not be executed. This behavior can be avoided by +storing the exit code in a variable: + +```yaml +job: + script: + - false && true; exit_code=$? + - if [ $exit_code -ne 0 ]; then echo "Previous command failed"; fi; +``` + #### YAML anchors for `script` > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/23005) in GitLab 12.5. @@ -189,7 +205,7 @@ a "key: value" pair. Be careful when using special characters: You can use [YAML anchors](#anchors) with scripts, which makes it possible to include a predefined list of commands in multiple jobs. -Example: +For example: ```yaml .something: &something @@ -262,13 +278,13 @@ For more information, see see [Available settings for `services`](../docker/usin `before_script` is used to define a command that should be run before each job, including deploy jobs, but after the restoration of any [artifacts](#artifacts). -This must be an an array. +This must be an array. Scripts specified in `before_script` are concatenated with any scripts specified in the main [`script`](#script), and executed together in a single shell. `after_script` is used to define the command that will be run after each -job, including failed ones. This must be an an array. +job, including failed ones. This must be an array. Scripts specified in `after_script` are executed in a new shell, separate from any `before_script` or `script` scripts. As a result, they: @@ -278,6 +294,10 @@ Scripts specified in `after_script` are executed in a new shell, separate from a - Command aliases and variables exported in `script` scripts. - Changes outside of the working tree (depending on the Runner executor), like software installed by a `before_script` or `script` script. +- Have a separate timeout, which is hard coded to 5 minutes. See + [related issue](https://gitlab.com/gitlab-org/gitlab-runner/issues/2716) for details. +- Do not affect the job's exit code. If the `script` section succeeds and the + `after_script` times out or fails, the job will exit with code `0` (`Job Succeeded`). It's possible to overwrite a globally defined `before_script` or `after_script` if you set it per-job: @@ -450,7 +470,9 @@ Jobs will run on your own Runners in parallel only if: ### `only`/`except` (basic) NOTE: **Note:** -These parameters will soon be [deprecated](https://gitlab.com/gitlab-org/gitlab/issues/27449) in favor of [`rules`](#rules) as it offers a more powerful syntax. +The [`rules`](#rules) syntax is now the preferred method of setting job policies. +`only` and `except` are [candidates for deprecation](https://gitlab.com/gitlab-org/gitlab/issues/27449), +and may be removed in the future. `only` and `except` are two parameters that set a job policy to limit when jobs are created: @@ -767,7 +789,7 @@ it is possible to define a job to be created based on files modified in a merge request. In order to deduce the correct base SHA of the source branch, we recommend combining -this keyword with `only: merge_requests`. This way, file differences are correctly +this keyword with `only: [merge_requests]`. This way, file differences are correctly calculated from any further commits, thus all changes in the merge requests are properly tested in pipelines. @@ -789,7 +811,7 @@ either files in `service-one` directory or the `Dockerfile`, GitLab creates and triggers the `docker build service one` job. Note that if [pipelines for merge requests](../merge_request_pipelines/index.md) is -combined with `only: change`, but `only: merge_requests` is omitted, there could be +combined with `only: [change]`, but `only: [merge_requests]` is omitted, there could be unwanted behavior. For example: @@ -1005,6 +1027,33 @@ Additional job configuration may be added to rules in the future. If something useful isn't available, please [open an issue](https://gitlab.com/gitlab-org/gitlab/issues). +### `workflow:rules` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/29654) in GitLab 12.5 + +The top-level `workflow:` key applies to the entirety of a pipeline, and will +determine whether or not a pipeline is created. It currently accepts a single +`rules:` key that operates similarly to [`rules:` defined within jobs](#rules), +enabling dynamic configuration of the pipeline. + +The configuration options currently available for `workflow:rules` are: + +- [`if`](#rulesif): Define a rule. +- [`when`](#when): May be set to `always` or `never` only. If not provided, the default value is `always`. + +The list of `if` rules is evaluated until a single one is matched. If none +match, the last `when` will be used: + +```yaml +workflow: + rules: + - if: $CI_COMMIT_REF_NAME =~ /-wip$/ + when: never + - if: $CI_COMMIT_TAG + when: never + - when: always +``` + ### `tags` `tags` is used to select specific Runners from the list of all Runners that are @@ -1099,6 +1148,8 @@ failure. 1. `always` - execute job regardless of the status of jobs from prior stages. 1. `manual` - execute job manually (added in GitLab 8.10). Read about [manual actions](#whenmanual) below. +1. `delayed` - execute job after a certain period (added in GitLab 11.14). + Read about [delayed actions](#whendelayed) below. For example: @@ -1232,11 +1283,13 @@ Delayed job are for executing scripts after a certain period. This is useful if you want to avoid jobs entering `pending` state immediately. You can set the period with `start_in` key. The value of `start_in` key is an elapsed time in seconds, unless a unit is -provided. `start_in` key must be less than or equal to one hour. Examples of valid values include: +provided. `start_in` key must be less than or equal to one week. Examples of valid values include: +- `'5'` - `10 seconds` - `30 minutes` -- `1 hour` +- `1 day` +- `1 week` When there is a delayed job in a stage, the pipeline will not progress until the delayed job has finished. This means this keyword can also be used for inserting delays between different stages. @@ -1397,6 +1450,11 @@ Also in the example, `GIT_STRATEGY` is set to `none` so that GitLab Runner won try to check out the code after the branch is deleted when the `stop_review_app` job is [automatically triggered](../environments.md#automatically-stopping-an-environment). +NOTE: **Note:** +The above example overwrites global variables. If your stop environment job depends +on global variables, you can use [anchor variables](#yaml-anchors-for-variables) when setting the `GIT_STRATEGY` +to change it without overriding the global variables. + The `stop_review_app` job is **required** to have the following keywords defined: - `when` - [reference](#when) @@ -1405,6 +1463,38 @@ The `stop_review_app` job is **required** to have the following keywords defined - `stage` should be the same as the `review_app` in order for the environment to stop automatically when the branch is deleted +#### `environment:kubernetes` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/27630) in GitLab 12.6. + +The `kubernetes` block is used to configure deployments to a +[Kubernetes cluster](../../user/project/clusters/index.md) that is associated with your project. + +For example: + +```yaml +deploy: + stage: deploy + script: make deploy-app + environment: + name: production + kubernetes: + namespace: production +``` + +This will set up the `deploy` job to deploy to the `production` +environment, using the `production` +[Kubernetes namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/). + +For more information, see +[Available settings for `kubernetes`](../environments.md#configuring-kubernetes-deployments). + +NOTE: **Note:** +Kubernetes configuration is not supported for Kubernetes clusters +that are [managed by GitLab](../../user/project/clusters/index.md#gitlab-managed-clusters). +To follow progress on support for GitLab-managed clusters, see the +[relevant issue](https://gitlab.com/gitlab-org/gitlab/issues/38054). + #### Dynamic environments > - [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. @@ -1460,8 +1550,10 @@ globally and all jobs will use that definition. #### `cache:paths` -Use the `paths` directive to choose which files or directories will be cached. You can only specify paths within your `$CI_PROJECT_DIR`. -Wildcards can be used that follow the [glob](https://en.wikipedia.org/wiki/Glob_(programming)) patterns and [filepath.Match](https://golang.org/pkg/path/filepath/#Match). +Use the `paths` directive to choose which files or directories will be cached. Paths +are relative to the project directory (`$CI_PROJECT_DIR`) and cannot directly link outside it. +Wildcards can be used that follow the [glob](https://en.wikipedia.org/wiki/Glob_(programming)) +patterns and [filepath.Match](https://golang.org/pkg/path/filepath/#Match). Cache all files in `binaries` that end in `.apk` and the `.config` file: @@ -1690,8 +1782,9 @@ be available for download in the GitLab UI. #### `artifacts:paths` -You can only use paths that are within the local working copy. -Wildcards can be used that follow the [glob](https://en.wikipedia.org/wiki/Glob_(programming)) patterns and [filepath.Match](https://golang.org/pkg/path/filepath/#Match). +Paths are relative to the project directory (`$CI_PROJECT_DIR`) and cannot directly +link outside it. Wildcards can be used that follow the [glob](https://en.wikipedia.org/wiki/Glob_(programming)) +patterns and [filepath.Match](https://golang.org/pkg/path/filepath/#Match). To restrict which jobs a specific job will fetch artifacts from, see [dependencies](#dependencies). @@ -2203,11 +2296,11 @@ This example creates three paths of execution: pipeline will be created with YAML error. - We are temporarily limiting the maximum number of jobs that a single job can need in the `needs:` array: - - For GitLab.com, the limit is five. For more information, see our + - For GitLab.com, the limit is ten. For more information, see our [infrastructure issue](https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/7541). - For self-managed instances, the limit is: - - Five by default (`ci_dag_limit_needs` feature flag is enabled). - - 50 if the `ci_dag_limit_needs` feature flag is disabled. + - 10, if the `ci_dag_limit_needs` feature flag is enabled (default). + - 50, if the `ci_dag_limit_needs` feature flag is disabled. - It is impossible for now to have `needs: []` (empty needs), the job always needs to depend on something, unless this is the job in the first stage. However, support for an empty needs array [is planned](https://gitlab.com/gitlab-org/gitlab/issues/30631). @@ -2219,6 +2312,49 @@ This example creates three paths of execution: - Related to the above, stages must be explicitly defined for all jobs that have the keyword `needs:` or are referred to by one. +#### Artifact downloads with `needs` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/14311) in GitLab v12.6. + +When using `needs`, artifact downloads are controlled with `artifacts: true` or `artifacts: false`. +The `dependencies` keyword should not be used with `needs`, as this is deprecated since GitLab 12.6. + +In the example below, the `rspec` job will download the `build_job` artifacts, while the +`rubocop` job will not: + +```yaml +build_job: + stage: build + artifacts: + paths: + - binaries/ + +rspec: + stage: test + needs: + - job: build_job + artifacts: true + +rubocop: + stage: test + needs: + - job: build_job + artifacts: false +``` + +Additionally, in the three syntax examples below, the `rspec` job will download the artifacts +from all three `build_jobs`, as `artifacts` is true for `build_job_1`, and will +**default** to true for both `build_job_2` and `build_job_3`. + +```yaml +rspec: + needs: + - job: build_job_1 + artifacts: true + - job: build_job_2 + - build_job_3 +``` + ### `coverage` > [Introduced][ce-7447] in GitLab 8.17. @@ -2344,12 +2480,13 @@ sequentially from `job_name 1/N` to `job_name N/N`. For every job, `CI_NODE_INDEX` and `CI_NODE_TOTAL` [environment variables](../variables/README.md#predefined-environment-variables) are set. -Marking a job to be run in parallel requires only a simple addition to your configuration file: +Marking a job to be run in parallel requires adding `parallel` to your configuration +file. For example: -```diff - test: - script: rspec -+ parallel: 5 +```yaml +test: + script: rspec + parallel: 5 ``` TIP: **Tip:** @@ -2397,7 +2534,7 @@ triggers being used. #### Simple `trigger` syntax -The most simple way to configure a downstream trigger to use `trigger` keyword +The simplest way to configure a downstream trigger is to use `trigger` keyword with a full path to a downstream project: ```yaml @@ -2429,7 +2566,7 @@ staging: It is possible to mirror the status from a triggered pipeline: -``` +```yaml trigger_job: trigger: project: my/project @@ -2438,7 +2575,7 @@ trigger_job: It is possible to mirror the status from an upstream pipeline: -``` +```yaml upstream_bridge: stage: test needs: @@ -3068,6 +3205,29 @@ which can be set in GitLab's UI. Learn more about [variables and their priority][variables]. +#### YAML anchors for variables + +[YAML anchors](#anchors) can be used with `variables`, to easily repeat assignment +of variables across multiple jobs. It can also enable more flexibility when a job +requires a specific `variables` block that would otherwise override the global variables. + +In the example below, we will override the `GIT_STRATEGY` variable without affecting +the use of the `SAMPLE_VARIABLE` variable: + +```yaml +# global variables +variables: &global-variables + SAMPLE_VARIABLE: sample_variable_value + +# a job that needs to set the GIT_STRATEGY variable, yet depend on global variables +job_no_git_strategy: + stage: cleanup + variables: + <<: *global-variables + GIT_STRATEGY: none + script: echo $SAMPLE_VARIABLE +``` + #### Git strategy > Introduced in GitLab 8.9 as an experimental feature. May change or be removed @@ -3129,7 +3289,7 @@ There are three possible values: `none`, `normal`, and `recursive`: - `normal` means that only the top-level submodules will be included. It is equivalent to: - ``` + ```shell git submodule sync git submodule update --init ``` @@ -3139,7 +3299,7 @@ There are three possible values: `none`, `normal`, and `recursive`: GitLab Runner with an executor not based on Docker, make sure the Git version meets that requirement. It is equivalent to: - ``` + ```shell git submodule sync --recursive git submodule update --init --recursive ``` @@ -3400,7 +3560,7 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya If you want to temporarily 'disable' a job, rather than commenting out all the lines where the job is defined: -``` +```yaml #hidden_job: # script: # - run test @@ -3541,6 +3701,10 @@ test:mysql: You can see that the hidden keys are conveniently used as templates. +NOTE: **Note:** +You can't use YAML anchors across multiple files when leveraging the [`include`](#include) +feature. Anchors are only valid within the file they were defined in. + ## Triggers Triggers can be used to force a rebuild of a specific branch, tag or commit, diff --git a/doc/development/README.md b/doc/development/README.md index 66df6f46e86..3a972c4c588 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -69,6 +69,8 @@ description: 'Learn how to contribute to GitLab.' - [Developing against interacting components or features](interacting_components.md) - [File uploads](uploads.md) - [Auto DevOps development guide](auto_devops.md) +- [Mass Inserting Models](mass_insert.md) +- [Cycle Analytics development guide](cycle_analytics.md) ## Performance guides @@ -101,6 +103,10 @@ description: 'Learn how to contribute to GitLab.' - [Swapping tables](swapping_tables.md) - [Deleting migrations](deleting_migrations.md) +### Debugging + +- Tracing the source of an SQL query using query comments with [Marginalia](database_query_comments.md) + ### Best practices - [Merge Request checklist](database_merge_request_checklist.md) diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index b0d94511c6e..1ef0b928820 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -1,5 +1,13 @@ # GraphQL API +## How GitLab implements GraphQL + +We use the [graphql-ruby gem](https://graphql-ruby.org/) written by [Robert Mosolgo](https://github.com/rmosolgo/). + +All GraphQL queries are directed to a single endpoint +([`app/controllers/graphql_controller.rb#execute`](https://gitlab.com/gitlab-org/gitlab/blob/master/app%2Fcontrollers%2Fgraphql_controller.rb)), +which is exposed as an API endpoint at `/api/graphql`. + ## Deep Dive In March 2019, Nick Thomas hosted a [Deep Dive](https://gitlab.com/gitlab-org/create-stage/issues/1) @@ -22,8 +30,31 @@ add a `HTTP_PRIVATE_TOKEN` header. ## Types +We use a code-first schema, and we declare what type everything is in Ruby. + +For example, `app/graphql/types/issue_type.rb`: + +```ruby +graphql_name 'Issue' + +field :iid, GraphQL::ID_TYPE, null: false +field :title, GraphQL::STRING_TYPE, null: false + +# we also have a method here that we've defined, that extends `field` +markdown_field :title_html, null: true +field :description, GraphQL::STRING_TYPE, null: true +markdown_field :description_html, null: true +``` + +We give each type a name (in this case `Issue`). + +The `iid`, `title` and `description` are _scalar_ GraphQL types. +`iid` is a `GraphQL::ID_TYPE`, a special string type that signifies a unique ID. +`title` and `description` are regular `GraphQL::STRING_TYPE` types. + When exposing a model through the GraphQL API, we do so by creating a -new type in `app/graphql/types`. +new type in `app/graphql/types`. You can also declare custom GraphQL data types +for scalar data types (e.g. `TimeType`). When exposing properties in a type, make sure to keep the logic inside the definition as minimal as possible. Instead, consider moving any @@ -293,6 +324,8 @@ If the: - Resource is part of a collection, the collection will be filtered to exclude the objects that the user's authorization checks failed against. +Also see [authorizing resources in a mutation](#authorizing-resources). + TIP: **Tip:** Try to load only what the currently authenticated user is allowed to view with our existing finders first, without relying on authorization @@ -391,6 +424,11 @@ end ## Resolvers +We define how the application serves the response using _resolvers_ +stored in the `app/graphql/resolvers` directory. +The resolver provides the actual implementation logic for retrieving +the objects in question. + To find objects to display in a field, we can add resolvers to `app/graphql/resolvers`. @@ -548,7 +586,7 @@ found, we should raise a `Gitlab::Graphql::Errors::ResourceNotAvailable` error. Which will be correctly rendered to the clients. -## Gitlab's custom scalars +## GitLab's custom scalars ### `Types::TimeType` @@ -618,7 +656,61 @@ it 'returns a successful response' do end ``` +## Notes about Query flow and GraphQL infrastructure + +GitLab's GraphQL infrastructure can be found in `lib/gitlab/graphql`. + +[Instrumentation](https://graphql-ruby.org/queries/instrumentation.html) is functionality +that wraps around a query being executed. It is implemented as a module that uses the `Instrumentation` class. + +Example: `Present` + +```ruby +module Present + #... some code above... + + def self.use(schema_definition) + schema_definition.instrument(:field, Instrumentation.new) + end +end +``` + +A [Query Analyzer](https://graphql-ruby.org/queries/analysis.html#analyzer-api) contains a series +of callbacks to validate queries before they are executed. Each field can pass through +the analyzer, and the final value is also available to you. + +[Multiplex queries](https://graphql-ruby.org/queries/multiplex.html) enable +multiple queries to be sent in a single request. This reduces the number of requests sent to the server. +(there are custom Multiplex Query Analyzers and Multiplex Instrumentation provided by graphql-ruby). + +### Query limits + +Queries and mutations are limited by depth, complexity, and recursion +to protect server resources from overly ambitious or malicious queries. +These values can be set as defaults and overridden in specific queries as needed. +The complexity values can be set per object as well, and the final query complexity is +evaluated based on how many objects are being returned. This is useful +for objects that are expensive (e.g. requiring Gitaly calls). + +For example, a conditional complexity method in a resolver: + +```ruby +def self.resolver_complexity(args, child_complexity:) + complexity = super + complexity += 2 if args[:labelName] + + complexity +end +``` + +More about complexity: +[graphql-ruby docs](https://graphql-ruby.org/queries/complexity_and_depth.html) + ## Documentation and Schema +Our schema is located at `app/graphql/gitlab_schema.rb`. +See the [schema reference](../api/graphql/reference/index.md) for details. + +This generated GraphQL documentation needs to be updated when the schema changes. For information on generating GraphQL documentation and schema files, see -[Rake tasks related to GraphQL](rake_tasks.md#update-graphql-documentation-and-schema-definitions). +[updating the schema documentation](rake_tasks.md#update-graphql-documentation-and-schema-definitions). diff --git a/doc/development/architecture.md b/doc/development/architecture.md index b579f812d99..eff83da523b 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -116,6 +116,7 @@ graph TB - ⚙ - Requires additional configuration, or GitLab Managed Apps - ⤓ - Manual installation required - ❌ - Not supported or no instructions available +- N/A - Not applicable Component statuses are linked to configuration documentation for each component. @@ -144,7 +145,7 @@ Component statuses are linked to configuration documentation for each component. | [Postgres Exporter](#postgres-exporter) | Prometheus endpoint with PostgreSQL metrics | [✅][postgres-exporter-omnibus] | [✅][postgres-exporter-charts] | [✅][postgres-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | | [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | [⚙][pgbouncer-exporter-omnibus] | [❌][pgbouncer-exporter-charts] | [❌][pgbouncer-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | | [GitLab Exporter](#gitlab-exporter) | Generates a variety of GitLab metrics | [✅][gitlab-exporter-omnibus] | [✅][gitlab-exporter-charts] | [✅][gitlab-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | -| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [❌][node-exporter-charts] | [❌][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [N/A][node-exporter-charts] | [N/A][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | | [Mattermost](#mattermost) | Open-source Slack alternative | [⚙][mattermost-omnibus] | [⤓][mattermost-charts] | [⤓][mattermost-charts] | [⤓](../user/project/integrations/mattermost.md) | ❌ | ❌ | CE & EE | | [MinIO](#minio) | Object storage service | [⤓][minio-omnibus] | [✅][minio-charts] | [✅][minio-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#storage-architecture) | ❌ | [⚙][minio-gdk] | CE & EE | | [Runner](#gitlab-runner) | Executes GitLab CI jobs | [⤓][runner-omnibus] | [✅][runner-charts] | [⚙][runner-charts] | [✅](../user/gitlab_com/index.md#shared-runners) | [⚙][runner-source] | [⚙][runner-gdk] | CE & EE | diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index 0a08360b727..8a1db615022 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -10,16 +10,16 @@ migrations automatically reschedule themselves for a later point in time. ## When To Use Background Migrations -> **Note:** -> When adding background migrations _you must_ make sure they are announced in the -> monthly release post along with an estimate of how long it will take to complete -> the migrations. - In the vast majority of cases you will want to use a regular Rails migration -instead. Background migrations should _only_ be used when migrating _data_ in +instead. Background migrations should be used when migrating _data_ in tables that have so many rows this process would take hours when performed in a regular Rails migration. +Background migrations _may_ also be used when executing numerous single-row queries +for every item on a large dataset. Typically, for single-record patterns, runtime is +largely dependent on the size of the dataset, hence it should be split accordingly +and put into background migrations. + Background migrations _may not_ be used to perform schema migrations, they should only be used for data migrations. @@ -29,6 +29,11 @@ Some examples where background migrations can be useful: - Populating one column based on JSON stored in another column. - Migrating data that depends on the output of external services (e.g. an API). +> **Note:** +> If the background migration is part of an important upgrade, make sure it's announced +> in the release post. Discuss with your Project Manager if you're not sure the migration falls +> into this category. + ## Isolation Background migrations must be isolated and can not use application code (e.g. diff --git a/doc/development/build_test_package.md b/doc/development/build_test_package.md index f58ac79b6f4..d478d6e1653 100644 --- a/doc/development/build_test_package.md +++ b/doc/development/build_test_package.md @@ -9,16 +9,16 @@ that will create: - A deb package for Ubuntu 16.04, available as a build artifact, and - A docker image, which is pushed to [Omnibus GitLab's container registry](https://gitlab.com/gitlab-org/omnibus-gitlab/container_registry) - (images titled `gitlab-foss` and `gitlab-ee` respectively and image tag is the + (images titled `gitlab-ce` and `gitlab-ee` respectively and image tag is the commit which triggered the pipeline). When you push a commit to either the GitLab CE or GitLab EE project, the pipeline for that commit will have a `build-package` manual action you can trigger. - + - + ## Specifying versions of components diff --git a/doc/development/changelog.md b/doc/development/changelog.md index af2c540cca5..f693267b554 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -33,7 +33,8 @@ the `author` field. GitLab team members **should not**. ## What warrants a changelog entry? -- Any change that introduces a database migration **must** have a changelog entry. +- Any change that introduces a database migration, whether it's regular, post, + or data migration, **must** have a changelog entry. - Any user-facing change **should** have a changelog entry. Example: "GitLab now uses system fonts for all text." - Performance improvements **should** have a changelog entry. @@ -43,8 +44,8 @@ the `author` field. GitLab team members **should not**. - Any docs-only changes **should not** have a changelog entry. - Any change behind a feature flag **should not** have a changelog entry. The entry should be added [in the merge request removing the feature flags](feature_flags/development.md). - If the change includes a database migration, there should be a changelog entry - for the migration change. + If the change includes a database migration (regular, post, or data migration), + there should be a changelog entry for the migration change. - A fix for a regression introduced and then fixed in the same release (i.e., fixing a bug introduced during a monthly release candidate) **should not** have a changelog entry. diff --git a/doc/development/chatops_on_gitlabcom.md b/doc/development/chatops_on_gitlabcom.md index 456dd1d4b4b..90b9cca54ac 100644 --- a/doc/development/chatops_on_gitlabcom.md +++ b/doc/development/chatops_on_gitlabcom.md @@ -14,7 +14,7 @@ tasks such as: To request access to Chatops on GitLab.com: 1. Log into <https://ops.gitlab.net/users/sign_in> **using the same username** as for GitLab.com (you may have to rename it). -1. Ask [a project member in the `chatops` project](https://ops.gitlab.net/gitlab-com/chatops/-/project_members) to add you by running `/chatops run member add <username> gitlab-com/chatops --ops`. +1. Ask in the [#production](https://gitlab.slack.com/messages/production) channel to add you by running `/chatops run member add <username> gitlab-com/chatops --ops`. ## See also diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 77c57bb332d..326ac7b3a37 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -181,6 +181,10 @@ vulnerabilities must be either empty or containing: Maintainers should **never** dismiss vulnerabilities to "empty" the list, without duly verifying them. +Note that certain Merge Requests may target a stable branch. These are rare +events. These types of Merge Requests cannot be merged by the Maintainer. +Instead these should be sent to the [Release Manager](https://about.gitlab.com/community/release-managers/). + ## Best practices ### Everyone diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index f32400d44a2..a385a7dc83a 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -40,17 +40,17 @@ scheduling into milestones. Labelling is a task for everyone. Most issues will have labels for at least one of the following: -- Type: ~feature, ~bug, ~backstage, etc. -- Stage: ~"devops::plan", ~"devops::create", etc. -- Group: ~"group::source code", ~"group::knowledge", ~"group::editor", etc. -- Category: ~"Category:Code Analytics", ~"Category:DevOps Score", ~"Category:Templates", etc. -- Feature: ~wiki, ~ldap, ~api, ~issues, ~"merge requests", etc. -- Department: ~UX, ~Quality -- Team: ~"Technical Writing", ~Delivery -- Specialization: ~frontend, ~backend, ~documentation -- Release Scoping: ~Deliverable, ~Stretch, ~"Next Patch Release" -- Priority: ~P1, ~P2, ~P3, ~P4 -- Severity: ~S1, ~S2, ~S3, ~S4 +- Type: `~feature`, `~bug`, `~backstage`, etc. +- Stage: `~"devops::plan"`, `~"devops::create"`, etc. +- Group: `~"group::source code"`, `~"group::knowledge"`, `~"group::editor"`, etc. +- Category: `~"Category:Code Analytics"`, `~"Category:DevOps Score"`, `~"Category:Templates"`, etc. +- Feature: `~wiki`, `~ldap`, `~api`, `~issues`, `~"merge requests"`, etc. +- Department: `~UX`, `~Quality` +- Team: `~"Technical Writing"`, `~Delivery` +- Specialization: `~frontend`, `~backend`, `~documentation` +- Release Scoping: `~Deliverable`, `~Stretch`, `~"Next Patch Release"` +- Priority: `~P1`, `~P2`, `~P3`, `~P4` +- Severity: ~`S1`, `~S2`, `~S3`, `~S4` All labels, their meaning and priority are defined on the [labels page](https://gitlab.com/gitlab-org/gitlab-foss/-/labels). @@ -210,7 +210,7 @@ If you are an expert in a particular area, it makes it easier to find issues to work on. You can also subscribe to those labels to receive an email each time an issue is labeled with a feature label corresponding to your expertise. -Examples of feature labels are ~wiki, ~ldap, ~api, ~issues, ~"merge requests" etc. +Examples of feature labels are `~wiki`, `~ldap`, `~api`, `~issues`, `~"merge requests"` etc. #### Naming and color convention diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md index 510e90524ed..e839ae0ea3a 100644 --- a/doc/development/contributing/merge_request_workflow.md +++ b/doc/development/contributing/merge_request_workflow.md @@ -248,3 +248,16 @@ request: 1. The [GitLab Development Kit](https://gitlab.com/gitlab-org/gitlab-development-kit). 1. The [CI environment preparation](https://gitlab.com/gitlab-org/gitlab/blob/master/scripts/prepare_build.sh). 1. The [Omnibus package creator](https://gitlab.com/gitlab-org/omnibus-gitlab). + +### Incremental improvements + +We allow engineering time to fix small problems (with or without an +issue) that are incremental improvements, such as: + +1. Unprioritized bug fixes (e.g. [Banner alerting of project move is +showing up everywhere](https://gitlab.com/gitlab-org/gitlab/merge_requests/18985)) +1. Documentation improvements +1. Rubocop or Code Quality improvements + +Tag a merge request with ~"Stuff that should Just Work" to track work in +this area. diff --git a/doc/development/contributing/style_guides.md b/doc/development/contributing/style_guides.md index f825b3d7088..d53675cc716 100644 --- a/doc/development/contributing/style_guides.md +++ b/doc/development/contributing/style_guides.md @@ -40,8 +40,8 @@ This saves you time as you don't have to wait for the same errors to be detected [rss-source]: https://github.com/rubocop-hq/ruby-style-guide/blob/master/README.adoc#source-code-layout [rss-naming]: https://github.com/rubocop-hq/ruby-style-guide/blob/master/README.adoc#naming-conventions [doc-guidelines]: ../documentation/index.md "Documentation guidelines" -[js-styleguide]: ../fe_guide/style_guide_js.md "JavaScript styleguide" -[scss-styleguide]: ../fe_guide/style_guide_scss.md "SCSS styleguide" +[js-styleguide]: ../fe_guide/style/javascript.md "JavaScript styleguide" +[scss-styleguide]: ../fe_guide/style/scss.md "SCSS styleguide" [newlines-styleguide]: ../newlines_styleguide.md "Newlines styleguide" [testing]: ../testing_guide/index.md [us-english]: https://en.wikipedia.org/wiki/American_English diff --git a/doc/development/cycle_analytics.md b/doc/development/cycle_analytics.md new file mode 100644 index 00000000000..284645cdae7 --- /dev/null +++ b/doc/development/cycle_analytics.md @@ -0,0 +1,246 @@ +# Cycle Analytics development guide + +Cycle analytics calculates the time between two arbitrary events recorded on domain objects and provides aggregated statistics about the duration. + +## Stage + +During development, events occur that move issues and merge requests through different stages of progress until they are considered finished. These stages can be expressed with the `Stage` model. + +Example stage: + +- Name: Development +- Start event: Issue created +- End event: Issue first mentioned in commit +- Parent: `Group: gitlab-org` + +### Events + +Events are the smallest building blocks of the cycle analytics feature. A stage consists of two events: + +- Start +- End + +These events play a key role in the duration calculation. + +Formula: `duration = end_event_time - start_event_time` + +To make the duration calculation flexible, each `Event` is implemented as a separate class. They're responsible for defining a timestamp expression that will be used in the calculation query. + +#### Implementing an `Event` class + +There are a few methods that are required to be implemented, the `StageEvent` base class describes them in great detail. The most important ones are: + +- `object_type` +- `timestamp_projection` + +The `object_type` method defines which domain object will be queried for the calculation. Currently two models are allowed: + +- `Issue` +- `MergeRequest` + +For the duration calculation the `timestamp_projection` method will be used. + +```ruby +def timestamp_projection + # your timestamp expression comes here +end + +# event will use the issue creation time in the duration calculation +def timestamp_projection + Issue.arel_table[:created_at] +end +``` + +NOTE: **Note:** +More complex expressions are also possible (e.g. using `COALESCE`). Look at the existing event classes for examples. + +In some cases, defining the `timestamp_projection` method is not enough. The calculation query should know which table contains the timestamp expression. Each `Event` class is responsible for making modifications to the calculation query to make the `timestamp_projection` work. This usually means joining an additional table. + +Example for joining the `issue_metrics` table and using the `first_mentioned_in_commit_at` column as the timestamp expression: + +```ruby +def object_type + Issue +end + +def timestamp_projection + IssueMetrics.arel_table[:first_mentioned_in_commit_at] +end + +def apply_query_customization(query) + # in this case the query attribute will be based on the Issue model: `Issue.where(...)` + query.joins(:metrics) +end +``` + +### Validating start and end events + +Some start/end event pairs are not "compatible" with each other. For example: + +- "Issue created" to "Merge Request created": The event classes are defined on different domain models, the `object_type` method is different. +- "Issue closed" to "Issue created": Issue must be created first before it can be closed. +- "Issue closed" to "Issue closed": Duration is always 0. + +The `StageEvents` module describes the allowed `start_event` and `end_event` pairings (`PAIRING_RULES` constant). If a new event is added, it needs to be registered in this module. +To add a new event: + +1. Add an entry in `ENUM_MAPPING` with a unique number, it'll be used in the `Stage` model as `enum`. +1. Define which events are compatible with the event in the `PAIRING_RULES` hash. + +Supported start/end event pairings: + +```mermaid +graph LR; + IssueCreated --> IssueClosed; + IssueCreated --> IssueFirstAddedToBoard; + IssueCreated --> IssueFirstAssociatedWithMilestone; + IssueCreated --> IssueFirstMentionedInCommit; + IssueCreated --> IssueLastEdited; + IssueCreated --> IssueLabelAdded; + IssueCreated --> IssueLabelRemoved; + MergeRequestCreated --> MergeRequestMerged; + MergeRequestCreated --> MergeRequestClosed; + MergeRequestCreated --> MergeRequestFirstDeployedToProduction; + MergeRequestCreated --> MergeRequestLastBuildStarted; + MergeRequestCreated --> MergeRequestLastBuildFinished; + MergeRequestCreated --> MergeRequestLastEdited; + MergeRequestCreated --> MergeRequestLabelAdded; + MergeRequestCreated --> MergeRequestLabelRemoved; + MergeRequestLastBuildStarted --> MergeRequestLastBuildFinished; + MergeRequestLastBuildStarted --> MergeRequestClosed; + MergeRequestLastBuildStarted --> MergeRequestFirstDeployedToProduction; + MergeRequestLastBuildStarted --> MergeRequestLastEdited; + MergeRequestLastBuildStarted --> MergeRequestMerged; + MergeRequestLastBuildStarted --> MergeRequestLabelAdded; + MergeRequestLastBuildStarted --> MergeRequestLabelRemoved; + MergeRequestMerged --> MergeRequestFirstDeployedToProduction; + MergeRequestMerged --> MergeRequestClosed; + MergeRequestMerged --> MergeRequestFirstDeployedToProduction; + MergeRequestMerged --> MergeRequestLastEdited; + MergeRequestMerged --> MergeRequestLabelAdded; + MergeRequestMerged --> MergeRequestLabelRemoved; + IssueLabelAdded --> IssueLabelAdded; + IssueLabelAdded --> IssueLabelRemoved; + IssueLabelAdded --> IssueClosed; + IssueLabelRemoved --> IssueClosed; + IssueFirstAddedToBoard --> IssueClosed; + IssueFirstAddedToBoard --> IssueFirstAssociatedWithMilestone; + IssueFirstAddedToBoard --> IssueFirstMentionedInCommit; + IssueFirstAddedToBoard --> IssueLastEdited; + IssueFirstAddedToBoard --> IssueLabelAdded; + IssueFirstAddedToBoard --> IssueLabelRemoved; + IssueFirstAssociatedWithMilestone --> IssueClosed; + IssueFirstAssociatedWithMilestone --> IssueFirstAddedToBoard; + IssueFirstAssociatedWithMilestone --> IssueFirstMentionedInCommit; + IssueFirstAssociatedWithMilestone --> IssueLastEdited; + IssueFirstAssociatedWithMilestone --> IssueLabelAdded; + IssueFirstAssociatedWithMilestone --> IssueLabelRemoved; + IssueFirstMentionedInCommit --> IssueClosed; + IssueFirstMentionedInCommit --> IssueFirstAssociatedWithMilestone; + IssueFirstMentionedInCommit --> IssueFirstAddedToBoard; + IssueFirstMentionedInCommit --> IssueLastEdited; + IssueFirstMentionedInCommit --> IssueLabelAdded; + IssueFirstMentionedInCommit --> IssueLabelRemoved; + IssueClosed --> IssueLastEdited; + IssueClosed --> IssueLabelAdded; + IssueClosed --> IssueLabelRemoved; + MergeRequestClosed --> MergeRequestFirstDeployedToProduction; + MergeRequestClosed --> MergeRequestLastEdited; + MergeRequestClosed --> MergeRequestLabelAdded; + MergeRequestClosed --> MergeRequestLabelRemoved; + MergeRequestFirstDeployedToProduction --> MergeRequestLastEdited; + MergeRequestFirstDeployedToProduction --> MergeRequestLabelAdded; + MergeRequestFirstDeployedToProduction --> MergeRequestLabelRemoved; + MergeRequestLastBuildFinished --> MergeRequestClosed; + MergeRequestLastBuildFinished --> MergeRequestFirstDeployedToProduction; + MergeRequestLastBuildFinished --> MergeRequestLastEdited; + MergeRequestLastBuildFinished --> MergeRequestMerged; + MergeRequestLastBuildFinished --> MergeRequestLabelAdded; + MergeRequestLastBuildFinished --> MergeRequestLabelRemoved; + MergeRequestLabelAdded --> MergeRequestLabelAdded; + MergeRequestLabelAdded --> MergeRequestLabelRemoved; + MergeRequestLabelRemoved --> MergeRequestLabelAdded; + MergeRequestLabelRemoved --> MergeRequestLabelRemoved; +``` + +### Parent + +Teams and organizations might define their own way of building software, thus stages can be completely different. For each stage, a parent object needs to be defined. + +Currently supported parents: + +- `Project` +- `Group` + +#### How parent relationship it work + +1. User navigates to the cycle analytics page. +1. User selects a group. +1. Backend loads the defined stages for the selected group. +1. Additions and modifications to the stages will be persisted within the selected group only. + +### Default stages + +The [original implementation](https://gitlab.com/gitlab-org/gitlab/issues/847) of cycle analytics defined 7 stages. These stages are always available for each parent, however altering these stages is not possible. + +To make things efficient and reduce the number of records created, the default stages are expressed as in-memory objects (not persisted). When the user creates a custom stage for the first time, all the stages will be persisted. This behaviour is implemented in the cycle analytics service objects. + +The reason for this was that we'd like to add the abilities to hide and order stages later on. + +## Data Collector + +`DataCollector` is the central point where the data will be queried from the database. The class always operates on a single stage and consists of the following components: + +- `BaseQueryBuilder`: + - Responsible for composing the initial query. + - Deals with `Stage` specific configuration: events and their query customizations. + - Parameters coming from the UI: date ranges. +- `Median`: Calculates the median duration for a stage using the query from `BaseQueryBuilder`. +- `RecordsFetcher`: Loads relevant records for a stage using the query from `BaseQueryBuilder` and specific `Finder` classes to apply visibility rules. +- `DataForDurationChart`: Loads calculated durations with the finish time (end event timestamp) for the scatterplot chart. + +For a new calculation or a query, implement it as a new method call in the `DataCollector` class. + +## Database query + +Structure of the database query: + +```sql +SELECT (customized by: Median or RecordsFetcher or DataForDurationChart) +FROM OBJECT_TYPE (Issue or MergeRequest) +INNER JOIN (several JOIN statements, depending on the events) +WHERE + (Filter by the PARENT model, example: filter Issues from Project A) + (Date range filter based on the OBJECT_TYPE.created_at) + (Check if the START_EVENT is earlier than END_EVENT, preventing negative duration) +``` + +Structure of the `SELECT` statement for `Median`: + +```sql +SELECT (calculate median from START_EVENT_TIME-END_EVENT_TIME) +``` + +Structure of the `SELECT` statement for `DataForDurationChart`: + +```sql +SELECT (START_EVENT_TIME-END_EVENT_TIME) as duration, END_EVENT.timestamp +``` + +## High-level overview + +- Rails Controller (`Analytics::CycleAnalytics` module): Cycle analytics exposes its data via JSON endpoints, implemented within the `analytics` workspace. Configuring the stages are also implements JSON endpoints (CRUD). +- Services (`Analytics::CycleAnalytics` module): All `Stage` related actions will be delegated to respective service objects. +- Models (`Analytics::CycleAnalytics` module): Models are used to persist the `Stage` objects `ProjectStage` and `GroupStage`. +- Feature classes (`Gitlab::Analytics::CycleAnalytics` module): + - Responsible for composing queries and define feature specific busines logic. + - `DataCollector`, `Event`, `StageEvents`, etc. + +## Testing + +Since we have a lots of events and possible pairings, testing each pairing is not possible. The rule is to have at least one test case using an `Event` class. + +Writing a test case for a stage using a new `Event` can be challenging since data must be created for both events. To make this a bit simpler, each test case must be implemented in the `data_collector_spec.rb` where the stage is tested through the `DataCollector`. Each test case will be turned into multiple tests, covering the following cases: + +- Different parents: `Group` or `Project` +- Different calculations: `Median`, `RecordsFetcher` or `DataForDurationChart` diff --git a/doc/development/dangerbot.md b/doc/development/dangerbot.md index 0b2e2b43512..40eb4294617 100644 --- a/doc/development/dangerbot.md +++ b/doc/development/dangerbot.md @@ -95,6 +95,12 @@ through in CI. However, you can speed these cycles up somewhat by emptying the `.gitlab/ci/rails.gitlab-ci.yml` file in your merge request. Just don't forget to revert the change before merging! +To enable the Dangerfile on another existing GitLab project, run the following extra steps, based on [this procedure](https://danger.systems/guides/getting_started.html#creating-a-bot-account-for-danger-to-use): + +1. Add `@gitlab-bot` to the project as a `reporter`. +1. Add the `@gitlab-bot`'s `GITLAB_API_PRIVATE_TOKEN` value as a value for a new CI/CD + variable named `DANGER_GITLAB_API_TOKEN`. + You should add the `~Danger bot` label to the merge request before sending it for review. diff --git a/doc/development/database_query_comments.md b/doc/development/database_query_comments.md new file mode 100644 index 00000000000..77943f2b261 --- /dev/null +++ b/doc/development/database_query_comments.md @@ -0,0 +1,56 @@ +# Database query comments with Marginalia + +The [Marginalia gem](https://github.com/basecamp/marginalia) is used to add +query comments containing application related context information to PostgreSQL +queries generated by ActiveRecord. + +It is very useful for tracing problematic queries back to the application source. + +An engineer during an on-call incident will have the full context of a query +and its application source from the comments. + +## Metadata information in comments + +Queries generated from **Rails** include the following metadata in comments: + +- `application` +- `controller` +- `action` +- `correlation_id` +- `line` + +Queries generated from **Sidekiq** workers will include the following metadata +in comments: + +- `application` +- `jid` +- `job_class` +- `correlation_id` +- `line` + +Examples of queries with comments as observed in `development.log`: + +1. Rails: + + ```sql + SELECT "project_features".* FROM "project_features" WHERE "project_features"."project_id" = $1 LIMIT $2 [["project_id", 5], ["LIMIT", 1]] /*application:web,controller:jobs,action:trace,correlation_id:rYF4mey9CH3,line:/app/policies/project_policy.rb:504:in `feature_available?'*/ + ``` + +1. Sidekiq: + + ```sql + SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = $1 LIMIT $2 [["id", 64], ["LIMIT", 1]] /*application:sidekiq,jid:e7d6668a39a991e323009833,job_class:ExpireJobCacheWorker,correlation_id:rYF4mey9CH3,line:/app/workers/expire_job_cache_worker.rb:14:in `perform'*/ + ``` + +## Enable/Disable the feature + +Enabling or disabling the feature requires a **restart/SIGHUP** of the Web and +Sidekiq workers, as the feature flag's state is memoized upon starting up. + +The `feature_flag` for this feature is **disabled** by default. You can enable +or disable it with: + +```ruby +Feature.enable(:marginalia) +Feature.disable(:marginalia) +``` diff --git a/doc/development/database_review.md b/doc/development/database_review.md index f3c19002417..b1c3ed47976 100644 --- a/doc/development/database_review.md +++ b/doc/development/database_review.md @@ -101,11 +101,15 @@ and details for a database reviewer: - Check consistency with `db/schema.rb` and that migrations are [reversible](migration_style_guide.md#reversibility) - Check queries timing (If any): Queries executed in a migration need to fit comfortably within `15s` - preferably much less than that - on GitLab.com. + - For column removals, make sure the column has been [ignored in a previous release](what_requires_downtime.md#dropping-columns) - Check [background migrations](background_migrations.md): - - Establish a time estimate for execution on GitLab.com. - - They should only be used when migrating data in larger tables. - - If a single `update` is below than `1s` the query can be placed + - Establish a time estimate for execution on GitLab.com. For historical purposes, + it's highly recommended to include this estimation on the merge request description. + - If a single `update` is below than `1s` the query can be placed directly in a regular migration (inside `db/migrate`). + - Background migrations are normally used, but not limited to: + - Migrating data in larger tables. + - Making numerous SQL queries per record in a dataset. - Review queries (for example, make sure batch sizes are fine) - Because execution time can be longer than for a regular migration, it's suggested to treat background migrations as post migrations: diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 7d575e9b0b1..61e42ecfe83 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -30,6 +30,23 @@ The source of the documentation exists within the codebase of each GitLab applic Documentation issues and merge requests are part of their respective repositories and all have the label `Documentation`. +### Branch naming + +The [CI pipeline for the main GitLab project](../pipelines.md) is configured to automatically +run only the jobs that match the type of contribution. If your contribution contains +**only** documentation changes, then only documentation-related jobs will be run, and +the pipeline will complete much faster than a code contribution. + +If you are submitting documentation-only changes to Runner, Omnibus, or Charts, +the fast pipeline is not determined automatically. Instead, create branches for +docs-only merge requests using the following guide: + +| Branch name | Valid example | +|:----------------------|:-----------------------------| +| Starting with `docs/` | `docs/update-api-issues` | +| Starting with `docs-` | `docs-update-api-issues` | +| Ending in `-docs` | `123-update-api-issues-docs` | + ## Contributing to docs [Contributions to GitLab docs](workflow.md) are welcome from the entire GitLab community. @@ -41,7 +58,7 @@ However, anyone can contribute [documentation improvements](improvement-workflow ## Markdown and styles [GitLab docs](https://gitlab.com/gitlab-org/gitlab-docs) uses [GitLab Kramdown](https://gitlab.com/gitlab-org/gitlab_kramdown) -as its markdown rendering engine. See the [GitLab Markdown Guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/) for a complete Kramdown reference. +as its Markdown rendering engine. See the [GitLab Markdown Guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/) for a complete Kramdown reference. Adhere to the [Documentation Style Guide](styleguide.md). If a style standard is missing, you are welcome to suggest one via a merge request. @@ -382,7 +399,7 @@ merge request with new or changed docs is submitted, are: - The `CHANGELOG.md` does not contain duplicate versions. - No files in `doc/` are executable. - No new `README.md` was added. - - [`markdownlint`](#markdownlint). + - [markdownlint](#markdownlint). - Nanoc tests, which you can [run locally](#previewing-the-changes-live) before pushing to GitLab by executing the command `bundle exec nanoc check internal_links` (or `internal_anchors`) on your local [`gitlab-docs`](https://gitlab.com/gitlab-org/gitlab-docs) directory: @@ -403,7 +420,7 @@ help you catch common issues before raising merge requests for review of documen The following are some suggested linters you can install locally and sample configuration: - [`proselint`](#proselint) -- [`markdownlint`](#markdownlint), which is the same as the test run in [`docs-lint`](#testing) +- [markdownlint](#markdownlint), which is the same as the test run in [`docs-lint`](#testing) NOTE: **Note:** This list does not limit what other linters you can add to your local documentation writing toolchain. @@ -447,18 +464,18 @@ noise. The following sample `proselint` configuration disables these checks: A file with `proselint` configuration must be placed in a [valid location](https://github.com/amperser/proselint#checks). For example, `~/.config/proselint/config`. -#### `markdownlint` +#### markdownlint -[`markdownlint`](https://github.com/DavidAnson/markdownlint) checks that markdown +[markdownlint](https://github.com/DavidAnson/markdownlint) checks that Markdown syntax follows [certain rules](https://github.com/DavidAnson/markdownlint/blob/master/doc/Rules.md#rules), and is used by the [`docs-lint` test](#testing) with a [configuration file](#markdownlint-configuration). Our [Documentation Style Guide](styleguide.md#markdown) and [Markdown Guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/) elaborate on which choices must be made when selecting Markdown syntax for GitLab documentation. This tool helps catch deviations from those guidelines. -`markdownlint` can be used [on the command line](https://github.com/igorshubovych/markdownlint-cli#markdownlint-cli--), +markdownlint can be used [on the command line](https://github.com/igorshubovych/markdownlint-cli#markdownlint-cli--), either on a single Markdown file or on all Markdown files in a project. For example, to run -`markdownlint` on all documentation in the [`gitlab` project](https://gitlab.com/gitlab-org/gitlab), +markdownlint on all documentation in the [`gitlab` project](https://gitlab.com/gitlab-org/gitlab), run the following commands from within your `gitlab` project root directory, which will automatically detect the [`.markdownlint.json`](#markdownlint-configuration) config file in the root of the project, and test all files in `/doc` and its subdirectories: @@ -473,7 +490,7 @@ If you wish to use a different config file, use the `-c` flag: markdownlint -c <config-file-name> 'doc/**/*.md' ``` -`markdownlint` can also be run from within text editors using [plugins/extensions](https://github.com/DavidAnson/markdownlint#related), +markdownlint can also be run from within text editors using [plugins/extensions](https://github.com/DavidAnson/markdownlint#related), such as: - [Sublime Text](https://packagecontrol.io/packages/SublimeLinter-contrib-markdownlint) @@ -485,9 +502,9 @@ is in use in the four repos that are the sources for <https://docs.gitlab.com>. plugin/extension has different requirements regarding the configuration file, which is explained in each editor's docs. -##### `markdownlint` configuration +##### markdownlint configuration -Each formatting issue that `markdownlint` checks has an associated +Each formatting issue that markdownlint checks has an associated [rule](https://github.com/DavidAnson/markdownlint/blob/master/doc/Rules.md#rules). These rules are configured in the `.markdownlint.json` files located in the root of four repos that are the sources for <https://docs.gitlab.com>: @@ -501,7 +518,7 @@ By default all rules are enabled, so the configuration file is used to disable u rules, and also to configure optional parameters for enabled rules as needed. You can also check [the issue](https://gitlab.com/gitlab-org/gitlab-foss/issues/64352) that tracked the changes required to implement these rules, and details which rules were -on or off when `markdownlint` was enabled on the docs. +on or off when markdownlint was enabled on the docs. ## Danger Bot diff --git a/doc/development/documentation/site_architecture/index.md b/doc/development/documentation/site_architecture/index.md index bf873995e54..8a5018df9b7 100644 --- a/doc/development/documentation/site_architecture/index.md +++ b/doc/development/documentation/site_architecture/index.md @@ -128,9 +128,9 @@ We can then loop over the `versions` array with something like: Note that the data file must have the `yaml` extension (not `yml`) and that we reference the array with a symbol (`:versions`). -## Bumping versions of CSS and Javascript +## Bumping versions of CSS and JavaScript -Whenever the custom CSS and Javascript files under `content/assets/` change, +Whenever the custom CSS and JavaScript files under `content/assets/` change, make sure to bump their version in the frontmatter. This method guarantees that your changes will take effect by clearing the cache of previous files. @@ -180,7 +180,7 @@ for its search function. This is how it works: 1. On the docs side, we use a [docsearch layout](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/layouts/docsearch.html) which is present on pretty much every page except <https://docs.gitlab.com/search/>, which uses its [own layout](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/layouts/instantsearch.html). In those layouts, - there's a javascript snippet which initiates docsearch by using an API key + there's a JavaScript snippet which initiates docsearch by using an API key and an index name (`gitlab`) that are needed for Algolia to show the results. NOTE: **For GitLab employees:** diff --git a/doc/development/documentation/site_architecture/release_process.md b/doc/development/documentation/site_architecture/release_process.md index 6f723531f4c..76bd74b0bdc 100644 --- a/doc/development/documentation/site_architecture/release_process.md +++ b/doc/development/documentation/site_architecture/release_process.md @@ -117,7 +117,7 @@ version and rotates the old one: There's a temporary hack for now: 1. Edit `content/404.html`, making sure all offline versions under - `content/_data/versions.yaml` are in the Javascript snippet at the end of + `content/_data/versions.yaml` are in the JavaScript snippet at the end of the document. 1. **Update the `:latest` and `:archives` Docker images:** diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md index e6d666473c3..385569fc8fa 100644 --- a/doc/development/documentation/styleguide.md +++ b/doc/development/documentation/styleguide.md @@ -102,7 +102,7 @@ Note that Kramdown-specific markup (e.g., `{:.class}`) will not render properly Hard-coded HTML is valid, although it's discouraged to be used while we have `/help`. HTML is permitted as long as: -- There's no equivalent markup in markdown. +- There's no equivalent markup in Markdown. - Advanced tables are necessary. - Special styling is required. - Reviewed and approved by a technical writer. @@ -111,11 +111,38 @@ Hard-coded HTML is valid, although it's discouraged to be used while we have `/h GitLab ensures that the Markdown used across all documentation is consistent, as well as easy to review and maintain, by [testing documentation changes](index.md#testing) with -[`markdownlint`](index.md#markdownlint). This lint test fails when any document has an issue +[markdownlint](index.md#markdownlint). This lint test fails when any document has an issue with Markdown formatting that may cause the page to render incorrectly within GitLab. It will also fail when a document is using non-standard Markdown (which may render correctly, but is not the current standard for GitLab documentation). +#### Markdown rule `MD044/proper-names` (capitalization) + +A rule that could cause confusion is `MD044/proper-names`, as it might not be immediately +clear what caused markdownlint to fail, or how to correct the failure. This rule +checks a list of known words, listed in the `.markdownlint.json` file in each project, +to verify that proper capitalization and backticks are used. Words in backticks will +be ignored by markdownlint. + +In general, product names should follow the exact capitalization of the official names +of the products, protocols, etc. + +Some examples that will fail if incorrect capitalization is used: + +- MinIO (needs capital `IO`) +- NGINX (needs all capitals) +- runit (needs lowercase `r`) + +Additionally, commands, parameters, values, filenames, etc. must be included in backticks. +For example: + +- "Change the `needs` keyword in your `.gitlab.yml`..." + - `needs` is a parameter, and `.gitlab.yml` is a file, so both need backticks. Additionally, + `.gitlab.yml` will fail markdownlint without backticks as it does not have capital G or L. +- "Run `git clone` to clone a Git repository..." + - `git clone` is a command, so it must be lowercase, while Git is the product, so + it must have a capital G. + ## Structure ### Organize by topic, not by type @@ -212,7 +239,7 @@ Do not include the same information in multiple places. [Link to a SSOT instead. - Use inclusive language and avoid jargon, as well as uncommon words. The docs should be clear and easy to understand. -- Write in the 3rd person (use "we," "you," "us," "one," instead of "I" or "me"). +- Do not write in the first person singular. Instead of "I" or "me," use "we," "you," "us," or "one." - Be clear, concise, and stick to the goal of the doc. - Write in US English with US grammar. - Capitalize "G" and "L" in GitLab. @@ -230,22 +257,27 @@ Do not include the same information in multiple places. [Link to a SSOT instead. "Create a new merge request for Z." - Avoid use of the future tense: - - Instead of, "After you execute this command, the result will be displayed," say "After you execute this command, the result is displayed." + - Instead of "after you execute this command, GitLab will display the result", use "after you execute this command, GitLab displays the result". - Only use the future tense to convey when the action or result will actually occur at a future time. - Do not use contractions: - - Instead of "don't," "can't," "doesn't," and so on, say "do not," "cannot," or "does not." + - Instead of "don't," "can't," "doesn't," and so on, use "do not," "cannot," or "does not." - Possible exceptions are cases when a more familiar tone is desired, such as a blog post or other casual context. - Do not use slashes to clump different words together or as a replacement for the word "or": - - Instead of "and/or," consider saying "or," or use another sensible construction. + - Instead of "and/or," consider using "or," or use another sensible construction. - Other examples include "clone/fetch," author/assignee," and "namespace/repository name." Break apart any such instances in an appropriate way. - Exceptions to this rule include commonly accepted technical terms such as CI/CD, TCP/IP, and so on. - Do not use "may" and "might" interchangeably: - Use "might" to indicate the probability of something occurring. "If you skip this step, the import process might fail." - Use "may" to indicate giving permission for someone to do something, or consider using "can" instead. "You may select either option on this screen." Or, "you can select either option on this screen." +- We recommend avoiding Latin abbreviations, such as "e.g.," "i.e.," or "etc.," +as even native users of English might misunderstand them. + - Instead of "i.e.", use "that is." + - Instead of "e.g.", use "for example." + - Instead of "etc.", either use "and so on" or consider editing it out, since it can be vague. ## Text -- [Write in markdown](#markdown). +- [Write in Markdown](#markdown). - Splitting long lines (preferably up to 100 characters) can make it easier to provide feedback on small chunks of text. - Insert an empty line for new paragraphs. - Use sentence case for titles, headings, labels, menu items, and buttons. @@ -453,7 +485,7 @@ to mix types, that is also possible, as long as you don't mix items at the same ## Quotes -Valid for markdown content only, not for frontmatter entries: +Valid for Markdown content only, not for frontmatter entries: - Standard quotes: double quotes (`"`). Example: "This is wrapped in double quotes". - Quote within a quote: double quotes (`"`) wrap single quotes (`'`). Example: "I am 'quoting' something within a quote". @@ -464,7 +496,7 @@ For other punctuation rules, please refer to the ## Headings - Add **only one H1** in each document, by adding `#` at the beginning of - it (when using markdown). The `h1` will be the document `<title>`. + it (when using Markdown). The `h1` will be the document `<title>`. - Start with an `h2` (`##`), and respect the order `h2` > `h3` > `h4` > `h5` > `h6`. Never skip the hierarchy level, such as `h2` > `h4` - Avoid putting numbers in headings. Numbers shift, hence documentation anchor @@ -490,7 +522,7 @@ For other punctuation rules, please refer to the ## Links -- Use inline link markdown markup `[Text](https://example.com)`. +- Use inline link Markdown markup `[Text](https://example.com)`. It's easier to read, review, and maintain. **Do not** use `[Text][identifier]`. - Use [meaningful anchor texts](https://www.futurehosting.com/blog/links-should-have-meaningful-anchor-text-heres-why/). @@ -533,7 +565,7 @@ For other punctuation rules, please refer to the [issue tags](../../issues/tags.md#stages) ``` -- Using the markdown extension is necessary for the [`/help`](index.md#gitlab-help) +- Using the Markdown extension is necessary for the [`/help`](index.md#gitlab-help) section of GitLab. ### Links requiring permissions @@ -656,7 +688,7 @@ to readers. To embed a video, follow the instructions below and make sure you have your MR reviewed and approved by a technical writer. -1. Copy the code below and paste it into your markdown file. +1. Copy the code below and paste it into your Markdown file. Leave a blank line above and below it. Do NOT edit the code (don't remove or add any spaces, etc). 1. On YouTube, visit the video URL you want to display. Copy @@ -694,12 +726,12 @@ This is how it renders on the GitLab Docs site: class is necessary to make sure the video is responsive and displays nicely on different mobile devices. > - The `<div class="video-fallback">` is a fallback necessary for GitLab's -`/help`, as GitLab's markdown processor does not support iframes. It's hidden on the docs site but will be displayed on GitLab's `/help`. +`/help`, as GitLab's Markdown processor does not support iframes. It's hidden on the docs site but will be displayed on GitLab's `/help`. ## Code blocks - Always wrap code added to a sentence in inline code blocks (`` ` ``). - E.g., `.gitlab-ci.yml`, `git add .`, `CODEOWNERS`, `only: master`. + E.g., `.gitlab-ci.yml`, `git add .`, `CODEOWNERS`, `only: [master]`. File names, commands, entries, and anything that refers to code should be added to code blocks. To make things easier for the user, always add a full code block for things that can be useful to copy and paste, as they can easily do it with the button on code blocks. @@ -725,7 +757,7 @@ nicely on different mobile devices. ``` ~~~ -- To display raw markdown instead of rendered markdown, you can use triple backticks +- To display raw Markdown instead of rendered Markdown, you can use triple backticks with `md`, like the `Markdown code` example above, unless you want to include triple backticks in the code block as well. In that case, use triple tildes (`~~~`) instead. - For a complete reference on code blocks, check the [Kramdown guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/#code-blocks). @@ -739,7 +771,7 @@ _Note that the alert boxes only work for one paragraph only. Multiple paragraphs lists, headers, etc will not render correctly. For multiple lines, use blockquotes instead._ Alert boxes only render on the GitLab Docs site (<https://docs.gitlab.com>). -Within GitLab itself, they will appear as plain markdown text (like the examples +Within GitLab itself, they will appear as plain Markdown text (like the examples above the rendered versions, below). ### Note @@ -1055,7 +1087,7 @@ Sometimes features are shipped with feature flags, either: - On by default, but providing the option to turn the feature off. - Off by default, but providing the option to turn the feature on. -When documenting feature flags for a feature, it's important that users know: +When documenting feature flags for a feature, include: - Why a feature flag is necessary. Some of the reasons are [outlined in the handbook](https://about.gitlab.com/handbook/product/#alpha-beta-ga). @@ -1080,6 +1112,9 @@ Feature.disable(:feature_flag) ``` ```` +For guidance on developing with feature flags, see +[Feature flags in development of GitLab](../feature_flags/index.md). + ## API Here is a list of must-have items. Use them in the exact order that appears diff --git a/doc/development/documentation/workflow.md b/doc/development/documentation/workflow.md index c373b976453..e48c940dc21 100644 --- a/doc/development/documentation/workflow.md +++ b/doc/development/documentation/workflow.md @@ -15,12 +15,31 @@ documentation is associated with: Documentation is not usually required when a "backstage feature" is added or changed, and does not directly affect the way that any user or administrator interacts with GitLab. +## Documentation labels + +Regardless of the type of issue or merge request, certain labels are required when documentation +is added or updated. The following are added by the issue or merge request author: + +- An appropriate [type label](../contributing/issue_workflow.md#type-labels). For example, + `~backstage`. +- The [stage label](../contributing/issue_workflow.md#stage-labels) and + [group label](../contributing/issue_workflow.md#group-labels). For example, `~devops::create` and + `~group::source code`. +- The `~documentation` [specialization label](../contributing/issue_workflow.md#specialization-labels). + +The following are also added by members of the Technical Writing team: + +- A documentation [scoped label](../../user/project/labels.md#scoped-labels-premium) with the + `docs::` prefix. For example, `~docs::improvement`. +- The `~Technical Writing` [team label](../contributing/issue_workflow.md#team-labels). + ## For a product change This documentation is required for any new or changed feature and is: -- Created or updated as part of feature development, typically via the same merge request as the - feature code. +- Created or updated as part of feature development, almost always in the same merge request as the + feature code. Including documentation in the same merge request as the code eliminates the + possibility that code and documentation get out of sync. - Required with the delivery of a feature for a specific milestone as part of GitLab's [definition of done](../contributing/merge_request_workflow.md#definition-of-done). - Often linked from the release post. @@ -100,28 +119,10 @@ Prior to merging, documentation changes committed by the developer must be revie - The code reviewer for the merge request. This is known as a technical review. - Optionally, others involved in the work, such as other developers or the Product Manager. -- Optionally, the Technical Writer for the DevOps stage group. +- The Technical Writer for the DevOps stage group, except in exceptional circumstances where a + [post-merge review](#post-merge-reviews) can be requested. - A maintainer of the project. -If not assigned to a Technical Writer for review prior to merging, a review must be scheduled -immediately after merge by the developer or maintainer. For this, -create an issue using the [Doc Review description template](https://gitlab.com/gitlab-org/gitlab/issues/new?issuable_template=Doc%20Review) -and link to it from the merged merge request that introduced the documentation change. - -To decide whether to request a Technical Writer review before or after merge, consider: - -- The amount of time left before the milestone release. If there is less than three days - remaining, seek a post-merge review and ping the writer via Slack to ensure the review is - completed in time. -- The size of the change and your degree of confidence in having early users (for example, - GitLab.com users) of features use your documentation as written. -- That pre-merge Technical Writer reviews should be most common when the code is complete well in - advance of a milestone release and for larger documentation changes. -- You can request a post-merge Technical Writer review if it's important to get the code part of - a merge request merged as soon as possible. -- The Technical Writer can also help decide that documentation can be merged without Technical - writer review, with the review to occur soon after merge. - #### Product Managers Product Managers are responsible for the [documentation requirements](#documentation-requirements) @@ -149,10 +150,12 @@ will do the following: Technical Writers are responsible for: +- Participating in issues discussions and reviewing MRs for the upcoming milestone. - Reviewing documentation requirements in issues when called upon. - Answering questions, and helping and providing advice throughout the authoring and editing process. -- Reviewing all new and updated documentation content, whether before merge or after it is merged. +- Reviewing all significant new and updated documentation content, whether before merge or after it + is merged. - Assisting the developer and Product Manager with feature documentation delivery. ##### Planning @@ -228,7 +231,7 @@ have this section by default. Anyone can add these details, but the Product Manager who assigns the issue to a specific release milestone will ensure these details are present and finalized by the time of that milestone's kickoff. -Developers, Technical Writers, and others may help further refine this plan at any time. +Developers, Technical Writers, and others may help further refine this plan at any time on request. The following details should be included: @@ -283,13 +286,23 @@ To update GitLab documentation: TIP: **Tip:** Work in a fork if you do not have developer access to the GitLab project. -Ping the Technical Writer for the relevant [DevOps stage group](https://about.gitlab.com/handbook/product/technical-writing/index.html#assignments) -in your issue or merge request, or within `#docs` if you are a member of GitLab's Slack workspace, if you: +Request help from the Technical Writing team if you: - Need help to choose the correct place for documentation. - Want to discuss a documentation idea or outline. - Want to request any other help. +To request help: + +1. Locate the the Technical Writer for the relevant + [DevOps stage group](https://about.gitlab.com/handbook/product/technical-writing/index.html#assignments). +1. Either: + - If urgent help is required, directly assign the Technical Writer in the issue or + [in the merge request](../../user/project/merge_requests/creating_merge_requests.md#multiple-assignees-starter). + - If non-urgent help is required, ping the Technical Writer in the issue or merge request. + +If you are a member of GitLab's Slack workspace, you can request help in `#docs`. + ### Reviewing and merging Anyone with Maintainer access to the relevant GitLab project can merge documentation changes. @@ -307,11 +320,10 @@ The process involves the following: or other appropriate colleague to confirm accuracy, clarity, and completeness. This can be skipped for minor fixes without substantive content changes. - Technical Writer (Optional). If not completed for a merge request prior to merging, must be scheduled - post-merge. To request a: + post-merge. Schedule post-merge reviews only if an urgent merge is required. To request a: - Pre-merge review, assign the Technical Writer listed for the applicable [DevOps stage group](https://about.gitlab.com/handbook/product/technical-writing/index.html#assignments). - - Post-merge review, [create an issue for one](https://gitlab.com/gitlab-org/gitlab/issues/new?issuable_template=Doc%20Review) - and link it from the MR that makes the documentation change. + - Post-merge review, see [Post-merge reviews](#post-merge-reviews). - Maintainer. For merge requests, Maintainers: - Can always request any of the above reviews. - Review before or after a Technical Writer review. @@ -319,7 +331,7 @@ The process involves the following: - Ensure the appropriate labels are applied, including any required to pick a merge request into a release. - Ensure that, if there has not been a Technical Writer review completed or scheduled, they - [create the required issue](https://gitlab.com/gitlab-org/gitlab/issues/new?issuable_template=Doc%20Review), assign to the technical writer of the given stage group, + [create the required issue](https://gitlab.com/gitlab-org/gitlab/issues/new?issuable_template=Doc%20Review), assign to the Technical Writer of the given stage group, and link it from the merge request. The process is reflected in the **Documentation** @@ -330,3 +342,50 @@ The process is reflected in the **Documentation** If you have ideas for further documentation resources please [create an issue](https://gitlab.com/gitlab-org/gitlab/issues/new?issuable_template=Documentation) using the Documentation template. + +## Post-merge reviews + +If not assigned to a Technical Writer for review prior to merging, a review must be scheduled +immediately after merge by the developer or maintainer. For this, +create an issue using the [Doc Review description template](https://gitlab.com/gitlab-org/gitlab/issues/new?issuable_template=Doc%20Review) +and link to it from the merged merge request that introduced the documentation change. + +Circumstances where a regular pre-merge Technical Writer review might be skipped include: + +- There is a short amount of time left before the milestone release. If there are less than three days + remaining, seek a post-merge review and ping the writer via Slack to ensure the review is + completed as soon as possible. +- The size of the change is small and you have a high degree of confidence + that early users of the feature (for example, GitLab.com users) can easily + use the documentation as written. + +Remember: + +- At GitLab, we treat documentation like code. As with code, documentation must be reviewed to + ensure quality. +- Documentation forms part of the GitLab [definition of done](../contributing/merge_request_workflow.md#definition-of-done). +- That pre-merge Technical Writer reviews should be most common when the code is complete well in + advance of a milestone release and for larger documentation changes. +- You can request a post-merge Technical Writer review of documentation if it's important to get the + code with which it ships merged as soon as possible. In this case, the author of the original MR + will address the feedback provided by the Technical Writer in a follow-up MR. +- The Technical Writer can also help decide that documentation can be merged without Technical + writer review, with the review to occur soon after merge. + +### Before merging + +Ensure the following if skipping an initial Technical Writer review: + +- That [product badges](styleguide.md#product-badges) are applied. +- That the GitLab [version](styleguide.md#text-for-documentation-requiring-version-text) that + introduced the feature has been included. +- That changes to headings don't affect in-app hyperlinks. +- Specific [user permissions](../../user/permissions.md) are documented. +- That new documents are linked from higher-level indexes, for discoverability. +- Style guide is followed: + - For [directories and files](styleguide.md#working-with-directories-and-files). + - For [images](styleguide.md#images). + +NOTE: **Note:** +Merge requests that change the location of documentation must always be reviewed by a Technical +Writer prior to merging. diff --git a/doc/development/experiment_guide/index.md b/doc/development/experiment_guide/index.md index 5155433c9ad..98086fc63ea 100644 --- a/doc/development/experiment_guide/index.md +++ b/doc/development/experiment_guide/index.md @@ -24,7 +24,7 @@ The author then adds a comment to this piece of code and adds a link to the issu ## How to create an A/B test -- [ ] Add the experiment to the `Gitlab::Experimentation::EXPERIMENTS` hash in [`experimentation.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib%2Fgitlab%2Fexperimentation.rb): +- Add the experiment to the `Gitlab::Experimentation::EXPERIMENTS` hash in [`experimentation.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib%2Fgitlab%2Fexperimentation.rb): ```ruby EXPERIMENTS = { @@ -40,7 +40,7 @@ The author then adds a comment to this piece of code and adds a link to the issu }.freeze ``` -- [ ] Use the experiment in a controller: +- Use the experiment in a controller: ```ruby class RegistrationController < Applicationcontroller @@ -55,8 +55,8 @@ The author then adds a comment to this piece of code and adds a link to the issu end ``` -- [ ] Track necessery events. See the [event tracking guide](../event_tracking/index.md) for details. -- [ ] After the merge request is merged, use [`chatops`](../../ci/chatops/README.md) to enable the feature flag and start the experiment. For visibility, please run the command in the `#s_growth` channel: +- Track necessary events. See the [event tracking guide](../event_tracking/index.md) for details. +- After the merge request is merged, use [`chatops`](../../ci/chatops/README.md) to enable the feature flag and start the experiment. For visibility, please run the command in the `#s_growth` channel: ``` /chatops run feature set --project=gitlab-org/gitlab experimental_sign_up_flow true diff --git a/doc/development/fe_guide/accessibility.md b/doc/development/fe_guide/accessibility.md index 64c793cfd64..4fd9a4fed60 100644 --- a/doc/development/fe_guide/accessibility.md +++ b/doc/development/fe_guide/accessibility.md @@ -16,5 +16,5 @@ useful compilation of accessibility-related material. [audit-rules]: https://github.com/GoogleChrome/accessibility-developer-tools/wiki/Audit-Rules [axe-website]: https://www.deque.com/axe/ [axe-firefox-extension]: https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/ -[axe-chrome-extension]: https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindnejefpokejbdd +[axe-chrome-extension]: https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd [awesome-a11y]: https://github.com/brunopulis/awesome-a11y diff --git a/doc/development/fe_guide/frontend_faq.md b/doc/development/fe_guide/frontend_faq.md index 36c8a03a241..cbe0a78370d 100644 --- a/doc/development/fe_guide/frontend_faq.md +++ b/doc/development/fe_guide/frontend_faq.md @@ -15,7 +15,7 @@ ## FAQ -### How do I find the Rails route for a page? +### 1. How do I find the Rails route for a page? #### Check the 'page' data attribute @@ -37,7 +37,7 @@ The output includes the request types available, route parameters and the releva bundle exec rake routes | grep "issues" ``` -### `modal_copy_button` vs `clipboard_button` +### 2. `modal_copy_button` vs `clipboard_button` The `clipboard_button` uses the `copy_to_clipboard.js` behaviour, which is initialized on page load, so if there are vue-based clipboard buttons that @@ -50,3 +50,26 @@ the instance of that component, which means that clipboard events are bound on mounting and destroyed when the button is, mitigating the above issue. It also has bindings to a particular container or modal ID available, to work with the focus trap created by our GlModal. + +### 3. A `gitlab-ui` component not conforming to [Pajamas Design System](https://design.gitlab.com/) + +Some [Pajamas Design System](https://design.gitlab.com/) components implemented in +`gitlab-ui` do not conform with the design system specs because they lack some +planned features or are not correctly styled yet. In the Pajamas website, a +banner on top of the component examples indicates that: + +> This component does not yet conform to the correct styling defined in our Design +> System. Refer to the Design System documentation when referencing visuals for this +> component. + +For example, at the time of writing, this type of warning can be observed for +[all form components](https://design.gitlab.com/components/forms). It, however, +doesn't imply that the component should not be used. + +GitLab always asks to use `<gl-*>` components whenever a suitable component exists. +It makes codebase unified and more comfortable to maintain/refactor in the future. + +Ensure a [Product Designer](https://about.gitlab.com/company/team/?department=ux-department) +reviews the use of the non-conforming component as part of the MR review. Make a +follow up issue and attach it to the component implementation epic found within the +[Components of Pajamas Design System epic](https://gitlab.com/groups/gitlab-org/-/epics/973). diff --git a/doc/development/fe_guide/graphql.md b/doc/development/fe_guide/graphql.md index 894a613ec2d..40b9fdef76e 100644 --- a/doc/development/fe_guide/graphql.md +++ b/doc/development/fe_guide/graphql.md @@ -39,6 +39,38 @@ To distinguish queries from mutations and fragments, the following naming conven - `addUser.mutation.graphql` for mutations; - `basicUser.fragment.graphql` for fragments. +### Fragments + +Fragments are a way to make your complex GraphQL queries more readable and re-usable. Here is an example of GraphQL fragment: + +```javascript +fragment DesignListItem on Design { + id + image + event + filename + notesCount +} +``` + +Fragments can be stored in separate files, imported and used in queries, mutations or other fragments. + +```javascript +#import "./designList.fragment.graphql" +#import "./diffRefs.fragment.graphql" + +fragment DesignItem on Design { + ...DesignListItem + fullPath + diffRefs { + ...DesignDiffRefs + } +} +``` + +More about fragments: +[GraphQL Docs](https://graphql.org/learn/queries/#fragments) + ## Usage in Vue To use Vue Apollo, import the [Vue Apollo][vue-apollo] plugin as well @@ -100,6 +132,8 @@ Read more about local state management with Apollo in the [Vue Apollo documentat ### Testing +#### Mocking response as component data + With [Vue test utils][vue-test-utils] it is easy to quickly test components that fetch GraphQL queries. The simplest way is to use `shallowMount` and then set the data on the component @@ -114,7 +148,100 @@ it('tests apollo component', () => { }); ``` -Another possible way is testing queries with mocked GraphQL schema. Read more about this way in [Vue Apollo testing documentation](https://vue-apollo.netlify.com/guide/testing.html#tests-with-mocked-graqhql-schema) +#### Testing loading state + +If we need to test how our component renders when results from the GraphQL API are still loading, we can mock a loading state into respective Apollo queries/mutations: + +```javascript + function createComponent({ + loading = false, + } = {}) { + const $apollo = { + queries: { + designs: { + loading, + }, + }; + + wrapper = shallowMount(Index, { + sync: false, + mocks: { $apollo } + }); + } + + it('renders loading icon', () => { + createComponent({ loading: true }); + + expect(wrapper.element).toMatchSnapshot(); +}) +``` + +#### Testing Apollo components + +If we use `ApolloQuery` or `ApolloMutation` in our components, in order to test their functionality we need to add a stub first: + +```javascript +import { ApolloMutation } from 'vue-apollo'; + +function createComponent(props = {}) { + wrapper = shallowMount(MyComponent, { + sync: false, + propsData: { + ...props, + }, + stubs: { + ApolloMutation, + }, + }); +} +``` + +`ApolloMutation` component exposes `mutate` method via scoped slot. If we want to test this method, we need to add it to mocks: + +```javascript +const mutate = jest.fn(() => Promise.resolve()); +const $apollo = { + mutate, +}; + +function createComponent(props = {}) { + wrapper = shallowMount(MyComponent, { + sync: false, + propsData: { + ...props, + }, + stubs: { + ApolloMutation, + }, + mocks: { + $apollo: + } + }); +} +``` + +Then we can check if `mutate` is called with correct variables: + +```javascript +const mutationVariables = { + mutation: createNoteMutation, + update: expect.anything(), + variables: { + input: { + noteableId: 'noteable-id', + body: 'test', + discussionId: '0', + }, + }, +}; + +it('calls mutation on submitting form ', () => { + createComponent() + findReplyForm().vm.$emit('submitForm'); + + expect(mutate).toHaveBeenCalledWith(mutationVariables); +}); +``` ## Usage outside of Vue diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 1cf798cedb6..f13ef767660 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -76,15 +76,18 @@ Read the [frontend's FAQ](frontend_faq.md) for common small pieces of helpful in ## Style Guides -### [JavaScript Style Guide](style_guide_js.md) +See the relevant style guides for our guidelines and for information on linting: -We use eslint to enforce our JavaScript style guides. Our guide is based on +- [JavaScript](style/javascript.md). Our guide is based on the excellent [Airbnb][airbnb-js-style-guide] style guide with a few small changes. +- [SCSS](style/scss.md): our SCSS conventions which are enforced through [`scss-lint`](https://github.com/brigade/scss-lint). +- [HTML](style/html.md). Guidelines for writing HTML code consistent with the rest of the codebase. +- [Vue](style/vue.md). Guidelines and conventions for Vue code may be found here. -### [SCSS Style Guide](style_guide_scss.md) +## Tooling -Our SCSS conventions which are enforced through [scss-lint](https://github.com/sds/scss-lint). +Our code is automatically formatted with [Prettier](https://prettier.io) to follow our guidelines. Read our [Tooling guide](tooling.md) for more detail. ## [Performance](performance.md) diff --git a/doc/development/fe_guide/style/html.md b/doc/development/fe_guide/style/html.md new file mode 100644 index 00000000000..1445da3f0e1 --- /dev/null +++ b/doc/development/fe_guide/style/html.md @@ -0,0 +1,53 @@ +# HTML style guide + +## Buttons + +### Button type + +Button tags requires a `type` attribute according to the [W3C HTML specification](https://www.w3.org/TR/2011/WD-html5-20110525/the-button-element.html#dom-button-type). + +```html +// bad +<button></button> + +// good +<button type="button"></button> +``` + +### Button role + +If an HTML element has an `onClick` handler but is not a button, it should have `role="button"`. This is [more accessible](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Roles/button_role). + +```html +// bad +<div onClick="doSomething"></div> + +// good +<div role="button" onClick="doSomething"></div> +``` + +## Links + +### Blank target + +Use `rel="noopener noreferrer"` whenever your links open in a new window, i.e. `target="_blank"`. This prevents a security vulnerability [documented by JitBit](https://www.jitbit.com/alexblog/256-targetblank---the-most-underestimated-vulnerability-ever/). + +```html +// bad +<a href="url" target="_blank"></a> + +// good +<a href="url" target="_blank" rel="noopener noreferrer"></a> +``` + +### Fake links + +**Do not use fake links.** Use a button tag if a link only invokes JavaScript click event handlers, which is more semantic. + +```html +// bad +<a class="js-do-something" href="#"></a> + +// good +<button class="js-do-something" type="button"></button> +``` diff --git a/doc/development/fe_guide/style/index.md b/doc/development/fe_guide/style/index.md new file mode 100644 index 00000000000..3b07a8557f5 --- /dev/null +++ b/doc/development/fe_guide/style/index.md @@ -0,0 +1,21 @@ +# GitLab development style guides + +See below the relevant style guides, guidelines, linting, and other information for developing GitLab. + +## JavaScript style guide + +We use `eslint` to enforce our [JavaScript style guides](javascript.md). Our guide is based on +the excellent [AirBnB](https://github.com/airbnb/javascript) style guide with a few small +changes. + +## SCSS style guide + +Our [SCSS conventions](scss.md) which are enforced through [`scss-lint`](https://github.com/brigade/scss-lint). + +## HTML style guide + +Guidelines for writing [HTML code](html.md) consistent with the rest of the codebase. + +## Vue style guide + +Guidelines and conventions for Vue code may be found within the [Vue style guide](vue.md). diff --git a/doc/development/fe_guide/style/javascript.md b/doc/development/fe_guide/style/javascript.md new file mode 100644 index 00000000000..f40e8c7b5df --- /dev/null +++ b/doc/development/fe_guide/style/javascript.md @@ -0,0 +1,275 @@ +--- +disqus_identifier: 'https://docs.gitlab.com/ee/development/fe_guide/style_guide_js.html' +--- + +# JavaScript style guide + +We use [Airbnb's JavaScript Style Guide](https://github.com/airbnb/javascript) and it's accompanying +linter to manage most of our JavaScript style guidelines. + +In addition to the style guidelines set by Airbnb, we also have a few specific rules +listed below. + +> **Tip:** +You can run eslint locally by running `yarn eslint` + +## Avoid forEach + +Avoid forEach when mutating data. Use `map`, `reduce` or `filter` instead of `forEach` +when mutating data. This will minimize mutations in functions, +which aligns with [Airbnb's style guide](https://github.com/airbnb/javascript#testing--for-real). + +```javascript +// bad +users.forEach((user, index) => { + user.id = index; +}); + +// good +const usersWithId = users.map((user, index) => { + return Object.assign({}, user, { id: index }); +}); +``` + +## Limit number of parameters + +If your function or method has more than 3 parameters, use an object as a parameter +instead. + +```javascript +// bad +function a(p1, p2, p3) { + // ... +}; + +// good +function a(p) { + // ... +}; +``` + +## Avoid classes to handle DOM events + +If the only purpose of the class is to bind a DOM event and handle the callback, prefer +using a function. + +```javascript +// bad +class myClass { + constructor(config) { + this.config = config; + } + + init() { + document.addEventListener('click', () => {}); + } +} + +// good + +const myFunction = () => { + document.addEventListener('click', () => { + // handle callback here + }); +} +``` + +## Pass element container to constructor + +When your class manipulates the DOM, receive the element container as a parameter. +This is more maintainable and performant. + +```javascript +// bad +class a { + constructor() { + document.querySelector('.b'); + } +} + +// good +class a { + constructor(options) { + options.container.querySelector('.b'); + } +} +``` + +## Use ParseInt + +Use `ParseInt` when converting a numeric string into a number. + +```javascript +// bad +Number('10') + +// good +parseInt('10', 10); +``` + +## CSS Selectors - Use `js-` prefix + +If a CSS class is only being used in JavaScript as a reference to the element, prefix +the class name with `js-`. + +```html +// bad +<button class="add-user"></button> + +// good +<button class="js-add-user"></button> +``` + +## ES Module Syntax + +Use ES module syntax to import modules: + +```javascript +// bad +const SomeClass = require('some_class'); + +// good +import SomeClass from 'some_class'; + +// bad +module.exports = SomeClass; + +// good +export default SomeClass; +``` + +_Note:_ We still use `require` in `scripts/` and `config/` files. + +## Absolute vs relative paths for modules + +Use relative paths if the module you are importing is less than two levels up. + +```javascript +// bad +import GitLabStyleGuide from '~/guides/GitLabStyleGuide'; + +// good +import GitLabStyleGuide from '../GitLabStyleGuide'; +``` + +If the module you are importing is two or more levels up, use an absolute path instead: + +```javascript +// bad +import GitLabStyleGuide from '../../../guides/GitLabStyleGuide'; + +// good +import GitLabStyleGuide from '~/GitLabStyleGuide'; +``` + +Additionally, **do not add to global namespace**. + +## Do not use `DOMContentLoaded` in non-page modules + +Imported modules should act the same each time they are loaded. `DOMContentLoaded` +events are only allowed on modules loaded in the `/pages/*` directory because those +are loaded dynamically with webpack. + +## Avoid XSS + +Do not use `innerHTML`, `append()` or `html()` to set content. It opens up too many +vulnerabilities. + +## ESLint + +ESLint behaviour can be found in our [tooling guide](../tooling.md). + +## IIFEs + +Avoid using IIFEs (Immediately-Invoked Function Expressions). Although +we have a lot of examples of files which wrap their contents in IIFEs, +this is no longer necessary after the transition from Sprockets to webpack. +Do not use them anymore and feel free to remove them when refactoring legacy code. + +## Global namespace + +Avoid adding to the global namespace. + +```javascript +// bad +window.MyClass = class { /* ... */ }; + +// good +export default class MyClass { /* ... */ } +``` + +## Side effects + +### Top-level side effects + +Top-level side effects are forbidden in any script which contains `export`: + +```javascript +// bad +export default class MyClass { /* ... */ } + +document.addEventListener("DOMContentLoaded", function(event) { + new MyClass(); +} +``` + +### Avoid side effects in constructors + +Avoid making asynchronous calls, API requests or DOM manipulations in the `constructor`. +Move them into separate functions instead. This will make tests easier to write and +avoids violating the [Single Responsibility Principle](https://en.wikipedia.org/wiki/Single_responsibility_principle). + +```javascript +// bad +class myClass { + constructor(config) { + this.config = config; + axios.get(this.config.endpoint) + } +} + +// good +class myClass { + constructor(config) { + this.config = config; + } + + makeRequest() { + axios.get(this.config.endpoint) + } +} +const instance = new myClass(); +instance.makeRequest(); +``` + +## Pure Functions and Data Mutation + +Strive to write many small pure functions and minimize where mutations occur + + ```javascript + // bad + const values = {foo: 1}; + + function impureFunction(items) { + const bar = 1; + + items.foo = items.a * bar + 2; + + return items.a; + } + + const c = impureFunction(values); + + // good + var values = {foo: 1}; + + function pureFunction (foo) { + var bar = 1; + + foo = foo * bar + 2; + + return foo; + } + + var c = pureFunction(values.foo); + ``` diff --git a/doc/development/fe_guide/style/scss.md b/doc/development/fe_guide/style/scss.md new file mode 100644 index 00000000000..4ec3c79b0ae --- /dev/null +++ b/doc/development/fe_guide/style/scss.md @@ -0,0 +1,285 @@ +--- +disqus_identifier: 'https://docs.gitlab.com/ee/development/fe_guide/style_guide_scss.html' +--- + +# SCSS style guide + +This style guide recommends best practices for SCSS to make styles easy to read, +easy to maintain, and performant for the end-user. + +## Rules + +### Utility Classes + +As part of the effort for [cleaning up our CSS and moving our components into `gitlab-ui`](https://gitlab.com/groups/gitlab-org/-/epics/950) +led by the [GitLab UI WG](https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/20623) we prefer the use of utility classes over adding new CSS. However, complex CSS can be addressed by adding component classes. + +#### Where are utility classes defined? + +- [Bootstrap's Utility Classes](https://getbootstrap.com/docs/4.3/utilities/) +- [`common.scss`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/assets/stylesheets/framework/common.scss) (old) +- [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/assets/stylesheets/utilities.scss) (new) + +#### Where should I put new utility classes? + +New utility classes should be added to [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/assets/stylesheets/utilities.scss). Existing classes include: + +| Name | Pattern | Example | +|------|---------|---------| +| Background color | `.bg-{variant}-{shade}` | `.bg-warning-400` | +| Text color | `.text-{variant}-{shade}` | `.text-success-500` | +| Font size | `.text-{size}` | `.text-2` | + +- `{variant}` is one of 'primary', 'secondary', 'success', 'warning', 'error' +- `{shade}` is one of the shades listed on [colors](https://design.gitlab.com/product-foundations/colors/) +- `{size}` is a number from 1-6 from our [Type scale](https://design.gitlab.com/product-foundations/typography/) + +#### When should I create component classes? + +We recommend a "utility-first" approach. + +1. Start with utility classes. +1. If composing utility classes into a component class removes code duplication and encapsulates a clear responsibility, do it. + +This encourages an organic growth of component classes and prevents the creation of one-off unreusable classes. Also, the kind of classes that emerge from "utility-first" tend to be design-centered (e.g. `.button`, `.alert`, `.card`) rather than domain-centered (e.g. `.security-report-widget`, `.commit-header-icon`). + +Examples of component classes that were created using "utility-first" include: + +- [`.circle-icon-container`](https://gitlab.com/gitlab-org/gitlab/blob/579fa8b8ec7eb38d40c96521f517c9dab8c3b97a/app/assets/stylesheets/framework/icons.scss#L85) +- [`.d-flex-center`](https://gitlab.com/gitlab-org/gitlab/blob/900083d89cd6af391d26ab7922b3f64fa2839bef/app/assets/stylesheets/framework/common.scss#L425) + +Inspiration: + +- <https://tailwindcss.com/docs/utility-first/> +- <https://tailwindcss.com/docs/extracting-components/> + +### Naming + +Filenames should use `snake_case`. + +CSS classes should use the `lowercase-hyphenated` format rather than +`snake_case` or `camelCase`. + +```scss +// Bad +.class_name { + color: #fff; +} + +// Bad +.className { + color: #fff; +} + +// Good +.class-name { + color: #fff; +} +``` + +### Formatting + +You should always use a space before a brace, braces should be on the same +line, each property should each get its own line, and there should be a space +between the property and its value. + +```scss +// Bad +.container-item { + width: 100px; height: 100px; + margin-top: 0; +} + +// Bad +.container-item +{ + width: 100px; + height: 100px; + margin-top: 0; +} + +// Bad +.container-item{ + width:100px; + height:100px; + margin-top:0; +} + +// Good +.container-item { + width: 100px; + height: 100px; + margin-top: 0; +} +``` + +Note that there is an exception for single-line rulesets, although these are +not typically recommended. + +```scss +p { margin: 0; padding: 0; } +``` + +### Colors + +HEX (hexadecimal) colors should use shorthand where possible, and should use +lower case letters to differentiate between letters and numbers, e.g. `#E3E3E3` +vs. `#e3e3e3`. + +```scss +// Bad +p { + color: #ffffff; +} + +// Bad +p { + color: #FFFFFF; +} + +// Good +p { + color: #fff; +} +``` + +### Indentation + +Indentation should always use two spaces for each indentation level. + +```scss +// Bad, four spaces +p { + color: #f00; +} + +// Good +p { + color: #f00; +} +``` + +### Semicolons + +Always include semicolons after every property. When the stylesheets are +minified, the semicolons will be removed automatically. + +```scss +// Bad +.container-item { + width: 100px; + height: 100px +} + +// Good +.container-item { + width: 100px; + height: 100px; +} +``` + +### Shorthand + +The shorthand form should be used for properties that support it. + +```scss +// Bad +margin: 10px 15px 10px 15px; +padding: 10px 10px 10px 10px; + +// Good +margin: 10px 15px; +padding: 10px; +``` + +### Zero Units + +Omit length units on zero values, they're unnecessary and not including them +is slightly more performant. + +```scss +// Bad +.item-with-padding { + padding: 0px; +} + +// Good +.item-with-padding { + padding: 0; +} +``` + +### Selectors with a `js-` Prefix + +Do not use any selector prefixed with `js-` for styling purposes. These +selectors are intended for use only with JavaScript to allow for removal or +renaming without breaking styling. + +### IDs + +Don't use ID selectors in CSS. + +```scss +// Bad +#my-element { + padding: 0; +} + +// Good +.my-element { + padding: 0; +} +``` + +### Variables + +Before adding a new variable for a color or a size, guarantee: + +- There isn't already one +- There isn't a similar one we can use instead. + +## Linting + +We use [SCSS Lint](https://github.com/sds/scss-lint) to check for style guide conformity. It uses the +ruleset in `.scss-lint.yml`, which is located in the home directory of the +project. + +To check if any warnings will be produced by your changes, you can run `rake +scss_lint` in the GitLab directory. SCSS Lint will also run in GitLab CI to +catch any warnings. + +If the Rake task is throwing warnings you don't understand, SCSS Lint's +documentation includes [a full list of their linters](https://github.com/sds/scss-lint/blob/master/lib/scss_lint/linter/README.md). + +### Fixing issues + +If you want to automate changing a large portion of the codebase to conform to +the SCSS style guide, you can use [CSSComb][csscomb]. First install +[Node][node] and [NPM][npm], then run `npm install csscomb -g` to install +CSSComb globally (system-wide). Run it in the GitLab directory with +`csscomb app/assets/stylesheets` to automatically fix issues with CSS/SCSS. + +Note that this won't fix every problem, but it should fix a majority. + +### Ignoring issues + +If you want a line or set of lines to be ignored by the linter, you can use +`// scss-lint:disable RuleName` ([more info](https://github.com/sds/scss-lint#disabling-linters-via-source)): + +```scss +// This lint rule is disabled because it is supported only in Chrome/Safari +// scss-lint:disable PropertySpelling +body { + text-decoration-skip: ink; +} +// scss-lint:enable PropertySpelling +``` + +Make sure a comment is added on the line above the `disable` rule, otherwise the +linter will throw a warning. `DisableLinterReason` is enabled to make sure the +style guide isn't being ignored, and to communicate to others why the style +guide is ignored in this instance. + +[csscomb]: https://github.com/csscomb/csscomb.js +[node]: https://github.com/nodejs/node +[npm]: https://www.npmjs.com/ diff --git a/doc/development/fe_guide/style/vue.md b/doc/development/fe_guide/style/vue.md new file mode 100644 index 00000000000..2499623e66a --- /dev/null +++ b/doc/development/fe_guide/style/vue.md @@ -0,0 +1,418 @@ +# Vue.js style guide + +## Linting + +We default to [eslint-vue-plugin](https://github.com/vuejs/eslint-plugin-vue), with the `plugin:vue/recommended`. +Please check this [rules](https://github.com/vuejs/eslint-plugin-vue#bulb-rules) for more documentation. + +## Basic Rules + +1. The service has it's own file +1. The store has it's own file +1. Use a function in the bundle file to instantiate the Vue component: + + ```javascript + // bad + class { + init() { + new Component({}) + } + } + + // good + document.addEventListener('DOMContentLoaded', () => new Vue({ + el: '#element', + components: { + componentName + }, + render: createElement => createElement('component-name'), + })); + ``` + +1. Do not use a singleton for the service or the store + + ```javascript + // bad + class Store { + constructor() { + if (!this.prototype.singleton) { + // do something + } + } + } + + // good + class Store { + constructor() { + // do something + } + } + ``` + +1. Use `.vue` for Vue templates. Do not use `%template` in HAML. + +## Naming + +1. **Extensions**: Use `.vue` extension for Vue components. Do not use `.js` as file extension ([#34371]). +1. **Reference Naming**: Use PascalCase for their instances: + + ```javascript + // bad + import cardBoard from 'cardBoard.vue' + + components: { + cardBoard, + }; + + // good + import CardBoard from 'cardBoard.vue' + + components: { + CardBoard, + }; + ``` + +1. **Props Naming:** Avoid using DOM component prop names. +1. **Props Naming:** Use kebab-case instead of camelCase to provide props in templates. + + ```javascript + // bad + <component class="btn"> + + // good + <component css-class="btn"> + + // bad + <component myProp="prop" /> + + // good + <component my-prop="prop" /> + ``` + +[#34371]: https://gitlab.com/gitlab-org/gitlab-foss/issues/34371 + +## Alignment + +1. Follow these alignment styles for the template method: + + 1. With more than one attribute, all attributes should be on a new line: + + ```javascript + // bad + <component v-if="bar" + param="baz" /> + + <button class="btn">Click me</button> + + // good + <component + v-if="bar" + param="baz" + /> + + <button class="btn"> + Click me + </button> + ``` + + 1. The tag can be inline if there is only one attribute: + + ```javascript + // good + <component bar="bar" /> + + // good + <component + bar="bar" + /> + + // bad + <component + bar="bar" /> + ``` + +## Quotes + +1. Always use double quotes `"` inside templates and single quotes `'` for all other JS. + + ```javascript + // bad + template: ` + <button :class='style'>Button</button> + ` + + // good + template: ` + <button :class="style">Button</button> + ` + ``` + +## Props + +1. Props should be declared as an object + + ```javascript + // bad + props: ['foo'] + + // good + props: { + foo: { + type: String, + required: false, + default: 'bar' + } + } + ``` + +1. Required key should always be provided when declaring a prop + + ```javascript + // bad + props: { + foo: { + type: String, + } + } + + // good + props: { + foo: { + type: String, + required: false, + default: 'bar' + } + } + ``` + +1. Default key should be provided if the prop is not required. + _Note:_ There are some scenarios where we need to check for the existence of the property. + On those a default key should not be provided. + + ```javascript + // good + props: { + foo: { + type: String, + required: false, + } + } + + // good + props: { + foo: { + type: String, + required: false, + default: 'bar' + } + } + + // good + props: { + foo: { + type: String, + required: true + } + } + ``` + +## Data + +1. `data` method should always be a function + + ```javascript + // bad + data: { + foo: 'foo' + } + + // good + data() { + return { + foo: 'foo' + }; + } + ``` + +## Directives + +1. Shorthand `@` is preferable over `v-on` + + ```javascript + // bad + <component v-on:click="eventHandler"/> + + // good + <component @click="eventHandler"/> + ``` + +1. Shorthand `:` is preferable over `v-bind` + + ```javascript + // bad + <component v-bind:class="btn"/> + + // good + <component :class="btn"/> + ``` + +1. Shorthand `#` is preferable over `v-slot` + + ```javascript + // bad + <template v-slot:header></template> + + // good + <template #header></template> + ``` + +## Closing tags + +1. Prefer self closing component tags + + ```javascript + // bad + <component></component> + + // good + <component /> + ``` + +## Component usage within templates + +1. Prefer a component's kebab-cased name over other styles when using it in a template + + ```javascript + // bad + <MyComponent /> + + // good + <my-component /> + ``` + +## Ordering + +1. Tag order in `.vue` file + + ``` + <script> + // ... + </script> + + <template> + // ... + </template> + + // We don't use scoped styles but there are few instances of this + <style> + // ... + </style> + ``` + +1. Properties in a Vue Component: + Check [order of properties in components rule](https://github.com/vuejs/eslint-plugin-vue/blob/master/docs/rules/order-in-components.md). + +## `:key` + +When using `v-for` you need to provide a *unique* `:key` attribute for each item. + +1. If the elements of the array being iterated have an unique `id` it is advised to use it: + + ```html + <div + v-for="item in items" + :key="item.id" + > + <!-- content --> + </div> + ``` + +1. When the elements being iterated don't have a unique id, you can use the array index as the `:key` attribute + + ```html + <div + v-for="(item, index) in items" + :key="index" + > + <!-- content --> + </div> + ``` + +1. When using `v-for` with `template` and there is more than one child element, the `:key` values must be unique. It's advised to use `kebab-case` namespaces. + + ```html + <template v-for="(item, index) in items"> + <span :key="`span-${index}`"></span> + <button :key="`button-${index}`"></button> + </template> + ``` + +1. When dealing with nested `v-for` use the same guidelines as above. + + ```html + <div + v-for="item in items" + :key="item.id" + > + <span + v-for="element in array" + :key="element.id" + > + <!-- content --> + </span> + </div> + ``` + +Useful links: + +1. [`key`](https://vuejs.org/v2/guide/list.html#key) +1. [Vue Style Guide: Keyed v-for](https://vuejs.org/v2/style-guide/#Keyed-v-for-essential ) + +## Vue and Bootstrap + +1. Tooltips: Do not rely on `has-tooltip` class name for Vue components + + ```javascript + // bad + <span + class="has-tooltip" + title="Some tooltip text"> + Text + </span> + + // good + <span + v-tooltip + title="Some tooltip text"> + Text + </span> + ``` + +1. Tooltips: When using a tooltip, include the tooltip directive, `./app/assets/javascripts/vue_shared/directives/tooltip.js` + +1. Don't change `data-original-title`. + + ```javascript + // bad + <span data-original-title="tooltip text">Foo</span> + + // good + <span title="tooltip text">Foo</span> + + $('span').tooltip('_fixTitle'); + ``` + +## The JavaScript/Vue Accord + +The goal of this accord is to make sure we are all on the same page. + +1. When writing Vue, you may not use jQuery in your application. + 1. If you need to grab data from the DOM, you may query the DOM 1 time while bootstrapping your application to grab data attributes using `dataset`. You can do this without jQuery. + 1. You may use a jQuery dependency in Vue.js following [this example from the docs](https://vuejs.org/v2/examples/select2.html). + 1. If an outside jQuery Event needs to be listen to inside the Vue application, you may use jQuery event listeners. + 1. We will avoid adding new jQuery events when they are not required. Instead of adding new jQuery events take a look at [different methods to do the same task](https://vuejs.org/v2/api/#vm-emit). +1. You may query the `window` object 1 time, while bootstrapping your application for application specific data (e.g. `scrollTo` is ok to access anytime). Do this access during the bootstrapping of your application. +1. You may have a temporary but immediate need to create technical debt by writing code that does not follow our standards, to be refactored later. Maintainers need to be ok with the tech debt in the first place. An issue should be created for that tech debt to evaluate it further and discuss. In the coming months you should fix that tech debt, with it's priority to be determined by maintainers. +1. When creating tech debt you must write the tests for that code before hand and those tests may not be rewritten. e.g. jQuery tests rewritten to Vue tests. +1. You may choose to use VueX as a centralized state management. If you choose not to use VueX, you must use the *store pattern* which can be found in the [Vue.js documentation](https://vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch). +1. Once you have chosen a centralized state management solution you must use it for your entire application. i.e. Don't mix and match your state management solutions. diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 43cd8180b6e..f3fa80325ef 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -1,731 +1,5 @@ -# Style guides and linting +--- +redirect_to: 'style/javascript.md' +--- -See the relevant style guides for our guidelines and for information on linting: - -## JavaScript - -We defer to [Airbnb][airbnb-js-style-guide] on most style-related -conventions and enforce them with eslint. - -See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab/blob/master/.eslintrc.yml) for specific rules and patterns. - -### Common - -#### ESlint - -1. **Never** disable eslint rules unless you have a good reason. - You may see a lot of legacy files with `/* eslint-disable some-rule, some-other-rule */` - at the top, but legacy files are a special case. Any time you develop a new feature or - refactor an existing one, you should abide by the eslint rules. - -1. **Never Ever EVER** disable eslint globally for a file - - ```javascript - // bad - /* eslint-disable */ - - // better - /* eslint-disable some-rule, some-other-rule */ - - // best - // nothing :) - ``` - -1. If you do need to disable a rule for a single violation, try to do it as locally as possible - - ```javascript - // bad - /* eslint-disable no-new */ - - import Foo from 'foo'; - - new Foo(); - - // better - import Foo from 'foo'; - - // eslint-disable-next-line no-new - new Foo(); - ``` - -1. There are few rules that we need to disable due to technical debt. Which are: - 1. [no-new](https://eslint.org/docs/rules/no-new) - 1. [class-methods-use-this](https://eslint.org/docs/rules/class-methods-use-this) - -1. When they are needed _always_ place ESlint directive comment blocks on the first line of a script, - followed by any global declarations, then a blank newline prior to any imports or code. - - ```javascript - // bad - /* global Foo */ - /* eslint-disable no-new */ - import Bar from './bar'; - - // good - /* eslint-disable no-new */ - /* global Foo */ - - import Bar from './bar'; - ``` - -1. **Never** disable the `no-undef` rule. Declare globals with `/* global Foo */` instead. - -1. When declaring multiple globals, always use one `/* global [name] */` line per variable. - - ```javascript - // bad - /* globals Flash, Cookies, jQuery */ - - // good - /* global Flash */ - /* global Cookies */ - /* global jQuery */ - ``` - -1. Use up to 3 parameters for a function or class. If you need more accept an Object instead. - - ```javascript - // bad - fn(p1, p2, p3, p4) {} - - // good - fn(options) {} - ``` - -#### Modules, Imports, and Exports - -1. Use ES module syntax to import modules - - ```javascript - // bad - const SomeClass = require('some_class'); - - // good - import SomeClass from 'some_class'; - - // bad - module.exports = SomeClass; - - // good - export default SomeClass; - ``` - - Import statements are following usual naming guidelines, for example object literals use camel case: - - ```javascript - // some_object file - export default { - key: 'value', - }; - - // bad - import ObjectLiteral from 'some_object'; - - // good - import objectLiteral from 'some_object'; - ``` - -1. Relative paths: when importing a module in the same directory, a child - directory, or an immediate parent directory prefer relative paths. When - importing a module which is two or more levels up, prefer either `~/` or `ee/`. - - In **app/assets/javascripts/my-feature/subdir**: - - ```javascript - // bad - import Foo from '~/my-feature/foo'; - import Bar from '~/my-feature/subdir/bar'; - import Bin from '~/my-feature/subdir/lib/bin'; - - // good - import Foo from '../foo'; - import Bar from './bar'; - import Bin from './lib/bin'; - ``` - - In **spec/javascripts**: - - ```javascript - // bad - import Foo from '../../app/assets/javascripts/my-feature/foo'; - - // good - import Foo from '~/my-feature/foo'; - ``` - - When referencing an **EE component**: - - ```javascript - // bad - import Foo from '../../../../../ee/app/assets/javascripts/my-feature/ee-foo'; - - // good - import Foo from 'ee/my-feature/foo'; - ``` - -1. Avoid using IIFE. Although we have a lot of examples of files which wrap their - contents in IIFEs (immediately-invoked function expressions), - this is no longer necessary after the transition from Sprockets to webpack. - Do not use them anymore and feel free to remove them when refactoring legacy code. - -1. Avoid adding to the global namespace. - - ```javascript - // bad - window.MyClass = class { /* ... */ }; - - // good - export default class MyClass { /* ... */ } - ``` - -1. Side effects are forbidden in any script which contains export - - ```javascript - // bad - export default class MyClass { /* ... */ } - - document.addEventListener("DOMContentLoaded", function(event) { - new MyClass(); - } - ``` - -#### Data Mutation and Pure functions - -1. Strive to write many small pure functions, and minimize where mutations occur. - - ```javascript - // bad - const values = {foo: 1}; - - function impureFunction(items) { - const bar = 1; - - items.foo = items.a * bar + 2; - - return items.a; - } - - const c = impureFunction(values); - - // good - var values = {foo: 1}; - - function pureFunction (foo) { - var bar = 1; - - foo = foo * bar + 2; - - return foo; - } - - var c = pureFunction(values.foo); - ``` - -1. Avoid constructors with side-effects. - Although we aim for code without side-effects we need some side-effects for our code to run. - - If the class won't do anything if we only instantiate it, it's ok to add side effects into the constructor (_Note:_ The following is just an example. If the only purpose of the class is to add an event listener and handle the callback a function will be more suitable.) - - ```javascript - // Bad - export class Foo { - constructor() { - this.init(); - } - init() { - document.addEventListener('click', this.handleCallback) - }, - handleCallback() { - - } - } - - // Good - export class Foo { - constructor() { - document.addEventListener() - } - handleCallback() { - } - } - ``` - - On the other hand, if a class only needs to extend a third party/add event listeners in some specific cases, they should be initialized outside of the constructor. - -1. Prefer `.map`, `.reduce` or `.filter` over `.forEach` - A forEach will most likely cause side effects, it will be mutating the array being iterated. Prefer using `.map`, - `.reduce` or `.filter` - - ```javascript - const users = [ { name: 'Foo' }, { name: 'Bar' } ]; - - // bad - users.forEach((user, index) => { - user.id = index; - }); - - // good - const usersWithId = users.map((user, index) => { - return Object.assign({}, user, { id: index }); - }); - ``` - -#### Parse Strings into Numbers - -1. `parseInt()` is preferable over `Number()` or `+` - - ```javascript - // bad - +'10' // 10 - - // good - Number('10') // 10 - - // better - parseInt('10', 10); - ``` - -#### CSS classes used for JavaScript - -1. If the class is being used in JavaScript it needs to be prepend with `js-` - - ```html - // bad - <button class="add-user"> - Add User - </button> - - // good - <button class="js-add-user"> - Add User - </button> - ``` - -### Vue.js - -#### `eslint-vue-plugin` - -We default to [eslint-vue-plugin][eslint-plugin-vue], with the `plugin:vue/recommended`. -Please check this [rules][eslint-plugin-vue-rules] for more documentation. - -#### Basic Rules - -1. The service has it's own file -1. The store has it's own file -1. Use a function in the bundle file to instantiate the Vue component: - - ```javascript - // bad - class { - init() { - new Component({}) - } - } - - // good - document.addEventListener('DOMContentLoaded', () => new Vue({ - el: '#element', - components: { - componentName - }, - render: createElement => createElement('component-name'), - })); - ``` - -1. Do not use a singleton for the service or the store - - ```javascript - // bad - class Store { - constructor() { - if (!this.prototype.singleton) { - // do something - } - } - } - - // good - class Store { - constructor() { - // do something - } - } - ``` - -1. Use `.vue` for Vue templates. Do not use `%template` in HAML. - -#### Naming - -1. **Extensions**: Use `.vue` extension for Vue components. Do not use `.js` as file extension ([#34371]). -1. **Reference Naming**: Use PascalCase for their instances: - - ```javascript - // bad - import cardBoard from 'cardBoard.vue' - - components: { - cardBoard, - }; - - // good - import CardBoard from 'cardBoard.vue' - - components: { - CardBoard, - }; - ``` - -1. **Props Naming:** Avoid using DOM component prop names. -1. **Props Naming:** Use kebab-case instead of camelCase to provide props in templates. - - ```javascript - // bad - <component class="btn"> - - // good - <component css-class="btn"> - - // bad - <component myProp="prop" /> - - // good - <component my-prop="prop" /> - ``` - -[#34371]: https://gitlab.com/gitlab-org/gitlab-foss/issues/34371 - -#### Alignment - -1. Follow these alignment styles for the template method: - - 1. With more than one attribute, all attributes should be on a new line: - - ```javascript - // bad - <component v-if="bar" - param="baz" /> - - <button class="btn">Click me</button> - - // good - <component - v-if="bar" - param="baz" - /> - - <button class="btn"> - Click me - </button> - ``` - - 1. The tag can be inline if there is only one attribute: - - ```javascript - // good - <component bar="bar" /> - - // good - <component - bar="bar" - /> - - // bad - <component - bar="bar" /> - ``` - -#### Quotes - -1. Always use double quotes `"` inside templates and single quotes `'` for all other JS. - - ```javascript - // bad - template: ` - <button :class='style'>Button</button> - ` - - // good - template: ` - <button :class="style">Button</button> - ` - ``` - -#### Props - -1. Props should be declared as an object - - ```javascript - // bad - props: ['foo'] - - // good - props: { - foo: { - type: String, - required: false, - default: 'bar' - } - } - ``` - -1. Required key should always be provided when declaring a prop - - ```javascript - // bad - props: { - foo: { - type: String, - } - } - - // good - props: { - foo: { - type: String, - required: false, - default: 'bar' - } - } - ``` - -1. Default key should be provided if the prop is not required. - _Note:_ There are some scenarios where we need to check for the existence of the property. - On those a default key should not be provided. - - ```javascript - // good - props: { - foo: { - type: String, - required: false, - } - } - - // good - props: { - foo: { - type: String, - required: false, - default: 'bar' - } - } - - // good - props: { - foo: { - type: String, - required: true - } - } - ``` - -#### Data - -1. `data` method should always be a function - - ```javascript - // bad - data: { - foo: 'foo' - } - - // good - data() { - return { - foo: 'foo' - }; - } - ``` - -#### Directives - -1. Shorthand `@` is preferable over `v-on` - - ```javascript - // bad - <component v-on:click="eventHandler"/> - - // good - <component @click="eventHandler"/> - ``` - -1. Shorthand `:` is preferable over `v-bind` - - ```javascript - // bad - <component v-bind:class="btn"/> - - // good - <component :class="btn"/> - ``` - -1. Shorthand `#` is preferable over `v-slot` - - ```javascript - // bad - <template v-slot:header></template> - - // good - <template #header></template> - ``` - -#### Closing tags - -1. Prefer self closing component tags - - ```javascript - // bad - <component></component> - - // good - <component /> - ``` - -#### Component usage within templates - -1. Prefer a component's kebab-cased name over other styles when using it in a template - - ```javascript - // bad - <MyComponent /> - - // good - <my-component /> - ``` - -#### Ordering - -1. Tag order in `.vue` file - - ``` - <script> - // ... - </script> - - <template> - // ... - </template> - - // We don't use scoped styles but there are few instances of this - <style> - // ... - </style> - ``` - -1. Properties in a Vue Component: - Check [order of properties in components rule][vue-order]. - -#### `:key` - -When using `v-for` you need to provide a *unique* `:key` attribute for each item. - -1. If the elements of the array being iterated have an unique `id` it is advised to use it: - - ```html - <div - v-for="item in items" - :key="item.id" - > - <!-- content --> - </div> - ``` - -1. When the elements being iterated don't have a unique id, you can use the array index as the `:key` attribute - - ```html - <div - v-for="(item, index) in items" - :key="index" - > - <!-- content --> - </div> - ``` - -1. When using `v-for` with `template` and there is more than one child element, the `:key` values must be unique. It's advised to use `kebab-case` namespaces. - - ```html - <template v-for="(item, index) in items"> - <span :key="`span-${index}`"></span> - <button :key="`button-${index}`"></button> - </template> - ``` - -1. When dealing with nested `v-for` use the same guidelines as above. - - ```html - <div - v-for="item in items" - :key="item.id" - > - <span - v-for="element in array" - :key="element.id" - > - <!-- content --> - </span> - </div> - ``` - -Useful links: - -1. [`key`](https://vuejs.org/v2/guide/list.html#key) -1. [Vue Style Guide: Keyed v-for](https://vuejs.org/v2/style-guide/#Keyed-v-for-essential ) - -#### Vue and Bootstrap - -1. Tooltips: Do not rely on `has-tooltip` class name for Vue components - - ```javascript - // bad - <span - class="has-tooltip" - title="Some tooltip text"> - Text - </span> - - // good - <span - v-tooltip - title="Some tooltip text"> - Text - </span> - ``` - -1. Tooltips: When using a tooltip, include the tooltip directive, `./app/assets/javascripts/vue_shared/directives/tooltip.js` - -1. Don't change `data-original-title`. - - ```javascript - // bad - <span data-original-title="tooltip text">Foo</span> - - // good - <span title="tooltip text">Foo</span> - - $('span').tooltip('_fixTitle'); - ``` - -### The JavaScript/Vue Accord - -The goal of this accord is to make sure we are all on the same page. - -1. When writing Vue, you may not use jQuery in your application. - 1. If you need to grab data from the DOM, you may query the DOM 1 time while bootstrapping your application to grab data attributes using `dataset`. You can do this without jQuery. - 1. You may use a jQuery dependency in Vue.js following [this example from the docs](https://vuejs.org/v2/examples/select2.html). - 1. If an outside jQuery Event needs to be listen to inside the Vue application, you may use jQuery event listeners. - 1. We will avoid adding new jQuery events when they are not required. Instead of adding new jQuery events take a look at [different methods to do the same task](https://vuejs.org/v2/api/#vm-emit). -1. You may query the `window` object 1 time, while bootstrapping your application for application specific data (e.g. `scrollTo` is ok to access anytime). Do this access during the bootstrapping of your application. -1. You may have a temporary but immediate need to create technical debt by writing code that does not follow our standards, to be refactored later. Maintainers need to be ok with the tech debt in the first place. An issue should be created for that tech debt to evaluate it further and discuss. In the coming months you should fix that tech debt, with it's priority to be determined by maintainers. -1. When creating tech debt you must write the tests for that code before hand and those tests may not be rewritten. e.g. jQuery tests rewritten to Vue tests. -1. You may choose to use VueX as a centralized state management. If you choose not to use VueX, you must use the *store pattern* which can be found in the [Vue.js documentation](https://vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch). -1. Once you have chosen a centralized state management solution you must use it for your entire application. i.e. Don't mix and match your state management solutions. - -## SCSS - -- [SCSS](style_guide_scss.md) - -[airbnb-js-style-guide]: https://github.com/airbnb/javascript -[eslintrc]: https://gitlab.com/gitlab-org/gitlab/blob/master/.eslintrc -[eslint-plugin-vue]: https://github.com/vuejs/eslint-plugin-vue -[eslint-plugin-vue-rules]: https://github.com/vuejs/eslint-plugin-vue#bulb-rules -[vue-order]: https://github.com/vuejs/eslint-plugin-vue/blob/master/docs/rules/order-in-components.md +This document was moved to [another location](style/javascript.md). diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md index 54d41b42c77..2b4e6427a18 100644 --- a/doc/development/fe_guide/style_guide_scss.md +++ b/doc/development/fe_guide/style_guide_scss.md @@ -1,281 +1,5 @@ -# SCSS styleguide +--- +redirect_to: 'style/scss.md' +--- -This style guide recommends best practices for SCSS to make styles easy to read, -easy to maintain, and performant for the end-user. - -## Rules - -### Utility Classes - -As part of the effort for [cleaning up our CSS and moving our components into GitLab-UI](https://gitlab.com/groups/gitlab-org/-/epics/950) -led by the [GitLab UI WG](https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/20623) we prefer the use of utility classes over adding new CSS. However, complex CSS can be addressed by adding component classes. - -#### Where are utility classes defined? - -- [Bootstrap's Utility Classes](https://getbootstrap.com/docs/4.3/utilities/) -- [`common.scss`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/assets/stylesheets/framework/common.scss) (old) -- [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/assets/stylesheets/utilities.scss) (new) - -#### Where should I put new utility classes? - -New utility classes should be added to [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/assets/stylesheets/utilities.scss). Existing classes include: - -| Name | Pattern | Example | -|------|---------|---------| -| Background color | `.bg-{variant}-{shade}` | `.bg-warning-400` | -| Text color | `.text-{variant}-{shade}` | `.text-success-500` | -| Font size | `.text-{size}` | `.text-2` | - -- `{variant}` is one of 'primary', 'secondary', 'success', 'warning', 'error' -- `{shade}` is one of the shades listed on [colors](https://design.gitlab.com/product-foundations/colors/) -- `{size}` is a number from 1-6 from our [Type scale](https://design.gitlab.com/product-foundations/typography) - -#### When should I create component classes? - -We recommend a "utility-first" approach. - -1. Start with utility classes. -1. If composing utility classes into a component class removes code duplication and encapsulates a clear responsibility, do it. - -This encourages an organic growth of component classes and prevents the creation of one-off unreusable classes. Also, the kind of classes that emerge from "utility-first" tend to be design-centered (e.g. `.button`, `.alert`, `.card`) rather than domain-centered (e.g. `.security-report-widget`, `.commit-header-icon`). - -Examples of component classes that were created using "utility-first" include: - -- [`.circle-icon-container`](https://gitlab.com/gitlab-org/gitlab/blob/579fa8b8ec7eb38d40c96521f517c9dab8c3b97a/app/assets/stylesheets/framework/icons.scss#L85) -- [`.d-flex-center`](https://gitlab.com/gitlab-org/gitlab/blob/900083d89cd6af391d26ab7922b3f64fa2839bef/app/assets/stylesheets/framework/common.scss#L425) - -Inspiration: - -- <https://tailwindcss.com/docs/utility-first/> -- <https://tailwindcss.com/docs/extracting-components/> - -### Naming - -Filenames should use `snake_case`. - -CSS classes should use the `lowercase-hyphenated` format rather than -`snake_case` or `camelCase`. - -```scss -// Bad -.class_name { - color: #fff; -} - -// Bad -.className { - color: #fff; -} - -// Good -.class-name { - color: #fff; -} -``` - -### Formatting - -You should always use a space before a brace, braces should be on the same -line, each property should each get its own line, and there should be a space -between the property and its value. - -```scss -// Bad -.container-item { - width: 100px; height: 100px; - margin-top: 0; -} - -// Bad -.container-item -{ - width: 100px; - height: 100px; - margin-top: 0; -} - -// Bad -.container-item{ - width:100px; - height:100px; - margin-top:0; -} - -// Good -.container-item { - width: 100px; - height: 100px; - margin-top: 0; -} -``` - -Note that there is an exception for single-line rulesets, although these are -not typically recommended. - -```scss -p { margin: 0; padding: 0; } -``` - -### Colors - -HEX (hexadecimal) colors should use shorthand where possible, and should use -lower case letters to differentiate between letters and numbers, e.g. `#E3E3E3` -vs. `#e3e3e3`. - -```scss -// Bad -p { - color: #ffffff; -} - -// Bad -p { - color: #FFFFFF; -} - -// Good -p { - color: #fff; -} -``` - -### Indentation - -Indentation should always use two spaces for each indentation level. - -```scss -// Bad, four spaces -p { - color: #f00; -} - -// Good -p { - color: #f00; -} -``` - -### Semicolons - -Always include semicolons after every property. When the stylesheets are -minified, the semicolons will be removed automatically. - -```scss -// Bad -.container-item { - width: 100px; - height: 100px -} - -// Good -.container-item { - width: 100px; - height: 100px; -} -``` - -### Shorthand - -The shorthand form should be used for properties that support it. - -```scss -// Bad -margin: 10px 15px 10px 15px; -padding: 10px 10px 10px 10px; - -// Good -margin: 10px 15px; -padding: 10px; -``` - -### Zero Units - -Omit length units on zero values, they're unnecessary and not including them -is slightly more performant. - -```scss -// Bad -.item-with-padding { - padding: 0px; -} - -// Good -.item-with-padding { - padding: 0; -} -``` - -### Selectors with a `js-` Prefix - -Do not use any selector prefixed with `js-` for styling purposes. These -selectors are intended for use only with JavaScript to allow for removal or -renaming without breaking styling. - -### IDs - -Don't use ID selectors in CSS. - -```scss -// Bad -#my-element { - padding: 0; -} - -// Good -.my-element { - padding: 0; -} -``` - -### Variables - -Before adding a new variable for a color or a size, guarantee: - -- There isn't already one -- There isn't a similar one we can use instead. - -## Linting - -We use [SCSS Lint](https://github.com/sds/scss-lint) to check for style guide conformity. It uses the -ruleset in `.scss-lint.yml`, which is located in the home directory of the -project. - -To check if any warnings will be produced by your changes, you can run `rake -scss_lint` in the GitLab directory. SCSS Lint will also run in GitLab CI to -catch any warnings. - -If the Rake task is throwing warnings you don't understand, SCSS Lint's -documentation includes [a full list of their linters][scss-lint-documentation](https://github.com/sds/scss-lint/blob/master/lib/scss_lint/linter/README.md). - -### Fixing issues - -If you want to automate changing a large portion of the codebase to conform to -the SCSS style guide, you can use [CSSComb][csscomb]. First install -[Node][node] and [NPM][npm], then run `npm install csscomb -g` to install -CSSComb globally (system-wide). Run it in the GitLab directory with -`csscomb app/assets/stylesheets` to automatically fix issues with CSS/SCSS. - -Note that this won't fix every problem, but it should fix a majority. - -### Ignoring issues - -If you want a line or set of lines to be ignored by the linter, you can use -`// scss-lint:disable RuleName` ([more info](https://github.com/sds/scss-lint#disabling-linters-via-source)): - -```scss -// This lint rule is disabled because it is supported only in Chrome/Safari -// scss-lint:disable PropertySpelling -body { - text-decoration-skip: ink; -} -// scss-lint:enable PropertySpelling -``` - -Make sure a comment is added on the line above the `disable` rule, otherwise the -linter will throw a warning. `DisableLinterReason` is enabled to make sure the -style guide isn't being ignored, and to communicate to others why the style -guide is ignored in this instance. - -[csscomb]: https://github.com/csscomb/csscomb.js -[node]: https://github.com/nodejs/node -[npm]: https://www.npmjs.com/ +This document was moved to [another location](style/scss.md). diff --git a/doc/development/fe_guide/tooling.md b/doc/development/fe_guide/tooling.md new file mode 100644 index 00000000000..066c2575e2d --- /dev/null +++ b/doc/development/fe_guide/tooling.md @@ -0,0 +1,154 @@ +# Tooling + +## ESLint + +We use ESLint to encapsulate and enforce frontend code standards. Our configuration may be found in the [`gitlab-eslint-config`](https://gitlab.com/gitlab-org/gitlab-eslint-config) project. + +### Disabling ESLint in new files + +Do not disable ESLint when creating new files. Existing files may have existing rules +disabled due to legacy compatibility reasons but they are in the process of being refactored. + +Do not disable specific ESLint rules. To avoid introducing technical debt, you may disable the following +rules only if you are invoking/instantiating existing code modules. + +- [`no-new`](https://eslint.org/docs/rules/no-new) +- [`class-method-use-this`](https://eslint.org/docs/rules/class-methods-use-this) + +NOTE: **Note:** +Disable these rules on a per-line basis. This makes it easier to refactor +in the future. E.g. use `eslint-disable-next-line` or `eslint-disable-line`. + +### Disabling ESLint for a single violation + +If you do need to disable a rule for a single violation, disable it for the smallest amount of code necessary: + +```javascript +// bad +/* eslint-disable no-new */ + +import Foo from 'foo'; + +new Foo(); + +// better +import Foo from 'foo'; + +// eslint-disable-next-line no-new +new Foo(); +``` + +### The `no-undef` rule and declaring globals + +**Never** disable the `no-undef` rule. Declare globals with `/* global Foo */` instead. + +When declaring multiple globals, always use one `/* global [name] */` line per variable. + +```javascript +// bad +/* globals Flash, Cookies, jQuery */ + +// good +/* global Flash */ +/* global Cookies */ +/* global jQuery */ +``` + +## Formatting with Prettier + +Our code is automatically formatted with [Prettier](https://prettier.io) to follow our style guides. Prettier is taking care of formatting .js, .vue, and .scss files based on the standard prettier rules. You can find all settings for Prettier in `.prettierrc`. + +### Editor + +The easiest way to include prettier in your workflow is by setting up your preferred editor (all major editors are supported) accordingly. We suggest setting up prettier to run automatically when each file is saved. Find [here](https://prettier.io/docs/en/editors.html) the best way to set it up in your preferred editor. + +Please take care that you only let Prettier format the same file types as the global Yarn script does (.js, .vue, and .scss). In VSCode by example you can easily exclude file formats in your settings file: + +``` + "prettier.disableLanguages": [ + "json", + "markdown" + ], +``` + +### Yarn Script + +The following yarn scripts are available to do global formatting: + +``` +yarn prettier-staged-save +``` + +Updates all currently staged files (based on `git diff`) with Prettier and saves the needed changes. + +``` +yarn prettier-staged +``` + +Checks all currently staged files (based on `git diff`) with Prettier and log which files would need manual updating to the console. + +``` +yarn prettier-all +``` + +Checks all files with Prettier and logs which files need manual updating to the console. + +``` +yarn prettier-all-save +``` + +Formats all files in the repository with Prettier. (This should only be used to test global rule updates otherwise you would end up with huge MR's). + +The source of these Yarn scripts can be found in `/scripts/frontend/prettier.js`. + +#### Scripts during Conversion period + +``` +node ./scripts/frontend/prettier.js check-all ./vendor/ +``` + +This will go over all files in a specific folder check it. + +``` +node ./scripts/frontend/prettier.js save-all ./vendor/ +``` + +This will go over all files in a specific folder and save it. + +### VSCode Settings + +#### Select Prettier as default formatter + +To select Prettier as a formatter, add the following properties to your User or Workspace Settings: + +```javascript +{ + "[html]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[javascript]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[vue]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + } +} +``` + +#### Format on Save + +To automatically format your files with Prettier, add the following properties to your User or Workspace Settings: + +```javascript +{ + "[html]": { + "editor.formatOnSave": true + }, + "[javascript]": { + "editor.formatOnSave": true + }, + "[vue]": { + "editor.formatOnSave": true + }, +} +``` diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index bca24e6ee0b..96bc89675fe 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -78,7 +78,7 @@ document.addEventListener('DOMContentLoaded', () => new Vue({ render(createElement) { return createElement('my-component', { props: { - endpoint: this.isLoading, + endpoint: this.endpoint, }, }); }, @@ -179,7 +179,7 @@ Check this [page](vuex.md) for more details. ## Style guide -Please refer to the Vue section of our [style guide](style_guide_js.md#vuejs) +Please refer to the Vue section of our [style guide](style/vue.md) for best practices while writing your Vue components and templates. ## Testing Vue Components @@ -284,7 +284,7 @@ need to test the rendered output. [Vue][vue-test] guide's to unit test show us e One should apply to be a Vue.js expert by opening an MR when the Merge Request's they create and review show: -- Deep understanding of Vue and Vuex reactivy +- Deep understanding of Vue and Vuex reactivity - Vue and Vuex code are structured according to both official and our guidelines - Full understanding of testing a Vue and Vuex application - Vuex code follows the [documented pattern](vuex.md#actions-pattern-request-and-receive-namespaces) diff --git a/doc/development/feature_flags/controls.md b/doc/development/feature_flags/controls.md index cbbbe1cd5ea..3799672ee11 100644 --- a/doc/development/feature_flags/controls.md +++ b/doc/development/feature_flags/controls.md @@ -19,7 +19,7 @@ run: ## Where to run commands To increase visibility, we recommend that GitLab team members run feature flag -related Chatops commands within certain slack channels based on the environment +related Chatops commands within certain Slack channels based on the environment and related feature. For the [staging](https://staging.gitlab.com) and [development](https://dev.gitlab.org) environments of GitLab.com, the commands should run in a channel for the stage the feature is relevant too. diff --git a/doc/development/feature_flags/development.md b/doc/development/feature_flags/development.md index c410c7eae41..cb5d5fe9ab4 100644 --- a/doc/development/feature_flags/development.md +++ b/doc/development/feature_flags/development.md @@ -38,7 +38,7 @@ Feature.enabled?(:feature_flag, project, default_enabled: true) The [`Project#feature_available?`][project-fa], [`Namespace#feature_available?`][namespace-fa] (EE), and [`License.feature_available?`][license-fa] (EE) methods all implicitly check for -a feature flag by the same name as the provided argument. +a by default enabled feature flag with the same name as the provided argument. For example if a feature is license-gated, there's no need to add an additional explicit feature flag check since the flag will be checked as part of the @@ -56,12 +56,19 @@ isn't gated by a License or Plan. unless the feature is explicitly disabled or limited to a percentage of users, the feature flag check will default to `true`.** -As an example, if you were to ship the backend half of a feature behind a flag, -you'd want to explicitly disable that flag until the frontend half is also ready -to be shipped. To make sure this feature is disabled for both GitLab.com and -self-managed instances you'd need to explicitly call `Feature.enabled?` method -before the `feature_available` method. This ensures the feature_flag is defaulting -to `false`. +This is relevant when developing the feature using +[several smaller merge requests](https://about.gitlab.com/handbook/values/#make-small-merge-requests), or when the feature is considered to be an +[alpha or beta](https://about.gitlab.com/handbook/product/#alpha-beta-ga), and +should not be available by default. + +As an example, if you were to ship the frontend half of a feature without the +backend, you'd want to disable the feature entirely until the backend half is +also ready to be shipped. To make sure this feature is disabled for both +GitLab.com and self-managed instances, you should use the +[`Namespace#alpha_feature_available?`](https://gitlab.com/gitlab-org/gitlab/blob/458749872f4a8f27abe8add930dbb958044cb926/ee/app/models/ee/namespace.rb#L113) or +[`Namespace#beta_feature_available?`](https://gitlab.com/gitlab-org/gitlab/blob/458749872f4a8f27abe8add930dbb958044cb926/ee/app/models/ee/namespace.rb#L100-112) +method, according to our [definitions](https://about.gitlab.com/handbook/product/#alpha-beta-ga). This ensures the feature is disabled unless the feature flag is +_explicitly_ enabled. ## Feature groups @@ -129,3 +136,9 @@ In the rails console (`rails c`), enter the following command to enable your fea ```ruby Feature.enable(:feature_flag_name) ``` + +Similarly, the following command will disable a feature flag: + +```ruby +Feature.disable(:feature_flag_name) +``` diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md index f1374b9e280..f5915f2c0a8 100644 --- a/doc/development/feature_flags/index.md +++ b/doc/development/feature_flags/index.md @@ -10,3 +10,6 @@ Before using feature flags for GitLab's development, read through the following: - [Process for using features flags](process.md). - [Developing with feature flags](development.md). - [Controlling feature flags](controls.md). + +When documenting feature flags, see [Feature flags](../documentation/styleguide.md#feature-flags) +in the Documentation Style Guide. diff --git a/doc/development/git_object_deduplication.md b/doc/development/git_object_deduplication.md index 6d9eb90d482..938882ba5a2 100644 --- a/doc/development/git_object_deduplication.md +++ b/doc/development/git_object_deduplication.md @@ -111,7 +111,7 @@ are as follows: contents of the pool repository are a Git clone of the source project repository. - The occasion for creating a pool is when an existing eligible - (public, hashed storage, non-forked) GitLab project gets forked and + (non-private, hashed storage, non-forked) GitLab project gets forked and this project does not belong to a pool repository yet. The fork parent project becomes the source project of the new pool, and both the fork parent and the fork child project become members of the new diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index 7d3c2b8fdf8..1fa555de994 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -277,14 +277,81 @@ Here are the steps to gate a new feature in Gitaly behind a feature flag. ### GitLab Rails -1. Add feature flag to `lib/gitlab/gitaly_client.rb` (in GitLab Rails): +1. In GitLab Rails: - ```ruby - SERVER_FEATURE_FLAGS = %w[go-find-all-tags].freeze - ``` + 1. Add the feature flag to `SERVER_FEATURE_FLAGS` in `lib/feature/gitaly.rb`: + + ```ruby + SERVER_FEATURE_FLAGS = %w[go-find-all-tags].freeze + ``` + + 1. Search for `["gitaly"]["features"]` (currently in `spec/requests/api/internal/base_spec.rb`) + and fix the expected results for the tests by adding the new feature flag into it: + + ```ruby + expect(json_response["gitaly"]["features"]).to eq('gitaly-feature-get-all-lfs-pointers-go' => 'true', 'gitaly-feature-go-find-all-tags' => 'true') + ``` -1. Test in rails console by setting feature flag: +1. Test in a Rails console by setting the feature flag: + + NOTE: **Note:** + Pay attention to the name of the flag and the one used in the Rails console. + There is a difference between them (dashes replaced by underscores and name + prefix is changed). ```ruby - Feature.enable('gitaly_go-find-all-tags') + Feature.enable('gitaly_go_find_all_tags') ``` + +### Testing with GDK + +To be sure that the flag is set correctly and it goes into Gitaly, you can check +the integration by using GDK: + +1. The state of the flag must be observable. To check it, you need to enable it + by fetching the Prometheus metrics: + 1. Navigate to GDK's root directory. + 1. Make sure you have the proper branch checked out for Gitaly. + 1. Recompile it with `make gitaly-setup` and restart the service with `gdk restart gitaly`. + 1. Make sure your setup is runnig: `gdk status | grep praefect`. + 1. Check what config file is used: `cat ./services/praefect/run | grep praefect` value of the `-config` flag + 1. Uncomment `prometheus_listen_addr` in the configuration file and run `gdk restart gitaly`. + +1. Make sure that the flag is not enabled yet: + 1. Perform whatever action is required to trigger your changes (project creation, + submitting commit, observing history, etc.). + 1. Check that the list of current metrics has the new counter for the feature flag: + + ```sh + curl --silent http://localhost:9236/metrics | grep go_find_all_tags + ``` + +1. Once you observe the metrics for the new feature flag and it increments, you + can enable the new feature: + 1. Navigate to GDK's root directory. + 1. Start a Rails console: + + ```sh + bundle install && bundle exec rails console + ``` + + 1. Check the list of feature flags: + + ```ruby + Feature::Gitaly.server_feature_flags + ``` + + It should be disabled `"gitaly-feature-go-find-all-tags"=>"false"`. + 1. Enable it: + + ```ruby + Feature.enable('gitaly_go_find_all_tags') + ``` + + 1. Exit the Rails console and perform whatever action is required to trigger + your changes (project creation, submitting commit, observing history, etc.). + 1. Verify the feature is on by observing the metrics for it: + + ```sh + curl --silent http://localhost:9236/metrics | grep go_find_all_tags + ``` diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index 33dd9dd9b6f..724bc240bc2 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -336,6 +336,49 @@ Generated docker images should have the program at their `Entrypoint` to create portable commands. That way, anyone can run the image, and without parameters it will display its help message (if `cli` has been used). +## Distributing Go binaries + +With the exception of [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner), +which publishes its own binaries, our Go binaries are created by projects +managed by the [Distribution group](https://about.gitlab.com/handbook/product/categories/#distribution-group). + +The [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab) project creates a +single, monolithic operating system package containing all the binaries, while +the [Cloud-Native GitLab (CNG)](https://gitlab.com/gitlab-org/build/CNG) project +publishes a set of Docker images and Helm charts to glue them together. + +Both approaches use the same version of Go for all projects, so it's important +to ensure all our Go-using projects have at least one Go version in common in +their test matrices. You can check the version of Go currently being used by +[Omnibus](https://gitlab.com/gitlab-org/gitlab-omnibus-builder/blob/master/docker/Dockerfile_debian_10#L59), +and the version being used for [CNG](https://gitlab.com/gitlab-org/build/cng/blob/master/ci_files/variables.yml#L12). + +### Updating Go version + +We should always use a [supported version](https://golang.org/doc/devel/release.html#policy) +of Go, i.e., one of the three most recent minor releases, and should always use +the most recent patch-level for that version, as it may contain security fixes. + +Changing the version affects every project being compiled, so it's important to +ensure that all projects have been updated to test against the new Go version +before changing the package builders to use it. Despite [Go's compatibility promise](https://golang.org/doc/go1compat), +changes between minor versions can expose bugs or cause problems in our projects. + +Once you've picked a new Go version to use, the steps to update Omnibus and CNG +are: + +- [Create a merge request in the CNG project](https://gitlab.com/gitlab-org/build/CNG/edit/master/ci_files/variables.yml?branch_name=update-go-version), + updating the `GO_VERSION` in `ci_files/variables.yml`. +- Create a merge request in the [`gitlab-omnibus-builder` project](https://gitlab.com/gitlab-org/gitlab-omnibus-builder), + updating every file in the `docker/` directory so the `GO_VERSION` is set + appropriately. [Here's an example](https://gitlab.com/gitlab-org/gitlab-omnibus-builder/merge_requests/125/diffs). +- Tag a new release of `gitlab-omnibus-builder` containing the change. +- [Create a merge request in the `gitlab-omnibus` project](https://gitlab.com/gitlab-org/omnibus-gitlab/edit/master/.gitlab-ci.yml?branch_name=update-gitlab-omnibus-builder-version), + updating the `BUILDER_IMAGE_REVISION` to match the newly-created tag. + +To reduce unnecessary differences between two distribution methods, Omnibus and +CNG **should always use the same Go version**. + --- [Return to Development documentation](../README.md). diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index da27ae9110b..7529278f902 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -155,7 +155,7 @@ refresh_service.execute(oldrev, newrev, ref) See ["Why is it bad style to `rescue Exception => e` in Ruby?"](https://stackoverflow.com/questions/10048173/why-is-it-bad-style-to-rescue-exception-e-in-ruby). _**Note:** This rule is [enforced automatically by -Rubocop](https://gitlab.com/gitlab-org/gitlab/blob/8-4-stable/.rubocop.yml#L911-914)._ +Rubocop](https://gitlab.com/gitlab-org/gitlab-foss/blob/8-4-stable/.rubocop.yml#L911-914)._ ## Do not use inline JavaScript in views diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index c8960ac0f61..903ca6ada4a 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -311,6 +311,45 @@ Developer documentation][mdn]. [mdn]: https://developer.mozilla.org/en-US/docs/Mozilla/Localization/Localization_content_best_practices#Splitting +##### Vue components interpolation + +When translating UI text in Vue components, you might want to include child components inside +the translation string. +You could not use a JavaScript-only solution to render the translation, +because Vue would not be aware of the child components and would render them as plain text. + +For this use case, you should use the `gl-sprintf` component which is maintained +in **GitLab UI**. + +The `gl-sprintf` component accepts a `message` property, which is the translatable string, +and it exposes a named slot for every placeholder in the string, which lets you include Vue +components easily. + +Assume you want to print the translatable string +`Pipeline %{pipelineId} triggered %{timeago} by %{author}`. To replace the `%{timeago}` and +`%{author}` placeholders with Vue components, here's how you would do that with `gl-sprintf`: + +```html +<template> + <div> + <gl-sprintf :message="__('Pipeline %{pipelineId} triggered %{timeago} by %{author}')"> + <template #pipelineId>{{ pipeline.id }}</template> + <template #timeago> + <timeago :time="pipeline.triggerTime" /> + </template> + <template #author> + <gl-avatar-labeled + :src="pipeline.triggeredBy.avatarPath" + :label="pipeline.triggeredBy.name" + /> + </template> + </gl-sprintf> + </div> +</template> +``` + +For more information, see the [`gl-sprintf`](https://gitlab-org.gitlab.io/gitlab-ui/?path=/story/base-sprintf--default) documentation. + ## Updating the PO files with the new content Now that the new content is marked for translation, we need to update diff --git a/doc/development/img/build_package_v12_6.png b/doc/development/img/build_package_v12_6.png Binary files differnew file mode 100644 index 00000000000..c3d99e6c6ce --- /dev/null +++ b/doc/development/img/build_package_v12_6.png diff --git a/doc/development/img/trigger_build_package_v12_6.png b/doc/development/img/trigger_build_package_v12_6.png Binary files differnew file mode 100644 index 00000000000..6f5879bd8c4 --- /dev/null +++ b/doc/development/img/trigger_build_package_v12_6.png diff --git a/doc/development/img/trigger_ss1.png b/doc/development/img/trigger_ss1.png Binary files differdeleted file mode 100644 index addbc551f73..00000000000 --- a/doc/development/img/trigger_ss1.png +++ /dev/null diff --git a/doc/development/img/trigger_ss2.png b/doc/development/img/trigger_ss2.png Binary files differdeleted file mode 100644 index 02ef3810a59..00000000000 --- a/doc/development/img/trigger_ss2.png +++ /dev/null diff --git a/doc/development/import_export.md b/doc/development/import_export.md index 38119d9bbd0..3ee723bc897 100644 --- a/doc/development/import_export.md +++ b/doc/development/import_export.md @@ -208,7 +208,7 @@ and it is useful for knowing which versions won't be compatible between them. ### When to bump the version up -We will have to bump the verision if we rename model/columns or perform any format +We will have to bump the version if we rename model/columns or perform any format modifications in the JSON structure or the file structure of the archive file. We do not need to bump the version up in any of the following cases: diff --git a/doc/development/internal_api.md b/doc/development/internal_api.md index b08112aacb2..dbb721b6018 100644 --- a/doc/development/internal_api.md +++ b/doc/development/internal_api.md @@ -4,8 +4,8 @@ The internal API is used by different GitLab components, it can not be used by other consumers. This documentation is intended for people working on the GitLab codebase. -This documentation does not yet include the internal api used by -GitLab pages. +This documentation does not yet include the internal API used by +GitLab Pages. ## Authentication @@ -19,7 +19,7 @@ file, and include the token Base64 encoded in a `secret_token` param or in the `Gitlab-Shared-Secret` header. NOTE: **Note:** -The internal api used by GitLab pages uses a different kind of +The internal API used by GitLab Pages uses a different kind of authentication. ## Git Authentication @@ -119,7 +119,7 @@ curl --request POST --header "Gitlab-Shared-Secret: <Base64 encoded token>" --da ## Authorized Keys Check This endpoint is called by the GitLab-shell authorized keys -check. Which is called by OpenSSH for [fast ssh key +check. Which is called by OpenSSH for [fast SSH key lookup](../administration/operations/fast_ssh_key_lookup.md). | Attribute | Type | Required | Description | @@ -287,12 +287,10 @@ Example response: } ``` -## Notify Post Receive [UNUSED] ? - ## PostReceive Called from Gitaly after a receiving a push. This triggers the -`PostReceive`-worker in sidekiq, processes the passed push options and +`PostReceive`-worker in Sidekiq, processes the passed push options and builds the response including messages that need to be displayed to the user. @@ -300,7 +298,7 @@ the user. |:----------|:-------|:---------|:------------| | `identifier` | string | yes | `user-[id]` or `key-[id]` Identifying the user performing the push | | `gl_repository` | string | yes | identifier of the repository being pushed to | -| `push_options` | [string] | no | array of push options | +| `push_options` | string array | no | array of push options | | `changes` | string | no | refs to be updated in the push in the format `oldrev newrev refname\n`. | ``` diff --git a/doc/development/issuable-like-models.md b/doc/development/issuable-like-models.md index ce19fd77496..3892f56156e 100644 --- a/doc/development/issuable-like-models.md +++ b/doc/development/issuable-like-models.md @@ -1,7 +1,9 @@ # Issuable-like Rails models utilities -GitLab Rails codebase contains several models that hold common functionality and behave similarly to an [Issue]. Other -examples of `Issuable`s are [Merge Requests] and [Epics]. +GitLab Rails codebase contains several models that hold common functionality and behave similarly to +[Issues](https://docs.gitlab.com/ee/user/project/issues/). Other examples of "issuables" +are [Merge Requests](https://docs.gitlab.com/ee/user/project/merge_requests/) and +[Epics](https://docs.gitlab.com/ee/user/group/epics/). This guide accumulates guidelines on working with such Rails models. @@ -13,7 +15,3 @@ There are max length constraints for the most important text fields for `Issuabl - `title_html`: 800 chars - `description`: 1 megabyte - `description_html`: 5 megabytes - -[Issue]: https://docs.gitlab.com/ee/user/project/issues -[Merge Requests]: https://docs.gitlab.com/ee/user/project/merge_requests -[Epics]: https://docs.gitlab.com/ee/user/group/epics diff --git a/doc/development/licensing.md b/doc/development/licensing.md index 40ff604c7c4..052c90c2b37 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -65,7 +65,6 @@ Libraries with the following licenses require legal approval for use: - [GNU GPL](https://choosealicense.com/licenses/gpl-3.0/) (version 1, [version 2][GPLv2], [version 3][GPLv3], or any future versions): GPL-licensed libraries cannot be linked to from non-GPL projects. - [GNU AGPLv3](https://choosealicense.com/licenses/agpl-3.0/): AGPL-licensed libraries cannot be linked to from non-GPL projects. - [Open Software License (OSL)][OSL]: is a copyleft license. In addition, the FSF [recommend against its use][OSL-GNU]. -- [Facebook BSD + PATENTS][Facebook]: is a 3-clause BSD license with a patent grant that has been deemed [Category X][x-list] by the Apache foundation. - [WTFPL][WTFPL]: is a public domain dedication [rejected by the OSI (3.2)][WTFPL-OSI]. Also has a strong language which is not in accordance with our diversity policy. ## GPL Cooperation Commitment @@ -124,7 +123,6 @@ Dependencies which are only used in development or test environment are exempt f [Org-Repo]: https://gitlab.com/gitlab-com/organization [UNLICENSE]: https://unlicense.org [OWFa1]: http://www.openwebfoundation.org/legal/the-owf-1-0-agreements/owfa-1-0 -[Facebook]: https://code.facebook.com/pages/850928938376556 [x-list]: https://www.apache.org/legal/resolved.html#category-x [Acceptable-Licenses]: #acceptable-licenses [Unacceptable-Licenses]: #unacceptable-licenses diff --git a/doc/development/logging.md b/doc/development/logging.md index b43f1029cc6..2eb140d3b7e 100644 --- a/doc/development/logging.md +++ b/doc/development/logging.md @@ -39,7 +39,7 @@ Note that currently on GitLab.com, any messages in `production.log` will NOT get indexed by Elasticsearch due to the sheer volume and noise. They do end up in Google Stackdriver, but it is still harder to search for logs there. See the [GitLab.com logging -documentation](https://gitlab.com/gitlab-com/runbooks/blob/master/howto/logging.md) +documentation](https://gitlab.com/gitlab-com/runbooks/blob/master/logging/doc/README.md) for more details. ## Use structured (JSON) logging @@ -127,6 +127,68 @@ importer progresses. Here's what to do: logger.info(message: "Import error", error_code: 1, error: "I/O failure") ``` +## Exception Handling + +It often happens that you catch the exception and want to track it. + +It should be noted that manual logging of exceptions is not allowed, as: + +1. Manual logged exceptions can leak confidential data, +1. Manual logged exception very often require to clean backtrace + which reduces the boilerplate, +1. Very often manually logged exception needs to be tracked to Sentry as well, +1. Manually logged exceptions does not use `correlation_id`, which makes hard + to pin them to request, user and context in which this exception was raised, +1. It is very likely that manually logged exceptions will end-up across + multiple files, which increases burden scraping all logging files. + +To avoid duplicating and having consistent behavior the `Gitlab::ErrorTracking` +provides helper methods to track exceptions: + +1. `Gitlab::ErrorTracking.track_and_raise_exception`: this method logs, + sends exception to Sentry (if configured) and re-raises the exception, +1. `Gitlab::ErrorTracking.track_exception`: this method only logs + and sends exception to Sentry (if configured), +1. `Gitlab::ErrorTracking.log_exception`: this method only logs the exception, + and DOES NOT send the exception to Sentry, +1. `Gitlab::ErrorTracking.track_and_raise_for_dev_exception`: this method logs, + sends exception to Sentry (if configured) and re-raises the exception + for development and test enviroments. + +It is advised to only use `Gitlab::ErrorTracking.track_and_raise_exception` +and `Gitlab::ErrorTracking.track_exception` as presented on below examples. + +Consider adding additional extra parameters to provide more context +for each tracked exception. + +### Example + +```ruby +class MyService < ::BaseService + def execute + project.perform_expensive_operation + + success + rescue => e + Gitlab::ErrorTracking.track_exception(e, project_id: project.id) + + error('Exception occurred') + end +end +``` + +```ruby +class MyService < ::BaseService + def execute + project.perform_expensive_operation + + success + rescue => e + Gitlab::ErrorTracking.track_and_raise_exception(e, project_id: project.id) + end +end +``` + ## Additional steps with new log files 1. Consider log retention settings. By default, Omnibus will rotate any diff --git a/doc/development/mass_insert.md b/doc/development/mass_insert.md new file mode 100644 index 00000000000..891ce0db87d --- /dev/null +++ b/doc/development/mass_insert.md @@ -0,0 +1,13 @@ +# Mass Inserting Rails Models + +Setting the environment variable [`MASS_INSERT=1`](rake_tasks.md#env-variables) +when running `rake setup` will create millions of records, but these records +aren't visible to the `root` user by default. + +To make any number of the mass-inserted projects visible to the `root` user, run +the following snippet in the rails console. + +```ruby +u = User.find(1) +Project.last(100).each { |p| p.set_create_timestamps && p.add_maintainer(u, current_user: u) } # Change 100 to whatever number of projects you need access to +``` diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md index 2e80e813a4b..ec50b1557d4 100644 --- a/doc/development/merge_request_performance_guidelines.md +++ b/doc/development/merge_request_performance_guidelines.md @@ -75,7 +75,7 @@ data set is for this feature to process, and what problems it might cause. If you would think about the following example that puts a strong emphasis of data set being processed. The problem is simple: you want to filter a list of files from -some git repository. Your feature requests a list of all files +some Git repository. Your feature requests a list of all files from the repository and perform search for the set of files. As an author you should in context of that problem consider the following: @@ -165,6 +165,54 @@ can quickly spiral out of control. There are some cases where this may be needed. If this is the case this should be clearly mentioned in the merge request description. +## Batch process + +**Summary:** Iterating a single process to external services (e.g. PostgreSQL, Redis, Object Storage, etc) +should be executed in a **batch-style** in order to reduce connection overheads. + +For fetching rows from various tables in a batch-style, please see [Eager Loading](#eager-loading) section. + +### Example: Delete multiple files from Object Storage + +When you delete multiple files from object storage (e.g. GCS), +executing a single REST API call multiple times is a quite expensive +process. Ideally, this should be done in a batch-style, for example, S3 provides +[batch deletion API](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html), +so it'd be a good idea to consider such an approach. + +The `FastDestroyAll` module might help this situation. It's a +small framework when you remove a bunch of database rows and its associated data +in a batch style. + +## Timeout + +**Summary:** You should set a reasonable timeout when the system invokes HTTP calls +to external services (e.g. Kubernetes), and it should be executed in Sidekiq, not +in Puma/Unicorn threads. + +Often, GitLab needs to communicate with an external service such as Kubernetes +clusters. In this case, it's hard to estimate when the external service finishes +the requested process, for example, if it's a user-owned cluster that is inactive for some reason, +GitLab might wait for the response forever ([Example](https://gitlab.com/gitlab-org/gitlab/issues/31475)). +This could result in Puma/Unicorn timeout and should be avoided at all cost. + +You should set a reasonable timeout, gracefully handle exceptions and surface the +errors in UI or logging internally. + +Using [`ReactiveCaching`](https://docs.gitlab.com/ee/development/utilities.html#reactivecaching) is one of the best solutions to fetch external data. + +## Keep database transaction minimal + +**Summary:** You should avoid accessing to external services (e.g. Gitaly) during database +transactions, otherwise it leads to severe contention problems +as an open transaction basically blocks the release of a Postgres backend connection. + +For keeping transaction as minimal as possible, please consider using `AfterCommitQueue` +module or `after_commit` AR hook. + +Here is [an example](https://gitlab.com/gitlab-org/gitlab/issues/36154#note_247228859) +that one request to Gitaly instance during transaction triggered a P1 issue. + ## Eager Loading **Summary:** always eager load associations when retrieving more than one row. diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 32c4313a1ed..6301ba778bc 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -517,9 +517,3 @@ _namespaces_ that have a `project_id`. The `path` column for these rows will be renamed to their previous value followed by an integer. For example: `users` would turn into `users0` - -### Moving migrations from EE to CE - -When migrations need to be moved from GitLab Enterprise Edition to GitLab Community Edition, -a migration file should be moved from `ee/db/{post_,}migrate` directory in the `gitlab` project to `db/{post_,}migrate` directory in the `gitlab-foss` project. This way -the schema number remains intact, there is no need to modify old migrations, and proper columns, tables or data are added in the Community Edition. diff --git a/doc/development/new_fe_guide/index.md b/doc/development/new_fe_guide/index.md index 227d03bd86f..152ddcdae64 100644 --- a/doc/development/new_fe_guide/index.md +++ b/doc/development/new_fe_guide/index.md @@ -15,10 +15,6 @@ Learn about all the dependencies that make up our frontend, including some of ou Learn about all the internal JavaScript modules that make up our frontend. -## [Style guides](style/index.md) - -Style guides to keep our code consistent. - ## [Tips](tips.md) Tips from our frontend team to develop more efficiently and effectively. diff --git a/doc/development/new_fe_guide/style/html.md b/doc/development/new_fe_guide/style/html.md index 1445da3f0e1..0b4fce13d90 100644 --- a/doc/development/new_fe_guide/style/html.md +++ b/doc/development/new_fe_guide/style/html.md @@ -1,53 +1,5 @@ -# HTML style guide +--- +redirect_to: '../../fe_guide/style/html.md' +--- -## Buttons - -### Button type - -Button tags requires a `type` attribute according to the [W3C HTML specification](https://www.w3.org/TR/2011/WD-html5-20110525/the-button-element.html#dom-button-type). - -```html -// bad -<button></button> - -// good -<button type="button"></button> -``` - -### Button role - -If an HTML element has an `onClick` handler but is not a button, it should have `role="button"`. This is [more accessible](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Roles/button_role). - -```html -// bad -<div onClick="doSomething"></div> - -// good -<div role="button" onClick="doSomething"></div> -``` - -## Links - -### Blank target - -Use `rel="noopener noreferrer"` whenever your links open in a new window, i.e. `target="_blank"`. This prevents a security vulnerability [documented by JitBit](https://www.jitbit.com/alexblog/256-targetblank---the-most-underestimated-vulnerability-ever/). - -```html -// bad -<a href="url" target="_blank"></a> - -// good -<a href="url" target="_blank" rel="noopener noreferrer"></a> -``` - -### Fake links - -**Do not use fake links.** Use a button tag if a link only invokes JavaScript click event handlers, which is more semantic. - -```html -// bad -<a class="js-do-something" href="#"></a> - -// good -<button class="js-do-something" type="button"></button> -``` +This document was moved to [another location](../../fe_guide/style/html.md). diff --git a/doc/development/new_fe_guide/style/index.md b/doc/development/new_fe_guide/style/index.md index f073dc56f1f..284862a2be9 100644 --- a/doc/development/new_fe_guide/style/index.md +++ b/doc/development/new_fe_guide/style/index.md @@ -1,15 +1,5 @@ -# Style guides +--- +redirect_to: '../../fe_guide/style/index.md' +--- -## [HTML style guide](html.md) - -## [SCSS style guide](scss.md) - -## [JavaScript style guide](javascript.md) - -## [Vue style guide](vue.md) - -## Tooling - -## [Prettier](prettier.md) - -Our code is automatically formatted with [Prettier](https://prettier.io) to follow our guidelines. +This document was moved to [another location](../../fe_guide/style/index.md). diff --git a/doc/development/new_fe_guide/style/javascript.md b/doc/development/new_fe_guide/style/javascript.md index d31edcb372d..003880c2592 100644 --- a/doc/development/new_fe_guide/style/javascript.md +++ b/doc/development/new_fe_guide/style/javascript.md @@ -1,195 +1,5 @@ -# JavaScript style guide +--- +redirect_to: '../../fe_guide/style/javascript.md' +--- -We use [Airbnb's JavaScript Style Guide](https://github.com/airbnb/javascript) and it's accompanying -linter to manage most of our JavaScript style guidelines. - -In addition to the style guidelines set by Airbnb, we also have a few specific rules -listed below. - -> **Tip:** -You can run eslint locally by running `yarn eslint` - -## Avoid forEach - -Avoid forEach when mutating data. Use `map`, `reduce` or `filter` instead of `forEach` -when mutating data. This will minimize mutations in functions, -which aligns with [Airbnb's style guide](https://github.com/airbnb/javascript#testing--for-real). - -```javascript -// bad -users.forEach((user, index) => { - user.id = index; -}); - -// good -const usersWithId = users.map((user, index) => { - return Object.assign({}, user, { id: index }); -}); -``` - -## Limit number of parameters - -If your function or method has more than 3 parameters, use an object as a parameter -instead. - -```javascript -// bad -function a(p1, p2, p3) { - // ... -}; - -// good -function a(p) { - // ... -}; -``` - -## Avoid side effects in constructors - -Avoid making asynchronous calls, API requests or DOM manipulations in the `constructor`. -Move them into separate functions instead. This will make tests easier to write and -code easier to maintain. - -```javascript -// bad -class myClass { - constructor(config) { - this.config = config; - axios.get(this.config.endpoint) - } -} - -// good -class myClass { - constructor(config) { - this.config = config; - } - - makeRequest() { - axios.get(this.config.endpoint) - } -} -const instance = new myClass(); -instance.makeRequest(); -``` - -## Avoid classes to handle DOM events - -If the only purpose of the class is to bind a DOM event and handle the callback, prefer -using a function. - -```javascript -// bad -class myClass { - constructor(config) { - this.config = config; - } - - init() { - document.addEventListener('click', () => {}); - } -} - -// good - -const myFunction = () => { - document.addEventListener('click', () => { - // handle callback here - }); -} -``` - -## Pass element container to constructor - -When your class manipulates the DOM, receive the element container as a parameter. -This is more maintainable and performant. - -```javascript -// bad -class a { - constructor() { - document.querySelector('.b'); - } -} - -// good -class a { - constructor(options) { - options.container.querySelector('.b'); - } -} -``` - -## Use ParseInt - -Use `ParseInt` when converting a numeric string into a number. - -```javascript -// bad -Number('10') - -// good -parseInt('10', 10); -``` - -## CSS Selectors - Use `js-` prefix - -If a CSS class is only being used in JavaScript as a reference to the element, prefix -the class name with `js-`. - -```html -// bad -<button class="add-user"></button> - -// good -<button class="js-add-user"></button> -``` - -## Absolute vs relative paths for modules - -Use relative paths if the module you are importing is less than two levels up. - -```javascript -// bad -import GitLabStyleGuide from '~/guides/GitLabStyleGuide'; - -// good -import GitLabStyleGuide from '../GitLabStyleGuide'; -``` - -If the module you are importing is two or more levels up, use an absolute path instead: - -```javascript -// bad -import GitLabStyleGuide from '../../../guides/GitLabStyleGuide'; - -// good -import GitLabStyleGuide from '~/GitLabStyleGuide'; -``` - -Additionally, **do not add to global namespace**. - -## Do not use `DOMContentLoaded` in non-page modules - -Imported modules should act the same each time they are loaded. `DOMContentLoaded` -events are only allowed on modules loaded in the `/pages/*` directory because those -are loaded dynamically with webpack. - -## Avoid XSS - -Do not use `innerHTML`, `append()` or `html()` to set content. It opens up too many -vulnerabilities. - -## Disabling ESLint in new files - -Do not disable ESLint when creating new files. Existing files may have existing rules -disabled due to legacy compatibility reasons but they are in the process of being refactored. - -Do not disable specific ESLint rules. Due to technical debt, you may disable the following -rules only if you are invoking/instantiating existing code modules. - -- [no-new](https://eslint.org/docs/rules/no-new) -- [class-method-use-this](https://eslint.org/docs/rules/class-methods-use-this) - -> Note: Disable these rules on a per line basis. This makes it easier to refactor -> in the future. E.g. use `eslint-disable-next-line` or `eslint-disable-line`. +This document was moved to [another location](../../fe_guide/style/javascript.md). diff --git a/doc/development/new_fe_guide/style/prettier.md b/doc/development/new_fe_guide/style/prettier.md index 17b209d419e..9a95aa96dff 100644 --- a/doc/development/new_fe_guide/style/prettier.md +++ b/doc/development/new_fe_guide/style/prettier.md @@ -1,98 +1,5 @@ -# Formatting with Prettier +--- +redirect_to: '../../fe_guide/tooling.md#formatting-with-prettier' +--- -Our code is automatically formatted with [Prettier](https://prettier.io) to follow our style guides. Prettier is taking care of formatting .js, .vue, and .scss files based on the standard prettier rules. You can find all settings for Prettier in `.prettierrc`. - -## Editor - -The easiest way to include prettier in your workflow is by setting up your preferred editor (all major editors are supported) accordingly. We suggest setting up prettier to run automatically when each file is saved. Find [here](https://prettier.io/docs/en/editors.html) the best way to set it up in your preferred editor. - -Please take care that you only let Prettier format the same file types as the global Yarn script does (.js, .vue, and .scss). In VSCode by example you can easily exclude file formats in your settings file: - -``` - "prettier.disableLanguages": [ - "json", - "markdown" - ], -``` - -## Yarn Script - -The following yarn scripts are available to do global formatting: - -``` -yarn prettier-staged-save -``` - -Updates all currently staged files (based on `git diff`) with Prettier and saves the needed changes. - -``` -yarn prettier-staged -``` - -Checks all currently staged files (based on `git diff`) with Prettier and log which files would need manual updating to the console. - -``` -yarn prettier-all -``` - -Checks all files with Prettier and logs which files need manual updating to the console. - -``` -yarn prettier-all-save -``` - -Formats all files in the repository with Prettier. (This should only be used to test global rule updates otherwise you would end up with huge MR's). - -The source of these Yarn scripts can be found in `/scripts/frontend/prettier.js`. - -### Scripts during Conversion period - -``` -node ./scripts/frontend/prettier.js check-all ./vendor/ -``` - -This will go over all files in a specific folder check it. - -``` -node ./scripts/frontend/prettier.js save-all ./vendor/ -``` - -This will go over all files in a specific folder and save it. - -## VSCode Settings - -### Select Prettier as default formatter - -To select Prettier as a formatter, add the following properties to your User or Workspace Settings: - -```javascript -{ - "[html]": { - "editor.defaultFormatter": "esbenp.prettier-vscode" - }, - "[javascript]": { - "editor.defaultFormatter": "esbenp.prettier-vscode" - }, - "[vue]": { - "editor.defaultFormatter": "esbenp.prettier-vscode" - } -} -``` - -### Format on Save - -To automatically format your files with Prettier, add the following properties to your User or Workspace Settings: - -```javascript -{ - "[html]": { - "editor.formatOnSave": true - }, - "[javascript]": { - "editor.formatOnSave": true - }, - "[vue]": { - "editor.formatOnSave": true - }, -} -``` +This document was moved to [another location](../../fe_guide/tooling.md#formatting-with-prettier). diff --git a/doc/development/new_fe_guide/style/scss.md b/doc/development/new_fe_guide/style/scss.md deleted file mode 100644 index 6f5e818d7db..00000000000 --- a/doc/development/new_fe_guide/style/scss.md +++ /dev/null @@ -1,3 +0,0 @@ -# SCSS style guide - -> TODO: Add content diff --git a/doc/development/new_fe_guide/style/vue.md b/doc/development/new_fe_guide/style/vue.md deleted file mode 100644 index fd9353e0d3f..00000000000 --- a/doc/development/new_fe_guide/style/vue.md +++ /dev/null @@ -1,3 +0,0 @@ -# Vue style guide - -> TODO: Add content diff --git a/doc/development/new_fe_guide/tips.md b/doc/development/new_fe_guide/tips.md index 879b54bd93c..657da1be1a7 100644 --- a/doc/development/new_fe_guide/tips.md +++ b/doc/development/new_fe_guide/tips.md @@ -23,3 +23,7 @@ Your feature flag can now be: - [Deleting a feature flag](../../api/features.md#delete-a-feature) - [Manage feature flags](../feature_flags/process.md) - [Feature flags API](../../api/features.md) + +## Running tests locally + +This can be done as outlined by the [frontend testing guide](../testing_guide/frontend_testing.md#running-frontend-tests). diff --git a/doc/development/packages.md b/doc/development/packages.md index 6d4a9ea9f41..7ae3cd53e66 100644 --- a/doc/development/packages.md +++ b/doc/development/packages.md @@ -5,7 +5,7 @@ This document will guide you through adding another [package management system]( See already supported package types in [Packages documentation](../administration/packages/index.md) Since GitLab packages' UI is pretty generic, it is possible to add basic new -package system support by solely backend changes. This guide is superficial and does +package system support with solely backend changes. This guide is superficial and does not cover the way the code should be written. However, you can find a good example by looking at existing merge requests with Maven and NPM support: @@ -14,6 +14,28 @@ by looking at existing merge requests with Maven and NPM support: - [Maven repository](https://gitlab.com/gitlab-org/gitlab/merge_requests/6607). - [Instance level endpoint for Maven repository](https://gitlab.com/gitlab-org/gitlab/merge_requests/8757) +## Suggested contributions + +The goal of the Package group is to build a set of features that, within three years, will allow ninety percent of our customers to store all of their packages in GitLab. To do that we need to ensure that we support the below package manager formats. + +| Format | Use case | +| ------ | ------ | +| [Bower](https://gitlab.com/gitlab-org/gitlab/issues/36888) | Boost your front end development by hosting your own Bower components. | +| [Chef](https://gitlab.com/gitlab-org/gitlab/issues/36889) | Configuration management with Chef using all the benefits of a repository manager. | +| [CocoaPods](https://gitlab.com/gitlab-org/gitlab/issues/36890) | Speed up development with Xcode and CocoaPods. | +| [Conda](https://gitlab.com/gitlab-org/gitlab/issues/36891) | Secure and private local Conda repositories. | +| [CRAN](https://gitlab.com/gitlab-org/gitlab/issues/36892) | Deploy and resolve CRAN packages for the R language. | +| [Debian](https://gitlab.com/gitlab-org/gitlab/issues/5835) | Host and provision Debian packages. | +| [Go](https://gitlab.com/gitlab-org/gitlab/issues/9773) | Resolve Go dependencies from and publish your Go packages to GitLab. | +| [Opkg](https://gitlab.com/gitlab-org/gitlab/issues/36894) | Optimize your work with OpenWrt using Opkg repositories. | +| [P2](https://gitlab.com/gitlab-org/gitlab/issues/36895) | Host all your Eclipse plugins in your own GitLab P2 repository. | +| [Puppet](https://gitlab.com/gitlab-org/gitlab/issues/36897) | Configuration management meets repository management with Puppet repositories. | +| [PyPi](https://gitlab.com/gitlab-org/gitlab/issues/10483) | Host PyPi distributions. | +| [RPM](https://gitlab.com/gitlab-org/gitlab/issues/5932) | Distribute RPMs directly from GitLab. | +| [RubyGems](https://gitlab.com/gitlab-org/gitlab/issues/803) | Use GitLab to host your own gems. | +| [SBT](https://gitlab.com/gitlab-org/gitlab/issues/36898) | Resolve dependencies from and deploy build output to SBT repositories when running SBT builds. | +| [Vagrant](https://gitlab.com/gitlab-org/gitlab/issues/36899) | Securely host your Vagrant boxes in local repositories. | + ## General information The existing database model requires the following: @@ -45,6 +67,29 @@ PUT https://gitlab.com/api/v4/projects/<your_project_id>/packages/npm/ Group-level and instance-level endpoints are good to have but are optional. +### Remote hierarchy + +Packages are scoped within various levels of access, which is generally configured by setting your remote. A +remote endpoint may be set at the project level, meaning when installing packages, only packages belonging to that +project will be visible. Alternatively, a group-level endpoint may be used to allow visibility to all packages +within a given group. Lastly, an instance-level endpoint can be used to allow visibility to all packages within an +entire GitLab instance. + +Using group and project level endpoints will allow for more flexibility in package naming, however, more remotes +will have to be managed. Using instance level endpoints requires [stricter naming conventions](#naming-conventions). + +The current state of existing package registries availability is: + +| Repository Type | Project Level | Group Level | Instance Level | +|-----------------|---------------|-------------|----------------| +| Maven | Yes | Yes | Yes | +| Conan | No - [open issue](https://gitlab.com/gitlab-org/gitlab/issues/11679) | No - [open issue](https://gitlab.com/gitlab-org/gitlab/issues/11679) | Yes | +| NPM | No - [open issue](https://gitlab.com/gitlab-org/gitlab/issues/36853) | Yes | No - [open issue](https://gitlab.com/gitlab-org/gitlab/issues/36853) | + +NOTE: **Note:** NPM is currently a hybrid of the instance level and group level. +It is using the top-level group or namespace as the defining portion of the name +(for example, `@my-group-name/my-package-name`). + ## Naming conventions To avoid name conflict for instance-level endpoints you will need to define a package naming convention @@ -61,14 +106,14 @@ model for that package type. ## File uploads -File uploads should be handled by GitLab workhorse using object accelerated uploads. What this means is that +File uploads should be handled by GitLab Workhorse using object accelerated uploads. What this means is that the workhorse proxy that checks all incoming requests to GitLab will intercept the upload request, upload the file, and forward a request to the main GitLab codebase only containing the metadata and file location rather than the file itself. An overview of this process can be found in the [development documentation](uploads.md#workhorse-object-storage-acceleration). In terms of code, this means a route will need to be added to the -[gitlab-workhorse project](https://gitlab.com/gitlab-org/gitlab-workhorse) for each level of remote being added +[GitLab Workhorse project](https://gitlab.com/gitlab-org/gitlab-workhorse) for each level of remote being added (instance, group, project). [This merge request](https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/412/diffs) demonstrates adding an instance-level endpoint for Conan to workhorse. You can also see the Maven project level endpoint implemented in the same file. @@ -135,7 +180,7 @@ process. These changes represent all that is needed to deliver a minimally usable package management system. -1. Empty file structure (api file, base service for this package) +1. Empty file structure (API file, base service for this package) 1. Authentication system for 'logging in' to the package manager 1. Identify metadata and create applicable tables 1. Workhorse route for [object storage accelerated uploads](uploads.md#workhorse-object-storage-acceleration) diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index 764bd68000d..99f92e6f39f 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -1,6 +1,6 @@ # Pipelines for the GitLab project -Pipelines for `gitlab-org/gitlab` and `gitlab-org/gitlab-foss` (as well as the +Pipelines for <https://gitlab.com/gitlab-org/gitlab> and <https://gitlab.com/gitlab-org/gitlab-foss> (as well as the `dev` instance's mirrors) are configured in the usual [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml) which itself includes files under @@ -15,33 +15,36 @@ as much as possible. The current stages are: -- `sync`: This stage is used to synchronize changes from gitlab-org/gitlab to - gitlab-org/gitlab-foss. +- `sync`: This stage is used to synchronize changes from <https://gitlab.com/gitlab-org/gitlab> to + <https://gitlab.com/gitlab-org/gitlab-foss>. - `prepare`: This stage includes jobs that prepare artifacts that are needed by jobs in subsequent stages. - `quick-test`: This stage includes test jobs that should run first and fail the pipeline early (currently used to run Geo tests when the branch name starts with `geo-`, `geo/`, or ends with `-geo`). - `test`: This stage includes most of the tests, DB/migration jobs, and static analysis jobs. +- `post-test`: This stage includes jobs that build reports or gather data from + the `test` stage's jobs (e.g. coverage, Knapsack metadata etc.). - `review-prepare`: This stage includes a job that build the CNG images that are later used by the (Helm) Review App deployment (see [Review Apps](testing_guide/review_apps.md) for details). - `review`: This stage includes jobs that deploy the GitLab and Docs Review Apps. - `qa`: This stage includes jobs that perform QA tasks against the Review App that is deployed in the previous stage. -- `post-test`: This stage includes jobs that build reports or gather data from - the previous stages' jobs (e.g. coverage, Knapsack metadata etc.). +- `post-qa`: This stage includes jobs that build reports or gather data from + the `qa` stage's jobs (e.g. Review App performance report). +- `notification`: This stage includes jobs that sends notifications about pipeline status. - `pages`: This stage includes a job that deploys the various reports as - GitLab pages (e.g. <https://gitlab-org.gitlab.io/gitlab/coverage-ruby/>, + GitLab Pages (e.g. <https://gitlab-org.gitlab.io/gitlab/coverage-ruby/>, <https://gitlab-org.gitlab.io/gitlab/coverage-javascript/>, <https://gitlab-org.gitlab.io/gitlab/webpack-report/>). ## Default image The default image is currently -`registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.3-golang-1.11-git-2.22-chrome-73.0-node-12.x-yarn-1.16-postgresql-9.6-graphicsmagick-1.3.33`. +`registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.3-golang-1.12-git-2.24-lfs-2.9-chrome-73.0-node-12.x-yarn-1.16-postgresql-9.6-graphicsmagick-1.3.33`. -It includes Ruby 2.6.3, Go 1.11, Git 2.22, Chrome 73, Node 12, Yarn 1.16, +It includes Ruby 2.6.3, Go 1.12, Git 2.24, Git LFS 2.9, Chrome 73, Node 12, Yarn 1.16, PostgreSQL 9.6, and Graphics Magick 1.3.33. The images used in our pipelines are configured in the @@ -191,8 +194,8 @@ subgraph "`review-prepare` stage" end subgraph "`review` stage" - G --> |needs| E; - G2 --> |needs| E; + G + G2 end subgraph "`qa` stage" @@ -209,6 +212,11 @@ subgraph "`qa` stage" dast -.-> |needs and depends on| G; end +subgraph "`notification` stage" + NOTIFICATION1["schedule:package-and-qa:notify-success<br>(on_success)"] -.-> |needs| P; + NOTIFICATION2["schedule:package-and-qa:notify-failure<br>(on_failure)"] -.-> |needs| P; + end + subgraph "`post-test` stage" M end diff --git a/doc/development/prometheus_metrics.md b/doc/development/prometheus_metrics.md index b479c053862..74dbcddc5db 100644 --- a/doc/development/prometheus_metrics.md +++ b/doc/development/prometheus_metrics.md @@ -22,7 +22,9 @@ The requirement for adding a new metric is to make each query to have an unique ### Update existing metrics -After you add or change existing _common_ metric you have to create a new database migration that will query and update all existing metrics. +After you add or change an existing common metric, you must [re-run the import script](../administration/raketasks/maintenance.md#import-common-metrics) that will query and update all existing metrics. + +Or, you can create a database migration: NOTE: **Note:** If a query metric (which is identified by `id:`) is removed it will not be removed from database by default. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 369806d462b..ff978ee2899 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -93,10 +93,10 @@ There are a few environment flags you can pass to change how projects are seeded In order to run the test you can use the following commands: - `bin/rake spec` to run the rspec suite -- `bin/rake spec:unit` to run the only the unit tests -- `bin/rake spec:integration` to run the only the integration tests -- `bin/rake spec:system` to run the only the system tests -- `bin/rake karma` to run the karma test suite +- `bin/rake spec:unit` to run only the unit tests +- `bin/rake spec:integration` to run only the integration tests +- `bin/rake spec:system` to run only the system tests +- `bin/rake karma` to run the Karma test suite Note: `bin/rake spec` takes significant time to pass. Instead of running full test suite locally you can save a lot of time by running @@ -188,11 +188,10 @@ task, then check the dimensions of the new spritesheet and update the ## Updating project templates Starting a project from a template needs this project to be exported. On a -up to date master branch with run: +up to date master branch run: ``` -gdk run -# In a new terminal window +gdk start bundle exec rake gitlab:update_project_templates git checkout -b update-project-templates git add vendor/project_templates diff --git a/doc/development/shell_scripting_guide/index.md b/doc/development/shell_scripting_guide/index.md index 60678497bb2..e0895a088ab 100644 --- a/doc/development/shell_scripting_guide/index.md +++ b/doc/development/shell_scripting_guide/index.md @@ -60,7 +60,7 @@ All projects with shell scripts should use this GitLab CI/CD job: ```yaml shell check: - image: koalaman/shellcheck-alpine + image: koalaman/shellcheck-alpine:stable stage: test before_script: - shellcheck --version diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index e433691c1ed..77663b0bb29 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -211,7 +211,7 @@ end We use the following approach to determine whether a worker is CPU-bound: -- In the sidekiq structured JSON logs, aggregate the worker `duration` and +- In the Sidekiq structured JSON logs, aggregate the worker `duration` and `cpu_s` fields. - `duration` refers to the total job execution duration, in seconds - `cpu_s` is derived from the diff --git a/doc/development/sql.md b/doc/development/sql.md index 8a8204ffe87..84ad11effc5 100644 --- a/doc/development/sql.md +++ b/doc/development/sql.md @@ -108,21 +108,114 @@ class AddUsersLowerUsernameEmailIndexes < ActiveRecord::Migration[4.2] disable_ddl_transaction! def up - return unless Gitlab::Database.postgresql? - execute 'CREATE INDEX CONCURRENTLY index_on_users_lower_username ON users (LOWER(username));' execute 'CREATE INDEX CONCURRENTLY index_on_users_lower_email ON users (LOWER(email));' end def down - return unless Gitlab::Database.postgresql? - remove_index :users, :index_on_users_lower_username remove_index :users, :index_on_users_lower_email end end ``` +## Reliably referencing database columns + +ActiveRecord by default returns all columns from the queried database table. In some cases the returned rows might need to be customized, for example: + +- Specify only a few columns to reduce the amount of data returned from the database. +- Include columns from `JOIN` relations. +- Perform calculations (`SUM`, `COUNT`). + +In this example we specify the columns, but not their tables: + +- `path` from the `projects` table +- `user_id` from the `merge_requests` table + +The query: + +```ruby +# bad, avoid +Project.select("path, user_id").joins(:merge_requests) # SELECT path, user_id FROM "projects" ... +``` + +Later on, a new feature adds an extra column to the `projects` table: `user_id`. During deployment there might be a short time window where the database migration is already executed, but the new version of the application code is not deployed yet. When the query mentioned above executes during this period, the query will fail with the following error message: `PG::AmbiguousColumn: ERROR: column reference "user_id" is ambiguous` + +The problem is caused by the way the attributes are selected from the database. The `user_id` column is present in both the `users` and `merge_requests` tables. The query planner cannot decide which table to use when looking up the `user_id` column. + +When writing a customized `SELECT` statement, it's better to **explicitly specify the columns with the table name**. + +### Good (prefer) + +```ruby +Project.select(:path, 'merge_requests.user_id').joins(:merge_requests) + +# SELECT "projects"."path", merge_requests.user_id as user_id FROM "projects" ... +``` + +```ruby +Project.select(:path, :'merge_requests.user_id').joins(:merge_requests) + +# SELECT "projects"."path", "merge_requests"."id" as user_id FROM "projects" ... +``` + +Example using Arel (`arel_table`): + +```ruby +Project.select(:path, MergeRequest.arel_table[:user_id]).joins(:merge_requests) + +# SELECT "projects"."path", "merge_requests"."user_id" FROM "projects" ... +``` + +When writing raw SQL query: + +```sql +SELECT projects.path, merge_requests.user_id FROM "projects"... +``` + +When the raw SQL query is parameterized (needs escaping): + +```ruby +include ActiveRecord::ConnectionAdapters::Quoting + +""" +SELECT + #{quote_table_name('projects')}.#{quote_column_name('path')}, + #{quote_table_name('merge_requests')}.#{quote_column_name('user_id')} +FROM ... +""" +``` + +### Bad (avoid) + +```ruby +Project.select('id, path, user_id').joins(:merge_requests).to_sql + +# SELECT id, path, user_id FROM "projects" ... +``` + +```ruby +Project.select("path", "user_id").joins(:merge_requests) +# SELECT "projects"."path", "user_id" FROM "projects" ... + +# or + +Project.select(:path, :user_id).joins(:merge_requests) +# SELECT "projects"."path", "user_id" FROM "projects" ... +``` + +When a column list is given, ActiveRecord tries to match the arguments against the columns defined in the `projects` table and prepend the table name automatically. In this case, the `id` column is not going to be a problem, but the `user_id` column could return unexpected data: + +```ruby +Project.select(:id, :user_id).joins(:merge_requests) + +# Before deployment (user_id is taken from the merge_requests table): +# SELECT "projects"."id", "user_id" FROM "projects" ... + +# After deployment (user_id is taken from the projects table): +# SELECT "projects"."id", "projects"."user_id" FROM "projects" ... +``` + ## Plucking IDs This can't be stressed enough: **never** use ActiveRecord's `pluck` to pluck a diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index fe3989474e6..356e3f7a227 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -133,6 +133,8 @@ CHROME_HEADLESS=0 bundle exec rspec some_spec.rb ``` The test will go by quickly, but this will give you an idea of what's happening. +Using `live_debug` with `CHROME_HEADLESS=0` pauses the open browser, and does not +open the page again. This can be used to debug and inspect elements. You can also add `byebug` or `binding.pry` to pause execution and [step through](../pry_debugging.md#stepping) the test. diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md index e2a0d267ba1..fc00fcea67e 100644 --- a/doc/development/testing_guide/end_to_end/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -54,9 +54,9 @@ In summary: - **Do**: Split tests across separate files, unless the tests share expensive setup. - **Don't**: Put new tests in an existing file without considering the impact on parallelization. -## Limit the use of `before(:all)` hook +## Limit the use of `before(:all)` and `after` hooks -Limit the use of `before(:all)` to perform setup tasks with only API calls, non UI operations +Limit the use of `before(:all)` hook to perform setup tasks with only API calls, non UI operations or basic UI operations such as login. We use [`capybara-screenshot`](https://github.com/mattheworiordan/capybara-screenshot) library to automatically save screenshots on failures. @@ -66,6 +66,10 @@ This library [saves the screenshots in the RSpec's `after` hook](https://github. Given this fact, we should limit the use of `before(:all)` to only those operations where a screenshot is not necessary in case of failure and QA logs would be enough for debugging. +Similarly, the `after` hook should only be used for non-UI operations. Any UI operations in `after` hook in a test file +would execute before the `after` hook that takes the screenshot. This would result in moving the UI status away from the +point of failure and so the screenshot would not be captured at the right moment. + ## Ensure tests do not leave the browser logged in All QA tests expect to be able to log in at the start of the test. @@ -74,7 +78,7 @@ That's not possible if a test leaves the browser logged in when it finishes. Nor For an example see: <https://gitlab.com/gitlab-org/gitlab/issues/34736> -Ideally, any actions peformed in an `after(:context)` (or [`before(:context)`](#limit-the-use-of-beforeall-hook)) block would be performed via the API. But if it's necessary to do so via the UI (e.g., if API functionality doesn't exist), make sure to log out at the end of the block. +Ideally, any actions peformed in an `after(:context)` (or [`before(:context)`](#limit-the-use-of-beforeall-and-after-hooks)) block would be performed via the API. But if it's necessary to do so via the UI (e.g., if API functionality doesn't exist), make sure to log out at the end of the block. ```ruby after(:all) do diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 2457d8ada5a..fb820ac22a2 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -26,7 +26,17 @@ If you don't exactly understand what we mean by **not everything needs to happen At GitLab we respect the [test pyramid](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/testing_guide/testing_levels.md), and so, we recommend you check the code coverage of a specific feature before writing end-to-end tests, for both [CE](https://gitlab-org.gitlab.io/gitlab-foss/coverage-ruby/#_AllFiles) and [EE](https://gitlab-org.gitlab.io/gitlab/coverage-ruby/#_AllFiles) projects. -Sometimes you may notice that there is already good coverage in other test levels, and we can stay confident that if we break a feature, we will still have quick feedback about it, even without having end-to-end tests. +Sometimes you may notice that there is already good coverage in lower test levels, and we can stay confident that if we break a feature, we will still have quick feedback about it, even without having end-to-end tests. + +> For analyzing the code coverage, you will also need to understand which application files implement specific functionalities. + +#### Some other guidelines are as follows + +- Take a look at the [How to test at the correct level?](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/testing_guide/testing_levels.md#how-to-test-at-the-correct-level) section of the [Testing levels](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/development/testing_guide/testing_levels.md) document + +- Look into the frequency in which such a feature is changed (_Stable features that don't change very often might not be worth covering with end-to-end tests if they're already covered in lower levels_) + +- Finally, discuss with the developer(s) involved in developing the feature and the tests themselves, to get their feeling If after this analysis you still think that end-to-end tests are needed, keep reading. @@ -581,7 +591,7 @@ filter_output = search_field_tag search_id, nil, class: "dropdown-input-field", > `data-qa-*` data attributes and CSS classes starting with `qa-` are used solely for the purpose of QA and testing. > By defining these, we add **testability** to the application. > -> When defining a data attribute like: `qa_selector: 'labels_block'`, it should match the element definition: `element :labels_block`. We use a [sanity test](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/qa/qa/page#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective selectors in the specified views. +> When defining a data attribute like: `qa_selector: 'labels_block'`, it should match the element definition: `element :labels_block`. We use a [sanity test](https://gitlab.com/gitlab-org/gitlab-foss/tree/master/doc/development/testing_guide/end_to_end/page_objects.md#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective selectors in the specified views. #### Updates in the `QA::Page::Base` class diff --git a/doc/development/testing_guide/end_to_end/resources.md b/doc/development/testing_guide/end_to_end/resources.md index 1e32db4f633..b8a093c54c6 100644 --- a/doc/development/testing_guide/end_to_end/resources.md +++ b/doc/development/testing_guide/end_to_end/resources.md @@ -44,6 +44,9 @@ create the resource via the public GitLab API: - `#api_post_path`: The `POST` path to create a new resource. - `#api_post_body`: The `POST` body (as a Ruby hash) to create a new resource. +> Be aware that many API resources are [paginated](../../../api/README.md#pagination). +> If you don't find the results you expect, check if there is more that one page of results. + Let's take the `Shirt` resource class, and add these three API methods: ```ruby diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 236f175cee5..8934f0d4b65 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -26,7 +26,7 @@ If you need to update an existing Karma test file (found in `spec/javascripts`), need to migrate the whole spec to Jest. Simply updating the Karma spec to test your change is fine. It is probably more appropriate to migrate to Jest in a separate merge request. -If you need to create a new test file, we strongly recommend creating one in Jest. This will +If you create a new test file, it needs to be created in Jest. This will help support our migration and we think you'll love using Jest. As always, please use discretion. Jest solves a lot of issues we experienced in Karma and @@ -552,6 +552,7 @@ For running the frontend tests, you need the following commands: - `rake frontend:fixtures` (re-)generates [fixtures](#frontend-test-fixtures). - `yarn test` executes the tests. +- `yarn jest` executes only the Jest tests. As long as the fixtures don't change, `yarn test` is sufficient (and saves you some time). @@ -593,6 +594,24 @@ glob otherwise your shell may split it into multiple arguments: yarn karma -f 'spec/javascripts/ide/**/file_spec.js' ``` +It is also possible to target individual Jest / RSpec tests: + +```bash +# Run specific jest file +yarn jest ./path/to/local_spec.js +# Run specific jest folder +yarn jest ./path/to/folder/ +# Run all jest files which path contain term +yarn jest term +``` + +```bash +# Run specific rspec file +rspec ./path/to/local_spec.rb +# Run specific block within rspec file +rspec ./path/to/local_spec.rb:15 +``` + ## Frontend test fixtures Code that is added to HAML templates (in `app/views/`) or makes Ajax requests to the backend has tests that require HTML or JSON from the backend. diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index ecfcbc731e1..b06ea41d91f 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -129,6 +129,10 @@ two node pools: ### Helm/Tiller +The Helm/Tiller version used is defined in the +[`registry.gitlab.com/gitlab-org/gitlab-build-images:gitlab-charts-build-base` image](https://gitlab.com/gitlab-org/gitlab-build-images/blob/master/Dockerfile.gitlab-charts-build-base#L4) +used by the `review-deploy` and `review-stop` jobs. + The `tiller` deployment (the Helm server) is deployed to a dedicated node pool that has the `app=helm` label and a specific [taint](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) @@ -182,22 +186,89 @@ secure note named `gitlab-{ce,ee} Review App's root password`. `review-qa-raise-e-12chm0-migrations.1-nqwtx`. 1. Click on the `Container logs` link. -### Diagnosing unhealthy review-app releases +## Diagnosing unhealthy Review App releases + +If [Review App Stability](https://app.periscopedata.com/app/gitlab/496118/Engineering-Productivity-Sandbox?widget=6690556&udv=785399) +dips this may be a signal that the `review-apps-ce/ee` cluster is unhealthy. +Leading indicators may be healthcheck failures leading to restarts or majority failure for Review App deployments. + +The [Review Apps Overview dashboard](https://app.google.stackdriver.com/dashboards/6798952013815386466?project=gitlab-review-apps&timeDomain=1d) +aids in identifying load spikes on the cluster, and if nodes are problematic or the entire cluster is trending towards unhealthy. + +### Node count is always increasing (i.e. never stabilizing or decreasing) + +**Potential cause:** + +That could be a sign that the [`schedule:review-cleanup`][gitlab-ci-yml] job is +failing to cleanup stale Review Apps and Kubernetes resources. + +**Where to look for further debugging:** + +Look at the latest `schedule:review-cleanup` job log, and identify look for any +unexpected failure. + +### p99 CPU utilization is at 100% for most of the nodes and/or many components + +**Potential cause:** + +This could be a sign that Helm is failing to deploy Review Apps. When Helm has a +lot of `FAILED` releases, it seems that the CPU utilization is increasing, probably +due to Helm or Kubernetes trying to recreate the components. + +**Where to look for further debugging:** + +Look at a recent `review-deploy` job log, and at the Tiller logs. + +**Useful commands:** + +```shell +# Identify if node spikes are common or load on specific nodes which may get rebalanced by the Kubernetes scheduler +› kubectl top nodes | sort --key 3 --numeric + +# Identify pods under heavy CPU load +› kubectl top pods | sort --key 2 --numeric +``` + +### The `logging/user/events/FailedMount` chart is going up + +**Potential cause:** + +This could be a sign that there are too many stale secrets and/or config maps. + +**Where to look for further debugging:** + +Look at [the list of Configurations](https://console.cloud.google.com/kubernetes/config?project=gitlab-review-apps) +or `kubectl get secret,cm --sort-by='{.metadata.creationTimestamp}' | grep 'review-'`. + +Any secrets or config maps older than 5 days are suspect and should be deleted. + +**Useful commands:** + +``` +# List secrets and config maps ordered by created date +› kubectl get secret,cm --sort-by='{.metadata.creationTimestamp}' | grep 'review-' + +# Delete all secrets that are 5 to 9 days old +› kubectl get secret --sort-by='{.metadata.creationTimestamp}' | grep '^review-' | grep '[5-9]d$' | cut -d' ' -f1 | xargs kubectl delete secret + +# Delete all secrets that are 10 to 99 days old +› kubectl get secret --sort-by='{.metadata.creationTimestamp}' | grep '^review-' | grep '[1-9][0-9]d$' | cut -d' ' -f1 | xargs kubectl delete secret + +# Delete all config maps that are 5 to 9 days old +› kubectl get cm --sort-by='{.metadata.creationTimestamp}' | grep 'review-' | grep -v 'dns-gitlab-review-app' | grep '[5-9]d$' | cut -d' ' -f1 | xargs kubectl delete cm + +# Delete all config maps that are 10 to 99 days old +› kubectl get cm --sort-by='{.metadata.creationTimestamp}' | grep 'review-' | grep -v 'dns-gitlab-review-app' | grep '[1-9][0-9]d$' | cut -d' ' -f1 | xargs kubectl delete cm +``` -If [Review App Stability](https://gitlab.com/gitlab-org/quality/team-tasks/issues/93) dips this may be a signal -that the `review-apps-ce/ee` cluster is unhealthy. Leading indicators may be healthcheck failures leading to restarts or majority failure for Review App deployments. +### Using K9s -The following items may help diagnose this: +[K9s] is a powerful command line dashboard which allows you to filter by labels. This can help identify trends with apps exceeding the [review-app resource requests](https://gitlab.com/gitlab-org/gitlab/blob/master/scripts/review_apps/base-config.yaml). Kubernetes will schedule pods to nodes based on resource requests and allow for CPU usage up to the limits. -- [Review Apps Health dashboard](https://app.google.stackdriver.com/dashboards/6798952013815386466?project=gitlab-review-apps&timeDomain=1d) - - Aids in identifying load spikes on the cluster, and if nodes are problematic or the entire cluster is trending towards unhealthy. -- `kubectl top nodes | sort --key 3 --numeric` - can identify if node spikes are common or load on specific nodes which may get rebalanced by the Kubernetes scheduler. -- `kubectl top pods | sort --key 2 --numeric` - -- [K9s] - K9s is a powerful command line dashboard which allows you to filter by labels. This can help identify trends with apps exceeding the [review-app resource requests](https://gitlab.com/gitlab-org/gitlab/blob/master/scripts/review_apps/base-config.yaml). Kubernetes will schedule pods to nodes based on resource requests and allow for CPU usage up to the limits. - - In K9s you can sort or add filters by typing the `/` character - - `-lrelease=<review-app-slug>` - filters down to all pods for a release. This aids in determining what is having issues in a single deployment - - `-lapp=<app>` - filters down to all pods for a specific app. This aids in determining resource usage by app. - - You can scroll to a Kubernetes resource and hit `d`(describe), `s`(shell), `l`(logs) for a deeper inspection +- In K9s you can sort or add filters by typing the `/` character + - `-lrelease=<review-app-slug>` - filters down to all pods for a release. This aids in determining what is having issues in a single deployment + - `-lapp=<app>` - filters down to all pods for a specific app. This aids in determining resource usage by app. +- You can scroll to a Kubernetes resource and hit `d`(describe), `s`(shell), `l`(logs) for a deeper inspection  diff --git a/doc/development/testing_guide/testing_migrations_guide.md b/doc/development/testing_guide/testing_migrations_guide.md index b28d17a4b55..3fef13afa9c 100644 --- a/doc/development/testing_guide/testing_migrations_guide.md +++ b/doc/development/testing_guide/testing_migrations_guide.md @@ -104,9 +104,9 @@ end ### Example database migration test This spec tests the -[`db/post_migrate/20170526185842_migrate_pipeline_stages.rb`](https://gitlab.com/gitlab-org/gitlab/blob/v11.6.5/db/post_migrate/20170526185842_migrate_pipeline_stages.rb) +[`db/post_migrate/20170526185842_migrate_pipeline_stages.rb`](https://gitlab.com/gitlab-org/gitlab-foss/blob/v11.6.5/db/post_migrate/20170526185842_migrate_pipeline_stages.rb) migration. You can find the complete spec in -[`spec/migrations/migrate_pipeline_stages_spec.rb`](https://gitlab.com/gitlab-org/gitlab/blob/v11.6.5/spec/migrations/migrate_pipeline_stages_spec.rb). +[`spec/migrations/migrate_pipeline_stages_spec.rb`](https://gitlab.com/gitlab-org/gitlab-foss/blob/v11.6.5/spec/migrations/migrate_pipeline_stages_spec.rb). ```ruby require 'spec_helper' @@ -171,9 +171,9 @@ end ### Example background migration test This spec tests the -[`lib/gitlab/background_migration/archive_legacy_traces.rb`](https://gitlab.com/gitlab-org/gitlab/blob/v11.6.5/lib/gitlab/background_migration/archive_legacy_traces.rb) +[`lib/gitlab/background_migration/archive_legacy_traces.rb`](https://gitlab.com/gitlab-org/gitlab-foss/blob/v11.6.5/lib/gitlab/background_migration/archive_legacy_traces.rb) background migration. You can find the complete spec on -[`spec/lib/gitlab/background_migration/archive_legacy_traces_spec.rb`](https://gitlab.com/gitlab-org/gitlab/blob/v11.6.5/spec/lib/gitlab/background_migration/archive_legacy_traces_spec.rb) +[`spec/lib/gitlab/background_migration/archive_legacy_traces_spec.rb`](https://gitlab.com/gitlab-org/gitlab-foss/blob/v11.6.5/spec/lib/gitlab/background_migration/archive_legacy_traces_spec.rb) ```ruby require 'spec_helper' diff --git a/doc/development/utilities.md b/doc/development/utilities.md index 25869a0d2b5..68851f4d550 100644 --- a/doc/development/utilities.md +++ b/doc/development/utilities.md @@ -53,7 +53,7 @@ Refer to <https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/utils/over - This utility can help you check if one method would override another or not. It is the same concept as Java's `@Override` annotation - or Scala's `override` keyword. However, you should only do this check when + or Scala's `override` keyword. However, we only run this check when `ENV['STATIC_VERIFICATION']` is set to avoid production runtime overhead. This is useful for checking: @@ -94,6 +94,15 @@ Refer to <https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/utils/over end ``` + Note that the check will only happen when either: + + - The overriding method is defined in a class, or: + - The overriding method is defined in a module, and it's prepended to + a class or a module. + + Because only a class or prepended module can actually override a method. + Including or extending a module into another cannot override anything. + ## `StrongMemoize` Refer to <https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/utils/strong_memoize.rb>: diff --git a/doc/development/verifying_database_capabilities.md b/doc/development/verifying_database_capabilities.md index 6b4995aebe2..1413c782c5d 100644 --- a/doc/development/verifying_database_capabilities.md +++ b/doc/development/verifying_database_capabilities.md @@ -6,22 +6,16 @@ necessary to add database (version) specific behaviour. To facilitate this we have the following methods that you can use: -- `Gitlab::Database.postgresql?`: returns `true` if PostgreSQL is being used. - You can normally just assume this is the case. - `Gitlab::Database.version`: returns the PostgreSQL version number as a string in the format `X.Y.Z`. This allows you to write code such as: ```ruby -if Gitlab::Database.postgresql? - if Gitlab::Database.version.to_f >= 9.6 - run_really_fast_query - else - run_fast_query - end +if Gitlab::Database.version.to_f >= 9.6 + run_really_fast_query else - run_query + run_fast_query end ``` diff --git a/doc/development/what_requires_downtime.md b/doc/development/what_requires_downtime.md index 00371057d3c..9e43758a4aa 100644 --- a/doc/development/what_requires_downtime.md +++ b/doc/development/what_requires_downtime.md @@ -37,11 +37,19 @@ information on how to use this method. ## Dropping Columns Removing columns is tricky because running GitLab processes may still be using -the columns. To work around this you will need two separate merge requests and -releases: one to ignore and then remove the column, and one to remove the ignore -rule. +the columns. To work around this safely, you will need three steps in three releases: -### Step 1: Ignoring The Column +1. Ignoring the column (release M) +1. Dropping the column (release M+1) +1. Removing the ignore rule (release M+2) + +The reason we spread this out across three releases is that dropping a column is +a destructive operation that can't be rolled back easily. + +Following this procedure helps us to make sure there are no deployments to GitLab.com +and upgrade processes for self-hosted installations that lump together any of these steps. + +### Step 1: Ignoring the column (release M) The first step is to ignore the column in the application code. This is necessary because Rails caches the columns and re-uses this cache in various @@ -50,18 +58,46 @@ places. This can be done by defining the columns to ignore. For example, to igno ```ruby class User < ApplicationRecord - self.ignored_columns += %i[updated_at] + include IgnorableColumns + ignore_column :updated_at, remove_with: '12.7', remove_after: '2019-12-22' end ``` -Once added you should create a _post-deployment_ migration that removes the -column. Both these changes should be submitted in the same merge request. +Multiple columns can be ignored, too: + +```ruby +ignore_columns %i[updated_at created_at], remove_with: '12.7', remove_after: '2019-12-22' +``` + +We require indication of when it is safe to remove the column ignore with: + +- `remove_with`: set to a GitLab release typically two releases (M+2) after adding the + column ignore. +- `remove_after`: set to a date after which we consider it safe to remove the column + ignore, typically within the development cycle of release M+2. + +This information allows us to reason better about column ignores and makes sure we +don't remove column ignores too early for both regular releases and deployments to GitLab.com. For +example, this avoids a situation where we deploy a bulk of changes that include both changes +to ignore the column and subsequently remove the column ignore (which would result in a downtime). + +In this example, the change to ignore the column went into release 12.5. + +### Step 2: Dropping the column (release M+1) + +Continuing our example, dropping the column goes into a _post-deployment_ migration in release 12.6: + +```ruby + remove_column :user, :updated_at +``` + +### Step 3: Removing the ignore rule (release M+2) -### Step 2: Removing The Ignore Rule +With the next release, in this example 12.7, we set up another merge request to remove the ignore rule. +This removes the `ignore_column` line and - if not needed anymore - also the inclusion of `IgnoreableColumns`. -Once the changes from step 1 have been released & deployed you can set up a -separate merge request that removes the ignore rule. This merge request can -simply remove the `self.ignored_columns` line. +This should only get merged with the release indicated with `remove_with` and once +the `remove_after` date has passed. ## Renaming Columns diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index 258a85d0474..0f95c9e0cb6 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -38,7 +38,7 @@ The following are guides to basic GitLab functionality: If you're familiar with Git on the command line, you can interact with your GitLab projects just as you would with any other Git repository. -These resources will help get further acclimated to working on the command line. +These resources will help you get further acclimated to working on the command line. - [Start using Git on the command line](start-using-git.md), for some simple Git commands. - [Command line basics](command-line-commands.md), to create and edit files using the command line. diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md index 1f43b151d5d..097794d39a7 100644 --- a/doc/gitlab-basics/start-using-git.md +++ b/doc/gitlab-basics/start-using-git.md @@ -61,10 +61,12 @@ To verify that you entered your email correctly, type: git config --global user.email ``` -You'll need to do this only once, since you are using the `--global` option. It tells -Git to always use this information for anything you do on that system. If you want -to override this with a different username or email address for specific projects or repositories, -you can run the command without the `--global` option when you’re in that project, and that will default to `--local`. You can read more on how Git manages configurations in the [Git Config](https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration) documentation. +You'll need to do this only once, since you are using the `--global` option. It +tells Git to always use this information for anything you do on that system. If +you want to override this with a different username or email address for specific +projects or repositories, you can run the command without the `--global` option +when you’re in that project, and that will default to `--local`. You can read more +on how Git manages configurations in the [Git Config](https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration) documentation. ## Check your information @@ -102,7 +104,10 @@ files to your local computer, automatically preserving the Git connection with t remote repository. You can either clone it via HTTPS or [SSH](../ssh/README.md). If you chose to clone -it via HTTPS, you'll have to enter your credentials every time you pull and push. You can read more about credential storage in the [Git Credentials documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). With SSH, you enter your credentials only once. +it via HTTPS, you'll have to enter your credentials every time you pull and push. +You can read more about credential storage in the +[Git Credentials documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). +With SSH, you enter your credentials only once. You can find both paths (HTTPS and SSH) by navigating to your project's landing page and clicking **Clone**. GitLab will prompt you with both paths, from which you can copy @@ -113,8 +118,8 @@ As an example, consider this repository path: - HTTPS: `https://gitlab.com/gitlab-org/gitlab.git` - SSH: `git@gitlab.com:gitlab-org/gitlab.git` -To get started, open a terminal window in the directory you wish to clone the repository -files into, and run one of the following commands. +To get started, open a terminal window in the directory you wish to clone the +repository files into, and run one of the following commands. Clone via HTTPS: @@ -134,9 +139,9 @@ on it locally. ### Switch to the master branch -You are always in a branch when working with Git. The main branch is the master branch, -but you can use the same command to switch to a different branch by changing `master` -to the branch name. +You are always in a branch when working with Git. The main branch is the master +branch, but you can use the same command to switch to a different branch by +changing `master` to the branch name. ```bash git checkout master @@ -145,10 +150,10 @@ git checkout master ### Download the latest changes in the project To work on an up-to-date copy of the project (it is important to do this every time -you start working on a project), you `pull` to get all the changes made by users since -the last time you cloned or pulled the project. Use `master` for the `<name-of-branch>` -to get the main branch code, or the branch name of the branch you are currently working -in. +you start working on a project), you `pull` to get all the changes made by users +since the last time you cloned or pulled the project. Use `master` for the +`<name-of-branch>` to get the main branch code, or the branch name of the branch +you are currently working in. ```bash git pull <REMOTE> <name-of-branch> @@ -156,10 +161,11 @@ git pull <REMOTE> <name-of-branch> When you clone a repository, `REMOTE` is typically `origin`. This is where the repository was cloned from, and it indicates the SSH or HTTPS URL of the repository -on the remote server. `<name-of-branch>` is usually `master`, but it may be any existing -branch. You can create additional named remotes and branches as necessary. +on the remote server. `<name-of-branch>` is usually `master`, but it may be any +existing branch. You can create additional named remotes and branches as necessary. -You can learn more on how Git manages remote repositories in the [Git Remote documentation](https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes). +You can learn more on how Git manages remote repositories in the +[Git Remote documentation](https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes). ### View your remote repositories @@ -184,9 +190,9 @@ so use something easy to remember and type. ### Create a branch -To create a new branch, to work from without affecting the `master` branch, type the -following (spaces won't be recognized in the branch name, so you will need to use a -hyphen or underscore): +To create a new branch, to work from without affecting the `master` branch, type +the following (spaces won't be recognized in the branch name, so you will need to +use a hyphen or underscore): ```bash git checkout -b <name-of-branch> @@ -245,6 +251,10 @@ The `.` character means _all file changes in the current directory and all subdi ### Send changes to GitLab.com +NOTE: **Note:** +To create a merge request from a fork to an upstream repository, see the +[forking workflow](../user/project/repository/forking_workflow.md) + To push all local commits (saved changes) to the remote repository: ```bash @@ -301,6 +311,21 @@ git checkout <name-of-branch> git merge master ``` +### Synchronize changes in a forked repository with the upstream + +[Forking a repository](../user/project/repository/forking_workflow.md lets you create +a copy of a repository in your namespace. Changes made to your copy of the repository +are not synchronized automatically with the original. +Your local fork (copy) contains changes made by you only, so to keep the project +in sync with the original project, you need to `pull` from the original repository. + +You must [create a link to the remote repository](#add-a-remote-repository) to pull +changes from the original repository. It is common to call this remote the `upstream`. + +You can now use the `upstream` as a [`<remote>` to `pull` new updates](#download-the-latest-changes-in-the-project) +from the original repository, and use the `origin` +to [push local changes](#send-changes-to-gitlabcom) and create merge requests. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/install/aws/index.md b/doc/install/aws/index.md index c1dde05196c..8165d3edabb 100644 --- a/doc/install/aws/index.md +++ b/doc/install/aws/index.md @@ -362,7 +362,7 @@ Choose the AMI: You should choose an instance type based on your workload. Consult [the hardware requirements](../requirements.md#hardware-requirements) to choose -one that fits your needs (at least `c4.xlarge`, which is enough to accommodate 100 users): +one that fits your needs (at least `c5.xlarge`, which is enough to accommodate 100 users): 1. Choose the your instance type. 1. Click **Next: Configure Instance Details**. diff --git a/doc/install/installation.md b/doc/install/installation.md index 98094ca1185..d420ac5e952 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -134,7 +134,7 @@ Make sure you have the right version of Git installed: # Install Git sudo apt-get install -y git-core -# Make sure Git is version 2.22.0 or higher +# Make sure Git is version 2.24.1 or higher (minimal supported version is 2.22.0) git --version ``` @@ -171,9 +171,9 @@ sudo make install # Download and compile from source cd /tmp -curl --remote-name --location --progress https://www.kernel.org/pub/software/scm/git/git-2.22.0.tar.gz -echo 'a4b7e4365bee43caa12a38d646d2c93743d755d1cea5eab448ffb40906c9da0b git-2.22.0.tar.gz' | shasum -a256 -c - && tar -xzf git-2.22.0.tar.gz -cd git-2.22.0/ +curl --remote-name --location --progress https://www.kernel.org/pub/software/scm/git/git-2.24.1.tar.gz +echo 'ad5334956301c86841eb1e5b1bb20884a6bad89a10a6762c958220c7cf64da02 git-2.24.1.tar.gz' | shasum -a256 -c - && tar -xzf git-2.24.1.tar.gz +cd git-2.24.1/ ./configure --with-libpcre make prefix=/usr/local all diff --git a/doc/install/requirements.md b/doc/install/requirements.md index ecd6516bd2e..106c7714bfe 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -128,6 +128,8 @@ CREATE EXTENSION pg_trgm; On some systems you may need to install an additional package (e.g. `postgresql-contrib`) for this extension to become available. +NOTE: **Note:** Support for PostgreSQL 9.6 and 10 will be removed in GitLab 13.0 so that GitLab can benefit from PostgreSQL 11 improvements, such as partitioning. For the schedule on adding support for PostgreSQL 11 and 12, see [the related epic](https://gitlab.com/groups/gitlab-org/-/epics/2184). For the release schedule for GitLab 13.0, see [GitLab's release and maintenance policy](../policy/maintenance.md). + #### Additional requirements for GitLab Geo If you are using [GitLab Geo](../development/geo.md): diff --git a/doc/integration/README.md b/doc/integration/README.md index 3f33aa94cb9..5cda537ac39 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -30,6 +30,7 @@ GitLab can be configured to authenticate access requests with the following auth - Use [OmniAuth](omniauth.md) to enable sign in via Twitter, GitHub, GitLab.com, Google, Bitbucket, Facebook, Shibboleth, SAML, Crowd, Azure or Authentiq ID. - Use GitLab as an [OpenID Connect](openid_connect_provider.md) identity provider. +- Authenticate to [Vault](vault.md) through GitLab OpenID Connect. - Configure GitLab as a [SAML](saml.md) 2.0 Service Provider. ## Security enhancements diff --git a/doc/integration/akismet.md b/doc/integration/akismet.md index cb8f25d2895..5b697d387e9 100644 --- a/doc/integration/akismet.md +++ b/doc/integration/akismet.md @@ -24,9 +24,9 @@ To use Akismet: 1. Click on **Show** to reveal the API key. -1. Go to Applications Settings on Admin Area (`admin/application_settings`) +1. Go to **Admin Area > Settings > Reporting** (`/admin/application_settings/reporting`). -1. Check the **Enable Akismet** checkbox +1. Check the **Enable Akismet** checkbox. 1. Fill in the API key from step 3. diff --git a/doc/integration/elasticsearch.md b/doc/integration/elasticsearch.md index 5c77bd5bcd9..62b3de72a3a 100644 --- a/doc/integration/elasticsearch.md +++ b/doc/integration/elasticsearch.md @@ -158,7 +158,10 @@ If you select `Limit namespaces and projects that can be indexed`, more options You can select namespaces and projects to index exclusively. Please note that if the namespace is a group it will include any sub-groups and projects belonging to those sub-groups to be indexed as well. +Elasticsearch only provides cross-group code/commit search (global) if all name-spaces are indexed. In this particular scenario where only a subset of namespaces are indexed, a global search will not provide a code or commit scope. This will be possible only in the scope of an indexed namespace. Currently there is no way to code/commit search in multiple indexed namespaces (when only a subset of namespaces has been indexed). For example if two groups are indexed, there is no way to run a single code search on both. You can only run a code search on the first group and then on the second. + You can filter the selection dropdown by writing part of the namespace or project name you're interested in. +  NOTE: **Note**: @@ -270,7 +273,7 @@ or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq decrease in indexing time. We'll enable them when indexing is done. This step is optional! ```bash - curl --request PUT localhost:9200/gitlab-production/_settings --data '{ + curl --request PUT localhost:9200/gitlab-production/_settings --header 'Content-Type: application/json' --data '{ "index" : { "refresh_interval" : "-1", "number_of_replicas" : 0 @@ -352,7 +355,7 @@ or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq 1. Enable replication and refreshing again after indexing (only if you previously disabled it): ```bash - curl --request PUT localhost:9200/gitlab-production/_settings --data '{ + curl --request PUT localhost:9200/gitlab-production/_settings --header 'Content-Type: application/json' ---data '{ "index" : { "number_of_replicas" : 1, "refresh_interval" : "1s" @@ -364,7 +367,7 @@ or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq For Elasticsearch 6.x, the index should be in read-only mode before proceeding with the force merge: ```bash - curl --request PUT localhost:9200/gitlab-production/_settings --data '{ + curl --request PUT localhost:9200/gitlab-production/_settings ---header 'Content-Type: application/json' --data '{ "settings": { "index.blocks.write": true } }' @@ -373,13 +376,13 @@ or creating [extra Sidekiq processes](../administration/operations/extra_sidekiq Then, initiate the force merge: ```bash - curl --request POST 'http://localhost:9200/gitlab-production/_forcemerge?max_num_segments=5' + curl --request POST 'localhost:9200/gitlab-production/_forcemerge?max_num_segments=5' ``` After this, if your index is in read-only mode, switch back to read-write: ```bash - curl --request PUT localhost:9200/gitlab-production/_settings --data '{ + curl --request PUT localhost:9200/gitlab-production/_settings ---header 'Content-Type: application/json' --data '{ "settings": { "index.blocks.write": false } }' @@ -467,7 +470,7 @@ However, some larger installations may wish to tune the merge policy settings: - Consider reducing the `index.merge.policy.max_merged_segment` size from the default 5 GB to maybe 2 GB or 3 GB. Merging only happens when a segment has at least 50% deletions. Smaller segment sizes will allow merging to happen more frequently. ```bash - curl --request PUT http://localhost:9200/gitlab-production/_settings --data '{ + curl --request PUT localhost:9200/gitlab-production/_settings ---header 'Content-Type: application/json' --data '{ "index" : { "merge.policy.max_merged_segment": "2gb" } @@ -477,7 +480,7 @@ However, some larger installations may wish to tune the merge policy settings: - You can also adjust `index.merge.policy.reclaim_deletes_weight`, which controls how aggressively deletions are targeted. But this can lead to costly merge decisions, so we recommend not changing this unless you understand the tradeoffs. ```bash - curl --request PUT http://localhost:9200/gitlab-production/_settings --data '{ + curl --request PUT localhost:9200/gitlab-production/_settings ---header 'Content-Type: application/json' --data '{ "index" : { "merge.policy.reclaim_deletes_weight": "3.0" } @@ -502,6 +505,9 @@ Here are some common pitfalls and how to overcome them: If you see `Elasticsearch::Model::Response::Records`, you are using Elasticsearch. + NOTE: **Note**: + The above instructions are used to verify that GitLab is using Elasticsearch only when indexing all namespaces. This is not to be used for scenarios that only index a [subset of namespaces](https://docs.gitlab.com/ee/integration/elasticsearch.html#limiting-namespaces-and-projects). + - **I updated GitLab and now I can't find anything** We continuously make updates to our indexing strategies and aim to support @@ -518,10 +524,13 @@ Here are some common pitfalls and how to overcome them: ```ruby u = User.find_by_username('your-username') - s = SearchService.new(u, {:search => 'search_term', :scope => ‘blobs’}) + s = SearchService.new(u, {:search => 'search_term', :scope => 'blobs'}) pp s.search_objects.to_a ``` + NOTE: **Note**: + The above instructions are not to be used for scenarios that only index a [subset of namespaces](https://docs.gitlab.com/ee/integration/elasticsearch.html#limiting-namespaces-and-projects). + See [Elasticsearch Index Scopes](#elasticsearch-index-scopes) for more information on searching for specific types of data. - **I indexed all the repositories but then switched Elasticsearch servers and now I can't find anything** diff --git a/doc/integration/github.md b/doc/integration/github.md index 23dd67f6891..f46038269c0 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -166,3 +166,36 @@ via Omnibus, or [restart GitLab] if you installed from source. [reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure [restart GitLab]: ../administration/restart_gitlab.md#installations-from-source + +## Troubleshooting + +### Error 500 when trying to sign in to GitLab via GitHub Enterprise + +Check the [`production.log`](../administration/logs.md#productionlog) +on your GitLab server to obtain further details. If you are getting the error like +`Faraday::ConnectionFailed (execution expired)` in the log, there may be a connectivity issue +between your GitLab instance and GitHub Enterprise. To verify it, [start the rails console](https://docs.gitlab.com/omnibus/maintenance/#starting-a-rails-console-session) +and run the commands below replacing `<github_url>` with the URL of your GitHub Enterprise instance: + +```ruby +uri = URI.parse("https://<github_url>") # replace `GitHub-URL` with the real one here +http = Net::HTTP.new(uri.host, uri.port) +http.use_ssl = true +http.verify_mode = 1 +response = http.request(Net::HTTP::Get.new(uri.request_uri)) +``` + +If you are getting a similar `execution expired` error, it confirms the theory about the +network connectivity. In that case, make sure that the GitLab server is able to reach your +GitHub enterprise instance. + +### Signing in using your GitHub account without a pre-existing GitLab account is not allowed + +If you're getting the message `Signing in using your GitHub account without a pre-existing +GitLab account is not allowed. Create a GitLab account first, and then connect it to your +GitHub account` when signing in, in GitLab: + +1. Go to your **Profile > Account**. +1. Under the "Social sign-in" section, click **Connect** near the GitHub icon. + +After that, you should be able to sign in via GitHub successfully. diff --git a/doc/integration/img/authorize_vault_with_gitlab_v12_6.png b/doc/integration/img/authorize_vault_with_gitlab_v12_6.png Binary files differnew file mode 100644 index 00000000000..dc5bc954cd7 --- /dev/null +++ b/doc/integration/img/authorize_vault_with_gitlab_v12_6.png diff --git a/doc/integration/img/gitlab_oauth_vault_v12_6.png b/doc/integration/img/gitlab_oauth_vault_v12_6.png Binary files differnew file mode 100644 index 00000000000..f952abc2c6d --- /dev/null +++ b/doc/integration/img/gitlab_oauth_vault_v12_6.png diff --git a/doc/integration/img/sign_into_vault_with_gitlab_v12_6.png b/doc/integration/img/sign_into_vault_with_gitlab_v12_6.png Binary files differnew file mode 100644 index 00000000000..8afa2c6aabd --- /dev/null +++ b/doc/integration/img/sign_into_vault_with_gitlab_v12_6.png diff --git a/doc/integration/img/signed_into_vault_via_oidc_v12_6.png b/doc/integration/img/signed_into_vault_via_oidc_v12_6.png Binary files differnew file mode 100644 index 00000000000..0ad81ef40e6 --- /dev/null +++ b/doc/integration/img/signed_into_vault_via_oidc_v12_6.png diff --git a/doc/integration/jenkins.md b/doc/integration/jenkins.md index a54f6843c53..fe607b2afdf 100644 --- a/doc/integration/jenkins.md +++ b/doc/integration/jenkins.md @@ -135,3 +135,32 @@ configured or there was an error reporting the status via the API. 1. [Configure the Jenkins server](#configure-the-jenkins-server) for GitLab API access 1. [Configure a Jenkins project](#configure-a-jenkins-project), including the 'Publish build status to GitLab' post-build action. + +### Merge Request event does not trigger a Jenkins Pipeline + +Check the `/var/log/gitlab/gitlab-rails/production.log` file for messages like: + +```plaintext +WebHook Error => Net::ReadTimeout +``` + +or + +```plaintext +WebHook Error => execution expired +``` + +If those are present, the request is exceeding the +[webhook timeout](../user/project/integrations/webhooks.md#receiving-duplicate-or-multiple-webhook-requests-triggered-by-one-event), +which is set to 10 seconds by default. + +To fix this the `gitlab_rails['webhook_timeout']` value will need to be increased +in the `gitlab.rb` config file, followed by the [`gitlab-ctl reconfigure` command](../administration/restart_gitlab.md). + +If you don't find the errors above, but do find *duplicate* entries like below (in `/var/log/gitlab/gitlab-rail`), this +could also indicate that [webhook requests are timing out](../user/project/integrations/webhooks.md#receiving-duplicate-or-multiple-webhook-requests-triggered-by-one-event): + +``` +2019-10-25_04:22:41.25630 2019-10-25T04:22:41.256Z 1584 TID-ovowh4tek WebHookWorker JID-941fb7f40b69dff3d833c99b INFO: start +2019-10-25_04:22:41.25630 2019-10-25T04:22:41.256Z 1584 TID-ovowh4tek WebHookWorker JID-941fb7f40b69dff3d833c99b INFO: start +``` diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 099cab0f5b8..a667c2e84c9 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -541,7 +541,7 @@ args: { } ``` -GitLab will sign the request with the provided private key. GitLab will include the configured public x500 certificate in the metadata for your Identity Provider to validate the signature of the received request with. For more information on this option, see the [ruby-saml gem documentation](https://github.com/onelogin/ruby-saml/tree/v1.7.0). The `ruby-saml` gem is used by the [omniauth-saml gem](https://github.com/omniauth/omniauth-saml) to implement the client side of the SAML authentication. +GitLab will sign the request with the provided private key. GitLab will include the configured public x500 certificate in the metadata for your Identity Provider to validate the signature of the received request with. For more information on this option, see the [Ruby SAML gem documentation](https://github.com/onelogin/ruby-saml/tree/v1.7.0). The Ruby SAML gem is used by the [OmniAuth SAML gem](https://github.com/omniauth/omniauth-saml) to implement the client side of the SAML authentication. ## Troubleshooting diff --git a/doc/integration/vault.md b/doc/integration/vault.md new file mode 100644 index 00000000000..68803fed35d --- /dev/null +++ b/doc/integration/vault.md @@ -0,0 +1,120 @@ +--- +type: reference, howto +--- + +# Vault Authentication with GitLab OpenID Connect + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/22323) in GitLab 9.0 + +[Vault](https://www.vaultproject.io/) is a secrets management application offered by HashiCorp. +It allows you to store and manage sensitive information such secret environment variables, encryption keys, and authentication tokens. +Vault offers Identity-based Access, which means Vault users can authenticate through several of their preferred cloud providers. + +In this document, we'll explain how Vault users can authenticate themselves through GitLab by utilizing our OpenID authentication feature. +The following assumes you already have Vault installed and running. + +1. **Get the OpenID Connect client ID and secret from GitLab:** + + First you'll need to create a GitLab application to obtain an application ID and secret for authenticating into Vault. To do this, sign in to GitLab and follow these steps: + + 1. On GitLab, click your avatar on the top-right corner, and select your user **Settings > Applications**. + 1. Fill out the application **Name** and [**Redirect URI**](https://www.vaultproject.io/docs/auth/jwt.html#redirect-uris), + making sure to select the **OpenID** scope. + 1. Save application. + 1. Copy client ID and secret, or keep the page open for reference. +  + +1. **Enable OIDC auth on Vault:** + + OpenID Connect is not enabled in Vault by default. This needs to be enabled in the terminal. + + Open a terminal session and run the following command to enable the OpenID Connect authentication provider in Vault: + + ```bash + vault auth enable oidc + ``` + + You should see the following output in the terminal: + + ```bash + Success! Enabled oidc auth method at: oidc/ + ``` + +1. **Write the OIDC config:** + + Next, Vault needs to be given the application ID and secret generated by GitLab. + + In the terminal session, run the following command to give Vault access to the GitLab application you've just created with an OpenID scope. This allows Vault to authenticate through GitLab. + + Replace `your_application_id` and `your_secret` in the example below with the application ID and secret generated for your app: + + ```bash + $ vault write auth/oidc/config \ + oidc_discovery_url="https://gitlab.com" \ + oidc_client_id="your_application_id" \ + oidc_client_secret="your_secret" \ + default_role="demo" \ + bound_issuer="localhost" + ``` + + You should see the following output in the terminal: + + ```bash + Success! Data written to: auth/oidc/config + ``` + +1. **Write the OIDC Role Config:** + + Now that Vault has a GitLab application ID and secret, it needs to know the [**Redirect URIs**](https://www.vaultproject.io/docs/auth/jwt.html#redirect-uris) and scopes given to GitLab during the application creation process. The redirect URIs need to match where your Vault instance is running. The `oidc_scopes` field needs to include the `openid`. Similarly to the previous step, replace `your_application_id` with the generated application ID from GitLab: + + This configuration is saved under the name of the role you are creating. In this case, we are creating a `demo` role. Later, we'll show how you can access this role through the Vault CLI. + + ```bash + vault write auth/oidc/role/demo \ + user_claim="sub" \ + allowed_redirect_uris="http://localhost:8250/oidc/callback,http://127.0.0.1:8200/ui/vault/auth/oidc/oidc/callback" \ + bound_audiences="your_application_id" \ + role_type="oidc" \ + oidc_scopes="openid" \ + policies=demo \ + ttl=1h + ``` + +1. **Sign in to Vault:** + + 1. Go to your Vault UI (example: [http://127.0.0.1:8200/ui/vault/auth?with=oidc](http://127.0.0.1:8200/ui/vault/auth?with=oidc)). + 1. If the `OIDC` method is not currently selected, open the dropdown and select it. + 1. Click the **Sign in With GitLab** button, which will open a modal window: +  + + 1. Click **Authorize** on the modal to allow Vault to sign in through GitLab. This will redirect you back to your Vault UI as a signed-in user. +  + +1. **Sign in using the Vault CLI** (optional): + + Vault also allows you to sign in via their CLI. + + After writing the same configurations from above, you can run the command below in your terminal to sign in with the role configuration created in step 4 above: + + ```bash + vault login -method=oidc port=8250 role=demo + ``` + + Here is a short explaination of what this command does: + + 1. In the **Write the OIDC Role Config** (step 4), we created a role called `demo`. We set `role=demo` so Vault knows which configuration we'd like to login in with. + 1. To set Vault to use the `OIDC` sign-in method, we set `-method=oidc`. + 1. To set the port that GitLab should redirect to, we set `port=8250` or another port number that matches the port given to GitLab when listing [Redirect URIs](https://www.vaultproject.io/docs/auth/jwt.html#redirect-uris). + + Once you run the command above, it will present a link in the terminal. + Click the link in the terminal and a tab will open in the browser confirming you're signed into Vault via OIDC: + +  + + The terminal will output: + + ``` + Success! You are now authenticated. The token information displayed below + is already stored in the token helper. You do NOT need to run "vault login" + again. Future Vault requests will automatically use this token. + ``` diff --git a/doc/policy/maintenance.md b/doc/policy/maintenance.md index ef94236d711..7617d0c8881 100644 --- a/doc/policy/maintenance.md +++ b/doc/policy/maintenance.md @@ -48,9 +48,9 @@ incremental upgrades (and installations) are as simple as possible. review process a new change goes through. 1. Ensuring that tests pass on older release is a considerable challenge in some cases, and as such is very time consuming. -Including new features in patch releases is not possible as that would break [Semantic Versioning]. -Breaking [Semantic Versioning] has the following consequences for users that -have to adhere to various internal requirements (e.g. org. compliance, verifying new features and similar): +Including new features in patch releases is not possible as that would break [Semantic Versioning](https://semver.org/). +Breaking [Semantic Versioning](https://semver.org/) has the following consequences for users that +have to adhere to various internal requirements (for example, org. compliance, verifying new features, and similar): 1. Inability to quickly upgrade to leverage bug fixes included in patch versions. 1. Inability to quickly upgrade to leverage security fixes included in patch versions. @@ -58,9 +58,12 @@ have to adhere to various internal requirements (e.g. org. compliance, verifying In cases where a strategic user has a requirement to test a feature before it is officially released, we can offer to create a Release Candidate (RC) version that will -include the specific feature. This should be needed only in extreme cases, and can be requested for consideration by raising an issue in [release/tasks] issue tracker. -It is important to note that the Release Candidate will also contain other -features and changes as it is not possible to easily isolate a specific feature (similar reasons as noted above). The Release Candidate will be no different than any code that is deployed to GitLab.com or is publicly accessible. +include the specific feature. This should be needed only in extreme cases, and can be requested for +consideration by raising an issue in the [release/tasks](https://gitlab.com/gitlab-org/release/tasks/issues/new?issuable_template=Backporting-request) issue tracker. +It is important to note that the Release Candidate will also contain other features and changes as +it is not possible to easily isolate a specific feature (similar reasons as noted above). The +Release Candidate will be no different than any code that is deployed to GitLab.com or is publicly +accessible. ### Backporting to older releases @@ -68,11 +71,16 @@ Backporting to more than one stable release is reserved for [security releases]( In some cases however, we may need to backport *a bug fix* to more than one stable release, depending on the severity of the bug. -Decision on whether backporting a change will be performed is done at the discretion of the [current release managers][release-managers], similar to what is described in the [managing bugs] process, based on *all* of the following: +The decision on whether backporting a change will be performed is done at the discretion of the +[current release managers](https://about.gitlab.com/community/release-managers/), similar to what is +described in the [managing bugs](https://gitlab.com/gitlab-org/gitlab/blob/master/PROCESS.md#managing-bugs) process, +based on *all* of the following: -1. Estimated [severity][severity-labels] of the bug: Highest possible impact to users based on the current definition of severity. +1. Estimated [severity](../development/contributing/issue_workflow.md#severity-labels) of the bug: + Highest possible impact to users based on the current definition of severity. -1. Estimated [priority][priority-labels] of the bug: Immediate impact on all impacted users based on the above estimated severity. +1. Estimated [priority](../development/contributing/issue_workflow.md#priority-labels) of the bug: + Immediate impact on all impacted users based on the above estimated severity. 1. Potentially incurring data loss and/or security breach. @@ -83,7 +91,8 @@ the current stable stable release, and two previous monthly releases. For instance, if we release `11.2.1` with a fix for a severe bug introduced in `11.0.0`, we could backport the fix to a new `11.0.x`, and `11.1.x` patch release. -To request backporting to more than one stable release for consideration, raise an issue in [release/tasks] issue tracker. +To request backporting to more than one stable release for consideration, raise an issue in the +[release/tasks](https://gitlab.com/gitlab-org/release/tasks/issues/new?issuable_template=Backporting-request) issue tracker. ### Security releases @@ -115,17 +124,24 @@ one major version. For example, it is safe to: - `8.9.0` -> `8.9.7` - `8.9.0` -> `8.9.1` - `8.9.2` -> `8.9.6` + - `9.5.5` -> `9.5.9` + - `10.6.3` -> `10.6.6` + - `11.11.1` -> `11.11.8` + - `12.0.4` -> `12.0.9` - Upgrade the minor version: - `8.9.4` -> `8.12.3` - `9.2.3` -> `9.5.5` + - `10.6.6` -> `10.8.7` + - `11.3.4` -> `11.11.8` Upgrading the major version requires more attention. We cannot guarantee that upgrading between major versions will be seamless. As previously mentioned, major versions are reserved for backwards incompatible changes. - We recommend that you first upgrade to the latest available minor version within your major version. By doing this, you can address any deprecation messages that could change behavior in the next major release. +To ensure background migrations are successful, increment by one minor version during the version jump before installing newer releases. +For example: `11.11.x` -> `12.0.x` Please see the table below for some examples: | Latest stable version | Your version | Recommended upgrade path | Note | @@ -133,14 +149,11 @@ Please see the table below for some examples: | 9.4.5 | 8.13.4 | `8.13.4` -> `8.17.7` -> `9.4.5` | `8.17.7` is the last version in version `8` | | 10.1.4 | 8.13.4 | `8.13.4 -> 8.17.7 -> 9.5.10 -> 10.1.4` | `8.17.7` is the last version in version `8`, `9.5.10` is the last version in version `9` | | 11.3.4 | 8.13.4 | `8.13.4` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> `11.3.4` | `8.17.7` is the last version in version `8`, `9.5.10` is the last version in version `9`, `10.8.7` is the last version in version `10` | -| 12.0.2 | 11.3.4 | `11.3.4` -> `11.11.x` -> `12.0.2` | `11.11.x` is the last version in version `11` +| 12.5.8 | 11.3.4 | `11.3.4` -> `11.11.8` -> `12.0.9` -> `12.5.8` | `11.11.8` is the last version in version `11` | + +To check the size of `background_migration` queue and to learn more about background migrations +see [Upgrading without downtime](../update/README.md#upgrading-without-downtime). More information about the release procedures can be found in our [release documentation](https://gitlab.com/gitlab-org/release/docs). You may also want to read our [Responsible Disclosure Policy](https://about.gitlab.com/security/disclosure/). - -[release-managers]: https://about.gitlab.com/community/release-managers/ -[priority-definition]: ../development/contributing/issue_workflow.md#priority-labels -[severity-labels]: ../development/contributing/issue_workflow.html#severity-labels -[managing bugs]: https://gitlab.com/gitlab-org/gitlab/blob/master/PROCESS.md#managing-bugs -[release/tasks]: https://gitlab.com/gitlab-org/release/tasks/issues diff --git a/doc/public_access/img/project_visibility_confirmation_v12_6.png b/doc/public_access/img/project_visibility_confirmation_v12_6.png Binary files differnew file mode 100644 index 00000000000..ac4d70ff11a --- /dev/null +++ b/doc/public_access/img/project_visibility_confirmation_v12_6.png diff --git a/doc/public_access/public_access.md b/doc/public_access/public_access.md index bb19436017a..1b6895aaef1 100644 --- a/doc/public_access/public_access.md +++ b/doc/public_access/public_access.md @@ -77,6 +77,16 @@ by accident. The restricted visibility settings do not apply to admin users. For details, see [Restricted visibility levels](../user/admin_area/settings/visibility_and_access_controls.md#restricted-visibility-levels). +## Reducing visibility + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/33358) in GitLab 12.6. + +Reducing a project's visibility level will remove the fork relationship between the project and +any forked project. This is a potentially destructive action which requires confirmation before +this can be saved. + + + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/push_rules/push_rules.md b/doc/push_rules/push_rules.md index 0771a3e4225..d778d6b929c 100644 --- a/doc/push_rules/push_rules.md +++ b/doc/push_rules/push_rules.md @@ -49,6 +49,13 @@ branch names globally in Push Rules, you can now sleep without the anxiety of your developers' mistakes. Every branch that doesn't match your push rule will get rejected. +### Custom Push Rules **(CORE ONLY)** + +It's possible to create custom push rules rather than the push rules available in +**Admin area > Push Rules** by using more advanced server-side Git hooks. + +See [custom server-side Git hooks](../administration/custom_hooks.md) for more information. + ## Enabling push rules NOTE: **Note:** diff --git a/doc/raketasks/generate_sample_prometheus_data.md b/doc/raketasks/generate_sample_prometheus_data.md new file mode 100644 index 00000000000..2489a2c2ad3 --- /dev/null +++ b/doc/raketasks/generate_sample_prometheus_data.md @@ -0,0 +1,16 @@ +# Generate Sample Prometheus Data + +This command will run Prometheus queries for each of the metrics of a specific environment +for a default time interval of 7 days ago to now. The results of each of query are stored +under a `sample_metrics` directory as a yaml file named by the metric's `identifier`. +When the environmental variable `USE_SAMPLE_METRICS` is set, the Prometheus API query is +re-routed to `Projects::Environments::SampleMetricsController` which loads the appropriate +data set if it is present within the `sample_metrics` directory. + +- This command requires an id from an Environment with an available Prometheus installation. + +**Example:** + +``` +bundle exec rake gitlab:generate_sample_prometheus_data[21] +``` diff --git a/doc/security/password_length_limits.md b/doc/security/password_length_limits.md index 9909ef4a8e4..235730eb825 100644 --- a/doc/security/password_length_limits.md +++ b/doc/security/password_length_limits.md @@ -4,7 +4,19 @@ type: reference, howto # Custom password length limits -The user password length is set to a minimum of 8 characters by default. +By default, GitLab supports passwords with: + +- A minimum length of 8. +- A maximum length of 128. + +GitLab administrators can modify password lengths: + +- Using configuration file. +- [From](https://gitlab.com/gitlab-org/gitlab/merge_requests/20661) GitLab 12.6, using the GitLab UI. + +## Modify maximum password length using configuration file + +The user password length is set to a maximum of 128 characters by default. To change that for installations from source: 1. Edit `devise_password_length.rb`: @@ -18,15 +30,35 @@ To change that for installations from source: 1. Change the new password length limits: ```ruby - config.password_length = 12..128 + config.password_length = 12..135 ``` In this example, the minimum length is 12 characters, and the maximum length - is 128 characters. + is 135 characters. 1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect. +NOTE: **Note:** +From GitLab 12.6, the minimum password length set in this configuration file will be ignored. Minimum password lengths will now have to be modified via the [GitLab UI](#modify-minimum-password-length-using-gitlab-ui) instead. + +## Modify minimum password length using GitLab UI + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/20661) in GitLab 12.6 + +The user password length is set to a minimum of 8 characters by default. +To change that using GitLab UI: + +In the Admin area under **Settings** (`/admin/application_settings`), go to section **Sign-up Restrictions**. + +[Minimum password length settings](../user/admin_area/img/minimum_password_length_settings_v12_6.png) + +Set the **Minimum password length** to a value greater than or equal to 8 and hit **Save changes** to save the changes. + +CAUTION: **Caution:** +Changing minimum or maximum limit does not affect existing user passwords in any manner. Existing users will not be asked to reset their password to adhere to the new limits. +The new limit restriction will only apply during new user sign-ups and when an existing user performs a password reset. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/security/rack_attack.md b/doc/security/rack_attack.md index 51b7d7db3e4..9cbb296338a 100644 --- a/doc/security/rack_attack.md +++ b/doc/security/rack_attack.md @@ -53,7 +53,8 @@ default['gitlab']['gitlab-rails']['rack_attack_protected_paths'] = [ '/users', '/users/confirmation', '/unsubscribes/', - '/import/github/personal_access_token' + '/import/github/personal_access_token', + '/admin/session' ] ``` diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 01d86331a0a..80b8db84bed 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -273,6 +273,65 @@ git config core.sshCommand "ssh -o IdentitiesOnly=yes -i ~/.ssh/private-key-file This will not use the SSH Agent and requires at least Git 2.10. +## Multiple accounts on a single GitLab instance + +The [per-repository](#per-repository-ssh-keys) method also works for using +multiple accounts within a single GitLab instance. + +Alternatively, it is possible to directly assign aliases to hosts in +`~.ssh/config`. SSH and, by extension, Git will fail to log in if there is +an `IdentityFile` set outside of a `Host` block in `.ssh/config`. This is +due to how SSH assembles `IdentityFile` entries and is not changed by +setting `IdentitiesOnly` to `yes`. `IdentityFile` entries should point to +the private key of an SSH key pair. + +NOTE: **Note:** +Private and public keys should be readable by the user only. Accomplish this +on Linux and macOS by running: `chmod 0400 ~/.ssh/<example_ssh_key>` and +`chmod 0400 ~/.ssh/<example_sh_key.pub>`. + +```conf +# User1 Account Identity +Host <user_1.gitlab.com> + Hostname gitlab.com + PreferredAuthentications publickey + IdentityFile ~/.ssh/<example_ssh_key1> + +# User2 Account Identity +Host <user_2.gitlab.com> + Hostname gitlab.com + PreferredAuthentications publickey + IdentityFile ~/.ssh/<example_ssh_key2> +``` + +NOTE: **Note:** +The example `Host` aliases are defined as `user_1.gitlab.com` and +`user_2.gitlab.com` for efficiency and transparency. Advanced configurations +are more difficult to maintain; using this type of alias makes it easier to +understand when using other tools such as `git remote` subcommands. SSH +would understand any string as a `Host` alias thus `Tanuki1` and `Tanuki2`, +despite giving very little context as to where they point, would also work. + +Cloning the `gitlab` repository normally looks like this: + +```sh +git clone git@gitlab.com:gitlab-org/gitlab.git +``` + +To clone it for `user_1`, replace `gitlab.com` with the SSH alias `user_1.gitlab.com`: + +```sh +git clone git@<user_1.gitlab.com>:gitlab-org/gitlab.git +``` + +Fix a previously cloned repository using the `git remote` command. + +The example below assumes the remote repository is aliased as `origin`. + +```sh +git remote set-url origin git@<user_1.gitlab.com>:gitlab-org/gitlab.git +``` + ## Deploy keys ### Per-repository deploy keys diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md index cae83d6186f..b406abaf481 100644 --- a/doc/subscriptions/index.md +++ b/doc/subscriptions/index.md @@ -223,6 +223,46 @@ The following table describes details of your subscription for groups: | Subscription start date | Date your subscription started. If this is for a Free plan, is the date you transitioned off your group's paid plan. | | Subscription end date | Date your current subscription will end. Does not apply to Free plans. | +#### CI pipeline minutes + +CI pipeline minutes are the execution time for your [pipelines](../ci/pipelines.md) on our shared runners. Each [GitLab.com tier](https://about.gitlab.com/pricing/) includes a monthly quota of CI pipeline minutes. The quota is applied per group, shared across all members of that group, its subgroups and nested projects. To view the usage, navigate to the group's page, then **Settings > Usage Quotas**. + +Only pipeline minutes for our shared runners are restricted. If you have a specific runner setup for your projects, there is no limit to your build time on GitLab.com. + +The minutes limit only applies to private projects. The available quota is reset on the first of each calendar month at midnight UTC. + +If you reach your limit, you can [purchase additional CI minutes](#extra-shared-runners-pipeline-minutes), or upgrade your account to [Silver or Gold](https://about.gitlab.com/pricing/). Note, your own runners can still be used even if you reach your limits. + +##### How pipeline quota usage is calculated + +Pipeline quota usage is calculated as the sum of the duration of each individual job. This is slightly different to how pipeline _duration_ is [calculated](https://docs.gitlab.com/ee/ci/pipelines.html#how-pipeline-duration-is-calculated). Pipeline quota usage doesn't consider the intersection of jobs. + +A simple example is: + +A (1, 3) +B (2, 4) +C (6, 7) + +In the example: + +A begins at 1 and ends at 3. +B begins at 2 and ends at 4. +C begins at 6 and ends at 7. +Visually, it can be viewed as: + +``` +0 1 2 3 4 5 6 7 + AAAAAAA + BBBBBBB + CCCC +``` + +The sum of each individual job is being calculated therefore in this example, `8` runner minutes would be used for this pipeline: + +``` +A + B + C = 3 + 3 + 2 => 8 +``` + #### Extra Shared Runners pipeline minutes If you're using GitLab.com, you can purchase additional CI minutes so your diff --git a/doc/topics/autodevops/img/autodevops_banner_v12_6.png b/doc/topics/autodevops/img/autodevops_banner_v12_6.png Binary files differnew file mode 100644 index 00000000000..51ccdeeaa52 --- /dev/null +++ b/doc/topics/autodevops/img/autodevops_banner_v12_6.png diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 93549ac4de5..33b13935de3 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -651,6 +651,8 @@ procfile exec` to replicate the environment where your application will run. #### Workers +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/30628) in GitLab 12.6, `.gitlab/auto-deploy-values.yaml` will be used by default for Helm upgrades. + Some web applications need to run extra deployments for "worker processes". For example, it is common in a Rails application to have a separate worker process to run background tasks like sending emails. @@ -672,13 +674,18 @@ need to: - Set a CI variable `K8S_SECRET_REDIS_URL`, which the URL of this instance to ensure it's passed into your deployments. -Once you have configured your worker to respond to health checks, you will -need to configure a CI variable `HELM_UPGRADE_EXTRA_ARGS` with the value -`--values helm-values.yaml`. +Once you have configured your worker to respond to health checks, run a Sidekiq +worker for your Rails application. For: + +- GitLab 12.6 and later, either: + - Add a file named `.gitlab/auto-deploy-values.yaml` to your repository. It will + be automatically used if found. + - Add a file with a different name or path to the repository, and override the value of the + `HELM_UPGRADE_VALUES_FILE` variable with the path and name. +- GitLab 12.5 and earlier, run the worker with the `--values` parameter that specifies + a file in the repository. -Then you can, for example, run a Sidekiq worker for your Rails application -by adding a file named `helm-values.yaml` to your repository with the following -content: +In any case, the file must contain the following: ```yml workers: @@ -766,10 +773,17 @@ or a `.buildpacks` file in your project: and add the URL of the buildpack to use on a line in the file. If you want to use multiple buildpacks, you can enter them in, one on each line. -CAUTION: **Caution:** -Using multiple buildpacks isn't yet supported by Auto DevOps. +#### Multiple buildpacks + +Using multiple buildpacks isn't fully supported by Auto DevOps because, when using the `.buildpacks` +file, Auto Test will not work. -CAUTION: **Caution:** When using the `.buildpacks` file, Auto Test will not work. The buildpack [heroku-buildpack-multi](https://github.com/heroku/heroku-buildpack-multi/) (which is used under the hood to parse the `.buildpacks` file) doesn't provide the necessary commands `bin/test-compile` and `bin/test`. Make sure to provide the project variable `BUILDPACK_URL` instead. +The buildpack [heroku-buildpack-multi](https://github.com/heroku/heroku-buildpack-multi/), +which is used under the hood to parse the `.buildpacks` file, doesn't provide the necessary commands +`bin/test-compile` and `bin/test`. + +If your goal is to use only a single custom buildpack, you should provide the project variable +`BUILDPACK_URL` instead. ### Custom `Dockerfile` @@ -976,6 +990,7 @@ applications. | `CANARY_PRODUCTION_REPLICAS` | Number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md) in the production environment. Takes precedence over `CANARY_REPLICAS`. Defaults to 1. | | `CANARY_REPLICAS` | Number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md). Defaults to 1. | | `HELM_RELEASE_NAME` | From GitLab 12.1, allows the `helm` release name to be overridden. Can be used to assign unique release names when deploying multiple projects to a single namespace. | +| `HELM_UPGRADE_VALUES_FILE` | From GitLab 12.6, allows the `helm upgrade` values file to be overridden. Defaults to `.gitlab/auto-deploy-values.yaml`. | | `HELM_UPGRADE_EXTRA_ARGS` | From GitLab 11.11, allows extra arguments in `helm` commands when deploying the application. Note that using quotes will not prevent word splitting. **Tip:** you can use this variable to [customize the Auto Deploy Helm chart](#custom-helm-chart) by applying custom override values with `--values my-values.yaml`. | | `INCREMENTAL_ROLLOUT_MODE` | From GitLab 11.4, if present, can be used to enable an [incremental rollout](#incremental-rollout-to-production-premium) of your application for the production environment. Set to `manual` for manual deployment jobs or `timed` for automatic rollout deployments with a 5 minute delay each one. | | `K8S_SECRET_*` | From GitLab 11.7, any variable prefixed with [`K8S_SECRET_`](#application-secret-variables) will be made available by Auto DevOps as environment variables to the deployed application. | @@ -1253,6 +1268,30 @@ Everything behaves the same way, except: 1. `timed rollout 50%` 1. `timed rollout 100%` +### Auto DevOps banner + +The following Auto DevOps banner will show for maintainers+ on new projects when Auto DevOps is not +enabled: + + + +The banner can be disabled for: + +- A user when they dismiss it themselves. +- A project by explicitly [disabling Auto DevOps](#enablingdisabling-auto-devops). +- An entire GitLab instance: + - By an administrator running the following in a Rails console: + + ```ruby + Feature.get(:auto_devops_banner_disabled).enable + ``` + + - Through the REST API with an admin access token: + + ```sh + curl --data "value=true" --header "PRIVATE-TOKEN: <personal_access_token>" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled + ``` + ## Currently supported languages Note that not all buildpacks support Auto Test yet, as it's a relatively new @@ -1325,27 +1364,6 @@ spec: service account for your project. For help debugging this issue, see [Troubleshooting failed deployment jobs](../../user/project/clusters/index.md#troubleshooting). -### Disable the banner instance wide - -If an administrator would like to disable the banners on an instance level, this -feature can be disabled either through the console: - -```sh -sudo gitlab-rails console -``` - -Then run: - -```ruby -Feature.get(:auto_devops_banner_disabled).enable -``` - -Or through the HTTP API with an admin access token: - -```sh -curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled -``` - [ce-37115]: https://gitlab.com/gitlab-org/gitlab-foss/issues/37115 [docker-in-docker]: ../../docker/using_docker_build.md#use-docker-in-docker-executor [review-app]: ../../ci/review_apps/index.md diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index 5391f6e5ad6..d4d559c9483 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -130,5 +130,25 @@ remote: Calculating new repository size... (cancelled after 729ms) This could be used to further investigate what operation is performing poorly and provide GitLab with more information on how to improve the service. +## `git clone` over HTTP fails with `transfer closed with outstanding read data remaining` error + +If the buffer size is lower than what is allowed in the request, the action will fail with an error similar to the one below: + +```text +error: RPC failed; curl 18 transfer closed with outstanding read data remaining +fatal: The remote end hung up unexpectedly +fatal: early EOF +fatal: index-pack failed +``` + +This can be fixed by increasing the existing `http.postBuffer` value to one greater than the repository size. For example, if `git clone` fails when cloning a 500M repository, the solution will be to set `http.postBuffer` to `524288000` so that the request only starts buffering after the first 524288000 bytes. + +NOTE: **Note:** +The default value of `http.postBuffer`, 1 MiB, is applied if the setting is not configured. + +```sh +git config http.postBuffer 524288000 +``` + [SSH troubleshooting]: ../../ssh/README.md#troubleshooting "SSH Troubleshooting" [Broken-Pipe]: https://stackoverflow.com/questions/19120120/broken-pipe-when-pushing-to-git-repository/36971469#36971469 "StackOverflow: 'Broken pipe when pushing to Git repository'" diff --git a/doc/topics/git/useful_git_commands.md b/doc/topics/git/useful_git_commands.md index cfe19c89618..abd06b95b1e 100644 --- a/doc/topics/git/useful_git_commands.md +++ b/doc/topics/git/useful_git_commands.md @@ -167,6 +167,14 @@ With HTTPS: GIT_TRACE_PACKET=1 GIT_TRACE=2 GIT_CURL_VERBOSE=1 git clone <url> ``` +### Debugging with Git embedded traces + +Git includes a complete set of [traces for debugging Git commands](https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables#_debugging), for example: + +- `GIT_TRACE_PERFORMANCE=1`: enables tracing of performance data, showing how long each particular `git` invocation takes. +- `GIT_TRACE_SETUP=1`: enables tracing of what `git` is discovering about the repository and environment it’s interacting with. +- `GIT_TRACE_PACKET=1`: enables packet-level tracing for network operations. + ## Rebasing ### Rebase your branch onto master diff --git a/doc/university/README.md b/doc/university/README.md index 9725cb14fc5..2609bc5728f 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -196,8 +196,8 @@ The GitLab University curriculum is composed of GitLab videos, screencasts, pres ## 4. External Articles 1. [2011 WSJ article by Marc Andreessen - Software is Eating the World](https://www.wsj.com/articles/SB10001424053111903480904576512250915629460) -1. [2014 Blog post by Chris Dixon - Software eats software development](http://cdixon.org/2014/04/13/software-eats-software-development/) -1. [2015 Venture Beat article - Actually, Open Source is Eating the World](http://venturebeat.com/2015/12/06/its-actually-open-source-software-thats-eating-the-world/) +1. [2014 Blog post by Chris Dixon - Software eats software development](https://cdixon.org/2014/04/13/software-eats-software-development) +1. [2015 Venture Beat article - Actually, Open Source is Eating the World](https://venturebeat.com/2015/12/06/its-actually-open-source-software-thats-eating-the-world/) ## 5. Resources for GitLab Team Members diff --git a/doc/university/training/end-user/README.md b/doc/university/training/end-user/README.md index 4c86aedff8f..be9db9229cd 100644 --- a/doc/university/training/end-user/README.md +++ b/doc/university/training/end-user/README.md @@ -4,7 +4,7 @@ comments: false # Training -This training material is the markdown used to generate training slides +This training material is the Markdown used to generate training slides which can be found at [End User Slides](https://gitlab-org.gitlab.io/end-user-training-slides/#/) through it's [RevealJS](https://gitlab.com/gitlab-org/end-user-training-slides) project. diff --git a/doc/university/training/index.md b/doc/university/training/index.md index 61fde9d8336..69f82392027 100644 --- a/doc/university/training/index.md +++ b/doc/university/training/index.md @@ -5,7 +5,7 @@ type: index # GitLab Training Material -All GitLab training material is stored in markdown format. Slides are +All GitLab training material is stored in Markdown format. Slides are generated using [Deskset](https://www.deckset.com/). All training material is open to public contribution. diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 7314e34666d..b00fc5d90cf 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -12,13 +12,7 @@ You can select the tag in the version dropdown in the top left corner of GitLab ### 0. Backup -It's useful to make a backup just in case things go south: - -```bash -cd /home/git/gitlab - -sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production -``` +It's useful to make a backup just in case things go south. Depending on the installation method, backup commands vary, see the [backing up and restoring GitLab](../raketasks/backup_restore.md#creating-a-backup-of-the-gitlab-system) documentation. ### 1. Stop server diff --git a/doc/update/upgrading_from_ce_to_ee.md b/doc/update/upgrading_from_ce_to_ee.md index b553b4aa405..52a65a89cbf 100644 --- a/doc/update/upgrading_from_ce_to_ee.md +++ b/doc/update/upgrading_from_ce_to_ee.md @@ -59,17 +59,22 @@ sudo -u git -H git checkout EE_BRANCH ```sh cd /home/git/gitlab -# MySQL installations (note: the line below states '--without postgres') -sudo -u git -H bundle install --without postgres development test --deployment +sudo -u git -H bundle install --deployment --without development test mysql aws kerberos -# PostgreSQL installations (note: the line below states '--without mysql') -sudo -u git -H bundle install --without mysql development test --deployment +# Optional: clean up old gems +sudo -u git -H bundle clean # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production -# Clean up assets and cache -sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production +# Compile GetText PO files +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production NODE_OPTIONS="--max_old_space_size=4096" + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production ``` ### 4. Install `gitlab-elasticsearch-indexer` (optional) **(STARTER ONLY)** diff --git a/doc/user/admin_area/appearance.md b/doc/user/admin_area/appearance.md index 1fea6ab8b02..b9eb9e2a731 100644 --- a/doc/user/admin_area/appearance.md +++ b/doc/user/admin_area/appearance.md @@ -46,8 +46,8 @@ of your GitLab instance. These messages will appear on all projects and pages of instance, including the sign in / sign up page. The default color is white text on an orange background, but this can be customized by clicking on **Customize colors**. -Limited [markdown](../markdown.md) is supported, such as bold, italics, and links, for -example. Other markdown features, including lists, images and quotes, are not supported, +Limited [Markdown](../markdown.md) is supported, such as bold, italics, and links, for +example. Other Markdown features, including lists, images and quotes, are not supported, as the header and footer messages can only be a single line.  @@ -61,7 +61,7 @@ to activate it in the GitLab instance. ## Sign in / Sign up pages You can replace the default message on the sign in / sign up page with your own message -and logo. You can make full use of [markdown](../markdown.md) in the description: +and logo. You can make full use of [Markdown](../markdown.md) in the description:  @@ -81,7 +81,7 @@ You can add also add a [customized help message](settings/help_page.md) below th ## New project pages You can add a new project guidelines message to the **New project page** within GitLab. -You can make full use of [markdown](../markdown.md) in the description: +You can make full use of [Markdown](../markdown.md) in the description:  diff --git a/doc/user/admin_area/broadcast_messages.md b/doc/user/admin_area/broadcast_messages.md index b0491499f88..bc51552603d 100644 --- a/doc/user/admin_area/broadcast_messages.md +++ b/doc/user/admin_area/broadcast_messages.md @@ -22,6 +22,7 @@ To add a broadcast message: 1. Navigate to the **Admin Area > Messages** page. 1. Add the text for the message to the **Message** field. Markdown and emoji are supported. 1. If required, click the **Customize colors** link to edit the background color and font color of the message. +1. If required, add a **Target Path** to only show the broadcast message on URLs matching that path. You can use the wildcard character `*` to match multiple URLs, for example `/users/*/issues`. 1. Select a date for the message to start and end. 1. Click the **Add broadcast message** button. diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md new file mode 100644 index 00000000000..30ebbb5b6db --- /dev/null +++ b/doc/user/admin_area/credentials_inventory.md @@ -0,0 +1,19 @@ +# Credentials inventory **(ULTIMATE ONLY)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/20912) in GitLab 12.6. + +## Overview + +GitLab administrators are responsible for the overall security of their instance. To assist, GitLab provides a Credentials inventory to keep track of all the credentials that can be used to access their self-managed instance. + +Using Credentials inventory, GitLab administrators can see all the personal access tokens and SSH keys that exist in their instance and: + +- Who they belong to. +- Their access scope. +- Their usage pattern. + +To access the Credentials inventory, navigate to **Admin Area > Credentials**. + +The following is an example of the Credentials inventory page: + + diff --git a/doc/user/admin_area/img/credentials_inventory_v12_6.png b/doc/user/admin_area/img/credentials_inventory_v12_6.png Binary files differnew file mode 100644 index 00000000000..ff46db61cdb --- /dev/null +++ b/doc/user/admin_area/img/credentials_inventory_v12_6.png diff --git a/doc/user/admin_area/img/minimum_password_length_settings_v12_6.png b/doc/user/admin_area/img/minimum_password_length_settings_v12_6.png Binary files differnew file mode 100644 index 00000000000..f75d9e9bb29 --- /dev/null +++ b/doc/user/admin_area/img/minimum_password_length_settings_v12_6.png diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index dbcf250bc57..fe8903a9f01 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -67,7 +67,7 @@ Omnibus installations should add this entry to `gitlab.rb`: gitlab_rails['license_file'] = "/path/to/license/file" ``` -CAUTION:: **Caution:** +CAUTION: **Caution:** These methods will only add a license at the time of installation. Use the admin area in the web ui to renew or upgrade licenses. diff --git a/doc/user/admin_area/monitoring/convdev.md b/doc/user/admin_area/monitoring/convdev.md deleted file mode 100644 index 6ad8a5a7ff0..00000000000 --- a/doc/user/admin_area/monitoring/convdev.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -redirect_to: '../../instance_statistics/convdev.md' ---- - -This document was moved to [another location](../../instance_statistics/convdev.md). diff --git a/doc/user/admin_area/monitoring/dev_ops_score.md b/doc/user/admin_area/monitoring/dev_ops_score.md new file mode 100644 index 00000000000..f8b66531f2f --- /dev/null +++ b/doc/user/admin_area/monitoring/dev_ops_score.md @@ -0,0 +1,5 @@ +--- +redirect_to: '../../instance_statistics/dev_ops_score.md' +--- + +This document was moved to [another location](../../instance_statistics/dev_ops_score.md). diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 103d7ecc573..68767efc72a 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -103,7 +103,7 @@ This check is being exempt from Rack Attack. ## Liveness DANGER: **Warning:** -In Gitlab [12.4](https://about.gitlab.com/upcoming-releases/) +In GitLab [12.4](https://about.gitlab.com/upcoming-releases/) the response body of the Liveness check was changed to match the example below. diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index e443127a8a0..9d82b3b4292 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -84,3 +84,35 @@ add the line below to `/etc/gitlab/gitlab.rb` before increasing the max attachme ``` nginx['client_max_body_size'] = "200m" ``` + +## Limiting lifetime of personal access tokens **(ULTIMATE ONLY)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/3649) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. + +Users can optionally specify an expiration date for +[personal access tokens](../../profile/personal_access_tokens.md). +This expiration date is not a requirement, and can be set to any arbitrary date. + +Since personal access tokens are the only token needed for programmatic access to GitLab, +organizations with security requirements may want to enforce more protection to require +regular rotation of these tokens. + +### Setting a limit + +Only a GitLab administrator can set a limit. Leaving it empty means +there are no restrictions. + +To set a limit on how long personal access tokens are valid: + +1. Navigate to **Admin Area > Settings > General**. +1. Expand the **Account and limit** section. +1. Fill in the **Maximun allowable lifetime for personal access tokens (days)** field. +1. Click **Save changes**. + +Once a lifetime for personal access tokens is set, GitLab will: + +- Apply the lifetime for new personal access tokens, and require users to set an expiration date + and a date no later than the allowed lifetime. +- After three hours, revoke old tokens with no expiration date or with a lifetime longer than the + allowed lifetime. Three hours is given to allow administrators to change the allowed lifetime, + or remove it, before revocation takes place. diff --git a/doc/user/admin_area/settings/protected_paths.md b/doc/user/admin_area/settings/protected_paths.md index 21c8d79b138..5d2548890e3 100644 --- a/doc/user/admin_area/settings/protected_paths.md +++ b/doc/user/admin_area/settings/protected_paths.md @@ -14,7 +14,8 @@ GitLab protects the following paths with Rack Attack by default: '/users', '/users/confirmation', '/unsubscribes/', -'/import/github/personal_access_token' +'/import/github/personal_access_token', +'/admin/session' ``` GitLab responds with HTTP status code `429` to POST requests at protected paths @@ -59,18 +60,14 @@ NOTE: **Note:** If Omnibus settings are present, applications settings will be a To migrate from Omnibus GitLab 12.3 and earlier settings: -1. Disable the Protected Paths throttle from Omnibus, by changing `rack_attack_enabled` value to `false` on [`rack_attack.rb.erb`](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/rack_attack.rb.erb#L18): - - ```ruby - rack_attack_enabled = false - ``` - 1. Customize and enable your protected paths settings by following [Configure using GitLab UI](#configure-using-gitlab-ui) section. -1. Restart GitLab: +1. SSH into your frontend nodes and add to `/etc/gitlab/gitlab.rb`: - ```bash - sudo gitlab-ctl restart + ```ruby + gitlab_rails['rack_attack_admin_area_protected_paths_enabled'] = true ``` +1. [Reconfigure GitLab](../../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + That's it. Protected paths throttle are now managed by GitLab admin settings. diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index ff26a1ee09c..851a984c285 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -19,6 +19,13 @@ their email address before they are allowed to sign in.  +## Minimum password length limit + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/20661) in GitLab 12.6 + +You can [change](../../../security/password_length_limits.md#modify-minimum-password-length-using-gitlab-ui) +the minimum number of characters a user must have in their password using the GitLab UI. + ## Whitelist email domains > [Introduced][ce-598] in GitLab 7.11.0 diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index 73406fd5037..74398128593 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -14,8 +14,11 @@ To access the visibility and access control options: ## Default branch protection -Branch protection specifies which roles can push to branches and which roles can delete -branches. +This global option defines the branch protection that applies to every repository's default branch. [Branch protection](../../project/protected_branches.md) specifies which roles can push to branches and which roles can delete +branches. In this case _Default_ refers to a repository's default branch, which in most cases is _master_. +branches. "Default" in this case refers to a repository's default branch, which in most cases would be "master". + +This setting applies only to each repositories' default branch. To protect other branches, you must configure branch protection in repository. For details, see [Protected Branches](../../project/protected_branches.md). To change the default branch protection: @@ -35,7 +38,7 @@ To change the default project creation protection: For more details, see [Default project-creation level](../../group/index.md#default-project-creation-level). -## Default project deletion protection +## Default project deletion protection **(PREMIUM ONLY)** By default, a project can be deleted by anyone with the **Owner** role, either at the project or group level. @@ -45,6 +48,17 @@ To ensure only admin users can delete projects: 1. Check the **Default project deletion protection** checkbox. 1. Click **Save changes**. +## Project deletion adjourned period **(PREMIUM ONLY)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/32935) in GitLab 12.6. + +By default, project marked for deletion will be permanently removed after 7 days. This period may be changed. + +To change this period: + +1. Select the desired option. +1. Click **Save changes**. + ## Default project visibility To set the default visibility levels for new projects: diff --git a/doc/user/analytics/cycle_analytics.md b/doc/user/analytics/cycle_analytics.md index c75f101b0e1..796cae70803 100644 --- a/doc/user/analytics/cycle_analytics.md +++ b/doc/user/analytics/cycle_analytics.md @@ -153,6 +153,27 @@ A few notes: cycles, calculate their median time and the result is what the dashboard of Cycle Analytics is showing. +## Days to completion chart + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/21631) in GitLab 12.6. + +This chart visually depicts the total number of days it takes for cycles to be completed. + +This chart uses the global page filters for displaying data based on the selected +group, projects, and timeframe. In addition, specific stages can be selected +from within the chart itself. + +### Enabling chart + +By default, this chart is disabled for self-managed instances. To enable it, ask an +administrator with Rails console access to run the following: + +```ruby +Feature.enable(:cycle_analytics_scatterplot_enabled) +``` + +This chart is enabled by default on GitLab.com. + ## Permissions The current permissions on the Project Cycle Analytics dashboard are: diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md new file mode 100644 index 00000000000..c8d7edff2d6 --- /dev/null +++ b/doc/user/application_security/configuration/index.md @@ -0,0 +1,27 @@ +--- +type: reference, howto +--- + +# Security Configuration **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/20711) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. + +## Overview + +The security configuration page displays the configuration state of each of the security +features and can be accessed through a project's sidebar nav. + + + +The page uses the project's latest default branch [CI pipeline](../../../ci/pipelines.md) to determine the configuration +state of each feature. If a job with the expected security report artifact exists in the pipeline, +the feature is considered configured. + +NOTE: **Note:** if the latest pipeline used [Auto DevOps](../../../topics/autodevops/index.md), +all security features will be configured by default. + +## Limitations + +It is not possible to enable or disable a feature using the configuration page. +However, instructions on how to enable or disable a feature can be found through +the links next to each feature on that page. diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 931755c6305..08242b3c65b 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -127,7 +127,7 @@ If you want to whitelist specific vulnerabilities, you'll need to: [overriding the Container Scanning template](#overriding-the-container-scanning-template) section of this document. 1. Define the whitelisted vulnerabilities in a YAML file named `clair-whitelist.yml` which must use the format described in the [following whitelist example file](https://github.com/arminc/clair-scanner/blob/v12/example-whitelist.yaml). - 1. Add the `clair-whitelist.yml` file to the git repository of your project + 1. Add the `clair-whitelist.yml` file to the Git repository of your project ### Overriding the Container Scanning template @@ -219,6 +219,94 @@ build_latest_vulnerabilities: The above template will work for a GitLab Docker registry running on a local installation, however, if you're using a non-GitLab Docker registry, you'll need to change the `$CI_REGISTRY` value and the `docker login` credentials to match the details of your local registry. +## Reports JSON format + +CAUTION: **Caution:** +The JSON report artifacts are not a public API of Container Scanning and their format may change in the future. + +The Container Scanning tool emits a JSON report file. Here is an example of the report structure with all important parts of +it highlighted: + +```json-doc +{ + "version": "2.3", + "vulnerabilities": [ + { + "category": "container_scanning", + "message": "CVE-2019-3462 in apt", + "description": "Incorrect sanitation of the 302 redirect field in HTTP transport method of apt versions 1.4.8 and earlier can lead to content injection by a MITM attacker, potentially leading to remote code execution on the target machine.", + "cve": "debian:9:apt:CVE-2019-3462", + "severity": "High", + "confidence": "Unknown", + "solution": "Upgrade apt from 1.4.8 to 1.4.9", + "scanner": { + "id": "klar", + "name": "klar" + }, + "location": { + "dependency": { + "package": { + "name": "apt" + }, + "version": "1.4.8" + }, + "operating_system": "debian:9", + "image": "registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256:bc09fe2e0721dfaeee79364115aeedf2174cce0947b9ae5fe7c33312ee019a4e" + }, + "identifiers": [ + { + "type": "cve", + "name": "CVE-2019-3462", + "value": "CVE-2019-3462", + "url": "https://security-tracker.debian.org/tracker/CVE-2019-3462" + } + ], + "links": [ + { + "url": "https://security-tracker.debian.org/tracker/CVE-2019-3462" + } + ] + } + ], + "remediations": [ + ] +} +``` + +Here is the description of the report file structure nodes and their meaning. All fields are mandatory to be present in +the report JSON unless stated otherwise. Presence of optional fields depends on the underlying analyzers being used. + +| Report JSON node | Description | +|------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `version` | Report syntax version used to generate this JSON. | +| `vulnerabilities` | Array of vulnerability objects. | +| `vulnerabilities[].category` | Where this vulnerability belongs (SAST, Container Scanning etc.). For Container Scanning, it will always be `container_scanning`. | +| `vulnerabilities[].message` | A short text that describes the vulnerability, it may include occurrence's specific information. Optional. | +| `vulnerabilities[].description` | A long text that describes the vulnerability. Optional. | +| `vulnerabilities[].cve` | A fingerprint string value that represents a concrete occurrence of the vulnerability. It's used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | +| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Undefined` (an analyzer has not provided this info), `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) only provides the following levels: `Unknown`, `Low`, `Medium`, `High`, `Critical`. | +| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Undefined` (an analyzer has not provided this info), `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) does not provide a confidence level, so this value is currently hardcoded to `Unknown`. | +| `vulnerabilities[].solution` | Explanation of how to fix the vulnerability. Optional. | +| `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | +| `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | +| `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | +| `vulnerabilities[].location` | A node that tells where the vulnerability is located. | +| `vulnerabilities[].location.dependency` | A node that describes the dependency of a project where the vulnerability is located. | +| `vulnerabilities[].location.dependency.package` | A node that provides the information on the package where the vulnerability is located. | +| `vulnerabilities[].location.dependency.package.name` | Name of the package where the vulnerability is located. | +| `vulnerabilities[].location.dependency.version` | Version of the vulnerable package. Optional. | +| `vulnerabilities[].location.operating_system` | The operating system that contains the vulnerable package. | +| `vulnerabilities[].location.image` | The Docker image that was analyzed. Optional. | +| `vulnerabilities[].identifiers` | An ordered array of references that identify a vulnerability on internal or external DBs. | +| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`). | +| `vulnerabilities[].identifiers[].name` | Name of the identifier for display purpose. | +| `vulnerabilities[].identifiers[].value` | Value of the identifier for matching purpose. | +| `vulnerabilities[].identifiers[].url` | URL to identifier's documentation. Optional. | +| `vulnerabilities[].links` | An array of references to external documentation pieces or articles that describe the vulnerability further. Optional. | +| `vulnerabilities[].links[].name` | Name of the vulnerability details link. Optional. | +| `vulnerabilities[].links[].url` | URL of the vulnerability details document. Optional. | +| `remediations` | Not supported yet. | + ## Troubleshooting ### docker: Error response from daemon: failed to copy xattrs diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index d285b5ff585..3a8a81f5f57 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -85,7 +85,7 @@ There are two ways to define the URL to be scanned by DAST: 1. Add it in an `environment_url.txt` file at the root of your project. This is great for testing in dynamic environments. In order to run DAST against - an app that is dynamically created during a Gitlab CI pipeline, have the app + an app that is dynamically created during a GitLab CI pipeline, have the app persist its domain in an `environment_url.txt` file, and DAST will automatically parse that file to find its scan target. You can see an [example](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) @@ -228,7 +228,7 @@ server { ###### Apache Apache can also be used as a [reverse proxy](https://httpd.apache.org/docs/2.4/mod/mod_proxy.html) -to add the Gitlab-DAST-Permission [header](https://httpd.apache.org/docs/current/mod/mod_headers.html). +to add the `Gitlab-DAST-Permission` [header](https://httpd.apache.org/docs/current/mod/mod_headers.html). To do so, add the following lines to `httpd.conf`: diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 0e46052b0bd..01feaaac423 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -55,7 +55,7 @@ The following languages and dependency managers are supported. | Language (package managers) | Supported | Scan tool(s) | |----------------------------- | --------- | ------------ | -| Java ([Gradle](https://gradle.org/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab/issues/13075 "Dependency Scanning for Gradle" )) | not available | +| Java ([Gradle](https://gradle.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | Java ([Maven](https://maven.apache.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | JavaScript ([npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/en/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | | Go ([Golang](https://golang.org/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab/issues/7132 "Dependency Scanning for Go")) | not available | @@ -64,6 +64,7 @@ The following languages and dependency managers are supported. | Python ([Pipfile](https://pipenv.kennethreitz.org/en/latest/basics/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab/issues/11756 "Pipfile.lock support for Dependency Scanning"))| not available | | Python ([poetry](https://poetry.eustace.io/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab/issues/7006 "Support Poetry in Dependency Scanning")) | not available | | Ruby ([gem](https://rubygems.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | +| Scala ([sbt](https://www.scala-sbt.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | ## Configuration @@ -127,23 +128,26 @@ dependency_scanning: Dependency Scanning can be [configured](#customizing-the-dependency-scanning-settings) using environment variables. -| Environment variable | Description | Example usage | -| --------------------------------------- | ----------- | ------------- | -| `DS_ANALYZER_IMAGES` | Comma separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). | | -| `DS_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | | -| `DS_ANALYZER_IMAGE_TAG` | Override the Docker tag of the official default images. Read more about [customizing analyzers](analyzers.md). | | -| `DS_PYTHON_VERSION` | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12296) in GitLab 12.1)| | -| `DS_PIP_DEPENDENCY_PATH` | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12412) in GitLab 12.2) | | -| `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | | -| `DS_DISABLE_DIND` | Disable Docker in Docker and run analyzers [individually](#disabling-docker-in-docker-for-dependency-scanning).| | -| `DS_PULL_ANALYZER_IMAGES` | Pull the images from the Docker registry (set to `0` to disable). | | -| `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, file or folder paths. Parent directories will also match patterns. | `DS_EXCLUDED_PATHS=doc,spec` | -| `DS_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | | -| `DS_PULL_ANALYZER_IMAGE_TIMEOUT` | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | | -| `DS_RUN_ANALYZER_TIMEOUT` | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | | -| `PIP_INDEX_URL` | Base URL of Python Package Index (default `https://pypi.org/simple`). | | -| `PIP_EXTRA_INDEX_URL` | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma separated. | | -| `MAVEN_CLI_OPTS` | List of command line arguments that will be passed to the maven analyzer during the project's build phase (see example for [using private repos](#using-private-maven-repos)). | | +| Environment variable | Description | +| --------------------------------------- | ----------- | +| `DS_ANALYZER_IMAGES` | Comma separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). | +| `DS_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | +| `DS_ANALYZER_IMAGE_TAG` | Override the Docker tag of the official default images. Read more about [customizing analyzers](analyzers.md). | +| `DS_PYTHON_VERSION` | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12296) in GitLab 12.1)| +| `DS_PIP_VERSION` | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the docker image is used. | +| `DS_PIP_DEPENDENCY_PATH` | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12412) in GitLab 12.2) | +| `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | +| `DS_DISABLE_DIND` | Disable Docker in Docker and run analyzers [individually](#disabling-docker-in-docker-for-dependency-scanning).| +| `DS_PULL_ANALYZER_IMAGES` | Pull the images from the Docker registry (set to `0` to disable). | +| `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, file or folder paths (e.g., `doc,spec`). Parent directories will also match patterns. | +| `DS_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | +| `DS_PULL_ANALYZER_IMAGE_TIMEOUT` | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | +| `DS_RUN_ANALYZER_TIMEOUT` | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | +| `PIP_INDEX_URL` | Base URL of Python Package Index (default `https://pypi.org/simple`). | +| `PIP_EXTRA_INDEX_URL` | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma separated. | +| `PIP_REQUIREMENTS_FILE` | Pip requirements file to be scanned. | +| `MAVEN_CLI_OPTS` | List of command line arguments that will be passed to the maven analyzer during the project's build phase (see example for [using private repos](#using-private-maven-repos)). | +| `BUNDLER_AUDIT_UPDATE_DISABLED` | Disable automatic updates for the `bundler-audit` analyzer (default: `"false"`). Useful if you're running Dependency Scanning in an offline, air-gapped environment.| ### Using private Maven repos diff --git a/doc/user/application_security/img/security_configuration_page_v12_6.png b/doc/user/application_security/img/security_configuration_page_v12_6.png Binary files differnew file mode 100644 index 00000000000..d838b648c1f --- /dev/null +++ b/doc/user/application_security/img/security_configuration_page_v12_6.png diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index b79edc9d5a8..a42cf7f09ff 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -18,10 +18,11 @@ SAST supports the following official analyzers: - [`eslint`](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) (ESLint (JavaScript and React)) - [`flawfinder`](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) (Flawfinder) - [`gosec`](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) (Gosec) +- [`kubesec`](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) (Kubesec) - [`nodejs-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan) (NodeJsScan) - [`phpcs-security-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) (PHP CS security-audit) - [`pmd-apex`](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) (PMD (Apex only)) -- [`secrets`](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) (Secrets (Gitleaks, TruffleHog & Diffence secret detectors)) +- [`secrets`](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) (Secrets (Gitleaks & TruffleHog secret detectors)) - [`security-code-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) (Security Code Scan (.NET)) - [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) (Sobelow (Elixir Phoenix)) - [`spotbugs`](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) (SpotBugs with the Find Sec Bugs plugin (Ant, Gradle and wrapper, Grails, Maven and wrapper, SBT)) @@ -116,24 +117,24 @@ Custom analyzers are not spawned automatically when [Docker In Docker](index.md# ## Analyzers Data -| Property \ Tool | Apex | Bandit | Brakeman | ESLint security | Find Sec Bugs | Flawfinder | Go AST Scanner | NodeJsScan | Php CS Security Audit | Security code Scan (.NET) | TSLint Security | Sobelow | -| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :-------------: | :----------------: | -| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | -| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | -| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | -| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | -| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | -| External id (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | +| Property \ Tool | Apex | Bandit | Brakeman | ESLint security | Find Sec Bugs | Flawfinder | Go AST Scanner | Kubesec Scanner | NodeJsScan | Php CS Security Audit | Security code Scan (.NET) | Sobelow | TSLint Security | +| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :----------------: | :-------------: | +| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | ✓ | +| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | +| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | ✓ | +| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | +| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | +| External id (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | +| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | +| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | - ✓ => we have that data - ⚠ => we have that data but it's partially reliable, or we need to extract it from unstructured content diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index f427f33c8c2..95027e99c00 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -73,6 +73,7 @@ The following table shows which languages, package managers and frameworks are s | Groovy ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Ant, Maven, SBT) | | Java ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (Ant, SBT) | | JavaScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 | +| Kubernetes manifests | [Kubesec](https://github.com/controlplaneio/kubesec) | 12.6 | | Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1 | | PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8 | | Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/PyCQA/bandit) | 10.3 | @@ -185,6 +186,22 @@ variables: This will create individual `<analyzer-name>-sast` jobs for each analyzer that runs in your CI/CD pipeline. +#### Enabling kubesec analyzer + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12752) in GitLab Ultimate 12.6. + +When [Docker in Docker is disabled](#disabling-docker-in-docker-for-sast), +you will need to set `SCAN_KUBERNETES_MANIFESTS` to `"true"` to enable the +kubesec analyzer. In `.gitlab-ci.yml`, define: + +```yaml +include: + template: SAST.gitlab-ci.yml + +variables: + SCAN_KUBERNETES_MANIFESTS: "true" +``` + ### Available variables SAST can be [configured](#customizing-the-sast-settings) using environment variables. @@ -206,14 +223,14 @@ The following are Docker image-related variables. Some analyzers make it possible to filter out vulnerabilities under a given threshold. -| Environment variable | Default value | Description | Example usage | -|----------------------|---------------|-------------|---| -| `SAST_BANDIT_EXCLUDED_PATHS` | - | comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html) | | -| `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | | -| `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | | -| `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | | -| `SAST_GOSEC_LEVEL` | 0 | Ignore gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | | -| `SAST_EXCLUDED_PATHS` | - | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, file or folder paths. Parent directories will also match patterns. | `SAST_EXCLUDED_PATHS=doc,spec` | +| Environment variable | Default value | Description | +|----------------------|---------------|-------------| +| `SAST_BANDIT_EXCLUDED_PATHS` | - | comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html) | +| `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | +| `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | +| `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | +| `SAST_GOSEC_LEVEL` | 0 | Ignore gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | +| `SAST_EXCLUDED_PATHS` | - | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, file or folder paths (e.g., `doc,spec` ). Parent directories will also match patterns. | #### Timeouts @@ -232,19 +249,20 @@ Timeout variables are not applicable for setups with [disabled Docker In Docker] Some analyzers can be customized with environment variables. -| Environment variable | Analyzer | Description | -|-------------------------|----------|----------| -| `ANT_HOME` | spotbugs | The `ANT_HOME` environment variable. | -| `ANT_PATH` | spotbugs | Path to the `ant` executable. | -| `GRADLE_PATH` | spotbugs | Path to the `gradle` executable. | -| `JAVA_OPTS` | spotbugs | Additional arguments for the `java` executable. | -| `JAVA_PATH` | spotbugs | Path to the `java` executable. | -| `SAST_JAVA_VERSION` | spotbugs | Which Java version to use. Supported versions are `8` and `11`. Defaults to `8`. | -| `MAVEN_CLI_OPTS` | spotbugs | Additional arguments for the `mvn` or `mvnw` executable. | -| `MAVEN_PATH` | spotbugs | Path to the `mvn` executable. | -| `MAVEN_REPO_PATH` | spotbugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). | -| `SBT_PATH` | spotbugs | Path to the `sbt` executable. | -| `FAIL_NEVER` | spotbugs | Set to `1` to ignore compilation failure. | +| Environment variable | Analyzer | Description | +|-----------------------------|----------|-------------| +| `SCAN_KUBERNETES_MANIFESTS` | kubesec | Set to `"true"` to scan Kubernetes manifests when [Docker in Docker](#disabling-docker-in-docker-for-sast) is disabled. | +| `ANT_HOME` | spotbugs | The `ANT_HOME` environment variable. | +| `ANT_PATH` | spotbugs | Path to the `ant` executable. | +| `GRADLE_PATH` | spotbugs | Path to the `gradle` executable. | +| `JAVA_OPTS` | spotbugs | Additional arguments for the `java` executable. | +| `JAVA_PATH` | spotbugs | Path to the `java` executable. | +| `SAST_JAVA_VERSION` | spotbugs | Which Java version to use. Supported versions are `8` and `11`. Defaults to `8`. | +| `MAVEN_CLI_OPTS` | spotbugs | Additional arguments for the `mvn` or `mvnw` executable. | +| `MAVEN_PATH` | spotbugs | Path to the `mvn` executable. | +| `MAVEN_REPO_PATH` | spotbugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). | +| `SBT_PATH` | spotbugs | Path to the `sbt` executable. | +| `FAIL_NEVER` | spotbugs | Set to `1` to ignore compilation failure. | #### Custom environment variables @@ -340,7 +358,7 @@ it highlighted: } ``` -Here is the description of the report file structure nodes and their meaning. All fields are mandatory to be present in +Here is the description of the report file structure nodes and their meaning. All fields are mandatory in the report JSON unless stated otherwise. Presence of optional fields depends on the underlying analyzers being used. | Report JSON node | Function | diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_4.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_4.png Binary files differdeleted file mode 100644 index 682dcbec63f..00000000000 --- a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_4.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png Binary files differnew file mode 100644 index 00000000000..c93a3ce8c35 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_3.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_3.png Binary files differdeleted file mode 100644 index 09979ba99b3..00000000000 --- a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_3.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png Binary files differnew file mode 100644 index 00000000000..670c90d10a3 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v12_6.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 7eb0d649648..bb2bf0b7806 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -42,7 +42,7 @@ At the pipeline level, the Security Dashboard displays the vulnerabilities prese Visit the page for any pipeline which has run any of the [supported reports](#supported-reports). Click the **Security** tab to view the Security Dashboard. - + ## Project Security Dashboard @@ -71,12 +71,12 @@ Once you're on the dashboard, at the top you should see a series of filters for: - Report type - Project -To the right of the filters, you should see a **Hide dismissed** toggle button ([available for GitLab.com Gold, planned for GitLab Ultimate 12.6](https://gitlab.com/gitlab-org/gitlab/issues/9102)). +To the right of the filters, you should see a **Hide dismissed** toggle button. NOTE: **Note:** The dashboard only shows projects with [security reports](#supported-reports) enabled in a group. - + Selecting one or more filters will filter the results in this page. Disabling the **Hide dismissed** toggle button will let you also see vulnerabilities that have been dismissed. @@ -97,6 +97,17 @@ vulnerabilities your projects had at various points in time. You can filter amon 90 days, with the default being 90. Hover over the chart to get more details about the open vulnerabilities at a specific time. +Below the timeline chart is a list of projects, grouped and sorted by the severity of the vulnerability found: + +- F: 1 or more "critical" +- D: 1 or more "high" or "unknown" +- C: 1 or more "medium" +- B: 1 or more "low" +- A: 0 vulnerabilities + +Projects with no vulnerability tests configured will not appear in the list. Additionally, dismissed +vulnerabilities are not included either. + Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). ## Keeping the dashboards up to date diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index 7ee1650f698..95dbe7d3b51 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -267,13 +267,19 @@ This feature: kubectl -n gitlab-managed-apps exec -it $(kubectl get pods -n gitlab-managed-apps | grep 'ingress-controller' | awk '{print $1}') -- tail -f /var/log/modsec/audit.log ``` -There is a small performance overhead by enabling `modsecurity`. However, if this is -considered significant for your application, you can toggle the feature flag back to -false by running the following command within the Rails console: - -```ruby -Feature.disable(:ingress_modsecurity) -``` +There is a small performance overhead by enabling `modsecurity`. If this is +considered significant for your application, you can either: + +- Disable ModSecurity's rule engine for your deployed application by setting + [the deployment variable](../../topics/autodevops/index.md) + `AUTO_DEVOPS_MODSECURITY_SEC_RULE_ENGINE` to `Off`. This will prevent ModSecurity from + processing any requests for the given application or environment. +- Toggle the feature flag to false by running the following command within your + instance's Rails console: + + ```ruby + Feature.disable(:ingress_modsecurity) + ``` Once disabled, you must [uninstall](#uninstalling-applications) and reinstall your Ingress application for the changes to take effect. @@ -429,6 +435,131 @@ administrator to run following command within a Rails console: Feature.enable(:enable_cluster_application_crossplane) ``` +## Install using GitLab CI (alpha) + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/20822) in GitLab 12.6. + +CAUTION: **Warning:** +This is an _alpha_ feature, and it is subject to change at any time without +prior notice. + +This alternative method allows users to install GitLab-managed +applications using GitLab CI. It also allows customization of the +install using Helm `values.yaml` files. + +Supported applications: + +- [Ingress](#install-ingress-using-gitlab-ci) +- [Sentry](#install-sentry-using-gitlab-ci) + +### Usage + +To install applications using GitLab CI: + +1. Connect the cluster to a [cluster management project](management_project.md). +1. In that project, add a `.gitlab-ci.yml` file with the following content: + + ```yaml + include: + - template: Managed-Cluster-Applications.gitlab-ci.yml + ``` + +1. Add a `.gitlab/managed-apps/config.yaml` file to define which + applications you would like to install. Define the `installed` key as + `true` to install the application and `false` to uninstall the + application. For example, to install Ingress: + + ```yaml + ingress: + installed: true + ``` + +1. Optionally, define `.gitlab/managed-apps/<application>/values.yaml` file to + customize values for the installed application. + +A GitLab CI pipeline will then run on the `master` branch to install the +applications you have configured. + +### Install Ingress using GitLab CI + +To install Ingress, define the `.gitlab/managed-apps/config.yaml` file +with: + +```yaml +ingress: + installed: true +``` + +Ingress will then be installed into the `gitlab-managed-apps` namespace +of your cluster. + +You can customize the installation of Ingress by defining +`.gitlab/managed-apps/ingress/values.yaml` file in your cluster +management project. Refer to the +[chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress) +for the available configuration options. + +### Install Sentry using GitLab CI + +NOTE: **Note:** +The Sentry Helm chart [recommends](https://github.com/helm/charts/blob/f6e5784f265dd459c5a77430185d0302ed372665/stable/sentry/values.yaml#L284-L285) at least 3GB of available RAM for database migrations. + +To install Sentry, define the `.gitlab/managed-apps/config.yaml` file +with: + +```yaml +sentry: + installed: true +``` + +Sentry will then be installed into the `gitlab-managed-apps` namespace +of your cluster. + +You can customize the installation of Sentry by defining +`.gitlab/managed-apps/sentry/values.yaml` file in your cluster +management project. Refer to the +[chart](https://github.com/helm/charts/tree/master/stable/sentry) +for the available configuration options. + +We recommend you pay close attention to the following configuration options: + +- `email`. Needed to invite users to your Sentry instance and to send error emails. +- `user`. Where you can set the login credentials for the default admin user. +- `postgresql`. For a PostgreSQL password that can be used when running future updates. + +NOTE: **Note:** +When upgrading it is important to provide the existing PostgreSQL password (given using the `postgresql.postgresqlPassword` key) or you will receive authentication errors. See the [PostgreSQL chart documentation](https://github.com/helm/charts/tree/master/stable/postgresql#upgrade) for more information. + +Here is an example configuration for Sentry: + +```yaml +# Admin user to create +user: + # Indicated to create the admin user or not, + # Default is true as the initial installation. + create: true + email: "<your email>" + password: "<your password>" + +email: + from_address: "<your from email>" + host: smtp + port: 25 + use_tls: false + user: "<your email username>" + password: "<your email password>" + enable_replies: false + +ingress: + enabled: true + hostname: "<sentry.example.com>" + +# Needs to be here between runs. +# See https://github.com/helm/charts/tree/master/stable/postgresql#upgrade for more info +postgresql: + postgresqlPassword: example-postgresql-password +``` + ## Upgrading applications > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/24789) in GitLab 11.8. @@ -470,6 +601,7 @@ The applications below can be uninstalled. | Knative | 12.1+ | The associated IP will be deleted and cannot be restored. | | Prometheus | 11.11+ | All data will be deleted and cannot be restored. | | Crossplane | 12.5+ | All data will be deleted and cannot be restored. | +| Sentry | 12.6+ | The PostgreSQL persistent volume will remain and should be manually removed for complete uninstall. | To uninstall an application: diff --git a/doc/user/clusters/crossplane.md b/doc/user/clusters/crossplane.md index 37210b22f6f..ee0bd4c33db 100644 --- a/doc/user/clusters/crossplane.md +++ b/doc/user/clusters/crossplane.md @@ -220,9 +220,9 @@ The Resource Classes allow you to define classes of service for a managed servic The Auto DevOps pipeline can be run with the following options: -The Environment variables, `AUTO_DEVOPS_POSTGRES_MANAGED` and `AUTO_DEVOPS_POSTGRES_MANAGED_CLASS_SELECTOR` need to be set to provision PostgresQL using Crossplane +The Environment variables, `AUTO_DEVOPS_POSTGRES_MANAGED` and `AUTO_DEVOPS_POSTGRES_MANAGED_CLASS_SELECTOR` need to be set to provision PostgreSQL using Crossplane -Alertnatively, the following options can be overridden from the values for the helm chart. +Alertnatively, the following options can be overridden from the values for the Helm chart. - `postgres.managed` set to true which will select a default resource class. The resource class needs to be marked with the annotation @@ -237,7 +237,7 @@ Alertnatively, the following options can be overridden from the values for the h The Auto DevOps pipeline should provision a PostgresqlInstance when it runs succesfully. -Verify creation of the PostgresQL Instance. +Verify creation of the PostgreSQL Instance. ```sh kubectl get postgresqlinstance @@ -286,7 +286,7 @@ serverCACertificateInstance: 41 bytes serverCACertificateSha1Fingerprint: 40 bytes ``` -## Connect to the PostgresQL instance +## Connect to the PostgreSQL instance Follow this [GCP guide](https://cloud.google.com/sql/docs/postgres/connect-kubernetes-engine) if you would like to connect to the newly provisioned Postgres database instance on CloudSQL. diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index 83b6f6fe300..57a1f46ac6e 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -55,7 +55,7 @@ To select a cluster management project to use: ### Configuring your pipeline After designating a project as the management project for the cluster, -write a [`.gitlab-ci,yml`](../../ci/yaml/README.md) in that project. For example: +write a [`.gitlab-ci.yml`](../../ci/yaml/README.md) in that project. For example: ```yaml configure cluster: diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index dcb75a19b2a..d4e485d7c32 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -386,7 +386,7 @@ from any device you're logged into. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/18008) in GitLab 11.6. As a reviewer, you're able to suggest code changes with a simple -markdown syntax in Merge Request Diff threads. Then, the +Markdown syntax in Merge Request Diff threads. Then, the Merge Request author (or other users with appropriate [permission](../permissions.md)) is able to apply these suggestions with a click, which will generate a commit in diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 5912fc8e9f9..f174b75abb6 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -14,6 +14,17 @@ Below are the fingerprints for GitLab.com's SSH host keys. | ED25519 | `2e:65:6a:c8:cf:bf:b2:8b:9a:bd:6d:9f:11:5c:12:16` | `eUXGGm1YGsMAS7vkcx6JOJdOGHPem5gQp4taiCfCLB8` | | RSA | `b6:03:0e:39:97:9e:d0:e7:24:ce:a3:77:3e:01:42:09` | `ROQFvPThGrW4RuWLoL9tq9I9zJ42fK4XywyRtbOz/EQ` | +## SSH `known_hosts` entries + +Add the following to `.ssh/known_hosts` to skip manual fingerprint +confirmation in SSH: + +``` +gitlab.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAfuCHKVTjquxvt6CM6tdG4SLp1Btn/nOeHHE5UOzRdf +gitlab.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsj2bNKTBSpIYDEGk9KxsGh3mySTRgMtXL583qmBpzeQ+jqCMRgBqB98u3z++J1sKlXHWfM9dyhSevkMwSbhoR8XIq/U0tCNyokEi/ueaBMCvbcTHhO7FcwzY92WK4Yt0aGROY5qX2UKSeOvuP4D6TPqKF1onrSzH9bx9XUf2lEdWT/ia1NEKjunUqu1xOB/StKDHMoX4/OKyIzuS0q/T1zOATthvasJFoPrAjkohTyaDUz2LN5JoH839hViyEG82yB+MjcFV5MU3N1l1QL3cVUCh93xSaua1N85qivl+siMkPGbO5xR/En4iEY6K2XPASUEMaieWVNTRCtJ4S8H+9 +gitlab.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFSMqzJeV9rUzU4kWitGjeR4PWSa29SPqJ1fVkhtj3Hw9xjLVXVYrU9QlYWrOLXBpQ6KWjbjTDTdDkoohFzgbEY= +``` + ## Mail configuration GitLab.com sends emails from the `mg.gitlab.com` domain via [Mailgun] and has @@ -63,6 +74,7 @@ Below are the current settings regarding [GitLab CI/CD](../../ci/README.md). | ----------- | ----------------- | ------------- | | Artifacts maximum size (uncompressed) | 1G | 100M | | Artifacts [expiry time](../../ci/yaml/README.md#artifactsexpire_in) | kept forever | deleted after 30 days unless otherwise specified | +| Scheduled Pipeline Cron | `*/5 * * * *` | `*/19 * * * *` | ## Repository size limit @@ -79,7 +91,7 @@ GitLab.com, CI/CD, and related services are deployed into Google Cloud Platform IP based firewall can be configured by looking up all [IP address ranges or CIDR blocks for GCP](https://cloud.google.com/compute/docs/faq#where_can_i_find_product_name_short_ip_ranges). -[Static endpoints](https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/5071) are being considered. +[Static endpoints](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/97) are being considered. ## Shared Runners @@ -95,6 +107,8 @@ installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default region of the VMs is US East1. Each instance is used only for one job, this ensures any sensitive data left on the system can't be accessed by other people their CI jobs. +The `gitlab-shared-runners-manager-X.gitlab.com` fleet of Runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They will not run untagged jobs and unlike the general fleet of shared Runners, the instances are re-used up to 40 times. + Jobs handled by the shared Runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), **will be timed out after 3 hours**, regardless of the timeout configured in a project. Check the issues [4010] and [4070] for the reference. @@ -341,19 +355,44 @@ GitLab.com: set to the default. - Does not have the user and IP rate limits settings enabled. +### Visibility settings + +On GitLab.com, projects, groups, and snippets created +As of GitLab 12.2 (July 2019), projects, groups, and snippets have the +[**Internal** visibility](../../public_access/public_access.md#internal-projects) setting [disabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/issues/12388). + +## GitLab.com Logging + +We use [Fluentd](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#fluentd) to parse our logs. Fluentd sends our logs to +[Stackdriver Logging](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#stackdriver) and [Cloud Pub/Sub](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#cloud-pubsub). +Stackdriver is used for storing logs long-term in Google Cold Storage (GCS). Cloud Pub/Sub +is used to forward logs to an [Elastic cluster](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#elastic) using [pubsubbeat](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#pubsubbeat-vms). + +You can view more information in our runbooks such as: + +- A [detailed list of what we're logging](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#what-are-we-logging) +- Our [current log retention policies](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#retention) +- A [diagram of our logging infrastructure](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#logging-infrastructure-overview) + ## GitLab.com at scale In addition to the GitLab Enterprise Edition Omnibus install, GitLab.com uses the following applications and settings to achieve scale. All settings are publicly available at [chef cookbooks](https://gitlab.com/gitlab-cookbooks). -### ELK +### Elastic Cluster -We use Elasticsearch, logstash, and Kibana for part of our monitoring solution: +We use Elasticsearch and Kibana for part of our monitoring solution: - [`gitlab-cookbooks` / `gitlab-elk` · GitLab](https://gitlab.com/gitlab-cookbooks/gitlab-elk) - [`gitlab-cookbooks` / `gitlab_elasticsearch` · GitLab](https://gitlab.com/gitlab-cookbooks/gitlab_elasticsearch) +### Fluentd + +We use Fluentd to unify our GitLab logs: + +- [`gitlab-cookbooks` / `gitlab_fluentd` · GitLab](https://gitlab.com/gitlab-cookbooks/gitlab_fluentd) + ### Prometheus Prometheus complete our monitoring stack: @@ -393,11 +432,3 @@ High Performance TCP/HTTP Load Balancer: [unicorn-worker-killer]: https://rubygems.org/gems/unicorn-worker-killer "unicorn-worker-killer" [4010]: https://gitlab.com/gitlab-com/infrastructure/issues/4010 "Find a good value for maximum timeout for Shared Runners" [4070]: https://gitlab.com/gitlab-com/infrastructure/issues/4070 "Configure per-runner timeout for shared-runners-manager-X on GitLab.com" - -## Group and project settings - -On GitLab.com, projects, groups, and snippets created -after July 2019 have the `Internal` visibility setting disabled. - -You can read more about the change in the -[relevant issue](https://gitlab.com/gitlab-org/gitlab/issues/12388). diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 1fe456902a2..2b36c3bdf5b 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -75,6 +75,21 @@ NOTE: **Note:** If you [install applications](#installing-applications) on your cluster, GitLab will create the resources required to run these even if you have chosen to manage your own cluster. +### Clearing the cluster cache + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/31759) in GitLab 12.6. + +If you choose to allow GitLab to manage your cluster for you, GitLab stores a cached +version of the namespaces and service accounts it creates for your projects. If you +modify these resources in your cluster manually, this cache can fall out of sync with +your cluster, which can cause deployment jobs to fail. + +To clear the cache: + +1. Navigate to your group’s **Kubernetes** page, and select your cluster. +1. Expand the **Advanced settings** section. +1. Click **Clear cluster cache**. + ## Base domain > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/24580) in GitLab 11.8. diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 5f45a462f94..ad16aaa34ff 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -431,6 +431,23 @@ To enable this feature: 1. Expand the **Permissions, LFS, 2FA** section, and select **Disable email notifications**. 1. Click **Save changes**. +#### Disabling group mentions + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/21301) in GitLab 12.6. + +You can prevent users from being added to a conversation and getting notified when +anyone mentions a group in which those users are members. + +Groups with disabled mentions are visualized accordingly in the autocompletion dropdown. + +This is particularly helpful for groups with a large number of users. + +To enable this feature: + +1. Navigate to the group's **Settings > General** page. +1. Expand the **Permissions, LFS, 2FA** section, and select **Disable group mentions**. +1. Click **Save changes**. + ### Advanced settings - **Projects**: View all projects within that group, add members to each project, diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 6fd56414796..5fe2d0da5c8 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -2,19 +2,23 @@ type: reference, howto --- -# SAML SSO for GitLab.com Groups **(SILVER ONLY)** +# SAML SSO for GitLab.com groups **(SILVER ONLY)** > Introduced in [GitLab.com Silver](https://about.gitlab.com/pricing/) 11.0. -NOTE: **Note:** -This topic is for SAML on GitLab.com Silver tier and above. For SAML on self-managed GitLab instances, see [SAML OmniAuth Provider](../../../integration/saml.md). - SAML on GitLab.com allows users to be automatically added to a group, and then allows those users to sign into GitLab.com. Users should already have an account on the GitLab instance, or can create one when logging in for the first time. User synchronization for GitLab.com is partially supported using [SCIM](scim_setup.md). -NOTE: **Note:** -SAML SSO for GitLab.com groups does not sync users between providers without using SCIM. If a group is not using SCIM, group Owners will still need to manage user accounts (for example, removing users when necessary). +## Important notes + +Note the following: + +- This topic is for SAML on GitLab.com Silver tier and above. For SAML on self-managed GitLab + instances, see [SAML OmniAuth Provider](../../../integration/saml.md). +- SAML SSO for GitLab.com groups requires SCIM to sync users between providers. If a + group is not using SCIM, group Owners will still need to manage user accounts (for example, + removing users when necessary). ## Configuring your Identity Provider @@ -68,16 +72,17 @@ When this option is enabled: - All existing and new users in the group will be required to log in via the SSO URL associated with the group. - On successfully authenticating, GitLab will prompt the user to create a new, dedicated account using the email address received from the configured identity provider. -- After the group managed account has been created, group activity will require the use of this user account. +- After the group-managed account has been created, group activity will require the use of this user account. -Since use of the group managed account requires the use of SSO, users of group managed accounts will lose access to these accounts when they are no longer able to authenticate with the connected identity provider. In the case of an offboarded employee who has been removed from your identity provider: +Since use of the group-managed account requires the use of SSO, users of group-managed accounts will lose access to these accounts when they are no longer able to authenticate with the connected identity provider. In the case of an offboarded employee who has been removed from your identity provider: - The user will be unable to access the group (their credentials will no longer work on the identity provider when prompted to SSO). - Contributions in the group (e.g. issues, merge requests) will remain intact. #### Assertions -When using Group Manged Accounts, the following user details need to be passed to GitLab as SAML Assertions in order for us to be able to create a user: +When using group-managed accounts, the following user details need to be passed to GitLab as SAML +assertions to be able to create a user. | Field | Supported keys | |-----------------|----------------| @@ -91,7 +96,7 @@ When using Group Manged Accounts, the following user details need to be passed t GitLab provides metadata XML that can be used to configure your Identity Provider. 1. Navigate to the group and click **Settings > SAML SSO**. -1. Copy the provided **GitLab metadata URL** +1. Copy the provided **GitLab metadata URL**. 1. Follow your Identity Provider's documentation and paste the metadata URL when it is requested. ## Configuring GitLab @@ -123,6 +128,25 @@ NOTE: **Note:** GitLab is unable to provide support for IdPs that are not listed When [configuring your identify provider](#configuring-your-identity-provider), please consider the notes below for specific providers to help avoid common issues and as a guide for terminology used. +### Azure setup notes + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo of the Azure SAML setup including SCIM, see [SCIM Provisioning on Azure Using SAML SSO for Groups Demo](https://youtu.be/24-ZxmTeEBU). + +| GitLab Setting | Azure Field | +|--------------|----------------| +| Identifier | Identifier (Entity ID) | +| Assertion consumer service URL | Reply URL (Assertion Consumer Service URL) | +| Identity provider single sign on URL | Login URL | +| Certificate fingerprint | Thumbprint | + +We recommend: + +- **Unique User Identifier (Name identifier)** set to `user.objectID`. +- **nameid-format** set to persistent. + +Set other user attributes and claims according to the [assertions table](#assertions). + ### Okta setup notes | GitLab Setting | Okta Field | @@ -193,6 +217,8 @@ For example, to unlink the `MyOrg` account, the following **Disconnect** button ## Troubleshooting +This section contains possible solutions for problems you might encounter. + ### SAML debugging tools SAML responses are base64 encoded, so we recommend the following browser plugins to decode them on the fly: diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index 392b27bb42f..a117364a355 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -2,7 +2,7 @@ type: howto, reference --- -# SCIM provisioning using SAML SSO for Groups **(SILVER ONLY)** +# SCIM provisioning using SAML SSO for GitLab.com groups **(SILVER ONLY)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/9388) in [GitLab.com Silver](https://about.gitlab.com/pricing/) 11.10. @@ -24,7 +24,7 @@ The following identity providers are supported: ## Requirements -- [Group SSO](index.md) needs to be configured. +- [Group SSO](index.md) must be configured. ## GitLab configuration @@ -64,15 +64,25 @@ You can then test the connection by clicking on **Test Connection**. If the conn 1. Click on `Synchronize Azure Active Directory Users to AppName`, to configure the attribute mapping. 1. Click **Delete** next to the `mail` mapping. -1. Map `userPrincipalName` to `emails[type eq "work"].value` and change it's **Matching precedence** to `2`. +1. Map `userPrincipalName` to `emails[type eq "work"].value` and change its **Matching precedence** to `2`. 1. Map `mailNickname` to `userName`. 1. Determine how GitLab will uniquely identify users. - Use `objectId` unless users already have SAML linked for your group. - If you already have users with SAML linked then use the `Name ID` value from the [SAML configuration](#azure). Using a different value will likely cause duplicate users and prevent users from accessing the GitLab group. -1. Create a new mapping by clicking **Add New Mapping** then set **Source attribute** to the unique identifier determined above, **Target attribute** to `id`, **Match objects using this attribute** to `Yes`, and **Matching precedence** to `1`. -1. Create a new mapping by clicking **Add New Mapping** then set **Source attribute** to the unique identifier determined above, and **Target attribute** to `externalId`. +1. Create a new mapping: + 1. Click **Add New Mapping**. + 1. Set: + - **Source attribute** to the unique identifier determined above. + - **Target attribute** to `id`. + - **Match objects using this attribute** to `Yes`. + - **Matching precedence** to `1`. +1. Create another new mapping: + 1. Click **Add New Mapping**. + 1. Set: + - **Source attribute** to the unique identifier determined above. + - **Target attribute** to `externalId`. 1. Click the `userPrincipalName` mapping and change **Match objects using this attribute** to `No`. Save your changes and you should have the following configuration: @@ -109,6 +119,8 @@ Once synchronized, changing the field mapped to `id` and `externalId` will likel ## Troubleshooting +This section contains possible solutions for problems you might encounter. + ### Testing Azure connection: invalid credentials When testing the connection, you may encounter an error: **You appear to have entered invalid credentials. Please confirm you are using the correct information for an administrative account**. If `Tenant URL` and `secret token` are correct, check whether your group path contains characters that may be considered invalid JSON primitives (such as `.`). Removing such characters from the group path typically resolves the error. diff --git a/doc/user/group/subgroups/img/group_members_filter_v12_6.png b/doc/user/group/subgroups/img/group_members_filter_v12_6.png Binary files differnew file mode 100644 index 00000000000..0207515ded0 --- /dev/null +++ b/doc/user/group/subgroups/img/group_members_filter_v12_6.png diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 52b7035389a..997cb1ba6c5 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -4,7 +4,7 @@ type: reference, howto, concepts # Subgroups ->[Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/2772) in GitLab 9.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/2772) in GitLab 9.0. Subgroups, also known as nested groups or hierarchical groups, allow you to have up to 20 levels of groups. @@ -142,6 +142,16 @@ From the image above, we can deduce the following things: - Administrator is the Owner and member of **all** subgroups and for that reason, as with User3, there is no indication of an ancestor group. +[From](https://gitlab.com/gitlab-org/gitlab/issues/21727) GitLab 12.6, you can filter +this list using dropdown on the right side: + + + +- **Show only direct members** displays only Administrator and User3, since these are + the only users that belong to group `four`, which is the one we're inspecting. +- **Show only inherited members** displays User0, User1 and User2, no matter which group + above the hierarchy is the source of inherited permissions. + ### Overriding the ancestor group membership NOTE: **Note:** @@ -186,7 +196,7 @@ Here's a list of what you can't do with subgroups: [ce-2772]: https://gitlab.com/gitlab-org/gitlab-foss/issues/2772 [permissions]: ../../permissions.md#group-members-permissions -[reserved]: ../../reserved_names.md +[reserved]: ../../reserved_names.md [issue]: https://gitlab.com/gitlab-org/gitlab-foss/issues/30472#note_27747600 <!-- ## Troubleshooting diff --git a/doc/user/incident_management/index.md b/doc/user/incident_management/index.md index 5ac27d227a1..febe1a2423a 100644 --- a/doc/user/incident_management/index.md +++ b/doc/user/incident_management/index.md @@ -91,7 +91,7 @@ Please refer to a list of [available slash commands](../../integration/slash_com ## Zoom in issues -In order to communicate synchronously for incidents management, GitLab allows to +In order to communicate synchronously for incidents management, GitLab allows you to associate a Zoom meeting with an issue. Once you start a Zoom call for a fire-fight, you need a way to associate the conference call with an issue, so that your team members can join swiftly without requesting a link. diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md new file mode 100644 index 00000000000..a50cdf1cf0e --- /dev/null +++ b/doc/user/infrastructure/index.md @@ -0,0 +1,6 @@ +# Infrastructure as Code + +GitLab can be used to manage infrastructure as code. The following are some examples: + +- [A generic tutorial for Terraform with GitLab](https://medium.com/@timhberry/terraform-pipelines-in-gitlab-415b9d842596). +- [Terraform at GitLab](https://about.gitlab.com/blog/2019/11/12/gitops-part-2/). diff --git a/doc/user/instance_statistics/convdev.md b/doc/user/instance_statistics/dev_ops_score.md index 705f9be3d94..fbe4cc3c6df 100644 --- a/doc/user/instance_statistics/convdev.md +++ b/doc/user/instance_statistics/dev_ops_score.md @@ -1,12 +1,12 @@ -# Conversational Development Index +# DevOps Score -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/30469) in GitLab 9.3. +> [Renamed from Conversational Development Index](https://gitlab.com/gitlab-org/gitlab/issues/20976) in GitLab 12.6. NOTE: **Note:** Your GitLab instance's [usage ping](../admin_area/settings/usage_statistics.md#usage-ping-core-only) must be activated in order to use this feature. -The [Conversational Development](http://conversationaldevelopment.com/2017/04/16/what-is-conversational-development/) Index (ConvDev Index) gives you an overview of your entire -instance's adoption of [Concurrent DevOps](https://about.gitlab.com/concurrent-devops/) +The DevOps Score gives you an overview of your entire instance's adoption of +[Concurrent DevOps](https://about.gitlab.com/concurrent-devops/) from planning to monitoring. This displays the usage of these GitLab features over @@ -16,7 +16,7 @@ of top-performing instances based on [usage ping data](../admin_area/settings/us collected. Your score is compared to the lead score of each feature and then expressed as a percentage at the bottom of said feature. Your overall index score is an average of all your feature score percentages - this percentage value is presented above all the of features on the page. - + The page also provides helpful links to articles and GitLab docs, to help you improve your scores. diff --git a/doc/user/instance_statistics/img/cohorts.png b/doc/user/instance_statistics/img/cohorts.png Binary files differindex 12e839e7cd2..4d070fdb654 100644 --- a/doc/user/instance_statistics/img/cohorts.png +++ b/doc/user/instance_statistics/img/cohorts.png diff --git a/doc/user/instance_statistics/img/convdev_index.png b/doc/user/instance_statistics/img/dev_ops_score.png Binary files differindex bee1317438d..bee1317438d 100644 --- a/doc/user/instance_statistics/img/convdev_index.png +++ b/doc/user/instance_statistics/img/dev_ops_score.png diff --git a/doc/user/instance_statistics/index.md b/doc/user/instance_statistics/index.md index fe8e8c96f81..53bf85b6e13 100644 --- a/doc/user/instance_statistics/index.md +++ b/doc/user/instance_statistics/index.md @@ -12,5 +12,5 @@ and can be accessed via the top bar. There are two kinds of statistics: -- [Conversational Development (ConvDev) Index](convdev.md): Provides an overview of your entire instance's feature usage. +- [Dev Ops Score](dev_ops_score.md): Provides an overview of your entire instance's feature usage. - [User Cohorts](user_cohorts.md): Display the monthly cohorts of new users and their activities over time. diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 3bd0dcafc19..fdf6cb3c7be 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -1,9 +1,9 @@ # GitLab Markdown -This markdown guide is **valid only for GitLab's internal markdown rendering system for entries and files**. +This Markdown guide is **valid only for GitLab's internal Markdown rendering system for entries and files**. It is **not** valid for the [GitLab documentation website](https://docs.gitlab.com) or [GitLab's main website](https://about.gitlab.com), as they both use -[Kramdown](https://kramdown.gettalong.org) as their markdown engine. The documentation +[Kramdown](https://kramdown.gettalong.org) as their Markdown engine. The documentation website uses an extended Kramdown gem, [GitLab Kramdown](https://gitlab.com/gitlab-org/gitlab_kramdown). Consult the [GitLab Kramdown Guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/) for a complete Kramdown reference. @@ -40,7 +40,7 @@ repositories are also processed with CommonMark. As of 11.8, the [Redcarpet Ruby has been removed and all issues and comments, including those from pre-11.1, are now processed using the [CommonMark Ruby Library](https://github.com/gjtorikian/commonmarker). -The documentation website had its [markdown engine migrated from Redcarpet to Kramdown](https://gitlab.com/gitlab-org/gitlab-docs/merge_requests/108) +The documentation website had its [Markdown engine migrated from Redcarpet to Kramdown](https://gitlab.com/gitlab-org/gitlab-docs/merge_requests/108) in October 2018. You may have older issues, merge requests, or Markdown documents in your @@ -71,7 +71,7 @@ the top list item (`C` in this case): - milk NOTE: **Note:** We will flag any significant differences between Redcarpet and CommonMark - markdown in this document. + Markdown in this document. If you have a large volume of Markdown files, it can be tedious to determine if they will display correctly or not. You can use the @@ -81,28 +81,28 @@ differences between how RedCarpet and CommonMark render the files. It can give an indication if anything needs to be changed - often nothing will need to change. -### GFM extends standard markdown +### GFM extends standard Markdown GitLab makes full use of the standard (CommonMark) formatting, but also includes additional functionality useful for GitLab users. -It makes use of [new markdown features](#new-GFM-markdown-extensions), -not found in standard markdown: +It makes use of [new Markdown features](#new-GFM-markdown-extensions), +not found in standard Markdown: - [Color "chips" written in HEX, RGB or HSL](#colors) -- [Diagrams and flowcharts using Mermaid](#diagrams-and-flowcharts-using-mermaid) +- [Diagrams and flowcharts](#diagrams-and-flowcharts) - [Emoji](#emoji) - [Front matter](#front-matter) - [Inline diffs](#inline-diff) - [Math equations and symbols written in LaTeX](#math) - [Special GitLab references](#special-gitlab-references) - [Task Lists](#task-lists) -- [Wiki specific markdown](#wiki-specific-markdown) +- [Wiki specific Markdown](#wiki-specific-markdown) -It also has [extended markdown features](#standard-markdown-and-extensions-in-gitlab), without -changing how standard markdown is used: +It also has [extended Markdown features](#standard-markdown-and-extensions-in-gitlab), without +changing how standard Markdown is used: -| Standard markdown | Extended markdown in GitLab | +| Standard Markdown | Extended Markdown in GitLab | | ------------------------------------- | ------------------------- | | [blockquotes](#blockquotes) | [multiline blockquotes](#multiline-blockquote) | | [code blocks](#code-spans-and-blocks) | [colored code and syntax highlighting](#colored-code-and-syntax-highlighting) | @@ -112,7 +112,7 @@ changing how standard markdown is used: | [linebreaks](#line-breaks) | [more linebreak control](#newlines) | | [links](#links) | [automatically linking URLs](#url-auto-linking) | -## New GFM markdown extensions +## New GFM Markdown extensions ### Colors @@ -151,13 +151,16 @@ Color written inside backticks will be followed by a color "chip": `HSL(540,70%,50%)` `HSLA(540,70%,50%,0.3)` -### Diagrams and flowcharts using Mermaid +### Diagrams and flowcharts + +It is possible to generate diagrams and flowcharts from text in GitLab using [Mermaid](https://mermaidjs.github.io/) or [PlantUML](http://plantuml.com). + +#### Mermaid > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/15107) in GitLab 10.3. -It is possible to generate diagrams and flowcharts from text using [Mermaid](https://mermaidjs.github.io/). -Visit the official page for more details. +Visit the [official page](https://mermaidjs.github.io/) for more details. In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block: @@ -179,8 +182,6 @@ graph TD; C-->D; ``` -#### Subgraphs - Subgraphs can also be included: ~~~ @@ -219,6 +220,10 @@ graph TB end ``` +#### PlantUML + +To make PlantUML available in GitLab, a GitLab administrator needs to enable it first. Read more in [PlantUML & GitLab](../administration/integration/plantuml.md). + ### Emoji > If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#emoji). @@ -258,7 +263,7 @@ this font installed by default. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/23331) in GitLab 11.6. -Front matter is metadata included at the beginning of a markdown document, preceding +Front matter is metadata included at the beginning of a Markdown document, preceding its content. This data can be used by static site generators such as [Jekyll](https://jekyllrb.com/docs/front-matter/), [Hugo](https://gohugo.io/content-management/front-matter/), and many other applications. @@ -529,9 +534,9 @@ This snippet links to `<wiki_root>/miscellaneous.md`: Metric charts can be embedded within GitLab Flavored Markdown. See [Embedding Metrics within GitLab flavored Markdown](../user/project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown) for more details. -## Standard markdown and extensions in GitLab +## Standard Markdown and extensions in GitLab -All standard markdown formatting should work as expected within GitLab. Some standard +All standard Markdown formatting should work as expected within GitLab. Some standard functionality is extended with additional features, without affecting the standard usage. If a functionality is extended, the new option will be listed as a sub-section. @@ -560,7 +565,7 @@ Quote break. > If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#multiline-blockquote). -GFM extends the standard markdown standard by also supporting multiline blockquotes +GFM extends the standard Markdown standard by also supporting multiline blockquotes fenced by `>>>`: ``` @@ -704,7 +709,7 @@ But let's throw in a <b>tag</b>. ### Emphasis -There are multiple ways to emphasize text in markdown. You can italicize, bold, strikethrough, +There are multiple ways to emphasize text in Markdown. You can italicize, bold, strikethrough, as well as combine these emphasis styles together. Examples: @@ -735,8 +740,8 @@ NOTE: **Note:** Strikethrough is not part of the core Markdown standard, but is It is not usually useful to italicize just _part_ of a word, especially when you're dealing with code and names that often appear with multiple underscores. As a result, -GFM extends the standard markdown standard by ignoring multiple underlines in words, -to allow better rendering of markdown documents discussing code: +GFM extends the standard Markdown standard by ignoring multiple underlines in words, +to allow better rendering of Markdown documents discussing code: ```md perform_complicated_task @@ -768,7 +773,7 @@ do*this*and*do*that*and*another thing ### Footnotes -Footnotes add a link to a note rendered at the end of a markdown file: +Footnotes add a link to a note rendered at the end of a Markdown file: ```markdown You can add footnotes to your text as follows.[^1] @@ -801,7 +806,7 @@ Alt-H2 #### Header IDs and links -GFM extends the standard markdown standard so that all Markdown-rendered headers automatically +GFM extends the standard Markdown standard so that all Markdown-rendered headers automatically get IDs, which can be linked to, except in comments. On hover, a link to those IDs becomes visible to make it easier to copy the link to @@ -906,7 +911,7 @@ Here's a sample video: > If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#audio). Similar to videos, link tags for files with an audio extension are automatically converted to -an audio player. The valid audio extensions are `.mp3`, `.ogg`, and `.wav`: +an audio player. The valid audio extensions are `.mp3`, `.oga`, `.ogg`, `.spx`, and `.wav`: ```md Here's a sample audio clip: @@ -920,7 +925,7 @@ Here's a sample audio clip: ### Inline HTML -> To see the markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#inline-html). +> To see the Markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#inline-html). You can also use raw HTML in your Markdown, and it'll usually work pretty well. @@ -948,7 +953,7 @@ class for the list of allowed HTML tags and attributes. In addition to the defa --- -It is still possible to use markdown inside HTML tags, but only if the lines containing markdown +It is still possible to use Markdown inside HTML tags, but only if the lines containing Markdown are separated into their own lines: ```html @@ -965,7 +970,7 @@ are separated into their own lines: </dl> ``` -<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, markdown will be fine in GitLab --> +<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, Markdown will be fine in GitLab --> <dl> <dt>Markdown in HTML</dt> @@ -981,7 +986,7 @@ are separated into their own lines: #### Details and Summary -> To see the markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#details-and-summary). +> To see the Markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#details-and-summary). Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) and [`<summary>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary) @@ -1029,7 +1034,7 @@ PASTE LOGS HERE </details> ```` -<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, markdown will be fine in GitLab --> +<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, Markdown will be fine in GitLab --> <details> <summary>Click me to collapse/fold.</summary> @@ -1070,7 +1075,7 @@ in the *same paragraph*. #### Newlines -GFM adheres to the markdown specification in how [paragraphs and line breaks are handled](https://spec.commonmark.org/current/). +GFM adheres to the Markdown specification in how [paragraphs and line breaks are handled](https://spec.commonmark.org/current/). A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines (i.e. two newlines at the end of the first paragraph), as [explained above](#line-breaks). @@ -1117,7 +1122,7 @@ There are two ways to create links, inline-style and reference-style: Using header ID anchors: -- This links to [a section on a different markdown page, using a "#" and the header ID](index.md#overview) +- This links to [a section on a different Markdown page, using a "#" and the header ID](index.md#overview) - This links to [a different section on the same page, using a "#" and the header ID](#header-ids-and-links) Using references: @@ -1140,7 +1145,7 @@ Some text to show that the reference links can follow later. Using header ID anchors: -- This links to [a section on a different markdown page, using a "#" and the header ID](index.md#overview) +- This links to [a section on a different Markdown page, using a "#" and the header ID](index.md#overview) - This links to [a different section on the same page, using a "#" and the header ID](#header-ids-and-links) Using references: @@ -1158,7 +1163,7 @@ Some text to show that the reference links can follow later. NOTE: **Note:** Relative links do not allow the referencing of project files in a wiki page, or a wiki page in a project file. The reason for this is that a wiki is always in a separate Git repository in GitLab. For example, `[I'm a reference-style link](style)` -will point the link to `wikis/style` only when the link is inside of a wiki markdown file. +will point the link to `wikis/style` only when the link is inside of a wiki Markdown file. #### URL auto-linking @@ -1314,7 +1319,7 @@ Tables aren't part of the core Markdown spec, but they are part of GFM. 1. The first line contains the headers, separated by "pipes" (`|`). 1. The second line separates the headers from the cells, and must contain three or more dashes. 1. The third, and any following lines, contain the cell values. - - You **can't** have cells separated over many lines in the markdown, they must be kept to single lines, + - You **can't** have cells separated over many lines in the Markdown, they must be kept to single lines, but they can be very long. You can also include HTML `<br>` tags to force newlines if needed. - The cell sizes **don't** have to match each other. They are flexible, but must be separated by pipes (`|`). @@ -1357,6 +1362,6 @@ to the sides of the "dash" lines in the second row. This will affect every cell - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). - The original [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) - at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. + at Daring Fireball is an excellent resource for a detailed explanation of standard Markdown. - The detailed specification for CommonMark can be found in the [CommonMark Spec](https://spec.commonmark.org/current/) - The [CommonMark Dingus](http://try.commonmark.org) is a handy tool for testing CommonMark syntax. diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index 953c7472f4d..2366d1ccc0d 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -1,6 +1,6 @@ # GitLab Conan Repository **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/8248) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.5. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/8248) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.6. With the GitLab Conan Repository, every project can have its own space to store Conan packages. @@ -27,7 +27,7 @@ get familiar with the package naming convention. ## Authenticating to the GitLab Conan Repository -You will need to generate a [personal access token](../../../user/profile/personal_access_tokens.md) for repository authentication. +You will need to generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. Now you can run conan commands using your token. diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index f6c9a2c9e34..9c1a9d5a41a 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -144,7 +144,6 @@ project or branch name. Special characters can include: - Leading underscore - Trailing hyphen/dash -- Double hyphen/dash To get around this, you can [change the group path](../../group/index.md#changing-a-groups-path), [change the project path](../../project/settings/index.md#renaming-a-repository) or change the branch diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 9873bd80e8b..ecaad960340 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -13,8 +13,12 @@ The Packages feature allows GitLab to act as a repository for the following: | [Conan Repository](conan_repository/index.md) **(PREMIUM)** | The GitLab Conan Repository enables every project in GitLab to have its own space to store [Conan](https://conan.io/) packages. | 12.4+ | | [Maven Repository](maven_repository/index.md) **(PREMIUM)** | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | | [NPM Registry](npm_registry/index.md) **(PREMIUM)** | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | +| [NuGet Repository](https://gitlab.com/gitlab-org/gitlab/issues/20050) **(PREMIUM)** | *COMING SOON* The GitLab NuGet Repository will enable every project in GitLab to have its own space to store [NuGet](https://www.nuget.org/) packages. | 12.7 (planned) | TIP: **Tip:** Don't you see your package management system supported yet? Consider contributing to GitLab. This [development documentation](../../development/packages.md) will -guide you through the process. +guide you through the process. Or check out how other members of the commmunity +are adding support for [PHP](https://gitlab.com/gitlab-org/gitlab/merge_requests/17417) or [Terraform](https://gitlab.com/gitlab-org/gitlab/merge_requests/18834). + +NOTE: **Note** We are especially interested in adding support for [PyPi](https://gitlab.com/gitlab-org/gitlab/issues/10483), [RubyGems](https://gitlab.com/gitlab-org/gitlab/issues/803), [Debian](https://gitlab.com/gitlab-org/gitlab/issues/5835), and [RPM](https://gitlab.com/gitlab-org/gitlab/issues/5932). diff --git a/doc/user/packages/maven_repository/img/maven_package_view.png b/doc/user/packages/maven_repository/img/maven_package_view.png Binary files differdeleted file mode 100644 index 2eb4b6f76b4..00000000000 --- a/doc/user/packages/maven_repository/img/maven_package_view.png +++ /dev/null diff --git a/doc/user/packages/maven_repository/img/maven_package_view_v12_6.png b/doc/user/packages/maven_repository/img/maven_package_view_v12_6.png Binary files differnew file mode 100644 index 00000000000..92cefc26660 --- /dev/null +++ b/doc/user/packages/maven_repository/img/maven_package_view_v12_6.png diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 8ed10c09891..da5139fcaf9 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -5,7 +5,7 @@ With the GitLab [Maven](https://maven.apache.org) Repository, every project can have its own space to store its Maven artifacts. - + ## Enabling the Maven Repository @@ -37,7 +37,7 @@ credentials do not work. ### Authenticating with a personal access token To authenticate with a [personal access token](../../profile/personal_access_tokens.md), -add a corresponding section to your +set the scope to `api` and add a corresponding section to your [`settings.xml`](https://maven.apache.org/settings.html) file: ```xml diff --git a/doc/user/packages/npm_registry/img/npm_package_view.png b/doc/user/packages/npm_registry/img/npm_package_view.png Binary files differdeleted file mode 100644 index e0634718c02..00000000000 --- a/doc/user/packages/npm_registry/img/npm_package_view.png +++ /dev/null diff --git a/doc/user/packages/npm_registry/img/npm_package_view_v12_5.png b/doc/user/packages/npm_registry/img/npm_package_view_v12_5.png Binary files differnew file mode 100644 index 00000000000..a6f823011eb --- /dev/null +++ b/doc/user/packages/npm_registry/img/npm_package_view_v12_5.png diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index d8b59ae63d0..7d5db5a60ef 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -5,7 +5,7 @@ With the GitLab NPM Registry, every project can have its own space to store NPM packages. - + NOTE: **Note:** Only [scoped](https://docs.npmjs.com/misc/scope) packages are supported. @@ -42,6 +42,20 @@ it is not possible due to a naming collision. For example: | `gitlab-org/gitlab` | `@gitlab-org/gitlab` | Yes | | `gitlab-org/gitlab` | `@foo/bar` | No | +The regex that is used for naming is validating all package names from all package managers: + +``` +/\A\@?(([\w\-\.\+]*)\/)*([\w\-\.]+)@?(([\w\-\.\+]*)\/)*([\w\-\.]*)\z/ +``` + +It allows for capital letters, while NPM does not, and allows for almost all of the +characters NPM allows with a few exceptions (`~` is not allowed). + +NOTE: **Note:** Capital letters are needed because the scope is required to be +identical to the top level namespace of the project. So, for example, if your +project path is `My-Group/project-foo`, your package must be named `@My-Group/any-package-name`. +`@my-group/any-package-name` will not work. + CAUTION: **When updating the path of a user/group or transferring a (sub)group/project:** If you update the root namespace of a project with NPM packages, your changes will be rejected. To be allowed to do that, make sure to remove any NPM package first. Don't forget to update your `.npmrc` files to follow the above naming convention and run `npm publish` if necessary. @@ -54,7 +68,7 @@ If a project is private or you want to upload an NPM package to GitLab, credentials will need to be provided for authentication. Support is available for [OAuth tokens](../../../api/oauth2.md#resource-owner-password-credentials-flow) or [personal access tokens](../../profile/personal_access_tokens.md). CAUTION: **2FA is only supported with personal access tokens:** -If you have 2FA enabled, you need to use a [personal access token](../../profile/personal_access_tokens.md) with OAuth headers. Standard OAuth tokens won't be able to authenticate to the GitLab NPM Registry. +If you have 2FA enabled, you need to use a [personal access token](../../profile/personal_access_tokens.md) with OAuth headers with the scope set to `api`. Standard OAuth tokens won't be able to authenticate to the GitLab NPM Registry. ### Authenticating with an OAuth token @@ -108,7 +122,7 @@ Then, you could run `npm publish` either locally or via GitLab CI/CD: - **GitLab CI/CD:** Set an `NPM_TOKEN` [variable](../../../ci/variables/README.md) under your project's **Settings > CI/CD > Variables**. - + ### Authenticating with a CI job token > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9104) in GitLab Premium 12.5. @@ -116,7 +130,7 @@ Then, you could run `npm publish` either locally or via GitLab CI/CD: If you’re using NPM with GitLab CI/CD, a CI job token can be used instead of a personal access token. The token will inherit the permissions of the user that generates the pipeline. -Add a corresponding section to your `.npmrc` file: +Add a corresponding section to your `.npmrc` file: ```ini @foo:registry=https://gitlab.com/api/v4/packages/npm/ @@ -156,9 +170,10 @@ a given scope, you will receive a `403 Forbidden!` error. ## Uploading a package with the same version twice -If you upload a package with a same name and version twice, GitLab will show -both packages in the UI, but the GitLab NPM Registry will expose the most recent -one as it supports only one package per version for `npm install`. +You cannot upload a package with the same name and version twice, unless you +delete the existing package first. This aligns with npmjs.org's behavior, with +the exception that npmjs.org does not allow users to ever publish the same version +more than once, even if it has been deleted. ## Troubleshooting @@ -211,3 +226,19 @@ And the `.npmrc` file should look like: //gitlab.com/api/v4/packages/npm/:_authToken=<your_oauth_token> @foo:registry=https://gitlab.com/api/v4/packages/npm/ ``` + +## NPM dependencies metadata + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/11867) in GitLab Premium 12.6. + +Starting from GitLab 12.6, new packages published to the GitLab NPM Registry expose the following attributes to the NPM client: + +- name +- version +- dist-tags +- dependencies + - dependencies + - devDependencies + - bundleDependencies + - peerDependencies + - deprecated diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 70660e5e22f..9cbf4fd6192 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -5,7 +5,7 @@ description: 'Understand and explore the user permission levels in GitLab, and w # Permissions Users have different abilities depending on the access level they have in a -particular group or project. If a user is both in a group's project and the +particular group or project. If a user is both in a project's group and the project itself, the highest permission level is used. On public and internal projects the Guest role is not enforced. All users will @@ -46,10 +46,11 @@ The following table depicts the various user permission levels in a project. | Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View Insights charts **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View approved/blacklisted licenses **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View approved/blacklisted licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View License Compliance reports **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View Security reports **(ULTIMATE)** | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | | View Dependency list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View License list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View licenses in Dependency list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View [Design Management](project/issues/design_management.md) pages **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | | View project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | @@ -252,7 +253,7 @@ project and should only have access to that project. External users: -- Cannot create groups or projects. +- Cannot create groups, projects, or personal snippets. - Can only access projects to which they are explicitly granted access, thus hiding all other internal or private ones from them (like being logged out). diff --git a/doc/user/profile/active_sessions.md b/doc/user/profile/active_sessions.md index fe2eeebdb99..f68b11a57ec 100644 --- a/doc/user/profile/active_sessions.md +++ b/doc/user/profile/active_sessions.md @@ -2,10 +2,9 @@ type: howto --- -# Active Sessions +# Active sessions -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/17867) -> in GitLab 10.8. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/17867) in GitLab 10.8. GitLab lists all devices that have logged into your account. This allows you to review the sessions, and revoke any you don't recognize. @@ -18,6 +17,13 @@ review the sessions, and revoke any you don't recognize.  +## Active sessions limit + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/31611) in GitLab 12.6. + +GitLab allows users to have up to 100 active sessions at once. If the number of active sessions +exceeds 100, the oldest ones are deleted. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index c73368fbbd2..6a0377f118d 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -34,7 +34,7 @@ namespace. This service account will be: -- Added to the installed Helm Tiller +- Added to the installed Helm Tiller. - Used by Helm to install and run [GitLab managed applications](index.md#installing-applications). Helm will also create additional service accounts and other resources for each @@ -111,6 +111,11 @@ If you don't want to use GitLab Runner in privileged mode, either: ## Add new cluster +New clusters can be added using GitLab for: + +- Google Kubernetes Engine. +- Amazon Elastic Kubernetes Service. + ### GKE cluster GitLab supports: @@ -206,46 +211,9 @@ GitLab supports: Before creating your first cluster on Amazon EKS with GitLab's integration, make sure the following requirements are met: -- Enable the `create_eks_clusters` feature flag for your GitLab instance. - An [Amazon Web Services](https://aws.amazon.com/) account is set up and you are able to log in. - You have permissions to manage IAM resources. -#### Enable the `create_eks_clusters` feature flag **(CORE ONLY)** - -NOTE: **Note:** -If you are running a self-managed instance, EKS cluster creation will not be available -unless the feature flag `create_eks_clusters` is enabled. This can be done from the Rails console -by instance administrators. - -Use these commands to start the Rails console: - -```sh -# Omnibus GitLab -gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production -``` - -Then run the following command to enable the feature flag: - -``` -Feature.enable(:create_eks_clusters) -``` - -You can also enable the feature flag only for specific projects with: - -``` -Feature.enable(:create_eks_clusters, Project.find_by_full_path('my_group/my_project')) -``` - -Run the following command to disable the feature flag: - -``` -Feature.disable(:create_eks_clusters) -``` - ##### Additional requirements for self-managed instances If you are using a self-managed GitLab instance, GitLab must first @@ -332,6 +300,7 @@ new Kubernetes cluster to your project: "iam:AttachRolePolicy", "iam:CreateRole", "iam:CreateInstanceProfile", + "iam:CreateServiceLinkedRole", "iam:GetRole", "iam:ListRoles", "iam:PassRole", diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index c5c2c2c07e7..6d863a8b888 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -128,9 +128,31 @@ automatically. If you are using [Auto DevOps](../../../topics/autodevops/index.m need to explicitly provide the `KUBE_NAMESPACE` [deployment variable](#deployment-variables) that will be used by your deployment jobs, otherwise a namespace will be created for you. -NOTE: **Note:** -If you [install applications](#installing-applications) on your cluster, GitLab will create -the resources required to run these even if you have chosen to manage your own cluster. +#### Important notes + +Note the following with GitLab and clusters: + +- If you [install applications](#installing-applications) on your cluster, GitLab will + create the resources required to run these even if you have chosen to manage your own + cluster. +- Be aware that manually managing resources that have been created by GitLab, like + namespaces and service accounts, can cause unexpected errors. If this occurs, try + [clearing the cluster cache](#clearing-the-cluster-cache). + +#### Clearing the cluster cache + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/31759) in GitLab 12.6. + +If you choose to allow GitLab to manage your cluster for you, GitLab stores a cached +version of the namespaces and service accounts it creates for your projects. If you +modify these resources in your cluster manually, this cache can fall out of sync with +your cluster, which can cause deployment jobs to fail. + +To clear the cache: + +1. Navigate to your project’s **Operations > Kubernetes** page, and select your cluster. +1. Expand the **Advanced settings** section. +1. Click **Clear cluster cache**. ### Base domain diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index 2c16748a3ee..0b74f1e73eb 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -4,9 +4,20 @@ GitLab allows users to easily deploy AWS Lambda functions and create rich server GitLab supports deployment of functions to AWS Lambda using a combination of: -- [Serverless Framework](https://serverless.com) +- [Serverless Framework with AWS](https://serverless.com/framework/docs/providers/aws/) - GitLab CI/CD +We have prepared an example with a step-by-step guide to create a simple function and deploy it on AWS. + +Additionally, in the [How To section](#how-to), you can read about different use cases, +like: + +- Running a function locally. +- Working with secrets. +- Setting up CORS. + +Alternatively, you can quickly [create a new project with a template](https://docs.gitlab.com/ee/gitlab-basics/create-project.html#project-templates). The [`Serverless Framework/JS` template](https://gitlab.com/gitlab-org/project-templates/serverless-framework/) already includes all parts described below. + ## Example In the following example, you will: @@ -23,13 +34,13 @@ The example consists of the following steps: 1. Crafting the `.gitlab-ci.yml` file 1. Setting up your AWS credentials with your GitLab account 1. Deploying your function -1. Testing your function +1. Testing the deployed function Lets take it step by step. ### Creating a Lambda handler function -Your Lambda function will be the primary handler of requests. In this case we will create a very simple Node.js "Hello" function: +Your Lambda function will be the primary handler of requests. In this case we will create a very simple Node.js `hello` function: ```javascript 'use strict'; @@ -46,8 +57,6 @@ module.exports.hello = async event => { ), }; }; - - ``` Place this code in the file `src/handler.js`. @@ -58,7 +67,7 @@ In our case, `module.exports.hello` defines the `hello` handler that will be ref You can learn more about the AWS Lambda Node.js function handler and all its various options here: <https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html> -### Creating a serverless.yml file +### Creating a `serverless.yml` file In the root of your project, create a `serverless.yml` file that will contain configuration specifics for the Serverless Framework. @@ -69,7 +78,7 @@ service: gitlab-example provider: name: aws runtime: nodejs10.x - + functions: hello: handler: src/handler.hello @@ -85,9 +94,9 @@ The `events` declaration will create a AWS API Gateway `GET` endpoint to receive You can read more about the available properties and additional configuration possibilities of the Serverless Framework here: <https://serverless.com/framework/docs/providers/aws/guide/serverless.yml/> -### Crafting the .gitlab-ci.yml file +### Crafting the `.gitlab-ci.yml` file -In a `.gitlab-ci.yml` file, place the following code: +In a `.gitlab-ci.yml` file in the root of your project, place the following code: ```yaml image: node:latest @@ -109,21 +118,22 @@ This example code does the following: 1. Uses the `node:latest` image for all GitLab CI builds 1. The `deploy` stage: - -- Installs the `serverless framework`. -- Deploys the serverless function to your AWS account using the AWS credentials defined above. + - Installs the Serverless Framework. + - Deploys the serverless function to your AWS account using the AWS credentials + defined above. + - Deploys the serverless function to your AWS account using the AWS credentials defined above ### Setting up your AWS credentials with your GitLab account -In order to interact with your AWS account, the .gitlab-ci.yml requires both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` be defined in your GitLab settings under **Settings > CI/CD > Variables**. +In order to interact with your AWS account, the GitLab CI/CD pipelines require both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be defined in your GitLab settings under **Settings > CI/CD > Variables**. For more information please see: <https://docs.gitlab.com/ee/ci/variables/README.html#via-the-ui> NOTE: **Note:** - The AWS credentials you provide must include IAM policies that provision correct access control to AWS Lambda, API Gateway, and CloudFormation resources. + The AWS credentials you provide must include IAM policies that provision correct access control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. ### Deploying your function -Deploying your function is very simple, just `git push` to your GitLab repository and the GitLab build pipeline will automatically deploy your function. +`git push` the changes to your GitLab repository and the GitLab build pipeline will automatically deploy your function. In your GitLab deploy stage log, there will be output containing your AWS Lambda endpoint URL. The log line will look similar to this: @@ -133,7 +143,7 @@ endpoints: GET - https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello ``` -### Testing your function +### Manually testing your function Running the following `curl` command should trigger your function. @@ -144,7 +154,7 @@ NOTE: **Note:** curl https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello ``` -Should output: +That should output: ```json { @@ -156,8 +166,123 @@ Hooray! You now have a AWS Lambda function deployed via GitLab CI. Nice work! -## Example code +## How To + +In this section, we show you how to build on the basic example to: + +- Run the function locally. +- Set up secret variables. +- Set up CORS. + +### Running function locally + +The `serverless-offline` plugin allows to run your code locally. To run your code locally: + +1. Add the following to your `serverless.yml`: + + ```yaml + plugins: + - serverless-offline + ``` + +1. Start the service by running the following command: + + ```shell + serverless offline + ``` + +Running the following `curl` command should trigger your function. + +```sh +curl http://localhost:3000/hello +``` + +It should output: + +```json +{ + "message": "Your function executed successfully!" +} +``` + +### Secret variables + +Secrets are injected into your functions using environment variables. + +By defining variables in the provider section of the `serverless.yml`, you add them to +the environment of the deployed function: + +```yaml +provider: + ... + environment: + A_VARIABLE: ${env:A_VARIABLE} +``` + +From there, you can reference them in your functions as well. +Remember to add `A_VARIABLE` to your GitLab CI variables under **Settings > CI/CD > Variables**, and it will get picked up and deployed with your function. + +NOTE: **Note:** +Anyone with access to the AWS environemnt may be able to see the values of those +variables persisted in the lambda definition. + +### Setting up CORS + +If you want to set up a web page that makes calls to your function, like we have done in the [template](https://gitlab.com/gitlab-org/project-templates/serverless-framework/), you need to deal with the Cross-Origin Resource Sharing (CORS). + +The quick way to do that is to add the `cors: true` flag to the HTTP endpoint in your `serverless.yml`: + +```yaml +functions: + hello: + handler: src/handler.hello + events: + - http: # Rewrite this part to enable CORS + path: hello + method: get + cors: true # <-- CORS here +``` + +You also need to return CORS specific headers in your function response: + +```javascript +'use strict'; + +module.exports.hello = async event => { + return { + statusCode: 200, + headers: { + // Uncomment the line below if you need access to cookies or authentication + // 'Access-Control-Allow-Credentials': true, + 'Access-Control-Allow-Origin': '*' + }, + body: JSON.stringify( + { + message: 'Your function executed successfully!' + }, + null, + 2 + ), + }; +}; +``` + +For more information, see the [Your CORS and API Gateway survival guide](https://serverless.com/blog/cors-api-gateway-survival-guide/) +blog post written by the Serverless Framework team. + +### Writing automated tests + +The [Serverless Framework](https://gitlab.com/gitlab-org/project-templates/serverless-framework/) +example project shows how to use Jest, Axios, and `serverless-offline` plugin to do +automated testing of both local and deployed serverless function. + +## Examples and template + +The example code is available: -To see the example code for this example please follow the link below: +- As a [cloneable repository](https://gitlab.com/gitlab-org/serverless/examples/serverless-framework-js). +- In a version with [tests and secret variables](https://gitlab.com/gitlab-org/project-templates/serverless-framework/). -- [Node.js example](https://gitlab.com/gitlab-org/serverless/examples/serverless-framework-js): Deploy a AWS Lambda Javascript function + API Gateway using Serverless Framework and GitLab CI/CD +You can also use a [template](https://docs.gitlab.com/ee/gitlab-basics/create-project.html#project-templates) +(based on the version with tests and secret variables) from within the GitLab UI (see +the `Serverless Framework/JS` template). diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index ffd7b0c0f2a..ae04dbab1a0 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -13,7 +13,7 @@ GitLab supports several ways deploy Serverless applications in both Kubernetes E Currently we support: -- [Knative](#knative): Build Knative applications with Knative and gitlabktl on GKE. +- [Knative](#knative): Build Knative applications with Knative and `gitlabktl` on GKE and EKS. - [AWS Lambda](aws.md): Create serverless applications via the Serverless Framework and GitLab CI. ## Knative @@ -36,10 +36,12 @@ With GitLab Serverless, you can deploy both functions-as-a-service (FaaS) and se To run Knative on GitLab, you will need: 1. **Existing GitLab project:** You will need a GitLab project to associate all resources. The simplest way to get started: - - - If you are planning on deploying functions, clone the [functions example project](https://gitlab.com/knative-examples/functions) to get started. - - If you are planning on deploying a serverless application, clone the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get started. - + - If you are planning on [deploying functions](#deploying-functions), + clone the [functions example project](https://gitlab.com/knative-examples/functions) to get + started. + - If you are planning on [deploying a serverless application](#deploying-serverless-applications), + clone the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get + started. 1. **Kubernetes Cluster:** An RBAC-enabled Kubernetes cluster is required to deploy Knative. The simplest way to get started is to add a cluster using [GitLab's GKE integration](../add_remove_clusters.md#gke-cluster). The set of minimum recommended cluster specifications to run Knative is 3 nodes, 6 vCPUs, and 22.50 GB memory. @@ -58,7 +60,7 @@ To run Knative on GitLab, you will need: 1. **`serverless.yml`** (for [functions only](#deploying-functions)): When using serverless to deploy functions, the `serverless.yml` file will contain the information for all the functions being hosted in the repository as well as a reference to the runtime being used. -1. **`Dockerfile`** (for [applications only](#deploying-serverless-applications): Knative requires a +1. **`Dockerfile`** (for [applications only](#deploying-serverless-applications)): Knative requires a `Dockerfile` in order to build your applications. It should be included at the root of your project's repo and expose port `8080`. `Dockerfile` is not require if you plan to build serverless functions using our [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes). @@ -87,7 +89,7 @@ The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22. for other platforms [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/). 1. The Ingress is now available at this address and will route incoming requests to the proper service based on the DNS - name in the request. To support this, a wildcard DNS A record should be created for the desired domain name. For example, + name in the request. To support this, a wildcard DNS record should be created for the desired domain name. For example, if your Knative base domain is `knative.info` then you need to create an A record or CNAME record with domain `*.knative.info` pointing the ip address or hostname of the Ingress. @@ -116,7 +118,8 @@ You must do the following: 1. Ensure GitLab can manage Knative: - For a non-GitLab managed cluster, ensure that the service account for the token - provided can manage resources in the `serving.knative.dev` API group. + provided can manage resources in the `serving.knative.dev` API group. It will also + need list access to the deployments in the `knative-serving` namespace. - For a GitLab managed cluster, if you added the cluster in [GitLab 12.1 or later](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/30235), then GitLab will already have the required access and you can proceed to the next step. @@ -153,6 +156,19 @@ You must do the following: - delete - patch - watch + --- + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRole + metadata: + name: gitlab-knative-version-role + rules: + - apiGroups: + - apps + resources: + - deployments + verbs: + - list + - get ``` Then run the following command: @@ -217,26 +233,40 @@ Or: ## Supported runtimes -Serverless functions for GitLab can be written in 6 supported languages: +Serverless functions for GitLab can be run using: + +- [GitLab-managed](#gitlab-managed-runtimes) runtimes. +- [OpenFaaS](#openfaas-runtimes) runtimes. -- NodeJS and Ruby, with GitLab-managed and OpenFaas runtimes. -- C#, Go, PHP, and Python with OpenFaaS runtimes only. +If a runtime is not available for the required programming language, consider deploying a +[serverless application](#deploying-serverless-applications). -### GitLab managed runtimes +### GitLab-managed runtimes -Currently the following [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes) are offered: +Currently the following GitLab-managed [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes) +are available: -- ruby -- node.js -- Dockerfile +- `go` (proof of concept) +- `nodejs` +- `ruby` -`Dockerfile` presence is assumed when a runtime is not specified. +You must provide a `Dockerfile` to run serverless functions if no runtime is specified. ### OpenFaaS runtimes > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/29253) in GitLab 12.5. [OpenFaaS classic runtimes](https://github.com/openfaas/templates#templates-in-store) can be used with GitLab serverless. + +OpenFaas runtimes are available for the following languages: + +- C# +- Go +- NodeJS +- PHP +- Python +- Ruby + Runtimes are specified using the pattern: `openfaas/classic/<template_name>`. The following example shows how to define a function in `serverless.yml` using an OpenFaaS runtime: @@ -311,17 +341,21 @@ project): provider: name: triggermesh - environment: + envs: FOO: value + secrets: + - my-secrets functions: echo-js: handler: echo-js source: ./echo-js - runtime: https://gitlab.com/gitlab-org/serverless/runtimes/nodejs + runtime: gitlab/runtimes/nodejs description: "node.js runtime function" - environment: + envs: MY_FUNCTION: echo-js + secrets: + - my-secrets ``` Explanation of the fields used above: @@ -338,7 +372,8 @@ Explanation of the fields used above: | Parameter | Description | |-----------|-------------| | `name` | Indicates which provider is used to execute the `serverless.yml` file. In this case, the TriggerMesh middleware. | -| `environment` | Includes the environment variables to be passed as part of function execution for **all** functions in the file, where `FOO` is the variable name and `BAR` are he variable contents. You may replace this with you own variables. | +| `envs` | Includes the environment variables to be passed as part of function execution for **all** functions in the file, where `FOO` is the variable name and `BAR` are he variable contents. You may replace this with you own variables. | +| `secrets` | Includes the contents of the Kubernetes secret as environment variables accessible to be passed as part of function execution for **all** functions in the file. The secrets are expected in ini format. | ### `functions` @@ -349,9 +384,29 @@ subsequent lines contain the function attributes. |-----------|-------------| | `handler` | The function's name. | | `source` | Directory with sources of a functions. | -| `runtime` (optional)| The runtime to be used to execute the function. When the runtime is not specified, we assume that `Dockerfile` is present in the function directory specified by `source`. | +| `runtime` (optional)| The runtime to be used to execute the function. This can be a runtime alias (see [Runtime aliases](#runtime-aliases)), or it can be a full URL to a custom runtime repository. When the runtime is not specified, we assume that `Dockerfile` is present in the function directory specified by `source`. | | `description` | A short description of the function. | -| `environment` | Sets an environment variable for the specific function only. | +| `envs` | Sets an environment variable for the specific function only. | +| `secrets` | Includes the contents of the Kubernetes secret as environment variables accessible to be passed as part of function execution for the specific function only. The secrets are expected in ini format. | + +### Deployment + +#### Runtime aliases + +The optional `runtime` parameter can refer to one of the following runtime aliases (also see [Supported runtimes](#supported-runtimes)): + +| Runtime alias | Maintained by | +|-------------|---------------| +| `gitlab/runtimes/go` | GitLab | +| `gitlab/runtimes/nodejs` | GitLab | +| `gitlab/runtimes/ruby` | GitLab | +| `openfaas/classic/csharp` | OpenFaaS | +| `openfaas/classic/go` | OpenFaaS | +| `openfaas/classic/node` | OpenFaaS | +| `openfaas/classic/php7` | OpenFaaS | +| `openfaas/classic/python` | OpenFaaS | +| `openfaas/classic/python3` | OpenFaaS | +| `openfaas/classic/ruby` | OpenFaaS | After the `gitlab-ci.yml` template has been added and the `serverless.yml` file has been created, pushing a commit to your project will result in a CI pipeline @@ -388,6 +443,33 @@ The sample function can now be triggered from any HTTP client using a simple `PO  +### Secrets + +To access your Kubernetes secrets from within your function, the secrets should be created under the namespace of your serverless deployment. + +#### CLI example + +```bash +kubectl create secret generic my-secrets -n "$KUBE_NAMESPACE" --from-literal MY_SECRET=imverysecure +``` + +#### Part of deployment job + +You can extend your `.gitlab-ci.yml` to create the secrets during deployment using the [environment variables](../../../../ci/variables/README.md) +stored securely under your GitLab project. + +```yaml +deploy:function: + stage: deploy + environment: production + extends: .serverless:deploy:functions + before_script: + - kubectl create secret generic my-secret + --from-literal MY_SECRET="$GITLAB_SECRET_VARIABLE" + --namespace "$KUBE_NAMESPACE" + --dry-run -o yaml | kubectl apply -f - +``` + ### Running functions locally Running a function locally is a good way to quickly verify behavior during development. @@ -427,9 +509,10 @@ To run a function locally: > Introduced in GitLab 11.5. -Serverless applications are the building block of serverless functions. They are useful in scenarios where an existing -runtime does not meet the needs of an application, such as one written in a language that has no runtime available. Note -though that serverless applications should be stateless! +Serverless applications are an alternative to [serverless functions](#deploying-functions). +They are useful in scenarios where an existing runtime does not meet the needs of an application, +such as one written in a language that has no runtime available. Note though that serverless +applications should be stateless! NOTE: **Note:** You can reference and import the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get started. @@ -780,3 +863,23 @@ The instructions below relate to installing and running Certbot on a Linux serve After your changes are running on your Knative cluster, you can begin using the HTTPS protocol for secure access your deployed Knative services. In the event a mistake is made during this process and you need to update the cert, you will need to edit the gateway `knative-ingress-gateway` to switch back to `PASSTHROUGH` mode. Once corrections are made, edit the file again so the gateway will use the new certificates. + +## Using an older version of `gitlabktl` + +There may be situations where you want to run an older version of `gitlabktl`. This +requires setting an older version of the `gitlabktl` image in the `.gitlab-ci.yml file.` + +To set an older version, add `image:` to the `functions:deploy` block. For example: + +```yaml +functions:deploy: + extends: .serverless:deploy:functions + environment: production + image: registry.gitlab.com/gitlab-org/gitlabktl:0.5.0 +``` + +Different versions are available by changing the version tag at the end of the registry URL in the +format `registry.gitlab.com/gitlab-org/gitlabktl:<version>`. + +For a full inventory of available `gitlabktl` versions, see the `gitlabktl` project's +[container registry](https://gitlab.com/gitlab-org/gitlabktl/container_registry). diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index b14d7f821bb..98e9188ed9b 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -14,7 +14,7 @@ With Deploy Boards you can gain more insight into deploys with benefits such as: - Following a deploy from the start, not just when it's done - Watching the rollout of a build across multiple servers -- Finer state detail (Waiting, Deploying, Finished, Unknown) +- Finer state detail (Succeeded, Running, Failed, Pending, Unknown) - See [Canary Deployments](canary_deployments.md) Here's an example of a Deploy Board of the production environment. diff --git a/doc/user/project/img/service_desk_disabled.png b/doc/user/project/img/service_desk_disabled.png Binary files differindex edae7e76986..ba11b508682 100644 --- a/doc/user/project/img/service_desk_disabled.png +++ b/doc/user/project/img/service_desk_disabled.png diff --git a/doc/user/project/img/service_desk_enabled.png b/doc/user/project/img/service_desk_enabled.png Binary files differindex 9c143ff58cd..aee2b53a680 100644 --- a/doc/user/project/img/service_desk_enabled.png +++ b/doc/user/project/img/service_desk_enabled.png diff --git a/doc/user/project/import/gemnasium.md b/doc/user/project/import/gemnasium.md index f1121745c7e..20614bf5cc5 100644 --- a/doc/user/project/import/gemnasium.md +++ b/doc/user/project/import/gemnasium.md @@ -98,7 +98,7 @@ back to both GitLab and GitHub when completed. 1. The result of the job will be visible directly from the pipeline view: -  +  NOTE: **Note:** If you don't commit very often to your project, you may want to use diff --git a/doc/user/project/index.md b/doc/user/project/index.md index c173d3d3e11..29de700af4d 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -200,9 +200,14 @@ When [renaming a user](../profile/index.md#changing-your-username), ## Use your project as a Go package -Any project can be used as a Go package including private projects in subgroups. To use packages -hosted in private projects with the `go get` command, use a [`.netrc` file](https://ec.haxx.se/usingcurl-netrc.html) -and a [personal access token](../profile/personal_access_tokens.md) in the password field. +Any project can be used as a Go package including private projects in subgroups. +GitLab responds correctly to `go get` and `godoc.org` discovery requests, +including the [`go-import`](https://golang.org/cmd/go/#hdr-Remote_import_paths) +and [`go-source`](https://github.com/golang/gddo/wiki/Source-Code-Links) meta +tags, respectively. To use packages hosted in private projects with the `go get` +command, use a [`.netrc` file](https://ec.haxx.se/usingcurl-netrc.html) and a +[personal access token](../profile/personal_access_tokens.md) in the password +field. For example: diff --git a/doc/user/project/integrations/github.md b/doc/user/project/integrations/github.md index 9d16748085c..b8b073af2a4 100644 --- a/doc/user/project/integrations/github.md +++ b/doc/user/project/integrations/github.md @@ -33,6 +33,9 @@ with `repo:status` access granted: 1. Optionally uncheck **Static status check names** checkbox to disable static status check names. 1. Save or optionally click "Test Settings". +Once the integration is configured, see [Pipelines for external pull requests](../../../ci/ci_cd_for_external_repos/#pipelines-for-external-pull-requests) +to configure pipelines to run for open pull requests. + #### Static / dynamic status check names > - Introduced in GitLab 11.5: using static status check names as opt-in option. diff --git a/doc/user/project/integrations/img/unify_circuit_configuration.png b/doc/user/project/integrations/img/unify_circuit_configuration.png Binary files differnew file mode 100644 index 00000000000..285d4f92030 --- /dev/null +++ b/doc/user/project/integrations/img/unify_circuit_configuration.png diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 874a1092b73..d08c8699eba 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -20,7 +20,7 @@ Here's how the integration responds when you take the following actions in GitLa - **Mention a Jira issue ID** in a commit message or MR (merge request). - GitLab hyperlinks to the Jira issue. - The Jira issue adds an issue link to the commit/MR in GitLab. - - The Jira issue adds a comment reflecting the comment made in GitLab, the comment author, and a link to the commit/MR in GitLab. + - The Jira issue adds a comment reflecting the comment made in GitLab, the comment author, and a link to the commit/MR in GitLab, unless this commenting to Jira is [disabled](#disabling-comments-on-jira-issues). - **Mention that a commit or MR 'closes', 'resolves', or 'fixes' a Jira issue ID**. When the commit is made on the project's default branch (usually master) or the change is merged to the default branch: - GitLab's merge request page displays a note that it "Closed" the Jira issue, with a link to the issue. (Note: Before the merge, an MR will display that it "Closes" the Jira issue.) - The Jira issue shows the activity and the Jira issue is closed, or otherwise transitioned. @@ -95,6 +95,15 @@ with all Jira projects in your Jira instance and you'll see the Jira link on the  +### Disabling comments on Jira issues + +When you reference a Jira issue, it will always link back to the source commit/MR in GitLab, however, you can control whether GitLab will also cross-post a comment to the Jira issue. That functionality is enabled by default. + +To disable the automated commenting on Jira issues: + +1. Open the [Integrations page](project_services.md#accessing-the-project-services) and select **Jira**. +1. In the **Event Action** section, uncheck **Comment**. + ## Jira issues By now you should have [configured Jira](#configuring-jira) and enabled the diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index 315039f82b3..f960c59850d 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -54,6 +54,7 @@ Click on the service links to see further configuration instructions and details | [Prometheus](prometheus.md) | Monitor the performance of your deployed apps | | Pushover | Pushover makes it easy to get real-time notifications on your Android device, iPhone, iPad, and Desktop | | [Redmine](redmine.md) | Redmine issue tracker | +| [Unify Circuit](unify_circuit.md) | Receive events notifications in Unify Circuit | | [YouTrack](youtrack.md) | YouTrack issue tracker | ## Push hooks limit diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index d3d4afefb59..3b7309ea7e4 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -476,12 +476,12 @@ Prometheus server. > [Introduced][ce-29691] in GitLab 12.2. -It is possible to display metrics charts within [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm). +It is possible to display metrics charts within [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm). The maximum number of embeds allowed in a GitLab Flavored Markdown field is 100. NOTE: **Note:** Requires [Kubernetes](prometheus_library/kubernetes.md) metrics. -To display a metric chart, include a link of the form `https://<root_url>/<project>/environments/<environment_id>/metrics`. +To display a metric chart, include a link of the form `https://<root_url>/<project>/-/environments/<environment_id>/metrics`. A single chart may also be embedded. You can generate a link to the chart via the dropdown located on the right side of the chart: @@ -520,7 +520,7 @@ The sharing dialog within Grafana provides the link, as highlighted below. NOTE: **Note:** For this embed to display correctly the Grafana instance must be available to the target user, either as a public dashboard or on the same network. -Copy the link and add an image tag as [inline HTML](../../markdown.md#inline-html) in your markdown. You may tweak the query parameters as required. For instance, removing the `&from=` and `&to=` parameters will give you a live chart. Here is example markup for a live chart from GitLab's public dashboard: +Copy the link and add an image tag as [inline HTML](../../markdown.md#inline-html) in your Markdown. You may tweak the query parameters as required. For instance, removing the `&from=` and `&to=` parameters will give you a live chart. Here is example markup for a live chart from GitLab's public dashboard: ```html <img src="https://dashboards.gitlab.com/render/d-solo/RZmbBr7mk/gitlab-triage?orgId=1&refresh=30s&var-env=gprd&var-environment=gprd&var-prometheus=prometheus-01-inf-gprd&var-prometheus_app=prometheus-app-01-inf-gprd&var-backend=All&var-type=All&var-stage=main&panelId=1247&width=1000&height=300"/> @@ -534,7 +534,7 @@ This will render like so: > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/31376) in GitLab 12.5. -Each project can support integration with one Grafana instance. This configuration allows a user to copy a link to a panel in Grafana, then paste it into a GitLab markdown field. The chart will be rendered in the GitLab chart format. +Each project can support integration with one Grafana instance. This configuration allows a user to copy a link to a panel in Grafana, then paste it into a GitLab Markdown field. The chart will be rendered in the GitLab chart format. Prerequisites for embedding from a Grafana instance: @@ -562,7 +562,7 @@ Prerequisites for embedding from a Grafana instance: 1. If your Prometheus queries use Grafana's custom template variables, ensure that "Template variables" and "Current time range" options are toggled to **On**. Of Grafana global template variables, only `$__interval`, `$__from`, and `$__to` are currently supported.  1. Click **Copy** to copy the URL to the clipboard. -1. In GitLab, paste the URL into a markdown field and save. The chart will take a few moments to render. +1. In GitLab, paste the URL into a Markdown field and save. The chart will take a few moments to render.  ## Troubleshooting @@ -574,6 +574,7 @@ If the "No data found" screen continues to appear, it could be due to: are not labeled correctly. To test this, connect to the Prometheus server and [run a query](prometheus_library/kubernetes.html#metrics-supported), replacing `$CI_ENVIRONMENT_SLUG` with the name of your environment. +- You may need to re-add the GitLab predefined common metrics. This can be done by running the [import common metrics rake task](../../../administration/raketasks/maintenance.md#import-common-metrics). [autodeploy]: ../../../topics/autodevops/index.md#auto-deploy [kubernetes]: https://kubernetes.io diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index 3c6919561fd..cf46456ca42 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -22,7 +22,7 @@ NGINX server metrics are detected, which tracks the pages and content directly s To get started with NGINX monitoring, you should first enable the [VTS statistics](https://github.com/vozlt/nginx-module-vts)) module for your NGINX server. This will capture and display statistics in an HTML readable form. Next, you should install and configure the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter) which parses these statistics and translates them into a Prometheus monitoring endpoint. -If you are using NGINX as your Kubernetes ingress, GitLab will [automatically detect](nginx_ingress.md) the metrics once enabled in 0.9.0 and later releases. +If you are using NGINX as your Kubernetes Ingress, GitLab will [automatically detect](nginx_ingress.md) the metrics once enabled in 0.9.0 and later releases. ## Specifying the Environment label diff --git a/doc/user/project/integrations/unify_circuit.md b/doc/user/project/integrations/unify_circuit.md new file mode 100644 index 00000000000..e357afb9224 --- /dev/null +++ b/doc/user/project/integrations/unify_circuit.md @@ -0,0 +1,27 @@ +# Unify Circuit service + +The Unify Circuit service sends notifications from GitLab to the conversation for which the webhook was created. + +## On Unify Circuit + +1. Open the conversation in which you want to see the notifications. +1. From the conversation menu, select **Configure Webhooks**. +1. Click **ADD WEBHOOK** and fill in the name of the bot that will post the messages. Optionally define avatar. +1. Click **SAVE** and copy the **Webhook URL** of your webhook. + +For more information, see the [Unify Circuit documentation for configuring incoming webhooks](https://www.circuit.com/unifyportalfaqdetail?articleId=164448). + +## On GitLab + +When you have the **Webhook URL** for your Unify Circuit conversation webhook, you can set up the GitLab service. + +1. Navigate to the [Integrations page](project_services.md#accessing-the-project-services) in your project's settings, i.e. **Project > Settings > Integrations**. +1. Select the **Unify Circuit** project service to configure it. +1. Check the **Active** checkbox to turn on the service. +1. Check the checkboxes corresponding to the GitLab events you want to receive in Unify Circuit. +1. Paste the **Webhook URL** that you copied from the Unify Circuit configuration step. +1. Configure the remaining options and click `Save changes`. + +Your Unify Circuit conversation will now start receiving GitLab event notifications as configured. + + diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index d0f538a4b52..bb946574371 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -47,6 +47,20 @@ and **per project and per group** for **GitLab Enterprise Edition**. Navigate to the webhooks page by going to your project's **Settings ➔ Integrations**. +## Maximum number of webhooks (per tier) + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/20730) in GitLab 12.6. + +A maximum number of project webhooks applies to each [GitLab.com +tier](https://about.gitlab.com/pricing/), as shown in the following table: + +| Tier | Number of webhooks per project | +|----------|--------------------------------| +| Free | 10 | +| Bronze | 20 | +| Silver | 30 | +| Gold | 100 | + ## Use-cases - You can set up a webhook in GitLab to send a notification to @@ -969,7 +983,7 @@ X-Gitlab-Event: Wiki Page Hook "http_url": "http://example.com/root/awesome-project.git" }, "wiki": { - "web_url": "http://example.com/root/awesome-project/wikis/home", + "web_url": "http://example.com/root/awesome-project/-/wikis/home", "git_ssh_url": "git@example.com:root/awesome-project.wiki.git", "git_http_url": "http://example.com/root/awesome-project.wiki.git", "path_with_namespace": "root/awesome-project.wiki", @@ -981,7 +995,7 @@ X-Gitlab-Event: Wiki Page Hook "format": "markdown", "message": "adding an awesome page to the wiki", "slug": "awesome", - "url": "http://example.com/root/awesome-project/wikis/awesome", + "url": "http://example.com/root/awesome-project/-/wikis/awesome", "action": "create" } } @@ -1076,6 +1090,7 @@ X-Gitlab-Event: Pipeline Hook "finished_at": null, "when": "manual", "manual": true, + "allow_failure": false, "user":{ "name": "Administrator", "username": "root", @@ -1097,6 +1112,7 @@ X-Gitlab-Event: Pipeline Hook "finished_at": null, "when": "on_success", "manual": false, + "allow_failure": false, "user":{ "name": "Administrator", "username": "root", @@ -1123,6 +1139,7 @@ X-Gitlab-Event: Pipeline Hook "finished_at": "2016-08-12 15:26:29 UTC", "when": "on_success", "manual": false, + "allow_failure": false, "user":{ "name": "Administrator", "username": "root", @@ -1149,6 +1166,7 @@ X-Gitlab-Event: Pipeline Hook "finished_at": "2016-08-12 15:25:26 UTC", "when": "on_success", "manual": false, + "allow_failure": false, "user":{ "name": "Administrator", "username": "root", @@ -1175,6 +1193,7 @@ X-Gitlab-Event: Pipeline Hook "finished_at": null, "when": "on_success", "manual": false, + "allow_failure": false, "user":{ "name": "Administrator", "username": "root", @@ -1218,6 +1237,7 @@ X-Gitlab-Event: Job Hook "build_duration": null, "build_allow_failure": false, "build_failure_reason": "script_failure", + "pipeline_id": 2366, "project_id": 380, "project_name": "gitlab-org/gitlab-test", "user": { @@ -1243,10 +1263,18 @@ X-Gitlab-Event: Job Hook "git_ssh_url": "git@192.168.64.1:gitlab-org/gitlab-test.git", "git_http_url": "http://192.168.64.1:3005/gitlab-org/gitlab-test.git", "visibility_level": 20 + }, + "runner": { + "active": true, + "is_shared": false, + "id": 380987, + "description": "shared-runners-manager-6.gitlab.com" } } ``` +Note that `commit.id` is the id of the pipeline, not the id of the commit. + ## Image URL rewriting From GitLab 11.2, simple image references are rewritten to use an absolute URL @@ -1271,7 +1299,7 @@ Markdown features, like link labels. ## Testing webhooks -You can trigger the webhook manually. Sample data from the project will be used.Sample data will take from the project. +You can trigger the webhook manually. Sample data from the project will be used. Sample data will take from the project. > For example: for triggering `Push Events` your project should have at least one commit.  diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index d8356abdd1c..e4264615488 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -12,7 +12,7 @@ requests are located. In GitLab, you can create project and group labels: -- **Project labels** can be assigned to epics, issues and merge requests in that project only. +- **Project labels** can be assigned to issues and merge requests in that project only. - **Group labels** can be assigned to any epics, issue and merge request in any project in that group, or any subgroups of the group. diff --git a/doc/user/project/members/img/project_members.png b/doc/user/project/members/img/project_members.png Binary files differnew file mode 100644 index 00000000000..5d44b5d957e --- /dev/null +++ b/doc/user/project/members/img/project_members.png diff --git a/doc/user/project/members/img/project_members_filter_v12_6.png b/doc/user/project/members/img/project_members_filter_v12_6.png Binary files differnew file mode 100644 index 00000000000..0207515ded0 --- /dev/null +++ b/doc/user/project/members/img/project_members_filter_v12_6.png diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 2f8394eb104..c069882e38f 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -10,6 +10,31 @@ or import a new user to your project. To view, edit, add, and remove project's members, go to your project's **Settings > Members**. +## Inherited membership + +When your project belongs to the group, group members inherit the membership and permission +level for the project from the group. + + + +From the image above, we can deduce the following things: + +- There are 3 members that have access to the project. +- User0 is a Reporter and has inherited their permissions from group `demo` + which contains current project. +- For User1 there is no indication of a group, therefore they belong directly + to the project we're inspecting. +- Administrator is the Owner and member of **all** groups and for that reason, + there is an indication of an ancestor group and inherited Owner permissions. + +[From](https://gitlab.com/gitlab-org/gitlab/issues/21727), you can filter this list +using dropdown on the right side: + + + +- **Show only direct members** displays only User1. +- **Show only inherited members** displays User0 and Administrator. + ## Add a user Right next to **People**, start typing the name or username of the user you diff --git a/doc/user/project/merge_requests/creating_merge_requests.md b/doc/user/project/merge_requests/creating_merge_requests.md index 084ebf32a92..1dec58a8bb0 100644 --- a/doc/user/project/merge_requests/creating_merge_requests.md +++ b/doc/user/project/merge_requests/creating_merge_requests.md @@ -31,6 +31,8 @@ also appear in the top right of the: In this case, the merge request will use the most recent branch you pushed changes to as the source branch, and `master` in the current project as the target. +You can also [create a new merge request directly from an issue](../repository/web_editor.md#create-a-new-branch-from-an-issue). + ## Workflow for new merge requests On the **New Merge Request** page, you can start by filling in the title and description diff --git a/doc/user/project/merge_requests/img/merge_request_tab_position_v12_6.png b/doc/user/project/merge_requests/img/merge_request_tab_position_v12_6.png Binary files differnew file mode 100644 index 00000000000..9284e58f456 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_request_tab_position_v12_6.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 1ca8c882ac7..203a2949243 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -40,6 +40,40 @@ B. Consider you're a web developer writing a webpage for your company's website: 1. Once approved, your merge request is [squashed and merged](squash_and_merge.md), and [deployed to staging with GitLab Pages](https://about.gitlab.com/blog/2016/08/26/ci-deployment-and-environments/) 1. Your production team [cherry picks](cherry_pick_changes.md) the merge commit into production +## Overview + +Merge requests (aka "MRs") display a great deal of information about the changes proposed. +The body of an MR contains its description, along with its widget (displaying information +about CI/CD pipelines, when present), followed by the discussion threads of the people +collaborating with that MR. + +MRs also contain navigation tabs from which you can see the discussion happening on the thread, +the list of commits, the list of pipelines and jobs, the code changes and inline code reviews. + +## Merge request navigation tabs at the top + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/33813) in GitLab 12.6. This positioning is experimental. + +So far, the navigation tabs present in merge requests to display **Discussion**, +**Commits**, **Pipelines**, and **Changes** were located after the merge request +widget. + +To facilitate this navigation without having to scroll up and down through the page +to find these tabs, based on user feedback, we are experimenting with a new positioning +of these tabs. They are now located at the top of the merge request, with a new +**Overview** tab, containing the description of the merge request followed by the +widget. Next to **Overview**, you can find **Pipelines**, **Commits**, and **Changes**. + + + +Please note this change is currently behind a feature flag which is enabled by default. For +self-managed instances, it can be disabled through the Rails console by a GitLab +administrator with the following command: + +```ruby +Feature.disable(:mr_tabs_position) +``` + ## Creating merge requests While making changes to files in the `master` branch of a repository is possible, it is not diff --git a/doc/user/project/merge_requests/merge_request_dependencies.md b/doc/user/project/merge_requests/merge_request_dependencies.md index c99e6663093..8b8aa30e75a 100644 --- a/doc/user/project/merge_requests/merge_request_dependencies.md +++ b/doc/user/project/merge_requests/merge_request_dependencies.md @@ -9,6 +9,8 @@ type: reference, concepts > - [Renamed](https://gitlab.com/gitlab-org/gitlab/merge_requests/17291) from "Cross-project dependencies" to "Merge Requests dependencies" in [GitLab Premium](https://about.gitlab.com/pricing/) 12.4. +> - Intra-project MR dependencies were [introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/16799) +in [GitLab Premium](https://about.gitlab.com/pricing/) 12.4. Merge request dependencies allows a required order of merging between merge requests to be expressed. If a merge request "depends on" another, @@ -20,10 +22,6 @@ only enforced for the dependent merge request. A merge request in a **CORE** or **STARTER** project can be a dependency of a **PREMIUM** merge request, but not vice-versa. -NOTE: **Note:** -A merge request can only depend on merge requests in a different project. Two -merge requests in the same project cannot depend on each other. - ## Use cases - Ensure changes to a library are merged before changes to a project that diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index 6630179ea47..e1ac8b2183c 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -69,7 +69,7 @@ For example, to that on merge requests there is always a passing job even though ```yaml enable_merge: - only: merge_requests + only: [merge_requests] script: - echo true ``` diff --git a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md index f693b0b1e72..97c16a9794d 100644 --- a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md +++ b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md @@ -68,7 +68,7 @@ list. By default, the diff shows only the parts of a file which are changed. To view more unchanged lines above or below a change click on the -**Expand up** or **Expand down** icons. You can also click on **Show all lines** +**Expand up** or **Expand down** icons. You can also click on **Show unchanged lines** to expand the entire file.  diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 21a4e3d8ead..eacc1fd12dc 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -68,8 +68,8 @@ If you are expanding from a few projects to a larger number of projects within t From the project milestone list page, you can promote a project milestone to a group milestone. This will merge all project milestones across all projects in this group with the same name into a single group milestones. All issues and merge requests that previously were assigned one of these project milestones will now be assigned the new group milestones. This action cannot be reversed and the changes are permanent. ->**Note:** -Not all features on the project milestone view are available on the group milestone view. If you promote a project milestone to a group milestone, you will lose these features. See [Milestone view](#milestone-view) to see which features are missing from the group milestone view. +CAUTION: **Caution:** +From GitLab 12.4 and earlier, some information is lost when you promote a project milestone to a group milestone. Not all features on the project milestone view are available on the group milestone view. If you promote a project milestone to a group milestone, you will lose these features. See [Milestone view](#milestone-view) to see which features are missing from the group milestone view.  @@ -105,14 +105,17 @@ The milestone view shows the title and description. There are also tabs below these that show the following: -- Issues - Shows all issues assigned to the milestone. These are displayed in three columns: Unstarted issues, ongoing issues, and completed issues. -- Merge requests - Shows all merge requests assigned to the milestone. These are displayed in four columns: Work in progress merge requests, waiting for merge, rejected, and closed. -- Participants - Shows all assignees of issues assigned to the milestone. -- Labels - Shows all labels that are used in issues assigned to the milestone. +- **Issues**: Shows all issues assigned to the milestone. These are displayed in three columns named: + - Unstarted Issues (open and unassigned) + - Ongoing Issues (open and assigned) + - Completed Issues (closed) +- **Merge Requests**: Shows all merge requests assigned to the milestone. These are displayed in four columns named: + - Work in progress (open and unassigned) + - Waiting for merge (open and unassigned) + - Rejected (closed) + - Merged +- **Participants**: Shows all assignees of issues assigned to the milestone. +- **Labels**: Shows all labels that are used in issues assigned to the milestone. ### Project Burndown Charts **(STARTER)** diff --git a/doc/user/project/operations/error_tracking.md b/doc/user/project/operations/error_tracking.md index 04eda026bc3..912d7fdbef5 100644 --- a/doc/user/project/operations/error_tracking.md +++ b/doc/user/project/operations/error_tracking.md @@ -32,7 +32,7 @@ GitLab provides an easy way to connect Sentry to your project: 1. Click **Save changes** for the changes to take effect. 1. You can now visit **Operations > Error Tracking** in your project's sidebar to [view a list](#error-tracking-list) of Sentry errors. -### Enabling Gitlab issues links +### Enabling GitLab issues links You may also want to enable Sentry's GitLab integration by following the steps in the [Sentry documentation](https://docs.sentry.io/workflow/integrations/global-integrations/#gitlab) @@ -42,9 +42,9 @@ NOTE: **Note:** You will need at least Reporter [permissions](../../permissions.md) to view the Error Tracking list. The Error Tracking list may be found at **Operations > Error Tracking** in your project's sidebar. -Errors can be filtered by title. +Errors can be filtered by title or sorted by Frequency, First Seen or Last Seen. Errors are always sorted in descending order by the field specified. - + ## Error Details @@ -52,7 +52,13 @@ From error list, users can navigate to the error details page by clicking the ti This page has: -- A link to Sentry issue. -- A full stack trace along with other details. +- A link to the Sentry issue. +- Other details about the issue, including a full stack trace. - +If the error has not been linked to an existing GitLab issue, a 'Create Issue' button will be visible: + + + +If a link does exist, it will be shown in the details and the 'Create Issue' button will be hidden: + + diff --git a/doc/user/project/operations/feature_flags.md b/doc/user/project/operations/feature_flags.md index c05f8fa8bc4..723f9d69995 100644 --- a/doc/user/project/operations/feature_flags.md +++ b/doc/user/project/operations/feature_flags.md @@ -75,7 +75,7 @@ To define specs for each environment: 1. Set the status and rollout strategy of the additional spec. This status and rollout strategy combination takes precedence over the default spec since we always use the most specific match available. 1. Click **Create feature flag** or **Update feature flag**. - + NOTE: **NOTE** We'd highly recommend you to use the [Environment](../../../ci/environments.md) @@ -119,17 +119,15 @@ CAUTION: **Caution:** If this strategy is selected, then the Unleash client **must** be given a user ID for the feature to be enabled. See the [Ruby example](#ruby-application-example) below. -### Target users strategy +#### User IDs -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8240) in GitLab 12.2. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8240) in GitLab 12.2. [Updated](https://gitlab.com/gitlab-org/gitlab/issues/34363) to be defined per environment in GitLab 12.6. A feature flag may be enabled for a list of target users. It is implemented using the Unleash [`userWithId`](https://unleash.github.io/docs/activation_strategy#userwithid) activation strategy. -The feature will always be enabled for all users in the list across all environments even if the matching environment spec **Status** is disabled. - - +User IDs should be a comma separated list of values. For example, `user@example.com, user2@example.com`, or `username1,username2,username3`, etc. CAUTION: **Caution:** The Unleash client **must** be given a user ID for the feature to be enabled for diff --git a/doc/user/project/operations/img/error_details_v12_6.png b/doc/user/project/operations/img/error_details_v12_6.png Binary files differnew file mode 100644 index 00000000000..b9152bd2c11 --- /dev/null +++ b/doc/user/project/operations/img/error_details_v12_6.png diff --git a/doc/user/project/operations/img/error_details_with_issue_v12_6.png b/doc/user/project/operations/img/error_details_with_issue_v12_6.png Binary files differnew file mode 100644 index 00000000000..963b70bd1e4 --- /dev/null +++ b/doc/user/project/operations/img/error_details_with_issue_v12_6.png diff --git a/doc/user/project/operations/img/error_tracking_list.png b/doc/user/project/operations/img/error_tracking_list.png Binary files differdeleted file mode 100644 index 79b464e021e..00000000000 --- a/doc/user/project/operations/img/error_tracking_list.png +++ /dev/null diff --git a/doc/user/project/operations/img/error_tracking_list_v12_6.png b/doc/user/project/operations/img/error_tracking_list_v12_6.png Binary files differnew file mode 100644 index 00000000000..b99c83c14d3 --- /dev/null +++ b/doc/user/project/operations/img/error_tracking_list_v12_6.png diff --git a/doc/user/project/operations/img/specs_list.png b/doc/user/project/operations/img/specs_list.png Binary files differdeleted file mode 100644 index 43d069c09ce..00000000000 --- a/doc/user/project/operations/img/specs_list.png +++ /dev/null diff --git a/doc/user/project/operations/img/specs_list_v12_6.png b/doc/user/project/operations/img/specs_list_v12_6.png Binary files differnew file mode 100644 index 00000000000..ea429802a40 --- /dev/null +++ b/doc/user/project/operations/img/specs_list_v12_6.png diff --git a/doc/user/project/operations/img/target_users_v12_2.png b/doc/user/project/operations/img/target_users_v12_2.png Binary files differdeleted file mode 100644 index c88d2b7be97..00000000000 --- a/doc/user/project/operations/img/target_users_v12_2.png +++ /dev/null diff --git a/doc/user/project/operations/index.md b/doc/user/project/operations/index.md index 2da9c3e70cf..df7ce61525e 100644 --- a/doc/user/project/operations/index.md +++ b/doc/user/project/operations/index.md @@ -6,6 +6,7 @@ your applications: - Collect [Prometheus metrics](../integrations/prometheus_library/index.md). - Deploy to different [environments](../../../ci/environments.md). - Connect your project to a [Kubernetes cluster](../clusters/index.md). +- Manage your infrastructure with [Infrastructure as Code](../../infrastructure/index.md) approaches. - Discover and view errors generated by your applications with [Error Tracking](error_tracking.md). - Create, toggle, and remove [Feature Flags](feature_flags.md). **(PREMIUM)** - [Trace](tracing.md) the performance and health of a deployed application. **(ULTIMATE)** diff --git a/doc/user/project/pages/getting_started/fork_sample_project.md b/doc/user/project/pages/getting_started/fork_sample_project.md index ac1a29ca2a0..9c58189e984 100644 --- a/doc/user/project/pages/getting_started/fork_sample_project.md +++ b/doc/user/project/pages/getting_started/fork_sample_project.md @@ -41,7 +41,7 @@ forking (copying) a [sample project from the most popular Static Site Generators and click **Run pipeline** to trigger GitLab CI/CD to build and deploy your site to the server. 1. Once the pipeline has finished successfully, find the link to visit your - website from your project's **Settings > Pages**. It can take aproximatelly + website from your project's **Settings > Pages**. It can take approximately 30 minutes to be deployed. You can also take some **optional** further steps: diff --git a/doc/user/project/pages/pages_access_control.md b/doc/user/project/pages/pages_access_control.md index cd715c6e3b9..1d8119cfe87 100644 --- a/doc/user/project/pages/pages_access_control.md +++ b/doc/user/project/pages/pages_access_control.md @@ -11,7 +11,7 @@ You can enable Pages access control on your project, so that only [members of your project](../../permissions.md#project-members-permissions) (at least Guest) can access your website: -1. Navigate to your project's **Settings > General > Permissions**. +1. Navigate to your project's **Settings > General** and expand **Visibility, project features, permissions**. 1. Toggle the **Pages** button to enable the access control. NOTE: **Note:** diff --git a/doc/user/project/pipelines/job_artifacts.md b/doc/user/project/pipelines/job_artifacts.md index 794c3030c6a..4ea46399635 100644 --- a/doc/user/project/pipelines/job_artifacts.md +++ b/doc/user/project/pipelines/job_artifacts.md @@ -103,8 +103,12 @@ It is possible to download the latest artifacts of a job via a well known URL so you can use it for scripting purposes. NOTE: **Note:** -The latest artifacts are considered as the artifacts created by jobs in the -latest pipeline that succeeded for the specific ref. +The latest artifacts are created by jobs in the **most recent** successful pipeline +for the specific ref. If you run two types of pipelines for the same ref, the latest +artifact will be determined by timing. For example, if a branch pipeline created +by merging a merge request runs at the same time as a scheduled pipeline, the +latest artifact will be from the pipeline that completed most recently. + Artifacts for other pipelines can be accessed with direct access to them. The structure of the URL to download the whole artifacts archive is the following: diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index 6480c7e0af9..ca888c69b37 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -67,26 +67,46 @@ For information about setting a maximum artifact size for a project, see ## Custom CI configuration path -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/12509) in GitLab 9.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/12509) in GitLab 9.4. +> - [Support for external `.gitlab-ci.yml` locations](https://gitlab.com/gitlab-org/gitlab/issues/14376) introduced in GitLab 12.6. By default we look for the `.gitlab-ci.yml` file in the project's root -directory. If you require a different location **within** the repository, -you can set a custom path that will be used to look up the configuration file, -this path should be **relative** to the root. +directory. If needed, you can specify an alternate path and file name, including locations outside the project. -Here are some valid examples: +To customize the path: -- `.gitlab-ci.yml` +1. Go to the project's **Settings > CI / CD**. +1. Expand the **General pipelines** section. +1. Provide a value in the **Custom CI configuration path** field. +1. Click **Save changes**. + +If the CI configuration is stored within the repository in a non-default +location, the path must be relative to the root directory. Examples of valid +paths and file names include: + +- `.gitlab-ci.yml` (default) - `.my-custom-file.yml` - `my/path/.gitlab-ci.yml` - `my/path/.my-custom-file.yml` -The path can be customized at a project level. To customize the path: +If the CI configuration will be hosted on an external site, the URL link must end with `.yml`: -1. Go to the project's **Settings > CI / CD**. -1. Expand the **General pipelines** section. -1. Provide a value in the **Custom CI configuration path** field. -1. Click **Save changes**. +- `http://example.com/generate/ci/config.yml` + +If the CI configuration will be hosted in a different project within GitLab, the path must be relative +to the root directory in the other project, with the group and project name added to the end: + +- `.gitlab-ci.yml@mygroup/another-project` +- `my/path/.my-custom-file.yml@mygroup/another-project` + +Hosting the configuration file in a separate project allows stricter control of the +configuration file. For example: + +- Create a public project to host the configuration file. +- Give write permissions on the project only to users who are allowed to edit the file. + +Other users and projects will be able to access the configuration file without being +able to edit it. ## Test coverage parsing @@ -132,6 +152,9 @@ Pipeline visibility is determined by: - Your current [user access level](../../permissions.md). - The **Public pipelines** project setting under your project's **Settings > CI/CD > General pipelines**. +NOTE: **Note:** +If the project visibility is set to **Private**, the [**Public pipelines** setting will have no effect](../../../ci/enable_or_disable_ci.md#per-project-user-setting). + This also determines the visibility of these related features: - Job output logs diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index 8952f845b96..11789f7d497 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -31,19 +31,25 @@ git push -o <push_option> ## Push options for GitLab CI/CD -If the `ci.skip` push option is used, the commit will be pushed, but no [CI pipeline](../../ci/pipelines.md) -will be created. +You can use push options to skip a CI/CD pipeline, or pass environment variables. -| Push option | Description | -| ----------- | ----------- | -| `ci.skip` | Do not create a CI pipeline for the latest push. | +| Push option | Description | +| ------------------------------ | ------------------------------------------------------------------------------------------- | +| `ci.skip` | Do not create a CI pipeline for the latest push. | +| `ci.variable="<name>=<value>"` | Provide [environment variables](../../ci/variables/README.md) to be used in a CI pipeline, if one is created due to the push. | -For example: +An example of using `ci.skip`: ```shell git push -o ci.skip ``` +An example of passing some environment variables for a pipeline: + +```shell +git push -o ci.variable="MAX_RETRIES=10" -o ci.variable="MAX_TIME=600" +``` + ## Push options for merge requests You can use Git push options to perform certain actions for merge requests at the same diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 61bc66a6a69..97ae429a33f 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -8,9 +8,9 @@ Quick actions are textual shortcuts for common actions on issues, epics, merge r and commits that are usually done by clicking buttons or dropdowns in GitLab's UI. You can enter these commands while creating a new issue or merge request, or in comments of issues, epics, merge requests, and commits. Each command should be -on a separate line in order to be properly detected and executed. Once executed, +on a separate line in order to be properly detected and executed. -> From [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/26672), an alert is displayed when a quick action is successfully applied. +> From [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/26672), once an action is executed, an alert is displayed when a quick action is successfully applied. ## Quick Actions for issues, merge requests and epics @@ -60,7 +60,7 @@ The following quick actions are applicable to descriptions, discussions and thre | `/remove_epic` | ✓ | | | Remove from epic **(ULTIMATE)** | | `/promote` | ✓ | | | Promote issue to epic **(ULTIMATE)** | | `/confidential` | ✓ | | | Make confidential | -| `/duplicate <#issue>` | ✓ | | | Mark this issue as a duplicate of another issue | +| `/duplicate <#issue>` | ✓ | | | Mark this issue as a duplicate of another issue and relate them for **(STARTER)** | | `/create_merge_request <branch name>` | ✓ | | | Create a new merge request starting from the current issue | | `/relate #issue1 #issue2` | ✓ | | | Mark issues as related **(STARTER)** | | `/move <path/to/project>` | ✓ | | | Move this issue to another project | diff --git a/doc/user/project/releases/img/edit_release_page_v12_5.png b/doc/user/project/releases/img/edit_release_page_v12_6.png Binary files differindex 8b9c502a2ef..8b9c502a2ef 100644 --- a/doc/user/project/releases/img/edit_release_page_v12_5.png +++ b/doc/user/project/releases/img/edit_release_page_v12_6.png diff --git a/doc/user/project/releases/img/release_edit_button_v12_5.png b/doc/user/project/releases/img/release_edit_button_v12_6.png Binary files differindex f60b0ecb1be..f60b0ecb1be 100644 --- a/doc/user/project/releases/img/release_edit_button_v12_5.png +++ b/doc/user/project/releases/img/release_edit_button_v12_6.png diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 9b7b20be98f..58e028c89be 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -32,7 +32,7 @@ your users to quickly scan the differences between each one you publish. NOTE: **Note:** [Git's tagging messages](https://git-scm.com/book/en/v2/Git-Basics-Tagging) and -Release descriptions are unrelated. Description supports [markdown](../../markdown.md). +Release descriptions are unrelated. Description supports [Markdown](../../markdown.md). ### Release assets @@ -92,17 +92,17 @@ project. ## Editing a release -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/26016) in GitLab 12.5. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/26016) in GitLab 12.6. To edit the details of a release, navigate to **Project overview > Releases** and click the edit button (pencil icon) in the top-right corner of the release you want to modify. - + This will bring you to the **Edit Release** page, from which you can change some of the release's details. - + Currently, it is only possible to edit the release title and notes. To change other release information, such as its tag, associated @@ -126,7 +126,7 @@ following modal window will be then displayed, from which you can select **New r ## Add release notes to Git tags You can add release notes to any Git tag using the notes feature. Release notes -behave like any other markdown form in GitLab so you can write text and +behave like any other Markdown form in GitLab so you can write text and drag and drop files to it. Release notes are stored in GitLab's database. There are several ways to add release notes: diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index 8756760fe4b..4cf0e458a53 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -37,6 +37,10 @@ After the forking is done, you can start working on the newly created repository. There, you will have full [Owner](../../permissions.md) access, so you can set it up as you please. +CAUTION: **CAUTION:** +From GitLab 12.6 onwards, if the [visibility of an upstream project is reduced](../../../public_access/public_access.md#reducing-visibility) +in any way, the fork relationship with all its forks will be removed. + ## Merging upstream Once you are ready to send your code back to the main project, you need diff --git a/doc/user/project/repository/git_blame.md b/doc/user/project/repository/git_blame.md new file mode 100644 index 00000000000..454b3f86df9 --- /dev/null +++ b/doc/user/project/repository/git_blame.md @@ -0,0 +1,50 @@ +--- +type: reference, howto +description: "Documentation on Git file blame." +--- + +# Git file blame + +> [Introduced](https://git.sphere.ly/staff/publicgitlab/commit/39c657930625ddc3ac8a921f01ffc83acadce68f) in GitLab 2.5 + +[Git blame](https://git-scm.com/docs/git-blame) provides more information +about every line in a file, including the last modified time, author, and +commit hash. + +You can find the **Blame** button with each file in a project. + + + +When you select the **Blame** button, you'll see a screen with the +noted information: + + + +If you hover over a commit in the UI, you'll see a precise date and time +for that commit. + +## Associated `git` command + +If you're running `git` from the command line, the equivalent command is +`git blame <filename>`. For example, if you want to find `blame` information +about a `README.md` file in the local directory, run the following command: + +```bash +git blame README.md +``` + +You'll see output similar to the following, which includes the commit time +in UTC format: + +```bash +62e2353a (Achilleas Pipinellis 2019-07-11 14:52:18 +0300 1) [](https://gitlab.com/gitlab-com/gitlab-docs/commits/master) +fb0fc7d6 (Achilleas Pipinellis 2016-11-07 22:21:22 +0100 2) +^764ca75 (Connor Shea 2016-10-05 23:40:24 -0600 3) # GitLab Documentation +^764ca75 (Connor Shea 2016-10-05 23:40:24 -0600 4) +0e62ed6d (Mike Jang 2019-11-26 21:44:53 +0000 5) This project hosts the repository used to generate the GitLab +0e62ed6d (Mike Jang 2019-11-26 21:44:53 +0000 6) documentation website and deployed to https://docs.gitlab.com. It uses the +``` + +## File blame through the API + +You can also get this information over the [Git file blame REST API](../../../api/repository_files.md#get-file-blame-from-repository). diff --git a/doc/user/project/repository/git_history.md b/doc/user/project/repository/git_history.md new file mode 100644 index 00000000000..9cd3d0d4ed0 --- /dev/null +++ b/doc/user/project/repository/git_history.md @@ -0,0 +1,67 @@ +--- +type: reference, howto +description: "Documentation on Git file history." +--- + +# Git file history + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/blob/9ba1224867665844b117fa037e1465bb706b3685/app/controllers/commits_controller.rb) in GitLab 0.8.0 + +Git file History provides information about the commit history associated +with a file. + +You can find the **History** button with each file in a project. + + + +When you select the **History** button, you'll see a screen with the +noted information: + + + +If you hover over a commit in the UI, you'll see a precise date and time +that commit was last modified. + +## Associated `git` command + +If you're running `git` from the command line, the equivalent command +is `git log <filename>`. For example, if you want to find `history` +information about a `README.md` file in the local directory, run the +following command: + +```bash +git log README.md +``` + +You'll see output similar to the following, which includes the commit +time in UTC format: + +```bash +commit 0e62ed6d9f39fa9bedf7efc6edd628b137fa781a +Author: Mike Jang <mjang@gitlab.com> +Date: Tue Nov 26 21:44:53 2019 +0000 + + Deemphasize GDK as a doc build tool + +commit 418879420b1e3a4662067bd07b64bb6988654697 +Author: Marcin Sedlak-Jakubowski <msedlakjakubowski@gitlab.com> +Date: Mon Nov 4 19:58:27 2019 +0100 + + Fix typo + +commit 21cc1fef11349417ed515557748369cfb235fc81 +Author: Jacques Erasmus <jerasmus@gitlab.com> +Date: Mon Oct 14 22:13:40 2019 +0000 + + Add support for modern JS + + Added rollup to the project + +commit 2f5e895aebfa5678e51db303b97de56c51e3cebe +Author: Achilleas Pipinellis <axil@gitlab.com> +Date: Fri Sep 13 14:03:01 2019 +0000 + + Remove gitlab-foss Git URLs as we don't need them anymore + + [ci skip] +``` diff --git a/doc/user/project/repository/img/file_blame_button_v12_6.png b/doc/user/project/repository/img/file_blame_button_v12_6.png Binary files differnew file mode 100644 index 00000000000..b5a18e6726f --- /dev/null +++ b/doc/user/project/repository/img/file_blame_button_v12_6.png diff --git a/doc/user/project/repository/img/file_blame_output_v12_6.png b/doc/user/project/repository/img/file_blame_output_v12_6.png Binary files differnew file mode 100644 index 00000000000..4aca40353d5 --- /dev/null +++ b/doc/user/project/repository/img/file_blame_output_v12_6.png diff --git a/doc/user/project/repository/img/file_history_button_v12_6.png b/doc/user/project/repository/img/file_history_button_v12_6.png Binary files differnew file mode 100644 index 00000000000..b5a18e6726f --- /dev/null +++ b/doc/user/project/repository/img/file_history_button_v12_6.png diff --git a/doc/user/project/repository/img/file_history_output_v12_6.png b/doc/user/project/repository/img/file_history_output_v12_6.png Binary files differnew file mode 100644 index 00000000000..9e9855203af --- /dev/null +++ b/doc/user/project/repository/img/file_history_output_v12_6.png diff --git a/doc/user/project/repository/img/web_editor_new_branch_from_issue.png b/doc/user/project/repository/img/web_editor_new_branch_from_issue.png Binary files differdeleted file mode 100644 index 4e156b8adc8..00000000000 --- a/doc/user/project/repository/img/web_editor_new_branch_from_issue.png +++ /dev/null diff --git a/doc/user/project/repository/img/web_editor_new_branch_from_issue_create_button_v12_6.png b/doc/user/project/repository/img/web_editor_new_branch_from_issue_create_button_v12_6.png Binary files differnew file mode 100644 index 00000000000..f40cc187b46 --- /dev/null +++ b/doc/user/project/repository/img/web_editor_new_branch_from_issue_create_button_v12_6.png diff --git a/doc/user/project/repository/img/web_editor_new_branch_from_issue_v_12_6.png b/doc/user/project/repository/img/web_editor_new_branch_from_issue_v_12_6.png Binary files differnew file mode 100644 index 00000000000..d5a92546d40 --- /dev/null +++ b/doc/user/project/repository/img/web_editor_new_branch_from_issue_v_12_6.png diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 5a6e011220c..fc422bb5aba 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -34,7 +34,7 @@ You can either use the user interface (UI), or connect your local computer with GitLab [through the command line](../../../gitlab-basics/command-line-commands.md#start-working-on-your-project). To configure [GitLab CI/CD](../../../ci/README.md) to build, test, and deploy -you code, add a file called [`.gitlab-ci.yml`](../../../ci/quick_start/README.md) +your code, add a file called [`.gitlab-ci.yml`](../../../ci/quick_start/README.md) to your repository's root. **From the user interface:** @@ -48,6 +48,8 @@ it's easier to do so [via GitLab UI](web_editor.md): - [File templates](web_editor.md#template-dropdowns) - [Create a directory](web_editor.md#create-a-directory) - [Start a merge request](web_editor.md#tips) +- [Find file history](git_history.md) +- [Identify changes by line (Git blame)](git_blame.md) **From the command line:** diff --git a/doc/user/project/repository/repository_mirroring.md b/doc/user/project/repository/repository_mirroring.md index a682983ab83..993c96d2ae4 100644 --- a/doc/user/project/repository/repository_mirroring.md +++ b/doc/user/project/repository/repository_mirroring.md @@ -143,6 +143,7 @@ Changes pushed to the upstream repository will be pulled into the GitLab reposit CAUTION: **Caution:** If you do manually update a branch in the GitLab repository, the branch will become diverged from upstream and GitLab will no longer automatically update this branch to prevent any changes from being lost. +Also note that deleted branches and tags in the upstream repository will not be reflected in the GitLab repository. ### How it works @@ -247,6 +248,10 @@ If you need to change the key at any time, you can remove and re-add the mirror to generate a new key. You'll have to update the other repository with the new key to keep the mirror running. +NOTE: **Note:** +The generated keys are stored in the GitLab database, not in the filesystem. Therefore, +SSH public key authentication for mirrors cannot be used in a pre-receive hook. + ### Overwrite diverged branches **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/merge_requests/4559) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.6. @@ -362,6 +367,7 @@ proxy_push() branch=$(expr "$refname" : "refs/heads/\(.*\)") if [ "$whitelisted" = "$branch" ]; then + unset GIT_QUARANTINE_PATH # handle https://git-scm.com/docs/git-receive-pack#_quarantine_environment error="$(git push --quiet $TARGET_REPO $NEWREV:$REFNAME 2>&1)" fail=$? @@ -396,6 +402,15 @@ else fi ``` +Note that this sample has a few limitations: + +- This example may not work verbatim for your use case and might need modification. + - It does not regard different types of authentication mechanisms for the mirror. + - It does not work with forced updates (rewriting history). + - Only branches that match the `whitelisted` patterns will be proxy pushed. +- The script circumvents the Git hook quarantine environment because the update of `$TARGET_REPO` + is seen as a ref update and Git will complain about it. + ### Mirroring with Perforce Helix via Git Fusion **(STARTER)** CAUTION: **Warning:** diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index 944720c3863..f41ff12d0a4 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -95,20 +95,31 @@ There are multiple ways to create a branch from GitLab's web interface. In case your development workflow dictates to have an issue for every merge request, you can quickly create a branch right on the issue page which will be -tied with the issue itself. You can see a **New branch** button after the issue -description, unless there is already a branch with the same name or a referenced -merge request. +tied with the issue itself. You can see a **Create merge request** dropdown +below the issue description unless there is already a branch with the same +name or a referenced merge request. - + -Once you click it, a new branch will be created that diverges from the default +This dropdown contains the options **Create merge request and branch** and **Create branch**. + + + +Once you choose one of these options, a new branch or branch and merge request +will be created, based on the default branch of your project, by default `master`. The branch name will be based on the title of the issue and as a prefix, it will have its internal ID. Thus, the example -screenshot above will yield a branch named -`23177-add-support-for-rich-references-to-referables`. - -Since GitLab 9.0, when you click the `New branch` in an empty repository project, GitLab automatically creates the master branch, commits a blank `README.md` file to it and creates and redirects you to a new branch based on the issue title. -If your [project is already configured with a deployment service][project-services-doc] (e.g. Kubernetes), GitLab takes one step further and prompts you to set up [auto deploy][auto-deploy-doc] by helping you create a `.gitlab-ci.yml` file. +screenshot above will create a branch named +`2-make-static-site-auto-deploy-and-serve`. + +When you click the **Create branch** button in an empty +repository project, GitLab automatically creates a `master` branch, commits +a blank `README.md` file to it, and creates and redirects you to a new branch +based on the issue title. +If your [project is already configured with a deployment service](../integrations/project_services.md), +such as Kubernetes, GitLab takes one step further and prompts you to set up +[auto deploy](../../../topics/autodevops/index.md#auto-deploy) +by helping you create a `.gitlab-ci.yml` file. After the branch is created, you can edit files in the repository to fix the issue. When a merge request is created based on the newly created branch, @@ -116,9 +127,6 @@ the description field will automatically display the [issue closing pattern](../ `Closes #ID`, where `ID` the ID of the issue. This will close the issue once the merge request is merged. -[project-services-doc]: ../integrations/project_services.md -[auto-deploy-doc]: ../../../topics/autodevops/index.md#auto-deploy - ### Create a new branch from a project's dashboard If you want to make changes to several files before creating a new merge @@ -152,7 +160,7 @@ SHA. From a project's files page, choose **New tag** from the dropdown. Give the tag a name such as `v1.0.0`. Choose the branch or SHA from which you would like to create this new tag. You can optionally add a message and -release notes. The release notes section supports markdown format and you can +release notes. The release notes section supports Markdown format and you can also upload an attachment. Click **Create tag** and you will be taken to the tag list page. diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index 0ca34c4ed02..fad4af7102a 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -55,9 +55,9 @@ you can skip the step 1 below; you only need to enable it per project. 1. [Set up incoming email](../../administration/incoming_email.md#set-it-up) for the GitLab instance. This must support [email sub-addressing](../../administration/incoming_email.md#email-sub-addressing). -1. Navigate to your project's **Settings** and scroll down to the **Service Desk** +1. Navigate to your project's **Settings > General** and scroll down to the **Service Desk** section. -1. If you have the correct access and an Premium license, +1. If you have the correct access and a Premium license, you will see an option to set up Service Desk:  @@ -79,6 +79,9 @@ you can skip the step 1 below; you only need to enable it per project. However the older format is still supported, allowing existing aliases or contacts to continue working._ +1. If you have [templates](description_templates.md) in your repository, then you can optionally + select one of these templates from the dropdown to append it to all Service Desk issues. + 1. Service Desk is now enabled for this project! You should be able to access it from your project's navigation **Issue submenu**:  diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 2dc507901d0..2c7a24da8f9 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -18,7 +18,7 @@ Adjust your project's name, description, avatar, [default branch](../repository/  -The project description also partially supports [standard markdown](../../markdown.md#standard-markdown-and-extensions-in-gitlab). You can use [emphasis](../../markdown.md#emphasis), [links](../../markdown.md#links), and [line-breaks](../../markdown.md#line-breaks) to add more context to the project description. +The project description also partially supports [standard Markdown](../../markdown.md#standard-markdown-and-extensions-in-gitlab). You can use [emphasis](../../markdown.md#emphasis), [links](../../markdown.md#links), and [line-breaks](../../markdown.md#line-breaks) to add more context to the project description. ### Sharing and permissions @@ -26,6 +26,10 @@ Set up your project's access, [visibility](../../../public_access/public_access.  +CAUTION: **Caution:** +[Reducing a project's visibility level](../../../public_access/public_access.md#reducing-visibility) +will remove the fork relationship between the project and any forked project. + If Issues are disabled, or you can't access Issues because you're not a project member, then Labels and Milestones links will be missing from the sidebar UI. diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 9836255932a..ce129f0663b 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -1,11 +1,5 @@ --- -comments: false +redirect_to: '../README.md' --- -# Workflow (Deprecated) - -This page was deprecated, with all content previously stored under the `/workflow` path moved -to other locations in the documentation site, organized by topic. You can use the search -box to find the content you are looking for, browse the main [GitLab Documentation page](../README.md), -or view the [issue that deprecated this page](https://gitlab.com/gitlab-org/gitlab/issues/32940) -for more details. +This document was moved to [another location](../README.md). |
