diff options
Diffstat (limited to 'doc')
176 files changed, 4854 insertions, 2681 deletions
diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index aaa43f67760..02de2caf558 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -73,6 +73,8 @@ From there, you can see the following actions: - User was added to project and with which [permissions] - Permission changes of a user assigned to a project - User was removed from project +- Project export was downloaded +- Project repository was downloaded ### Instance events **(PREMIUM ONLY)** diff --git a/doc/administration/auth/ldap-ee.md b/doc/administration/auth/ldap-ee.md index 1b7af2d945a..34f3cfa353f 100644 --- a/doc/administration/auth/ldap-ee.md +++ b/doc/administration/auth/ldap-ee.md @@ -4,37 +4,27 @@ type: reference # LDAP Additions in GitLab EE **(STARTER ONLY)** -This is a continuation of the main [LDAP documentation](ldap.md), detailing LDAP -features specific to GitLab Enterprise Edition Starter, Premium and Ultimate. +This section documents LDAP features specific to to GitLab Enterprise Edition +[Starter](https://about.gitlab.com/pricing/#self-managed) and above. -## Overview - -[LDAP](https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol) -stands for **Lightweight Directory Access Protocol**, which -is a standard application protocol for -accessing and maintaining distributed directory information services -over an Internet Protocol (IP) network. - -GitLab integrates with LDAP to support **user authentication**. This integration -works with most LDAP-compliant directory servers, including Microsoft Active -Directory, Apple Open Directory, Open LDAP, and 389 Server. -**GitLab Enterprise Edition** includes enhanced integration, including group -membership syncing. +For documentation relevant to both Community Edition and Enterprise Edition, +see the main [LDAP documentation](ldap.md). ## Use cases -- User sync: Once a day, GitLab will update users against LDAP +- User sync: Once a day, GitLab will update users against LDAP. - Group sync: Once an hour, GitLab will update group membership - based on LDAP group members + based on LDAP group members. -## Multiple LDAP servers +## Multiple LDAP servers **(STARTER ONLY)** With GitLab Enterprise Edition Starter, you can configure multiple LDAP servers that your GitLab instance will connect to. -To add another LDAP server, you can start by duplicating the settings under -[the main configuration](ldap.md#configuration) and edit them to match the -additional LDAP server. +To add another LDAP server: + +1. Duplicating the settings under [the main configuration](ldap.md#configuration). +1. Edit them to match the additional LDAP server. Be sure to choose a different provider ID made of letters a-z and numbers 0-9. This ID will be stored in the database so that GitLab can remember which LDAP @@ -47,19 +37,19 @@ users against LDAP. The process will execute the following access checks: -1. Ensure the user is still present in LDAP -1. If the LDAP server is Active Directory, ensure the user is active (not - blocked/disabled state). This will only be checked if - `active_directory: true` is set in the LDAP configuration [^1] +- Ensure the user is still present in LDAP. +- If the LDAP server is Active Directory, ensure the user is active (not + blocked/disabled state). This will only be checked if + `active_directory: true` is set in the LDAP configuration. [^1] The user will be set to `ldap_blocked` state in GitLab if the above conditions fail. This means the user will not be able to login or push/pull code. The process will also update the following user information: -1. Email address -1. If `sync_ssh_keys` is set, SSH public keys -1. If Kerberos is enabled, Kerberos identity +- Email address. +- If `sync_ssh_keys` is set, SSH public keys. +- If Kerberos is enabled, Kerberos identity. NOTE: **Note:** The LDAP sync process updates existing users while new users will @@ -72,9 +62,6 @@ first time GitLab will trigger a sync for groups the user should be a member of. That way they don't need to wait for the hourly sync to be granted access to their groups and projects. -In GitLab Premium, we can also add a GitLab group to sync with one or multiple LDAP groups or we can -also add a filter. The filter must comply with the syntax defined in [RFC 2254](https://tools.ietf.org/search/rfc2254). - A group sync process will run every hour on the hour, and `group_base` must be set in LDAP configuration for LDAP synchronizations based on group CN to work. This allows GitLab group membership to be automatically updated based on LDAP group members. @@ -120,13 +107,26 @@ following. 1. [Restart GitLab][restart] for the changes to take effect. ---- - To take advantage of group sync, group owners or maintainers will need to create an -LDAP group link in their group **Settings > LDAP Groups** page. Multiple LDAP -groups and/or filters can be linked with a single GitLab group. When the link is -created, an access level/role is specified (Guest, Reporter, Developer, Maintainer, -or Owner). +LDAP group link in their group **Settings > LDAP Groups** page. + +Multiple LDAP groups and [filters](#filters-premium-only) can be linked with +a single GitLab group. When the link is created, an access level/role is +specified (Guest, Reporter, Developer, Maintainer, or Owner). + +### Filters **(PREMIUM ONLY)** + +In GitLab Premium, you can add an LDAP user filter for group synchronization. +Filters allow for complex logic without creating a special LDAP group. + +To sync GitLab group membership based on an LDAP filter: + +1. Open the **LDAP Synchronization** page for the GitLab group. +1. Select **LDAP user filter** as the **Sync method**. +1. Enter an LDAP user filter in the **LDAP user filter** field. + +The filter must comply with the +syntax defined in [RFC 2254](https://tools.ietf.org/search/rfc2254). ## Administrator sync @@ -190,10 +190,13 @@ group, as opposed to the full DN. ## Global group memberships lock "Lock memberships to LDAP synchronization" setting allows instance administrators -to lock down user abilities to invite new members to a group. When enabled following happens: +to lock down user abilities to invite new members to a group. + +When enabled, the following applies: -1. Only administrator can manage memberships of any group including access levels. -1. Users are not allowed to share project with other groups or invite members to a project created in a group. +- Only administrator can manage memberships of any group including access levels. +- Users are not allowed to share project with other groups or invite members to + a project created in a group. ## Adjusting LDAP user sync schedule @@ -333,10 +336,18 @@ administrative duties. ### Supported LDAP group types/attributes -GitLab supports LDAP groups that use member attributes `member`, `submember`, -`uniquemember`, `memberof` and `memberuid`. This means group sync supports, at -least, LDAP groups with object class `groupOfNames`, `posixGroup`, and -`groupOfUniqueName`. Other object classes should work fine as long as members +GitLab supports LDAP groups that use member attributes: + +- `member` +- `submember` +- `uniquemember` +- `memberof` +- `memberuid`. + +This means group sync supports, at least, LDAP groups with object class: +`groupOfNames`, `posixGroup`, and `groupOfUniqueName`. + +Other object classes should work fine as long as members are defined as one of the mentioned attributes. This also means GitLab supports Microsoft Active Directory, Apple Open Directory, Open LDAP, and 389 Server. Other LDAP servers should work, too. @@ -344,11 +355,12 @@ Other LDAP servers should work, too. Active Directory also supports nested groups. Group sync will recursively resolve membership if `active_directory: true` is set in the configuration file. -> **Note:** Nested group membership will only be resolved if the nested group - also falls within the configured `group_base`. For example, if GitLab sees a - nested group with DN `cn=nested_group,ou=special_groups,dc=example,dc=com` but - the configured `group_base` is `ou=groups,dc=example,dc=com`, `cn=nested_group` - will be ignored. +NOTE: **Note:** +Nested group membership will only be resolved if the nested group +also falls within the configured `group_base`. For example, if GitLab sees a +nested group with DN `cn=nested_group,ou=special_groups,dc=example,dc=com` but +the configured `group_base` is `ou=groups,dc=example,dc=com`, `cn=nested_group` +will be ignored. ### Queries @@ -403,7 +415,7 @@ main: # 'main' is the GitLab 'provider ID' of this LDAP server [^1]: In Active Directory, a user is marked as disabled/blocked if the user account control attribute (`userAccountControl:1.2.840.113556.1.4.803`) - has bit 2 set. See https://ctogonewild.com/2009/09/03/bitmask-searches-in-ldap/ + has bit 2 set. See <https://ctogonewild.com/2009/09/03/bitmask-searches-in-ldap/> for more information. ### User DN has changed @@ -423,10 +435,10 @@ things to check to debug the situation. - Ensure LDAP configuration has a `group_base` specified. This configuration is required for group sync to work properly. - Ensure the correct LDAP group link is added to the GitLab group. Check group - links by visiting the GitLab group, then **Settings dropdown -> LDAP groups**. -- Check that the user has an LDAP identity + links by visiting the GitLab group, then **Settings dropdown > LDAP groups**. +- Check that the user has an LDAP identity: 1. Sign in to GitLab as an administrator user. - 1. Navigate to **Admin area -> Users**. + 1. Navigate to **Admin area > Users**. 1. Search for the user 1. Open the user, by clicking on their name. Do not click 'Edit'. 1. Navigate to the **Identities** tab. There should be an LDAP identity with @@ -437,7 +449,7 @@ Often, the best way to learn more about why group sync is behaving a certain way is to enable debug logging. There is verbose output that details every step of the sync. -1. Start a Rails console +1. Start a Rails console: ```bash # For Omnibus installations @@ -446,6 +458,7 @@ step of the sync. # For installations from source sudo -u git -H bundle exec rails console production ``` + 1. Set the log level to debug (only for this session): ```ruby @@ -540,8 +553,9 @@ and more DNs may be added, or existing entries modified, based on additional LDAP group lookups. The very last occurrence of this entry should indicate exactly which users GitLab believes should be added to the group. -> **Note:** 10 is 'Guest', 20 is 'Reporter', 30 is 'Developer', 40 is 'Maintainer' - and 50 is 'Owner' +NOTE: **Note:** +10 is 'Guest', 20 is 'Reporter', 30 is 'Developer', 40 is 'Maintainer' +and 50 is 'Owner'. ```bash Resolved 'my_group' group member access: {"uid=john0,ou=people,dc=example,dc=com"=>30, diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index be05a4d63a7..186bf4c4825 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -7,28 +7,44 @@ type: reference # LDAP GitLab integrates with LDAP to support user authentication. -This integration works with most LDAP-compliant directory -servers, including Microsoft Active Directory, Apple Open Directory, Open LDAP, -and 389 Server. GitLab Enterprise Editions include enhanced integration, + +This integration works with most LDAP-compliant directory servers, including: + +- Microsoft Active Directory +- Apple Open Directory +- Open LDAP +- 389 Server. + +GitLab Enterprise Editions (EE) include enhanced integration, including group membership syncing as well as multiple LDAP servers support. -## GitLab EE +For more details about EE-specific LDAP features, see the +[LDAP Enterprise Edition documentation](ldap-ee.md). -The information on this page is relevant for both GitLab CE and EE. For more -details about EE-specific LDAP features, see the -[LDAP Enterprise Edition documentation](ldap-ee.md). **(STARTER ONLY)** +NOTE: **Note:** +The information on this page is relevant for both GitLab CE and EE. + +## Overview + +[LDAP](https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol) +stands for **Lightweight Directory Access Protocol**, which is a standard +application protocol for accessing and maintaining distributed directory +information services over an Internet Protocol (IP) network. ## Security -GitLab assumes that LDAP users are not able to change their LDAP 'mail', 'email' -or 'userPrincipalName' attribute. An LDAP user who is allowed to change their -email on the LDAP server can potentially -[take over any account](#enabling-ldap-sign-in-for-existing-gitlab-users) -on your GitLab server. +GitLab assumes that LDAP users: + +- Are not able to change their LDAP `mail`, `email`, or `userPrincipalName` attribute. + An LDAP user who is allowed to change their email on the LDAP server can potentially + [take over any account](#enabling-ldap-sign-in-for-existing-gitlab-users) + on your GitLab server. +- Have unique email addresses, otherwise it is possible for LDAP users with the same + email address to share the same GitLab account. We recommend against using LDAP integration if your LDAP users are -allowed to change their 'mail', 'email' or 'userPrincipalName' attribute on -the LDAP server. +allowed to change their 'mail', 'email' or 'userPrincipalName' attribute on +the LDAP server or share email addresses. ### User deletion @@ -64,9 +80,12 @@ NOTE: **Note**: In GitLab Enterprise Edition Starter, you can configure multiple LDAP servers to connect to one GitLab server. -For a complete guide on configuring LDAP with GitLab Community Edition, please check -the admin guide [How to configure LDAP with GitLab CE](how_to_configure_ldap_gitlab_ce/index.md). -For GitLab Enterprise Editions, see also [How to configure LDAP with GitLab EE](how_to_configure_ldap_gitlab_ee/index.md). **(STARTER ONLY)** +For a complete guide on configuring LDAP with: + +- GitLab Community Edition, see + [How to configure LDAP with GitLab CE](how_to_configure_ldap_gitlab_ce/index.md). +- Enterprise Editions, see + [How to configure LDAP with GitLab EE](how_to_configure_ldap_gitlab_ee/index.md). **(STARTER ONLY)** To enable LDAP integration you need to add your LDAP server settings in `/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml` for Omnibus @@ -384,7 +403,7 @@ production: Tip: If you want to limit access to the nested members of an Active Directory group, you can use the following syntax: -``` +```text (memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com) ``` @@ -402,13 +421,13 @@ The `user_filter` DN can contain special characters. For example: - A comma: - ``` + ```text OU=GitLab, Inc,DC=gitlab,DC=com ``` - Open and close brackets: - ``` + ```text OU=Gitlab (Inc),DC=gitlab,DC=com ``` @@ -417,13 +436,13 @@ The `user_filter` DN can contain special characters. For example: - Escape commas with `\2C`. For example: - ``` + ```text OU=GitLab\2C Inc,DC=gitlab,DC=com ``` - Escape open and close brackets with `\28` and `\29`, respectively. For example: - ``` + ```text OU=Gitlab \28Inc\29,DC=gitlab,DC=com ``` @@ -507,11 +526,11 @@ timeout), the login is rejected and a message will be logged to ### Debug LDAP user filter with ldapsearch -This example uses ldapsearch and assumes you are using ActiveDirectory. The +This example uses `ldapsearch` and assumes you are using ActiveDirectory. The following query returns the login names of the users that will be allowed to log in to GitLab if you configure your own user_filter. -``` +```sh ldapsearch -H ldaps://$host:$port -D "$bind_dn" -y bind_dn_password.txt -b "$base" "$user_filter" sAMAccountName ``` diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 113514e1ee8..32462a95a1a 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -21,7 +21,7 @@ administrators can add custom git hooks to any GitLab project. ## Create a custom Git hook for a repository Server-side Git hooks are typically placed in the repository's `hooks` -subdirectory. In GitLab, hook directories are are symlinked to the gitlab-shell +subdirectory. In GitLab, hook directories are symlinked to the gitlab-shell `hooks` directory for ease of maintenance between gitlab-shell upgrades. Custom hooks are implemented differently, but the behavior is exactly the same once the hook is created. Follow the steps below to set up a custom hook for a diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index f0d329d5296..dc49a95e008 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -1,5 +1,11 @@ # Geo Replication **(PREMIUM ONLY)** +> - Introduced in GitLab Enterprise Edition 8.9. +> - Using Geo in combination with +> [High Availability](../../high_availability/README.md) +> is considered **Generally Available** (GA) in +> [GitLab Premium](https://about.gitlab.com/pricing/) 10.4. + Geo is the solution for widely distributed development teams. ## Overview @@ -8,14 +14,14 @@ Fetching large repositories can take a long time for teams located far from a si Geo provides local, read-only instances of your GitLab instances, reducing the time it takes to clone and fetch large repositories and speeding up development. +> **Notes:** +> > - Geo is part of [GitLab Premium](https://about.gitlab.com/pricing/#self-managed). -> - Introduced in GitLab Enterprise Edition 8.9. > - We recommend you use: > - At least GitLab Enterprise Edition 10.0 for basic Geo features. > - The latest version for a better experience. > - Make sure that all nodes run the same GitLab version. > - Geo requires PostgreSQL 9.6 and Git 2.9, in addition to GitLab's usual [minimum requirements](../../../install/requirements.md). -> - Using Geo in combination with [High Availability](../../high_availability/README.md) is considered **Generally Available** (GA) in GitLab [GitLab Premium](https://about.gitlab.com/pricing/) 10.4. For a video introduction to Geo, see [Introduction to GitLab Geo - GitLab Features](https://www.youtube.com/watch?v=-HDLxSjEh6w). @@ -239,35 +245,48 @@ This list of limitations only reflects the latest version of GitLab. If you are - Object pools for forked project deduplication work only on the **primary** node, and are duplicated on the **secondary** node. - [External merge request diffs](../../merge_request_diffs.md) will not be replicated if they are on-disk, and viewing merge requests will fail. However, external MR diffs in object storage **are** supported. The default configuration (in-database) does work. -### Limitations on replication - -Only the following items are replicated to the **secondary** node: - -- All database content. For example, snippets, epics, issues, merge requests, groups, and project metadata. -- Project repositories. -- Project wiki repositories. -- User uploads. For example, attachments to issues, merge requests, epics, and avatars. -- CI job artifacts and traces. +### Limitations on replication/verification + +The following table lists the GitLab features along with their replication +and verification status on a **secondary** node. + +You can keep track of the progress to include the missing items in: + +- [ee-893](https://gitlab.com/groups/gitlab-org/-/epics/893). +- [ee-1430](https://gitlab.com/groups/gitlab-org/-/epics/1430). + +| Feature | Replicated | Verified | +|-----------|------------|----------| +| All database content (e.g. snippets, epics, issues, merge requests, groups, and project metadata) | Yes | Yes | +| Project repository | Yes | Yes | +| Project wiki repository | Yes | Yes | +| Project designs repository | No | No | +| Uploads (e.g. attachments to issues, merge requests, epics, and avatars) | Yes | Yes, only on transfer, or manually (1) | +| LFS Objects | Yes | Yes, only on transfer, or manually (1) | +| CI job artifacts (other than traces) | Yes | No, only manually (1) | +| Archived traces | Yes | Yes, only on transfer, or manually (1) | +| Personal snippets | Yes | Yes | +| Version-controlled personal snippets ([unsupported](https://gitlab.com/gitlab-org/gitlab-ce/issues/13426)) | No | No | +| Project snippets | Yes | Yes | +| Version-controlled project snippets ([unsupported](https://gitlab.com/gitlab-org/gitlab-ce/issues/13426)) | No | No | +| Object pools for forked project deduplication | No | No | +| [Server-side Git Hooks](../../custom_hooks.md) | No | No | +| [Elasticsearch integration](../../../integration/elasticsearch.md) | No | No | +| [GitLab Pages](../../pages/index.md) | No | No | +| [Container Registry](../../container_registry.md) ([track progress](https://gitlab.com/gitlab-org/gitlab-ee/issues/2870)) | No | No | +| [NPM Registry](../../npm_registry.md) | No | No | +| [Maven Packages](../../maven_packages.md) | No | No | +| [External merge request diffs](../../merge_request_diffs.md) | No, if they are on-disk | No | +| Content in object storage ([track progress](https://gitlab.com/groups/gitlab-org/-/epics/1526)) | No | No | + +1. The integrity can be verified manually using [Integrity Check Rake Task](../../raketasks/check.md) on both nodes and comparing the output between them. DANGER: **DANGER** -Data not on this list is unavailable on the **secondary** node. Failing over without manually replicating data not on this list will cause the data to be **lost**. - -### Examples of data not replicated - -Take special note that these examples of GitLab features are both: - -- Commonly used. -- **Not** replicated by Geo at present. - -Examples include: - -- [Elasticsearch integration](../../../integration/elasticsearch.md). -- [Container Registry](../../container_registry.md). [Object Storage](object_storage.md) can mitigate this. -- [GitLab Pages](../../pages/index.md). -- [Mattermost integration](https://docs.gitlab.com/omnibus/gitlab-mattermost/). - -CAUTION: **Caution:** -If you wish to use them on a **secondary** node, or to execute a failover successfully, you will need to replicate their data using some other means. +Features not on this list, or with **No** in the **Replicated** column, +are not replicated on the **secondary** node. Failing over without manually +replicating data from those features will cause the data to be **lost**. +If you wish to use those features on a **secondary** node, or to execute a failover +successfully, you must replicate their data using some other means. ## Frequently Asked Questions diff --git a/doc/administration/geo/replication/updating_the_geo_nodes.md b/doc/administration/geo/replication/updating_the_geo_nodes.md index 550b3b07a95..39174780e24 100644 --- a/doc/administration/geo/replication/updating_the_geo_nodes.md +++ b/doc/administration/geo/replication/updating_the_geo_nodes.md @@ -10,10 +10,23 @@ all you need to do is update GitLab itself: 1. Log into each node (**primary** and **secondary** nodes). 1. [Update GitLab][update]. -1. [Update tracking database on **secondary** node](#update-tracking-database-on-secondary-node) when - the tracking database is enabled. 1. [Test](#check-status-after-updating) **primary** and **secondary** nodes, and check version in each. +### Check status after updating + +Now that the update process is complete, you may want to check whether +everything is working correctly: + +1. Run the Geo raketask on all nodes, everything should be green: + + ```sh + sudo gitlab-rake gitlab:geo:check + ``` + +1. Check the **primary** node's Geo dashboard for any errors. +1. Test the data replication by pushing code to the **primary** node and see if it + is received by **secondary** nodes. + ## Upgrading to GitLab 12.1 By default, GitLab 12.1 will attempt to automatically upgrade the embedded PostgreSQL server to 10.7 from 9.6. Please see [the omnibus documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance) for the recommended procedure. @@ -419,22 +432,7 @@ is prepended with the relevant node for better clarity: sudo gitlab-ctl start ``` -## Check status after updating - -Now that the update process is complete, you may want to check whether -everything is working correctly: - -1. Run the Geo raketask on all nodes, everything should be green: - - ```sh - sudo gitlab-rake gitlab:geo:check - ``` - -1. Check the **primary** node's Geo dashboard for any errors. -1. Test the data replication by pushing code to the **primary** node and see if it - is received by **secondary** nodes. - -## Update tracking database on **secondary** node +### Update tracking database on **secondary** node After updating a **secondary** node, you might need to run migrations on the tracking database. The tracking database was added in GitLab 9.1, diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 0f547ef03bf..432056d48c7 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -469,7 +469,16 @@ One current feature of GitLab that still requires a shared directory (NFS) is There is [work in progress](https://gitlab.com/gitlab-org/gitlab-pages/issues/196) to eliminate the need for NFS to support GitLab Pages. -## Troubleshooting +## Troubleshooting Gitaly + +### Commits, pushes, and clones return a 401 + +``` +remote: GitLab: 401 Unauthorized +``` + +You will need to sync your `gitlab-secrets.json` file with your GitLab +app nodes. ### `gitaly-debug` @@ -510,3 +519,94 @@ implemented as calls to `gitaly-ruby`: ``` sum(rate(grpc_client_handled_total[5m])) by (grpc_method) > 0 ``` + +### Repository changes fail with a `401 Unauthorized` error + +If you're running Gitaly on its own server and notice that users can +successfully clone and fetch repositories (via both SSH and HTTPS), but can't +push to them or make changes to the repository in the web UI without getting a +`401 Unauthorized` message, then it's possible Gitaly is failing to authenticate +with the other nodes due to having the [wrong secrets file](#3-gitaly-server-configuration). + +Confirm the following are all true: + +- When any user performs a `git push` to any repository on this Gitaly node, it + fails with the following error (note the `401 Unauthorized`): + + ```sh + remote: GitLab: 401 Unauthorized + To <REMOTE_URL> + ! [remote rejected] branch-name -> branch-name (pre-receive hook declined) + error: failed to push some refs to '<REMOTE_URL>' + ``` + +- When any user adds or modifies a file from the repository using the GitLab + UI, it immediatley fails with a red `401 Unauthorized` banner. +- Creating a new project and [initializing it with a README](../../gitlab-basics/create-project.md#blank-projects) + successfully creates the project but doesn't create the README. +- When [tailing the logs](https://docs.gitlab.com/omnibus/settings/logs.md#tail-logs-in-a-console-on-the-server) on an app node and reproducing the error, you get `401` errors + when reaching the `/api/v4/internal/allowed` endpoint: + + ```sh + # api_json.log + { + "time": "2019-07-18T00:30:14.967Z", + "severity": "INFO", + "duration": 0.57, + "db": 0, + "view": 0.57, + "status": 401, + "method": "POST", + "path": "\/api\/v4\/internal\/allowed", + "params": [ + { + "key": "action", + "value": "git-receive-pack" + }, + { + "key": "changes", + "value": "REDACTED" + }, + { + "key": "gl_repository", + "value": "REDACTED" + }, + { + "key": "project", + "value": "\/path\/to\/project.git" + }, + { + "key": "protocol", + "value": "web" + }, + { + "key": "env", + "value": "{\"GIT_ALTERNATE_OBJECT_DIRECTORIES\":[],\"GIT_ALTERNATE_OBJECT_DIRECTORIES_RELATIVE\":[],\"GIT_OBJECT_DIRECTORY\":null,\"GIT_OBJECT_DIRECTORY_RELATIVE\":null}" + }, + { + "key": "user_id", + "value": "2" + }, + { + "key": "secret_token", + "value": "[FILTERED]" + } + ], + "host": "gitlab.example.com", + "ip": "REDACTED", + "ua": "Ruby", + "route": "\/api\/:version\/internal\/allowed", + "queue_duration": 4.24, + "gitaly_calls": 0, + "gitaly_duration": 0, + "correlation_id": "XPUZqTukaP3" + } + + # nginx_access.log + [IP] - - [18/Jul/2019:00:30:14 +0000] "POST /api/v4/internal/allowed HTTP/1.1" 401 30 "" "Ruby" + ``` + +To fix this problem, confirm that your [`gitlab-secrets.json` file](#3-gitaly-server-configuration) +on the Gitaly node matches the one on all other nodes. If it doesn't match, +update the secrets file on the Gitaly node to match the others, then +[reconfigure the node](../restart_gitlab.md#omnibus-gitlab-reconfigure). diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 41ef68f5b57..56665ba8b9a 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -164,25 +164,24 @@ contention due to certain workloads. #### Reference Architecture -- **Status:** Work-in-progress - **Supported Users (approximate):** 10,000 -- **Related Issues:** [gitlab-com/support/support-team-meta#1513](https://gitlab.com/gitlab-com/support/support-team-meta/issues/1513), - [gitlab-org/quality/team-tasks#110](https://gitlab.com/gitlab-org/quality/team-tasks/issues/110) - -The Support and Quality teams are in the process of building and performance testing -an environment that will support about 10,000 users. The specifications below -are a work-in-progress representation of the work so far. Quality will be -certifying this environment in FY20-Q2. The specifications may be adjusted -prior to certification based on performance testing. - -- 3 PostgreSQL - 4 CPU, 8GB RAM per node -- 1 PgBouncer - 2 CPU, 4GB RAM -- 2 Redis - 2 CPU, 8GB RAM per node -- 3 Consul/Sentinel - 2 CPU, 2GB RAM per node -- 4 Sidekiq - 4 CPU, 8GB RAM per node -- 5 GitLab application nodes - 20 CPU, 64GB RAM per node -- 1 Gitaly - 20 CPU, 64GB RAM -- 1 Monitoring node - 4 CPU, 8GB RAM +- **Known Issues:** While validating the reference architecture, slow endpoints were discovered and are being investigated. [gitlab-org/gitlab-ce/issues/64335](https://gitlab.com/gitlab-org/gitlab-ce/issues/64335) + +The Support and Quality teams built, performance tested, and validated an +environment that supports about 10,000 users. The specifications below are a +representation of the work so far. The specifications may be adjusted in the +future based on additional testing and iteration. + +NOTE: **Note:** The specifications here were performance tested against a specific coded workload. Your exact needs may be more, depending on your workload. Your workload is influenced by factors such as - but not limited to - how active your users are, how much automation you use, mirroring, and repo/change size. + +- 3 PostgreSQL - 4 CPU, 16GiB memory per node +- 1 PgBouncer - 2 CPU, 4GiB memory +- 2 Redis - 2 CPU, 8GiB memory per node +- 3 Consul/Sentinel - 2 CPU, 2GiB memory per node +- 4 Sidekiq - 4 CPU, 16GiB memory per node +- 5 GitLab application nodes - 16 CPU, 64GiB memory per node +- 1 Gitaly - 16 CPU, 64GiB memory +- 1 Monitoring node - 2 CPU, 8GiB memory, 100GiB local storage ### Fully Distributed diff --git a/doc/administration/index.md b/doc/administration/index.md index 00c8863f200..febee3e5af8 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -185,3 +185,5 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Debugging tips](troubleshooting/debug.md): Tips to debug problems when things go wrong - [Log system](logs.md): Where to look for logs. - [Sidekiq Troubleshooting](troubleshooting/sidekiq.md): Debug when Sidekiq appears hung and is not processing jobs. +- Useful [diagnostics tools](troubleshooting/diagnostics_tools.md) that are sometimes used by the GitLab + Support team. diff --git a/doc/administration/monitoring/performance/img/performance_bar.png b/doc/administration/monitoring/performance/img/performance_bar.png Binary files differindex 8a6f8b3b273..89b09054d46 100644 --- a/doc/administration/monitoring/performance/img/performance_bar.png +++ b/doc/administration/monitoring/performance/img/performance_bar.png diff --git a/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png b/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png Binary files differdeleted file mode 100644 index a55ce753101..00000000000 --- a/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png +++ /dev/null diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 15eab7dcab0..2cc78ccc03c 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -14,8 +14,6 @@ It allows you to see (from left to right):  - time taken and number of [Rugged] calls, click through for details of these calls  -- profile of the code used to generate the page, line by line. In the profile view, the numbers in the left panel represent wall time, cpu time, and number of calls (based on [rblineprof](https://github.com/tmm1/rblineprof)). -  - time taken and number of Redis calls, click through for details of these calls  - time taken and number of Ruby GC calls diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index ea69378b249..e787af798bc 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -71,10 +71,10 @@ sudo service sshd reload Confirm that SSH is working by removing your user's SSH key in the UI, adding a new one, and attempting to pull a repo. -> **Note:** For Omnibus Docker, `AuthorizedKeysCommand` is setup by default in +NOTE: **Note:** For Omnibus Docker, `AuthorizedKeysCommand` is setup by default in GitLab 11.11 and later. -> **Warning:** Do not disable writes until SSH is confirmed to be working +CAUTION: **Caution:** Do not disable writes until SSH is confirmed to be working perfectly, because the file will quickly become out-of-date. In the case of lookup failures (which are common), the `authorized_keys` diff --git a/doc/administration/plugins.md b/doc/administration/plugins.md index 4302667caf5..4cf3c607dae 100644 --- a/doc/administration/plugins.md +++ b/doc/administration/plugins.md @@ -52,7 +52,37 @@ as appropriate. The plugins file list is updated for each event, there is no need to restart GitLab to apply a new plugin. If a plugin executes with non-zero exit code or GitLab fails to execute it, a -message will be logged to `plugin.log`. +message will be logged to: + +- `gitlab-rails/plugin.log` in an Omnibus installation. +- `log/plugin.log` in a source installation. + +## Creating plugins + +Below is an example that will only response on the event `project_create` and +will inform the admins from the GitLab instance that a new project has been created. + +```ruby +# By using the embedded ruby version we eliminate the possibility that our chosen language +# would be unavailable from +#!/opt/gitlab/embedded/bin/ruby +require 'json' +require 'mail' + +# The incoming variables are in JSON format so we need to parse it first. +ARGS = JSON.parse(STDIN.read) + +# We only want to trigger this plugin on the event project_create +return unless ARGS['event_name'] == 'project_create' + +# We will inform our admins of our gitlab instance that a new project is created +Mail.deliver do + from 'info@gitlab_instance.com' + to 'admin@gitlab_instance.com' + subject "new project " + ARGS['name'] + body ARGS['owner_name'] + 'created project ' + ARGS['name'] +end +``` ## Validation diff --git a/doc/administration/raketasks/ldap.md b/doc/administration/raketasks/ldap.md index 91fc0133d56..e880d76e756 100644 --- a/doc/administration/raketasks/ldap.md +++ b/doc/administration/raketasks/ldap.md @@ -28,6 +28,31 @@ limit by passing a number to the check task: rake gitlab:ldap:check[50] ``` +## Run a Group Sync + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14735) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.3. + +The following task will run a [group sync](../auth/ldap-ee.md#group-sync) immediately. This is valuable +when you'd like to update all configured group memberships against LDAP without +waiting for the next scheduled group sync to be run. + +NOTE: **NOTE:** +If you'd like to change the frequency at which a group sync is performed, +[adjust the cron schedule](../auth/ldap-ee.md#adjusting-ldap-group-sync-schedule) +instead. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:ldap:group_sync +``` + +**Source Installation** + +```bash +bundle exec rake gitlab:ldap:group_sync +``` + ## Rename a provider If you change the LDAP server ID in `gitlab.yml` or `gitlab.rb` you will need diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index ad3a9b19c3c..b1a870210a8 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -57,10 +57,10 @@ storage2: Now that you've read that big fat warning above, let's edit the configuration files and add the full paths of the alternative repository storage paths. In -the example below, we add two more mountpoints that are named `nfs` and `cephfs` +the example below, we add two more mountpoints that are named `nfs_1` and `nfs_2` respectively. -NOTE: **Note:** This example uses NFS and CephFS. We do not recommend using EFS for storage as it may impact GitLab's performance. See the [relevant documentation](high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. +NOTE: **Note:** This example uses NFS. We do not recommend using EFS for storage as it may impact GitLab's performance. See the [relevant documentation](high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. **For installations from source** @@ -73,10 +73,10 @@ NOTE: **Note:** This example uses NFS and CephFS. We do not recommend using EFS storages: # You must have at least a 'default' storage path. default: path: /home/git/repositories - nfs: - path: /mnt/nfs/repositories - cephfs: - path: /mnt/cephfs/repositories + nfs_1: + path: /mnt/nfs1/repositories + nfs_2: + path: /mnt/nfs2/repositories ``` 1. [Restart GitLab][restart-gitlab] for the changes to take effect. @@ -96,8 +96,8 @@ working, you can remove the `repos_path` line. ```ruby git_data_dirs({ "default" => { "path" => "/var/opt/gitlab/git-data" }, - "nfs" => { "path" => "/mnt/nfs/git-data" }, - "cephfs" => { "path" => "/mnt/cephfs/git-data" } + "nfs_1" => { "path" => "/mnt/nfs1/git-data" }, + "nfs_2" => { "path" => "/mnt/nfs2/git-data" } }) ``` diff --git a/doc/administration/troubleshooting/diagnostics_tools.md b/doc/administration/troubleshooting/diagnostics_tools.md new file mode 100644 index 00000000000..ab3b25f0e97 --- /dev/null +++ b/doc/administration/troubleshooting/diagnostics_tools.md @@ -0,0 +1,27 @@ +--- +type: reference +--- + +# Diagnostics tools + +These are some of the diagnostics tools the GitLab Support team uses during troubleshooting. +They are listed here for transparency, and they may be useful for users with experience +with troubleshooting GitLab. If you are currently having an issue with GitLab, you +may want to check your [support options](https://about.gitlab.com/support/) first, +before attempting to use these tools. + +## gitlabsos + +The [gitlabsos](https://gitlab.com/gitlab-com/support/toolbox/gitlabsos/) utility +provides a unified method of gathering info and logs from GitLab and the system it's +running on. + +## strace-parser + +[strace-parser](https://gitlab.com/wchandler/strace-parser) is a small tool to analyze +and summarize raw strace data. + +## Pritaly + +[Pritaly](https://gitlab.com/wchandler/pritaly) takes Gitaly logs and colorizes output +or converts the logs to JSON. diff --git a/doc/api/README.md b/doc/api/README.md index 70540420544..6cd89e34921 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -695,6 +695,13 @@ The correct encoding for the query parameter would be: There are many unofficial GitLab API Clients for most of the popular programming languages. Visit the [GitLab website] for a complete list. +## Rate limits + +For administrator documentation on rate limit settings, check out +[Rate limits](../security/rate_limits.md). To find the settings that are +specifically used by GitLab.com, see +[GitLab.com-specific rate limits](../user/gitlab_com/index.md). + [GitLab website]: https://about.gitlab.com/applications/#api-clients "Clients using the GitLab API" [lib-api-url]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/lib/api/api.rb [ce-3749]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3749 diff --git a/doc/api/container_registry.md b/doc/api/container_registry.md index 174b93a4f7a..bf544f64178 100644 --- a/doc/api/container_registry.md +++ b/doc/api/container_registry.md @@ -6,6 +6,8 @@ This is the API docs of the [GitLab Container Registry](../user/project/containe ## List registry repositories +### Within a project + Get a list of registry repositories in a project. ``` @@ -14,7 +16,8 @@ GET /projects/:id/registry/repositories | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) accessible by the authenticated user. | +| `tags` | boolean | no | If the param is included as true, each repository will include an array of `"tags"` in the response. | ```bash curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/registry/repositories" @@ -28,6 +31,7 @@ Example response: "id": 1, "name": "", "path": "group/project", + "project_id": 9, "location": "gitlab.example.com:5000/group/project", "created_at": "2019-01-10T13:38:57.391Z" }, @@ -35,12 +39,77 @@ Example response: "id": 2, "name": "releases", "path": "group/project/releases", + "project_id": 9, "location": "gitlab.example.com:5000/group/project/releases", "created_at": "2019-01-10T13:39:08.229Z" } ] ``` +### Within a group + +Get a list of registry repositories in a group. + +``` +GET /groups/:id/registry/repositories +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) accessible by the authenticated user. | +| `tags` | boolean | no | If the param is included as true, each repository will include an array of `"tags"` in the response. | + +```bash +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/2/registry/repositories?tags=1" +``` + +Example response: + +```json +[ + { + "id": 1, + "name": "", + "path": "group/project", + "project_id": 9, + "location": "gitlab.example.com:5000/group/project", + "created_at": "2019-01-10T13:38:57.391Z", + "tags": [ + { + "name": "0.0.1", + "path": "group/project:0.0.1", + "location": "gitlab.example.com:5000/group/project:0.0.1" + } + ] + }, + { + "id": 2, + "name": "", + "path": "group/other_project", + "project_id": 11, + "location": "gitlab.example.com:5000/group/other_project", + "created_at": "2019-01-10T13:39:08.229Z", + "tags": [ + { + "name": "0.0.1", + "path": "group/other_project:0.0.1", + "location": "gitlab.example.com:5000/group/other_project:0.0.1" + }, + { + "name": "0.0.2", + "path": "group/other_project:0.0.2", + "location": "gitlab.example.com:5000/group/other_project:0.0.2" + }, + { + "name": "latest", + "path": "group/other_project:latest", + "location": "gitlab.example.com:5000/group/other_project:latest" + } + ] + } +] +``` + ## Delete registry repository Delete a repository in registry. @@ -62,6 +131,8 @@ curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://git ## List repository tags +### Within a project + Get a list of tags for given registry repository. ``` @@ -70,7 +141,7 @@ GET /projects/:id/registry/repositories/:repository_id/tags | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) accessible by the authenticated user. | | `repository_id` | integer | yes | The ID of registry repository. | ```bash @@ -104,7 +175,7 @@ GET /projects/:id/registry/repositories/:repository_id/tags/:tag_name | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) accessible by the authenticated user. | | `repository_id` | integer | yes | The ID of registry repository. | | `tag_name` | string | yes | The name of tag. | diff --git a/doc/api/jobs.md b/doc/api/jobs.md index 0e45ee1a583..2a1c1b5f6f3 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -491,7 +491,7 @@ Parameters Example request: ```sh -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/5/artifacts/some/release/file.pdf" +curl --location --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/5/artifacts/some/release/file.pdf" ``` Possible response status codes: @@ -526,7 +526,7 @@ Parameters: Example request: ```sh -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/raw/some/release/file.pdf?job=pdf" +curl --location --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/raw/some/release/file.pdf?job=pdf" ``` Possible response status codes: @@ -551,7 +551,7 @@ GET /projects/:id/jobs/:job_id/trace | job_id | integer | yes | ID of a job. | ```sh -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/8/trace" +curl --location --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/8/trace" ``` Possible response status codes: diff --git a/doc/api/oauth2.md b/doc/api/oauth2.md index 76e3a0fa1a4..f9382361187 100644 --- a/doc/api/oauth2.md +++ b/doc/api/oauth2.md @@ -50,11 +50,14 @@ The web application flow is: `/oauth/authorize` endpoint with the following GET parameters: ``` - https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=code&state=YOUR_UNIQUE_STATE_HASH + https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=code&state=YOUR_UNIQUE_STATE_HASH&scope=REQUESTED_SCOPES ``` - This will ask the user to approve the applications access to their account and - then redirect back to the `REDIRECT_URI` you provided. The redirect will + This will ask the user to approve the applications access to their account + based on the scopes specified in `REQUESTED_SCOPES` and then redirect back to + the `REDIRECT_URI` you provided. The [scope parameter](https://github.com/doorkeeper-gem/doorkeeper/wiki/Using-Scopes#requesting-particular-scopes) + is a space separated list of scopes you want to have access to (e.g. `scope=read_user+profile` + would request `read_user` and `profile` scopes). The redirect will include the GET `code` parameter, for example: ``` @@ -110,11 +113,14 @@ To request the access token, you should redirect the user to the `/oauth/authorize` endpoint using `token` response type: ``` -https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=token&state=YOUR_UNIQUE_STATE_HASH +https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=token&state=YOUR_UNIQUE_STATE_HASH&scope=REQUESTED_SCOPES ``` -This will ask the user to approve the application's access to their account and -then redirect them back to the `REDIRECT_URI` you provided. The redirect +This will ask the user to approve the applications access to their account +based on the scopes specified in `REQUESTED_SCOPES` and then redirect back to +the `REDIRECT_URI` you provided. The [scope parameter](https://github.com/doorkeeper-gem/doorkeeper/wiki/Using-Scopes#requesting-particular-scopes) + is a space separated list of scopes you want to have access to (e.g. `scope=read_user+profile` +would request `read_user` and `profile` scopes). The redirect will include a fragment with `access_token` as well as token details in GET parameters, for example: diff --git a/doc/api/pipeline_triggers.md b/doc/api/pipeline_triggers.md index 736312df116..e207ff8e98a 100644 --- a/doc/api/pipeline_triggers.md +++ b/doc/api/pipeline_triggers.md @@ -120,35 +120,6 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" --form descript } ``` -## Take ownership of a project trigger - -Update an owner of a project trigger. - -``` -POST /projects/:id/triggers/:trigger_id/take_ownership -``` - -| Attribute | Type | required | Description | -|---------------|---------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `trigger_id` | integer | yes | The trigger id | - -``` -curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/triggers/10/take_ownership" -``` - -```json -{ - "id": 10, - "description": "my trigger", - "created_at": "2016-01-07T09:53:58.235Z", - "last_used": null, - "token": "6d056f63e50fe6f8c5f8f4aa10edb7", - "updated_at": "2016-01-07T09:53:58.235Z", - "owner": null -} -``` - ## Remove a project trigger Remove a project's build trigger. diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index 87c7f371de1..b292c9dd7de 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -80,6 +80,75 @@ X-Gitlab-Size: 1476 ... ``` +## Get file blame from repository + +Allows you to receive blame information. Each blame range contains lines and corresponding commit info. + +``` +GET /projects/:id/repository/files/:file_path/blame +``` + +```bash +curl --request GET --header 'PRIVATE-TOKEN: <your_access_token>' 'https://gitlab.example.com/api/v4/projects/13083/repository/files/path%2Fto%2Ffile.rb/blame?ref=master' +``` + +Example response: + +```json +[ + { + "commit": { + "id": "d42409d56517157c48bf3bd97d3f75974dde19fb", + "message": "Add feature\n\nalso fix bug\n", + "parent_ids": [ + "cc6e14f9328fa6d7b5a0d3c30dc2002a3f2a3822" + ], + "authored_date": "2015-12-18T08:12:22.000Z", + "author_name": "John Doe", + "author_email": "john.doe@example.com", + "committed_date": "2015-12-18T08:12:22.000Z", + "committer_name": "John Doe", + "committer_email": "john.doe@example.com" + }, + "lines": [ + "require 'fileutils'", + "require 'open3'", + "" + ] + }, + ... +] +``` + +Parameters: + +- `file_path` (required) - Url encoded full path to new file. Ex. lib%2Fclass%2Erb +- `ref` (required) - The name of branch, tag or commit + +NOTE: **Note:** +`HEAD` method return just file metadata as in [Get file from repository](repository_files.md#get-file-from-repository). + +```bash +curl --head --header 'PRIVATE-TOKEN: <your_access_token>' 'https://gitlab.example.com/api/v4/projects/13083/repository/files/path%2Fto%2Ffile.rb/blame?ref=master' +``` + +Example response: + +```text +HTTP/1.1 200 OK +... +X-Gitlab-Blob-Id: 79f7bbd25901e8334750839545a9bd021f0e4c83 +X-Gitlab-Commit-Id: d5a3ff139356ce33e37e73add446f16869741b50 +X-Gitlab-Content-Sha256: 4c294617b60715c1d218e61164a3abd4808a4284cbc30e6728a01ad9aada4481 +X-Gitlab-Encoding: base64 +X-Gitlab-File-Name: file.rb +X-Gitlab-File-Path: path/to/file.rb +X-Gitlab-Last-Commit-Id: 570e7b2abdd848b95f2f578043fc23bd6f6fd24d +X-Gitlab-Ref: master +X-Gitlab-Size: 1476 +... +``` + ## Get raw file from repository ``` diff --git a/doc/api/services.md b/doc/api/services.md index df15e6892b0..45b49d7eb92 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -972,22 +972,28 @@ Parameters: | `channel` | string | false | Default channel to use if others are not configured | | `notify_only_broken_pipelines` | boolean | false | Send notifications for broken pipelines | | `notify_only_default_branch` | boolean | false | Send notifications only for the default branch | -| `push_events` | boolean | false | Enable notifications for push events | -| `issues_events` | boolean | false | Enable notifications for issue events | +| `commit_events` | boolean | false | Enable notifications for commit events | +| `confidential_issue_channel` | string | false | The name of the channel to receive confidential issues events notifications | | `confidential_issues_events` | boolean | false | Enable notifications for confidential issue events | +| `confidential_note_channel` | string | false | The name of the channel to receive confidential note events notifications | +| `confidential_note_events` | boolean | false | Enable notifications for confidential note events | +| `deployment_channel` | string | false | The name of the channel to receive deployment events notifications | +| `deployment_events` | boolean | false | Enable notifications for deployment events | +| `issue_channel` | string | false | The name of the channel to receive issues events notifications | +| `issues_events` | boolean | false | Enable notifications for issue events | +| `job_events` | boolean | false | Enable notifications for job events | +| `merge_request_channel` | string | false | The name of the channel to receive merge request events notifications | | `merge_requests_events` | boolean | false | Enable notifications for merge request events | -| `tag_push_events` | boolean | false | Enable notifications for tag push events | +| `note_channel` | string | false | The name of the channel to receive note events notifications | | `note_events` | boolean | false | Enable notifications for note events | +| `pipeline_channel` | string | false | The name of the channel to receive pipeline events notifications | | `pipeline_events` | boolean | false | Enable notifications for pipeline events | -| `wiki_page_events` | boolean | false | Enable notifications for wiki page events | | `push_channel` | string | false | The name of the channel to receive push events notifications | -| `issue_channel` | string | false | The name of the channel to receive issues events notifications | -| `confidential_issue_channel` | string | false | The name of the channel to receive confidential issues events notifications | -| `merge_request_channel` | string | false | The name of the channel to receive merge request events notifications | -| `note_channel` | string | false | The name of the channel to receive note events notifications | +| `push_events` | boolean | false | Enable notifications for push events | | `tag_push_channel` | string | false | The name of the channel to receive tag push events notifications | -| `pipeline_channel` | string | false | The name of the channel to receive pipeline events notifications | +| `tag_push_events` | boolean | false | Enable notifications for tag push events | | `wiki_page_channel` | string | false | The name of the channel to receive wiki page events notifications | +| `wiki_page_events` | boolean | false | Enable notifications for wiki page events | ### Delete Slack service diff --git a/doc/api/settings.md b/doc/api/settings.md index 68da88af698..83125aff264 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -64,7 +64,10 @@ Example response: "performance_bar_allowed_group_id": 42, "instance_statistics_visibility_private": false, "user_show_add_ssh_key_message": true, - "local_markdown_version": 0 + "local_markdown_version": 0, + "allow_local_requests_from_hooks_and_services": true, + "allow_local_requests_from_web_hooks_and_services": true, + "allow_local_requests_from_system_hooks": false } ``` @@ -138,7 +141,10 @@ Example response: "user_show_add_ssh_key_message": true, "file_template_project_id": 1, "local_markdown_version": 0, - "geo_node_allowed_ips": "0.0.0.0/0, ::/0" + "geo_node_allowed_ips": "0.0.0.0/0, ::/0", + "allow_local_requests_from_hooks_and_services": true, + "allow_local_requests_from_web_hooks_and_services": true, + "allow_local_requests_from_system_hooks": false } ``` @@ -177,7 +183,9 @@ are listed in the descriptions of the relevant settings. | `akismet_api_key` | string | required by: `akismet_enabled` | API key for akismet spam protection. | | `akismet_enabled` | boolean | no | (**If enabled, requires:** `akismet_api_key`) Enable or disable akismet spam protection. | | `allow_group_owners_to_manage_ldap` | boolean | no | **(PREMIUM)** Set to `true` to allow group owners to manage LDAP | -| `allow_local_requests_from_hooks_and_services` | boolean | no | Allow requests to the local network from hooks and services. | +| `allow_local_requests_from_hooks_and_services` | boolean | no | (Deprecated: Use `allow_local_requests_from_web_hooks_and_services` instead) Allow requests to the local network from hooks and services. | +| `allow_local_requests_from_web_hooks_and_services` | boolean | no | Allow requests to the local network from web hooks and services. | +| `allow_local_requests_from_system_hooks` | boolean | no | Allow requests to the local network from system hooks. | | `authorized_keys_enabled` | boolean | no | By default, we write to the `authorized_keys` file to support Git over SSH without additional configuration. GitLab can be optimized to authenticate SSH keys via the database file. Only disable this if you have configured your OpenSSH server to use the AuthorizedKeysCommand. | | `auto_devops_domain` | string | no | Specify a domain to use by default for every project's Auto Review Apps and Auto Deploy stages. | | `auto_devops_enabled` | boolean | no | Enable Auto DevOps for projects by default. It will automatically build, test, and deploy applications based on a predefined CI/CD configuration. | @@ -230,7 +238,7 @@ are listed in the descriptions of the relevant settings. | `gravatar_enabled` | boolean | no | Enable Gravatar. | | `hashed_storage_enabled` | boolean | no | Create new projects using hashed storage paths: Enable immutable, hash-based paths and repository names to store repositories on disk. This prevents repositories from having to be moved or renamed when the Project URL changes and may improve disk I/O performance. (EXPERIMENTAL) | | `help_page_hide_commercial_content` | boolean | no | Hide marketing-related entries from help. | -| `help_page_support_url` | string | no | Alternate support URL for help page. | +| `help_page_support_url` | string | no | Alternate support URL for help page and help dropdown. | | `help_page_text` | string | no | Custom text displayed on the help page. | | `help_text` | string | no | **(PREMIUM)** GitLab server administrator information | | `hide_third_party_offers` | boolean | no | Do not display offers from third parties within GitLab. | diff --git a/doc/ci/README.md b/doc/ci/README.md index 7048ceaac41..6923d07bb1d 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -13,6 +13,11 @@ through the [continuous methodologies](introduction/index.md#introduction-to-cic - Continuous Delivery (CD) - Continuous Deployment (CD) +NOTE: **Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more.** +Watch our +["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) +webcast to learn about continuous methods and how GitLab’s built-in CI can help you simplify and scale software development. + ## Overview Continuous Integration works by pushing small code chunks to your @@ -155,6 +160,7 @@ for your CI/CD infrastructure: - [Why we chose GitLab CI for our CI/CD solution](https://about.gitlab.com/2016/10/17/gitlab-ci-oohlala/) - [Building our web-app on GitLab CI](https://about.gitlab.com/2016/07/22/building-our-web-app-on-gitlab-ci/) +- [5 Teams that made the switch to GitLab CI/CD](https://about.gitlab.com/2019/04/25/5-teams-that-made-the-switch-to-gitlab-ci-cd/) See also the [Why CI/CD?](https://docs.google.com/presentation/d/1OGgk2Tcxbpl7DJaIOzCX4Vqg3dlwfELC3u2jEeCBbDk) presentation. diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index b3110b435db..8fe11140cc7 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -14,9 +14,9 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab create a **CI/CD for external repo**, select **Repo by URL** and create the project. -  +  - GitLab will import the repository and enable [Pull Mirroring][pull-mirroring]. + GitLab will import the repository and enable [Pull Mirroring][pull-mirroring]. 1. In GitLab create a [Personal Access Token](../../user/profile/personal_access_tokens.md) @@ -26,19 +26,19 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify GitLab of new commits. - The web hook URL should be set to the GitLab API to trigger pull mirroring, - using the Personal Access Token we just generated for authentication. + The web hook URL should be set to the GitLab API to trigger pull mirroring, + using the Personal Access Token we just generated for authentication. - ```text - https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> - ``` + ```text + https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> + ``` - The web hook Trigger should be set to 'Repository Push'. + The web hook Trigger should be set to 'Repository Push'. -  +  - After saving, test the web hook by pushing a change to your Bitbucket - repository. + After saving, test the web hook by pushing a change to your Bitbucket + repository. 1. In Bitbucket, create an **App Password** from **Bitbucket Settings > App Passwords** to authenticate the build status script setting commit build @@ -49,104 +49,104 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab, from **Settings > CI/CD > Environment variables**, add variables to allow communication with Bitbucket via the Bitbucket API: - `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above. + `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above. - `BITBUCKET_USERNAME`: the username of the Bitbucket account. + `BITBUCKET_USERNAME`: the username of the Bitbucket account. - `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ. + `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ. - `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ. + `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ. 1. In Bitbucket, add a script to push the pipeline status to Bitbucket. - > Note: changes made in GitLab will be overwritten by any changes made - > upstream in Bitbucket. - - Create a file `build_status` and insert the script below and run - `chmod +x build_status` in your terminal to make the script executable. - - ```bash - #!/usr/bin/env bash - - # Push GitLab CI/CD build status to Bitbucket Cloud - - if [ -z "$BITBUCKET_ACCESS_TOKEN" ]; then - echo "ERROR: BITBUCKET_ACCESS_TOKEN is not set" - exit 1 - fi - if [ -z "$BITBUCKET_USERNAME" ]; then - echo "ERROR: BITBUCKET_USERNAME is not set" - exit 1 - fi - if [ -z "$BITBUCKET_NAMESPACE" ]; then - echo "Setting BITBUCKET_NAMESPACE to $CI_PROJECT_NAMESPACE" - BITBUCKET_NAMESPACE=$CI_PROJECT_NAMESPACE - fi - if [ -z "$BITBUCKET_REPOSITORY" ]; then - echo "Setting BITBUCKET_REPOSITORY to $CI_PROJECT_NAME" - BITBUCKET_REPOSITORY=$CI_PROJECT_NAME - fi - - BITBUCKET_API_ROOT="https://api.bitbucket.org/2.0" - BITBUCKET_STATUS_API="$BITBUCKET_API_ROOT/repositories/$BITBUCKET_NAMESPACE/$BITBUCKET_REPOSITORY/commit/$CI_COMMIT_SHA/statuses/build" - BITBUCKET_KEY="ci/gitlab-ci/$CI_JOB_NAME" - - case "$BUILD_STATUS" in - running) - BITBUCKET_STATE="INPROGRESS" - BITBUCKET_DESCRIPTION="The build is running!" - ;; - passed) - BITBUCKET_STATE="SUCCESSFUL" - BITBUCKET_DESCRIPTION="The build passed!" - ;; - failed) - BITBUCKET_STATE="FAILED" - BITBUCKET_DESCRIPTION="The build failed." - ;; - esac - - echo "Pushing status to $BITBUCKET_STATUS_API..." - curl --request POST $BITBUCKET_STATUS_API \ - --user $BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN \ - --header "Content-Type:application/json" \ - --silent \ - --data "{ \"state\": \"$BITBUCKET_STATE\", \"key\": \"$BITBUCKET_KEY\", \"description\": - \"$BITBUCKET_DESCRIPTION\",\"url\": \"$CI_PROJECT_URL/-/jobs/$CI_JOB_ID\" }" - ``` + > Note: changes made in GitLab will be overwritten by any changes made + > upstream in Bitbucket. + + Create a file `build_status` and insert the script below and run + `chmod +x build_status` in your terminal to make the script executable. + + ```bash + #!/usr/bin/env bash + + # Push GitLab CI/CD build status to Bitbucket Cloud + + if [ -z "$BITBUCKET_ACCESS_TOKEN" ]; then + echo "ERROR: BITBUCKET_ACCESS_TOKEN is not set" + exit 1 + fi + if [ -z "$BITBUCKET_USERNAME" ]; then + echo "ERROR: BITBUCKET_USERNAME is not set" + exit 1 + fi + if [ -z "$BITBUCKET_NAMESPACE" ]; then + echo "Setting BITBUCKET_NAMESPACE to $CI_PROJECT_NAMESPACE" + BITBUCKET_NAMESPACE=$CI_PROJECT_NAMESPACE + fi + if [ -z "$BITBUCKET_REPOSITORY" ]; then + echo "Setting BITBUCKET_REPOSITORY to $CI_PROJECT_NAME" + BITBUCKET_REPOSITORY=$CI_PROJECT_NAME + fi + + BITBUCKET_API_ROOT="https://api.bitbucket.org/2.0" + BITBUCKET_STATUS_API="$BITBUCKET_API_ROOT/repositories/$BITBUCKET_NAMESPACE/$BITBUCKET_REPOSITORY/commit/$CI_COMMIT_SHA/statuses/build" + BITBUCKET_KEY="ci/gitlab-ci/$CI_JOB_NAME" + + case "$BUILD_STATUS" in + running) + BITBUCKET_STATE="INPROGRESS" + BITBUCKET_DESCRIPTION="The build is running!" + ;; + passed) + BITBUCKET_STATE="SUCCESSFUL" + BITBUCKET_DESCRIPTION="The build passed!" + ;; + failed) + BITBUCKET_STATE="FAILED" + BITBUCKET_DESCRIPTION="The build failed." + ;; + esac + + echo "Pushing status to $BITBUCKET_STATUS_API..." + curl --request POST $BITBUCKET_STATUS_API \ + --user $BITBUCKET_USERNAME:$BITBUCKET_ACCESS_TOKEN \ + --header "Content-Type:application/json" \ + --silent \ + --data "{ \"state\": \"$BITBUCKET_STATE\", \"key\": \"$BITBUCKET_KEY\", \"description\": + \"$BITBUCKET_DESCRIPTION\",\"url\": \"$CI_PROJECT_URL/-/jobs/$CI_JOB_ID\" }" + ``` 1. Still in Bitbucket, create a `.gitlab-ci.yml` file to use the script to push pipeline success and failures to Bitbucket. - ```yaml - stages: - - test - - ci_status - - unit-tests: - script: - - echo "Success. Add your tests!" - - success: - stage: ci_status - before_script: - - "" - after_script: - - "" - script: - - BUILD_STATUS=passed BUILD_KEY=push ./build_status - when: on_success - - failure: - stage: ci_status - before_script: - - "" - after_script: - - "" - script: - - BUILD_STATUS=failed BUILD_KEY=push ./build_status - when: on_failure - ``` + ```yaml + stages: + - test + - ci_status + + unit-tests: + script: + - echo "Success. Add your tests!" + + success: + stage: ci_status + before_script: + - "" + after_script: + - "" + script: + - BUILD_STATUS=passed BUILD_KEY=push ./build_status + when: on_success + + failure: + stage: ci_status + before_script: + - "" + after_script: + - "" + script: + - BUILD_STATUS=failed BUILD_KEY=push ./build_status + when: on_failure + ``` GitLab is now configured to mirror changes from Bitbucket, run CI/CD pipelines configured in `.gitlab-ci.yml` and push the status to Bitbucket. diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 9f1ce1fc230..278a0d6e934 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -48,42 +48,42 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. 1. During GitLab Runner installation select `shell` as method of executing job scripts or use command: - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor shell \ - --description "My Runner" - ``` + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor shell \ + --description "My Runner" + ``` 1. Install Docker Engine on server. - For more information how to install Docker Engine on different systems - checkout the [Supported installations](https://docs.docker.com/engine/installation/). + For more information how to install Docker Engine on different systems + checkout the [Supported installations](https://docs.docker.com/engine/installation/). 1. Add `gitlab-runner` user to `docker` group: - ```bash - sudo usermod -aG docker gitlab-runner - ``` + ```bash + sudo usermod -aG docker gitlab-runner + ``` 1. Verify that `gitlab-runner` has access to Docker: - ```bash - sudo -u gitlab-runner -H docker info - ``` + ```bash + sudo -u gitlab-runner -H docker info + ``` - You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: + You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`: - ```yaml - before_script: - - docker info + ```yaml + before_script: + - docker info - build_image: - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` + build_image: + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` 1. You can now use `docker` command (and **install** `docker-compose` if needed). @@ -91,7 +91,7 @@ NOTE: **Note:** By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful). -### Use docker-in-docker executor +### Use docker-in-docker workflow with Docker executor The second approach is to use the special docker-in-docker (dind) [Docker image](https://hub.docker.com/_/docker/) with all tools installed @@ -107,83 +107,83 @@ In order to do that, follow the steps: 1. Register GitLab Runner from the command line to use `docker` and `privileged` mode: - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:stable" \ - --docker-privileged - ``` - - The above command will register a new Runner to use the special - `docker:stable` image which is provided by Docker. **Notice that it's using - the `privileged` mode to start the build and service containers.** If you - want to use [docker-in-docker] mode, you always have to use `privileged = true` - in your Docker containers. - - DANGER: **Danger:** - By enabling `--docker-privileged`, you are effectively disabling all of - the security mechanisms of containers and exposing your host to privilege - escalation which can lead to container breakout. For more information, check - out the official Docker documentation on - [Runtime privilege and Linux capabilities][docker-cap]. - - The above command will create a `config.toml` entry similar to this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:stable" - privileged = true - disable_cache = false - volumes = ["/cache"] - [runners.cache] - Insecure = false - ``` + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:stable" \ + --docker-privileged + ``` + + The above command will register a new Runner to use the special + `docker:stable` image which is provided by Docker. **Notice that it's using + the `privileged` mode to start the build and service containers.** If you + want to use [docker-in-docker] mode, you always have to use `privileged = true` + in your Docker containers. + + DANGER: **Danger:** + By enabling `--docker-privileged`, you are effectively disabling all of + the security mechanisms of containers and exposing your host to privilege + escalation which can lead to container breakout. For more information, check + out the official Docker documentation on + [Runtime privilege and Linux capabilities][docker-cap]. + + The above command will create a `config.toml` entry similar to this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:stable" + privileged = true + disable_cache = false + volumes = ["/cache"] + [runners.cache] + Insecure = false + ``` 1. You can now use `docker` in the build script (note the inclusion of the `docker:dind` service): - ```yaml - image: docker:stable - - variables: - # When using dind service we need to instruct docker, to talk with the - # daemon started inside of the service. The daemon is available with - # a network connection instead of the default /var/run/docker.sock socket. - # - # The 'docker' hostname is the alias of the service container as described at - # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services - # - # Note that if you're using the Kubernetes executor, the variable should be set to - # tcp://localhost:2375/ because of how the Kubernetes executor connects services - # to the job container - # DOCKER_HOST: tcp://localhost:2375/ - # - # For non-Kubernetes executors, we use tcp://docker:2375/ - DOCKER_HOST: tcp://docker:2375/ - # When using dind, it's wise to use the overlayfs driver for - # improved performance. - DOCKER_DRIVER: overlay2 - - services: - - docker:dind - - before_script: - - docker info - - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` + ```yaml + image: docker:stable + + variables: + # When using dind service we need to instruct docker, to talk with the + # daemon started inside of the service. The daemon is available with + # a network connection instead of the default /var/run/docker.sock socket. + # + # The 'docker' hostname is the alias of the service container as described at + # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services + # + # Note that if you're using the Kubernetes executor, the variable should be set to + # tcp://localhost:2375/ because of how the Kubernetes executor connects services + # to the job container + # DOCKER_HOST: tcp://localhost:2375/ + # + # For non-Kubernetes executors, we use tcp://docker:2375/ + DOCKER_HOST: tcp://docker:2375/ + # When using dind, it's wise to use the overlayfs driver for + # improved performance. + DOCKER_DRIVER: overlay2 + + services: + - docker:dind + + before_script: + - docker info + + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` Docker-in-Docker works well, and is the recommended configuration, but it is not without its own challenges: @@ -202,14 +202,14 @@ not without its own challenges: and use it as your mount point (for a more thorough explanation, check [issue #41227](https://gitlab.com/gitlab-org/gitlab-ce/issues/41227)): - ```yaml - variables: - MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt + ```yaml + variables: + MOUNT_POINT: /builds/$CI_PROJECT_PATH/mnt - script: - - mkdir -p "$MOUNT_POINT" - - docker run -v "$MOUNT_POINT:/mnt" my-docker-image - ``` + script: + - mkdir -p "$MOUNT_POINT" + - docker run -v "$MOUNT_POINT:/mnt" my-docker-image + ``` An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>. @@ -230,54 +230,54 @@ In order to do that, follow the steps: 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: - ```bash - sudo gitlab-runner register -n \ - --url https://gitlab.com/ \ - --registration-token REGISTRATION_TOKEN \ - --executor docker \ - --description "My Docker Runner" \ - --docker-image "docker:stable" \ - --docker-volumes /var/run/docker.sock:/var/run/docker.sock - ``` - - The above command will register a new Runner to use the special - `docker:stable` image which is provided by Docker. **Notice that it's using - the Docker daemon of the Runner itself, and any containers spawned by docker - commands will be siblings of the Runner rather than children of the runner.** - This may have complications and limitations that are unsuitable for your workflow. - - The above command will create a `config.toml` entry similar to this: - - ```toml - [[runners]] - url = "https://gitlab.com/" - token = REGISTRATION_TOKEN - executor = "docker" - [runners.docker] - tls_verify = false - image = "docker:stable" - privileged = false - disable_cache = false - volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] - [runners.cache] - Insecure = false - ``` + ```bash + sudo gitlab-runner register -n \ + --url https://gitlab.com/ \ + --registration-token REGISTRATION_TOKEN \ + --executor docker \ + --description "My Docker Runner" \ + --docker-image "docker:stable" \ + --docker-volumes /var/run/docker.sock:/var/run/docker.sock + ``` + + The above command will register a new Runner to use the special + `docker:stable` image which is provided by Docker. **Notice that it's using + the Docker daemon of the Runner itself, and any containers spawned by docker + commands will be siblings of the Runner rather than children of the runner.** + This may have complications and limitations that are unsuitable for your workflow. + + The above command will create a `config.toml` entry similar to this: + + ```toml + [[runners]] + url = "https://gitlab.com/" + token = REGISTRATION_TOKEN + executor = "docker" + [runners.docker] + tls_verify = false + image = "docker:stable" + privileged = false + disable_cache = false + volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] + [runners.cache] + Insecure = false + ``` 1. You can now use `docker` in the build script (note that you don't need to include the `docker:dind` service as when using the Docker in Docker executor): - ```yaml - image: docker:stable + ```yaml + image: docker:stable - before_script: - - docker info + before_script: + - docker info - build: - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests - ``` + build: + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests + ``` While the above method avoids using Docker in privileged mode, you should be aware of the following implications: @@ -293,9 +293,9 @@ aware of the following implications: work as expected since volume mounting is done in the context of the host machine, not the build container. For example: - ```sh - docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests - ``` + ```sh + docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests + ``` ## Making docker-in-docker builds faster with Docker layer caching @@ -366,23 +366,23 @@ which can be avoided if a different driver is used, for example `overlay2`. 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: - ```sh - sudo lsmod | grep overlay - ``` + ```sh + sudo lsmod | grep overlay + ``` - If you see no result, then it isn't loaded. To load it use: + If you see no result, then it isn't loaded. To load it use: - ```sh - sudo modprobe overlay - ``` + ```sh + sudo modprobe overlay + ``` - If everything went fine, you need to make sure module is loaded on reboot. - On Ubuntu systems, this is done by editing `/etc/modules`. Just add the - following line into it: + If everything went fine, you need to make sure module is loaded on reboot. + On Ubuntu systems, this is done by editing `/etc/modules`. Just add the + following line into it: - ```text - overlay - ``` + ```text + overlay + ``` ### Use driver per project @@ -450,9 +450,9 @@ For all projects, mostly suitable for public ones: your Docker images and has read/write access to the Registry. This is ephemeral, so it's only valid for one job. You can use the following example as-is: - ```sh - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - ``` + ```sh + docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + ``` For private and internal projects: @@ -465,9 +465,9 @@ For private and internal projects: Replace the `<username>` and `<access_token>` in the following example: - ```sh - docker login -u <username> -p <access_token> $CI_REGISTRY - ``` + ```sh + docker login -u <username> -p <access_token> $CI_REGISTRY + ``` - **Using the GitLab Deploy Token**: You can create and use a [special deploy token](../../user/project/deploy_tokens/index.md#gitlab-deploy-token) @@ -475,9 +475,9 @@ For private and internal projects: Once created, you can use the special environment variables, and GitLab CI/CD will fill them in for you. You can use the following example as-is: - ```sh - docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY - ``` + ```sh + docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY + ``` ### Container Registry examples diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index f3896c5232c..d5056568dff 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -305,25 +305,25 @@ For example, the following two definitions are equal: 1. Using a string as an option to `image` and `services`: - ```yaml - image: "registry.example.com/my/image:latest" + ```yaml + image: "registry.example.com/my/image:latest" - services: - - postgresql:9.4 - - redis:latest - ``` + services: + - postgresql:9.4 + - redis:latest + ``` 1. Using a map as an option to `image` and `services`. The use of `image:name` is required: - ```yaml - image: - name: "registry.example.com/my/image:latest" + ```yaml + image: + name: "registry.example.com/my/image:latest" - services: - - name: postgresql:9.4 - - name: redis:latest - ``` + services: + - name: postgresql:9.4 + - name: redis:latest + ``` ### Available settings for `image` @@ -526,6 +526,7 @@ it's provided as an environment variable. This is because GitLab Runnner uses ** runtime. ### Using statically-defined credentials + There are two approaches that you can take in order to access a private registry. Both require setting the environment variable `DOCKER_AUTH_CONFIG` with appropriate authentication info. @@ -555,18 +556,18 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: - **First way -** Do a `docker login` on your local machine: - ```bash - docker login registry.example.com:5000 --username my_username --password my_password - ``` + ```bash + docker login registry.example.com:5000 --username my_username --password my_password + ``` - Then copy the content of `~/.docker/config.json`. + Then copy the content of `~/.docker/config.json`. - If you don't need access to the registry from your computer, you - can do a `docker logout`: + If you don't need access to the registry from your computer, you + can do a `docker logout`: - ```bash - docker logout registry.example.com:5000 - ``` + ```bash + docker logout registry.example.com:5000 + ``` - **Second way -** In some setups, it's possible that Docker client will use the available system keystore to store the result of `docker @@ -575,12 +576,12 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`: `${username}:${password}` manually. Open a terminal and execute the following command: - ```bash - echo -n "my_username:my_password" | base64 + ```bash + echo -n "my_username:my_password" | base64 - # Example output to copy - bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= - ``` + # Example output to copy + bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= + ``` #### Configuring a job @@ -590,25 +591,25 @@ follow these steps: 1. Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: - ```json - { - "auths": { - "registry.example.com:5000": { - "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" - } - } - } - ``` + ```json + { + "auths": { + "registry.example.com:5000": { + "auth": "bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=" + } + } + } + ``` 1. You can now use any private image from `registry.example.com:5000` defined in `image` and/or `services` in your `.gitlab-ci.yml` file: - ```yaml - image: registry.example.com:5000/namespace/image:tag - ``` + ```yaml + image: registry.example.com:5000/namespace/image:tag + ``` - In the example above, GitLab Runner will look at `registry.example.com:5000` for the - image `namespace/image:tag`. + In the example above, GitLab Runner will look at `registry.example.com:5000` for the + image `namespace/image:tag`. You can add configuration for as many registries as you want, adding more registries to the `"auths"` hash as described above. @@ -637,10 +638,10 @@ To add `DOCKER_AUTH_CONFIG` to a Runner: 1. Modify the Runner's `config.toml` file as follows: - ```toml - [[runners]] - environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=\"}}}"] - ``` + ```toml + [[runners]] + environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=\"}}}"] + ``` 1. Restart the Runner service. @@ -662,16 +663,17 @@ To configure credentials store, follow these steps: Make sure helper program is available in GitLab Runner `$PATH`. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: + - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the - Docker configuration file as the value: + Docker configuration file as the value: - ```json - { - "credsStore": "osxkeychain" - } - ``` + ```json + { + "credsStore": "osxkeychain" + } + ``` - Or, if you are running self-hosted Runners, add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner will read this config file @@ -693,17 +695,18 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th 1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: + - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) `DOCKER_AUTH_CONFIG` with the content of the - Docker configuration file as the value: + Docker configuration file as the value: - ```json - { - "credHelpers": { - "aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login" - } - } - ``` + ```json + { + "credHelpers": { + "aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login" + } + } + ``` - Or, if you are running self-hosted Runners, add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`. @@ -713,12 +716,12 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th 1. You can now use any private image from `aws_account_id.dkr.ecr.region.amazonaws.com` defined in `image` and/or `services` in your `.gitlab-ci.yml` file: - ```yaml - image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest - ``` + ```yaml + image: aws_account_id.dkr.ecr.region.amazonaws.com/private/image:latest + ``` - In the example above, GitLab Runner will look at `aws_account_id.dkr.ecr.region.amazonaws.com` for the - image `private/image:latest`. + In the example above, GitLab Runner will look at `aws_account_id.dkr.ecr.region.amazonaws.com` for the + image `private/image:latest`. You can add configuration for as many registries as you want, adding more registries to the `"credHelpers"` hash as described above. diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index 50f1ac3d54a..925653f9fdf 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -11,7 +11,8 @@ Requires GitLab Runner 11.2 and above. container images from a Dockerfile, inside a container or Kubernetes cluster. kaniko solves two problems with using the -[docker-in-docker build](using_docker_build.md#use-docker-in-docker-executor) method: +[docker-in-docker +build](using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor) method: - Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities) in order to function, which is a significant security concern. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 9295dcfd4e0..00995f881da 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -33,7 +33,7 @@ The following table lists examples with step-by-step tutorials that are containe | Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | | PHP with PHPunit, atoum | [Testing PHP projects](php.md). | | PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP with Laravel, Ennvoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | +| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | | Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | | Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | | Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index c9f700ed190..940c4711132 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -39,9 +39,10 @@ project: 1. Create a new project by selecting **Import project from ➔ Repo by URL** 1. Add the following URL: - ``` - https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git - ``` + ``` + https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git + ``` + 1. Click **Create project** This application is nothing more than a basic class with a stub for a JUnit based test suite. @@ -63,15 +64,15 @@ The application is ready to use, but you need some additional steps to deploy it 1. Copy the snippet in the `pom.xml` file for your project, just after the `dependencies` section. The snippet should look like this: - ```xml - <distributionManagement> - <repository> - <id>central</id> - <name>83d43b5afeb5-releases</name> - <url>${env.MAVEN_REPO_URL}/libs-release-local</url> - </repository> - </distributionManagement> - ``` + ```xml + <distributionManagement> + <repository> + <id>central</id> + <name>83d43b5afeb5-releases</name> + <url>${env.MAVEN_REPO_URL}/libs-release-local</url> + </repository> + </distributionManagement> + ``` Another step you need to do before you can deploy the dependency to Artifactory is to configure the authentication data. It is a simple task, but Maven requires @@ -86,18 +87,18 @@ parameter in `.gitlab-ci.yml` to use the custom location instead of the default 1. Create a file called `settings.xml` in the `.m2` folder 1. Copy the following content into a `settings.xml` file: - ```xml - <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" - xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> - <servers> - <server> - <id>central</id> - <username>${env.MAVEN_REPO_USER}</username> - <password>${env.MAVEN_REPO_PASS}</password> - </server> - </servers> - </settings> - ``` + ```xml + <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" + xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> + <servers> + <server> + <id>central</id> + <username>${env.MAVEN_REPO_USER}</username> + <password>${env.MAVEN_REPO_PASS}</password> + </server> + </servers> + </settings> + ``` Username and password will be replaced by the correct values using variables. @@ -187,9 +188,10 @@ We'll use again a Maven app that can be cloned from our example project: 1. Create a new project by selecting **Import project from ➔ Repo by URL** 1. Add the following URL: - ``` - https://gitlab.com/gitlab-examples/maven/simple-maven-app.git - ``` + ``` + https://gitlab.com/gitlab-examples/maven/simple-maven-app.git + ``` + 1. Click **Create project** This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java` @@ -204,13 +206,13 @@ Since Maven doesn't know how to resolve the dependency, you need to modify the c 1. Copy the snippet in the `dependencies` section of the `pom.xml` file. The snippet should look like this: - ```xml - <dependency> - <groupId>com.example.dep</groupId> - <artifactId>simple-maven-dep</artifactId> - <version>1.0</version> - </dependency> - ``` + ```xml + <dependency> + <groupId>com.example.dep</groupId> + <artifactId>simple-maven-dep</artifactId> + <version>1.0</version> + </dependency> + ``` ### Configure the Artifactory repository location diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md index 8ecac4a5a4f..3266e5dc62e 100644 --- a/doc/ci/examples/browser_performance.md +++ b/doc/ci/examples/browser_performance.md @@ -15,7 +15,7 @@ your code by using GitLab CI/CD and [sitespeed.io](https://www.sitespeed.io) using Docker-in-Docker. First, you need GitLab Runner with -[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). +[docker-in-docker build](../docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). Once you set up the Runner, add a new job to `.gitlab-ci.yml` that generates the expected report: @@ -155,4 +155,4 @@ performance: paths: - performance.json - sitespeed-results/ -```
\ No newline at end of file +``` diff --git a/doc/ci/examples/code_quality.md b/doc/ci/examples/code_quality.md index 43f773dab7c..69bad6b4c25 100644 --- a/doc/ci/examples/code_quality.md +++ b/doc/ci/examples/code_quality.md @@ -14,7 +14,7 @@ This example shows how to run Code Quality on your code by using GitLab CI/CD and Docker. First, you need GitLab Runner with -[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). +[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). Once you set up the Runner, include the CodeQuality template in your CI config: @@ -34,6 +34,12 @@ For [GitLab Starter][ee] users, this information will be automatically extracted and shown right in the merge request widget. [Learn more on Code Quality in merge requests](../../user/project/merge_requests/code_quality.md). +CAUTION: **Caution:** +On self-managed instances, if a malicious actor compromises the Code Quality job +definition they will be able to execute privileged docker commands on the Runner +host. Having proper access control policies mitigates this attack vector by +allowing access only to trusted actors. + ## Previous job definitions CAUTION: **Caution:** diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md index a5fed00972f..0e595e1a0be 100644 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md @@ -188,28 +188,27 @@ when running our Phoenix in our `localhost`. - Open `hello_gitlab_ci/config/test.exs` on your favorite code editor - Go to **Configure your database** session and edit the block to include `System.get_env`: - ```elixir - # Configure your database - config :hello_gitlab_ci, HelloGitlabCi.Repo, - adapter: Ecto.Adapters.Postgres, - username: System.get_env("POSTGRES_USER") || "postgres", - password: System.get_env("POSTGRES_PASSWORD") || "postgres", - database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test", - hostname: System.get_env("POSTGRES_HOST") || "localhost", - pool: Ecto.Adapters.SQL.Sandbox - ``` - - We'll need these system variables later on. + ```elixir + # Configure your database + config :hello_gitlab_ci, HelloGitlabCi.Repo, + adapter: Ecto.Adapters.Postgres, + username: System.get_env("POSTGRES_USER") || "postgres", + password: System.get_env("POSTGRES_PASSWORD") || "postgres", + database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test", + hostname: System.get_env("POSTGRES_HOST") || "localhost", + pool: Ecto.Adapters.SQL.Sandbox + ``` + + We'll need these system variables later on. - Create an empty file named `.gitkeep` into `hello_gitlab_ci/priv/repo/migrations` - As our project is still fresh, we don't have any data on our database, so, the `migrations` -directory will be empty. - Without `.gitkeep`, git will not upload this empty directory and we'll got an error when running our -test on GitLab. + As our project is still fresh, we don't have any data on our database, so, the `migrations` + directory will be empty. + Without `.gitkeep`, git will not upload this empty directory and we'll got an error when running our + test on GitLab. - > **Note:** - If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir. + > **Note:** If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir. Now, let's run a local test and see if everything we did didn't break anything. @@ -248,64 +247,64 @@ project. - The fastest and easiest way to do this, is to click on **Set up CI** on project's main page: -  +  - On next screen, we can select a template ready to go. Click on **Apply a GitLab CI/CD Yaml template** and select **Elixir**: -  +  - This template file tells GitLab CI/CD about what we wish to do every time a new commit is made. - However, we have to adapt it to run a Phoenix app. + This template file tells GitLab CI/CD about what we wish to do every time a new commit is made. + However, we have to adapt it to run a Phoenix app. - The first line tells GitLab what Docker image will be used. - Remember when we learn about Runners, the isolated virtual machine where GitLab CI/CD build and test - our application? This virtual machine must have all dependencies to run our application. This is - where a Docker image is needed. The correct image will provide the entire system for us. + Remember when we learn about Runners, the isolated virtual machine where GitLab CI/CD build and test + our application? This virtual machine must have all dependencies to run our application. This is + where a Docker image is needed. The correct image will provide the entire system for us. - As a suggestion, you can use [trenpixster's elixir image][docker-image], which already has all - dependencies for Phoenix installed, such as Elixir, Erlang, NodeJS and PostgreSQL: + As a suggestion, you can use [trenpixster's elixir image][docker-image], which already has all + dependencies for Phoenix installed, such as Elixir, Erlang, NodeJS and PostgreSQL: - ```yml - image: trenpixster/elixir:latest - ``` + ```yml + image: trenpixster/elixir:latest + ``` - At `services` session, we'll only use `postgres`, so we'll delete `mysql` and `redis` lines: - ```yml - services: - - postgres:latest - ``` + ```yml + services: + - postgres:latest + ``` - Now, we'll create a new entry called `variables`, before `before_script` session: - ```yml - variables: - POSTGRES_DB: hello_gitlab_ci_test - POSTGRES_HOST: postgres - POSTGRES_USER: postgres - POSTGRES_PASSWORD: "postgres" - MIX_ENV: "test" - ``` + ```yml + variables: + POSTGRES_DB: hello_gitlab_ci_test + POSTGRES_HOST: postgres + POSTGRES_USER: postgres + POSTGRES_PASSWORD: "postgres" + MIX_ENV: "test" + ``` - Here, we are setting up the values for GitLab CI/CD authenticate into PostgreSQL, as we did on - `config/test.exs` earlier. + Here, we are setting up the values for GitLab CI/CD authenticate into PostgreSQL, as we did on + `config/test.exs` earlier. - In `before_script` session, we'll add some commands to prepare everything to the test: - ```yml - before_script: - - apt-get update && apt-get -y install postgresql-client - - mix local.hex --force - - mix deps.get --only test - - mix ecto.create - - mix ecto.migrate - ``` - - It's important to install `postgresql-client` to let GitLab CI/CD access PostgreSQL and create our - database with the login information provided earlier. More important is to respect the indentation, - to avoid syntax errors when running the build. + ```yml + before_script: + - apt-get update && apt-get -y install postgresql-client + - mix local.hex --force + - mix deps.get --only test + - mix ecto.create + - mix ecto.migrate + ``` + + It's important to install `postgresql-client` to let GitLab CI/CD access PostgreSQL and create our + database with the login information provided earlier. More important is to respect the indentation, + to avoid syntax errors when running the build. - And finally, we'll let `mix` session intact. diff --git a/doc/ci/img/manual_job_variables.png b/doc/ci/img/manual_job_variables.png Binary files differnew file mode 100644 index 00000000000..c7d62477cdd --- /dev/null +++ b/doc/ci/img/manual_job_variables.png diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index 6401ff90a0a..366aca3442e 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -9,6 +9,11 @@ In this document we'll present an overview of the concepts of Continuous Integra Continuous Delivery, and Continuous Deployment, as well as an introduction to GitLab CI/CD. +NOTE: **Out-of-the-box management systems can decrease hours spent on maintaining toolchains by 10% or more.** +Watch our +["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) +webcast to learn about continuous methods and how GitLab’s built-in CI can help you simplify and scale software development. + ## Introduction to CI/CD methodologies The continuous methodologies of software development are based on diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 463b9194c58..ced4344a0b0 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -171,6 +171,11 @@ In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the upstream pipeline will be passed to the `downstream-job` job, and will be available within the context of all downstream builds. +NOTE: **Tip:** +Upstream pipelines take precedence over downstream ones. If there are two +variables with the same name defined in both upstream and downstream projects, +the ones defined in the upstream project will take precedence. + ### Limitations Because bridge jobs are a little different to regular jobs, it is not diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 98f30350968..ed8d0e3bc35 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -6,6 +6,11 @@ type: reference > Introduced in GitLab 8.8. +NOTE: **Tip:** +Watch our +["Mastering continuous software development"](https://about.gitlab.com/webcast/mastering-ci-cd/) +webcast to see a comprehensive demo of GitLab CI/CD pipeline. + ## Introduction Pipelines are the top-level component of continuous integration, delivery, and deployment. @@ -318,6 +323,20 @@ stage has a job with a manual action.  +### Specifying variables when running manual jobs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/30485) in GitLab 12.2. + +When running manual jobs you can supply additional job specific variables. + +You can do this from the job page of the manual job you want to run with +additional variables. + +This is useful when you want to alter the execution of a job by using +environment variables. + + + ### Delay a job in a pipeline graph > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21767) in GitLab 11.4. diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 69591ed605c..d9f022a7125 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -57,44 +57,44 @@ to access it. This is where an SSH key pair comes in handy. 1. Modify your `.gitlab-ci.yml` with a `before_script` action. In the following example, a Debian based image is assumed. Edit to your needs: - ```yaml - before_script: - ## - ## Install ssh-agent if not already installed, it is required by Docker. - ## (change apt-get to yum if you use an RPM-based image) - ## - - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - - ## - ## Run ssh-agent (inside the build environment) - ## - - eval $(ssh-agent -s) - - ## - ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store - ## We're using tr to fix line endings which makes ed25519 keys work - ## without extra base64 encoding. - ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556 - ## - - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null - - ## - ## Create the SSH directory and give it the right permissions - ## - - mkdir -p ~/.ssh - - chmod 700 ~/.ssh - - ## - ## Optionally, if you will be using any Git commands, set the user name and - ## and email. - ## - #- git config --global user.email "user@example.com" - #- git config --global user.name "User name" - ``` - - NOTE: **Note:** - The [`before_script`](../yaml/README.md#before_script-and-after_script) can be set globally - or per-job. + ```yaml + before_script: + ## + ## Install ssh-agent if not already installed, it is required by Docker. + ## (change apt-get to yum if you use an RPM-based image) + ## + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + + ## + ## Run ssh-agent (inside the build environment) + ## + - eval $(ssh-agent -s) + + ## + ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store + ## We're using tr to fix line endings which makes ed25519 keys work + ## without extra base64 encoding. + ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556 + ## + - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null + + ## + ## Create the SSH directory and give it the right permissions + ## + - mkdir -p ~/.ssh + - chmod 700 ~/.ssh + + ## + ## Optionally, if you will be using any Git commands, set the user name and + ## and email. + ## + #- git config --global user.email "user@example.com" + #- git config --global user.name "User name" + ``` + + NOTE: **Note:** + The [`before_script`](../yaml/README.md#before_script-and-after_script) can be set globally + or per-job. 1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys). @@ -118,9 +118,9 @@ on, and use that key for all projects that are run on this machine. 1. Then from the terminal login as the `gitlab-runner` user: - ``` - sudo su - gitlab-runner - ``` + ``` + sudo su - gitlab-runner + ``` 1. Generate the SSH key pair as described in the instructions to [generate an SSH key](../../ssh/README.md#generating-a-new-ssh-key-pair). diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index d1f9aa03b6b..2a382f18038 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -97,17 +97,6 @@ overview of the time the triggers were last used.  -## Taking ownership of a trigger - -> **Note**: -GitLab 9.0 introduced a trigger ownership to solve permission problems. - -Each created trigger when run will impersonate their associated user including -their access to projects and their project permissions. - -You can take ownership of existing triggers by clicking *Take ownership*. -From now on the trigger will be run as you. - ## Revoking a trigger You can revoke a trigger any time by going at your project's @@ -282,8 +271,7 @@ Old triggers, created before GitLab 9.0 will be marked as legacy. Triggers with the legacy label do not have an associated user and only have access to the current project. They are considered deprecated and will be -removed with one of the future versions of GitLab. You are advised to -[take ownership](#taking-ownership-of-a-trigger) of any legacy triggers. +removed with one of the future versions of GitLab. [ee-2017]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2017 [ee-2346]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2346 diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 4d6ca8cff6d..c63b1e104ed 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -370,8 +370,11 @@ variables take precedence over those defined in `.gitlab-ci.yml`. ## Unsupported variables There are cases where some variables cannot be used in the context of a -`.gitlab-ci.yml` definition (for example under `script`). Read more -about which variables are [not supported](where_variables_can_be_used.md). +`.gitlab-ci.yml` definition (for example under `script`). Read more about which variables are [not supported](where_variables_can_be_used.md). + +## Where variables can be used + +Click [here](where_variables_can_be_used.md) for a section that describes where and how the different types of variables can be used. ## Advanced use @@ -481,81 +484,86 @@ Below you can find supported syntax reference: 1. Equality matching using a string - > Example: `$VARIABLE == "some value"` + Examples: - > Example: `$VARIABLE != "some value"` (introduced in GitLab 11.11) + - `$VARIABLE == "some value"` + - `$VARIABLE != "some value"` (introduced in GitLab 11.11) - You can use equality operator `==` or `!=` to compare a variable content to a - string. We support both, double quotes and single quotes to define a string - value, so both `$VARIABLE == "some value"` and `$VARIABLE == 'some value'` - are supported. `"some value" == $VARIABLE` is correct too. + You can use equality operator `==` or `!=` to compare a variable content to a + string. We support both, double quotes and single quotes to define a string + value, so both `$VARIABLE == "some value"` and `$VARIABLE == 'some value'` + are supported. `"some value" == $VARIABLE` is correct too. 1. Checking for an undefined value - > Example: `$VARIABLE == null` + Examples: - > Example: `$VARIABLE != null` (introduced in GitLab 11.11) + - `$VARIABLE == null` + - `$VARIABLE != null` (introduced in GitLab 11.11) - It sometimes happens that you want to check whether a variable is defined - or not. To do that, you can compare a variable to `null` keyword, like - `$VARIABLE == null`. This expression is going to evaluate to truth if - variable is not defined when `==` is used, or to falsey if `!=` is used. + It sometimes happens that you want to check whether a variable is defined + or not. To do that, you can compare a variable to `null` keyword, like + `$VARIABLE == null`. This expression is going to evaluate to truth if + variable is not defined when `==` is used, or to falsey if `!=` is used. 1. Checking for an empty variable - > Example: `$VARIABLE == ""` + Examples: + + - `$VARIABLE == ""` + - `$VARIABLE != ""` (introduced in GitLab 11.11) - > Example: `$VARIABLE != ""` (introduced in GitLab 11.11) - - If you want to check whether a variable is defined, but is empty, you can - simply compare it against an empty string, like `$VAR == ''` or non-empty - string `$VARIABLE != ""`. + If you want to check whether a variable is defined, but is empty, you can + simply compare it against an empty string, like `$VAR == ''` or non-empty + string `$VARIABLE != ""`. 1. Comparing two variables - > Example: `$VARIABLE_1 == $VARIABLE_2` + Examples: - > Example: `$VARIABLE_1 != $VARIABLE_2` (introduced in GitLab 11.11) + - `$VARIABLE_1 == $VARIABLE_2` + - `$VARIABLE_1 != $VARIABLE_2` (introduced in GitLab 11.11) - It is possible to compare two variables. This is going to compare values - of these variables. + It is possible to compare two variables. This is going to compare values + of these variables. 1. Variable presence check - > Example: `$STAGING` + Example: `$STAGING` - If you only want to create a job when there is some variable present, - which means that it is defined and non-empty, you can simply use - variable name as an expression, like `$STAGING`. If `$STAGING` variable - is defined, and is non empty, expression will evaluate to truth. - `$STAGING` value needs to a string, with length higher than zero. - Variable that contains only whitespace characters is not an empty variable. + If you only want to create a job when there is some variable present, + which means that it is defined and non-empty, you can simply use + variable name as an expression, like `$STAGING`. If `$STAGING` variable + is defined, and is non empty, expression will evaluate to truth. + `$STAGING` value needs to a string, with length higher than zero. + Variable that contains only whitespace characters is not an empty variable. 1. Pattern matching (introduced in GitLab 11.0) - > Example: `$VARIABLE =~ /^content.*/` + Examples: - > Example: `$VARIABLE_1 !~ /^content.*/` (introduced in GitLab 11.11) + - `$VARIABLE =~ /^content.*/` + - `$VARIABLE_1 !~ /^content.*/` (introduced in GitLab 11.11) - It is possible perform pattern matching against a variable and regular - expression. Expression like this evaluates to truth if matches are found - when using `=~`. It evaluates to truth if matches are not found when `!~` is used. + It is possible perform pattern matching against a variable and regular + expression. Expression like this evaluates to truth if matches are found + when using `=~`. It evaluates to truth if matches are not found when `!~` is used. - Pattern matching is case-sensitive by default. Use `i` flag modifier, like - `/pattern/i` to make a pattern case-insensitive. + Pattern matching is case-sensitive by default. Use `i` flag modifier, like + `/pattern/i` to make a pattern case-insensitive. 1. Conjunction / Disjunction ([introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27925) in GitLab 12.0) - > Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 == "something"` - - > Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 =~ /thing$/ && $VARIABLE3` + Examples: - > Example: `$VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/ && $VARIABLE3` + - `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 == "something"` + - `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 =~ /thing$/ && $VARIABLE3` + - `$VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/ && $VARIABLE3` - It is possible to join multiple conditions using `&&` or `||`. Any of the otherwise - supported syntax may be used in a conjunctive or disjunctive statement. - Precedence of operators follows standard Ruby 2.5 operation - [precedence](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html). + It is possible to join multiple conditions using `&&` or `||`. Any of the otherwise + supported syntax may be used in a conjunctive or disjunctive statement. + Precedence of operators follows standard Ruby 2.5 operation + [precedence](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html). ## Debug tracing diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 09b9fc87986..a6051e87366 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1771,18 +1771,41 @@ sequentially from `job_name 1/N` to `job_name N/N`. For every job, `CI_NODE_INDEX` and `CI_NODE_TOTAL` [environment variables](../variables/README.md#predefined-environment-variables) are set. -A simple example: +Marking a job to be run in parallel requires only a simple addition to your configuration file: -```yaml -test: - script: rspec - parallel: 5 +```diff + test: + script: rspec ++ parallel: 5 ``` - TIP: **Tip:** Parallelize tests suites across parallel jobs. Different languages have different tools to facilitate this. +A simple example using [Sempahore Test Boosters](https://github.com/renderedtext/test-boosters) and RSpec to run some Ruby tests: + +```ruby +# Gemfile +source 'https://rubygems.org' + +gem 'rspec' +gem 'semaphore_test_boosters' +``` + +```yaml +test: + parallel: 3 + script: + - bundle + - bundle exec rspec_booster --job $CI_NODE_INDEX/$CI_NODE_TOTAL +``` + +CAUTION: **Caution:** +Please be aware that semaphore_test_boosters reports usages statistics to the author. + +You can then navigate to the **Jobs** tab of a new pipeline build and see your RSpec +job split into three separate jobs. + ### `trigger` **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/8997) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8. @@ -2445,20 +2468,20 @@ There are three possible values: `none`, `normal`, and `recursive`: - `normal` means that only the top-level submodules will be included. It is equivalent to: - ``` - git submodule sync - git submodule update --init - ``` + ``` + git submodule sync + git submodule update --init + ``` - `recursive` means that all submodules (including submodules of submodules) will be included. This feature needs Git v1.8.1 and later. When using a GitLab Runner with an executor not based on Docker, make sure the Git version meets that requirement. It is equivalent to: - ``` - git submodule sync --recursive - git submodule update --init --recursive - ``` + ``` + git submodule sync --recursive + git submodule update --init --recursive + ``` Note that for this feature to work correctly, the submodules must be configured (in `.gitmodules`) with either: diff --git a/doc/development/README.md b/doc/development/README.md index ea5d9e10e2c..44283a3ab0c 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -113,6 +113,10 @@ description: 'Learn how to contribute to GitLab.' - [Database helper modules](database_helpers.md) - [Code comments](code_comments.md) +## Case studies + +- [Database case study: Filtering by label](filtering_by_label.md) + ## Integration guides - [Jira Connect app](integrations/jira_connect.md) @@ -146,6 +150,10 @@ description: 'Learn how to contribute to GitLab.' - [Go Guidelines](go_guide/index.md) +## Shell Scripting guides + +- [Shell scripting standards and style guidelines](shell_scripting_guide/index.md) + ## Other GitLab Development Kit (GDK) guides - [Run full Auto DevOps cycle in a GDK instance](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/auto_devops.md) diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 79c701d7abf..a38794c49af 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -92,9 +92,6 @@ The following team labels are **true** teams per our [organization structure](ht The descriptions on the [labels page](https://gitlab.com/gitlab-org/gitlab-ce/-/labels) explain what falls under the responsibility of each team. -Within those team labels, we also have the ~backend and ~frontend labels to -indicate if an issue needs backend work, frontend work, or both. - Team labels are always capitalized so that they show up as the first label for any issue. @@ -107,15 +104,6 @@ The current stage labels can be found by [searching the labels list for `devops: These labels are [scoped labels](../../user/project/labels.md#scoped-labels-premium) and thus are mutually exclusive. -They differ from the [Team labels](#team-labels) because teams may work on -issues outside their stage. - -Normally there is a 1:1 relationship between Stage labels and Team labels, but -any issue can be picked up by any team, depending on current priorities. -So, an issue labeled ~"devops:create" may be scheduled by the ~Plan team, for -example. In such cases, it's usual to include both team labels so each team can -be aware of the progress. - The Stage labels are used to generate the [direction pages][direction-pages] automatically. [devops-stages]: https://about.gitlab.com/direction/#devops-stages @@ -130,9 +118,16 @@ The current group labels can be found by [searching the labels list for `group:: These labels are [scoped labels](../../user/project/labels.md#scoped-labels-premium) and thus are mutually exclusive. -Groups are nested beneath a particular stage, so only one stage label and one group label -can be applied to a single issue. You can find the groups listed in the -[Product Categories pages][product-categories]. +You can find the groups listed in the [Product Stages, Groups, and Categories][product-categories] page. + +We use the term group to map down product requirements from our product stages. +As a team needs some way to collect the work their members are planning to be assigned to, we use the `~group::` labels to do so. + +Normally there is a 1:1 relationship between Stage labels and Group labels. In the spirit of "Everyone can contribute", +any issue can be picked up by any group, depending on current priorities. For example, an issue labeled ~"devops::create" may be picked up by the ~"group::access" group. + +We also use stage and group labels to help quantify our [throughput](https://about.gitlab.com/handbook/engineering/management/throughput). +Please read [Stage and Group labels in Throughtput](https://about.gitlab.com/handbook/engineering/management/throughput/#stage-and-group-labels-in-throughput) for more information on how the labels are used in this context. [structure-groups]: https://about.gitlab.com/company/team/structure/#groups [product-categories]: https://about.gitlab.com/handbook/product/categories/ @@ -245,7 +240,7 @@ know how difficult the issue is. Additionally: as suitable for people that have never contributed to GitLab before on the [Up For Grabs campaign](http://up-for-grabs.net) - We encourage people that have never contributed to any open source project to - look for [`Accepting merge requests` issues with a weight of 1][firt-timers] + look for [`Accepting merge requests` issues with a weight of 1][first-timers] If you've decided that you would like to work on an issue, please @-mention the [appropriate product manager](https://about.gitlab.com/handbook/product/#who-to-talk-to-for-what) @@ -258,8 +253,8 @@ GitLab team members who apply the ~"Accepting merge requests" label to an issue should update the issue description with a responsible product manager, inviting any potential community contributor to @-mention per above. -[up-for-grabs]: https://gitlab.com/groups/gitlab-org/-/issues?state=opened&label_name[]=Accepting+merge+requests&assignee_id=0&sort=weight -[firt-timers]: https://gitlab.com/groups/gitlab-org/-/issues?state=opened&label_name[]=Accepting+merge+requests&assignee_id=0&sort=weight&weight=1 +[up-for-grabs]: https://gitlab.com/groups/gitlab-org/-/issues?state=opened&label_name[]=Accepting+merge+requests&assignee_id=None&sort=weight +[first-timers]: https://gitlab.com/groups/gitlab-org/-/issues?state=opened&label_name[]=Accepting+merge+requests&assignee_id=None&sort=weight&weight=1 ## Issue triaging diff --git a/doc/development/database_review.md b/doc/development/database_review.md index 1413c2f69fb..3d10a0c84e5 100644 --- a/doc/development/database_review.md +++ b/doc/development/database_review.md @@ -68,6 +68,17 @@ make sure you have applied the ~database label and rerun the `danger-review` CI job, or pick someone from the [`@gl-database` team](https://gitlab.com/groups/gl-database/-/group_members). +### How to prepare for speedy database reviews + +In order to make reviewing easier and therefore faster, please consider preparing a comment +and details for a database reviewer: + +- Provide queries in SQL form rather than ActiveRecord. +- Format any queries with a SQL query formatter, for example with [sqlformat.darold.net](http://sqlformat.darold.net). +- Consider providing query plans via a link to [explain.depesz.com](https://explain.depesz.com) or another tool instead of textual form. +- For query changes, it is best to provide the SQL query along with a plan *before* and *after* the change. This helps to spot differences quickly. +- When providing query plans, make sure to use good parameter values, so that the query executed is a good example and also hits enough data. Usually, the `gitlab-org` namespace (`namespace_id = 9970`) and the `gitlab-org/gitlab-ce` project (`project_id = 13083`) provides enough data to serve as a good example. + ### How to review for database - Check migrations diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index d717b17136e..19b26618882 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -111,18 +111,18 @@ For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to 1. Copy `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md` 1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with: - ```md - This document was moved to [another location](../../administration/lfs.md). - ``` + ```md + This document was moved to [another location](../../administration/lfs.md). + ``` 1. Find and replace any occurrences of the old location with the new one. A quick way to find them is to use `git grep`. First go to the root directory where you cloned the `gitlab-ce` repository and then do: - ```sh - git grep -n "workflow/lfs/lfs_administration" - git grep -n "lfs/lfs_administration" - ``` + ```sh + git grep -n "workflow/lfs/lfs_administration" + git grep -n "lfs/lfs_administration" + ``` NOTE: **Note:** If the document being moved has any Disqus comments on it, there are extra steps @@ -296,45 +296,45 @@ You can combine one or more of the following: 1. **Linking to an anchor link.** Use `anchor` as part of the `help_page_path` method: - ```haml - = link_to 'Help page', help_page_path('user/permissions', anchor: 'anchor-link') - ``` + ```haml + = link_to 'Help page', help_page_path('user/permissions', anchor: 'anchor-link') + ``` 1. **Opening links in a new tab.** This should be the default behavior: - ```haml - = link_to 'Help page', help_page_path('user/permissions'), target: '_blank' - ``` + ```haml + = link_to 'Help page', help_page_path('user/permissions'), target: '_blank' + ``` 1. **Linking to a circle icon.** Usually used in settings where a long description cannot be used, like near checkboxes. You can basically use any font awesome icon, but prefer the `question-circle`: - ```haml - = link_to icon('question-circle'), help_page_path('user/permissions') - ``` + ```haml + = link_to icon('question-circle'), help_page_path('user/permissions') + ``` 1. **Using a button link.** Useful in places where text would be out of context with the rest of the page layout: - ```haml - = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' - ``` + ```haml + = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' + ``` 1. **Using links inline of some text.** - ```haml - Description to #{link_to 'Help page', help_page_path('user/permissions')}. - ``` + ```haml + Description to #{link_to 'Help page', help_page_path('user/permissions')}. + ``` 1. **Adding a period at the end of the sentence.** Useful when you don't want the period to be part of the link: - ```haml - = succeed '.' do - Learn more in the - = link_to 'Help page', help_page_path('user/permissions') - ``` + ```haml + = succeed '.' do + Learn more in the + = link_to 'Help page', help_page_path('user/permissions') + ``` ### GitLab `/help` tests diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md index dd798777c12..e84d65f424e 100644 --- a/doc/development/documentation/styleguide.md +++ b/doc/development/documentation/styleguide.md @@ -36,8 +36,8 @@ For the Troubleshooting sections, people in GitLab Support can merge additions t Include any media types/sources if the content is relevant to readers. You can freely include or link presentations, diagrams, videos, etc.; no matter who it was originally composed for, if it is helpful to any of our audiences, we can include it. - - If you use an image that has a separate source file (for example, a vector or diagram format), link the image to the source file so that it may be reused or updated by anyone. - - Do not copy and paste content from other sources unless it is a limited quotation with the source cited. Typically it is better to either rephrase relevant information in your own words or link out to the other source. +- If you use an image that has a separate source file (for example, a vector or diagram format), link the image to the source file so that it may be reused or updated by anyone. +- Do not copy and paste content from other sources unless it is a limited quotation with the source cited. Typically it is better to either rephrase relevant information in your own words or link out to the other source. ### No special types @@ -237,14 +237,14 @@ Do not include the same information in multiple places. [Link to a SSOT instead. - Use sentence case for titles, headings, labels, menu items, and buttons. - Insert an empty line between different markups (e.g., after every paragraph, header, list, etc). Example: - ```md - ## Header + ```md + ## Header - Paragraph. + Paragraph. - - List item 1 - - List item 2 - ``` + - List item 1 + - List item 2 + ``` ### Tables overlapping the TOC @@ -303,12 +303,12 @@ Check specific punctuation rules for [list items](#list-items) below. - Be consistent throughout the list: if the majority of the items do not end in a period, do not end any of the items in a period, even if they consist of a complete sentence. The opposite is also valid: if the majority of the items end with a period, end all with a period. - Separate list items from explanatory text with a colon (`:`). For example: - ```md - The list is as follows: + ```md + The list is as follows: - - First item: this explains the first item. - - Second item: this explains the second item. - ``` + - First item: this explains the first item. + - Second item: this explains the second item. + ``` **Examples:** @@ -520,16 +520,16 @@ To embed a video, follow the instructions below and make sure you have your MR reviewed and approved by a technical writer. 1. Copy the code below and paste it into your markdown file. - Leave a blank line above and below it. Do NOT edit the code - (don't remove or add any spaces, etc). + Leave a blank line above and below it. Do NOT edit the code + (don't remove or add any spaces, etc). 1. On YouTube, visit the video URL you want to display. Copy - the regular URL from your browser (`https://www.youtube.com/watch?v=VIDEO-ID`) - and replace the video title and link in the line under `<div class="video-fallback">`. + the regular URL from your browser (`https://www.youtube.com/watch?v=VIDEO-ID`) + and replace the video title and link in the line under `<div class="video-fallback">`. 1. On YouTube, click **Share**, then **Embed**. 1. Copy the `<iframe>` source (`src`) **URL only** - (`https://www.youtube.com/embed/VIDEO-ID`), - and paste it, replacing the content of the `src` field in the - `iframe` tag. + (`https://www.youtube.com/embed/VIDEO-ID`), + and paste it, replacing the content of the `src` field in the + `iframe` tag. ```html leave a blank line here @@ -611,7 +611,7 @@ In most cases, content considered for a note should be included: #### When to use Use a note when there is a reason that most or all readers who browse the -section should see the content. That is, if missed, it’s likely to cause +section should see the content. That is, if missed, it’s likely to cause major trouble for a minority of users or significant trouble for a majority of users. @@ -747,24 +747,24 @@ a helpful link back to how the feature was developed. - For features that need to declare the GitLab version that the feature was introduced. Text similar to the following should be added immediately below the heading as a blockquote: - ```md - > Introduced in GitLab 11.3. - ``` + ```md + > Introduced in GitLab 11.3. + ``` - Whenever possible, version text should have a link to the issue, merge request, or epic that introduced the feature. An issue is preferred over a merge request, and a merge request is preferred over an epic. For example: - ```md - > [Introduced](<link-to-issue>) in GitLab 11.3. - ``` + ```md + > [Introduced](<link-to-issue>) in GitLab 11.3. + ``` - If the feature is only available in GitLab Enterprise Edition, mention the [paid tier](https://about.gitlab.com/handbook/marketing/product-marketing/#tiers) the feature is available in: - ```md - > [Introduced](<link-to-issue>) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.3. - ``` + ```md + > [Introduced](<link-to-issue>) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.3. + ``` ### Removing version text @@ -871,14 +871,14 @@ When there is a list of steps to perform, usually that entails editing the configuration file and reconfiguring/restarting GitLab. In such case, follow the style below as a guide: -```md +````md **For Omnibus installations** 1. Edit `/etc/gitlab/gitlab.rb`: - ```ruby - external_url "https://gitlab.example.com" - ``` + ```ruby + external_url "https://gitlab.example.com" + ``` 1. Save the file and [reconfigure] GitLab for the changes to take effect. @@ -888,17 +888,16 @@ the style below as a guide: 1. Edit `config/gitlab.yml`: - ```yaml - gitlab: - host: "gitlab.example.com" - ``` + ```yaml + gitlab: + host: "gitlab.example.com" + ``` 1. Save the file and [restart] GitLab for the changes to take effect. - [reconfigure]: path/to/administration/restart_gitlab.md#omnibus-gitlab-reconfigure [restart]: path/to/administration/restart_gitlab.md#installations-from-source -``` +```` In this case: @@ -917,9 +916,9 @@ on this document. Further explanation is given below. - Every method must have the REST API request. For example: - ``` - GET /projects/:id/repository/branches - ``` + ``` + GET /projects/:id/repository/branches + ``` - Every method must have a detailed [description of the parameters](#method-description). @@ -971,7 +970,7 @@ You can use the following fake tokens as examples. | Token type | Token value | |:----------------------|:-------------------------------------------------------------------| -| Private user token | `<your_access_token>` | +| Private user token | `<your_access_token>` | | Personal access token | `n671WNGecHugsdEDPsyo` | | Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` | | Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` | diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index 7131b717353..2217dedccd3 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -125,20 +125,24 @@ This also applies to views. ### EE features based on CE features For features that build on existing CE features, write a module in the `EE` -namespace and `prepend` it in the CE class, on the last line of the file that -the class resides in. This makes conflicts less likely to happen during CE to EE -merges because only one line is added to the CE class - the `prepend` line. For -example, to prepend a module into the `User` class you would use the following -approach: +namespace and inject it in the CE class, on the last line of the file that the +class resides in. This makes conflicts less likely to happen during CE to EE +merges because only one line is added to the CE class - the line that injects +the module. For example, to prepend a module into the `User` class you would use +the following approach: ```ruby class User < ActiveRecord::Base # ... lots of code here ... end -User.prepend(EE::User) +User.prepend_if_ee('EE::User') ``` +Do not use methods such as `prepend`, `extend`, and `include`. Instead, use +`prepend_if_ee`, `extend_if_ee`, or `include_if_ee`. These methods take a +_String_ containing the full module name as the argument, not the module itself. + Since the module would require an `EE` namespace, the file should also be put in an `ee/` sub-directory. For example, we want to extend the user model in EE, so we have a module called `::EE::User` put inside @@ -255,7 +259,7 @@ class ApplicationController < ActionController::Base # ... end -ApplicationController.prepend(EE::ApplicationController) +ApplicationController.prepend_if_ee('EE::ApplicationController') ``` And create a new file in the `ee/` sub-directory with the altered @@ -504,9 +508,9 @@ EE-specific LDAP classes in `ee/lib/ee/gitlab/ldap`. ### Code in `lib/api/` -It can be very tricky to extend EE features by a single line of `prepend`, -and for each different [Grape](https://github.com/ruby-grape/grape) feature, -we might need different strategies to extend it. To apply different strategies +It can be very tricky to extend EE features by a single line of `prepend_if_ee`, +and for each different [Grape](https://github.com/ruby-grape/grape) feature, we +might need different strategies to extend it. To apply different strategies easily, we would use `extend ActiveSupport::Concern` in the EE module. Put the EE module files following @@ -543,12 +547,12 @@ constants. We can define `params` and utilize `use` in another `params` definition to include params defined in EE. However, we need to define the "interface" first in CE in order for EE to override it. We don't have to do this in other places -due to `prepend`, but Grape is complex internally and we couldn't easily do -that, so we'll follow regular object-oriented practices that we define the +due to `prepend_if_ee`, but Grape is complex internally and we couldn't easily +do that, so we'll follow regular object-oriented practices that we define the interface first here. For example, suppose we have a few more optional params for EE. We can move the -params out of the `Grape::API` class to a helper module, so we can `prepend` it +params out of the `Grape::API` class to a helper module, so we can inject it before it would be used in the class. ```ruby @@ -583,7 +587,7 @@ module API end end -API::Helpers::ProjectsHelpers.prepend(EE::API::Helpers::ProjectsHelpers) +API::Helpers::ProjectsHelpers.prepend_if_ee('EE::API::Helpers::ProjectsHelpers') ``` We could override it in EE module: @@ -624,7 +628,7 @@ module API end end -API::JobArtifacts.prepend(EE::API::JobArtifacts) +API::JobArtifacts.prepend_if_ee('EE::API::JobArtifacts') ``` And then we can follow regular object-oriented practices to override it: @@ -677,7 +681,7 @@ module API end end -API::MergeRequests.prepend(EE::API::MergeRequests) +API::MergeRequests.prepend_if_ee('EE::API::MergeRequests') ``` Note that `update_merge_request_ee` doesn't do anything in CE, but @@ -717,8 +721,8 @@ Sometimes we need to use different arguments for a particular API route, and we can't easily extend it with an EE module because Grape has different context in different blocks. In order to overcome this, we need to move the data to a class method that resides in a separate module or class. This allows us to extend that -module or class before its data is used, without having to place a `prepend` in -the middle of CE code. +module or class before its data is used, without having to place a +`prepend_if_ee` in the middle of CE code. For example, in one place we need to pass an extra argument to `at_least_one_of` so that the API could consider an EE-only argument as the @@ -739,7 +743,7 @@ module API end end -API::MergeRequests::Parameters.prepend(EE::API::MergeRequests::Parameters) +API::MergeRequests::Parameters.prepend_if_ee('EE::API::MergeRequests::Parameters') # api/merge_requests.rb module API @@ -789,7 +793,7 @@ class Identity < ActiveRecord::Base [:provider] end - prepend EE::Identity + prepend_if_ee('EE::Identity') validates :extern_uid, allow_blank: true, @@ -841,7 +845,7 @@ class Identity < ActiveRecord::Base end end -Identity::UniquenessScopes.prepend(EE::Identity::UniquenessScopes) +Identity::UniquenessScopes.prepend_if_ee('EE::Identity::UniquenessScopes') # app/models/identity.rb class Identity < ActiveRecord::Base diff --git a/doc/development/fe_guide/components.md b/doc/development/fe_guide/components.md index 52462a4bec9..096ce8ca25a 100644 --- a/doc/development/fe_guide/components.md +++ b/doc/development/fe_guide/components.md @@ -14,28 +14,29 @@ See also the [corresponding UX guide](https://design.gitlab.com/#/components/dro 1. Use the HTML structure provided by the [docs][bootstrap-dropdowns] 1. Add a specific class to the top level `.dropdown` element - ```Haml - .dropdown.my-dropdown - %button{ type: 'button', data: { toggle: 'dropdown' }, 'aria-haspopup': true, 'aria-expanded': false } - %span.dropdown-toggle-text - Toggle Dropdown - = icon('chevron-down') - - %ul.dropdown-menu - %li - %a - item! - ``` - - Or use the helpers - ```Haml - .dropdown.my-dropdown - = dropdown_toggle('Toogle!', { toggle: 'dropdown' }) - = dropdown_content - %li - %a - item! - ``` + ```Haml + .dropdown.my-dropdown + %button{ type: 'button', data: { toggle: 'dropdown' }, 'aria-haspopup': true, 'aria-expanded': false } + %span.dropdown-toggle-text + Toggle Dropdown + = icon('chevron-down') + + %ul.dropdown-menu + %li + %a + item! + ``` + + Or use the helpers + + ```Haml + .dropdown.my-dropdown + = dropdown_toggle('Toogle!', { toggle: 'dropdown' }) + = dropdown_content + %li + %a + item! + ``` [bootstrap-dropdowns]: https://getbootstrap.com/docs/3.3/javascript/#dropdowns diff --git a/doc/development/fe_guide/event_tracking.md b/doc/development/fe_guide/event_tracking.md index 716f6ad7f92..1e6287d8f6d 100644 --- a/doc/development/fe_guide/event_tracking.md +++ b/doc/development/fe_guide/event_tracking.md @@ -1,79 +1,76 @@ # Event Tracking -We use [Snowplow](https://github.com/snowplow/snowplow) for tracking custom events (available in GitLab [Enterprise Edition](https://about.gitlab.com/pricing/) only). +We use a tracking interface that wraps up [Snowplow](https://github.com/snowplow/snowplow) for tracking custom events. Snowplow implements page tracking, but also exposes custom event tracking. -## Generic tracking function - -In addition to Snowplow's built-in method for tracking page views, we use a generic tracking function which enables us to selectively apply listeners to events. - -The generic tracking function can be imported in EE-specific JS files as follows: +The tracking interface can be imported in JS files as follows: ```javascript -import { trackEvent } from `ee/stats`; +import Tracking from `~/tracking`; ``` -This gives the user access to the `trackEvent` method, which takes the following parameters: +## Tracking in HAML or Vue templates -| parameter | type | description | required | -| ---------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -| `category` | string | Describes the page that you're capturing click events on. Unless infeasible, please use the Rails page attribute `document.body.dataset.page` by default. | true | -| `eventName` | string | Describes the action the user is taking. The first word should always describe the action. For example, clicks should be `click` and activations should be `activate`. Use underscores to describe what was acted on. For example, activating a form field would be `activate_form_input`. Clicking on a dropdown is `click_dropdown`. | true | -| `additionalData` | object | Additional data such as `label`, `property`, and `value` as described [in our Feature Instrumentation taxonomy](https://about.gitlab.com/handbook/product/feature-instrumentation/#taxonomy). | false | +To avoid having to do create a bunch of custom javascript event handlers, when working within HAML or Vue templates, we can add `data-track-*` attributes to elements of interest. This way, all elements that have a `data-track-event` attribute to automatically have event tracking bound. -Read more about instrumentation and the taxonomy in the [Product Handbook](https://about.gitlab.com/handbook/product/feature-instrumentation). - -### Tracking in `.js` and `.vue` files +Below is an example of `data-track-*` attributes assigned to a button in HAML: -The most simple use case is to add tracking programmatically to an event of interest in Javascript. +```haml +%button.btn{ data: { track_event: "click_button", track_label: "template_preview", track_property: "my-template", track_value: "" } } +``` -The following example demonstrates how to track a click on a button in Javascript by calling the `trackEvent` method explicitly: +We can then setup tracking for large sections of a page, or an entire page by telling the Tracking interface to bind to it. ```javascript -import { trackEvent } from `ee/stats`; +import Tracking from '~/tracking'; -trackEvent('dashboard:projects:index', 'click_button', { - label: 'create_from_template', - property: 'template_preview', - value: 'rails', +// for the entire document +new Tracking().bind(); + +// for a container element +document.addEventListener('DOMContentLoaded', () => { + new Tracking('my_category').bind(document.getElementById('my-container')); }); + ``` -### Tracking in HAML templates +When you instantiate a Tracking instance you can provide a category. If none is provided, `document.body.dataset.page` will be used. When you bind the Tracking instance you can provide an element. If no element is provided to bind to, the `document` is assumed. -Sometimes we want to track clicks for multiple elements on a page. Creating event handlers for all elements could soon turn into a tedious task. +Below is a list of supported `data-track-*` attributes: -There's a more convenient solution to this problem. When working with HAML templates, we can add `data-track-*` attributes to elements of interest. This way, all elements that have both `data-track-label` and `data-track-event` attributes assigned get marked for event tracking. All we have to do is call the `bindTrackableContainer` method on a container which allows for better scoping. +| attribute | required | description | +|:----------------------|:---------|:------------| +| `data-track-event` | true | Action the user is taking. Clicks should be `click` and activations should be `activate`, so for example, focusing a form field would be `activate_form_input`, and clicking a button would be `click_button`. | +| `data-track-label` | false | The `label` as described [in our Feature Instrumentation taxonomy](https://about.gitlab.com/handbook/product/feature-instrumentation/#taxonomy) | +| `data-track-property` | false | The `property` as described [in our Feature Instrumentation taxonomy](https://about.gitlab.com/handbook/product/feature-instrumentation/#taxonomy) +| `data-track-value` | false | The `value` as described [in our Feature Instrumentation taxonomy](https://about.gitlab.com/handbook/product/feature-instrumentation/#taxonomy). If omitted, this will be the elements `value` property or an empty string. For checkboxes, the default value will be the element's checked attribute or `false` when unchecked. -Below is an example of `data-track-*` attributes assigned to a button in HAML: -```ruby -%button.btn{ data: { track_label: "template_preview", track_property: "my-template", track_event: "click_button", track_value: "" } } -``` - -By calling `bindTrackableContainer('.my-container')`, click handlers get bound to all elements located in `.my-container` provided that they have the necessary `data-track-*` attributes assigned to them. +## Tracking in raw Javascript -```javascript -import Stats from 'ee/stats'; +Custom events can be tracked by directly calling the `Tracking.event` static function, which accepts the following arguments: -document.addEventListener('DOMContentLoaded', () => { - Stats.bindTrackableContainer('.my-container', 'category'); -}); -``` +| argument | type | default value | description | +|:-----------|:-------|:---------------------------|:------------| +| `category` | string | document.body.dataset.page | Page or subsection of a page that events are being captured within. | +| `event` | string | 'generic' | Action the user is taking. Clicks should be `click` and activations should be `activate`, so for example, focusing a form field would be `activate_form_input`, and clicking a button would be `click_button`. | +| `data` | object | {} | Additional data such as `label`, `property`, and `value` as described [in our Feature Instrumentation taxonomy](https://about.gitlab.com/handbook/product/feature-instrumentation/#taxonomy). These will be set as empty strings if you don't provide them. | -The second parameter in `bindTrackableContainer` is optional. If omitted, the value of `document.body.dataset.page` will be used as category instead. +Tracking can be programmatically added to an event of interest in Javascript, and the following example demonstrates tracking a click on a button by calling `Tracking.event` manually. -Below is a list of supported `data-track-*` attributes: +```javascript +import Tracking from `~/tracking`; -| attribute | description | required | -| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -| `data-track-label` | The `label` in `trackEvent` | true | -| `data-track-event` | The `eventName` in `trackEvent` | true | -| `data-track-property` | The `property` in `trackEvent`. If omitted, an empty string will be used as a default value. | false | -| `data-track-value` | The `value` in `trackEvent`. If omitted, this will be `target.value` or empty string. For checkboxes, the default value being tracked will be the element's checked attribute if `data-track-value` is omitted. | false | +document.getElementById('my_button').addEventListener('click', () => { + Tracking.event('dashboard:projects:index', 'click_button', { + label: 'create_from_template', + property: 'template_preview', + value: 'rails', + }); +}) +``` -Since Snowplow is an Enterprise Edition feature, it's necessary to create a CE backport when adding `data-track-*` attributes to HAML templates in most cases. -## Testing +## Toggling tracking on or off Snowplow can be enabled by navigating to: diff --git a/doc/development/fe_guide/performance.md b/doc/development/fe_guide/performance.md index 2628e95dbc1..676bce32998 100644 --- a/doc/development/fe_guide/performance.md +++ b/doc/development/fe_guide/performance.md @@ -30,8 +30,8 @@ To improve the time to first render we are using lazy loading for images. This w the actual image source on the `data-src` attribute. After the HTML is rendered and JavaScript is loaded, the value of `data-src` will be moved to `src` automatically if the image is in the current viewport. -- Prepare images in HTML for lazy loading by renaming the `src` attribute to `data-src` AND adding the class `lazy`. -- If you are using the Rails `image_tag` helper, all images will be lazy-loaded by default unless `lazy: false` is provided. +- Prepare images in HTML for lazy loading by renaming the `src` attribute to `data-src` AND adding the class `lazy`. +- If you are using the Rails `image_tag` helper, all images will be lazy-loaded by default unless `lazy: false` is provided. If you are asynchronously adding content which contains lazy images then you need to call the function `gl.lazyLoader.searchLazyImages()` which will search for lazy images and load them if needed. @@ -96,26 +96,26 @@ bundle and included on the page. DOM has loaded, you should attach an event handler to the `DOMContentLoaded` event with: - ```javascript - import initMyWidget from './my_widget'; + ```javascript + import initMyWidget from './my_widget'; - document.addEventListener('DOMContentLoaded', () => { - initMyWidget(); - }); - ``` + document.addEventListener('DOMContentLoaded', () => { + initMyWidget(); + }); + ``` - **Supporting Module Placement:** - - If a class or a module is _specific to a particular route_, try to locate - it close to the entry point it will be used. For instance, if - `my_widget.js` is only imported within `pages/widget/show/index.js`, you - should place the module at `pages/widget/show/my_widget.js` and import it - with a relative path (e.g. `import initMyWidget from './my_widget';`). - - If a class or module is _used by multiple routes_, place it within a - shared directory at the closest common parent directory for the entry - points that import it. For example, if `my_widget.js` is imported within - both `pages/widget/show/index.js` and `pages/widget/run/index.js`, then - place the module at `pages/widget/shared/my_widget.js` and import it with - a relative path if possible (e.g. `../shared/my_widget`). + - If a class or a module is _specific to a particular route_, try to locate + it close to the entry point it will be used. For instance, if + `my_widget.js` is only imported within `pages/widget/show/index.js`, you + should place the module at `pages/widget/show/my_widget.js` and import it + with a relative path (e.g. `import initMyWidget from './my_widget';`). + - If a class or module is _used by multiple routes_, place it within a + shared directory at the closest common parent directory for the entry + points that import it. For example, if `my_widget.js` is imported within + both `pages/widget/show/index.js` and `pages/widget/run/index.js`, then + place the module at `pages/widget/shared/my_widget.js` and import it with + a relative path if possible (e.g. `../shared/my_widget`). - **Enterprise Edition Caveats:** For GitLab Enterprise Edition, page-specific entry points will override their @@ -161,7 +161,7 @@ General tips: - Use code-splitting dynamic imports wherever possible to lazy-load code that is not needed initially. - [High Performance Animations][high-perf-animations] -------- +--- ## Additional Resources diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 18ef754642d..b0bbb4cc4b2 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -20,33 +20,33 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ 1. **Never Ever EVER** disable eslint globally for a file - ```javascript - // bad - /* eslint-disable */ + ```javascript + // bad + /* eslint-disable */ - // better - /* eslint-disable some-rule, some-other-rule */ + // better + /* eslint-disable some-rule, some-other-rule */ - // best - // nothing :) - ``` + // best + // nothing :) + ``` 1. If you do need to disable a rule for a single violation, try to do it as locally as possible - ```javascript - // bad - /* eslint-disable no-new */ + ```javascript + // bad + /* eslint-disable no-new */ - import Foo from 'foo'; + import Foo from 'foo'; - new Foo(); + new Foo(); - // better - import Foo from 'foo'; + // better + import Foo from 'foo'; - // eslint-disable-next-line no-new - new Foo(); - ``` + // eslint-disable-next-line no-new + new Foo(); + ``` 1. There are few rules that we need to disable due to technical debt. Which are: 1. [no-new][eslint-new] @@ -55,113 +55,113 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ 1. When they are needed _always_ place ESlint directive comment blocks on the first line of a script, followed by any global declarations, then a blank newline prior to any imports or code. - ```javascript - // bad - /* global Foo */ - /* eslint-disable no-new */ - import Bar from './bar'; + ```javascript + // bad + /* global Foo */ + /* eslint-disable no-new */ + import Bar from './bar'; - // good - /* eslint-disable no-new */ - /* global Foo */ + // good + /* eslint-disable no-new */ + /* global Foo */ - import Bar from './bar'; - ``` + import Bar from './bar'; + ``` 1. **Never** disable the `no-undef` rule. Declare globals with `/* global Foo */` instead. 1. When declaring multiple globals, always use one `/* global [name] */` line per variable. - ```javascript - // bad - /* globals Flash, Cookies, jQuery */ + ```javascript + // bad + /* globals Flash, Cookies, jQuery */ - // good - /* global Flash */ - /* global Cookies */ - /* global jQuery */ - ``` + // good + /* global Flash */ + /* global Cookies */ + /* global jQuery */ + ``` 1. Use up to 3 parameters for a function or class. If you need more accept an Object instead. - ```javascript - // bad - fn(p1, p2, p3, p4) {} + ```javascript + // bad + fn(p1, p2, p3, p4) {} - // good - fn(options) {} - ``` + // good + fn(options) {} + ``` #### Modules, Imports, and Exports 1. Use ES module syntax to import modules - ```javascript - // bad - const SomeClass = require('some_class'); + ```javascript + // bad + const SomeClass = require('some_class'); - // good - import SomeClass from 'some_class'; + // good + import SomeClass from 'some_class'; - // bad - module.exports = SomeClass; + // bad + module.exports = SomeClass; - // good - export default SomeClass; - ``` + // good + export default SomeClass; + ``` - Import statements are following usual naming guidelines, for example object literals use camel case: + Import statements are following usual naming guidelines, for example object literals use camel case: - ```javascript - // some_object file - export default { - key: 'value', - }; + ```javascript + // some_object file + export default { + key: 'value', + }; - // bad - import ObjectLiteral from 'some_object'; + // bad + import ObjectLiteral from 'some_object'; - // good - import objectLiteral from 'some_object'; - ``` + // good + import objectLiteral from 'some_object'; + ``` 1. Relative paths: when importing a module in the same directory, a child directory, or an immediate parent directory prefer relative paths. When importing a module which is two or more levels up, prefer either `~/` or `ee/`. - In **app/assets/javascripts/my-feature/subdir**: + In **app/assets/javascripts/my-feature/subdir**: - ```javascript - // bad - import Foo from '~/my-feature/foo'; - import Bar from '~/my-feature/subdir/bar'; - import Bin from '~/my-feature/subdir/lib/bin'; + ```javascript + // bad + import Foo from '~/my-feature/foo'; + import Bar from '~/my-feature/subdir/bar'; + import Bin from '~/my-feature/subdir/lib/bin'; - // good - import Foo from '../foo'; - import Bar from './bar'; - import Bin from './lib/bin'; - ``` + // good + import Foo from '../foo'; + import Bar from './bar'; + import Bin from './lib/bin'; + ``` - In **spec/javascripts**: + In **spec/javascripts**: - ```javascript - // bad - import Foo from '../../app/assets/javascripts/my-feature/foo'; + ```javascript + // bad + import Foo from '../../app/assets/javascripts/my-feature/foo'; - // good - import Foo from '~/my-feature/foo'; - ``` + // good + import Foo from '~/my-feature/foo'; + ``` - When referencing an **EE component**: + When referencing an **EE component**: - ```javascript - // bad - import Foo from '../../../../../ee/app/assets/javascripts/my-feature/ee-foo'; + ```javascript + // bad + import Foo from '../../../../../ee/app/assets/javascripts/my-feature/ee-foo'; - // good - import Foo from 'ee/my-feature/foo'; - ``` + // good + import Foo from 'ee/my-feature/foo'; + ``` 1. Avoid using IIFE. Although we have a lot of examples of files which wrap their contents in IIFEs (immediately-invoked function expressions), @@ -170,136 +170,136 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ 1. Avoid adding to the global namespace. - ```javascript - // bad - window.MyClass = class { /* ... */ }; + ```javascript + // bad + window.MyClass = class { /* ... */ }; - // good - export default class MyClass { /* ... */ } - ``` + // good + export default class MyClass { /* ... */ } + ``` 1. Side effects are forbidden in any script which contains export - ```javascript - // bad - export default class MyClass { /* ... */ } + ```javascript + // bad + export default class MyClass { /* ... */ } - document.addEventListener("DOMContentLoaded", function(event) { - new MyClass(); - } - ``` + document.addEventListener("DOMContentLoaded", function(event) { + new MyClass(); + } + ``` #### Data Mutation and Pure functions 1. Strive to write many small pure functions, and minimize where mutations occur. - ```javascript - // bad - const values = {foo: 1}; + ```javascript + // bad + const values = {foo: 1}; - function impureFunction(items) { - const bar = 1; + function impureFunction(items) { + const bar = 1; - items.foo = items.a * bar + 2; + items.foo = items.a * bar + 2; - return items.a; - } + return items.a; + } - const c = impureFunction(values); + const c = impureFunction(values); - // good - var values = {foo: 1}; + // good + var values = {foo: 1}; - function pureFunction (foo) { - var bar = 1; + function pureFunction (foo) { + var bar = 1; - foo = foo * bar + 2; + foo = foo * bar + 2; - return foo; - } + return foo; + } - var c = pureFunction(values.foo); + var c = pureFunction(values.foo); ``` 1. Avoid constructors with side-effects. Although we aim for code without side-effects we need some side-effects for our code to run. - If the class won't do anything if we only instantiate it, it's ok to add side effects into the constructor (_Note:_ The following is just an example. If the only purpose of the class is to add an event listener and handle the callback a function will be more suitable.) - - ```javascript - // Bad - export class Foo { - constructor() { - this.init(); - } - init() { - document.addEventListener('click', this.handleCallback) - }, - handleCallback() { - - } - } - - // Good - export class Foo { - constructor() { - document.addEventListener() - } - handleCallback() { - } - } - ``` - - On the other hand, if a class only needs to extend a third party/add event listeners in some specific cases, they should be initialized outside of the constructor. + If the class won't do anything if we only instantiate it, it's ok to add side effects into the constructor (_Note:_ The following is just an example. If the only purpose of the class is to add an event listener and handle the callback a function will be more suitable.) + + ```javascript + // Bad + export class Foo { + constructor() { + this.init(); + } + init() { + document.addEventListener('click', this.handleCallback) + }, + handleCallback() { + + } + } + + // Good + export class Foo { + constructor() { + document.addEventListener() + } + handleCallback() { + } + } + ``` + + On the other hand, if a class only needs to extend a third party/add event listeners in some specific cases, they should be initialized outside of the constructor. 1. Prefer `.map`, `.reduce` or `.filter` over `.forEach` A forEach will most likely cause side effects, it will be mutating the array being iterated. Prefer using `.map`, `.reduce` or `.filter` - ```javascript - const users = [ { name: 'Foo' }, { name: 'Bar' } ]; + ```javascript + const users = [ { name: 'Foo' }, { name: 'Bar' } ]; - // bad - users.forEach((user, index) => { - user.id = index; - }); + // bad + users.forEach((user, index) => { + user.id = index; + }); - // good - const usersWithId = users.map((user, index) => { - return Object.assign({}, user, { id: index }); - }); - ``` + // good + const usersWithId = users.map((user, index) => { + return Object.assign({}, user, { id: index }); + }); + ``` #### Parse Strings into Numbers 1. `parseInt()` is preferable over `Number()` or `+` - ```javascript - // bad - +'10' // 10 + ```javascript + // bad + +'10' // 10 - // good - Number('10') // 10 + // good + Number('10') // 10 - // better - parseInt('10', 10); - ``` + // better + parseInt('10', 10); + ``` #### CSS classes used for JavaScript 1. If the class is being used in Javascript it needs to be prepend with `js-` - ```html - // bad - <button class="add-user"> - Add User - </button> + ```html + // bad + <button class="add-user"> + Add User + </button> - // good - <button class="js-add-user"> - Add User - </button> - ``` + // good + <button class="js-add-user"> + Add User + </button> + ``` ### Vue.js @@ -314,43 +314,44 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. 1. The store has it's own file 1. Use a function in the bundle file to instantiate the Vue component: - ```javascript - // bad - class { - init() { - new Component({}) - } - } - - // good - document.addEventListener('DOMContentLoaded', () => new Vue({ - el: '#element', - components: { - componentName - }, - render: createElement => createElement('component-name'), - })); - ``` + ```javascript + // bad + class { + init() { + new Component({}) + } + } + + // good + document.addEventListener('DOMContentLoaded', () => new Vue({ + el: '#element', + components: { + componentName + }, + render: createElement => createElement('component-name'), + })); + ``` 1. Do not use a singleton for the service or the store - ```javascript - // bad - class Store { - constructor() { - if (!this.prototype.singleton) { - // do something - } - } - } - - // good - class Store { - constructor() { - // do something - } - } - ``` + ```javascript + // bad + class Store { + constructor() { + if (!this.prototype.singleton) { + // do something + } + } + } + + // good + class Store { + constructor() { + // do something + } + } + ``` + 1. Use `.vue` for Vue templates. Do not use `%template` in HAML. #### Naming @@ -358,38 +359,38 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. 1. **Extensions**: Use `.vue` extension for Vue components. Do not use `.js` as file extension ([#34371]). 1. **Reference Naming**: Use PascalCase for their instances: - ```javascript - // bad - import cardBoard from 'cardBoard.vue' + ```javascript + // bad + import cardBoard from 'cardBoard.vue' - components: { - cardBoard, - }; + components: { + cardBoard, + }; - // good - import CardBoard from 'cardBoard.vue' + // good + import CardBoard from 'cardBoard.vue' - components: { - CardBoard, - }; - ``` + components: { + CardBoard, + }; + ``` 1. **Props Naming:** Avoid using DOM component prop names. 1. **Props Naming:** Use kebab-case instead of camelCase to provide props in templates. - ```javascript - // bad - <component class="btn"> + ```javascript + // bad + <component class="btn"> - // good - <component css-class="btn"> + // good + <component css-class="btn"> - // bad - <component myProp="prop" /> + // bad + <component myProp="prop" /> - // good - <component my-prop="prop" /> - ``` + // good + <component my-prop="prop" /> + ``` [#34371]: https://gitlab.com/gitlab-org/gitlab-ce/issues/34371 @@ -399,205 +400,205 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. 1. With more than one attribute, all attributes should be on a new line: - ```javascript - // bad - <component v-if="bar" - param="baz" /> + ```javascript + // bad + <component v-if="bar" + param="baz" /> - <button class="btn">Click me</button> + <button class="btn">Click me</button> - // good - <component - v-if="bar" - param="baz" - /> + // good + <component + v-if="bar" + param="baz" + /> - <button class="btn"> - Click me - </button> - ``` + <button class="btn"> + Click me + </button> + ``` 1. The tag can be inline if there is only one attribute: - ```javascript - // good - <component bar="bar" /> + ```javascript + // good + <component bar="bar" /> - // good - <component - bar="bar" - /> + // good + <component + bar="bar" + /> - // bad - <component - bar="bar" /> - ``` + // bad + <component + bar="bar" /> + ``` #### Quotes 1. Always use double quotes `"` inside templates and single quotes `'` for all other JS. - ```javascript - // bad - template: ` - <button :class='style'>Button</button> - ` + ```javascript + // bad + template: ` + <button :class='style'>Button</button> + ` - // good - template: ` - <button :class="style">Button</button> - ` - ``` + // good + template: ` + <button :class="style">Button</button> + ` + ``` #### Props 1. Props should be declared as an object - ```javascript - // bad - props: ['foo'] - - // good - props: { - foo: { - type: String, - required: false, - default: 'bar' - } - } - ``` + ```javascript + // bad + props: ['foo'] + + // good + props: { + foo: { + type: String, + required: false, + default: 'bar' + } + } + ``` 1. Required key should always be provided when declaring a prop - ```javascript - // bad - props: { - foo: { - type: String, - } - } - - // good - props: { - foo: { - type: String, - required: false, - default: 'bar' - } - } - ``` + ```javascript + // bad + props: { + foo: { + type: String, + } + } + + // good + props: { + foo: { + type: String, + required: false, + default: 'bar' + } + } + ``` 1. Default key should be provided if the prop is not required. _Note:_ There are some scenarios where we need to check for the existence of the property. On those a default key should not be provided. - ```javascript - // good - props: { - foo: { - type: String, - required: false, - } - } - - // good - props: { - foo: { - type: String, - required: false, - default: 'bar' - } - } - - // good - props: { - foo: { - type: String, - required: true - } - } - ``` + ```javascript + // good + props: { + foo: { + type: String, + required: false, + } + } + + // good + props: { + foo: { + type: String, + required: false, + default: 'bar' + } + } + + // good + props: { + foo: { + type: String, + required: true + } + } + ``` #### Data 1. `data` method should always be a function - ```javascript - // bad - data: { - foo: 'foo' - } - - // good - data() { - return { - foo: 'foo' - }; - } - ``` + ```javascript + // bad + data: { + foo: 'foo' + } + + // good + data() { + return { + foo: 'foo' + }; + } + ``` #### Directives 1. Shorthand `@` is preferable over `v-on` - ```javascript - // bad - <component v-on:click="eventHandler"/> + ```javascript + // bad + <component v-on:click="eventHandler"/> - // good - <component @click="eventHandler"/> - ``` + // good + <component @click="eventHandler"/> + ``` 1. Shorthand `:` is preferable over `v-bind` - ```javascript - // bad - <component v-bind:class="btn"/> + ```javascript + // bad + <component v-bind:class="btn"/> - // good - <component :class="btn"/> - ``` + // good + <component :class="btn"/> + ``` 1. Shorthand `#` is preferable over `v-slot` - ```javascript - // bad - <template v-slot:header></template> + ```javascript + // bad + <template v-slot:header></template> - // good - <template #header></template> - ``` + // good + <template #header></template> + ``` #### Closing tags 1. Prefer self closing component tags - ```javascript - // bad - <component></component> + ```javascript + // bad + <component></component> - // good - <component /> - ``` + // good + <component /> + ``` #### Ordering 1. Tag order in `.vue` file - ``` - <script> - // ... - </script> - - <template> - // ... - </template> - - // We don't use scoped styles but there are few instances of this - <style> - // ... - </style> - ``` + ``` + <script> + // ... + </script> + + <template> + // ... + </template> + + // We don't use scoped styles but there are few instances of this + <style> + // ... + </style> + ``` 1. Properties in a Vue Component: Check [order of properties in components rule][vue-order]. @@ -608,50 +609,50 @@ When using `v-for` you need to provide a *unique* `:key` attribute for each item 1. If the elements of the array being iterated have an unique `id` it is advised to use it: - ```html - <div - v-for="item in items" - :key="item.id" - > - <!-- content --> - </div> - ``` + ```html + <div + v-for="item in items" + :key="item.id" + > + <!-- content --> + </div> + ``` 1. When the elements being iterated don't have a unique id, you can use the array index as the `:key` attribute - ```html - <div - v-for="(item, index) in items" - :key="index" - > - <!-- content --> - </div> - ``` + ```html + <div + v-for="(item, index) in items" + :key="index" + > + <!-- content --> + </div> + ``` 1. When using `v-for` with `template` and there is more than one child element, the `:key` values must be unique. It's advised to use `kebab-case` namespaces. - ```html - <template v-for="(item, index) in items"> - <span :key="`span-${index}`"></span> - <button :key="`button-${index}`"></button> - </template> - ``` + ```html + <template v-for="(item, index) in items"> + <span :key="`span-${index}`"></span> + <button :key="`button-${index}`"></button> + </template> + ``` 1. When dealing with nested `v-for` use the same guidelines as above. - ```html - <div - v-for="item in items" - :key="item.id" - > - <span - v-for="element in array" - :key="element.id" - > - <!-- content --> - </span> - </div> - ``` + ```html + <div + v-for="item in items" + :key="item.id" + > + <span + v-for="element in array" + :key="element.id" + > + <!-- content --> + </span> + </div> + ``` Useful links: @@ -662,35 +663,35 @@ Useful links: 1. Tooltips: Do not rely on `has-tooltip` class name for Vue components - ```javascript - // bad - <span - class="has-tooltip" - title="Some tooltip text"> - Text - </span> - - // good - <span - v-tooltip - title="Some tooltip text"> - Text - </span> - ``` + ```javascript + // bad + <span + class="has-tooltip" + title="Some tooltip text"> + Text + </span> + + // good + <span + v-tooltip + title="Some tooltip text"> + Text + </span> + ``` 1. Tooltips: When using a tooltip, include the tooltip directive, `./app/assets/javascripts/vue_shared/directives/tooltip.js` 1. Don't change `data-original-title`. - ```javascript - // bad - <span data-original-title="tooltip text">Foo</span> + ```javascript + // bad + <span data-original-title="tooltip text">Foo</span> - // good - <span title="tooltip text">Foo</span> + // good + <span title="tooltip text">Foo</span> - $('span').tooltip('_fixTitle'); - ``` + $('span').tooltip('_fixTitle'); + ``` ### The Javascript/Vue Accord diff --git a/doc/development/fe_guide/vuex.md b/doc/development/fe_guide/vuex.md index bf248b7f8af..9eeaee4482f 100644 --- a/doc/development/fe_guide/vuex.md +++ b/doc/development/fe_guide/vuex.md @@ -1,15 +1,18 @@ # Vuex + To manage the state of an application you should use [Vuex][vuex-docs]. _Note:_ All of the below is explained in more detail in the official [Vuex documentation][vuex-docs]. ## Separation of concerns + Vuex is composed of State, Getters, Mutations, Actions and Modules. When a user clicks on an action, we need to `dispatch` it. This action will `commit` a mutation that will change the state. _Note:_ The action itself will not update the state, only a mutation should update the state. ## File structure + When using Vuex at GitLab, separate this concerns into different files to improve readability: ``` @@ -21,10 +24,12 @@ When using Vuex at GitLab, separate this concerns into different files to improv ├── state.js # state └── mutation_types.js # mutation types ``` + The following example shows an application that lists and adds users to the state. (For a more complex example implementation take a look at the security applications store in [here](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/ee/app/assets/javascripts/vue_shared/security_reports/store)) ### `index.js` + This is the entry point for our store. You can use the following as a guide: ```javascript @@ -47,6 +52,7 @@ export default createStore(); ``` ### `state.js` + The first thing you should do before writing any code is to design the state. Often we need to provide data from haml to our Vue application. Let's store it in the state for better access. @@ -66,9 +72,11 @@ Often we need to provide data from haml to our Vue application. Let's store it i ``` #### Access `state` properties + You can use `mapState` to access state properties in the components. ### `actions.js` + An action is a payload of information to send data from our application to our store. An action is usually composed by a `type` and a `payload` and they describe what happened. @@ -110,6 +118,7 @@ In this file, we will write the actions that will call the respective mutations: ``` #### Actions Pattern: `request` and `receive` namespaces + When a request is made we often want to show a loading state to the user. Instead of creating an action to toggle the loading state and dispatch it in the component, @@ -136,6 +145,7 @@ By following this pattern we guarantee: 1. Actions are simple and straightforward #### Dispatching actions + To dispatch an action from a component, use the `mapActions` helper: ```javascript @@ -154,6 +164,7 @@ import { mapActions } from 'vuex'; ``` ### `mutations.js` + The mutations specify how the application state changes in response to actions sent to the store. The only way to change state in a Vuex store should be by committing a mutation. @@ -193,6 +204,7 @@ Remember that actions only describe that something happened, they don't describe ``` ### `getters.js` + Sometimes we may need to get derived state based on store state, like filtering for a specific prop. Using a getter will also cache the result based on dependencies due to [how computed props work](https://vuejs.org/v2/guide/computed.html#Computed-Caching-vs-Methods) This can be done through the `getters`: @@ -219,6 +231,7 @@ import { mapGetters } from 'vuex'; ``` ### `mutation_types.js` + From [vuex mutations docs][vuex-mutations]: > It is a commonly seen pattern to use constants for mutation types in various Flux implementations. This allows the code to take advantage of tooling like linters, and putting all constants in a single file allows your collaborators to get an at-a-glance view of what mutations are possible in the entire application. @@ -227,6 +240,7 @@ export const ADD_USER = 'ADD_USER'; ``` ### How to include the store in your application + The store should be included in the main component of your application: ```javascript @@ -241,6 +255,7 @@ The store should be included in the main component of your application: ``` ### Communicating with the Store + ```javascript <script> import { mapActions, mapState, mapGetters } from 'vuex'; @@ -298,29 +313,33 @@ export default { 1. Do not call a mutation directly. Always use an action to commit a mutation. Doing so will keep consistency throughout the application. From Vuex docs: - > why don't we just call store.commit('action') directly? Well, remember that mutations must be synchronous? Actions aren't. We can perform asynchronous operations inside an action. + > why don't we just call store.commit('action') directly? Well, remember that mutations must be synchronous? Actions aren't. We can perform asynchronous operations inside an action. - ```javascript - // component.vue + ```javascript + // component.vue - // bad - created() { - this.$store.commit('mutation'); - } + // bad + created() { + this.$store.commit('mutation'); + } + + // good + created() { + this.$store.dispatch('action'); + } + ``` - // good - created() { - this.$store.dispatch('action'); - } - ``` 1. Use mutation types instead of hardcoding strings. It will be less error prone. 1. The State will be accessible in all components descending from the use where the store is instantiated. ### Testing Vuex + #### Testing Vuex concerns + Refer to [vuex docs][vuex-testing] regarding testing Actions, Getters and Mutations. #### Testing components that need a store + Smaller components might use `store` properties to access the data. In order to write unit tests for those components, we need to include the store and provide the correct state: @@ -363,6 +382,7 @@ describe('component', () => { ``` #### Testing Vuex actions and getters + Because we're currently using [`babel-plugin-rewire`](https://github.com/speedskater/babel-plugin-rewire), you may encounter the following error when testing your Vuex actions and getters: `[vuex] actions should be function or object with "handler" function` diff --git a/doc/development/filtering_by_label.md b/doc/development/filtering_by_label.md new file mode 100644 index 00000000000..6e6b71b1787 --- /dev/null +++ b/doc/development/filtering_by_label.md @@ -0,0 +1,166 @@ +# Filtering by label + +## Introduction + +GitLab has [labels](../user/project/labels.md) that can be assigned to issues, +merge requests, and epics. Labels on those objects are a many-to-many relation +through the polymorphic `label_links` table. + +To filter these objects by multiple labels - for instance, 'all open +issues with the label ~Plan and the label ~backend' - we generate a +query containing a `GROUP BY` clause. In a simple form, this looks like: + +```sql +SELECT + issues.* +FROM + issues + INNER JOIN label_links ON label_links.target_id = issues.id + AND label_links.target_type = 'Issue' + INNER JOIN labels ON labels.id = label_links.label_id +WHERE + issues.project_id = 13083 + AND (issues.state IN ('opened')) + AND labels.title IN ('Plan', + 'backend') +GROUP BY + issues.id +HAVING (COUNT(DISTINCT labels.title) = 2) +ORDER BY + issues.updated_at DESC, + issues.id DESC +LIMIT 20 OFFSET 0 +``` + +In particular, note that: + +1. We `GROUP BY issues.id` so that we can ... +2. Use the `HAVING (COUNT(DISTINCT labels.title) = 2)` condition to ensure that + all matched issues have both labels. + +This is more complicated than is ideal. It makes the query construction more +prone to errors (such as +[gitlab-org/gitlab-ce#15557](https://gitlab.com/gitlab-org/gitlab-ce/issues/15557)). + +## Attempt A: WHERE EXISTS + +### Attempt A1: use multiple subqueries with WHERE EXISTS + +In +[gitlab-org/gitlab-ce#37137](https://gitlab.com/gitlab-org/gitlab-ce/issues/37137) +and its associated merge request +[gitlab-org/gitlab-ce!14022](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14022), +we tried to replace the `GROUP BY` with multiple uses of `WHERE EXISTS`. For the +example above, this would give: + +```sql +WHERE (EXISTS ( + SELECT + TRUE + FROM + label_links + INNER JOIN labels ON labels.id = label_links.label_id + WHERE + labels.title = 'Plan' + AND target_type = 'Issue' + AND target_id = issues.id)) +AND (EXISTS ( + SELECT + TRUE + FROM + label_links + INNER JOIN labels ON labels.id = label_links.label_id + WHERE + labels.title = 'backend' + AND target_type = 'Issue' + AND target_id = issues.id)) +``` + +While this worked without schema changes, and did improve readability somewhat, +it did not improve query performance. + +## Attempt B: Denormalize using an array column + +Having [removed MySQL support in GitLab +12.1](https://about.gitlab.com/2019/06/27/removing-mysql-support/), using +[Postgres's arrays](https://www.postgresql.org/docs/9.6/arrays.html) became more +tractable as we didn't have to support two databases. We discussed denormalizing +the `label_links` table for querying in +[gitlab-org/gitlab-ce#49651](https://gitlab.com/gitlab-org/gitlab-ce/issues/49651), +with two options: label IDs and titles. + +We can think of both of those as array columns on `issues`, `merge_requests`, +and `epics`: `issues.label_ids` would be an array column of label IDs, and +`issues.label_titles` would be an array of label titles. + +These array columns can be complemented with [GIN +indexes](https://www.postgresql.org/docs/9.6/gin-intro.html) to improve +matching. + +### Attempt B1: store label IDs for each object + +This has some strong advantages over titles: + +1. Unless a label is deleted, or a project is moved, we never need to + bulk-update the denormalized column. +2. It uses less storage than the titles. + +Unfortunately, our application design makes this hard. If we were able to query +just by label ID easily, we wouldn't need the `INNER JOIN labels` in the initial +query at the start of this document. GitLab allows users to filter by label +title across projects and even across groups, so a filter by the label ~Plan may +include labels with multiple distinct IDs. + +We do not want users to have to know about the different IDs, which means that +given this data set: + +| Project | ~Plan label ID | ~backend label ID | +| --- | --- | --- | +| A | 11 | 12 | +| B | 21 | 22 | +| C | 31 | 32 | + +We would need something like: + +```sql +WHERE + label_ids @> ARRAY[11, 12] + OR label_ids @> ARRAY[21, 22] + OR label_ids @> ARRAY[31, 32] +``` + +This can get even more complicated when we consider that in some cases, there +might be two ~backend labels - with different IDs - that could apply to the same +object, so the number of combinations would balloon further. + +### Attempt B2: store label titles for each object + +From the perspective of updating the labelable object, this is the worst +option. We have to bulk update the objects when: + +1. The objects are moved from one project to another. +1. The project is moved from one group to another. +1. The label is renamed. +1. The label is deleted. + +It also uses much more storage. Querying is simple, though: + +```sql +WHERE + label_titles @> ARRAY['Plan', 'backend'] +``` + +And our [tests in +gitlab-org/gitlab-ce#49651](https://gitlab.com/gitlab-org/gitlab-ce/issues/49651#note_188777346) +showed that this could be fast. + +However, at present, the disadvantages outweigh the advantages. + +## Conclusion + +We have yet to find a method that is demonstratably better than the current +method, when considering: + +1. Query performance. +1. Readability. +1. Ease of maintaining schema consistency. diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 17462887162..141f5a8d6d9 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -21,18 +21,18 @@ The following tools are used: 1. [`gettext_i18n_rails`](https://github.com/grosser/gettext_i18n_rails): this gem allow us to translate content from models, views and controllers. Also it gives us access to the following raketasks: - - `rake gettext:find`: Parses almost all the files from the - Rails application looking for content that has been marked for - translation. Finally, it updates the PO files with the new content that - it has found. - - `rake gettext:pack`: Processes the PO files and generates the - MO files that are binary and are finally used by the application. + - `rake gettext:find`: Parses almost all the files from the + Rails application looking for content that has been marked for + translation. Finally, it updates the PO files with the new content that + it has found. + - `rake gettext:pack`: Processes the PO files and generates the + MO files that are binary and are finally used by the application. 1. [`gettext_i18n_rails_js`](https://github.com/webhippie/gettext_i18n_rails_js): this gem is useful to make the translations available in JavaScript. It provides the following raketask: - - `rake gettext:po_to_json`: Reads the contents from the PO files and - generates JSON files containing all the available translations. + - `rake gettext:po_to_json`: Reads the contents from the PO files and + generates JSON files containing all the available translations. 1. PO editor: there are multiple applications that can help us to work with PO files, a good option is [Poedit](https://poedit.net/download) which is @@ -139,60 +139,61 @@ For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. Make s - In Ruby/HAML: - ```ruby - _("Hello %{name}") % { name: 'Joe' } => 'Hello Joe' - ``` + ```ruby + _("Hello %{name}") % { name: 'Joe' } => 'Hello Joe' + ``` - In JavaScript: - ```js - import { __, sprintf } from '~/locale'; + ```js + import { __, sprintf } from '~/locale'; - sprintf(__('Hello %{username}'), { username: 'Joe' }); // => 'Hello Joe' - ``` + sprintf(__('Hello %{username}'), { username: 'Joe' }); // => 'Hello Joe' + ``` - By default, `sprintf` escapes the placeholder values. - If you want to take care of that yourself, you can pass `false` as third argument. + By default, `sprintf` escapes the placeholder values. + If you want to take care of that yourself, you can pass `false` as third argument. - ```js - import { __, sprintf } from '~/locale'; + ```js + import { __, sprintf } from '~/locale'; - sprintf(__('This is %{value}'), { value: '<strong>bold</strong>' }); // => 'This is <strong>bold</strong>' - sprintf(__('This is %{value}'), { value: '<strong>bold</strong>' }, false); // => 'This is <strong>bold</strong>' - ``` + sprintf(__('This is %{value}'), { value: '<strong>bold</strong>' }); // => 'This is <strong>bold</strong>' + sprintf(__('This is %{value}'), { value: '<strong>bold</strong>' }, false); // => 'This is <strong>bold</strong>' + ``` ### Plurals - In Ruby/HAML: - ```ruby - n_('Apple', 'Apples', 3) - # => 'Apples' - ``` + ```ruby + n_('Apple', 'Apples', 3) + # => 'Apples' + ``` - Using interpolation: - ```ruby - n_("There is a mouse.", "There are %d mice.", size) % size - # => When size == 1: 'There is a mouse.' - # => When size == 2: 'There are 2 mice.' - ``` + Using interpolation: - Avoid using `%d` or count variables in singular strings. This allows more natural translation in some languages. + ```ruby + n_("There is a mouse.", "There are %d mice.", size) % size + # => When size == 1: 'There is a mouse.' + # => When size == 2: 'There are 2 mice.' + ``` + + Avoid using `%d` or count variables in singular strings. This allows more natural translation in some languages. - In JavaScript: - ```js - n__('Apple', 'Apples', 3) - // => 'Apples' - ``` + ```js + n__('Apple', 'Apples', 3) + // => 'Apples' + ``` - Using interpolation: + Using interpolation: - ```js - n__('Last day', 'Last %d days', x) - // => When x == 1: 'Last day' - // => When x == 2: 'Last 2 days' - ``` + ```js + n__('Last day', 'Last %d days', x) + // => When x == 1: 'Last day' + // => When x == 2: 'Last 2 days' + ``` ### Namespaces @@ -202,17 +203,17 @@ Namespaces should be PascalCase. - In Ruby/HAML: - ```ruby - s_('OpenedNDaysAgo|Opened') - ``` + ```ruby + s_('OpenedNDaysAgo|Opened') + ``` - In case the translation is not found it will return `Opened`. + In case the translation is not found it will return `Opened`. - In JavaScript: - ```js - s__('OpenedNDaysAgo|Opened') - ``` + ```js + s__('OpenedNDaysAgo|Opened') + ``` Note: The namespace should be removed from the translation. See the [translation guidelines for more details](translation.md#namespaced-strings). @@ -235,12 +236,12 @@ This makes use of [`Intl.DateTimeFormat`]. - In Ruby/HAML, we have two ways of adding format to dates and times: 1. **Through the `l` helper**, i.e. `l(active_session.created_at, format: :short)`. We have some predefined formats for - [dates](https://gitlab.com/gitlab-org/gitlab-ce/blob/v11.7.0/config/locales/en.yml#L54) and [times](https://gitlab.com/gitlab-org/gitlab-ce/blob/v11.7.0/config/locales/en.yml#L261). - If you need to add a new format, because other parts of the code could benefit from it, - you'll need to add it to [en.yml](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/locales/en.yml) file. + [dates](https://gitlab.com/gitlab-org/gitlab-ce/blob/v11.7.0/config/locales/en.yml#L54) and [times](https://gitlab.com/gitlab-org/gitlab-ce/blob/v11.7.0/config/locales/en.yml#L261). + If you need to add a new format, because other parts of the code could benefit from it, + you'll need to add it to [en.yml](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/locales/en.yml) file. 1. **Through `strftime`**, i.e. `milestone.start_date.strftime('%b %-d')`. We use `strftime` in case none of the formats - defined on [en.yml](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/locales/en.yml) matches the date/time - specifications we need, and if there is no need to add it as a new format because is very particular (i.e. it's only used in a single view). + defined on [en.yml](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/locales/en.yml) matches the date/time + specifications we need, and if there is no need to add it as a new format because is very particular (i.e. it's only used in a single view). ## Best practices @@ -268,40 +269,40 @@ should be externalized as follows: This also applies when using links in between translated sentences, otherwise these texts are not translatable in certain languages. - In Ruby/HAML, instead of: - - ```haml - - zones_link = link_to(s_('ClusterIntegration|zones'), 'https://cloud.google.com/compute/docs/regions-zones/regions-zones', target: '_blank', rel: 'noopener noreferrer') - = s_('ClusterIntegration|Learn more about %{zones_link}').html_safe % { zones_link: zones_link } - ``` - - Set the link starting and ending HTML fragments as variables like so: - - ```haml - - zones_link_url = 'https://cloud.google.com/compute/docs/regions-zones/regions-zones' - - zones_link_start = '<a href="%{url}" target="_blank" rel="noopener noreferrer">'.html_safe % { url: zones_link_url } - = s_('ClusterIntegration|Learn more about %{zones_link_start}zones%{zones_link_end}').html_safe % { zones_link_start: zones_link_start, zones_link_end: '</a>'.html_safe } - ``` + + ```haml + - zones_link = link_to(s_('ClusterIntegration|zones'), 'https://cloud.google.com/compute/docs/regions-zones/regions-zones', target: '_blank', rel: 'noopener noreferrer') + = s_('ClusterIntegration|Learn more about %{zones_link}').html_safe % { zones_link: zones_link } + ``` + + Set the link starting and ending HTML fragments as variables like so: + + ```haml + - zones_link_url = 'https://cloud.google.com/compute/docs/regions-zones/regions-zones' + - zones_link_start = '<a href="%{url}" target="_blank" rel="noopener noreferrer">'.html_safe % { url: zones_link_url } + = s_('ClusterIntegration|Learn more about %{zones_link_start}zones%{zones_link_end}').html_safe % { zones_link_start: zones_link_start, zones_link_end: '</a>'.html_safe } + ``` - In JavaScript, instead of: - ```js - {{ - sprintf(s__("ClusterIntegration|Learn more about %{link}"), { - link: '<a href="https://cloud.google.com/compute/docs/regions-zones/regions-zones" target="_blank" rel="noopener noreferrer">zones</a>' - }) - }} - ``` - - Set the link starting and ending HTML fragments as variables like so: - - ```js - {{ - sprintf(s__("ClusterIntegration|Learn more about %{linkStart}zones%{linkEnd}"), { - linkStart: '<a href="https://cloud.google.com/compute/docs/regions-zones/regions-zones" target="_blank" rel="noopener noreferrer">' - linkEnd: '</a>', - }) - }} - ``` + ```js + {{ + sprintf(s__("ClusterIntegration|Learn more about %{link}"), { + link: '<a href="https://cloud.google.com/compute/docs/regions-zones/regions-zones" target="_blank" rel="noopener noreferrer">zones</a>' + }) + }} + ``` + + Set the link starting and ending HTML fragments as variables like so: + + ```js + {{ + sprintf(s__("ClusterIntegration|Learn more about %{linkStart}zones%{linkEnd}"), { + linkStart: '<a href="https://cloud.google.com/compute/docs/regions-zones/regions-zones" target="_blank" rel="noopener noreferrer">' + linkEnd: '</a>', + }) + }} + ``` The reasoning behind this is that in some languages words change depending on context. For example in Japanese は is added to the subject of a sentence and を to the object. This is impossible to translate correctly if we extract individual words from the sentence. @@ -374,29 +375,29 @@ Let's suppose you want to add translations for a new language, let's say French. 1. The first step is to register the new language in `lib/gitlab/i18n.rb`: - ```ruby - ... - AVAILABLE_LANGUAGES = { - ..., - 'fr' => 'Français' - }.freeze - ... - ``` + ```ruby + ... + AVAILABLE_LANGUAGES = { + ..., + 'fr' => 'Français' + }.freeze + ... + ``` 1. Next, you need to add the language: - ```sh - bin/rake gettext:add_language[fr] - ``` + ```sh + bin/rake gettext:add_language[fr] + ``` - If you want to add a new language for a specific region, the command is similar, - you just need to separate the region with an underscore (`_`). For example: + If you want to add a new language for a specific region, the command is similar, + you just need to separate the region with an underscore (`_`). For example: - ```sh - bin/rake gettext:add_language[en_GB] - ``` + ```sh + bin/rake gettext:add_language[en_GB] + ``` - Please note that you need to specify the region part in capitals. + Please note that you need to specify the region part in capitals. 1. Now that the language is added, a new directory has been created under the path: `locale/fr/`. You can now start using your PO editor to edit the PO file @@ -406,9 +407,9 @@ Let's suppose you want to add translations for a new language, let's say French. in order to generate the binary MO files and finally update the JSON files containing the translations: - ```sh - bin/rake gettext:compile - ``` + ```sh + bin/rake gettext:compile + ``` 1. In order to see the translated content we need to change our preferred language which can be found under the user's **Settings** (`/profile`). @@ -416,7 +417,7 @@ Let's suppose you want to add translations for a new language, let's say French. 1. After checking that the changes are ok, you can proceed to commit the new files. For example: - ```sh - git add locale/fr/ app/assets/javascripts/locale/fr/ - git commit -m "Add French translations for Cycle Analytics page" - ``` + ```sh + git add locale/fr/ app/assets/javascripts/locale/fr/ + git commit -m "Add French translations for Cycle Analytics page" + ``` diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index 2dc06ba10a5..882e2230636 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -65,28 +65,24 @@ Below you can read more about how to use it and how does it work. Currently, we are using _multi-project pipeline_-like approach to run QA pipelines. - - -<details> -<summary>Show mermaid source</summary> -<pre> +```mermaid graph LR A1 -.->|1. Triggers an omnibus-gitlab pipeline and wait for it to be done| A2 - B2[<b>`Trigger-qa` stage</b><br />`Trigger:qa-test` job] -.->|2. Triggers a gitlab-qa pipeline and wait for it to be done| A3 + B2[`Trigger-qa` stage<br>`Trigger:qa-test` job] -.->|2. Triggers a gitlab-qa pipeline and wait for it to be done| A3 -subgraph gitlab-ce/ee pipeline - A1[<b>`test` stage</b><br />`package-and-qa` job] +subgraph "gitlab-ce/ee pipeline" + A1[`test` stage<br>`package-and-qa` job] end -subgraph omnibus-gitlab pipeline - A2[<b>`Trigger-docker` stage</b><br />`Trigger:gitlab-docker` job] -->|once done| B2 +subgraph "omnibus-gitlab pipeline" + A2[`Trigger-docker` stage<br>`Trigger:gitlab-docker` job] -->|once done| B2 end -subgraph gitlab-qa pipeline - A3>QA jobs run] -.->|3. Reports back the pipeline result to the `package-and-qa` job<br />and post the result on the original commit tested| A1 +subgraph "gitlab-qa pipeline" + A3>QA jobs run] -.->|3. Reports back the pipeline result to the `package-and-qa` job<br>and post the result on the original commit tested| A1 end -</pre> -</details> +``` + 1. Developer triggers a manual action, that can be found in CE / EE merge requests. This starts a chain of pipelines in multiple projects. diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index 14a169dcc1d..e1df8be8b6f 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -110,7 +110,7 @@ end ``` > Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later. - +> > The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects] before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. Below are the steps that the test covers: @@ -211,7 +211,7 @@ A pre-condition for the entire test suite is defined in the `before :context` bl > For our test suite, due to the need of the tests being completely independent of each other, we won't use the `before :context` block. The `before :context` block would make the tests dependent on each other because the first test changes the label of the issue, and the second one depends on the `'animal::fox'` label being set. -> **Tip:** In case of a test suite with only one `it` block it's ok to use only the `before` block (see below) with all the test's pre-conditions. +TIP: **Tip:** In case of a test suite with only one `it` block it's ok to use only the `before` block (see below) with all the test's pre-conditions. #### `before` @@ -274,11 +274,11 @@ end In the `before` block we create all the application state needed for the tests to run. We do that by using the `Runtime::Browser.visit` method to go to the login page, by performing a `sign_in_using_credentials` from the `Login` Page Object, by fabricating resources via APIs (`issue`, and `Resource::Label`), and by using the `issue.visit!` to visit the issue page. > A project is created in the background by creating the `issue` resource. - +> > When creating the [Resources], notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. - +> > What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage. - +> > Soon we will cover the use of the already existing resources' methods and the creation of your own `fabricate_via_api` methods for resources where this is still not available, but first, let's optimize our implementation. ### 6. Optimization @@ -362,7 +362,7 @@ First, in the [issue resource](https://gitlab.com/gitlab-org/gitlab-ee/blob/d358 Add the following `attribute :id` and `attribute :labels` right above the [`attribute :title`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L15). > This line is needed to allow for the issue fabrication, and for labels to be automatically added to the issue when fabricating it via API. - +> > We add the attributes above the existing attribute to keep them alphabetically organized. Then, let's initialize an instance variable for labels to allow an empty array as default value when such information is not passed during the resource fabrication, since this optional. [Between the attributes and the `fabricate!` method](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a1f1408728f19b2aa15887cd20bddab7e70c8bd/qa/qa/resource/issue.rb#L18), add the following: @@ -437,7 +437,7 @@ By defining the `resource_web_url(resource)` method, we override the one from th By defining the `api_get_path` method, we **would** allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single label, but since there's no path available for that in the publich API, we raise a `NotImplementedError` instead. -By defining the `api_post_path` method, we allow for the [`ApiFabricator `](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new label in a specific project. +By defining the `api_post_path` method, we allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new label in a specific project. By defining the `api_post_body` method, we we allow for the [`ApiFabricator.api_post`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/api_fabricator.rb#L68) method to know which data to send when making the `POST` request. @@ -580,7 +580,7 @@ filter_output = search_field_tag search_id, nil, class: "dropdown-input-field", > `data-qa-*` data attributes and CSS classes starting with `qa-` are used solely for the purpose of QA and testing. > By defining these, we add **testability** to the application. - +> > When defining a data attribute like: `qa_selector: 'labels_block'`, it should match the element definition: `element :labels_block`. We use a [sanity test](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/page#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective selectors in the specified views. #### Updates in the `QA::Page::Base` class @@ -599,8 +599,6 @@ This method receives an element (`name`) and the `keys` that it will send to tha As you might remember, in the Issue Page Object we call this method like this: `send_keys_to_element(:dropdown_input_field, [label, :enter])`. -___ - With that, you should be able to start writing end-to-end tests yourself. *Congratulations!* [Page Objects]: page_objects.md diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md index 6a888142575..97560e616a1 100644 --- a/doc/development/testing_guide/end_to_end/style_guide.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -45,7 +45,7 @@ Notice that in the above example, before clicking the `:operations_environments_ > We can create these methods as helpers to abstract multi-step navigation. -### Element naming convention +## Element naming convention When adding new elements to a page, it's important that we have a uniform element naming convention. @@ -67,7 +67,7 @@ We follow a simple formula roughly based on hungarian notation. *Note: This list is a work in progress. This list will eventually be the end-all enumeration of all available types. I.e., any element that does not end with something in this list is bad form.* -#### Examples +### Examples **Good** @@ -98,3 +98,47 @@ view '...' do element :ssh_clone_url end ``` + +## Block argument naming + +To have a standard on how we call pages when using the `.perform` method, we use the name of page object being called, all lowercased, and separated by underscore, if needed (see good and bad examples below.) This also applies to resources. We chose not to simply use `page` because that would shadow the Capybara DSL, potentially leading to confusion and bugs. + +### Examples + +**Good** + +```ruby +# qa/specs/features/browser_ui/1_manage/project/add_project_member_spec.rb + +Page::Project::Settings::Members.perform do |members| + members.do_something +end +``` + +```ruby +# qa/specs/features/ee/browser_ui/3_create/merge_request/add_batch_comments_in_merge_request_spec.rb + +Resource::MergeRequest.fabricate! do |merge_request| + merge_request.do_something_else +end +``` + +**Bad** + +```ruby +# qa/specs/features/browser_ui/1_manage/project/add_project_member_spec.rb + +Page::Project::Settings::Members.perform do |project_settings_members_page| + project_settings_members_page.do_something +end +``` + +```ruby +# qa/specs/features/ee/browser_ui/3_create/merge_request/add_batch_comments_in_merge_request_spec.rb + +Resource::MergeRequest.fabricate! do |merge_request_page| + merge_request_page.do_something_else +end +``` + +> Besides the advantage of having a standard in place, by following this standard we also write shorter lines of code.
\ No newline at end of file diff --git a/doc/development/testing_guide/img/qa_on_merge_requests_cicd_architecture.png b/doc/development/testing_guide/img/qa_on_merge_requests_cicd_architecture.png Binary files differdeleted file mode 100644 index 5b93a05db96..00000000000 --- a/doc/development/testing_guide/img/qa_on_merge_requests_cicd_architecture.png +++ /dev/null diff --git a/doc/development/testing_guide/img/review_apps_cicd_architecture.png b/doc/development/testing_guide/img/review_apps_cicd_architecture.png Binary files differdeleted file mode 100644 index 1ee28d3db91..00000000000 --- a/doc/development/testing_guide/img/review_apps_cicd_architecture.png +++ /dev/null diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index 96761622cfe..7843fc4c874 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -8,38 +8,33 @@ Review Apps are automatically deployed by each pipeline, both in ### CI/CD architecture diagram - - -<details> -<summary>Show mermaid source</summary> -<pre> +```mermaid graph TD build-qa-image -.->|once the `prepare` stage is done| gitlab:assets:compile review-build-cng -->|triggers a CNG-mirror pipeline and wait for it to be done| CNG-mirror review-build-cng -.->|once the `test` stage is done| review-deploy review-deploy -.->|once the `review` stage is done| review-qa-smoke -subgraph 1. gitlab-ce/ee `prepare` stage +subgraph "1. gitlab-ce/ee `prepare` stage" build-qa-image end -subgraph 2. gitlab-ce/ee `test` stage +subgraph "2. gitlab-ce/ee `test` stage" gitlab:assets:compile -->|plays dependent job once done| review-build-cng end -subgraph 3. gitlab-ce/ee `review` stage - review-deploy["review-deploy<br /><br />Helm deploys the Review App using the Cloud<br/>Native images built by the CNG-mirror pipeline.<br /><br />Cloud Native images are deployed to the `review-apps-ce` or `review-apps-ee`<br />Kubernetes (GKE) cluster, in the GCP `gitlab-review-apps` project."] +subgraph "3. gitlab-ce/ee `review` stage" + review-deploy["review-deploy<br><br>Helm deploys the Review App using the Cloud<br/>Native images built by the CNG-mirror pipeline.<br><br>Cloud Native images are deployed to the `review-apps-ce` or `review-apps-ee`<br>Kubernetes (GKE) cluster, in the GCP `gitlab-review-apps` project."] end -subgraph 4. gitlab-ce/ee `qa` stage - review-qa-smoke[review-qa-smoke<br /><br />gitlab-qa runs the smoke suite against the Review App.] +subgraph "4. gitlab-ce/ee `qa` stage" + review-qa-smoke[review-qa-smoke<br><br>gitlab-qa runs the smoke suite against the Review App.] end -subgraph CNG-mirror pipeline +subgraph "CNG-mirror pipeline" CNG-mirror>Cloud Native images are built]; end -</pre> -</details> +``` ### Detailed explanation @@ -115,6 +110,28 @@ On every [pipeline][gitlab-pipeline] in the `qa` stage, the browser performance testing using a [Sitespeed.io Container](../../user/project/merge_requests/browser_performance_testing.md). +## Cluster configuration + +### Node pools + +Both `review-apps-ce` and `review-apps-ee` clusters are currently set up with +two node pools: + +- a node pool of non-preemptible `n1-standard-2` (2 vCPU, 7.5 GB memory) nodes + dedicated to the `tiller` deployment (see below) with a single node. +- a node pool of preemptible `n1-standard-2` (2 vCPU, 7.5 GB memory) nodes, + with a minimum of 1 node and a maximum of 250 nodes. + +### Helm/Tiller + +The `tiller` deployment (the Helm server) is deployed to a dedicated node pool +that has the `app=helm` label and a specific +[taint](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) +to prevent other pods from being scheduled on this node pool. + +This is to ensure Tiller isn't affected by "noisy" neighbors that could put +their node under pressure. + ## How to: ### Log into my Review App @@ -241,15 +258,6 @@ thousands of unused Docker images.** CNG-mirror project to store these Docker images so that we can just wipe out the registry at some point, and use a new fresh, empty one. -**How big are the Kubernetes clusters (`review-apps-ce` and `review-apps-ee`)?** - - > The clusters are currently set up with a single pool of preemptible nodes, - with a minimum of 1 node and a maximum of 500 nodes. - -**What are the machine running on the cluster?** - - > We're currently using `n1-standard-1` (1 vCPU, 3.75 GB memory) machines. - **How do we secure this from abuse? Apps are open to the world so we need to find a way to limit it to only us.** diff --git a/doc/development/utilities.md b/doc/development/utilities.md index 0e396baccff..4021756343c 100644 --- a/doc/development/utilities.md +++ b/doc/development/utilities.md @@ -6,44 +6,44 @@ We developed a number of utilities to ease development. - Deep merges an array of hashes: - ``` ruby - Gitlab::Utils::MergeHash.merge( - [{ hello: ["world"] }, - { hello: "Everyone" }, - { hello: { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } }, - "Goodbye", "Hallo"] - ) - ``` - - Gives: - - ``` ruby - [ - { - hello: - [ - "world", - "Everyone", - { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } - ] - }, - "Goodbye" - ] - ``` + ``` ruby + Gitlab::Utils::MergeHash.merge( + [{ hello: ["world"] }, + { hello: "Everyone" }, + { hello: { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } }, + "Goodbye", "Hallo"] + ) + ``` + + Gives: + + ``` ruby + [ + { + hello: + [ + "world", + "Everyone", + { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } + ] + }, + "Goodbye" + ] + ``` - Extracts all keys and values from a hash into an array: - ``` ruby - Gitlab::Utils::MergeHash.crush( - { hello: "world", this: { crushes: ["an entire", "hash"] } } - ) - ``` + ``` ruby + Gitlab::Utils::MergeHash.crush( + { hello: "world", this: { crushes: ["an entire", "hash"] } } + ) + ``` - Gives: + Gives: - ``` ruby - [:hello, "world", :this, :crushes, "an entire", "hash"] - ``` + ``` ruby + [:hello, "world", :this, :crushes, "an entire", "hash"] + ``` ## [`Override`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/utils/override.rb) @@ -53,9 +53,9 @@ We developed a number of utilities to ease development. `ENV['STATIC_VERIFICATION']` is set to avoid production runtime overhead. This is useful to check: - - If we have typos in overriding methods. - - If we renamed the overridden methods, making original overriding methods - overrides nothing. + - If we have typos in overriding methods. + - If we renamed the overridden methods, making original overriding methods + overrides nothing. Here's a simple example: @@ -94,47 +94,47 @@ We developed a number of utilities to ease development. - Memoize the value even if it is `nil` or `false`. - We often do `@value ||= compute`, however this doesn't work well if - `compute` might eventually give `nil` and we don't want to compute again. - Instead we could use `defined?` to check if the value is set or not. - However it's tedious to write such pattern, and `StrongMemoize` would - help us use such pattern. + We often do `@value ||= compute`, however this doesn't work well if + `compute` might eventually give `nil` and we don't want to compute again. + Instead we could use `defined?` to check if the value is set or not. + However it's tedious to write such pattern, and `StrongMemoize` would + help us use such pattern. - Instead of writing patterns like this: + Instead of writing patterns like this: - ``` ruby - class Find - def result - return @result if defined?(@result) + ``` ruby + class Find + def result + return @result if defined?(@result) - @result = search - end + @result = search end - ``` + end + ``` - We could write it like: + We could write it like: - ``` ruby - class Find - include Gitlab::Utils::StrongMemoize + ``` ruby + class Find + include Gitlab::Utils::StrongMemoize - def result - strong_memoize(:result) do - search - end + def result + strong_memoize(:result) do + search end end - ``` + end + ``` - Clear memoization - ``` ruby - class Find - include Gitlab::Utils::StrongMemoize - end + ``` ruby + class Find + include Gitlab::Utils::StrongMemoize + end - Find.new.clear_memoization(:result) - ``` + Find.new.clear_memoization(:result) + ``` ## [`RequestCache`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/cache/request_cache.rb) diff --git a/doc/development/what_requires_downtime.md b/doc/development/what_requires_downtime.md index 24edd05da2f..f0da1cc2ddc 100644 --- a/doc/development/what_requires_downtime.md +++ b/doc/development/what_requires_downtime.md @@ -140,7 +140,7 @@ done without requiring downtime. However, this does require that any application changes are deployed _first_. Thus, changing the constraints of a column should happen in a post-deployment migration. NOTE: Avoid using `change_column` as it produces inefficient query because it re-defines -the whole column type. For example, to add a NOT NULL constraint, prefer `change_column_null ` +the whole column type. For example, to add a NOT NULL constraint, prefer `change_column_null` ## Changing Column Types diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index fd16047b8e4..fc3d36910f2 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -1,15 +1,18 @@ --- comments: false +type: index --- # GitLab basics guides -This section provides resources to help you start with GitLab by focusing on basic functionality. +This section provides resources to help you start working with GitLab and Git by focusing +on the basic features that you will need to use. This documentation is split into the following groups: - [GitLab-specific functionality](#gitlab-basics), for basic GitLab features. -- [General Git functionality](#git-basics), for working with Git in conjunction with GitLab. +- [General Git functionality](#working-with-git-from-the-command-line), for working + with Git in conjunction with GitLab. ## GitLab basics @@ -17,22 +20,26 @@ The following are guides to basic GitLab functionality: - [Create and add your SSH public key](create-your-ssh-keys.md), for enabling Git over SSH. - [Create a project](create-project.md), to start using GitLab. -- [Create a group](../user/group/index.md#create-a-new-group), to combine and administer projects together. +- [Create a group](../user/group/index.md#create-a-new-group), to combine and administer + projects together. - [Create a branch](create-branch.md), to make changes to files stored in a project's repository. - [Fork a project](fork-project.md), to duplicate projects so they can be worked on in parallel. - [Add a file](add-file.md), to add new files to a project's repository. -- [Add an image](add-image.md), to add new images to a project's repository. -- [Create an issue](../user/project/issues/managing_issues.md#create-a-new-issue), to start collaborating within a project. -- [Create a merge request](add-merge-request.md), to request changes made in a branch be merged into a project's repository. -- See how these features come together in the [GitLab Flow introduction video](https://youtu.be/InKNIvky2KE) and [GitLab Flow page](../workflow/gitlab_flow.md). +- [Create an issue](../user/project/issues/managing_issues.md#create-a-new-issue), + to start collaborating within a project. +- [Create a merge request](add-merge-request.md), to request changes made in a branch + be merged into a project's repository. +- See how these features come together in the [GitLab Flow introduction video](https://youtu.be/InKNIvky2KE) + and [GitLab Flow page](../workflow/gitlab_flow.md). -## Git basics +## Working with Git from the command line -If you're familiar with Git on the command line, you can interact with your GitLab projects just as you would with any other Git repository. +If you're familiar with Git on the command line, you can interact with your GitLab +projects just as you would with any other Git repository. These resources will help get further acclimated to working on the command line. - [Start using Git on the command line](start-using-git.md), for some simple Git commands. - [Command line basics](command-line-commands.md), to create and edit files using the command line. -More Git resources are available at GitLab's [Git documentation](../topics/git/index.md). +More Git resources are available in GitLab's [Git documentation](../topics/git/index.md). diff --git a/doc/gitlab-basics/add-file.md b/doc/gitlab-basics/add-file.md index e9fbcbc23a9..41cc8bc4aeb 100644 --- a/doc/gitlab-basics/add-file.md +++ b/doc/gitlab-basics/add-file.md @@ -1,5 +1,91 @@ -# How to add a file +--- +type: howto +--- -You can create a file in your [terminal](command-line-commands.md) and push -to GitLab or you can use the +# Add a file to a repository + +Adding files to a repository is a small, but key task. Bringing files in to a repository, +such as code, images, or documents, allows them to be tracked by Git, even though they +may have been created elsewhere. + +You can add a file to a repository in your [terminal](#add-a-file-using-the-command-line), and +then push to GitLab. You can also use the [web interface](../user/project/repository/web_editor.md#upload-a-file), +which may be a simpler solution. + +If you need to create a file first, for example a `README.md` text file, that can +also be done from the [terminal](command-line-commands.md#create-a-text-file-in-the-current-directory) or [web interface](../user/project/repository/web_editor.md#create-a-file). + +## Add a file using the command line + +Open a [terminal/shell](command-line-commands.md), and change into the folder of your +GitLab project. This usually means running the following command until you get +to the desired destination: + +```sh +cd <destination folder> +``` + +[Create a branch](create-branch.md) to add your file to, before it is added to the master +(main) branch of the project. It is not strictly necessary, but working directly in +the `master` branch is not recommended unless your project is very small, and you are +the only person working on it. You can [switch to an existing branch](start-using-git.md#work-on-an-existing-branch), +if you have one already. + +Using your standard tool for copying files (for example, Finder in macOS, or File Explorer +in Windows), put the file into a directory within the GitLab project. + +Check if your file is actually present in the directory (if you are in Windows, +use `dir` instead): + +```sh +ls +``` + +You should see the name of the file in the list shown. + +Check the status: + +```sh +git status +``` + +Your file's name should appear in red, so `git` took notice of it! Now add it +to the repository: + +```sh +git add <name of file> +``` + +Check the status again, your file's name should have turned green: + +```sh +git status +``` + +Commit (save) your file to the repository: + +```sh +git commit -m "DESCRIBE COMMIT IN A FEW WORDS" +``` + +Now you can push (send) your changes (in the branch `<branch-name>`) to GitLab +(the git remote named 'origin'): + +```sh +git push origin <branch-name> +``` + +Your image will be added to your branch in your repository in GitLab. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/gitlab-basics/add-image.md b/doc/gitlab-basics/add-image.md index 17c210c43e0..11a4a6bbd97 100644 --- a/doc/gitlab-basics/add-image.md +++ b/doc/gitlab-basics/add-image.md @@ -1,56 +1,5 @@ -# How to add an image +--- +redirect_to: 'add-file.md' +--- -Using your standard tool for copying files (e.g. Finder in Mac OS, or Explorer -in Windows, or...), put the image file into the GitLab project. You can find the -project as a regular folder in your files. - -Go to your [shell](command-line-commands.md), and move into the folder of your -GitLab project. This usually means running the following command until you get -to the desired destination: - -``` -cd NAME-OF-FOLDER-YOU'D-LIKE-TO-OPEN -``` - -Check if your image is actually present in the directory (if you are in Windows, -use `dir` instead): - -``` -ls -``` - -You should see the name of the image in the list shown. - -Check the status: - -``` -git status -``` - -Your image's name should appear in red, so `git` took notice of it! Now add it -to the repository: - -``` -git add NAME-OF-YOUR-IMAGE -``` - -Check the status again, your image's name should have turned green: - -``` -git status -``` - -Commit: - -``` -git commit -m "DESCRIBE COMMIT IN A FEW WORDS" -``` - -Now you can push (send) your changes (in the branch NAME-OF-BRANCH) to GitLab -(the git remote named 'origin'): - -``` -git push origin NAME-OF-BRANCH -``` - -Your image will be added to your branch in your repository in GitLab. +This document was moved to [another location](add-file.md). diff --git a/doc/gitlab-basics/add-merge-request.md b/doc/gitlab-basics/add-merge-request.md index 7bb2e014738..1a6a26152fa 100644 --- a/doc/gitlab-basics/add-merge-request.md +++ b/doc/gitlab-basics/add-merge-request.md @@ -1,9 +1,16 @@ +--- +type: howto +--- + # How to create a merge request -Merge requests are useful to integrate separate changes that you've made to a -project, on different branches. This is a brief guide on how to create a merge -request. For more information, check the -[merge requests documentation](../user/project/merge_requests/index.md). +Merge requests are how you integrate separate changes that you've made in a +[branch](create-branch.md) to a [project](create-project.md). + +This is a brief guide on how to create a merge request. For more detailed information, +check the [merge requests documentation](../user/project/merge_requests/index.md), or +you can watch our [GitLab Flow video](https://www.youtube.com/watch?v=InKNIvky2KE) for +a quick overview of working with merge requests. --- @@ -12,19 +19,31 @@ request. For more information, check the 1. Go to the project where you'd like to merge your changes and click on the **Merge requests** tab. 1. Click on **New merge request** on the right side of the screen. -1. From there on, you have the option to select the source branch and the target +1. From there, you have the option to select the source branch and the target branch you'd like to compare to. The default target project is the upstream repository, but you can choose to compare across any of its forks. -  +  1. When ready, click on the **Compare branches and continue** button. 1. At a minimum, add a title and a description to your merge request. Optionally, - select a user to review your merge request and to accept or close it. You may - also select a milestone and labels. + select a user to review your merge request. You may also select a milestone and + labels. -  +  1. When ready, click on the **Submit merge request** button. -Your merge request will be ready to be approved and merged. +Your merge request will be ready to be reviewed, approved, and merged. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/gitlab-basics/basic-git-commands.md b/doc/gitlab-basics/basic-git-commands.md index c2a3415cbc4..4e0cc2de2bc 100644 --- a/doc/gitlab-basics/basic-git-commands.md +++ b/doc/gitlab-basics/basic-git-commands.md @@ -1,3 +1,5 @@ -# Basic Git commands +--- +redirect_to: 'start-using-git.md' +--- -This section is now merged into [Start using Git](start-using-git.md). +This document was moved to [another location](start-using-git.md). diff --git a/doc/gitlab-basics/command-line-commands.md b/doc/gitlab-basics/command-line-commands.md index b8ebbbea9d4..ed70d3ce598 100644 --- a/doc/gitlab-basics/command-line-commands.md +++ b/doc/gitlab-basics/command-line-commands.md @@ -1,75 +1,47 @@ -# Command Line basic commands - -## Start working on your project - -In Git, when you copy a project you say you "clone" it. To work on a git project locally (from your own computer), you will need to clone it. To do this, sign in to GitLab. - -When you are on your Dashboard, click on the project that you'd like to clone. -To work in the project, you can copy a link to the Git repository through a SSH -or a HTTPS protocol. SSH is easier to use after it's been -[set up](create-your-ssh-keys.md). While you are at the **Project** tab, select -HTTPS or SSH from the dropdown menu and copy the link using the _Copy URL to clipboard_ -button (you'll have to paste it on your shell in the next step). - - - -## Working with project files on the command line - -This section has examples of some basic shell commands that you might find useful. For more information, search the web for _bash commands_. - -Alternatively, you can edit files using your choice of editor (IDE) or the GitLab user interface. - -### Clone your project - -Go to your computer's shell and type the following command with your SSH or HTTPS URL: +--- +type: howto, reference +--- -``` -git clone PASTE HTTPS OR SSH HERE -``` - -A clone of the project will be created in your computer. - -NOTE: **Note:** -If you clone your project via a URL that contains special characters, make sure -that characters are URL-encoded. - -### Go into a project directory to work in it - -``` -cd NAME-OF-PROJECT -``` +# Command Line basic commands -### Go back one directory +When [working with Git from the command line](start-using-git.md), you will need to +use more than just the Git commands. There are several basic commands that you should +learn, in order to make full use of the command line. -``` -cd .. -``` +## Start working on your project -### List what’s in the current directory +To work on a git project locally (from your own computer), with the command line, +first you will need to [clone (copy) it](start-using-git.md#clone-a-repository) to +your computer. -``` -ls -``` +## Working with files on the command line -### List what’s in the current directory that starts with `a` +This section has examples of some basic shell commands that you might find useful. +For more information, search the web for _bash commands_. -``` -ls a* -``` +Alternatively, you can edit files using your choice of editor (IDE), or the GitLab user +interface (not locally). -### List what’s in the current directory that ends with `.md` +### Common commands -``` -ls *.md -``` +The list below is not exhaustive, but contains many of the most commonly used commands. -### Create a new directory +| Command | Description | +|--------------------------------|---------------------------------------------| +| `cd NAME-OF-DIRECTORY` | Go into a directory to work in it | +| `cd ..` | Go back one directory | +| `ls` | List what’s in the current directory | +| `ls a*` | List what’s in the current directory that starts with `a` | +| `ls *.md` | List what’s in the current directory that ends with `.md` | +| `mkdir NAME-OF-YOUR-DIRECTORY` | Create a new directory | +| `cat README.md` | Display the contents of a [text file you created previously](#create-a-text-file-in-the-current-directory) | +| `pwd` | Show the current directory | +| `clear` | Clear the shell window | -``` -mkdir NAME-OF-YOUR-DIRECTORY -``` +### Create a text file in the current directory -### Create a README.md file in the current directory +To create a text file from the command line, for example `README.md`, follow these +steps: ``` touch README.md @@ -80,37 +52,37 @@ nano README.md #### Press: enter ``` -### Show the contents of the README.md file +### Remove a file or directory -``` -cat README.md -``` - -### Remove a file +It is easy to delete (remove) a file or directory, but be careful: DANGER: **Danger:** -This will permanently delete the file. +This will **permanently** delete a file. ``` rm NAME-OF-FILE ``` -### Remove a directory and all of its contents - DANGER: **Danger:** -This will permanently delete the directory and all of its contents. +This will **permanently** delete a directory and **all** of its contents. ``` rm -r NAME-OF-DIRECTORY ``` -### View command history +### View and Execute commands from history + +You can view the history of all the commands you executed from the command line, +and then execute any of them again, if needed. + +First, list the commands you executed previously: ``` history ``` -### Execute command 123 from history +Then, choose a command from the list and check the number next to the command (`123`, +for example) . Execute the same full command with: ``` !123 @@ -118,28 +90,32 @@ history ### Carry out commands for which the account you are using lacks authority -You will be asked for an administrator’s password. +Not all commands can be executed from a basic user account on a computer, you may +need administrator's rights to execute commands that affect the system, or try to access +protected data, for example. You can use `sudo` to execute these commands, but you +will likely be asked for an administrator password. ``` -sudo COMMAND +sudo RESTRICTED-COMMAND ``` CAUTION: **Caution:** Be careful of the commands you run with `sudo`. Certain commands may cause -damage to your data and system. +damage to your data or system. -### Show which directory I am in +## Sample Git taskflow -``` -pwd -``` - -### Clear the shell window +If you are completely new to Git, looking through some [sample taskflows](https://rogerdudler.github.io/git-guide/) +will help you understand the best practices for using these commands as you work. -``` -clear -``` +<!-- ## Troubleshooting -### Sample Git taskflow +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -If you are completely new to Git, looking through some [sample taskflows](https://rogerdudler.github.io/git-guide/) will help you understand best practices for using these commands as you work. +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/gitlab-basics/create-branch.md b/doc/gitlab-basics/create-branch.md index ad94f0dad29..e2a2fb52af8 100644 --- a/doc/gitlab-basics/create-branch.md +++ b/doc/gitlab-basics/create-branch.md @@ -1,12 +1,16 @@ +--- +type: howto +--- + # How to create a branch -A branch is an independent line of development. +A branch is an independent line of development in a [project](../user/project/index.md). -New commits are recorded in the history for the current branch, which results -in taking the source from someone’s repository (the place where the history of -your work is stored) at certain point in time, and apply your own changes to it -in the history of the project. +When you create a new branch (in your [terminal](basic-git-commands.md) or with +[the web interface](../user/project/repository/web_editor.md#create-a-new-branch)), +you are creating a snapshot of a certain branch, usually the main `master` branch, +at it's current state. From there, you can start to make your own changes without +affecting the main codebase. The history of your changes will be tracked in your branch. -To add changes to your GitLab project, you should create a branch. You can do -it in your [terminal](basic-git-commands.md) or by -[using the web interface](../user/project/repository/web_editor.md#create-a-new-branch). +When your changes are ready, you then merge them into the rest of the codebase with a +[merge request](add-merge-request.md). diff --git a/doc/gitlab-basics/create-project.md b/doc/gitlab-basics/create-project.md index 2caf7dbbc7a..8bbaf5d1927 100644 --- a/doc/gitlab-basics/create-project.md +++ b/doc/gitlab-basics/create-project.md @@ -1,10 +1,13 @@ -# Create a project +--- +type: howto +--- -[Projects](../user/project/index.md) combine many features of GitLab together. +# Creating projects -NOTE: **Note:** -For a list of words that cannot be used as project names see -[Reserved project and group names](../user/reserved_names.md). +Most work in GitLab is done within a [Project](../user/project/index.md). Files and +code are saved in projects, and most features are used within the scope of projects. + +## Create a project in GitLab To create a project in GitLab: @@ -14,11 +17,14 @@ To create a project in GitLab: - Create a [blank project](#blank-projects). - Create a project using with one of the available [project templates](#project-templates). - [Import a project](../user/project/import/index.md) from a different repository, - if enabled on your GitLab instance. Contact your GitLab admin if this - is unavailable. + if enabled on your GitLab instance. Contact your GitLab admin if this is unavailable. - Run [CI/CD pipelines for external repositories](../ci/ci_cd_for_external_repos/index.md). **(PREMIUM)** -## Blank projects +NOTE: **Note:** +For a list of words that cannot be used as project names see +[Reserved project and group names](../user/reserved_names.md). + +### Blank projects To create a new blank project on the **New project** page: @@ -37,23 +43,25 @@ To create a new blank project on the **New project** page: can be cloned. 1. Click **Create project**. -## Project templates +### Project templates -Project templates can pre-populate your project with necessary files to get you started quickly. +Project templates can pre-populate a new project with the necessary files to get you +started quickly. There are two types of project templates: - [Built-in templates](#built-in-templates), sourced from the following groups: - [`project-templates`](https://gitlab.com/gitlab-org/project-templates) - [`pages`](https://gitlab.com/pages) -- [Custom project templates](#custom-project-templates-premium-only), for custom templates configured by GitLab administrators and users. +- [Custom project templates](#custom-project-templates-premium), for custom templates + configured by GitLab administrators and users. -### Built-in templates +#### Built-in templates Built-in templates are project templates that are: -- Developed and maintained in the - [`project-templates`](https://gitlab.com/gitlab-org/project-templates) and [`pages`](https://gitlab.com/pages) groups. +- Developed and maintained in the [`project-templates`](https://gitlab.com/gitlab-org/project-templates) + and [`pages`](https://gitlab.com/pages) groups. - Released with GitLab. To use a built-in template on the **New project** page: @@ -62,22 +70,25 @@ To use a built-in template on the **New project** page: 1. From the list of available built-in templates, click the: - **Preview** button to look at the template source itself. - **Use template** button to start creating the project. -1. Finish creating the project by filling out the project's details. The process is the same as for - using a [blank project](#blank-projects). +1. Finish creating the project by filling out the project's details. The process is + the same as creating a [blank project](#blank-projects). TIP: **Tip:** -You can improve the existing built-in templates or contribute new ones on the -[`project-templates`](https://gitlab.com/gitlab-org/project-templates) and [`pages`](https://gitlab.com/pages) groups. +You can improve the existing built-in templates or contribute new ones in the +[`project-templates`](https://gitlab.com/gitlab-org/project-templates) and +[`pages`](https://gitlab.com/pages) groups. -### Custom project templates **(PREMIUM ONLY)** +#### Custom project templates **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/6860) in [GitLab Premium](https://about.gitlab.com/pricing) 11.2. -Creating new projects based on custom project templates is a convenient option to bootstrap a project. +Creating new projects based on custom project templates is a convenient option to +bootstrap a project. -Custom projects are available from the **Instance** or **Group** tabs under the **Create from template** tab, -depending on the type of template. +Custom projects are available at the [instance-level](../user/admin_area/custom_project_templates.md) +from the **Instance** tab, or at the [group-level](../user/group/custom_project_templates.md) +from the **Group** tab, under the **Create from template** tab. To use a custom project template on the **New project** page: @@ -85,23 +96,19 @@ To use a custom project template on the **New project** page: 1. From the list of available custom templates, click the: - **Preview** button to look at the template source itself. - **Use template** button to start creating the project. -1. Finish creating the project by filling out the project's details. The process is the same as for - using a [blank project](#blank-projects). - -For information on configuring custom project templates, see: - -- [Custom instance-level project templates](../user/admin_area/custom_project_templates.md), for instance-level templates. -- [Custom group-level project templates](../user/group/custom_project_templates.md), for group-level templates. +1. Finish creating the project by filling out the project's details. The process is + the same as creating a [blank project](#blank-projects). ## Push to create a new project > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/26388) in GitLab 10.5. -When you create a new repo locally, instead of going to GitLab to manually -create a new project and then push the repo, you can directly push it to -GitLab to create the new project, all without leaving your terminal. If you have access to that -namespace, we will automatically create a new project under that GitLab namespace with its -visibility set to Private by default (you can later change it in the [project's settings](../public_access/public_access.md#how-to-change-project-visibility)). +When you create a new repository locally, instead of going to GitLab to manually +create a new project and then [clone the repo](start-using-git.md#clone-a-repository) +locally, you can directly push it to GitLab to create the new project, all without leaving +your terminal. If you have access rights to the associated namespace, GitLab will +automatically create a new project under that GitLab namespace with its visibility +set to Private by default (you can later change it in the [project's settings](../public_access/public_access.md#how-to-change-project-visibility)). This can be done by using either SSH or HTTPS: @@ -127,3 +134,15 @@ remote: To view the project, visit: remote: https://gitlab.example.com/namespace/nonexistent-project remote: ``` + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/gitlab-basics/create-your-ssh-keys.md b/doc/gitlab-basics/create-your-ssh-keys.md index aac73d4c9c5..338b96374aa 100644 --- a/doc/gitlab-basics/create-your-ssh-keys.md +++ b/doc/gitlab-basics/create-your-ssh-keys.md @@ -1,22 +1,27 @@ -# Create and add your SSH public key +--- +type: howto +--- -This topic describes how to: +# Create and add your SSH public key -- Create an SSH key pair to use with GitLab. -- Add the SSH public key to your GitLab account. +It is best practice to use [Git over SSH instead of Git over HTTP](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols). +In order to use SSH, you will need to -You do this to use [Git over SSH instead of Git over HTTP](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols). +1. [Create an SSH key pair](#creating-your-ssh-key-pair) on your local computer. +1. [Add the key to GitLab](#adding-your-ssh-public-key-to-gitlab). ## Creating your SSH key pair -1. Go to your [command line](start-using-git.md). -1. Follow the [instructions](../ssh/README.md#generating-a-new-ssh-key-pair) to generate your SSH key pair. +1. Go to your [command line](start-using-git.md#open-a-shell). +1. Follow the [instructions](../ssh/README.md#generating-a-new-ssh-key-pair) to generate + your SSH key pair. ## Adding your SSH public key to GitLab -To add the SSH public key to GitLab, -see [Adding an SSH key to your GitLab account](../ssh/README.md#adding-an-ssh-key-to-your-gitlab-account). +To add the SSH public key to GitLab, see +[Adding an SSH key to your GitLab account](../ssh/README.md#adding-an-ssh-key-to-your-gitlab-account). NOTE: **Note:** -Once you add a key, you cannot edit it. If the paste -[didn't work](../ssh/README.md#testing-that-everything-is-set-up-correctly), you need to remove the key and re-add it. +Once you add a key, you cannot edit it. If it didn't paste properly, it +[will not work](../ssh/README.md#testing-that-everything-is-set-up-correctly), and +you will need to remove the key from GitLab and try adding it again. diff --git a/doc/gitlab-basics/fork-project.md b/doc/gitlab-basics/fork-project.md index a128a7c7dd3..5c19985121d 100644 --- a/doc/gitlab-basics/fork-project.md +++ b/doc/gitlab-basics/fork-project.md @@ -1,20 +1,11 @@ +--- +type: howto +--- + # How to fork a project A fork is a copy of an original repository that you put in another namespace where you can experiment and apply changes that you can later decide whether or not to share, without affecting the original project. -It takes just a few steps to fork a project in GitLab. - -1. Go to a project's dashboard under the **Project** tab and click on the - **Fork** button. - -  - -1. You will be asked where to fork the repository. Click on the user or group - to where you'd like to add the forked project. - -  - -1. After a few moments, depending on the repository's size, the forking will - complete. +It takes just a few steps to [fork a project in GitLab](../workflow/forking_workflow.md#creating-a-fork). diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md index b3c5d32f2f5..3e3f96fb31f 100644 --- a/doc/gitlab-basics/start-using-git.md +++ b/doc/gitlab-basics/start-using-git.md @@ -1,38 +1,43 @@ +--- +type: howto, tutorial +--- + # Start using Git on the command line -If you want to start using Git and GitLab, make sure that you have created and/or signed into an account on GitLab. +While GitLab has a powerful user interface, if you want to use Git itself, you will +have to do so from the command line. If you want to start using Git and GitLab together, +make sure that you have created and/or signed into an account on GitLab. ## Open a shell -Depending on your operating system, you will need to use a shell of your preference. Here are some suggestions: - -- [Terminal](http://blog.teamtreehouse.com/introduction-to-the-mac-os-x-command-line) on Mac OSX +Depending on your operating system, you will need to use a shell of your preference. +Here are some suggestions: +- [Terminal](http://blog.teamtreehouse.com/introduction-to-the-mac-os-x-command-line) on macOS - [GitBash](https://msysgit.github.io) on Windows - - [Linux Terminal](http://www.howtogeek.com/140679/beginner-geek-how-to-start-using-the-linux-terminal/) on Linux ## Check if Git has already been installed -Git is usually preinstalled on Mac and Linux. - -Type the following command and then press enter: +Git is usually preinstalled on Mac and Linux, so run the following command: ```bash git --version ``` -You should receive a message that tells you which Git version you have on your computer. If you don’t receive a "Git version" message, it means that you need to [download Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). +You should receive a message that tells you which Git version you have on your computer. +If you don’t receive a "Git version" message, it means that you need to +[download Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). -If Git doesn't automatically download, there's an option on the website to [download manually](https://git-scm.com/downloads). Then follow the steps on the installation window. - -After you are finished installing Git, open a new shell and type `git --version` again to verify that it was correctly installed. +After you are finished installing Git, open a new shell and type `git --version` again +to verify that it was correctly installed. ## Add your Git username and set your email -It is important to configure your Git username and email address, since every Git commit will use this information to identify you as the author. +It is important to configure your Git username and email address, since every Git +commit will use this information to identify you as the author. -On your shell, type the following command to add your username: +In your shell, type the following command to add your username: ```bash git config --global user.name "YOUR_USERNAME" @@ -56,7 +61,10 @@ To verify that you entered your email correctly, type: git config --global user.email ``` -You'll need to do this only once, since you are using the `--global` option. It tells Git to always use this information for anything you do on that system. If you want to override this with a different username or email address for specific projects, you can run the command without the `--global` option when you’re in that project. +You'll need to do this only once, since you are using the `--global` option. It tells +Git to always use this information for anything you do on that system. If you want +to override this with a different username or email address for specific projects, +you can run the command without the `--global` option when you’re in that project. ## Check your information @@ -68,12 +76,12 @@ git config --global --list ## Basic Git commands -Start using Git via the command line with the most basic -commands as described below. +Start using Git via the command line with the most basic commands as described below. -## Initialize a local directory for Git version control +### Initialize a local directory for Git version control -If you have an existing local directory that you want to *initialize* for version control, use the `init` command to instruct Git to begin tracking the directory: +If you have an existing local directory that you want to *initialize* for version +control, use the `init` command to instruct Git to begin tracking the directory: ```bash git init @@ -81,34 +89,33 @@ git init This creates a `.git` directory that contains the Git configuration files. -Once the directory has been initialized, you can [add a remote repository](#add-a-remote-repository) and [send changes to GitLab.com](#send-changes-to-gitlabcom). View the instructions on [Create a project](../gitlab-basics/create-project.html#push-to-create-a-new-project) to create a new project on GitLab with your changes. +Once the directory has been initialized, you can [add a remote repository](#add-a-remote-repository) +and [send changes to GitLab.com](#send-changes-to-gitlabcom). You will also need to +[create a new project in GitLab](../gitlab-basics/create-project.html#push-to-create-a-new-project) +for your Git repository. ### Clone a repository -To start working locally on an existing remote repository, -clone it with the command `git clone <repository path>`. -By cloning a repository, you'll download a copy of its -files into your local computer, preserving the Git -connection with the remote repository. +To start working locally on an existing remote repository, clone it with the command +`git clone <repository path>`. By cloning a repository, you'll download a copy of its +files to your local computer, automatically preserving the Git connection with the +remote repository. -You can either clone it via HTTPS or [SSH](../ssh/README.md). -If you chose to clone it via HTTPS, you'll have to enter your -credentials every time you pull and push. With SSH, you enter -your credentials once and can pull and push straightaway. +You can either clone it via HTTPS or [SSH](../ssh/README.md). If you chose to clone +it via HTTPS, you'll have to enter your credentials every time you pull and push. +With SSH, you enter your credentials only once. -You can find both paths (HTTPS and SSH) by navigating to -your project's landing page and clicking **Clone**. GitLab -will prompt you with both paths, from which you can copy +You can find both paths (HTTPS and SSH) by navigating to your project's landing page +and clicking **Clone**. GitLab will prompt you with both paths, from which you can copy and paste in your command line. -As an example, consider a repository path: +As an example, consider this repository path: - HTTPS: `https://gitlab.com/gitlab-org/gitlab-ce.git` -- SSH: `` git@gitlab.com:gitlab-org/gitlab-ce.git `` +- SSH: `git@gitlab.com:gitlab-org/gitlab-ce.git` -To get started, open a terminal window in the directory -you wish to clone the repository files into, and run one -of the following commands. +To get started, open a terminal window in the directory you wish to clone the repository +files into, and run one of the following commands. Clone via HTTPS: @@ -122,13 +129,15 @@ Clone via SSH: git clone git@gitlab.com:gitlab-org/gitlab-ce.git ``` -Both commands will download a copy of the files in a -folder named after the project's name. - -You can then navigate to the directory and start working +Both commands will download a copy of the files in a folder named after the project's +name. You can then navigate to the directory and start working on it locally. -### Go to the master branch to pull the latest changes from there +### Switch to the master branch + +You are always in a branch when working with Git. The main branch is the master branch, +but you can use the same command to switch to a different branch by changing `master` +to the branch name. ```bash git checkout master @@ -136,13 +145,20 @@ git checkout master ### Download the latest changes in the project -This is for you to work on an up-to-date copy (it is important to do this every time you start working on a project), while you set up tracking branches. You pull from remote repositories to get all the changes made by users since the last time you cloned or pulled the project. Later, you can push your local commits to the remote repositories. +To work on an up-to-date copy of the project (it is important to do this every time +you start working on a project), you `pull` to get all the changes made by users since +the last time you cloned or pulled the project. Use `master` for the `<name-of-branch>` +to get the main branch code, or the branch name of the branch you are currently working +in. ```bash -git pull REMOTE NAME-OF-BRANCH +git pull REMOTE <name-of-branch> ``` -When you first clone a repository, REMOTE is typically "origin". This is where the repository came from, and it indicates the SSH or HTTPS URL of the repository on the remote server. NAME-OF-BRANCH is usually "master", but it may be any existing branch. +When you first clone a repository, REMOTE is typically `origin`. This is where the +repository was cloned from, and it indicates the SSH or HTTPS URL of the repository +on the remote server. `<name-of-branch>` is usually `master`, but it may be any existing +branch. ### View your remote repositories @@ -157,17 +173,20 @@ git remote -v To add a link to a remote repository: ```bash -git remote add SOURCE-NAME REPOSITORY-PATH +git remote add <source-name> <repository-path> ``` -You'll use this source name every time you [push changes to GitLab.com](#send-changes-to-gitlabcom), so use something easy to remember and type. +You'll use this source name every time you [push changes to GitLab.com](#send-changes-to-gitlabcom), +so use something easy to remember and type. ### Create a branch -To create a branch, type the following (spaces won't be recognized in the branch name, so you will need to use a hyphen or underscore): +To create a new branch, to work from without affecting the `master` branch, type the +following (spaces won't be recognized in the branch name, so you will need to use a +hyphen or underscore): ```bash -git checkout -b NAME-OF-BRANCH +git checkout -b <name-of-branch>> ``` ### Work on an existing branch @@ -175,12 +194,14 @@ git checkout -b NAME-OF-BRANCH To switch to an existing branch, so you can work on it: ```bash -git checkout NAME-OF-BRANCH +git checkout <name-of-branch> ``` ### View the changes you've made -It's important to be aware of what's happening and the status of your changes. When you add, change, or delete files/folders, Git knows about it. To check the status of your changes: +It's important to be aware of what's happening and the status of your changes. When +you add, change, or delete files/folders, Git knows about it. To check the status of +your changes: ```bash git status @@ -188,7 +209,8 @@ git status ### View differences -To view the differences between your local, unstaged changes and the repository versions that you cloned or pulled, type: +To view the differences between your local, unstaged changes and the repository versions +that you cloned or pulled, type: ```bash git diff @@ -196,16 +218,19 @@ git diff ### Add and commit local changes -You'll see your local changes in red when you type `git status`. These changes may be new, modified, or deleted files/folders. Use `git add` to stage a local file/folder for committing. Then use `git commit` to commit the staged files: +You'll see any local changes in red when you type `git status`. These changes may +be new, modified, or deleted files/folders. Use `git add` to first stage (prepare) +a local file/folder for committing. Then use `git commit` to commit (save) the staged +files: ```bash -git add FILE OR FOLDER +git add <file-name OR folder-name> git commit -m "COMMENT TO DESCRIBE THE INTENTION OF THE COMMIT" ``` ### Add all changes to commit -To add and commit all local changes in one command: +To add and commit (save) all local changes quickly: ```bash git add . @@ -217,35 +242,32 @@ The `.` character typically means _all_ in Git. ### Send changes to GitLab.com -To push all local commits to the remote repository: +To push all local commits (saved changes) to the remote repository: ```bash -git push REMOTE NAME-OF-BRANCH +git push <remote> <name-of-branch> ``` -For example, to push your local commits to the _master_ branch of the _origin_ remote: +For example, to push your local commits to the _`master`_ branch of the _`origin`_ remote: ```bash git push origin master ``` -### Delete all changes in the Git repository +### Delete all changes in the branch -To delete all local changes in the repository that have not been added to the staging area, and leave unstaged files/folders, type: +To delete all local changes in the branch that have not been added to the staging +area, and leave unstaged files/folders, type: ```bash git checkout . ``` -### Delete all untracked changes in the Git repository - -```bash -git clean -f -``` +Note that this removes *changes* to files, not the files themselves. ### Unstage all changes that have been added to the staging area -To undo the most recent add, but not committed, files/folders: +To undo the most recently added, but not committed, changes to files/folders: ```bash git reset . @@ -259,25 +281,31 @@ To undo the most recent commit, type: git reset HEAD~1 ``` -This leaves the files and folders unstaged in your local repository. +This leaves the changed files and folders unstaged in your local repository. CAUTION: **Warning:** -A Git commit is mostly irreversible, particularly if you already pushed it to the remote repository. Although you can undo a commit, the best option is to avoid the situation altogether. +A Git commit should not usually be reverse, particularly if you already pushed it +to the remote repository. Although you can undo a commit, the best option is to avoid +the situation altogether by working carefully. -### Merge created branch with master branch +### Merge a branch with master branch -You need to be in the created branch. +When you are ready to make all the changes in a branch a permanent addition to +the master branch, you `merge` the two together: ```bash -git checkout NAME-OF-BRANCH +git checkout <name-of-branch> git merge master ``` -### Merge master branch with created branch +<!-- ## Troubleshooting -You need to be in the master branch. +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -```bash -git checkout master -git merge NAME-OF-BRANCH -``` +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/installation.md b/doc/install/installation.md index 06ec00cecc4..a12edef4e41 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -6,7 +6,7 @@ type: howto This is the official installation guide to set up a production GitLab server using the source files. To set up a **development installation** or for many -other installation options, see the [main installation page](index.md). +other installation options, see the [main installation page](README.md). It was created for and tested on **Debian/Ubuntu** operating systems. Read [requirements.md](requirements.md) for hardware and operating system requirements. If you want to install on RHEL/CentOS, we recommend using the @@ -938,7 +938,7 @@ To use GitLab with Puma: cd /home/git/gitlab # Copy config file for the web server - sudo -u git -H config/puma.rb.example config/puma.rb + sudo -u git -H cp config/puma.rb.example config/puma.rb ``` 1. Edit the system `init.d` script to use `EXPERIMENTAL_PUMA=1` flag. If you have `/etc/default/gitlab`, then you should edit it instead. diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 25ab608de3a..cfabc09646d 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -62,17 +62,19 @@ NOTE: **Note:** Since file system performance may affect GitLab's overall perfor ### CPU +This is the recommended minimum hardware for a handful of example GitLab user base sizes. Your exact needs may be more, depending on your workload. Your workload is influenced by factors such as - but not limited to - how active your users are, how much automation you use, mirroring, and repo/change size. + - 1 core supports up to 100 users but the application can be a bit slower due to having all workers and background jobs running on the same core -- **2 cores** is the **recommended** number of cores and supports up to 500 users -- 4 cores supports up to 2,000 users -- 8 cores supports up to 5,000 users -- 16 cores supports up to 10,000 users -- 32 cores supports up to 20,000 users -- 64 cores supports up to 40,000 users -- More users? Run it on [multiple application servers](https://about.gitlab.com/high-availability/) +- **2 cores** is the **recommended** minimum number of cores and supports up to 100 users +- 4 cores supports up to 500 users +- 8 cores supports up to 1,000 users +- 32 cores supports up to 5,000 users +- More users? Run it high-availability on [multiple application servers](https://about.gitlab.com/high-availability/) ### Memory +This is the recommended minimum hardware for a handful of example GitLab user base sizes. Your exact needs may be more, depending on your workload. Your workload is influenced by factors such as - but not limited to - how active your users are, how much automation you use, mirroring, and repo/change size. + You need at least 8GB of addressable memory (RAM + swap) to install and use GitLab! The operating system and any other running applications will also be using memory so keep in mind that you need at least 4GB available before running GitLab. With @@ -80,13 +82,11 @@ less memory GitLab will give strange errors during the reconfigure run and 500 errors during usage. - 4GB RAM + 4GB swap supports up to 100 users but it will be very slow -- **8GB RAM** is the **recommended** memory size for all installations and supports up to 100 users -- 16GB RAM supports up to 2,000 users -- 32GB RAM supports up to 4,000 users -- 64GB RAM supports up to 8,000 users -- 128GB RAM supports up to 16,000 users -- 256GB RAM supports up to 32,000 users -- More users? Run it on [multiple application servers](https://about.gitlab.com/high-availability/) +- **8GB RAM** is the **recommended** minimum memory size for all installations and supports up to 100 users +- 16GB RAM supports up to 500 users +- 32GB RAM supports up to 1,000 users +- 128GB RAM supports up to 5,000 users +- More users? Run it high-availability on [multiple application servers](https://about.gitlab.com/high-availability/) We recommend having at least [2GB of swap on your server](https://askubuntu.com/a/505344/310789), even if you currently have enough available RAM. Having swap will help reduce the chance of errors occurring @@ -146,8 +146,8 @@ CREATE EXTENSION postgres_fdw; ## Unicorn Workers -For most instances we recommend using: CPU cores + 1 = unicorn workers. -So for a machine with 2 cores, 3 unicorn workers is ideal. +For most instances we recommend using: (CPU cores * 1.5) + 1 = unicorn workers. +For example a node with 4 cores would have 7 unicorn workers. For all machines that have 2GB and up we recommend a minimum of three unicorn workers. If you have a 1GB machine we recommend to configure only two Unicorn workers to prevent excessive swapping. diff --git a/doc/integration/elasticsearch.md b/doc/integration/elasticsearch.md index 626bd259ed6..1c80fc543af 100644 --- a/doc/integration/elasticsearch.md +++ b/doc/integration/elasticsearch.md @@ -24,18 +24,21 @@ special searches: ## Installing Elasticsearch Elasticsearch is _not_ included in the Omnibus packages. You will have to -install it yourself whether you are using the Omnibus package or installed -GitLab from source. Providing detailed information on installing Elasticsearch -is out of the scope of this document. +[install it yourself](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html "Elasticsearch installation documentation") +whether you are using the Omnibus package or installed GitLab from source. +Providing detailed information on installing Elasticsearch is out of the scope +of this document. + +NOTE: **Note:** +Elasticsearch should be installed on a separate server, whether you install +it yourself or by using the +[Amazon Elasticsearch](http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html) +service. Running Elasticsearch on the same server as GitLab is not recommended +and it will likely cause performance degradation on the GitLab installation. Once the data is added to the database or repository and [Elasticsearch is enabled in the admin area](#enabling-elasticsearch) the search index will be -updated automatically. Elasticsearch can be installed on the same machine as -GitLab or on a separate server, or you can use the [Amazon Elasticsearch](http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg.html) -service. - -You can follow the steps as described in the [official web site](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html "Elasticsearch installation documentation") or -use the packages that are available for your OS. +updated automatically. ## Elasticsearch repository indexer (beta) diff --git a/doc/integration/jenkins.md b/doc/integration/jenkins.md index 50cb3d50009..6755640d39c 100644 --- a/doc/integration/jenkins.md +++ b/doc/integration/jenkins.md @@ -35,6 +35,9 @@ and [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF For a real use case, read the blog post [Continuous integration: From Jenkins to GitLab using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/). +NOTE: **Moving from a traditional CI plug-in to a single application for the entire software development lifecycle can decrease hours spent on maintaining toolchains by 10% or more.** +Visit the ['GitLab vs. Jenkins' comparison page](https://about.gitlab.com/devops-tools/jenkins-vs-gitlab.html) to learn how our built-in CI compares to Jenkins. + ## Requirements - [Jenkins GitLab Plugin](https://wiki.jenkins.io/display/JENKINS/GitLab+Plugin) diff --git a/doc/public_access/public_access.md b/doc/public_access/public_access.md index d04428fdbfe..ea4702acc41 100644 --- a/doc/public_access/public_access.md +++ b/doc/public_access/public_access.md @@ -37,7 +37,7 @@ visibility setting keep this setting. You can read more about the change in the ### Private projects -Private projects can only be cloned and viewed by project members. +Private projects can only be cloned and viewed by project members (except for guests). They will appear in the public access directory (`/public`) for project members only. diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 0d86df04367..f8da09e5fe1 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -960,3 +960,22 @@ want to run the chown against your custom location instead of [reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure [restart GitLab]: ../administration/restart_gitlab.md#installations-from-source + +### Backup fails to complete with Gzip error + +While running the backup, you may receive a gzip error: + +```sh +sudo /opt/gitlab/bin/gitlab-rake gitlab:backup:create +Dumping ... +... +gzip: stdout: Input/output error + +Backup failed +``` + +If this happens, check the following: + +1. Confirm there is sufficent diskspace for the gzip operation. +1. If NFS is being used, check if the mount option `timeo` is set. The default is `600`, and changing this to smaller values have resulted in this error. + diff --git a/doc/security/README.md b/doc/security/README.md index c48d5bc2065..5d498ac7602 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -7,7 +7,7 @@ type: index - [Password length limits](password_length_limits.md) - [Restrict SSH key technologies and minimum length](ssh_keys_restrictions.md) -- [Rack attack](rack_attack.md) +- [Rate limits](rate_limits.md) - [Webhooks and insecure internal web services](webhooks.md) - [Information exclusivity](information_exclusivity.md) - [Reset your root password](reset_root_password.md) diff --git a/doc/security/img/outbound_requests_section.png b/doc/security/img/outbound_requests_section.png Binary files differdeleted file mode 100644 index f7783f34cdd..00000000000 --- a/doc/security/img/outbound_requests_section.png +++ /dev/null diff --git a/doc/security/img/outbound_requests_section_v12_2.png b/doc/security/img/outbound_requests_section_v12_2.png Binary files differnew file mode 100644 index 00000000000..4fd3c7d9fce --- /dev/null +++ b/doc/security/img/outbound_requests_section_v12_2.png diff --git a/doc/security/rack_attack.md b/doc/security/rack_attack.md index 1e5678ec47c..c772f783f71 100644 --- a/doc/security/rack_attack.md +++ b/doc/security/rack_attack.md @@ -2,7 +2,9 @@ type: reference, howto --- -# Rack Attack +# Rack Attack initializer + +## Overview [Rack Attack](https://github.com/kickstarter/rack-attack), also known as Rack::Attack, is a Ruby gem that is meant to protect GitLab with the ability to customize throttling and @@ -14,19 +16,72 @@ If you find throttling is not enough to protect you against abusive clients, Rack Attack offers IP whitelisting, blacklisting, Fail2ban style filtering, and tracking. -**Note:** Starting with 11.2, Rack Attack is disabled by default. To continue -using Rack Attack, please enable it by [configuring `gitlab.rb` as described in Settings](#settings). +For more information on how to use these options see the [Rack Attack README](https://github.com/kickstarter/rack-attack/blob/master/README.md). + +NOTE: **Note:** See +[User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md) +for simpler throttles that are configured in UI. + +NOTE: **Note:** Starting with 11.2, Rack Attack is disabled by default. If your +instance is not exposed to the public internet, it is recommended that you leave +Rack Attack disabled. + +## Behavior + +If set up as described in the [Settings](#settings) section below, two behaviors +will be enabled: + +- Protected paths will be throttled +- Failed authentications for Git and container registry requests will trigger a temporary IP ban + +### Protected paths throttle + +GitLab responds with HTTP status code 429 to POST requests at protected paths +over 10 requests per minute per IP address. + +By default, protected paths are: + +```ruby +default['gitlab']['gitlab-rails']['rack_attack_protected_paths'] = [ + '/users/password', + '/users/sign_in', + '/api/#{API::API.version}/session.json', + '/api/#{API::API.version}/session', + '/users', + '/users/confirmation', + '/unsubscribes/', + '/import/github/personal_access_token' +] +``` + +This header is included in responses to blocked requests: + +``` +Retry-After: 60 +``` + +For example, the following are limited to a maximum 10 requests per minute: + +- user sign-in +- user sign-up (if enabled) +- user password reset + +After trying for 10 times, the client will +have to wait a minute before to be able to try again. + +### Git and container registry failed authentication ban + +GitLab responds with HTTP status code 403 for 1 hour, if 30 failed +authentication requests were received in a 3-minute period from a single IP address. -By default, user sign-in, user sign-up (if enabled), and user password reset is -limited to 6 requests per minute. After trying for 6 times, the client will -have to wait for the next minute to be able to try again. +This applies only to Git requests and container registry (`/jwt/auth`) requests +(combined). -If you installed or upgraded GitLab by following the [official guides](../install/README.md), -Rack Attack should be disabled by default. If your instance is not exposed to any incoming -connections, it is recommended that you leave Rack Attack disabled. +This limit is reset by requests that authenticate successfully. For example, 29 +failed authentication requests followed by 1 successful request, followed by 29 +more failed authentication requests would not trigger a ban. -For more information on how to use these options check out -[rack-attack README](https://github.com/kickstarter/rack-attack/blob/master/README.md). +No response headers are provided. ## Settings diff --git a/doc/security/rate_limits.md b/doc/security/rate_limits.md new file mode 100644 index 00000000000..0e5bdcd9c79 --- /dev/null +++ b/doc/security/rate_limits.md @@ -0,0 +1,32 @@ +--- +type: reference, howto +--- + +# Rate limits + +NOTE: **Note:** +For GitLab.com, please see +[GitLab.com-specific rate limits](../user/gitlab_com/index.md#gitlabcom-specific-rate-limits). + +Rate limiting is a common technique used to improve the security and durability +of a web application. + +For example, a simple script can make thousands of web requests per second. +Whether malicious, apathetic, or just a bug, your application and infrastructure +may not be able to cope with the load. For more details, see +[Denial-of-service attack](https://en.wikipedia.org/wiki/Denial-of-service_attack). +Most cases can be mitigated by limiting the rate of requests from a single IP address. + +Most [brute-force attacks](https://en.wikipedia.org/wiki/Brute-force_attack) are +similarly mitigated by a rate limit. + +## Admin Area settings + +See +[User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md). + +## Rack Attack initializer + +This method of rate limiting is cumbersome, but has some advantages. It allows +throttling of specific paths, and is also integrated into Git and container +registry requests. See [Rack Attack initializer](rack_attack.md). diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md index 1194234a295..7ece9407ac0 100644 --- a/doc/security/webhooks.md +++ b/doc/security/webhooks.md @@ -34,15 +34,16 @@ to 127.0.0.1, ::1 and 0.0.0.0, as well as IPv4 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 and IPv6 site-local (ffc0::/10) addresses won't be allowed. This behavior can be overridden by enabling the option *"Allow requests to the -local network from hooks and services"* in the *"Outbound requests"* section +local network from web hooks and services"* in the *"Outbound requests"* section inside the Admin area under **Settings** (`/admin/application_settings/network`): - + ->**Note:** -*System hooks* are exempt from this protection because they are set up by -admins. +NOTE: **Note:** +*System hooks* are enabled to make requests to local network by default since they are +set up by administrators. However, you can turn this off by disabling the +**Allow requests to the local network from system hooks** option. <!-- ## Troubleshooting diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index c6219126b30..63a72f8f193 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -87,6 +87,10 @@ Auto DevOps provides great defaults for all the stages; you can, however, For an overview on the creation of Auto DevOps, read the blog post [From 2/3 of the Self-Hosted Git Market, to the Next-Generation CI System, to Auto DevOps](https://about.gitlab.com/2017/06/29/whats-next-for-gitlab-ci/). +NOTE: **Note** +Kubernetes clusters can [be used without](../../user/project/clusters/index.md) +Auto DevOps. + ## Requirements To make full use of Auto DevOps, you will need: @@ -721,47 +725,34 @@ The following variables can be used for setting up the Auto DevOps domain, providing a custom Helm chart, or scaling your application. PostgreSQL can also be customized, and you can easily use a [custom buildpack](#custom-buildpacks). +#### Build and deployment + +The following table lists variables related to building and deploying +applications. + | **Variable** | **Description** | |-----------------------------------------|------------------------------------| -| `AUTO_DEVOPS_CHART` | The Helm Chart used to deploy your apps; defaults to the one [provided by GitLab](https://gitlab.com/gitlab-org/charts/auto-deploy-app). | -| `AUTO_DEVOPS_CHART_REPOSITORY` | The Helm Chart repository used to search for charts; defaults to `https://charts.gitlab.io`. | -| `AUTO_DEVOPS_CHART_REPOSITORY_NAME` | From Gitlab 11.11, this variable can be used to set the name of the helm repository; defaults to "gitlab" | -| `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From Gitlab 11.11, this variable can be used to set a username to connect to the helm repository. Defaults to no credentials. (Also set AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD) | -| `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From Gitlab 11.11, this variable can be used to set a password to connect to the helm repository. Defaults to no credentials. (Also set AUTO_DEVOPS_CHART_REPOSITORY_USERNAME) | -| `REPLICAS` | The number of replicas to deploy; defaults to 1. | -| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment. Takes precedence over `REPLICAS` and defaults to 1. For zero downtime upgrades, set to 2 or greater. | -| `CANARY_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md); defaults to 1. | -| `CANARY_PRODUCTION_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md) in the production environment. This takes precedence over `CANARY_REPLICAS`; defaults to 1. | | `ADDITIONAL_HOSTS` | Fully qualified domain names specified as a comma-separated list that are added to the ingress hosts. | | `<ENVIRONMENT>_ADDITIONAL_HOSTS` | For a specific environment, the fully qualified domain names specified as a comma-separated list that are added to the ingress hosts. This takes precedence over `ADDITIONAL_HOSTS`. | -| `POSTGRES_ENABLED` | Whether PostgreSQL is enabled; defaults to `"true"`. Set to `false` to disable the automatic deployment of PostgreSQL. | -| `POSTGRES_USER` | The PostgreSQL user; defaults to `user`. Set it to use a custom username. | -| `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. | -| `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-environment-variables). Set it to use a custom database name. | -| `POSTGRES_VERSION` | Tag for the [`postgres` Docker image](https://hub.docker.com/_/postgres) to use. Defaults to `9.6.2`. | -| `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142`. | -| `SAST_CONFIDENCE_LEVEL` | The minimum confidence level of security issues you want to be reported; `1` for Low, `2` for Medium, `3` for High; defaults to `3`. | -| `DS_DISABLE_REMOTE_CHECKS` | Whether remote Dependency Scanning checks are disabled; defaults to `"false"`. Set to `"true"` to disable checks that send data to GitLab central servers. [Read more about remote checks](../../user/application_security/dependency_scanning/index.md#remote-checks). | -| `DB_INITIALIZE` | From GitLab 11.4, this variable can be used to specify the command to run to initialize the application's PostgreSQL database. It runs inside the application pod. | -| `DB_MIGRATE` | From GitLab 11.4, this variable can be used to specify the command to run to migrate the application's PostgreSQL database. It runs inside the application pod. | -| `STAGING_ENABLED` | From GitLab 10.8, this variable can be used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). | -| `CANARY_ENABLED` | From GitLab 11.0, this variable can be used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments-premium). | -| `INCREMENTAL_ROLLOUT_MODE` | From GitLab 11.4, this variable, if present, can be used to enable an [incremental rollout](#incremental-rollout-to-production-premium) of your application for the production environment. Set to `manual` for manual deployment jobs or `timed` for automatic rollout deployments with a 5 minute delay each one. | -| `TEST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `test` job. If the variable is present, the job will not be created. | -| `CODE_QUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `codequality` job. If the variable is present, the job will not be created. | -| `LICENSE_MANAGEMENT_DISABLED` | From GitLab 11.0, this variable can be used to disable the `license_management` job. If the variable is present, the job will not be created. | -| `SAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast` job. If the variable is present, the job will not be created. | -| `DEPENDENCY_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dependency_scanning` job. If the variable is present, the job will not be created. | -| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast:container` job. If the variable is present, the job will not be created. | -| `REVIEW_DISABLED` | From GitLab 11.0, this variable can be used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs will not be created. | -| `DAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dast` job. If the variable is present, the job will not be created. | -| `PERFORMANCE_DISABLED` | From GitLab 11.0, this variable can be used to disable the `performance` job. If the variable is present, the job will not be created. | +| `AUTO_DEVOPS_CHART` | Helm Chart used to deploy your apps. Defaults to the one [provided by GitLab](https://gitlab.com/gitlab-org/charts/auto-deploy-app). | +| `AUTO_DEVOPS_CHART_REPOSITORY` | Helm Chart repository used to search for charts. Defaults to `https://charts.gitlab.io`. | +| `AUTO_DEVOPS_CHART_REPOSITORY_NAME` | From Gitlab 11.11, used to set the name of the helm repository. Defaults to `gitlab`. | +| `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From Gitlab 11.11, used to set a username to connect to the helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD`. | +| `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From Gitlab 11.11, used to set a password to connect to the helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME`. | +| `BUILDPACK_URL` | Buildpack's full URL. Can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`. For example `https://github.com/heroku/heroku-buildpack-ruby.git#v142`. | +| `CANARY_ENABLED` | From GitLab 11.0, used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments-premium). | +| `CANARY_PRODUCTION_REPLICAS` | Number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md) in the production environment. Takes precedence over `CANARY_REPLICAS`. Defaults to 1. | +| `CANARY_REPLICAS` | Number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md). Defaults to 1. | +| `HELM_RELEASE_NAME` | From GitLab 12.1, allows the `helm` release name to be overridden. Can be used to assign unique release names when deploying multiple projects to a single namespace. | +| `HELM_UPGRADE_EXTRA_ARGS` | From GitLab 11.11, allows extra arguments in `helm` commands when deploying the application. Note that using quotes will not prevent word splitting. **Tip:** you can use this variable to [customize the Auto Deploy helm chart](#custom-helm-chart) by applying custom override values with `--values my-values.yaml`. | +| `INCREMENTAL_ROLLOUT_MODE` | From GitLab 11.4, if present, can be used to enable an [incremental rollout](#incremental-rollout-to-production-premium) of your application for the production environment. Set to `manual` for manual deployment jobs or `timed` for automatic rollout deployments with a 5 minute delay each one. | | `K8S_SECRET_*` | From GitLab 11.7, any variable prefixed with [`K8S_SECRET_`](#application-secret-variables) will be made available by Auto DevOps as environment variables to the deployed application. | -| `KUBE_INGRESS_BASE_DOMAIN` | From GitLab 11.8, this variable can be used to set a domain per cluster. See [cluster domains](../../user/project/clusters/index.md#base-domain) for more information. | -| `ROLLOUT_RESOURCE_TYPE` | From GitLab 11.9, this variable allows specification of the resource type being deployed when using a custom helm chart. Default value is `deployment`. | -| `ROLLOUT_STATUS_DISABLED` | From GitLab 12.0, this variable allows to disable rollout status check because it doesn't support all resource types, for example, `cronjob`. | -| `HELM_UPGRADE_EXTRA_ARGS` | From GitLab 11.11, this variable allows extra arguments in `helm` commands when deploying the application. Note that using quotes will not prevent word splitting. **Tip:** you can use this variable to [customize the Auto Deploy helm chart](#custom-helm-chart) by applying custom override values with `--values my-values.yaml`. | -| `HELM_RELEASE_NAME` | From GitLab 12.1, this variable allows the `helm` release name to be overridden, this can be used to assign unique release names when deploying multiple projects to a single namespace. | +| `KUBE_INGRESS_BASE_DOMAIN` | From GitLab 11.8, can be used to set a domain per cluster. See [cluster domains](../../user/project/clusters/index.md#base-domain) for more information. | +| `PRODUCTION_REPLICAS` | Number of replicas to deploy in the production environment. Takes precedence over `REPLICAS` and defaults to 1. For zero downtime upgrades, set to 2 or greater. | +| `REPLICAS` | Number of replicas to deploy. Defaults to 1. | +| `ROLLOUT_RESOURCE_TYPE` | From GitLab 11.9, allows specification of the resource type being deployed when using a custom helm chart. Default value is `deployment`. | +| `ROLLOUT_STATUS_DISABLED` | From GitLab 12.0, used to disable rollout status check because it doesn't support all resource types, for example, `cronjob`. | +| `STAGING_ENABLED` | From GitLab 10.8, used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). | TIP: **Tip:** Set up the replica variables using a @@ -773,6 +764,42 @@ You should *not* scale your application using Kubernetes directly. This can cause confusion with Helm not detecting the change, and subsequent deploys with Auto DevOps can undo your changes. +#### Database + +The following table lists variables related to the database. + +| **Variable** | **Description** | +| `DB_INITIALIZE` | From GitLab 11.4, used to specify the command to run to initialize the application's PostgreSQL database. Runs inside the application pod. | +| `DB_MIGRATE` | From GitLab 11.4, used to specify the command to run to migrate the application's PostgreSQL database. Runs inside the application pod. | +| `POSTGRES_ENABLED` | Whether PostgreSQL is enabled. Defaults to `"true"`. Set to `false` to disable the automatic deployment of PostgreSQL. | +| `POSTGRES_USER` | The PostgreSQL user. Defaults to `user`. Set it to use a custom username. | +| `POSTGRES_PASSWORD` | The PostgreSQL password. Defaults to `testing-password`. Set it to use a custom password. | +| `POSTGRES_DB` | The PostgreSQL database name. Defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-environment-variables). Set it to use a custom database name. | +| `POSTGRES_VERSION` | Tag for the [`postgres` Docker image](https://hub.docker.com/_/postgres) to use. Defaults to `9.6.2`. | + +#### Security tools + +The following table lists variables related to security tools. + +| **Variable** | **Description** | +| `SAST_CONFIDENCE_LEVEL` | Minimum confidence level of security issues you want to be reported; `1` for Low, `2` for Medium, `3` for High. Defaults to `3`. | +| `DS_DISABLE_REMOTE_CHECKS` | Whether remote Dependency Scanning checks are disabled. Defaults to `"false"`. Set to `"true"` to disable checks that send data to GitLab central servers. [Read more about remote checks](../../user/application_security/dependency_scanning/index.md#remote-checks). | + +#### Disable jobs + +The following table lists variables used to disable jobs. + +| **Variable** | **Description** | +| `CODE_QUALITY_DISABLED` | From GitLab 11.0, used to disable the `codequality` job. If the variable is present, the job will not be created. | +| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, used to disable the `sast:container` job. If the variable is present, the job will not be created. | +| `DAST_DISABLED` | From GitLab 11.0, used to disable the `dast` job. If the variable is present, the job will not be created. | +| `DEPENDENCY_SCANNING_DISABLED` | From GitLab 11.0, used to disable the `dependency_scanning` job. If the variable is present, the job will not be created. | +| `LICENSE_MANAGEMENT_DISABLED` | From GitLab 11.0, used to disable the `license_management` job. If the variable is present, the job will not be created. | +| `PERFORMANCE_DISABLED` | From GitLab 11.0, used to disable the `performance` job. If the variable is present, the job will not be created. | +| `REVIEW_DISABLED` | From GitLab 11.0, used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs will not be created. | +| `SAST_DISABLED` | From GitLab 11.0, used to disable the `sast` job. If the variable is present, the job will not be created. | +| `TEST_DISABLED` | From GitLab 11.0, used to disable the `test` job. If the variable is present, the job will not be created. | + #### Application secret variables > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/49056) in GitLab 11.7. @@ -1042,6 +1069,12 @@ Container Registry. **Restarting a pod, scaling a service, or other actions whic require on-going access to the registry may fail**. On-going secure access is planned for a subsequent release. +### Private registry support + +There is no documented way of using private container registry with Auto DevOps. +We strongly advise using GitLab Container Registry with Auto DevOps in order to +simplify configuration and prevent any unforeseen issues. + ## Troubleshooting - Auto Build and Auto Test may fail in detecting your language/framework. There diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index cdcd8215b23..6a539b526f3 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -48,6 +48,7 @@ The following are resources about version control concepts: The following resources may help you become more efficient at using Git: +- [Useful Git commands](useful_git_commands.md) collected by the GitLab support team. - [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) - [Eight Tips to help you work better with Git](https://about.gitlab.com/2015/02/19/8-tips-to-help-you-work-better-with-git/) @@ -71,6 +72,7 @@ The following are advanced topics for those who want to get the most out of Git: - [Custom Git Hooks](../../administration/custom_hooks.md) - [Git Attributes](../../user/project/git_attributes.md) - Git Submodules: [Using Git submodules with GitLab CI](../../ci/git_submodules.md#using-git-submodules-with-gitlab-ci) +- [Partial Clone](partial_clone.md) ## API @@ -82,6 +84,8 @@ Git-related queries from GitLab. The following relate to Git Large File Storage: - [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) -- [GitLab Git LFS documentation](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) +- [Migrate an existing Git repo with Git LFS](migrate_to_git_lfs/index.md) +- [GitLab Git LFS user documentation](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) +- [GitLab Git LFS admin documentation](../../workflow/lfs/lfs_administration.md) - [Git-Annex to Git-LFS migration guide](../../workflow/lfs/migrate_from_git_annex_to_git_lfs.md) - [Towards a production quality open source Git LFS server](https://about.gitlab.com/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) diff --git a/doc/topics/git/migrate_to_git_lfs/index.md b/doc/topics/git/migrate_to_git_lfs/index.md new file mode 100644 index 00000000000..c879e404997 --- /dev/null +++ b/doc/topics/git/migrate_to_git_lfs/index.md @@ -0,0 +1,174 @@ +--- +type: tutorial, concepts +description: "How to migrate an existing Git repository to Git LFS with BFG." +last_updated: 2019-07-11 +--- + +# Migrate a Git repo into Git LFS with BFG + +Using Git LFS can help you to reduce the size of your Git +repository and improve its performance. + +However, simply adding the +large files that are already in your repository to Git LFS, +will not actually reduce the size of your repository because +the files are still referenced by previous commits. + +Through the method described on this document, first migrate +to Git LFS with [BFG](https://rtyley.github.io/bfg-repo-cleaner/) +through a mirror repo, then clean up the repository's history, +and lastly create LFS tracking rules to prevent new binary files +from being added. + +This tutorial was inspired by the guide +[Use BFG to migrate a repo to Git LFS](https://confluence.atlassian.com/bitbucket/use-bfg-to-migrate-a-repo-to-git-lfs-834233484.html). +For more information on Git LFS, see the [references](#references) +below. + +CAUTION: **Warning:** +The method described on this guide rewrites Git history. Make +sure to back up your repo before beginning and use it at your +own risk. + +## Requirements + +Before beginning, make sure: + +- You have enough LFS storage for the files you want to convert. + Storage is required for the entire history of all files. +- All the team members you share the repository with have pushed all changes. + Branches based on the repository before applying this method cannot be merged. + Branches based on the repo before applying this method cannot be merged. + +To follow this tutorial, you'll need: + +- Maintainer permissions to the existing Git repository + you'd like to migrate to LFS with access through the command line. +- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) + and [Java Runtime Environment](https://www.java.com/en/download/manual.jsp) + (Java 7 or above) installed locally. +- BFG installed locally: + + ```bash + brew install bfg + ``` + +- Git LFS installed locally: + + ```bash + brew install git-lfs + ``` + +NOTE: **Note:** +This guide was tested on macOS Mojave. + +## Steps + +Consider an example upstream project, `git@gitlab.com:gitlab-tests/test-git-lfs-repo-migration.git`. + +1. Back up your repository: + + Create a copy of your repository so that you can + recover it in case something goes wrong. + +1. Clone `--mirror` the repo: + + Cloning with the mirror flag will create a bare repository. + This ensures you get all the branches within the repo. + + It creates a directory called `<repo-name>.git` + (in our example, `test-git-lfs-repo-migration.git`), + mirroring the upstream project: + + ```bash + git clone --mirror git@gitlab.com:gitlab-tests/test-git-lfs-repo-migration.git + ``` + +1. Convert the Git history with BFG: + + ```bash + bfg --convert-to-git-lfs "*.{png,mp4,jpg,gif}" --no-blob-protection test-git-lfs-repo-migration.git + ``` + + It is scanning all the history, and looking for any files with + that extension, and then converting them to an LFS pointer. + +1. Clean up the repository: + + ```bash + # cd path/to/mirror/repo: + cd test-git-lfs-repo-migration.git + # clean up the repo: + git reflog expire --expire=now --all && git gc --prune=now --aggressive + ``` + + You can also take a look on how to further [clean the repo](../../../user/project/repository/reducing_the_repo_size_using_git.md), + but it's not necessary for the purposes of this guide. + +1. Install Git LFS in the mirror repository: + + ```bash + git lfs install + ``` + +1. [Unprotect the default branch](../../../user/project/protected_branches.md), + so that we can force-push the rewritten repository: + + 1. Navigate to your project's **Settings > Repository** and + expand **Protected Branches**. + 1. Scroll down to locate the protected branches and click + **Unprotect** the default branch. + +1. Force-push to GitLab: + + ```bash + git push --force + ``` + +1. Track the files you want with LFS: + + ```bash + # cd path/to/upstream/repo: + cd test-git-lfs-repo-migration + # You may need to reset your local copy with upstream's `master` after force-pushing from the mirror: + git reset --hard origin/master + # Track the files with LFS: + git lfs track "*.gif" "*.png" "*.jpg" "*.psd" "*.mp4" ".gitattributes" "img/" + ``` + + Now all existing the files you converted, as well as the new + ones you add, will be properly tracked with LFS. + +1. [Re-protect the default branch](../../../user/project/protected_branches.md): + + 1. Navigate to your project's **Settings > Repository** and + expand **Protected Branches**. + 1. Select the default branch from the **Branch** dropdown menu, + and set up the + **Allowed to push** and **Allowed to merge** rules. + 1. Click **Protect**. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + +## References + +- [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) +- [Migrate from Git Annex to Git LFS](../../../workflow/lfs/migrate_from_git_annex_to_git_lfs.md) +- [GitLab's Git LFS user documentation](../../../workflow/lfs/manage_large_binaries_with_git_lfs.md) +- [GitLab's Git LFS administrator documentation](../../../workflow/lfs/lfs_administration.md) +- Alternative method to [migrate an existing repo to Git LFS](https://github.com/git-lfs/git-lfs/wiki/Tutorial#migrating-existing-repository-data-to-lfs) + +<!-- +Test project: +https://gitlab.com/gitlab-tests/test-git-lfs-repo-migration +--> diff --git a/doc/topics/git/numerous_undo_possibilities_in_git/index.md b/doc/topics/git/numerous_undo_possibilities_in_git/index.md index 84201e11831..5cae532bf54 100644 --- a/doc/topics/git/numerous_undo_possibilities_in_git/index.md +++ b/doc/topics/git/numerous_undo_possibilities_in_git/index.md @@ -110,21 +110,21 @@ At this point there are 3 options to undo the local changes you have: - Discard all local changes, but save them for possible re-use [later](#quickly-save-local-changes): - ```shell - git stash - ``` + ```shell + git stash + ``` - Discarding local changes (permanently) to a file: - ```shell - git checkout -- <file> - ``` + ```shell + git checkout -- <file> + ``` - Discard all local changes to all files permanently: - ```shell - git reset --hard - ``` + ```shell + git reset --hard + ``` Before executing `git reset --hard`, keep in mind that there is also a way to just temporary store the changes without committing them using `git stash`. @@ -182,27 +182,27 @@ Now you have 4 options to undo your changes: - Unstage the file to current commit (HEAD): - ```shell - git reset HEAD <file> - ``` + ```shell + git reset HEAD <file> + ``` - Unstage everything - retain changes: - ```shell - git reset - ``` + ```shell + git reset + ``` - Discard all local changes, but save them for [later](#quickly-save-local-changes): - ```shell - git stash - ``` + ```shell + git stash + ``` - Discard everything permanently: - ```shell - git reset --hard - ``` + ```shell + git reset --hard + ``` ## Committed local changes @@ -240,21 +240,21 @@ In our example we will end up with commit `B`, that introduced bug/error. We hav - Undo (swap additions and deletions) changes introduced by commit `B`: - ```shell - git revert commit-B-id - ``` + ```shell + git revert commit-B-id + ``` - Undo changes on a single file or directory from commit `B`, but retain them in the staged state: - ```shell - git checkout commit-B-id <file> - ``` + ```shell + git checkout commit-B-id <file> + ``` - Undo changes on a single file or directory from commit `B`, but retain them in the unstaged state: - ```shell - git reset commit-B-id <file> - ``` + ```shell + git reset commit-B-id <file> + ``` - There is one command we also must not forget: **creating a new branch** from the point where changes are not applicable or where the development has hit a @@ -270,14 +270,14 @@ In our example we will end up with commit `B`, that introduced bug/error. We hav you can [cherry-pick](../../../user/project/merge_requests/cherry_pick_changes.md#cherry-picking-a-commit) that commit into a new merge request. -  +  - ```shell - git checkout commit-B-id - git checkout -b new-path-of-feature - # Create <commit F> - git commit -a - ``` + ```shell + git checkout commit-B-id + git checkout -b new-path-of-feature + # Create <commit F> + git commit -a + ``` ### With history modification @@ -297,9 +297,9 @@ delete commit `B`. - Rebase the range from current commit D to A: - ```shell - git rebase -i A - ``` + ```shell + git rebase -i A + ``` - Command opens your favorite editor where you write `drop` in front of commit `B`, but you leave default `pick` with all other commits. Save and exit the @@ -310,9 +310,9 @@ In case you want to modify something introduced in commit `B`. - Rebase the range from current commit D to A: - ```shell - git rebase -i A - ``` + ```shell + git rebase -i A + ``` - Command opens your favorite text editor where you write `edit` in front of commit `B`, but leave default `pick` with all other commits. Save and exit the editor to @@ -320,9 +320,9 @@ In case you want to modify something introduced in commit `B`. - Now do your edits and commit changes: - ```shell - git commit -a - ``` + ```shell + git commit -a + ``` You can find some more examples in [below section where we explain how to modify history](#how-modifying-history-is-done) diff --git a/doc/topics/git/partial_clone.md b/doc/topics/git/partial_clone.md new file mode 100644 index 00000000000..f2951308ba1 --- /dev/null +++ b/doc/topics/git/partial_clone.md @@ -0,0 +1,147 @@ +# Partial Clone for Large Repositories + +CAUTION: **Alpha:** +Partial Clone is an experimental feature, and will significantly increase +Gitaly resource utilization when performing a partial clone, and decrease +performance of subsequent fetch operations. + +As Git repositories become very large, usability decreases as performance +decreases. One major challenge is cloning the repository, because Git will +download the entire repository including every commit and every version of +every object. This can be slow to transfer, and require large amounts of disk +space. + +Historically, performing a **shallow clone** +([`--depth`](https://www.git-scm.com/docs/git-clone#Documentation/git-clone.txt---depthltdepthgt)) +has been the only way to reduce the amount of data transferred when cloning +a Git repository. This does not, however, allow filtering by sub-tree which is +important for monolithic repositories containing many projects, or by object +size preventing unnecessary large objects being downloaded. + +[Partial clone](https://github.com/git/git/blob/master/Documentation/technical/partial-clone.txt) +is a performance optimization that "allows Git to function without having a +complete copy of the repository. The goal of this work is to allow Git better +handle extremely large repositories." + +Specifically, using partial clone, it should be possible for Git to natively +support: + +- large objects, instead of using [Git LFS](https://git-lfs.github.com/) +- enormous repositories + +Briefly, partial clone works by: + +- excluding objects from being transferred when cloning or fetching a + repository using a new `--filter` flag +- downloading missing objects on demand + +Follow [Git for enormous repositories](https://gitlab.com/groups/gitlab-org/-/epics/773) for roadmap and updates. + +## Enabling partial clone + +GitLab 12.1 uses Git 2.21.0 which has an arbitrary file access security +vulnerability when `uploadpack.allowFilter` is enabled, and should not be +enabled in production environments. + +A feature flag is planned to enable `uploadpack.allowFilter` and +`uploadpack.allowAnySHA1InWant` once the version of Git used by GitLab has been +updated to Git 2.22.0. + +Follow [this issue](https://gitlab.com/gitlab-org/gitaly/issues/1553) for +updated. + +## Excluding objects by size + +Partial Clone allows large objects to be stored directly in the Git repository, +and be excluded from clones as desired by the user. This eliminates the error +prone process of deciding which objects should be stored in LFS or not. Using +partial clone, all files – large or small – may be treated the same. + +With the `uploadpack.allowFilter` and `uploadpack.allowAnySHA1InWant` options +enabled on the Git server: + +```bash +# clone the repo, excluding blobs larger than 1 megabyte +git clone --filter=blob:limit=1m <url> + +# in the checkout step of the clone, and any subsequent operations +# any blobs that are needed will be downloaded on demand +git checkout feature-branch +``` + +## Excluding objects by path + +Partial Clone allows clones to be filtered by path using a format similar to a +`.gitignore` file stored inside the repository. + +With the `uploadpack.allowFilter` and `uploadpack.allowAnySHA1InWant` options +enabled on the Git server: + +1. **Create a filter spec.** For example, consider a monolithic repository with + many applications, each in a different subdirectory in the root. Create a file + `shiny-app/.filterspec` using the GitLab web interface: + + ```.gitignore + # Only the paths listed in the file will be downloaded when performing a + # partial clone using `--filter=sparse:oid=shiny-app/.gitfilterspec` + + # Explicitly include filterspec needed to configure sparse checkout with + # git config --local core.sparsecheckout true + # git show master:snazzy-app/.gitfilterspec >> .git/info/sparse-checkout + shiny-app/.gitfilterspec + + # Shiny App + shiny-app/ + + # Dependencies + shimmery-app/ + shared-component-a/ + shared-component-b/ + ``` + +2. *Create a new Git repository and fetch.* Support for `--filter=sparse:oid` + using the clone command is incomplete, so we will emulate the clone command + by hand, using `git init` and `git fetch`. Follow + [gitaly#1769](https://gitlab.com/gitlab-org/gitaly/issues/1769) for updates. + + ```bash + # Create a new directory for the Git repository + mkdir jumbo-repo && cd jumbo-repo + + # Initialize a new Git repository + git init + + # Add the remote + git remote add origin git@gitlab.com/example/jumbo-repo + + # Enable partial clone support for the remote + git config --local extensions.partialClone origin + + # Fetch the filtered set of objects using the filterspec stored on the + # server. WARNING: this step is slow! + git fetch --filter=sparse:oid=master:shiny-app/.gitfilterspec origin + + # Optional: observe there are missing objects that we have not fetched + git rev-list --all --quiet --objects --missing=print | wc -l + ``` + + CAUTION: **IDE and Shell integrations:** + Git integrations with `bash`, `zsh`, etc and editors that automatically + show Git status information often run `git fetch` which will fetch the + entire repository. You many need to disable or reconfigure these + integrations. + +3. **Sparse checkout** must be enabled and configured to prevent objects from + other paths being downloaded automatically when checking out branches. Follow + [gitaly#1765](https://gitlab.com/gitlab-org/gitaly/issues/1765) for updates. + + ```bash + # Enable sparse checkout + git config --local core.sparsecheckout true + + # Configure sparse checkout + git show master:snazzy-app/.gitfilterspec >> .git/info/sparse-checkout + + # Checkout master + git checkout master + ``` diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index 417d91bf834..11284da30af 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -51,11 +51,11 @@ Configuring *both* the client and the server is unnecessary. - On UNIX, edit `~/.ssh/config` (create the file if it doesn’t exist) and add or edit: - ```text - Host your-gitlab-instance-url.com - ServerAliveInterval 60 - ServerAliveCountMax 5 - ``` + ```text + Host your-gitlab-instance-url.com + ServerAliveInterval 60 + ServerAliveCountMax 5 + ``` - On Windows, if you are using PuTTY, go to your session properties, then navigate to "Connection" and under "Sending of null packets to keep diff --git a/doc/topics/git/useful_git_commands.md b/doc/topics/git/useful_git_commands.md new file mode 100644 index 00000000000..84406805350 --- /dev/null +++ b/doc/topics/git/useful_git_commands.md @@ -0,0 +1,210 @@ +--- +type: reference +--- + +# Useful Git commands + +Here are some useful Git commands collected by the GitLab support team. You may not +need to use often, but they can can come in handy when needed. + +## Remotes + +### Add another URL to a remote, so both remotes get updated on each push + +```sh +git remote set-url --add <remote_name> <remote_url> +``` + +## Staging and reverting changes + +### Remove last commit and leave the changes in unstaged + +```sh +git reset --soft HEAD^ +``` + +### Unstage a certain number of commits from HEAD + +To unstage 3 commits, for example, run: + +```sh +git reset HEAD^3 +``` + +### Unstage changes to a certain file from HEAD + +```sh +git reset <filename> +``` + +### Revert a file to HEAD state and remove changes + +There are two options to revert changes to a file: + +- `git checkout <filename>` +- `git reset --hard <filename>` + +### Undo a previous commit by creating a new replacement commit + +```sh +git revert <commit-sha> +``` + +### Create a new message for last commit + +```sh +git commit --amend +``` + +### Add a file to the last commit + +```sh +git add <filename> +git commit --amend +``` + +Append `--no-edit` to the `commit` command if you do not want to edit the commit +message. + +## Stashing + +### Stash changes + +```sh +git stash save +``` + +The default behavor of `stash` is to save, so you can also use just: + +```sh +git stash +``` + +### Unstash your changes + +```sh +git stash apply +``` + +### Discard your stashed changes + +```sh +git stash drop +``` + +### Apply and drop your stashed changes + +```sh +git stash pop +``` + +## Refs and Log + +### Use reflog to show the log of reference changes to HEAD + +```sh +git reflog +``` + +### Check the Git history of a file + +The basic command to check the git history of a file: + +```sh +git log <file> +``` + +If you get this error message: + +```text +fatal: ambiguous argument <file_name>: unknown revision or path not in the working tree. +Use '--' to separate paths from revisions, like this: +``` + +Use this to check the Git history of the file: + +```sh +git log -- <file> +``` + +### Find the tags that contain a particular SHA + +```sh +git tag --contains <sha> +``` + +### Check the content of each change to a file + +```sh +gitk <file> +``` + +### Check the content of each change to a file, follows it past file renames + +```sh +gitk --follow <file> +``` + +## Debugging + +### Use a custom SSH key for a git command + +```text +GIT_SSH_COMMAND="ssh -i ~/.ssh/gitlabadmin" git <command> +``` + +### Debug cloning + +With SSH: + +```text +GIT_SSH_COMMAND="ssh -vvv" git clone <git@url> +``` + +With HTTPS: + +```text +GIT_TRACE_PACKET=1 GIT_TRACE=2 GIT_CURL_VERBOSE=1 git clone <url> +``` + +## Rebasing + +### Rebase your branch onto master + +The -i flag stands for 'interactive': + +```sh +git rebase -i master +``` + +### Continue the rebase if paused + +```sh +git rebase --continue +``` + +### Use git rerere + +To _reuse_ recorded solutions to the same problems when repeated: + +```sh +git rerere +``` + +To enable `rerere` functionality: + +```sh +git config --global rerere.enabled true +``` + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/university/training/end-user/README.md b/doc/university/training/end-user/README.md index 99fb5e83387..423ba1cfbd7 100644 --- a/doc/university/training/end-user/README.md +++ b/doc/university/training/end-user/README.md @@ -54,6 +54,7 @@ project. --- ## Git Setup + Workshop Time! --- @@ -88,7 +89,7 @@ git config --global --list ``` - You might want or be required to use an SSH key. - - Instructions: [SSH](http://doc.gitlab.com/ce/ssh/README.html) + - Instructions: [SSH](http://doc.gitlab.com/ce/ssh/README.html) --- @@ -119,13 +120,13 @@ cd ~/workspace ### Git Workflow - Untracked files - - New files that Git has not been told to track previously. + - New files that Git has not been told to track previously. - Working area (Workspace) - - Files that have been modified but are not committed. + - Files that have been modified but are not committed. - Staging area (Index) - - Modified files that have been marked to go in the next commit. + - Modified files that have been marked to go in the next commit. - Upstream - - Hosted repository on a shared server + - Hosted repository on a shared server --- @@ -229,8 +230,6 @@ git push origin squash_some_bugs --- -### Feedback and Collaboration - - Review the Thoughtbot code-review guide for suggestions to follow when reviewing merge requests:[Thoughtbot](https://github.com/thoughtbot/guides/tree/master/code-review) - See GitLab merge requests for examples: [Merge Requests](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests) @@ -266,20 +265,22 @@ git push origin squash_some_bugs ### Example 1/2 - git checkout -b conflicts_branch +```sh +git checkout -b conflicts_branch - # vi conflicts.rb - # Add 'Line4' and 'Line5' +# vi conflicts.rb +# Add 'Line4' and 'Line5' - git commit -am "add line4 and line5" - git push origin conflicts_branch +git commit -am "add line4 and line5" +git push origin conflicts_branch - git checkout master +git checkout master - # vi conflicts.rb - # Add 'Line6' and 'Line7' - git commit -am "add line6 and line7" - git push origin master +# vi conflicts.rb +# Add 'Line6' and 'Line7' +git commit -am "add line6 and line7" +git push origin master +``` --- @@ -287,20 +288,22 @@ git push origin squash_some_bugs Create a merge request on the GitLab web UI. You'll see a conflict warning. - git checkout conflicts_branch - git fetch - git rebase master +```sh +git checkout conflicts_branch +git fetch +git rebase master - # Fix conflicts by editing the files. +# Fix conflicts by editing the files. - git add conflicts.rb - # No need to commit this file +git add conflicts.rb +# No need to commit this file - git rebase --continue +git rebase --continue - # Remember that we have rewritten our commit history so we - # need to force push so that our remote branch is restructured - git push origin conflicts_branch -f +# Remember that we have rewritten our commit history so we +# need to force push so that our remote branch is restructured +git push origin conflicts_branch -f +``` --- @@ -321,20 +324,28 @@ Create a merge request on the GitLab web UI. You'll see a conflict warning. To remove files from stage use reset HEAD. Where HEAD is the last commit of the current branch: - git reset HEAD <file> +```sh +git reset HEAD <file> +``` This will unstage the file but maintain the modifications. To revert the file back to the state it was in before the changes we can use: - git checkout -- <file> +```sh +git checkout -- <file> +``` To remove a file from disk and repo use 'git rm' and to rm a dir use the '-r' flag: - git rm '*.txt' - git rm -r <dirname> +```sh +git rm '*.txt' +git rm -r <dirname> +``` If we want to remove a file from the repository but keep it on disk, say we forgot to add it to our .gitignore file then use `--cache`: - git rm <filename> --cache +```sh +git rm <filename> --cache +``` --- @@ -342,19 +353,27 @@ If we want to remove a file from the repository but keep it on disk, say we forg Undo last commit putting everything back into the staging area: - git reset --soft HEAD^ +```sh +git reset --soft HEAD^ +``` Add files and change message with: - git commit --amend -m "New Message" +```sh +git commit --amend -m "New Message" +``` Undo last and remove changes - git reset --hard HEAD^ +```sh +git reset --hard HEAD^ +``` Same as last one but for two commits back: - git reset --hard HEAD^^ +```sh +git reset --hard HEAD^^ +``` Don't reset after pushing @@ -373,35 +392,38 @@ Don't reset after pushing 1. Pull for updates 1. Push changes ----- +--- - # Change file edit_this_file.rb - git status - git commit -am "kjkfjkg" - git log - git commit --amend -m "New comment added" - git log - git reset --soft HEAD^ - git log - git pull origin master - git push origin master +```sh +# Change file edit_this_file.rb +git status +git commit -am "kjkfjkg" +git log +git commit --amend -m "New comment added" +git log +git reset --soft HEAD^ +git log +git pull origin master +git push origin master +``` --- -### Note +### git revert vs git reset -git revert vs git reset Reset removes the commit while revert removes the changes but leaves the commit Revert is safer considering we can revert a revert - # Changed file - git commit -am "bug introduced" - git revert HEAD - # New commit created reverting changes - # Now we want to re apply the reverted commit - git log # take hash from the revert commit - git revert <rev commit hash> - # reverted commit is back (new commit created again) +```sh +# Changed file +git commit -am "bug introduced" +git revert HEAD +# New commit created reverting changes +# Now we want to re apply the reverted commit +git log # take hash from the revert commit +git revert <rev commit hash> +# reverted commit is back (new commit created again) +``` --- @@ -415,11 +437,11 @@ Revert is safer considering we can revert a revert ### Version Control - - Local VCS was used with a filesystem or a simple db. - - Centralized VCS such as Subversion includes collaboration but - still is prone to data loss as the main server is the single point of - failure. - - Distributed VCS enables the team to have a complete copy of the project - and work with little dependency to the main server. In case of a main - server failing the project can be recovered by any of the latest copies - from the team +- Local VCS was used with a filesystem or a simple db. +- Centralized VCS such as Subversion includes collaboration but + still is prone to data loss as the main server is the single point of + failure. +- Distributed VCS enables the team to have a complete copy of the project + and work with little dependency to the main server. In case of a main + server failing the project can be recovered by any of the latest copies + from the team diff --git a/doc/university/training/topics/git_add.md b/doc/university/training/topics/git_add.md index 4c61d5eb175..7152fc2030b 100644 --- a/doc/university/training/topics/git_add.md +++ b/doc/university/training/topics/git_add.md @@ -10,32 +10,32 @@ Adds content to the index or staging area. - Adds a list of file: - ```bash - git add <files> - ``` + ```bash + git add <files> + ``` - Adds all files including deleted ones: - ```bash - git add -A - ``` + ```bash + git add -A + ``` ## Git add continued - Add all text files in current dir: - ```bash - git add *.txt - ``` + ```bash + git add *.txt + ``` - Add all text file in the project: - ```bash - git add "*.txt*" - ``` + ```bash + git add "*.txt*" + ``` - Adds all files in directory: - ```bash - git add views/layouts/ - ``` + ```bash + git add views/layouts/ + ``` diff --git a/doc/university/training/topics/git_log.md b/doc/university/training/topics/git_log.md index 11addcd3ee1..bae734554f5 100644 --- a/doc/university/training/topics/git_log.md +++ b/doc/university/training/topics/git_log.md @@ -8,33 +8,33 @@ Git log lists commit history. It allows searching and filtering. - Initiate log: - ```sh - git log - ``` + ```sh + git log + ``` - Retrieve set number of records: - ```sh - git log -n 2 - ``` + ```sh + git log -n 2 + ``` - Search commits by author. Allows user name or a regular expression. - ```sh - git log --author="user_name" - ``` + ```sh + git log --author="user_name" + ``` - Search by comment message: - ```sh - git log --grep="<pattern>" - ``` + ```sh + git log --grep="<pattern>" + ``` - Search by date: - ```sh - git log --since=1.month.ago --until=3.weeks.ago - ``` + ```sh + git log --since=1.month.ago --until=3.weeks.ago + ``` ## Git Log Workflow diff --git a/doc/university/training/topics/rollback_commits.md b/doc/university/training/topics/rollback_commits.md index 1e6602deef2..c17e8d59737 100644 --- a/doc/university/training/topics/rollback_commits.md +++ b/doc/university/training/topics/rollback_commits.md @@ -8,29 +8,29 @@ comments: false - Undo last commit putting everything back into the staging area: - ```sh - git reset --soft HEAD^ - ``` + ```sh + git reset --soft HEAD^ + ``` - Add files and change message with: - ```sh - git commit --amend -m "New Message" - ``` + ```sh + git commit --amend -m "New Message" + ``` - Undo last and remove changes: - ```sh - git reset --hard HEAD^ - ``` + ```sh + git reset --hard HEAD^ + ``` - Same as last one but for two commits back: - ```sh - git reset --hard HEAD^^ - ``` + ```sh + git reset --hard HEAD^^ + ``` -** Don't reset after pushing ** +**Don't reset after pushing** ## Reset Workflow diff --git a/doc/university/training/topics/stash.md b/doc/university/training/topics/stash.md index 02b2f610266..21abad88cfa 100644 --- a/doc/university/training/topics/stash.md +++ b/doc/university/training/topics/stash.md @@ -9,47 +9,47 @@ and we need to change to a different branch. - Stash: - ```sh - git stash save - # or - git stash - # or with a message - git stash save "this is a message to display on the list" - ``` + ```sh + git stash save + # or + git stash + # or with a message + git stash save "this is a message to display on the list" + ``` - Apply stash to keep working on it: - ```sh - git stash apply - # or apply a specific one from out stack - git stash apply stash@{3} - ``` + ```sh + git stash apply + # or apply a specific one from out stack + git stash apply stash@{3} + ``` - Every time we save a stash it gets stacked so by using list we can see all our stashes. - ```sh - git stash list - # or for more information (log methods) - git stash list --stat - ``` + ```sh + git stash list + # or for more information (log methods) + git stash list --stat + ``` - To clean our stack we need to manually remove them: - ```sh - # drop top stash - git stash drop - # or - git stash drop <name> - # to clear all history we can use - git stash clear - ``` + ```sh + # drop top stash + git stash drop + # or + git stash drop <name> + # to clear all history we can use + git stash clear + ``` - Apply and drop on one command: - ```sh - git stash pop - ``` + ```sh + git stash pop + ``` - If we meet conflicts we need to either reset or commit our changes. - Conflicts through `pop` will not drop a stash afterwards. diff --git a/doc/university/training/topics/unstage.md b/doc/university/training/topics/unstage.md index af16afdc5d1..fa1f63f9ec4 100644 --- a/doc/university/training/topics/unstage.md +++ b/doc/university/training/topics/unstage.md @@ -6,27 +6,27 @@ comments: false ## Unstage -- To remove files from stage use reset HEAD where HEAD is the last commit of the current branch. This will unstage the file but maintain the modifications. +- To remove files from stage use reset HEAD where HEAD is the last commit of the current branch. This will unstage the file but maintain the modifications. - ```bash - git reset HEAD <file> - ``` + ```bash + git reset HEAD <file> + ``` - To revert the file back to the state it was in before the changes we can use: - ```bash - git checkout -- <file> - ``` + ```bash + git checkout -- <file> + ``` - To remove a file from disk and repo use 'git rm' and to rm a dir use the '-r' flag: - ```sh - git rm '*.txt' - git rm -r <dirname> - ``` + ```sh + git rm '*.txt' + git rm -r <dirname> + ``` - If we want to remove a file from the repository but keep it on disk, say we forgot to add it to our `.gitignore` file then use `--cache`: - ```sh - git rm <filename> --cache - ``` + ```sh + git rm <filename> --cache + ``` diff --git a/doc/update/mysql_to_postgresql.md b/doc/update/mysql_to_postgresql.md index 1e424134242..f6a1b6abdbf 100644 --- a/doc/update/mysql_to_postgresql.md +++ b/doc/update/mysql_to_postgresql.md @@ -72,15 +72,15 @@ need to enable the bundled PostgreSQL: 1. Stop GitLab: - ```bash - sudo gitlab-ctl stop - ``` + ```bash + sudo gitlab-ctl stop + ``` 1. Edit `/etc/gitlab/gitlab.rb` to enable bundled PostgreSQL: - ``` - postgresql['enable'] = true - ``` + ``` + postgresql['enable'] = true + ``` 1. Edit `/etc/gitlab/gitlab.rb` to use the bundled PostgreSQL. Please check all the settings beginning with `db_`, such as `gitlab_rails['db_adapter']` @@ -91,22 +91,22 @@ need to enable the bundled PostgreSQL: for the changes to take effect. 1. Start Unicorn and PostgreSQL so that we can prepare the schema: - ```bash - sudo gitlab-ctl start unicorn - sudo gitlab-ctl start postgresql - ``` + ```bash + sudo gitlab-ctl start unicorn + sudo gitlab-ctl start postgresql + ``` 1. Run the following commands to prepare the schema: - ```bash - sudo gitlab-rake db:create db:migrate - ``` + ```bash + sudo gitlab-rake db:create db:migrate + ``` 1. Stop Unicorn to prevent other database access from interfering with the loading of data: - ```bash - sudo gitlab-ctl stop unicorn - ``` + ```bash + sudo gitlab-ctl stop unicorn + ``` After these steps, you'll have a fresh PostgreSQL database with up-to-date schema. @@ -116,57 +116,57 @@ new PostgreSQL one: 1. Save the following snippet in a `commands.load` file, and edit with your MySQL database `username`, `password` and `host`: - ``` - LOAD DATABASE - FROM mysql://username:password@host/gitlabhq_production - INTO postgresql://gitlab-psql@unix://var/opt/gitlab/postgresql:/gitlabhq_production + ``` + LOAD DATABASE + FROM mysql://username:password@host/gitlabhq_production + INTO postgresql://gitlab-psql@unix://var/opt/gitlab/postgresql:/gitlabhq_production - WITH include no drop, truncate, disable triggers, create no tables, - create no indexes, preserve index names, no foreign keys, - data only + WITH include no drop, truncate, disable triggers, create no tables, + create no indexes, preserve index names, no foreign keys, + data only - ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public' + ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public' - ; - ``` + ; + ``` 1. Start the migration: - ```bash - sudo -u gitlab-psql pgloader commands.load - ``` + ```bash + sudo -u gitlab-psql pgloader commands.load + ``` 1. Once the migration finishes, you should see a summary table that looks like the following: - ``` - table name read imported errors total time - ----------------------------------------------- --------- --------- --------- -------------- - fetch meta data 119 119 0 0.388s - Truncate 119 119 0 1.134s - ----------------------------------------------- --------- --------- --------- -------------- - public.abuse_reports 0 0 0 0.490s - public.appearances 0 0 0 0.488s - . - . - . - public.web_hook_logs 0 0 0 1.080s - ----------------------------------------------- --------- --------- --------- -------------- - COPY Threads Completion 4 4 0 2.008s - Reset Sequences 113 113 0 0.304s - Install Comments 0 0 0 0.000s - ----------------------------------------------- --------- --------- --------- -------------- - Total import time 1894 1894 0 12.497s - ``` - - If there is no output for more than 30 minutes, it's possible `pgloader` encountered an error. See - the [troubleshooting guide](#troubleshooting) for more details. + ``` + table name read imported errors total time + ----------------------------------------------- --------- --------- --------- -------------- + fetch meta data 119 119 0 0.388s + Truncate 119 119 0 1.134s + ----------------------------------------------- --------- --------- --------- -------------- + public.abuse_reports 0 0 0 0.490s + public.appearances 0 0 0 0.488s + . + . + . + public.web_hook_logs 0 0 0 1.080s + ----------------------------------------------- --------- --------- --------- -------------- + COPY Threads Completion 4 4 0 2.008s + Reset Sequences 113 113 0 0.304s + Install Comments 0 0 0 0.000s + ----------------------------------------------- --------- --------- --------- -------------- + Total import time 1894 1894 0 12.497s + ``` + + If there is no output for more than 30 minutes, it's possible `pgloader` encountered an error. See + the [troubleshooting guide](#troubleshooting) for more details. 1. Start GitLab: - ```bash - sudo gitlab-ctl start - ``` + ```bash + sudo gitlab-ctl start + ``` You can now verify that everything works as expected by visiting GitLab. diff --git a/doc/user/admin_area/settings/img/user_and_ip_rate_limits.png b/doc/user/admin_area/settings/img/user_and_ip_rate_limits.png Binary files differnew file mode 100644 index 00000000000..2bd85a2fd96 --- /dev/null +++ b/doc/user/admin_area/settings/img/user_and_ip_rate_limits.png diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index 5427d04cd7d..2a12614e325 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -18,6 +18,7 @@ include: - [Third party offers](third_party_offers.md) - [Usage statistics](usage_statistics.md) - [Visibility and access controls](visibility_and_access_controls.md) +- [User and IP rate limits](user_and_ip_rate_limits.md) - [Custom templates repository](instance_template_repository.md) **(PREMIUM)** NOTE: **Note:** diff --git a/doc/user/admin_area/settings/user_and_ip_rate_limits.md b/doc/user/admin_area/settings/user_and_ip_rate_limits.md new file mode 100644 index 00000000000..e3a495750f2 --- /dev/null +++ b/doc/user/admin_area/settings/user_and_ip_rate_limits.md @@ -0,0 +1,32 @@ +--- +type: reference +--- + +# User and IP rate limits + +Rate limiting is a common technique used to improve the security and durability +of a web application. For more details, see +[Rate limits](../../../security/rate_limits.md). + +The following limits can be enforced in **Admin Area > Network > User and +IP rate limits**: + +- Unauthenticated requests +- Authenticated API requests +- Authenticated web requests + +These limits are disabled by default. + + + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 88e2d1ef22b..fa84f995e58 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -7,6 +7,11 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/4348) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4. +NOTE: **4 of the top 6 attacks were application based.** +Download our whitepaper, +["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) +to learn how to protect your organization. + Running [static checks](../sast/index.md) on your code is the first step to detect vulnerabilities that can put the security of your code at risk. Yet, once deployed, your application is exposed to a new category of possible attacks, diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 6a810757a28..10b4d9d4c7c 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -52,14 +52,15 @@ The following languages and dependency managers are supported. | Language (package managers) | Supported | Scan tool(s) | |----------------------------- | --------- | ------------ | +| Java ([Gradle](https://gradle.org/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/13075 "Dependency Scanning for Gradle" )) | not available | +| Java ([Maven](https://maven.apache.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | JavaScript ([npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/en/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js) | +| Go ([Golang](https://golang.org/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/7132 "Dependency Scanning for Go")) | not available | +| PHP ([Composer](https://getcomposer.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | Python ([pip](https://pip.pypa.io/en/stable/)) (only `requirements.txt` supported) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Python ([Pipfile](https://docs.pipenv.org/en/latest/basics/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/11756 "Pipfile.lock support for Dependency Scanning"))| not available | +| Python ([poetry](https://poetry.eustace.io/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/7006 "Support Poetry in Dependency Scanning")) | not available | | Ruby ([gem](https://rubygems.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | -| Java ([Maven](https://maven.apache.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| PHP ([Composer](https://getcomposer.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Python ([poetry](https://poetry.eustace.io/)) | no ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/7006 "Support Poetry in Dependency Scanning")) | not available | -| Python ([Pipfile](https://docs.pipenv.org/en/latest/basics/)) | no ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/11756 "Pipfile.lock support for Dependency Scanning"))| not available | -| Go ([Golang](https://golang.org/)) | no ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/7132 "Dependency Scanning for Go")) | not available | ## Remote checks diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 31f0b5a050c..4dcb416c110 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -148,6 +148,38 @@ Clicking on this button will create a merge request to apply the solution onto t  +## Security approvals in merge requests **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9928) in [GitLab Ultimate](https://about.gitlab.com/pricing) 12.2. + +Merge Request Approvals can be configured to require approval from a member +of your security team when a vulnerability would be introduced by a merge request. + +This threshold is defined as `high`, `critical`, or `unknown` +severity. When any vulnerabilities are present within a merge request, an +approval will be required from the `Vulnerability-Check` approver group. + +### Enabling Security Approvals within a project + +To enable Security Approvals, a [project approval rule](../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) +must be created with the case-sensitive name `Vulnerability-Check`. This approval +group must be set with an "Approvals required" count greater than zero. + +Once this group has been added to your project, the approval rule will be enabled +for all Merge Requests. + +Any code changes made will cause the count of approvals required to reset. + +An approval will be required when a security report: + +- Contains a new vulnerability of `high`, `critical`, or `unknown` severity. +- Is not generated during pipeline execution. + +An approval will be optional when a security report: + +- Contains no new vulnerabilities. +- Contains only new vulnerabilities of `low` or `medium` severity. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 59835aeba01..cb533538047 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -29,6 +29,7 @@ SAST supports the following official analyzers: - [Security Code Scan (.NET)](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) - [TSLint (Typescript)](https://gitlab.com/gitlab-org/security-products/analyzers/tslint) - [Sobelow (Elixir Phoenix)](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) +- [PMD (Apex only)](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) The analyzers are published as Docker images that SAST will use to launch dedicated containers for each analysis. @@ -116,24 +117,24 @@ custom analyzer can scan the source code. ## Analyzers Data -| Property \ Tool | Bandit | Brakeman | ESLint security | Find Sec Bugs | Flawfinder | Go AST Scanner | NodeJsScan | Php CS Security Audit | Security code Scan (.NET) | TSLint Security | Sobelow | -| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :-------------: | :----------------: | -| Severity | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | -| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Description | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | -| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| End line | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | -| Start column | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | -| End column | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | -| External id (e.g. CVE) | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| URLs | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal doc/explanation | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| Solution | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Confidence | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| Affected item (e.g. class or package) | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Source code extract | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | +| Property \ Tool | Apex | Bandit | Brakeman | ESLint security | Find Sec Bugs | Flawfinder | Go AST Scanner | NodeJsScan | Php CS Security Audit | Security code Scan (.NET) | TSLint Security | Sobelow | +| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :-------------: | :----------------: | +| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | 𐄂 | +| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | +| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | +| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | +| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | +| External id (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | - ✓ => we have that data - ⚠ => we have that data but it's partially reliable, or we need to extract it from unstructured content diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index aac881112ff..5149f628345 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -59,6 +59,7 @@ The following table shows which languages, package managers and frameworks are s |-----------------------------------------------------------------------------|----------------------------------------------------------------------------------------|------------------------------| | .NET | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | | Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11.9 | +| Apex (Salesforce) | [pmd](https://pmd.github.io/pmd/index.html) | 12.1 | | C/C++ | [Flawfinder](https://www.dwheeler.com/flawfinder/) | 10.7 | | Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10 | | Go | [Gosec](https://github.com/securego/gosec) | 10.7 | diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index e1a6f8fcf71..c0106b2cb9e 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -95,7 +95,7 @@ CI/CD](../../ci/README.md), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security -implications](../project/clusters/index.md/#security-implications) before doing so. +implications](../project/clusters/index.md#security-implications) before doing so. NOTE: **Note:** The @@ -252,9 +252,12 @@ The applications below can be uninstalled. | Application | GitLab version | Notes | | ----------- | -------------- | ----- | +| Cert-Manager | 12.2+ | The associated private key will be deleted and cannot be restored. Deployed applications will continue to use HTTPS, but certificates will not be renewed. Before uninstalling, you may wish to [back up your configuration](https://docs.cert-manager.io/en/latest/tasks/backup-restore-crds.html) or [revoke your certificates](https://letsencrypt.org/docs/revoking/) | | GitLab Runner | 12.2+ | Any running pipelines will be canceled. | +| Helm | 12.2+ | The associated Tiller pod will be deleted and cannot be restored. | | Ingress | 12.1+ | The associated load balancer and IP will be deleted and cannot be restored. Furthermore, it can only be uninstalled if JupyterHub is not installed. | | JupyterHub | 12.1+ | All data not committed to GitLab will be deleted and cannot be restored. | +| Knative | 12.1+ | The associated IP will be deleted and cannot be restored. | | Prometheus | 11.11+ | All data will be deleted and cannot be restored. | To uninstall an application: diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 7858c419e04..e6c27c33654 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -298,6 +298,79 @@ Web front-ends: - `memory_limit_min` = 1024MiB - `memory_limit_max` = 1280MiB +## GitLab.com-specific rate limits + +NOTE: **Note:** +See [Rate limits](../../security/rate_limits.md) for administrator +documentation. + +IP blocks usually happen when GitLab.com receives unusual traffic from a single +IP address that the system views as potentially malicious based on rate limit +settings. After the unusual traffic ceases, the IP address will be automatically +released depending on the type of block, as described below. + +If you receive a `403 Forbidden` error for all requests to GitLab.com, please +check for any automated processes that may be triggering a block. For +assistance, contact [GitLab Support](https://support.gitlab.com) +with details, such as the affected IP address. + +### HAProxy API throttle + +GitLab.com responds with HTTP status code 429 to API requests over 10 requests +per second per IP address. + +The following example headers are included for all API requests: + +``` +RateLimit-Limit: 600 +RateLimit-Observed: 6 +RateLimit-Remaining: 594 +RateLimit-Reset: 1563325137 +RateLimit-ResetTime: Wed, 17 Jul 2019 00:58:57 GMT +``` + +Source: + +- Search for `rate_limit_http_rate_per_minute` and `rate_limit_sessions_per_second` in [GitLab.com's current HAProxy settings](https://gitlab.com/gitlab-cookbooks/gitlab-haproxy/blob/master/attributes/default.rb). + +### Rack Attack initializer + +#### Protected paths throttle + +GitLab.com responds with HTTP status code 429 to POST requests at protected +paths over 10 requests per **minute** per IP address. + +See the source below for which paths are protected. This includes user creation, +user confirmation, user sign in, and password reset. + +This header is included in responses to blocked requests: + +``` +Retry-After: 60 +``` + +Source: + +- Search for `rate_limit_requests_per_period`, `rate_limit_period`, and `rack_attack_protected_paths` in [GitLab.com's current Rails app settings](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb). + +#### Git and container registry failed authentication ban + +GitLab.com responds with HTTP status code 403 for 1 hour, if 30 failed +authentication requests were received in a 3-minute period from a single IP address. + +This applies only to Git requests and container registry (`/jwt/auth`) requests +(combined). + +This limit is reset by requests that authenticate successfully. For example, 29 +failed authentication requests followed by 1 successful request, followed by 29 +more failed authentication requests would not trigger a ban. + +No response headers are provided. + +### Admin Area settings + +GitLab.com does not currently use these settings. + ## GitLab.com at scale In addition to the GitLab Enterprise Edition Omnibus install, GitLab.com uses diff --git a/doc/user/group/bulk_editing/index.md b/doc/user/group/bulk_editing/index.md index 5b5f75c2dd9..c8715577eb2 100644 --- a/doc/user/group/bulk_editing/index.md +++ b/doc/user/group/bulk_editing/index.md @@ -5,22 +5,21 @@ > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/12719) for merge requests in GitLab [GitLab Premium](https://about.gitlab.com/pricing/) 12.2. -> NOTE: **Note:** -> -> - A permission level of `Reporter` or higher is required in order to manage issues. -> - A permission level of `Developer` or higher is required in order to manage merge requests. - Milestones can be updated simultaneously across multiple issues or merge requests by using the bulk editing feature.  +NOTE: **Note:** +A permission level of `Reporter` or higher is required in order to manage issues, and +a permission level of `Developer` or higher is required in order to manage merge requests. + To bulk update group issue or merge request milestones: 1. Navigate to the issues or merge requests list. 1. Click the **Edit issues** or **Edit merge requests** button. - - This will open a sidebar on the right-hand side of your screen where an editable field - for milestones will be displayed. - - Checkboxes will also appear beside each issue or merge request. + - This will open a sidebar on the right-hand side of your screen where an editable field + for milestones will be displayed. + - Checkboxes will also appear beside each issue or merge request. 1. Check the checkbox beside each issue to be edited. 1. Select the desired milestone from the sidebar. 1. Click **Update all**. diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 14c831fe671..98e1c654d0e 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -331,7 +331,7 @@ This will disable the option for all users who previously had permissions to operate project memberships, so no new users can be added. Furthermore, any request to add a new user to a project through API will not be possible. -#### IP access restriction **(ULTIMATE)** +#### IP access restriction **(ULTIMATE ONLY)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/1985) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index bc74725bbc9..64ac4f15d5f 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -78,7 +78,7 @@ During this configuration, note the following: - It is recommended to set a notification email and check the **Send an email notification when a failure occurs** checkbox. - For mappings, we will only leave `Synchronize Azure Active Directory Users to AppName` enabled. -You can then test the connection by clicking on **Test Connection**. If the connection is successful, be sure to save your configuration before moving on. +You can then test the connection by clicking on **Test Connection**. If the connection is successful, be sure to save your configuration before moving on. See below for [troubleshooting](#troubleshooting). #### Configure attribute mapping @@ -118,14 +118,22 @@ You can then test the connection by clicking on **Test Connection**. If the conn Once enabled, the synchronization details and any errors will appear on the bottom of the **Provisioning** screen, together with a link to the audit logs. -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### Testing Azure connection: invalid credentials -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +When testing the connection, you may encounter an error: **You appear to have entered invalid credentials. Please confirm you are using the correct information for an administrative account**. If `Tenant URL` and `secret token` are correct, check whether your group path contains characters that may be considered invalid JSON primitives (such as `.`). Removing such characters from the group path typically resolves the error. + +### Azure: (Field) can't be blank sync error + +When checking the Audit Logs for the Provisioning, you can sometimes see the +error `Namespace can't be blank, Name can't be blank, and User can't be blank.` + +This is likely caused because not all required fields (such as first name and +last name) are present for all users being mapped. + +As a workaround, try an alternate mapping: + +1. Follow the Azure mapping instructions from above. +1. Delete the `name.formatted` target attribute entry. +1. Change the `displayName` source attribute to have `name.formatted` target attribute. diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 9380dcf2a32..edb1e904f2b 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -185,6 +185,49 @@ graph TD; C-->D; ``` +#### Subgraphs + +NOTE: **Note:** GitLab 12.1 and up now [requires quotes around subgraph +titles that contain multiple words](https://github.com/knsv/mermaid/pull/845). + +Subgraphs can also be included: + +~~~ +```mermaid +graph TB + + SubGraph1 --> SubGraph1Flow + subgraph "SubGraph 1 Flow" + SubGraph1Flow(SubNode 1) + SubGraph1Flow -- Choice1 --> DoChoice1 + SubGraph1Flow -- Choice2 --> DoChoice2 + end + + subgraph "Main Graph" + Node1[Node 1] --> Node2[Node 2] + Node2 --> SubGraph1[Jump to SubGraph1] + SubGraph1 --> FinalThing[Final Thing] +end +``` +~~~ + +```mermaid +graph TB + + SubGraph1 --> SubGraph1Flow + subgraph "SubGraph 1 Flow" + SubGraph1Flow(SubNode 1) + SubGraph1Flow -- Choice1 --> DoChoice1 + SubGraph1Flow -- Choice2 --> DoChoice2 + end + + subgraph "Main Graph" + Node1[Node 1] --> Node2[Node 2] + Node2 --> SubGraph1[Jump to SubGraph1] + SubGraph1 --> FinalThing[Final Thing] +end +``` + ### Emoji > If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#emoji). diff --git a/doc/user/permissions.md b/doc/user/permissions.md index e6822f0c52c..d92435ef724 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -64,7 +64,6 @@ The following table depicts the various user permission levels in a project. | Lock issue threads | | ✓ | ✓ | ✓ | ✓ | | Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | | Manage related issues **(STARTER)** | | ✓ | ✓ | ✓ | ✓ | -| Create issue from vulnerability **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | | Manage labels | | ✓ | ✓ | ✓ | ✓ | | Create code snippets | | ✓ | ✓ | ✓ | ✓ | | See a commit status | | ✓ | ✓ | ✓ | ✓ | @@ -94,6 +93,8 @@ The following table depicts the various user permission levels in a project. | Remove a container registry image | | | ✓ | ✓ | ✓ | | Create/edit/delete project milestones | | | ✓ | ✓ | ✓ | | Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| View dependency list **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| Create issue from vulnerability **(ULTIMATE)** | | | ✓ | ✓ | ✓ | | Dismiss vulnerability **(ULTIMATE)** | | | ✓ | ✓ | ✓ | | Apply code change suggestions | | | ✓ | ✓ | ✓ | | Create and edit wiki pages | | | ✓ | ✓ | ✓ | @@ -211,7 +212,8 @@ group. | Create project in group | | | ✓ | ✓ | ✓ | | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | | Enable/disable a dependency proxy **(PREMIUM)** | | | ✓ | ✓ | ✓ | -| Create subgroup | | | | ✓ (1) | ✓ | +| Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| Create subgroup | | | | ✓ (1) | ✓ | | Edit group | | | | | ✓ | | Manage group members | | | | | ✓ | | Remove group | | | | | ✓ | diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index cbee79de493..5b0954195f8 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Deleting a User account Users can be deleted from a GitLab instance, either by: @@ -64,8 +68,20 @@ for such records. Any commits made by a deleted user will still display the username of the original user. When a user is deleted from an [abuse report](../../admin_area/abuse_reports.md) -or spam log, these associated records are not ghosted and will be removed, along -with any groups the user is a sole owner of. - -Administrators can also request this behavior when deleting users from the -[API](../../../api/users.md#user-deletion) or the Admin Area. +or spam log, these associated +records are not ghosted and will be removed, along with any groups the user +is a sole owner of. Administrators can also request this behavior when +deleting users from the [API](../../../api/users.md#user-deletion) or the +Admin Area. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index d3a634c7b9d..e2b1a20a605 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Two-Factor Authentication Two-factor Authentication (2FA) provides an additional level of security to your @@ -15,7 +19,7 @@ When you enable 2FA, don't forget to back up your [recovery codes](#recovery-cod In addition to time-based one time passwords (TOTP), GitLab supports U2F (universal 2nd factor) devices as the second factor of authentication. Once -enabled, in addition to supplying your username and password to login, you'll +enabled, in addition to supplying your username and password to log in, you'll be prompted to activate your U2F device (usually by pressing a button on it), and it will perform secure authentication on your behalf. @@ -238,3 +242,15 @@ Sign in and re-enable two-factor authentication as soon as possible. - The user logs out and attempts to log in via `first.host.xyz` - U2F authentication succeeds. - The user logs out and attempts to log in via `second.host.xyz` - U2F authentication fails, because the U2F key has only been registered on `first.host.xyz`. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/profile/active_sessions.md b/doc/user/profile/active_sessions.md index 28e3f4904a9..2d7bd25fc27 100644 --- a/doc/user/profile/active_sessions.md +++ b/doc/user/profile/active_sessions.md @@ -1,14 +1,31 @@ +--- +type: howto +--- + # Active Sessions > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17867) > in GitLab 10.8. GitLab lists all devices that have logged into your account. This allows you to -review the sessions. +review the sessions, and revoke any you don't recognize. ## Listing all active sessions -1. On the upper right corner, click on your avatar and go to your **Settings**. -1. Navigate to the **Active Sessions** tab. +1. Click your avatar. +1. Select **Settings**. +1. Click **Active Sessions** in the sidebar.  + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index fc5eb8c7b56..06710065dfd 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -1,7 +1,12 @@ +--- +type: index, howto +--- + # User account -When signed into their GitLab account, users can customize their -experience according to the best approach to their cases. +Each GitLab account has a user profile, and settings. Your [profile](#user-profile) +contains information about you, and your GitLab activity. Your [settings](#profile-settings) +allow you to customize some aspects of GitLab to suit yourself. ## Signing in @@ -10,8 +15,10 @@ See the [authentication topic](../../topics/authentication/index.md) for more de ## User profile -Your profile is available from the up-right corner menu bar (user's avatar) > **Profile**, -or from `https://example.gitlab.com/username`. +To access your profile: + +1. Click on your avatar. +1. Select **Profile**. On your profile page, you will see the following information: @@ -24,8 +31,10 @@ On your profile page, you will see the following information: ## Profile settings -You can edit your account settings by navigating from the up-right corner menu bar -(user's avatar) > **Settings**, or visiting `https://example.gitlab.com/profile`. +To access your profile settings: + +1. Click on your avatar. +1. Select **Settings**. From there, you can: @@ -56,8 +65,8 @@ before proceeding. To change your `username`: 1. Navigate to your [profile's](#profile-settings) **Settings > Account**. -1. Enter a new username under "Change username". -1. Hit **Update username**. +1. Enter a new username under **Change username**. +1. Click **Update username**. CAUTION: **Caution:** It is currently not possible to change your username if it contains a @@ -86,12 +95,15 @@ The following information will be hidden from the user profile page (`https://gi To enable private profile: -1. Navigate to your personal [profile settings](#profile-settings). -1. Check the "Private profile" option. -1. Hit **Update profile settings**. +1. Click your avatar. +1. Select **Profile**. +1. Click **Edit profile** (pencil icon). +1. Check the **Private profile** option in the **Main settings** section. +1. Click **Update profile settings**. NOTE: **Note:** -You and GitLab admins can see your the abovementioned information on your profile even if it is private. +All your profile information can be seen by yourself, and GitLab admins, even if +the **Private profile** option is enabled. ## Add details of external accounts @@ -99,9 +111,15 @@ GitLab allows you to add links to certain other external accounts you might have To add links to other accounts: -1. Navigate to your **User Settings > Profile**. -1. In the **Main settings** section, locate and fill out fields for links to external accounts like Skype and Twitter. -1. Click the **Update profile settings** button. +1. Click your avatar. +1. Select **Profile**. +1. Click **Edit profile** (pencil icon). +1. Complete the desired fields for external accounts, in the **Main settings** + section: + - Skype + - Twitter + - LinkedIn +1. Click **Update profile settings**. ## Private contributions @@ -111,9 +129,11 @@ Enabling private contributions will include contributions to private projects, i To enable private contributions: -1. Navigate to your personal [profile settings](#profile-settings). -1. Check the "Private contributions" option. -1. Hit **Update profile settings**. +1. Click on your avatar. +1. Select **Profile**. +1. Click **Edit profile** (pencil icon). +1. Check the **Private contributions** option. +1. Click **Update profile settings**. ## Current status @@ -124,22 +144,24 @@ This may be helpful when you are out of office or otherwise not available. Other users can then take your status into consideration when responding to your issues or assigning work to you. Please be aware that your status is publicly visible even if your [profile is private](#private-profile). +Status messages are restricted to 100 characters of plain text. +They may however contain emoji codes such as `I'm on vacation :palm_tree:`. + To set your current status: -1. Open the user menu in the top-right corner of the navigation bar. -1. Hit **Set status**, or **Edit status** if you have already set a status. -1. Set the emoji and/or status message to your liking. -1. Hit **Set status**. Alternatively, you can also hit **Remove status** to remove your user status entirely. +1. Click your avatar. +1. Click **Set status**, or **Edit status** if you have already set a status. +1. Set the desired emoji and/or status message. +1. Click **Set status**. Alternatively, you can click **Remove status** to remove your user status entirely. or -1. Navigate to your personal [profile settings](#profile-settings). -1. In the text field below `Your status`, enter your status message. -1. Select an emoji from the dropdown if you like. -1. Hit **Update profile settings**. - -Status messages are restricted to 100 characters of plain text. -They may however contain emoji codes such as `I'm on vacation :palm_tree:`. +1. Click your avatar. +1. Select **Profile**. +1. Click **Edit profile** (pencil icon). +1. Enter your status message in the **Your status** text field. +1. Click **Add status emoji** (smiley face), and select the desired emoji. +1. Click **Update profile settings**. You can also set your current status [using the API](../../api/users.md#user-status). @@ -153,34 +175,36 @@ Any of your own verified email addresses can be used as the commit email. To change your commit email: -1. Click on your avatar at the top-right corner of the navigation bar. -1. From the menu that appears, click **Settings**. -1. In the **Main settings** section, locate **Commit email** dropdown. +1. Click your avatar. +1. Select **Profile**. +1. Click **Edit profile** (pencil icon). +1. Click **Commit email** dropdown. 1. Select any of the verified emails. -1. Press **Update profile settings**. +1. Click **Update profile settings**. ### Private commit email > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22560) in GitLab 11.5. -GitLab provides users with an automatically generated private commit email option, -which allows the user to not make their email information public. +GitLab provides the user with an automatically generated private commit email option, +which allows the user to keep their email information private. To enable this option: -1. Click on your avatar at the top-right corner of the navigation bar. -1. From the menu that appears, click **Settings**. -1. In the **Main settings** section, locate **Commit email** dropdown. -1. Select the "Use a private email" option. -1. Press **Update profile settings**. +1. Click your avatar. +1. Select **Profile**. +1. Click **Edit profile** (pencil icon). +1. Click **Commit email** dropdown. +1. Select **Use a private email** option. +1. Click **Update profile settings**. Once this option is enabled, every Git-related action will be performed using the private commit email. -In order to stay fully anonymous, you can also copy this private commit email +To stay fully anonymous, you can also copy this private commit email and configure it on your local machine using the following command: ```sh -git config --global user.email <YOUR_PRIVATE_COMMIT_EMAIL> +git config --global user.email <your email address> ``` ## Troubleshooting @@ -203,3 +227,15 @@ to get you a new `_gitlab_session` and keep you signed in through browser restar After your `remember_user_token` expires and your `_gitlab_session` is cleared/expired, you will be asked to sign in again to verify your identity (which is for security reasons). + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 0b224fc7e01..248188a6457 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -1,3 +1,7 @@ +--- +type: concepts, howto +--- + # Personal access tokens > [Introduced][ce-3749] in GitLab 8.8. @@ -52,3 +56,15 @@ the following table. [container registry]: ../project/container_registry.md [users]: ../../api/users.md [usage]: ../../api/README.md#personal-access-tokens + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index b1fde3b577b..5bd4b263a58 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -1,11 +1,17 @@ +--- +type: concepts, howto +--- + # Profile preferences A user's profile preferences page allows the user to customize various aspects of GitLab to their liking. -To navigate to your profile's preferences, click your avatar icon in the top -right corner, select **Settings** and then choose **Preferences** from the -left sidebar. +To navigate to your profile's preferences: + +1. Click your avatar. +1. Select **Settings**. +1. Click **Preferences** in the sidebar. ## Navigation theme @@ -15,7 +21,7 @@ and left side navigation. Using individual color themes might help you differentiate between your different GitLab instances. -The default palette is Indigo. You can choose between 10 different themes: +The default theme is Indigo. You can choose between 10 themes: - Indigo - Light Indigo @@ -39,7 +45,7 @@ for syntax highlighting. For a list of supported languages visit the rouge websi Changing this setting allows you to customize the color theme when viewing any syntax highlighted code on GitLab. -The default syntax theme is White, and you can choose among 5 different colors: +The default syntax theme is White, and you can choose among 5 different themes: - White - Dark @@ -102,7 +108,7 @@ Select your preferred language from a list of supported languages. ### First day of the week -The first day of the week can be customised for calendar views and date pickers. +The first day of the week can be customized for calendar views and date pickers. You can choose one of the following options as the first day of the week: @@ -111,3 +117,15 @@ You can choose one of the following options as the first day of the week: - Monday If you select **System Default**, the system-wide default setting will be used. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 670dde6bb00..f0d80dad94f 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -1,11 +1,10 @@ # Kubernetes clusters -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) for -> projects in GitLab 10.1. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/34758) for -> [groups](../../group/clusters/index.md) in GitLab 11.6. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/39840) for -> [instances](../../instance/clusters/index.md) in GitLab 11.11. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) in GitLab 10.1 for projects. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/34758) in +> GitLab 11.6 for [groups](../../group/clusters/index.md). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/39840) in +> GitLab 11.11 for [instances](../../instance/clusters/index.md). GitLab provides many features with a Kubernetes integration. Kubernetes can be integrated with projects, but also: @@ -76,6 +75,9 @@ you will need the Kubernetes project integration enabled. [Read more about Auto DevOps](../../../topics/autodevops/index.md) +NOTE: **Note** +Kubernetes clusters can be used without Auto DevOps. + ### Web terminals NOTE: **Note:** @@ -477,10 +479,10 @@ For example, let's say the following Kubernetes clusters exist in a project: | Cluster | Environment scope | | ----------- | ----------------- | | Development | `*` | -| Staging | `staging` | | Production | `production` | -And the following environments are set in [`.gitlab-ci.yml`](../../../ci/yaml/README.md): +And the following environments are set in +[`.gitlab-ci.yml`](../../../ci/yaml/README.md): ```yaml stages: @@ -508,9 +510,12 @@ deploy to production: The result will then be: -- The development cluster will be used for the "test" job. -- The staging cluster will be used for the "deploy to staging" job. -- The production cluster will be used for the "deploy to production" job. +- The Development cluster details will be available in the `deploy to staging` + job. +- The production cluster details will be available in the `deploy to production` + job. +- No cluster details will be available in the `test` job because it doesn't + define any environment. ### Multiple Kubernetes clusters **(PREMIUM)** @@ -615,8 +620,16 @@ use an A record. If your external endpoint is a hostname, use a CNAME record. ## Deploying to a Kubernetes cluster -A Kubernetes cluster can be the destination for a deployment job, using -special [deployment variables](#deployment-variables). +A Kubernetes cluster can be the destination for a deployment job. If + +- The cluster is integrated with GitLab, special + [deployment variables](#deployment-variables) are made available to your job + and configuration is not required. You can immediately begin interacting with + the cluster from your jobs using tools such as `kubectl` or `helm`. +- You don't use GitLab's cluster integration you can still deploy to your + cluster. However, you will need configure Kubernetes tools yourself + using [environment variables](../../../ci/variables/README.md#creating-a-custom-environment-variable) + before you can interact with the cluster from your jobs. ### Deployment variables diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index 864cd75823c..163f3befeee 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -5,6 +5,10 @@ GitLab makes it easy to view the logs of running pods in [connected Kubernetes clusters](index.md). By displaying the logs directly in GitLab, developers can avoid having to manage console tools or jump to a different interface. +NOTE: **Kubernetes + GitLab** +Everything you need to build, test, deploy, and run your app at scale. +[Learn more](https://about.gitlab.com/solutions/kubernetes/). + ## Overview [Kubernetes](https://kubernetes.io) pod logs can be viewed directly within GitLab. Logs can be displayed by clicking on a specific pod from [Deploy Boards](../deploy_boards.md): diff --git a/doc/user/project/img/key_value_labels.png b/doc/user/project/img/key_value_labels.png Binary files differdeleted file mode 100644 index bec901f127f..00000000000 --- a/doc/user/project/img/key_value_labels.png +++ /dev/null diff --git a/doc/user/project/img/labels_default.png b/doc/user/project/img/labels_default.png Binary files differdeleted file mode 100644 index 221c5cf55a2..00000000000 --- a/doc/user/project/img/labels_default.png +++ /dev/null diff --git a/doc/user/project/img/labels_default_v12_1.png b/doc/user/project/img/labels_default_v12_1.png Binary files differnew file mode 100644 index 00000000000..4a0a6ba9ce0 --- /dev/null +++ b/doc/user/project/img/labels_default_v12_1.png diff --git a/doc/user/project/img/labels_delete_v12_1.png b/doc/user/project/img/labels_delete_v12_1.png Binary files differnew file mode 100644 index 00000000000..2e0ea934519 --- /dev/null +++ b/doc/user/project/img/labels_delete_v12_1.png diff --git a/doc/user/project/img/labels_drag_priority_v12_1.gif b/doc/user/project/img/labels_drag_priority_v12_1.gif Binary files differnew file mode 100644 index 00000000000..a568490da5f --- /dev/null +++ b/doc/user/project/img/labels_drag_priority_v12_1.gif diff --git a/doc/user/project/img/labels_epic_sidebar.png b/doc/user/project/img/labels_epic_sidebar.png Binary files differdeleted file mode 100644 index f8162d89e9d..00000000000 --- a/doc/user/project/img/labels_epic_sidebar.png +++ /dev/null diff --git a/doc/user/project/img/labels_epic_sidebar_v12_1.png b/doc/user/project/img/labels_epic_sidebar_v12_1.png Binary files differnew file mode 100644 index 00000000000..7e05c6d1ce4 --- /dev/null +++ b/doc/user/project/img/labels_epic_sidebar_v12_1.png diff --git a/doc/user/project/img/labels_generate_default.png b/doc/user/project/img/labels_generate_default.png Binary files differdeleted file mode 100644 index 982a4df999c..00000000000 --- a/doc/user/project/img/labels_generate_default.png +++ /dev/null diff --git a/doc/user/project/img/labels_generate_default_v12_1.png b/doc/user/project/img/labels_generate_default_v12_1.png Binary files differnew file mode 100644 index 00000000000..48adc9a5699 --- /dev/null +++ b/doc/user/project/img/labels_generate_default_v12_1.png diff --git a/doc/user/project/img/labels_group_issues.png b/doc/user/project/img/labels_group_issues.png Binary files differdeleted file mode 100644 index cea1d304d31..00000000000 --- a/doc/user/project/img/labels_group_issues.png +++ /dev/null diff --git a/doc/user/project/img/labels_group_issues_v12_1.png b/doc/user/project/img/labels_group_issues_v12_1.png Binary files differnew file mode 100644 index 00000000000..bfe425f20ac --- /dev/null +++ b/doc/user/project/img/labels_group_issues_v12_1.png diff --git a/doc/user/project/img/labels_key_value_v12_1.png b/doc/user/project/img/labels_key_value_v12_1.png Binary files differnew file mode 100644 index 00000000000..81d5c416c95 --- /dev/null +++ b/doc/user/project/img/labels_key_value_v12_1.png diff --git a/doc/user/project/img/labels_list.png b/doc/user/project/img/labels_list.png Binary files differdeleted file mode 100644 index 6940dde68bf..00000000000 --- a/doc/user/project/img/labels_list.png +++ /dev/null diff --git a/doc/user/project/img/labels_list_v12_1.png b/doc/user/project/img/labels_list_v12_1.png Binary files differnew file mode 100644 index 00000000000..c414ab42fbc --- /dev/null +++ b/doc/user/project/img/labels_list_v12_1.png diff --git a/doc/user/project/img/new_label_from_sidebar.gif b/doc/user/project/img/labels_new_label_from_sidebar.gif Binary files differindex 572b29a86e1..572b29a86e1 100644 --- a/doc/user/project/img/new_label_from_sidebar.gif +++ b/doc/user/project/img/labels_new_label_from_sidebar.gif diff --git a/doc/user/project/img/labels_prioritized.png b/doc/user/project/img/labels_prioritized.png Binary files differdeleted file mode 100644 index 7ce2d08b38c..00000000000 --- a/doc/user/project/img/labels_prioritized.png +++ /dev/null diff --git a/doc/user/project/img/labels_prioritized_v12_1.png b/doc/user/project/img/labels_prioritized_v12_1.png Binary files differnew file mode 100644 index 00000000000..8ea69949d8e --- /dev/null +++ b/doc/user/project/img/labels_prioritized_v12_1.png diff --git a/doc/user/project/img/labels_promotion.png b/doc/user/project/img/labels_promotion.png Binary files differdeleted file mode 100644 index 762a3773692..00000000000 --- a/doc/user/project/img/labels_promotion.png +++ /dev/null diff --git a/doc/user/project/img/labels_promotion_v12_1.png b/doc/user/project/img/labels_promotion_v12_1.png Binary files differnew file mode 100644 index 00000000000..238d50981ae --- /dev/null +++ b/doc/user/project/img/labels_promotion_v12_1.png diff --git a/doc/user/project/img/labels_subscriptions.png b/doc/user/project/img/labels_subscriptions.png Binary files differdeleted file mode 100644 index f3c4235d051..00000000000 --- a/doc/user/project/img/labels_subscriptions.png +++ /dev/null diff --git a/doc/user/project/img/labels_subscriptions_v12_1.png b/doc/user/project/img/labels_subscriptions_v12_1.png Binary files differnew file mode 100644 index 00000000000..57c437ac8a4 --- /dev/null +++ b/doc/user/project/img/labels_subscriptions_v12_1.png diff --git a/doc/user/project/integrations/jira_server_configuration.md b/doc/user/project/integrations/jira_server_configuration.md index 5116cbfe9ac..1efd0ff3944 100644 --- a/doc/user/project/integrations/jira_server_configuration.md +++ b/doc/user/project/integrations/jira_server_configuration.md @@ -3,8 +3,8 @@ We need to create a user in Jira which will have access to all projects that need to integrate with GitLab. -As an example, we'll create a user named `gitlab` and add it to the `Jira-developers` -group. +As an example, we'll create a user named `gitlab` and add it to a new group +named `gitlab-developers`. NOTE: **Note** It is important that the Jira user created for the integration is given 'write' diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index ba982c9cf6f..44439b59e77 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -98,7 +98,10 @@ You can view the performance dashboard for an environment by [clicking on the mo > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3799) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.6. -Additional metrics can be monitored by adding them on the Prometheus integration page. Once saved, they will be displayed on the environment performance dashboard. +Custom metrics can be monitored by adding them on the Prometheus integration page. Once saved, they will be displayed on the environment performance dashboard provided that either: + +- A [connected Kubernetes cluster](../clusters/index.md#adding-and-removing-clusters) with the environment scope of `*` is used and [Prometheus installed on the cluster](#enabling-prometheus-integration), or +- Prometheus is [manually configured](#manual-configuration-of-prometheus).  @@ -323,8 +326,9 @@ Once enabled, an issue will be opened automatically when an alert is triggered w - `starts_at`: Alert start time via `startsAt` - `full_query`: Alert query extracted from `generatorURL` - Optional list of attached annotations extracted from `annotations/*` +- Alert [GFM](../../markdown.md): GitLab Flavored Markdown from `annotations/gitlab_incident_markdown` -To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template. +To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template, which will apply to all incidents. To limit quick actions or other information to only specific types of alerts, use the `annotations/gitlab_incident_markdown` field. Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-ce/issues/63373), GitLab will tag each incident issue with the `incident` label automatically. If the label does not yet exist, it will be created automatically as well. diff --git a/doc/user/project/issues/csv_export.md b/doc/user/project/issues/csv_export.md index d3f53c09fb3..27ce3a814f9 100644 --- a/doc/user/project/issues/csv_export.md +++ b/doc/user/project/issues/csv_export.md @@ -72,4 +72,4 @@ Data will be encoded with a comma as the column delimiter, with `"` used to quot ## Limitations -As the issues will be sent as an email attachment, there is a limit on how much data can be exported. Currently this limit is 20MB to ensure successful delivery across a range of email providers. If this limit is reached we suggest narrowing the search before export, perhaps by exporting open and closed issues separately. +As the issues will be sent as an email attachment, there is a limit on how much data can be exported. Currently this limit is 15MB to ensure successful delivery across a range of email providers. If this limit is reached we suggest narrowing the search before export, perhaps by exporting open and closed issues separately. diff --git a/doc/user/project/issues/related_issues.md b/doc/user/project/issues/related_issues.md index 9c72fe33d0d..d7178506b64 100644 --- a/doc/user/project/issues/related_issues.md +++ b/doc/user/project/issues/related_issues.md @@ -19,7 +19,7 @@ Issues from a different project require additional information like the group and the project name. For example: - same project: `#44` -- same group: `project#44 ` +- same group: `project#44` - different group: `group/project#44` Valid references will be added to a temporary list that you can review. diff --git a/doc/user/project/issues/sorting_issue_lists.md b/doc/user/project/issues/sorting_issue_lists.md index ba120783430..0fe86e6f410 100644 --- a/doc/user/project/issues/sorting_issue_lists.md +++ b/doc/user/project/issues/sorting_issue_lists.md @@ -9,7 +9,7 @@ similar to [issue boards](../issue_board.md#issue-ordering-in-a-list). ## Manual sorting -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/62178) in GitLab 12.1. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/62178) in GitLab 12.2. When you select **Manual** sorting, you can change the order by dragging and dropping the issues. The changed order will persist. Everyone who visits the same list will see the reordered list, with some exceptions. diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index fdfeb4aa4cd..a00c1c980d2 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -2,43 +2,52 @@ ## Overview -Labels allow you to categorize issues or merge requests using descriptive titles like `bug`, `feature request`, or `docs`. Each label also has a customizable color. They allow you to quickly and dynamically filter and manage issues or merge requests you care about, and are visible throughout GitLab in most places where issues and merge requests are located. +Labels allow you to categorize issues or merge requests using descriptive titles like +`bug`, `feature request`, or `docs`. Each label also has a customizable color. They +allow you to quickly and dynamically filter and manage issues or merge requests you +care about, and are visible throughout GitLab in most places where issues and merge +requests are located. ## Project labels and group labels In GitLab, you can create project and group labels: - **Project labels** can be assigned to issues or merge requests in that project only. -- **Group labels** can be assigned to any issue or merge request of any project in that group or any subgroups of the group. +- **Group labels** can be assigned to any issue or merge request in any project in + that group, or any subgroups of the group. ## Scoped labels **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9175) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.10. -Scoped labels allow teams to use the simple and familiar feature of labels to +Scoped labels allow teams to use the simple and familiar label feature to annotate their issues, merge requests, and epics to achieve custom fields and custom workflow states by leveraging a special label title syntax. -A scoped label is a kind of label defined only by a special double-colon syntax +A scoped label is a kind of label defined by a special double-colon syntax in the label’s title, using the format `key::value`. For example: - + An issue, epic, or merge request cannot have two scoped labels with the same key. -For example, if an issue is already labeled `priority::3` and you apply the label `priority::2` to it, -`priority::3` is automatically removed. +For example, if an issue is already labeled `priority::3`, and then you apply the +`priority::2` label to it, `priority::3` is automatically removed. This functionality is demonstrated in a video titled [Use scoped labels in GitLab 11.10 for custom fields and custom workflows](https://www.youtube.com/watch?v=4BCBby6du3c). ### Labels with multiple colon pairs -If labels have multiple instances of `::`, the longest path from left to right, until the last `::`, is considered the "key" or the "scope". +If labels have multiple instances of `::`, the longest path from left to right, until +the last `::`, is considered the "key" or the "scope". -For example, `nested::key1::value1` and `nested::key1::value2` cannot both exist on the same issue. Adding the latter label will automatically remove the former due to the shared scope of `nested::key1`. +For example, `nested::key1::value1` and `nested::key1::value2` cannot both exist on +the same issue. Adding the latter label will automatically remove the former due to +the shared scope of `nested::key1`. -`nested::key1::value1` and `nested::key2::value1` can both exist on the same issue, as these are considered to use two different label scopes, `nested::key1` and `nested::key2`. +`nested::key1::value1` and `nested::key2::value1` can both exist on the same issue, +as these are considered to use two different label scopes, `nested::key1` and `nested::key2`. -### Workflows with scoped labels **(PREMIUM)** +### Workflows with scoped labels Suppose you wanted a custom field in issues to track the platform operating system that your features target, where each issue should only target one platform. You @@ -58,101 +67,154 @@ be able to advance workflow states consistently in issues themselves. ## Creating labels ->**Note:** +NOTE: **Note:** A permission level of Reporter or higher is required to create labels. ### New project label To create a **project label**, navigate to **Issues > Labels** in the project. -This page only shows project labels in this project and group labels of this project's parent group. +This page only shows the project labels in this project, and the group labels of the +project's parent group. -Click the **New label** button. Enter the title, an optional description, and the background color. Click **Create label** to create the label. +Click the **New label** button. Enter the title, an optional description, and the +background color. Click **Create label** to create the label. -If a project has no labels, you can generate a default set of project labels from its empty label list page: +If a project has no labels, you can generate a default set of project labels from +its empty label list page: - + GitLab will add the following default labels to the project: - + ### New group label -To create a **group label**, follow similar steps from above to project labels. Navigate to **Issues > Labels** in the group and create it from there. -This page only shows group labels in this group. -Alternatively, you can create group labels also from Epic sidebar. Please note that the created label will belong to the immediate group to which epic belongs. +To create a **group label**, navigate to **Issues > Labels** in the **group** and create +it from there. This page only shows group labels in this group. + +Alternatively, you can create group labels from the Epic sidebar. Please note that the +created label will belong to the immediate group to which the epic belongs. **(ULTIMATE)** - + Group labels appear in every label list page of the group's child projects. - + ### New project label from sidebar -From the sidebar of an issue or a merge request, you can create a new **project label** inline immediately, instead of navigating to the project label list page. +From the sidebar of an issue or a merge request, you can create a new **project label** +inline immediately, instead of navigating to the project label list page. - + ## Editing labels NOTE: **Note:** A permission level of Reporter or higher is required to edit labels. -You can update a label by navigating to **Issues > Labels** in the project or group and clicking the pencil icon. +To update a label, navigate to **Issues > Labels** in the project or group +and click the pencil icon. The title, description and color can be changed. -You can delete a label by clicking the trash icon. +To delete a label, click the three dots next to the `Subscribe` button, and select +**Delete**. + + ### Promoting project labels to group labels -If you are expanding from a few projects to a larger number of projects within the same group, you may want to share the same label among multiple projects in the same group. If you previously created a project label and now want to make it available for other projects, you can promote it to a group label. +If you are expanding from a few projects to a larger number of projects within the +same group, you may want to share the same label among multiple projects in the same +group. If you previously created a project label and now want to make it available +for other projects, you can promote it to a group label. -From the project label list page, you can promote a project label to a group label. This will merge all project labels across all projects in this group with the same name into a single group label. All issues and merge requests that previously were assigned one of these project labels will now be assigned the new group label. This action cannot be reversed and the changes are permanent. +From the project label list page, you can promote a project label to a group label. +This will merge all project labels with the same name into a single group label, across +all projects in this group. All issues and merge requests that were previously +assigned one of these project labels will now be assigned the new group label. This +action cannot be reversed and the changes are permanent. - + ## Assigning labels from the sidebar -Every issue and merge request can be assigned any number of labels. The labels are visible on every issue and merge request page, in the sidebar. They are also visible in the issue board. From the sidebar, you can assign or unassign a label to the object (i.e. label or unlabel it). You can also perform this as a [quick action](quick_actions.md) in a comment. +Every issue and merge request can be assigned any number of labels. The labels are +visible on every issue and merge request page, in the sidebar. They are also visible on: + +- Every issue and merge request page in the sidebar. +- The issue board. + +From the sidebar, you can assign or unassign a label to the object (i.e. label or +unlabel it). You can also perform this as a [quick action](quick_actions.md), +in a comment. | View labels in sidebar | Assign labels from sidebar | -|:---:|:---:| +|:----------------------:|:--------------------------:| |  |  | ## Searching for project labels -You can search for project labels by navigating from the left sidebar to your -project's **Issues > Labels** and entering your query to the search bar on the -top-right: +To search for project labels, go to **Issues > Labels** in the left sidebar, and enter +your search query in the **Filter** field.  -GitLab will consider the label title and description for the search. +GitLab will check both the label titles and descriptions for the search. + +## Filtering by label + +The following can be filtered labels: -## Filtering issues, merge requests and epics by label +- Issue lists +- Merge Request lists +- Epic lists **(ULTIMATE)** +- Issue Boards ### Filtering in list pages -From the project issue list page and the project merge request list page, you can [filter](../search/index.md#issues-and-merge-requests) by both group (including subgroup ancestors) labels and project labels. +- From the project issue list page and the project merge request list page, you can + [filter](../search/index.md#issues-and-merge-requests) by: + - Group labels (including subgroup ancestors) + - Project labels -From the group issue list page and the group merge request list page, you can [filter](../search/index.md#issues-and-merge-requests) by both group labels (including subgroup ancestors and subgroup descendants) and project labels. +- From the group issue list page and the group merge request list page, you can + [filter](../search/index.md#issues-and-merge-requests) by: + - Group labels (including subgroup ancestors and subgroup descendants) + - Project labels -From the group epic list page, you can [filter](../search/index.md#issues-and-merge-requests) by both current group labels as well as descendant group labels. +- You can [filter](../search/index.md#issues-and-merge-requests) the group epic list + page by: **(ULTIMATE)** + - Current group labels + - Descendant group labels - + ### Filtering in issue boards -- From [project boards](issue_board.md), you can filter by both group labels and project labels in the [search and filter bar](../search/index.md#issue-boards). -- From [group issue boards](issue_board.md#group-issue-boards-premium), you can filter by only group labels in the [search and filter bar](../search/index.md#issue-boards). **(PREMIUM)** -- From [project boards](issue_board.md), you can filter by both group labels and project labels in the [issue board configuration](issue_board.md#configurable-issue-boards-starter). **(STARTER)** -- From [group issue boards](issue_board.md#group-issue-boards-premium), you can filter by only group labels in the [issue board configuration](issue_board.md#configurable-issue-boards-starter). **(STARTER)** +- From [project boards](issue_board.md), you can use the [search and filter bar](../search/index.md#issue-boards) + to filter by: + - Group labels + - Project labels + +- From [group issue boards](issue_board.md#group-issue-boards-premium), you can use the + [search and filter bar](../search/index.md#issue-boards) to filter by group labels only. **(PREMIUM)** + +- From [project boards](issue_board.md), in the [issue board configuration](issue_board.md#configurable-issue-boards-starter), + you can filter by: **(STARTER)** + - Group labels + - Project labels + +- From [group issue boards](issue_board.md#group-issue-boards-premium), in the [issue board configuration](issue_board.md#configurable-issue-boards-starter), + you can filter by group labels only. **(STARTER)** ## Subscribing to labels -From the project label list page and the group label list page, you can subscribe to [notifications](../../workflow/notifications.md) of a given label, to alert you that the label has been assigned to an issue or merge request. +From the project label list page and the group label list page, you can subscribe +to [notifications](../../workflow/notifications.md) of a given label, to alert you +that the label has been assigned to an issue or merge request. - + ## Label priority @@ -161,26 +223,43 @@ From the project label list page and the group label list page, you can subscrib > - Introduced in GitLab 8.9. > - Priority sorting is based on the highest priority label only. [This discussion](https://gitlab.com/gitlab-org/gitlab-ce/issues/18554) considers changing this. -Labels can have relative priorities, which are used in the "Label priority" and "Priority" sort orders of the issue and merge request list pages. +Labels can have relative priorities, which are used in the "Label priority" and +"Priority" sort orders of the issue and merge request list pages. + +From the project label list page, star a label to indicate that it has a priority. + + + +Drag starred labels up and down the list to change their priority. Higher in the list +means higher priority. Prioritization happens at the project level, only on the project +label list page, and not on the group label list page. -From the project label list page, star a label to indicate that it has a priority. Drag starred labels up and down to change their priority. Higher means higher priority. Prioritization happens at the project level, only on the project label list page, and not on the group label list page. However, both project and group labels can be prioritized on the project label list page since both types are displayed on the project label list page. +However, both project and group +labels can be prioritized on the project label list page since both types are displayed +on the project label list page. - + -On the project and group issue and merge request list pages, you can sort by `Label priority` and `Priority`, which account for objects (issues and merge requests) that have prioritized labels assigned to them. +On the merge request and issue pages, for both groups and projects, you can sort by `Label priority` +and `Priority`, which account for objects (issues and merge requests) that have prioritized +labels assigned to them. If you sort by `Label priority`, GitLab considers this sort comparison order: - Object with a higher priority prioritized label. - Object without a prioritized label. -Ties are broken arbitrarily. (Note that we _only_ consider the highest prioritized label in an object, and not any of the lower prioritized labels. [This discussion](https://gitlab.com/gitlab-org/gitlab-ce/issues/18554) considers changing this.) +Ties are broken arbitrarily. Note that we _only_ consider the highest prioritized label +in an object, and not any of the lower prioritized labels. [This discussion](https://gitlab.com/gitlab-org/gitlab-ce/issues/18554) +considers changing this.  If you sort by `Priority`, GitLab considers this sort comparison order: -- Object's assigned [milestone](milestones/index.md)'s due date is sooner, provided the object has a milestone and the milestone has a due date. If this isn't the case, consider the object having a due date in the infinite future. +- Due date of the assigned [milestone](milestones/index.md) is sooner, provided + the object has a milestone and the milestone has a due date. If this isn't the case, + consider the object having a due date in the infinite future. - Object with a higher priority prioritized label. - Object without a prioritized label. diff --git a/doc/user/project/merge_requests/merge_request_approvals.md b/doc/user/project/merge_requests/merge_request_approvals.md index 220795d6f15..656459b3b03 100644 --- a/doc/user/project/merge_requests/merge_request_approvals.md +++ b/doc/user/project/merge_requests/merge_request_approvals.md @@ -331,6 +331,16 @@ the dropdown) `approver` and select the user.  +## Security approvals in merge requests **(ULTIMATE)** + +> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing) 12.2. + +Merge Request Approvals can be configured to require approval from a member +of your security team when a vulnerability would be introduced by a merge request. + +For more information, see +[Security approvals in merge requests](../../application_security/index.md#security-approvals-in-merge-requests-ultimate). + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/milestones/burndown_charts.md b/doc/user/project/milestones/burndown_charts.md index 84e08b4eeb8..d4dd9c7d4c3 100644 --- a/doc/user/project/milestones/burndown_charts.md +++ b/doc/user/project/milestones/burndown_charts.md @@ -1,6 +1,9 @@ +--- +type: reference +--- + # Burndown Charts **(STARTER)** -> **Notes:** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1540) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.1 for project milestones. > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/5354) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.8 for group milestones. > - [Added](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6495) to [GitLab Starter](https://about.gitlab.com/pricing/) 11.2 for group milestones. @@ -20,7 +23,8 @@ yourself to have the same sense of progress. GitLab Starter plots it for you and presents it in a clear and beautiful chart. -For an overview, check the video demonstration on [Mapping Work Versus Time, With Burndown Charts](https://about.gitlab.com/2017/04/25/mapping-work-to-do-versus-time-with-burndown-charts/). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, check the video demonstration on [Mapping work versus time with Burndown Charts](https://www.youtube.com/watch?v=zJU2MuRChzs). ## Use cases @@ -68,3 +72,15 @@ The Burndown Chart can also be toggled to display the cumulative open issue weight for a given day. When using this feature, make sure issue weights have been properly assigned, since an open issue with no weight adds zero to the cumulative value. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 142462e1879..453983fa882 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -1,3 +1,7 @@ +--- +type: index, reference +--- + # Milestones ## Overview @@ -145,3 +149,15 @@ The milestone sidebar on the milestone view shows the following: For project milestones only, the milestone sidebar shows the total issue weight of all issues that have the milestone assigned.  + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md index 494dd539da7..03ae24242e3 100644 --- a/doc/user/project/new_ci_build_permissions_model.md +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -90,8 +90,7 @@ to steal the tokens of other jobs. Since 9.0 [pipeline triggers][triggers] do support the new permission model. The new triggers do impersonate their associated user including their access -to projects and their project permissions. To migrate trigger to use new permission -model use **Take ownership**. +to projects and their project permissions. ## Before GitLab 8.12 diff --git a/doc/user/project/packages/npm_registry.md b/doc/user/project/packages/npm_registry.md index ca0aa9965ef..e01bac6b26f 100644 --- a/doc/user/project/packages/npm_registry.md +++ b/doc/user/project/packages/npm_registry.md @@ -81,6 +81,10 @@ domain name. You should now be able to download and upload NPM packages to your project. +NOTE: **Note:** +If you encounter an error message with [Yarn](https://yarnpkg.com/en/), see the +[troubleshooting section](#troubleshooting). + ## Uploading packages Before you will be able to upload a package, you need to specify the registry @@ -116,3 +120,29 @@ a given scope, you will receive a `403 Forbidden!` error. If you upload a package with a same name and version twice, GitLab will show both packages in the UI, but the GitLab NPM Registry will expose the most recent one as it supports only one package per version for `npm install`. + +## Troubleshooting + +### Error running yarn with NPM registry + +If you are using [yarn](https://yarnpkg.com/en/) with the NPM registry, you may get +an error message like: + +```sh +yarn install v1.15.2 +warning package.json: No license field +info No lockfile found. +warning XXX: No license field +[1/4] 🔍 Resolving packages... +[2/4] 🚚 Fetching packages... +error An unexpected error occurred: "https://gitlab.com/api/v4/projects/XXX/packages/npm/XXX/XXX/-/XXX/XXX-X.X.X.tgz: Request failed \"404 Not Found\"". +info If you think this is a bug, please open a bug report with the information provided in "/Users/XXX/gitlab-migration/module-util/yarn-error.log". +info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command +``` + +In this case, try adding this to your `.npmrc` file (and replace `<your_oauth_token>` +with your with your OAuth or personal access token): + +```text +//gitlab.com/api/v4/projects/:_authToken=<your_oauth_token> +``` diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index 54ecc42d2b9..6a9900d48f9 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -191,7 +191,7 @@ can use the following setup: 1. In Cloudflare, create a DNS `TXT` record to verify your domain. 1. In GitLab, verify your domain. 1. In Cloudflare, create a DNS `CNAME` record pointing `www` to `domain.com`. -1. In Cloudflare, add a Page Rule pointing `www.domain,com` to `domain.com`: +1. In Cloudflare, add a Page Rule pointing `www.domain.com` to `domain.com`: - Navigate to your domain's dashboard and click **Page Rules** on the top nav. - Click **Create Page Rule**. diff --git a/doc/user/project/pipelines/job_artifacts.md b/doc/user/project/pipelines/job_artifacts.md index 2411744c874..ef90899c512 100644 --- a/doc/user/project/pipelines/job_artifacts.md +++ b/doc/user/project/pipelines/job_artifacts.md @@ -1,24 +1,24 @@ +--- +type: reference, howto +--- + # Introduction to job artifacts -> **Notes:** -> -> - Since GitLab 8.2 and GitLab Runner 0.7.0, job artifacts that are created by -> GitLab Runner are uploaded to GitLab and are downloadable as a single archive -> (`tar.gz`) using the GitLab UI. -> - Starting with GitLab 8.4 and GitLab Runner 1.0, the artifacts archive format -> changed to `ZIP`, and it is now possible to browse its contents, with the added -> ability of downloading the files separately. -> - Starting with GitLab 8.17, builds are renamed to jobs. -> - The artifacts browser will be available only for new artifacts that are sent -> to GitLab using GitLab Runner version 1.0 and up. It will not be possible to -> browse old artifacts already uploaded to GitLab. ->- This is the user documentation. For the administration guide see -> [administration/job_artifacts](../../../administration/job_artifacts.md). - -Artifacts is a list of files and directories which are attached to a job -after it finishes. This feature is enabled by default in all +> - Introduced in GitLab 8.2 and GitLab Runner 0.7.0. +> - Starting with GitLab 8.4 and GitLab Runner 1.0, the artifacts archive format changed to `ZIP`, and it is now possible to browse its contents, with the added ability of downloading the files separately. +> - In GitLab 8.17, builds were renamed to jobs. +> - The artifacts browser will be available only for new artifacts that are sent to GitLab using GitLab Runner version 1.0 and up. It will not be possible to browse old artifacts already uploaded to GitLab. + +Artifacts are a list of files and directories which created by a job +once it finishes. This feature is [enabled by default](../../../administration/job_artifacts.md) in all GitLab installations. +Job artifacts that are created by GitLab Runner are uploaded to GitLab and are downloadable as a single archive using the GitLab UI. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, watch the video [GitLab CI Pipeline, Artifacts, and Environments](https://www.youtube.com/watch?v=PCKDICEe10s). +Watch also [GitLab CI pipeline tutorial for beginners](https://www.youtube.com/watch?v=Jav4vbUrqII). + ## Defining artifacts in `.gitlab-ci.yml` A simple example of using the artifacts definition in `.gitlab-ci.yml` would be @@ -50,11 +50,8 @@ For more examples on artifacts, follow the [artifacts reference in ## Browsing artifacts -> **Note:** > With GitLab 9.2, PDFs, images, videos and other formats can be previewed > directly in the job artifacts browser without the need to download them. -> -> **Note:** > With [GitLab 10.1][ce-14399], HTML files in a public project can be previewed > directly in a new tab without the need to download them when > [GitLab Pages](../../../administration/pages/index.md) is enabled. @@ -108,7 +105,7 @@ inside GitLab that make that possible. It is possible to download the latest artifacts of a job via a well known URL so you can use it for scripting purposes. ->**Note:** +NOTE: **Note:** The latest artifacts are considered as the artifacts created by jobs in the latest pipeline that succeeded for the specific ref. Artifacts for other pipelines can be accessed with direct access to them. @@ -169,9 +166,9 @@ https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/artifacts/master/file/htmlcov/ind The latest builds are also exposed in the UI in various places. Specifically, look for the download button in: -- the main project's page -- the branches page -- the tags page +- The main project's page +- The branches page +- The tags page If the latest job has failed to upload the artifacts, you can see that information in the UI. @@ -201,3 +198,15 @@ In order to retrieve a job artifact of a different project, you might need to us [expiry date]: ../../../ci/yaml/README.md#artifactsexpire_in [ce-14399]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14399 + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/pipelines/schedules.md b/doc/user/project/pipelines/schedules.md index dd5427ab35a..4e25d8545e9 100644 --- a/doc/user/project/pipelines/schedules.md +++ b/doc/user/project/pipelines/schedules.md @@ -1,7 +1,9 @@ +--- +type: reference, howto +--- + # Pipeline schedules -> **Notes**: -> > - Introduced in GitLab 9.1 as [Trigger Schedule](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10533). > - [Renamed to Pipeline Schedule](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10853) in GitLab 9.2. > - Cron notation is parsed by [Fugit](https://github.com/floraison/fugit). @@ -118,3 +120,15 @@ on the target branch, the schedule will stop creating new pipelines. This can happen if, for example, the owner is blocked or removed from the project, or the target branch or tag is protected. In this case, someone with sufficient privileges must take ownership of the schedule. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index df82daa3da3..456209c2180 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -1,3 +1,7 @@ +--- +type: reference, howto +--- + # Pipelines settings To reach the pipelines settings navigate to your project's @@ -5,6 +9,10 @@ To reach the pipelines settings navigate to your project's The following settings can be configured per project. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, watch the video [GitLab CI Pipeline, Artifacts, and Environments](https://www.youtube.com/watch?v=PCKDICEe10s). +Watch also [GitLab CI pipeline tutorial for beginners](https://www.youtube.com/watch?v=Jav4vbUrqII). + ## Git strategy With Git strategy, you can choose the default way your repository is fetched @@ -68,6 +76,13 @@ Here are some valid examples: - `my/path/.gitlab-ci.yml` - `my/path/.my-custom-file.yml` +The path can be customized at a project level. To customize the path: + +1. Go to the project's **Settings > CI / CD**. +1. Expand the **General pipelines** section. +1. Provide a value in the **Custom CI config path** field. +1. Click **Save changes**. + ## Test coverage parsing If you use test coverage in your code, GitLab can capture its output in the @@ -216,3 +231,15 @@ https://example.gitlab.com/<namespace>/<project>/badges/<branch>/coverage.svg?st ## Environment Variables [Environment variables](../../../ci/variables/README.html#gitlab-cicd-environment-variables) can be set in an environment to be available to a runner. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 9a3c02e1f50..e3515711f17 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -1,3 +1,7 @@ +--- +type: reference, howto +--- + # Protected Branches [Permissions](../permissions.md) in GitLab are fundamentally defined around the @@ -9,13 +13,13 @@ created protected branches. By default, a protected branch does four simple things: -- it prevents its creation, if not already created, from everybody except users - with Maintainer permission -- it prevents pushes from everybody except users with Maintainer permission -- it prevents **anyone** from force pushing to the branch -- it prevents **anyone** from deleting the branch +- It prevents its creation, if not already created, from everybody except users + with Maintainer permission. +- It prevents pushes from everybody except users with Maintainer permission. +- It prevents **anyone** from force pushing to the branch. +- It prevents **anyone** from deleting the branch. ->**Note**: +NOTE: **Note:** A GitLab admin is allowed to push to the protected branches. See the [Changelog](#changelog) section for changes over time. @@ -68,7 +72,7 @@ they are set to "Maintainers" by default. ## Restricting push and merge access to certain users **(STARTER)** -> This feature was [introduced][ce-5081] in [GitLab Starter][ee] 8.11. +> [Introduced][ce-5081] in [GitLab Starter][ee] 8.11. With GitLab Enterprise Edition you can restrict access to protected branches by choosing a role (Maintainers, Developers) as well as certain users. From the @@ -174,11 +178,21 @@ for details about the pipelines security model. - Allow developers to merge into a protected branch without having push access [gitlab-org/gitlab-ce!4892][ce-4892] - Allow specifying protected branches using wildcards [gitlab-org/gitlab-ce!4665][ce-4665] ---- - [ce-4665]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4665 "Allow specifying protected branches using wildcards" [ce-4892]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4892 "Allow developers to merge into a protected branch without having push access" [ce-5081]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5081 "Allow creating protected branches that can't be pushed to" [ce-21393]: https://gitlab.com/gitlab-org/gitlab-ce/issues/21393 [perm]: ../permissions.md [ee]: https://about.gitlab.com/pricing/ + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 26bec323f02..687c4e1ea6f 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -1,3 +1,7 @@ +--- +type: reference, howto +--- + # Protected Tags > [Introduced][ce-10356] in GitLab 9.1. @@ -14,19 +18,19 @@ Protected tags will prevent anyone from updating or deleting the tag, as and wil To protect a tag, you need to have at least Maintainer permission level. -1. Navigate to the project's Settings -> Repository page +1. Navigate to the project's **Settings > Repository**:  -1. From the **Tag** dropdown menu, select the tag you want to protect or type and click `Create wildcard`. In the screenshot below, we chose to protect all tags matching `v*`. +1. From the **Tag** dropdown menu, select the tag you want to protect or type and click **Create wildcard**. In the screenshot below, we chose to protect all tags matching `v*`:  -1. From the `Allowed to create` dropdown, select who will have permission to create matching tags and then click `Protect`. +1. From the **Allowed to create** dropdown, select who will have permission to create matching tags and then click **Protect**:  -1. Once done, the protected tag will appear in the "Protected tags" list. +1. Once done, the protected tag will appear in the **Protected tags** list:  @@ -45,13 +49,23 @@ matching the wildcard. For example: Two different wildcards can potentially match the same tag. For example, `*-stable` and `production-*` would both match a `production-stable` tag. In that case, if _any_ of these protected tags have a setting like -"Allowed to create", then `production-stable` will also inherit this setting. +**Allowed to create**, then `production-stable` will also inherit this setting. If you click on a protected tag's name, you will be presented with a list of all matching tags:  ---- +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> [ce-10356]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10356 "Protected Tags" diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index f81669a41c9..437899dce1e 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -1,13 +1,18 @@ -# GitLab quick actions +--- +type: reference +--- + +# GitLab Quick Actions Quick actions are textual shortcuts for common actions on issues, epics, merge requests, and commits that are usually done by clicking buttons or dropdowns in GitLab's UI. You can enter these commands while creating a new issue or merge request, or in comments of issues, epics, merge requests, and commits. Each command should be on a separate line in order to be properly detected and executed. Once executed, -the commands are removed from the text body and not visible to anyone else. -## Quick actions for issues and merge requests +> From [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26672), an alert is displayed when a quick action is successfully applied. + +## Quick Actions for issues and merge requests The following quick actions are applicable to both issues and merge requests threads, discussions, and descriptions: @@ -65,6 +70,19 @@ Many quick actions require a parameter, for example: username, milestone, and label. [Autocomplete characters](autocomplete_characters.md) can make it easier to enter a parameter, compared to selecting items from a list. +## Quick actions parameters + +The easiest way to set parameters for quick actions is to use autocomplete. If +you manually enter a parameter, it must be enclosed in double quotation marks +(`"`), unless it contains only: + +1. ASCII letters. +2. Numerals. +3. Underscore, hyphen, question mark, dot, and ampersand. + +Parameters are also case-sensitive. Autocomplete handles this, and the insertion +of quotation marks, automatically. + ## Quick actions for commit messages The following quick actions are applicable for commit messages: @@ -96,3 +114,15 @@ The following quick actions are applicable for epics threads and description: | `/remove_child_epic <&epic | group&epic | Epic URL>` | Removes child epic from epic ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-ee/issues/7330)) | | `/parent_epic <&epic | group&epic | Epic URL>` | Sets parent epic to epic ([introduced in GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-ee/issues/10556)) | | `/remove_parent_epic` | Removes parent epic from epic ([introduced in GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-ee/issues/10556)) | + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 00a4f6c6a6b..aae253467f0 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -1,3 +1,7 @@ +--- +type: reference, howto +--- + # Releases > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/41766) in GitLab 11.7. @@ -60,3 +64,15 @@ Navigate to **Project > Releases** in order to see the list of releases for a gi project.  + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/repository/img/compare_branches.png b/doc/user/project/repository/branches/img/compare_branches.png Binary files differindex 52d5c518c45..52d5c518c45 100644 --- a/doc/user/project/repository/img/compare_branches.png +++ b/doc/user/project/repository/branches/img/compare_branches.png diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index a81c9197ec1..a864665602e 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -1,17 +1,41 @@ +--- +type: concepts, howto +--- + # Branches -Read through GiLab's branching documentation: +A branch is a version of a project's working tree. You create a branch for each +set of related changes you make. This keeps each set of changes separate from +each other, allowing changes to be made in parallel, without affecting each +other. + +After pushing your changes to a new branch, you can: + +- Create a [merge request](../../merge_requests/index.md) +- Perform inline code review +- [Discuss](../../../discussions/index.md) your implementation with your team +- Preview changes submitted to a new branch with [Review Apps](../../../../ci/review_apps/index.md). + +With [GitLab Starter](https://about.gitlab.com/pricing/), you can also request +[approval](../../merge_requests/merge_request_approvals.md) from your managers. + +For more information on managing branches using the GitLab UI, see: + +- [Default branches](#default-branch) +- [Create a branch](../web_editor.md#create-a-new-branch) +- [Protected branches](../../protected_branches.md#protected-branches) +- [Delete merged branches](#delete-merged-branches) +- [Branch filter search box](#branch-filter-search-box) + +You can also manage branches using the +[command line](../../../../gitlab-basics/start-using-git.md#create-a-branch). -- [Create a branch](../web_editor.md#create-a-new-branch). -- [Default branch](#default-branch). -- [Protected branches](../../protected_branches.md#protected-branches). -- [Delete merged branches](#delete-merged-branches). -- [Branch filter search box](#branch-filter-search-box). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i>Watch the video [GitLab Flow](https://www.youtube.com/watch?v=InKNIvky2KE). See also: - [Branches API](../../../../api/branches.md), for information on operating on repository branches using the GitLab API. -- [GitLab Flow](../../../../university/training/gitlab_flow.md). Use the best of GitLab for your branching strategies. +- [GitLab Flow](../../../../university/training/gitlab_flow.md) documentation. - [Getting started with Git](../../../../topics/git/index.md) and GitLab. ## Default branch @@ -29,6 +53,17 @@ The default branch is also protected against accidental deletion. Read through the documentation on [protected branches](../../protected_branches.md#protected-branches) to learn more. +## Compare + +To compare branches in a repository: + +1. Navigate to your project's repository. +1. Select **Repository > Compare** in the sidebar. +1. Select branches to compare using the [branch filter search box](#branch-filter-search-box) +1. Click **Compare** to view the changes inline: + + + ## Delete merged branches > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6449) in GitLab 8.14. @@ -57,3 +92,15 @@ Sometimes when you have hundreds of branches you may want a more flexible matchi - `^feature` will only match branch names that begin with 'feature'. - `feature$` will only match branch names that end with 'feature'. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index 3b0a045ef9c..b929c6a681a 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -1,38 +1,39 @@ -# Signing commits with GPG +--- +type: concepts, howto +--- -NOTE: **Note:** -The term GPG is used for all OpenPGP/PGP/GPG related material and -implementations. +# Signing commits with GPG > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9546) in GitLab 9.5. > - Subkeys support was added in GitLab 10.1. -GitLab can show whether a commit is verified or not when signed with a GPG key. -All you need to do is upload the public GPG key in your profile settings. +You can use a GPG key to sign Git commits made in a GitLab repository. Signed +commits are labeled **Verified** if the identity of the committer can be +verified. To verify the identity of a committer, GitLab requires their public +GPG key. -GPG verified tags are not supported yet. +NOTE: **Note:** +The term GPG is used for all OpenPGP/PGP/GPG related material and +implementations. -## Getting started with GPG +GPG verified tags are not supported yet. -Here are a few guides to get you started with GPG: - -- [Git Tools - Signing Your Work](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) -- [Managing OpenPGP Keys](https://riseup.net/en/security/message-security/openpgp/gpg-keys) -- [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices) -- [Creating a new GPG key with subkeys](https://www.void.gr/kargig/blog/2013/12/02/creating-a-new-gpg-key-with-subkeys/) (advanced) +See the [further reading](#further-reading) section for more details on GPG. ## How GitLab handles GPG GitLab uses its own keyring to verify the GPG signature. It does not access any public key server. -In order to have a commit verified on GitLab the corresponding public key needs -to be uploaded to GitLab. For a signature to be verified three conditions need -to be met: +For a commit to be verified by GitLab: -1. The public key needs to be added your GitLab account -1. One of the emails in the GPG key matches a **verified** email address you use in GitLab -1. The committer's email matches the verified email from the gpg key +- The committer must have a GPG public/private key pair. +- The committer's public key must have been uploaded to their GitLab + account. +- One of the emails in the GPG key must match a **verified** email address + used by the committer in GitLab. +- The committer's email address must match the verified email address from the + GPG key. ## Generating a GPG key @@ -65,8 +66,7 @@ started: Your selection? 1 ``` -1. The next question is key length. We recommend to choose the highest value - which is `4096`: +1. The next question is key length. We recommend you choose `4096`: ``` RSA keys may be between 1024 and 4096 bits long. @@ -74,8 +74,8 @@ started: Requested keysize is 4096 bits ``` -1. Next, you need to specify the validity period of your key. This is something - subjective, and you can use the default value which is to never expire: +1. Specify the validity period of your key. This is something + subjective, and you can use the default value, which is to never expire: ``` Please specify how long the key should be valid. @@ -94,9 +94,9 @@ started: Is this correct? (y/N) y ``` -1. Enter you real name, the email address to be associated with this key (should - match a verified email address you use in GitLab) and an optional comment - (press <kbd>Enter</kbd> to skip): +1. Enter your real name, the email address to be associated with this key + (should match a verified email address you use in GitLab) and an optional + comment (press <kbd>Enter</kbd> to skip): ``` GnuPG needs to construct a user ID to identify your key. @@ -270,3 +270,24 @@ via [push rules](../../../../push_rules/push_rules.md). ## GPG signing API Learn how to [get the GPG signature from a commit via API](../../../../api/commits.md#get-gpg-signature-of-a-commit). + +## Further reading + +For more details about GPG, see: + +- [Git Tools - Signing Your Work](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) +- [Managing OpenPGP Keys](https://riseup.net/en/security/message-security/openpgp/gpg-keys) +- [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices) +- [Creating a new GPG key with subkeys](https://www.void.gr/kargig/blog/2013/12/02/creating-a-new-gpg-key-with-subkeys/) (advanced) + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 5b5685b3418..84d63d29929 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -1,3 +1,7 @@ +--- +type: concepts, howto +--- + # Repository A [repository](https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository) @@ -111,33 +115,7 @@ GitLab. ## Branches -When you submit changes in a new [branch](branches/index.md), you create a new version -of that project's file tree. Your branch contains all the changes -you are presenting, which are detected by Git line by line. - -To continue your workflow, once you pushed your changes to a new branch, -you can create a [merge request](../merge_requests/index.md), perform -inline code review, and [discuss](../../discussions/index.md) -your implementation with your team. -You can live preview changes submitted to a new branch with -[Review Apps](../../../ci/review_apps/index.md). - -With [GitLab Starter](https://about.gitlab.com/pricing/), you can also request -[approval](../merge_requests/merge_request_approvals.md) from your managers. - -To create, delete, and view [branches](branches/index.md) via GitLab's UI: - -- [Default branches](branches/index.md#default-branch) -- [Create a branch](web_editor.md#create-a-new-branch) -- [Protected branches](../protected_branches.md#protected-branches) -- [Delete merged branches](branches/index.md#delete-merged-branches) -- [Branch filter search box](branches/index.md#branch-filter-search-box) - -Alternatively, you can use the -[command line](../../../gitlab-basics/start-using-git.md#create-a-branch). - -To learn more about branching strategies read through the -[GitLab Flow](../../../university/training/gitlab_flow.md) documentation. +For details, see [Branches](branches/index.md). ## Commits @@ -213,14 +191,6 @@ detected, add the following to `.gitattributes` in the root of your repository. > *.proto linguist-detectable=true -## Compare - -Select branches to compare using the [branch filter search box](branches/index.md#branch-filter-search-box), then click the **Compare** button to view the changes inline: - - - -Find it under your project's **Repository > Compare**. - ## Locked files **(PREMIUM)** Use [File Locking](../file_lock.md) to @@ -256,3 +226,15 @@ By clicking the download icon, a dropdown will open with links to download the f `tar`, `tar.gz`, and `tar.bz2`. - **Artifacts:** allows users to download the artifacts of the latest CI build. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 7c711bc0b3b..3adf66e4b6f 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Reducing the repository size using Git A GitLab Enterprise Edition administrator can set a [repository size limit](../../admin_area/settings/account_and_limit_settings.md) @@ -139,3 +143,15 @@ purposes! ``` Your repository should now be below the size limit. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index 0116f0fe7ca..09a5cdabc00 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # GitLab Web Editor Sometimes it's easier to make quick changes directly from the GitLab interface @@ -169,3 +173,15 @@ through the web editor, you can choose to use another of your linked email addresses from the **User Settings > Edit Profile** page. [ce-2808]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2808 + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/workflow/git_annex.md b/doc/workflow/git_annex.md index 84d25951908..9543859f86f 100644 --- a/doc/workflow/git_annex.md +++ b/doc/workflow/git_annex.md @@ -54,13 +54,13 @@ sudo yum install epel-release && sudo yum install git-annex For omnibus-gitlab packages, only one configuration setting is needed. The Omnibus package will internally set the correct options in all locations. -1. In `/etc/gitlab/gitlab.rb` add the following line: +1. In `/etc/gitlab/gitlab.rb` add the following line: - ```ruby - gitlab_shell['git_annex_enabled'] = true - ``` + ```ruby + gitlab_shell['git_annex_enabled'] = true + ``` -1. Save the file and [reconfigure GitLab][] for the changes to take effect. +1. Save the file and [reconfigure GitLab][] for the changes to take effect. ### Configuration for installations from source @@ -69,20 +69,20 @@ There are 2 settings to enable git-annex on your GitLab server. One is located in `config/gitlab.yml` of the GitLab repository and the other one is located in `config.yml` of gitlab-shell. -1. In `config/gitlab.yml` add or edit the following lines: +1. In `config/gitlab.yml` add or edit the following lines: - ```yaml - gitlab_shell: - git_annex_enabled: true - ``` + ```yaml + gitlab_shell: + git_annex_enabled: true + ``` -1. In `config.yml` of gitlab-shell add or edit the following lines: +1. In `config.yml` of gitlab-shell add or edit the following lines: - ```yaml - git_annex_enabled: true - ``` + ```yaml + git_annex_enabled: true + ``` -1. Save the files and [restart GitLab][] for the changes to take effect. +1. Save the files and [restart GitLab][] for the changes to take effect. ## Using GitLab git-annex diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index b6bba57049d..264372a512d 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -84,6 +84,10 @@ that are on the remote repository, eg. for a branch from origin: git lfs fetch origin master ``` +### Migrate an existing repo to Git LFS + +Read the documentation on how to [migrate an existing Git repo with Git LFS](../../topics/git/migrate_to_git_lfs/index.md). + ## File Locking > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35856) in GitLab 10.5. diff --git a/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md b/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md index 71c73e3dffe..75673d23799 100644 --- a/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md +++ b/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md @@ -46,25 +46,24 @@ need to do (we assume you have [git-annex enabled](../git_annex.md#using-gitlab- repository and that you have made backups in case something goes wrong). Fire up a terminal, navigate to your Git repository and: - 1. Disable `git-annex`: - ```bash - git annex sync --content - git annex direct - git annex uninit - git annex indirect - ``` + ```bash + git annex sync --content + git annex direct + git annex uninit + git annex indirect + ``` 1. Enable `git-lfs`: - ``` - git lfs install - git lfs track <files> - git add . - git commit -m "commit message" - git push - ``` + ``` + git lfs install + git lfs track <files> + git add . + git commit -m "commit message" + git push + ``` ### Disabling Git Annex in your repo @@ -86,72 +85,72 @@ if the server also has Git Annex 6 installed. Read more in the 1. Backup your repository - ```bash - cd repository - git annex sync --content - cd .. - git clone repository repository-backup - cd repository-backup - git annex get - cd .. - ``` + ```bash + cd repository + git annex sync --content + cd .. + git clone repository repository-backup + cd repository-backup + git annex get + cd .. + ``` 1. Use `annex direct`: - ```bash - cd repository - git annex direct - ``` + ```bash + cd repository + git annex direct + ``` - The output should be similar to this: + The output should be similar to this: - ```bash - commit - On branch master - Your branch is up-to-date with 'origin/master'. - nothing to commit, working tree clean - ok - direct debian.iso ok - direct ok - ``` + ```bash + commit + On branch master + Your branch is up-to-date with 'origin/master'. + nothing to commit, working tree clean + ok + direct debian.iso ok + direct ok + ``` 1. Disable Git Annex with [`annex uninit`][uninit]: - ```bash - git annex uninit - ``` + ```bash + git annex uninit + ``` - The output should be similar to this: + The output should be similar to this: - ```bash - unannex debian.iso ok - Deleted branch git-annex (was 2534d2c). - ``` + ```bash + unannex debian.iso ok + Deleted branch git-annex (was 2534d2c). + ``` - This will `unannex` every file in the repository, leaving the original files. + This will `unannex` every file in the repository, leaving the original files. 1. Switch back to `indirect` mode: - ```bash - git annex indirect - ``` - - The output should be similar to this: + ```bash + git annex indirect + ``` - ```bash - (merging origin/git-annex into git-annex...) - (recording state in git...) - commit (recording state in git...) + The output should be similar to this: - ok - (recording state in git...) - [master fac3194] commit before switching to indirect mode - 1 file changed, 1 deletion(-) - delete mode 120000 alpine-virt-3.4.4-x86_64.iso - ok - indirect ok - ok - ``` + ```bash + (merging origin/git-annex into git-annex...) + (recording state in git...) + commit (recording state in git...) + + ok + (recording state in git...) + [master fac3194] commit before switching to indirect mode + 1 file changed, 1 deletion(-) + delete mode 120000 alpine-virt-3.4.4-x86_64.iso + ok + indirect ok + ok + ``` --- @@ -216,16 +215,16 @@ branches created by Git Annex: `git-annex`, and all under `synced/`.  -You can also do this on the commandline with: +You can also do this on the command line with: - ```bash - git branch -d synced/master - git branch -d synced/git-annex - git push origin :synced/master - git push origin :synced/git-annex - git push origin :git-annex - git remote prune origin - ``` +```bash +git branch -d synced/master +git branch -d synced/git-annex +git push origin :synced/master +git push origin :synced/git-annex +git push origin :git-annex +git remote prune origin +``` If there are still some Annex objects inside your repository (`.git/annex/`) or references inside `.git/config`, run `annex uninit` again: diff --git a/doc/workflow/shortcuts.md b/doc/workflow/shortcuts.md index fd67ea8ce87..d61d7eafd18 100644 --- a/doc/workflow/shortcuts.md +++ b/doc/workflow/shortcuts.md @@ -35,11 +35,11 @@ You can see GitLab's keyboard shortcuts by using <kbd>shift</kbd> + <kbd>?</kbd> | Keyboard Shortcut | Description | | ----------------- | ----------- | -| <kbd>g</kbd> + <kbd>a</kbd> | Go to the activity feed | -| <kbd>g</kbd> + <kbd>p</kbd> | Go to projects | -| <kbd>g</kbd> + <kbd>i</kbd> | Go to issues | -| <kbd>g</kbd> + <kbd>m</kbd> | Go to merge requests | -| <kbd>g</kbd> + <kbd>t</kbd> | Go to todos | +| <kbd>Shift</kbd> + <kbd>a</kbd> | Go to the activity feed | +| <kbd>Shift</kbd> + <kbd>p</kbd> | Go to projects | +| <kbd>Shift</kbd> + <kbd>i</kbd> | Go to issues | +| <kbd>Shift</kbd> + <kbd>m</kbd> | Go to merge requests | +| <kbd>Shift</kbd> + <kbd>t</kbd> | Go to todos | ## Project diff --git a/doc/workflow/time_tracking.md b/doc/workflow/time_tracking.md index b55c6b2e3df..fad853f8a44 100644 --- a/doc/workflow/time_tracking.md +++ b/doc/workflow/time_tracking.md @@ -22,8 +22,8 @@ below. ## How to enter data -Time Tracking uses two [quick actions] that GitLab introduced with this new -feature: `/spend` and `/estimate`. +Time Tracking uses two [quick actions](../user/project/quick_actions.md) +that GitLab introduced with this new feature: `/spend` and `/estimate`. Quick actions can be used in the body of an issue or a merge request, but also in a comment in both an issue or a merge request. @@ -73,16 +73,15 @@ The following time units are available: Default conversion rates are 1mo = 4w, 1w = 5d and 1d = 8h. -### Limit displayed units to hours +### Limit displayed units to hours **(CORE ONLY)** -> Introduced in GitLab 12.1. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29469/) in GitLab 12.1. -The display of time units can be limited to hours through the option in **Admin Area > Settings > Preferences** under 'Localization'. +In GitLab self-managed instances, the display of time units can be limited to +hours through the option in **Admin Area > Settings > Preferences** under **Localization**. With this option enabled, `75h` is displayed instead of `1w 4d 3h`. ## Other interesting links - [Time Tracking landing page on about.gitlab.com](https://about.gitlab.com/solutions/time-tracking/) - -[quick actions]: ../user/project/quick_actions.md diff --git a/doc/workflow/workflow.md b/doc/workflow/workflow.md index f70e41df842..7fac41c3b6f 100644 --- a/doc/workflow/workflow.md +++ b/doc/workflow/workflow.md @@ -1,31 +1,31 @@ # Feature branch workflow -1. Clone project: +1. Clone project: - ```bash - git clone git@example.com:project-name.git - ``` + ```bash + git clone git@example.com:project-name.git + ``` -1. Create branch with your feature: +1. Create branch with your feature: - ```bash - git checkout -b $feature_name - ``` + ```bash + git checkout -b $feature_name + ``` -1. Write code. Commit changes: +1. Write code. Commit changes: - ```bash - git commit -am "My feature is ready" - ``` + ```bash + git commit -am "My feature is ready" + ``` -1. Push your branch to GitLab: +1. Push your branch to GitLab: - ```bash - git push origin $feature_name - ``` + ```bash + git push origin $feature_name + ``` -1. Review your code on commits page. +1. Review your code on commits page. -1. Create a merge request. +1. Create a merge request. -1. Your team lead will review the code & merge it to the main branch. +1. Your team lead will review the code & merge it to the main branch. |
