diff options
| author | Kamil Trzciński <ayufan@ayufan.eu> | 2018-02-28 20:35:22 +0100 |
|---|---|---|
| committer | Kamil Trzciński <ayufan@ayufan.eu> | 2018-02-28 20:35:22 +0100 |
| commit | e0401df1214397626e65e58166988fe62715d372 (patch) | |
| tree | 087d8ca4a1611aa50a8ac98e66f7d1657ff1f90f /doc | |
| parent | 2b7b60728426c10ef1188a1073d3630805773a35 (diff) | |
| parent | 11c67e7c2f992299ff5918ce67995b73d1e0be6d (diff) | |
| download | gitlab-ce-e0401df1214397626e65e58166988fe62715d372.tar.gz | |
Merge commit '11c67e7c2f992299ff5918ce67995b73d1e0be6d' into object-storage-ee-to-ce-backport
Diffstat (limited to 'doc')
253 files changed, 3832 insertions, 1033 deletions
diff --git a/doc/README.md b/doc/README.md index b3e7c9bd0bf..c6500a37aa9 100644 --- a/doc/README.md +++ b/doc/README.md @@ -53,6 +53,7 @@ Shortcuts to GitLab's most visited docs: ### Projects and groups - [Projects](user/project/index.md): + - [Project settings](user/project/settings/index.md) - [Create a project](gitlab-basics/create-project.md) - [Fork a project](gitlab-basics/fork-project.md) - [Importing and exporting projects between instances](user/project/settings/import_export.md). @@ -68,17 +69,18 @@ Shortcuts to GitLab's most visited docs: Manage your [repositories](user/project/repository/index.md) from the UI (user interface): -- Files +- [Files](user/project/repository/index.md#files) - [Create a file](user/project/repository/web_editor.md#create-a-file) - [Upload a file](user/project/repository/web_editor.md#upload-a-file) - [File templates](user/project/repository/web_editor.md#template-dropdowns) - [Create a directory](user/project/repository/web_editor.md#create-a-directory) - [Start a merge request](user/project/repository/web_editor.md#tips) (when committing via UI) -- Branches +- [Branches](user/project/repository/branches/index.md) + - [Default branch](user/project/repository/branches/index.md#default-branch) - [Create a branch](user/project/repository/web_editor.md#create-a-new-branch) - [Protected branches](user/project/protected_branches.md#protected-branches) - [Delete merged branches](user/project/repository/branches/index.md#delete-merged-branches) -- Commits +- [Commits](user/project/repository/index.md#commits) - [Signing commits](user/project/repository/gpg_signed_commits/index.md): use GPG to sign your commits. ### Issues and Merge Requests (MRs) diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index d22815dfa5e..ad903aef896 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -256,7 +256,7 @@ production: ``` Tip: If you want to limit access to the nested members of an Active Directory -group you can use the following syntax: +group, you can use the following syntax: ``` (memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com) @@ -287,11 +287,11 @@ LDAP email address, and then sign into GitLab via their LDAP credentials. There are two encryption methods, `simple_tls` and `start_tls`. -For either encryption method, if setting `validate_certificates: false`, TLS +For either encryption method, if setting `verify_certificates: false`, TLS encryption is established with the LDAP server before any LDAP-protocol data is exchanged but no validation of the LDAP server's SSL certificate is performed. ->**Note**: Before GitLab 9.5, `validate_certificates: false` is the default if +>**Note**: Before GitLab 9.5, `verify_certificates: false` is the default if unspecified. ## Limitations diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 5732b6a1ca4..e3b10119090 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -32,6 +32,14 @@ prometheus_listen_addr = "localhost:9236" Changes to `/home/git/gitaly/config.toml` are applied when you run `service gitlab restart`. +## Client-side GRPC logs + +Gitaly uses the [gRPC](https://grpc.io/) RPC framework. The Ruby gRPC +client has its own log file which may contain useful information when +you are seeing Gitaly errors. You can control the log level of the +gRPC client with the `GRPC_LOG_LEVEL` environment variable. The +default level is `WARN`. + ## Running Gitaly on its own server > This is an optional way to deploy Gitaly which can benefit GitLab @@ -145,8 +153,8 @@ Omnibus installations: ```ruby # /etc/gitlab/gitlab.rb git_data_dirs({ - { 'default' => { 'path' => '/mnt/gitlab/default', 'gitaly_address' => 'tcp://gitlab.internal:9999' } }, - { 'storage1' => { 'path' => '/mnt/gitlab/storage1', 'gitaly_address' => 'tcp://gitlab.internal:9999' } }, + 'default' => { 'path' => '/mnt/gitlab/default', 'gitaly_address' => 'tcp://gitlab.internal:9999' }, + 'storage1' => { 'path' => '/mnt/gitlab/storage1', 'gitaly_address' => 'tcp://gitlab.internal:9999' }, }) gitlab_rails['gitaly_token'] = 'abc123secret' diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 582f3c67b0d..59305774c4a 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -171,9 +171,9 @@ and [projects APIs](../api/projects.md). ## Implementation details When GitLab receives an artifacts archive, an archive metadata file is also -generated. This metadata file describes all the entries that are located in the -artifacts archive itself. The metadata file is in a binary format, with -additional GZIP compression. +generated by [GitLab Workhorse]. This metadata file describes all the entries +that are located in the artifacts archive itself. +The metadata file is in a binary format, with additional GZIP compression. GitLab does not extract the artifacts archive in order to save space, memory and disk I/O. It instead inspects the metadata file which contains all the diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md new file mode 100644 index 00000000000..bac8fa4bd9d --- /dev/null +++ b/doc/administration/raketasks/storage.md @@ -0,0 +1,107 @@ +# Repository Storage Rake Tasks + +This is a collection of rake tasks you can use to help you list and migrate +existing projects from Legacy storage to the new Hashed storage type. + +You can read more about the storage types [here][storage-types]. + +## List projects on Legacy storage + +To have a simple summary of projects using **Legacy** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:legacy_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:legacy_projects + +``` + +------ + +To list projects using **Legacy** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:list_legacy_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:list_legacy_projects + +``` + +## List projects on Hashed storage + +To have a simple summary of projects using **Hashed** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:hashed_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:hashed_projects + +``` + +------ + +To list projects using **Hashed** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:list_hashed_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:list_hashed_projects + +``` + +## Migrate existing projects to Hashed storage + +Before migrating your existing projects, you should +[enable hashed storage][storage-migration] for the new projects as well. + +This task will schedule all your existing projects to be migrated to the +**Hashed** storage type: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:migrate_to_hashed +``` + +**Source Installation** + +```bash +rake gitlab:storage:migrate_to_hashed + +``` + +You can monitor the progress in the _Admin > Monitoring > Background jobs_ screen. +There is a specific Queue you can watch to see how long it will take to finish: **project_migrate_hashed_storage** + +After it reaches zero, you can confirm every project has been migrated by running the commands above. +If you find it necessary, you can run this migration script again to schedule missing projects. + +Any error or warning will be logged in the sidekiq log file. + + +[storage-types]: ../repository_storage_types.md +[storage-migration]: ../repository_storage_types.md#how-to-migrate-to-hashed-storage diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md index e99a7ee29cc..1304476e678 100644 --- a/doc/administration/reply_by_email.md +++ b/doc/administration/reply_by_email.md @@ -77,6 +77,33 @@ and use [an application password](https://support.google.com/mail/answer/185833) To set up a basic Postfix mail server with IMAP access on Ubuntu, follow the [Postfix setup documentation](reply_by_email_postfix_setup.md). +### Security Concerns + +**WARNING:** Be careful when choosing the domain used for receiving incoming +email. + +For the sake of example, suppose your top-level company domain is `hooli.com`. +All employees in your company have an email address at that domain via Google +Apps, and your company's private Slack instance requires a valid `@hooli.com` +email address in order to sign up. + +If you also host a public-facing GitLab instance at `hooli.com` and set your +incoming email domain to `hooli.com`, an attacker could abuse the "Create new +issue by email" feature by using a project's unique address as the email when +signing up for Slack, which would send a confirmation email, which would create +a new issue on the project owned by the attacker, allowing them to click the +confirmation link and validate their account on your company's private Slack +instance. + +We recommend receiving incoming email on a subdomain, such as +`incoming.hooli.com`, and ensuring that you do not employ any services that +authenticate solely based on access to an email domain such as `*.hooli.com.` +Alternatively, use a dedicated domain for GitLab email communications such as +`hooli-gitlab.com`. + +See GitLab issue [#30366](https://gitlab.com/gitlab-org/gitlab-ce/issues/30366) +for a real-world example of this exploit. + ### Omnibus package installations 1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the @@ -141,7 +168,7 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow the # The IDLE command timeout. gitlab_rails['incoming_email_idle_timeout'] = 60 ``` - + ```ruby # Configuration for Microsoft Exchange mail server w/ IMAP enabled, assumes mailbox incoming@exchange.example.com gitlab_rails['incoming_email_enabled'] = true @@ -253,7 +280,7 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow the # The IDLE command timeout. idle_timeout: 60 ``` - + ```yaml # Configuration for Microsoft Exchange mail server w/ IMAP enabled, assumes mailbox incoming@exchange.example.com incoming_email: diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md new file mode 100644 index 00000000000..fa882bbe28a --- /dev/null +++ b/doc/administration/repository_storage_types.md @@ -0,0 +1,69 @@ +# Repository Storage Types + +> [Introduced][ce-28283] in GitLab 10.0. + +## Legacy Storage + +Legacy Storage is the storage behavior prior to version 10.0. For historical reasons, GitLab replicated the same +mapping structure from the projects URLs: + + * Project's repository: `#{namespace}/#{project_name}.git` + * Project's wiki: `#{namespace}/#{project_name}.wiki.git` + +This structure made simple to migrate from existing solutions to GitLab and easy for Administrators to find where the +repository is stored. + +On the other hand this has some drawbacks: + +Storage location will concentrate huge amount of top-level namespaces. The impact can be reduced by the introduction of [multiple storage paths][storage-paths]. + +Because Backups are a snapshot of the same URL mapping, if you try to recover a very old backup, you need to verify +if any project has taken the place of an old removed project sharing the same URL. This means that `mygroup/myproject` +from your backup may not be the same original project that is today in the same URL. + +Any change in the URL will need to be reflected on disk (when groups / users or projects are renamed). This can add a lot +of load in big installations, and can be even worst if they are using any type of network based filesystem. + +Last, for GitLab Geo, this storage type means we have to synchronize the disk state, replicate renames in the correct +order or we may end-up with wrong repository or missing data temporarily. + +## Hashed Storage + +Hashed Storage is the new storage behavior we are rolling out with 10.0. It's not enabled by default yet, but we +encourage everyone to try-it and take the time to fix any script you may have that depends on the old behavior. + +Instead of coupling project URL and the folder structure where the repository will be stored on disk, we are coupling +a hash, based on the project's ID. + +This makes the folder structure immutable, and therefore eliminates any requirement to synchronize state from URLs to +disk structure. This means that renaming a group, user or project will cost only the database transaction, and will take +effect immediately. + +The hash also helps to spread the repositories more evenly on the disk, so the top-level directory will contain less +folders than the total amount of top-level namespaces. + +Hash format is based on hexadecimal representation of SHA256: `SHA256(project.id)`. +Top-level folder uses first 2 characters, followed by another folder with the next 2 characters. They are both stored in +a special folder `@hashed`, to co-exist with existing Legacy projects: + +```ruby +# Project's repository: +"@hashed/#{hash[0..1]}/#{hash[2..3]}/#{hash}.git" + +# Wiki's repository: +"@hashed/#{hash[0..1]}/#{hash[2..3]}/#{hash}.wiki.git" +``` + +This new format also makes possible to restore backups with confidence, as when restoring a repository from the backup, +you will never mistakenly restore a repository in the wrong project (considering the backup is made after the migration). + +### How to migrate to Hashed Storage + +In GitLab, go to **Admin > Settings**, find the **Repository Storage** section and select +"_Create new projects using hashed storage paths_". + +To migrate your existing projects to the new storage type, check the specific [rake tasks]. + +[ce-28283]: https://gitlab.com/gitlab-org/gitlab-ce/issues/28283 +[rake tasks]: raketasks/storage.md#migrate-existing-projects-to-hashed-storage +[storage-paths]: repository_storage_types.md diff --git a/doc/api/README.md b/doc/api/README.md index 6cbea29bda6..3fd4c97e536 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -14,6 +14,7 @@ following locations: - [Project-level Variables](project_level_variables.md) - [Group-level Variables](group_level_variables.md) - [Commits](commits.md) +- [Custom Attributes](custom_attributes.md) - [Deployments](deployments.md) - [Deploy Keys](deploy_keys.md) - [Environments](environments.md) diff --git a/doc/api/commits.md b/doc/api/commits.md index 2a78553782f..5a4a8d888b3 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -181,6 +181,12 @@ Example response: "parent_ids": [ "ae1d9fb46aa2b07ee9836d49862ec4e2c46fbbba" ], + "last_pipeline" : { + "id": 8, + "ref": "master", + "sha": "2dc6aa325a317eda67812f05600bdf0fcdc70ab0" + "status": "created" + } "stats": { "additions": 15, "deletions": 10, diff --git a/doc/api/custom_attributes.md b/doc/api/custom_attributes.md new file mode 100644 index 00000000000..8b26f7093ab --- /dev/null +++ b/doc/api/custom_attributes.md @@ -0,0 +1,105 @@ +# Custom Attributes API + +Every API call to custom attributes must be authenticated as administrator. + +## List custom attributes + +Get all custom attributes on a user. + +``` +GET /users/:id/custom_attributes +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes +``` + +Example response: + +```json +[ + { + "key": "location", + "value": "Antarctica" + }, + { + "key": "role", + "value": "Developer" + } +] +``` + +## Single custom attribute + +Get a single custom attribute on a user. + +``` +GET /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` + +Example response: + +```json +{ + "key": "location", + "value": "Antarctica" +} +``` + +## Set custom attribute + +Set a custom attribute on a user. The attribute will be updated if it already exists, +or newly created otherwise. + +``` +PUT /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | +| `value` | string | yes | The value of the custom attribute | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "value=Greenland" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` + +Example response: + +```json +{ + "key": "location", + "value": "Greenland" +} +``` + +## Delete custom attribute + +Delete a custom attribute on a user. + +``` +DELETE /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` diff --git a/doc/api/issues.md b/doc/api/issues.md index 8ca66049d31..ec8ff3cd3f3 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -95,6 +95,7 @@ Example response: "username" : "root" }, "updated_at" : "2016-01-04T15:31:51.081Z", + "closed_at" : null, "id" : 76, "title" : "Consequatur vero maxime deserunt laboriosam est voluptas dolorem.", "created_at" : "2016-01-04T15:31:51.081Z", @@ -109,7 +110,8 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, - "confidential": false + "confidential": false, + "discussion_locked": false } ] ``` @@ -205,6 +207,7 @@ Example response: "title" : "Ut commodi ullam eos dolores perferendis nihil sunt.", "updated_at" : "2016-01-04T15:31:46.176Z", "created_at" : "2016-01-04T15:31:46.176Z", + "closed_at" : null, "user_notes_count": 1, "due_date": null, "web_url": "http://example.com/example/example/issues/1", @@ -214,7 +217,8 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, - "confidential": false + "confidential": false, + "discussion_locked": false } ] ``` @@ -311,6 +315,7 @@ Example response: "title" : "Ut commodi ullam eos dolores perferendis nihil sunt.", "updated_at" : "2016-01-04T15:31:46.176Z", "created_at" : "2016-01-04T15:31:46.176Z", + "closed_at" : "2016-01-05T15:31:46.176Z", "user_notes_count": 1, "due_date": "2016-07-22", "web_url": "http://example.com/example/example/issues/1", @@ -320,7 +325,8 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, - "confidential": false + "confidential": false, + "discussion_locked": false } ] ``` @@ -358,7 +364,8 @@ Example response: "id" : 11, "title" : "v3.0", "created_at" : "2016-01-04T15:31:39.788Z", - "updated_at" : "2016-01-04T15:31:39.788Z" + "updated_at" : "2016-01-04T15:31:39.788Z", + "closed_at" : "2016-01-05T15:31:46.176Z" }, "author" : { "state" : "active", @@ -403,6 +410,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -465,6 +473,7 @@ Example response: }, "description" : null, "updated_at" : "2016-01-07T12:44:33.959Z", + "closed_at" : null, "milestone" : null, "subscribed" : true, "user_notes_count": 0, @@ -477,6 +486,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -510,6 +520,8 @@ PUT /projects/:id/issues/:issue_iid | `state_event` | string | no | The state event of an issue. Set `close` to close the issue and `reopen` to reopen it | | `updated_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` (requires admin or project owner rights) | | `due_date` | string | no | Date time string in the format YEAR-MONTH-DAY, e.g. `2016-03-11` | +| `discussion_locked` | boolean | no | Flag indicating if the issue's discussion is locked. If the discussion is locked only project members can add or edit comments. | + ```bash curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/4/issues/85?state_event=close @@ -533,6 +545,7 @@ Example response: "project_id" : 4, "description" : null, "updated_at" : "2016-01-07T12:55:16.213Z", + "closed_at" : "2016-01-08T12:55:16.213Z", "iid" : 15, "labels" : [ "bug" @@ -552,6 +565,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -615,6 +629,7 @@ Example response: "state": "opened", "created_at": "2016-04-05T21:41:45.652Z", "updated_at": "2016-04-07T12:20:17.596Z", + "closed_at": null, "labels": [], "milestone": null, "assignees": [{ @@ -650,6 +665,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -692,6 +708,7 @@ Example response: "state": "opened", "created_at": "2016-04-05T21:41:45.652Z", "updated_at": "2016-04-07T12:20:17.596Z", + "closed_at": null, "labels": [], "milestone": null, "assignees": [{ @@ -727,6 +744,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -757,6 +775,44 @@ POST /projects/:id/issues/:issue_iid/unsubscribe curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/93/unsubscribe ``` +Example response: + +```json +{ + "id": 93, + "iid": 12, + "project_id": 5, + "title": "Incidunt et rerum ea expedita iure quibusdam.", + "description": "Et cumque architecto sed aut ipsam.", + "state": "opened", + "created_at": "2016-04-05T21:41:45.217Z", + "updated_at": "2016-04-07T13:02:37.905Z", + "labels": [], + "milestone": null, + "assignee": { + "name": "Edwardo Grady", + "username": "keyon", + "id": 21, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/3e6f06a86cf27fa8b56f3f74f7615987?s=80&d=identicon", + "web_url": "https://gitlab.example.com/keyon" + }, + "author": { + "name": "Vivian Hermann", + "username": "orville", + "id": 11, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", + "web_url": "https://gitlab.example.com/orville" + }, + "subscribed": false, + "due_date": null, + "web_url": "http://example.com/example/example/issues/12", + "confidential": false, + "discussion_locked": false +} +``` + ## Create a todo Manually creates a todo for the current user on an issue. If @@ -849,7 +905,8 @@ Example response: "downvotes": 0, "due_date": null, "web_url": "http://example.com/example/example/issues/110", - "confidential": false + "confidential": false, + "discussion_locked": false }, "target_url": "https://gitlab.example.com/gitlab-org/gitlab-ci/issues/10", "body": "Vel voluptas atque dicta mollitia adipisci qui at.", diff --git a/doc/api/jobs.md b/doc/api/jobs.md index d60c7c12881..e7060e154f4 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -336,7 +336,7 @@ Parameters | Attribute | Type | Required | Description | |-------------|---------|----------|-------------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `ref_name` | string | yes | The ref from a repository | +| `ref_name` | string | yes | The ref from a repository (can only be branch or tag name, not HEAD or SHA) | | `job` | string | yes | The name of the job | Example request: diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index bff8a2d3e4d..50a971102fb 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -121,6 +121,15 @@ GET /projects/:id/merge_requests?labels=bug,reproduced GET /projects/:id/merge_requests?my_reaction_emoji=star ``` +`project_id` represents the ID of the project where the MR resides. +`project_id` will always equal `target_project_id`. + +In the case of a merge request from the same project, +`source_project_id`, `target_project_id` and `project_id` +will be the same. In the case of a merge request from a fork, +`target_project_id` and `project_id` will be the same and +`source_project_id` will be the fork project's ID. + Parameters: | Attribute | Type | Required | Description | @@ -192,6 +201,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -267,6 +277,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -378,6 +389,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -471,6 +483,7 @@ POST /projects/:id/merge_requests "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -500,6 +513,7 @@ PUT /projects/:id/merge_requests/:merge_request_iid | `labels` | string | no | Labels for MR as a comma-separated list | | `milestone_id` | integer | no | The ID of a milestone | | `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | +| `discussion_locked` | boolean | no | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | Must include at least one non-required attribute from above. @@ -554,6 +568,7 @@ Must include at least one non-required attribute from above. "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -658,6 +673,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -734,6 +750,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -1028,7 +1045,8 @@ Example response: "id": 14, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a7fa515d53450023c83d62986d0658a8?s=80&d=identicon", - "web_url": "https://gitlab.example.com/francisca" + "web_url": "https://gitlab.example.com/francisca", + "discussion_locked": false }, "assignee": { "name": "Dr. Gabrielle Strosin", diff --git a/doc/api/projects.md b/doc/api/projects.md index 3144220e588..07331d05231 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -635,6 +635,98 @@ POST /projects/:id/fork | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | | `namespace` | integer/string | yes | The ID or path of the namespace that the project will be forked to | +## List Forks of a project + +>**Note:** This feature was introduced in GitLab 10.1 + +List the projects accessible to the calling user that have an established, forked relationship with the specified project + +``` +GET /projects/:id/forks +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of projects matching the search criteria | +| `simple` | boolean | no | Return only the ID, URL, name, and path of each project | +| `owned` | boolean | no | Limit by projects owned by the current user | +| `membership` | boolean | no | Limit by projects that the current user is a member of | +| `starred` | boolean | no | Limit by projects starred by the current user | +| `statistics` | boolean | no | Include project statistics | +| `with_issues_enabled` | boolean | no | Limit by enabled issues feature | +| `with_merge_requests_enabled` | boolean | no | Limit by enabled merge requests feature | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/5/forks" +``` + +Example responses: + +```json +[ + { + "id": 3, + "description": null, + "default_branch": "master", + "visibility": "internal", + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site", + "tag_list": [ + "example", + "disapora project" + ], + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "jobs_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, + "container_registry_enabled": false, + "created_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "id": 3, + "name": "Diaspora", + "path": "diaspora", + "kind": "group", + "full_path": "diaspora" + }, + "import_status": "none", + "archived": true, + "avatar_url": "http://example.com/uploads/project/avatar/3/uploads/avatar.png", + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 1, + "public_jobs": true, + "shared_with_groups": [], + "only_allow_merge_if_pipeline_succeeds": false, + "only_allow_merge_if_all_discussions_are_resolved": false, + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } + } +] +``` + ## Star a project Stars a given project. Returns status code `304` if the project is already starred. diff --git a/doc/api/tags.md b/doc/api/tags.md index 32fe5eea692..bebe6536b6e 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -131,7 +131,7 @@ Parameters: "message": null } ``` -The message will be `nil` when creating a lightweight tag otherwise +The message will be `null` when creating a lightweight tag otherwise it will contain the annotation. In case of an error, diff --git a/doc/api/users.md b/doc/api/users.md index 6d5db16b36a..1643c584244 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -154,6 +154,12 @@ You can search users by creation date time range with: GET /users?created_before=2001-01-02T00:00:00.060Z&created_after=1999-01-02T00:00:00.060 ``` +You can filter by [custom attributes](custom_attributes.md) with: + +``` +GET /users?custom_attributes[key]=value&custom_attributes[other_key]=other_value +``` + ## Single user Get a single user. diff --git a/doc/api/wikis.md b/doc/api/wikis.md index 10eebc4a3c1..15ce5f96b60 100644 --- a/doc/api/wikis.md +++ b/doc/api/wikis.md @@ -155,3 +155,5 @@ curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gi ``` On success the HTTP status code is `204` and no JSON response is expected. + +[ce-13372]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13372 diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index f28c9791bee..4586caa457d 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -250,6 +250,8 @@ By default, when using `docker:dind`, Docker uses the `vfs` storage driver which copies the filesystem on every run. This is a very disk-intensive operation which can be avoided if a different driver is used, for example `overlay2`. +### Requirements + 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: @@ -271,14 +273,27 @@ which can be avoided if a different driver is used, for example `overlay2`. overlay ``` -1. Use the driver by defining a variable at the top of your `.gitlab-ci.yml`: +### Use driver per project - ``` - variables: - DOCKER_DRIVER: overlay2 - ``` - -> **Note:** +You can enable the driver for each project individually by editing the project's `.gitlab-ci.yml`: + +``` +variables: + DOCKER_DRIVER: overlay2 +``` + +### Use driver for every project + +To enable the driver for every project, you can set the environment variable for every build by adding `environment` in the `[[runners]]` section of `config.toml`: + +```toml +environment = ["DOCKER_DRIVER=overlay2"] +``` + +If you're running multiple Runners you will have to modify all configuration files. + +> **Notes:** +- More information about the Runner configuration is available in the [Runner documentation](https://docs.gitlab.com/runner/configuration/). - For more information about using OverlayFS with Docker, you can read [Use the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/). diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 796a025b951..b8f9988e3ef 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -1,51 +1,46 @@ -## Enable or disable GitLab CI +## Enable or disable GitLab CI/CD -_To effectively use GitLab CI, you need a valid [`.gitlab-ci.yml`](yaml/README.md) +To effectively use GitLab CI/CD, you need a valid [`.gitlab-ci.yml`](yaml/README.md) file present at the root directory of your project and a [runner](runners/README.md) properly set up. You can read our -[quick start guide](quick_start/README.md) to get you started._ +[quick start guide](quick_start/README.md) to get you started. -If you are using an external CI server like Jenkins or Drone CI, it is advised -to disable GitLab CI in order to not have any conflicts with the commits status +If you are using an external CI/CD server like Jenkins or Drone CI, it is advised +to disable GitLab CI/CD in order to not have any conflicts with the commits status API. --- -GitLab CI is exposed via the `/pipelines` and `/builds` pages of a project. -Disabling GitLab CI in a project does not delete any previous jobs. -In fact, the `/pipelines` and `/builds` pages can still be accessed, although +GitLab CI/CD is exposed via the `/pipelines` and `/jobs` pages of a project. +Disabling GitLab CI/CD in a project does not delete any previous jobs. +In fact, the `/pipelines` and `/jobs` pages can still be accessed, although it's hidden from the left sidebar menu. -GitLab CI is enabled by default on new installations and can be disabled either +GitLab CI/CD is enabled by default on new installations and can be disabled either individually under each project's settings, or site-wide by modifying the settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations respectively. ### Per-project user setting -The setting to enable or disable GitLab CI can be found with the name **Pipelines** -under the **Sharing & Permissions** area of a project's settings along with -**Merge Requests**. Choose one of **Disabled**, **Only team members** and -**Everyone with access** and hit **Save changes** for the settings to take effect. +The setting to enable or disable GitLab CI/CD can be found under your project's +**Settings > General > Permissions**. Choose one of "Disabled", "Only team members" +or "Everyone with access" and hit **Save changes** for the settings to take effect. - + ---- - -### Site-wide administrator setting +### Site-wide admin setting -You can disable GitLab CI site-wide, by modifying the settings in `gitlab.yml` +You can disable GitLab CI/CD site-wide, by modifying the settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations respectively. Two things to note: -1. Disabling GitLab CI, will affect only newly-created projects. Projects that +1. Disabling GitLab CI/CD, will affect only newly-created projects. Projects that had it enabled prior to this modification, will work as before. -1. Even if you disable GitLab CI, users will still be able to enable it in the +1. Even if you disable GitLab CI/CD, users will still be able to enable it in the project's settings. ---- - For installations from source, open `gitlab.yml` with your editor and set `builds` to `false`: diff --git a/doc/ci/environments.md b/doc/ci/environments.md index cbf06afa294..c03e16b1b38 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -26,7 +26,7 @@ so every environment can have one or more deployments. GitLab keeps track of your deployments, so you always know what is currently being deployed on your servers. If you have a deployment service such as [Kubernetes][kubernetes-service] enabled for your project, you can use it to assist with your deployments, and -can even access a web terminal for your environment from within GitLab! +can even access a [web terminal](#web-terminals) for your environment from within GitLab! To better understand how environments and deployments work, let's consider an example. We assume that you have already created a project in GitLab and set up @@ -119,7 +119,7 @@ where you can find information of the last deployment status of an environment. Here's how the Environments page looks so far. - + There's a bunch of information there, specifically you can see: @@ -229,7 +229,7 @@ You can find it in the pipeline, job, environment, and deployment views. | Pipelines | Single pipeline | Environments | Deployments | jobs | | --------- | ----------------| ------------ | ----------- | -------| -|  |  |  |  |  | +|  |  |  |  |  | Clicking on the play button in either of these places will trigger the `deploy_prod` job, and the deployment will be recorded under a new @@ -240,55 +240,18 @@ Remember that if your environment's name is `production` (all lowercase), then it will get recorded in [Cycle Analytics](../user/project/cycle_analytics.md). Double the benefit! -## Web terminals - ->**Note:** -Web terminals were added in GitLab 8.15 and are only available to project -masters and owners. - -If you deploy to your environments with the help of a deployment service (e.g., -the [Kubernetes service][kubernetes-service], GitLab can open -a terminal session to your environment! This is a very powerful feature that -allows you to debug issues without leaving the comfort of your web browser. To -enable it, just follow the instructions given in the service documentation. - -Once enabled, your environments will gain a "terminal" button: - - - -You can also access the terminal button from the page for a specific environment: - - - -Wherever you find it, clicking the button will take you to a separate page to -establish the terminal session: - - - -This works just like any other terminal - you'll be in the container created -by your deployment, so you can run shell commands and get responses in real -time, check the logs, try out configuration or code tweaks, etc. You can open -multiple terminals to the same environment - they each get their own shell -session - and even a multiplexer like `screen` or `tmux`! - ->**Note:** -Container-based deployments often lack basic tools (like an editor), and may -be stopped or restarted at any time. If this happens, you will lose all your -changes! Treat this as a debugging tool, not a comprehensive online IDE. - ---- - -While this is fine for deploying to some stable environments like staging or -production, what happens for branches? So far we haven't defined anything -regarding deployments for branches other than `master`. Dynamic environments -will help us achieve that. - ## Dynamic environments As the name suggests, it is possible to create environments on the fly by just declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is the basis of [Review apps](review_apps/index.md). +>**Note:** +The `name` and `url` parameters can use any of the defined CI variables, +including predefined, secure variables and `.gitlab-ci.yml` +[`variables`](yaml/README.md#variables). +You however cannot use variables defined under `script` or on the Runner's side. + GitLab Runner exposes various [environment variables][variables] when a job runs, and as such, you can use them as environment names. Let's add another job in our example which will deploy to all branches except `master`: @@ -434,11 +397,12 @@ Let's briefly see where URL that's defined in the environments is exposed. ## Making use of the environment URL -The environment URL is exposed in a few places within GitLab. +The [environment URL](yaml/README.md#environments-url) is exposed in a few +places within GitLab. | In a merge request widget as a link | In the Environments view as a button | In the Deployments view as a button | | -------------------- | ------------ | ----------- | -|  |  |  | +|  |  |  | If a merge request is eventually merged to the default branch (in our case `master`) and that branch also deploys to an environment (in our case `staging` @@ -446,8 +410,7 @@ and/or `production`) you can see this information in the merge request itself.  -### <a name="route-map"></a>Go directly from source files to public pages on the environment - +### Go directly from source files to public pages on the environment > Introduced in GitLab 8.17. @@ -599,7 +562,7 @@ exist, you should see something like: >**Notes:** > -- For the monitor dashboard to appear, you need to: +- For the monitoring dashboard to appear, you need to: - Have enabled the [Prometheus integration][prom] - Configured Prometheus to collect at least one [supported metric](../user/project/integrations/prometheus_library/metrics.md) - With GitLab 9.2, all deployments to an environment are shown directly on the @@ -609,10 +572,9 @@ If you have enabled [Prometheus for monitoring system and response metrics](http Once configured, GitLab will attempt to retrieve [supported performance metrics](https://docs.gitlab.com/ee/user/project/integrations/prometheus_library/metrics.html) for any environment which has had a successful deployment. If monitoring data was -successfully retrieved, a Monitoring button will appear on the environment's -detail page. +successfully retrieved, a Monitoring button will appear for each environment. - + Clicking on the Monitoring button will display a new page, showing up to the last 8 hours of performance data. It may take a minute or two for data to appear @@ -624,6 +586,50 @@ version of the app, all without leaving GitLab.  +## Web terminals + +>**Note:** +Web terminals were added in GitLab 8.15 and are only available to project +masters and owners. + +If you deploy to your environments with the help of a deployment service (e.g., +the [Kubernetes service][kubernetes-service]), GitLab can open +a terminal session to your environment! This is a very powerful feature that +allows you to debug issues without leaving the comfort of your web browser. To +enable it, just follow the instructions given in the service integration +documentation. + +Once enabled, your environments will gain a "terminal" button: + + + +You can also access the terminal button from the page for a specific environment: + + + +Wherever you find it, clicking the button will take you to a separate page to +establish the terminal session: + + + +This works just like any other terminal - you'll be in the container created +by your deployment, so you can run shell commands and get responses in real +time, check the logs, try out configuration or code tweaks, etc. You can open +multiple terminals to the same environment - they each get their own shell +session - and even a multiplexer like `screen` or `tmux`! + +>**Note:** +Container-based deployments often lack basic tools (like an editor), and may +be stopped or restarted at any time. If this happens, you will lose all your +changes! Treat this as a debugging tool, not a comprehensive online IDE. + +--- + +While this is fine for deploying to some stable environments like staging or +production, what happens for branches? So far we haven't defined anything +regarding deployments for branches other than `master`. Dynamic environments +will help us achieve that. + ## Checkout deployments locally Since 8.13, a reference in the git repository is saved for each deployment, so diff --git a/doc/ci/img/builds_tab.png b/doc/ci/img/builds_tab.png Binary files differdeleted file mode 100644 index 2d7eec8a949..00000000000 --- a/doc/ci/img/builds_tab.png +++ /dev/null diff --git a/doc/ci/img/deployments_view.png b/doc/ci/img/deployments_view.png Binary files differindex 7ded0c97b72..436fed5f465 100644 --- a/doc/ci/img/deployments_view.png +++ b/doc/ci/img/deployments_view.png diff --git a/doc/ci/img/environments_available.png b/doc/ci/img/environments_available.png Binary files differnew file mode 100644 index 00000000000..2991a309655 --- /dev/null +++ b/doc/ci/img/environments_available.png diff --git a/doc/ci/img/environments_available_staging.png b/doc/ci/img/environments_available_staging.png Binary files differdeleted file mode 100644 index 5c031ad0d9d..00000000000 --- a/doc/ci/img/environments_available_staging.png +++ /dev/null diff --git a/doc/ci/img/environments_dynamic_groups.png b/doc/ci/img/environments_dynamic_groups.png Binary files differindex 0f42b368c5b..45124b3d8d8 100644 --- a/doc/ci/img/environments_dynamic_groups.png +++ b/doc/ci/img/environments_dynamic_groups.png diff --git a/doc/ci/img/environments_link_url.png b/doc/ci/img/environments_link_url.png Binary files differdeleted file mode 100644 index 44010f6aa6f..00000000000 --- a/doc/ci/img/environments_link_url.png +++ /dev/null diff --git a/doc/ci/img/environments_link_url_deployments.png b/doc/ci/img/environments_link_url_deployments.png Binary files differdeleted file mode 100644 index 4f90143527a..00000000000 --- a/doc/ci/img/environments_link_url_deployments.png +++ /dev/null diff --git a/doc/ci/img/environments_link_url_mr.png b/doc/ci/img/environments_link_url_mr.png Binary files differindex 64f134e0b0d..7ce46063062 100644 --- a/doc/ci/img/environments_link_url_mr.png +++ b/doc/ci/img/environments_link_url_mr.png diff --git a/doc/ci/img/environments_manual_action_builds.png b/doc/ci/img/environments_manual_action_builds.png Binary files differdeleted file mode 100644 index e7cf63a1031..00000000000 --- a/doc/ci/img/environments_manual_action_builds.png +++ /dev/null diff --git a/doc/ci/img/environments_manual_action_deployments.png b/doc/ci/img/environments_manual_action_deployments.png Binary files differindex 2b3f6f3edad..93beaa0de54 100644 --- a/doc/ci/img/environments_manual_action_deployments.png +++ b/doc/ci/img/environments_manual_action_deployments.png diff --git a/doc/ci/img/environments_manual_action_environments.png b/doc/ci/img/environments_manual_action_environments.png Binary files differindex e0c07604e7f..9490be63f14 100644 --- a/doc/ci/img/environments_manual_action_environments.png +++ b/doc/ci/img/environments_manual_action_environments.png diff --git a/doc/ci/img/environments_manual_action_jobs.png b/doc/ci/img/environments_manual_action_jobs.png Binary files differnew file mode 100644 index 00000000000..9ae223cf77f --- /dev/null +++ b/doc/ci/img/environments_manual_action_jobs.png diff --git a/doc/ci/img/environments_manual_action_pipelines.png b/doc/ci/img/environments_manual_action_pipelines.png Binary files differindex 82bbae88027..129e44f6fb0 100644 --- a/doc/ci/img/environments_manual_action_pipelines.png +++ b/doc/ci/img/environments_manual_action_pipelines.png diff --git a/doc/ci/img/environments_manual_action_single_pipeline.png b/doc/ci/img/environments_manual_action_single_pipeline.png Binary files differindex 36337cb1870..1eeb4379eb7 100644 --- a/doc/ci/img/environments_manual_action_single_pipeline.png +++ b/doc/ci/img/environments_manual_action_single_pipeline.png diff --git a/doc/ci/img/environments_monitoring.png b/doc/ci/img/environments_monitoring.png Binary files differindex d9c46ea4c95..dcffdd1fdb8 100644 --- a/doc/ci/img/environments_monitoring.png +++ b/doc/ci/img/environments_monitoring.png diff --git a/doc/ci/img/environments_mr_review_app.png b/doc/ci/img/environments_mr_review_app.png Binary files differindex 7bff84362a3..4bb643d708f 100644 --- a/doc/ci/img/environments_mr_review_app.png +++ b/doc/ci/img/environments_mr_review_app.png diff --git a/doc/ci/img/environments_terminal_button_on_index.png b/doc/ci/img/environments_terminal_button_on_index.png Binary files differindex 6f05b2aa343..061bb7c3c87 100644 --- a/doc/ci/img/environments_terminal_button_on_index.png +++ b/doc/ci/img/environments_terminal_button_on_index.png diff --git a/doc/ci/img/environments_terminal_button_on_show.png b/doc/ci/img/environments_terminal_button_on_show.png Binary files differindex 9469fab99ab..4d24304bc93 100644 --- a/doc/ci/img/environments_terminal_button_on_show.png +++ b/doc/ci/img/environments_terminal_button_on_show.png diff --git a/doc/ci/img/environments_view.png b/doc/ci/img/environments_view.png Binary files differdeleted file mode 100644 index 821352188ef..00000000000 --- a/doc/ci/img/environments_view.png +++ /dev/null diff --git a/doc/ci/img/permissions_settings.png b/doc/ci/img/permissions_settings.png Binary files differdeleted file mode 100644 index 1454c75fd24..00000000000 --- a/doc/ci/img/permissions_settings.png +++ /dev/null diff --git a/doc/ci/img/prometheus_environment_detail_with_metrics.png b/doc/ci/img/prometheus_environment_detail_with_metrics.png Binary files differdeleted file mode 100644 index 214b10624a9..00000000000 --- a/doc/ci/img/prometheus_environment_detail_with_metrics.png +++ /dev/null diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 88e53ff40e8..2d56b2540ef 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -106,7 +106,7 @@ What is important is that each job is run independently from each other. If you want to check whether your `.gitlab-ci.yml` file is valid, there is a Lint tool under the page `/ci/lint` of your GitLab instance. You can also find -a "CI Lint" button to go to this page under **Pipelines ➔ Pipelines** and +a "CI Lint" button to go to this page under **CI/CD ➔ Pipelines** and **Pipelines ➔ Jobs** in your project. For more information and a complete `.gitlab-ci.yml` syntax, please read @@ -155,7 +155,7 @@ Find more information about different Runners in the [Runners](../runners/README.md) documentation. You can find whether any Runners are assigned to your project by going to -**Settings ➔ Pipelines**. Setting up a Runner is easy and straightforward. The +**Settings ➔ CI/CD**. Setting up a Runner is easy and straightforward. The official Runner supported by GitLab is written in Go and its documentation can be found at <https://docs.gitlab.com/runner/>. @@ -168,7 +168,7 @@ Follow the links above to set up your own Runner or use a Shared Runner as described in the next section. Once the Runner has been set up, you should see it on the Runners page of your -project, following **Settings ➔ Pipelines**. +project, following **Settings ➔ CI/CD**.  @@ -181,7 +181,7 @@ These are special virtual machines that run on GitLab's infrastructure and can build any project. To enable the **Shared Runners** you have to go to your project's -**Settings ➔ Pipelines** and click **Enable shared runners**. +**Settings ➔ CI/CD** and click **Enable shared runners**. [Read more on Shared Runners](../runners/README.md). diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index f5d3b524d6e..8b51d112a2c 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -35,7 +35,7 @@ are: A Runner that is specific only runs for the specified project(s). A shared Runner can run jobs for every project that has enabled the option **Allow shared Runners** -under **Settings ➔ Pipelines**. +under **Settings ➔ CI/CD**. Projects with high demand of CI activity can also benefit from using specific Runners. By having dedicated Runners you are guaranteed that the Runner is not @@ -61,7 +61,7 @@ You can only register a shared Runner if you are an admin of the GitLab instance Shared Runners are enabled by default as of GitLab 8.2, but can be disabled with the **Disable shared Runners** button which is present under each project's -**Settings ➔ Pipelines** page. Previous versions of GitLab defaulted shared +**Settings ➔ CI/CD** page. Previous versions of GitLab defaulted shared Runners to disabled. ## Registering a specific Runner @@ -76,7 +76,7 @@ Registering a specific can be done in two ways: To create a specific Runner without having admin rights to the GitLab instance, visit the project you want to make the Runner work for in GitLab: -1. Go to **Settings ➔ Pipelines** to obtain the token +1. Go to **Settings ➔ CI/CD** to obtain the token 1. [Register the Runner][register] ### Making an existing shared Runner specific @@ -101,7 +101,7 @@ can be changed afterwards under each Runner's settings. To lock/unlock a Runner: -1. Visit your project's **Settings ➔ Pipelines** +1. Visit your project's **Settings ➔ CI/CD** 1. Find the Runner you wish to lock/unlock and make sure it's enabled 1. Click the pencil button 1. Check the **Lock to current projects** option @@ -115,7 +115,7 @@ you can enable the Runner also on any other project where you have Master permis To enable/disable a Runner in your project: -1. Visit your project's **Settings ➔ Pipelines** +1. Visit your project's **Settings ➔ CI/CD** 1. Find the Runner you wish to enable/disable 1. Click **Enable for this project** or **Disable for this project** @@ -136,7 +136,7 @@ Whenever a Runner is protected, the Runner picks only jobs created on To protect/unprotect Runners: -1. Visit your project's **Settings ➔ Pipelines** +1. Visit your project's **Settings ➔ CI/CD** 1. Find a Runner you want to protect/unprotect and make sure it's enabled 1. Click the pencil button besides the Runner name 1. Check the **Protected** option @@ -220,7 +220,7 @@ each Runner's settings. To make a Runner pick tagged/untagged jobs: -1. Visit your project's **Settings ➔ Pipelines** +1. Visit your project's **Settings ➔ CI/CD** 1. Find the Runner you wish and make sure it's enabled 1. Click the pencil button 1. Check the **Run untagged jobs** option @@ -228,7 +228,8 @@ To make a Runner pick tagged/untagged jobs: ### Be careful with sensitive information -If you can run a job on a Runner, you can get access to any code it runs +With some [Runner Executors](https://docs.gitlab.com/runner/executors/README.html), +if you can run a job on the Runner, you can get access to any code it runs and get the token of the Runner. With shared Runners, this means that anyone that runs jobs on the Runner, can access anyone else's code that runs on the Runner. @@ -237,7 +238,8 @@ In addition, because you can get access to the Runner token, it is possible to create a clone of a Runner and submit false jobs, for example. The above is easily avoided by restricting the usage of shared Runners -on large public GitLab instances and controlling access to your GitLab instance. +on large public GitLab instances, controlling access to your GitLab instance, +and using more secure [Runner Executors](https://docs.gitlab.com/runner/executors/README.html). ### Forks diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index cdb9858e179..e5a2bbd1773 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -34,7 +34,7 @@ instructions to [generate an SSH key](../../ssh/README.md). Do not add a passphrase to the SSH key, or the `before_script` will prompt for it. Then, create a new **Secret Variable** in your project settings on GitLab -following **Settings > Pipelines** and look for the "Secret Variables" section. +following **Settings > CI/CD** and look for the "Secret Variables" section. As **Key** add the name `SSH_PRIVATE_KEY` and in the **Value** field paste the content of your _private_ key that you created earlier. diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 7ec7136d8c6..56a16f77e7f 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -19,7 +19,7 @@ A unique trigger token can be obtained when [adding a new trigger](#adding-a-new ## Adding a new trigger You can add a new trigger by going to your project's -**Settings ➔ Pipelines** under **Triggers**. The **Add trigger** button will +**Settings ➔ CI/CD** under **Triggers**. The **Add trigger** button will create a new token which you can then use to trigger a rerun of this particular project's pipeline. @@ -43,7 +43,7 @@ From now on the trigger will be run as you. ## Revoking a trigger You can revoke a trigger any time by going at your project's -**Settings ➔ Pipelines** under **Triggers** and hitting the **Revoke** button. +**Settings ➔ CI/CD** under **Triggers** and hitting the **Revoke** button. The action is irreversible. ## Triggering a pipeline @@ -64,7 +64,7 @@ POST /projects/:id/trigger/pipeline The required parameters are the [trigger's `token`](#authentication-tokens) and the Git `ref` on which the trigger will be performed. Valid refs are the branch and the tag. The `:id` of a project can be found by -[querying the API](../../api/projects.md) or by visiting the **Pipelines** +[querying the API](../../api/projects.md) or by visiting the **CI/CD** settings page which provides self-explanatory examples. When a rerun of a pipeline is triggered, the information is exposed in GitLab's diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 6513b31826a..17839cbaef1 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -149,28 +149,31 @@ script: ## Secret variables ->**Notes:** -- This feature requires GitLab Runner 0.4.0 or higher. -- Group-level secret variables added in GitLab 9.4. -- Be aware that secret variables are not masked, and their values can be shown - in the job logs if explicitly asked to do so. If your project is public or - internal, you can set the pipelines private from your project's Pipelines - settings. Follow the discussion in issue [#13784][ce-13784] for masking the - secret variables. - -GitLab CI allows you to define per-project or per-group **secret variables** -that are set in the build environment. The secret variables are stored out of -the repository (`.gitlab-ci.yml`) and are securely passed to GitLab Runner -making them available in the build environment. It's the recommended method to -use for storing things like passwords, secret keys and credentials. +NOTE: **Note:** +Group-level secret variables were added in GitLab 9.4. + +CAUTION: **Important:** +Be aware that secret variables are not masked, and their values can be shown +in the job logs if explicitly asked to do so. If your project is public or +internal, you can set the pipelines private from your [project's Pipelines +settings](../../user/project/pipelines/settings.md#visibility-of-pipelines). +Follow the discussion in issue [#13784][ce-13784] for masking the secret variables. + +GitLab CI allows you to define per-project or per-group secret variables +that are set in the pipeline environment. The secret variables are stored out of +the repository (not in `.gitlab-ci.yml`) and are securely passed to GitLab Runner +making them available during a pipeline run. It's the recommended method to +use for storing things like passwords, SSH keys and credentials. Project-level secret variables can be added by going to your project's -**Settings ➔ Pipelines**, then finding the section called **Secret variables**. +**Settings > CI/CD**, then finding the section called **Secret variables**. Likewise, group-level secret variables can be added by going to your group's -**Settings ➔ Pipelines**, then finding the section called **Secret variables**. +**Settings > CI/CD**, then finding the section called **Secret variables**. Any variables of [subgroups] will be inherited recursively. + + Once you set them, they will be available for all subsequent pipelines. You can also [protect your variables](#protected-secret-variables). @@ -185,8 +188,8 @@ protected, it would only be securely passed to pipelines running on the protected variables. Protected variables can be added by going to your project's -**Settings ➔ Pipelines**, then finding the section called -**Secret variables**, and check *Protected*. +**Settings > CI/CD**, then finding the section called +**Secret variables**, and check "Protected". Once you set them, they will be available for all subsequent pipelines. @@ -202,7 +205,7 @@ are set in the build environment. These variables are only defined for the project services that you are using to learn which variables they define. An example project service that defines deployment variables is -[Kubernetes Service](../../user/project/integrations/kubernetes.md). +[Kubernetes Service](../../user/project/integrations/kubernetes.md#deployment-variables). ## Debug tracing @@ -439,7 +442,7 @@ export CI_REGISTRY_USER="gitlab-ci-token" export CI_REGISTRY_PASSWORD="longalfanumstring" ``` -[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 +[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI secret variables" [eep]: https://about.gitlab.com/gitlab-ee/ "Available only in GitLab Enterprise Edition Premium" [envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md diff --git a/doc/ci/variables/img/secret_variables.png b/doc/ci/variables/img/secret_variables.png Binary files differnew file mode 100644 index 00000000000..f70935069d9 --- /dev/null +++ b/doc/ci/variables/img/secret_variables.png diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index d0ac3ec6163..38bd0450a09 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -252,6 +252,8 @@ The `cache:key` variable can use any of the [predefined variables](../variables/ The default key is **default** across the project, therefore everything is shared between each pipelines and jobs by default, starting from GitLab 9.0. +>**Note:** The `cache:key` variable cannot contain the `/` character. + --- **Example configurations** @@ -276,7 +278,7 @@ To enable per-job and per-branch caching: ```yaml cache: - key: "$CI_JOB_NAME/$CI_COMMIT_REF_NAME" + key: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME" untracked: true ``` @@ -284,7 +286,7 @@ To enable per-branch and per-stage caching: ```yaml cache: - key: "$CI_JOB_STAGE/$CI_COMMIT_REF_NAME" + key: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME" untracked: true ``` @@ -293,7 +295,7 @@ If you use **Windows Batch** to run your shell scripts you need to replace ```yaml cache: - key: "%CI_JOB_STAGE%/%CI_COMMIT_REF_NAME%" + key: "%CI_JOB_STAGE%-%CI_COMMIT_REF_NAME%" untracked: true ``` @@ -302,7 +304,7 @@ If you use **Windows PowerShell** to run your shell scripts you need to replace ```yaml cache: - key: "$env:CI_JOB_STAGE/$env:CI_COMMIT_REF_NAME" + key: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_NAME" untracked: true ``` @@ -727,6 +729,9 @@ deployment to the `production` environment. - Before GitLab 8.11, the name of an environment could be defined as a string like `environment: production`. The recommended way now is to define it under the `name` keyword. +- The `name` parameter can use any of the defined CI variables, + including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). + You however cannot use variables defined under `script`. The `environment` name can contain: @@ -762,6 +767,9 @@ deploy to production: - Introduced in GitLab 8.11. - Before GitLab 8.11, the URL could be added only in GitLab's UI. The recommended way now is to define it in `.gitlab-ci.yml`. +- The `url` parameter can use any of the defined CI variables, + including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). + You however cannot use variables defined under `script`. This is an optional value that when set, it exposes buttons in various places in GitLab which when clicked take you to the defined URL. @@ -841,10 +849,9 @@ The `stop_review_app` job is **required** to have the following keywords defined **Notes:** - [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. - The `$CI_ENVIRONMENT_SLUG` was [introduced][ce-7983] in GitLab 8.15. - -`environment` can also represent a configuration hash with `name` and `url`. -These parameters can use any of the defined [CI variables](#variables) -(including predefined, secure variables and `.gitlab-ci.yml` variables). +- The `name` and `url` parameters can use any of the defined CI variables, + including predefined, secure variables and `.gitlab-ci.yml` [`variables`](#variables). + You however cannot use variables defined under `script`. For example: @@ -1366,25 +1373,31 @@ variables: GIT_DEPTH: "3" ``` -## Hidden keys +## Hidden keys (jobs) > Introduced in GitLab 8.6 and GitLab Runner v1.1.1. -Keys that start with a dot (`.`) will be not processed by GitLab CI. You can -use this feature to ignore jobs, or use the -[special YAML features](#special-yaml-features) and transform the hidden keys -into templates. +If you want to temporarily 'disable' a job, rather than commenting out all the +lines where the job is defined: + +``` +#hidden_job: +# script: +# - run test +``` -In the following example, `.key_name` will be ignored: +you can instead start its name with a dot (`.`) and it will not be processed by +GitLab CI. In the following example, `.hidden_job` will be ignored: ```yaml -.key_name: +.hidden_job: script: - - rake spec + - run test ``` -Hidden keys can be hashes like normal CI jobs, but you are also allowed to use -different types of structures to leverage special YAML features. +Use this feature to ignore jobs, or use the +[special YAML features](#special-yaml-features) and transform the hidden keys +into templates. ## Special YAML features @@ -1400,7 +1413,7 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya YAML has a handy feature called 'anchors', which lets you easily duplicate content across your document. Anchors can be used to duplicate/inherit -properties, and is a perfect example to be used with [hidden keys](#hidden-keys) +properties, and is a perfect example to be used with [hidden keys](#hidden-keys-jobs) to provide templates for your jobs. The following example uses anchors and map merging. It will create two jobs, @@ -1557,6 +1570,11 @@ Read more on [GitLab Pages user documentation](../../user/project/pages/index.md Each instance of GitLab CI has an embedded debug tool called Lint. You can find the link under `/ci/lint` of your gitlab instance. +## Using reserved keywords + +If you get validation error when using specific values (e.g., `true` or `false`), +try to quote them, or change them to a different form (e.g., `/bin/true`). + ## Skipping jobs If your commit message contains `[ci skip]` or `[skip ci]`, using any diff --git a/doc/development/README.md b/doc/development/README.md index dd150421b65..b648c7ce086 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -43,6 +43,8 @@ - [Object state models](object_state_models.md) - [Building a package for testing purposes](build_test_package.md) - [Manage feature flags](feature_flags.md) +- [View sent emails or preview mailers](emails.md) +- [Working with Gitaly](gitaly.md) ## Databases @@ -60,10 +62,13 @@ - [Ordering Table Columns](ordering_table_columns.md) - [Verifying Database Capabilities](verifying_database_capabilities.md) - [Hash Indexes](hash_indexes.md) +- [Swapping Tables](swapping_tables.md) -## i18n +## Internationalization (i18n) -- [Internationalization for GitLab](i18n_guide.md) +- [Introduction](i18n/index.md) +- [Externalization](i18n/externalization.md) +- [Translation](i18n/translation.md) ## Compliance diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index f83a60e49e8..5452b0e7a2f 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -215,14 +215,29 @@ same time will ensure that both existing and new data is migrated. In the next release we can remove the `after_commit` hooks and related code. We will also need to add a post-deployment migration that consumes any remaining -jobs. Such a migration would look like this: +jobs and manually run on any un-migrated rows. Such a migration would look like +this: ```ruby class ConsumeRemainingExtractServicesUrlJobs < ActiveRecord::Migration disable_ddl_transaction! + class Service < ActiveRecord::Base + include ::EachBatch + + self.table_name = 'services' + end + def up + # This must be included Gitlab::BackgroundMigration.steal('ExtractServicesUrl') + + # This should be included, but can be skipped - see below + Service.where(url: nil).each_batch(of: 50) do |batch| + range = batch.pluck('MIN(id)', 'MAX(id)').first + + Gitlab::BackgroundMigration::ExtractServicesUrl.new.perform(*range) + end end def down @@ -230,6 +245,15 @@ class ConsumeRemainingExtractServicesUrlJobs < ActiveRecord::Migration end ``` +The final step runs for any un-migrated rows after all of the jobs have been +processed. This is in case a Sidekiq process running the background migrations +received SIGKILL, leading to the jobs being lost. (See +[more reliable Sidekiq queue][reliable-sidekiq] for more information.) + +If the application does not depend on the data being 100% migrated (for +instance, the data is advisory, and not mission-critical), then this final step +can be skipped. + This migration will then process any jobs for the ExtractServicesUrl migration and continue once all jobs have been processed. Once done you can safely remove the `services.properties` column. @@ -254,6 +278,9 @@ for more details. 1. Make sure that background migration jobs are idempotent. 1. Make sure that tests you write are not false positives. +1. Make sure that if the data being migrated is critical and cannot be lost, the + clean-up migration also checks the final state of the data before completing. [migrations-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md [issue-rspec-hooks]: https://gitlab.com/gitlab-org/gitlab-ce/issues/35351 +[reliable-sidekiq]: https://gitlab.com/gitlab-org/gitlab-ce/issues/36791 diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 64a89976300..7165b8062a7 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -9,8 +9,18 @@ There are a few rules to get your merge request accepted: **approved by a [backend maintainer][projects]**. 1. If your merge request includes only frontend changes [^1], it must be **approved by a [frontend maintainer][projects]**. + 1. If your merge request includes UX changes [^1], it must + be **approved by a [UX team member][team]**. + 1. If your merge request includes adding a new JavaScript library [^1], it must be + **approved by a [frontend lead][team]**. + 1. If your merge request includes adding a new UI/UX paradigm [^1], it must be + **approved by a [UX lead][team]**. 1. If your merge request includes frontend and backend changes [^1], it must be **approved by a [frontend and a backend maintainer][projects]**. + 1. If your merge request includes UX and frontend changes [^1], it must + be **approved by a [UX team member and a frontend maintainer][team]**. + 1. If your merge request includes UX, frontend and backend changes [^1], it must + be **approved by a [UX team member, a frontend and a backend maintainer][team]**. 1. If your merge request includes a new dependency or a filesystem change, it must be **approved by a [Build team member][team]**. See [how to work with the Build team][build handbook] for more details. 1. To lower the amount of merge requests maintainers need to review, you can diff --git a/doc/development/emails.md b/doc/development/emails.md new file mode 100644 index 00000000000..18f47f44cb5 --- /dev/null +++ b/doc/development/emails.md @@ -0,0 +1,23 @@ +# Dealing with email in development + +## Sent emails + +To view rendered emails "sent" in your development instance, visit +[`/rails/letter_opener`](http://localhost:3000/rails/letter_opener). + +## Mailer previews + +Rails provides a way to preview our mailer templates in HTML and plaintext using +dummy data. + +The previews live in [`spec/mailers/previews`][previews] and can be viewed at +[`/rails/mailers`](http://localhost:3000/rails/mailers). + +See the [Rails guides] for more info. + +[previews]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/spec/mailers/previews +[Rails guides]: http://guides.rubyonrails.org/action_mailer_basics.html#previewing-emails + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md new file mode 100644 index 00000000000..a76e978bd26 --- /dev/null +++ b/doc/development/fe_guide/icons.md @@ -0,0 +1,40 @@ +# Icons + +We are using SVG Icons in GitLab with a SVG Sprite, due to this the icons are only loaded once and then referenced through an ID. The sprite SVG is located under `/assets/icons.svg`. Our goal is to replace one by one all inline SVG Icons (as those currently bloat the HTML) and also all Font Awesome usages. + +### Usage in HAML/Rails + +To use a sprite Icon in HAML or Rails we use a specific helper function : + +`sprite_icon(icon_name, size: nil, css_class: '')` + +**icon_name** Use the icon_name that you can find in the SVG Sprite (Overview is available under `/assets/sprite.symbol.html`). +**size (optional)** Use one of the following sizes : 16,24,32,48,72 (this will be translated into a `s16` class) +**css_class (optional)** If you want to add additional css classes + +**Example** + +`= sprite_icon('issues', size: 72, css_class: 'icon-danger')` + +**Output from example above** + +`<svg class="s72 icon-danger"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="/assets/icons.svg#issues"></use></svg>` + +### Usage in HTML/JS + +Please use the following function inside JS to render an icon : +`gl.utils.spriteIcon(iconName)` + +## Adding a new icon to the sprite + +All Icons and Illustrations are managed in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository which is added as a dev-dependency. + +To upgrade to a new SVG Sprite version run `yarn upgrade https://gitlab.com/gitlab-org/gitlab-svgs` and then run `yarn run svg`. This task will copy the svg sprite and all illustrations in the correct folders. + +# SVG Illustrations + +Please use from now on for any SVG based illustrations simple `img` tags to show an illustration by simply using either `image_tag` or `image_path` helpers. Please use the class `svg-content` around it to ensure nice rendering. The illustrations are also organised in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository (as they are then automatically optimised). + +**Example** + +`= image_tag 'illustrations/merge_requests.svg'` diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 64bcb4a0257..031b12a8e91 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -29,34 +29,6 @@ For our currently-supported browsers, see our [requirements][requirements]. ## Development Process -When you are assigned an issue please follow the next steps: - -### Divide a big feature into small Merge Requests -1. Big Merge Request are painful to review. In order to make this process easier we -must break a big feature into smaller ones and create a Merge Request for each step. -1. First step is to create a branch from `master`, let's call it `new-feature`. This branch -will be the recipient of all the smaller Merge Requests. Only this one will be merged to master. -1. Don't do any work on this one, let's keep it synced with master. -1. Create a new branch from `new-feature`, let's call it `new-feature-step-1`. We advise you -to clearly identify which step the branch represents. -1. Do the first part of the modifications in this branch. The target branch of this Merge Request -should be `new-feature`. -1. Once `new-feature-step-1` gets merged into `new-feature` we can continue our work. Create a new -branch from `new-feature`, let's call it `new-feature-step-2` and repeat the process done before. - -```shell -master -└─ new-feature - ├─ new-feature-step-1 - ├─ new-feature-step-2 - └─ new-feature-step-3 -``` - -**Tips** -- Make sure `new-feature` branch is always synced with `master`: merge master frequently. -- Do the same for the feature branch you have opened. This can be accomplished by merging `master` into `new-feature` and `new-feature` into `new-feature-step-*` -- Avoid rewriting history. - ### Share your work early 1. Before writing code guarantee your vision of the architecture is aligned with GitLab's architecture. @@ -98,6 +70,11 @@ Vue specific design patterns and practices. --- +## [Icons](icons.md) +How we use SVG for our Icons. + +--- + ## Style Guides ### [JavaScript Style Guide](style_guide_js.md) diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 4f20aa070de..77ae6d2a0ea 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -311,6 +311,7 @@ A forEach will cause side effects, it will be mutating the array being iterated. #### Alignment 1. Follow these alignment styles for the template method: + 1. With more than one attribute, all attributes should be on a new line: ```javascript // bad <component v-if="bar" @@ -327,9 +328,16 @@ A forEach will cause side effects, it will be mutating the array being iterated. <button class="btn"> Click me </button> + ``` + 1. The tag can be inline if there is only one attribute: + ```javascript + // good + <component bar="bar" /> - // if props fit in one line then keep it on the same line - <component bar="bar" /> + // good + <component + bar="bar" + /> ``` #### Quotes @@ -381,9 +389,12 @@ A forEach will cause side effects, it will be mutating the array being iterated. } ``` -1. Default key should always be provided if the prop is not required: +1. Default key should be provided if the prop is not required. +_Note:_ There are some scenarios where we need to check for the existence of the property. +On those a default key should not be provided. + ```javascript - // bad + // good props: { foo: { type: String, @@ -459,7 +470,25 @@ A forEach will cause side effects, it will be mutating the array being iterated. ``` #### Ordering -1. Order for a Vue Component: + +1. Tag order in `.vue` file + + ``` + <script> + // ... + </script> + + <template> + // ... + </template> + + // We don't use scoped styles but there are few instances of this + <style> + // ... + </style> + ``` + +1. Properties in a Vue Component: 1. `name` 1. `props` 1. `mixins` @@ -479,6 +508,7 @@ A forEach will cause side effects, it will be mutating the array being iterated. 1. `beforeDestroy` 1. `destroyed` + #### Vue and Bootstrap 1. Tooltips: Do not rely on `has-tooltip` class name for Vue components @@ -512,11 +542,11 @@ A forEach will cause side effects, it will be mutating the array being iterated. ``` ### The Javascript/Vue Accord -The goal of this accord is to make sure we are all on the same page. +The goal of this accord is to make sure we are all on the same page. -1. When writing Vue, you may not use jQuery in your application. +1. When writing Vue, you may not use jQuery in your application. 1. If you need to grab data from the DOM, you may query the DOM 1 time while bootstrapping your application to grab data attributes using `dataset`. You can do this without jQuery. - 1. You may use a jQuery dependency in Vue.js following [this example from the docs](https://vuejs.org/v2/examples/select2.html). + 1. You may use a jQuery dependency in Vue.js following [this example from the docs](https://vuejs.org/v2/examples/select2.html). 1. If an outside jQuery Event needs to be listen to inside the Vue application, you may use jQuery event listeners. 1. We will avoid adding new jQuery events when they are not required. Instead of adding new jQuery events take a look at [different methods to do the same task](https://vuejs.org/v2/api/#vm-emit). 1. You may query the `window` object 1 time, while bootstrapping your application for application specific data (e.g. `scrollTo` is ok to access anytime). Do this access during the bootstrapping of your application. diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 2607353782a..277e0cd5f00 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -428,7 +428,7 @@ is a good example of this pattern. ## Style guide -Please refer to the Vue section of our [style guide](style_guide_js.md#vuejs) +Please refer to the Vue section of our [style guide](style_guide_js.md#vue-js) for best practices while writing your Vue components and templates. ## Testing Vue Components diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md new file mode 100644 index 00000000000..e41d258bec6 --- /dev/null +++ b/doc/development/gitaly.md @@ -0,0 +1,84 @@ +# GitLab Developers Guide to Working with Gitaly + +[Gitaly](https://gitlab.com/gitlab-org/gitaly) is a high-level Git RPC service used by GitLab CE/EE, +Workhorse and GitLab-Shell. All Rugged operations in GitLab CE/EE are currently being phased out to +be replaced by Gitaly API calls. + +Visit the [Gitaly Migration Board](https://gitlab.com/gitlab-org/gitaly/boards/331341) for current +status of the migration. + +## Feature Flags + +Gitaly makes heavy use of [feature flags](feature_flags.md). + +Each Rugged-to-Gitaly migration goes through a [series of phases](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/MIGRATION_PROCESS.md): + +* **Opt-In**: by default the Rugged implementation is used. + * Production instances can choose to enable the Gitaly endpoint by enabling the feature flag. + * For testing purposes, you may wish to enable all feature flags by default. This can be done by exporting the following + environment variable: `GITALY_FEATURE_DEFAULT_ON=1`. + * On developer instances (ie, when `Rails.env.development?` is true), the Gitaly endpoint + is enabled by default, but can be _disabled_ using feature flags. +* **Opt-Out**: by default, the Gitaly endpoint is used, but the feature can be explicitly disabled using the feature flag. +* **Mandatory**: The migration is complete and cannot be disabled. The old codepath is removed. + +### Enabling and Disabling Feature + +In the Rails console, type: + +```ruby +Feature.enable(:gitaly_feature_name) +Feature.disable(:gitaly_feature_name) +``` + +Where `gitaly_feature_name` is the name of the Gitaly feature. This can be determined by finding the appropriate +`gitaly_migrate` code block, for example: + +```ruby +gitaly_migrate(:tag_names) do +... +end +``` + +Since Gitaly features are always prefixed with `gitaly_`, the name of the feature flag in this case would be `gitaly_tag_names`. + +## Gitaly-Related Test Failures + +If your test-suite is failing with Gitaly issues, as a first step, try running: + +```shell +rm -rf tmp/tests/gitaly +``` + +## `TooManyInvocationsError` errors + +During development and testing, you may experience `Gitlab::GitalyClient::TooManyInvocationsError` failures. +The `GitalyClient` will attempt to block against potential n+1 issues by raising this error +when Gitaly is called more than 30 times in a single Rails request or Sidekiq execution. + +As a temporary measure, export `GITALY_DISABLE_REQUEST_LIMITS=1` to suppress the error. This will disable the n+1 detection +in your development environment. + +Please raise an issue in the GitLab CE or EE repositories to report the issue. Include the labels ~Gitaly +~performance ~"technical debt". Please ensure that the issue contains the full stack trace and error message of the +`TooManyInvocationsError`. Also include any known failing tests if possible. + +Isolate the source of the n+1 problem. This will normally be a loop that results in Gitaly being called for each +element in an array. If you are unable to isolate the problem, please contact a member +of the [Gitaly Team](https://gitlab.com/groups/gl-gitaly/group_members) for assistance. + +Once the source has been found, wrap it in an `allow_n_plus_1_calls` block, as follows: + +```ruby +# n+1: link to n+1 issue +Gitlab::GitalyClient.allow_n_plus_1_calls do + # original code + commits.each { |commit| ... } +end +``` + +Once the code is wrapped in this block, this code-path will be excluded from n+1 detection. + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md new file mode 100644 index 00000000000..167260b6e0e --- /dev/null +++ b/doc/development/i18n/externalization.md @@ -0,0 +1,296 @@ +# Internationalization for GitLab + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10669) in GitLab 9.2. + +For working with internationalization (i18n), +[GNU gettext](https://www.gnu.org/software/gettext/) is used given it's the most +used tool for this task and there are a lot of applications that will help us to +work with it. + +## Setting up GitLab Development Kit (GDK) + +In order to be able to work on the [GitLab Community Edition](https://gitlab.com/gitlab-org/gitlab-ce) +project you must download and configure it through [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/set-up-gdk.md). + +Once you have the GitLab project ready, you can start working on the translation. + +## Tools + +The following tools are used: + +1. [`gettext_i18n_rails`](https://github.com/grosser/gettext_i18n_rails): this + gem allow us to translate content from models, views and controllers. Also + it gives us access to the following raketasks: + - `rake gettext:find`: Parses almost all the files from the + Rails application looking for content that has been marked for + translation. Finally, it updates the PO files with the new content that + it has found. + - `rake gettext:pack`: Processes the PO files and generates the + MO files that are binary and are finally used by the application. + +1. [`gettext_i18n_rails_js`](https://github.com/webhippie/gettext_i18n_rails_js): + this gem is useful to make the translations available in JavaScript. It + provides the following raketask: + - `rake gettext:po_to_json`: Reads the contents from the PO files and + generates JSON files containing all the available translations. + +1. PO editor: there are multiple applications that can help us to work with PO + files, a good option is [Poedit](https://poedit.net/download) which is + available for macOS, GNU/Linux and Windows. + +## Preparing a page for translation + +We basically have 4 types of files: + +1. Ruby files: basically Models and Controllers. +1. HAML files: these are the view files. +1. ERB files: used for email templates. +1. JavaScript files: we mostly need to work with VUE JS templates. + +### Ruby files + +If there is a method or variable that works with a raw string, for instance: + +```ruby +def hello + "Hello world!" +end +``` + +Or: + +```ruby +hello = "Hello world!" +``` + +You can easily mark that content for translation with: + +```ruby +def hello + _("Hello world!") +end +``` + +Or: + +```ruby +hello = _("Hello world!") +``` + +### HAML files + +Given the following content in HAML: + +```haml +%h1 Hello world! +``` + +You can mark that content for translation with: + +```haml +%h1= _("Hello world!") +``` + +### ERB files + +Given the following content in ERB: + +```erb +<h1>Hello world!</h1> +``` + +You can mark that content for translation with: + +```erb +<h1><%= _("Hello world!") %></h1> +``` + +### JavaScript files + +In JavaScript we added the `__()` (double underscore parenthesis) function +for translations. + +### Updating the PO files with the new content + +Now that the new content is marked for translation, we need to update the PO +files with the following command: + +```sh +bundle exec rake gettext:find +``` + +This command will update the `locale/**/gitlab.edit.po` file with the +new content that the parser has found. + +New translations will be added with their default content and will be marked +fuzzy. To use the translation, look for the `#, fuzzy` mention in `gitlab.edit.po` +and remove it. + +We need to make sure we remove the `fuzzy` translations before generating the +`locale/**/gitlab.po` file. When they aren't removed, the resulting `.po` will +be treated as a binary file which could overwrite translations that were merged +before the new translations. + +When we are just preparing a page to be translated, but not actually adding any +translations. There's no need to generate `.po` files. + +Translations that aren't used in the source code anymore will be marked with +`~#`; these can be removed to keep our translation files clutter-free. + +### Validating PO files + +To make sure we keep our translation files up to date, there's a linter that is +running on CI as part of the `static-analysis` job. + +To lint the adjustments in PO files locally you can run `rake gettext:lint`. + +The linter will take the following into account: + +- Valid PO-file syntax +- Variable usage + - Only one unnamed (`%d`) variable, since the order of variables might change + in different languages + - All variables used in the message-id are used in the translation + - There should be no variables used in a translation that aren't in the + message-id +- Errors during translation. + +The errors are grouped per file, and per message ID: + +``` +Errors in `locale/zh_HK/gitlab.po`: + PO-syntax errors + SimplePoParser::ParserErrorSyntax error in lines + Syntax error in msgctxt + Syntax error in msgid + Syntax error in msgstr + Syntax error in message_line + There should be only whitespace until the end of line after the double quote character of a message text. + Parseing result before error: '{:msgid=>["", "You are going to remove %{project_name_with_namespace}.\\n", "Removed project CANNOT be restored!\\n", "Are you ABSOLUTELY sure?"]}' + SimplePoParser filtered backtrace: SimplePoParser::ParserError +Errors in `locale/zh_TW/gitlab.po`: + 1 pipeline + <%d 條流水線> is using unknown variables: [%d] + Failure translating to zh_TW with []: too few arguments +``` + +In this output the `locale/zh_HK/gitlab.po` has syntax errors. +The `locale/zh_TW/gitlab.po` has variables that are used in the translation that +aren't in the message with id `1 pipeline`. + +## Working with special content + +### Interpolation + +- In Ruby/HAML: + + ```ruby + _("Hello %{name}") % { name: 'Joe' } + ``` + +- In JavaScript: Not supported at this moment. + +### Plurals + +- In Ruby/HAML: + + ```ruby + n_('Apple', 'Apples', 3) => 'Apples' + ``` + + Using interpolation: + ```ruby + n_("There is a mouse.", "There are %d mice.", size) % size + ``` + +- In JavaScript: + + ```js + n__('Apple', 'Apples', 3) => 'Apples' + ``` + + Using interpolation: + + ```js + n__('Last day', 'Last %d days', 30) => 'Last 30 days' + ``` + +### Namespaces + +Sometimes you need to add some context to the text that you want to translate +(if the word occurs in a sentence and/or the word is ambiguous). + +- In Ruby/HAML: + + ```ruby + s_('OpenedNDaysAgo|Opened') + ``` + + In case the translation is not found it will return `Opened`. + +- In JavaScript: + + ```js + s__('OpenedNDaysAgo|Opened') + ``` + +### Just marking content for parsing + +Sometimes there are some dynamic translations that can't be found by the +parser when running `bundle exec rake gettext:find`. For these scenarios you can +use the [`_N` method](https://github.com/grosser/gettext_i18n_rails/blob/c09e38d481e0899ca7d3fc01786834fa8e7aab97/Readme.md#unfound-translations-with-rake-gettextfind). + +There is also and alternative method to [translate messages from validation errors](https://github.com/grosser/gettext_i18n_rails/blob/c09e38d481e0899ca7d3fc01786834fa8e7aab97/Readme.md#option-a). + +## Adding a new language + +Let's suppose you want to add translations for a new language, let's say French. + +1. The first step is to register the new language in `lib/gitlab/i18n.rb`: + + ```ruby + ... + AVAILABLE_LANGUAGES = { + ..., + 'fr' => 'Français' + }.freeze + ... + ``` + +1. Next, you need to add the language: + + ```sh + bundle exec rake gettext:add_language[fr] + ``` + + If you want to add a new language for a specific region, the command is similar, + you just need to separate the region with an underscore (`_`). For example: + + ```sh + bundle exec rake gettext:add_language[en_GB] + ``` + + Please note that you need to specify the region part in capitals. + +1. Now that the language is added, a new directory has been created under the + path: `locale/fr/`. You can now start using your PO editor to edit the PO file + located in: `locale/fr/gitlab.edit.po`. + +1. After you're done updating the translations, you need to process the PO files + in order to generate the binary MO files and finally update the JSON files + containing the translations: + + ```sh + bundle exec rake gettext:compile + ``` + +1. In order to see the translated content we need to change our preferred language + which can be found under the user's **Settings** (`/profile`). + +1. After checking that the changes are ok, you can proceed to commit the new files. + For example: + + ```sh + git add locale/fr/ app/assets/javascripts/locale/fr/ + git commit -m "Add French translations for Cycle Analytics page" + ``` diff --git a/doc/development/i18n/img/crowdin-editor.png b/doc/development/i18n/img/crowdin-editor.png Binary files differnew file mode 100644 index 00000000000..5c31d8f0cec --- /dev/null +++ b/doc/development/i18n/img/crowdin-editor.png diff --git a/doc/development/i18n/index.md b/doc/development/i18n/index.md new file mode 100644 index 00000000000..4cb2624c098 --- /dev/null +++ b/doc/development/i18n/index.md @@ -0,0 +1,76 @@ +# Translate GitLab to your language + +The text in GitLab's user interface is in American English by default. +Each string can be translated to other languages. +As each string is translated, it is added to the languages translation file, +and will be available in future releases of GitLab. + +Contributions to translations are always needed. +Many strings are not yet available for translation because they have not been externalized. +Helping externalize strings benefits all languages. +Some translations are incomplete or inconsistent. +Translating strings will help complete and improve each language. + +## How to contribute + +There are many ways you can contribute in translating GitLab. + +### Externalize strings + +Before a string can be translated, it must be externalized. +This is the process where English strings in the GitLab source code are wrapped in a function that +retrieves the translated string for the user's language. + +As new features are added and existing features are updated, the surrounding strings are being +externalized, however, there are many parts of GitLab that still need more work to externalize all +strings. + +See [Externalization for GitLab](externalization.md). + +### Translate strings + +The translation process is managed at [translate.gitlab.com](https://translate.gitlab.com) +using [Crowdin](https://crowdin.com/). +You will need to create an account before you can submit translations. +Once you are signed in, select the language you wish to contribute translations to. + +Voting for translations is also valuable, helping to confirm good and flag inaccurate translations. + +See [Translation guidelines](translation.md). + +### Proof reading + +Proof reading helps ensure the accuracy and consistency of translations. +All translations are proof read before being accepted. +If a translations requires changes, you will be notified with a comment explaining why. + +Community assistance proof reading translations is encouraged and appreciated. +Requests to become a proof reader will be considered on the merits of previous translations. + +- Bulgarian +- Chinese Simplified + - [Huang Tao](https://crowdin.com/profile/htve) +- Chinese Traditional + - [Huang Tao](https://crowdin.com/profile/htve) +- Chinese Traditional, Hong Kong + - [Huang Tao](https://crowdin.com/profile/htve) +- Dutch +- Esperanto +- French +- German +- Italian +- Japanese +- Korean + - [Huang Tao](https://crowdin.com/profile/htve) +- Portuguese, Brazilian +- Russian + - [Alexy Lustin](https://crowdin.com/profile/lustin) + - [Nikita Grylov](https://crowdin.com/profile/nixel2007) +- Spanish +- Ukrainian + +If you would like to be added as a proof reader, please [open an issue](https://gitlab.com/gitlab-org/gitlab-ce/issues). + +## Release + +Translations are typically included in the next major or minor release. diff --git a/doc/development/i18n/translation.md b/doc/development/i18n/translation.md new file mode 100644 index 00000000000..79707aaf671 --- /dev/null +++ b/doc/development/i18n/translation.md @@ -0,0 +1,64 @@ +# Translating GitLab + +For managing the translation process we use [Crowdin](https://crowdin.com). + +## Using Crowdin + +The first step is to get familiar with Crowdin. + +### Sign In + +To contribute translations at [translate.gitlab.com](https://translate.gitlab.com) +you must create a Crowdin account. +You may create a new account or use any of their supported sign in services. + +### Language Selections + +GitLab is being translated into many languages. + +1. Select the language you would like to contribute translations to by clicking the flag +1. You will see a list of files and folders. + Click `gitlab.pot` to open the translation editor. + +### Translation Editor + +The online translation editor is the easiest way to contribute translations. + + + +1. Strings for translation are listed in the left panel +1. Translations are entered into the central panel. + Multiple translations will be required for strings that contains plurals. + The string to be translated is shown above with glossary terms highlighted. + If the string to be translated is not clear, you can 'Request Context' + +A glossary of common terms is available in the right panel by clicking Terms. +Comments can be added to discuss a translation with the community. + +Remember to **Save** each translation. + +## Translation Guidelines + +Be sure to check the following guidelines before you translate any strings. + +### Technical terms + +Technical terms should be treated like proper nouns and not be translated. +This helps maintain a logical connection and consistency between tools (e.g. `git` client) and +GitLab. + +Technical terms that should always be in English are noted in the glossary when using +[translate.gitlab.com](https://translate.gitlab.com). + +### Formality + +The level of formality used in software varies by language. +For example, in French we translate `you` as the informal `tu`. + +You can refer to other translated strings and notes in the glossary to assist determining a +suitable level of formality. + +### Updating the glossary + +To propose additions to the glossary please +[open an issue](https://gitlab.com/gitlab-org/gitlab-ce/issues). diff --git a/doc/development/i18n_guide.md b/doc/development/i18n_guide.md index bd0ef39ca62..f6e949b5fd8 100644 --- a/doc/development/i18n_guide.md +++ b/doc/development/i18n_guide.md @@ -1,297 +1 @@ -# Internationalization for GitLab - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10669) in GitLab 9.2. - -For working with internationalization (i18n) we use -[GNU gettext](https://www.gnu.org/software/gettext/) given it's the most used -tool for this task and we have a lot of applications that will help us to work -with it. - -## Setting up GitLab Development Kit (GDK) - -In order to be able to work on the [GitLab Community Edition](https://gitlab.com/gitlab-org/gitlab-ce) project we must download and -configure it through [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit), we can do it by following this [guide](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/set-up-gdk.md). - -Once we have the GitLab project ready we can start working on the -translation of the project. - -## Tools - -We use a couple of gems: - -1. [`gettext_i18n_rails`](https://github.com/grosser/gettext_i18n_rails): this - gem allow us to translate content from models, views and controllers. Also - it gives us access to the following raketasks: - - `rake gettext:find`: Parses almost all the files from the - Rails application looking for content that has been marked for - translation. Finally, it updates the PO files with the new content that - it has found. - - `rake gettext:pack`: Processes the PO files and generates the - MO files that are binary and are finally used by the application. - -1. [`gettext_i18n_rails_js`](https://github.com/webhippie/gettext_i18n_rails_js): - this gem is useful to make the translations available in JavaScript. It - provides the following raketask: - - `rake gettext:po_to_json`: Reads the contents from the PO files and - generates JSON files containing all the available translations. - -1. PO editor: there are multiple applications that can help us to work with PO - files, a good option is [Poedit](https://poedit.net/download) which is - available for macOS, GNU/Linux and Windows. - -## Preparing a page for translation - -We basically have 4 types of files: - -1. Ruby files: basically Models and Controllers. -1. HAML files: these are the view files. -1. ERB files: used for email templates. -1. JavaScript files: we mostly need to work with VUE JS templates. - -### Ruby files - -If there is a method or variable that works with a raw string, for instance: - -```ruby -def hello - "Hello world!" -end -``` - -Or: - -```ruby -hello = "Hello world!" -``` - -You can easily mark that content for translation with: - -```ruby -def hello - _("Hello world!") -end -``` - -Or: - -```ruby -hello = _("Hello world!") -``` - -### HAML files - -Given the following content in HAML: - -```haml -%h1 Hello world! -``` - -You can mark that content for translation with: - -```haml -%h1= _("Hello world!") -``` - -### ERB files - -Given the following content in ERB: - -```erb -<h1>Hello world!</h1> -``` - -You can mark that content for translation with: - -```erb -<h1><%= _("Hello world!") %></h1> -``` - -### JavaScript files - -In JavaScript we added the `__()` (double underscore parenthesis) function -for translations. - -### Updating the PO files with the new content - -Now that the new content is marked for translation, we need to update the PO -files with the following command: - -```sh -bundle exec rake gettext:find -``` - -This command will update the `locale/**/gitlab.edit.po` file with the -new content that the parser has found. - -New translations will be added with their default content and will be marked -fuzzy. To use the translation, look for the `#, fuzzy` mention in `gitlab.edit.po` -and remove it. - -We need to make sure we remove the `fuzzy` translations before generating the -`locale/**/gitlab.po` file. When they aren't removed, the resulting `.po` will -be treated as a binary file which could overwrite translations that were merged -before the new translations. - -When we are just preparing a page to be translated, but not actually adding any -translations. There's no need to generate `.po` files. - -Translations that aren't used in the source code anymore will be marked with -`~#`; these can be removed to keep our translation files clutter-free. - -### Validating PO files - -To make sure we keep our translation files up to date, there's a linter that is -running on CI as part of the `static-analysis` job. - -To lint the adjustments in PO files locally you can run `rake gettext:lint`. - -The linter will take the following into account: - -- Valid PO-file syntax -- Variable usage - - Only one unnamed (`%d`) variable, since the order of variables might change - in different languages - - All variables used in the message-id are used in the translation - - There should be no variables used in a translation that aren't in the - message-id -- Errors during translation. - -The errors are grouped per file, and per message ID: - -``` -Errors in `locale/zh_HK/gitlab.po`: - PO-syntax errors - SimplePoParser::ParserErrorSyntax error in lines - Syntax error in msgctxt - Syntax error in msgid - Syntax error in msgstr - Syntax error in message_line - There should be only whitespace until the end of line after the double quote character of a message text. - Parseing result before error: '{:msgid=>["", "You are going to remove %{project_name_with_namespace}.\\n", "Removed project CANNOT be restored!\\n", "Are you ABSOLUTELY sure?"]}' - SimplePoParser filtered backtrace: SimplePoParser::ParserError -Errors in `locale/zh_TW/gitlab.po`: - 1 pipeline - <%d 條流水線> is using unknown variables: [%d] - Failure translating to zh_TW with []: too few arguments -``` - -In this output the `locale/zh_HK/gitlab.po` has syntax errors. -The `locale/zh_TW/gitlab.po` has variables that are used in the translation that -aren't in the message with id `1 pipeline`. - -## Working with special content - -### Interpolation - -- In Ruby/HAML: - - ```ruby - _("Hello %{name}") % { name: 'Joe' } - ``` - -- In JavaScript: Not supported at this moment. - -### Plurals - -- In Ruby/HAML: - - ```ruby - n_('Apple', 'Apples', 3) => 'Apples' - ``` - - Using interpolation: - ```ruby - n_("There is a mouse.", "There are %d mice.", size) % size - ``` - -- In JavaScript: - - ```js - n__('Apple', 'Apples', 3) => 'Apples' - ``` - - Using interpolation: - - ```js - n__('Last day', 'Last %d days', 30) => 'Last 30 days' - ``` - -### Namespaces - -Sometimes you need to add some context to the text that you want to translate -(if the word occurs in a sentence and/or the word is ambiguous). - -- In Ruby/HAML: - - ```ruby - s_('OpenedNDaysAgo|Opened') - ``` - - In case the translation is not found it will return `Opened`. - -- In JavaScript: - - ```js - s__('OpenedNDaysAgo|Opened') - ``` - -### Just marking content for parsing - -Sometimes there are some dynamic translations that can't be found by the -parser when running `bundle exec rake gettext:find`. For these scenarios you can -use the [`_N` method](https://github.com/grosser/gettext_i18n_rails/blob/c09e38d481e0899ca7d3fc01786834fa8e7aab97/Readme.md#unfound-translations-with-rake-gettextfind). - -There is also and alternative method to [translate messages from validation errors](https://github.com/grosser/gettext_i18n_rails/blob/c09e38d481e0899ca7d3fc01786834fa8e7aab97/Readme.md#option-a). - -## Adding a new language - -Let's suppose you want to add translations for a new language, let's say French. - -1. The first step is to register the new language in `lib/gitlab/i18n.rb`: - - ```ruby - ... - AVAILABLE_LANGUAGES = { - ..., - 'fr' => 'Français' - }.freeze - ... - ``` - -1. Next, you need to add the language: - - ```sh - bundle exec rake gettext:add_language[fr] - ``` - - If you want to add a new language for a specific region, the command is similar, - you just need to separate the region with an underscore (`_`). For example: - - ```sh - bundle exec rake gettext:add_language[en_GB] - ``` - - Please note that you need to specify the region part in capitals. - -1. Now that the language is added, a new directory has been created under the - path: `locale/fr/`. You can now start using your PO editor to edit the PO file - located in: `locale/fr/gitlab.edit.po`. - -1. After you're done updating the translations, you need to process the PO files - in order to generate the binary MO files and finally update the JSON files - containing the translations: - - ```sh - bundle exec rake gettext:compile - ``` - -1. In order to see the translated content we need to change our preferred language - which can be found under the user's **Settings** (`/profile`). - -1. After checking that the changes are ok, you can proceed to commit the new files. - For example: - - ```sh - git add locale/fr/ app/assets/javascripts/locale/fr/ - git commit -m "Add French translations for Cycle Analytics page" - ``` +This document was moved to [a new location](i18n/index.md). diff --git a/doc/development/img/manual_build_docs.png b/doc/development/img/manual_build_docs.png Binary files differindex fef767c2a79..615facabb5f 100644 --- a/doc/development/img/manual_build_docs.png +++ b/doc/development/img/manual_build_docs.png diff --git a/doc/development/licensing.md b/doc/development/licensing.md index 9a5811d8474..a75cdf22f40 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -65,6 +65,7 @@ Libraries with the following licenses are unacceptable for use: - [GNU AGPLv3][AGPLv3]: AGPL-licensed libraries cannot be linked to from non-GPL projects. - [Open Software License (OSL)][OSL]: is a copyleft license. In addition, the FSF [recommend against its use][OSL-GNU]. - [Facebook BSD + PATENTS][Facebook]: is a 3-clause BSD license with a patent grant that has been deemed [Category X][x-list] by the Apache foundation. +- [WTFPL][WTFPL]: is a public domain dedication [rejected by the OSI (3.2)][WTFPL-OSI]. Also has a strong language which is not in accordance with our diversity policy. ## Requesting Approval for Licenses @@ -108,3 +109,5 @@ Gems which are included only in the "development" or "test" groups by Bundler ar [x-list]: https://www.apache.org/legal/resolved.html#category-x [Acceptable-Licenses]: #acceptable-licenses [Unacceptable-Licenses]: #unacceptable-licenses +[WTFPL]: https://wtfpl.net +[WTFPL-OSI]: https://opensource.org/minutes20090304 diff --git a/doc/development/swapping_tables.md b/doc/development/swapping_tables.md new file mode 100644 index 00000000000..6b990ece72c --- /dev/null +++ b/doc/development/swapping_tables.md @@ -0,0 +1,53 @@ +# Swapping Tables + +Sometimes you need to replace one table with another. For example, when +migrating data in a very large table it's often better to create a copy of the +table and insert & migrate the data into this new table in the background. + +Let's say you want to swap the table "events" with "events_for_migration". In +this case you need to follow 3 steps: + +1. Rename "events" to "events_temporary" +2. Rename "events_for_migration" to "events" +3. Rename "events_temporary" to "events_for_migration" + +Rails allows you to do this using the `rename_table` method: + +```ruby +rename_table :events, :events_temporary +rename_table :events_for_migration, :events +rename_table :events_temporary, :events_for_migration +``` + +This does not require any downtime as long as the 3 `rename_table` calls are +executed in the _same_ database transaction. Rails by default uses database +transactions for migrations, but if it doesn't you'll need to start one +manually: + +```ruby +Event.transaction do + rename_table :events, :events_temporary + rename_table :events_for_migration, :events + rename_table :events_temporary, :events_for_migration +end +``` + +Once swapped you _have to_ reset the primary key of the new table. For +PostgreSQL you can use the `reset_pk_sequence!` method like so: + +```ruby +reset_pk_sequence!('events') +``` + +For MySQL however you need to do run the following: + +```ruby +amount = Event.pluck('COALESCE(MAX(id), 1)').first + +execute "ALTER TABLE events AUTO_INCREMENT = #{amount}" +``` + +Failure to reset the primary keys will result in newly created rows starting +with an ID value of 1. Depending on the existing data this can then lead to +duplicate key constraints from popping up, preventing users from creating new +data. diff --git a/doc/development/testing.md b/doc/development/testing.md index 83269303005..4d5b90de6fc 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -150,6 +150,16 @@ always in-sync with the codebase. [GitLab QA]: https://gitlab.com/gitlab-org/gitlab-qa [part of GitLab Rails]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa +## Test for what should not be there + +This is particularly important for permission calls and might be called a +negative assertion: make sure only the bare minimum is returned and nothing else. + +See an issue about [leaking tokens] as an example of a vulnerability that is +captured by such a test. + +[leaking tokens]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37948 + ## How to test at the correct level? As many things in life, deciding what to test at each level of testing is a @@ -292,7 +302,7 @@ range of inputs, might look like this: ```ruby describe "#==" do - using Rspec::Parameterized::TableSyntax + using RSpec::Parameterized::TableSyntax let(:project1) { create(:project) } let(:project2) { create(:project) } @@ -493,24 +503,24 @@ Here are some things to keep in mind regarding test performance: Our current CI parallelization setup is as follows: -1. The `knapsack` job in the prepare stage that is supposed to ensure we have a - `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file: +1. The `retrieve-tests-metadata` job in the `prepare` stage ensures that we have + a `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file: - The `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file is fetched from S3, if it's not here we initialize the file with `{}`. -1. Each `rspec x y` job are run with `knapsack rspec` and should have an evenly - distributed share of tests: +1. Each `rspec-pg x y`/`rspec-mysql x y` job is run with `knapsack rspec` and + should have an evenly distributed share of tests: - It works because the jobs have access to the `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` since the "artifacts from all previous stages are passed by default". [^1] - - the jobs set their own report path to + - The jobs set their own report path to `KNAPSACK_REPORT_PATH=knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json`. - - if knapsack is doing its job, test files that are run should be listed under + - If knapsack is doing its job, test files that are run should be listed under `Report specs`, not under `Leftover specs`. -1. The `update-knapsack` job takes all the +1. The `update-tests-metadata` job takes all the `knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json` - files from the `rspec x y` jobs and merge them all together into a single - `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that is then - uploaded to S3. + files from the `rspec-pg x y`/`rspec-mysql x y`jobs and merge them all together + into a single `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that + is then uploaded to S3. After that, the next pipeline will use the up-to-date `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. The same strategy diff --git a/doc/development/ux_guide/animation.md b/doc/development/ux_guide/animation.md index 5dae4bcc905..d190ee1b0ff 100644 --- a/doc/development/ux_guide/animation.md +++ b/doc/development/ux_guide/animation.md @@ -39,6 +39,12 @@ When information is updating in place, a quick, subtle animation is needed. The  +### Skeleton loading + +Skeleton loading is explained in the [component section](components.html#skeleton-loading) of the UX guide. It includes a horizontally pulsating animation that shows motion as if it's growing. It's timing is a slower `linear 1s`. + + + ### Moving transitions When elements move on screen, there should be a quick animation so it is clear to users what moved where. The timing of this animation differs based on the amount of movement and change. Consider animations between `200ms` and `400ms`. @@ -51,7 +57,9 @@ View the [interactive example](http://codepen.io/awhildy/full/ALyKPE/) here.  #### Autoscroll the page + Another example of a moving transition is when you have to autoscroll the page to keep an active element visible. View the [interactive example](http://codepen.io/awhildy/full/PbxgVo/) here. -
\ No newline at end of file + + diff --git a/doc/development/ux_guide/basics.md b/doc/development/ux_guide/basics.md index a436e9b1948..e215026bcca 100644 --- a/doc/development/ux_guide/basics.md +++ b/doc/development/ux_guide/basics.md @@ -32,19 +32,17 @@ This is the typeface used for code blocks and references to commits, branches, a --- ## Icons -GitLab uses Font Awesome icons throughout our interface. -| | | -| :-----------: | :---- | -|  | The trash icon is used for destructive actions that deletes information. | -|  | The pencil icon is used for editing content such as comments.| -|  | The bell icon is for notifications, such as Todos. | -|  | The eye icon is for subscribing to updates. For example, you can subscribe to a label and get updated on issues with that label. | -|  | The standard RSS icon is used for linking to RSS/atom feeds. | -|  | An 'x' is used for closing UI elements such as dropdowns. | -|  | A plus is used when creating new objects, such as issues, projects, etc. | - -> TODO: update this section, add more general guidance to icon usage and personality, etc. +GitLab has a strong, unique personality. When you look at any screen, you should know immediately that it is GitLab. +Iconography is a powerful visual cue to the user and is a great way for us to reflect our particular sense of style. + +- **Standard size:** 16px * 16px +- **Border thickness:** 2px +- **Border radius:** 3px + + + +> TODO: List all icons, proper usage, hover, and active states. --- diff --git a/doc/development/ux_guide/components.md b/doc/development/ux_guide/components.md index ac7c1b6207d..fa31c496b30 100644 --- a/doc/development/ux_guide/components.md +++ b/doc/development/ux_guide/components.md @@ -42,6 +42,37 @@ By default, tooltips should be placed below the referring element. However, if t --- +## Popovers + +Popovers provide additional, useful, unique information about the referring elements and can provide one or multiple actionable elements. They inform the user of additional information within the context of their original view, but without forcing the user to act upon it like a modal. Popovers are different from tooltips, which do not provide rich markup and actionable items. A popover can contain a header section with a different background color. + +Popovers are summoned: + +* Upon hover or touch on an element + +### Usage +A popover should be used: +* When you don't want to let the user lose context, but still want to provide additional useful unique information about referring elements +* When it isn’t critical for the user to act upon the information +* When you want to give a user a summary of extended information and the option to switch context if they want to dive in deeper. + +### Styling + +A popover can contain a header section with a different background color if that improves readability and separation of content within. + + + +This example shows two sections, where each section includes an actionable element. The first section shows a summary of the content shown when clicking the "read more" link. With this information the user can decide to dive deeper or start their GitLab Enterprise Edition trial immediately. + +### Placement +By default, tooltips should be placed below the referring element. However, if there isn’t enough space in the viewport or it blocks related content, the tooltip should be moved to the side or above as needed. + + + +In this example we let the user know more about the setting they are deciding over, without loosing context. If they want to know even more they can do so, but with the expectation of opening that content in a new view. + +--- + ## Anchor links Anchor links are used for navigational actions and lone, secondary commands (such as 'Reset filters' on the Issues List) when deemed appropriate by the UX team. @@ -204,6 +235,25 @@ Cover blocks are generally used to create a heading element for a page, such as --- +## Skeleton loading + +Skeleton loading is a way to convey to the user what kind of content is currently being loaded. It's a paradigm with which content can independently and asynchronously be loaded, while still adhering to the structure and look of the completely loaded view. + +### Requirements + +* A skeleton should represent an organism in a recognisable way +* Atom elements within organisms (for reference see this article on [atomic design methodology](http://atomicdesign.bradfrost.com/chapter-2/)) may be represented in a maximum of 3 repetitions, if applicable. +* Skeletons should only be presented in grayscale using the HEX colors: `#fafafa` or `#ffffff` (except for shadows) +* Animate the grey atoms in a pulsating way to show motion, as if "loading". The pulse animation transitions colors horizontally from left to right, starting with `#f2f2f2` to `#fafafa`. + + + +### Usage + +Skeleton loading can replace any existing UI elements for the period in which they are loaded and should aim for maintaining a similar structure visually. + +--- + ## Panels > TODO: Catalog how we are currently using panels and rationalize how they relate to alerts diff --git a/doc/development/ux_guide/illustrations.md b/doc/development/ux_guide/illustrations.md new file mode 100644 index 00000000000..7e16c300921 --- /dev/null +++ b/doc/development/ux_guide/illustrations.md @@ -0,0 +1,86 @@ +# Illustrations + +The illustrations should always align with topics and goals in specific context. + +## Principles + +#### Be simple. +- For clarity, we use simple and specific elements to create our illustrations. + +#### Be optimistic. +- We are an open-minded, optimistic, and friendly team. We should reflect those values in our illustrations to connect with our brand experience. + +#### Be gentle. +- Our illustrations assist users in understanding context and guide users in the right direction. Illustrations are supportive, so they should be obvious but not aggressive. + + +## Style + +#### Shapes +- All illustrations are geometric rather than organic. +- The illustrations are made by circles, rectangles, squares, and triangles. + +<img src="img/illustrations-geometric.png" width=224px alt="Example for geometric" /> + +#### Stroke +- Standard border thickness: **4px** +- Depending on the situation, border thickness can be changed to **3px**. For example, when the illustration size is small, an illustration with 4px border thickness would look tight. In this case, the border thickness can be changed to 3px. +- We use **rounded caps** and **rounded corner**. + +| Do | Don't | +| -------- | -------- | +| <img src="img/illustrations-caps-do.png" width= 133px alt="Do: caps and corner" /> | <img src="img/illustrations-caps-don't.png" width= 133px alt="Don't: caps and corner"/> | + +#### Radius +- Standard corner radius: **10px** +- Depending on the situation, corner radius can be changed to **5px**. For example, when the illustration size is small, an illustration with 10px corner radius would be over-rounded. In this case, the corner radius can be changed to 5px. + +<img src="img/illustrations-border-radius.png" width= 464px alt="Example for border radius"/> + +#### Size +Depends on the situation, the illustration size can be the 3 types below: + +**Large** +* Use case: Empty states, error pages(e.g. 404, 403) +* For vertical layout, the illustration should not larger than **430*300 px**. +* For horizontal layout, the illustration should not larger than **430*380 px**. + +| Vertical layout | Horizontal layout | +| --------------- | ----------------- | +| <img src="img/illustration-size-large-vertical.png" /> | <img src="img/illustration-size-large-horizontal.png" /> + +**Medium** +* Use case: Banner +* The illustration should not larger than **240*160 px** +* The illustration should keep simple and clear. We recommend not including too many elements in the medium size illustration. + +<img src="img/illustration-size-medium.png" width=983px /> + +**Small** +* Use case: Graphics for explanatory text, graphics for status. +* The illustration should not larger than **160*90 px**. +* The illustration should keep simple and clear. We recommend not including too many elements in the small size illustration. + +<img src="img/illustration-size-small.png" width=983px /> + +**Illustration on mobile** +- Keep the proportions in original ratio. + +#### Colors palette + +For consistency, we recommend choosing colors from our color palette. + +| Orange | Purple | Grey | +| -------- | -------- | -------- | +| <img src="img/illustrations-color-orange.png" width= 160px alt="Orange" /> | <img src="img/illustrations-color-purple.png" width= 160px alt="Purple" /> | <img src="img/illustrations-color-grey.png" width= 160px alt="Grey" /> | +| #FC6D26 | #6B4FBB | #EEEEEE | + +#### Don't +- Don't include the typography in the illustration. +- Don't include tanuki in the illustration. If necessary, we recommend having tanuki monochromatic. + +--- + +| Orange | Purple | +| -------- | -------- | +| <img src="img/illustrations-palette-oragne.png" width= 160px alt="Palette - Orange" /> | <img src="img/illustrations-palette-purple.png" width= 160px alt="Palette - Purple" /> | diff --git a/doc/development/ux_guide/img/icon-spec.png b/doc/development/ux_guide/img/icon-spec.png Binary files differnew file mode 100644 index 00000000000..56b19610dc1 --- /dev/null +++ b/doc/development/ux_guide/img/icon-spec.png diff --git a/doc/development/ux_guide/img/illustration-size-large-horizontal.png b/doc/development/ux_guide/img/illustration-size-large-horizontal.png Binary files differnew file mode 100644 index 00000000000..8aa835adccc --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-large-horizontal.png diff --git a/doc/development/ux_guide/img/illustration-size-large-vertical.png b/doc/development/ux_guide/img/illustration-size-large-vertical.png Binary files differnew file mode 100644 index 00000000000..813b6a065e5 --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-large-vertical.png diff --git a/doc/development/ux_guide/img/illustration-size-medium.png b/doc/development/ux_guide/img/illustration-size-medium.png Binary files differnew file mode 100644 index 00000000000..55cfe1dcb91 --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-medium.png diff --git a/doc/development/ux_guide/img/illustration-size-small.png b/doc/development/ux_guide/img/illustration-size-small.png Binary files differnew file mode 100644 index 00000000000..0124f58f48e --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-small.png diff --git a/doc/development/ux_guide/img/illustrations-border-radius.png b/doc/development/ux_guide/img/illustrations-border-radius.png Binary files differnew file mode 100644 index 00000000000..4e2fef5c7f5 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-border-radius.png diff --git a/doc/development/ux_guide/img/illustrations-caps-do.png b/doc/development/ux_guide/img/illustrations-caps-do.png Binary files differnew file mode 100644 index 00000000000..7a2c74382f6 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-caps-do.png diff --git a/doc/development/ux_guide/img/illustrations-caps-don't.png b/doc/development/ux_guide/img/illustrations-caps-don't.png Binary files differnew file mode 100644 index 00000000000..848f72dbe30 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-caps-don't.png diff --git a/doc/development/ux_guide/img/illustrations-color-grey.png b/doc/development/ux_guide/img/illustrations-color-grey.png Binary files differnew file mode 100644 index 00000000000..63855026c2b --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-grey.png diff --git a/doc/development/ux_guide/img/illustrations-color-orange.png b/doc/development/ux_guide/img/illustrations-color-orange.png Binary files differnew file mode 100644 index 00000000000..96765c8c28c --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-orange.png diff --git a/doc/development/ux_guide/img/illustrations-color-purple.png b/doc/development/ux_guide/img/illustrations-color-purple.png Binary files differnew file mode 100644 index 00000000000..745d2c853ba --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-purple.png diff --git a/doc/development/ux_guide/img/illustrations-geometric.png b/doc/development/ux_guide/img/illustrations-geometric.png Binary files differnew file mode 100644 index 00000000000..33f05547bac --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-geometric.png diff --git a/doc/development/ux_guide/img/illustrations-palette-oragne.png b/doc/development/ux_guide/img/illustrations-palette-oragne.png Binary files differnew file mode 100644 index 00000000000..15f35912646 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-palette-oragne.png diff --git a/doc/development/ux_guide/img/illustrations-palette-purple.png b/doc/development/ux_guide/img/illustrations-palette-purple.png Binary files differnew file mode 100644 index 00000000000..e0f5839705e --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-palette-purple.png diff --git a/doc/development/ux_guide/img/popover-placement-above.png b/doc/development/ux_guide/img/popover-placement-above.png Binary files differnew file mode 100644 index 00000000000..1aa044bfc9c --- /dev/null +++ b/doc/development/ux_guide/img/popover-placement-above.png diff --git a/doc/development/ux_guide/img/popover-placement-below.png b/doc/development/ux_guide/img/popover-placement-below.png Binary files differnew file mode 100644 index 00000000000..2d6ab8a1618 --- /dev/null +++ b/doc/development/ux_guide/img/popover-placement-below.png diff --git a/doc/development/ux_guide/img/skeleton-loading.gif b/doc/development/ux_guide/img/skeleton-loading.gif Binary files differnew file mode 100644 index 00000000000..5877139171d --- /dev/null +++ b/doc/development/ux_guide/img/skeleton-loading.gif diff --git a/doc/development/ux_guide/index.md b/doc/development/ux_guide/index.md index 8a849f239dc..42bcf234e12 100644 --- a/doc/development/ux_guide/index.md +++ b/doc/development/ux_guide/index.md @@ -21,6 +21,11 @@ Guidance on the timing, curving and motion for GitLab. --- +### [Illustrations](illustrations.md) +Guidelines for principals and styles related to illustrations for GitLab. + +--- + ### [Copy](copy.md) Conventions on text and messaging within labels, buttons, and other components. diff --git a/doc/development/verifying_database_capabilities.md b/doc/development/verifying_database_capabilities.md index cc6d62957e3..ffdeff47d4a 100644 --- a/doc/development/verifying_database_capabilities.md +++ b/doc/development/verifying_database_capabilities.md @@ -24,3 +24,15 @@ else run_query end ``` + +# Read-only database + +The database can be used in read-only mode. In this case we have to +make sure all GET requests don't attempt any write operations to the +database. If one of those requests wants to write to the database, it needs +to be wrapped in a `Gitlab::Database.read_only?` or `Gitlab::Database.read_write?` +guard, to make sure it doesn't for read-only databases. + +We have a Rails Middleware that filters any potentially writing +operations (the CUD operations of CRUD) and prevent the user from trying +to update the database and getting a 500 error (see `Gitlab::Middleware::ReadOnly`). diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 479258f743e..68ba3dd2da3 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -1,6 +1,6 @@ # Writing documentation - - **General Documentation**: written by the developers responsible by creating features. Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. + - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. - **Technical Articles**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). @@ -69,6 +69,51 @@ Use the [writing method](https://about.gitlab.com/handbook/product/technical-wri All the docs follow the same [styleguide](doc_styleguide.md). +### Contributing to docs + +Whenever a feature is changed, updated, introduced, or deprecated, the merge +request introducing these changes must be accompanied by the documentation +(either updating existing ones or creating new ones). This is also valid when +changes are introduced to the UI. + +The one resposible for writing the first piece of documentation is the developer who +wrote the code. It's the job of the Product Manager to ensure all features are +shipped with its docs, whether is a small or big change. At the pace GitLab evolves, +this is the only way to keep the docs up-to-date. If you have any questions about it, +please ask a Technical Writer. Otherwise, when your content is ready, assign one of +them to review it for you. + +We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything +is documented. + +### Feature overview and use cases + +Every major feature (regardless if present in GitLab Community or Enterprise editions) +should present, at the beginning of the document, two main sections: **overview** and +**use cases**. Every GitLab EE-only feature should also contain these sections. + +**Overview**: at the name suggests, the goal here is to provide an overview of the feature. +Describe what is it, what it does, why it is important/cool/nice-to-have, +what problem it solves, and what you can do with this feature that you couldn't +do before. + +**Use cases**: provide at least two, ideally three, use cases for every major feature. +You should answer this question: what can you do with this feature/change? Use cases +are examples of how this feauture or change can be used in real life. + +Examples: +- CE and EE: [Issues](../user/project/issues/index.md#use-cases) +- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) +- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) +- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) + +Note that if you don't have anything to add between the doc title (`<h1>`) and +the header `## Overview`, you can omit the header, but keep the content of the +overview there. + +> **Overview** and **use cases** are required to **every** Enterprise Edition feature, +and for every **major** feature present in Community Edition. + ### Markdown Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. @@ -106,21 +151,84 @@ CE and EE. ## Previewing the changes live -If you want to preview your changes live, you can use the manual `build-docs` -job in your merge request. +If you want to preview the doc changes of your merge request live, you can use +the manual `review-docs-deploy` job in your merge request. + +TIP: **Tip:** +If your branch contains only documentation changes, you can use +[special branch names](#testing) to avoid long running pipelines.  This job will: 1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) - project named after the scheme: `<CE/EE-branch-slug>-built-from-ce-ee` -1. Trigger a pipeline and build the docs site with your changes - -Look for the docs URL at the output of the `build-docs` job. - ->**Note:** + project named after the scheme: `preview-<branch-slug>` +1. Trigger a cross project pipeline and build the docs site with your changes + +After a few minutes, the Review App will be deployed and you will be able to +preview the changes. The docs URL can be found in two places: + +- In the merge request widget +- In the output of the `review-docs-deploy` job, which also includes the + triggered pipeline so that you can investigate whether something went wrong + +In case the Review App URL returns 404, follow these steps to debug: + +1. **Did you follow the URL from the merge request widget?** If yes, then check if + the link is the same as the one in the job output. It can happen that if the + branch name slug is longer than 35 characters, it is automatically + truncated. That means that the merge request widget will not show the proper + URL due to a limitation of how `environment: url` works, but you can find the + real URL from the output of the `review-docs-deploy` job. +1. **Did you follow the URL from the job output?** If yes, then it means that + either the site is not yet deployed or something went wrong with the remote + pipeline. Give it a few minutes and it should appear online, otherwise you + can check the status of the remote pipeline from the link in the job output. + If the pipeline failed or got stuck, drop a line in the `#docs` chat channel. + +TIP: **Tip:** +Someone that has no merge rights to the CE/EE projects (think of forks from +contributors) will not be able to run the manual job. In that case, you can +ask someone from the GitLab team who has the permissions to do that for you. + +NOTE: **Note:** Make sure that you always delete the branch of the merge request you were working on. If you don't, the remote docs branch won't be removed either, and the server where the Review Apps are hosted will eventually be out of disk space. + +### Behind the scenes + +If you want to know the hot details, here's what's really happening: + +1. You manually run the `review-docs-deploy` job in a CE/EE merge request. +1. The job runs the [`scirpts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) + script with the `deploy` flag, which in turn: + 1. Takes your branch name and applies the following: + - The slug of the branch name is used to avoid special characters since + ultimately this will be used by NGINX. + - The `preview-` prefix is added to avoid conflicts if there's a remote branch + with the same name that you created in the merge request. + - The final branch name is truncated to 42 characters to avoid filesystem + limitations with long branch names (> 63 chars). + 1. The remote branch is then created if it doesn't exist (meaning you can + re-run the manual job as many times as you want and this step will be skipped). + 1. A new cross-project pipeline is triggered in the docs project. + 1. The preview URL is shown both at the job output and in the merge request + widget. You also get the link to the remote pipeline. +1. In the docs project, the pipeline is created and it + [skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55) + to lower the build time. +1. Once the docs site is built, the HTML files are uploaded as artifacts. +1. A specific Runner tied only to the docs project, runs the Review App job + that downloads the artifacts and uses `rsync` to transfer the files over + to a location where NGINX serves them. + +The following GitLab features are used among others: + +- [Manual actions](../ci/yaml/README.md#manual-actions) +- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Review Apps](../ci/review_apps/index.md) +- [Artifacts](../ci/yaml/README.md#artifacts) +- [Specific Runner](../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index bc75dc1447e..f9ba1508705 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -1,11 +1,12 @@ # Database MySQL >**Note:** -We do not recommend using MySQL due to various issues. For example, case +- We do not recommend using MySQL due to various issues. For example, case [(in)sensitivity](https://dev.mysql.com/doc/refman/5.0/en/case-sensitivity.html) and [problems](https://bugs.mysql.com/bug.php?id=65830) that [suggested](https://bugs.mysql.com/bug.php?id=50909) [fixes](https://bugs.mysql.com/bug.php?id=65830) [have](https://bugs.mysql.com/bug.php?id=63164). +- We recommend using MySQL version 5.6 or later. Please see the following [issue][ce-38152]. ## Initial database setup @@ -13,7 +14,7 @@ and [problems](https://bugs.mysql.com/bug.php?id=65830) that # Install the database packages sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev -# Ensure you have MySQL version 5.5.14 or later +# Ensure you have MySQL version 5.6 or later mysql --version # Pick a MySQL root password (can be anything), type it and press enter @@ -75,7 +76,7 @@ log_bin_trust_function_creators=1 ### MySQL utf8mb4 support -After installation or upgrade, remember to [convert any new tables](#convert) to `utf8mb4`/`utf8mb4_general_ci`. +After installation or upgrade, remember to [convert any new tables](#tables-and-data-conversion-to-utf8mb4) to `utf8mb4`/`utf8mb4_general_ci`. --- @@ -230,7 +231,6 @@ We need to check, enable and probably convert your existing GitLab DB tables to > Now, ensure that [innodb_file_format](https://dev.mysql.com/doc/refman/5.6/en/tablespace-enabling.html) and [innodb_large_prefix](http://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_large_prefix) are **persisted** in your `my.cnf` file. #### Tables and data conversion to utf8mb4 -<a name="convert"></a> Now that you have a persistent MySQL setup, you can safely upgrade tables after setup or upgrade time: @@ -294,3 +294,4 @@ Details can be found in the [PostgreSQL][postgres-text-type] and [postgres-text-type]: http://www.postgresql.org/docs/9.2/static/datatype-character.html [mysql-text-types]: http://dev.mysql.com/doc/refman/5.7/en/string-type-overview.html +[ce-38152]: https://gitlab.com/gitlab-org/gitlab-ce/issues/38152 diff --git a/doc/install/installation.md b/doc/install/installation.md index 66eb7675896..af6c797dc00 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -133,9 +133,9 @@ Remove the old Ruby 1.8 if present: Download Ruby and compile it: mkdir /tmp/ruby && cd /tmp/ruby - curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.3.tar.gz - echo '1014ee699071aa2ddd501907d18cbe15399c997d ruby-2.3.3.tar.gz' | shasum -c - && tar xzf ruby-2.3.3.tar.gz - cd ruby-2.3.3 + curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.5.tar.gz + echo '3247e217d6745c27ef23bdc77b6abdb4b57a118f ruby-2.3.5.tar.gz' | shasum -c - && tar xzf ruby-2.3.5.tar.gz + cd ruby-2.3.5 ./configure --disable-install-rdoc make sudo make install @@ -299,9 +299,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 9-5-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-0-stable gitlab -**Note:** You can change `9-5-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `10-0-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It diff --git a/doc/install/kubernetes/gitlab_chart.md b/doc/install/kubernetes/gitlab_chart.md index a339bc23809..ddfd47df099 100644 --- a/doc/install/kubernetes/gitlab_chart.md +++ b/doc/install/kubernetes/gitlab_chart.md @@ -1,9 +1,14 @@ # GitLab Helm Chart -> **Note:** -* > **Note**: This chart will be replaced by the [gitlab-omnibus](gitlab_omnibus.md) chart, once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). -* Officially supported cloud providers are Google Container Service and Azure Container Service. +> **Note**: +* This chart will be replaced by the [gitlab-omnibus](gitlab_omnibus.md) chart, once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). For more information on available charts, please see our [overview](index.md#chart-overview). +* These charts have been tested on Google Container Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/charts.gitlab.io/issues). + + +For more information on available GitLab Helm Charts, please see our [overview](index.md#chart-overview). + +## Introduction -The `gitlab` Helm chart deploys just GitLab into your Kubernetes cluster, and offers extensive configuration options. For most deployments we recommended the [gitlab-omnibus](gitlab_omnibus.md) chart, +The `gitlab` Helm chart deploys just GitLab into your Kubernetes cluster, and offers extensive configuration options. This chart requires advanced knowledge of Kubernetes to successfully use. For most deployments we **strongly recommended** the [gitlab-omnibus](gitlab_omnibus.md) chart, which will replace this chart once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). Due to the difficulty in supporting upgrades to the `omnibus-gitlab` chart, migrating will require exporting data out of this instance and importing it into the new deployment. This chart includes the following: @@ -17,7 +22,7 @@ This chart includes the following: ## Prerequisites -- _At least_ 3 GB of RAM available on your cluster, in chunks of 1 GB. 41GB of storage and 2 CPU are also required. +- _At least_ 3 GB of RAM available on your cluster. 41GB of storage and 2 CPU are also required. - Kubernetes 1.4+ with Beta APIs enabled - [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) provisioner support in the underlying infrastructure - The ability to point a DNS entry or URL at your GitLab install diff --git a/doc/install/kubernetes/gitlab_omnibus.md b/doc/install/kubernetes/gitlab_omnibus.md index d7fd8613633..8c110a37380 100644 --- a/doc/install/kubernetes/gitlab_omnibus.md +++ b/doc/install/kubernetes/gitlab_omnibus.md @@ -2,13 +2,15 @@ > **Note:** * This Helm chart is in beta, while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being worked on. * GitLab is working on a [cloud native set of Charts](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md) which will eventually replace these. -* Officially supported cloud providers are Google Container Service and Azure Container Service. +* These charts have been tested on Google Container Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/charts.gitlab.io/issues). This work is based partially on: https://github.com/lwolf/kubernetes-gitlab/. GitLab would like to thank Sergey Nuzhdin for his work. +For more information on available GitLab Helm Charts, please see our [overview](index.md#chart-overview). + ## Introduction -This chart provides an easy way to get started with GitLab, provisioning an installation with nearly all functionality enabled. SSL is automatically provisioned as well via [Let's Encrypt](https://letsencrypt.org/). +This chart provides an easy way to get started with GitLab, provisioning an installation with nearly all functionality enabled. SSL is automatically provisioned via [Let's Encrypt](https://letsencrypt.org/). The deployment includes: @@ -19,7 +21,13 @@ The deployment includes: - [NGINX Ingress](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) - Persistent Volume Claims for Data, Registry, Postgres, and Redis -A video demonstration of GitLab utilizing this chart [is available](https://about.gitlab.com/handbook/sales/demo/). +### Limitations + +* This chart is suited for small to medium size deployments, because [High Availability](https://docs.gitlab.com/ee/administration/high_availability/) and [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html) will not be supported. +* It is in beta. Additional features to support production deployments, like backups, are [in development](https://gitlab.com/charts/charts.gitlab.io/issues/68). Once completed, this chart will be generally available. +* A new generation of [cloud native charts](index.md#upcoming-cloud-native-helm-charts) is in development, and will eventually deprecate these. Due to the difficulty in supporting upgrades to the new architecture, migrating will require exporting data out of this instance and importing it into the new deployment. We do not expect these to be production ready before the second half of 2018. + +For more information on available GitLab Helm Charts, please see our [overview](index.md#chart-overview). ## Prerequisites @@ -46,7 +54,7 @@ Finally, set the `baseIP` setting to this IP address when [deploying GitLab](#co #### Load Balancer IP -If you do not specify a `baseIP`, an ephemeral IP will be assigned to the Load Balancer or Ingress. You can retrieve this IP by running the following command *after* deploying GitLab: +If you do not specify a `baseIP`, an IP will be assigned to the Load Balancer or Ingress. You can retrieve this IP by running the following command *after* deploying GitLab: `kubectl get svc -w --namespace nginx-ingress nginx` @@ -140,32 +148,51 @@ helm install --name gitlab --set baseDomain=gitlab.io,baseIP=1.1.1.1,gitlab=ee,g ## Updating GitLab using the Helm Chart +>**Note**: If you are upgrading from a previous version to 0.1.35 or above, you will need to change the access mode values for GitLab's storage. To do this, set the following in `values.yaml` or on the CLI: +``` +gitlabDataAccessMode=ReadWriteMany +gitlabRegistryAccessMode=ReadWriteMany +gitlabConfigAccessMode=ReadWriteMany +``` + Once your GitLab Chart is installed, configuration changes and chart updates -should we done using `helm upgrade` +should be done using `helm upgrade`: + +```bash +helm upgrade -f values.yaml gitlab gitlab/gitlab-omnibus +``` + +## Upgrading from CE to EE using the Helm Chart + +If you have installed the Community Edition using this chart, upgrading to Enterprise Edition is easy. + +If you are using a `values.yaml` file to specify the configuration options, edit the file and set `gitlab=ee`. If you would like to run a specific version of GitLab EE, set `gitlabEEImage` to be the desired GitLab [docker image](https://hub.docker.com/r/gitlab/gitlab-ee/tags/). Then you can use `helm upgrade` to update your GitLab instance to EE: ```bash -helm upgrade -f <CONFIG_VALUES_FILE> <RELEASE-NAME> gitlab/gitlab +helm upgrade -f values.yaml gitlab gitlab/gitlab-omnibus ``` -where: +You can also upgrade and specify these options via the command line: -- `<CONFIG_VALUES_FILE>` is the path to values file containing your custom - [configuration] (#configuring-and-installing-gitlab). -- `<RELEASE-NAME>` is the name you gave the chart when installing it. - In the [Install section](#installing-gitlab-using-the-helm-chart) we called it `gitlab`. +```bash +helm upgrade gitlab --set gitlab=ee,gitlabEEImage=gitlab/gitlab-ee:9.5.5-ee.0 gitlab/gitlab-omnibus +``` ## Uninstalling GitLab using the Helm Chart To uninstall the GitLab Chart, run the following: ```bash -helm delete <RELEASE-NAME> +helm delete gitlab ``` -where: +## Troubleshooting + +### Storage errors when updating `gitlab-omnibus` versions prior to 0.1.35 + +Users upgrading `gitlab-omnibus` from a version prior to 0.1.35, may see an error like: `Error: UPGRADE FAILED: PersistentVolumeClaim "gitlab-gitlab-config-storage" is invalid: spec: Forbidden: field is immutable after creation`. -- `<RELEASE-NAME>` is the name you gave the chart when installing it. - In the [Install section](#installing) we called it `gitlab`. +This is due to a change in the access mode for GitLab storage in version 0.1.35. To successfully upgrade, the access mode flags must be set to `ReadWriteMany` as detailed in the [update section](#updating-gitlab-using-the-helm-chart). [kube-srv]: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types [storageclass]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#storageclasses diff --git a/doc/install/kubernetes/gitlab_runner_chart.md b/doc/install/kubernetes/gitlab_runner_chart.md index d31c763ed64..5e0d7493b61 100644 --- a/doc/install/kubernetes/gitlab_runner_chart.md +++ b/doc/install/kubernetes/gitlab_runner_chart.md @@ -1,6 +1,6 @@ # GitLab Runner Helm Chart > **Note:** -Officially supported cloud providers are Google Container Service and Azure Container Service. +These charts have been tested on Google Container Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/charts.gitlab.io/issues). The `gitlab-runner` Helm chart deploys a GitLab Runner instance into your Kubernetes cluster. @@ -11,6 +11,8 @@ This chart configures the Runner to: - For each new job it receives from [GitLab CI](https://about.gitlab.com/features/gitlab-ci-cd/), it will provision a new pod within the specified namespace to run it. +For more information on available GitLab Helm Charts, please see our [overview](index.md#chart-overview). + ## Prerequisites - Your GitLab Server's API is reachable from the cluster diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index fb6c0c2d263..aed00ae9e2c 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -1,58 +1,63 @@ # Installing GitLab on Kubernetes -> Officially supported cloud providers are Google Container Service and Azure Container Service. +> **Note**: These charts have been tested on Google Container Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/charts.gitlab.io/issues). -The easiest method to deploy GitLab in [Kubernetes](https://kubernetes.io/) is +The easiest method to deploy GitLab on [Kubernetes](https://kubernetes.io/) is to take advantage of GitLab's Helm charts. [Helm] is a package management tool for Kubernetes, allowing apps to be easily managed via their Charts. A [Chart] is a detailed description of the application including how it should be deployed, upgraded, and configured. -GitLab provides [official Helm Charts](#official-gitlab-helm-charts-recommended) which are the recommended way to run GitLab within Kubernetes. +## Chart Overview -There are also two other sets of charts: -* Our [upcoming cloud native Charts](#upcoming-cloud-native-helm-charts), which are in development but will eventually replace the current official charts. -* [Community contributed charts](#community-contributed-helm-charts). These charts should be considered deprecated, in favor of the official charts. +* **[GitLab-Omnibus](gitlab_omnibus.md)**: The best way to run GitLab on Kubernetes today. It is suited for small to medium deployments, and is in beta while support for backups and other features are added. +* **[Cloud Native GitLab Chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md)**: The next generation GitLab chart, currently in development. Will support large deployments with horizontal scaling of individual GitLab components. +* Other Charts + * [GitLab Runner Chart](gitlab_runner_chart.md): For deploying just the GitLab Runner. + * [Advanced GitLab Installation](gitlab_chart.md): Provides additional deployment options, but provides less functionality out-of-the-box. It's beta, no longer actively developed, and will be deprecated by [gitlab-omnibus](#gitlab-omnibus-chart-recommended) once it supports these options. + * [Community Contributed Charts](#community-contributed-charts): Community contributed charts, deprecated by the official GitLab chart. -## Official GitLab Helm Charts +## GitLab-Omnibus Chart (Recommended) +> **Note**: This chart is in beta while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being added. -These charts utilize our [GitLab Omnibus Docker images](https://docs.gitlab.com/omnibus/docker/README.html). You can report any issues and feedback related to these charts at -https://gitlab.com/charts/charts.gitlab.io/issues. +This chart is the best available way to operate GitLab on Kubernetes. It deploys and configures nearly all features of GitLab, including: a [Runner](https://docs.gitlab.com/runner/), [Container Registry](../../user/project/container_registry.html#gitlab-container-registry), [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/), [automatic SSL](https://github.com/kubernetes/charts/tree/master/stable/kube-lego), and a [load balancer](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). It is based on our [GitLab Omnibus Docker Images](https://docs.gitlab.com/omnibus/docker/README.html). -### Deploying GitLab on Kubernetes -> **Note**: This chart will eventually be replaced by the [cloud native charts](#upcoming-cloud-native-helm-charts), which are presently in development. +Once the [cloud native charts](#upcoming-cloud-native-helm-charts) are ready for production use, this chart will be deprecated. Due to the difficulty in supporting upgrades to the new architecture, migrating will require exporting data out of this instance and importing it into the new deployment. -The best way to deploy GitLab on Kubernetes is to use the [gitlab-omnibus](gitlab_omnibus.md) chart. +Learn more about the [gitlab-omnibus chart.](gitlab_omnibus.md) -It includes everything needed to run GitLab, including: a [Runner](https://docs.gitlab.com/runner/), [Container Registry](https://docs.gitlab.com/ee/user/project/container_registry.html#gitlab-container-registry), [automatic SSL](https://github.com/kubernetes/charts/tree/master/stable/kube-lego), and an [Ingress](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). This chart is in beta while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being completed. +## Cloud Native GitLab Chart -### Deploying just the GitLab Runner +GitLab is working towards building a [cloud native GitLab chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). A key part of this effort is to isolate each service into its [own Docker container and Helm chart](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420), rather than utilizing the all-in-one container image of the [current charts](#official-gitlab-helm-charts-recommended). -To deploy just the [GitLab Runner](https://docs.gitlab.com/runner/), utilize the [gitlab-runner](gitlab_runner_chart.md) chart. +By offering individual containers and charts, we will be able to provide a number of benefits: +* Easier horizontal scaling of each service, +* Smaller, more efficient images, +* Potential for rolling updates and canaries within a service, +* and plenty more. -It offers a quick way to configure and deploy the Runner on Kubernetes, regardless of where your GitLab server may be running. +This is a large project and will be worked on over the span of multiple releases. For the most up-to-date status and release information, please see our [tracking issue](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420). We do not expect these to be production ready before the second half of 2018. -### Advanced deployment of GitLab -> **Note**: This chart will be replaced by the [gitlab-omnibus](gitlab_omnibus.md) chart, once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). +Learn more about the [cloud native GitLab chart.](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md) -If advanced configuration of GitLab is required, the beta [gitlab](gitlab_chart.md) chart can be used which deploys the GitLab service along with optional Postgres and Redis. It offers extensive configuration, but requires deep knowledge of Kubernetes and Helm to use. +## Other Charts -For most deployments we recommend using our [gitlab-omnibus](gitlab_omnibus.md) chart. +### GitLab Runner Chart -## Upcoming Cloud Native Helm Charts +If you already have a GitLab instance running, inside or outside of Kubernetes, and you'd like to leverage the Runner's [Kubernetes capabilities](https://docs.gitlab.com/runner/executors/kubernetes.html), it can be deployed with the GitLab Runner chart. -GitLab is working towards a building a [cloud native deployment method](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). A key part of this effort is to isolate each service into it's [own Docker container and Helm chart](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420), rather than utilizing the all-in-one container image of the [current charts](#official-gitlab-helm-charts-recommended). +Learn more about [gitlab-runner chart.](gitlab_runner_chart.md) -By offering individual containers and charts, we will be able to provide a number of benefits: -* Easier horizontal scaling of each service -* Smaller more efficient images -* Potential for rolling updates and canaries within a service -* and plenty more. +### Advanced GitLab Installation + +If advanced configuration of GitLab is required, the beta [gitlab](gitlab_chart.md) chart can be used which deploys the core GitLab service along with optional Postgres and Redis. It offers extensive configuration, but offers limited functionality out-of-the-box; it's lacking Pages support, the container registry, and Mattermost. It requires deep knowledge of Kubernetes and Helm to use. + +This chart will be deprecated and replaced by the [gitlab-omnibus](gitlab_omnibus.md) chart, once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). It's beta quality, and since it is not actively under development, it will never be GA. -This is a large project and will be worked on over the span of multiple releases. For the most up to date status and release information, please see our [tracking issue](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420). +Learn more about the [gitlab chart.](gitlab_chart.md) -## Community Contributed Helm Charts +### Community Contributed Charts -The community has also [contributed GitLab charts](https://github.com/kubernetes/charts/tree/master/stable/gitlab-ce) to the [Helm Stable Repository](https://github.com/kubernetes/charts#repository-structure). These charts should be considered [deprecated](https://github.com/kubernetes/charts/issues/1138) in favor of the [official Charts](#official-gitlab-helm-charts-recommended). +The community has also contributed GitLab [CE](https://github.com/kubernetes/charts/tree/master/stable/gitlab-ce) and [EE](https://github.com/kubernetes/charts/tree/master/stable/gitlab-ee) charts to the [Helm Stable Repository](https://github.com/kubernetes/charts#repository-structure). These charts should be considered [deprecated](https://github.com/kubernetes/charts/issues/1138) in favor of the [official Charts](gitlab_omnibus.md). [chart]: https://github.com/kubernetes/charts [helm]: https://github.com/kubernetes/helm/blob/master/README.md diff --git a/doc/install/requirements.md b/doc/install/requirements.md index f672b358096..17fe80fa93d 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -82,11 +82,11 @@ errors during usage. We recommend having at least 2GB of swap on your server, even if you currently have enough available RAM. Having swap will help reduce the chance of errors occurring -if your available memory changes. We also recommend [configuring the kernels swappiness setting](https://askubuntu.com/a/103916) +if your available memory changes. We also recommend [configuring the kernel's swappiness setting](https://askubuntu.com/a/103916) to a low value like `10` to make the most of your RAM while still having the swap available when needed. -Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about many you need of those. +Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about how many you need of those. ## Database diff --git a/doc/integration/azure.md b/doc/integration/azure.md index 5e3e9f5ab77..f3c9c498634 100644 --- a/doc/integration/azure.md +++ b/doc/integration/azure.md @@ -74,6 +74,9 @@ To enable the Microsoft Azure OAuth2 OmniAuth provider you must register your ap tenant_id: "TENANT ID" } } ``` + The `base_azure_url` is optional and can be added for different locales; + e.g. `base_azure_url: "https://login.microsoftonline.de"`. + 1. Replace 'CLIENT ID', 'CLIENT SECRET' and 'TENANT ID' with the values you got above. 1. Save the configuration file. diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index ae69d7f92f2..e4c09b2b507 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -370,7 +370,7 @@ This is recommended to reduce cron spam. ## Restore -GitLab provides a simple command line interface to backup your whole installation, +GitLab provides a simple command line interface to restore your whole installation, and is flexible enough to fit your needs. The [restore prerequisites section](#restore-prerequisites) includes crucial @@ -445,6 +445,14 @@ Restoring repositories: Deleting tmp directories...[DONE] ``` +Next, restore `/home/git/gitlab/.secret` if necessary as mentioned above. + +Restart GitLab: + +```shell +sudo service gitlab restart +``` + ### Restore for Omnibus installations This procedure assumes that: @@ -480,10 +488,12 @@ restore: sudo gitlab-rake gitlab:backup:restore BACKUP=1493107454_2017_04_25_9.1.0 ``` +Next, restore `/etc/gitlab/gitlab-secrets.json` if necessary as mentioned above. + Restart and check GitLab: ```shell -sudo gitlab-ctl start +sudo gitlab-ctl restart sudo gitlab-rake gitlab:check SANITIZE=true ``` diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index fac91935a45..597c98fbf6b 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -11,6 +11,7 @@ This page gathers all the resources for the topic **Authentication** within GitL - [Security Webcast with Yubico](https://about.gitlab.com/2016/08/31/gitlab-and-yubico-security-webcast/) - **Integrations:** - [GitLab as OAuth2 authentication service provider](../../integration/oauth_provider.md#introduction-to-oauth) + - [GitLab as OpenID Connect identity provider](../../integration/openid_connect_provider.md) ## GitLab administrators diff --git a/doc/topics/autodevops/img/auto_devops_settings.png b/doc/topics/autodevops/img/auto_devops_settings.png Binary files differdeleted file mode 100644 index 57bd7650a30..00000000000 --- a/doc/topics/autodevops/img/auto_devops_settings.png +++ /dev/null diff --git a/doc/topics/autodevops/img/auto_monitoring.png b/doc/topics/autodevops/img/auto_monitoring.png Binary files differindex 5661b50841b..92902e3ca72 100644 --- a/doc/topics/autodevops/img/auto_monitoring.png +++ b/doc/topics/autodevops/img/auto_monitoring.png diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index babf44d2665..5561784ed0b 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -1,73 +1,51 @@ # Auto DevOps -> [Introduced][ce-37115] in GitLab 10.0. Auto DevOps is currently in Beta and -**not recommended for production use**. Access to the Container Registry is only -available while the pipeline is running. Restarting a pod, scaling a service, or -other actions which require on-going access **will fail** even for public -projects. On-going secure access is planned for a subsequent release. - -Auto DevOps brings best practices to your project in an easy and default way. A -typical web project starts with Continuous Integration (CI), then adds automated -deployment to production, and maybe some time in the future adds some kind of -monitoring. With Auto DevOps, every project has a complete workflow, with -no configuration, including: - -- [Auto Build](#auto-build) -- [Auto Test](#auto-test) -- [Auto Code Quality](#auto-code-quality) -- [Auto Review Apps](#auto-review-apps) -- [Auto Deploy](#auto-deploy) -- [Auto Monitoring](#-auto-monitoring) +DANGER: Auto DevOps is currently in **Beta** and _not recommended for production use_. + +> [Introduced][ce-37115] in GitLab 10.0. + +Auto DevOps automatically detects, builds, tests, deploys, and monitors your +applications. ## Overview -You will need [Kubernetes](https://kubernetes.io/) and -[Prometheus](https://prometheus.io/) to make full use of Auto DevOps, but -even projects using only [GitLab Runners](https://docs.gitlab.com/runner/) will -be able to make use of Auto Build, Auto Test, and Auto Code Quality. - -Auto DevOps makes use of an open source tool called -[Herokuish](https://github.com/gliderlabs/herokuish) which uses [Heroku -buildpacks](https://devcenter.heroku.com/articles/buildpacks) to automatically -detect, build, and test applications. Auto DevOps supports all of the languages -and frameworks that are [supported by -Herokuish](https://github.com/gliderlabs/herokuish#buildpacks) such as Ruby, -Rails, Node, PHP, Python, and Java, and [custom buildpacks can be -specified](#using-custom-buildpacks). *GitLab is in no way affiliated with Heroku -or Glider Labs.* - -Projects can [customize](#customizing) the process by specifying [custom -buildpacks](#custom-buildpack), [custom `Dockerfile`s](#custom-dockerfile), -[custom Helm charts](#custom-helm-chart), or even copying the complete CI/CD -configuration into your project to enable staging and canary deployments, and -more. +With Auto DevOps, the software development process becomes easier to set up +as every project can have a complete workflow from build to deploy and monitoring, +with minimal to zero configuration. -## Quick start +Comprised of a set of stages, Auto DevOps brings these best practices to your +project in an easy and automatic way: -If you are using GitLab.com, see our [quick start guide](quick_start_guide.md) -for using Auto DevOps with GitLab.com and an external Kubernetes cluster on -Google Cloud. +1. [Auto Build](#auto-build) +1. [Auto Test](#auto-test) +1. [Auto Code Quality](#auto-code-quality) +1. [Auto Review Apps](#auto-review-apps) +1. [Auto Deploy](#auto-deploy) +1. [Auto Monitoring](#auto-monitoring) -For self-hosted installations, the easiest way to make use of Auto DevOps is to -install GitLab inside a Kubernetes cluster using the [GitLab-Omnibus Helm -Chart](../../install/kubernetes/gitlab_omnibus.md) which automatically installs -and configures everything you need. +As Auto DevOps relies on many different components, it's good to have a basic +knowledge of the following: -## Prerequisites +- [Kubernetes](https://kubernetes.io/docs/home/) +- [Helm](https://docs.helm.sh/) +- [Docker](https://docs.docker.com) +- [GitLab Runner](https://docs.gitlab.com/runner/) +- [Prometheus](https://prometheus.io/docs/introduction/overview/) -You will need one or more GitLab Runners, a Kubernetes cluster, and Prometheus -installed in the cluster to make full use of Auto DevOps. If you do not have -Kubernetes or Prometheus installed then Auto Review Apps, Auto Deploy, and Auto -Monitoring will be silently skipped. +Auto DevOps provides great defaults for all the stages; you can, however, +[customize](#customizing) almost everything to your needs. + +## Prerequisites -If you are using GitLab outside of Kubernetes, for example with GitLab.com, then -you should take these prerequisites into account: +TIP: **Tip:** +For self-hosted installations, the easiest way to make use of Auto DevOps is to +install GitLab inside a Kubernetes cluster using the [GitLab Omnibus Helm Chart] +which automatically installs and configures everything you need! -1. **Base domain** - You will need a base domain configured with wildcard DNS to - be used by all of your Auto DevOps applications. +To make full use of Auto DevOps, you will need: -1. **GitLab Runner** - Your Runner needs to be configured to be able to run Docker. - Generally this means using the +1. **GitLab Runner** (needed for all stages) - Your Runner needs to be + configured to be able to run Docker. Generally this means using the [Docker](https://docs.gitlab.com/runner/executors/docker.html) or [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html), with [privileged mode enabled](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). @@ -78,35 +56,98 @@ you should take these prerequisites into account: should be registered as [shared Runners](../../ci/runners/README.md#registering-a-shared-runner) for the entire GitLab instance, or [specific Runners](../../ci/runners/README.md#registering-a-specific-runner) that are assigned to specific projects. - -1. **Kubernetes** - To enable deploys, you will need Kubernetes 1.5+, with NGINX - ingress and wildcard SSL termination, for example using the - [`nginx-ingress`](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) - and [`kube-lego`](https://github.com/kubernetes/charts/tree/master/stable/kube-lego) - Helm charts respectively. The [Kubernetes service][kubernetes-service] +1. **Base domain** (needed for Auto Review Apps and Auto Deploy) - You will need + a domain configured with wildcard DNS which is gonna be used by all of your + Auto DevOps applications. [Read the specifics](#auto-devops-base-domain). +1. **Kubernetes** (needed for Auto Review Apps, Auto Deploy, and Auto Monitoring) - + To enable deployments, you will need Kubernetes 1.5+. The [Kubernetes service][kubernetes-service] integration will need to be enabled for the project, or enabled as a [default service template](../../user/project/integrations/services_templates.md) for the entire GitLab installation. - -1. **Prometheus** - To enable Auto Monitoring, you will need Prometheus installed - somewhere (inside or outside your cluster) and configured to scrape your - Kubernetes cluster. To get response metrics (in addition to system metrics), - you need to [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-prometheus-to-monitor-for-nginx-ingress-metrics). + 1. **A load balancer** - You can use NGINX ingress by deploying it to your + Kubernetes cluster using the + [`nginx-ingress`](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) + Helm chart. + 1. **Wildcard TLS termination** - You can deploy the + [`kube-lego`](https://github.com/kubernetes/charts/tree/master/stable/kube-lego) + Helm chart to your Kubernetes cluster to automatically issue certificates + for your domains using Let's Encrypt. +1. **Prometheus** (needed for Auto Monitoring) - To enable Auto Monitoring, you + will need Prometheus installed somewhere (inside or outside your cluster) and + configured to scrape your Kubernetes cluster. To get response metrics + (in addition to system metrics), you need to + [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-prometheus-to-monitor-for-nginx-ingress-metrics). The [Prometheus service](../../user/project/integrations/prometheus.md) integration needs to be enabled for the project, or enabled as a [default service template](../../user/project/integrations/services_templates.md) for the entire GitLab installation. -## Enabling Auto DevOps +NOTE: **Note:** +If you do not have Kubernetes or Prometheus installed, then Auto Review Apps, +Auto Deploy, and Auto Monitoring will be silently skipped. + +### Auto DevOps base domain + +The Auto DevOps base domain is required if you want to make use of [Auto +Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It is defined +under the project's CI/CD settings while [enabling Auto DevOps](#enabling-auto-devops). +It can also be set at the project or group level as a variable, `AUTO_DEVOPS_DOMAIN`. + +A wildcard DNS A record matching the base domain is required, for example, +given a base domain of `example.com`, you'd need a DNS entry like: + +``` +*.example.com 3600 A 1.2.3.4 +``` -In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps -section. Select "Enable Auto DevOps", add in your base domain, and save. +where `example.com` is the domain name under which the deployed apps will be served, +and `1.2.3.4` is the IP address of your load balancer; generally NGINX +([see prerequisites](#prerequisites)). How to set up the DNS record is beyond +the scope of this document; you should check with your DNS provider. - +Once set up, all requests will hit the load balancer, which in turn will route +them to the Kubernetes pods that run your application(s). + +NOTE: **Note:** +If GitLab is installed using the [GitLab Omnibus Helm Chart], there are two +options: provide a static IP, or have one assigned. For more information see the +relevant docs on the [network prerequisites](../../install/kubernetes/gitlab_omnibus.md#networking-prerequisites). + +## Quick start + +If you are using GitLab.com, see our [quick start guide](quick_start_guide.md) +for using Auto DevOps with GitLab.com and an external Kubernetes cluster on +Google Cloud. + +## Enabling Auto DevOps + +NOTE: **Note:** +If you haven't done already, read the [prerequisites](#prerequisites) to make +full use of Auto DevOps. If this is your fist time, we recommend you follow the +[quick start guide](#quick-start). + +1. Go to your project's **Settings > CI/CD > General pipelines settings** and + find the Auto DevOps section +1. Select "Enable Auto DevOps" +1. Optionally, but recommended, add in the [base domain](#auto-devops-base-domain) + that will be used by Kubernetes to deploy your application +1. Hit **Save changes** for the changes to take effect + +Now that it's enabled, there are a few more steps depending on whether your project +has a `.gitlab-ci.yml` or not: + +- **For projects with no `.gitlab-ci.yml` present:** + A pipeline needs to be triggered either by pushing a new commit to the + repository or manually visiting `https://example.gitlab.com/<username>/<project>/pipelines/new` + and creating a new pipeline for your default branch, generally `master`. +- **For projects with a `.gitlab-ci.yml` present:** + All you need to do is remove your existing `.gitlab-ci.yml`, and you can even + do that in a branch to test Auto DevOps before committing to `master`. ## Stages of Auto DevOps -The following sections describe the stages of Auto DevOps. +The following sections describe the stages of Auto DevOps. Read them carefully +to understand how each one works. ### Auto Build @@ -118,18 +159,24 @@ Auto Build creates a build of the application in one of two ways: to automatically detect and build the application into a Docker image. Either way, the resulting Docker image is automatically pushed to the -[Container Registry][container-registry], tagged with the commit SHA. +[Container Registry][container-registry] and tagged with the commit SHA. + +CAUTION: **Important:** +If you are also using Auto Review Apps and Auto Deploy and choose to provide +your own `Dockerfile`, make sure you expose your application to port +`5000` as this is the port assumed by the default Helm chart. ### Auto Test -Auto Test automatically tests your application using +Auto Test automatically runs the appropriate tests for your application using [Herokuish](https://github.com/gliderlabs/herokuish) and [Heroku -buildpacks](https://devcenter.heroku.com/articles/buildpacks). Auto Test will -analyze your project to detect the language and framework, and run appropriate -tests. Several languages and frameworks are detected automatically, but if your -language is not detected, you may succeed with a [custom -buildpack](#custom-buildpack). +buildpacks](https://devcenter.heroku.com/articles/buildpacks) by analyzing +your project to detect the language and framework. Several languages and +frameworks are detected automatically, but if your language is not detected, +you may succeed with a [custom buildpack](#custom-buildpacks). Check the +[currently supported languages](#currently-supported-languages). +NOTE: **Note:** Auto Test uses tests you already have in your application. If there are no tests, it's up to you to add them. @@ -137,99 +184,110 @@ tests, it's up to you to add them. Auto Code Quality uses the open source [`codeclimate` image](https://hub.docker.com/r/codeclimate/codeclimate/) to run -static analysis and other code checks on the current code, creating a report -that is uploaded as an artifact. In GitLab EE, differences between the source -and target branches are shown in the merge request widget. *GitLab is in no way -affiliated with Code Climate.* +static analysis and other code checks on the current code. The report is +created, and is uploaded as an artifact which you can later download and check +out. In GitLab Enterprise Edition Starter, differences between the source and +target branches are +[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html). ### Auto Review Apps -Auto Review Apps create a [Review App][review-app] for each branch. Review Apps -are temporary application environments based on the branch's code so developers, -designers, QA, product managers, and other reviewers can actually see and -interact with code changes as part of the review process. - -The review app will have a unique URL based on the project name, the branch +NOTE: **Note:** +This is an optional step, since many projects do not have a Kubernetes cluster +available. If the [prerequisites](#prerequisites) are not met, the job will +silently be skipped. + +CAUTION: **Caution:** +Your apps should *not* be manipulated outside of Helm (using Kubernetes directly.) +This can cause confusion with Helm not detecting the change, and subsequent +deploys with Auto DevOps can undo your changes. Also, if you change something +and want to undo it by deploying again, Helm may not detect that anything changed +in the first place, and thus not realize that it needs to re-apply the old config. + +[Review Apps][review-app] are temporary application environments based on the +branch's code so developers, designers, QA, product managers, and other +reviewers can actually see and interact with code changes as part of the review +process. Auto Review Apps create a Review App for each branch. + +The Review App will have a unique URL based on the project name, the branch name, and a unique number, combined with the Auto DevOps base domain. For example, `user-project-branch-1234.example.com`. A link to the Review App shows up in the merge request widget for easy discovery. When the branch is deleted, for example after the merge request is merged, the Review App will automatically be deleted. -This is an optional step, since many projects do not have a Kubernetes cluster -available. If the Kubernetes service is not configured, or if the variable -`AUTO_DEVOPS_DOMAIN` is not available (usually set automatically by the Auto -DevOps setting), the job will silently be skipped. - ### Auto Deploy -After a branch or merge request is merged into `master`, Auto Deploy deploys the -application to a `production` environment in the Kubernetes cluster, with a -namespace based on the project name and unique project ID. e.g. `project-4321`. +NOTE: **Note:** This is an optional step, since many projects do not have a Kubernetes cluster -available. If the Kubernetes service is not configured, or if the variable -`AUTO_DEVOPS_DOMAIN` is not available (usually set automatically by the Auto -DevOps setting), the job will silently be skipped. +available. If the [prerequisites](#prerequisites) are not met, the job will +silently be skipped. + +CAUTION: **Caution:** +Your apps should *not* be manipulated outside of Helm (using Kubernetes directly.) +This can cause confusion with Helm not detecting the change, and subsequent +deploys with Auto DevOps can undo your changes. Also, if you change something +and want to undo it by deploying again, Helm may not detect that anything changed +in the first place, and thus not realize that it needs to re-apply the old config. + +After a branch or merge request is merged into the project's default branch (usually +`master`), Auto Deploy deploys the application to a `production` environment in +the Kubernetes cluster, with a namespace based on the project name and unique +project ID, for example `project-4321`. Auto Deploy doesn't include deployments to staging or canary by default, but the -Auto DevOps template contains job definitions for these tasks if you want to +[Auto DevOps template] contains job definitions for these tasks if you want to enable them. +You can make use of [environment variables](#helm-chart-variables) to automatically +scale your pod replicas. + ### Auto Monitoring +NOTE: **Note:** +Check the [prerequisites](#prerequisites) for Auto Monitoring to make this stage +work. + Once your application is deployed, Auto Monitoring makes it possible to monitor your application's server and response metrics right out of the box. Auto Monitoring uses [Prometheus](../../user/project/integrations/prometheus.md) to get system metrics such as CPU and memory usage directly from [Kubernetes](../../user/project/integrations/prometheus_library/kubernetes.md), and response metrics such as HTTP error rates, latency, and throughput from the -[NGINX -server](../../user/project/integrations/prometheus_library/nginx_ingress.md). +[NGINX server](../../user/project/integrations/prometheus_library/nginx_ingress.md). -* Response Metrics: latency, throughput, error rate -* System Metrics: CPU utilization, memory utilization +The metrics include: -To view the metrics, open the [Monitoring dashboard for a deployed environment](../../ci/environments.md#monitoring-environments). +- **Response Metrics:** latency, throughput, error rate +- **System Metrics:** CPU utilization, memory utilization - - -### Configuring Auto Monitoring - -If GitLab has been deployed using the -[omnibus-gitlab](../../install/kubernetes/gitlab_omnibus.md) Helm chart, no +If GitLab has been deployed using the [GitLab Omnibus Helm Chart], no configuration is required. -If you have installed GitLab using a different method: +If you have installed GitLab using a different method, you need to: 1. [Deploy Prometheus](../../user/project/integrations/prometheus.md#configuring-your-own-prometheus-server-within-kubernetes) into your Kubernetes cluster -1. If you would like response metrics, ensure you are running at least version 0.9.0 of NGINX Ingress and [enable Prometheus metrics](https://github.com/kubernetes/ingress/blob/master/examples/customization/custom-vts-metrics/nginx/nginx-vts-metrics-conf.yaml). -1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) the NGINX Ingress deployment to be scraped by Prometheus using `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. - -## Customizing +1. If you would like response metrics, ensure you are running at least version + 0.9.0 of NGINX Ingress and + [enable Prometheus metrics](https://github.com/kubernetes/ingress/blob/master/examples/customization/custom-vts-metrics/nginx/nginx-vts-metrics-conf.yaml). +1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) + the NGINX Ingress deployment to be scraped by Prometheus using + `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. -### PostgreSQL Database Support +To view the metrics, open the +[Monitoring dashboard for a deployed environment](../../ci/environments.md#monitoring-environments). -In order to support applications that require a database, -[PostgreSQL][postgresql] is provisioned by default. Credentials to access the -database are preconfigured, but can be customized by setting the associated -[variables](#postgresql-variables). These credentials can be used for defining a -`DATABASE_URL` of the format: -`postgres://user:password@postgres-host:postgres-port/postgres-database`. - -PostgreSQL provisioning can be disabled by creating a project variable -`POSTGRES_ENABLED` set to `false`. - -#### PostgreSQL Variables + -Any variables set at the project or group level will override variables set in -the CI/CD configuration. +## Customizing -1. `POSTGRES_ENABLED: "false"`: disable automatic deployment of PostgreSQL -1. `POSTGRES_USER: "my-user"`: use custom username for PostgreSQL -1. `POSTGRES_PASSWORD: "password"`: use custom password for PostgreSQL -1. `POSTGRES_DB: "my-database"`: use custom database name for PostgreSQL +While Auto DevOps provides great defaults to get you started, you can customize +almost everything to fit your needs; from custom [buildpacks](#custom-buildpacks), +to [`Dockerfile`s](#custom-dockerfile), [Helm charts](#custom-helm-chart), or +even copying the complete [CI/CD configuration](#customizing-gitlab-ci-yml) +into your project to enable staging and canary deployments, and more. -### Custom buildpack +### Custom buildpacks If the automatic buildpack detection fails for your project, or if you want to use a custom buildpack, you can override the buildpack using a project variable @@ -237,12 +295,12 @@ or a `.buildpack` file in your project: - **Project variable** - Create a project variable `BUILDPACK_URL` with the URL of the buildpack to use. - - **`.buildpack` file** - Add a file in your project's repo called `.buildpack` and add the URL of the buildpack to use on a line in the file. If you want to - use multiple buildpacks, you can enter them in, one on each line + use multiple buildpacks, you can enter them in, one on each line. - >**Note:** Using multiple buildpacks may break Auto Test. +CAUTION: **Caution:** +Using multiple buildpacks isn't yet supported by Auto DevOps. ### Custom `Dockerfile` @@ -253,37 +311,138 @@ Dockerfile is based on [Alpine](https://hub.docker.com/_/alpine/). ### Custom Helm Chart -Auto DevOps uses Helm to deploy your application to Kubernetes. You can override -the Helm chart used by bundling up a chart into your project repo or by -specifying a project variable. - -**Bundled chart** - If your project has a `chart` directory with a `Chart.yaml` -file in it, Auto DevOps will detect the chart and use it instead of the default -chart. This can be a great way to control exactly how your application is -deployed. +Auto DevOps uses [Helm](https://helm.sh/) to deploy your application to Kubernetes. +You can override the Helm chart used by bundling up a chart into your project +repo or by specifying a project variable: -**Project variable** - Create a project variable `AUTO_DEVOPS_CHART` with the -URL of a custom chart to use. +- **Bundled chart** - If your project has a `./charts` directory with a `Chart.yaml` + file in it, Auto DevOps will detect the chart and use it instead of the [default + one](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). + This can be a great way to control exactly how your application is deployed. +- **Project variable** - Create a [project variable](../../ci/variables/README.md#secret-variables) + `AUTO_DEVOPS_CHART` with the URL of a custom chart to use. -### Enable staging, canaries, and more with custom `.gitlab-ci.yml` +### Customizing `.gitlab-ci.yml` If you want to modify the CI/CD pipeline used by Auto DevOps, you can copy the -Auto DevOps template into your project's repo and edit as you see fit. +[Auto DevOps template] into your project's repo and edit as you see fit. -From your project home page, click on the `Set up CI` button, or click on the `+` -button and `New file` and pick `.gitlab-ci.yml` as the template type, or view an -existing `.gitlab-ci.yml` file. Then select "Auto DevOps" from the template -dropdown. You will then be able to edit or add any jobs needed. +Assuming that your project is new or it doesn't have a `.gitlab-ci.yml` file +present: -For example, if you want deploys to go to a staging environment instead of -directly to a production environment, you can enable the `staging` job by +1. From your project home page, either click on the "Set up CI" button, or click + on the plus button and (`+`), then "New file" +1. Pick `.gitlab-ci.yml` as the template type +1. Select "Auto-DevOps" from the template dropdown +1. Edit the template or add any jobs needed +1. Give an appropriate commit message and hit "Commit changes" + +TIP: **Tip:** The Auto DevOps template includes useful comments to help you +customize it. For example, if you want deployments to go to a staging environment +instead of directly to a production one, you can enable the `staging` job by renaming `.staging` to `staging`. Then make sure to uncomment the `when` key of the `production` job to turn it into a manual action instead of deploying automatically. +### PostgreSQL database support + +In order to support applications that require a database, +[PostgreSQL][postgresql] is provisioned by default. The credentials to access +the database are preconfigured, but can be customized by setting the associated +[variables](#environment-variables). These credentials can be used for defining a +`DATABASE_URL` of the format: + +```yaml +postgres://user:password@postgres-host:postgres-port/postgres-database +``` + +### Environment variables + +The following variables can be used for setting up the Auto DevOps domain, +providing a custom Helm chart, or scaling your application. PostgreSQL can be +also be customized, and you can easily use a [custom buildpack](#custom-buildpacks). + +| **Variable** | **Description** | +| ------------ | --------------- | +| `AUTO_DEVOPS_DOMAIN` | The [Auto DevOps domain](#auto-devops-domain); by default set automatically by the [Auto DevOps setting](#enabling-auto-devops). | +| `AUTO_DEVOPS_CHART` | The Helm Chart used to deploy your apps; defaults to the one [provided by GitLab](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). | +| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment; defaults to 1. | +| `CANARY_PRODUCTION_REPLICAS`| The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) in the production environment. | +| `POSTGRES_ENABLED` | Whether PostgreSQL is enabled; defaults to `"true"`. Set to `false` to disable the automatic deployment of PostgreSQL. | +| `POSTGRES_USER` | The PostgreSQL user; defaults to `user`. Set it to use a custom username. | +| `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. | +| `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-variables-environment-variables). Set it to use a custom database name. | +| `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142`| + +TIP: **Tip:** +Set up the replica variables using a +[project variable](../../ci/variables/README.md#secret-variables) +and scale your application by just redeploying it! + +CAUTION: **Caution:** +You should *not* scale your application using Kubernetes directly. This can +cause confusion with Helm not detecting the change, and subsequent deploys with +Auto DevOps can undo your changes. + +#### Advanced replica variables setup + +Apart from the two replica-related variables for production mentioned above, +you can also use others for different environments. + +There's a very specific mapping between Kubernetes' label named `track`, +GitLab CI/CD environment names, and the replicas environment variable. +The general rule is: `TRACK_ENV_REPLICAS`. Where: + +- `TRACK`: The capitalized value of the `track` + [Kubernetes label](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) + in the Helm Chart app definition. If not set, it will not be taken into account + to the variable name. +- `ENV`: The capitalized environment name of the deploy job that is set in + `.gitlab-ci.yml`. + +That way, you can define your own `TRACK_ENV_REPLICAS` variables with which +you will be able to scale the pod's replicas easily. + +In the example below, the environment's name is `qa` which would result in +looking for the `QA_REPLICAS` environment variable: + +```yaml +QA testing: + stage: deploy + environment: + name: qa + script: + - deploy qa +``` + +If, in addition, there was also a `track: foo` defined in the application's Helm +chart, like: + +```yaml +replicaCount: 1 +image: + repository: gitlab.example.com/group/project + tag: stable + pullPolicy: Always + secrets: + - name: gitlab-registry +application: + track: foo + tier: web +service: + enabled: true + name: web + type: ClusterIP + url: http://my.host.com/ + externalPort: 5000 + internalPort: 5000 +``` + +then the environment variable would be `FOO_QA_REPLICAS`. + ## Currently supported languages ->**Note:** +NOTE: **Note:** Not all buildpacks support Auto Test yet, as it's a relatively new enhancement. All of Heroku's [officially supported languages](https://devcenter.heroku.com/articles/heroku-ci#currently-supported-languages) @@ -294,22 +453,28 @@ multi-buildpack does not. As of GitLab 10.0, the supported buildpacks are: ``` -* heroku-buildpack-multi v1.0.0 -* heroku-buildpack-ruby v168 -* heroku-buildpack-nodejs v99 -* heroku-buildpack-clojure v77 -* heroku-buildpack-python v99 -* heroku-buildpack-java v53 -* heroku-buildpack-gradle v23 -* heroku-buildpack-scala v78 -* heroku-buildpack-play v26 -* heroku-buildpack-php v122 -* heroku-buildpack-go v72 -* heroku-buildpack-erlang fa17af9 -* buildpack-nginx v8 +- heroku-buildpack-multi v1.0.0 +- heroku-buildpack-ruby v168 +- heroku-buildpack-nodejs v99 +- heroku-buildpack-clojure v77 +- heroku-buildpack-python v99 +- heroku-buildpack-java v53 +- heroku-buildpack-gradle v23 +- heroku-buildpack-scala v78 +- heroku-buildpack-play v26 +- heroku-buildpack-php v122 +- heroku-buildpack-go v72 +- heroku-buildpack-erlang fa17af9 +- buildpack-nginx v8 ``` -## Private Project Support - Experimental +## Limitations + +The following restrictions apply. + +### Private project support + +CAUTION: **Caution:** Private project support in Auto DevOps is experimental. When a project has been marked as private, GitLab's [Container Registry][container-registry] requires authentication when downloading @@ -319,8 +484,8 @@ Authentication credentials will be valid while the pipeline is running, allowing for a successful initial deployment. After the pipeline completes, Kubernetes will no longer be able to access the -container registry. **Restarting a pod, scaling a service, or other actions which -require on-going access to the registry will fail**. On-going secure access is +Container Registry. **Restarting a pod, scaling a service, or other actions which +require on-going access to the registry may fail**. On-going secure access is planned for a subsequent release. ## Troubleshooting @@ -330,13 +495,36 @@ planned for a subsequent release. key files the buildpack is looking for. For example, for ruby apps, you must have a `Gemfile` to be properly detected, even though it is possible to write a Ruby app without a `Gemfile`. Try specifying a [custom - buildpack](#custom-buildpack). + buildpack](#custom-buildpacks). - Auto Test may fail because of a mismatch between testing frameworks. In this case, you may need to customize your `.gitlab-ci.yml` with your test commands. +### Disable the banner instance wide + +If an administrator would like to disable the banners on an instance level, this +feature can be disabled either through the console: + +```sh +sudo gitlab-rails console +``` + +Then run: + +```ruby +Feature.get(:auto_devops_banner_disabled).enable +``` + +Or through the HTTP API with an admin access token: + +```sh +curl --data "value=true" --header "PRIVATE-TOKEN: private_token" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled +``` + [ce-37115]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37115 [kubernetes-service]: ../../user/project/integrations/kubernetes.md [docker-in-docker]: ../../docker/using_docker_build.md#use-docker-in-docker-executor [review-app]: ../../ci/review_apps/index.md [container-registry]: ../../user/project/container_registry.md [postgresql]: https://www.postgresql.org/ +[Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml +[GitLab Omnibus Helm Chart]: ../../install/kubernetes/gitlab_omnibus.md diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index f23c9d794b4..ffe05519d7b 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -1,14 +1,15 @@ # Auto DevOps: quick start guide -> [Introduced][ce-37115] in GitLab 10.0. Auto DevOps is currently in Beta and -**not recommended for production use**. +DANGER: Auto DevOps is currently in **Beta** and _not recommended for production use_. + +> [Introduced][ce-37115] in GitLab 10.0. This is a step-by-step guide to deploying a project hosted on GitLab.com to Google Cloud, using Auto DevOps. We made a minimal [Ruby -application](https://gitlab.com/gitlab-examples/minimal-ruby-app) to use as an -example for this guide. It contains two files: +application](https://gitlab.com/auto-devops-examples/minimal-ruby-app) to use +as an example for this guide. It contains two main files: * `server.rb` - our application. It will start an HTTP server on port 5000 and render "Hello, world!" @@ -18,45 +19,66 @@ example for this guide. It contains two files: ## Fork sample project on GitLab.com Let’s start by forking our sample application. Go to [the project -page](https://gitlab.com/gitlab-examples/minimal-ruby-app) and press the `Fork` -button. Soon you should have a project under your namespace with the necessary -files. +page](https://gitlab.com/auto-devops-examples/minimal-ruby-app) and press the +**Fork** button. Soon you should have a project under your namespace with the +necessary files. ## Setup your own cluster on Google Container Engine -If you do not already have a Google Cloud account, create one at https://console.cloud.google.com. +If you do not already have a Google Cloud account, create one at +https://console.cloud.google.com. -Visit the [`Container Engine`](https://console.cloud.google.com/kubernetes/list) tab and create a new cluster. You can change the name and leave the rest of the default settings. Once you have your cluster running, you need to connect to the cluster by following the Google interface. +Visit the [**Container Engine**](https://console.cloud.google.com/kubernetes/list) +tab and create a new cluster. You can change the name and leave the rest of the +default settings. Once you have your cluster running, you need to connect to the +cluster by following the Google interface. ## Connect to Kubernetes cluster You need to have the Google Cloud SDK installed. e.g. -On OSX, install [homebrew](https://brew.sh): +On macOS, install [homebrew](https://brew.sh): 1. Install Brew Caskroom: `brew install caskroom/cask/brew-cask` 2. Install Google Cloud SDK: `brew cask install google-cloud-sdk` -3. Add `kubectl`: `gcloud components install kubectl` +3. Add `kubectl` with: `gcloud components install kubectl` 4. Log in: `gcloud auth login` -Now go back to the Google interface, find your cluster, and follow the instructions under `Connect to the cluster` and open the Kubernetes Dashboard. It will look something like `gcloud container clusters get-credentials ruby-autodeploy \ --zone europe-west2-c --project api-project-XXXXXXX` and then `kubectl proxy`. +Now go back to the Google interface, find your cluster, follow the instructions +under "Connect to the cluster" and open the Kubernetes Dashboard. It will look +something like: + +```sh +gcloud container clusters get-credentials ruby-autodeploy \ --zone europe-west2-c --project api-project-XXXXXXX +``` + +Finally, run `kubectl proxy`.  ## Copy credentials to GitLab.com project -Once you have the Kubernetes Dashboard interface running, you should visit `Secrets` under the `Config` section. There you should find the settings we need for GitLab integration: ca.crt and token. +Once you have the Kubernetes Dashboard interface running, you should visit +**Secrets** under the "Config" section. There, you should find the settings we +need for GitLab integration: `ca.crt` and token.  -You need to copy-paste the ca.crt and token into your project on GitLab.com in the Kubernetes integration page under project **Settings > Integrations > Project services > Kubernetes**. Don't actually copy the namespace though. Each project should have a unique namespace, and by leaving it blank, GitLab will create one for you. +You need to copy-paste the `ca.crt` and token into your project on GitLab.com in +the Kubernetes integration page under project +**Settings > Integrations > Project services > Kubernetes**. Don't actually copy +the namespace though. Each project should have a unique namespace, and by leaving +it blank, GitLab will create one for you.  -For API URL, you should use the `Endpoint` IP from your cluster page on Google Cloud Platform. +For the API URL, you should use the "Endpoint" IP from your cluster page on +Google Cloud Platform. ## Expose application to the world -In order to be able to visit your application, you need to install an NGINX ingress controller and point your domain name to its external IP address. +In order to be able to visit your application, you need to install an NGINX +ingress controller and point your domain name to its external IP address. Let's +see how that's done. ### Set up Ingress controller @@ -68,28 +90,47 @@ helm init helm install --name ruby-app stable/nginx-ingress ``` -This should create several services including `ruby-app-nginx-ingress-controller`. You can list your services by running `kubectl get svc` to confirm that. +This should create several services including `ruby-app-nginx-ingress-controller`. +You can list your services by running `kubectl get svc` to confirm that. ### Point DNS at Cluster IP -Find out the external IP address of the `ruby-app-nginx-ingress-controller` by running: +Find out the external IP address of the `ruby-app-nginx-ingress-controller` by +running: ```sh kubectl get svc ruby-app-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' ``` -Use this IP address to configure your DNS. This part heavily depends on your preferences and domain provider. But in case you are not sure, just create an A record with a wildcard host like `*.<your-domain>`. +Use this IP address to configure your DNS. This part heavily depends on your +preferences and domain provider. But in case you are not sure, just create an +A record with a wildcard host like `*.<your-domain>`. -Use `nslookup minimal-ruby-app-staging.<yourdomain>` to confirm that domain is assigned to the cluster IP. +Use `nslookup minimal-ruby-app-staging.<yourdomain>` to confirm that domain is +assigned to the cluster IP. ## Set up Auto DevOps -In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps section. Select "Enable Auto DevOps", add in your base domain, and save. +In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps +section. Select "Enable Auto DevOps", add in your base domain, and save. + +Next, a pipeline needs to be triggered. Since the test project doesn't have a +`.gitlab-ci.yml`, you need to either push a change to the repository or +manually visit `https://gitlab.com/<username>/minimal-ruby-app/pipelines/new`, +where `<username>` is your username. - +This will create a new pipeline with several jobs: `build`, `test`, `codequality`, +and `production`. The `build` job will create a Docker image with your new +change and push it to the Container Registry. The `test` job will test your +changes, whereas the `codequality` job will run static analysis on your changes. +Finally, the `production` job will deploy your changes to a production application. -Then trigger your first pipeline run. This will create a new pipeline with several jobs: `build`, `test`, `codequality`, and `production`. The `build` job will create a docker image with your new change and push it to the GitLab Container Registry. The `test` job will test your change. The `codequality` job will run static analysis on your change. The `production` job will deploy your change to a production application. Once the deploy job succeeds you should be able to see your application by visiting the Kubernetes dashboard. Select the namespace of your project, which will look like `minimal-ruby-app-23`, but with a unique ID for your project, and your app will be listed as "production" under the Deployment tab. +Once the deploy job succeeds you should be able to see your application by +visiting the Kubernetes dashboard. Select the namespace of your project, which +will look like `minimal-ruby-app-23`, but with a unique ID for your project, +and your app will be listed as "production" under the Deployment tab. -Once its ready - just visit http://minimal-ruby-app.example.com to see “Hello, world!” +Once its ready, just visit `http://minimal-ruby-app.example.com` to see the +famous "Hello, world!"! [ce-37115]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37115 diff --git a/doc/university/training/gitlab_flow.md b/doc/university/training/gitlab_flow.md index a7db1f2e069..a7db1f2e069 100755..100644 --- a/doc/university/training/gitlab_flow.md +++ b/doc/university/training/gitlab_flow.md diff --git a/doc/university/training/index.md b/doc/university/training/index.md index 03179ff5a77..03179ff5a77 100755..100644 --- a/doc/university/training/index.md +++ b/doc/university/training/index.md diff --git a/doc/university/training/topics/additional_resources.md b/doc/university/training/topics/additional_resources.md index 3ed601625cf..3ed601625cf 100755..100644 --- a/doc/university/training/topics/additional_resources.md +++ b/doc/university/training/topics/additional_resources.md diff --git a/doc/university/training/topics/agile_git.md b/doc/university/training/topics/agile_git.md index e6e4fea9b51..e6e4fea9b51 100755..100644 --- a/doc/university/training/topics/agile_git.md +++ b/doc/university/training/topics/agile_git.md diff --git a/doc/university/training/topics/bisect.md b/doc/university/training/topics/bisect.md index a60c4365e0c..a60c4365e0c 100755..100644 --- a/doc/university/training/topics/bisect.md +++ b/doc/university/training/topics/bisect.md diff --git a/doc/university/training/topics/cherry_picking.md b/doc/university/training/topics/cherry_picking.md index af7a70a2818..af7a70a2818 100755..100644 --- a/doc/university/training/topics/cherry_picking.md +++ b/doc/university/training/topics/cherry_picking.md diff --git a/doc/university/training/topics/env_setup.md b/doc/university/training/topics/env_setup.md index 8149379b36f..8149379b36f 100755..100644 --- a/doc/university/training/topics/env_setup.md +++ b/doc/university/training/topics/env_setup.md diff --git a/doc/university/training/topics/explore_gitlab.md b/doc/university/training/topics/explore_gitlab.md index b65457728c0..b65457728c0 100755..100644 --- a/doc/university/training/topics/explore_gitlab.md +++ b/doc/university/training/topics/explore_gitlab.md diff --git a/doc/university/training/topics/feature_branching.md b/doc/university/training/topics/feature_branching.md index 4b34406ea75..4b34406ea75 100755..100644 --- a/doc/university/training/topics/feature_branching.md +++ b/doc/university/training/topics/feature_branching.md diff --git a/doc/university/training/topics/getting_started.md b/doc/university/training/topics/getting_started.md index ec7bb2631aa..ec7bb2631aa 100755..100644 --- a/doc/university/training/topics/getting_started.md +++ b/doc/university/training/topics/getting_started.md diff --git a/doc/university/training/topics/git_add.md b/doc/university/training/topics/git_add.md index 9ffb4b9c859..9ffb4b9c859 100755..100644 --- a/doc/university/training/topics/git_add.md +++ b/doc/university/training/topics/git_add.md diff --git a/doc/university/training/topics/git_intro.md b/doc/university/training/topics/git_intro.md index ca1ff29d93b..ca1ff29d93b 100755..100644 --- a/doc/university/training/topics/git_intro.md +++ b/doc/university/training/topics/git_intro.md diff --git a/doc/university/training/topics/git_log.md b/doc/university/training/topics/git_log.md index 32ebceff491..32ebceff491 100755..100644 --- a/doc/university/training/topics/git_log.md +++ b/doc/university/training/topics/git_log.md diff --git a/doc/university/training/topics/gitlab_flow.md b/doc/university/training/topics/gitlab_flow.md index 8e5d3baf959..8e5d3baf959 100755..100644 --- a/doc/university/training/topics/gitlab_flow.md +++ b/doc/university/training/topics/gitlab_flow.md diff --git a/doc/university/training/topics/merge_conflicts.md b/doc/university/training/topics/merge_conflicts.md index 77807b3e7ef..77807b3e7ef 100755..100644 --- a/doc/university/training/topics/merge_conflicts.md +++ b/doc/university/training/topics/merge_conflicts.md diff --git a/doc/university/training/topics/merge_requests.md b/doc/university/training/topics/merge_requests.md index 5b446f02f63..5b446f02f63 100755..100644 --- a/doc/university/training/topics/merge_requests.md +++ b/doc/university/training/topics/merge_requests.md diff --git a/doc/university/training/topics/rollback_commits.md b/doc/university/training/topics/rollback_commits.md index cf647284604..cf647284604 100755..100644 --- a/doc/university/training/topics/rollback_commits.md +++ b/doc/university/training/topics/rollback_commits.md diff --git a/doc/university/training/topics/stash.md b/doc/university/training/topics/stash.md index c1bdda32645..c1bdda32645 100755..100644 --- a/doc/university/training/topics/stash.md +++ b/doc/university/training/topics/stash.md diff --git a/doc/university/training/topics/subtree.md b/doc/university/training/topics/subtree.md index 5d869af64c1..5d869af64c1 100755..100644 --- a/doc/university/training/topics/subtree.md +++ b/doc/university/training/topics/subtree.md diff --git a/doc/university/training/topics/tags.md b/doc/university/training/topics/tags.md index e9607b5a875..e9607b5a875 100755..100644 --- a/doc/university/training/topics/tags.md +++ b/doc/university/training/topics/tags.md diff --git a/doc/university/training/topics/unstage.md b/doc/university/training/topics/unstage.md index 17dbb64b9e6..17dbb64b9e6 100755..100644 --- a/doc/university/training/topics/unstage.md +++ b/doc/university/training/topics/unstage.md diff --git a/doc/university/training/user_training.md b/doc/university/training/user_training.md index 9e38df26b6a..9e38df26b6a 100755..100644 --- a/doc/university/training/user_training.md +++ b/doc/university/training/user_training.md diff --git a/doc/update/10.0-to-10.1.md b/doc/update/10.0-to-10.1.md new file mode 100644 index 00000000000..4a9384f3ad6 --- /dev/null +++ b/doc/update/10.0-to-10.1.md @@ -0,0 +1,356 @@ +# From 10.0 to 10.1 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.5.tar.gz +echo '3247e217d6745c27ef23bdc77b6abdb4b57a118f ruby-2.3.5.tar.gz' | shasum -c - && tar xzf ruby-2.3.5.tar.gz +cd ruby-2.3.5 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets and +it has a minimum requirement of node v4.3.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v4.3.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-1-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-1-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell'. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-0-stable:config/gitlab.yml.example origin/10-1-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-0-stable:lib/support/nginx/gitlab-ssl origin/10-1-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-0-stable:lib/support/nginx/gitlab origin/10-1-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-1-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-1-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-0-stable:lib/support/init.d/gitlab.default.example origin/10-1-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (9.5) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 9.4 to 9.5](9.4-to-9.5.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-1-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-1-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/update/8.17-to-9.0.md b/doc/update/8.17-to-9.0.md index 2abc57da1a0..baab217b6b7 100644 --- a/doc/update/8.17-to-9.0.md +++ b/doc/update/8.17-to-9.0.md @@ -236,7 +236,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-17-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-0-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.0-to-9.1.md b/doc/update/9.0-to-9.1.md index 3fd1d023d2a..6f1870a1366 100644 --- a/doc/update/9.0-to-9.1.md +++ b/doc/update/9.0-to-9.1.md @@ -236,7 +236,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-0-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-1-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.1-to-9.2.md b/doc/update/9.1-to-9.2.md index 5f7a616cc7d..ce72b313031 100644 --- a/doc/update/9.1-to-9.2.md +++ b/doc/update/9.1-to-9.2.md @@ -194,7 +194,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-1-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-2-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md index 9d0b0da7edb..779ced0cf75 100644 --- a/doc/update/9.2-to-9.3.md +++ b/doc/update/9.2-to-9.3.md @@ -230,7 +230,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-2-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-3-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.3-to-9.4.md b/doc/update/9.3-to-9.4.md index 9ee01bc9c51..78d8a6c7de5 100644 --- a/doc/update/9.3-to-9.4.md +++ b/doc/update/9.3-to-9.4.md @@ -243,7 +243,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-3-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-4-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.4-to-9.5.md b/doc/update/9.4-to-9.5.md index 1b5a15589af..a7255142ef5 100644 --- a/doc/update/9.4-to-9.5.md +++ b/doc/update/9.4-to-9.5.md @@ -252,7 +252,7 @@ ActionMailer::Base.delivery_method = :smtp See [smtp_settings.rb.sample] as an example. -[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-4-stable/config/initializers/smtp_settings.rb.sample#L13 +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-5-stable/config/initializers/smtp_settings.rb.sample#L13 #### Init script diff --git a/doc/update/9.5-to-10.0.md b/doc/update/9.5-to-10.0.md new file mode 100644 index 00000000000..8581e6511f2 --- /dev/null +++ b/doc/update/9.5-to-10.0.md @@ -0,0 +1,356 @@ +# From 9.5 to 10.0 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.3.tar.gz +echo '1014ee699071aa2ddd501907d18cbe15399c997d ruby-2.3.3.tar.gz' | shasum -c - && tar xzf ruby-2.3.3.tar.gz +cd ruby-2.3.3 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets and +it has a minimum requirement of node v4.3.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v4.3.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-0-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-0-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell'. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.0 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/9-5-stable:config/gitlab.yml.example origin/10-0-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/9-5-stable:lib/support/nginx/gitlab-ssl origin/10-0-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/9-5-stable:lib/support/nginx/gitlab origin/10-0-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-0-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-0-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/9-5-stable:lib/support/init.d/gitlab.default.example origin/10-0-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (9.5) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 9.4 to 9.5](9.4-to-9.5.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-0-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-0-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/update/mysql_to_postgresql.md b/doc/update/mysql_to_postgresql.md index a7de5648c0e..fff47180099 100644 --- a/doc/update/mysql_to_postgresql.md +++ b/doc/update/mysql_to_postgresql.md @@ -1,78 +1,267 @@ -# Migrating GitLab from MySQL to Postgres -*Make sure you view this [guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/update/mysql_to_postgresql.md#migrating-gitlab-from-mysql-to-postgres) for the most up to date instructions.* +--- +last_updated: 2017-10-05 +--- -If you are replacing MySQL with Postgres while keeping GitLab on the same server all you need to do is to export from MySQL, convert the resulting SQL file, and import it into Postgres. If you are also moving GitLab to another server, or if you are switching to omnibus-gitlab, you may want to use a GitLab backup file. The second part of this documents explains the procedure to do this. +# Migrating from MySQL to PostgreSQL -## Export from MySQL and import into Postgres +> **Note:** This guide assumes you have a working Omnibus GitLab instance with +> MySQL and want to migrate to bundled PostgreSQL database. -Use this if you are keeping GitLab on the same server. +## Prerequisites -``` -sudo service gitlab stop +First, we'll need to enable the bundled PostgreSQL database with up-to-date +schema. Next, we'll use [pgloader](http://pgloader.io) to migrate the data +from the old MySQL database to the new PostgreSQL one. -# Update /home/git/gitlab/config/database.yml +Here's what you'll need to have installed: -git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab -cd mysql-postgresql-converter -mysqldump --compatible=postgresql --default-character-set=utf8 -r gitlabhq_production.mysql -u root gitlabhq_production -p -python db_converter.py gitlabhq_production.mysql gitlabhq_production.psql -ed -s gitlabhq_production.psql < move_drop_indexes.ed +- pgloader 3.4.1+ +- Omnibus GitLab +- MySQL -# Import the database dump as the application database user -sudo -u git psql -f gitlabhq_production.psql -d gitlabhq_production +## Enable bundled PostgreSQL database -# Install gems for PostgreSQL (note: the line below states '--without ... mysql') -sudo -u git -H bundle install --without development test mysql --deployment +1. Stop GitLab: -sudo service gitlab start -``` + ``` bash + sudo gitlab-ctl stop + ``` -## Converting a GitLab backup file from MySQL to Postgres -**Note:** Please make sure to have Python 2.7.x (or higher) installed. +1. Edit `/etc/gitlab/gitlab.rb` to enable bundled PostgreSQL: -GitLab backup files (`<timestamp>_gitlab_backup.tar`) contain a SQL dump. Using the lanyrd database converter we can replace a MySQL database dump inside the tar file with a Postgres database dump. This can be useful if you are moving to another server. + ``` + postgresql['enable'] = true + ``` -``` -# Stop GitLab -sudo service gitlab stop +1. Edit `/etc/gitlab/gitlab.rb` to use the bundled PostgreSQL. Please check + all the settings beginning with `db_`, such as `gitlab_rails['db_adapter']` + and alike. You could just comment all of them out so that we'll just use + the defaults. + +1. [Reconfigure GitLab] for the changes to take effect: + + ``` bash + sudo gitlab-ctl reconfigure + ``` + +1. Start Unicorn and PostgreSQL so that we can prepare the schema: + + ``` bash + sudo gitlab-ctl start unicorn + sudo gitlab-ctl start postgresql + ``` + +1. Run the following commands to prepare the schema: + + ``` bash + sudo gitlab-rake db:create db:migrate + ``` + +1. Stop Unicorn to prevent other database access from interfering with the loading of data: + + ``` bash + sudo gitlab-ctl stop unicorn + ``` + +After these steps, you'll have a fresh PostgreSQL database with up-to-date schema. + +## Migrate data from MySQL to PostgreSQL + +Now, you can use pgloader to migrate the data from MySQL to PostgreSQL: -# Create the backup -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +1. Save the following snippet in a `commands.load` file, and edit with your + database `username`, `password` and `host`: -# Note the filename of the backup that was created. We will call it -# TIMESTAMP_gitlab_backup.tar below. + ``` + LOAD DATABASE + FROM mysql://username:password@host/gitlabhq_production + INTO postgresql://gitlab-psql@unix://var/opt/gitlab/postgresql:/gitlabhq_production -# Move the backup file we will convert to its own directory -sudo -u git -H mkdir -p tmp/backups/postgresql -sudo -u git -H mv tmp/backups/TIMESTAMP_gitlab_backup.tar tmp/backups/postgresql/ + WITH include no drop, truncate, disable triggers, create no tables, + create no indexes, preserve index names, no foreign keys, + data only -# Create a separate database dump with PostgreSQL compatibility -cd tmp/backups/postgresql -sudo -u git -H mysqldump --compatible=postgresql --default-character-set=utf8 -r gitlabhq_production.mysql -u root gitlabhq_production -p + ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public' -# Clone the database converter -sudo -u git -H git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab + ; + ``` -# Convert gitlabhq_production.mysql -sudo -u git -H mkdir db -sudo -u git -H python mysql-postgresql-converter/db_converter.py gitlabhq_production.mysql db/database.sql -sudo -u git -H ed -s db/database.sql < mysql-postgresql-converter/move_drop_indexes.ed +1. Start the migration: -# Compress database backup -# Warning: If you have Gitlab 7.12.0 or older skip this step and import the database.sql directly into the backup with: -# sudo -u git -H tar rf TIMESTAMP_gitlab_backup.tar db/database.sql -# The compressed databasedump is not supported at 7.12.0 and older. -sudo -u git -H gzip db/database.sql + ``` bash + sudo -u gitlab-psql pgloader commands.load + ``` -# Replace the MySQL dump in TIMESTAMP_gitlab_backup.tar. +1. Once the migration finishes, you should see a summary table that looks like +the following: -# Warning: if you forget to replace TIMESTAMP below, tar will create a new file -# 'TIMESTAMP_gitlab_backup.tar' without giving an error. -sudo -u git -H tar rf TIMESTAMP_gitlab_backup.tar db/database.sql.gz + ``` + table name read imported errors total time + ----------------------------------------------- --------- --------- --------- -------------- + fetch meta data 119 119 0 0.388s + Truncate 119 119 0 1.134s + ----------------------------------------------- --------- --------- --------- -------------- + public.abuse_reports 0 0 0 0.490s + public.appearances 0 0 0 0.488s + public.approvals 0 0 0 0.273s + public.application_settings 1 1 0 0.266s + public.approvers 0 0 0 0.339s + public.approver_groups 0 0 0 0.357s + public.audit_events 1 1 0 0.410s + public.award_emoji 0 0 0 0.441s + public.boards 0 0 0 0.505s + public.broadcast_messages 0 0 0 0.498s + public.chat_names 0 0 0 0.576s + public.chat_teams 0 0 0 0.617s + public.ci_builds 0 0 0 0.611s + public.ci_group_variables 0 0 0 0.620s + public.ci_pipelines 0 0 0 0.599s + public.ci_pipeline_schedules 0 0 0 0.622s + public.ci_pipeline_schedule_variables 0 0 0 0.573s + public.ci_pipeline_variables 0 0 0 0.594s + public.ci_runners 0 0 0 0.533s + public.ci_runner_projects 0 0 0 0.584s + public.ci_sources_pipelines 0 0 0 0.564s + public.ci_stages 0 0 0 0.595s + public.ci_triggers 0 0 0 0.569s + public.ci_trigger_requests 0 0 0 0.596s + public.ci_variables 0 0 0 0.565s + public.container_repositories 0 0 0 0.605s + public.conversational_development_index_metrics 0 0 0 0.571s + public.deployments 0 0 0 0.607s + public.emails 0 0 0 0.602s + public.deploy_keys_projects 0 0 0 0.557s + public.events 160 160 0 0.677s + public.environments 0 0 0 0.567s + public.features 0 0 0 0.639s + public.events_for_migration 160 160 0 0.582s + public.feature_gates 0 0 0 0.579s + public.forked_project_links 0 0 0 0.660s + public.geo_nodes 0 0 0 0.686s + public.geo_event_log 0 0 0 0.626s + public.geo_repositories_changed_events 0 0 0 0.677s + public.geo_node_namespace_links 0 0 0 0.618s + public.geo_repository_renamed_events 0 0 0 0.696s + public.gpg_keys 0 0 0 0.704s + public.geo_repository_deleted_events 0 0 0 0.638s + public.historical_data 0 0 0 0.729s + public.geo_repository_updated_events 0 0 0 0.634s + public.index_statuses 0 0 0 0.746s + public.gpg_signatures 0 0 0 0.667s + public.issue_assignees 80 80 0 0.769s + public.identities 0 0 0 0.655s + public.issue_metrics 80 80 0 0.781s + public.issues 80 80 0 0.720s + public.labels 0 0 0 0.795s + public.issue_links 0 0 0 0.707s + public.label_priorities 0 0 0 0.793s + public.keys 0 0 0 0.734s + public.lfs_objects 0 0 0 0.812s + public.label_links 0 0 0 0.725s + public.licenses 0 0 0 0.813s + public.ldap_group_links 0 0 0 0.751s + public.members 52 52 0 0.830s + public.lfs_objects_projects 0 0 0 0.738s + public.merge_requests_closing_issues 0 0 0 0.825s + public.lists 0 0 0 0.769s + public.merge_request_diff_commits 0 0 0 0.840s + public.merge_request_metrics 0 0 0 0.837s + public.merge_requests 0 0 0 0.753s + public.merge_request_diffs 0 0 0 0.771s + public.namespaces 30 30 0 0.874s + public.merge_request_diff_files 0 0 0 0.775s + public.notes 0 0 0 0.849s + public.milestones 40 40 0 0.799s + public.oauth_access_grants 0 0 0 0.979s + public.namespace_statistics 0 0 0 0.797s + public.oauth_applications 0 0 0 0.899s + public.notification_settings 72 72 0 0.818s + public.oauth_access_tokens 0 0 0 0.807s + public.pages_domains 0 0 0 0.958s + public.oauth_openid_requests 0 0 0 0.832s + public.personal_access_tokens 0 0 0 0.965s + public.projects 8 8 0 0.987s + public.path_locks 0 0 0 0.925s + public.plans 0 0 0 0.923s + public.project_features 8 8 0 0.985s + public.project_authorizations 66 66 0 0.969s + public.project_import_data 8 8 0 1.002s + public.project_statistics 8 8 0 1.001s + public.project_group_links 0 0 0 0.949s + public.project_mirror_data 0 0 0 0.972s + public.protected_branch_merge_access_levels 0 0 0 1.017s + public.protected_branches 0 0 0 0.969s + public.protected_branch_push_access_levels 0 0 0 0.991s + public.protected_tags 0 0 0 1.009s + public.protected_tag_create_access_levels 0 0 0 0.985s + public.push_event_payloads 0 0 0 1.041s + public.push_rules 0 0 0 0.999s + public.redirect_routes 0 0 0 1.020s + public.remote_mirrors 0 0 0 1.034s + public.releases 0 0 0 0.993s + public.schema_migrations 896 896 0 1.057s + public.routes 38 38 0 1.021s + public.services 0 0 0 1.055s + public.sent_notifications 0 0 0 1.003s + public.slack_integrations 0 0 0 1.022s + public.spam_logs 0 0 0 1.024s + public.snippets 0 0 0 1.058s + public.subscriptions 0 0 0 1.069s + public.taggings 0 0 0 1.099s + public.timelogs 0 0 0 1.104s + public.system_note_metadata 0 0 0 1.038s + public.tags 0 0 0 1.034s + public.trending_projects 0 0 0 1.140s + public.uploads 0 0 0 1.129s + public.todos 80 80 0 1.085s + public.users_star_projects 0 0 0 1.153s + public.u2f_registrations 0 0 0 1.061s + public.web_hooks 0 0 0 1.179s + public.users 26 26 0 1.163s + public.user_agent_details 0 0 0 1.068s + public.web_hook_logs 0 0 0 1.080s + ----------------------------------------------- --------- --------- --------- -------------- + COPY Threads Completion 4 4 0 2.008s + Reset Sequences 113 113 0 0.304s + Install Comments 0 0 0 0.000s + ----------------------------------------------- --------- --------- --------- -------------- + Total import time 1894 1894 0 12.497s + ``` + + If there is no output for more than 30 minutes, it's possible pgloader encountered an error. See + the [troubleshooting guide](#Troubleshooting) for more details. + +1. Start GitLab: + + ``` bash + sudo gitlab-ctl start + ``` + +Now, you can verify that everything worked by visiting GitLab. + +## Troubleshooting + +### Permissions + +Note that the PostgreSQL user that you use for the above MUST have **superuser** privileges. Otherwise, you may see +a similar message to the following: -# Done! TIMESTAMP_gitlab_backup.tar can now be restored into a Postgres GitLab -# installation. -# See https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/raketasks/backup_restore.md for more information about backups. ``` +debugger invoked on a CL-POSTGRES-ERROR:INSUFFICIENT-PRIVILEGE in thread + #<THREAD "lparallel" RUNNING {10078A3513}>: + Database error 42501: permission denied: "RI_ConstraintTrigger_a_20937" is a system trigger + QUERY: ALTER TABLE ci_builds DISABLE TRIGGER ALL; + 2017-08-23T00:36:56.782000Z ERROR Database error 42501: permission denied: "RI_ConstraintTrigger_c_20864" is a system trigger + QUERY: ALTER TABLE approver_groups DISABLE TRIGGER ALL; +``` + +### Experiencing 500 errors after the migration + +If you experience 500 errors after the migration, try to clear the cache: + +``` bash +sudo gitlab-rake cache:clear +``` + +[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 30107360446..b2679d1ff22 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -74,7 +74,15 @@ cd /home/git/gitlab sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production ``` -### 5. Update gitlab-shell to the corresponding version +### 5. Update gitaly to the corresponding version + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake "gitlab:gitaly:install[/home/git/gitaly]" RAILS_ENV=production +``` + +### 6. Update gitlab-shell to the corresponding version ```bash cd /home/git/gitlab-shell @@ -84,14 +92,14 @@ sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` -b v`ca sudo -u git -H sh -c 'if [ -x bin/compile ]; then bin/compile; fi' ``` -### 6. Start application +### 7. Start application ```bash sudo service gitlab start sudo service nginx restart ``` -### 7. Check application status +### 8. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/user/admin_area/monitoring/convdev.md b/doc/user/admin_area/monitoring/convdev.md index 3d93c7557a4..a98602c4d70 100644 --- a/doc/user/admin_area/monitoring/convdev.md +++ b/doc/user/admin_area/monitoring/convdev.md @@ -23,7 +23,7 @@ If you have just started using GitLab, it may take a few weeks for data to be collected before this feature is available. This feature is accessible only to a system admin, at -**Admin area > Monitoring > ConvDev Index**. +**Admin area > Overview > ConvDev Index**. [ce-30469]: https://gitlab.com/gitlab-org/gitlab-ce/issues/30469 [ping]: ../settings/usage_statistics.md#usage-ping diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 70934f9960a..843fb4ce26b 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -18,7 +18,7 @@ traffic until the system is ready or restart the container as needed. To access monitoring resources, the client IP needs to be included in a whitelist. -[Read how to add IPs to a whitelist for the monitoring endpoints.][admin]. +[Read how to add IPs to a whitelist for the monitoring endpoints][admin]. ## Using the endpoint diff --git a/doc/user/admin_area/monitoring/img/convdev_index.png b/doc/user/admin_area/monitoring/img/convdev_index.png Binary files differindex 4e47ff2228d..ffe18d76c96 100644 --- a/doc/user/admin_area/monitoring/img/convdev_index.png +++ b/doc/user/admin_area/monitoring/img/convdev_index.png diff --git a/doc/user/discussions/img/discussion_lock_system_notes.png b/doc/user/discussions/img/discussion_lock_system_notes.png Binary files differnew file mode 100644 index 00000000000..8e8e8e0bc3d --- /dev/null +++ b/doc/user/discussions/img/discussion_lock_system_notes.png diff --git a/doc/user/discussions/img/lock_form_member.png b/doc/user/discussions/img/lock_form_member.png Binary files differnew file mode 100644 index 00000000000..01c6308d24c --- /dev/null +++ b/doc/user/discussions/img/lock_form_member.png diff --git a/doc/user/discussions/img/lock_form_non_member.png b/doc/user/discussions/img/lock_form_non_member.png Binary files differnew file mode 100644 index 00000000000..3bb70b69580 --- /dev/null +++ b/doc/user/discussions/img/lock_form_non_member.png diff --git a/doc/user/discussions/img/turn_off_lock.png b/doc/user/discussions/img/turn_off_lock.png Binary files differnew file mode 100644 index 00000000000..dd05b398a8b --- /dev/null +++ b/doc/user/discussions/img/turn_off_lock.png diff --git a/doc/user/discussions/img/turn_on_lock.png b/doc/user/discussions/img/turn_on_lock.png Binary files differnew file mode 100644 index 00000000000..9597da4e14d --- /dev/null +++ b/doc/user/discussions/img/turn_on_lock.png diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index efea99eb120..ab46befb91c 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -153,12 +153,52 @@ comments in greater detail.  +## Locking discussions + +> [Introduced][ce-14531] in GitLab 10.1. + +There might be some cases where a discussion is better off if it's locked down. +For example: + +- Discussions that are several years old and the issue/merge request is closed, + but people continue to try to resurrect the discussion. +- Discussions where someone or a group of people are trolling, are abusive, or + in-general are causing the discussion to be unproductive. + +In locked discussions, only team members can write new comments and edit the old +ones. + +To lock or unlock a discussion, you need to have at least Master [permissions]: + +1. Find the "Lock" section in the sidebar and click **Edit** +1. In the dialog that will appear, you can choose to turn on or turn off the + discussion lock +1. Optionally, leave a comment to explain your reasoning behind that action + +| Turn off discussion lock | Turn on discussion lock | +| :-----------: | :----------: | +|  |  | + +Every change is indicated by a system note in the issue's or merge request's +comments. + + + +Once an issue or merge request is locked, project members can see the indicator +in the comment area, whereas non project members can only see the information +that the discussion is locked. + +| Team member | Not a member | +| :-----------: | :----------: | +|  |  | + [ce-5022]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5022 [ce-7125]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7125 [ce-7527]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7527 [ce-7180]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7180 [ce-8266]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8266 [ce-14053]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14053 +[ce-14531]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14531 [resolve-discussion-button]: img/resolve_discussion_button.png [resolve-comment-button]: img/resolve_comment_button.png [discussion-view]: img/discussion_view.png diff --git a/doc/user/group/index.md b/doc/user/group/index.md index db0242f1324..a1671f9dd91 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -188,6 +188,27 @@ Besides giving you the option to edit any settings you've previously set when [creating the group](#create-a-new-group), you can also access further configurations for your group. +#### Changing a group's path + +> **Note:** If you want to retain ownership over the original namespace and +protect the URL redirects, then instead of changing a group's path or renaming a +username, you can create a new group and transfer projects to it. + +Changing a group's path can have unintended side effects. + +* Existing web URLs for the group and anything under it (i.e. projects) will +redirect to the new URLs +* Existing Git remote URLs for projects under the group will no longer work, but +Git responses will show an error with the new remote URL +* The original namespace can be claimed again by any group or user, which will +destroy web redirects and Git remote warnings +* If you are vacating the path so it can be claimed by another group or user, +you may need to rename the group name as well since both names and paths must be +unique + +> It is currently not possible to rename a namespace if it contains a +project with container registry tags, because the project cannot be moved. + #### Enforce 2FA to group members Add a security layer to your group by diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index d2478aea4bd..161a3af9903 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -84,10 +84,13 @@ structure. a subgroup. For more information check the [permissions table][permissions]. - For a list of words that are not allowed to be used as group names see the [reserved names][reserved]. +- Users can always create subgroups if they are explicitly added as an Owner to + a parent group even if group creation is disabled by an administrator in their + settings. To create a subgroup: -1. In the group's dashboard go to the **Subgroups** page and click **Create subgroup**. +1. In the group's dashboard go to the **Subgroups** page and click **New subgroup**.  @@ -100,9 +103,7 @@ To create a subgroup: 1. Click the **Create group** button and you will be taken to the new group's dashboard page. ---- - -You can follow the same process to create any subsequent groups. +Follow the same process to create any subsequent groups. ## Membership diff --git a/doc/user/markdown.md b/doc/user/markdown.md index b42b8f0a525..454988b9b80 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -596,6 +596,30 @@ See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubyd <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> </dl> +#### Details and Summary + +Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) and [`<summary>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary) tags. This is especially useful for collapsing long logs so they take up less screen space. + +<p> +<details> +<summary>Click me to collapse/fold.</summary> +These details will remain hidden until expanded. + +<pre><code>PASTE LOGS HERE</code></pre> +</details> +</p> + +**Note:** Unfortunately Markdown is not supported inside these tags, as described by the [markdown specification](https://daringfireball.net/projects/markdown/syntax#html). You can work around this by using HTML, for example you can use `<pre><code>` tags instead of [code fences](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting). + +```html +<details> +<summary>Click me to collapse/fold.</summary> +These details will remain hidden until expanded. + +<pre><code>PASTE LOGS HERE</code></pre> +</details> +``` + ### Horizontal Rule ``` diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 0c17905aa8c..be72d42625a 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -25,6 +25,7 @@ The following table depicts the various user permission levels in a project. | Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | | View confidential issues | (✓) [^2] | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| Lock comments | | | | ✓ | ✓ | | See a list of jobs | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | | See a job log | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | | Download and browse job artifacts | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | @@ -71,6 +72,7 @@ The following table depicts the various user permission levels in a project. | Switch visibility level | | | | | ✓ | | Transfer project to another namespace | | | | | ✓ | | Remove project | | | | | ✓ | +| Delete issues | | | | | ✓ | | Force push to protected branches [^4] | | | | | | | Remove protected branches [^4] | | | | | | | Remove pages | | | | | ✓ | @@ -253,7 +255,7 @@ only. [^1]: On public and internal projects, all users are able to perform this action. [^2]: Guest users can only view the confidential issues they created themselves -[^3]: If **Public pipelines** is enabled in **Project Settings > Pipelines** +[^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** [^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner [^5]: Only if user is not external one. [^6]: Only if user is a member of the project. diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 7d25970fcb1..5ebb88bf324 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -8,10 +8,27 @@ experience according to the best approach to their cases. Your `username` is a unique [`namespace`](../group/index.md#namespaces) related to your user ID. +### Changing your username + You can change your `username` from your -[profile settings](#profile-settings). To avoid breaking -paths when you change your `username`, we suggest you follow -[this procedure from the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#how-to-change-your-username-at-gitlabcom). +[profile settings](#profile-settings). + +> **Note:** If you want to retain ownership over the original namespace and +protect the URL redirects, then instead of changing your username, you can +create a new group and transfer projects to it. +Alternatively, you can follow [this detailed procedure from the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#how-to-change-your-username-at-gitlabcom). + +Changing your username can have unintended side effects. + +* Existing web URLs for the user and anything under it (i.e. projects) will +redirect to the new URLs +* Existing Git remote URLs for projects under the user will no longer work, but +Git responses will show an error with the new remote URL +* The original namespace can be claimed again by any group or user, which will +destroy any web redirects and Git remote warnings + +> It is currently not possible to rename a namespace if it contains a +project with container registry tags, because the project cannot be moved. ## User profile diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 5c615daf464..2c4dfcff4a6 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -17,25 +17,25 @@ have its own space to store its Docker images. You can read more about Docker Registry at https://docs.docker.com/registry/introduction/. ---- - ## Enable the Container Registry for your project +NOTE: **Note:** +If you cannot find the Container Registry entry under your project's settings, +that means that it is not enabled in your GitLab instance. Ask your administrator +to enable it. + 1. First, ask your system administrator to enable GitLab Container Registry following the [administration documentation](../../administration/container_registry.md). If you are using GitLab.com, this is enabled by default so you can start using the Registry immediately. - -1. Go to your project's settings and enable the **Container Registry** feature - on your project. For new projects this might be enabled by default. For - existing projects (prior GitLab 8.8), you will have to explicitly enable it. - -  - +1. Go to your [project's General settings](settings/index.md#sharing-and-permissions) + and enable the **Container Registry** feature on your project. For new + projects this might be enabled by default. For existing projects + (prior GitLab 8.8), you will have to explicitly enable it. 1. Hit **Save changes** for the changes to take effect. You should now be able - to see the **Registry** link in the project menu. + to see the **Registry** link in the sidebar. -  + ## Build and push images diff --git a/doc/user/project/img/container_registry.png b/doc/user/project/img/container_registry.png Binary files differnew file mode 100644 index 00000000000..abbaf838538 --- /dev/null +++ b/doc/user/project/img/container_registry.png diff --git a/doc/user/project/img/container_registry_enable.png b/doc/user/project/img/container_registry_enable.png Binary files differdeleted file mode 100644 index d067a8be1ca..00000000000 --- a/doc/user/project/img/container_registry_enable.png +++ /dev/null diff --git a/doc/user/project/img/container_registry_tab.png b/doc/user/project/img/container_registry_tab.png Binary files differdeleted file mode 100644 index a85237271d9..00000000000 --- a/doc/user/project/img/container_registry_tab.png +++ /dev/null diff --git a/doc/user/project/img/issue_board.png b/doc/user/project/img/issue_board.png Binary files differindex cf7f519f783..5f6dc9e4e8b 100644 --- a/doc/user/project/img/issue_board.png +++ b/doc/user/project/img/issue_board.png diff --git a/doc/user/project/img/issue_board_move_issue_card_list.png b/doc/user/project/img/issue_board_move_issue_card_list.png Binary files differindex c6b17ada40e..3666dbb87ab 100644 --- a/doc/user/project/img/issue_board_move_issue_card_list.png +++ b/doc/user/project/img/issue_board_move_issue_card_list.png diff --git a/doc/user/project/img/labels_assign_label_in_new_issue.png b/doc/user/project/img/labels_assign_label_in_new_issue.png Binary files differdeleted file mode 100644 index badfbed0bbe..00000000000 --- a/doc/user/project/img/labels_assign_label_in_new_issue.png +++ /dev/null diff --git a/doc/user/project/img/labels_default.png b/doc/user/project/img/labels_default.png Binary files differindex 474953d565b..7934e3bfb5e 100644 --- a/doc/user/project/img/labels_default.png +++ b/doc/user/project/img/labels_default.png diff --git a/doc/user/project/img/labels_filter.png b/doc/user/project/img/labels_filter.png Binary files differindex 3aca77f0070..6a1ebfc2ecb 100644 --- a/doc/user/project/img/labels_filter.png +++ b/doc/user/project/img/labels_filter.png diff --git a/doc/user/project/img/labels_filter_by_priority.png b/doc/user/project/img/labels_filter_by_priority.png Binary files differindex 5609a1f6d7f..419e555e709 100644 --- a/doc/user/project/img/labels_filter_by_priority.png +++ b/doc/user/project/img/labels_filter_by_priority.png diff --git a/doc/user/project/img/labels_new_label.png b/doc/user/project/img/labels_new_label.png Binary files differindex b44b4bd296d..e26425d0188 100644 --- a/doc/user/project/img/labels_new_label.png +++ b/doc/user/project/img/labels_new_label.png diff --git a/doc/user/project/img/labels_prioritize.png b/doc/user/project/img/labels_prioritize.png Binary files differindex 3e888f36364..d602a3c90ec 100644 --- a/doc/user/project/img/labels_prioritize.png +++ b/doc/user/project/img/labels_prioritize.png diff --git a/doc/user/project/img/project_repository_settings.png b/doc/user/project/img/project_repository_settings.png Binary files differindex 1aa7efc36f1..aa4d4452c87 100644 --- a/doc/user/project/img/project_repository_settings.png +++ b/doc/user/project/img/project_repository_settings.png diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 016f98966e3..6423beefc77 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -42,6 +42,11 @@ The importer will create any new namespaces (groups) if they don't exist or in the case the namespace is taken, the repository will be imported under the user's namespace that started the import process. +The importer will also import branches on forks of projects related to open pull +requests. These branches will be imported with a naming scheume similar to +GH-SHA-Username/Pull-Request-number/fork-name/branch. This may lead to a discrepency +in branches compared to the GitHub Repository. + ## Importing your GitHub repositories The importer page is visible when you create a new project. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index d6b3d59d407..03bbc46bd8c 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -20,6 +20,8 @@ When you create a project in GitLab, you'll have access to a large number of - [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards) (**EES/EEP**): Allow your teams to create their own workflows (Issue Boards) for the same project - [Repositories](repository/index.md): Host your code in a fully integrated platform + - [Branches](repository/branches/index.md): use Git branching strategies to + collaborate on code - [Protected branches](protected_branches.md): Prevent collaborators from messing with history or pushing code without review - [Protected tags](protected_tags.md): Control over who has @@ -87,6 +89,10 @@ You can [fork a project](../../gitlab-basics/fork-project.md) in order to: from your fork to the upstream project - Fork a sample project to work on the top of that +## Project settings + +Read through the documentation on [project settings](settings/index.md). + ## Import or export a project - [Import a project](import/index.md) from: diff --git a/doc/user/project/integrations/img/kubernetes_configuration.png b/doc/user/project/integrations/img/kubernetes_configuration.png Binary files differindex 349a2dc8456..e535e2b8d46 100644 --- a/doc/user/project/integrations/img/kubernetes_configuration.png +++ b/doc/user/project/integrations/img/kubernetes_configuration.png diff --git a/doc/user/project/integrations/img/webhook_logs.png b/doc/user/project/integrations/img/webhook_logs.png Binary files differindex 917068d9398..917068d9398 100755..100644 --- a/doc/user/project/integrations/img/webhook_logs.png +++ b/doc/user/project/integrations/img/webhook_logs.png diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index f4000523938..e9738b683f9 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -1,3 +1,7 @@ +--- +last_updated: 2017-09-25 +--- + # GitLab Kubernetes / OpenShift integration GitLab can be configured to interact with Kubernetes, or other systems using the @@ -6,62 +10,114 @@ Kubernetes API (such as OpenShift). Each project can be configured to connect to a different Kubernetes cluster, see the [configuration](#configuration) section. -If you have a single cluster that you want to use for all your projects, -you can pre-fill the settings page with a default template. To configure the -template, see the [Services Templates](services_templates.md) document. - ## Configuration Navigate to the [Integrations page](project_services.md#accessing-the-project-services) -of your project and select the **Kubernetes** service to configure it. +of your project and select the **Kubernetes** service to configure it. Fill in +all the needed parameters, check the "Active" checkbox and hit **Save changes** +for the changes to take effect.  -The Kubernetes service takes the following arguments: - -1. API URL -1. Custom CA bundle -1. Kubernetes namespace -1. Service token - -The API URL is the URL that GitLab uses to access the Kubernetes API. Kubernetes -exposes several APIs - we want the "base" URL that is common to all of them, -e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. - -GitLab authenticates against Kubernetes using service tokens, which are -scoped to a particular `namespace`. If you don't have a service token yet, -you can follow the -[Kubernetes documentation](http://kubernetes.io/docs/user-guide/service-accounts/) -to create one. You can also view or create service tokens in the -[Kubernetes dashboard](http://kubernetes.io/docs/user-guide/ui/) - visit -`Config -> Secrets`. - -Fill in the service token and namespace according to the values you just got. -If the API is using a self-signed TLS certificate, you'll also need to include -the `ca.crt` contents as the `Custom CA bundle`. +The Kubernetes service takes the following parameters: + +- **API URL** - + It's the URL that GitLab uses to access the Kubernetes API. Kubernetes + exposes several APIs, we want the "base" URL that is common to all of them, + e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. +- **CA certificate** (optional) - + If the API is using a self-signed TLS certificate, you'll also need to include + the `ca.crt` contents here. +- **Project namespace** (optional) - The following apply: + - By default you don't have to fill it in; by leaving it blank, GitLab will + create one for you. + - Each project should have a unique namespace. + - The project namespace is not necessarily the namespace of the secret, if + you're using a secret with broader permissions, like the secret from `default`. + - You should **not** use `default` as the project namespace. + - If you or someone created a secret specifically for the project, usually + with limited permissions, the secret's namespace and project namespace may + be the same. +- **Token** - + GitLab authenticates against Kubernetes using service tokens, which are + scoped to a particular `namespace`. If you don't have a service token yet, + you can follow the + [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) + to create one. You can also view or create service tokens in the + [Kubernetes dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/#config) + (under **Config > Secrets**). + +TIP: **Tip:** +If you have a single cluster that you want to use for all your projects, +you can pre-fill the settings page with a default template. To configure the +template, see [Services Templates](services_templates.md). ## Deployment variables -The Kubernetes service exposes following +The Kubernetes service exposes the following [deployment variables](../../../ci/variables/README.md#deployment-variables) in the -GitLab CI build environment: +GitLab CI/CD build environment: -- `KUBE_URL` - equal to the API URL -- `KUBE_TOKEN` +- `KUBE_URL` - Equal to the API URL. +- `KUBE_TOKEN` - The Kubernetes token. - `KUBE_NAMESPACE` - The Kubernetes namespace is auto-generated if not specified. The default value is `<project_name>-<project_id>`. You can overwrite it to use different one if needed, otherwise the `KUBE_NAMESPACE` variable will receive the default value. -- `KUBE_CA_PEM_FILE` - only present if a custom CA bundle was specified. Path +- `KUBE_CA_PEM_FILE` - Only present if a custom CA bundle was specified. Path to a file containing PEM data. -- `KUBE_CA_PEM` (deprecated)- only if a custom CA bundle was specified. Raw PEM data. -- `KUBECONFIG` - Path to a file containing kubeconfig for this deployment. CA bundle would be embedded if specified. +- `KUBE_CA_PEM` (deprecated) - Only if a custom CA bundle was specified. Raw PEM data. +- `KUBECONFIG` - Path to a file containing `kubeconfig` for this deployment. + CA bundle would be embedded if specified. + +## What you can get with the Kubernetes integration + +Here's what you can do with GitLab if you enable the Kubernetes integration. + +### Deploy Boards (EEP) + +> Available in [GitLab Enterprise Edition Premium][ee]. -## Web terminals +GitLab's Deploy Boards offer a consolidated view of the current health and +status of each CI [environment](../../../ci/environments.md) running on Kubernetes, +displaying the status of the pods in the deployment. Developers and other +teammates can view the progress and status of a rollout, pod by pod, in the +workflow they already use without any need to access Kubernetes. ->**NOTE:** -Added in GitLab 8.15. You must be the project owner or have `master` permissions -to use terminals. Support is currently limited to the first container in the +[> Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) + +### Canary Deployments (EEP) + +> Available in [GitLab Enterprise Edition Premium][ee]. + +Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) +and visualize your canary deployments right inside the Deploy Board, without +the need to leave GitLab. + +[> Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) + +### Kubernetes monitoring + +Automatically detect and monitor Kubernetes metrics. Automatic monitoring of +[NGINX ingress](./prometheus_library/nginx.md) is also supported. + +[> Read more about Kubernetes monitoring](prometheus_library/kubernetes.md) + +### Auto DevOps + +Auto DevOps automatically detects, builds, tests, deploys, and monitors your +applications. + +To make full use of Auto DevOps(Auto Deploy, Auto Review Apps, and Auto Monitoring) +you will need the Kubernetes project integration enabled. + +[> Read more about Auto DevOps](../../../topics/autodevops/index.md) + +### Web terminals + +NOTE: **Note:** +Introduced in GitLab 8.15. You must be the project owner or have `master` permissions +to use terminals. Support is limited to the first container in the first pod of your environment. When enabled, the Kubernetes service adds [web terminal](../../../ci/environments.md#web-terminals) @@ -70,3 +126,5 @@ Docker and Kubernetes, so you get a new shell session within your existing containers. To use this integration, you should deploy to Kubernetes using the deployment variables above, ensuring any pods you create are labelled with `app=$CI_ENVIRONMENT_SLUG`. GitLab will do the rest! + +[ee]: https://about.gitlab.com/gitlab-ee/ diff --git a/doc/user/project/integrations/prometheus_library/cloudwatch.md b/doc/user/project/integrations/prometheus_library/cloudwatch.md index cc5cee36d28..34a0b97a171 100644 --- a/doc/user/project/integrations/prometheus_library/cloudwatch.md +++ b/doc/user/project/integrations/prometheus_library/cloudwatch.md @@ -1,8 +1,13 @@ # Monitoring AWS Resources + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 GitLab has support for automatically detecting and monitoring AWS resources, starting with the [Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/). This is provided by leveraging the official [Cloudwatch exporter](https://github.com/prometheus/cloudwatch_exporter), which translates [Cloudwatch metrics](https://aws.amazon.com/cloudwatch/) into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md index f2939f047a3..518018e5839 100644 --- a/doc/user/project/integrations/prometheus_library/haproxy.md +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -3,11 +3,15 @@ GitLab has support for automatically detecting and monitoring HAProxy. This is provided by leveraging the [HAProxy Exporter](https://github.com/prometheus/haproxy_exporter), which translates HAProxy statistics into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | | ---- | ----- | -| Throughput (req/sec) | sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | +| Throughput (req/sec) | sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) by (code) | | HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_requests_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_requests_total{%{environment_filter}}[2m])) | ## Configuring Prometheus to monitor for HAProxy metrics diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 9f0308d8111..518683965e8 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -1,14 +1,20 @@ # Monitoring Kubernetes + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8935) in GitLab 9.0 -GitLab has support for automatically detecting and monitoring Kubernetes metrics. Kubernetes exposes Node level metrics out of the box via the built-in [Prometheus metrics support in cAdvisor](https://github.com/google/cadvisor). No additional services or exporters are needed. +GitLab has support for automatically detecting and monitoring Kubernetes metrics. + +## Requirements + +The [Prometheus](../prometheus.md) and [Kubernetes](../kubernetes.md) +integration services must be enabled. ## Metrics supported | Name | Query | | ---- | ----- | | Average Memory Usage (MB) | (sum(container_memory_usage_bytes{container_name!="POD",%{environment_filter}}) / count(container_memory_usage_bytes{container_name!="POD",%{environment_filter}})) /1024/1024 | -| Average CPU Utilization (%) | sum(rate(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}[2m])) / count(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}) * 100 | +| Average CPU Utilization (%) | sum(rate(container_cpu_usage_seconds_total{container_name!="POD",%{environment_filter}}[2m])) by (cpu) * 100 | ## Configuring Prometheus to monitor for Kubernetes node metrics @@ -23,4 +29,4 @@ Prometheus server up and running. You have two options here: In order to isolate and only display relevant metrics for a given environment however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). -If you are using [GitLab Auto-Deploy][../../../ci/autodeploy/index.md] and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. +If you are using [GitLab Auto-Deploy](../../../../ci/autodeploy/index.md) and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index 12e3321f5f3..7fb8369d3c1 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -1,13 +1,18 @@ # Monitoring NGINX + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 GitLab has support for automatically detecting and monitoring NGINX. This is provided by leveraging the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter), which translates [VTS statistics](https://github.com/vozlt/nginx-module-vts) into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | | ---- | ----- | -| Throughput (req/sec) | sum(rate(nginx_requests_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) | +| Throughput (req/sec) | sum(rate(nginx_responses_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) by (status_code) | | Latency (ms) | avg(nginx_upstream_response_msecs_avg{%{environment_filter}}) | | HTTP Error Rate (HTTP Errors / sec) | rate(nginx_responses_total{status_code="5xx", %{environment_filter}}[2m])) | diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 84ee8bc45e5..e6f13d0630b 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -1,25 +1,44 @@ # Monitoring NGINX Ingress Controller + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13438) in GitLab 9.5 GitLab has support for automatically detecting and monitoring the Kubernetes NGINX ingress controller. This is provided by leveraging the built in Prometheus metrics included in [version 0.9.0](https://github.com/kubernetes/ingress/blob/master/controllers/nginx/Changelog.md#09-beta1) of the ingress. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | | ---- | ----- | -| Throughput (req/sec) | sum(rate(nginx_upstream_requests_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) | +| Throughput (req/sec) | sum(rate(nginx_upstream_responses_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) by (status_code) | | Latency (ms) | avg(nginx_upstream_response_msecs_avg{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}) | | HTTP Error Rate (HTTP Errors / sec) | sum(rate(nginx_upstream_responses_total{status_code="5xx", upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) | -## Configuring Prometheus to monitor for NGINX ingress metrics +## Configuring NGINX ingress monitoring + +If you have deployed with the [gitlab-omnibus](https://docs.gitlab.com/ee/install/kubernetes/gitlab_omnibus.md) Helm chart, and your application is running in the same cluster, no further action is required. The ingress metrics will be automatically enabled and annotated for Prometheus monitoring. Simply ensure Prometheus monitoring is [enabled for your project](../prometheus.md), which is on by default. + +For other deployments, there is some configuration required depending on your installation: +* NGINX Ingress should be version 0.9.0 or above +* NGINX Ingress should be annotated for Prometheus monitoring +* Prometheus should be configured to monitor annotated pods + +### Setting up NGINX Ingress for Prometheus monitoring + +Version 0.9.0 and above of [NGINX ingress](https://github.com/kubernetes/ingress/tree/master/controllers/nginx) have built-in support for exporting Prometheus metrics. To enable, a ConfigMap setting must be passed: `enable-vts-status: "true"`. Once enabled, a Prometheus metrics endpoint will start running on port 10254. + +With metric data now available, Prometheus needs to be configured to collect it. The easiest way to do this is to leverage Prometheus' [built-in Kubernetes service discovery](https://prometheus.io/docs/operating/configuration/#kubernetes_sd_config), which automatically detects a variety of Kubernetes components and makes them available for monitoring. Since NGINX ingress metrics are exposed per pod, a scrape job for Kubernetes pods is required. A sample pod scraping configuration [is available](https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus-kubernetes.yml#L248). This configuration will detect pods and enable collection of metrics **only if** they have been specifically annotated for monitoring. -The easiest way to get started is to use at least version 0.9.0 of [NGINX ingress](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). If you are using NGINX as your Kubernetes ingress, there is [direct support](https://github.com/kubernetes/ingress/pull/423) for enabling Prometheus monitoring in the 0.9.0 release. +Depending on how NGINX ingress was deployed, typically a DaemonSet or Deployment, edit the corresponding YML spec. Two new annotations need to be added: +* `prometheus.io/scrape: "true"` +* `prometheus.io/port: "10254"` -If you have deployed with the [gitlab-omnibus](https://docs.gitlab.com/ee/install/kubernetes/gitlab_omnibus.md) Helm chart, these metrics will be automatically enabled and annotated for Prometheus monitoring. +Prometheus should now be collecting NGINX ingress metrics. To validate view the Prometheus Targets, available under `Status > Targets` on the Prometheus dashboard. New entries for NGINX should be listed in the kubernetes pod monitoring job, `kubernetes-pods`. ## Specifying the Environment label -In order to isolate and only display relevant metrics for a given environment -however, GitLab needs a method to detect which labels are associated. To do this, GitLab will search metrics with appropriate labels. In this case, the `upstream` label must be of the form `<Kubernetes Namespace>-<CI_ENVIRONMENT_SLUG>-*`. +In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do this, GitLab will search for metrics with appropriate labels. In this case, the `upstream` label must be of the form `<KUBE_NAMESPACE>-<CI_ENVIRONMENT_SLUG>-*`. If you have used [Auto Deploy](https://docs.gitlab.com/ee/ci/autodeploy/index.html) to deploy your app, this format will be used automatically and metrics will be detected with no action on your part. diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index e2cc67726e0..96a5a23ee13 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -12,6 +12,8 @@ Other interesting links: - [GitLab Issue Board landing page on about.gitlab.com][landing] - [YouTube video introduction to Issue Boards][youtube] + + ## Overview The Issue Board builds on GitLab's existing @@ -89,10 +91,6 @@ two defaults: - **Backlog** (default): shows all open issues that does not belong to one of lists. Always appears on the very left. - **Closed** (default): shows all closed issues. Always appears on the very right. - - ---- - In short, here's a list of actions you can take in an Issue Board: - [Create a new list](#creating-a-new-list). diff --git a/doc/user/project/issues/deleting_issues.md b/doc/user/project/issues/deleting_issues.md new file mode 100644 index 00000000000..d7442104c53 --- /dev/null +++ b/doc/user/project/issues/deleting_issues.md @@ -0,0 +1,11 @@ +# Deleting Issues + +> [Introduced][ce-2982] in GitLab 8.6 + +Please read through the [GitLab Issue Documentation](index.md) for an overview on GitLab Issues. + +You can delete an issue by editing it and clicking on the delete button. + + + +>**Note:** Only [project owners](../../permissions.md) can delete issues.
\ No newline at end of file diff --git a/doc/user/project/issues/img/button_close_issue.png b/doc/user/project/issues/img/button_close_issue.png Binary files differindex 8fb2e23f58a..05d257ce9bf 100755..100644 --- a/doc/user/project/issues/img/button_close_issue.png +++ b/doc/user/project/issues/img/button_close_issue.png diff --git a/doc/user/project/issues/img/closing_and_related_issues.png b/doc/user/project/issues/img/closing_and_related_issues.png Binary files differindex c6543e85fdb..c6543e85fdb 100755..100644 --- a/doc/user/project/issues/img/closing_and_related_issues.png +++ b/doc/user/project/issues/img/closing_and_related_issues.png diff --git a/doc/user/project/issues/img/confidential_issues_create.png b/doc/user/project/issues/img/confidential_issues_create.png Binary files differindex 0a141eb39f8..0a141eb39f8 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_create.png +++ b/doc/user/project/issues/img/confidential_issues_create.png diff --git a/doc/user/project/issues/img/confidential_issues_search_guest.png b/doc/user/project/issues/img/confidential_issues_search_guest.png Binary files differindex dc1b4ba8ad7..dc1b4ba8ad7 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_search_guest.png +++ b/doc/user/project/issues/img/confidential_issues_search_guest.png diff --git a/doc/user/project/issues/img/confidential_issues_search_master.png b/doc/user/project/issues/img/confidential_issues_search_master.png Binary files differindex fc01f4da9db..fc01f4da9db 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_search_master.png +++ b/doc/user/project/issues/img/confidential_issues_search_master.png diff --git a/doc/user/project/issues/img/delete_issue.png b/doc/user/project/issues/img/delete_issue.png Binary files differnew file mode 100644 index 00000000000..a356f52044e --- /dev/null +++ b/doc/user/project/issues/img/delete_issue.png diff --git a/doc/user/project/issues/img/due_dates_create.png b/doc/user/project/issues/img/due_dates_create.png Binary files differindex ece35d44213..ece35d44213 100755..100644 --- a/doc/user/project/issues/img/due_dates_create.png +++ b/doc/user/project/issues/img/due_dates_create.png diff --git a/doc/user/project/issues/img/due_dates_edit_sidebar.png b/doc/user/project/issues/img/due_dates_edit_sidebar.png Binary files differindex d1c7d1eb7e9..d1c7d1eb7e9 100755..100644 --- a/doc/user/project/issues/img/due_dates_edit_sidebar.png +++ b/doc/user/project/issues/img/due_dates_edit_sidebar.png diff --git a/doc/user/project/issues/img/due_dates_issues_index_page.png b/doc/user/project/issues/img/due_dates_issues_index_page.png Binary files differindex 94679436b32..94679436b32 100755..100644 --- a/doc/user/project/issues/img/due_dates_issues_index_page.png +++ b/doc/user/project/issues/img/due_dates_issues_index_page.png diff --git a/doc/user/project/issues/img/due_dates_todos.png b/doc/user/project/issues/img/due_dates_todos.png Binary files differindex 4c124c97f67..4c124c97f67 100755..100644 --- a/doc/user/project/issues/img/due_dates_todos.png +++ b/doc/user/project/issues/img/due_dates_todos.png diff --git a/doc/user/project/issues/img/group_issues_list_view.png b/doc/user/project/issues/img/group_issues_list_view.png Binary files differindex 5d20e8cbc89..bba964076d0 100644 --- a/doc/user/project/issues/img/group_issues_list_view.png +++ b/doc/user/project/issues/img/group_issues_list_view.png diff --git a/doc/user/project/issues/img/issue_board.png b/doc/user/project/issues/img/issue_board.png Binary files differindex 1759b28a9ef..87b1016cc76 100755..100644 --- a/doc/user/project/issues/img/issue_board.png +++ b/doc/user/project/issues/img/issue_board.png diff --git a/doc/user/project/issues/img/issue_template.png b/doc/user/project/issues/img/issue_template.png Binary files differindex c63229a4af2..0e4c8df897b 100755..100644 --- a/doc/user/project/issues/img/issue_template.png +++ b/doc/user/project/issues/img/issue_template.png diff --git a/doc/user/project/issues/img/issues_main_view.png b/doc/user/project/issues/img/issues_main_view.png Binary files differindex 4faa42e40ee..a929916c682 100644 --- a/doc/user/project/issues/img/issues_main_view.png +++ b/doc/user/project/issues/img/issues_main_view.png diff --git a/doc/user/project/issues/img/issues_main_view_numbered.jpg b/doc/user/project/issues/img/issues_main_view_numbered.jpg Binary files differindex 4b5d7fba459..b4b68476d24 100644 --- a/doc/user/project/issues/img/issues_main_view_numbered.jpg +++ b/doc/user/project/issues/img/issues_main_view_numbered.jpg diff --git a/doc/user/project/issues/img/mention_in_issue.png b/doc/user/project/issues/img/mention_in_issue.png Binary files differindex c762a812138..c762a812138 100755..100644 --- a/doc/user/project/issues/img/mention_in_issue.png +++ b/doc/user/project/issues/img/mention_in_issue.png diff --git a/doc/user/project/issues/img/mention_in_merge_request.png b/doc/user/project/issues/img/mention_in_merge_request.png Binary files differindex 681e086d6e0..681e086d6e0 100755..100644 --- a/doc/user/project/issues/img/mention_in_merge_request.png +++ b/doc/user/project/issues/img/mention_in_merge_request.png diff --git a/doc/user/project/issues/img/merge_request_closes_issue.png b/doc/user/project/issues/img/merge_request_closes_issue.png Binary files differindex 6fd27738843..6fd27738843 100755..100644 --- a/doc/user/project/issues/img/merge_request_closes_issue.png +++ b/doc/user/project/issues/img/merge_request_closes_issue.png diff --git a/doc/user/project/issues/img/new_issue.png b/doc/user/project/issues/img/new_issue.png Binary files differindex e72ac49d6b9..07d65a93070 100755..100644 --- a/doc/user/project/issues/img/new_issue.png +++ b/doc/user/project/issues/img/new_issue.png diff --git a/doc/user/project/issues/img/new_issue_from_issue_board.png b/doc/user/project/issues/img/new_issue_from_issue_board.png Binary files differindex 9c2b3ff50fa..da892eff0a6 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_issue_board.png +++ b/doc/user/project/issues/img/new_issue_from_issue_board.png diff --git a/doc/user/project/issues/img/new_issue_from_open_issue.png b/doc/user/project/issues/img/new_issue_from_open_issue.png Binary files differindex 2aed5372830..c6f3f0617ab 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_open_issue.png +++ b/doc/user/project/issues/img/new_issue_from_open_issue.png diff --git a/doc/user/project/issues/img/new_issue_from_projects_dashboard.png b/doc/user/project/issues/img/new_issue_from_projects_dashboard.png Binary files differindex cddf36b7457..4b9535f6b15 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_projects_dashboard.png +++ b/doc/user/project/issues/img/new_issue_from_projects_dashboard.png diff --git a/doc/user/project/issues/img/new_issue_from_tracker_list.png b/doc/user/project/issues/img/new_issue_from_tracker_list.png Binary files differindex 7e5413f0b7d..66793cb44fa 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_tracker_list.png +++ b/doc/user/project/issues/img/new_issue_from_tracker_list.png diff --git a/doc/user/project/issues/img/project_issues_list_view.png b/doc/user/project/issues/img/project_issues_list_view.png Binary files differindex 2fcc9e8d9da..584a81aab8a 100644 --- a/doc/user/project/issues/img/project_issues_list_view.png +++ b/doc/user/project/issues/img/project_issues_list_view.png diff --git a/doc/user/project/issues/img/sidebar_confidential_issue.png b/doc/user/project/issues/img/sidebar_confidential_issue.png Binary files differindex d99a1ca756e..d99a1ca756e 100755..100644 --- a/doc/user/project/issues/img/sidebar_confidential_issue.png +++ b/doc/user/project/issues/img/sidebar_confidential_issue.png diff --git a/doc/user/project/issues/img/sidebar_move_issue.png b/doc/user/project/issues/img/sidebar_move_issue.png Binary files differindex 111f7861364..1e688cec894 100644 --- a/doc/user/project/issues/img/sidebar_move_issue.png +++ b/doc/user/project/issues/img/sidebar_move_issue.png diff --git a/doc/user/project/issues/img/sidebar_not_confidential_issue.png b/doc/user/project/issues/img/sidebar_not_confidential_issue.png Binary files differindex 2e6cbbc5b3a..2e6cbbc5b3a 100755..100644 --- a/doc/user/project/issues/img/sidebar_not_confidential_issue.png +++ b/doc/user/project/issues/img/sidebar_not_confidential_issue.png diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 0f187946a4a..3e81dcb78c6 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -90,6 +90,10 @@ Learn distinct ways to [close issues](closing_issues.md) in GitLab. Read through the [documentation on moving issues](moving_issues.md). +## Deleting issues + +Read through the [documentation on deleting issues](deleting_issues.md) + ## Create a merge request from an issue Learn more about it on the [GitLab Issues Functionalities documentation](issues_functionalities.md#18-new-merge-request). diff --git a/doc/user/project/issues/issues_functionalities.md b/doc/user/project/issues/issues_functionalities.md index 074b2c19c43..66140f389af 100644 --- a/doc/user/project/issues/issues_functionalities.md +++ b/doc/user/project/issues/issues_functionalities.md @@ -167,6 +167,7 @@ Once you wrote your comment, you can either: #### 18. New Merge Request - Create a new merge request (with a new source branch named after the issue) in one action. -The merge request will automatically close that issue as soon as merged. +The merge request will automatically inherit the milestone and labels of the issue. The merge +request will automatically close that issue as soon as merged. - Optionally, you can just create a [new branch](../repository/web_editor.md#create-a-new-branch-from-an-issue) named after that issue. diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 8ec7adad172..21a2e1213ec 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -20,8 +20,6 @@ Head over a single project and navigate to **Issues > Labels**. The first time you visit this page, you'll notice that there are no labels created yet. - - Creating a new label from scratch is as easy as pressing the **New label** button. From there on you can choose the name, give it an optional description, a color and you are set. @@ -32,21 +30,23 @@ When you are ready press the **Create label** button to create the new label. --- -## Default Labels - -It's possible to populate the labels for your project from a set of predefined labels. - -### Generate GitLab's predefined label set +## Default labels - +The very first time you visit the labels area, it's gonna be empty. In that +case, it's possible to populate the labels for your project from a set of +predefined labels. Click the link to 'Generate a default set of labels' and GitLab will -generate a set of predefined labels for you. There are 8 default generated labels -in total and you can see them in the screenshot below. - - +generate them for you. There are 8 default generated labels in total: ---- +- bug +- confirmed +- critical +- discussion +- documentation +- enhancement +- suggestion +- support ## Labels Overview @@ -102,30 +102,25 @@ If you work on a large or popular project, try subscribing only to the labels that are relevant to you. You’ll notice it’ll be much easier to focus on what’s important. -## Create a new label right from the issue tracker - -> Introduced in GitLab 8.6. +## Create a new label when inside an issue -There are times when you are already in the issue tracker searching for a +There are times when you are already inside an issue searching to assign a label, only to realize it doesn't exist. Instead of going to the **Labels** page and being distracted from your original purpose, you can create new labels on the fly. -Select **Create new** from the labels dropdown list, provide a name, pick a -color and hit **Create**. +Expand the issue sidebar and select **Create new label** from the labels dropdown +list. Provide a name, pick a color and hit **Create**. The new label will be +ready to used right away! -  ## Assigning labels to issues and merge requests There are generally two ways to assign a label to an issue or merge request. -You can assign a label when you first create or edit an issue or merge request. - - - ---- +The first one is to assign a label when you first create or edit an issue or +merge request. The second way is by using the right sidebar when inside an issue or merge request. Expand it and hit **Edit** in the labels area. Start typing the name diff --git a/doc/user/project/merge_requests/cherry_pick_changes.md b/doc/user/project/merge_requests/cherry_pick_changes.md index 64b94d81024..22ef11e4049 100644 --- a/doc/user/project/merge_requests/cherry_pick_changes.md +++ b/doc/user/project/merge_requests/cherry_pick_changes.md @@ -2,24 +2,19 @@ > [Introduced][ce-3514] in GitLab 8.7. ---- - GitLab implements Git's powerful feature to [cherry-pick any commit][git-cherry-pick] -with introducing a **Cherry-pick** button in Merge Requests and commit details. +with introducing a **Cherry-pick** button in merge requests and commit details. -## Cherry-picking a Merge Request +## Cherry-picking a merge request -After the Merge Request has been merged, a **Cherry-pick** button will be available -to cherry-pick the changes introduced by that Merge Request: +After the merge request has been merged, a **Cherry-pick** button will be available +to cherry-pick the changes introduced by that merge request.  ---- - -You can cherry-pick the changes directly into the selected branch or you can opt to -create a new Merge Request with the cherry-pick changes: - - +After you click that button, a modal will appear where you can choose to +cherry-pick the changes directly into the selected branch or you can opt to +create a new merge request with the cherry-pick changes ## Cherry-picking a Commit @@ -27,15 +22,9 @@ You can cherry-pick a Commit from the Commit details page:  ---- - -Similar to cherry-picking a Merge Request, you can opt to cherry-pick the changes -directly into the target branch or create a new Merge Request to cherry-pick the -changes: - - - ---- +Similar to cherry-picking a merge request, you can opt to cherry-pick the changes +directly into the target branch or create a new merge request to cherry-pick the +changes. Please note that when cherry-picking merge commits, the mainline will always be the first parent. If you want to use a different mainline then you need to do that diff --git a/doc/user/project/merge_requests/fast_forward_merge.md b/doc/user/project/merge_requests/fast_forward_merge.md new file mode 100644 index 00000000000..085170d9f03 --- /dev/null +++ b/doc/user/project/merge_requests/fast_forward_merge.md @@ -0,0 +1,35 @@ +# Fast-forward merge requests + +Retain a linear Git history and a way to accept merge requests without +creating merge commits. + +## Overview + +When the fast-forward merge ([`--ff-only`][ffonly]) setting is enabled, no merge +commits will be created and all merges are fast-forwarded, which means that +merging is only allowed if the branch could be fast-forwarded. + +When a fast-forward merge is not possible, the user must rebase the branch manually. + +## Use cases + +Sometimes, a workflow policy might mandate a clean commit history without +merge commits. In such cases, the fast-forward merge is the perfect candidate. + +## Enabling fast-forward merges + +1. Navigate to your project's **Settings** and search for the 'Merge method' +1. Select the **Fast-forward merge** option +1. Hit **Save changes** for the changes to take effect + +Now, when you visit the merge request page, you will be able to accept it +**only if a fast-forward merge is possible**. + + + +If the target branch is ahead of the source branch, you need to rebase the +source branch locally before you will be able to do a fast-forward merge. + + + +[ffonly]: https://git-scm.com/docs/git-merge#git-merge---ff-only diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png b/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png Binary files differindex 5ab094ab367..7dc344f8cf6 100644 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png +++ b/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_commit_modal.png b/doc/user/project/merge_requests/img/cherry_pick_changes_commit_modal.png Binary files differdeleted file mode 100644 index 42dcb9203ec..00000000000 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_commit_modal.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png b/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png Binary files differindex 71227747182..811b0998f85 100644 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png +++ b/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_mr_modal.png b/doc/user/project/merge_requests/img/cherry_pick_changes_mr_modal.png Binary files differdeleted file mode 100644 index 604eb22f51c..00000000000 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_mr_modal.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/commit_compare.png b/doc/user/project/merge_requests/img/commit_compare.png Binary files differdeleted file mode 100644 index e612a39716e..00000000000 --- a/doc/user/project/merge_requests/img/commit_compare.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/ff_merge_mr.png b/doc/user/project/merge_requests/img/ff_merge_mr.png Binary files differnew file mode 100644 index 00000000000..241cc990343 --- /dev/null +++ b/doc/user/project/merge_requests/img/ff_merge_mr.png diff --git a/doc/user/project/merge_requests/img/ff_merge_rebase_locally.png b/doc/user/project/merge_requests/img/ff_merge_rebase_locally.png Binary files differnew file mode 100644 index 00000000000..fb412296efc --- /dev/null +++ b/doc/user/project/merge_requests/img/ff_merge_rebase_locally.png diff --git a/doc/user/project/merge_requests/img/group_merge_requests_list_view.png b/doc/user/project/merge_requests/img/group_merge_requests_list_view.png Binary files differindex 02a88d0112f..7d0756505db 100644 --- a/doc/user/project/merge_requests/img/group_merge_requests_list_view.png +++ b/doc/user/project/merge_requests/img/group_merge_requests_list_view.png diff --git a/doc/user/project/merge_requests/img/merge_request.png b/doc/user/project/merge_requests/img/merge_request.png Binary files differnew file mode 100644 index 00000000000..f9ca6348953 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_request.png diff --git a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png b/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png Binary files differindex 33f5a4a7a02..d7f0535d3c5 100644 --- a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png +++ b/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png diff --git a/doc/user/project/merge_requests/img/revert_changes_commit_modal.png b/doc/user/project/merge_requests/img/revert_changes_commit_modal.png Binary files differdeleted file mode 100644 index ef7b6dae553..00000000000 --- a/doc/user/project/merge_requests/img/revert_changes_commit_modal.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/revert_changes_mr_modal.png b/doc/user/project/merge_requests/img/revert_changes_mr_modal.png Binary files differdeleted file mode 100644 index f6540c9dd33..00000000000 --- a/doc/user/project/merge_requests/img/revert_changes_mr_modal.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/versions.png b/doc/user/project/merge_requests/img/versions.png Binary files differindex 33c58d2abff..3883fb4bc1c 100644 --- a/doc/user/project/merge_requests/img/versions.png +++ b/doc/user/project/merge_requests/img/versions.png diff --git a/doc/user/project/merge_requests/img/versions_compare.png b/doc/user/project/merge_requests/img/versions_compare.png Binary files differindex db978ea7b1d..f5bd85dc7c1 100644 --- a/doc/user/project/merge_requests/img/versions_compare.png +++ b/doc/user/project/merge_requests/img/versions_compare.png diff --git a/doc/user/project/merge_requests/img/versions_dropdown.png b/doc/user/project/merge_requests/img/versions_dropdown.png Binary files differindex 889a2d93e6c..cc70a5bf14b 100644 --- a/doc/user/project/merge_requests/img/versions_dropdown.png +++ b/doc/user/project/merge_requests/img/versions_dropdown.png diff --git a/doc/user/project/merge_requests/img/wip_blocked_accept_button.png b/doc/user/project/merge_requests/img/wip_blocked_accept_button.png Binary files differindex 047b0b4620f..0c492aca363 100644 --- a/doc/user/project/merge_requests/img/wip_blocked_accept_button.png +++ b/doc/user/project/merge_requests/img/wip_blocked_accept_button.png diff --git a/doc/user/project/merge_requests/img/wip_mark_as_wip.png b/doc/user/project/merge_requests/img/wip_mark_as_wip.png Binary files differindex 8bd206bc24a..e405879b28a 100644 --- a/doc/user/project/merge_requests/img/wip_mark_as_wip.png +++ b/doc/user/project/merge_requests/img/wip_mark_as_wip.png diff --git a/doc/user/project/merge_requests/img/wip_unmark_as_wip.png b/doc/user/project/merge_requests/img/wip_unmark_as_wip.png Binary files differindex c0bfa6a35a2..d7f8c419945 100644 --- a/doc/user/project/merge_requests/img/wip_unmark_as_wip.png +++ b/doc/user/project/merge_requests/img/wip_unmark_as_wip.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 26c6277d33a..6289fcf3c2b 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -3,6 +3,8 @@ Merge requests allow you to exchange changes you made to source code and collaborate with other people on the same project. + + ## Overview A Merge Request (**MR**) is the basis of GitLab as a code collaboration @@ -23,12 +25,14 @@ With GitLab merge requests, you can: - Organize your issues and merge requests consistently throughout the project with [labels](../../project/labels.md) - Add a time estimation and the time spent with that merge request with [Time Tracking](../../../workflow/time_tracking.html#time-tracking) - [Resolve merge conflicts from the UI](#resolve-conflicts) +- Enable [fast-forward merge requests](#fast-forward-merge-requests) +- Enable [semi-linear history merge requests](#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch + With **[GitLab Enterprise Edition][ee]**, you can also: - View the deployment process across projects with [Multi-Project Pipeline Graphs](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html#multi-project-pipeline-graphs) (available only in GitLab Enterprise Edition Premium) - Request [approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your managers (available in GitLab Enterprise Edition Starter) -- Enable [fast-forward merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/fast_forward_merge.html) (available in GitLab Enterprise Edition Starter) - [Squash and merge](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html) for a cleaner commit history (available in GitLab Enterprise Edition Starter) - Enable [semi-linear history merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/index.html#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch (available in GitLab Enterprise Edition Starter) - Analise the impact of your changes with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) (available in GitLab Enterprise Edition Starter) @@ -89,6 +93,22 @@ in a merged merge requests or a commit. [Learn more about cherry-picking changes.](cherry_pick_changes.md) +## Semi-linear history merge requests + +A merge commit is created for every merge, but the branch is only merged if +a fast-forward merge is possible. This ensures that if the merge request build +succeeded, the target branch build will also succeed after merging. + +Navigate to a project's settings, select the **Merge commit with semi-linear +history** option under **Merge Requests: Merge method** and save your changes. + +## Fast-forward merge requests + +If you prefer a linear Git history and a way to accept merge requests without +creating merge commits, you can configure this on a per-project basis. + +[Read more about fast-forward merge requests.](fast_forward_merge.md) + ## Merge when pipeline succeeds When reviewing a merge request that looks ready to merge but still has one or @@ -254,4 +274,4 @@ git checkout origin/merge-requests/1 ``` [protected branches]: ../protected_branches.md -[ee]: https://about.gitlab.com/gitlab-ee/ "GitLab Enterprise Edition"
\ No newline at end of file +[ee]: https://about.gitlab.com/gitlab-ee/ "GitLab Enterprise Edition" diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index 5ead9f4177f..8cf8a59dbfe 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -2,51 +2,39 @@ > [Introduced][ce-1990] in GitLab 8.5. ---- - GitLab implements Git's powerful feature to [revert any commit][git-revert] -with introducing a **Revert** button in Merge Requests and commit details. +with introducing a **Revert** button in merge requests and commit details. ## Reverting a Merge Request -_**Note:** The **Revert** button will only be available for Merge Requests -created since GitLab 8.5. However, you can still revert a Merge Request -by reverting the merge commit from the list of Commits page._ +NOTE: **Note:** +The **Revert** button will only be available for merge requests +created since GitLab 8.5. However, you can still revert a merge request +by reverting the merge commit from the list of Commits page. After the Merge Request has been merged, a **Revert** button will be available -to revert the changes introduced by that Merge Request: - - - ---- - -You can revert the changes directly into the selected branch or you can opt to -create a new Merge Request with the revert changes: +to revert the changes introduced by that merge request. - + ---- +After you click that button, a modal will appear where you can choose to +revert the changes directly into the selected branch or you can opt to +create a new merge request with the revert changes. -After the Merge Request has been reverted, the **Revert** button will not be +After the merge request has been reverted, the **Revert** button will not be available anymore. ## Reverting a Commit You can revert a Commit from the Commit details page: - - ---- - -Similar to reverting a Merge Request, you can opt to revert the changes -directly into the target branch or create a new Merge Request to revert the -changes: - - + ---- +Similar to reverting a merge request, you can opt to revert the changes +directly into the target branch or create a new merge request to revert the +changes. -After the Commit has been reverted, the **Revert** button will not be available +After the commit has been reverted, the **Revert** button will not be available anymore. Please note that when reverting merge commits, the mainline will always be the diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 46fa4378fe7..453e10184f0 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -62,7 +62,7 @@ which is highly recommendable and much faster than hardcoding. If you set up a GitLab Pages project on GitLab.com, it will automatically be accessible under a -[subdomain of `namespace.pages.io`](https://docs.gitlab.com/ce/user/project/pages/). +[subdomain of `namespace.pages.io`](introduction.md#gitlab-pages-on-gitlab-com). The `namespace` is defined by your username on GitLab.com, or the group name you created this project under. @@ -73,6 +73,8 @@ Pages wildcard domain. This guide is valid for any GitLab instance, you just need to replace Pages wildcard domain on GitLab.com (`*.gitlab.io`) with your own. +Learn more about [namespaces](../../group/index.md#namespaces). + ### Practical examples #### Project Websites diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index 53fd1786cfa..0096f8507d2 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -1,9 +1,14 @@ +--- +last_updated: 2017-09-28 +--- + # GitLab Pages from A to Z: Part 3 -> **Article [Type](../../../development/writing_documentation.html#types-of-technical-articles)**: user guide || +> **[Article Type](../../../development/writing_documentation.md#types-of-technical-articles)**: user guide || > **Level**: beginner || > **Author**: [Marcia Ramos](https://gitlab.com/marcia) || -> **Publication date:** 2017/02/22 +> **Publication date:** 2017-02-22 || +> **Last updated**: 2017-09-28 - [Part 1: Static sites and GitLab Pages domains](getting_started_part_one.md) - [Part 2: Quick start guide - Setting up GitLab Pages](getting_started_part_two.md) @@ -16,6 +21,21 @@ As described in the previous part of this series, setting up GitLab Pages with c These steps assume you've already [set your site up](getting_started_part_two.md) and and it's served under the default Pages domain `namespace.gitlab.io`, or `namespace.gitlab.io/project-name`. +### Adding your custom domain to GitLab Pages + +To use one or more custom domain with your Pages site, there are two things +you should consider first, which we'll cover in this guide: + +1. Either if you're adding a **root domain** or a **subdomain**, for which +you'll need to set up [DNS records](#dns-records) +1. Whether you want to add an [SSL/TLS certificate](#ssl-tls-certificates) or not + +To finish the association, you need to [add your domain to your project's Pages settings](#add-your-custom-domain-to-gitlab-pages-settings). + +Let's start from the beginning with [DNS records](#dns-records). +If you already know how they work and want to skip the introduction to DNS, +you may be interested in skipping it until the [TL;DR](#tl-dr) section below. + ### DNS Records A Domain Name System (DNS) web service routes visitors to websites @@ -99,6 +119,29 @@ domain. E.g., **do not** point your `subdomain.domain.com` to `namespace.gitlab.io.` or `namespace.gitlab.io/`. > - GitLab Pages IP on GitLab.com [has been changed](https://about.gitlab.com/2017/03/06/we-are-changing-the-ip-of-gitlab-pages-on-gitlab-com/) from `104.208.235.32` to `52.167.214.135`. +### Add your custom domain to GitLab Pages settings + +Once you've set the DNS record, you'll need navigate to your project's +**Setting > Pages** and click **+ New domain** to add your custom domain to +GitLab Pages. You can choose whether to add an [SSL/TLS certificate](#ssl-tls-certificates) +to make your website accessible under HTTPS or leave it blank. If don't add a certificate, +your site will be accessible only via HTTP: + + + +You can add more than one alias (custom domains and subdomains) to the same project. +An alias can be understood as having many doors leading to the same room. + +All the aliases you've set to your site will be listed on **Setting > Pages**. +From that page, you can view, add, and remove them. + +Note that [DNS propagation may take some time (up to 24h)](http://www.inmotionhosting.com/support/domain-names/dns-nameserver-changes/domain-names-dns-changes), +although it's usually a matter of minutes to complete. Until it does, visit attempts +to your domain will respond with a 404. + +Read through the [general documentation on GitLab Pages](introduction.md#add-a-custom-domain-to-your-pages-website) to learn more about adding +custom domains to GitLab Pages sites. + ### SSL/TLS Certificates Every GitLab Pages project on GitLab.com will be available under diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 9ecf7a3a8e7..4fcdfa7b281 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -28,7 +28,8 @@ In general there are two types of pages one might create: - Pages per project (`username.example.io/projectname` or `groupname.example.io/projectname`) In GitLab, usernames and groupnames are unique and we often refer to them -as namespaces. There can be only one namespace in a GitLab instance. Below you +as [namespaces](../../group/index.md#namespaces). There can be only one namespace +in a GitLab instance. Below you can see the connection between the type of GitLab Pages, what the project name that is created on GitLab looks like and the website URL it will be ultimately be served on. @@ -98,6 +99,9 @@ The steps to create a project page for a user or a group are identical: A user's project will be served under `http(s)://username.example.io/projectname` whereas a group's project under `http(s)://groupname.example.io/projectname`. +For practical examples for group and project Pages, read through the guide +[GitLab Pages from A to Z: Part 1 - Static sites and GitLab Pages domains](getting_started_part_one.md#practical-examples). + ## Quick Start Read through [GitLab Pages Quick Start Guide][pages-quick] or watch the video tutorial on @@ -111,6 +115,9 @@ The key thing about GitLab Pages is the `.gitlab-ci.yml` file, something that gives you absolute control over the build process. You can actually watch your website being built live by following the CI job traces. +For a simplified user guide on setting up GitLab CI/CD for Pages, read through +the article [GitLab Pages from A to Z: Part 4 - Creating and Tweaking `.gitlab-ci.yml` for GitLab Pages](getting_started_part_four.md#creating-and-tweaking-gitlab-ci-yml-for-gitlab-pages) + > **Note:** > Before reading this section, make sure you familiarize yourself with GitLab CI > and the specific syntax of[`.gitlab-ci.yml`][yaml] by @@ -311,6 +318,9 @@ Visit the GitLab Pages group for a full list of example projects: ### Add a custom domain to your Pages website +For a complete guide on Pages domains, read through the article +[GitLab Pages from A to Z: Part 3 - Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](getting_started_part_three.md#setting-up-custom-domains-dns-records-and-ssl-tls-certificates) + If this setting is enabled by your GitLab administrator, you should be able to see the **New Domain** button when visiting your project's settings through the gear icon in the top right and then navigating to **Pages**. @@ -349,6 +359,9 @@ private key when adding a new domain.  +For a complete guide on Pages domains, read through the article +[GitLab Pages from A to Z: Part 3 - Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](getting_started_part_three.md#setting-up-custom-domains-dns-records-and-ssl-tls-certificates) + ### Custom error codes pages You can provide your own 403 and 404 error pages by creating the `403.html` and @@ -387,6 +400,8 @@ If you are using GitLab.com to host your website, then: The rest of the guide still applies. +See also: [GitLab Pages from A to Z: Part 1 - Static sites and GitLab Pages domains](getting_started_part_one.md#gitlab-pages-domain). + ## Limitations When using Pages under the general domain of a GitLab instance (`*.example.io`), diff --git a/doc/user/project/pipelines/img/job_artifacts_browser.png b/doc/user/project/pipelines/img/job_artifacts_browser.png Binary files differindex 145fe156bbb..d3d8de5ac60 100644 --- a/doc/user/project/pipelines/img/job_artifacts_browser.png +++ b/doc/user/project/pipelines/img/job_artifacts_browser.png diff --git a/doc/user/project/pipelines/job_artifacts.md b/doc/user/project/pipelines/job_artifacts.md index 4e93e680fd2..9ef6f9185c9 100644 --- a/doc/user/project/pipelines/job_artifacts.md +++ b/doc/user/project/pipelines/job_artifacts.md @@ -50,6 +50,10 @@ For more examples on artifacts, follow the [artifacts reference in With GitLab 9.2, PDFs, images, videos and other formats can be previewed directly in the job artifacts browser without the need to download them. +>**Note:** +With [GitLab 10.1][ce-14399], HTML files in a public project can be previewed +directly in a new tab without the need to download them. + After a job finishes, if you visit the job's specific page, there are three buttons. You can download the artifacts archive or browse its contents, whereas the **Keep** button appears only if you have set an [expiry date] to the @@ -64,7 +68,8 @@ archive. If your artifacts contained directories, then you are also able to browse inside them. Below you can see how browsing looks like. In this case we have browsed inside -the archive and at this point there is one directory and one HTML file. +the archive and at this point there is one directory, a couple files, and +one HTML file that you can view directly online (opens in a new tab).  @@ -158,3 +163,4 @@ information in the UI. [expiry date]: ../../../ci/yaml/README.md#artifacts-expire_in +[ce-14399]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14399 diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index dbc1305101f..56f58fd755a 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -1,7 +1,7 @@ # Pipelines settings To reach the pipelines settings navigate to your project's -**Settings ➔ Pipelines**. +**Settings ➔ CI/CD**. The following settings can be configured per project. diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index 1948627ee79..e1d3aebb8b3 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -1,5 +1,32 @@ # Branches +Read through GiLab's branching documentation: + +- [Create a branch](../web_editor.md#create-a-new-branch) +- [Default branch](#default-branch) +- [Protected branches](../../protected_branches.md#protected-branches) +- [Delete merged branches](#delete-merged-branches) + +See also: + +- [GitLab Flow](../../../../university/training/gitlab_flow.md#gitlab-flow): use the best of GitLab for your branching strategies +- [Getting started with Git](../../../../topics/git/index.md) and GitLab + +## Default branch + +When you create a new [project](../../index.md), GitLab sets `master` as the default +branch for your project. You can choose another branch to be your project's +default under your project's **Settings > General**. + +The default branch is the branched affected by the +[issue closing pattern](../../issues/automatic_issue_closing.md), +which means that _an issue will be closed when a merge request is merged to +the **default branch**_. + +The default branch is also protected against accidental deletion. Read through +the documentation on [protected branches](../../protected_branches.md#protected-branches) +to learn more. + ## Delete merged branches > [Introduced][ce-6449] in GitLab 8.14. diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index 20aadb8f7ff..6b9976d133c 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -26,7 +26,7 @@ to be uploaded to GitLab. For a signature to be verified three conditions need to be met: 1. The public key needs to be added your GitLab account -1. One of the emails in the GPG key matches your **primary** email +1. One of the emails in the GPG key matches a **verified** email address you use in GitLab 1. The committer's email matches the verified email from the gpg key ## Generating a GPG key @@ -94,7 +94,7 @@ started: ``` 1. Enter you real name, the email address to be associated with this key (should - match the primary email address you use in GitLab) and an optional comment + match a verified email address you use in GitLab) and an optional comment (press <kbd>Enter</kbd> to skip): ``` @@ -113,25 +113,25 @@ started: 1. Use the following command to list the private GPG key you just created: ``` - gpg --list-secret-keys mr@robot.sh + gpg --list-secret-keys --keyid-format LONG mr@robot.sh ``` Replace `mr@robot.sh` with the email address you entered above. 1. Copy the GPG key ID that starts with `sec`. In the following example, that's - `0x30F2B65B9246B6CA`: + `30F2B65B9246B6CA`: ``` - sec rsa4096/0x30F2B65B9246B6CA 2017-08-18 [SC] + sec rsa4096/30F2B65B9246B6CA 2017-08-18 [SC] D5E4F29F3275DC0CDA8FFC8730F2B65B9246B6CA uid [ultimate] Mr. Robot <mr@robot.sh> - ssb rsa4096/0xB7ABC0813E4028C0 2017-08-18 [E] + ssb rsa4096/B7ABC0813E4028C0 2017-08-18 [E] ``` 1. Export the public key of that ID (replace your key ID from the previous step): ``` - gpg --armor --export 0x30F2B65B9246B6CA + gpg --armor --export 30F2B65B9246B6CA ``` 1. Finally, copy the public key and [add it in your profile settings](#adding-a-gpg-key-to-your-account) @@ -167,28 +167,28 @@ key to use. 1. Use the following command to list the private GPG key you just created: ``` - gpg --list-secret-keys mr@robot.sh + gpg --list-secret-keys --keyid-format LONG mr@robot.sh ``` Replace `mr@robot.sh` with the email address you entered above. 1. Copy the GPG key ID that starts with `sec`. In the following example, that's - `0x30F2B65B9246B6CA`: + `30F2B65B9246B6CA`: ``` - sec rsa4096/0x30F2B65B9246B6CA 2017-08-18 [SC] + sec rsa4096/30F2B65B9246B6CA 2017-08-18 [SC] D5E4F29F3275DC0CDA8FFC8730F2B65B9246B6CA uid [ultimate] Mr. Robot <mr@robot.sh> - ssb rsa4096/0xB7ABC0813E4028C0 2017-08-18 [E] + ssb rsa4096/B7ABC0813E4028C0 2017-08-18 [E] ``` 1. Tell Git to use that key to sign the commits: ``` - git config --global user.signingkey 0x30F2B65B9246B6CA + git config --global user.signingkey 30F2B65B9246B6CA ``` - Replace `0x30F2B65B9246B6CA` with your GPG key ID. + Replace `30F2B65B9246B6CA` with your GPG key ID. ## Signing commits diff --git a/doc/user/project/repository/img/compare_branches.png b/doc/user/project/repository/img/compare_branches.png Binary files differindex 353bd72ef4e..d7ab587f030 100755..100644 --- a/doc/user/project/repository/img/compare_branches.png +++ b/doc/user/project/repository/img/compare_branches.png diff --git a/doc/user/project/repository/img/contributors_graph.png b/doc/user/project/repository/img/contributors_graph.png Binary files differindex c31da7aa1ff..c31da7aa1ff 100755..100644 --- a/doc/user/project/repository/img/contributors_graph.png +++ b/doc/user/project/repository/img/contributors_graph.png diff --git a/doc/user/project/repository/img/repo_graph.png b/doc/user/project/repository/img/repo_graph.png Binary files differindex 28da8ad9589..28da8ad9589 100755..100644 --- a/doc/user/project/repository/img/repo_graph.png +++ b/doc/user/project/repository/img/repo_graph.png diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 235af83353d..9501db88f57 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -55,7 +55,7 @@ Use GitLab's [file finder](../../../workflow/file_finder.md) to search for files ## Branches -When you submit changes in a new branch, you create a new version +When you submit changes in a new [branch](branches/index.md), you create a new version of that project's file tree. Your branch contains all the changes you are presenting, which are detected by Git line by line. @@ -70,8 +70,9 @@ With [GitLab Enterprise Edition](https://about.gitlab.com/gitlab-ee/) subscriptions, you can also request [approval](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#merge-request-approvals) from your managers. -To create, delete, and branches via GitLab's UI: +To create, delete, and [branches](branches/index.md) via GitLab's UI: +- [Default branches](branches/index.md#default-branch) - [Create a branch](web_editor.md#create-a-new-branch) - [Protected branches](../protected_branches.md#protected-branches) - [Delete merged branches](branches/index.md#delete-merged-branches) diff --git a/doc/user/project/settings/img/general_settings.png b/doc/user/project/settings/img/general_settings.png Binary files differnew file mode 100644 index 00000000000..96f5b84871f --- /dev/null +++ b/doc/user/project/settings/img/general_settings.png diff --git a/doc/user/project/settings/img/merge_requests_settings.png b/doc/user/project/settings/img/merge_requests_settings.png Binary files differnew file mode 100644 index 00000000000..b1f2dfa7376 --- /dev/null +++ b/doc/user/project/settings/img/merge_requests_settings.png diff --git a/doc/user/project/settings/img/sharing_and_permissions_settings.png b/doc/user/project/settings/img/sharing_and_permissions_settings.png Binary files differnew file mode 100644 index 00000000000..0f9cf9512af --- /dev/null +++ b/doc/user/project/settings/img/sharing_and_permissions_settings.png diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md new file mode 100644 index 00000000000..a234a647b77 --- /dev/null +++ b/doc/user/project/settings/index.md @@ -0,0 +1,52 @@ +# Project settings + +You can adjust your [project](../index.md) settings by navigating +to your project's homepage and clicking **Settings**. + +## General settings + +Adjust your project's path and name, description, avatar, [default branch](../repository/branches/index.md#default-branch), and tags: + + + +### Sharing and permissions + +Set up your project's access, [visibility](../../../public_access/public_access.md), and enable [Container Registry](../container_registry.md) for your projects: + + + +### Issue settings + +Add an [issue description template](../description_templates.md#description-templates) to your project, so that every new issue will start with a custom template. + +### Merge request settings + +Set up your project's merge request settings: + +- Set up the merge request method (merge commit, [fast-forward merge](../merge_requests/fast_forward_merge.html)). +- Merge request [description templates](../description_templates.md#description-templates). +- Enable [merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#merge-request-approvals), _available in [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/)_. +- Enable [merge only of pipeline succeeds](../merge_requests/merge_when_pipeline_succeeds.md). +- Enable [merge only when all discussions are resolved](../../discussions/index.md#only-allow-merge-requests-to-be-merged-if-all-discussions-are-resolved). + + + +### Service Desk + +Enable [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html) for your project to offer customer support. Service Desk is available in [GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/). + +### Export project + +Learn how to [export a project](import_export.md#importing-the-project) in GitLab. + +### Advanced settings + +Here you can run housekeeping, archive, rename, transfer, or remove a project. + +#### Archiving a project + +>**Note:** Only Project Owners and Admin users have the permission to archive a project + +It's possible to mark a project as archived via the Project Settings. An archived project will be hidden by default in the project listings. + +An archived project can be fully restored and will therefore retain it's repository and all associated resources whilst in an archived state. diff --git a/doc/user/search/img/issues_any_assignee.png b/doc/user/search/img/issues_any_assignee.png Binary files differindex 2f902bcc66c..2f902bcc66c 100755..100644 --- a/doc/user/search/img/issues_any_assignee.png +++ b/doc/user/search/img/issues_any_assignee.png diff --git a/doc/user/search/img/issues_assigned_to_you.png b/doc/user/search/img/issues_assigned_to_you.png Binary files differindex 36c670eedd5..36c670eedd5 100755..100644 --- a/doc/user/search/img/issues_assigned_to_you.png +++ b/doc/user/search/img/issues_assigned_to_you.png diff --git a/doc/user/search/img/issues_author.png b/doc/user/search/img/issues_author.png Binary files differindex 792f9746db6..792f9746db6 100755..100644 --- a/doc/user/search/img/issues_author.png +++ b/doc/user/search/img/issues_author.png diff --git a/doc/user/search/img/issues_mrs_shortcut.png b/doc/user/search/img/issues_mrs_shortcut.png Binary files differindex 6380b337b54..6380b337b54 100755..100644 --- a/doc/user/search/img/issues_mrs_shortcut.png +++ b/doc/user/search/img/issues_mrs_shortcut.png diff --git a/doc/user/search/img/left_menu_bar.png b/doc/user/search/img/left_menu_bar.png Binary files differindex d68a71cba8e..d68a71cba8e 100755..100644 --- a/doc/user/search/img/left_menu_bar.png +++ b/doc/user/search/img/left_menu_bar.png diff --git a/doc/user/search/img/project_search.png b/doc/user/search/img/project_search.png Binary files differindex 3150b40de29..3150b40de29 100755..100644 --- a/doc/user/search/img/project_search.png +++ b/doc/user/search/img/project_search.png diff --git a/doc/user/search/img/search_issues_board.png b/doc/user/search/img/search_issues_board.png Binary files differindex 84048ae6a02..84048ae6a02 100755..100644 --- a/doc/user/search/img/search_issues_board.png +++ b/doc/user/search/img/search_issues_board.png diff --git a/doc/user/search/img/sort_projects.png b/doc/user/search/img/sort_projects.png Binary files differindex 9bf2770b299..9bf2770b299 100755..100644 --- a/doc/user/search/img/sort_projects.png +++ b/doc/user/search/img/sort_projects.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 21e96d8b11c..2b23c494dc4 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -31,8 +31,8 @@ on the search field on the top-right of your screen: If you want to search for issues present in a specific project, navigate to a project's **Issues** tab, and click on the field **Search or filter results...**. It will -display a dropdown menu, from which you can add filters per author, assignee, milestone, label, -and weight. When done, press **Enter** on your keyboard to filter the issues. +display a dropdown menu, from which you can add filters per author, assignee, milestone, +label, weight, and 'my-reaction' (based on your emoji votes). When done, press **Enter** on your keyboard to filter the issues.  @@ -63,8 +63,6 @@ the same way as you do for projects.  The same process is valid for merge requests. Navigate to your project's **Merge Requests** tab. -The search and filter UI currently uses dropdowns. In a future release, the same -dynamic UI as above will be carried over here. ## Search history diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 673e08287a3..6b2aba47f54 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -36,6 +36,7 @@ - [Revert changes in the UI](../user/project/merge_requests/revert_changes.md) - [Merge requests versions](../user/project/merge_requests/versions.md) - ["Work In Progress" merge requests](../user/project/merge_requests/work_in_progress_merge_requests.md) + - [Fast-forward merge requests](../user/project/merge_requests/fast_forward_merge.md) - [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) - [Importing from SVN, GitHub, Bitbucket, etc](importing/README.md) - [Todos](todos.md) |
