diff options
Diffstat (limited to 'doc')
64 files changed, 3148 insertions, 618 deletions
diff --git a/doc/README.md b/doc/README.md index fb7a23e2750..05fa444657c 100644 --- a/doc/README.md +++ b/doc/README.md @@ -77,6 +77,7 @@ Manage your [repositories](user/project/repository/index.md) from the UI (user i - [Discussions](user/discussions/index.md): Threads, comments, and resolvable discussions in issues, commits, and merge requests. - [Issues](user/project/issues/index.md) - [Project issue Board](user/project/issue_board.md) +- [Group Issue Board](user/project/issue_board.md#group-issue-board) - [Issues and merge requests templates](user/project/description_templates.md): Create templates for submitting new issues and merge requests. - [Labels](user/project/labels.md): Categorize your issues or merge requests based on descriptive titles. - [Merge Requests](user/project/merge_requests/index.md) diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md index b6320aba83e..d18dddf09c0 100644 --- a/doc/administration/monitoring/index.md +++ b/doc/administration/monitoring/index.md @@ -7,3 +7,4 @@ Explore our features to monitor your GitLab instance: - [GitHub imports](github_imports.md): Monitor the health and progress of GitLab's GitHub importer with various Prometheus metrics. - [Monitoring uptime](../../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. - [IP whitelists](ip_whitelist.md): Configure GitLab for monitoring endpoints that provide health check information when probed. +- [nginx_status](https://docs.gitlab.com/omnibus/settings/nginx.html#enabling-disabling-nginx_status): Monitor your Nginx server status diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index d1ed152b58c..51c62742d01 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -78,34 +78,45 @@ Example output: ## Uploaded Files Integrity -The uploads check Rake task will loop through all uploads in the database -and run two checks to determine the integrity of each file: +Various types of file can be uploaded to a GitLab installation by users. +Checksums are generated and stored in the database upon upload, and integrity +checks using those checksums can be run. These checks also detect missing files. -1. Check if the file exist on the file system. -1. Check if the checksum of the file on the file system matches the checksum in the database. +Currently, integrity checks are supported for the following types of file: + +* CI artifacts +* LFS objects +* User uploads **Omnibus Installation** ``` +sudo gitlab-rake gitlab:artifacts:check +sudo gitlab-rake gitlab:lfs:check sudo gitlab-rake gitlab:uploads:check ``` **Source Installation** ```bash +sudo -u git -H bundle exec rake gitlab:artifacts:check RAILS_ENV=production +sudo -u git -H bundle exec rake gitlab:lfs:check RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:uploads:check RAILS_ENV=production ``` -This task also accepts some environment variables which you can use to override +These tasks also accept some environment variables which you can use to override certain values: -Variable | Type | Description --------- | ---- | ----------- -`BATCH` | integer | Specifies the size of the batch. Defaults to 200. -`ID_FROM` | integer | Specifies the ID to start from, inclusive of the value. -`ID_TO` | integer | Specifies the ID value to end at, inclusive of the value. +Variable | Type | Description +--------- | ------- | ----------- +`BATCH` | integer | Specifies the size of the batch. Defaults to 200. +`ID_FROM` | integer | Specifies the ID to start from, inclusive of the value. +`ID_TO` | integer | Specifies the ID value to end at, inclusive of the value. +`VERBOSE` | boolean | Causes failures to be listed individually, rather than being summarized. ```bash +sudo gitlab-rake gitlab:artifacts:check BATCH=100 ID_FROM=50 ID_TO=250 +sudo gitlab-rake gitlab:lfs:check BATCH=100 ID_FROM=50 ID_TO=250 sudo gitlab-rake gitlab:uploads:check BATCH=100 ID_FROM=50 ID_TO=250 ``` diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index ecf92c379fd..2b4252a7b1d 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -240,3 +240,31 @@ sudo gitlab-rake gitlab:tcp_check[example.com,80] cd /home/git/gitlab sudo -u git -H bundle exec rake gitlab:tcp_check[example.com,80] RAILS_ENV=production ``` + +## Clear exclusive lease (DANGER) + +GitLab uses a shared lock mechanism: `ExclusiveLease` to prevent simultaneous operations +in a shared resource. An example is running periodic garbage collection on repositories. + +In very specific situations, a operation locked by an Exclusive Lease can fail without +releasing the lock. If you can't wait for it to expire, you can run this task to manually +clear it. + +To clear all exclusive leases: + +DANGER: **DANGER**: +Don't run it while GitLab or Sidekiq is running + +```bash +sudo gitlab-rake gitlab:exclusive_lease:clear +``` + +To specify a lease `type` or lease `type + id`, specify a scope: + +```bash +# to clear all leases for repository garbage collection: +sudo gitlab-rake gitlab:exclusive_lease:clear[project_housekeeping:*] + +# to clear a lease for repository garbage collection in a specific project: (id=4) +sudo gitlab-rake gitlab:exclusive_lease:clear[project_housekeeping:4] +``` diff --git a/doc/api/README.md b/doc/api/README.md index b193ef4ab7f..ae4481b400e 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -24,9 +24,11 @@ following locations: - [GitLab CI Config templates](templates/gitlab_ci_ymls.md) - [Groups](groups.md) - [Group Access Requests](access_requests.md) +- [Group Badges](group_badges.md) - [Group Members](members.md) - [Issues](issues.md) - [Issue Boards](boards.md) +- [Group Issue Boards](group_boards.md) - [Jobs](jobs.md) - [Keys](keys.md) - [Labels](labels.md) @@ -35,6 +37,7 @@ following locations: - [Group milestones](group_milestones.md) - [Namespaces](namespaces.md) - [Notes](notes.md) (comments) +- [Discussions](discussions.md) (threaded comments) - [Notification settings](notification_settings.md) - [Open source license templates](templates/licenses.md) - [Pages Domains](pages_domains.md) @@ -43,6 +46,7 @@ following locations: - [Pipeline Schedules](pipeline_schedules.md) - [Projects](projects.md) including setting Webhooks - [Project Access Requests](access_requests.md) +- [Project Badges](project_badges.md) - [Project import/export](project_import_export.md) - [Project Members](members.md) - [Project Snippets](project_snippets.md) diff --git a/doc/api/branches.md b/doc/api/branches.md index 80744258acb..01bb30c3859 100644 --- a/doc/api/branches.md +++ b/doc/api/branches.md @@ -13,6 +13,7 @@ GET /projects/:id/repository/branches | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `search` | string | no | Return list of branches matching the search criteria. | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/repository/branches diff --git a/doc/api/discussions.md b/doc/api/discussions.md new file mode 100644 index 00000000000..c341b7f2009 --- /dev/null +++ b/doc/api/discussions.md @@ -0,0 +1,411 @@ +# Discussions API + +Discussions are set of related notes on snippets or issues. + +## Issues + +### List project issue discussions + +Gets a list of all discussions for a single issue. + +``` +GET /projects/:id/issues/:issue_iid/discussions +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ------------ | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `issue_iid` | integer | yes | The IID of an issue | + +```json +[ + { + "id": "6a9c1750b37d513a43987b574953fceb50b03ce7", + "individual_note": false, + "notes": [ + { + "id": 1126, + "type": "DiscussionNote", + "body": "discussion text", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-03T21:54:39.668Z", + "updated_at": "2018-03-03T21:54:39.668Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Issue", + "noteable_iid": null + }, + { + "id": 1129, + "type": "DiscussionNote", + "body": "reply to the discussion", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T13:38:02.127Z", + "updated_at": "2018-03-04T13:38:02.127Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Issue", + "noteable_iid": null + } + ] + }, + { + "id": "87805b7c09016a7058e91bdbe7b29d1f284a39e6", + "individual_note": true, + "notes": [ + { + "id": 1128, + "type": null, + "body": "a single comment", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T09:17:22.520Z", + "updated_at": "2018-03-04T09:17:22.520Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Issue", + "noteable_iid": null + } + ] + } +] +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/discussions +``` + +### Get single issue discussion + +Returns a single discussion for a specific project issue + +``` +GET /projects/:id/issues/:issue_iid/discussions/:discussion_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `issue_iid` | integer | yes | The IID of an issue | +| `discussion_id` | integer | yes | The ID of a discussion | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7 +``` + +### Create new issue discussion + +Creates a new discussion to a single project issue. This is similar to creating +a note but but another comments (replies) can be added to it later. + +``` +POST /projects/:id/issues/:issue_iid/discussions +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `issue_iid` | integer | yes | The IID of an issue | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/discussions?body=comment +``` + +### Add note to existing issue discussion + +Adds a new note to the discussion. + +``` +POST /projects/:id/issues/:issue_iid/discussions/:discussion_id/notes +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `issue_iid` | integer | yes | The IID of an issue | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes?body=comment +``` + +### Modify existing issue discussion note + +Modify existing discussion note of an issue. + +``` +PUT /projects/:id/issues/:issue_iid/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `issue_iid` | integer | yes | The IID of an issue | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes/1108?body=comment +``` + +### Delete an issue discussion note + +Deletes an existing discussion note of an issue. + +``` +DELETE /projects/:id/issues/:issue_iid/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `issue_iid` | integer | yes | The IID of an issue | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/discussions/636 +``` + +## Snippets + +### List project snippet discussions + +Gets a list of all discussions for a single snippet. + +``` +GET /projects/:id/snippets/:snippet_id/discussions +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `snippet_id` | integer | yes | The ID of an snippet | + +```json +[ + { + "id": "6a9c1750b37d513a43987b574953fceb50b03ce7", + "individual_note": false, + "notes": [ + { + "id": 1126, + "type": "DiscussionNote", + "body": "discussion text", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-03T21:54:39.668Z", + "updated_at": "2018-03-03T21:54:39.668Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Snippet", + "noteable_id": null + }, + { + "id": 1129, + "type": "DiscussionNote", + "body": "reply to the discussion", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T13:38:02.127Z", + "updated_at": "2018-03-04T13:38:02.127Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Snippet", + "noteable_id": null + } + ] + }, + { + "id": "87805b7c09016a7058e91bdbe7b29d1f284a39e6", + "individual_note": true, + "notes": [ + { + "id": 1128, + "type": null, + "body": "a single comment", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T09:17:22.520Z", + "updated_at": "2018-03-04T09:17:22.520Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Snippet", + "noteable_id": null + } + ] + } +] +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions +``` + +### Get single snippet discussion + +Returns a single discussion for a specific project snippet + +``` +GET /projects/:id/snippets/:snippet_id/discussions/:discussion_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `snippet_id` | integer | yes | The ID of an snippet | +| `discussion_id` | integer | yes | The ID of a discussion | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7 +``` + +### Create new snippet discussion + +Creates a new discussion to a single project snippet. This is similar to creating +a note but but another comments (replies) can be added to it later. + +``` +POST /projects/:id/snippets/:snippet_id/discussions +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `snippet_id` | integer | yes | The ID of an snippet | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions?body=comment +``` + +### Add note to existing snippet discussion + +Adds a new note to the discussion. + +``` +POST /projects/:id/snippets/:snippet_id/discussions/:discussion_id/notes +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `snippet_id` | integer | yes | The ID of an snippet | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes?body=comment +``` + +### Modify existing snippet discussion note + +Modify existing discussion note of an snippet. + +``` +PUT /projects/:id/snippets/:snippet_id/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `snippet_id` | integer | yes | The ID of an snippet | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes/1108?body=comment +``` + +### Delete an snippet discussion note + +Deletes an existing discussion note of an snippet. + +``` +DELETE /projects/:id/snippets/:snippet_id/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `snippet_id` | integer | yes | The ID of an snippet | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions/636 +``` diff --git a/doc/api/group_badges.md b/doc/api/group_badges.md new file mode 100644 index 00000000000..3e0683f378d --- /dev/null +++ b/doc/api/group_badges.md @@ -0,0 +1,191 @@ +# Group badges API + +## Placeholder tokens + +Badges support placeholders that will be replaced in real time in both the link and image URL. The allowed placeholders are: + +- **%{project_path}**: will be replaced by the project path. +- **%{project_id}**: will be replaced by the project id. +- **%{default_branch}**: will be replaced by the project default branch. +- **%{commit_sha}**: will be replaced by the last project's commit sha. + +Because these enpoints aren't inside a project's context, the information used to replace the placeholders will be +from the first group's project by creation date. If the group hasn't got any project the original URL with the placeholders will be returned. + +## List all badges of a group + +Gets a list of a group's badges. + +``` +GET /groups/:id/badges +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/:id/badges +``` + +Example response: + +```json +[ + { + "id": 1, + "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", + "rendered_image_url": "https://shields.io/my/badge", + "kind": "group" + }, + { + "id": 2, + "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", + "rendered_image_url": "https://shields.io/my/badge", + "kind": "group" + }, +] +``` + +## Get a badge of a group + +Gets a badge of a group. + +``` +GET /groups/:id/badges/:badge_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `badge_id` | integer | yes | The badge ID | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/:id/badges/:badge_id +``` + +Example response: + +```json +{ + "id": 1, + "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", + "rendered_image_url": "https://shields.io/my/badge", + "kind": "group" +} +``` + +## Add a badge to a group + +Adds a badge to a group. + +``` +POST /groups/:id/badges +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `link_url` | string | yes | URL of the badge link | +| `image_url` | string | yes | URL of the badge image | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "link_url=https://gitlab.com/gitlab-org/gitlab-ce/commits/master&image_url=https://shields.io/my/badge1&position=0" https://gitlab.example.com/api/v4/groups/:id/badges +``` + +Example response: + +```json +{ + "id": 1, + "link_url": "https://gitlab.com/gitlab-org/gitlab-ce/commits/master", + "image_url": "https://shields.io/my/badge1", + "rendered_link_url": "https://gitlab.com/gitlab-org/gitlab-ce/commits/master", + "rendered_image_url": "https://shields.io/my/badge1", + "kind": "group" +} +``` + +## Edit a badge of a group + +Updates a badge of a group. + +``` +PUT /groups/:id/badges/:badge_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `badge_id` | integer | yes | The badge ID | +| `link_url` | string | no | URL of the badge link | +| `image_url` | string | no | URL of the badge image | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/:id/badges/:badge_id +``` + +Example response: + +```json +{ + "id": 1, + "link_url": "https://gitlab.com/gitlab-org/gitlab-ce/commits/master", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "https://gitlab.com/gitlab-org/gitlab-ce/commits/master", + "rendered_image_url": "https://shields.io/my/badge", + "kind": "group" +} +``` + +## Remove a badge from a group + +Removes a badge from a group. + +``` +DELETE /groups/:id/badges/:badge_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `badge_id` | integer | yes | The badge ID | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/:id/badges/:badge_id +``` + +## Preview a badge from a group + +Returns how the `link_url` and `image_url` final URLs would be after resolving the placeholder interpolation. + +``` +GET /groups/:id/badges/render +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `link_url` | string | yes | URL of the badge link| +| `image_url` | string | yes | URL of the badge image | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/:id/badges/render?link_url=http%3A%2F%2Fexample.com%2Fci_status.svg%3Fproject%3D%25%7Bproject_path%7D%26ref%3D%25%7Bdefault_branch%7D&image_url=https%3A%2F%2Fshields.io%2Fmy%2Fbadge +``` + +Example response: + +```json +{ + "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", + "rendered_image_url": "https://shields.io/my/badge", +} +``` diff --git a/doc/api/group_boards.md b/doc/api/group_boards.md new file mode 100644 index 00000000000..45a8544d6b1 --- /dev/null +++ b/doc/api/group_boards.md @@ -0,0 +1,284 @@ +# Group Issue Boards API + +Every API call to group boards must be authenticated. + +If a user is not a member of a group and the group is private, a `GET` +request will result in `404` status code. + +## Group Board + +Lists Issue Boards in the given group. + +``` +GET /groups/:id/boards +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/5/boards +``` + +Example response: + +```json +[ + { + "id": 1, + "group_id": 5, + "lists" : [ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } + ] + } +] +``` + +## Single board + +Gets a single board. + +``` +GET /groups/:id/boards/:board_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/5/boards/1 +``` + +Example response: + +```json + { + "id": 1, + "group_id": 5, + "lists" : [ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } + ] + } +``` + +## List board lists + +Get a list of the board's lists. +Does not include `backlog` and `closed` lists + +``` +GET /groups/:id/boards/:board_id/lists +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/5/boards/1/lists +``` + +Example response: + +```json +[ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } +] +``` + +## Single board list + +Get a single board list. + +``` +GET /groups/:id/boards/:board_id/lists/:list_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/5/boards/1/lists/1 +``` + +Example response: + +```json +{ + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 +} +``` + +## New board list + +Creates a new Issue Board list. + +``` +POST /groups/:id/boards/:board_id/lists +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `label_id` | integer | yes | The ID of a label | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/5/boards/1/lists?label_id=5 +``` + +Example response: + +```json +{ + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 +} +``` + +## Edit board list + +Updates an existing Issue Board list. This call is used to change list position. + +``` +PUT /groups/:id/boards/:board_id/lists/:list_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | +| `position` | integer | yes | The position of the list | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/group/5/boards/1/lists/1?position=2 +``` + +Example response: + +```json +{ + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 +} +``` + +## Delete a board list + +Only for admins and group owners. Soft deletes the board list in question. + +``` +DELETE /groups/:id/boards/:board_id/lists/:list_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/5/boards/1/lists/1 +``` diff --git a/doc/api/groups.md b/doc/api/groups.md index f50558b58a6..1aed8aac64e 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -525,3 +525,7 @@ And to switch pages add: ``` [ce-15142]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15142 + +## Group badges + +Read more in the [Group Badges](group_badges.md) documentation. diff --git a/doc/api/issues.md b/doc/api/issues.md index da89db17cd9..a4a51101297 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -46,6 +46,10 @@ GET /issues?my_reaction_emoji=star | `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | | `search` | string | no | Search issues against their `title` and `description` | +| `created_after` | datetime | no | Return issues created on or after the given time | +| `created_before` | datetime | no | Return issues created on or before the given time | +| `updated_after` | datetime | no | Return issues updated on or after the given time | +| `updated_before` | datetime | no | Return issues updated on or before the given time | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/issues @@ -152,6 +156,10 @@ GET /groups/:id/issues?my_reaction_emoji=star | `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | | `search` | string | no | Search group issues against their `title` and `description` | +| `created_after` | datetime | no | Return issues created on or after the given time | +| `created_before` | datetime | no | Return issues created on or before the given time | +| `updated_after` | datetime | no | Return issues updated on or after the given time | +| `updated_before` | datetime | no | Return issues updated on or before the given time | ```bash @@ -259,8 +267,10 @@ GET /projects/:id/issues?my_reaction_emoji=star | `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | | `search` | string | no | Search project issues against their `title` and `description` | -| `created_after` | datetime | no | Return issues created after the given time (inclusive) | -| `created_before` | datetime | no | Return issues created before the given time (inclusive) | +| `created_after` | datetime | no | Return issues created on or after the given time | +| `created_before` | datetime | no | Return issues created on or before the given time | +| `updated_after` | datetime | no | Return issues updated on or after the given time | +| `updated_before` | datetime | no | Return issues updated on or before the given time | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/4/issues diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index cb9b0618767..b9a4f661777 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -41,8 +41,10 @@ Parameters: | `milestone` | string | no | Return merge requests for a specific milestone | | `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | | `labels` | string | no | Return merge requests matching a comma separated list of labels | -| `created_after` | datetime | no | Return merge requests created after the given time (inclusive) | -| `created_before` | datetime | no | Return merge requests created before the given time (inclusive) | +| `created_after` | datetime | no | Return merge requests created on or after the given time | +| `created_before` | datetime | no | Return merge requests created on or before the given time | +| `updated_after` | datetime | no | Return merge requests updated on or after the given time | +| `updated_before` | datetime | no | Return merge requests updated on or before the given time | | `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` | | `author_id` | integer | no | Returns merge requests created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me` | | `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` | @@ -158,8 +160,10 @@ Parameters: | `milestone` | string | no | Return merge requests for a specific milestone | | `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | | `labels` | string | no | Return merge requests matching a comma separated list of labels | -| `created_after` | datetime | no | Return merge requests created after the given time (inclusive) | -| `created_before` | datetime | no | Return merge requests created before the given time (inclusive) | +| `created_after` | datetime | no | Return merge requests created on or after the given time | +| `created_before` | datetime | no | Return merge requests created on or before the given time | +| `updated_after` | datetime | no | Return merge requests updated on or after the given time | +| `updated_before` | datetime | no | Return merge requests updated on or before the given time | | `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13060] in GitLab 9.5)_ | | `author_id` | integer | no | Returns merge requests created by the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | | `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | @@ -494,6 +498,8 @@ Parameters: ## List MR pipelines +> [Introduced][ce-15454] in GitLab 10.5.0. + Get a list of merge request pipelines. ``` @@ -523,18 +529,19 @@ Creates a new merge request. POST /projects/:id/merge_requests ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `source_branch` | string | yes | The source branch | -| `target_branch` | string | yes | The target branch | -| `title` | string | yes | Title of MR | -| `assignee_id` | integer | no | Assignee user ID | -| `description` | string | no | Description of MR | -| `target_project_id` | integer | no | The target project (numeric id) | -| `labels` | string | no | Labels for MR as a comma-separated list | -| `milestone_id` | integer | no | The ID of a milestone | -| `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `source_branch` | string | yes | The source branch | +| `target_branch` | string | yes | The target branch | +| `title` | string | yes | Title of MR | +| `assignee_id` | integer | no | Assignee user ID | +| `description` | string | no | Description of MR | +| `target_project_id` | integer | no | The target project (numeric id) | +| `labels` | string | no | Labels for MR as a comma-separated list | +| `milestone_id` | integer | no | The ID of a milestone | +| `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | +| `allow_maintainer_to_push` | boolean | no | Whether or not a maintainer of the target project can push to the source branch | ```json { @@ -542,7 +549,7 @@ POST /projects/:id/merge_requests "iid": 1, "target_branch": "master", "source_branch": "test1", - "project_id": 3, + "project_id": 4, "title": "test1", "state": "opened", "upvotes": 0, @@ -563,7 +570,7 @@ POST /projects/:id/merge_requests "state": "active", "created_at": "2012-04-29T08:46:00Z" }, - "source_project_id": 4, + "source_project_id": 3, "target_project_id": 4, "labels": [ ], "description": "fixed login page css paddings", @@ -590,6 +597,7 @@ POST /projects/:id/merge_requests "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, + "allow_maintainer_to_push": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -607,19 +615,20 @@ Updates an existing merge request. You can change the target branch, title, or e PUT /projects/:id/merge_requests/:merge_request_iid ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `merge_request_iid` | integer | yes | The ID of a merge request | -| `target_branch` | string | no | The target branch | -| `title` | string | no | Title of MR | -| `assignee_id` | integer | no | The ID of the user to assign the merge request to. Set to `0` or provide an empty value to unassign all assignees. | -| `milestone_id` | integer | no | The ID of a milestone to assign the merge request to. Set to `0` or provide an empty value to unassign a milestone.| -| `labels` | string | no | Comma-separated label names for a merge request. Set to an empty string to unassign all labels. | -| `description` | string | no | Description of MR | -| `state_event` | string | no | New state (close/reopen) | -| `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | -| `discussion_locked` | boolean | no | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `merge_request_iid` | integer | yes | The ID of a merge request | +| `target_branch` | string | no | The target branch | +| `title` | string | no | Title of MR | +| `assignee_id` | integer | no | The ID of the user to assign the merge request to. Set to `0` or provide an empty value to unassign all assignees. | +| `milestone_id` | integer | no | The ID of a milestone to assign the merge request to. Set to `0` or provide an empty value to unassign a milestone.| +| `labels` | string | no | Comma-separated label names for a merge request. Set to an empty string to unassign all labels. | +| `description` | string | no | Description of MR | +| `state_event` | string | no | New state (close/reopen) | +| `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | +| `discussion_locked` | boolean | no | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | +| `allow_maintainer_to_push` | boolean | no | Whether or not a maintainer of the target project can push to the source branch | Must include at least one non-required attribute from above. @@ -628,7 +637,7 @@ Must include at least one non-required attribute from above. "id": 1, "iid": 1, "target_branch": "master", - "project_id": 3, + "project_id": 4, "title": "test1", "state": "opened", "upvotes": 0, @@ -649,7 +658,7 @@ Must include at least one non-required attribute from above. "state": "active", "created_at": "2012-04-29T08:46:00Z" }, - "source_project_id": 4, + "source_project_id": 3, "target_project_id": 4, "labels": [ ], "description": "description1", @@ -676,6 +685,7 @@ Must include at least one non-required attribute from above. "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, + "allow_maintainer_to_push": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -1449,3 +1459,4 @@ Example response: [ce-13060]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13060 [ce-14016]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14016 +[ce-15454]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15454 diff --git a/doc/api/notes.md b/doc/api/notes.md index 1b68bd99ce2..aa38d22845c 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -15,7 +15,7 @@ GET /projects/:id/issues/:issue_iid/notes?sort=asc&order_by=updated_at | Attribute | Type | Required | Description | | ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | `issue_iid` | integer | yes | The IID of an issue | `sort` | string | no | Return issue notes sorted in `asc` or `desc` order. Default is `desc` | `order_by` | string | no | Return issue notes ordered by `created_at` or `updated_at` fields. Default is `created_at` @@ -63,6 +63,10 @@ GET /projects/:id/issues/:issue_iid/notes?sort=asc&order_by=updated_at ] ``` +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/notes +``` + ### Get single issue note Returns a single note for a specific project issue @@ -73,14 +77,17 @@ GET /projects/:id/issues/:issue_iid/notes/:note_id Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `issue_iid` (required) - The IID of a project issue - `note_id` (required) - The ID of an issue note +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/notes/1 +``` + ### Create new issue note -Creates a new note to a single project issue. If you create a note where the body -only contains an Award Emoji, you'll receive this object back. +Creates a new note to a single project issue. ``` POST /projects/:id/issues/:issue_iid/notes @@ -88,11 +95,15 @@ POST /projects/:id/issues/:issue_iid/notes Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `issue_id` (required) - The IID of an issue - `body` (required) - The content of a note - `created_at` (optional) - Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/notes?body=note +``` + ### Modify existing issue note Modify existing note of an issue. @@ -103,11 +114,15 @@ PUT /projects/:id/issues/:issue_iid/notes/:note_id Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `issue_iid` (required) - The IID of an issue - `note_id` (required) - The ID of a note - `body` (required) - The content of a note +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/11/notes?body=note +``` + ### Delete an issue note Deletes an existing note of an issue. @@ -120,7 +135,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | | `issue_iid` | integer | yes | The IID of an issue | | `note_id` | integer | yes | The ID of a note | @@ -141,11 +156,15 @@ GET /projects/:id/snippets/:snippet_id/notes?sort=asc&order_by=updated_at | Attribute | Type | Required | Description | | ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | `snippet_id` | integer | yes | The ID of a project snippet | `sort` | string | no | Return snippet notes sorted in `asc` or `desc` order. Default is `desc` | `order_by` | string | no | Return snippet notes ordered by `created_at` or `updated_at` fields. Default is `created_at` +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/notes +``` + ### Get single snippet note Returns a single note for a given snippet. @@ -156,7 +175,7 @@ GET /projects/:id/snippets/:snippet_id/notes/:note_id Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `snippet_id` (required) - The ID of a project snippet - `note_id` (required) - The ID of a snippet note @@ -179,6 +198,10 @@ Parameters: } ``` +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/notes/11 +``` + ### Create new snippet note Creates a new note for a single snippet. Snippet notes are comments users can post to a snippet. @@ -190,10 +213,14 @@ POST /projects/:id/snippets/:snippet_id/notes Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `snippet_id` (required) - The ID of a snippet - `body` (required) - The content of a note +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippet/11/notes?body=note +``` + ### Modify existing snippet note Modify existing note of a snippet. @@ -204,11 +231,15 @@ PUT /projects/:id/snippets/:snippet_id/notes/:note_id Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `snippet_id` (required) - The ID of a snippet - `note_id` (required) - The ID of a note - `body` (required) - The content of a note +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/snippets/11/notes?body=note +``` + ### Delete a snippet note Deletes an existing note of a snippet. @@ -221,7 +252,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | | `snippet_id` | integer | yes | The ID of a snippet | | `note_id` | integer | yes | The ID of a note | @@ -242,11 +273,15 @@ GET /projects/:id/merge_requests/:merge_request_iid/notes?sort=asc&order_by=upda | Attribute | Type | Required | Description | | ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | `merge_request_iid` | integer | yes | The IID of a project merge request | `sort` | string | no | Return merge request notes sorted in `asc` or `desc` order. Default is `desc` | `order_by` | string | no | Return merge request notes ordered by `created_at` or `updated_at` fields. Default is `created_at` +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/notes +``` + ### Get single merge request note Returns a single note for a given merge request. @@ -257,7 +292,7 @@ GET /projects/:id/merge_requests/:merge_request_iid/notes/:note_id Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `merge_request_iid` (required) - The IID of a project merge request - `note_id` (required) - The ID of a merge request note @@ -283,6 +318,10 @@ Parameters: } ``` +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/notes/1 +``` + ### Create new merge request note Creates a new note for a single merge request. @@ -295,7 +334,7 @@ POST /projects/:id/merge_requests/:merge_request_iid/notes Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `merge_request_iid` (required) - The IID of a merge request - `body` (required) - The content of a note @@ -309,11 +348,15 @@ PUT /projects/:id/merge_requests/:merge_request_iid/notes/:note_id Parameters: -- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) - `merge_request_iid` (required) - The IID of a merge request - `note_id` (required) - The ID of a note - `body` (required) - The content of a note +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/notes?body=note +``` + ### Delete a merge request note Deletes an existing note of a merge request. @@ -326,7 +369,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | | `merge_request_iid` | integer | yes | The IID of a merge request | | `note_id` | integer | yes | The ID of a note | diff --git a/doc/api/project_badges.md b/doc/api/project_badges.md new file mode 100644 index 00000000000..3f6e348b5b4 --- /dev/null +++ b/doc/api/project_badges.md @@ -0,0 +1,188 @@ +# Project badges API + +## Placeholder tokens + +Badges support placeholders that will be replaced in real time in both the link and image URL. The allowed placeholders are: + +- **%{project_path}**: will be replaced by the project path. +- **%{project_id}**: will be replaced by the project id. +- **%{default_branch}**: will be replaced by the project default branch. +- **%{commit_sha}**: will be replaced by the last project's commit sha. + +## List all badges of a project + +Gets a list of a project's badges and its group badges. + +``` +GET /projects/:id/badges +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/:id/badges +``` + +Example response: + +```json +[ + { + "id": 1, + "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", + "rendered_image_url": "https://shields.io/my/badge", + "kind": "project" + }, + { + "id": 2, + "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", + "rendered_image_url": "https://shields.io/my/badge", + "kind": "group" + }, +] +``` + +## Get a badge of a project + +Gets a badge of a project. + +``` +GET /projects/:id/badges/:badge_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `badge_id` | integer | yes | The badge ID | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/:id/badges/:badge_id +``` + +Example response: + +```json +{ + "id": 1, + "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", + "rendered_image_url": "https://shields.io/my/badge", + "kind": "project" +} +``` + +## Add a badge to a project + +Adds a badge to a project. + +``` +POST /projects/:id/badges +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project ](README.md#namespaced-path-encoding) owned by the authenticated user | +| `link_url` | string | yes | URL of the badge link | +| `image_url` | string | yes | URL of the badge image | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "link_url=https://gitlab.com/gitlab-org/gitlab-ce/commits/master&image_url=https://shields.io/my/badge1&position=0" https://gitlab.example.com/api/v4/projects/:id/badges +``` + +Example response: + +```json +{ + "id": 1, + "link_url": "https://gitlab.com/gitlab-org/gitlab-ce/commits/master", + "image_url": "https://shields.io/my/badge1", + "rendered_link_url": "https://gitlab.com/gitlab-org/gitlab-ce/commits/master", + "rendered_image_url": "https://shields.io/my/badge1", + "kind": "project" +} +``` + +## Edit a badge of a project + +Updates a badge of a project. + +``` +PUT /projects/:id/badges/:badge_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `badge_id` | integer | yes | The badge ID | +| `link_url` | string | no | URL of the badge link | +| `image_url` | string | no | URL of the badge image | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/:id/badges/:badge_id +``` + +Example response: + +```json +{ + "id": 1, + "link_url": "https://gitlab.com/gitlab-org/gitlab-ce/commits/master", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "https://gitlab.com/gitlab-org/gitlab-ce/commits/master", + "rendered_image_url": "https://shields.io/my/badge", + "kind": "project" +} +``` + +## Remove a badge from a project + +Removes a badge from a project. Only project's badges will be removed by using this endpoint. + +``` +DELETE /projects/:id/badges/:badge_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `badge_id` | integer | yes | The badge ID | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/:id/badges/:badge_id +``` + +## Preview a badge from a project + +Returns how the `link_url` and `image_url` final URLs would be after resolving the placeholder interpolation. + +``` +GET /projects/:id/badges/render +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `link_url` | string | yes | URL of the badge link| +| `image_url` | string | yes | URL of the badge image | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/:id/badges/render?link_url=http%3A%2F%2Fexample.com%2Fci_status.svg%3Fproject%3D%25%7Bproject_path%7D%26ref%3D%25%7Bdefault_branch%7D&image_url=https%3A%2F%2Fshields.io%2Fmy%2Fbadge +``` + +Example response: + +```json +{ + "link_url": "http://example.com/ci_status.svg?project=%{project_path}&ref=%{default_branch}", + "image_url": "https://shields.io/my/badge", + "rendered_link_url": "http://example.com/ci_status.svg?project=example-org/example-project&ref=master", + "rendered_image_url": "https://shields.io/my/badge", +} +``` diff --git a/doc/api/project_import_export.md b/doc/api/project_import_export.md index e442442c750..677765368a8 100644 --- a/doc/api/project_import_export.md +++ b/doc/api/project_import_export.md @@ -1,9 +1,89 @@ -# Project import API +# Project import/export API [Introduced][ce-41899] in GitLab 10.6 [See also the project import/export documentation](../user/project/settings/import_export.md) +## Schedule an export + +Start a new export. + +```http +POST /projects/:id/export +``` + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | + +```console +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/1/export +``` + +```json +{ + "message": "202 Accepted" +} +``` + +## Export status + +Get the status of export. + +```http +GET /projects/:id/export +``` + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | + +```console +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/1/export +``` + +Status can be one of `none`, `started`, or `finished`. + +`_links` are only present when export has finished. + +```json +{ + "id": 1, + "description": "Itaque perspiciatis minima aspernatur corporis consequatur.", + "name": "Gitlab Test", + "name_with_namespace": "Gitlab Org / Gitlab Test", + "path": "gitlab-test", + "path_with_namespace": "gitlab-org/gitlab-test", + "created_at": "2017-08-29T04:36:44.383Z", + "export_status": "finished", + "_links": { + "api_url": "https://gitlab.example.com/api/v4/projects/1/export/download", + "web_url": "https://gitlab.example.com/gitlab-org/gitlab-test/download_export", + } +} +``` + +## Export download + +Download the finished export. + +```http +GET /projects/:id/export/download +``` + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | + +```console +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --remote-header-name --remote-name https://gitlab.example.com/api/v4/projects/5/export/download +``` + +```console +ls *export.tar.gz +2017-12-05_22-11-148_namespace_project_export.tar.gz +``` + ## Import a file ```http diff --git a/doc/api/projects.md b/doc/api/projects.md index b6442cfac22..271ee91dc72 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -1340,3 +1340,7 @@ Read more in the [Project import/export](project_import_export.md) documentation ## Project members Read more in the [Project members](members.md) documentation. + +## Project badges + +Read more in the [Project Badges](project_badges.md) documentation. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index ffebe1618d3..f69729f602d 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -32,34 +32,38 @@ There's also a collection of repositories with [example projects](https://gitlab - **Debian**: [Continuous Deployment with GitLab: how to build and deploy a Debian Package with GitLab CI](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) - **Maven**: [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) +### Game development + +- [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md) + ### Miscellaneous - [Using `dpl` as deployment tool](deployment/README.md) - [The `.gitlab-ci.yml` file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml) -### Code quality analysis +## Code quality analysis [Analyze code quality with the Code Climate CLI](code_climate.md). -### Static Application Security Testing (SAST) +## Static Application Security Testing (SAST) - **(Ultimate)** [Scan your code for vulnerabilities](https://docs.gitlab.com/ee/ci/examples/sast.html) - [Scan your Docker images for vulnerabilities](sast_docker.md) -### Dynamic Application Security Testing (DAST) +## Dynamic Application Security Testing (DAST) Scan your app for vulnerabilities with GitLab [Dynamic Application Security Testing (DAST)](dast.md). -### Browser Performance Testing with Sitespeed.io +## Browser Performance Testing with Sitespeed.io Analyze your [browser performance with Sitespeed.io](browser_performance.md). -### GitLab CI/CD for Review Apps +## GitLab CI/CD for Review Apps - [Example project](https://gitlab.com/gitlab-examples/review-apps-nginx/) that shows how to use GitLab CI/CD for [Review Apps](../review_apps/index.html). - [Dockerizing GitLab Review Apps](https://about.gitlab.com/2017/07/11/dockerizing-review-apps/) -### GitLab CI/CD for GitLab Pages +## GitLab CI/CD for GitLab Pages See the documentation on [GitLab Pages](../../user/project/pages/index.md) for a complete overview. diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png Binary files differnew file mode 100644 index 00000000000..76e0295722b --- /dev/null +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/aws_config_window.png diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png Binary files differnew file mode 100644 index 00000000000..050a97d2726 --- /dev/null +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/gitlab_config.png diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png Binary files differnew file mode 100644 index 00000000000..4ab5d5f401a --- /dev/null +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/img/test_pipeline_pass.png diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md new file mode 100644 index 00000000000..bfc8558a580 --- /dev/null +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -0,0 +1,526 @@ +--- +author: Ryan Hall +author_gitlab: blitzgren +level: intermediary +article_type: tutorial +date: 2018-03-07 +--- + +# DevOps and Game Dev with GitLab CI/CD + +With advances in WebGL and WebSockets, browsers are extremely viable as game development +platforms without the use of plugins like Adobe Flash. Furthermore, by using GitLab and [AWS](https://aws.amazon.com/), +single game developers, as well as game dev teams, can easily host browser-based games online. + +In this tutorial, we'll focus on DevOps, as well as testing and hosting games with Continuous +Integration/Deployment methods. We assume you are familiar with GitLab, javascript, +and the basics of game development. + +## The game + +Our [demo game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) consists of a simple spaceship traveling in space that shoots by clicking the mouse in a given direction. + +Creating a strong CI/CD pipeline at the beginning of developing another game, [Dark Nova](http://darknova.io/about), +was essential for the fast pace the team worked at. This tutorial will build upon my +[previous introductory article](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) and go through the following steps: + +1. Using code from the previous article to start with a barebones [Phaser](https://phaser.io) game built by a gulp file +1. Adding and running unit tests +1. Creating a `Weapon` class that can be triggered to spawn a `Bullet` in a given direction +1. Adding a `Player` class that uses this weapon and moves around the screen +1. Adding the sprites we will use for the `Player` and `Weapon` +1. Testing and deploying with Continuous Integration and Continuous Deployment methods + +By the end, we'll have the core of a [playable game](http://gitlab-game-demo.s3-website-us-east-1.amazonaws.com/) +that's tested and deployed on every push to the `master` branch of the [codebase](https://gitlab.com/blitzgren/gitlab-game-demo). +This will also provide +boilerplate code for starting a browser-based game with the following components: + +- Written in [Typescript](https://www.typescriptlang.org/) and [PhaserJs](https://phaser.io) +- Building, running, and testing with [Gulp](http://gulpjs.com/) +- Unit tests with [Chai](http://chaijs.com/) and [Mocha](https://mochajs.org/) +- CI/CD with GitLab +- Hosting the codebase on GitLab.com +- Hosting the game on AWS +- Deploying to AWS + +## Requirements and setup + +Please refer to my previous article [DevOps and Game Dev](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) to learn the foundational +development tools, running a Hello World-like game, and building this game using GitLab +CI/CD from every new push to master. The `master` branch for this game's [repository](https://gitlab.com/blitzgren/gitlab-game-demo) +contains a completed version with all configurations. If you would like to follow along +with this article, you can clone and work from the `devops-article` branch: + +```sh +git clone git@gitlab.com:blitzgren/gitlab-game-demo.git +git checkout devops-article +``` + +Next, we'll create a small subset of tests that exemplify most of the states I expect +this `Weapon` class to go through. To get started, create a folder called `lib/tests` +and add the following code to a new file `weaponTests.ts`: + +```ts +import { expect } from 'chai'; +import { Weapon, BulletFactory } from '../lib/weapon'; + +describe('Weapon', () => { + var subject: Weapon; + var shotsFired: number = 0; + // Mocked bullet factory + var bulletFactory: BulletFactory = <BulletFactory>{ + generate: function(px, py, vx, vy, rot) { + shotsFired++; + } + }; + var parent: any = { x: 0, y: 0 }; + + beforeEach(() => { + shotsFired = 0; + subject = new Weapon(bulletFactory, parent, 0.25, 1); + }); + + it('should shoot if not in cooldown', () => { + subject.trigger(true); + subject.update(0.1); + expect(shotsFired).to.equal(1); + }); + + it('should not shoot during cooldown', () => { + subject.trigger(true); + subject.update(0.1); + subject.update(0.1); + expect(shotsFired).to.equal(1); + }); + + it('should shoot after cooldown ends', () => { + subject.trigger(true); + subject.update(0.1); + subject.update(0.3); // longer than timeout + expect(shotsFired).to.equal(2); + }); + + it('should not shoot if not triggered', () => { + subject.update(0.1); + subject.update(0.1); + expect(shotsFired).to.equal(0); + }); +}); +``` + +To build and run these tests using gulp, let's also add the following gulp functions +to the existing `gulpfile.js` file: + +```ts +gulp.task('build-test', function () { + return gulp.src('src/tests/**/*.ts', { read: false }) + .pipe(tap(function (file) { + // replace file contents with browserify's bundle stream + file.contents = browserify(file.path, { debug: true }) + .plugin(tsify, { project: "./tsconfig.test.json" }) + .bundle(); + })) + .pipe(buffer()) + .pipe(sourcemaps.init({loadMaps: true}) ) + .pipe(gulp.dest('built/tests')); +}); + +gulp.task('run-test', function() { + gulp.src(['./built/tests/**/*.ts']).pipe(mocha()); +}); +``` + +We will start implementing the first part of our game and get these `Weapon` tests to pass. +The `Weapon` class will expose a method to trigger the generation of a bullet at a given +direction and speed. Later we will implement a `Player` class that ties together the user input +to trigger the weapon. In the `src/lib` folder create a `weapon.ts` file. We'll add two classes +to it: `Weapon` and `BulletFactory` which will encapsulate Phaser's **sprite** and +**group** objects, and the logic specific to our game. + +```ts +export class Weapon { + private isTriggered: boolean = false; + private currentTimer: number = 0; + + constructor(private bulletFactory: BulletFactory, private parent: Phaser.Sprite, private cooldown: number, private bulletSpeed: number) { + } + + public trigger(on: boolean): void { + this.isTriggered = on; + } + + public update(delta: number): void { + this.currentTimer -= delta; + + if (this.isTriggered && this.currentTimer <= 0) { + this.shoot(); + } + } + + private shoot(): void { + // Reset timer + this.currentTimer = this.cooldown; + + // Get velocity direction from player rotation + var parentRotation = this.parent.rotation + Math.PI / 2; + var velx = Math.cos(parentRotation); + var vely = Math.sin(parentRotation); + + // Apply a small forward offset so bullet shoots from head of ship instead of the middle + var posx = this.parent.x - velx * 10 + var posy = this.parent.y - vely * 10; + + this.bulletFactory.generate(posx, posy, -velx * this.bulletSpeed, -vely * this.bulletSpeed, this.parent.rotation); + } +} + +export class BulletFactory { + + constructor(private bullets: Phaser.Group, private poolSize: number) { + // Set all the defaults for this BulletFactory's bullet object + this.bullets.enableBody = true; + this.bullets.physicsBodyType = Phaser.Physics.ARCADE; + this.bullets.createMultiple(30, 'bullet'); + this.bullets.setAll('anchor.x', 0.5); + this.bullets.setAll('anchor.y', 0.5); + this.bullets.setAll('outOfBoundsKill', true); + this.bullets.setAll('checkWorldBounds', true); + } + + public generate(posx: number, posy: number, velx: number, vely: number, rot: number): Phaser.Sprite { + // Pull a bullet from Phaser's Group pool + var bullet = this.bullets.getFirstExists(false); + + // Set the few unique properties about this bullet: rotation, position, and velocity + if (bullet) { + bullet.reset(posx, posy); + bullet.rotation = rot; + bullet.body.velocity.x = velx; + bullet.body.velocity.y = vely; + } + + return bullet; + } +} +``` + +Lastly, we'll redo our entry point, `game.ts`, to tie together both `Player` and `Weapon` objects +as well as add them to the update loop. Here is what the updated `game.ts` file looks like: + +```ts +import { Player } from "./player"; +import { Weapon, BulletFactory } from "./weapon"; + +window.onload = function() { + var game = new Phaser.Game(800, 600, Phaser.AUTO, 'gameCanvas', { preload: preload, create: create, update: update }); + var player: Player; + var weapon: Weapon; + + // Import all assets prior to loading the game + function preload () { + game.load.image('player', 'assets/player.png'); + game.load.image('bullet', 'assets/bullet.png'); + } + + // Create all entities in the game, after Phaser loads + function create () { + // Create and position the player + var playerSprite = game.add.sprite(400, 550, 'player'); + playerSprite.anchor.setTo(0.5); + player = new Player(game.input, playerSprite, 150); + + var bulletFactory = new BulletFactory(game.add.group(), 30); + weapon = new Weapon(bulletFactory, player.sprite, 0.25, 1000); + + player.loadWeapon(weapon); + } + + // This function is called once every tick, default is 60fps + function update() { + var deltaSeconds = game.time.elapsedMS / 1000; // convert to seconds + player.update(deltaSeconds); + weapon.update(deltaSeconds); + } +} +``` + +Run `gulp serve` and you can run around and shoot. Wonderful! Let's update our CI +pipeline to include running the tests along with the existing build job. + +## Continuous Integration + +To ensure our changes don't break the build and all tests still pass, we utilize +Continuous Integration (CI) to run these checks automatically for every push. +Read through this article to understand [Continuous Integration, Continuous Delivery, and Continuous Deployment](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/), +and how these methods are leveraged by GitLab. +From the [last tutorial](https://ryanhallcs.wordpress.com/2017/03/15/devops-and-game-dev/) we already have a `gitlab-ci.yml` file set up for building our app from +every push. We need to set up a new CI job for testing, which GitLab CI/CD will run after the build job using our generated artifacts from gulp. + +Please read through the [documentation on CI/CD configuration](../../../ci/yaml/README.md) file to explore its contents and adjust it to your needs. + +### Build your game with GitLab CI/CD + +We need to update our build job to ensure tests get run as well. Add `gulp build-test` +to the end of the `script` array for the existing `build` job. Once these commands run, +we know we will need to access everything in the `built` folder, given by GitLab CI/CD's `artifacts`. +We'll also cache `node_modules` to avoid having to do a full re-pull of those dependencies: +just pack them up in the cache. Here is the full `build` job: + +```yml +build: + stage: build + script: + - npm i gulp -g + - npm i + - gulp + - gulp build-test + cache: + policy: push + paths: + - node_modules + artifacts: + paths: + - built +``` + +### Test your game with GitLab CI/CD + +For testing locally, we simply run `gulp run-tests`, which requires gulp to be installed +globally like in the `build` job. We pull `node_modules` from the cache, so the `npm i` +command won't have to do much. In preparation for deployment, we know we will still need +the `built` folder in the artifacts, which will be brought over as default behavior from +the previous job. Lastly, by convention, we let GitLab CI/CD know this needs to be run after +the `build` job by giving it a `test` [stage](../../../ci/yaml/README.md#stages). +Following the YAML structure, the `test` job should look like this: + +```yml +test: + stage: test + script: + - npm i gulp -g + - npm i + - gulp run-test + cache: + policy: push + paths: + - node_modules/ + artifacts: + paths: + - built/ +``` + +We have added unit tests for a `Weapon` class that shoots on a specified interval. +The `Player` class implements `Weapon` along with the ability to move around and shoot. Also, +we've added test artifacts and a test stage to our GitLab CI/CD pipeline using `.gitlab-ci.yml`, +allowing us to run our tests by every push. +Our entire `.gitlab-ci.yml` file should now look like this: + +```yml +image: node:6 + +build: + stage: build + script: + - npm i gulp -g + - npm i + - gulp + - gulp build-test + cache: + policy: push + paths: + - node_modules/ + artifacts: + paths: + - built/ + +test: + stage: test + script: + - npm i gulp -g + - npm i + - gulp run-test + cache: + policy: pull + paths: + - node_modules/ + artifacts: + paths: + - built/ +``` + +### Run your CI/CD pipeline + +That's it! Add all your new files, commit, and push. For a reference of what our repo should +look like at this point, please refer to the [final commit related to this article on my sample repository](https://gitlab.com/blitzgren/gitlab-game-demo/commit/8b36ef0ecebcf569aeb251be4ee13743337fcfe2). +By applying both build and test stages, GitLab will run them sequentially at every push to +our repository. If all goes well you'll end up with a green check mark on each job for the pipeline: + + + +You can confirm that the tests passed by clicking on the `test` job to enter the full build logs. +Scroll to the bottom and observe, in all its passing glory: + +```sh +$ gulp run-test +[18:37:24] Using gulpfile /builds/blitzgren/gitlab-game-demo/gulpfile.js +[18:37:24] Starting 'run-test'... +[18:37:24] Finished 'run-test' after 21 ms + + + Weapon + ✓ should shoot if not in cooldown + ✓ should not shoot during cooldown + ✓ should shoot after cooldown ends + ✓ should not shoot if not triggered + + + 4 passing (18ms) + +Uploading artifacts... +built/: found 17 matching files +Uploading artifacts to coordinator... ok id=17095874 responseStatus=201 Created token=aaaaaaaa Job succeeded +``` + +## Continuous Deployment + +We have our codebase built and tested on every push. To complete the full pipeline with Continuous Deployment, +let's set up [free web hosting with AWS S3](https://aws.amazon.com/s/dm/optimization/server-side-test/free-tier/free_np/) and a job through which our build artifacts get +deployed. GitLab also has a free static site hosting service we could use, [GitLab Pages](https://about.gitlab.com/features/pages/), +however Dark Nova specifically uses other AWS tools that necessitates using `AWS S3`. +Read through this article that describes [deploying to both S3 and GitLab Pages](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) +and further delves into the principles of GitLab CI/CD than discussed in this article. + +### Set up S3 Bucket + +1. Log into your AWS account and go to [S3](https://console.aws.amazon.com/s3/home) +1. Click the **Create Bucket** link at the top +1. Enter a name of your choosing and click next +1. Keep the default **Properties** and click next +1. Click the **Manage group permissions** and allow **Read** for the **Everyone** group, click next +1. Create the bucket, and select it in your S3 bucket list +1. On the right side, click **Properties** and enable the **Static website hosting** category +1. Update the radio button to the **Use this bucket to host a website** selection. Fill in `index.html` and `error.html` respectively + +### Set up AWS Secrets + +We need to be able to deploy to AWS with our AWS account credentials, but we certainly +don't want to put secrets into source code. Luckily GitLab provides a solution for this +with [Secret Variables](../../../ci/variables/README.md). This can get complicated +due to [IAM](https://aws.amazon.com/iam/) management. As a best practice, you shouldn't +use root security credentials. Proper IAM credential management is beyond the scope of this +article, but AWS will remind you that using root credentials is unadvised and against their +best practices, as they should. Feel free to follow best practices and use a custom IAM user's +credentials, which will be the same two credentials (Key ID and Secret). It's a good idea to +fully understand [IAM Best Practices in AWS](http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html). We need to add these credentials to GitLab: + +1. Log into your AWS account and go to the [Security Credentials page](https://console.aws.amazon.com/iam/home#/security_credential) +1. Click the **Access Keys** section and **Create New Access Key**. Create the key and keep the id and secret around, you'll need them later +  +1. Go to your GitLab project, click **Settings > CI/CD** on the left sidebar +1. Expand the **Secret Variables** section +  +1. Add a key named `AWS_KEY_ID` and copy the key id from Step 2 into the **Value** textbox +1. Add a key named `AWS_KEY_SECRET` and copy the key secret from Step 2 into the **Value** textbox + +### Deploy your game with GitLab CI/CD + +To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on +the Shared Runner. The Shared Runner also needs to be able to authenticate with your AWS +account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID` +and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the secret variables we +set up in the prior section using the `variables` portion of the `deploy` job. At the end, +we add directives to ensure deployment `only` happens on pushes to `master`. This way, every +single branch still runs through CI, and only merging (or committing directly) to master will +trigger the `deploy` job of our pipeline. Put these together to get the following: + +```yml +deploy: + stage: deploy + variables: + AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" + AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" + script: + - apt-get update + - apt-get install -y python3-dev python3-pip + - easy_install3 -U pip + - pip3 install --upgrade awscli + - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete + only: + - master +``` + +Be sure to update the region and S3 URL in that last script command to fit your setup. +Our final configuration file `.gitlab-ci.yml` looks like: + +```yml +image: node:6 + +build: + stage: build + script: + - npm i gulp -g + - npm i + - gulp + - gulp build-test + cache: + policy: push + paths: + - node_modules/ + artifacts: + paths: + - built/ + +test: + stage: test + script: + - npm i gulp -g + - gulp run-test + cache: + policy: pull + paths: + - node_modules/ + artifacts: + paths: + - built/ + +deploy: + stage: deploy + variables: + AWS_ACCESS_KEY_ID: "$AWS_KEY_ID" + AWS_SECRET_ACCESS_KEY: "$AWS_KEY_SECRET" + script: + - apt-get update + - apt-get install -y python3-dev python3-pip + - easy_install3 -U pip + - pip3 install --upgrade awscli + - aws s3 sync ./built s3://gitlab-game-demo --region "us-east-1" --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers --cache-control "no-cache, no-store, must-revalidate" --delete + only: + - master +``` + +## Conclusion + +Within the [demo repository](https://gitlab.com/blitzgren/gitlab-game-demo) you can also find a handful of boilerplate code to get +[Typescript](https://www.typescriptlang.org/), [Mocha](https://mochajs.org/), [Gulp](http://gulpjs.com/) and [Phaser](https://phaser.io) all playing +together nicely with GitLab CI/CD, which is the result of lessons learned while making [Dark Nova](http://darknova.io/). +Using a combination of free and open source software, we have a full CI/CD pipeline, a game foundation, +and unit tests, all running and deployed at every push to master - with shockingly little code. +Errors can be easily debugged through GitLab's build logs, and within minutes of a successful commit, +you can see the changes live on your game. + +Setting up Continous Integration and Continuous Deployment from the start with Dark Nova enables +rapid but stable development. We can easily test changes in a separate [environment](../../../ci/environments.md#introduction-to-environments-and-deployments), +or multiple environments if needed. Balancing and updating a multiplayer game can be ongoing +and tedious, but having faith in a stable deployment with GitLab CI/CD allows +a lot of breathing room in quickly getting changes to players. + +## Further settings + +Here are some ideas to further investigate that can speed up or improve your pipeline: + +- [Yarn](https://yarnpkg.com) instead of npm +- Setup a custom [Docker](../../../ci/docker/using_docker_images.md#define-image-and-services-from-gitlab-ci-yml) image that can preload dependencies and tools (like AWS CLI) +- Forward a [custom domain](http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) to your game's S3 static website +- Combine jobs if you find it unnecessary for a small project +- Avoid the queues and set up your own [custom GitLab CI/CD runner](https://about.gitlab.com/2016/03/01/gitlab-runner-with-docker/) diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 03aa6ff8e7c..f879ed62010 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -163,8 +163,7 @@ in your `.gitlab-ci.yml`. Behind the scenes, this works by increasing a counter in the database, and the value of that counter is used to create the key for the cache. After a push, a -new key is generated and the old cache is not valid anymore. Eventually, the -Runner's garbage collector will remove it form the filesystem. +new key is generated and the old cache is not valid anymore. ## How shared Runners pick jobs diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 23ce6a5f210..bd4aeb006bd 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -12,7 +12,7 @@ this order: 1. [Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables) (take precedence over all) 1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) 1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) -1. YAML-defined [job-level variables](../yaml/README.md#job-variables) +1. YAML-defined [job-level variables](../yaml/README.md#variables) 1. YAML-defined [global variables](../yaml/README.md#variables) 1. [Deployment variables](#deployment-variables) 1. [Predefined variables](#predefined-variables-environment-variables) (are the diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 80ab63468f2..ae200f9b6e2 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -3,18 +3,19 @@ This document describes the usage of `.gitlab-ci.yml`, the file that is used by GitLab Runner to manage your project's jobs. -If you want a quick introduction to GitLab CI, follow our -[quick start guide](../quick_start/README.md). - -## .gitlab-ci.yml - From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML) file (`.gitlab-ci.yml`) for the project configuration. It is placed in the root of your repository and contains definitions of how your project should be built. +If you want a quick introduction to GitLab CI, follow our +[quick start guide](../quick_start/README.md). + +## Jobs + The YAML file defines a set of jobs with constraints stating when they should -be run. The jobs are defined as top-level elements with a name and always have -to contain at least the `script` clause: +be run. You can specify an unlimited number of jobs which are defined as +top-level elements with an arbitrary name and always have to contain at least +the `script` clause. ```yaml job1: @@ -24,9 +25,8 @@ job2: script: "execute-script-for-job2" ``` -The above example is the simplest possible CI configuration with two separate +The above example is the simplest possible CI/CD configuration with two separate jobs, where each of the jobs executes a different command. - Of course a command can execute code directly (`./configure;make;make install`) or run a script (`test.sh`) in the repository. @@ -34,78 +34,115 @@ Jobs are picked up by [Runners](../runners/README.md) and executed within the environment of the Runner. What is important, is that each job is run independently from each other. -The YAML syntax allows for using more complex job specifications than in the -above example: +Each job must have a unique name, but there are a few **reserved `keywords` that +cannot be used as job names**: -```yaml -image: ruby:2.1 -services: - - postgres +- `image` +- `services` +- `stages` +- `types` +- `before_script` +- `after_script` +- `variables` +- `cache` -before_script: - - bundle install +A job is defined by a list of parameters that define the job behavior. -after_script: - - rm secrets +| Keyword | Required | Description | +|---------------|----------|-------------| +| script | yes | Defines a shell script which is executed by Runner | +| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | +| services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | +| stage | no | Defines a job stage (default: `test`) | +| type | no | Alias for `stage` | +| variables | no | Define job variables on a job level | +| only | no | Defines a list of git refs for which job is created | +| except | no | Defines a list of git refs for which job is not created | +| tags | no | Defines a list of tags which are used to select Runner | +| allow_failure | no | Allow job to fail. Failed job doesn't contribute to commit status | +| when | no | Define when to run job. Can be `on_success`, `on_failure`, `always` or `manual` | +| dependencies | no | Define other jobs that a job depends on so that you can pass artifacts between them| +| artifacts | no | Define list of [job artifacts](#artifacts) | +| cache | no | Define list of files that should be cached between subsequent runs | +| before_script | no | Override a set of commands that are executed before job | +| after_script | no | Override a set of commands that are executed after job | +| environment | no | Defines a name of environment to which deployment is done by this job | +| coverage | no | Define code coverage settings for a given job | +| retry | no | Define how many times a job can be auto-retried in case of a failure | -stages: - - build - - test - - deploy +### `pages` -job1: - stage: build +`pages` is a special job that is used to upload static content to GitLab that +can be used to serve your website. It has a special syntax, so the two +requirements below must be met: + +1. Any static content must be placed under a `public/` directory +1. `artifacts` with a path to the `public/` directory must be defined + +The example below simply moves all files from the root of the project to the +`public/` directory. The `.public` workaround is so `cp` doesn't also copy +`public/` to itself in an infinite loop: + +``` +pages: + stage: deploy script: - - execute-script-for-job1 + - mkdir .public + - cp -r * .public + - mv .public public + artifacts: + paths: + - public only: - - master - tags: - - docker + - master ``` -There are a few reserved `keywords` that **cannot** be used as job names: - -| Keyword | Required | Description | -|---------------|----------|-------------| -| image | no | Use docker image, covered in [Use Docker](../docker/README.md) | -| services | no | Use docker services, covered in [Use Docker](../docker/README.md) | -| stages | no | Define build stages | -| types | no | Alias for `stages` (deprecated) | -| before_script | no | Define commands that run before each job's script | -| after_script | no | Define commands that run after each job's script | -| variables | no | Define build variables | -| cache | no | Define list of files that should be cached between subsequent runs | +Read more on [GitLab Pages user documentation](../../user/project/pages/index.md). -### image and services +## `image` and `services` This allows to specify a custom Docker image and a list of services that can be used for time of the job. The configuration of this feature is covered in [a separate document](../docker/README.md). -### before_script - -`before_script` is used to define the command that should be run before all -jobs, including deploy jobs, but after the restoration of artifacts. This can -be an array or a multi-line string. - -### after_script +## `before_script` and `after_script` > Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 +`before_script` is used to define the command that should be run before all +jobs, including deploy jobs, but after the restoration of [artifacts](#artifacts). +This can be an array or a multi-line string. + `after_script` is used to define the command that will be run after for all jobs, including failed ones. This has to be an array or a multi-line string. -> **Note:** The `before_script` and the main `script` are concatenated and run in a single context/container. The `after_script` is run separately, so depending on the executor, changes done outside of the working tree might not be visible, e.g. software installed in the `before_script`. -### stages +It's possible to overwrite the globally defined `before_script` and `after_script` +if you set it per-job: -`stages` is used to define stages that can be used by jobs. -The specification of `stages` allows for having flexible multi stage pipelines. +```yaml +before_script: +- global before script + +job: + before_script: + - execute this instead of global before script + script: + - my command + after_script: + - execute this after my script +``` + +## `stages` +`stages` is used to define stages that can be used by jobs and is defined +globally. + +The specification of `stages` allows for having flexible multi stage pipelines. The ordering of elements in `stages` defines the ordering of jobs' execution: 1. Jobs of the same stage are run in parallel. @@ -134,280 +171,45 @@ There are also two edge cases worth mentioning: `test` and `deploy` are allowed to be used as job's stage by default. 2. If a job doesn't specify a `stage`, the job is assigned the `test` stage. -### types - -> Deprecated, and could be removed in one of the future releases. Use [stages](#stages) instead. - -Alias for [stages](#stages). - -### variables - -> Introduced in GitLab Runner v0.5.0. - -GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the -job environment. The variables are stored in the Git repository and are meant -to store non-sensitive project configuration, for example: - -```yaml -variables: - DATABASE_URL: "postgres://postgres@postgres/my_database" -``` - ->**Note:** -Integers (as well as strings) are legal both for variable's name and value. -Floats are not legal and cannot be used. - -These variables can be later used in all executed commands and scripts. -The YAML-defined variables are also set to all created service containers, -thus allowing to fine tune them. Variables can be also defined on a -[job level](#job-variables). - -Except for the user defined variables, there are also the ones set up by the -Runner itself. One example would be `CI_COMMIT_REF_NAME` which has the value of -the branch or tag name for which project is built. Apart from the variables -you can set in `.gitlab-ci.yml`, there are also the so called secret variables -which can be set in GitLab's UI. - -[Learn more about variables.][variables] - -### cache - -> -**Notes:** -- Introduced in GitLab Runner v0.7.0. -- Prior to GitLab 9.2, caches were restored after artifacts. -- From GitLab 9.2, caches are restored before artifacts. - -`cache` is used to specify a list of files and directories which should be -cached between jobs. You can only use paths that are within the project -workspace. - -**By default caching is enabled and shared between pipelines and jobs, -starting from GitLab 9.0** - -If `cache` is defined outside the scope of jobs, it means it is set -globally and all jobs will use that definition. - -Cache all files in `binaries` and `.config`: - -```yaml -rspec: - script: test - cache: - paths: - - binaries/ - - .config -``` - -Cache all Git untracked files: - -```yaml -rspec: - script: test - cache: - untracked: true -``` - -Cache all Git untracked files and files in `binaries`: - -```yaml -rspec: - script: test - cache: - untracked: true - paths: - - binaries/ -``` - -Locally defined cache overrides globally defined options. The following `rspec` -job will cache only `binaries/`: - -```yaml -cache: - paths: - - my/files - -rspec: - script: test - cache: - key: rspec - paths: - - binaries/ -``` - -Note that since cache is shared between jobs, if you're using different -paths for different jobs, you should also set a different **cache:key** -otherwise cache content can be overwritten. - -The cache is provided on a best-effort basis, so don't expect that the cache -will be always present. For implementation details, please check GitLab Runner. - -#### cache:key - -> Introduced in GitLab Runner v1.0.0. - -The `key` directive allows you to define the affinity of caching -between jobs, allowing to have a single cache for all jobs, -cache per-job, cache per-branch or any other way you deem proper. - -This allows you to fine tune caching, allowing you to cache data between -different jobs or even different branches. - -The `cache:key` variable can use any of the [predefined variables](../variables/README.md). - -The default key is **default** across the project, therefore everything is -shared between each pipelines and jobs by default, starting from GitLab 9.0. - ->**Note:** The `cache:key` variable cannot contain the `/` character, or the equivalent URI encoded `%2F`; a value made only of dots (`.`, `%2E`) is also forbidden. - ---- - -**Example configurations** - -To enable per-job caching: - -```yaml -cache: - key: "$CI_JOB_NAME" - untracked: true -``` - -To enable per-branch caching: - -```yaml -cache: - key: "$CI_COMMIT_REF_SLUG" - untracked: true -``` - -To enable per-job and per-branch caching: - -```yaml -cache: - key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" - untracked: true -``` - -To enable per-branch and per-stage caching: - -```yaml -cache: - key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG" - untracked: true -``` - -If you use **Windows Batch** to run your shell scripts you need to replace -`$` with `%`: - -```yaml -cache: - key: "%CI_JOB_STAGE%-%CI_COMMIT_REF_SLUG%" - untracked: true -``` +## `stage` -If you use **Windows PowerShell** to run your shell scripts you need to replace -`$` with `$env:`: - -```yaml -cache: - key: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_SLUG" - untracked: true -``` - -### cache:policy - -> Introduced in GitLab 9.4. - -The default behaviour of a caching job is to download the files at the start of -execution, and to re-upload them at the end. This allows any changes made by the -job to be persisted for future runs, and is known as the `pull-push` cache -policy. - -If you know the job doesn't alter the cached files, you can skip the upload step -by setting `policy: pull` in the job specification. Typically, this would be -twinned with an ordinary cache job at an earlier stage to ensure the cache -is updated from time to time: +`stage` is defined per-job and relies on [`stages`](#stages) which is defined +globally. It allows to group jobs into different stages, and jobs of the same +`stage` are executed in `parallel`. For example: ```yaml stages: - - setup + - build - test + - deploy -prepare: - stage: setup - cache: - key: gems - paths: - - vendor/bundle - script: - - bundle install --deployment - -rspec: - stage: test - cache: - key: gems - paths: - - vendor/bundle - policy: pull - script: - - bundle exec rspec ... -``` - -This helps to speed up job execution and reduce load on the cache server, -especially when you have a large number of cache-using jobs executing in -parallel. - -Additionally, if you have a job that unconditionally recreates the cache without -reference to its previous contents, you can use `policy: push` in that job to -skip the download step. - -## Jobs +job 1: + stage: build + script: make build dependencies -`.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job -must have a unique name, which is not one of the keywords mentioned above. -A job is defined by a list of parameters that define the job behavior. +job 2: + stage: build + script: make build artifacts -```yaml -job_name: - script: - - rake spec - - coverage +job 3: stage: test - only: - - master - except: - - develop - tags: - - ruby - - postgres - allow_failure: true + script: make test + +job 4: + stage: deploy + script: make deploy ``` -| Keyword | Required | Description | -|---------------|----------|-------------| -| script | yes | Defines a shell script which is executed by Runner | -| image | no | Use docker image, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | -| services | no | Use docker services, covered in [Using Docker Images](../docker/using_docker_images.md#define-image-and-services-from-gitlab-ciyml) | -| stage | no | Defines a job stage (default: `test`) | -| type | no | Alias for `stage` | -| variables | no | Define job variables on a job level | -| only | no | Defines a list of git refs for which job is created | -| except | no | Defines a list of git refs for which job is not created | -| tags | no | Defines a list of tags which are used to select Runner | -| allow_failure | no | Allow job to fail. Failed job doesn't contribute to commit status | -| when | no | Define when to run job. Can be `on_success`, `on_failure`, `always` or `manual` | -| dependencies | no | Define other jobs that a job depends on so that you can pass artifacts between them| -| artifacts | no | Define list of [job artifacts](../../user/project/pipelines/job_artifacts.md) | -| cache | no | Define list of files that should be cached between subsequent runs | -| before_script | no | Override a set of commands that are executed before job | -| after_script | no | Override a set of commands that are executed after job | -| environment | no | Defines a name of environment to which deployment is done by this job | -| coverage | no | Define code coverage settings for a given job | -| retry | no | Define how many times a job can be auto-retried in case of a failure | +## `types` -### script +CAUTION: **Deprecated:** +`types` is deprecated, and could be removed in one of the future releases. +Use [stages](#stages) instead. -`script` is a shell script which is executed by the Runner. For example: +## `script` + +`script` is the only required keyword that a job needs. It's a shell script +which is executed by the Runner. For example: ```yaml job: @@ -429,13 +231,7 @@ that the YAML parser knows to interpret the whole thing as a string rather than a "key: value" pair. Be careful when using special characters: `:`, `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``. -### stage - -`stage` allows to group jobs into different stages. Jobs of the same `stage` -are executed in `parallel`. For more info about the use of `stage` please check -[stages](#stages). - -### only and except (simplified) +## `only` and `except` (simplified) `only` and `except` are two parameters that set a job policy to limit when jobs are created: @@ -505,12 +301,13 @@ job: The above example will run `job` for all branches on `gitlab-org/gitlab-ce`, except master. -### only and except (complex) +## `only` and `except` (complex) > Introduced in GitLab 10.0 -> This an _alpha_ feature, and it it subject to change at any time without - prior notice! +CAUTION: **Warning:** +This an _alpha_ feature, and it it subject to change at any time without +prior notice! Since GitLab 10.0 it is possible to define a more elaborate only/except job policy configuration. @@ -535,24 +332,7 @@ job: kubernetes: active ``` -### Job variables - -It is possible to define job variables using a `variables` keyword on a job -level. It works basically the same way as its [global-level equivalent](#variables), -but allows you to define job-specific variables. - -When the `variables` keyword is used on a job level, it overrides the global YAML -job variables and predefined ones. To turn off global defined variables -in your job, define an empty hash: - -```yaml -job_name: - variables: {} -``` - -Job variables priority is defined in the [variables documentation][variables]. - -### tags +## `tags` `tags` is used to select specific Runners from the list of all Runners that are allowed to run this project. @@ -573,7 +353,7 @@ job: The specification above, will make sure that `job` is built by a Runner that has both `ruby` AND `postgres` tags defined. -### allow_failure +## `allow_failure` `allow_failure` is used when you want to allow a job to fail without impacting the rest of the CI suite. Failed jobs don't contribute to the commit status. @@ -606,7 +386,7 @@ job3: - deploy_to_staging ``` -### when +## `when` `when` is used to implement jobs that are run in case of failure or despite the failure. @@ -619,7 +399,7 @@ failure. fails. 1. `always` - execute job regardless of the status of jobs from prior stages. 1. `manual` - execute job manually (added in GitLab 8.10). Read about - [manual actions](#manual-actions) below. + [manual actions](#when-manual) below. For example: @@ -667,42 +447,41 @@ The above script will: success or failure. 3. Allow you to manually execute `deploy_job` from GitLab's UI. -#### Manual actions - -> Introduced in GitLab 8.10. -> Blocking manual actions were introduced in GitLab 9.0 -> Protected actions were introduced in GitLab 9.2 +### `when:manual` -Manual actions are a special type of job that are not executed automatically; -they need to be explicitly started by a user. Manual actions can be started -from pipeline, build, environment, and deployment views. +> **Notes:** +- Introduced in GitLab 8.10. +- Blocking manual actions were introduced in GitLab 9.0. +- Protected actions were introduced in GitLab 9.2. -An example usage of manual actions is deployment to production. +Manual actions are a special type of job that are not executed automatically, +they need to be explicitly started by a user. An example usage of manual actions +would be a deployment to a production environment. Manual actions can be started +from the pipeline, job, environment, and deployment views. Read more at the +[environments documentation][env-manual]. -Read more at the [environments documentation][env-manual]. - -Manual actions can be either optional or blocking. Blocking manual action will -block execution of the pipeline at stage this action is defined in. It is +Manual actions can be either optional or blocking. Blocking manual actions will +block the execution of the pipeline at the stage this action is defined in. It's possible to resume execution of the pipeline when someone executes a blocking -manual actions by clicking a _play_ button. +manual action by clicking a _play_ button. -When pipeline is blocked it will not be merged if Merge When Pipeline Succeeds +When a pipeline is blocked, it will not be merged if Merge When Pipeline Succeeds is set. Blocked pipelines also do have a special status, called _manual_. - Manual actions are non-blocking by default. If you want to make manual action blocking, it is necessary to add `allow_failure: false` to the job's definition in `.gitlab-ci.yml`. -Optional manual actions have `allow_failure: true` set by default. +Optional manual actions have `allow_failure: true` set by default and their +Statuses do not contribute to the overall pipeline status. So, if a manual +action fails, the pipeline will eventually succeed. -**Statuses of optional actions do not contribute to overall pipeline status.** +Manual actions are considered to be write actions, so permissions for +[protected branches](../../user/project/protected_branches.md) are used when +user wants to trigger an action. In other words, in order to trigger a manual +action assigned to a branch that the pipeline is running for, user needs to +have ability to merge to this branch. -**Manual actions are considered to be write actions, so permissions for -protected branches are used when user wants to trigger an action. In other -words, in order to trigger a manual action assigned to a branch that the -pipeline is running for, user needs to have ability to merge to this branch.** - -### environment +## `environment` > **Notes:** @@ -727,7 +506,7 @@ deploy to production: In the above example, the `deploy to production` job will be marked as doing a deployment to the `production` environment. -#### environment:name +### `environment:name` > **Notes:** @@ -766,7 +545,7 @@ deploy to production: name: production ``` -#### environment:url +### `environment:url` > **Notes:** @@ -793,7 +572,7 @@ deploy to production: url: https://prod.example.com ``` -#### environment:on_stop +### `environment:on_stop` > **Notes:** @@ -808,7 +587,7 @@ the environment. Read the `environment:action` section for an example. -#### environment:action +### `environment:action` > [Introduced][ce-6669] in GitLab 8.13. @@ -849,7 +628,7 @@ The `stop_review_app` job is **required** to have the following keywords defined - `stage` should be the same as the `review_app` in order for the environment to stop automatically when the branch is deleted -#### dynamic environments +### Dynamic environments > **Notes:** @@ -885,13 +664,204 @@ The common use case is to create dynamic environments for branches and use them as Review Apps. You can see a simple example using Review Apps at <https://gitlab.com/gitlab-examples/review-apps-nginx/>. -### artifacts +## `cache` + +> +**Notes:** +- Introduced in GitLab Runner v0.7.0. +- `cache` can be set globally and per-job. +- From GitLab 9.0, caching is enabled and shared between pipelines and jobs + by default. +- From GitLab 9.2, caches are restored before [artifacts](#artifacts). + +`cache` is used to specify a list of files and directories which should be +cached between jobs. You can only use paths that are within the project +workspace. + +If `cache` is defined outside the scope of jobs, it means it is set +globally and all jobs will use that definition. + +Cache all files in `binaries` and `.config`: + +```yaml +rspec: + script: test + cache: + paths: + - binaries/ + - .config +``` + +Cache all Git untracked files: + +```yaml +rspec: + script: test + cache: + untracked: true +``` + +Cache all Git untracked files and files in `binaries`: + +```yaml +rspec: + script: test + cache: + untracked: true + paths: + - binaries/ +``` + +Locally defined cache overrides globally defined options. The following `rspec` +job will cache only `binaries/`: + +```yaml +cache: + paths: + - my/files + +rspec: + script: test + cache: + key: rspec + paths: + - binaries/ +``` + +Note that since cache is shared between jobs, if you're using different +paths for different jobs, you should also set a different **cache:key** +otherwise cache content can be overwritten. + +NOTE: **Note:** +The cache is provided on a best-effort basis, so don't expect that the cache +will be always present. + +### `cache:key` + +> Introduced in GitLab Runner v1.0.0. + +The `key` directive allows you to define the affinity of caching +between jobs, allowing to have a single cache for all jobs, +cache per-job, cache per-branch or any other way that fits your needs. + +This way, you can fine tune caching, allowing you to cache data between +different jobs or even different branches. + +The `cache:key` variable can use any of the +[predefined variables](../variables/README.md), and the default key, if not set, +is set as `$CI_JOB_NAME-$CI_COMMIT_REF_NAME` which translates as "per-job and +per-branch". It is the default across the project, therefore everything is +shared between pipelines and jobs running on the same branch by default. + +NOTE: **Note:** +The `cache:key` variable cannot contain the `/` character, or the equivalent +URI-encoded `%2F`; a value made only of dots (`.`, `%2E`) is also forbidden. + +**Example configurations** + +To enable per-job caching: + +```yaml +cache: + key: "$CI_JOB_NAME" + untracked: true +``` + +To enable per-branch caching: + +```yaml +cache: + key: "$CI_COMMIT_REF_SLUG" + untracked: true +``` + +To enable per-job and per-branch caching: + +```yaml +cache: + key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG" + untracked: true +``` + +To enable per-branch and per-stage caching: + +```yaml +cache: + key: "$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG" + untracked: true +``` + +If you use **Windows Batch** to run your shell scripts you need to replace +`$` with `%`: + +```yaml +cache: + key: "%CI_JOB_STAGE%-%CI_COMMIT_REF_SLUG%" + untracked: true +``` + +If you use **Windows PowerShell** to run your shell scripts you need to replace +`$` with `$env:`: + +```yaml +cache: + key: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_SLUG" + untracked: true +``` + +### `cache:policy` + +> Introduced in GitLab 9.4. + +The default behaviour of a caching job is to download the files at the start of +execution, and to re-upload them at the end. This allows any changes made by the +job to be persisted for future runs, and is known as the `pull-push` cache +policy. + +If you know the job doesn't alter the cached files, you can skip the upload step +by setting `policy: pull` in the job specification. Typically, this would be +twinned with an ordinary cache job at an earlier stage to ensure the cache +is updated from time to time: + +```yaml +stages: + - setup + - test + +prepare: + stage: setup + cache: + key: gems + paths: + - vendor/bundle + script: + - bundle install --deployment + +rspec: + stage: test + cache: + key: gems + paths: + - vendor/bundle + policy: pull + script: + - bundle exec rspec ... +``` + +This helps to speed up job execution and reduce load on the cache server, +especially when you have a large number of cache-using jobs executing in +parallel. + +Additionally, if you have a job that unconditionally recreates the cache without +reference to its previous contents, you can use `policy: push` in that job to +skip the download step. + +## `artifacts` > **Notes:** - Introduced in GitLab Runner v0.7.0 for non-Windows platforms. - Windows support was added in GitLab Runner v.1.0.0. -- Prior to GitLab 9.2, caches were restored after artifacts. - From GitLab 9.2, caches are restored before artifacts. - Currently not all executors are supported. - Job artifacts are only collected for successful jobs by default. @@ -960,7 +930,9 @@ release-job: The artifacts will be sent to GitLab after the job finishes successfully and will be available for download in the GitLab UI. -#### artifacts:name +[Read more about artifacts.](../../user/project/pipelines/job_artifacts.md) + +### `artifacts:name` > Introduced in GitLab 8.6 and GitLab Runner v1.1.0. @@ -970,10 +942,6 @@ useful when you'd like to download the archive from GitLab. The `artifacts:name` variable can make use of any of the [predefined variables](../variables/README.md). The default name is `artifacts`, which becomes `artifacts.zip` when downloaded. ---- - -**Example configurations** - To create an archive with a name of the current job: ```yaml @@ -1033,7 +1001,7 @@ job: untracked: true ``` -#### artifacts:when +### `artifacts:when` > Introduced in GitLab 8.9 and GitLab Runner v1.3.0. @@ -1046,11 +1014,7 @@ failure. 1. `on_failure` - upload artifacts only when the job fails. 1. `always` - upload artifacts regardless of the job status. ---- - -**Example configurations** - -To upload artifacts only when job fails. +To upload artifacts only when job fails: ```yaml job: @@ -1058,22 +1022,23 @@ job: when: on_failure ``` -#### artifacts:expire_in +### `artifacts:expire_in` > Introduced in GitLab 8.9 and GitLab Runner v1.3.0. -`artifacts:expire_in` is used to delete uploaded artifacts after the specified -time. By default, artifacts are stored on GitLab forever. `expire_in` allows you -to specify how long artifacts should live before they expire, counting from the -time they are uploaded and stored on GitLab. +`expire_in` allows you to specify how long artifacts should live before they +expire and therefore deleted, counting from the time they are uploaded and +stored on GitLab. If the expiry time is not defined, it defaults to the +[instance wide setting](../../user/admin_area/settings/continuous_integration.md#default-artifacts-expiration) +(30 days by default, forever on GitLab.com). You can use the **Keep** button on the job page to override expiration and keep artifacts forever. -After expiry, artifacts are actually deleted hourly by default (via a cron job), -but they are not accessible after expiry. +After their expiry, artifacts are deleted hourly by default (via a cron job), +and are not accessible anymore. -The value of `expire_in` is an elapsed time. Examples of parseable values: +The value of `expire_in` is an elapsed time. Examples of parsable values: - '3 mins 4 sec' - '2 hrs 20 min' @@ -1082,10 +1047,6 @@ The value of `expire_in` is an elapsed time. Examples of parseable values: - '47 yrs 6 mos and 4d' - '3 weeks and 2 days' ---- - -**Example configurations** - To expire artifacts 1 week after being uploaded: ```yaml @@ -1094,7 +1055,7 @@ job: expire_in: 1 week ``` -### dependencies +## `dependencies` > Introduced in GitLab 8.6 and GitLab Runner v1.1.1. @@ -1153,7 +1114,7 @@ deploy: script: make deploy ``` -#### When a dependent job will fail +### When a dependent job will fail > Introduced in GitLab 10.3. @@ -1167,27 +1128,9 @@ You can ask your administrator to [flip this switch](../../administration/job_artifacts.md#validation-for-dependencies) and bring back the old behavior. -### before_script and after_script - -It's possible to overwrite the globally defined `before_script` and `after_script`: +## `coverage` -```yaml -before_script: -- global before script - -job: - before_script: - - execute this instead of global before script - script: - - my command - after_script: - - execute this after my script -``` - -### coverage - -**Notes:** -- [Introduced][ce-7447] in GitLab 8.17. +> [Introduced][ce-7447] in GitLab 8.17. `coverage` allows you to configure how code coverage will be extracted from the job output. @@ -1205,10 +1148,9 @@ job1: coverage: '/Code coverage: \d+\.\d+/' ``` -### retry +## `retry` -**Notes:** -- [Introduced][ce-3442] in GitLab 9.5. +> [Introduced][ce-3442] in GitLab 9.5. `retry` allows you to configure how many times a job is going to be retried in case of a failure. @@ -1228,16 +1170,57 @@ test: retry: 2 ``` -## Git Strategy +## `variables` + +> Introduced in GitLab Runner v0.5.0. + +NOTE: **Note:** +Integers (as well as strings) are legal both for variable's name and value. +Floats are not legal and cannot be used. + +GitLab CI/CD allows you to define variables inside `.gitlab-ci.yml` that are +then passed in the job environment. They can be set globally and per-job. +When the `variables` keyword is used on a job level, it overrides the global +YAML variables and predefined ones. + +They are stored in the Git repository and are meant to store non-sensitive +project configuration, for example: + +```yaml +variables: + DATABASE_URL: "postgres://postgres@postgres/my_database" +``` + +These variables can be later used in all executed commands and scripts. +The YAML-defined variables are also set to all created service containers, +thus allowing to fine tune them. + +To turn off global defined variables in a specific job, define an empty hash: + +```yaml +job_name: + variables: {} +``` + +Except for the user defined variables, there are also the ones [set up by the +Runner itself](../variables/README.md#predefined-variables-environment-variables). +One example would be `CI_COMMIT_REF_NAME` which has the value of +the branch or tag name for which project is built. Apart from the variables +you can set in `.gitlab-ci.yml`, there are also the so called +[secret variables](../variables/README.md#secret-variables) +which can be set in GitLab's UI. + +[Learn more about variables and their priority.][variables] + +### Git strategy > Introduced in GitLab 8.9 as an experimental feature. May change or be removed completely in future releases. `GIT_STRATEGY=none` requires GitLab Runner v1.7+. You can set the `GIT_STRATEGY` used for getting recent application code, either -in the global [`variables`](#variables) section or the [`variables`](#job-variables) -section for individual jobs. If left unspecified, the default from project -settings will be used. +globally or per-job in the [`variables`](#variables) section. If left +unspecified, the default from project settings will be used. There are three possible values: `clone`, `fetch`, and `none`. @@ -1269,44 +1252,13 @@ variables: GIT_STRATEGY: none ``` -## Git Checkout - -> Introduced in GitLab Runner 9.3 - -The `GIT_CHECKOUT` variable can be used when the `GIT_STRATEGY` is set to either -`clone` or `fetch` to specify whether a `git checkout` should be run. If not -specified, it defaults to true. Like `GIT_STRATEGY`, it can be set in either the -global [`variables`](#variables) section or the [`variables`](#job-variables) -section for individual jobs. - -If set to `false`, the Runner will: - -- when doing `fetch` - update the repository and leave working copy on - the current revision, -- when doing `clone` - clone the repository and leave working copy on the - default branch. - -Having this setting set to `true` will mean that for both `clone` and `fetch` -strategies the Runner will checkout the working copy to a revision related -to the CI pipeline: - -```yaml -variables: - GIT_STRATEGY: clone - GIT_CHECKOUT: "false" -script: - - git checkout master - - git merge $CI_BUILD_REF_NAME -``` - -## Git Submodule Strategy +### Git submodule strategy > Requires GitLab Runner v1.10+. The `GIT_SUBMODULE_STRATEGY` variable is used to control if / how Git -submodules are included when fetching the code before a build. Like -`GIT_STRATEGY`, it can be set in either the global [`variables`](#variables) -section or the [`variables`](#job-variables) section for individual jobs. +submodules are included when fetching the code before a build. You can set them +globally or per-job in the [`variables`](#variables) section. There are three possible values: `none`, `normal`, and `recursive`: @@ -1336,8 +1288,36 @@ Note that for this feature to work correctly, the submodules must be configured - a relative path to another repository on the same GitLab server. See the [Git submodules](../git_submodules.md) documentation. +### Git checkout + +> Introduced in GitLab Runner 9.3 + +The `GIT_CHECKOUT` variable can be used when the `GIT_STRATEGY` is set to either +`clone` or `fetch` to specify whether a `git checkout` should be run. If not +specified, it defaults to true. You can set them globally or per-job in the +[`variables`](#variables) section. + +If set to `false`, the Runner will: + +- when doing `fetch` - update the repository and leave working copy on + the current revision, +- when doing `clone` - clone the repository and leave working copy on the + default branch. + +Having this setting set to `true` will mean that for both `clone` and `fetch` +strategies the Runner will checkout the working copy to a revision related +to the CI pipeline: -## Job stages attempts +```yaml +variables: + GIT_STRATEGY: clone + GIT_CHECKOUT: "false" +script: + - git checkout master + - git merge $CI_BUILD_REF_NAME +``` + +### Job stages attempts > Introduced in GitLab, it requires GitLab Runner v1.9+. @@ -1359,10 +1339,9 @@ variables: GET_SOURCES_ATTEMPTS: 3 ``` -You can set them in the global [`variables`](#variables) section or the -[`variables`](#job-variables) section for individual jobs. +You can set them globally or per-job in the [`variables`](#variables) section. -## Shallow cloning +### Shallow cloning > Introduced in GitLab 8.9 as an experimental feature. May change in future releases or be removed completely. @@ -1393,7 +1372,17 @@ variables: GIT_DEPTH: "3" ``` -## Hidden keys (jobs) +You can set it globally or per-job in the [`variables`](#variables) section. + +## Special YAML features + +It's possible to use special YAML features like anchors (`&`), aliases (`*`) +and map merging (`<<`), which will allow you to greatly reduce the complexity +of `.gitlab-ci.yml`. + +Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/). + +### Hidden keys (jobs) > Introduced in GitLab 8.6 and GitLab Runner v1.1.1. @@ -1419,14 +1408,6 @@ Use this feature to ignore jobs, or use the [special YAML features](#special-yaml-features) and transform the hidden keys into templates. -## Special YAML features - -It's possible to use special YAML features like anchors (`&`), aliases (`*`) -and map merging (`<<`), which will allow you to greatly reduce the complexity -of `.gitlab-ci.yml`. - -Read more about the various [YAML features](https://learnxinyminutes.com/docs/yaml/). - ### Anchors > Introduced in GitLab 8.6 and GitLab Runner v1.1.1. @@ -1556,34 +1537,10 @@ with an API call. [Read more in the triggers documentation.](../triggers/README.md) -### pages - -`pages` is a special job that is used to upload static content to GitLab that -can be used to serve your website. It has a special syntax, so the two -requirements below must be met: - -1. Any static content must be placed under a `public/` directory -1. `artifacts` with a path to the `public/` directory must be defined - -The example below simply moves all files from the root of the project to the -`public/` directory. The `.public` workaround is so `cp` doesn't also copy -`public/` to itself in an infinite loop: - -``` -pages: - stage: deploy - script: - - mkdir .public - - cp -r * .public - - mv .public public - artifacts: - paths: - - public - only: - - master -``` +## Skipping jobs -Read more on [GitLab Pages user documentation](../../user/project/pages/index.md). +If your commit message contains `[ci skip]` or `[skip ci]`, using any +capitalization, the commit will be created but the pipeline will be skipped. ## Validate the .gitlab-ci.yml @@ -1595,11 +1552,6 @@ You can find the link under `/ci/lint` of your gitlab instance. If you get validation error when using specific values (e.g., `true` or `false`), try to quote them, or change them to a different form (e.g., `/bin/true`). -## Skipping jobs - -If your commit message contains `[ci skip]` or `[skip ci]`, using any -capitalization, the commit will be created but the jobs will be skipped. - ## Examples Visit the [examples README][examples] to see a list of examples using GitLab diff --git a/doc/development/automatic_ce_ee_merge.md b/doc/development/automatic_ce_ee_merge.md index cf6314f9521..a85e5b1b1cc 100644 --- a/doc/development/automatic_ce_ee_merge.md +++ b/doc/development/automatic_ce_ee_merge.md @@ -103,6 +103,99 @@ Notes: - You can use [`git rerere`](https://git-scm.com/blog/2010/03/08/rerere.html) to avoid resolving the same conflicts multiple times. +### Cherry-picking from CE to EE + +For avoiding merge conflicts, we use a method of creating equivalent branches +for CE and EE. If the `ee-compat-check` job fails, this process is required. + +This method only requires that you have cloned both CE and EE into your computer. +If you don't have them yet, please go ahead and clone them: + +- Clone CE repo: `git clone git@gitlab.com:gitlab-org/gitlab-ce.git` +- Clone EE repo: `git clone git@gitlab.com:gitlab-org/gitlab-ee.git` + +And the only additional setup we need is to add CE as remote of EE and vice-versa: + +- Open two terminal windows, one in CE, and another one in EE: + - In EE: `git remote add ce git@gitlab.com:gitlab-org/gitlab-ce.git` + - In CE: `git remote add ee git@gitlab.com:gitlab-org/gitlab-ee.git` + +That's all setup we need, so that we can cherry-pick a commit from CE to EE, and +from EE to CE. + +Now, every time you create an MR for CE and EE: + +1. Open two terminal windows, one in CE, and another one in EE +1. In the CE terminal: + 1. Create the CE branch, e.g., `branch-example` + 1. Make your changes and push a commit (commit A) + 1. Create the CE merge request in GitLab +1. In the EE terminal: + 1. Create the EE-equivalent branch ending with `-ee`, e.g., + `git checkout -b branch-example-ee` + 1. Fetch the CE branch: `git fetch ce branch-example` + 1. Cherry-pick the commit A: `git cherry-pick commit-A-SHA` + 1. If Git prompts you to fix the conflicts, do a `git status` + to check which files contain conflicts, fix them, save the files + 1. Add the changes with `git add .` but **DO NOT commit** them + 1. Continue cherry-picking: `git cherry-pick --continue` + 1. Push to EE: `git push origin branch-example-ee` +1. Create the EE-equivalent MR and link to the CE MR from the +description "Ports [CE-MR-LINK] to EE" +1. Once all the jobs are passing in both CE and EE, you've addressed the +feedback from your own team, and got them approved, the merge requests can be merged. +1. When both MRs are ready, the EE merge request will be merged first, and the +CE-equivalent will be merged next. + +**Important notes:** + +- The commit SHA can be easily found from the GitLab UI. From a merge request, +open the tab **Commits** and click the copy icon to copy the commit SHA. +- To cherry-pick a **commit range**, such as [A > B > C > D] use: + + ```shell + git cherry-pick "oldest-commit-SHA^..newest-commit-SHA" + ``` + + For example, suppose the commit A is the oldest, and its SHA is `4f5e4018c09ed797fdf446b3752f82e46f5af502`, + and the commit D is the newest, and its SHA is `80e1c9e56783bd57bd7129828ec20b252ebc0538`. + The cherry-pick command will be: + + ```shell + git cherry-pick "4f5e4018c09ed797fdf446b3752f82e46f5af502^..80e1c9e56783bd57bd7129828ec20b252ebc0538" + ``` + +- To cherry-pick a **merge commit**, use the flag `-m 1`. For example, suppose that the +merge commit SHA is `138f5e2f20289bb376caffa0303adb0cac859ce1`: + + ```shell + git cherry-pick -m 1 138f5e2f20289bb376caffa0303adb0cac859ce1 + ``` +- To cherry-pick multiple commits, such as B and D in a range [A > B > C > D], use: + + ```shell + git cherry-pick commmit-B-SHA commit-D-SHA + ``` + + For example, suppose commit B SHA = `4f5e4018c09ed797fdf446b3752f82e46f5af502`, + and the commit D SHA = `80e1c9e56783bd57bd7129828ec20b252ebc0538`. + The cherry-pick command will be: + + ```shell + git cherry-pick 4f5e4018c09ed797fdf446b3752f82e46f5af502 80e1c9e56783bd57bd7129828ec20b252ebc0538 + ``` + + This case is particularly useful when you have a merge commit in a sequence of + commits and you want to cherry-pick all but the merge commit. + +- If you push more commits to the CE branch, you can safely repeat the procedure +to cherry-pick them to the EE-equivalent branch. You can do that as many times as +necessary, using the same CE and EE branches. +- If you submitted the merge request to the CE repo and the `ee-compat-check` job passed, +you are not required to submit the EE-equivalent MR, but it's still recommended. If the +job failed, you are required to submit the EE MR so that you can fix the conflicts in EE +before merging your changes into CE. + --- [Return to Development documentation](README.md) diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index c0ce49eb40b..856ef882453 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -45,7 +45,7 @@ We basically have 4 types of files: 1. Ruby files: basically Models and Controllers. 1. HAML files: these are the view files. 1. ERB files: used for email templates. -1. JavaScript files: we mostly need to work with VUE JS templates. +1. JavaScript files: we mostly need to work with Vue templates. ### Ruby files diff --git a/doc/development/new_fe_guide/dependencies.md b/doc/development/new_fe_guide/dependencies.md new file mode 100644 index 00000000000..3417d77a06d --- /dev/null +++ b/doc/development/new_fe_guide/dependencies.md @@ -0,0 +1,3 @@ +# Dependencies + +> TODO: Add Dependencies
\ No newline at end of file diff --git a/doc/development/new_fe_guide/development/accessibility.md b/doc/development/new_fe_guide/development/accessibility.md new file mode 100644 index 00000000000..ed35f08432f --- /dev/null +++ b/doc/development/new_fe_guide/development/accessibility.md @@ -0,0 +1,3 @@ +# Accessibility + +> TODO: Add content diff --git a/doc/development/new_fe_guide/development/components.md b/doc/development/new_fe_guide/development/components.md new file mode 100644 index 00000000000..637099d1e83 --- /dev/null +++ b/doc/development/new_fe_guide/development/components.md @@ -0,0 +1,3 @@ +# Components + +> TODO: Add content diff --git a/doc/development/new_fe_guide/development/design_patterns.md b/doc/development/new_fe_guide/development/design_patterns.md new file mode 100644 index 00000000000..ee06566ed30 --- /dev/null +++ b/doc/development/new_fe_guide/development/design_patterns.md @@ -0,0 +1,3 @@ +# Design patterns + +> TODO: Add content diff --git a/doc/development/new_fe_guide/development/index.md b/doc/development/new_fe_guide/development/index.md new file mode 100644 index 00000000000..cee8e43ebad --- /dev/null +++ b/doc/development/new_fe_guide/development/index.md @@ -0,0 +1,29 @@ +# Development + +## [Design patterns](design_patterns.md) + +Examples of proven design patterns used in our codebase. + +## [Components](components.md) + +Documentation on existing components and how to best create a new component. + +## [Accessibility](accessibility.md) + +Learn how to implement an accessible frontend. + +## [Network requests](network_requests.md) + +Learn how to handle network requests in our codebase. + +## [Security](security.md) + +Learn how to ensure that our frontend is secure. + +## [Performance](performance.md) + +Learn how to keep our frontend performant. + +## [Testing](testing.md) + +Learn how to keep our frontend tested. diff --git a/doc/development/new_fe_guide/development/network_requests.md b/doc/development/new_fe_guide/development/network_requests.md new file mode 100644 index 00000000000..047c00313bc --- /dev/null +++ b/doc/development/new_fe_guide/development/network_requests.md @@ -0,0 +1,3 @@ +# Network requests + +> TODO: Add content diff --git a/doc/development/new_fe_guide/development/performance.md b/doc/development/new_fe_guide/development/performance.md new file mode 100644 index 00000000000..26b07874f0f --- /dev/null +++ b/doc/development/new_fe_guide/development/performance.md @@ -0,0 +1,3 @@ +# Performance + +> TODO: Add content diff --git a/doc/development/new_fe_guide/development/security.md b/doc/development/new_fe_guide/development/security.md new file mode 100644 index 00000000000..debda7de0c6 --- /dev/null +++ b/doc/development/new_fe_guide/development/security.md @@ -0,0 +1,3 @@ +# Security + +> TODO: Add content diff --git a/doc/development/new_fe_guide/development/testing.md b/doc/development/new_fe_guide/development/testing.md new file mode 100644 index 00000000000..c359bd83ed1 --- /dev/null +++ b/doc/development/new_fe_guide/development/testing.md @@ -0,0 +1,3 @@ +# Testing + +> TODO: Add content diff --git a/doc/development/new_fe_guide/index.md b/doc/development/new_fe_guide/index.md new file mode 100644 index 00000000000..08c6a266e7f --- /dev/null +++ b/doc/development/new_fe_guide/index.md @@ -0,0 +1,28 @@ +# Frontend Development Guidelines + +This guide contains all the information to successfully contribute to GitLab's frontend. +This is a living document, and we welcome contributions, feedback and suggestions. + +## [Principles](principles.md) + +Ensure that your frontend contribution starts off in the right direction. + +## [Initiatives](initiatives.md) + +High level overview of where we are going from a frontend perspective. + +## [Development](development/index.md) + +Guidance on topics related to development. + +## [Dependencies](dependencies.md) + +Learn about all the dependencies that make up our frontend, including some of our own custom built libraries. + +## [Style](style/index.md) + +Style guides to keep our code consistent. + +## [Tips](tips.md) + +Tips from our frontend team to develop more efficiently and effectively. diff --git a/doc/development/new_fe_guide/initiatives.md b/doc/development/new_fe_guide/initiatives.md new file mode 100644 index 00000000000..c81ed3579f0 --- /dev/null +++ b/doc/development/new_fe_guide/initiatives.md @@ -0,0 +1,3 @@ +# Initiatives + +> TODO: Add Initiatives diff --git a/doc/development/new_fe_guide/principles.md b/doc/development/new_fe_guide/principles.md new file mode 100644 index 00000000000..2126d202a7e --- /dev/null +++ b/doc/development/new_fe_guide/principles.md @@ -0,0 +1,3 @@ +# Principles + +> TODO: Add principles diff --git a/doc/development/new_fe_guide/style/html.md b/doc/development/new_fe_guide/style/html.md new file mode 100644 index 00000000000..5489def5d6e --- /dev/null +++ b/doc/development/new_fe_guide/style/html.md @@ -0,0 +1,3 @@ +# HTML style guide + +> TODO: Add content diff --git a/doc/development/new_fe_guide/style/index.md b/doc/development/new_fe_guide/style/index.md new file mode 100644 index 00000000000..d2d576b3b46 --- /dev/null +++ b/doc/development/new_fe_guide/style/index.md @@ -0,0 +1,9 @@ +# Style + +## [HTML style guide](html.md) + +## [SCSS style guide](scss.md) + +## [JavaScript style guide](javascript.md) + +## [Vue style guide](vue.md) diff --git a/doc/development/new_fe_guide/style/javascript.md b/doc/development/new_fe_guide/style/javascript.md new file mode 100644 index 00000000000..480d50a211f --- /dev/null +++ b/doc/development/new_fe_guide/style/javascript.md @@ -0,0 +1,3 @@ +# JavaScript style guide + +> TODO: Add content diff --git a/doc/development/new_fe_guide/style/scss.md b/doc/development/new_fe_guide/style/scss.md new file mode 100644 index 00000000000..6f5e818d7db --- /dev/null +++ b/doc/development/new_fe_guide/style/scss.md @@ -0,0 +1,3 @@ +# SCSS style guide + +> TODO: Add content diff --git a/doc/development/new_fe_guide/style/vue.md b/doc/development/new_fe_guide/style/vue.md new file mode 100644 index 00000000000..fd9353e0d3f --- /dev/null +++ b/doc/development/new_fe_guide/style/vue.md @@ -0,0 +1,3 @@ +# Vue style guide + +> TODO: Add content diff --git a/doc/development/new_fe_guide/tips.md b/doc/development/new_fe_guide/tips.md new file mode 100644 index 00000000000..f0cdf52d618 --- /dev/null +++ b/doc/development/new_fe_guide/tips.md @@ -0,0 +1,3 @@ +# Tips + +> TODO: Add tips diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 403c9d08752..40c21e5355c 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -19,7 +19,7 @@ The one responsible for writing the first piece of documentation is the develope wrote the code. It's the job of the Product Manager to ensure all features are shipped with its docs, whether is a small or big change. At the pace GitLab evolves, this is the only way to keep the docs up-to-date. If you have any questions about it, -please ask a Technical Writer. Otherwise, when your content is ready, assign one of +ask a Technical Writer. Otherwise, when your content is ready, assign one of them to review it for you. We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything @@ -27,6 +27,8 @@ is documented. Whenever you submit a merge request for the documentation, use the documentation MR description template. +Please check the [documentation workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) before getting started. + ### Documentation directory structure The documentation is structured based on the GitLab UI structure itself, @@ -40,7 +42,7 @@ all docs should be linked. Every new document should be cross-linked to its rela The directories `/workflow/`, `/gitlab-basics/`, `/university/`, and `/articles/` have been deprecated and the majority their docs have been moved to their correct location -in small iterations. Please don't create new docs in these folders. +in small iterations. Don't create new docs in these folders. To move a document from its location to another directory, read the section [changing document location](doc_styleguide.md#changing-document-location) of the doc style guide. @@ -116,6 +118,49 @@ choices: If your branch name matches any of the above, it will run only the docs tests. If it doesn't, the whole test suite will run (including docs). +### Merge requests for GitLab documentation + +Before getting started, make sure you read the introductory section +"[contributing to docs](#contributing-to-docs)" above and the +[tech writing workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) +for GitLab Team members. + +- Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/merge_request_templates/Documentation.md) +- Use the correct [branch name](#branch-naming) +- Label the MR `Documentation` +- Assign the correct milestone (see note below) + + +NOTE: **Note:** +If the release version you want to add the documentation to has already been +frozen or released, use the label `Pick into X.Y` to get it merged into +the correct release. Avoid picking into a past release as much as you can, as +it increases the work of the release managers. + +#### Cherry-picking from CE to EE + +As we have the `master` branch of CE merged into EE once a day, it's common to +run into merge conflicts. To avoid them, we [test for merge conflicts against EE](#testing) +with the `ee-compat-check` job, and use the following method of creating equivalent +branches for CE and EE. + +Follow this [method for cherry-picking from CE to EE](automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee), with a few adjustments: + +- Create the [CE branch](#branch-naming) starting with `docs-`, + e.g.: `git checkout -b docs-example` +- Create the EE-equivalent branch ending with `-ee`, e.g., + `git checkout -b docs-example-ee` +- Once all the jobs are passing in CE and EE, and you've addressed the +feedback from your own team, assign the CE MR to a technical writer for review +- When both MRs are ready, the EE merge request will be merged first, and the +CE-equivalent will be merged next. +- Note that the review will occur only in the CE MR, as the EE MR +contains the same commits as the CE MR. +- If you have a few more changes that apply to the EE-version only, you can submit +a couple more commits to the EE branch, but ask the reviewer to review the EE merge request +additionally to the CE MR. If there are many EE-only changes though, start a new MR +to EE only. + ### Previewing the changes live If you want to preview the doc changes of your merge request live, you can use diff --git a/doc/install/installation.md b/doc/install/installation.md index 170d92faa09..6eb767b00b3 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -299,9 +299,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-5-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-6-stable gitlab -**Note:** You can change `10-5-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `10-6-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It diff --git a/doc/install/requirements.md b/doc/install/requirements.md index b2c9177e6eb..1f2b4d9d3d9 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -137,6 +137,14 @@ CREATE EXTENSION pg_trgm; On some systems you may need to install an additional package (e.g. `postgresql-contrib`) for this extension to become available. +#### Additional requirements for GitLab Geo + +If you are using [GitLab Geo](https://docs.gitlab.com/ee/development/geo.html), the [tracking database](https://docs.gitlab.com/ee/development/geo.html#geo-tracking-database) also requires the `postgres_fdw` extension. + +``` +CREATE EXTENSION postgres_fdw; +``` + ## Unicorn Workers It's possible to increase the amount of unicorn workers and this will usually help to reduce the response time of the applications and increase the ability to handle parallel requests. diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 33a2d7a88a7..aa14a39e4c9 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -35,8 +35,8 @@ to clipboard step. If you don't see the string or would like to generate a SSH key pair with a custom name continue onto the next step. -> -**Note:** Public SSH key may also be named as follows: +Note that Public SSH key may also be named as follows: + - `id_dsa.pub` - `id_ecdsa.pub` - `id_ed25519.pub` @@ -73,7 +73,7 @@ custom name continue onto the next step. key pair, but it is not required and you can skip creating a password by pressing enter. - >**Note:** + NOTE: **Note:** If you want to change the password of your SSH key pair, you can use `ssh-keygen -p <keyname>`. @@ -162,11 +162,13 @@ That's why it needs to uniquely map to a single user. ## Deploy keys +### Per-repository deploy keys + Deploy keys allow read-only or read-write (if enabled) access to one or multiple projects with a single SSH key pair. This is really useful for cloning repositories to your Continuous -Integration (CI) server. By using deploy keys, you don't have to setup a +Integration (CI) server. By using deploy keys, you don't have to set up a dummy user account. If you are a project master or owner, you can add a deploy key in the @@ -185,6 +187,47 @@ a group. Deploy keys can be shared between projects, you just need to add them to each project. +### Global shared deploy keys + +Global Shared Deploy keys allow read-only or read-write (if enabled) access to +be configured on any repository in the entire GitLab installation. + +This is really useful for integrating repositories to secured, shared Continuous +Integration (CI) services or other shared services. +GitLab administrators can set up the Global Shared Deploy key in GitLab and +add the private key to any shared systems. Individual repositories opt into +exposing their repsitory using these keys when a project masters (or higher) +authorizes a Global Shared Deploy key to be used with their project. + +Global Shared Keys can provide greater security compared to Per-Project Deploy +Keys since an administrator of the target integrated system is the only one +who needs to know and configure the private key. + +GitLab administrators set up Global Deploy keys in the Admin area under the +section **Deploy Keys**. Ensure keys have a meaningful title as that will be +the primary way for project masters and owners to identify the correct Global +Deploy key to add. For instance, if the key gives access to a SaaS CI instance, +use the name of that service in the key name if that is all it is used for. +When creating Global Shared Deploy keys, give some thought to the granularity +of keys - they could be of very narrow usage such as just a specific service or +of broader usage for something like "Anywhere you need to give read access to +your repository". + +Once a GitLab administrator adds the Global Deployment key, project masters +and owners can add it in project's **Settings > Repository** section by expanding the +**Deploy Key** section and clicking **Enable** next to the appropriate key listed +under **Public deploy keys available to any project**. + +NOTE: **Note:** +The heading **Public deploy keys available to any project** only appears +if there is at least one Global Deploy Key configured. + +CAUTION: **Warning:** +Defining Global Deploy Keys does not expose any given repository via +the key until that respository adds the Global Deploy Key to their project. +In this way the Global Deploy Keys enable access by other systems, but do +not implicitly give any access just by setting them up. + ## Applications ### Eclipse diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 5f5ba2b69bc..ec091549c05 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -309,6 +309,18 @@ enable them. You can make use of [environment variables](#helm-chart-variables) to automatically scale your pod replicas. +It's important to note that when a project is deployed to a Kubernetes cluster, +it relies on a Docker image that has been pushed to the +[GitLab Container Registry](../../user/project/container_registry.md). Kubernetes +fetches this image and uses it to run the application. If the project is public, +the image can be accessed by Kubernetes without any authentication, allowing us +to have deployments more usable. If the project is private/internal, the +Registry requires credentials to pull the image. Currently, this is addressed +by providing `CI_JOB_TOKEN` as the password that can be used, but this token will +no longer be valid as soon as the deployment job finishes. This means that +Kubernetes can run the application, but in case it should be restarted or +executed somewhere else, it cannot be accessed again. + ### Auto Monitoring NOTE: **Note:** diff --git a/doc/update/10.5-to-10.6.md b/doc/update/10.5-to-10.6.md new file mode 100644 index 00000000000..af8343b5958 --- /dev/null +++ b/doc/update/10.5-to-10.6.md @@ -0,0 +1,361 @@ +--- +comments: false +--- + +# From 10.5 to 10.6 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +NOTE: If you installed GitLab from source, make sure `rsync` is installed. + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.6.tar.gz +echo '4e6a0f828819e15d274ae58485585fc8b7caace0 ruby-2.3.6.tar.gz' | shasum -c - && tar xzf ruby-2.3.6.tar.gz +cd ruby-2.3.6 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets. +We require a minimum version of node v6.0.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v6.0.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-6-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-6-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-5-stable:config/gitlab.yml.example origin/10-6-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-5-stable:lib/support/nginx/gitlab-ssl origin/10-6-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-5-stable:lib/support/nginx/gitlab origin/10-6-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-6-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-6-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-5-stable:lib/support/init.d/gitlab.default.example origin/10-6-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (10.5) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 10.4 to 10.5](10.4-to-10.5.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-6-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-6-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/user/admin_area/settings/img/update-available.png b/doc/user/admin_area/settings/img/update-available.png Binary files differnew file mode 100644 index 00000000000..0dafdad618e --- /dev/null +++ b/doc/user/admin_area/settings/img/update-available.png diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index d874688cc29..381efdf5d67 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -8,20 +8,26 @@ under **Admin area > Settings > Usage statistics**. ## Version check -GitLab can inform you when an update is available and the importance of it. +If enabled, version check will inform you if a new version is available and the +importance of it through a status. This is shown on the help page (i.e. `/help`) +for all signed in users, and on the admin pages. The statuses are: -No information other than the GitLab version and the instance's hostname (through the HTTP referer) -are collected. +* Green: You are running the latest version of GitLab. +* Orange: An updated version of GitLab is available. +* Red: The version of GitLab you are running is vulnerable. You should install + the latest version with security fixes as soon as possible. -In the **Overview** tab you can see if your GitLab version is up to date. There -are three cases: 1) you are up to date (green), 2) there is an update available -(yellow) and 3) your version is vulnerable and a security fix is released (red). + -In any case, you will see a message informing you of the state and the -importance of the update. +GitLab Inc. collects your instance's version and hostname (through the HTTP +referer) as part of the version check. No other information is collected. -If enabled, the version status will also be shown in the help page (`/help`) -for all signed in users. +This information is used, among other things, to identify to which versions +patches will need to be back ported, making sure active GitLab instances remain +secure. + +If you disable version check, this information will not be collected. Enable or +disable the version check at **Admin area > Settings > Usage statistics**. ## Usage ping diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 914a80bcd6a..a520279c29e 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -25,7 +25,8 @@ The following table depicts the various user permission levels in a project. | Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | | View confidential issues | (✓) [^2] | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| Lock discussions (issues and merge requests) | | | | ✓ | ✓ | +| Lock issue discussions | | ✓ | ✓ | ✓ | ✓ | +| Lock merge request discussions | | | ✓ | ✓ | ✓ | | See a list of jobs | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | | See a job log | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | | Download and browse job artifacts | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 4ac54f96aa2..661697aaeb7 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -109,6 +109,41 @@ you will be notified. You can now proceed to install some pre-defined applications and then enable the Kubernetes cluster integration. +## Security implications + +CAUTION: **Important:** +The whole cluster security is based on a model where [developers](../../permissions.md) +are trusted, so **only trusted users should be allowed to control your clusters**. + +The default cluster configuration grants access to a wide set of +functionalities needed to successfully build and deploy a containerized +application. Bare in mind that the same credentials are used for all the +applications running on the cluster. + +When GitLab creates the cluster, it enables and uses the legacy +[Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/). +The newer [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/) +authorization will be supported in a +[future release](https://gitlab.com/gitlab-org/gitlab-ce/issues/29398). + +### Security of GitLab Runners + +GitLab Runners have the [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#the-privileged-mode) +enabled by default, which allows them to execute special commands and running +Docker in Docker. This functionality is needed to run some of the [Auto DevOps] +jobs. This implies the containers are running in privileged mode and you should, +therefore, be aware of some important details. + +The privileged flag gives all capabilities to the running container, which in +turn can do almost everything that the host can do. Be aware of the +inherent security risk associated with performing `docker run` operations on +arbitrary images as they effectively have root access. + +If you don't want to use GitLab Runner in privileged mode, first make sure that +you don't have it installed via the applications, and then use the +[Runner's Helm chart](../../../install/kubernetes/gitlab_runner_chart.md) to +install it manually. + ## Installing applications GitLab provides a one-click install for various applications which will be @@ -118,16 +153,16 @@ added directly to your configured cluster. Those applications are needed for | Application | GitLab version | Description | | ----------- | :------------: | ----------- | | [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | +| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps] or deploy your own web apps. | | [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications | -| [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI](https://about.gitlab.com/features/gitlab-ci-cd/), the open-source continuous integration service included with GitLab that coordinates the jobs. | +| [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](#security-implications) before doing so. | ## Getting the external IP address NOTE: **Note:** You need a load balancer installed in your cluster in order to obtain the external IP address with the following procedure. It can be deployed using the -[**Ingress** application](#installing-appplications). +[**Ingress** application](#installing-applications). In order to publish your web application, you first need to find the external IP address associated to your load balancer. @@ -329,3 +364,4 @@ the deployment variables above, ensuring any pods you create are labelled with [permissions]: ../../permissions.md [ee]: https://about.gitlab.com/products/ +[Auto DevOps]: ../../../topics/autodevops/index.md diff --git a/doc/user/project/import/img/import_projects_from_repo_url.png b/doc/user/project/import/img/import_projects_from_repo_url.png Binary files differnew file mode 100644 index 00000000000..ec867da1087 --- /dev/null +++ b/doc/user/project/import/img/import_projects_from_repo_url.png diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index e2b285678c3..72cc58546b7 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -10,6 +10,7 @@ 1. [From Perforce](perforce.md) 1. [From SVN](svn.md) 1. [From TFS](tfs.md) +1. [From repo by URL](repo_by_url.md) In addition to the specific migration documentation above, you can import any Git repository via HTTP from the New Project page. Be aware that if the diff --git a/doc/user/project/import/perforce.md b/doc/user/project/import/perforce.md index aa7508e1e8e..a1ea716b606 100644 --- a/doc/user/project/import/perforce.md +++ b/doc/user/project/import/perforce.md @@ -48,3 +48,9 @@ Here's a few links to get you started: - [git-p4 manual page](https://www.kernel.org/pub/software/scm/git/docs/git-p4.html) - [git-p4 example usage](https://git.wiki.kernel.org/index.php/Git-p4_Usage) - [Git book migration guide](https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git#_perforce_import) + +Note that `git p4` and `git filter-branch` are not very good at +creating small and efficient Git pack files. So it might be a good +idea to spend time and CPU to properly repack your repository before +sending it for the first time to your GitLab server. See +[this StackOverflow question](https://stackoverflow.com/questions/28720151/git-gc-aggressive-vs-git-repack/). diff --git a/doc/user/project/import/repo_by_url.md b/doc/user/project/import/repo_by_url.md new file mode 100644 index 00000000000..f43e384de88 --- /dev/null +++ b/doc/user/project/import/repo_by_url.md @@ -0,0 +1,12 @@ +# Import project from repo by URL + +You can import your existing repositories by providing the Git URL: + +1. From your GitLab dashboard click **New project** +1. Switch to the **Import project** tab +1. Click on the **Repo by URL** button +1. Fill in the "Git repository URL" and the remaining project fields +1. Click **Create project** to being the import process +1. Once complete, you will be redirected to your newly created project + + diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 02adc562028..8ac753c07bf 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -13,21 +13,18 @@ integration services must be enabled. | Name | Query | | ---- | ----- | -| Average Memory Usage (MB) | (sum(avg(container_memory_usage_bytes{container_name!="POD",environment="%{ci_environment_slug}"}) without (job))) / count(avg(container_memory_usage_bytes{container_name!="POD",environment="%{ci_environment_slug}"}) without (job)) /1024/1024 | -| Average CPU Utilization (%) | sum(avg(rate(container_cpu_usage_seconds_total{container_name!="POD",environment="%{ci_environment_slug}"}[2m])) without (job)) * 100 | +| Average Memory Usage (MB) | avg(sum(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}) by (job)) without (job) / count(avg(container_memory_usage_bytes{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}) without (job)) /1024/1024 | +| Average CPU Utilization (%) | avg(sum(rate(container_cpu_usage_seconds_total{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}[15m])) by (job)) without (job) / count(sum(rate(container_cpu_usage_seconds_total{container_name!="POD",pod_name=~"^%{ci_environment_slug}-([^c].*|c([^a]|a([^n]|n([^a]|a([^r]|r[^y])))).*|)-(.*)",namespace="%{kube_namespace}"}[15m])) by (pod_name)) | -## Configuring Prometheus to monitor for Kubernetes node metrics +## Configuring Prometheus to monitor for Kubernetes metrics -In order for Prometheus to collect Kubernetes metrics, you first must have a -Prometheus server up and running. You have two options here: +Prometheus needs to be deployed into the cluster and configured properly in order to gather Kubernetes metrics. GitLab supports two methods for doing so: -- If you have an Omnibus based GitLab installation within your Kubernetes cluster, you can leverage the bundled Prometheus server to [monitor Kubernetes](../../../../administration/monitoring/prometheus/index.md#configuring-prometheus-to-monitor-kubernetes). -- To configure your own Prometheus server, you can follow the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/) or [our guide](../../../../administration/monitoring/prometheus/index.md#configuring-your-own-prometheus-server-within-kubernetes). +- GitLab [integrates with Kubernetes](../../clusters/index.md), and can [deploy Prometheus into a connected cluster](../prometheus.html#managed-prometheus-on-kubernetes). It is automatically configured to collect Kubernetes metrics. +- To configure your own Prometheus server, you can follow the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/). ## Specifying the Environment In order to isolate and only display relevant CPU and Memory metrics for a given environment, GitLab needs a method to detect which containers it is running. Because these metrics are tracked at the container level, traditional Kubernetes labels are not available. Instead, the [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) or [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) name should begin with [CI_ENVIRONMENT_SLUG](../../../../ci/variables/README.md#predefined-variables-environment-variables). It can be followed by a `-` and additional content if desired. For example, a deployment name of `review-homepage-5620p5` would match the `review/homepage` environment. - -If you are using [GitLab Auto-Deploy](../../../../ci/autodeploy/index.md) and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index bc6306927e1..d403d5698a9 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -235,6 +235,27 @@ to another list the label changes and a system not is recorded. [Developers and up](../permissions.md) can use all the functionality of the Issue Board, that is create/delete lists and drag issues around. +## Group Issue Board + +>Introduced in GitLab 10.6 + +Group issue board is analogous to project-level issue board and it is accessible at the group +navigation level. A group-level issue board allows you to view all issues from all projects in that group +(currently, it does not see issues from projects in subgroups). Similarly, you can only filter by group labels for these +boards. When updating milestones and labels for an issue through the sidebar update mechanism, again only +group-level objects are available. + +## Features per tier + +Different issue board features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), as shown in the following table: + +| Tier | Number of project issue boards | Board with configuration in project issue boards | Number of group issue boards | Board with configuration in group issue boards | +| --- | --- | --- | --- | --- | +| Libre | 1 | No | 1 | No | +| Starter | Multiple | Yes | 1 | No | +| Premium | Multiple | Yes | Multiple | Yes | +| Ultimate | Multiple | Yes | Multiple | Yes | + ## Tips A few things to remember: diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index f5c748a03b3..5d819998dd9 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -16,19 +16,29 @@ say 'Project Acme', in GitLab is to make the 'Engineering' group the owner of 'P Acme'. But what if 'Project Acme' already belongs to another group, say 'Open Source'? This is where the group sharing feature can be of use. -To share 'Project Acme' with the 'Engineering' group, go to the project settings page for 'Project Acme' and use the left navigation menu to go to the **Settings > Members** section. +To share 'Project Acme' with the 'Engineering' group: - +1. For 'Project Acme' use the left navigation menu to go to **Settings > Members** -Then select the 'Share with group' tab by clicking it. +  -Now you can add the 'Engineering' group with the maximum access level of your choice. Click 'Share' to share it. +1. Select the 'Share with group' tab +1. Add the 'Engineering' group with the maximum access level of your choice +1. Click **Share** to share it - +  -After sharing 'Project Acme' with 'Engineering', the project will be listed on the group dashboard. +1. After sharing 'Project Acme' with 'Engineering', the project will be listed + on the group dashboard - +  + +Note that you can only share a project with: + +- groups for which you have an explicitly defined membership +- groups that contain a nested subgroup or project for which you have an explicitly defined role + +Admins are able to share projects with any group in the system. ## Maximum access level diff --git a/doc/user/project/merge_requests/img/allow_maintainer_push.png b/doc/user/project/merge_requests/img/allow_maintainer_push.png Binary files differnew file mode 100644 index 00000000000..1631527071b --- /dev/null +++ b/doc/user/project/merge_requests/img/allow_maintainer_push.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index d3220598933..10d67729734 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -28,6 +28,7 @@ With GitLab merge requests, you can: - Enable [fast-forward merge requests](#fast-forward-merge-requests) - Enable [semi-linear history merge requests](#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch - [Create new merge requests by email](#create-new-merge-requests-by-email) +- Allow maintainers of the target project to push directly to the fork by [allowing edits from maintainers](maintainer_access.md) With **[GitLab Enterprise Edition][ee]**, you can also: diff --git a/doc/user/project/merge_requests/maintainer_access.md b/doc/user/project/merge_requests/maintainer_access.md new file mode 100644 index 00000000000..7feccc28f6b --- /dev/null +++ b/doc/user/project/merge_requests/maintainer_access.md @@ -0,0 +1,13 @@ +# Allow maintainer pushes for merge requests across forks + +This feature is available for merge requests across forked projects that are +publicly accessible. It makes it easier for maintainers of projects to collaborate +on merge requests across forks. + +When enabling this feature for a merge request, you give can give members with push access to the target project rights to edit files on the source branch of the merge request. + +The feature can only be enabled by users who already have push access to the source project. And only lasts while the merge request is open. + +Enable this functionality while creating a merge request: + + |
