diff options
| author | Simon Knox <psimyn@gmail.com> | 2017-10-03 08:41:33 +1100 |
|---|---|---|
| committer | Simon Knox <psimyn@gmail.com> | 2017-10-03 08:41:33 +1100 |
| commit | d54983885bee00ff7b49c4352639beca4a082f46 (patch) | |
| tree | 24c684554dfbfcc1a1fb3aa70b51f1ffdf041a48 /doc | |
| parent | 06b31461f34bac86d31d898e4f0e5b573d6b0345 (diff) | |
| parent | 0781e956e4a4174494aa28b62b8dfc92a92e8e84 (diff) | |
| download | gitlab-ce-d54983885bee00ff7b49c4352639beca4a082f46.tar.gz | |
Merge remote-tracking branch 'origin' into 37229-mr-widget-status-icon
Diffstat (limited to 'doc')
118 files changed, 1477 insertions, 350 deletions
diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md new file mode 100644 index 00000000000..bac8fa4bd9d --- /dev/null +++ b/doc/administration/raketasks/storage.md @@ -0,0 +1,107 @@ +# Repository Storage Rake Tasks + +This is a collection of rake tasks you can use to help you list and migrate +existing projects from Legacy storage to the new Hashed storage type. + +You can read more about the storage types [here][storage-types]. + +## List projects on Legacy storage + +To have a simple summary of projects using **Legacy** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:legacy_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:legacy_projects + +``` + +------ + +To list projects using **Legacy** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:list_legacy_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:list_legacy_projects + +``` + +## List projects on Hashed storage + +To have a simple summary of projects using **Hashed** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:hashed_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:hashed_projects + +``` + +------ + +To list projects using **Hashed** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:list_hashed_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:list_hashed_projects + +``` + +## Migrate existing projects to Hashed storage + +Before migrating your existing projects, you should +[enable hashed storage][storage-migration] for the new projects as well. + +This task will schedule all your existing projects to be migrated to the +**Hashed** storage type: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:migrate_to_hashed +``` + +**Source Installation** + +```bash +rake gitlab:storage:migrate_to_hashed + +``` + +You can monitor the progress in the _Admin > Monitoring > Background jobs_ screen. +There is a specific Queue you can watch to see how long it will take to finish: **project_migrate_hashed_storage** + +After it reaches zero, you can confirm every project has been migrated by running the commands above. +If you find it necessary, you can run this migration script again to schedule missing projects. + +Any error or warning will be logged in the sidekiq log file. + + +[storage-types]: ../repository_storage_types.md +[storage-migration]: ../repository_storage_types.md#how-to-migrate-to-hashed-storage diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md new file mode 100644 index 00000000000..fa882bbe28a --- /dev/null +++ b/doc/administration/repository_storage_types.md @@ -0,0 +1,69 @@ +# Repository Storage Types + +> [Introduced][ce-28283] in GitLab 10.0. + +## Legacy Storage + +Legacy Storage is the storage behavior prior to version 10.0. For historical reasons, GitLab replicated the same +mapping structure from the projects URLs: + + * Project's repository: `#{namespace}/#{project_name}.git` + * Project's wiki: `#{namespace}/#{project_name}.wiki.git` + +This structure made simple to migrate from existing solutions to GitLab and easy for Administrators to find where the +repository is stored. + +On the other hand this has some drawbacks: + +Storage location will concentrate huge amount of top-level namespaces. The impact can be reduced by the introduction of [multiple storage paths][storage-paths]. + +Because Backups are a snapshot of the same URL mapping, if you try to recover a very old backup, you need to verify +if any project has taken the place of an old removed project sharing the same URL. This means that `mygroup/myproject` +from your backup may not be the same original project that is today in the same URL. + +Any change in the URL will need to be reflected on disk (when groups / users or projects are renamed). This can add a lot +of load in big installations, and can be even worst if they are using any type of network based filesystem. + +Last, for GitLab Geo, this storage type means we have to synchronize the disk state, replicate renames in the correct +order or we may end-up with wrong repository or missing data temporarily. + +## Hashed Storage + +Hashed Storage is the new storage behavior we are rolling out with 10.0. It's not enabled by default yet, but we +encourage everyone to try-it and take the time to fix any script you may have that depends on the old behavior. + +Instead of coupling project URL and the folder structure where the repository will be stored on disk, we are coupling +a hash, based on the project's ID. + +This makes the folder structure immutable, and therefore eliminates any requirement to synchronize state from URLs to +disk structure. This means that renaming a group, user or project will cost only the database transaction, and will take +effect immediately. + +The hash also helps to spread the repositories more evenly on the disk, so the top-level directory will contain less +folders than the total amount of top-level namespaces. + +Hash format is based on hexadecimal representation of SHA256: `SHA256(project.id)`. +Top-level folder uses first 2 characters, followed by another folder with the next 2 characters. They are both stored in +a special folder `@hashed`, to co-exist with existing Legacy projects: + +```ruby +# Project's repository: +"@hashed/#{hash[0..1]}/#{hash[2..3]}/#{hash}.git" + +# Wiki's repository: +"@hashed/#{hash[0..1]}/#{hash[2..3]}/#{hash}.wiki.git" +``` + +This new format also makes possible to restore backups with confidence, as when restoring a repository from the backup, +you will never mistakenly restore a repository in the wrong project (considering the backup is made after the migration). + +### How to migrate to Hashed Storage + +In GitLab, go to **Admin > Settings**, find the **Repository Storage** section and select +"_Create new projects using hashed storage paths_". + +To migrate your existing projects to the new storage type, check the specific [rake tasks]. + +[ce-28283]: https://gitlab.com/gitlab-org/gitlab-ce/issues/28283 +[rake tasks]: raketasks/storage.md#migrate-existing-projects-to-hashed-storage +[storage-paths]: repository_storage_types.md diff --git a/doc/api/README.md b/doc/api/README.md index 6cbea29bda6..3fd4c97e536 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -14,6 +14,7 @@ following locations: - [Project-level Variables](project_level_variables.md) - [Group-level Variables](group_level_variables.md) - [Commits](commits.md) +- [Custom Attributes](custom_attributes.md) - [Deployments](deployments.md) - [Deploy Keys](deploy_keys.md) - [Environments](environments.md) diff --git a/doc/api/commits.md b/doc/api/commits.md index 2a78553782f..5a4a8d888b3 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -181,6 +181,12 @@ Example response: "parent_ids": [ "ae1d9fb46aa2b07ee9836d49862ec4e2c46fbbba" ], + "last_pipeline" : { + "id": 8, + "ref": "master", + "sha": "2dc6aa325a317eda67812f05600bdf0fcdc70ab0" + "status": "created" + } "stats": { "additions": 15, "deletions": 10, diff --git a/doc/api/custom_attributes.md b/doc/api/custom_attributes.md new file mode 100644 index 00000000000..8b26f7093ab --- /dev/null +++ b/doc/api/custom_attributes.md @@ -0,0 +1,105 @@ +# Custom Attributes API + +Every API call to custom attributes must be authenticated as administrator. + +## List custom attributes + +Get all custom attributes on a user. + +``` +GET /users/:id/custom_attributes +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes +``` + +Example response: + +```json +[ + { + "key": "location", + "value": "Antarctica" + }, + { + "key": "role", + "value": "Developer" + } +] +``` + +## Single custom attribute + +Get a single custom attribute on a user. + +``` +GET /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` + +Example response: + +```json +{ + "key": "location", + "value": "Antarctica" +} +``` + +## Set custom attribute + +Set a custom attribute on a user. The attribute will be updated if it already exists, +or newly created otherwise. + +``` +PUT /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | +| `value` | string | yes | The value of the custom attribute | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "value=Greenland" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` + +Example response: + +```json +{ + "key": "location", + "value": "Greenland" +} +``` + +## Delete custom attribute + +Delete a custom attribute on a user. + +``` +DELETE /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` diff --git a/doc/api/jobs.md b/doc/api/jobs.md index d60c7c12881..e7060e154f4 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -336,7 +336,7 @@ Parameters | Attribute | Type | Required | Description | |-------------|---------|----------|-------------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `ref_name` | string | yes | The ref from a repository | +| `ref_name` | string | yes | The ref from a repository (can only be branch or tag name, not HEAD or SHA) | | `job` | string | yes | The name of the job | Example request: diff --git a/doc/api/projects.md b/doc/api/projects.md index 3144220e588..07331d05231 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -635,6 +635,98 @@ POST /projects/:id/fork | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | | `namespace` | integer/string | yes | The ID or path of the namespace that the project will be forked to | +## List Forks of a project + +>**Note:** This feature was introduced in GitLab 10.1 + +List the projects accessible to the calling user that have an established, forked relationship with the specified project + +``` +GET /projects/:id/forks +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of projects matching the search criteria | +| `simple` | boolean | no | Return only the ID, URL, name, and path of each project | +| `owned` | boolean | no | Limit by projects owned by the current user | +| `membership` | boolean | no | Limit by projects that the current user is a member of | +| `starred` | boolean | no | Limit by projects starred by the current user | +| `statistics` | boolean | no | Include project statistics | +| `with_issues_enabled` | boolean | no | Limit by enabled issues feature | +| `with_merge_requests_enabled` | boolean | no | Limit by enabled merge requests feature | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/5/forks" +``` + +Example responses: + +```json +[ + { + "id": 3, + "description": null, + "default_branch": "master", + "visibility": "internal", + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site", + "tag_list": [ + "example", + "disapora project" + ], + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "jobs_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, + "container_registry_enabled": false, + "created_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "id": 3, + "name": "Diaspora", + "path": "diaspora", + "kind": "group", + "full_path": "diaspora" + }, + "import_status": "none", + "archived": true, + "avatar_url": "http://example.com/uploads/project/avatar/3/uploads/avatar.png", + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 1, + "public_jobs": true, + "shared_with_groups": [], + "only_allow_merge_if_pipeline_succeeds": false, + "only_allow_merge_if_all_discussions_are_resolved": false, + "request_access_enabled": false, + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members" + } + } +] +``` + ## Star a project Stars a given project. Returns status code `304` if the project is already starred. diff --git a/doc/api/users.md b/doc/api/users.md index 6d5db16b36a..1643c584244 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -154,6 +154,12 @@ You can search users by creation date time range with: GET /users?created_before=2001-01-02T00:00:00.060Z&created_after=1999-01-02T00:00:00.060 ``` +You can filter by [custom attributes](custom_attributes.md) with: + +``` +GET /users?custom_attributes[key]=value&custom_attributes[other_key]=other_value +``` + ## Single user Get a single user. diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index f28c9791bee..4586caa457d 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -250,6 +250,8 @@ By default, when using `docker:dind`, Docker uses the `vfs` storage driver which copies the filesystem on every run. This is a very disk-intensive operation which can be avoided if a different driver is used, for example `overlay2`. +### Requirements + 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: @@ -271,14 +273,27 @@ which can be avoided if a different driver is used, for example `overlay2`. overlay ``` -1. Use the driver by defining a variable at the top of your `.gitlab-ci.yml`: +### Use driver per project - ``` - variables: - DOCKER_DRIVER: overlay2 - ``` - -> **Note:** +You can enable the driver for each project individually by editing the project's `.gitlab-ci.yml`: + +``` +variables: + DOCKER_DRIVER: overlay2 +``` + +### Use driver for every project + +To enable the driver for every project, you can set the environment variable for every build by adding `environment` in the `[[runners]]` section of `config.toml`: + +```toml +environment = ["DOCKER_DRIVER=overlay2"] +``` + +If you're running multiple Runners you will have to modify all configuration files. + +> **Notes:** +- More information about the Runner configuration is available in the [Runner documentation](https://docs.gitlab.com/runner/configuration/). - For more information about using OverlayFS with Docker, you can read [Use the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/). diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index f69d71a5c39..38bd0450a09 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -252,6 +252,8 @@ The `cache:key` variable can use any of the [predefined variables](../variables/ The default key is **default** across the project, therefore everything is shared between each pipelines and jobs by default, starting from GitLab 9.0. +>**Note:** The `cache:key` variable cannot contain the `/` character. + --- **Example configurations** @@ -276,7 +278,7 @@ To enable per-job and per-branch caching: ```yaml cache: - key: "$CI_JOB_NAME/$CI_COMMIT_REF_NAME" + key: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME" untracked: true ``` @@ -284,7 +286,7 @@ To enable per-branch and per-stage caching: ```yaml cache: - key: "$CI_JOB_STAGE/$CI_COMMIT_REF_NAME" + key: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME" untracked: true ``` @@ -293,7 +295,7 @@ If you use **Windows Batch** to run your shell scripts you need to replace ```yaml cache: - key: "%CI_JOB_STAGE%/%CI_COMMIT_REF_NAME%" + key: "%CI_JOB_STAGE%-%CI_COMMIT_REF_NAME%" untracked: true ``` @@ -302,7 +304,7 @@ If you use **Windows PowerShell** to run your shell scripts you need to replace ```yaml cache: - key: "$env:CI_JOB_STAGE/$env:CI_COMMIT_REF_NAME" + key: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_NAME" untracked: true ``` @@ -1568,6 +1570,11 @@ Read more on [GitLab Pages user documentation](../../user/project/pages/index.md Each instance of GitLab CI has an embedded debug tool called Lint. You can find the link under `/ci/lint` of your gitlab instance. +## Using reserved keywords + +If you get validation error when using specific values (e.g., `true` or `false`), +try to quote them, or change them to a different form (e.g., `/bin/true`). + ## Skipping jobs If your commit message contains `[ci skip]` or `[skip ci]`, using any diff --git a/doc/development/README.md b/doc/development/README.md index 3096d9f25f0..1448a4c0414 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -44,6 +44,7 @@ - [Building a package for testing purposes](build_test_package.md) - [Manage feature flags](feature_flags.md) - [View sent emails or preview mailers](emails.md) +- [Working with Gitaly](gitaly.md) ## Databases diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index f83a60e49e8..5452b0e7a2f 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -215,14 +215,29 @@ same time will ensure that both existing and new data is migrated. In the next release we can remove the `after_commit` hooks and related code. We will also need to add a post-deployment migration that consumes any remaining -jobs. Such a migration would look like this: +jobs and manually run on any un-migrated rows. Such a migration would look like +this: ```ruby class ConsumeRemainingExtractServicesUrlJobs < ActiveRecord::Migration disable_ddl_transaction! + class Service < ActiveRecord::Base + include ::EachBatch + + self.table_name = 'services' + end + def up + # This must be included Gitlab::BackgroundMigration.steal('ExtractServicesUrl') + + # This should be included, but can be skipped - see below + Service.where(url: nil).each_batch(of: 50) do |batch| + range = batch.pluck('MIN(id)', 'MAX(id)').first + + Gitlab::BackgroundMigration::ExtractServicesUrl.new.perform(*range) + end end def down @@ -230,6 +245,15 @@ class ConsumeRemainingExtractServicesUrlJobs < ActiveRecord::Migration end ``` +The final step runs for any un-migrated rows after all of the jobs have been +processed. This is in case a Sidekiq process running the background migrations +received SIGKILL, leading to the jobs being lost. (See +[more reliable Sidekiq queue][reliable-sidekiq] for more information.) + +If the application does not depend on the data being 100% migrated (for +instance, the data is advisory, and not mission-critical), then this final step +can be skipped. + This migration will then process any jobs for the ExtractServicesUrl migration and continue once all jobs have been processed. Once done you can safely remove the `services.properties` column. @@ -254,6 +278,9 @@ for more details. 1. Make sure that background migration jobs are idempotent. 1. Make sure that tests you write are not false positives. +1. Make sure that if the data being migrated is critical and cannot be lost, the + clean-up migration also checks the final state of the data before completing. [migrations-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md [issue-rspec-hooks]: https://gitlab.com/gitlab-org/gitlab-ce/issues/35351 +[reliable-sidekiq]: https://gitlab.com/gitlab-org/gitlab-ce/issues/36791 diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 64a89976300..7165b8062a7 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -9,8 +9,18 @@ There are a few rules to get your merge request accepted: **approved by a [backend maintainer][projects]**. 1. If your merge request includes only frontend changes [^1], it must be **approved by a [frontend maintainer][projects]**. + 1. If your merge request includes UX changes [^1], it must + be **approved by a [UX team member][team]**. + 1. If your merge request includes adding a new JavaScript library [^1], it must be + **approved by a [frontend lead][team]**. + 1. If your merge request includes adding a new UI/UX paradigm [^1], it must be + **approved by a [UX lead][team]**. 1. If your merge request includes frontend and backend changes [^1], it must be **approved by a [frontend and a backend maintainer][projects]**. + 1. If your merge request includes UX and frontend changes [^1], it must + be **approved by a [UX team member and a frontend maintainer][team]**. + 1. If your merge request includes UX, frontend and backend changes [^1], it must + be **approved by a [UX team member, a frontend and a backend maintainer][team]**. 1. If your merge request includes a new dependency or a filesystem change, it must be **approved by a [Build team member][team]**. See [how to work with the Build team][build handbook] for more details. 1. To lower the amount of merge requests maintainers need to review, you can diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md new file mode 100644 index 00000000000..a76e978bd26 --- /dev/null +++ b/doc/development/fe_guide/icons.md @@ -0,0 +1,40 @@ +# Icons + +We are using SVG Icons in GitLab with a SVG Sprite, due to this the icons are only loaded once and then referenced through an ID. The sprite SVG is located under `/assets/icons.svg`. Our goal is to replace one by one all inline SVG Icons (as those currently bloat the HTML) and also all Font Awesome usages. + +### Usage in HAML/Rails + +To use a sprite Icon in HAML or Rails we use a specific helper function : + +`sprite_icon(icon_name, size: nil, css_class: '')` + +**icon_name** Use the icon_name that you can find in the SVG Sprite (Overview is available under `/assets/sprite.symbol.html`). +**size (optional)** Use one of the following sizes : 16,24,32,48,72 (this will be translated into a `s16` class) +**css_class (optional)** If you want to add additional css classes + +**Example** + +`= sprite_icon('issues', size: 72, css_class: 'icon-danger')` + +**Output from example above** + +`<svg class="s72 icon-danger"><use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="/assets/icons.svg#issues"></use></svg>` + +### Usage in HTML/JS + +Please use the following function inside JS to render an icon : +`gl.utils.spriteIcon(iconName)` + +## Adding a new icon to the sprite + +All Icons and Illustrations are managed in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository which is added as a dev-dependency. + +To upgrade to a new SVG Sprite version run `yarn upgrade https://gitlab.com/gitlab-org/gitlab-svgs` and then run `yarn run svg`. This task will copy the svg sprite and all illustrations in the correct folders. + +# SVG Illustrations + +Please use from now on for any SVG based illustrations simple `img` tags to show an illustration by simply using either `image_tag` or `image_path` helpers. Please use the class `svg-content` around it to ensure nice rendering. The illustrations are also organised in the [gitlab-svgs](https://gitlab.com/gitlab-org/gitlab-svgs) repository (as they are then automatically optimised). + +**Example** + +`= image_tag 'illustrations/merge_requests.svg'` diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 64bcb4a0257..031b12a8e91 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -29,34 +29,6 @@ For our currently-supported browsers, see our [requirements][requirements]. ## Development Process -When you are assigned an issue please follow the next steps: - -### Divide a big feature into small Merge Requests -1. Big Merge Request are painful to review. In order to make this process easier we -must break a big feature into smaller ones and create a Merge Request for each step. -1. First step is to create a branch from `master`, let's call it `new-feature`. This branch -will be the recipient of all the smaller Merge Requests. Only this one will be merged to master. -1. Don't do any work on this one, let's keep it synced with master. -1. Create a new branch from `new-feature`, let's call it `new-feature-step-1`. We advise you -to clearly identify which step the branch represents. -1. Do the first part of the modifications in this branch. The target branch of this Merge Request -should be `new-feature`. -1. Once `new-feature-step-1` gets merged into `new-feature` we can continue our work. Create a new -branch from `new-feature`, let's call it `new-feature-step-2` and repeat the process done before. - -```shell -master -└─ new-feature - ├─ new-feature-step-1 - ├─ new-feature-step-2 - └─ new-feature-step-3 -``` - -**Tips** -- Make sure `new-feature` branch is always synced with `master`: merge master frequently. -- Do the same for the feature branch you have opened. This can be accomplished by merging `master` into `new-feature` and `new-feature` into `new-feature-step-*` -- Avoid rewriting history. - ### Share your work early 1. Before writing code guarantee your vision of the architecture is aligned with GitLab's architecture. @@ -98,6 +70,11 @@ Vue specific design patterns and practices. --- +## [Icons](icons.md) +How we use SVG for our Icons. + +--- + ## Style Guides ### [JavaScript Style Guide](style_guide_js.md) diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md new file mode 100644 index 00000000000..f0be3a6b141 --- /dev/null +++ b/doc/development/gitaly.md @@ -0,0 +1,54 @@ +# GitLab Developers Guide to Working with Gitaly + +[Gitaly](https://gitlab.com/gitlab-org/gitaly) is a high-level Git RPC service used by GitLab CE/EE, +Workhorse and GitLab-Shell. All Rugged operations in GitLab CE/EE are currently being phased out to +be replaced by Gitaly API calls. + +Visit the [Gitaly Migration Board](https://gitlab.com/gitlab-org/gitaly/boards/331341) for current +status of the migration. + +## Feature Flags + +Gitaly makes heavy use of [feature flags](feature_flags.md). + +Each Rugged-to-Gitaly migration goes through a [series of phases](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/MIGRATION_PROCESS.md): +* **Opt-In**: by default the Rugged implementation is used. + * Production instances can choose to enable the Gitaly endpoint by enabling the feature flag. + * For testing purposes, you may wish to enable all feature flags by default. This can be done by exporting the following + environment variable: `GITALY_FEATURE_DEFAULT_ON=1`. + * On developer instances (ie, when `Rails.env.development?` is true), the Gitaly endpoint + is enabled by default, but can be _disabled_ using feature flags. +* **Opt-Out**: by default, the Gitaly endpoint is used, but the feature can be explicitly disabled using the feature flag. +* **Madatory**: The migration is complete and cannot be disabled. The old codepath is removed. + +### Enabling and Disabling Feature + +In the Rails console, type: + +```ruby +Feature.enable(:gitaly_feature_name) +Feature.disable(:gitaly_feature_name) +``` + +Where `gitaly_feature_name` is the name of the Gitaly feature. This can be determined by finding the appropriate +`gitaly_migrate` code block, for example: + +```ruby +gitaly_migrate(:tag_names) do +... +end +``` + +Since Gitaly features are always prefixed with `gitaly_`, the name of the feature flag in this case would be `gitaly_tag_names`. + +## Gitaly-Related Test Failures + +If your test-suite is failing with Gitaly issues, as a first step, try running: + +```shell +rm -rf tmp/tests/gitaly +``` + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/img/manual_build_docs.png b/doc/development/img/manual_build_docs.png Binary files differindex fef767c2a79..615facabb5f 100644 --- a/doc/development/img/manual_build_docs.png +++ b/doc/development/img/manual_build_docs.png diff --git a/doc/development/licensing.md b/doc/development/licensing.md index 9a5811d8474..a75cdf22f40 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -65,6 +65,7 @@ Libraries with the following licenses are unacceptable for use: - [GNU AGPLv3][AGPLv3]: AGPL-licensed libraries cannot be linked to from non-GPL projects. - [Open Software License (OSL)][OSL]: is a copyleft license. In addition, the FSF [recommend against its use][OSL-GNU]. - [Facebook BSD + PATENTS][Facebook]: is a 3-clause BSD license with a patent grant that has been deemed [Category X][x-list] by the Apache foundation. +- [WTFPL][WTFPL]: is a public domain dedication [rejected by the OSI (3.2)][WTFPL-OSI]. Also has a strong language which is not in accordance with our diversity policy. ## Requesting Approval for Licenses @@ -108,3 +109,5 @@ Gems which are included only in the "development" or "test" groups by Bundler ar [x-list]: https://www.apache.org/legal/resolved.html#category-x [Acceptable-Licenses]: #acceptable-licenses [Unacceptable-Licenses]: #unacceptable-licenses +[WTFPL]: https://wtfpl.net +[WTFPL-OSI]: https://opensource.org/minutes20090304 diff --git a/doc/development/testing.md b/doc/development/testing.md index 83269303005..d856b003353 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -150,6 +150,16 @@ always in-sync with the codebase. [GitLab QA]: https://gitlab.com/gitlab-org/gitlab-qa [part of GitLab Rails]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa +## Test for what should not be there + +This is particularly important for permission calls and might be called a +negative assertion: make sure only the bare minimum is returned and nothing else. + +See an issue about [leaking tokens] as an example of a vulnerability that is +captured by such a test. + +[leaking tokens]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37948 + ## How to test at the correct level? As many things in life, deciding what to test at each level of testing is a @@ -493,24 +503,24 @@ Here are some things to keep in mind regarding test performance: Our current CI parallelization setup is as follows: -1. The `knapsack` job in the prepare stage that is supposed to ensure we have a - `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file: +1. The `retrieve-tests-metadata` job in the `prepare` stage ensures that we have + a `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file: - The `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file is fetched from S3, if it's not here we initialize the file with `{}`. -1. Each `rspec x y` job are run with `knapsack rspec` and should have an evenly - distributed share of tests: +1. Each `rspec-pg x y`/`rspec-mysql x y` job is run with `knapsack rspec` and + should have an evenly distributed share of tests: - It works because the jobs have access to the `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` since the "artifacts from all previous stages are passed by default". [^1] - - the jobs set their own report path to + - The jobs set their own report path to `KNAPSACK_REPORT_PATH=knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json`. - - if knapsack is doing its job, test files that are run should be listed under + - If knapsack is doing its job, test files that are run should be listed under `Report specs`, not under `Leftover specs`. -1. The `update-knapsack` job takes all the +1. The `update-tests-metadata` job takes all the `knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json` - files from the `rspec x y` jobs and merge them all together into a single - `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that is then - uploaded to S3. + files from the `rspec-pg x y`/`rspec-mysql x y`jobs and merge them all together + into a single `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that + is then uploaded to S3. After that, the next pipeline will use the up-to-date `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. The same strategy diff --git a/doc/development/ux_guide/basics.md b/doc/development/ux_guide/basics.md index a436e9b1948..e215026bcca 100644 --- a/doc/development/ux_guide/basics.md +++ b/doc/development/ux_guide/basics.md @@ -32,19 +32,17 @@ This is the typeface used for code blocks and references to commits, branches, a --- ## Icons -GitLab uses Font Awesome icons throughout our interface. -| | | -| :-----------: | :---- | -|  | The trash icon is used for destructive actions that deletes information. | -|  | The pencil icon is used for editing content such as comments.| -|  | The bell icon is for notifications, such as Todos. | -|  | The eye icon is for subscribing to updates. For example, you can subscribe to a label and get updated on issues with that label. | -|  | The standard RSS icon is used for linking to RSS/atom feeds. | -|  | An 'x' is used for closing UI elements such as dropdowns. | -|  | A plus is used when creating new objects, such as issues, projects, etc. | - -> TODO: update this section, add more general guidance to icon usage and personality, etc. +GitLab has a strong, unique personality. When you look at any screen, you should know immediately that it is GitLab. +Iconography is a powerful visual cue to the user and is a great way for us to reflect our particular sense of style. + +- **Standard size:** 16px * 16px +- **Border thickness:** 2px +- **Border radius:** 3px + + + +> TODO: List all icons, proper usage, hover, and active states. --- diff --git a/doc/development/ux_guide/illustrations.md b/doc/development/ux_guide/illustrations.md new file mode 100644 index 00000000000..7e16c300921 --- /dev/null +++ b/doc/development/ux_guide/illustrations.md @@ -0,0 +1,86 @@ +# Illustrations + +The illustrations should always align with topics and goals in specific context. + +## Principles + +#### Be simple. +- For clarity, we use simple and specific elements to create our illustrations. + +#### Be optimistic. +- We are an open-minded, optimistic, and friendly team. We should reflect those values in our illustrations to connect with our brand experience. + +#### Be gentle. +- Our illustrations assist users in understanding context and guide users in the right direction. Illustrations are supportive, so they should be obvious but not aggressive. + + +## Style + +#### Shapes +- All illustrations are geometric rather than organic. +- The illustrations are made by circles, rectangles, squares, and triangles. + +<img src="img/illustrations-geometric.png" width=224px alt="Example for geometric" /> + +#### Stroke +- Standard border thickness: **4px** +- Depending on the situation, border thickness can be changed to **3px**. For example, when the illustration size is small, an illustration with 4px border thickness would look tight. In this case, the border thickness can be changed to 3px. +- We use **rounded caps** and **rounded corner**. + +| Do | Don't | +| -------- | -------- | +| <img src="img/illustrations-caps-do.png" width= 133px alt="Do: caps and corner" /> | <img src="img/illustrations-caps-don't.png" width= 133px alt="Don't: caps and corner"/> | + +#### Radius +- Standard corner radius: **10px** +- Depending on the situation, corner radius can be changed to **5px**. For example, when the illustration size is small, an illustration with 10px corner radius would be over-rounded. In this case, the corner radius can be changed to 5px. + +<img src="img/illustrations-border-radius.png" width= 464px alt="Example for border radius"/> + +#### Size +Depends on the situation, the illustration size can be the 3 types below: + +**Large** +* Use case: Empty states, error pages(e.g. 404, 403) +* For vertical layout, the illustration should not larger than **430*300 px**. +* For horizontal layout, the illustration should not larger than **430*380 px**. + +| Vertical layout | Horizontal layout | +| --------------- | ----------------- | +| <img src="img/illustration-size-large-vertical.png" /> | <img src="img/illustration-size-large-horizontal.png" /> + +**Medium** +* Use case: Banner +* The illustration should not larger than **240*160 px** +* The illustration should keep simple and clear. We recommend not including too many elements in the medium size illustration. + +<img src="img/illustration-size-medium.png" width=983px /> + +**Small** +* Use case: Graphics for explanatory text, graphics for status. +* The illustration should not larger than **160*90 px**. +* The illustration should keep simple and clear. We recommend not including too many elements in the small size illustration. + +<img src="img/illustration-size-small.png" width=983px /> + +**Illustration on mobile** +- Keep the proportions in original ratio. + +#### Colors palette + +For consistency, we recommend choosing colors from our color palette. + +| Orange | Purple | Grey | +| -------- | -------- | -------- | +| <img src="img/illustrations-color-orange.png" width= 160px alt="Orange" /> | <img src="img/illustrations-color-purple.png" width= 160px alt="Purple" /> | <img src="img/illustrations-color-grey.png" width= 160px alt="Grey" /> | +| #FC6D26 | #6B4FBB | #EEEEEE | + +#### Don't +- Don't include the typography in the illustration. +- Don't include tanuki in the illustration. If necessary, we recommend having tanuki monochromatic. + +--- + +| Orange | Purple | +| -------- | -------- | +| <img src="img/illustrations-palette-oragne.png" width= 160px alt="Palette - Orange" /> | <img src="img/illustrations-palette-purple.png" width= 160px alt="Palette - Purple" /> | diff --git a/doc/development/ux_guide/img/icon-spec.png b/doc/development/ux_guide/img/icon-spec.png Binary files differnew file mode 100644 index 00000000000..56b19610dc1 --- /dev/null +++ b/doc/development/ux_guide/img/icon-spec.png diff --git a/doc/development/ux_guide/img/illustration-size-large-horizontal.png b/doc/development/ux_guide/img/illustration-size-large-horizontal.png Binary files differnew file mode 100755 index 00000000000..8aa835adccc --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-large-horizontal.png diff --git a/doc/development/ux_guide/img/illustration-size-large-vertical.png b/doc/development/ux_guide/img/illustration-size-large-vertical.png Binary files differnew file mode 100644 index 00000000000..813b6a065e5 --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-large-vertical.png diff --git a/doc/development/ux_guide/img/illustration-size-medium.png b/doc/development/ux_guide/img/illustration-size-medium.png Binary files differnew file mode 100755 index 00000000000..55cfe1dcb91 --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-medium.png diff --git a/doc/development/ux_guide/img/illustration-size-small.png b/doc/development/ux_guide/img/illustration-size-small.png Binary files differnew file mode 100644 index 00000000000..0124f58f48e --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-small.png diff --git a/doc/development/ux_guide/img/illustrations-border-radius.png b/doc/development/ux_guide/img/illustrations-border-radius.png Binary files differnew file mode 100755 index 00000000000..4e2fef5c7f5 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-border-radius.png diff --git a/doc/development/ux_guide/img/illustrations-caps-do.png b/doc/development/ux_guide/img/illustrations-caps-do.png Binary files differnew file mode 100755 index 00000000000..7a2c74382f6 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-caps-do.png diff --git a/doc/development/ux_guide/img/illustrations-caps-don't.png b/doc/development/ux_guide/img/illustrations-caps-don't.png Binary files differnew file mode 100755 index 00000000000..848f72dbe30 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-caps-don't.png diff --git a/doc/development/ux_guide/img/illustrations-color-grey.png b/doc/development/ux_guide/img/illustrations-color-grey.png Binary files differnew file mode 100755 index 00000000000..63855026c2b --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-grey.png diff --git a/doc/development/ux_guide/img/illustrations-color-orange.png b/doc/development/ux_guide/img/illustrations-color-orange.png Binary files differnew file mode 100755 index 00000000000..96765c8c28c --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-orange.png diff --git a/doc/development/ux_guide/img/illustrations-color-purple.png b/doc/development/ux_guide/img/illustrations-color-purple.png Binary files differnew file mode 100755 index 00000000000..745d2c853ba --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-purple.png diff --git a/doc/development/ux_guide/img/illustrations-geometric.png b/doc/development/ux_guide/img/illustrations-geometric.png Binary files differnew file mode 100755 index 00000000000..33f05547bac --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-geometric.png diff --git a/doc/development/ux_guide/img/illustrations-palette-oragne.png b/doc/development/ux_guide/img/illustrations-palette-oragne.png Binary files differnew file mode 100755 index 00000000000..15f35912646 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-palette-oragne.png diff --git a/doc/development/ux_guide/img/illustrations-palette-purple.png b/doc/development/ux_guide/img/illustrations-palette-purple.png Binary files differnew file mode 100755 index 00000000000..e0f5839705e --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-palette-purple.png diff --git a/doc/development/ux_guide/index.md b/doc/development/ux_guide/index.md index 8a849f239dc..42bcf234e12 100644 --- a/doc/development/ux_guide/index.md +++ b/doc/development/ux_guide/index.md @@ -21,6 +21,11 @@ Guidance on the timing, curving and motion for GitLab. --- +### [Illustrations](illustrations.md) +Guidelines for principals and styles related to illustrations for GitLab. + +--- + ### [Copy](copy.md) Conventions on text and messaging within labels, buttons, and other components. diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 479258f743e..68ba3dd2da3 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -1,6 +1,6 @@ # Writing documentation - - **General Documentation**: written by the developers responsible by creating features. Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. + - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. - **Technical Articles**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). @@ -69,6 +69,51 @@ Use the [writing method](https://about.gitlab.com/handbook/product/technical-wri All the docs follow the same [styleguide](doc_styleguide.md). +### Contributing to docs + +Whenever a feature is changed, updated, introduced, or deprecated, the merge +request introducing these changes must be accompanied by the documentation +(either updating existing ones or creating new ones). This is also valid when +changes are introduced to the UI. + +The one resposible for writing the first piece of documentation is the developer who +wrote the code. It's the job of the Product Manager to ensure all features are +shipped with its docs, whether is a small or big change. At the pace GitLab evolves, +this is the only way to keep the docs up-to-date. If you have any questions about it, +please ask a Technical Writer. Otherwise, when your content is ready, assign one of +them to review it for you. + +We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything +is documented. + +### Feature overview and use cases + +Every major feature (regardless if present in GitLab Community or Enterprise editions) +should present, at the beginning of the document, two main sections: **overview** and +**use cases**. Every GitLab EE-only feature should also contain these sections. + +**Overview**: at the name suggests, the goal here is to provide an overview of the feature. +Describe what is it, what it does, why it is important/cool/nice-to-have, +what problem it solves, and what you can do with this feature that you couldn't +do before. + +**Use cases**: provide at least two, ideally three, use cases for every major feature. +You should answer this question: what can you do with this feature/change? Use cases +are examples of how this feauture or change can be used in real life. + +Examples: +- CE and EE: [Issues](../user/project/issues/index.md#use-cases) +- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) +- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) +- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) + +Note that if you don't have anything to add between the doc title (`<h1>`) and +the header `## Overview`, you can omit the header, but keep the content of the +overview there. + +> **Overview** and **use cases** are required to **every** Enterprise Edition feature, +and for every **major** feature present in Community Edition. + ### Markdown Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. @@ -106,21 +151,84 @@ CE and EE. ## Previewing the changes live -If you want to preview your changes live, you can use the manual `build-docs` -job in your merge request. +If you want to preview the doc changes of your merge request live, you can use +the manual `review-docs-deploy` job in your merge request. + +TIP: **Tip:** +If your branch contains only documentation changes, you can use +[special branch names](#testing) to avoid long running pipelines.  This job will: 1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) - project named after the scheme: `<CE/EE-branch-slug>-built-from-ce-ee` -1. Trigger a pipeline and build the docs site with your changes - -Look for the docs URL at the output of the `build-docs` job. - ->**Note:** + project named after the scheme: `preview-<branch-slug>` +1. Trigger a cross project pipeline and build the docs site with your changes + +After a few minutes, the Review App will be deployed and you will be able to +preview the changes. The docs URL can be found in two places: + +- In the merge request widget +- In the output of the `review-docs-deploy` job, which also includes the + triggered pipeline so that you can investigate whether something went wrong + +In case the Review App URL returns 404, follow these steps to debug: + +1. **Did you follow the URL from the merge request widget?** If yes, then check if + the link is the same as the one in the job output. It can happen that if the + branch name slug is longer than 35 characters, it is automatically + truncated. That means that the merge request widget will not show the proper + URL due to a limitation of how `environment: url` works, but you can find the + real URL from the output of the `review-docs-deploy` job. +1. **Did you follow the URL from the job output?** If yes, then it means that + either the site is not yet deployed or something went wrong with the remote + pipeline. Give it a few minutes and it should appear online, otherwise you + can check the status of the remote pipeline from the link in the job output. + If the pipeline failed or got stuck, drop a line in the `#docs` chat channel. + +TIP: **Tip:** +Someone that has no merge rights to the CE/EE projects (think of forks from +contributors) will not be able to run the manual job. In that case, you can +ask someone from the GitLab team who has the permissions to do that for you. + +NOTE: **Note:** Make sure that you always delete the branch of the merge request you were working on. If you don't, the remote docs branch won't be removed either, and the server where the Review Apps are hosted will eventually be out of disk space. + +### Behind the scenes + +If you want to know the hot details, here's what's really happening: + +1. You manually run the `review-docs-deploy` job in a CE/EE merge request. +1. The job runs the [`scirpts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) + script with the `deploy` flag, which in turn: + 1. Takes your branch name and applies the following: + - The slug of the branch name is used to avoid special characters since + ultimately this will be used by NGINX. + - The `preview-` prefix is added to avoid conflicts if there's a remote branch + with the same name that you created in the merge request. + - The final branch name is truncated to 42 characters to avoid filesystem + limitations with long branch names (> 63 chars). + 1. The remote branch is then created if it doesn't exist (meaning you can + re-run the manual job as many times as you want and this step will be skipped). + 1. A new cross-project pipeline is triggered in the docs project. + 1. The preview URL is shown both at the job output and in the merge request + widget. You also get the link to the remote pipeline. +1. In the docs project, the pipeline is created and it + [skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55) + to lower the build time. +1. Once the docs site is built, the HTML files are uploaded as artifacts. +1. A specific Runner tied only to the docs project, runs the Review App job + that downloads the artifacts and uses `rsync` to transfer the files over + to a location where NGINX serves them. + +The following GitLab features are used among others: + +- [Manual actions](../ci/yaml/README.md#manual-actions) +- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Review Apps](../ci/review_apps/index.md) +- [Artifacts](../ci/yaml/README.md#artifacts) +- [Specific Runner](../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index 5c128f54a76..f9ba1508705 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -1,11 +1,12 @@ # Database MySQL >**Note:** -We do not recommend using MySQL due to various issues. For example, case +- We do not recommend using MySQL due to various issues. For example, case [(in)sensitivity](https://dev.mysql.com/doc/refman/5.0/en/case-sensitivity.html) and [problems](https://bugs.mysql.com/bug.php?id=65830) that [suggested](https://bugs.mysql.com/bug.php?id=50909) [fixes](https://bugs.mysql.com/bug.php?id=65830) [have](https://bugs.mysql.com/bug.php?id=63164). +- We recommend using MySQL version 5.6 or later. Please see the following [issue][ce-38152]. ## Initial database setup @@ -13,7 +14,7 @@ and [problems](https://bugs.mysql.com/bug.php?id=65830) that # Install the database packages sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev -# Ensure you have MySQL version 5.5.14 or later +# Ensure you have MySQL version 5.6 or later mysql --version # Pick a MySQL root password (can be anything), type it and press enter @@ -293,3 +294,4 @@ Details can be found in the [PostgreSQL][postgres-text-type] and [postgres-text-type]: http://www.postgresql.org/docs/9.2/static/datatype-character.html [mysql-text-types]: http://dev.mysql.com/doc/refman/5.7/en/string-type-overview.html +[ce-38152]: https://gitlab.com/gitlab-org/gitlab-ce/issues/38152 diff --git a/doc/install/kubernetes/gitlab_omnibus.md b/doc/install/kubernetes/gitlab_omnibus.md index 9d1280c3dc6..8c110a37380 100644 --- a/doc/install/kubernetes/gitlab_omnibus.md +++ b/doc/install/kubernetes/gitlab_omnibus.md @@ -148,32 +148,51 @@ helm install --name gitlab --set baseDomain=gitlab.io,baseIP=1.1.1.1,gitlab=ee,g ## Updating GitLab using the Helm Chart +>**Note**: If you are upgrading from a previous version to 0.1.35 or above, you will need to change the access mode values for GitLab's storage. To do this, set the following in `values.yaml` or on the CLI: +``` +gitlabDataAccessMode=ReadWriteMany +gitlabRegistryAccessMode=ReadWriteMany +gitlabConfigAccessMode=ReadWriteMany +``` + Once your GitLab Chart is installed, configuration changes and chart updates -should we done using `helm upgrade` +should be done using `helm upgrade`: ```bash -helm upgrade -f <CONFIG_VALUES_FILE> <RELEASE-NAME> gitlab/gitlab +helm upgrade -f values.yaml gitlab gitlab/gitlab-omnibus ``` -where: +## Upgrading from CE to EE using the Helm Chart + +If you have installed the Community Edition using this chart, upgrading to Enterprise Edition is easy. -- `<CONFIG_VALUES_FILE>` is the path to values file containing your custom - [configuration] (#configuring-and-installing-gitlab). -- `<RELEASE-NAME>` is the name you gave the chart when installing it. - In the [Install section](#installing-gitlab-using-the-helm-chart) we called it `gitlab`. +If you are using a `values.yaml` file to specify the configuration options, edit the file and set `gitlab=ee`. If you would like to run a specific version of GitLab EE, set `gitlabEEImage` to be the desired GitLab [docker image](https://hub.docker.com/r/gitlab/gitlab-ee/tags/). Then you can use `helm upgrade` to update your GitLab instance to EE: + +```bash +helm upgrade -f values.yaml gitlab gitlab/gitlab-omnibus +``` + +You can also upgrade and specify these options via the command line: + +```bash +helm upgrade gitlab --set gitlab=ee,gitlabEEImage=gitlab/gitlab-ee:9.5.5-ee.0 gitlab/gitlab-omnibus +``` ## Uninstalling GitLab using the Helm Chart To uninstall the GitLab Chart, run the following: ```bash -helm delete <RELEASE-NAME> +helm delete gitlab ``` -where: +## Troubleshooting + +### Storage errors when updating `gitlab-omnibus` versions prior to 0.1.35 + +Users upgrading `gitlab-omnibus` from a version prior to 0.1.35, may see an error like: `Error: UPGRADE FAILED: PersistentVolumeClaim "gitlab-gitlab-config-storage" is invalid: spec: Forbidden: field is immutable after creation`. -- `<RELEASE-NAME>` is the name you gave the chart when installing it. - In the [Install section](#installing) we called it `gitlab`. +This is due to a change in the access mode for GitLab storage in version 0.1.35. To successfully upgrade, the access mode flags must be set to `ReadWriteMany` as detailed in the [update section](#updating-gitlab-using-the-helm-chart). [kube-srv]: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types [storageclass]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#storageclasses diff --git a/doc/install/requirements.md b/doc/install/requirements.md index f672b358096..17fe80fa93d 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -82,11 +82,11 @@ errors during usage. We recommend having at least 2GB of swap on your server, even if you currently have enough available RAM. Having swap will help reduce the chance of errors occurring -if your available memory changes. We also recommend [configuring the kernels swappiness setting](https://askubuntu.com/a/103916) +if your available memory changes. We also recommend [configuring the kernel's swappiness setting](https://askubuntu.com/a/103916) to a low value like `10` to make the most of your RAM while still having the swap available when needed. -Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about many you need of those. +Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about how many you need of those. ## Database diff --git a/doc/integration/azure.md b/doc/integration/azure.md index 5e3e9f5ab77..f3c9c498634 100644 --- a/doc/integration/azure.md +++ b/doc/integration/azure.md @@ -74,6 +74,9 @@ To enable the Microsoft Azure OAuth2 OmniAuth provider you must register your ap tenant_id: "TENANT ID" } } ``` + The `base_azure_url` is optional and can be added for different locales; + e.g. `base_azure_url: "https://login.microsoftonline.de"`. + 1. Replace 'CLIENT ID', 'CLIENT SECRET' and 'TENANT ID' with the values you got above. 1. Save the configuration file. diff --git a/doc/topics/autodevops/img/auto_devops_settings.png b/doc/topics/autodevops/img/auto_devops_settings.png Binary files differdeleted file mode 100644 index 57bd7650a30..00000000000 --- a/doc/topics/autodevops/img/auto_devops_settings.png +++ /dev/null diff --git a/doc/topics/autodevops/img/auto_monitoring.png b/doc/topics/autodevops/img/auto_monitoring.png Binary files differindex 5661b50841b..92902e3ca72 100644 --- a/doc/topics/autodevops/img/auto_monitoring.png +++ b/doc/topics/autodevops/img/auto_monitoring.png diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index b31b8eaaca0..5561784ed0b 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -1,73 +1,51 @@ # Auto DevOps -> [Introduced][ce-37115] in GitLab 10.0. Auto DevOps is currently in Beta and -**not recommended for production use**. Access to the Container Registry is only -available while the pipeline is running. Restarting a pod, scaling a service, or -other actions which require on-going access **will fail** even for public -projects. On-going secure access is planned for a subsequent release. - -Auto DevOps brings best practices to your project in an easy and default way. A -typical web project starts with Continuous Integration (CI), then adds automated -deployment to production, and maybe some time in the future adds some kind of -monitoring. With Auto DevOps, every project has a complete workflow, with -no configuration, including: - -- [Auto Build](#auto-build) -- [Auto Test](#auto-test) -- [Auto Code Quality](#auto-code-quality) -- [Auto Review Apps](#auto-review-apps) -- [Auto Deploy](#auto-deploy) -- [Auto Monitoring](#-auto-monitoring) +DANGER: Auto DevOps is currently in **Beta** and _not recommended for production use_. + +> [Introduced][ce-37115] in GitLab 10.0. + +Auto DevOps automatically detects, builds, tests, deploys, and monitors your +applications. ## Overview -You will need [Kubernetes](https://kubernetes.io/) and -[Prometheus](https://prometheus.io/) to make full use of Auto DevOps, but -even projects using only [GitLab Runners](https://docs.gitlab.com/runner/) will -be able to make use of Auto Build, Auto Test, and Auto Code Quality. - -Auto DevOps makes use of an open source tool called -[Herokuish](https://github.com/gliderlabs/herokuish) which uses [Heroku -buildpacks](https://devcenter.heroku.com/articles/buildpacks) to automatically -detect, build, and test applications. Auto DevOps supports all of the languages -and frameworks that are [supported by -Herokuish](https://github.com/gliderlabs/herokuish#buildpacks) such as Ruby, -Rails, Node, PHP, Python, and Java, and [custom buildpacks can be -specified](#using-custom-buildpacks). *GitLab is in no way affiliated with Heroku -or Glider Labs.* - -Projects can [customize](#customizing) the process by specifying [custom -buildpacks](#custom-buildpack), [custom `Dockerfile`s](#custom-dockerfile), -[custom Helm charts](#custom-helm-chart), or even copying the complete CI/CD -configuration into your project to enable staging and canary deployments, and -more. +With Auto DevOps, the software development process becomes easier to set up +as every project can have a complete workflow from build to deploy and monitoring, +with minimal to zero configuration. -## Quick start +Comprised of a set of stages, Auto DevOps brings these best practices to your +project in an easy and automatic way: -If you are using GitLab.com, see our [quick start guide](quick_start_guide.md) -for using Auto DevOps with GitLab.com and an external Kubernetes cluster on -Google Cloud. +1. [Auto Build](#auto-build) +1. [Auto Test](#auto-test) +1. [Auto Code Quality](#auto-code-quality) +1. [Auto Review Apps](#auto-review-apps) +1. [Auto Deploy](#auto-deploy) +1. [Auto Monitoring](#auto-monitoring) -For self-hosted installations, the easiest way to make use of Auto DevOps is to -install GitLab inside a Kubernetes cluster using the [GitLab-Omnibus Helm -Chart](../../install/kubernetes/gitlab_omnibus.md) which automatically installs -and configures everything you need. +As Auto DevOps relies on many different components, it's good to have a basic +knowledge of the following: -## Prerequisites +- [Kubernetes](https://kubernetes.io/docs/home/) +- [Helm](https://docs.helm.sh/) +- [Docker](https://docs.docker.com) +- [GitLab Runner](https://docs.gitlab.com/runner/) +- [Prometheus](https://prometheus.io/docs/introduction/overview/) -You will need one or more GitLab Runners, a Kubernetes cluster, and Prometheus -installed in the cluster to make full use of Auto DevOps. If you do not have -Kubernetes or Prometheus installed then Auto Review Apps, Auto Deploy, and Auto -Monitoring will be silently skipped. +Auto DevOps provides great defaults for all the stages; you can, however, +[customize](#customizing) almost everything to your needs. -If you are using GitLab outside of Kubernetes, for example with GitLab.com, then -you should take these prerequisites into account: +## Prerequisites + +TIP: **Tip:** +For self-hosted installations, the easiest way to make use of Auto DevOps is to +install GitLab inside a Kubernetes cluster using the [GitLab Omnibus Helm Chart] +which automatically installs and configures everything you need! -1. **Base domain** - You will need a base domain configured with wildcard DNS to - be used by all of your Auto DevOps applications. +To make full use of Auto DevOps, you will need: -1. **GitLab Runner** - Your Runner needs to be configured to be able to run Docker. - Generally this means using the +1. **GitLab Runner** (needed for all stages) - Your Runner needs to be + configured to be able to run Docker. Generally this means using the [Docker](https://docs.gitlab.com/runner/executors/docker.html) or [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html), with [privileged mode enabled](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). @@ -78,35 +56,98 @@ you should take these prerequisites into account: should be registered as [shared Runners](../../ci/runners/README.md#registering-a-shared-runner) for the entire GitLab instance, or [specific Runners](../../ci/runners/README.md#registering-a-specific-runner) that are assigned to specific projects. - -1. **Kubernetes** - To enable deploys, you will need Kubernetes 1.5+, with NGINX - ingress and wildcard SSL termination, for example using the - [`nginx-ingress`](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) - and [`kube-lego`](https://github.com/kubernetes/charts/tree/master/stable/kube-lego) - Helm charts respectively. The [Kubernetes service][kubernetes-service] +1. **Base domain** (needed for Auto Review Apps and Auto Deploy) - You will need + a domain configured with wildcard DNS which is gonna be used by all of your + Auto DevOps applications. [Read the specifics](#auto-devops-base-domain). +1. **Kubernetes** (needed for Auto Review Apps, Auto Deploy, and Auto Monitoring) - + To enable deployments, you will need Kubernetes 1.5+. The [Kubernetes service][kubernetes-service] integration will need to be enabled for the project, or enabled as a [default service template](../../user/project/integrations/services_templates.md) for the entire GitLab installation. - -1. **Prometheus** - To enable Auto Monitoring, you will need Prometheus installed - somewhere (inside or outside your cluster) and configured to scrape your - Kubernetes cluster. To get response metrics (in addition to system metrics), - you need to [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-prometheus-to-monitor-for-nginx-ingress-metrics). + 1. **A load balancer** - You can use NGINX ingress by deploying it to your + Kubernetes cluster using the + [`nginx-ingress`](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) + Helm chart. + 1. **Wildcard TLS termination** - You can deploy the + [`kube-lego`](https://github.com/kubernetes/charts/tree/master/stable/kube-lego) + Helm chart to your Kubernetes cluster to automatically issue certificates + for your domains using Let's Encrypt. +1. **Prometheus** (needed for Auto Monitoring) - To enable Auto Monitoring, you + will need Prometheus installed somewhere (inside or outside your cluster) and + configured to scrape your Kubernetes cluster. To get response metrics + (in addition to system metrics), you need to + [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-prometheus-to-monitor-for-nginx-ingress-metrics). The [Prometheus service](../../user/project/integrations/prometheus.md) integration needs to be enabled for the project, or enabled as a [default service template](../../user/project/integrations/services_templates.md) for the entire GitLab installation. -## Enabling Auto DevOps +NOTE: **Note:** +If you do not have Kubernetes or Prometheus installed, then Auto Review Apps, +Auto Deploy, and Auto Monitoring will be silently skipped. + +### Auto DevOps base domain + +The Auto DevOps base domain is required if you want to make use of [Auto +Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It is defined +under the project's CI/CD settings while [enabling Auto DevOps](#enabling-auto-devops). +It can also be set at the project or group level as a variable, `AUTO_DEVOPS_DOMAIN`. + +A wildcard DNS A record matching the base domain is required, for example, +given a base domain of `example.com`, you'd need a DNS entry like: + +``` +*.example.com 3600 A 1.2.3.4 +``` + +where `example.com` is the domain name under which the deployed apps will be served, +and `1.2.3.4` is the IP address of your load balancer; generally NGINX +([see prerequisites](#prerequisites)). How to set up the DNS record is beyond +the scope of this document; you should check with your DNS provider. + +Once set up, all requests will hit the load balancer, which in turn will route +them to the Kubernetes pods that run your application(s). -In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps -section. Select "Enable Auto DevOps", add in your base domain, and save. +NOTE: **Note:** +If GitLab is installed using the [GitLab Omnibus Helm Chart], there are two +options: provide a static IP, or have one assigned. For more information see the +relevant docs on the [network prerequisites](../../install/kubernetes/gitlab_omnibus.md#networking-prerequisites). - +## Quick start + +If you are using GitLab.com, see our [quick start guide](quick_start_guide.md) +for using Auto DevOps with GitLab.com and an external Kubernetes cluster on +Google Cloud. + +## Enabling Auto DevOps + +NOTE: **Note:** +If you haven't done already, read the [prerequisites](#prerequisites) to make +full use of Auto DevOps. If this is your fist time, we recommend you follow the +[quick start guide](#quick-start). + +1. Go to your project's **Settings > CI/CD > General pipelines settings** and + find the Auto DevOps section +1. Select "Enable Auto DevOps" +1. Optionally, but recommended, add in the [base domain](#auto-devops-base-domain) + that will be used by Kubernetes to deploy your application +1. Hit **Save changes** for the changes to take effect + +Now that it's enabled, there are a few more steps depending on whether your project +has a `.gitlab-ci.yml` or not: + +- **For projects with no `.gitlab-ci.yml` present:** + A pipeline needs to be triggered either by pushing a new commit to the + repository or manually visiting `https://example.gitlab.com/<username>/<project>/pipelines/new` + and creating a new pipeline for your default branch, generally `master`. +- **For projects with a `.gitlab-ci.yml` present:** + All you need to do is remove your existing `.gitlab-ci.yml`, and you can even + do that in a branch to test Auto DevOps before committing to `master`. ## Stages of Auto DevOps -The following sections describe the stages of Auto DevOps. +The following sections describe the stages of Auto DevOps. Read them carefully +to understand how each one works. ### Auto Build @@ -118,18 +159,24 @@ Auto Build creates a build of the application in one of two ways: to automatically detect and build the application into a Docker image. Either way, the resulting Docker image is automatically pushed to the -[Container Registry][container-registry], tagged with the commit SHA. +[Container Registry][container-registry] and tagged with the commit SHA. + +CAUTION: **Important:** +If you are also using Auto Review Apps and Auto Deploy and choose to provide +your own `Dockerfile`, make sure you expose your application to port +`5000` as this is the port assumed by the default Helm chart. ### Auto Test -Auto Test automatically tests your application using +Auto Test automatically runs the appropriate tests for your application using [Herokuish](https://github.com/gliderlabs/herokuish) and [Heroku -buildpacks](https://devcenter.heroku.com/articles/buildpacks). Auto Test will -analyze your project to detect the language and framework, and run appropriate -tests. Several languages and frameworks are detected automatically, but if your -language is not detected, you may succeed with a [custom -buildpack](#custom-buildpack). +buildpacks](https://devcenter.heroku.com/articles/buildpacks) by analyzing +your project to detect the language and framework. Several languages and +frameworks are detected automatically, but if your language is not detected, +you may succeed with a [custom buildpack](#custom-buildpacks). Check the +[currently supported languages](#currently-supported-languages). +NOTE: **Note:** Auto Test uses tests you already have in your application. If there are no tests, it's up to you to add them. @@ -137,99 +184,110 @@ tests, it's up to you to add them. Auto Code Quality uses the open source [`codeclimate` image](https://hub.docker.com/r/codeclimate/codeclimate/) to run -static analysis and other code checks on the current code, creating a report -that is uploaded as an artifact. In GitLab EE, differences between the source -and target branches are shown in the merge request widget. *GitLab is in no way -affiliated with Code Climate.* +static analysis and other code checks on the current code. The report is +created, and is uploaded as an artifact which you can later download and check +out. In GitLab Enterprise Edition Starter, differences between the source and +target branches are +[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html). ### Auto Review Apps -Auto Review Apps create a [Review App][review-app] for each branch. Review Apps -are temporary application environments based on the branch's code so developers, -designers, QA, product managers, and other reviewers can actually see and -interact with code changes as part of the review process. - -The review app will have a unique URL based on the project name, the branch +NOTE: **Note:** +This is an optional step, since many projects do not have a Kubernetes cluster +available. If the [prerequisites](#prerequisites) are not met, the job will +silently be skipped. + +CAUTION: **Caution:** +Your apps should *not* be manipulated outside of Helm (using Kubernetes directly.) +This can cause confusion with Helm not detecting the change, and subsequent +deploys with Auto DevOps can undo your changes. Also, if you change something +and want to undo it by deploying again, Helm may not detect that anything changed +in the first place, and thus not realize that it needs to re-apply the old config. + +[Review Apps][review-app] are temporary application environments based on the +branch's code so developers, designers, QA, product managers, and other +reviewers can actually see and interact with code changes as part of the review +process. Auto Review Apps create a Review App for each branch. + +The Review App will have a unique URL based on the project name, the branch name, and a unique number, combined with the Auto DevOps base domain. For example, `user-project-branch-1234.example.com`. A link to the Review App shows up in the merge request widget for easy discovery. When the branch is deleted, for example after the merge request is merged, the Review App will automatically be deleted. -This is an optional step, since many projects do not have a Kubernetes cluster -available. If the Kubernetes service is not configured, or if the variable -`AUTO_DEVOPS_DOMAIN` is not available (usually set automatically by the Auto -DevOps setting), the job will silently be skipped. - ### Auto Deploy -After a branch or merge request is merged into `master`, Auto Deploy deploys the -application to a `production` environment in the Kubernetes cluster, with a -namespace based on the project name and unique project ID. e.g. `project-4321`. +NOTE: **Note:** This is an optional step, since many projects do not have a Kubernetes cluster -available. If the Kubernetes service is not configured, or if the variable -`AUTO_DEVOPS_DOMAIN` is not available (usually set automatically by the Auto -DevOps setting), the job will silently be skipped. +available. If the [prerequisites](#prerequisites) are not met, the job will +silently be skipped. + +CAUTION: **Caution:** +Your apps should *not* be manipulated outside of Helm (using Kubernetes directly.) +This can cause confusion with Helm not detecting the change, and subsequent +deploys with Auto DevOps can undo your changes. Also, if you change something +and want to undo it by deploying again, Helm may not detect that anything changed +in the first place, and thus not realize that it needs to re-apply the old config. + +After a branch or merge request is merged into the project's default branch (usually +`master`), Auto Deploy deploys the application to a `production` environment in +the Kubernetes cluster, with a namespace based on the project name and unique +project ID, for example `project-4321`. Auto Deploy doesn't include deployments to staging or canary by default, but the -Auto DevOps template contains job definitions for these tasks if you want to +[Auto DevOps template] contains job definitions for these tasks if you want to enable them. +You can make use of [environment variables](#helm-chart-variables) to automatically +scale your pod replicas. + ### Auto Monitoring +NOTE: **Note:** +Check the [prerequisites](#prerequisites) for Auto Monitoring to make this stage +work. + Once your application is deployed, Auto Monitoring makes it possible to monitor your application's server and response metrics right out of the box. Auto Monitoring uses [Prometheus](../../user/project/integrations/prometheus.md) to get system metrics such as CPU and memory usage directly from [Kubernetes](../../user/project/integrations/prometheus_library/kubernetes.md), and response metrics such as HTTP error rates, latency, and throughput from the -[NGINX -server](../../user/project/integrations/prometheus_library/nginx_ingress.md). +[NGINX server](../../user/project/integrations/prometheus_library/nginx_ingress.md). -* Response Metrics: latency, throughput, error rate -* System Metrics: CPU utilization, memory utilization +The metrics include: -To view the metrics, open the [Monitoring dashboard for a deployed environment](../../ci/environments.md#monitoring-environments). +- **Response Metrics:** latency, throughput, error rate +- **System Metrics:** CPU utilization, memory utilization - - -### Configuring Auto Monitoring - -If GitLab has been deployed using the -[omnibus-gitlab](../../install/kubernetes/gitlab_omnibus.md) Helm chart, no +If GitLab has been deployed using the [GitLab Omnibus Helm Chart], no configuration is required. -If you have installed GitLab using a different method: +If you have installed GitLab using a different method, you need to: 1. [Deploy Prometheus](../../user/project/integrations/prometheus.md#configuring-your-own-prometheus-server-within-kubernetes) into your Kubernetes cluster -1. If you would like response metrics, ensure you are running at least version 0.9.0 of NGINX Ingress and [enable Prometheus metrics](https://github.com/kubernetes/ingress/blob/master/examples/customization/custom-vts-metrics/nginx/nginx-vts-metrics-conf.yaml). -1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) the NGINX Ingress deployment to be scraped by Prometheus using `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. - -## Customizing +1. If you would like response metrics, ensure you are running at least version + 0.9.0 of NGINX Ingress and + [enable Prometheus metrics](https://github.com/kubernetes/ingress/blob/master/examples/customization/custom-vts-metrics/nginx/nginx-vts-metrics-conf.yaml). +1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) + the NGINX Ingress deployment to be scraped by Prometheus using + `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. -### PostgreSQL Database Support +To view the metrics, open the +[Monitoring dashboard for a deployed environment](../../ci/environments.md#monitoring-environments). -In order to support applications that require a database, -[PostgreSQL][postgresql] is provisioned by default. Credentials to access the -database are preconfigured, but can be customized by setting the associated -[variables](#postgresql-variables). These credentials can be used for defining a -`DATABASE_URL` of the format: -`postgres://user:password@postgres-host:postgres-port/postgres-database`. - -PostgreSQL provisioning can be disabled by creating a project variable -`POSTGRES_ENABLED` set to `false`. - -#### PostgreSQL Variables + -Any variables set at the project or group level will override variables set in -the CI/CD configuration. +## Customizing -1. `POSTGRES_ENABLED: "false"`: disable automatic deployment of PostgreSQL -1. `POSTGRES_USER: "my-user"`: use custom username for PostgreSQL -1. `POSTGRES_PASSWORD: "password"`: use custom password for PostgreSQL -1. `POSTGRES_DB: "my-database"`: use custom database name for PostgreSQL +While Auto DevOps provides great defaults to get you started, you can customize +almost everything to fit your needs; from custom [buildpacks](#custom-buildpacks), +to [`Dockerfile`s](#custom-dockerfile), [Helm charts](#custom-helm-chart), or +even copying the complete [CI/CD configuration](#customizing-gitlab-ci-yml) +into your project to enable staging and canary deployments, and more. -### Custom buildpack +### Custom buildpacks If the automatic buildpack detection fails for your project, or if you want to use a custom buildpack, you can override the buildpack using a project variable @@ -237,12 +295,12 @@ or a `.buildpack` file in your project: - **Project variable** - Create a project variable `BUILDPACK_URL` with the URL of the buildpack to use. - - **`.buildpack` file** - Add a file in your project's repo called `.buildpack` and add the URL of the buildpack to use on a line in the file. If you want to - use multiple buildpacks, you can enter them in, one on each line + use multiple buildpacks, you can enter them in, one on each line. - >**Note:** Using multiple buildpacks may break Auto Test. +CAUTION: **Caution:** +Using multiple buildpacks isn't yet supported by Auto DevOps. ### Custom `Dockerfile` @@ -253,37 +311,138 @@ Dockerfile is based on [Alpine](https://hub.docker.com/_/alpine/). ### Custom Helm Chart -Auto DevOps uses Helm to deploy your application to Kubernetes. You can override -the Helm chart used by bundling up a chart into your project repo or by -specifying a project variable. - -**Bundled chart** - If your project has a `chart` directory with a `Chart.yaml` -file in it, Auto DevOps will detect the chart and use it instead of the default -chart. This can be a great way to control exactly how your application is -deployed. +Auto DevOps uses [Helm](https://helm.sh/) to deploy your application to Kubernetes. +You can override the Helm chart used by bundling up a chart into your project +repo or by specifying a project variable: -**Project variable** - Create a project variable `AUTO_DEVOPS_CHART` with the -URL of a custom chart to use. +- **Bundled chart** - If your project has a `./charts` directory with a `Chart.yaml` + file in it, Auto DevOps will detect the chart and use it instead of the [default + one](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). + This can be a great way to control exactly how your application is deployed. +- **Project variable** - Create a [project variable](../../ci/variables/README.md#secret-variables) + `AUTO_DEVOPS_CHART` with the URL of a custom chart to use. -### Enable staging, canaries, and more with custom `.gitlab-ci.yml` +### Customizing `.gitlab-ci.yml` If you want to modify the CI/CD pipeline used by Auto DevOps, you can copy the -Auto DevOps template into your project's repo and edit as you see fit. +[Auto DevOps template] into your project's repo and edit as you see fit. + +Assuming that your project is new or it doesn't have a `.gitlab-ci.yml` file +present: -From your project home page, click on the `Set up CI` button, or click on the `+` -button and `New file` and pick `.gitlab-ci.yml` as the template type, or view an -existing `.gitlab-ci.yml` file. Then select "Auto DevOps" from the template -dropdown. You will then be able to edit or add any jobs needed. +1. From your project home page, either click on the "Set up CI" button, or click + on the plus button and (`+`), then "New file" +1. Pick `.gitlab-ci.yml` as the template type +1. Select "Auto-DevOps" from the template dropdown +1. Edit the template or add any jobs needed +1. Give an appropriate commit message and hit "Commit changes" -For example, if you want deploys to go to a staging environment instead of -directly to a production environment, you can enable the `staging` job by +TIP: **Tip:** The Auto DevOps template includes useful comments to help you +customize it. For example, if you want deployments to go to a staging environment +instead of directly to a production one, you can enable the `staging` job by renaming `.staging` to `staging`. Then make sure to uncomment the `when` key of the `production` job to turn it into a manual action instead of deploying automatically. +### PostgreSQL database support + +In order to support applications that require a database, +[PostgreSQL][postgresql] is provisioned by default. The credentials to access +the database are preconfigured, but can be customized by setting the associated +[variables](#environment-variables). These credentials can be used for defining a +`DATABASE_URL` of the format: + +```yaml +postgres://user:password@postgres-host:postgres-port/postgres-database +``` + +### Environment variables + +The following variables can be used for setting up the Auto DevOps domain, +providing a custom Helm chart, or scaling your application. PostgreSQL can be +also be customized, and you can easily use a [custom buildpack](#custom-buildpacks). + +| **Variable** | **Description** | +| ------------ | --------------- | +| `AUTO_DEVOPS_DOMAIN` | The [Auto DevOps domain](#auto-devops-domain); by default set automatically by the [Auto DevOps setting](#enabling-auto-devops). | +| `AUTO_DEVOPS_CHART` | The Helm Chart used to deploy your apps; defaults to the one [provided by GitLab](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). | +| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment; defaults to 1. | +| `CANARY_PRODUCTION_REPLICAS`| The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) in the production environment. | +| `POSTGRES_ENABLED` | Whether PostgreSQL is enabled; defaults to `"true"`. Set to `false` to disable the automatic deployment of PostgreSQL. | +| `POSTGRES_USER` | The PostgreSQL user; defaults to `user`. Set it to use a custom username. | +| `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. | +| `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-variables-environment-variables). Set it to use a custom database name. | +| `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142`| + +TIP: **Tip:** +Set up the replica variables using a +[project variable](../../ci/variables/README.md#secret-variables) +and scale your application by just redeploying it! + +CAUTION: **Caution:** +You should *not* scale your application using Kubernetes directly. This can +cause confusion with Helm not detecting the change, and subsequent deploys with +Auto DevOps can undo your changes. + +#### Advanced replica variables setup + +Apart from the two replica-related variables for production mentioned above, +you can also use others for different environments. + +There's a very specific mapping between Kubernetes' label named `track`, +GitLab CI/CD environment names, and the replicas environment variable. +The general rule is: `TRACK_ENV_REPLICAS`. Where: + +- `TRACK`: The capitalized value of the `track` + [Kubernetes label](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/) + in the Helm Chart app definition. If not set, it will not be taken into account + to the variable name. +- `ENV`: The capitalized environment name of the deploy job that is set in + `.gitlab-ci.yml`. + +That way, you can define your own `TRACK_ENV_REPLICAS` variables with which +you will be able to scale the pod's replicas easily. + +In the example below, the environment's name is `qa` which would result in +looking for the `QA_REPLICAS` environment variable: + +```yaml +QA testing: + stage: deploy + environment: + name: qa + script: + - deploy qa +``` + +If, in addition, there was also a `track: foo` defined in the application's Helm +chart, like: + +```yaml +replicaCount: 1 +image: + repository: gitlab.example.com/group/project + tag: stable + pullPolicy: Always + secrets: + - name: gitlab-registry +application: + track: foo + tier: web +service: + enabled: true + name: web + type: ClusterIP + url: http://my.host.com/ + externalPort: 5000 + internalPort: 5000 +``` + +then the environment variable would be `FOO_QA_REPLICAS`. + ## Currently supported languages ->**Note:** +NOTE: **Note:** Not all buildpacks support Auto Test yet, as it's a relatively new enhancement. All of Heroku's [officially supported languages](https://devcenter.heroku.com/articles/heroku-ci#currently-supported-languages) @@ -294,22 +453,28 @@ multi-buildpack does not. As of GitLab 10.0, the supported buildpacks are: ``` -* heroku-buildpack-multi v1.0.0 -* heroku-buildpack-ruby v168 -* heroku-buildpack-nodejs v99 -* heroku-buildpack-clojure v77 -* heroku-buildpack-python v99 -* heroku-buildpack-java v53 -* heroku-buildpack-gradle v23 -* heroku-buildpack-scala v78 -* heroku-buildpack-play v26 -* heroku-buildpack-php v122 -* heroku-buildpack-go v72 -* heroku-buildpack-erlang fa17af9 -* buildpack-nginx v8 +- heroku-buildpack-multi v1.0.0 +- heroku-buildpack-ruby v168 +- heroku-buildpack-nodejs v99 +- heroku-buildpack-clojure v77 +- heroku-buildpack-python v99 +- heroku-buildpack-java v53 +- heroku-buildpack-gradle v23 +- heroku-buildpack-scala v78 +- heroku-buildpack-play v26 +- heroku-buildpack-php v122 +- heroku-buildpack-go v72 +- heroku-buildpack-erlang fa17af9 +- buildpack-nginx v8 ``` -## Private Project Support - Experimental +## Limitations + +The following restrictions apply. + +### Private project support + +CAUTION: **Caution:** Private project support in Auto DevOps is experimental. When a project has been marked as private, GitLab's [Container Registry][container-registry] requires authentication when downloading @@ -319,27 +484,10 @@ Authentication credentials will be valid while the pipeline is running, allowing for a successful initial deployment. After the pipeline completes, Kubernetes will no longer be able to access the -container registry. **Restarting a pod, scaling a service, or other actions which -require on-going access to the registry will fail**. On-going secure access is +Container Registry. **Restarting a pod, scaling a service, or other actions which +require on-going access to the registry may fail**. On-going secure access is planned for a subsequent release. -## Disable the banner instance wide - -If an administrater would like to disable the banners on an instance level, this -feature can be disabled either through the console: - -```basb -$ gitlab-rails console -[1] pry(main)> Feature.get(:auto_devops_banner_disabled).disable -=> true -``` - -Or through the HTTP API with the admin access token: - -``` -curl --data "value=true" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled -``` - ## Troubleshooting - Auto Build and Auto Test may fail in detecting your language/framework. There @@ -347,13 +495,36 @@ curl --data "value=true" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https:// key files the buildpack is looking for. For example, for ruby apps, you must have a `Gemfile` to be properly detected, even though it is possible to write a Ruby app without a `Gemfile`. Try specifying a [custom - buildpack](#custom-buildpack). + buildpack](#custom-buildpacks). - Auto Test may fail because of a mismatch between testing frameworks. In this case, you may need to customize your `.gitlab-ci.yml` with your test commands. +### Disable the banner instance wide + +If an administrator would like to disable the banners on an instance level, this +feature can be disabled either through the console: + +```sh +sudo gitlab-rails console +``` + +Then run: + +```ruby +Feature.get(:auto_devops_banner_disabled).enable +``` + +Or through the HTTP API with an admin access token: + +```sh +curl --data "value=true" --header "PRIVATE-TOKEN: private_token" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled +``` + [ce-37115]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37115 [kubernetes-service]: ../../user/project/integrations/kubernetes.md [docker-in-docker]: ../../docker/using_docker_build.md#use-docker-in-docker-executor [review-app]: ../../ci/review_apps/index.md [container-registry]: ../../user/project/container_registry.md [postgresql]: https://www.postgresql.org/ +[Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml +[GitLab Omnibus Helm Chart]: ../../install/kubernetes/gitlab_omnibus.md diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 564dd3222ac..ffe05519d7b 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -1,14 +1,15 @@ # Auto DevOps: quick start guide -> [Introduced][ce-37115] in GitLab 10.0. Auto DevOps is currently in Beta and -**not recommended for production use**. +DANGER: Auto DevOps is currently in **Beta** and _not recommended for production use_. + +> [Introduced][ce-37115] in GitLab 10.0. This is a step-by-step guide to deploying a project hosted on GitLab.com to Google Cloud, using Auto DevOps. We made a minimal [Ruby -application](https://gitlab.com/gitlab-examples/minimal-ruby-app) to use as an -example for this guide. It contains two files: +application](https://gitlab.com/auto-devops-examples/minimal-ruby-app) to use +as an example for this guide. It contains two main files: * `server.rb` - our application. It will start an HTTP server on port 5000 and render "Hello, world!" @@ -113,11 +114,9 @@ assigned to the cluster IP. In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps section. Select "Enable Auto DevOps", add in your base domain, and save. - - Next, a pipeline needs to be triggered. Since the test project doesn't have a `.gitlab-ci.yml`, you need to either push a change to the repository or -manually visit `https://gitlab.com/<username>/minimal-ruby-app/pipelines/run`, +manually visit `https://gitlab.com/<username>/minimal-ruby-app/pipelines/new`, where `<username>` is your username. This will create a new pipeline with several jobs: `build`, `test`, `codequality`, diff --git a/doc/university/training/gitlab_flow.md b/doc/university/training/gitlab_flow.md index a7db1f2e069..a7db1f2e069 100755..100644 --- a/doc/university/training/gitlab_flow.md +++ b/doc/university/training/gitlab_flow.md diff --git a/doc/university/training/index.md b/doc/university/training/index.md index 03179ff5a77..03179ff5a77 100755..100644 --- a/doc/university/training/index.md +++ b/doc/university/training/index.md diff --git a/doc/university/training/topics/additional_resources.md b/doc/university/training/topics/additional_resources.md index 3ed601625cf..3ed601625cf 100755..100644 --- a/doc/university/training/topics/additional_resources.md +++ b/doc/university/training/topics/additional_resources.md diff --git a/doc/university/training/topics/agile_git.md b/doc/university/training/topics/agile_git.md index e6e4fea9b51..e6e4fea9b51 100755..100644 --- a/doc/university/training/topics/agile_git.md +++ b/doc/university/training/topics/agile_git.md diff --git a/doc/university/training/topics/bisect.md b/doc/university/training/topics/bisect.md index a60c4365e0c..a60c4365e0c 100755..100644 --- a/doc/university/training/topics/bisect.md +++ b/doc/university/training/topics/bisect.md diff --git a/doc/university/training/topics/cherry_picking.md b/doc/university/training/topics/cherry_picking.md index af7a70a2818..af7a70a2818 100755..100644 --- a/doc/university/training/topics/cherry_picking.md +++ b/doc/university/training/topics/cherry_picking.md diff --git a/doc/university/training/topics/env_setup.md b/doc/university/training/topics/env_setup.md index 8149379b36f..8149379b36f 100755..100644 --- a/doc/university/training/topics/env_setup.md +++ b/doc/university/training/topics/env_setup.md diff --git a/doc/university/training/topics/explore_gitlab.md b/doc/university/training/topics/explore_gitlab.md index b65457728c0..b65457728c0 100755..100644 --- a/doc/university/training/topics/explore_gitlab.md +++ b/doc/university/training/topics/explore_gitlab.md diff --git a/doc/university/training/topics/feature_branching.md b/doc/university/training/topics/feature_branching.md index 4b34406ea75..4b34406ea75 100755..100644 --- a/doc/university/training/topics/feature_branching.md +++ b/doc/university/training/topics/feature_branching.md diff --git a/doc/university/training/topics/getting_started.md b/doc/university/training/topics/getting_started.md index ec7bb2631aa..ec7bb2631aa 100755..100644 --- a/doc/university/training/topics/getting_started.md +++ b/doc/university/training/topics/getting_started.md diff --git a/doc/university/training/topics/git_add.md b/doc/university/training/topics/git_add.md index 9ffb4b9c859..9ffb4b9c859 100755..100644 --- a/doc/university/training/topics/git_add.md +++ b/doc/university/training/topics/git_add.md diff --git a/doc/university/training/topics/git_intro.md b/doc/university/training/topics/git_intro.md index ca1ff29d93b..ca1ff29d93b 100755..100644 --- a/doc/university/training/topics/git_intro.md +++ b/doc/university/training/topics/git_intro.md diff --git a/doc/university/training/topics/git_log.md b/doc/university/training/topics/git_log.md index 32ebceff491..32ebceff491 100755..100644 --- a/doc/university/training/topics/git_log.md +++ b/doc/university/training/topics/git_log.md diff --git a/doc/university/training/topics/gitlab_flow.md b/doc/university/training/topics/gitlab_flow.md index 8e5d3baf959..8e5d3baf959 100755..100644 --- a/doc/university/training/topics/gitlab_flow.md +++ b/doc/university/training/topics/gitlab_flow.md diff --git a/doc/university/training/topics/merge_conflicts.md b/doc/university/training/topics/merge_conflicts.md index 77807b3e7ef..77807b3e7ef 100755..100644 --- a/doc/university/training/topics/merge_conflicts.md +++ b/doc/university/training/topics/merge_conflicts.md diff --git a/doc/university/training/topics/merge_requests.md b/doc/university/training/topics/merge_requests.md index 5b446f02f63..5b446f02f63 100755..100644 --- a/doc/university/training/topics/merge_requests.md +++ b/doc/university/training/topics/merge_requests.md diff --git a/doc/university/training/topics/rollback_commits.md b/doc/university/training/topics/rollback_commits.md index cf647284604..cf647284604 100755..100644 --- a/doc/university/training/topics/rollback_commits.md +++ b/doc/university/training/topics/rollback_commits.md diff --git a/doc/university/training/topics/stash.md b/doc/university/training/topics/stash.md index c1bdda32645..c1bdda32645 100755..100644 --- a/doc/university/training/topics/stash.md +++ b/doc/university/training/topics/stash.md diff --git a/doc/university/training/topics/subtree.md b/doc/university/training/topics/subtree.md index 5d869af64c1..5d869af64c1 100755..100644 --- a/doc/university/training/topics/subtree.md +++ b/doc/university/training/topics/subtree.md diff --git a/doc/university/training/topics/tags.md b/doc/university/training/topics/tags.md index e9607b5a875..e9607b5a875 100755..100644 --- a/doc/university/training/topics/tags.md +++ b/doc/university/training/topics/tags.md diff --git a/doc/university/training/topics/unstage.md b/doc/university/training/topics/unstage.md index 17dbb64b9e6..17dbb64b9e6 100755..100644 --- a/doc/university/training/topics/unstage.md +++ b/doc/university/training/topics/unstage.md diff --git a/doc/university/training/user_training.md b/doc/university/training/user_training.md index 9e38df26b6a..9e38df26b6a 100755..100644 --- a/doc/university/training/user_training.md +++ b/doc/university/training/user_training.md diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 70934f9960a..843fb4ce26b 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -18,7 +18,7 @@ traffic until the system is ready or restart the container as needed. To access monitoring resources, the client IP needs to be included in a whitelist. -[Read how to add IPs to a whitelist for the monitoring endpoints.][admin]. +[Read how to add IPs to a whitelist for the monitoring endpoints][admin]. ## Using the endpoint diff --git a/doc/user/group/index.md b/doc/user/group/index.md index db0242f1324..a1671f9dd91 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -188,6 +188,27 @@ Besides giving you the option to edit any settings you've previously set when [creating the group](#create-a-new-group), you can also access further configurations for your group. +#### Changing a group's path + +> **Note:** If you want to retain ownership over the original namespace and +protect the URL redirects, then instead of changing a group's path or renaming a +username, you can create a new group and transfer projects to it. + +Changing a group's path can have unintended side effects. + +* Existing web URLs for the group and anything under it (i.e. projects) will +redirect to the new URLs +* Existing Git remote URLs for projects under the group will no longer work, but +Git responses will show an error with the new remote URL +* The original namespace can be claimed again by any group or user, which will +destroy web redirects and Git remote warnings +* If you are vacating the path so it can be claimed by another group or user, +you may need to rename the group name as well since both names and paths must be +unique + +> It is currently not possible to rename a namespace if it contains a +project with container registry tags, because the project cannot be moved. + #### Enforce 2FA to group members Add a security layer to your group by diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index d2478aea4bd..161a3af9903 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -84,10 +84,13 @@ structure. a subgroup. For more information check the [permissions table][permissions]. - For a list of words that are not allowed to be used as group names see the [reserved names][reserved]. +- Users can always create subgroups if they are explicitly added as an Owner to + a parent group even if group creation is disabled by an administrator in their + settings. To create a subgroup: -1. In the group's dashboard go to the **Subgroups** page and click **Create subgroup**. +1. In the group's dashboard go to the **Subgroups** page and click **New subgroup**.  @@ -100,9 +103,7 @@ To create a subgroup: 1. Click the **Create group** button and you will be taken to the new group's dashboard page. ---- - -You can follow the same process to create any subsequent groups. +Follow the same process to create any subsequent groups. ## Membership diff --git a/doc/user/markdown.md b/doc/user/markdown.md index b42b8f0a525..454988b9b80 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -596,6 +596,30 @@ See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubyd <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> </dl> +#### Details and Summary + +Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) and [`<summary>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary) tags. This is especially useful for collapsing long logs so they take up less screen space. + +<p> +<details> +<summary>Click me to collapse/fold.</summary> +These details will remain hidden until expanded. + +<pre><code>PASTE LOGS HERE</code></pre> +</details> +</p> + +**Note:** Unfortunately Markdown is not supported inside these tags, as described by the [markdown specification](https://daringfireball.net/projects/markdown/syntax#html). You can work around this by using HTML, for example you can use `<pre><code>` tags instead of [code fences](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting). + +```html +<details> +<summary>Click me to collapse/fold.</summary> +These details will remain hidden until expanded. + +<pre><code>PASTE LOGS HERE</code></pre> +</details> +``` + ### Horizontal Rule ``` diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 7d25970fcb1..5ebb88bf324 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -8,10 +8,27 @@ experience according to the best approach to their cases. Your `username` is a unique [`namespace`](../group/index.md#namespaces) related to your user ID. +### Changing your username + You can change your `username` from your -[profile settings](#profile-settings). To avoid breaking -paths when you change your `username`, we suggest you follow -[this procedure from the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#how-to-change-your-username-at-gitlabcom). +[profile settings](#profile-settings). + +> **Note:** If you want to retain ownership over the original namespace and +protect the URL redirects, then instead of changing your username, you can +create a new group and transfer projects to it. +Alternatively, you can follow [this detailed procedure from the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#how-to-change-your-username-at-gitlabcom). + +Changing your username can have unintended side effects. + +* Existing web URLs for the user and anything under it (i.e. projects) will +redirect to the new URLs +* Existing Git remote URLs for projects under the user will no longer work, but +Git responses will show an error with the new remote URL +* The original namespace can be claimed again by any group or user, which will +destroy any web redirects and Git remote warnings + +> It is currently not possible to rename a namespace if it contains a +project with container registry tags, because the project cannot be moved. ## User profile diff --git a/doc/user/project/integrations/img/kubernetes_configuration.png b/doc/user/project/integrations/img/kubernetes_configuration.png Binary files differindex 349a2dc8456..e535e2b8d46 100644 --- a/doc/user/project/integrations/img/kubernetes_configuration.png +++ b/doc/user/project/integrations/img/kubernetes_configuration.png diff --git a/doc/user/project/integrations/img/webhook_logs.png b/doc/user/project/integrations/img/webhook_logs.png Binary files differindex 917068d9398..917068d9398 100755..100644 --- a/doc/user/project/integrations/img/webhook_logs.png +++ b/doc/user/project/integrations/img/webhook_logs.png diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index f4000523938..e9738b683f9 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -1,3 +1,7 @@ +--- +last_updated: 2017-09-25 +--- + # GitLab Kubernetes / OpenShift integration GitLab can be configured to interact with Kubernetes, or other systems using the @@ -6,62 +10,114 @@ Kubernetes API (such as OpenShift). Each project can be configured to connect to a different Kubernetes cluster, see the [configuration](#configuration) section. -If you have a single cluster that you want to use for all your projects, -you can pre-fill the settings page with a default template. To configure the -template, see the [Services Templates](services_templates.md) document. - ## Configuration Navigate to the [Integrations page](project_services.md#accessing-the-project-services) -of your project and select the **Kubernetes** service to configure it. +of your project and select the **Kubernetes** service to configure it. Fill in +all the needed parameters, check the "Active" checkbox and hit **Save changes** +for the changes to take effect.  -The Kubernetes service takes the following arguments: - -1. API URL -1. Custom CA bundle -1. Kubernetes namespace -1. Service token - -The API URL is the URL that GitLab uses to access the Kubernetes API. Kubernetes -exposes several APIs - we want the "base" URL that is common to all of them, -e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. - -GitLab authenticates against Kubernetes using service tokens, which are -scoped to a particular `namespace`. If you don't have a service token yet, -you can follow the -[Kubernetes documentation](http://kubernetes.io/docs/user-guide/service-accounts/) -to create one. You can also view or create service tokens in the -[Kubernetes dashboard](http://kubernetes.io/docs/user-guide/ui/) - visit -`Config -> Secrets`. - -Fill in the service token and namespace according to the values you just got. -If the API is using a self-signed TLS certificate, you'll also need to include -the `ca.crt` contents as the `Custom CA bundle`. +The Kubernetes service takes the following parameters: + +- **API URL** - + It's the URL that GitLab uses to access the Kubernetes API. Kubernetes + exposes several APIs, we want the "base" URL that is common to all of them, + e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. +- **CA certificate** (optional) - + If the API is using a self-signed TLS certificate, you'll also need to include + the `ca.crt` contents here. +- **Project namespace** (optional) - The following apply: + - By default you don't have to fill it in; by leaving it blank, GitLab will + create one for you. + - Each project should have a unique namespace. + - The project namespace is not necessarily the namespace of the secret, if + you're using a secret with broader permissions, like the secret from `default`. + - You should **not** use `default` as the project namespace. + - If you or someone created a secret specifically for the project, usually + with limited permissions, the secret's namespace and project namespace may + be the same. +- **Token** - + GitLab authenticates against Kubernetes using service tokens, which are + scoped to a particular `namespace`. If you don't have a service token yet, + you can follow the + [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) + to create one. You can also view or create service tokens in the + [Kubernetes dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/#config) + (under **Config > Secrets**). + +TIP: **Tip:** +If you have a single cluster that you want to use for all your projects, +you can pre-fill the settings page with a default template. To configure the +template, see [Services Templates](services_templates.md). ## Deployment variables -The Kubernetes service exposes following +The Kubernetes service exposes the following [deployment variables](../../../ci/variables/README.md#deployment-variables) in the -GitLab CI build environment: +GitLab CI/CD build environment: -- `KUBE_URL` - equal to the API URL -- `KUBE_TOKEN` +- `KUBE_URL` - Equal to the API URL. +- `KUBE_TOKEN` - The Kubernetes token. - `KUBE_NAMESPACE` - The Kubernetes namespace is auto-generated if not specified. The default value is `<project_name>-<project_id>`. You can overwrite it to use different one if needed, otherwise the `KUBE_NAMESPACE` variable will receive the default value. -- `KUBE_CA_PEM_FILE` - only present if a custom CA bundle was specified. Path +- `KUBE_CA_PEM_FILE` - Only present if a custom CA bundle was specified. Path to a file containing PEM data. -- `KUBE_CA_PEM` (deprecated)- only if a custom CA bundle was specified. Raw PEM data. -- `KUBECONFIG` - Path to a file containing kubeconfig for this deployment. CA bundle would be embedded if specified. +- `KUBE_CA_PEM` (deprecated) - Only if a custom CA bundle was specified. Raw PEM data. +- `KUBECONFIG` - Path to a file containing `kubeconfig` for this deployment. + CA bundle would be embedded if specified. + +## What you can get with the Kubernetes integration + +Here's what you can do with GitLab if you enable the Kubernetes integration. + +### Deploy Boards (EEP) + +> Available in [GitLab Enterprise Edition Premium][ee]. -## Web terminals +GitLab's Deploy Boards offer a consolidated view of the current health and +status of each CI [environment](../../../ci/environments.md) running on Kubernetes, +displaying the status of the pods in the deployment. Developers and other +teammates can view the progress and status of a rollout, pod by pod, in the +workflow they already use without any need to access Kubernetes. ->**NOTE:** -Added in GitLab 8.15. You must be the project owner or have `master` permissions -to use terminals. Support is currently limited to the first container in the +[> Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) + +### Canary Deployments (EEP) + +> Available in [GitLab Enterprise Edition Premium][ee]. + +Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) +and visualize your canary deployments right inside the Deploy Board, without +the need to leave GitLab. + +[> Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) + +### Kubernetes monitoring + +Automatically detect and monitor Kubernetes metrics. Automatic monitoring of +[NGINX ingress](./prometheus_library/nginx.md) is also supported. + +[> Read more about Kubernetes monitoring](prometheus_library/kubernetes.md) + +### Auto DevOps + +Auto DevOps automatically detects, builds, tests, deploys, and monitors your +applications. + +To make full use of Auto DevOps(Auto Deploy, Auto Review Apps, and Auto Monitoring) +you will need the Kubernetes project integration enabled. + +[> Read more about Auto DevOps](../../../topics/autodevops/index.md) + +### Web terminals + +NOTE: **Note:** +Introduced in GitLab 8.15. You must be the project owner or have `master` permissions +to use terminals. Support is limited to the first container in the first pod of your environment. When enabled, the Kubernetes service adds [web terminal](../../../ci/environments.md#web-terminals) @@ -70,3 +126,5 @@ Docker and Kubernetes, so you get a new shell session within your existing containers. To use this integration, you should deploy to Kubernetes using the deployment variables above, ensuring any pods you create are labelled with `app=$CI_ENVIRONMENT_SLUG`. GitLab will do the rest! + +[ee]: https://about.gitlab.com/gitlab-ee/ diff --git a/doc/user/project/integrations/prometheus_library/cloudwatch.md b/doc/user/project/integrations/prometheus_library/cloudwatch.md index cc5cee36d28..34a0b97a171 100644 --- a/doc/user/project/integrations/prometheus_library/cloudwatch.md +++ b/doc/user/project/integrations/prometheus_library/cloudwatch.md @@ -1,8 +1,13 @@ # Monitoring AWS Resources + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 GitLab has support for automatically detecting and monitoring AWS resources, starting with the [Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/). This is provided by leveraging the official [Cloudwatch exporter](https://github.com/prometheus/cloudwatch_exporter), which translates [Cloudwatch metrics](https://aws.amazon.com/cloudwatch/) into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md index d4b5911a91c..518018e5839 100644 --- a/doc/user/project/integrations/prometheus_library/haproxy.md +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -3,6 +3,10 @@ GitLab has support for automatically detecting and monitoring HAProxy. This is provided by leveraging the [HAProxy Exporter](https://github.com/prometheus/haproxy_exporter), which translates HAProxy statistics into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 4d39ae0c4fa..518683965e8 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -1,7 +1,13 @@ # Monitoring Kubernetes + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8935) in GitLab 9.0 -GitLab has support for automatically detecting and monitoring Kubernetes metrics. Kubernetes exposes Node level metrics out of the box via the built-in [Prometheus metrics support in cAdvisor](https://github.com/google/cadvisor). No additional services or exporters are needed. +GitLab has support for automatically detecting and monitoring Kubernetes metrics. + +## Requirements + +The [Prometheus](../prometheus.md) and [Kubernetes](../kubernetes.md) +integration services must be enabled. ## Metrics supported @@ -23,4 +29,4 @@ Prometheus server up and running. You have two options here: In order to isolate and only display relevant metrics for a given environment however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). -If you are using [GitLab Auto-Deploy][../../../ci/autodeploy/index.md] and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. +If you are using [GitLab Auto-Deploy](../../../../ci/autodeploy/index.md) and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index bab22f9a384..7fb8369d3c1 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -1,8 +1,13 @@ # Monitoring NGINX + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 GitLab has support for automatically detecting and monitoring NGINX. This is provided by leveraging the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter), which translates [VTS statistics](https://github.com/vozlt/nginx-module-vts) into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 17a47cfa646..e6f13d0630b 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -1,8 +1,13 @@ # Monitoring NGINX Ingress Controller + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13438) in GitLab 9.5 GitLab has support for automatically detecting and monitoring the Kubernetes NGINX ingress controller. This is provided by leveraging the built in Prometheus metrics included in [version 0.9.0](https://github.com/kubernetes/ingress/blob/master/controllers/nginx/Changelog.md#09-beta1) of the ingress. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/issues/img/button_close_issue.png b/doc/user/project/issues/img/button_close_issue.png Binary files differindex 8fb2e23f58a..8fb2e23f58a 100755..100644 --- a/doc/user/project/issues/img/button_close_issue.png +++ b/doc/user/project/issues/img/button_close_issue.png diff --git a/doc/user/project/issues/img/closing_and_related_issues.png b/doc/user/project/issues/img/closing_and_related_issues.png Binary files differindex c6543e85fdb..c6543e85fdb 100755..100644 --- a/doc/user/project/issues/img/closing_and_related_issues.png +++ b/doc/user/project/issues/img/closing_and_related_issues.png diff --git a/doc/user/project/issues/img/confidential_issues_create.png b/doc/user/project/issues/img/confidential_issues_create.png Binary files differindex 0a141eb39f8..0a141eb39f8 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_create.png +++ b/doc/user/project/issues/img/confidential_issues_create.png diff --git a/doc/user/project/issues/img/confidential_issues_search_guest.png b/doc/user/project/issues/img/confidential_issues_search_guest.png Binary files differindex dc1b4ba8ad7..dc1b4ba8ad7 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_search_guest.png +++ b/doc/user/project/issues/img/confidential_issues_search_guest.png diff --git a/doc/user/project/issues/img/confidential_issues_search_master.png b/doc/user/project/issues/img/confidential_issues_search_master.png Binary files differindex fc01f4da9db..fc01f4da9db 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_search_master.png +++ b/doc/user/project/issues/img/confidential_issues_search_master.png diff --git a/doc/user/project/issues/img/due_dates_create.png b/doc/user/project/issues/img/due_dates_create.png Binary files differindex ece35d44213..ece35d44213 100755..100644 --- a/doc/user/project/issues/img/due_dates_create.png +++ b/doc/user/project/issues/img/due_dates_create.png diff --git a/doc/user/project/issues/img/due_dates_edit_sidebar.png b/doc/user/project/issues/img/due_dates_edit_sidebar.png Binary files differindex d1c7d1eb7e9..d1c7d1eb7e9 100755..100644 --- a/doc/user/project/issues/img/due_dates_edit_sidebar.png +++ b/doc/user/project/issues/img/due_dates_edit_sidebar.png diff --git a/doc/user/project/issues/img/due_dates_issues_index_page.png b/doc/user/project/issues/img/due_dates_issues_index_page.png Binary files differindex 94679436b32..94679436b32 100755..100644 --- a/doc/user/project/issues/img/due_dates_issues_index_page.png +++ b/doc/user/project/issues/img/due_dates_issues_index_page.png diff --git a/doc/user/project/issues/img/due_dates_todos.png b/doc/user/project/issues/img/due_dates_todos.png Binary files differindex 4c124c97f67..4c124c97f67 100755..100644 --- a/doc/user/project/issues/img/due_dates_todos.png +++ b/doc/user/project/issues/img/due_dates_todos.png diff --git a/doc/user/project/issues/img/issue_board.png b/doc/user/project/issues/img/issue_board.png Binary files differindex 1759b28a9ef..1759b28a9ef 100755..100644 --- a/doc/user/project/issues/img/issue_board.png +++ b/doc/user/project/issues/img/issue_board.png diff --git a/doc/user/project/issues/img/issue_template.png b/doc/user/project/issues/img/issue_template.png Binary files differindex c63229a4af2..c63229a4af2 100755..100644 --- a/doc/user/project/issues/img/issue_template.png +++ b/doc/user/project/issues/img/issue_template.png diff --git a/doc/user/project/issues/img/mention_in_issue.png b/doc/user/project/issues/img/mention_in_issue.png Binary files differindex c762a812138..c762a812138 100755..100644 --- a/doc/user/project/issues/img/mention_in_issue.png +++ b/doc/user/project/issues/img/mention_in_issue.png diff --git a/doc/user/project/issues/img/mention_in_merge_request.png b/doc/user/project/issues/img/mention_in_merge_request.png Binary files differindex 681e086d6e0..681e086d6e0 100755..100644 --- a/doc/user/project/issues/img/mention_in_merge_request.png +++ b/doc/user/project/issues/img/mention_in_merge_request.png diff --git a/doc/user/project/issues/img/merge_request_closes_issue.png b/doc/user/project/issues/img/merge_request_closes_issue.png Binary files differindex 6fd27738843..6fd27738843 100755..100644 --- a/doc/user/project/issues/img/merge_request_closes_issue.png +++ b/doc/user/project/issues/img/merge_request_closes_issue.png diff --git a/doc/user/project/issues/img/new_issue.png b/doc/user/project/issues/img/new_issue.png Binary files differindex e72ac49d6b9..e72ac49d6b9 100755..100644 --- a/doc/user/project/issues/img/new_issue.png +++ b/doc/user/project/issues/img/new_issue.png diff --git a/doc/user/project/issues/img/new_issue_from_issue_board.png b/doc/user/project/issues/img/new_issue_from_issue_board.png Binary files differindex 9c2b3ff50fa..9c2b3ff50fa 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_issue_board.png +++ b/doc/user/project/issues/img/new_issue_from_issue_board.png diff --git a/doc/user/project/issues/img/new_issue_from_open_issue.png b/doc/user/project/issues/img/new_issue_from_open_issue.png Binary files differindex 2aed5372830..2aed5372830 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_open_issue.png +++ b/doc/user/project/issues/img/new_issue_from_open_issue.png diff --git a/doc/user/project/issues/img/new_issue_from_projects_dashboard.png b/doc/user/project/issues/img/new_issue_from_projects_dashboard.png Binary files differindex cddf36b7457..cddf36b7457 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_projects_dashboard.png +++ b/doc/user/project/issues/img/new_issue_from_projects_dashboard.png diff --git a/doc/user/project/issues/img/new_issue_from_tracker_list.png b/doc/user/project/issues/img/new_issue_from_tracker_list.png Binary files differindex 7e5413f0b7d..7e5413f0b7d 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_tracker_list.png +++ b/doc/user/project/issues/img/new_issue_from_tracker_list.png diff --git a/doc/user/project/issues/img/sidebar_confidential_issue.png b/doc/user/project/issues/img/sidebar_confidential_issue.png Binary files differindex d99a1ca756e..d99a1ca756e 100755..100644 --- a/doc/user/project/issues/img/sidebar_confidential_issue.png +++ b/doc/user/project/issues/img/sidebar_confidential_issue.png diff --git a/doc/user/project/issues/img/sidebar_not_confidential_issue.png b/doc/user/project/issues/img/sidebar_not_confidential_issue.png Binary files differindex 2e6cbbc5b3a..2e6cbbc5b3a 100755..100644 --- a/doc/user/project/issues/img/sidebar_not_confidential_issue.png +++ b/doc/user/project/issues/img/sidebar_not_confidential_issue.png diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 46fa4378fe7..453e10184f0 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -62,7 +62,7 @@ which is highly recommendable and much faster than hardcoding. If you set up a GitLab Pages project on GitLab.com, it will automatically be accessible under a -[subdomain of `namespace.pages.io`](https://docs.gitlab.com/ce/user/project/pages/). +[subdomain of `namespace.pages.io`](introduction.md#gitlab-pages-on-gitlab-com). The `namespace` is defined by your username on GitLab.com, or the group name you created this project under. @@ -73,6 +73,8 @@ Pages wildcard domain. This guide is valid for any GitLab instance, you just need to replace Pages wildcard domain on GitLab.com (`*.gitlab.io`) with your own. +Learn more about [namespaces](../../group/index.md#namespaces). + ### Practical examples #### Project Websites diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index 53fd1786cfa..0096f8507d2 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -1,9 +1,14 @@ +--- +last_updated: 2017-09-28 +--- + # GitLab Pages from A to Z: Part 3 -> **Article [Type](../../../development/writing_documentation.html#types-of-technical-articles)**: user guide || +> **[Article Type](../../../development/writing_documentation.md#types-of-technical-articles)**: user guide || > **Level**: beginner || > **Author**: [Marcia Ramos](https://gitlab.com/marcia) || -> **Publication date:** 2017/02/22 +> **Publication date:** 2017-02-22 || +> **Last updated**: 2017-09-28 - [Part 1: Static sites and GitLab Pages domains](getting_started_part_one.md) - [Part 2: Quick start guide - Setting up GitLab Pages](getting_started_part_two.md) @@ -16,6 +21,21 @@ As described in the previous part of this series, setting up GitLab Pages with c These steps assume you've already [set your site up](getting_started_part_two.md) and and it's served under the default Pages domain `namespace.gitlab.io`, or `namespace.gitlab.io/project-name`. +### Adding your custom domain to GitLab Pages + +To use one or more custom domain with your Pages site, there are two things +you should consider first, which we'll cover in this guide: + +1. Either if you're adding a **root domain** or a **subdomain**, for which +you'll need to set up [DNS records](#dns-records) +1. Whether you want to add an [SSL/TLS certificate](#ssl-tls-certificates) or not + +To finish the association, you need to [add your domain to your project's Pages settings](#add-your-custom-domain-to-gitlab-pages-settings). + +Let's start from the beginning with [DNS records](#dns-records). +If you already know how they work and want to skip the introduction to DNS, +you may be interested in skipping it until the [TL;DR](#tl-dr) section below. + ### DNS Records A Domain Name System (DNS) web service routes visitors to websites @@ -99,6 +119,29 @@ domain. E.g., **do not** point your `subdomain.domain.com` to `namespace.gitlab.io.` or `namespace.gitlab.io/`. > - GitLab Pages IP on GitLab.com [has been changed](https://about.gitlab.com/2017/03/06/we-are-changing-the-ip-of-gitlab-pages-on-gitlab-com/) from `104.208.235.32` to `52.167.214.135`. +### Add your custom domain to GitLab Pages settings + +Once you've set the DNS record, you'll need navigate to your project's +**Setting > Pages** and click **+ New domain** to add your custom domain to +GitLab Pages. You can choose whether to add an [SSL/TLS certificate](#ssl-tls-certificates) +to make your website accessible under HTTPS or leave it blank. If don't add a certificate, +your site will be accessible only via HTTP: + + + +You can add more than one alias (custom domains and subdomains) to the same project. +An alias can be understood as having many doors leading to the same room. + +All the aliases you've set to your site will be listed on **Setting > Pages**. +From that page, you can view, add, and remove them. + +Note that [DNS propagation may take some time (up to 24h)](http://www.inmotionhosting.com/support/domain-names/dns-nameserver-changes/domain-names-dns-changes), +although it's usually a matter of minutes to complete. Until it does, visit attempts +to your domain will respond with a 404. + +Read through the [general documentation on GitLab Pages](introduction.md#add-a-custom-domain-to-your-pages-website) to learn more about adding +custom domains to GitLab Pages sites. + ### SSL/TLS Certificates Every GitLab Pages project on GitLab.com will be available under diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 9ecf7a3a8e7..4fcdfa7b281 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -28,7 +28,8 @@ In general there are two types of pages one might create: - Pages per project (`username.example.io/projectname` or `groupname.example.io/projectname`) In GitLab, usernames and groupnames are unique and we often refer to them -as namespaces. There can be only one namespace in a GitLab instance. Below you +as [namespaces](../../group/index.md#namespaces). There can be only one namespace +in a GitLab instance. Below you can see the connection between the type of GitLab Pages, what the project name that is created on GitLab looks like and the website URL it will be ultimately be served on. @@ -98,6 +99,9 @@ The steps to create a project page for a user or a group are identical: A user's project will be served under `http(s)://username.example.io/projectname` whereas a group's project under `http(s)://groupname.example.io/projectname`. +For practical examples for group and project Pages, read through the guide +[GitLab Pages from A to Z: Part 1 - Static sites and GitLab Pages domains](getting_started_part_one.md#practical-examples). + ## Quick Start Read through [GitLab Pages Quick Start Guide][pages-quick] or watch the video tutorial on @@ -111,6 +115,9 @@ The key thing about GitLab Pages is the `.gitlab-ci.yml` file, something that gives you absolute control over the build process. You can actually watch your website being built live by following the CI job traces. +For a simplified user guide on setting up GitLab CI/CD for Pages, read through +the article [GitLab Pages from A to Z: Part 4 - Creating and Tweaking `.gitlab-ci.yml` for GitLab Pages](getting_started_part_four.md#creating-and-tweaking-gitlab-ci-yml-for-gitlab-pages) + > **Note:** > Before reading this section, make sure you familiarize yourself with GitLab CI > and the specific syntax of[`.gitlab-ci.yml`][yaml] by @@ -311,6 +318,9 @@ Visit the GitLab Pages group for a full list of example projects: ### Add a custom domain to your Pages website +For a complete guide on Pages domains, read through the article +[GitLab Pages from A to Z: Part 3 - Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](getting_started_part_three.md#setting-up-custom-domains-dns-records-and-ssl-tls-certificates) + If this setting is enabled by your GitLab administrator, you should be able to see the **New Domain** button when visiting your project's settings through the gear icon in the top right and then navigating to **Pages**. @@ -349,6 +359,9 @@ private key when adding a new domain.  +For a complete guide on Pages domains, read through the article +[GitLab Pages from A to Z: Part 3 - Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](getting_started_part_three.md#setting-up-custom-domains-dns-records-and-ssl-tls-certificates) + ### Custom error codes pages You can provide your own 403 and 404 error pages by creating the `403.html` and @@ -387,6 +400,8 @@ If you are using GitLab.com to host your website, then: The rest of the guide still applies. +See also: [GitLab Pages from A to Z: Part 1 - Static sites and GitLab Pages domains](getting_started_part_one.md#gitlab-pages-domain). + ## Limitations When using Pages under the general domain of a GitLab instance (`*.example.io`), diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index dfe43c6b691..29e04a0ccf0 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -113,25 +113,25 @@ started: 1. Use the following command to list the private GPG key you just created: ``` - gpg --list-secret-keys --keyid-format 0xLONG mr@robot.sh + gpg --list-secret-keys --keyid-format LONG mr@robot.sh ``` Replace `mr@robot.sh` with the email address you entered above. 1. Copy the GPG key ID that starts with `sec`. In the following example, that's - `0x30F2B65B9246B6CA`: + `30F2B65B9246B6CA`: ``` - sec rsa4096/0x30F2B65B9246B6CA 2017-08-18 [SC] + sec rsa4096/30F2B65B9246B6CA 2017-08-18 [SC] D5E4F29F3275DC0CDA8FFC8730F2B65B9246B6CA uid [ultimate] Mr. Robot <mr@robot.sh> - ssb rsa4096/0xB7ABC0813E4028C0 2017-08-18 [E] + ssb rsa4096/B7ABC0813E4028C0 2017-08-18 [E] ``` 1. Export the public key of that ID (replace your key ID from the previous step): ``` - gpg --armor --export 0x30F2B65B9246B6CA + gpg --armor --export 30F2B65B9246B6CA ``` 1. Finally, copy the public key and [add it in your profile settings](#adding-a-gpg-key-to-your-account) @@ -167,28 +167,28 @@ key to use. 1. Use the following command to list the private GPG key you just created: ``` - gpg --list-secret-keys --keyid-format 0xLONG mr@robot.sh + gpg --list-secret-keys --keyid-format LONG mr@robot.sh ``` Replace `mr@robot.sh` with the email address you entered above. 1. Copy the GPG key ID that starts with `sec`. In the following example, that's - `0x30F2B65B9246B6CA`: + `30F2B65B9246B6CA`: ``` - sec rsa4096/0x30F2B65B9246B6CA 2017-08-18 [SC] + sec rsa4096/30F2B65B9246B6CA 2017-08-18 [SC] D5E4F29F3275DC0CDA8FFC8730F2B65B9246B6CA uid [ultimate] Mr. Robot <mr@robot.sh> - ssb rsa4096/0xB7ABC0813E4028C0 2017-08-18 [E] + ssb rsa4096/B7ABC0813E4028C0 2017-08-18 [E] ``` 1. Tell Git to use that key to sign the commits: ``` - git config --global user.signingkey 0x30F2B65B9246B6CA + git config --global user.signingkey 30F2B65B9246B6CA ``` - Replace `0x30F2B65B9246B6CA` with your GPG key ID. + Replace `30F2B65B9246B6CA` with your GPG key ID. ## Signing commits diff --git a/doc/user/project/repository/img/contributors_graph.png b/doc/user/project/repository/img/contributors_graph.png Binary files differindex c31da7aa1ff..c31da7aa1ff 100755..100644 --- a/doc/user/project/repository/img/contributors_graph.png +++ b/doc/user/project/repository/img/contributors_graph.png diff --git a/doc/user/project/repository/img/repo_graph.png b/doc/user/project/repository/img/repo_graph.png Binary files differindex 28da8ad9589..28da8ad9589 100755..100644 --- a/doc/user/project/repository/img/repo_graph.png +++ b/doc/user/project/repository/img/repo_graph.png diff --git a/doc/user/project/settings/img/general_settings.png b/doc/user/project/settings/img/general_settings.png Binary files differindex 96f5b84871f..96f5b84871f 100755..100644 --- a/doc/user/project/settings/img/general_settings.png +++ b/doc/user/project/settings/img/general_settings.png diff --git a/doc/user/project/settings/img/merge_requests_settings.png b/doc/user/project/settings/img/merge_requests_settings.png Binary files differindex b1f2dfa7376..b1f2dfa7376 100755..100644 --- a/doc/user/project/settings/img/merge_requests_settings.png +++ b/doc/user/project/settings/img/merge_requests_settings.png diff --git a/doc/user/project/settings/img/sharing_and_permissions_settings.png b/doc/user/project/settings/img/sharing_and_permissions_settings.png Binary files differindex 7767a3d7187..0f9cf9512af 100755..100644 --- a/doc/user/project/settings/img/sharing_and_permissions_settings.png +++ b/doc/user/project/settings/img/sharing_and_permissions_settings.png diff --git a/doc/user/search/img/issues_any_assignee.png b/doc/user/search/img/issues_any_assignee.png Binary files differindex 2f902bcc66c..2f902bcc66c 100755..100644 --- a/doc/user/search/img/issues_any_assignee.png +++ b/doc/user/search/img/issues_any_assignee.png diff --git a/doc/user/search/img/issues_assigned_to_you.png b/doc/user/search/img/issues_assigned_to_you.png Binary files differindex 36c670eedd5..36c670eedd5 100755..100644 --- a/doc/user/search/img/issues_assigned_to_you.png +++ b/doc/user/search/img/issues_assigned_to_you.png diff --git a/doc/user/search/img/issues_author.png b/doc/user/search/img/issues_author.png Binary files differindex 792f9746db6..792f9746db6 100755..100644 --- a/doc/user/search/img/issues_author.png +++ b/doc/user/search/img/issues_author.png diff --git a/doc/user/search/img/issues_mrs_shortcut.png b/doc/user/search/img/issues_mrs_shortcut.png Binary files differindex 6380b337b54..6380b337b54 100755..100644 --- a/doc/user/search/img/issues_mrs_shortcut.png +++ b/doc/user/search/img/issues_mrs_shortcut.png diff --git a/doc/user/search/img/left_menu_bar.png b/doc/user/search/img/left_menu_bar.png Binary files differindex d68a71cba8e..d68a71cba8e 100755..100644 --- a/doc/user/search/img/left_menu_bar.png +++ b/doc/user/search/img/left_menu_bar.png diff --git a/doc/user/search/img/project_search.png b/doc/user/search/img/project_search.png Binary files differindex 3150b40de29..3150b40de29 100755..100644 --- a/doc/user/search/img/project_search.png +++ b/doc/user/search/img/project_search.png diff --git a/doc/user/search/img/search_issues_board.png b/doc/user/search/img/search_issues_board.png Binary files differindex 84048ae6a02..84048ae6a02 100755..100644 --- a/doc/user/search/img/search_issues_board.png +++ b/doc/user/search/img/search_issues_board.png diff --git a/doc/user/search/img/sort_projects.png b/doc/user/search/img/sort_projects.png Binary files differindex 9bf2770b299..9bf2770b299 100755..100644 --- a/doc/user/search/img/sort_projects.png +++ b/doc/user/search/img/sort_projects.png |
