diff options
Diffstat (limited to 'doc')
280 files changed, 6631 insertions, 1842 deletions
diff --git a/doc/README.md b/doc/README.md index 8bb8e147cd1..b3e7c9bd0bf 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,9 +1,13 @@ +--- +toc: false +--- + # GitLab Documentation Welcome to [GitLab](https://about.gitlab.com/), a Git-based fully featured platform for software development! -We offer four different products for you and your company: +GitLab offers the most scalable Git-based fully integrated platform for software development, with flexible products and subscription plans: - **GitLab Community Edition (CE)** is an [opensource product](https://gitlab.com/gitlab-org/gitlab-ce/), self-hosted, free to use. Every feature available in GitLab CE is also available on GitLab Enterprise Edition (Starter and Premium) and GitLab.com. @@ -20,12 +24,16 @@ plus premium features available in each version: **Enterprise Edition Starter** Shortcuts to GitLab's most visited docs: -| [GitLab CI](ci/README.md) | Other | +| [GitLab CI/CD](ci/README.md) | Other | | :----- | :----- | | [Quick start guide](ci/quick_start/README.md) | [API](api/README.md) | | [Configuring `.gitlab-ci.yml`](ci/yaml/README.md) | [SSH authentication](ssh/README.md) | | [Using Docker images](ci/docker/using_docker_images.md) | [GitLab Pages](user/project/pages/index.md) | +- [User documentation](user/index.md) +- [Administrator documentation](#administrator-documentation) +- [Technical Articles](articles/index.md) + ## Getting started with GitLab - [GitLab Basics](gitlab-basics/README.md): Start working on your command line and on GitLab. @@ -33,10 +41,10 @@ Shortcuts to GitLab's most visited docs: - See also [GitLab Workflow - an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/). - [GitLab Markdown](user/markdown.md): GitLab's advanced formatting system (GitLab Flavored Markdown). - [GitLab Quick Actions](user/project/quick_actions.md): Textual shortcuts for common actions on issues or merge requests that are usually done by clicking buttons or dropdowns in GitLab's UI. +- [Auto DevOps](topics/autodevops/index.md) ### User account -- [User documentation](user/index.md): Learn how to use GitLab and explore its features - [User account](user/profile/index.md): Manage your account - [Authentication](topics/authentication/index.md): Account security with two-factor authentication, setup your ssh keys and deploy keys for secure access to your projects. - [Profile settings](user/profile/index.md#profile-settings): Manage your profile settings, two factor authentication and more. @@ -70,12 +78,14 @@ Manage your [repositories](user/project/repository/index.md) from the UI (user i - [Create a branch](user/project/repository/web_editor.md#create-a-new-branch) - [Protected branches](user/project/protected_branches.md#protected-branches) - [Delete merged branches](user/project/repository/branches/index.md#delete-merged-branches) +- Commits + - [Signing commits](user/project/repository/gpg_signed_commits/index.md): use GPG to sign your commits. ### Issues and Merge Requests (MRs) - [Discussions](user/discussions/index.md) Threads, comments, and resolvable discussions in issues, commits, and merge requests. - [Issues](user/project/issues/index.md) -- [Issue Board](user/project/issue_board.md) +- [Project issue Board](user/project/issue_board.md) - [Issues and merge requests templates](user/project/description_templates.md): Create templates for submitting new issues and merge requests. - [Labels](user/project/labels.md): Categorize your issues or merge requests based on descriptive titles. - [Merge Requests](user/project/merge_requests/index.md) @@ -91,11 +101,10 @@ Manage your [repositories](user/project/repository/index.md) from the UI (user i - [Git](topics/git/index.md): Getting started with Git, branching strategies, Git LFS, advanced use. - [Git cheatsheet](https://gitlab.com/gitlab-com/marketing/raw/master/design/print/git-cheatsheet/print-pdf/git-cheatsheet.pdf): Download a PDF describing the most used Git operations. - [GitLab Flow](workflow/gitlab_flow.md): explore the best of Git with the GitLab Flow strategy. -- [Signing commits](workflow/gpg_signed_commits/index.md): use GPG to sign your commits. ### Migrate and import your projects from other platforms -- [Importing to GitLab](workflow/importing/README.md): Import your projects from GitHub, Bitbucket, GitLab.com, FogBugz and SVN into GitLab. +- [Importing to GitLab](user/project/import/index.md): Import your projects from GitHub, Bitbucket, GitLab.com, FogBugz and SVN into GitLab. - [Migrating from SVN](workflow/importing/migrating_from_svn.md): Convert a SVN repository to Git and GitLab. ### Continuous Integration, Delivery, and Deployment @@ -152,7 +161,6 @@ have access to GitLab administration tools and settings. ### Integrations - [Integrations](integration/README.md): How to integrate with systems such as JIRA, Redmine, Twitter. -- [Koding](administration/integration/koding.md): Set up Koding to use with GitLab. - [Mattermost](user/project/integrations/mattermost.md): Set up GitLab with Mattermost. ### Monitoring diff --git a/doc/administration/auth/authentiq.md b/doc/administration/auth/authentiq.md index 1528f1d2b17..252ff1f4b15 100644 --- a/doc/administration/auth/authentiq.md +++ b/doc/administration/auth/authentiq.md @@ -4,7 +4,7 @@ To enable the Authentiq OmniAuth provider for passwordless authentication you mu Authentiq will generate a Client ID and the accompanying Client Secret for you to use. -1. Get your Client credentials (Client ID and Client Secret) at [Authentiq](https://www.authentiq.com/register). +1. Get your Client credentials (Client ID and Client Secret) at [Authentiq](https://www.authentiq.com/developers). 2. On your GitLab server, open the configuration file: diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index 425c924cdf2..d22815dfa5e 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -87,9 +87,12 @@ main: # 'main' is the GitLab 'provider ID' of this LDAP server encryption: 'plain' # Enables SSL certificate verification if encryption method is - # "start_tls" or "simple_tls". (Defaults to false for backward- - # compatibility) - verify_certificates: false + # "start_tls" or "simple_tls". Defaults to true since GitLab 10.0 for + # security. This may break installations upon upgrade to 10.0, that did + # not know their LDAP SSL certificates were not setup properly. For + # example, when using self-signed certificates, the ca_file path may + # need to be specified. + verify_certificates: true # Specifies the path to a file containing a PEM-format CA certificate, # e.g. if you need to use an internal CA. diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index 90a2e9298bf..e09ccaba08c 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -42,6 +42,10 @@ GitLab does not recommend using EFS with GitLab. are allocated. For smaller volumes, users may experience decent performance for a period of time due to 'Burst Credits'. Over a period of weeks to months credits may run out and performance will bottom out. +- To keep "Burst Credits" available, it may be necessary to provision more space + with 'dummy data'. However, this may get expensive. +- Another option to maintain "Burst Credits" is to use FS Cache on the server so + that AWS doesn't always have to go into EFS to access files. - For larger volumes, allocated IOPS may not be the problem. Workloads where many small files are written in a serialized manner are not well-suited for EFS. EBS with an NFS server on top will perform much better. diff --git a/doc/administration/integration/koding.md b/doc/administration/integration/koding.md index b95c425842c..67f9f01efb8 100644 --- a/doc/administration/integration/koding.md +++ b/doc/administration/integration/koding.md @@ -1,6 +1,10 @@ # Koding & GitLab -> [Introduced][ce-5909] in GitLab 8.11. +>**Notes:** +- **As of GitLab 10.0, the Koding integration is deprecated and will be removed + in a future version. The option to configure it is removed from GitLab's admin + area.** +- [Introduced][ce-5909] in GitLab 8.11. This document will guide you through installing and configuring Koding with GitLab. diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index b21817c1fd3..652ca9cf454 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -71,6 +71,15 @@ And in Markdown using fenced code blocks: Alice -> Bob : Go Away ``` +And in reStructuredText using a directive: + +``` +.. plantuml:: + + Bob -> Alice: hello + Alice -> Bob: Go Away +``` + The above blocks will be converted to an HTML img tag with source pointing to the PlantUML instance. If the PlantUML server is correctly configured, this should render a nice diagram instead of the block: @@ -94,4 +103,4 @@ Some parameters can be added to the AsciiDoc block definition: Markdown does not support any parameters and will always use PNG format. -[ce-8537]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8537
\ No newline at end of file +[ce-8537]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8537 diff --git a/doc/administration/logs.md b/doc/administration/logs.md index 4b8d5c5cc87..76e071dc673 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -9,6 +9,33 @@ documentation](http://docs.gitlab.com/ee/administration/audit_events.html) System log files are typically plain text in a standard log file format. This guide talks about how to read and use these system log files. +## `production_json.log` + +This file lives in `/var/log/gitlab/gitlab-rails/production_json.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/production_json.log` for +installations from source. (When Gitlab is running in an environment +other than production, the corresponding logfile is shown here.) + +It contains a structured log for Rails controller requests received from +GitLab, thanks to [Lograge](https://github.com/roidrage/lograge/). Note that +requests from the API [are not yet logged to this +file](https://gitlab.com/gitlab-org/gitlab-ce/issues/36189). + +Each line contains a JSON line that can be ingested by Elasticsearch, Splunk, etc. For example: + +```json +{"method":"GET","path":"/gitlab/gitlab-ce/issues/1234","format":"html","controller":"Projects::IssuesController","action":"show","status":200,"duration":229.03,"view":174.07,"db":13.24,"time":"2017-08-08T20:15:54.821Z","params":{"namespace_id":"gitlab","project_id":"gitlab-ce","id":"1234"},"remote_ip":"18.245.0.1","user_id":1,"username":"admin"} +``` + +In this example, you can see this was a GET request for a specific issue. Notice each line also contains performance data: + +1. `duration`: the total time taken to retrieve the request +2. `view`: total time taken inside the Rails views +3. `db`: total time to retrieve data from the database + +In addition, the log contains the IP address from which the request originated +(`remote_ip`) as well as the user's ID (`user_id`), and username (`username`). + ## `production.log` This file lives in `/var/log/gitlab/gitlab-rails/production.log` for diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index ee680c7b258..68efe0aae5c 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -5,17 +5,17 @@ activated, it looks as follows:  -It allows you to: +It allows you to see (from left to right): -- see the current host serving the page -- see the timing of the page (backend, frontend) -- the number of DB queries, the time it took, and the detail of these queries +- the current host serving the page +- the timing of the page (backend, frontend) +- time taken and number of DB queries, click through for details of these queries  -- the number of calls to Redis, and the time it took -- the number of background jobs created by Sidekiq, and the time it took -- the number of Ruby GC calls, and the time it took -- profile the code used to generate the page, line by line +- time taken and number of calls to Redis +- time taken and number of background jobs created by Sidekiq +- profile of the code used to generate the page, line by line for either _all_, _app & lib_ , or _views_. In the profile view, the numbers in the left panel represent wall time, cpu time, and number of calls (based on [rblineprof](https://github.com/tmm1/rblineprof)).  +- time taken and number of Ruby GC calls ## Enable the Performance Bar via the Admin panel diff --git a/doc/api/README.md b/doc/api/README.md index 8acb2145f1a..6cbea29bda6 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -55,27 +55,14 @@ following locations: - [Tags](tags.md) - [Todos](todos.md) - [Users](users.md) -- [Validate CI configuration](ci/lint.md) +- [Validate CI configuration](lint.md) - [V3 to V4](v3_to_v4.md) - [Version](version.md) - -The following documentation is for the [internal CI API](ci/README.md): - -- [Builds](ci/builds.md) -- [Runners](ci/runners.md) +- [Wikis](wikis.md) ## Road to GraphQL -Going forward, we will start on moving to -[GraphQL](http://graphql.org/learn/best-practices/) and deprecate the use of -controller-specific endpoints. GraphQL has a number of benefits: - -1. We avoid having to maintain two different APIs. -2. Callers of the API can request only what they need. -3. It is versioned by default. - -It will co-exist with the current v4 REST API. If we have a v5 API, this should -be a compatibility layer on top of GraphQL. +We have changed our plans to move to GraphQL. After reviewing the GraphQL license, anything related to the Facebook BSD plus patent license will not be allowed at GitLab. ## Basic usage @@ -251,8 +238,8 @@ The following table gives an overview of how the API functions generally behave. | ------------ | ----------- | | `GET` | Access one or more resources and return the result as JSON. | | `POST` | Return `201 Created` if the resource is successfully created and return the newly created resource as JSON. | -| `GET` / `PUT` / `DELETE` | Return `200 OK` if the resource is accessed, modified or deleted successfully. The (modified) result is returned as JSON. | -| `DELETE` | Designed to be idempotent, meaning a request to a resource still returns `200 OK` even it was deleted before or is not available. The reasoning behind this, is that the user is not really interested if the resource existed before or not. | +| `GET` / `PUT` | Return `200 OK` if the resource is accessed or modified successfully. The (modified) result is returned as JSON. | +| `DELETE` | Returns `204 No Content` if the resuource was deleted successfully. | The following table shows the possible return codes for API requests. @@ -268,6 +255,7 @@ The following table shows the possible return codes for API requests. | `404 Not Found` | A resource could not be accessed, e.g., an ID for a resource could not be found. | | `405 Method Not Allowed` | The request is not supported. | | `409 Conflict` | A conflicting resource already exists, e.g., creating a project with a name that already exists. | +| `412` | Indicates the request was denied. May happen if the `If-Unmodified-Since` header is provided when trying to delete a resource, which was modified in between. | | `422 Unprocessable` | The entity could not be processed. | | `500 Server Error` | While handling the request something went wrong server-side. | diff --git a/doc/api/ci/README.md b/doc/api/ci/README.md deleted file mode 100644 index 96a281e27c8..00000000000 --- a/doc/api/ci/README.md +++ /dev/null @@ -1,24 +0,0 @@ -# GitLab CI API - -## Purpose - -The main purpose of GitLab CI API is to provide the necessary data and context -for GitLab CI Runners. - -All relevant information about the consumer API can be found in a -[separate document](../../api/README.md). - -## API Prefix - -The current CI API prefix is `/ci/api/v1`. - -You need to prepend this prefix to all examples in this documentation, like: - -```bash -GET /ci/api/v1/builds/:id/artifacts -``` - -## Resources - -- [Builds](builds.md) -- [Runners](runners.md) diff --git a/doc/api/ci/builds.md b/doc/api/ci/builds.md deleted file mode 100644 index c8374d94716..00000000000 --- a/doc/api/ci/builds.md +++ /dev/null @@ -1,147 +0,0 @@ -# Builds API - -API used by runners to receive and update builds. - ->**Note:** -This API is intended to be used only by Runners as their own -communication channel. For the consumer API see the -[Jobs API](../jobs.md). - -## Authentication - -This API uses two types of authentication: - -1. Unique Runner's token which is the token assigned to the Runner after it - has been registered. - -2. Using the build authorization token. - This is project's CI token that can be found under the **Builds** section of - a project's settings. The build authorization token can be passed as a - parameter or a value of `BUILD-TOKEN` header. - -These two methods of authentication are interchangeable. - -## Builds - -### Runs oldest pending build by runner - -``` -POST /ci/api/v1/builds/register -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|---------------------| -| `token` | string | yes | Unique runner token | - - -``` -curl --request POST "https://gitlab.example.com/ci/api/v1/builds/register" --form "token=t0k3n" -``` - -**Responses:** - -| Status | Data |Description | -|--------|------|---------------------------------------------------------------------------| -| `201` | yes | When a build is scheduled for a runner | -| `204` | no | When no builds are scheduled for a runner (for GitLab Runner >= `v1.3.0`) | -| `403` | no | When invalid token is used or no token is sent | -| `404` | no | When no builds are scheduled for a runner (for GitLab Runner < `v1.3.0`) **or** when the runner is set to `paused` in GitLab runner's configuration page | - -### Update details of an existing build - -``` -PUT /ci/api/v1/builds/:id -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|----------------------| -| `id` | integer | yes | The ID of a project | -| `token` | string | yes | Unique runner token | -| `state` | string | no | The state of a build | -| `trace` | string | no | The trace of a build | - -``` -curl --request PUT "https://gitlab.example.com/ci/api/v1/builds/1234" --form "token=t0k3n" --form "state=running" --form "trace=Running git clone...\n" -``` - -### Incremental build trace update - -Using this method you need to send trace content as a request body. You also need to provide the `Content-Range` header -with a range of sent trace part. Note that you need to send parts in the proper order, so the begining of the part -must start just after the end of the previous part. If you provide the wrong part, then GitLab CI API will return `416 -Range Not Satisfiable` response with a header `Range: 0-X`, where `X` is the current trace length. - -For example, if you receive `Range: 0-11` in the response, then your next part must contain a `Content-Range: 11-...` -header and a trace part covered by this range. - -For a valid update API will return `202` response with: -* `Build-Status: {status}` header containing current status of the build, -* `Range: 0-{length}` header with the current trace length. - -``` -PATCH /ci/api/v1/builds/:id/trace.txt -``` - -Parameters: - -| Attribute | Type | Required | Description | -|-----------|---------|----------|----------------------| -| `id` | integer | yes | The ID of a build | - -Headers: - -| Attribute | Type | Required | Description | -|-----------------|---------|----------|-----------------------------------| -| `BUILD-TOKEN` | string | yes | The build authorization token | -| `Content-Range` | string | yes | Bytes range of trace that is sent | - -``` -curl --request PATCH "https://gitlab.example.com/ci/api/v1/builds/1234/trace.txt" --header "BUILD-TOKEN=build_t0k3n" --header "Content-Range=0-21" --data "Running git clone...\n" -``` - - -### Upload artifacts to build - -``` -POST /ci/api/v1/builds/:id/artifacts -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|-------------------------------| -| `id` | integer | yes | The ID of a build | -| `token` | string | yes | The build authorization token | -| `file` | mixed | yes | Artifacts file | - -``` -curl --request POST "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" --form "token=build_t0k3n" --form "file=@/path/to/file" -``` - -### Download the artifacts file from build - -``` -GET /ci/api/v1/builds/:id/artifacts -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|-------------------------------| -| `id` | integer | yes | The ID of a build | -| `token` | string | yes | The build authorization token | - -``` -curl "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" --form "token=build_t0k3n" -``` - -### Remove the artifacts file from build - -``` -DELETE /ci/api/v1/builds/:id/artifacts -``` - -| Attribute | Type | Required | Description | -|-----------|---------|----------|-------------------------------| -| ` id` | integer | yes | The ID of a build | -| `token` | string | yes | The build authorization token | - -``` -curl --request DELETE "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" --form "token=build_t0k3n" -``` diff --git a/doc/api/ci/runners.md b/doc/api/ci/runners.md deleted file mode 100644 index 342c039dad8..00000000000 --- a/doc/api/ci/runners.md +++ /dev/null @@ -1,59 +0,0 @@ -# Register and Delete Runners API - -API used by Runners to register and delete themselves. - ->**Note:** -This API is intended to be used only by Runners as their own -communication channel. For the consumer API see the -[new Runners API](../runners.md). - -## Authentication - -This API uses two types of authentication: - -1. Unique Runner's token, which is the token assigned to the Runner after it - has been registered. This token can be found on the Runner's edit page (go to - **Project > Runners**, select one of the Runners listed under **Runners activated for - this project**). - -2. Using Runners' registration token. - This is a token that can be found in project's settings. - It can also be found in the **Admin > Runners** settings area. - There are two types of tokens you can pass: shared Runner registration - token or project specific registration token. - -## Register a new runner - -Used to make GitLab CI aware of available runners. - -```sh -POST /ci/api/v1/runners/register -``` - -| Attribute | Type | Required | Description | -| --------- | ------- | --------- | ----------- | -| `token` | string | yes | Runner's registration token | - -Example request: - -```sh -curl --request POST "https://gitlab.example.com/ci/api/v1/runners/register" --form "token=t0k3n" -``` - -## Delete a Runner - -Used to remove a Runner. - -```sh -DELETE /ci/api/v1/runners/delete -``` - -| Attribute | Type | Required | Description | -| --------- | ------- | --------- | ----------- | -| `token` | string | yes | Unique Runner's token | - -Example request: - -```sh -curl --request DELETE "https://gitlab.example.com/ci/api/v1/runners/delete" --form "token=t0k3n" -``` diff --git a/doc/api/deploy_keys.md b/doc/api/deploy_keys.md index 4fa800ecb9c..273d5a56b6f 100644 --- a/doc/api/deploy_keys.md +++ b/doc/api/deploy_keys.md @@ -106,7 +106,7 @@ Example response: Creates a new deploy key for a project. If the deploy key already exists in another project, it will be joined to current -project only if original one was is accessible by the same user. +project only if original one is accessible by the same user. ``` POST /projects/:id/deploy_keys diff --git a/doc/api/environments.md b/doc/api/environments.md index 5ca766bf87d..e8deb3e07e9 100644 --- a/doc/api/environments.md +++ b/doc/api/environments.md @@ -94,7 +94,7 @@ Example response: ## Delete an environment -It returns `200` if the environment was successfully deleted, and `404` if the environment does not exist. +It returns `204` if the environment was successfully deleted, and `404` if the environment does not exist. ``` DELETE /projects/:id/environments/:environment_id diff --git a/doc/api/events.md b/doc/api/events.md index 3d5170f3f1e..129af0afa35 100644 --- a/doc/api/events.md +++ b/doc/api/events.md @@ -79,7 +79,6 @@ Example response: "target_id":160, "target_type":"Issue", "author_id":25, - "data":null, "target_title":"Qui natus eos odio tempore et quaerat consequuntur ducimus cupiditate quis.", "created_at":"2017-02-09T10:43:19.667Z", "author":{ @@ -99,7 +98,6 @@ Example response: "target_id":159, "target_type":"Issue", "author_id":21, - "data":null, "target_title":"Nostrum enim non et sed optio illo deleniti non.", "created_at":"2017-02-09T10:43:19.426Z", "author":{ @@ -151,7 +149,6 @@ Example response: "target_id": 830, "target_type": "Issue", "author_id": 1, - "data": null, "target_title": "Public project search field", "author": { "name": "Dmitriy Zaporozhets", @@ -166,7 +163,7 @@ Example response: { "title": null, "project_id": 15, - "action_name": "opened", + "action_name": "pushed", "target_id": null, "target_type": null, "author_id": 1, @@ -179,31 +176,14 @@ Example response: "web_url": "http://localhost:3000/root" }, "author_username": "john", - "data": { - "before": "50d4420237a9de7be1304607147aec22e4a14af7", - "after": "c5feabde2d8cd023215af4d2ceeb7a64839fc428", - "ref": "refs/heads/master", - "user_id": 1, - "user_name": "Dmitriy Zaporozhets", - "repository": { - "name": "gitlabhq", - "url": "git@dev.gitlab.org:gitlab/gitlabhq.git", - "description": "GitLab: self hosted Git management software. \r\nDistributed under the MIT License.", - "homepage": "https://dev.gitlab.org/gitlab/gitlabhq" - }, - "commits": [ - { - "id": "c5feabde2d8cd023215af4d2ceeb7a64839fc428", - "message": "Add simple search to projects in public area", - "timestamp": "2013-05-13T18:18:08+00:00", - "url": "https://dev.gitlab.org/gitlab/gitlabhq/commit/c5feabde2d8cd023215af4d2ceeb7a64839fc428", - "author": { - "name": "Dmitriy Zaporozhets", - "email": "dmitriy.zaporozhets@gmail.com" - } - } - ], - "total_commits_count": 1 + "push_data": { + "commit_count": 1, + "action": "pushed", + "ref_type": "branch", + "commit_from": "50d4420237a9de7be1304607147aec22e4a14af7", + "commit_to": "c5feabde2d8cd023215af4d2ceeb7a64839fc428", + "ref": "master", + "commit_title": "Add simple search to projects in public area" }, "target_title": null }, @@ -214,7 +194,6 @@ Example response: "target_id": 840, "target_type": "Issue", "author_id": 1, - "data": null, "target_title": "Finish & merge Code search PR", "author": { "name": "Dmitriy Zaporozhets", @@ -233,7 +212,6 @@ Example response: "target_id": 1312, "target_type": "Note", "author_id": 1, - "data": null, "target_title": null, "created_at": "2015-12-04T10:33:58.089Z", "note": { @@ -305,7 +283,6 @@ Example response: "target_iid":160, "target_type":"Issue", "author_id":25, - "data":null, "target_title":"Qui natus eos odio tempore et quaerat consequuntur ducimus cupiditate quis.", "created_at":"2017-02-09T10:43:19.667Z", "author":{ @@ -326,7 +303,6 @@ Example response: "target_iid":159, "target_type":"Issue", "author_id":21, - "data":null, "target_title":"Nostrum enim non et sed optio illo deleniti non.", "created_at":"2017-02-09T10:43:19.426Z", "author":{ diff --git a/doc/api/group_level_variables.md b/doc/api/group_level_variables.md index e19be7b35c4..33c6da08018 100644 --- a/doc/api/group_level_variables.md +++ b/doc/api/group_level_variables.md @@ -1,5 +1,7 @@ # Group-level Variables API +> [Introduced][ce-34519] in GitLab 9.5 + ## List group variables Get list of a group's variables. @@ -123,3 +125,5 @@ DELETE /groups/:id/variables/:key ``` curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/groups/1/variables/VARIABLE_1" ``` + +[ce-34519]: https://gitlab.com/gitlab-org/gitlab-ce/issues/34519 diff --git a/doc/api/group_milestones.md b/doc/api/group_milestones.md index dbfc7529125..a96fb3124fc 100644 --- a/doc/api/group_milestones.md +++ b/doc/api/group_milestones.md @@ -1,5 +1,8 @@ # Group milestones API +> **Notes:** +> [Introduced][ce-12819] in GitLab 9.5. + ## List group milestones Returns a list of group milestones. @@ -118,3 +121,5 @@ Parameters: - `id` (required) - The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user - `milestone_id` (required) - The ID of a group milestone + +[ce-12819]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12819 diff --git a/doc/api/groups.md b/doc/api/groups.md index 2b3d8e125c8..c2daa8bc029 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -2,7 +2,8 @@ ## List groups -Get a list of groups. (As user: my groups or all available, as admin: all groups). +Get a list of visible groups for the authenticated user. When accessed without +authentication, only public groups are returned. Parameters: @@ -43,7 +44,8 @@ You can search for groups by name or path, see below. ## List a group's projects -Get a list of projects in this group. +Get a list of projects in this group. When accessed without authentication, only +public projects are returned. ``` GET /groups/:id/projects @@ -109,7 +111,8 @@ Example response: ## Details of a group -Get all details of a group. +Get all details of a group. This endpoint can be accessed without authentication +if the group is publicly accessible. ``` GET /groups/:id diff --git a/doc/api/issues.md b/doc/api/issues.md index f30ed08d0fa..8ca66049d31 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -30,20 +30,22 @@ GET /issues?milestone=1.0.0&state=opened GET /issues?iids[]=42&iids[]=43 GET /issues?author_id=5 GET /issues?assignee_id=5 -``` - -| Attribute | Type | Required | Description | -|-------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------| -| `state` | string | no | Return all issues or just those that are `opened` or `closed` | -| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | -| `milestone` | string | no | The milestone title | -| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `author_id` | integer | no | Return issues created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me`. _([Introduced][ce-13004] in GitLab 9.5)_ | -| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | -| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | -| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | -| `search` | string | no | Search issues against their `title` and `description` | +GET /issues?my_reaction_emoji=star +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| `state` | string | no | Return all issues or just those that are `opened` or `closed` | +| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | +| `milestone` | string | no | The milestone title | +| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me`. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | +| `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | +| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Search issues against their `title` and `description` | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/issues @@ -101,6 +103,12 @@ Example response: "user_notes_count": 1, "due_date": "2016-07-22", "web_url": "http://example.com/example/example/issues/6", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + }, "confidential": false } ] @@ -125,21 +133,23 @@ GET /groups/:id/issues?iids[]=42&iids[]=43 GET /groups/:id/issues?search=issue+title+or+description GET /groups/:id/issues?author_id=5 GET /groups/:id/issues?assignee_id=5 +GET /groups/:id/issues?my_reaction_emoji=star ``` -| Attribute | Type | Required | Description | -|-------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | -| `state` | string | no | Return all issues or just those that are `opened` or `closed` | -| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | -| `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | -| `milestone` | string | no | The milestone title | -| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | -| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | -| `search` | string | no | Search group issues against their `title` and `description` | +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `state` | string | no | Return all issues or just those that are `opened` or `closed` | +| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | +| `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | +| `milestone` | string | no | The milestone title | +| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | +| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Search group issues against their `title` and `description` | ```bash @@ -198,6 +208,12 @@ Example response: "user_notes_count": 1, "due_date": null, "web_url": "http://example.com/example/example/issues/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + }, "confidential": false } ] @@ -222,23 +238,25 @@ GET /projects/:id/issues?iids[]=42&iids[]=43 GET /projects/:id/issues?search=issue+title+or+description GET /projects/:id/issues?author_id=5 GET /projects/:id/issues?assignee_id=5 -``` - -| Attribute | Type | Required | Description | -|-------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `iids[]` | Array[integer] | no | Return only the milestone having the given `iid` | -| `state` | string | no | Return all issues or just those that are `opened` or `closed` | -| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | -| `milestone` | string | no | The milestone title | -| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | -| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | -| `search` | string | no | Search project issues against their `title` and `description` | -| `created_after` | datetime | no | Return issues created after the given time (inclusive) | -| `created_before` | datetime | no | Return issues created before the given time (inclusive) | +GET /projects/:id/issues?my_reaction_emoji=star +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `iids[]` | Array[integer] | no | Return only the milestone having the given `iid` | +| `state` | string | no | Return all issues or just those that are `opened` or `closed` | +| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | +| `milestone` | string | no | The milestone title | +| `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | +| `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | +| `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Search project issues against their `title` and `description` | +| `created_after` | datetime | no | Return issues created after the given time (inclusive) | +| `created_before` | datetime | no | Return issues created before the given time (inclusive) | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/4/issues @@ -296,6 +314,12 @@ Example response: "user_notes_count": 1, "due_date": "2016-07-22", "web_url": "http://example.com/example/example/issues/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + }, "confidential": false } ] @@ -372,6 +396,12 @@ Example response: "user_notes_count": 1, "due_date": null, "web_url": "http://example.com/example/example/issues/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + }, "confidential": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", @@ -440,6 +470,12 @@ Example response: "user_notes_count": 0, "due_date": null, "web_url": "http://example.com/example/example/issues/14", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + }, "confidential": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", @@ -509,6 +545,12 @@ Example response: "user_notes_count": 0, "due_date": "2016-07-22", "web_url": "http://example.com/example/example/issues/15", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + }, "confidential": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", @@ -558,7 +600,7 @@ POST /projects/:id/issues/:issue_iid/move | `to_project_id` | integer | yes | The ID of the new project | ```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/4/issues/85/move +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data '{"to_project_id": 5}' https://gitlab.example.com/api/v4/projects/4/issues/85/move ``` Example response: @@ -601,6 +643,12 @@ Example response: }, "due_date": null, "web_url": "http://example.com/example/example/issues/11", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + }, "confidential": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", @@ -672,6 +720,12 @@ Example response: }, "due_date": null, "web_url": "http://example.com/example/example/issues/11", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + }, "confidential": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", @@ -1001,7 +1055,13 @@ Example response: "user_notes_count": 1, "should_remove_source_branch": null, "force_remove_source_branch": false, - "web_url": "https://gitlab.example.com/gitlab-org/gitlab-test/merge_requests/6432" + "web_url": "https://gitlab.example.com/gitlab-org/gitlab-test/merge_requests/6432", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } } ] ``` @@ -1039,3 +1099,4 @@ Example response: ``` [ce-13004]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13004 +[ce-14016]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14016 diff --git a/doc/api/jobs.md b/doc/api/jobs.md index 297115e94ac..d60c7c12881 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -320,11 +320,11 @@ Response: [ce-2893]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2893 -## Download the artifacts file +## Download the artifacts archive > [Introduced][ce-5347] in GitLab 8.10. -Download the artifacts file from the given reference name and job provided the +Download the artifacts archive from the given reference name and job provided the job finished successfully. ``` @@ -354,6 +354,40 @@ Example response: [ce-5347]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5347 +## Download a single artifact file + +> Introduced in GitLab 10.0 + +Download a single artifact file from within the job's artifacts archive. + +Only a single file is going to be extracted from the archive and streamed to a client. + +``` +GET /projects/:id/jobs/:job_id/artifacts/*artifact_path +``` + +Parameters + +| Attribute | Type | Required | Description | +|-----------------|---------|----------|-------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `job_id ` | integer | yes | The unique job identifier | +| `artifact_path` | string | yes | Path to a file inside the artifacts archive | + +Example request: + +``` +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/jobs/5/artifacts/some/release/file.pdf" +``` + +Example response: + +| Status | Description | +|-----------|--------------------------------------| +| 200 | Sends a single artifact file | +| 400 | Invalid path provided | +| 404 | Build not found or no file/artifacts | + ## Get a trace file Get a trace of a specific job of a project diff --git a/doc/api/keys.md b/doc/api/keys.md index 376ac27df3a..ddcf7830621 100644 --- a/doc/api/keys.md +++ b/doc/api/keys.md @@ -32,6 +32,7 @@ Parameters: "twitter": "", "website_url": "", "email": "john@example.com", + "theme_id": 2, "color_scheme_id": 1, "projects_limit": 10, "current_sign_in_at": null, diff --git a/doc/api/ci/lint.md b/doc/api/lint.md index e4a6dc809b1..bd5a216a99d 100644 --- a/doc/api/ci/lint.md +++ b/doc/api/lint.md @@ -5,7 +5,7 @@ Checks if your .gitlab-ci.yml file is valid. ``` -POST ci/lint +POST /lint ``` | Attribute | Type | Required | Description | @@ -49,3 +49,4 @@ Example responses: ``` [ce-5953]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5953 + diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 802e5362d70..bff8a2d3e4d 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -22,24 +22,26 @@ GET /merge_requests?state=all GET /merge_requests?milestone=release GET /merge_requests?labels=bug,reproduced GET /merge_requests?author_id=5 +GET /merge_requests?my_reaction_emoji=star GET /merge_requests?scope=assigned-to-me ``` Parameters: -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, or `merged`| -| `order_by`| string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | -| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | -| `milestone` | string | no | Return merge requests for a specific milestone | -| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | -| `labels` | string | no | Return merge requests matching a comma separated list of labels | -| `created_after` | datetime | no | Return merge requests created after the given time (inclusive) | -| `created_before` | datetime | no | Return merge requests created before the given time (inclusive) | -| `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` | -| `author_id` | integer | no | Returns merge requests created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me` | -| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` | +| Attribute | Type | Required | Description | +| ------------------- | -------- | -------- | ---------------------------------------------------------------------------------------------------------------------- | +| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, or `merged` | +| `order_by` | string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | +| `milestone` | string | no | Return merge requests for a specific milestone | +| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | +| `labels` | string | no | Return merge requests matching a comma separated list of labels | +| `created_after` | datetime | no | Return merge requests created after the given time (inclusive) | +| `created_before` | datetime | no | Return merge requests created before the given time (inclusive) | +| `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` | +| `author_id` | integer | no | Returns merge requests created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me` | +| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` | +| `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | ```json [ @@ -92,7 +94,13 @@ Parameters: "user_notes_count": 1, "should_remove_source_branch": true, "force_remove_source_branch": false, - "web_url": "http://example.com/example/example/merge_requests/1" + "web_url": "http://example.com/example/example/merge_requests/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } } ] ``` @@ -110,25 +118,27 @@ GET /projects/:id/merge_requests?state=all GET /projects/:id/merge_requests?iids[]=42&iids[]=43 GET /projects/:id/merge_requests?milestone=release GET /projects/:id/merge_requests?labels=bug,reproduced +GET /projects/:id/merge_requests?my_reaction_emoji=star ``` Parameters: -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a project | -| `iids[]` | Array[integer] | no | Return the request having the given `iid` | -| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, or `merged`| -| `order_by`| string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | -| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | -| `milestone` | string | no | Return merge requests for a specific milestone | -| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | -| `labels` | string | no | Return merge requests matching a comma separated list of labels | -| `created_after` | datetime | no | Return merge requests created after the given time (inclusive) | -| `created_before` | datetime | no | Return merge requests created before the given time (inclusive) | -| `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13060] in GitLab 9.5)_ | -| `author_id` | integer | no | Returns merge requests created by the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | -| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ | +| `id` | integer | yes | The ID of a project | +| `iids[]` | Array[integer] | no | Return the request having the given `iid` | +| `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, or `merged` | +| `order_by` | string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | +| `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | +| `milestone` | string | no | Return merge requests for a specific milestone | +| `view` | string | no | If `simple`, returns the `iid`, URL, title, description, and basic state of merge request | +| `labels` | string | no | Return merge requests matching a comma separated list of labels | +| `created_after` | datetime | no | Return merge requests created after the given time (inclusive) | +| `created_before` | datetime | no | Return merge requests created before the given time (inclusive) | +| `scope` | string | no | Return merge requests for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13060] in GitLab 9.5)_ | +| `author_id` | integer | no | Returns merge requests created by the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | +| `assignee_id` | integer | no | Returns merge requests assigned to the given user `id` _([Introduced][ce-13060] in GitLab 9.5)_ | +| `my_reaction_emoji` | string | no | Return merge requests reacted by the authenticated user by the given `emoji` _([Introduced][ce-14016] in GitLab 10.0)_ | ```json [ @@ -181,7 +191,13 @@ Parameters: "user_notes_count": 1, "should_remove_source_branch": true, "force_remove_source_branch": false, - "web_url": "http://example.com/example/example/merge_requests/1" + "web_url": "http://example.com/example/example/merge_requests/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } } ] ``` @@ -250,7 +266,13 @@ Parameters: "user_notes_count": 1, "should_remove_source_branch": true, "force_remove_source_branch": false, - "web_url": "http://example.com/example/example/merge_requests/1" + "web_url": "http://example.com/example/example/merge_requests/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } } ``` @@ -356,6 +378,12 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } "changes": [ { "old_path": "VERSION", @@ -442,7 +470,13 @@ POST /projects/:id/merge_requests "user_notes_count": 0, "should_remove_source_branch": true, "force_remove_source_branch": false, - "web_url": "http://example.com/example/example/merge_requests/1" + "web_url": "http://example.com/example/example/merge_requests/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } } ``` @@ -519,7 +553,13 @@ Must include at least one non-required attribute from above. "user_notes_count": 1, "should_remove_source_branch": true, "force_remove_source_branch": false, - "web_url": "http://example.com/example/example/merge_requests/1" + "web_url": "http://example.com/example/example/merge_requests/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } } ``` @@ -617,7 +657,13 @@ Parameters: "user_notes_count": 1, "should_remove_source_branch": true, "force_remove_source_branch": false, - "web_url": "http://example.com/example/example/merge_requests/1" + "web_url": "http://example.com/example/example/merge_requests/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } } ``` @@ -687,7 +733,13 @@ Parameters: "user_notes_count": 1, "should_remove_source_branch": true, "force_remove_source_branch": false, - "web_url": "http://example.com/example/example/merge_requests/1" + "web_url": "http://example.com/example/example/merge_requests/1", + "time_stats": { + "time_estimate": 0, + "total_time_spent": 0, + "human_time_estimate": null, + "human_total_time_spent": null + } } ``` @@ -1267,3 +1319,4 @@ Example response: ``` [ce-13060]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13060 +[ce-14016]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14016 diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index 8133251dffe..5c0bebbaeb0 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -28,12 +28,14 @@ Example response: [ { "id": 1, + "name": "user1", "path": "user1", "kind": "user", "full_path": "user1" }, { "id": 2, + "name": "group1", "path": "group1", "kind": "group", "full_path": "group1", @@ -42,6 +44,7 @@ Example response: }, { "id": 3, + "name": "bar", "path": "bar", "kind": "group", "full_path": "foo/bar", @@ -77,6 +80,7 @@ Example response: [ { "id": 4, + "name": "twitter", "path": "twitter", "kind": "group", "full_path": "twitter", diff --git a/doc/api/pipeline_schedules.md b/doc/api/pipeline_schedules.md index 433654c18cc..c28f48e5fc6 100644 --- a/doc/api/pipeline_schedules.md +++ b/doc/api/pipeline_schedules.md @@ -84,7 +84,13 @@ curl --header "PRIVATE-TOKEN: k5ESFgWY2Qf5xEvDcFxZ" "https://gitlab.example.com/ "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", "web_url": "https://gitlab.example.com/root" - } + }, + "variables": [ + { + "key": "TEST_VARIABLE_1", + "value": "TEST_1" + } + ] } ``` @@ -271,3 +277,86 @@ curl --request DELETE --header "PRIVATE-TOKEN: k5ESFgWY2Qf5xEvDcFxZ" "https://gi } } ``` + +## Pipeline schedule variable + +> [Introduced][ce-34518] in GitLab 10.0. + +## Create a new pipeline schedule variable + +Create a new variable of a pipeline schedule. + +``` +POST /projects/:id/pipeline_schedules/:pipeline_schedule_id/variables +``` + +| Attribute | Type | required | Description | +|------------------------|----------------|----------|--------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `pipeline_schedule_id` | integer | yes | The pipeline schedule id | +| `key` | string | yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | +| `value` | string | yes | The `value` of a variable | + +```sh +curl --request POST --header "PRIVATE-TOKEN: k5ESFgWY2Qf5xEvDcFxZ" --form "key=NEW_VARIABLE" --form "value=new value" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13/variables" +``` + +```json +{ + "key": "NEW_VARIABLE", + "value": "new value" +} +``` + +## Edit a pipeline schedule variable + +Updates the variable of a pipeline schedule. + +``` +PUT /projects/:id/pipeline_schedules/:pipeline_schedule_id/variables/:key +``` + +| Attribute | Type | required | Description | +|------------------------|----------------|----------|--------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `pipeline_schedule_id` | integer | yes | The pipeline schedule id | +| `key` | string | yes | The `key` of a variable | +| `value` | string | yes | The `value` of a variable | + +```sh +curl --request PUT --header "PRIVATE-TOKEN: k5ESFgWY2Qf5xEvDcFxZ" --form "value=updated value" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13/variables/NEW_VARIABLE" +``` + +```json +{ + "key": "NEW_VARIABLE", + "value": "updated value" +} +``` + +## Delete a pipeline schedule variable + +Delete the variable of a pipeline schedule. + +``` +DELETE /projects/:id/pipeline_schedules/:pipeline_schedule_id/variables/:key +``` + +| Attribute | Type | required | Description | +|------------------------|----------------|----------|--------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `pipeline_schedule_id` | integer | yes | The pipeline schedule id | +| `key` | string | yes | The `key` of a variable | + +```sh +curl --request DELETE --header "PRIVATE-TOKEN: k5ESFgWY2Qf5xEvDcFxZ" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13/variables/NEW_VARIABLE" +``` + +```json +{ + "key": "NEW_VARIABLE", + "value": "updated value" +} +``` + +[ce-34518]: https://gitlab.com/gitlab-org/gitlab-ce/issues/34518
\ No newline at end of file diff --git a/doc/api/project_snippets.md b/doc/api/project_snippets.md index d74398c6e65..ad2521230e6 100644 --- a/doc/api/project_snippets.md +++ b/doc/api/project_snippets.md @@ -95,8 +95,7 @@ Parameters: ## Delete snippet -Deletes an existing project snippet. This is an idempotent function and deleting a non-existent -snippet still returns a `200 OK` status code. +Deletes an existing project snippet. This returns a `204 No Content` status code if the operation was successfully or `404` if the resource was not found. ``` DELETE /projects/:id/snippets/:snippet_id @@ -150,4 +149,4 @@ Example response: } ``` -[ce-[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508 +[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508 diff --git a/doc/api/projects.md b/doc/api/projects.md index d3f8e509612..3144220e588 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -1,6 +1,6 @@ # Projects API -### Project visibility level +## Project visibility level Project in GitLab can be either private, internal or public. This is determined by the `visibility` field in the project. @@ -16,16 +16,15 @@ Values for the project visibility level are: * `public`: The project can be cloned without any authentication. -## List projects +## List all projects -Get a list of visible projects for authenticated user. When accessed without authentication, only public projects are returned. +Get a list of all visible projects across GitLab for the authenticated user. +When accessed without authentication, only public projects are returned. ``` GET /projects ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `archived` | boolean | no | Limit by archived status | @@ -70,6 +69,7 @@ Parameters: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -137,6 +137,7 @@ Parameters: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -191,16 +192,15 @@ Parameters: ] ``` -### List a user's projects +## List user projects -Get a list of visible projects for the given user. When accessed without authentication, only public projects are returned. +Get a list of visible projects for the given user. When accessed without +authentication, only public projects are returned. ``` GET /users/:user_id/projects ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `user_id` | string | yes | The ID or username of the user | @@ -246,6 +246,7 @@ Parameters: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -313,6 +314,7 @@ Parameters: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -367,7 +369,7 @@ Parameters: ] ``` -### Get single project +## Get single project Get a specific project. This endpoint can be accessed without authentication if the project is publicly accessible. @@ -376,8 +378,6 @@ the project is publicly accessible. GET /projects/:id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -411,6 +411,7 @@ Parameters: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -480,17 +481,14 @@ Parameters: Get the users list of a project. - -Parameters: +``` +GET /projects/:id/users +``` | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `search` | string | no | Search for specific users | -``` -GET /projects/:id/users -``` - ```json [ { @@ -512,11 +510,11 @@ GET /projects/:id/users ] ``` -### Get project events +## Get project events -Please refer to the [Events API documentation](events.md#list-a-projects-visible-events) +Please refer to the [Events API documentation](events.md#list-a-projects-visible-events). -### Create project +## Create project Creates a new project owned by the authenticated user. @@ -524,8 +522,6 @@ Creates a new project owned by the authenticated user. POST /projects ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `name` | string | yes if path is not provided | The name of the new project. Equals path if not provided. | @@ -537,6 +533,7 @@ Parameters: | `jobs_enabled` | boolean | no | Enable jobs for this project | | `wiki_enabled` | boolean | no | Enable wiki for this project | | `snippets_enabled` | boolean | no | Enable snippets for this project | +| `resolve_outdated_diff_discussions` | boolean | no | Automatically resolve merge request diffs discussions on lines changed with a push | | `container_registry_enabled` | boolean | no | Enable container registry for this project | | `shared_runners_enabled` | boolean | no | Enable shared runners for this project | | `visibility` | string | no | See [project visibility level](#project-visibility-level) | @@ -551,7 +548,7 @@ Parameters: | `printing_merge_request_link_enabled` | boolean | no | Show link to create/view merge request when pushing from the command line | | `ci_config_path` | string | no | The path to CI config file | -### Create project for user +## Create project for user Creates a new project owned by the specified user. Available only for admins. @@ -559,8 +556,6 @@ Creates a new project owned by the specified user. Available only for admins. POST /projects/user/:user_id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `user_id` | integer | yes | The user ID of the project owner | @@ -574,6 +569,7 @@ Parameters: | `jobs_enabled` | boolean | no | Enable jobs for this project | | `wiki_enabled` | boolean | no | Enable wiki for this project | | `snippets_enabled` | boolean | no | Enable snippets for this project | +| `resolve_outdated_diff_discussions` | boolean | no | Automatically resolve merge request diffs discussions on lines changed with a push | | `container_registry_enabled` | boolean | no | Enable container registry for this project | | `shared_runners_enabled` | boolean | no | Enable shared runners for this project | | `visibility` | string | no | See [project visibility level](#project-visibility-level) | @@ -588,7 +584,7 @@ Parameters: | `printing_merge_request_link_enabled` | boolean | no | Show link to create/view merge request when pushing from the command line | | `ci_config_path` | string | no | The path to CI config file | -### Edit project +## Edit project Updates an existing project. @@ -596,8 +592,6 @@ Updates an existing project. PUT /projects/:id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -610,6 +604,7 @@ Parameters: | `jobs_enabled` | boolean | no | Enable jobs for this project | | `wiki_enabled` | boolean | no | Enable wiki for this project | | `snippets_enabled` | boolean | no | Enable snippets for this project | +| `resolve_outdated_diff_discussions` | boolean | no | Automatically resolve merge request diffs discussions on lines changed with a push | | `container_registry_enabled` | boolean | no | Enable container registry for this project | | `shared_runners_enabled` | boolean | no | Enable shared runners for this project | | `visibility` | string | no | See [project visibility level](#project-visibility-level) | @@ -623,24 +618,24 @@ Parameters: | `avatar` | mixed | no | Image file for avatar of the project | | `ci_config_path` | string | no | The path to CI config file | -### Fork project +## Fork project Forks a project into the user namespace of the authenticated user or the one provided. -The forking operation for a project is asynchronous and is completed in a background job. The request will return immediately. To determine whether the fork of the project has completed, query the `import_status` for the new project. +The forking operation for a project is asynchronous and is completed in a +background job. The request will return immediately. To determine whether the +fork of the project has completed, query the `import_status` for the new project. ``` POST /projects/:id/fork ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | | `namespace` | integer/string | yes | The ID or path of the namespace that the project will be forked to | -### Star a project +## Star a project Stars a given project. Returns status code `304` if the project is already starred. @@ -648,8 +643,6 @@ Stars a given project. Returns status code `304` if the project is already starr POST /projects/:id/star ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -683,6 +676,7 @@ Example response: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -717,7 +711,7 @@ Example response: } ``` -### Unstar a project +## Unstar a project Unstars a given project. Returns status code `304` if the project is not starred. @@ -758,6 +752,7 @@ Example response: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -792,7 +787,7 @@ Example response: } ``` -### Archive a project +## Archive a project Archives the project if the user is either admin or the project owner of this project. This action is idempotent, thus archiving an already archived project will not change the project. @@ -839,6 +834,7 @@ Example response: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -885,7 +881,7 @@ Example response: } ``` -### Unarchive a project +## Unarchive a project Unarchives the project if the user is either admin or the project owner of this project. This action is idempotent, thus unarchiving an non-archived project will not change the project. @@ -932,6 +928,7 @@ Example response: "jobs_enabled": true, "wiki_enabled": true, "snippets_enabled": false, + "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", @@ -978,7 +975,7 @@ Example response: } ``` -### Remove project +## Remove project Removes a project including all associated resources (issues, merge requests etc.) @@ -986,15 +983,11 @@ Removes a project including all associated resources (issues, merge requests etc DELETE /projects/:id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | -## Uploads - -### Upload a file +## Upload a file Uploads a file to the specified project to be used in an issue or merge request description, or a comment. @@ -1002,8 +995,6 @@ Uploads a file to the specified project to be used in an issue or merge request POST /projects/:id/uploads ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1028,15 +1019,11 @@ Returned object: } ``` -**Note**: The returned `url` is relative to the project path. +>**Note**: The returned `url` is relative to the project path. In Markdown contexts, the link is automatically expanded when the format in `markdown` is used. -## Project members - -Please consult the [Project Members](members.md) documentation. - -### Share project with group +## Share project with group Allow to share project with group. @@ -1044,8 +1031,6 @@ Allow to share project with group. POST /projects/:id/share ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1053,7 +1038,7 @@ Parameters: | `group_access` | integer | yes | The permissions level to grant the group | | `expires_at` | string | no | Share expiration date in ISO 8601 format: 2016-09-26 | -### Delete a shared project link within a group +## Delete a shared project link within a group Unshare the project from the group. Returns `204` and no content on success. @@ -1061,8 +1046,6 @@ Unshare the project from the group. Returns `204` and no content on success. DELETE /projects/:id/share/:group_id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1085,8 +1068,6 @@ Get a list of project hooks. GET /projects/:id/hooks ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1099,8 +1080,6 @@ Get a specific hook for a project. GET /projects/:id/hooks/:hook_id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1132,8 +1111,6 @@ Adds a hook to a specified project. POST /projects/:id/hooks ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1157,8 +1134,6 @@ Edits a hook for a specified project. PUT /projects/:id/hooks/:hook_id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1184,8 +1159,6 @@ Either the hook is available or not. DELETE /projects/:id/hooks/:hook_id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1194,126 +1167,16 @@ Parameters: Note the JSON response differs if the hook is available or not. If the project hook is available before it is returned in the JSON response or an empty response is returned. -## Branches - -For more information please consult the [Branches](branches.md) documentation. - -### List branches - -Lists all branches of a project. - -``` -GET /projects/:id/repository/branches -``` - -Parameters: - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | - -```json -[ - { - "name": "async", - "commit": { - "id": "a2b702edecdf41f07b42653eb1abe30ce98b9fca", - "parent_ids": [ - "3f94fc7c85061973edc9906ae170cc269b07ca55" - ], - "message": "give Caolan credit where it's due (up top)", - "author_name": "Jeremy Ashkenas", - "author_email": "jashkenas@example.com", - "authored_date": "2010-12-08T21:28:50+00:00", - "committer_name": "Jeremy Ashkenas", - "committer_email": "jashkenas@example.com", - "committed_date": "2010-12-08T21:28:50+00:00" - }, - "protected": false, - "developers_can_push": false, - "developers_can_merge": false - }, - { - "name": "gh-pages", - "commit": { - "id": "101c10a60019fe870d21868835f65c25d64968fc", - "parent_ids": [ - "9c15d2e26945a665131af5d7b6d30a06ba338aaa" - ], - "message": "Underscore.js 1.5.2", - "author_name": "Jeremy Ashkenas", - "author_email": "jashkenas@example.com", - "authored_date": "2013-09-07T12:58:21+00:00", - "committer_name": "Jeremy Ashkenas", - "committer_email": "jashkenas@example.com", - "committed_date": "2013-09-07T12:58:21+00:00" - }, - "protected": false, - "developers_can_push": false, - "developers_can_merge": false - } -] -``` - -### Single branch - -A specific branch of a project. - -``` -GET /projects/:id/repository/branches/:branch -``` - -Parameters: - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | -| `branch` | string | yes | The name of the branch | -| `developers_can_push` | boolean | no | Flag if developers can push to the branch | -| `developers_can_merge` | boolean | no | Flag if developers can merge to the branch | - -### Protect single branch - -Protects a single branch of a project. - -``` -PUT /projects/:id/repository/branches/:branch/protect -``` - -Parameters: - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | -| `branch` | string | yes | The name of the branch | - -### Unprotect single branch - -Unprotects a single branch of a project. - -``` -PUT /projects/:id/repository/branches/:branch/unprotect -``` - -Parameters: - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | -| `branch` | string | yes | The name of the branch | - ## Admin fork relation Allows modification of the forked relationship between existing projects. Available only for admins. -### Create a forked from/to relation between existing projects. +### Create a forked from/to relation between existing projects ``` POST /projects/:id/fork/:forked_from_id ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1325,8 +1188,6 @@ Parameters: DELETE /projects/:id/fork ``` -Parameter: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | @@ -1341,8 +1202,6 @@ accessible. GET /projects ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `search` | string | yes | A string contained in the project name | @@ -1355,14 +1214,20 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/a ## Start the Housekeeping task for a Project ->**Note:** This feature was introduced in GitLab 9.0 +> Introduced in GitLab 9.0. ``` POST /projects/:id/housekeeping ``` -Parameters: - | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | + +## Branches + +Read more in the [Branches](branches.md) documentation. + +## Project members + +Read more in the [Project members](members.md) documentation. diff --git a/doc/api/runners.md b/doc/api/runners.md index 16d362a3530..6304a496f94 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -138,7 +138,8 @@ Example response: "ruby", "mysql" ], - "version": null + "version": null, + "access_level": "ref_protected" } ``` @@ -156,6 +157,9 @@ PUT /runners/:id | `description` | string | no | The description of a runner | | `active` | boolean | no | The state of a runner; can be set to `true` or `false` | | `tag_list` | array | no | The list of tags for a runner; put array of tags, that should be finally assigned to a runner | +| `run_untagged` | boolean | no | Flag indicating the runner can execute untagged jobs | +| `locked` | boolean | no | Flag indicating the runner is locked | +| `access_level` | string | no | The access_level of the runner; `not_protected` or `ref_protected` | ``` curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/runners/6" --form "description=test-1-20150125-test" --form "tag_list=ruby,mysql,tag1,tag2" @@ -190,7 +194,8 @@ Example response: "tag1", "tag2" ], - "version": null + "version": null, + "access_level": "ref_protected" } ``` diff --git a/doc/api/session.md b/doc/api/session.md index f79eac11689..b97e26f34a2 100644 --- a/doc/api/session.md +++ b/doc/api/session.md @@ -39,6 +39,7 @@ Example response: "twitter": "", "website_url": "", "email": "john@example.com", + "theme_id": 1, "color_scheme_id": 1, "projects_limit": 10, "current_sign_in_at": "2015-07-07T07:10:58.392Z", diff --git a/doc/api/settings.md b/doc/api/settings.md index 94a9f8265fb..b78f1252108 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -48,7 +48,11 @@ Example response: "plantuml_enabled": false, "plantuml_url": null, "terminal_max_session_time": 0, - "polling_interval_multiplier": 1.0 + "polling_interval_multiplier": 1.0, + "rsa_key_restriction": 0, + "dsa_key_restriction": 0, + "ecdsa_key_restriction": 0, + "ed25519_key_restriction": 0, } ``` @@ -88,6 +92,10 @@ PUT /application/settings | `plantuml_url` | string | yes (if `plantuml_enabled` is `true`) | The PlantUML instance URL for integration. | | `terminal_max_session_time` | integer | no | Maximum time for web terminal websocket connection (in seconds). Set to 0 for unlimited time. | | `polling_interval_multiplier` | decimal | no | Interval multiplier used by endpoints that perform polling. Set to 0 to disable polling. | +| `rsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded RSA key. Default is `0` (no restriction). `-1` disables RSA keys. +| `dsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded DSA key. Default is `0` (no restriction). `-1` disables DSA keys. +| `ecdsa_key_restriction` | integer | no | The minimum allowed curve size (in bits) of an uploaded ECDSA key. Default is `0` (no restriction). `-1` disables ECDSA keys. +| `ed25519_key_restriction` | integer | no | The minimum allowed curve size (in bits) of an uploaded ED25519 key. Default is `0` (no restriction). `-1` disables ED25519 keys. ```bash curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/application/settings?signup_enabled=false&default_project_visibility=internal @@ -125,6 +133,10 @@ Example response: "plantuml_enabled": false, "plantuml_url": null, "terminal_max_session_time": 0, - "polling_interval_multiplier": 1.0 + "polling_interval_multiplier": 1.0, + "rsa_key_restriction": 0, + "dsa_key_restriction": 0, + "ecdsa_key_restriction": 0, + "ed25519_key_restriction": 0, } ``` diff --git a/doc/api/users.md b/doc/api/users.md index 57a13eb477d..6d5db16b36a 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -72,6 +72,7 @@ GET /users "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 2, "projects_limit": 100, @@ -105,6 +106,7 @@ GET /users "organization": "", "last_sign_in_at": null, "confirmed_at": "2012-05-30T16:53:06.148Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 3, "projects_limit": 100, @@ -215,6 +217,7 @@ Parameters: "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 2, "projects_limit": 100, @@ -299,10 +302,7 @@ e.g. when renaming the email address to some existing one. ## User deletion Deletes a user. Available only for administrators. -This is an idempotent function, calling this function for a non-existent user id -still returns a status code `200 OK`. -The JSON response differs if the user was actually deleted or not. -In the former the user is returned and in the latter not. +This returns a `204 No Content` status code if the operation was successfully or `404` if the resource was not found. ``` DELETE /users/:id @@ -344,6 +344,7 @@ GET /user "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 2, "projects_limit": 100, @@ -390,6 +391,7 @@ GET /user "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", + "theme_id": 1, "last_activity_on": "2012-05-23", "color_scheme_id": 2, "projects_limit": 100, @@ -524,8 +526,7 @@ Parameters: ## Delete SSH key for current user Deletes key owned by currently authenticated user. -This is an idempotent function and calling it on a key that is already deleted -or not available results in `200 OK`. +This returns a `204 No Content` status code if the operation was successfully or `404` if the resource was not found. ``` DELETE /user/keys/:key_id @@ -548,7 +549,216 @@ Parameters: - `id` (required) - id of specified user - `key_id` (required) - SSH key ID -Will return `200 OK` on success, or `404 Not found` if either user or key cannot be found. +## List all GPG keys + +Get a list of currently authenticated user's GPG keys. + +``` +GET /user/gpg_keys +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/user/gpg_keys +``` + +Example response: + +```json +[ + { + "id": 1, + "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\r\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\r\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\r\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\r\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\r\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\r\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\r\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\r\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\r\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\r\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\r\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\r\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\r\n=XQoy\r\n-----END PGP PUBLIC KEY BLOCK-----", + "created_at": "2017-09-05T09:17:46.264Z" + } +] +``` + +## Get a specific GPG key + +Get a specific GPG key of currently authenticated user. + +``` +GET /user/gpg_keys/:key_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `key_id` | integer | yes | The ID of the GPG key | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/user/gpg_keys/1 +``` + +Example response: + +```json + { + "id": 1, + "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\r\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\r\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\r\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\r\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\r\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\r\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\r\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\r\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\r\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\r\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\r\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\r\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\r\n=XQoy\r\n-----END PGP PUBLIC KEY BLOCK-----", + "created_at": "2017-09-05T09:17:46.264Z" + } +``` + +## Add a GPG key + +Creates a new GPG key owned by the currently authenticated user. + +``` +POST /user/gpg_keys +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| key | string | yes | The new GPG key | + +```bash +curl --data "key=-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFV..." --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/user/gpg_keys +``` + +Example response: + +```json +[ + { + "id": 1, + "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\r\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\r\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\r\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\r\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\r\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\r\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\r\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\r\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\r\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\r\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\r\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\r\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\r\n=XQoy\r\n-----END PGP PUBLIC KEY BLOCK-----", + "created_at": "2017-09-05T09:17:46.264Z" + } +] +``` + +## Delete a GPG key + +Delete a GPG key owned by currently authenticated user. + +``` +DELETE /user/gpg_keys/:key_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `key_id` | integer | yes | The ID of the GPG key | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/user/gpg_keys/1 +``` + +Returns `204 No Content` on success, or `404 Not found` if the key cannot be found. + +## List all GPG keys for given user + +Get a list of a specified user's GPG keys. Available only for admins. + +``` +GET /users/:id/gpg_keys +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/2/gpg_keys +``` + +Example response: + +```json +[ + { + "id": 1, + "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\r\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\r\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\r\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\r\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\r\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\r\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\r\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\r\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\r\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\r\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\r\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\r\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\r\n=XQoy\r\n-----END PGP PUBLIC KEY BLOCK-----", + "created_at": "2017-09-05T09:17:46.264Z" + } +] +``` + +## Get a specific GPG key for a given user + +Get a specific GPG key for a given user. Available only for admins. + +``` +GET /users/:id/gpg_keys/:key_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the user | +| `key_id` | integer | yes | The ID of the GPG key | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/2/gpg_keys/1 +``` + +Example response: + +```json + { + "id": 1, + "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\r\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\r\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\r\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\r\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\r\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\r\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\r\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\r\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\r\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\r\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\r\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\r\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\r\n=XQoy\r\n-----END PGP PUBLIC KEY BLOCK-----", + "created_at": "2017-09-05T09:17:46.264Z" + } +``` + +## Add a GPG key for a given user + +Create new GPG key owned by the specified user. Available only for admins. + +``` +POST /users/:id/gpg_keys +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the user | +| `key_id` | integer | yes | The ID of the GPG key | + +```bash +curl --data "key=-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFV..." --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/2/gpg_keys +``` + +Example response: + +```json +[ + { + "id": 1, + "key": "-----BEGIN PGP PUBLIC KEY BLOCK-----\r\n\r\nxsBNBFVjnlIBCACibzXOLCiZiL2oyzYUaTOCkYnSUhymg3pdbfKtd4mpBa58xKBj\r\nt1pTHVpw3Sk03wmzhM/Ndlt1AV2YhLv++83WKr+gAHFYFiCV/tnY8bx3HqvVoy8O\r\nCfxWhw4QZK7+oYzVmJj8ZJm3ZjOC4pzuegNWlNLCUdZDx9OKlHVXLCX1iUbjdYWa\r\nqKV6tdV8hZolkbyjedQgrpvoWyeSHHpwHF7yk4gNJWMMI5rpcssL7i6mMXb/sDzO\r\nVaAtU5wiVducsOa01InRFf7QSTxoAm6Xy0PGv/k48M6xCALa9nY+BzlOv47jUT57\r\nvilf4Szy9dKD0v9S0mQ+IHB+gNukWrnwtXx5ABEBAAHNFm5hbWUgKGNvbW1lbnQp\r\nIDxlbUBpbD7CwHUEEwECACkFAlVjnlIJEINgJNgv009/AhsDAhkBBgsJCAcDAgYV\r\nCAIJCgsEFgIDAQAAxqMIAFBHuBA8P1v8DtHonIK8Lx2qU23t8Mh68HBIkSjk2H7/\r\noO2cDWCw50jZ9D91PXOOyMPvBWV2IE3tARzCvnNGtzEFRtpIEtZ0cuctxeIF1id5\r\ncrfzdMDsmZyRHAOoZ9VtuD6mzj0ybQWMACb7eIHjZDCee3Slh3TVrLy06YRdq2I4\r\nbjMOPePtK5xnIpHGpAXkB3IONxyITpSLKsA4hCeP7gVvm7r7TuQg1ygiUBlWbBYn\r\niE5ROzqZjG1s7dQNZK/riiU2umGqGuwAb2IPvNiyuGR3cIgRE4llXH/rLuUlspAp\r\no4nlxaz65VucmNbN1aMbDXLJVSqR1DuE00vEsL1AItI=\r\n=XQoy\r\n-----END PGP PUBLIC KEY BLOCK-----", + "created_at": "2017-09-05T09:17:46.264Z" + } +] +``` + +## Delete a GPG key for a given user + +Delete a GPG key owned by a specified user. Available only for admins. + +``` +DELETE /users/:id/gpg_keys/:key_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the user | +| `key_id` | integer | yes | The ID of the GPG key | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/2/gpg_keys/1 +``` ## List emails @@ -654,8 +864,7 @@ Parameters: ## Delete email for current user Deletes email owned by currently authenticated user. -This is an idempotent function and calling it on a email that is already deleted -or not available results in `200 OK`. +This returns a `204 No Content` status code if the operation was successfully or `404` if the resource was not found. ``` DELETE /user/emails/:email_id @@ -678,8 +887,6 @@ Parameters: - `id` (required) - id of specified user - `email_id` (required) - email ID -Will return `200 OK` on success, or `404 Not found` if either user or email cannot be found. - ## Block user Blocks the specified user. Available only for admin. diff --git a/doc/api/wikis.md b/doc/api/wikis.md new file mode 100644 index 00000000000..10eebc4a3c1 --- /dev/null +++ b/doc/api/wikis.md @@ -0,0 +1,157 @@ +# Wikis API + +> [Introduced][ce-13372] in GitLab 10.0. + +Available only in APIv4. + +## List wiki pages + +Get all wiki pages for a given project. + +``` +GET /projects/:id/wikis +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `with_content` | boolean | no | Include pages' content | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/1/wikis?with_content=1 +``` + +Example response: + +```json +[ + { + "content" : "Here is an instruction how to deploy this project.", + "format" : "markdown", + "slug" : "deploy", + "title" : "deploy" + }, + { + "content" : "Our development process is described here.", + "format" : "markdown", + "slug" : "development", + "title" : "development" + },{ + "content" : "* [Deploy](deploy)\n* [Development](development)", + "format" : "markdown", + "slug" : "home", + "title" : "home" + } +] +``` + +## Get a wiki page + +Get a wiki page for a given project. + +``` +GET /projects/:id/wikis/:slug +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `slug` | string | yes | The slug (a unique string) of the wiki page | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/1/wikis/home +``` + +Example response: + +```json +[ + { + "content" : "home page", + "format" : "markdown", + "slug" : "home", + "title" : "home" + } +] +``` + +## Create a new wiki page + +Creates a new wiki page for the given repository with the given title, slug, and content. + +``` +POST /projects/:id/wikis +``` + +| Attribute | Type | Required | Description | +| ------------- | ------- | -------- | ---------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `content` | string | yes | The content of the wiki page | +| `title` | string | yes | The title of the wiki page | +| `format` | string | no | The format of the wiki page. Available formats are: `markdown` (default), `rdoc`, and `asciidoc` | + +```bash +curl --data "format=rdoc&title=Hello&content=Hello world" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/wikis" +``` + +Example response: + +```json +{ + "content" : "Hello world", + "format" : "markdown", + "slug" : "Hello", + "title" : "Hello" +} +``` + +## Edit an existing wiki page + +Updates an existing wiki page. At least one parameter is required to update the wiki page. + +``` +PUT /projects/:id/wikis/:slug +``` + +| Attribute | Type | Required | Description | +| --------------- | ------- | --------------------------------- | ------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `content` | string | yes if `title` is not provided | The content of the wiki page | +| `title` | string | yes if `content` is not provided | The title of the wiki page | +| `format` | string | no | The format of the wiki page. Available formats are: `markdown` (default), `rdoc`, and `asciidoc` | +| `slug` | string | yes | The slug (a unique string) of the wiki page | + + +```bash +curl --request PUT --data "format=rdoc&content=documentation&title=Docs" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/wikis/foo" +``` + +Example response: + +```json +{ + "content" : "documentation", + "format" : "markdown", + "slug" : "Docs", + "title" : "Docs" +} +``` + +## Delete a wiki page + +Deletes a wiki page with a given slug. + +``` +DELETE /projects/:id/wikis/:slug +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `slug` | string | yes | The slug (a unique string) of the wiki page | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/wikis/foo" +``` + +On success the HTTP status code is `204` and no JSON response is expected. diff --git a/doc/articles/artifactory_and_gitlab/index.md b/doc/articles/artifactory_and_gitlab/index.md new file mode 100644 index 00000000000..c64851bad2b --- /dev/null +++ b/doc/articles/artifactory_and_gitlab/index.md @@ -0,0 +1,278 @@ +# How to deploy Maven projects to Artifactory with GitLab CI/CD + +> **Article [Type](../../development/writing_documentation.md#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Fabio Busatto](https://gitlab.com/bikebilly) || +> **Publication date:** 2017-08-15 + +## Introduction + +In this article, we will show how you can leverage the power of [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) +to build a [Maven](https://maven.apache.org/) project, deploy it to [Artifactory](https://www.jfrog.com/artifactory/), and then use it from another Maven application as a dependency. + +You'll create two different projects: + +- `simple-maven-dep`: the app built and deployed to Artifactory (available at https://gitlab.com/gitlab-examples/maven/simple-maven-dep) +- `simple-maven-app`: the app using the previous one as a dependency (available at https://gitlab.com/gitlab-examples/maven/simple-maven-app) + +We assume that you already have a GitLab account on [GitLab.com](https://gitlab.com/), and that you know the basic usage of Git and [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/). +We also assume that an Artifactory instance is available and reachable from the internet, and that you have valid credentials to deploy on it. + +## Create the simple Maven dependency + +First of all, you need an application to work with: in this specific case we will +use a simple one, but it could be any Maven application. This will be the +dependency you want to package and deploy to Artifactory, in order to be +available to other projects. + +### Prepare the dependency application + +For this article you'll use a Maven app that can be cloned from our example +project: + +1. Log in to your GitLab account +1. Create a new project by selecting **Import project from ➔ Repo by URL** +1. Add the following URL: + + ``` + https://gitlab.com/gitlab-examples/maven/simple-maven-dep.git + ``` +1. Click **Create project** + +This application is nothing more than a basic class with a stub for a JUnit based test suite. +It exposes a method called `hello` that accepts a string as input, and prints a hello message on the screen. + +The project structure is really simple, and you should consider these two resources: + +- `pom.xml`: project object model (POM) configuration file +- `src/main/java/com/example/dep/Dep.java`: source of our application + +### Configure the Artifactory deployment + +The application is ready to use, but you need some additional steps to deploy it to Artifactory: + +1. Log in to Artifactory with your user's credentials. +1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel. +1. Copy to clipboard the configuration snippet under the **Deploy** paragraph. +1. Change the `url` value in order to have it configurable via secret variables. +1. Copy the snippet in the `pom.xml` file for your project, just after the + `dependencies` section. The snippet should look like this: + + ```xml + <distributionManagement> + <repository> + <id>central</id> + <name>83d43b5afeb5-releases</name> + <url>${env.MAVEN_REPO_URL}/libs-release-local</url> + </repository> + </distributionManagement> + ``` + +Another step you need to do before you can deploy the dependency to Artifactory +is to configure the authentication data. It is a simple task, but Maven requires +it to stay in a file called `settings.xml` that has to be in the `.m2` subdirectory +in the user's homedir. + +Since you want to use GitLab Runner to automatically deploy the application, you +should create the file in the project's home directory and set a command line +parameter in `.gitlab-ci.yml` to use the custom location instead of the default one: + +1. Create a folder called `.m2` in the root of your repository +1. Create a file called `settings.xml` in the `.m2` folder +1. Copy the following content into a `settings.xml` file: + + ```xml + <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" + xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> + <servers> + <server> + <id>central</id> + <username>${env.MAVEN_REPO_USER}</username> + <password>${env.MAVEN_REPO_PASS}</password> + </server> + </servers> + </settings> + ``` + + Username and password will be replaced by the correct values using secret variables. + +### Configure GitLab CI/CD for `simple-maven-dep` + +Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) to automatically build, test and deploy the dependency! + +GitLab CI/CD uses a file in the root of the repo, named `.gitlab-ci.yml`, to read the definitions for jobs +that will be executed by the configured GitLab Runners. You can read more about this file in the [GitLab Documentation](https://docs.gitlab.com/ee/ci/yaml/). + +First of all, remember to set up secret variables for your deployment. Navigate to your project's **Settings > CI/CD** page +and add the following secret variables (replace them with your current values, of course): + +- **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL) +- **MAVEN_REPO_USER**: `gitlab` (your Artifactory username) +- **MAVEN_REPO_PASS**: `AKCp2WXr3G61Xjz1PLmYa3arm3yfBozPxSta4taP3SeNu2HPXYa7FhNYosnndFNNgoEds8BCS` (your Artifactory Encrypted Password) + +Now it's time to define jobs in `.gitlab-ci.yml` and push it to the repo: + +```yaml +image: maven:latest + +variables: + MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" + MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" + +cache: + paths: + - .m2/repository/ + - target/ + +build: + stage: build + script: + - mvn $MAVEN_CLI_OPTS compile + +test: + stage: test + script: + - mvn $MAVEN_CLI_OPTS test + +deploy: + stage: deploy + script: + - mvn $MAVEN_CLI_OPTS deploy + only: + - master +``` + +GitLab Runner will use the latest [Maven Docker image](https://hub.docker.com/_/maven/), which already contains all the tools and the dependencies you need to manage the project, +in order to run the jobs. + +Environment variables are set to instruct Maven to use the `homedir` of the repo instead of the user's home when searching for configuration and dependencies. + +Caching the `.m2/repository folder` (where all the Maven files are stored), and the `target` folder (where our application will be created), is useful for speeding up the process +by running all Maven phases in a sequential order, therefore, executing `mvn test` will automatically run `mvn compile` if necessary. + +Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application. + +Deploy to Artifactory is done as defined by the secret variables we have just set up. +The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published. + +Done! Now you have all the changes in the GitLab repo, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening. +If the deployment has been successful, the deploy job log will output: + +``` +[INFO] ------------------------------------------------------------------------ +[INFO] BUILD SUCCESS +[INFO] ------------------------------------------------------------------------ +[INFO] Total time: 1.983 s +``` + +>**Note**: +the `mvn` command downloads a lot of files from the internet, so you'll see a lot of extra activity in the log the first time you run it. + +Yay! You did it! Checking in Artifactory will confirm that you have a new artifact available in the `libs-release-local` repo. + +## Create the main Maven application + +Now that you have the dependency available on Artifactory, it's time to use it! +Let's see how we can have it as a dependency to our main application. + +### Prepare the main application + +We'll use again a Maven app that can be cloned from our example project: + +1. Create a new project by selecting **Import project from ➔ Repo by URL** +1. Add the following URL: + + ``` + https://gitlab.com/gitlab-examples/maven/simple-maven-app.git + ``` +1. Click **Create project** + +This one is a simple app as well. If you look at the `src/main/java/com/example/app/App.java` +file you can see that it imports the `com.example.dep.Dep` class and calls the `hello` method passing `GitLab` as a parameter. + +Since Maven doesn't know how to resolve the dependency, you need to modify the configuration: + +1. Go back to Artifactory +1. Browse the `libs-release-local` repository +1. Select the `simple-maven-dep-1.0.jar` file +1. Find the configuration snippet from the **Dependency Declaration** section of the main panel +1. Copy the snippet in the `dependencies` section of the `pom.xml` file. + The snippet should look like this: + + ```xml + <dependency> + <groupId>com.example.dep</groupId> + <artifactId>simple-maven-dep</artifactId> + <version>1.0</version> + </dependency> + ``` + +### Configure the Artifactory repository location + +At this point you defined the dependency for the application, but you still miss where you can find the required files. +You need to create a `.m2/settings.xml` file as you did for the dependency project, and let Maven know the location using environment variables. + +Here is how you can get the content of the file directly from Artifactory: + +1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel +1. Click on **Generate Maven Settings** +1. Click on **Generate Settings** +1. Copy to clipboard the configuration file +1. Save the file as `.m2/settings.xml` in your repo + +Now you are ready to use the Artifactory repository to resolve dependencies and use `simple-maven-dep` in your main application! + +### Configure GitLab CI/CD for `simple-maven-app` + +You need a last step to have everything in place: configure the `.gitlab-ci.yml` file for this project, as you already did for `simple-maven-dep`. + +You want to leverage [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/) to automatically build, test and run your awesome application, +and see if you can get the greeting as expected! + +All you need to do is to add the following `.gitlab-ci.yml` to the repo: + +```yaml +image: maven:latest + +stages: + - build + - test + - run + +variables: + MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode" + MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" + +cache: + paths: + - .m2/repository/ + - target/ + +build: + stage: build + script: + - mvn $MAVEN_CLI_OPTS compile + +test: + stage: test + script: + - mvn $MAVEN_CLI_OPTS test + +run: + stage: run + script: + - mvn $MAVEN_CLI_OPTS package + - mvn $MAVEN_CLI_OPTS exec:java -Dexec.mainClass="com.example.app.App" +``` + +It is very similar to the configuration used for `simple-maven-dep`, but instead of the `deploy` job there is a `run` job. +Probably something that you don't want to use in real projects, but here it is useful to see the application executed automatically. + +And that's it! In the `run` job output log you will find a friendly hello to GitLab! + +## Conclusion + +In this article we covered the basic steps to use an Artifactory Maven repository to automatically publish and consume artifacts. + +A similar approach could be used to interact with any other Maven compatible Binary Repository Manager. +Obviously, you can improve these examples, optimizing the `.gitlab-ci.yml` file to better suit your needs, and adapting to your workflow. diff --git a/doc/articles/how_to_configure_ldap_gitlab_ce/index.md b/doc/articles/how_to_configure_ldap_gitlab_ce/index.md index 130e8f542b4..25a24bc1d32 100644 --- a/doc/articles/how_to_configure_ldap_gitlab_ce/index.md +++ b/doc/articles/how_to_configure_ldap_gitlab_ce/index.md @@ -3,7 +3,7 @@ > **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** admin guide || > **Level:** intermediary || > **Author:** [Chris Wilson](https://gitlab.com/MrChrisW) || -> **Publication date:** 2017/05/03 +> **Publication date:** 2017-05-03 ## Introduction diff --git a/doc/articles/how_to_install_git/index.md b/doc/articles/how_to_install_git/index.md index 66d866b2d09..37b60501ce2 100644 --- a/doc/articles/how_to_install_git/index.md +++ b/doc/articles/how_to_install_git/index.md @@ -3,7 +3,7 @@ > **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** user guide || > **Level:** beginner || > **Author:** [Sean Packham](https://gitlab.com/SeanPackham) || -> **Publication date:** 2017/05/15 +> **Publication date:** 2017-05-15 To begin contributing to GitLab projects you will need to install the Git client on your computer. diff --git a/doc/articles/index.md b/doc/articles/index.md index 558c624fe39..798d4cbf4ff 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -17,8 +17,8 @@ Explore GitLab's supported [authentications methods](../topics/authentication/in | Article title | Category | Publishing date | | :------------ | :------: | --------------: | | **LDAP** | -| [How to configure LDAP with GitLab CE](how_to_configure_ldap_gitlab_ce/index.md)| Admin guide | 2017/05/03 | -| [How to configure LDAP with GitLab EE](https://docs.gitlab.com/ee/articles/how_to_configure_ldap_gitlab_ee/) | Admin guide | 2017/05/03 | +| [How to configure LDAP with GitLab CE](how_to_configure_ldap_gitlab_ce/index.md)| Admin guide | 2017-05-03 | +| [How to configure LDAP with GitLab EE](https://docs.gitlab.com/ee/articles/how_to_configure_ldap_gitlab_ee/) | Admin guide | 2017-05-03 | ## Build, test, and deploy with GitLab CI/CD @@ -26,17 +26,19 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM | Article title | Category | Publishing date | | :------------ | :------: | --------------: | -| [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017/07/13 | -| [Dockerizing GitLab Review Apps](https://about.gitlab.com/2017/07/11/dockerizing-review-apps/) | Concepts | 2017/07/11 | -| [Continuous Integration: From Jenkins to GitLab Using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/) | Concepts | 2017/07/27 | -| [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) | Tutorial | 2016/12/14 | -| [Setting up GitLab CI for Android projects](https://about.gitlab.com/2016/11/30/setting-up-gitlab-ci-for-android-projects/) | Tutorial | 2016/11/30 | -| [Automated Debian Package Build with GitLab CI](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) | Tutorial | 2016/10/12 | -| [Building an Elixir Release into a Docker image using GitLab CI](https://about.gitlab.com/2016/08/11/building-an-elixir-release-into-docker-image-using-gitlab-ci-part-1/) | Tutorial | 2016/08/11 | -| [Continuous Delivery with GitLab and Convox](https://about.gitlab.com/2016/06/09/continuous-delivery-with-gitlab-and-convox/) | Technical overview | 2016/06/09 | -| [GitLab Container Registry](https://about.gitlab.com/2016/05/23/gitlab-container-registry/) | Technical overview | 2016/05/23 | -| [How to use GitLab CI and MacStadium to build your macOS or iOS projects](https://about.gitlab.com/2017/05/15/how-to-use-macstadium-and-gitlab-ci-to-build-your-macos-or-ios-projects/) | Technical overview | 2017/05/15 | -| [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) | Tutorial | 2016/03/10 | +| [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 | +| [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) | Tutorial | 2017-08-15 | +| [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 | +| [Dockerizing GitLab Review Apps](https://about.gitlab.com/2017/07/11/dockerizing-review-apps/) | Concepts | 2017-07-11 | +| [Continuous Integration: From Jenkins to GitLab Using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/) | Concepts | 2017-07-27 | +| [Continuous Delivery of a Spring Boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) | Tutorial | 2016-12-14 | +| [Setting up GitLab CI for Android projects](https://about.gitlab.com/2016/11/30/setting-up-gitlab-ci-for-android-projects/) | Tutorial | 2016-11-30 | +| [Automated Debian Package Build with GitLab CI](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) | Tutorial | 2016-10-12 | +| [Building an Elixir Release into a Docker image using GitLab CI](https://about.gitlab.com/2016/08/11/building-an-elixir-release-into-docker-image-using-gitlab-ci-part-1/) | Tutorial | 2016-08-11 | +| [Continuous Delivery with GitLab and Convox](https://about.gitlab.com/2016/06/09/continuous-delivery-with-gitlab-and-convox/) | Technical overview | 2016-06-09 | +| [GitLab Container Registry](https://about.gitlab.com/2016/05/23/gitlab-container-registry/) | Technical overview | 2016-05-23 | +| [How to use GitLab CI and MacStadium to build your macOS or iOS projects](https://about.gitlab.com/2017/05/15/how-to-use-macstadium-and-gitlab-ci-to-build-your-macos-or-ios-projects/) | Technical overview | 2017-05-15 | +| [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) | Tutorial | 2016-03-10 | ## Git @@ -44,10 +46,11 @@ Learn how to use [Git with GitLab](../topics/git/index.md): | Article title | Category | Publishing date | | :------------ | :------: | --------------: | -| [Why Git is Worth the Learning Curve](https://about.gitlab.com/2017/05/17/learning-curve-is-the-biggest-challenge-developers-face-with-git/) | Concepts | 2017/05/17 | -| [How to install Git](how_to_install_git/index.md) | Tutorial | 2017/05/15 | -| [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) | Tutorial | 2017/01/30 | -| [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) | Technical overview | 2016/12/08 | +| [Numerous _undo_ possibilities in Git](numerous_undo_possibilities_in_git/index.md) | Tutorial | 2017-08-17 | +| [Why Git is Worth the Learning Curve](https://about.gitlab.com/2017/05/17/learning-curve-is-the-biggest-challenge-developers-face-with-git/) | Concepts | 2017-05-17 | +| [How to install Git](how_to_install_git/index.md) | Tutorial | 2017-05-15 | +| [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) | Tutorial | 2017-01-30 | +| [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) | Technical overview | 2016-12-08 | ## GitLab Pages @@ -56,32 +59,33 @@ Learn how to deploy a static website with [GitLab Pages](../user/project/pages/i | Article title | Category | Publishing date | | :------------ | :------: | --------------: | | **Series: GitLab Pages from A to Z:** | -| [- Part 1: Static sites and GitLab Pages domains](../user/project/pages/getting_started_part_one.md)| User guide | 2017/02/22 | -| [- Part 2: Quick start guide - Setting up GitLab Pages](../user/project/pages/getting_started_part_two.md)| User guide | 2017/02/22 | -| [- Part 3: Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](../user/project/pages/getting_started_part_three.md)| User guide | 2017/02/22 | -| [- Part 4: Creating and tweaking `.gitlab-ci.yml` for GitLab Pages](../user/project/pages/getting_started_part_four.md)| User guide | 2017/02/22 | -| [Setting up GitLab Pages with CloudFlare Certificates](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/) | Tutorial | 2017/02/07 | -| [Building a new GitLab Docs site with Nanoc, GitLab CI, and GitLab Pages](https://about.gitlab.com/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) | Tutorial | 2016/12/07 | -| [Publish Code Coverage Report with GitLab Pages](https://about.gitlab.com/2016/11/03/publish-code-coverage-report-with-gitlab-pages/) | Tutorial | 2016/11/03 | -| [GitLab CI: Deployment & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) | Tutorial | 2016/08/26 | -| [Posting to your GitLab Pages blog from iOS](https://about.gitlab.com/2016/08/19/posting-to-your-gitlab-pages-blog-from-ios/) | Tutorial | 2016/08/19 | +| [- Part 1: Static sites and GitLab Pages domains](../user/project/pages/getting_started_part_one.md)| User guide | 2017-02-22 | +| [- Part 2: Quick start guide - Setting up GitLab Pages](../user/project/pages/getting_started_part_two.md)| User guide | 2017-02-22 | +| [- Part 3: Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](../user/project/pages/getting_started_part_three.md)| User guide | 2017-02-22 | +| [- Part 4: Creating and tweaking `.gitlab-ci.yml` for GitLab Pages](../user/project/pages/getting_started_part_four.md)| User guide | 2017-02-22 | +| [Setting up GitLab Pages with CloudFlare Certificates](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/) | Tutorial | 2017-02-07 | +| [Building a new GitLab Docs site with Nanoc, GitLab CI, and GitLab Pages](https://about.gitlab.com/2016/12/07/building-a-new-gitlab-docs-site-with-nanoc-gitlab-ci-and-gitlab-pages/) | Tutorial | 2016-12-07 | +| [Publish Code Coverage Report with GitLab Pages](https://about.gitlab.com/2016/11/03/publish-code-coverage-report-with-gitlab-pages/) | Tutorial | 2016-11-03 | +| [GitLab CI: Deployment & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) | Tutorial | 2016-08-26 | +| [Posting to your GitLab Pages blog from iOS](https://about.gitlab.com/2016/08/19/posting-to-your-gitlab-pages-blog-from-ios/) | Tutorial | 2016-08-19 | | **Series: Static Site Generator:** | -| [- Part 1: Dynamic vs Static Websites](https://about.gitlab.com/2016/06/03/ssg-overview-gitlab-pages-part-1-dynamic-x-static/) | Tutorial | 2016/06/03 | -| [- Part 2: Modern Static Site Generators](https://about.gitlab.com/2016/06/10/ssg-overview-gitlab-pages-part-2/) | Tutorial | 2016/06/10 | -| [- Part 3: Build any SSG site with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) | Tutorial | 2016/06/17 | -| [Securing your GitLab Pages with TLS and Let's Encrypt](https://about.gitlab.com/2016/04/11/tutorial-securing-your-gitlab-pages-with-tls-and-letsencrypt/) | Tutorial | 2016/04/11 | -| [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) | Tutorial | 2016/04/07 | +| [- Part 1: Dynamic vs Static Websites](https://about.gitlab.com/2016/06/03/ssg-overview-gitlab-pages-part-1-dynamic-x-static/) | Tutorial | 2016-06-03 | +| [- Part 2: Modern Static Site Generators](https://about.gitlab.com/2016/06/10/ssg-overview-gitlab-pages-part-2/) | Tutorial | 2016-06-10 | +| [- Part 3: Build any SSG site with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) | Tutorial | 2016-06-17 | +| [Securing your GitLab Pages with TLS and Let's Encrypt](https://about.gitlab.com/2016/04/11/tutorial-securing-your-gitlab-pages-with-tls-and-letsencrypt/) | Tutorial | 2016-04-11 | +| [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) | Tutorial | 2016-04-07 | ## Install and maintain GitLab -Install, upgrade, integrate, migrate to GitLab: +[Admin](../README.md#administrator-documentation), [install](../install/README.md), +upgrade, integrate, migrate to GitLab: | Article title | Category | Publishing date | | :------------ | :------: | --------------: | -| [Video Tutorial: Idea to Production on Google Container Engine (GKE)](https://about.gitlab.com/2017/01/23/video-tutorial-idea-to-production-on-google-container-engine-gke/) | Tutorial | 2017/01/23 | -| [How to Setup a GitLab Instance on Microsoft Azure](https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/) | Tutorial | 2016/07/13 | -| [Get started with OpenShift Origin 3 and GitLab](openshift_and_gitlab/index.md) | Tutorial | 2016/06/28 | -| [Getting started with GitLab and DigitalOcean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/) | Tutorial | 2016/04/27 | +| [Video Tutorial: Idea to Production on Google Container Engine (GKE)](https://about.gitlab.com/2017/01/23/video-tutorial-idea-to-production-on-google-container-engine-gke/) | Tutorial | 2017-01-23 | +| [How to Setup a GitLab Instance on Microsoft Azure](https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/) | Tutorial | 2016-07-13 | +| [Get started with OpenShift Origin 3 and GitLab](openshift_and_gitlab/index.md) | Tutorial | 2016-06-28 | +| [Getting started with GitLab and DigitalOcean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/) | Tutorial | 2016-04-27 | ## Software development @@ -89,25 +93,25 @@ Explore the best of GitLab's software development's capabilities: | Article title | Category | Publishing date | | :------------ | :------: | --------------: | -| [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017/07/13 | -| [From 2/3 of the Self-Hosted Git Market, to the Next-Generation CI System, to Auto DevOps](https://about.gitlab.com/2017/06/29/whats-next-for-gitlab-ci/)| Concepts | 2017/06/29 | -| [Fast and Natural Continuous Integration with GitLab CI](https://about.gitlab.com/2017/05/22/fast-and-natural-continuous-integration-with-gitlab-ci/) | Concepts | 2017/05/22 | -| [Demo: Auto-Deploy from GitLab to an OpenShift Container Cluster](https://about.gitlab.com/2017/05/16/devops-containers-gitlab-openshift/) | Technical overview | 2017/05/16 | -| [Demo: GitLab Service Desk](https://about.gitlab.com/2017/05/09/demo-service-desk/) | Feature highlight | 2017/05/09 | -| [Demo: Mapping Work Versus Time, With Burndown Charts](https://about.gitlab.com/2017/04/25/mapping-work-to-do-versus-time-with-burndown-charts/) | Feature highlight | 2017/04/25 | -| [Demo: Cloud Native Development with GitLab](https://about.gitlab.com/2017/04/18/cloud-native-demo/) | Feature highlight | 2017/04/18 | -| [Demo: Mastering Code Review With GitLab](https://about.gitlab.com/2017/03/17/demo-mastering-code-review-with-gitlab/) | Feature highlight | 2017/03/17 | -| [In 13 minutes from Kubernetes to a complete application development tool](https://about.gitlab.com/2016/11/14/idea-to-production/) | Technical overview | 2016/11/14 | -| [GitLab Workflow, an Overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) | Technical overview | 2016/10/25 | -| [Trends in Version Control Land: Microservices](https://about.gitlab.com/2016/08/16/trends-in-version-control-land-microservices/) | Concepts | 2016/08/16 | -| [Continuous Integration, Delivery, and Deployment with GitLab](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) | Concepts | 2016/08/05 | -| [Trends in Version Control Land: Innersourcing](https://about.gitlab.com/2016/07/07/trends-version-control-innersourcing/) | Concepts | 2016/07/07 | -| [Tutorial: It's all connected in GitLab](https://about.gitlab.com/2016/03/08/gitlab-tutorial-its-all-connected/) | Technical overview | 2016/03/08 | +| [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 | +| [From 2/3 of the Self-Hosted Git Market, to the Next-Generation CI System, to Auto DevOps](https://about.gitlab.com/2017/06/29/whats-next-for-gitlab-ci/)| Concepts | 2017-06-29 | +| [Fast and Natural Continuous Integration with GitLab CI](https://about.gitlab.com/2017/05/22/fast-and-natural-continuous-integration-with-gitlab-ci/) | Concepts | 2017-05-22 | +| [Demo: Auto-Deploy from GitLab to an OpenShift Container Cluster](https://about.gitlab.com/2017/05/16/devops-containers-gitlab-openshift/) | Technical overview | 2017-05-16 | +| [Demo: GitLab Service Desk](https://about.gitlab.com/2017/05/09/demo-service-desk/) | Feature highlight | 2017-05-09 | +| [Demo: Mapping Work Versus Time, With Burndown Charts](https://about.gitlab.com/2017/04/25/mapping-work-to-do-versus-time-with-burndown-charts/) | Feature highlight | 2017-04-25 | +| [Demo: Cloud Native Development with GitLab](https://about.gitlab.com/2017/04/18/cloud-native-demo/) | Feature highlight | 2017-04-18 | +| [Demo: Mastering Code Review With GitLab](https://about.gitlab.com/2017/03/17/demo-mastering-code-review-with-gitlab/) | Feature highlight | 2017-03-17 | +| [In 13 minutes from Kubernetes to a complete application development tool](https://about.gitlab.com/2016/11/14/idea-to-production/) | Technical overview | 2016-11-14 | +| [GitLab Workflow, an Overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) | Technical overview | 2016-10-25 | +| [Trends in Version Control Land: Microservices](https://about.gitlab.com/2016/08/16/trends-in-version-control-land-microservices/) | Concepts | 2016-08-16 | +| [Continuous Integration, Delivery, and Deployment with GitLab](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) | Concepts | 2016-08-05 | +| [Trends in Version Control Land: Innersourcing](https://about.gitlab.com/2016/07/07/trends-version-control-innersourcing/) | Concepts | 2016-07-07 | +| [Tutorial: It's all connected in GitLab](https://about.gitlab.com/2016/03/08/gitlab-tutorial-its-all-connected/) | Technical overview | 2016-03-08 | ## Technologies | Article title | Category | Publishing date | | :------------ | :------: | --------------: | -| [Why we are not leaving the cloud](https://about.gitlab.com/2017/03/02/why-we-are-not-leaving-the-cloud/) | Concepts | 2017/03/02 | -| [Why We Chose Vue.js](https://about.gitlab.com/2016/10/20/why-we-chose-vue/) | Concepts | 2016/10/20 | -| [Markdown Kramdown Tips & Tricks](https://about.gitlab.com/2016/07/19/markdown-kramdown-tips-and-tricks/) | Technical overview | 2016/07/19 | +| [Why we are not leaving the cloud](https://about.gitlab.com/2017/03/02/why-we-are-not-leaving-the-cloud/) | Concepts | 2017-03-02 | +| [Why We Chose Vue.js](https://about.gitlab.com/2016/10/20/why-we-chose-vue/) | Concepts | 2016-10-20 | +| [Markdown Kramdown Tips & Tricks](https://about.gitlab.com/2016/07/19/markdown-kramdown-tips-and-tricks/) | Technical overview | 2016-07-19 | diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png Binary files differnew file mode 100644 index 00000000000..a56c07a0da7 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png Binary files differnew file mode 100644 index 00000000000..b1406fed6b8 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg Binary files differnew file mode 100644 index 00000000000..d1f0cbc08ab --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png Binary files differnew file mode 100644 index 00000000000..9aae11b8679 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png Binary files differnew file mode 100644 index 00000000000..a06b6d417cd --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png Binary files differnew file mode 100644 index 00000000000..d357ecda7d2 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png Binary files differnew file mode 100644 index 00000000000..3bb21fd12b4 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png b/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png Binary files differnew file mode 100644 index 00000000000..bc188f83fb1 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png Binary files differnew file mode 100644 index 00000000000..baf8dec499c --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png Binary files differnew file mode 100644 index 00000000000..d96c43bcf16 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png b/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png Binary files differnew file mode 100644 index 00000000000..997db10189f --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png b/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png Binary files differnew file mode 100644 index 00000000000..6dbc29fc25c --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png b/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png Binary files differnew file mode 100644 index 00000000000..8a6dcccfa38 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png Binary files differnew file mode 100644 index 00000000000..658c0b5bcac --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/index.md b/doc/articles/laravel_with_gitlab_and_envoy/index.md new file mode 100644 index 00000000000..e0d8fb8d081 --- /dev/null +++ b/doc/articles/laravel_with_gitlab_and_envoy/index.md @@ -0,0 +1,680 @@ +# Test and deploy Laravel applications with GitLab CI/CD and Envoy + +> **[Article Type](../../development/writing_documentation.md#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Mehran Rasulian](https://gitlab.com/mehranrasulian) || +> **Publication date:** 2017-08-31 + +## Introduction + +GitLab features our applications with Continuous Integration, and it is possible to easily deploy the new code changes to the production server whenever we want. + +In this tutorial, we'll show you how to initialize a [Laravel](http://laravel.com/) application and setup our [Envoy](https://laravel.com/docs/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../../ci/README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/). + +We assume you have a basic experience with Laravel, Linux servers, +and you know how to use GitLab. + +Laravel is a high quality web framework written in PHP. +It has a great community with a [fantastic documentation](https://laravel.com/docs). +Aside from the usual routing, controllers, requests, responses, views, and (blade) templates, out of the box Laravel provides plenty of additional services such as cache, events, localization, authentication and many others. + +We will use [Envoy](https://laravel.com/docs/master/envoy) as an SSH task runner based on PHP. +It uses a clean, minimal [Blade syntax](https://laravel.com/docs/blade) to setup tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/artisan). + +## Initialize our Laravel app on GitLab + +We assume [you have installed a new laravel project](https://laravel.com/docs/installation#installation), so let's start with a unit test, and initialize Git for the project. + +### Unit Test + +Every new installation of Laravel (currently 5.4) comes with two type of tests, 'Feature' and 'Unit', placed in the tests directory. +Here's a unit test from `test/Unit/ExampleTest.php`: + +```php +<?php + +namespace Tests\Unit; + +... + +class ExampleTest extends TestCase +{ + public function testBasicTest() + { + $this->assertTrue(true); + } +} +``` + +This test is as simple as asserting that the given value is true. + +Laravel uses `PHPUnit` for tests by default. +If we run `vendor/bin/phpunit` we should see the green output: + +```bash +vendor/bin/phpunit +OK (1 test, 1 assertions) +``` + +This test will be used later for continuously testing our app with GitLab CI/CD. + +### Push to GitLab + +Since we have our app up and running locally, it's time to push the codebase to our remote repository. +Let's create [a new project](../../gitlab-basics/create-project.md) in GitLab named `laravel-sample`. +After that, follow the command line instructions displayed on the project's homepage to initiate the repository on our machine and push the first commit. + + +```bash +cd laravel-sample +git init +git remote add origin git@gitlab.example.com:<USERNAME>/laravel-sample.git +git add . +git commit -m 'Initial Commit' +git push -u origin master +``` + +## Configure the production server + +Before we begin setting up Envoy and GitLab CI/CD, let's quickly make sure the production server is ready for deployment. +We have installed LEMP stack which stands for Linux, Nginx, MySQL and PHP on our Ubuntu 16.04. + +### Create a new user + +Let's now create a new user that will be used to deploy our website and give it +the needed permissions using [Linux ACL](https://serversforhackers.com/video/linux-acls): + +```bash +# Create user deployer +sudo adduser deployer +# Give the read-write-execute permissions to deployer user for directory /var/www +sudo setfacl -R -m u:deployer:rwx /var/www +``` + +If you don't have ACL installed on your Ubuntu server, use this command to install it: + +```bash +sudo apt install acl +``` + +### Add SSH key + +Let's suppose we want to deploy our app to the production server from a private repository on GitLab. First, we need to [generate a new SSH key pair **with no passphrase**](../../ssh/README.md) for the deployer user. + +After that, we need to copy the private key, which will be used to connect to our server as the deployer user with SSH, to be able to automate our deployment process: + +```bash +# As the deployer user on server +# +# Copy the content of public key to authorized_keys +cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys +# Copy the private key text block +cat ~/.ssh/id_rsa +``` + +Now, let's add it to your GitLab project as a [secret variable](../../ci/variables/README.md#secret-variables). +Secret variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. +They can be added per project by navigating to the project's **Settings** > **CI/CD**. + + + +To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. +We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our remote server as the deployer user without entering its password. + +We also need to add the public key to **Project** > **Settings** > **Repository** as [Deploy Keys](../../ssh/README.md/#deploy-keys), which gives us the ability to access our repository from the server through [SSH protocol](../../gitlab-basics/command-line-commands.md/#start-working-on-your-project). + + +```bash +# As the deployer user on the server +# +# Copy the public key +cat ~/.ssh/id_rsa.pub +``` + + + +To the field **Title**, add any name you want, and paste the public key into the **Key** field. + +Now, let's clone our repository on the server just to make sure the `deployer` user has access to the repository. + +```bash +# As the deployer user on server +# +git clone git@gitlab.example.com:<USERNAME>/laravel-sample.git +``` + +>**Note:** +Answer **yes** if asked `Are you sure you want to continue connecting (yes/no)?`. +It adds GitLab.com to the known hosts. + +### Configuring Nginx + +Now, let's make sure our web server configuration points to the `current/public` rather than `public`. + +Open the default Nginx server block configuration file by typing: + +```bash +sudo nano /etc/nginx/sites-available/default +``` + +The configuration should be like this. + +``` +server { + root /var/www/app/current/public; + server_name example.com; + # Rest of the configuration +} +``` + +>**Note:** +You may replace the app's name in `/var/www/app/current/public` with the folder name of your application. + +## Setting up Envoy + +So we have our Laravel app ready for production. +The next thing is to use Envoy to perform the deploy. + +To use Envoy, we should first install it on our local machine [using the given instructions by Laravel](https://laravel.com/docs/envoy/#introduction). + +### How Envoy works + +The pros of Envoy is that it doesn't require Blade engine, it just uses Blade syntax to define tasks. +To start, we create an `Envoy.blade.php` in the root of our app with a simple task to test Envoy. + + +```php +@servers(['web' => 'remote_username@remote_host']) + +@task('list', [on => 'web']) + ls -l +@endtask +``` + +As you may expect, we have an array within `@servers` directive at the top of the file, which contains a key named `web` with a value of the server's address (e.g. `deployer@192.168.1.1`). +Then within our `@task` directive we define the bash commands that should be run on the server when the task is executed. + +On the local machine use the `run` command to run Envoy tasks. + +```bash +envoy run list +``` + +It should execute the `list` task we defined earlier, which connects to the server and lists directory contents. + +Envoy is not a dependency of Laravel, therefore you can use it for any PHP application. + +### Zero downtime deployment + +Every time we deploy to the production server, Envoy downloads the latest release of our app from GitLab repository and replace it with preview's release. +Envoy does this without any [downtime](https://en.wikipedia.org/wiki/Downtime), +so we don't have to worry during the deployment while someone might be reviewing the site. +Our deployment plan is to clone the latest release from GitLab repository, install the Composer dependencies and finally, activate the new release. + +#### @setup directive + +The first step of our deployment process is to define a set of variables within [@setup](https://laravel.com/docs/envoy/#setup) directive. +You may change the `app` to your application's name: + + +```php +... + +@setup + $repository = 'git@gitlab.example.com:<USERNAME>/laravel-sample.git'; + $releases_dir = '/var/www/app/releases'; + $app_dir = '/var/www/app'; + $release = date('YmdHis'); + $new_release_dir = $releases_dir .'/'. $release; +@endsetup + +... +``` + +- `$repository` is the address of our repository +- `$releases_dir` directory is where we deploy the app +- `$app_dir` is the actual location of the app that is live on the server +- `$release` contains a date, so every time that we deploy a new release of our app, we get a new folder with the current date as name +- `$new_release_dir` is the full path of the new release which is used just to make the tasks cleaner + +#### @story directive + +The [@story](https://laravel.com/docs/envoy/#stories) directive allows us define a list of tasks that can be run as a single task. +Here we have three tasks called `clone_repository`, `run_composer`, `update_symlinks`. These variables are usable to making our task's codes more cleaner: + + +```php +... + +@story('deploy') + clone_repository + run_composer + update_symlinks +@endstory + +... +``` + +Let's create these three tasks one by one. + +#### Clone the repository + +The first task will create the `releases` directory (if it doesn't exist), and then clone the `master` branch of the repository (by default) into the new release directory, given by the `$new_release_dir` variable. +The `releases` directory will hold all our deployments: + +```php +... + +@task('clone_repository') + echo 'Cloning repository' + [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} + git clone --depth 1 {{ $repository }} {{ $new_release_dir }} +@endtask + +... +``` + +While our project grows, its Git history will be very very long over time. +Since we are creating a directory per release, it might not be necessary to have the history of the project downloaded for each release. +The `--depth 1` option is a great solution which saves systems time and disk space as well. + +#### Installing dependencies with Composer + +As you may know, this task just navigates to the new release directory and runs Composer to install the application dependencies: + +```php +... + +@task('run_composer') + echo "Starting deployment ({{ $release }})" + cd {{ $new_release_dir }} + composer install --prefer-dist --no-scripts -q -o +@endtask + +... +``` + +#### Activate new release + +Next thing to do after preparing the requirements of our new release, is to remove the storage directory from it and to create two symbolic links to point the application's `storage` directory and `.env` file to the new release. +Then, we need to create another symbolic link to the new release with the name of `current` placed in the app directory. +The `current` symbolic link always points to the latest release of our app: + +```php +... + +@task('update_symlinks') + echo "Linking storage directory" + rm -rf {{ $new_release_dir }}/storage + ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage + + echo 'Linking .env file' + ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env + + echo 'Linking current release' + ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current +@endtask +``` + +As you see, we use `-nfs` as an option for `ln` command, which says that the `storage`, `.env` and `current` no longer points to the preview's release and will point them to the new release by force (`f` from `-nfs` means force), which is the case when we are doing multiple deployments. + +### Full script + +The script is ready, but make sure to change the `deployer@192.168.1.1` to your server and also change `/var/www/app` with the directory you want to deploy your app. + +At the end, our `Envoy.blade.php` file will look like this: + +```php +@servers(['web' => 'deployer@192.168.1.1']) + +@setup + $repository = 'git@gitlab.example.com:<USERNAME>/laravel-sample.git'; + $releases_dir = '/var/www/app/releases'; + $app_dir = '/var/www/app'; + $release = date('YmdHis'); + $new_release_dir = $releases_dir .'/'. $release; +@endsetup + +@story('deploy') + clone_repository + run_composer + update_symlinks +@endstory + +@task('clone_repository') + echo 'Cloning repository' + [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} + git clone --depth 1 {{ $repository }} {{ $new_release_dir }} +@endtask + +@task('run_composer') + echo "Starting deployment ({{ $release }})" + cd {{ $new_release_dir }} + composer install --prefer-dist --no-scripts -q -o +@endtask + +@task('update_symlinks') + echo "Linking storage directory" + rm -rf {{ $new_release_dir }}/storage + ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage + + echo 'Linking .env file' + ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env + + echo 'Linking current release' + ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current +@endtask +``` + +One more thing we should do before any deployment is to manually copy our application `storage` folder to the `/var/www/app` directory on the server for the first time. +You might want to create another Envoy task to do that for you. +We also create the `.env` file in the same path to setup our production environment variables for Laravel. +These are persistent data and will be shared to every new release. + +Now, we would need to deploy our app by running `envoy run deploy`, but it won't be necessary since GitLab can handle that for us with CI's [environments](../../ci/environments.md), which will be described [later](#setting-up-gitlab-ci-cd) in this tutorial. + +Now it's time to commit [Envoy.blade.php](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Envoy.blade.php) and push it to the `master` branch. +To keep things simple, we commit directly to `master`, without using [feature-branches](../../workflow/gitlab_flow.md/#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial. +In a real world project, teams may use [Issue Tracker](../../user/project/issues/index.md) and [Merge Requests](../../user/project/merge_requests/index.md) to move their code across branches: + +```bash +git add Envoy.blade.php +git commit -m 'Add Envoy' +git push origin master +``` + +## Continuous Integration with GitLab + +We have our app ready on GitLab, and we also can deploy it manually. +But let's take a step forward to do it automatically with [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) method. +We need to check every commit with a set of automated tests to become aware of issues at the earliest, and then, we can deploy to the target environment if we are happy with the result of the tests. + +[GitLab CI/CD](../../ci/README.md) allows us to use [Docker](https://docker.com/) engine to handle the process of testing and deploying our app. +In the case you're not familiar with Docker, refer to [How to Automate Docker Deployments](http://paislee.io/how-to-automate-docker-deployments/). + +To be able to build, test, and deploy our app with GitLab CI/CD, we need to prepare our work environment. +To do that, we'll use a Docker image which has the minimum requirements that a Laravel app needs to run. +[There are other ways](../../ci/examples/php.md/#test-php-projects-using-the-docker-executor) to do that as well, but they may lead our builds run slowly, which is not what we want when there are faster options to use. + +With Docker images our builds run incredibly faster! + +### Create a Container Image + +Let's create a [Dockerfile](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Dockerfile) in the root directory of our app with the following content: + +```bash +# Set the base image for subsequent instructions +FROM php:7.1 + +# Update packages +RUN apt-get update + +# Install PHP and composer dependencies +RUN apt-get install -qq git curl libmcrypt-dev libjpeg-dev libpng-dev libfreetype6-dev libbz2-dev + +# Clear out the local repository of retrieved package files +RUN apt-get clean + +# Install needed extensions +# Here you can install any other extension that you need during the test and deployment process +RUN docker-php-ext-install mcrypt pdo_mysql zip + +# Install Composer +RUN curl --silent --show-error https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer + +# Install Laravel Envoy +RUN composer global require "laravel/envoy=~1.0" +``` + +We added the [official PHP 7.1 Docker image](https://hub.docker.com/r/_/php/), which consist of a minimum installation of Debian Jessie with PHP pre-installed, and works perfectly for our use case. + +We used `docker-php-ext-install` (provided by the official PHP Docker image) to install the PHP extensions we need. + +#### Setting Up GitLab Container Registry + +Now that we have our `Dockerfile` let's build and push it to our [GitLab Container Registry](../../user/project/container_registry.md). + +> The registry is the place to store and tag images for later use. Developers may want to maintain their own registry for private, company images, or for throw-away images used only in testing. Using GitLab Container Registry means you don't need to set up and administer yet another service or use a public registry. + +On your GitLab project repository navigate to the **Registry** tab. + + + +You may need to [enable Container Registry](../../user/project/container_registry.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Sharing and permissions**. + + + +To start using Container Registry on our machine, we first need to login to the GitLab registry using our GitLab username and password: + +```bash +docker login registry.gitlab.com +``` +Then we can build and push our image to GitLab: + +```bash +docker build -t registry.gitlab.com/<USERNAME>/laravel-sample . + +docker push registry.gitlab.com/<USERNAME>/laravel-sample +``` + +>**Note:** +To run the above commands, we first need to have [Docker](https://docs.docker.com/engine/installation/) installed on our machine. + +Congratulations! You just pushed the first Docker image to the GitLab Registry, and if you refresh the page you should be able to see it: + + + +>**Note:** +You can also [use GitLab CI/CD](https://about.gitlab.com/2016/05/23/gitlab-container-registry/#use-with-gitlab-ci) to build and push your Docker images, rather than doing that on your machine. + +We'll use this image further down in the `.gitlab-ci.yml` configuration file to handle the process of testing and deploying our app. + +Let's commit the `Dockerfile` file. + +```bash +git add Dockerfile +git commit -m 'Add Dockerfile' +git push origin master +``` + +### Setting up GitLab CI/CD + +In order to build and test our app with GitLab CI/CD, we need a file called `.gitlab-ci.yml` in our repository's root. It is similar to Circle CI and Travis CI, but built-in GitLab. + +Our `.gitlab-ci.yml` file will look like this: + +```yaml +image: registry.gitlab.com/<USERNAME>/laravel-sample:latest + +services: + - mysql:5.7 + +variables: + MYSQL_DATABASE: homestead + MYSQL_ROOT_PASSWORD: secret + DB_HOST: mysql + DB_USERNAME: root + +stages: + - test + - deploy + +unit_test: + stage: test + script: + - composer install + - cp .env.example .env + - php artisan key:generate + - php artisan migrate + - vendor/bin/phpunit + +deploy_production: + stage: deploy + script: + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + - eval $(ssh-agent -s) + - ssh-add <(echo "$SSH_PRIVATE_KEY") + - mkdir -p ~/.ssh + - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' + + - ~/.composer/vendor/bin/envoy run deploy + environment: + name: production + url: http://192.168.1.1 + when: manual + only: + - master +``` + +That's a lot to take in, isn't it? Let's run through it step by step. + +#### Image and Services + +[GitLab Runners](../../ci/runners/README.md) run the script defined by `.gitlab-ci.yml`. +The `image` keyword tells the Runners which image to use. +The `services` keyword defines additional images [that are linked to the main image](../../ci/docker/using_docker_images.md/#what-is-a-service). +Here we use the container image we created before as our main image and also use MySQL 5.7 as a service. + +```yaml +image: registry.gitlab.com/<USERNAME>/laravel-sample:latest + +services: + - mysql:5.7 + +... +``` + +>**Note:** +If you wish to test your app with different PHP versions and [database management systems](../../ci/services/README.md), you can define different `image` and `services` keywords for each test job. + +#### Variables + +GitLab CI/CD allows us to use [environment variables](../../ci/yaml/README.md#variables) in our jobs. +We defined MySQL as our database management system, which comes with a superuser root created by default. + +So we should adjust the configuration of MySQL instance by defining `MYSQL_DATABASE` variable as our database name and `MYSQL_ROOT_PASSWORD` variable as the password of `root`. +Find out more about MySQL variables at the [official MySQL Docker Image](https://hub.docker.com/r/_/mysql/). + +Also set the variables `DB_HOST` to `mysql` and `DB_USERNAME` to `root`, which are Laravel specific variables. +We define `DB_HOST` as `mysql` instead of `127.0.0.1`, as we use MySQL Docker image as a service which [is linked to the main Docker image](../../ci/docker/using_docker_images.md/#how-services-are-linked-to-the-build). + +```yaml +... + +variables: + MYSQL_DATABASE: homestead + MYSQL_ROOT_PASSWORD: secret + DB_HOST: mysql + DB_USERNAME: root + +... +``` + +#### Unit Test as the first job + +We defined the required shell scripts as an array of the [script](../../ci/yaml/README.md#script) variable to be executed when running `unit_test` job. + +These scripts are some Artisan commands to prepare the Laravel, and, at the end of the script, we'll run the tests by `PHPUnit`. + +```yaml +... + +unit_test: + script: + # Install app dependencies + - composer install + # Setup .env + - cp .env.example .env + # Generate an environment key + - php artisan key:generate + # Run migrations + - php artisan migrate + # Run tests + - vendor/bin/phpunit + +... +``` + +#### Deploy to production + +The job `deploy_production` will deploy the app to the production server. +To deploy our app with Envoy, we had to set up the `$SSH_PRIVATE_KEY` variable as an [SSH private key](../../ci/ssh_keys/README.md/#ssh-keys-when-using-the-docker-executor). +If the SSH keys have added successfully, we can run Envoy. + +As mentioned before, GitLab supports [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) methods as well. +The [environment](../../ci/yaml/README.md#environment) keyword tells GitLab that this job deploys to the `production` environment. +The `url` keyword is used to generate a link to our application on the GitLab Environments page. +The `only` keyword tells GitLab CI that the job should be executed only when the pipeline is building the `master` branch. +Lastly, `when: manual` is used to turn the job from running automatically to a manual action. + +```yaml +... + +deploy_production: + script: + # Add the private SSH key to the build environment + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + - eval $(ssh-agent -s) + - ssh-add <(echo "$SSH_PRIVATE_KEY") + - mkdir -p ~/.ssh + - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' + + # Run Envoy + - ~/.composer/vendor/bin/envoy run deploy + + environment: + name: production + url: http://192.168.1.1 + when: manual + only: + - master +``` + +You may also want to add another job for [staging environment](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments), to final test your application before deploying to production. + +### Turn on GitLab CI/CD + +We have prepared everything we need to test and deploy our app with GitLab CI/CD. +To do that, commit and push `.gitlab-ci.yml` to the `master` branch. It will trigger a pipeline, which you can watch live under your project's **Pipelines**. + + + +Here we see our **Test** and **Deploy** stages. +The **Test** stage has the `unit_test` build running. +click on it to see the Runner's output. + + + +After our code passed through the pipeline successfully, we can deploy to our production server by clicking the **play** button on the right side. + + + +Once the deploy pipeline passed successfully, navigate to **Pipelines > Environments**. + + + +If something doesn't work as expected, you can roll back to the latest working version of your app. + + + +By clicking on the external link icon specified on the right side, GitLab opens the production website. +Our deployment successfully was done and we can see the application is live. + + + +In the case that you're interested to know how is the application directory structure on the production server after deployment, here are three directories named `current`, `releases` and `storage`. +As you know, the `current` directory is a symbolic link that points to the latest release. +The `.env` file consists of our Laravel environment variables. + + + +If you navigate to the `current` directory, you should see the application's content. +As you see, the `.env` is pointing to the `/var/www/app/.env` file and also `storage` is pointing to the `/var/www/app/storage/` directory. + + + +## Conclusion + +We configured GitLab CI to perform automated tests and used the method of [Continuous Delivery](https://continuousdelivery.com/) to deploy to production a Laravel application with Envoy, directly from the codebase. + +Envoy also was a great match to help us deploy the application without writing our custom bash script and doing Linux magics. diff --git a/doc/articles/numerous_undo_possibilities_in_git/img/branching.png b/doc/articles/numerous_undo_possibilities_in_git/img/branching.png Binary files differnew file mode 100644 index 00000000000..9a80c211c99 --- /dev/null +++ b/doc/articles/numerous_undo_possibilities_in_git/img/branching.png diff --git a/doc/articles/numerous_undo_possibilities_in_git/img/rebase_reset.png b/doc/articles/numerous_undo_possibilities_in_git/img/rebase_reset.png Binary files differnew file mode 100644 index 00000000000..ac7ea9ecddc --- /dev/null +++ b/doc/articles/numerous_undo_possibilities_in_git/img/rebase_reset.png diff --git a/doc/articles/numerous_undo_possibilities_in_git/img/revert.png b/doc/articles/numerous_undo_possibilities_in_git/img/revert.png Binary files differnew file mode 100644 index 00000000000..13b3a35ca45 --- /dev/null +++ b/doc/articles/numerous_undo_possibilities_in_git/img/revert.png diff --git a/doc/articles/numerous_undo_possibilities_in_git/index.md b/doc/articles/numerous_undo_possibilities_in_git/index.md new file mode 100644 index 00000000000..895bbccec08 --- /dev/null +++ b/doc/articles/numerous_undo_possibilities_in_git/index.md @@ -0,0 +1,497 @@ +# Numerous undo possibilities in Git + +> **Article [Type](../../development/writing_documentation.md#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Crt Mori](https://gitlab.com/Letme) || +> **Publication date:** 2017-08-17 + +## Introduction + +In this tutorial, we will show you different ways of undoing your work in Git, for which +we will assume you have a basic working knowledge of. Check GitLab's +[Git documentation](../../topics/git/index.md#git-documentation) for reference. +Also, we will only provide some general info of the commands, which is enough +to get you started for the easy cases/examples, but for anything more advanced please refer to the [Git book](https://git-scm.com/book/en/v2). + +We will explain a few different techniques to undo your changes based on the stage +of the change in your current development. Also, keep in mind that [nothing in +Git is really deleted.][git-autoclean-ref] +This means that until Git automatically cleans detached commits (which cannot be +accessed by branch or tag) it will be possible to view them with `git reflog` command +and access them with direct commit-id. Read more about _[redoing the undo](#redoing-the-undo)_ on the section below. + +This guide is organized depending on the [stage of development][git-basics] +where you want to undo your changes from and if they were shared with other developers +or not. Because Git is tracking changes a created or edited file is in the unstaged state +(if created it is untracked by Git). After you add it to a repository (`git add`) you put +a file into the **staged** state, which is then committed (`git commit`) to your +local repository. After that, file can be shared with other developers (`git push`). +Here's what we'll cover in this tutorial: + + - [Undo local changes](#undo-local-changes) which were not pushed to remote repository + + - Before you commit, in both unstaged and staged state + - After you committed + + - Undo changes after they are pushed to remote repository + + - [Without history modification](#undo-remote-changes-without-changing-history) (preferred way) + - [With history modification](#undo-remote-changes-with-modifying-history) (requires + coordination with team and force pushes). + + - [Usecases when modifying history is generally acceptable](#where-modifying-history-is-generally-acceptable) + - [How to modify history](#how-modifying-history-is-done) + - [How to remove sensitive information from repository](#deleting-sensitive-information-from-commits) + + +### Branching strategy + +[Git][git-official] is a de-centralized version control system, which means that beside regular +versioning of the whole repository, it has possibilities to exchange changes +with other repositories. To avoid chaos with +[multiple sources of truth][git-distributed], various +development workflows have to be followed, and it depends on your internal +workflow how certain changes or commits can be undone or changed. +[GitLab Flow][gitlab-flow] provides a good +balance between developers clashing with each other while +developing the same feature and cooperating seamlessly, but it does not enable +joined development of the same feature by multiple developers by default. +When multiple developers develop the same feature on the same branch, clashing +with every synchronization is unavoidable, but a proper or chosen Git Workflow will +prevent that anything is lost or out of sync when feature is complete. You can also +read through this blog post on [Git Tips & Tricks][gitlab-git-tips-n-tricks] +to learn how to easily **do** things in Git. + + +## Undo local changes + +Until you push your changes to any remote repository, they will only affect you. +That broadens your options on how to handle undoing them. Still, local changes +can be on various stages and each stage has a different approach on how to tackle them. + + +### Unstaged local changes (before you commit) + +When a change is made, but it is not added to the staged tree, Git itself +proposes a solution to discard changes to certain file. + +Suppose you edited a file to change the content using your favorite editor: + +```shell +vim <file> +``` + +Since you did not `git add <file>` to staging, it should be under unstaged files (or +untracked if file was created). You can confirm that with: + +```shell +$ git status +On branch master +Your branch is up-to-date with 'origin/master'. +Changes not staged for commit: + (use "git add <file>..." to update what will be committed) + (use "git checkout -- <file>..." to discard changes in working directory) + + modified: <file> +no changes added to commit (use "git add" and/or "git commit -a") +``` + +At this point there are 3 options to undo the local changes you have: + + - Discard all local changes, but save them for possible re-use [later](#quickly-save-local-changes) + + ```shell + git stash + ``` + + - Discarding local changes (permanently) to a file + + ```shell + git checkout -- <file> + ``` + + - Discard all local changes to all files permanently + + ```shell + git reset --hard + ``` + + +Before executing `git reset --hard`, keep in mind that there is also a way to +just temporary store the changes without committing them using `git stash`. +This command resets the changes to all files, but it also saves them in case +you would like to apply them at some later time. You can read more about it in +[section below](#quickly-save-local-changes). + +### Quickly save local changes + +You are working on a feature when a boss drops by with an urgent task. Since your +feature is not complete, but you need to swap to another branch, you can use +`git stash` to save what you had done, swap to another branch, commit, push, +test, then get back to previous feature branch, do `git stash pop` and continue +where you left. + +The example above shows that discarding all changes is not always a preferred option, +but Git provides a way to save them for later, while resetting the repository to state without +them. This is achieved by Git stashing command `git stash`, which in fact saves your +current work and runs `git reset --hard`, but it also has various +additional options like: + + - `git stash save`, which enables including temporary commit message, which will help you identify changes, among with other options + - `git stash list`, which lists all previously stashed commits (yes, there can be more) that were not `pop`ed + - `git stash pop`, which redoes previously stashed changes and removes them from stashed list + - `git stash apply`, which redoes previously stashed changes, but keeps them on stashed list + +### Staged local changes (before you commit) + +Let's say you have added some files to staging, but you want to remove them from the +current commit, yet you want to retain those changes - just move them outside +of the staging tree. You also have an option to discard all changes with +`git reset --hard` or think about `git stash` [as described earlier.](#quickly-save-local-changes) + +Lets start the example by editing a file, with your favorite editor, to change the +content and add it to staging + +``` +vim <file> +git add <file> +``` + +The file is now added to staging as confirmed by `git status` command: + +```shell +$ git status +On branch master +Your branch is up-to-date with 'origin/master'. +Changes to be committed: + (use "git reset HEAD <file>..." to unstage) + + new file: <file> +``` + +Now you have 4 options to undo your changes: + + - Unstage the file to current commit (HEAD) + + ```shell + git reset HEAD <file> + ``` + + - Unstage everything - retain changes + + ```shell + git reset + ``` + + - Discard all local changes, but save them for [later](#quickly-save-local-changes) + + ```shell + git stash + ``` + + - Discard everything permanently + + ```shell + git reset --hard + ``` + +## Committed local changes + +Once you commit, your changes are recorded by the version control system. +Because you haven't pushed to your remote repository yet, your changes are +still not public (or shared with other developers). At this point, undoing +things is a lot easier, we have quite some workaround options. Once you push +your code, you'll have less options to troubleshoot your work. + +### Without modifying history + +Through the development process some of the previously committed changes do not +fit anymore in the end solution, or are source of the bugs. Once you find the +commit which triggered bug, or once you have a faulty commit, you can simply +revert it with `git revert commit-id`. This command inverts (swaps) the additions and +deletions in that commit, so that it does not modify history. Retaining history +can be helpful in future to notice that some changes have been tried +unsuccessfully in the past. + +In our example we will assume there are commits `A`,`B`,`C`,`D`,`E` committed in this order: `A-B-C-D-E`, +and `B` is the commit you want to undo. There are many different ways to identify commit +`B` as bad, one of them is to pass a range to `git bisect` command. The provided range includes +last known good commit (we assume `A`) and first known bad commit (where bug was detected - we will assume `E`). + +```shell +git bisect A..E +``` + +Bisect will provide us with commit-id of the middle commit to test, and then guide us +through simple bisection process. You can read more about it [in official Git Tools][git-debug] +In our example we will end up with commit `B`, that introduced bug/error. We have +4 options on how to remove it (or part of it) from our repository. + +- Undo (swap additions and deletions) changes introduced by commit `B`. + + ```shell + git revert commit-B-id + ``` + +- Undo changes on a single file or directory from commit `B`, but retain them in the staged state + + ```shell + git checkout commit-B-id <file> + ``` + +- Undo changes on a single file or directory from commit `B`, but retain them in the unstaged state + + ```shell + git reset commit-B-id <file> + ``` + + - There is one command we also must not forget: **creating a new branch** + from the point where changes are not applicable or where the development has hit a + dead end. For example you have done commits `A-B-C-D` on your feature-branch + and then you figure `C` and `D` are wrong. At this point you either reset to `B` + and do commit `F` (which will cause problems with pushing and if forced pushed also with other developers) + since branch now looks `A-B-F`, which clashes with what other developers have locally (you will + [change history](#with-history-modification)), or you simply checkout commit `B` create + a new branch and do commit `F`. In the last case, everyone else can still do their work while you + have your new way to get it right and merge it back in later. Alternatively, with GitLab, + you can [cherry-pick](../../user/project/merge_requests/cherry_pick_changes.md#cherry-picking-a-commit) + that commit into a new merge request. + +  + + ```shell + git checkout commit-B-id + git checkout -b new-path-of-feature + # Create <commit F> + git commit -a + ``` + +### With history modification + +There is one command for history modification and that is `git rebase`. Command +provides interactive mode (`-i` flag) which enables you to: + + - **reword** commit messages (there is also `git commit --amend` for editing + last commit message) + - **edit** the commit content (changes introduced by commit) and message + - **squash** multiple commits into a single one, and have a custom or aggregated + commit message + - **drop** commits - simply delete them + - and few more options + +Let us check few examples. Again there are commits `A-B-C-D` where you want to +delete commit `B`. + +- Rebase the range from current commit D to A: + + ```shell + git rebase -i A + ``` + +- Command opens your favorite editor where you write `drop` in front of commit + `B`, but you leave default `pick` with all other commits. Save and exit the + editor to perform a rebase. Remember: if you want to cancel delete whole + file content before saving and exiting the editor + +In case you want to modify something introduced in commit `B`. + +- Rebase the range from current commit D to A: + + ```shell + git rebase -i A + ``` + +- Command opens your favorite text editor where you write `edit` in front of commit + `B`, but leave default `pick` with all other commits. Save and exit the editor to + perform a rebase + +- Now do your edits and commit changes: + + ```shell + git commit -a + ``` + +You can find some more examples in [below section where we explain how to modify +history](#how-modifying-history-is-done) + + +### Redoing the Undo + +Sometimes you realize that the changes you undid were useful and you want them +back. Well because of first paragraph you are in luck. Command `git reflog` +enables you to *recall* detached local commits by referencing or applying them +via commit-id. Although, do not expect to see really old commits in reflog, because +Git regularly [cleans the commits which are *unreachable* by branches or tags][git-autoclean-ref]. + +To view repository history and to track older commits you can use below command: + +```shell +$ git reflog show + +# Example output: +b673187 HEAD@{4}: merge 6e43d5987921bde189640cc1e37661f7f75c9c0b: Merge made by the 'recursive' strategy. +eb37e74 HEAD@{5}: rebase -i (finish): returning to refs/heads/master +eb37e74 HEAD@{6}: rebase -i (pick): Commit C +97436c6 HEAD@{7}: rebase -i (start): checkout 97436c6eec6396c63856c19b6a96372705b08b1b +... +88f1867 HEAD@{12}: commit: Commit D +97436c6 HEAD@{13}: checkout: moving from 97436c6eec6396c63856c19b6a96372705b08b1b to test +97436c6 HEAD@{14}: checkout: moving from master to 97436c6 +05cc326 HEAD@{15}: commit: Commit C +6e43d59 HEAD@{16}: commit: Commit B +``` + +Output of command shows repository history. In first column there is commit-id, +in following column, number next to `HEAD` indicates how many commits ago something +was made, after that indicator of action that was made (commit, rebase, merge, ...) +and then on end description of that action. + +## Undo remote changes without changing history + +This topic is roughly same as modifying committed local changes without modifying +history. **It should be the preferred way of undoing changes on any remote repository +or public branch.** Keep in mind that branching is the best solution when you want +to retain the history of faulty development, yet start anew from certain point. Branching +enables you to include the existing changes in new development (by merging) and +it also provides a clear timeline and development structure. + + + +If you want to revert changes introduced in certain `commit-id` you can simply +revert that `commit-id` (swap additions and deletions) in newly created commit: +You can do this with + +```shell +git revert commit-id +``` + +or creating a new branch: + +```shell +git checkout commit-id +git checkout -b new-path-of-feature +``` + +## Undo remote changes with modifying history + +This is useful when you want to *hide* certain things - like secret keys, +passwords, SSH keys, etc. It is and should not be used to hide mistakes, as +it will make it harder to debug in case there are some other bugs. The main +reason for this is that you loose the real development progress. **Also keep in +mind that, even with modified history, commits are just detached and can still be +accessed through commit-id** - at least until all repositories perform +the cleanup of detached commits (happens automatically). + + + +### Where modifying history is generally acceptable + +Modified history breaks the development chain of other developers, as changed +history does not have matching commits'ids. For that reason it should not +be used on any public branch or on branch that *might* be used by other +developers. When contributing to big open source repositories (e.g. [GitLab CE][gitlab-ce]), +it is acceptable to *squash* commits into a single one, to present +a nicer history of your contribution. +Keep in mind that this also removes the comments attached to certain commits +in merge requests, so if you need to retain traceability in GitLab, then +modifying history is not acceptable. +A feature-branch of a merge request is a public branch and might be used by +other developers, but project process and rules might allow or require +you to use `git rebase` (command that changes history) to reduce number of +displayed commits on target branch after reviews are done (for example +GitLab). There is a `git merge --squash` command which does exactly that +(squashes commits on feature-branch to a single commit on target branch +at merge). + +>**Note:** +Never modify the commit history of `master` or shared branch + +### How modifying history is done + +After you know what you want to modify (how far in history or how which range of +old commits), use `git rebase -i commit-id`. This command will then display all the commits from +current version to chosen commit-id and allow modification, squashing, deletion +of that commits. + +```shell +$ git rebase -i commit1-id..commit3-id +pick <commit1-id> <commit1-commit-message> +pick <commit2-id> <commit2-commit-message> +pick <commit3-id> <commit3-commit-message> + +# Rebase commit1-id..commit3-id onto <commit4-id> (3 command(s)) +# +# Commands: +# p, pick = use commit +# r, reword = use commit, but edit the commit message +# e, edit = use commit, but stop for amending +# s, squash = use commit, but meld into previous commit +# f, fixup = like "squash", but discard this commit's log message +# x, exec = run command (the rest of the line) using shell +# d, drop = remove commit +# +# These lines can be re-ordered; they are executed from top to bottom. +# +# If you remove a line here THAT COMMIT WILL BE LOST. +# +# However, if you remove everything, the rebase will be aborted. +# +# Note that empty commits are commented out +``` + +>**Note:** +It is important to notice that comment from the output clearly states that, if +you decide to abort, then do not just close your editor (as that will in-fact +modify history), but remove all uncommented lines and save. + +That is one of the reasons why `git rebase` should be used carefully on +shared and remote branches. But don't worry, there will be nothing broken until +you push back to the remote repository (so you can freely explore the +different outcomes locally). + +```shell +# Modify history from commit-id to HEAD (current commit) +git rebase -i commit-id +``` + +### Deleting sensitive information from commits + +Git also enables you to delete sensitive information from your past commits and +it does modify history in the progress. That is why we have included it in this +section and not as a standalone topic. To do so, you should run the +`git filter-branch`, which enables you to rewrite history with +[certain filters][git-filters-manual]. +This command uses rebase to modify history and if you want to remove certain +file from history altogether use: + +```shell +git filter-branch --tree-filter 'rm filename' HEAD +``` + +Since `git filter-branch` command might be slow on big repositories, there are +tools that can use some of Git specifics to enable faster execution of common +tasks (which is exactly what removing sensitive information file is about). +An alternative is [BFG Repo-cleaner][bfg-repo-cleaner]. Keep in mind that these +tools are faster because they do not provide a same fully feature set as `git filter-branch` +does, but focus on specific usecases. + +## Conclusion + +There are various options of undoing your work with any version control system, but +because of de-centralized nature of Git, these options are multiplied (or limited) +depending on the stage of your process. Git also enables rewriting history, but that +should be avoided as it might cause problems when multiple developers are +contributing to the same codebase. + +<!-- Identifiers, in alphabetical order --> + +[bfg-repo-cleaner]: https://rtyley.github.io/bfg-repo-cleaner/ +[git-autoclean-ref]: https://git-scm.com/book/en/v2/Git-Internals-Maintenance-and-Data-Recovery +[git-basics]: https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository +[git-debug]: https://git-scm.com/book/en/v2/Git-Tools-Debugging-with-Git +[git-distributed]: https://git-scm.com/about/distributed +[git-filters-manual]: https://git-scm.com/docs/git-filter-branch#_options +[git-official]: https://git-scm.com/ +[gitlab-ce]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#contribution-acceptance-criteria +[gitlab-flow]: https://about.gitlab.com/2014/09/29/gitlab-flow/ +[gitlab-git-tips-n-tricks]: https://about.gitlab.com/2016/12/08/git-tips-and-tricks/ diff --git a/doc/articles/openshift_and_gitlab/index.md b/doc/articles/openshift_and_gitlab/index.md index 7f76e577efa..c0bbcfe2a8a 100644 --- a/doc/articles/openshift_and_gitlab/index.md +++ b/doc/articles/openshift_and_gitlab/index.md @@ -3,7 +3,7 @@ > **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** tutorial || > **Level:** intermediary || > **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) || -> **Publication date:** 2016/06/28 +> **Publication date:** 2016-06-28 ## Introduction diff --git a/doc/ci/README.md b/doc/ci/README.md index 10ea9467942..5cfd82de381 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -44,6 +44,10 @@ digging into specific reference guides. - [User permissions](../user/permissions.md#gitlab-ci) - [Jobs permissions](../user/permissions.md#jobs-permissions) +## Auto DevOps + +- [Auto DevOps](../topics/autodevops/index.md) + ## GitLab CI + Docker Leverage the power of Docker to run your CI pipelines. @@ -111,7 +115,9 @@ Here is an collection of tutorials and guides on setting up your CI pipeline. - [Phoenix](examples/test-phoenix-application.md) - [Run PHP Composer & NPM scripts then deploy them to a staging server](examples/deployment/composer-npm-deploy.md) - [Analyze code quality with the Code Climate CLI](examples/code_climate.md) -- **Blog posts** +- **Articles** + - [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](../articles/laravel_with_gitlab_and_envoy/index.md) + - [How to deploy Maven projects to Artifactory with GitLab CI/CD](../articles/artifactory_and_gitlab/index.md) - [Automated Debian packaging](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) - [Spring boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) - [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) diff --git a/doc/ci/api/README.md b/doc/ci/api/README.md deleted file mode 100644 index 98f37935427..00000000000 --- a/doc/ci/api/README.md +++ /dev/null @@ -1 +0,0 @@ -This document was moved to a [new location](../../api/ci/README.md). diff --git a/doc/ci/api/builds.md b/doc/ci/api/builds.md deleted file mode 100644 index 0563a367609..00000000000 --- a/doc/ci/api/builds.md +++ /dev/null @@ -1 +0,0 @@ -This document was moved to a [new location](../../api/ci/builds.md). diff --git a/doc/ci/api/runners.md b/doc/ci/api/runners.md deleted file mode 100644 index 1027363851c..00000000000 --- a/doc/ci/api/runners.md +++ /dev/null @@ -1 +0,0 @@ -This document was moved to a [new location](../../api/ci/runners.md). diff --git a/doc/ci/autodeploy/img/auto_deploy_btn.png b/doc/ci/autodeploy/img/auto_deploy_btn.png Binary files differnew file mode 100644 index 00000000000..25915ed1c9d --- /dev/null +++ b/doc/ci/autodeploy/img/auto_deploy_btn.png diff --git a/doc/ci/autodeploy/img/auto_deploy_dropdown.png b/doc/ci/autodeploy/img/auto_deploy_dropdown.png Binary files differindex b93b0a08fea..5815937a4af 100644 --- a/doc/ci/autodeploy/img/auto_deploy_dropdown.png +++ b/doc/ci/autodeploy/img/auto_deploy_dropdown.png diff --git a/doc/ci/autodeploy/img/auto_monitoring.png b/doc/ci/autodeploy/img/auto_monitoring.png Binary files differnew file mode 100644 index 00000000000..5661b50841b --- /dev/null +++ b/doc/ci/autodeploy/img/auto_monitoring.png diff --git a/doc/ci/autodeploy/img/guide_connect_cluster.png b/doc/ci/autodeploy/img/guide_connect_cluster.png Binary files differnew file mode 100644 index 00000000000..b856b81a1d0 --- /dev/null +++ b/doc/ci/autodeploy/img/guide_connect_cluster.png diff --git a/doc/ci/autodeploy/img/guide_integration.png b/doc/ci/autodeploy/img/guide_integration.png Binary files differnew file mode 100644 index 00000000000..723b2619ea2 --- /dev/null +++ b/doc/ci/autodeploy/img/guide_integration.png diff --git a/doc/ci/autodeploy/img/guide_secret.png b/doc/ci/autodeploy/img/guide_secret.png Binary files differnew file mode 100644 index 00000000000..01f5aa49908 --- /dev/null +++ b/doc/ci/autodeploy/img/guide_secret.png diff --git a/doc/ci/autodeploy/index.md b/doc/ci/autodeploy/index.md index 9fa2b2c4969..474cb28b9e4 100644 --- a/doc/ci/autodeploy/index.md +++ b/doc/ci/autodeploy/index.md @@ -1,8 +1,16 @@ -# Auto deploy +# Auto Deploy > [Introduced][mr-8135] in GitLab 8.15. -> Auto deploy is an experimental feature and is not recommended for Production use at this time. -> As of GitLab 9.1, access to the container registry is only available while the Pipeline is running. Restarting a pod, scaling a service, or other actions which require on-going access will fail. On-going secure access is planned for a subsequent release. +> Auto deploy is an experimental feature and is **not recommended for Production use** at this time. + +> As of GitLab 9.1, access to the container registry is only available while the +Pipeline is running. Restarting a pod, scaling a service, or other actions which +require on-going access **will fail**. On-going secure access is planned for a +subsequent release. + +> As of GitLab 10.0, Auto Deploy templates are **deprecated** and the +functionality has been included in [Auto +DevOps](../../topics/autodevops/index.md). Auto deploy is an easy way to configure GitLab CI for the deployment of your application. GitLab Community maintains a list of `.gitlab-ci.yml` @@ -11,9 +19,23 @@ powering them. These scripts are responsible for packaging your application, setting up the infrastructure and spinning up necessary services (for example a database). -You can use [project services][project-services] to store credentials to -your infrastructure provider and they will be available during the -deployment. +## How it works + +The Autodeploy templates are based on the [kubernetes-deploy][kube-deploy] +project which is used to simplify the deployment process to Kubernetes by +providing intelligent `build`, `deploy`, and `destroy` commands which you can +use in your `.gitlab-ci.yml` as is. It uses [Herokuish](https://github.com/gliderlabs/herokuish), +which uses [Heroku buildpacks](https://devcenter.heroku.com/articles/buildpacks) +to do some of the work, plus some of GitLab's own tools to package it all up. For +your convenience, a [Docker image][kube-image] is also provided. + +You can use the [Kubernetes project service](../../user/project/integrations/kubernetes.md) +to store credentials to your infrastructure provider and they will be available +during the deployment. + +## Quick start + +We made a [simple guide](quick_start_guide.md) to using Auto Deploy with GitLab.com. ## Supported templates @@ -22,20 +44,27 @@ The list of supported auto deploy templates is available in the ## Configuration +>**Note:** +In order to understand why the following steps are required, read the +[how it works](#how-it-works) section. + +To configure Autodeploy, you will need to: + 1. Enable a deployment [project service][project-services] to store your -credentials. For example, if you want to deploy to OpenShift you have to -enable [Kubernetes service][kubernetes-service]. -1. Configure GitLab Runner to use Docker or Kubernetes executor with -[privileged mode enabled][docker-in-docker]. + credentials. For example, if you want to deploy to OpenShift you have to + enable [Kubernetes service][kubernetes-service]. +1. Configure GitLab Runner to use the + [Docker or Kubernetes executor](https://docs.gitlab.com/runner/executors/) with + [privileged mode enabled][docker-in-docker]. 1. Navigate to the "Project" tab and click "Set up auto deploy" button.  1. Select a template.  1. Commit your changes and create a merge request. 1. Test your deployment configuration using a [Review App][review-app] that was -created automatically for you. + created automatically for you. -## Private Project Support +## Private project support > Experimental support [introduced][mr-2] in GitLab 9.1. @@ -43,7 +72,7 @@ When a project has been marked as private, GitLab's [Container Registry][contain After the pipeline completes, Kubernetes will no longer be able to access the container registry. Restarting a pod, scaling a service, or other actions which require on-going access to the registry will fail. On-going secure access is planned for a subsequent release. -## PostgreSQL Database Support +## PostgreSQL database support > Experimental support [introduced][mr-8] in GitLab 9.1. @@ -51,13 +80,38 @@ In order to support applications that require a database, [PostgreSQL][postgresq PostgreSQL provisioning can be disabled by setting the variable `DISABLE_POSTGRES` to `"yes"`. -### PostgreSQL Variables +The following PostgreSQL variables are supported: 1. `DISABLE_POSTGRES: "yes"`: disable automatic deployment of PostgreSQL 1. `POSTGRES_USER: "my-user"`: use custom username for PostgreSQL 1. `POSTGRES_PASSWORD: "password"`: use custom password for PostgreSQL 1. `POSTGRES_DB: "my database"`: use custom database name for PostgreSQL +## Auto Monitoring + +> Introduced in [GitLab 9.5](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13438). + +Apps auto-deployed using one the [Kubernetes templates](#supported-templates) can also be automatically monitored for: + +* Response Metrics: latency, throughput, error rate +* System Metrics: CPU utilization, memory utilization + +Metrics are gathered from [nginx-ingress](../../user/project/integrations/prometheus_library/nginx_ingress.md) and [Kubernetes](../../user/project/integrations/prometheus_library/kubernetes.md). + +To view the metrics, open the [Monitoring dashboard for a deployed environment](../environments.md#monitoring-environments). + + + +### Configuring Auto Monitoring + +If GitLab has been deployed using the [omnibus-gitlab](../../install/kubernetes/gitlab_omnibus.md) Helm chart, no configuration is required. + +If you have installed GitLab using a different method: + +1. [Deploy Prometheus](../../user/project/integrations/prometheus.md#configuring-your-own-prometheus-server-within-kubernetes) into your Kubernetes cluster +1. If you would like response metrics, ensure you are running at least version 0.9.0 of NGINX Ingress and [enable Prometheus metrics](https://github.com/kubernetes/ingress/blob/master/examples/customization/custom-vts-metrics/nginx/nginx-vts-metrics-conf.yaml). +1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) the NGINX Ingress deployment to be scraped by Prometheus using `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. + [mr-8135]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8135 [mr-2]: https://gitlab.com/gitlab-examples/kubernetes-deploy/merge_requests/2 [mr-8]: https://gitlab.com/gitlab-examples/kubernetes-deploy/merge_requests/8 @@ -67,5 +121,7 @@ PostgreSQL provisioning can be disabled by setting the variable `DISABLE_POSTGRE [kubernetes-service]: ../../user/project/integrations/kubernetes.md [docker-in-docker]: ../docker/using_docker_build.md#use-docker-in-docker-executor [review-app]: ../review_apps/index.md +[kube-image]: https://gitlab.com/gitlab-examples/kubernetes-deploy/container_registry "Kubernetes deploy Container Registry" +[kube-deploy]: https://gitlab.com/gitlab-examples/kubernetes-deploy "Kubernetes deploy example project" [container-registry]: https://docs.gitlab.com/ce/user/project/container_registry.html [postgresql]: https://www.postgresql.org/ diff --git a/doc/ci/autodeploy/quick_start_guide.md b/doc/ci/autodeploy/quick_start_guide.md new file mode 100644 index 00000000000..f76c2a2cf31 --- /dev/null +++ b/doc/ci/autodeploy/quick_start_guide.md @@ -0,0 +1,95 @@ +# Auto Deploy: quick start guide + +This is a step-by-step guide to deploying a project hosted on GitLab.com to Google Cloud, using Auto Deploy. + +We made a minimal [Ruby application](https://gitlab.com/gitlab-examples/minimal-ruby-app) to use as an example for this guide. It contains two files: + +* `server.rb` - our application. It will start an HTTP server on port 5000 and render “Hello, world!” +* `Dockerfile` - to build our app into a container image. It will use a ruby base image and run `server.rb` + +## Fork sample project on GitLab.com + +Let’s start by forking our sample application. Go to [the project page](https://gitlab.com/gitlab-examples/minimal-ruby-app) and press the `Fork` button. Soon you should have a project under your namespace with the necessary files. + +## Setup your own cluster on Google Container Engine + +If you do not already have a Google Cloud account, create one at https://console.cloud.google.com. + +Visit the [`Container Engine`](https://console.cloud.google.com/kubernetes/list) tab and create a new cluster. You can change the name and leave the rest of the default settings. Once you have your cluster running, you need to connect to the cluster by following the Google interface. + +## Connect to Kubernetes cluster + +You need to have the Google Cloud SDK installed. e.g. +On OSX, install [homebrew](https://brew.sh): + +1. Install Brew Caskroom: `brew install caskroom/cask/brew-cask` +2. Install Google Cloud SDK: `brew cask install google-cloud-sdk` +3. Add `kubectl`: `gcloud components install kubectl` +4. Log in: `gcloud auth login` + +Now go back to the Google interface, find your cluster, and follow the instructions under `Connect to the cluster` and open the Kubernetes Dashboard. It will look something like `gcloud container clusters get-credentials ruby-autodeploy \ --zone europe-west2-c --project api-project-XXXXXXX` and then `kubectl proxy`. + + + +## Copy credentials to GitLab.com project + +Once you have the Kubernetes Dashboard interface running, you should visit `Secrets` under the `Config` section. There you should find the settings we need for GitLab integration: ca.crt and token. + + + +You need to copy-paste the ca.crt and token into your project on GitLab.com in the Kubernetes integration page under project `Settings` > `Integrations` > `Project services` > `Kubernetes`. Don't actually copy the namespace though. Each project should have a unique namespace, and by leaving it blank, GitLab will create one for you. + + + +For API URL, you should use the `Endpoint` IP from your cluster page on Google Cloud Platform. + +## Expose the application to the internet + +In order to be able to visit your application, you need to install an NGINX ingress controller and point your domain name to its external IP address. + +### Set up Ingress controller + +You’ll need to make sure you have an ingress controller. If you don’t have one, do: + +```sh +brew install kubernetes-helm +helm init +helm install --name ruby-app stable/nginx-ingress +``` + +This should create several services including `ruby-app-nginx-ingress-controller`. You can list your services by running `kubectl get svc` to confirm that. + +### Point DNS at Cluster IP + +Find out the external IP address of the `ruby-app-nginx-ingress-controller` by running: + +```sh +kubectl get svc ruby-app-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' +``` + +Use this IP address to configure your DNS. This part heavily depends on your preferences and domain provider. But in case you are not sure, just create an A record with a wildcard host like `*.<your-domain>` pointing to the external IP address you found above. + +Use `nslookup minimal-ruby-app-staging.<yourdomain>` to confirm that domain is assigned to the cluster IP. + +## Setup Auto Deploy + +Visit the home page of your GitLab.com project and press "Set up Auto Deploy" button. + + + +You will be redirected to the "New file" page where you can apply one of the Auto Deploy templates. Select "Kubernetes" to apply the template, then in the file, replace `domain.example.com` with your domain name and make any other adjustments you need. + + + +Change the target branch to `master`, and submit your changes. This should create +a new pipeline with several jobs. If you made only the domain name change, the +pipeline will have three jobs: `build`, `staging`, and `production`. + +The `build` job will create a Docker image with your new change and push it to +the GitLab Container Registry. The `staging` job will deploy this image on your +cluster. Once the deploy job succeeds you should be able to see your application by +visiting the Kubernetes dashboard. Select the namespace of your project, which +will look like `ruby-autodeploy-23`, but with a unique ID for your project, and +your app will be listed as "staging" under the "Deployment" tab. + +Once its ready - just visit http://minimal-ruby-app-staging.yourdomain.com to see “Hello, world!” diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index f7c2a0ef0ca..f28c9791bee 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -134,7 +134,7 @@ In order to do that, follow the steps: # When using dind, it's wise to use the overlayfs driver for # improved performance. variables: - DOCKER_DRIVER: overlay + DOCKER_DRIVER: overlay2 services: - docker:dind @@ -248,7 +248,7 @@ aware of the following implications: By default, when using `docker:dind`, Docker uses the `vfs` storage driver which copies the filesystem on every run. This is a very disk-intensive operation -which can be avoided if a different driver is used, for example `overlay`. +which can be avoided if a different driver is used, for example `overlay2`. 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: @@ -275,8 +275,12 @@ which can be avoided if a different driver is used, for example `overlay`. ``` variables: - DOCKER_DRIVER: overlay + DOCKER_DRIVER: overlay2 ``` + +> **Note:** +- For more information about using OverlayFS with Docker, you can read + [Use the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/). ## Using the GitLab Container Registry diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index d3433594eb7..fa823ea4721 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -96,7 +96,7 @@ services: - tutum/wordpress:latest ``` -If you don't [specify a service alias](#available-settings-for-services-entry), +If you don't [specify a service alias](#available-settings-for-services), when the job is run, `tutum/wordpress` will be started and you will have access to it from your build container under two hostnames to choose from: @@ -117,7 +117,7 @@ following these rules: created (requires GitLab Runner v1.1.0 or higher) To override the default behavior, you can -[specify a service alias](#available-settings-for-services-entry). +[specify a service alias](#available-settings-for-services). ## Define `image` and `services` from `.gitlab-ci.yml` @@ -183,8 +183,7 @@ test: ## Extended Docker configuration options -> **Note:** -This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. +> Introduced in GitLab and GitLab Runner 9.4. When configuring the `image` or `services` entries, you can use a string or a map as options: @@ -221,28 +220,29 @@ For example, the following two definitions are equal: ### Available settings for `image` -> **Note:** -This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. +> Introduced in GitLab and GitLab Runner 9.4. -| Setting | Required | Description | -|------------|----------|-------------| -| `name` | yes, when used with any other option | Full name of the image that should be used. It should contain the Registry part if needed. | -| `entrypoint` | no | Command or script that should be executed as the container's entrypoint. It will be translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`][entrypoint] directive, where each shell token is a separate string in the array. | +| Setting | Required | GitLab version | Description | +|------------|----------|----------------| ----------- | +| `name` | yes, when used with any other option | 9.4 |Full name of the image that should be used. It should contain the Registry part if needed. | +| `entrypoint` | no | 9.4 |Command or script that should be executed as the container's entrypoint. It will be translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`][entrypoint] directive, where each shell token is a separate string in the array. | ### Available settings for `services` -> **Note:** -This feature requires GitLab 9.4 and GitLab Runner 9.4 or higher. +> Introduced in GitLab and GitLab Runner 9.4. -| Setting | Required | Description | -|------------|----------|-------------| -| `name` | yes, when used with any other option | Full name of the image that should be used. It should contain the Registry part if needed. | -| `entrypoint` | no | Command or script that should be executed as the container's entrypoint. It will be translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`][entrypoint] directive, where each shell token is a separate string in the array. | -| `command` | no | Command or script that should be used as the container's command. It will be translated to arguments passed to Docker after the image's name. The syntax is similar to [`Dockerfile`'s `CMD`][cmd] directive, where each shell token is a separate string in the array. | -| `alias` | no | Additional alias that can be used to access the service from the job's container. Read [Accessing the services](#accessing-the-services) for more information. | +| Setting | Required | GitLab version | Description | +|------------|----------|----------------| ----------- | +| `name` | yes, when used with any other option | 9.4 | Full name of the image that should be used. It should contain the Registry part if needed. | +| `entrypoint` | no | 9.4 |Command or script that should be executed as the container's entrypoint. It will be translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`][entrypoint] directive, where each shell token is a separate string in the array. | +| `command` | no | 9.4 |Command or script that should be used as the container's command. It will be translated to arguments passed to Docker after the image's name. The syntax is similar to [`Dockerfile`'s `CMD`][cmd] directive, where each shell token is a separate string in the array. | +| `alias` | no | 9.4 |Additional alias that can be used to access the service from the job's container. Read [Accessing the services](#accessing-the-services) for more information. | ### Starting multiple services from the same image +> Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended +configuration options](#extended-docker-configuration-options). + Before the new extended Docker configuration options, the following configuration would not work properly: @@ -274,6 +274,9 @@ in `.gitlab-ci.yml` file. ### Setting a command for the service +> Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended +configuration options](#extended-docker-configuration-options). + Let's assume you have a `super/sql:latest` image with some SQL database inside it and you would like to use it as a service for your job. Let's also assume that this image doesn't start the database process while starting @@ -313,6 +316,9 @@ As you can see, the syntax of `command` is similar to [Dockerfile's `CMD`][cmd]. ### Overriding the entrypoint of an image +> Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended +configuration options](#extended-docker-configuration-options). + Let's assume you have a `super/sql:experimental` image with some SQL database inside it and you would like to use it as a base image for your job because you want to execute some tests with this database binary. Let's also assume that @@ -382,15 +388,40 @@ that runner. As an example, let's assume that you want to use the `registry.example.com/private/image:latest` image which is private and requires you to login into a private container registry. + +Let's also assume that these are the login credentials: + +| Key | Value | +|----------|----------------------| +| registry | registry.example.com | +| username | my_username | +| password | my_password | + To configure access for `registry.example.com`, follow these steps: -1. Do a `docker login` on your computer: +1. Find what the value of `DOCKER_AUTH_CONFIG` should be. There are two ways to + accomplish this: + - **First way -** Do a `docker login` on your local machine: - ```bash - docker login registry.example.com --username my_username --password my_password - ``` + ```bash + docker login registry.example.com --username my_username --password my_password + ``` + + Then copy the content of `~/.docker/config.json`. + - **Second way -** In some setups, it's possible that Docker client will use + the available system keystore to store the result of `docker login`. In + that case, it's impossible to read `~/.docker/config.json`, so you will + need to prepare the required base64-encoded version of + `${username}:${password}` manually. Open a terminal and execute the + following command: + + ```bash + echo -n "my_username:my_password" | base64 + + # Example output to copy + bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= + ``` -1. Copy the content of `~/.docker/config.json` 1. Create a [secret variable] `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: @@ -404,7 +435,8 @@ To configure access for `registry.example.com`, follow these steps: } ``` -1. Do a `docker logout` on your computer if you don't need access to the +1. Optionally,if you followed the first way of finding the `DOCKER_AUTH_CONFIG` + value, do a `docker logout` on your computer if you don't need access to the registry from it: ```bash @@ -412,7 +444,7 @@ To configure access for `registry.example.com`, follow these steps: ``` 1. You can now use any private image from `registry.example.com` defined in - `image` and/or `services` in your [`.gitlab-ci.yml` file][yaml-priv-reg]: + `image` and/or `services` in your `.gitlab-ci.yml` file: ```yaml image: my.registry.tld:5000/namespace/image:tag diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 6a7f694d705..cbf06afa294 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -274,9 +274,7 @@ session - and even a multiplexer like `screen` or `tmux`! >**Note:** Container-based deployments often lack basic tools (like an editor), and may be stopped or restarted at any time. If this happens, you will lose all your -changes! Treat this as a debugging tool, not a comprehensive online IDE. You -can use [Koding](../administration/integration/koding.md) for online -development. +changes! Treat this as a debugging tool, not a comprehensive online IDE. --- @@ -607,10 +605,9 @@ exist, you should see something like: - With GitLab 9.2, all deployments to an environment are shown directly on the monitoring dashboard -If you have enabled Prometheus for collecting metrics, you can monitor the performance behavior of your app -through the environments. +If you have enabled [Prometheus for monitoring system and response metrics](https://docs.gitlab.com/ee/user/project/integrations/prometheus.html), you can monitor the performance behavior of your app running in each environment. -Once configured, GitLab will attempt to retrieve performance metrics for any +Once configured, GitLab will attempt to retrieve [supported performance metrics](https://docs.gitlab.com/ee/user/project/integrations/prometheus_library/metrics.html) for any environment which has had a successful deployment. If monitoring data was successfully retrieved, a Monitoring button will appear on the environment's detail page. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 2458cb959ab..f094546c3bd 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -50,12 +50,15 @@ Apart from those, here is an collection of tutorials and guides on setting up yo - **Articles:** - [Setting up GitLab CI for Android projects](https://about.gitlab.com/2016/11/30/setting-up-gitlab-ci-for-android-projects/) +### Code quality analysis + +- [Analyze code quality with the Code Climate CLI](code_climate.md) + ### Other - [Using `dpl` as deployment tool](deployment/README.md) - [Repositories with examples for various languages](https://gitlab.com/groups/gitlab-examples) - [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml) -- [Analyze code quality with the Code Climate CLI](code_climate.md) - **Articles:** - [Continuous Deployment with GitLab: how to build and deploy a Debian Package with GitLab CI](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md index 5659a8c2a2a..4d0ba8bfef3 100644 --- a/doc/ci/examples/code_climate.md +++ b/doc/ci/examples/code_climate.md @@ -5,10 +5,10 @@ GitLab CI and Docker. First, you need GitLab Runner with [docker-in-docker executor][dind]. -Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called `codeclimate`: +Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called `codequality`: ```yaml -codeclimate: +codequality: image: docker:latest variables: DOCKER_DRIVER: overlay @@ -22,7 +22,7 @@ codeclimate: paths: [codeclimate.json] ``` -This will create a `codeclimate` job in your CI pipeline and will allow you to +This will create a `codequality` job in your CI pipeline and will allow you to download and analyze the report artifact in JSON format. For GitLab [Enterprise Edition Starter][ee] users, this information can be automatically diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 5a2b61fb0cb..ac4a9b0ed27 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -222,6 +222,30 @@ total running time should be: Pipeline status and test coverage report badges are available. You can find their respective link in the [Pipelines settings] page. +## Security on protected branches + +A strict security model is enforced when pipelines are executed on +[protected branches](../user/project/protected_branches.md). + +The following actions are allowed on protected branches only if the user is +[allowed to merge or push](../user/project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings) +on that specific branch: +- run **manual pipelines** (using Web UI or Pipelines API) +- run **scheduled pipelines** +- run pipelines using **triggers** +- trigger **manual actions** on existing pipelines +- **retry/cancel** existing jobs (using Web UI or Pipelines API) + +**Secret variables** marked as **protected** are accessible only to jobs that +run on protected branches, avoiding untrusted users to get unintended access to +sensitive information like deployment credentials and tokens. + +**Runners** marked as **protected** can run jobs only on protected +branches, avoiding untrusted code to be executed on the protected runner and +preserving deployment keys and other credentials from being unintentionally +accessed. In order to ensure that jobs intended to be executed on protected +runners will not use regular runners, they must be tagged accordingly. + [jobs]: #jobs [jobs-yaml]: yaml/README.md#jobs [manual]: yaml/README.md#manual diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 76d746155eb..f5d3b524d6e 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -107,6 +107,43 @@ To lock/unlock a Runner: 1. Check the **Lock to current projects** option 1. Click **Save changes** for the changes to take effect +## Assigning a Runner to another project + +If you are Master on a project where a specific Runner is assigned to, and the +Runner is not [locked only to that project](#locking-a-specific-runner-from-being-enabled-for-other-projects), +you can enable the Runner also on any other project where you have Master permissions. + +To enable/disable a Runner in your project: + +1. Visit your project's **Settings ➔ Pipelines** +1. Find the Runner you wish to enable/disable +1. Click **Enable for this project** or **Disable for this project** + +> **Note**: +Consider that if you don't lock your specific Runner to a specific project, any +user with Master role in you project can assign your runner to another arbitrary +project without requiring your authorization, so use it with caution. + +## Protected Runners + +> +[Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13194) +in GitLab 10.0. + +You can protect Runners from revealing sensitive information. +Whenever a Runner is protected, the Runner picks only jobs created on +[protected branches] or [protected tags], and ignores other jobs. + +To protect/unprotect Runners: + +1. Visit your project's **Settings ➔ Pipelines** +1. Find a Runner you want to protect/unprotect and make sure it's enabled +1. Click the pencil button besides the Runner name +1. Check the **Protected** option +1. Click **Save changes** for the changes to take effect + + + ## How shared Runners pick jobs Shared Runners abide to a process queue we call fair usage. The fair usage @@ -218,3 +255,5 @@ We're always looking for contributions that can mitigate these [install]: http://docs.gitlab.com/runner/install/ [fifo]: https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics) [register]: http://docs.gitlab.com/runner/register/ +[protected branches]: ../../user/project/protected_branches.md +[protected tags]: ../../user/project/protected_tags.md diff --git a/doc/ci/runners/img/protected_runners_check_box.png b/doc/ci/runners/img/protected_runners_check_box.png Binary files differnew file mode 100644 index 00000000000..fb58498c7ce --- /dev/null +++ b/doc/ci/runners/img/protected_runners_check_box.png diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index cf25a8b618f..cdb9858e179 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -42,7 +42,7 @@ It is also good practice to check the server's own public key to make sure you are not being targeted by a man-in-the-middle attack. To do this, add another variable named `SSH_SERVER_HOSTKEYS`. To find out the hostkeys of your server, run the `ssh-keyscan YOUR_SERVER` command from a trusted network (ideally, from the -server itself), and paste its output into the `SSH_SERVER_HOSTKEY` variable. If +server itself), and paste its output into the `SSH_SERVER_HOSTKEYS` variable. If you need to connect to multiple servers, concatenate all the server public keys that you collected into the **Value** of the variable. There must be one key per line. diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 22e7f6879ed..6513b31826a 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -57,6 +57,7 @@ future GitLab releases.** | **CI_RUNNER_TAGS** | 8.10 | 0.5 | The defined runner tags | | **CI_PIPELINE_ID** | 8.10 | 0.5 | The unique id of the current pipeline that GitLab CI uses internally | | **CI_PIPELINE_TRIGGERED** | all | all | The flag to indicate that job was [triggered] | +| **CI_PIPELINE_SOURCE** | 10.0 | all | The source for this pipeline, one of: push, web, trigger, schedule, api, external. Pipelines created before 9.5 will have unknown as source | | **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the job is run | | **CI_PROJECT_ID** | all | all | The unique id of the current project that GitLab CI uses internally | | **CI_PROJECT_NAME** | 8.10 | 0.5 | The project name that is currently being built (actually it is project folder name) | @@ -77,6 +78,8 @@ future GitLab releases.** | **GITLAB_CI** | all | all | Mark that job is executed in GitLab CI environment | | **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the job | | **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the job | +| **GITLAB_USER_LOGIN** | 10.0 | all | The login username of the user who started the job | +| **GITLAB_USER_NAME** | 10.0 | all | The real name of the user who started the job | | **RESTORE_CACHE_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to restore the cache running a job | ## 9.0 Renaming @@ -85,6 +88,11 @@ To follow conventions of naming across GitLab, and to futher move away from the `build` term and toward `job` CI variables have been renamed for the 9.0 release. +>**Note:** +Starting with GitLab 9.0, we have deprecated the `$CI_BUILD_*` variables. **You are +strongly advised to use the new variables as we will remove the old ones in +future GitLab releases.** + | 8.x name | 9.0+ name | | --------------------- |------------------------ | | `CI_BUILD_ID` | `CI_JOB_ID` | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 1869782fe6e..d0ac3ec6163 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -130,7 +130,7 @@ There are also two edge cases worth mentioning: ### types -> Deprecated, and will be removed in 10.0. Use [stages](#stages) instead. +> Deprecated, and could be removed in one of the future releases. Use [stages](#stages) instead. Alias for [stages](#stages). @@ -427,16 +427,16 @@ a "key: value" pair. Be careful when using special characters: are executed in `parallel`. For more info about the use of `stage` please check [stages](#stages). -### only and except +### only and except (simplified) -`only` and `except` are two parameters that set a refs policy to limit when -jobs are built: +`only` and `except` are two parameters that set a job policy to limit when +jobs are created: 1. `only` defines the names of branches and tags for which the job will run. 2. `except` defines the names of branches and tags for which the job will **not** run. -There are a few rules that apply to the usage of refs policy: +There are a few rules that apply to the usage of job policy: * `only` and `except` are inclusive. If both `only` and `except` are defined in a job specification, the ref is filtered by `only` and `except`. @@ -497,6 +497,36 @@ job: The above example will run `job` for all branches on `gitlab-org/gitlab-ce`, except master. +### only and except (complex) + +> Introduced in GitLab 10.0 + +> This an _alpha_ feature, and it it subject to change at any time without + prior notice! + +Since GitLab 10.0 it is possible to define a more elaborate only/except job +policy configuration. + +GitLab now supports both, simple and complex strategies, so it is possible to +use an array and a hash configuration scheme. + +Two keys are now available: `refs` and `kubernetes`. Refs strategy equals to +simplified only/except configuration, whereas kubernetes strategy accepts only +`active` keyword. + +See the example below. Job is going to be created only when pipeline has been +scheduled or runs for a `master` branch, and only if kubernetes service is +active in the project. + +```yaml +job: + only: + refs: + - master + - schedules + kubernetes: active +``` + ### Job variables It is possible to define job variables using a `variables` keyword on a job @@ -1065,6 +1095,8 @@ a list of all previous jobs from which the artifacts should be downloaded. You can only define jobs from stages that are executed before the current one. An error will be shown if you define jobs from the current stage or next ones. Defining an empty array will skip downloading any artifacts for that job. +The status of the previous job is not considered when using `dependencies`, so +if it failed or it is a manual job that was not run, no error occurs. --- diff --git a/doc/development/README.md b/doc/development/README.md index 58993c52dcd..dd150421b65 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -46,6 +46,7 @@ ## Databases +- [Merge Request Checklist](database_merge_request_checklist.md) - [What requires downtime?](what_requires_downtime.md) - [Adding database indexes](adding_database_indexes.md) - [Post Deployment Migrations](post_deployment_migrations.md) @@ -56,6 +57,9 @@ - [Background Migrations](background_migrations.md) - [Storing SHA1 Hashes As Binary](sha1_as_binary.md) - [Iterating Tables In Batches](iterating_tables_in_batches.md) +- [Ordering Table Columns](ordering_table_columns.md) +- [Verifying Database Capabilities](verifying_database_capabilities.md) +- [Hash Indexes](hash_indexes.md) ## i18n diff --git a/doc/development/changelog.md b/doc/development/changelog.md index ce39a379a0e..f869938fe11 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -15,11 +15,14 @@ following format: title: "Going through change[log]s" merge_request: 1972 author: Ozzy Osbourne +type: added ``` The `merge_request` value is a reference to a merge request that adds this entry, and the `author` key is used to give attribution to community contributors. **Both are optional**. +The `type` field maps the category of the change, +valid options are: added, fixed, changed, deprecated, removed, security, other. **Type field is mandatory**. Community contributors and core team members are encouraged to add their name to the `author` field. GitLab team members **should not**. @@ -94,6 +97,19 @@ Its simplest usage is to provide the value for `title`: $ bin/changelog 'Hey DZ, I added a feature to GitLab!' ``` +At this point the script would ask you to select the category of the change (mapped to the `type` field in the entry): + +```text +>> Please specify the category of your change: +1. New feature +2. Bug fix +3. Feature change +4. New deprecation +5. Feature removal +6. Security fix +7. Other +``` + The entry filename is based on the name of the current Git branch. If you run the command above on a branch called `feature/hey-dz`, it will generate a `changelogs/unreleased/feature-hey-dz.yml` file. @@ -106,26 +122,29 @@ create changelogs/unreleased/my-feature.yml title: Hey DZ, I added a feature to GitLab! merge_request: author: +type: ``` If you're working on the GitLab EE repository, the entry will be added to `changelogs/unreleased-ee/` instead. #### Arguments -| Argument | Shorthand | Purpose | -| ----------------- | --------- | --------------------------------------------- | -| [`--amend`] | | Amend the previous commit | -| [`--force`] | `-f` | Overwrite an existing entry | -| [`--merge-request`] | `-m` | Set merge request ID | -| [`--dry-run`] | `-n` | Don't actually write anything, just print | -| [`--git-username`] | `-u` | Use Git user.name configuration as the author | -| [`--help`] | `-h` | Print help message | +| Argument | Shorthand | Purpose | +| ----------------- | --------- | ---------------------------------------------------------------------------------------------------------- | +| [`--amend`] | | Amend the previous commit | +| [`--force`] | `-f` | Overwrite an existing entry | +| [`--merge-request`] | `-m` | Set merge request ID | +| [`--dry-run`] | `-n` | Don't actually write anything, just print | +| [`--git-username`] | `-u` | Use Git user.name configuration as the author | +| [`--type`] | `-t` | The category of the change, valid options are: added, fixed, changed, deprecated, removed, security, other | +| [`--help`] | `-h` | Print help message | [`--amend`]: #-amend [`--force`]: #-force-or-f [`--merge-request`]: #-merge-request-or-m [`--dry-run`]: #-dry-run-or-n [`--git-username`]: #-git-username-or-u +[`--type`]: #-type-or-t [`--help`]: #-help ##### `--amend` @@ -147,6 +166,7 @@ create changelogs/unreleased/feature-hey-dz.yml title: Added an awesome new feature to GitLab merge_request: author: +type: ``` ##### `--force` or `-f` @@ -164,6 +184,7 @@ create changelogs/unreleased/feature-hey-dz.yml title: Hey DZ, I added a feature to GitLab! merge_request: 1983 author: +type: ``` ##### `--merge-request` or `-m` @@ -178,6 +199,7 @@ create changelogs/unreleased/feature-hey-dz.yml title: Hey DZ, I added a feature to GitLab! merge_request: 1983 author: +type: ``` ##### `--dry-run` or `-n` @@ -192,6 +214,7 @@ create changelogs/unreleased/feature-hey-dz.yml title: Added an awesome new feature to GitLab merge_request: author: +type: $ ls changelogs/unreleased/ ``` @@ -211,6 +234,21 @@ create changelogs/unreleased/feature-hey-dz.yml title: Hey DZ, I added a feature to GitLab! merge_request: author: Jane Doe +type: +``` + +##### `--type` or `-t` + +Use the **`--type`** or **`-t`** argument to provide the `type` value: + +```text +$ bin/changelog 'Hey DZ, I added a feature to GitLab!' -t added +create changelogs/unreleased/feature-hey-dz.yml +--- +title: Hey DZ, I added a feature to GitLab! +merge_request: +author: +type: added ``` ### History and Reasoning diff --git a/doc/development/database_merge_request_checklist.md b/doc/development/database_merge_request_checklist.md new file mode 100644 index 00000000000..75c395b61ef --- /dev/null +++ b/doc/development/database_merge_request_checklist.md @@ -0,0 +1,15 @@ +# Merge Request Checklist + +When creating a merge request that performs database related changes (schema +changes, adjusting queries to optimise performance, etc) you should use the +merge request template called "Database Changes". This template contains a +checklist of steps to follow to make sure the changes are up to snuff. + +To use the checklist, create a new merge request and click on the "Choose a +template" dropdown, then click "Database Changes". + +An example of this checklist can be found at +https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12463. + +The source code of the checklist can be found in at +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/merge_request_templates/Database%20Changes.md diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 90d1d9657b9..798f40eef3d 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -113,13 +113,12 @@ merge request. ## Links -- If a link makes the paragraph to span across multiple lines, do not use - the regular Markdown approach: `[Text](https://example.com)`. Instead use - `[Text][identifier]` and at the very bottom of the document add: - `[identifier]: https://example.com`. This is another way to create Markdown - links which keeps the document clear and concise. Bonus points if you also - add an alternative text: `[identifier]: https://example.com "Alternative text"` - that appears when hovering your mouse on a link +- Use the regular inline link markdown markup `[Text](https://example.com)`. + It's easier to read, review, and maintain. +- If there's a link that repeats several times through the same document, + you can use `[Text][identifier]` and at the bottom of the section or the + document add: `[identifier]: https://example.com`, in which case, we do + encourage you to also add an alternative text: `[identifier]: https://example.com "Alternative text"` that appears when hovering your mouse on a link. ### Linking to inline docs diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 6ade3231fac..4f20aa070de 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -102,42 +102,41 @@ followed by any global declarations, then a blank newline prior to any imports o 1. Relative paths: when importing a module in the same directory, a child directory, or an immediate parent directory prefer relative paths. When -importing a module which is two or more levels up, prefer either `~/` or `ee/` -. +importing a module which is two or more levels up, prefer either `~/` or `ee/`. -In **app/assets/javascripts/my-feature/subdir**: + In **app/assets/javascripts/my-feature/subdir**: -``` javascript -// bad -import Foo from '~/my-feature/foo'; -import Bar from '~/my-feature/subdir/bar'; -import Bin from '~/my-feature/subdir/lib/bin'; + ```javascript + // bad + import Foo from '~/my-feature/foo'; + import Bar from '~/my-feature/subdir/bar'; + import Bin from '~/my-feature/subdir/lib/bin'; -// good -import Foo from '../foo'; -import Bar from './bar'; -import Bin from './lib/bin'; -``` + // good + import Foo from '../foo'; + import Bar from './bar'; + import Bin from './lib/bin'; + ``` -In **spec/javascripts**: + In **spec/javascripts**: -``` javascript -// bad -import Foo from '../../app/assets/javascripts/my-feature/foo'; + ```javascript + // bad + import Foo from '../../app/assets/javascripts/my-feature/foo'; -// good -import Foo from '~/my-feature/foo'; -``` + // good + import Foo from '~/my-feature/foo'; + ``` -When referencing an **EE component**: + When referencing an **EE component**: -``` javascript -// bad -import Foo from '../../../../../ee/app/assets/javascripts/my-feature/ee-foo'; + ```javascript + // bad + import Foo from '../../../../../ee/app/assets/javascripts/my-feature/ee-foo'; -// good -import Foo from 'ee/my-feature/foo'; -``` + // good + import Foo from 'ee/my-feature/foo'; + ``` 1. Avoid using IIFE. Although we have a lot of examples of files which wrap their contents in IIFEs (immediately-invoked function expressions), @@ -145,24 +144,23 @@ this is no longer necessary after the transition from Sprockets to webpack. Do not use them anymore and feel free to remove them when refactoring legacy code. 1. Avoid adding to the global namespace. - ```javascript - // bad - window.MyClass = class { /* ... */ }; + ```javascript + // bad + window.MyClass = class { /* ... */ }; - // good - export default class MyClass { /* ... */ } - ``` + // good + export default class MyClass { /* ... */ } + ``` 1. Side effects are forbidden in any script which contains exports - ```javascript - // bad - export default class MyClass { /* ... */ } - - document.addEventListener("DOMContentLoaded", function(event) { - new MyClass(); - } - ``` + ```javascript + // bad + export default class MyClass { /* ... */ } + document.addEventListener("DOMContentLoaded", function(event) { + new MyClass(); + } + ``` #### Data Mutation and Pure functions 1. Strive to write many small pure functions, and minimize where mutations occur. @@ -414,19 +412,19 @@ A forEach will cause side effects, it will be mutating the array being iterated. #### Data 1. `data` method should always be a function - ```javascript - // bad - data: { - foo: 'foo' - } - - // good - data() { - return { + ```javascript + // bad + data: { foo: 'foo' - }; - } - ``` + } + + // good + data() { + return { + foo: 'foo' + }; + } + ``` #### Directives @@ -481,7 +479,8 @@ A forEach will cause side effects, it will be mutating the array being iterated. 1. `beforeDestroy` 1. `destroyed` -#### Vue and Boostrap +#### Vue and Bootstrap + 1. Tooltips: Do not rely on `has-tooltip` class name for Vue components ```javascript // bad @@ -512,6 +511,19 @@ A forEach will cause side effects, it will be mutating the array being iterated. $('span').tooltip('fixTitle'); ``` +### The Javascript/Vue Accord +The goal of this accord is to make sure we are all on the same page. + +1. When writing Vue, you may not use jQuery in your application. + 1. If you need to grab data from the DOM, you may query the DOM 1 time while bootstrapping your application to grab data attributes using `dataset`. You can do this without jQuery. + 1. You may use a jQuery dependency in Vue.js following [this example from the docs](https://vuejs.org/v2/examples/select2.html). + 1. If an outside jQuery Event needs to be listen to inside the Vue application, you may use jQuery event listeners. + 1. We will avoid adding new jQuery events when they are not required. Instead of adding new jQuery events take a look at [different methods to do the same task](https://vuejs.org/v2/api/#vm-emit). +1. You may query the `window` object 1 time, while bootstrapping your application for application specific data (e.g. `scrollTo` is ok to access anytime). Do this access during the bootstrapping of your application. +1. You may have a temporary but immediate need to create technical debt by writing code that does not follow our standards, to be refactored later. Maintainers need to be ok with the tech debt in the first place. An issue should be created for that tech debt to evaluate it further and discuss. In the coming months you should fix that tech debt, with it's priority to be determined by maintainers. +1. When creating tech debt you must write the tests for that code before hand and those tests may not be rewritten. e.g. jQuery tests rewritten to Vue tests. +1. You may choose to use VueX as a centralized state management. If you choose not to use VueX, you must use the *store pattern* which can be found in the [Vue.js documentation](https://vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch). +1. Once you have chosen a centralized state management solution you must use it for your entire application. i.e. Don't mix and match your state management solutions. ## SCSS - [SCSS](style_guide_scss.md) diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 0742b202807..2607353782a 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -28,8 +28,9 @@ As always, the Frontend Architectural Experts are available to help with any Vue All new features built with Vue.js must follow a [Flux architecture][flux]. The main goal we are trying to achieve is to have only one data flow and only one data entry. -In order to achieve this goal, each Vue bundle needs a Store - where we keep all the data -, -a Service - that we use to communicate with the server - and a main Vue component. +In order to achieve this goal, you can either use [vuex](#vuex) or use the [store pattern][state-management], explained below: + +Each Vue bundle needs a Store - where we keep all the data -,a Service - that we use to communicate with the server - and a main Vue component. Think of the Main Vue Component as the entry point of your application. This is the only smart component that should exist in each Vue feature. @@ -74,6 +75,59 @@ provided as a prop to the main component. Don't forget to follow [these steps.][page_specific_javascript] +### Bootstrapping Gotchas +#### Providing data from Haml to JavaScript +While mounting a Vue application may be a need to provide data from Rails to JavaScript. +To do that, provide the data through `data` attributes in the HTML element and query them while mounting the application. + +_Note:_ You should only do this while initing the application, because the mounted element will be replaced with Vue-generated DOM. + +The advantage of providing data from the DOM to the Vue instance through `props` in the `render` function +instead of querying the DOM inside the main vue component is that makes tests easier by avoiding the need to +create a fixture or an HTML element in the unit test. See the following example: + +```javascript +// haml +.js-vue-app{ data: { endpoint: 'foo' }} + +document.addEventListener('DOMContentLoaded', () => new Vue({ + el: '.js-vue-app', + data() { + const dataset = this.$options.el.dataset; + return { + endpoint: dataset.endpoint, + }; + }, + render(createElement) { + return createElement('my-component', { + props: { + endpoint: this.isLoading, + }, + }); + }, +})); +``` + +#### Accessing the `gl` object +When we need to query the `gl` object for data that won't change during the application's lyfecyle, we should do it in the same place where we query the DOM. +By following this practice, we can avoid the need to mock the `gl` object, which will make tests easier. +It should be done while initializing our Vue instance, and the data should be provided as `props` to the main component: + +##### example: +```javascript + +document.addEventListener('DOMContentLoaded', () => new Vue({ + el: '.js-vue-app', + render(createElement) { + return createElement('my-component', { + props: { + username: gon.current_username, + }, + }); + }, +})); +``` + ### A folder for Components This folder holds all components that are specific of this new feature. @@ -89,6 +143,29 @@ in one table would not be a good use of this pattern. You can read more about components in Vue.js site, [Component System][component-system] +#### Components Gotchas +1. Using SVGs in components: To use an SVG in a template we need to make it a property we can access through the component. +A `prop` and a property returned by the `data` functions require `vue` to set a `getter` and a `setter` for each of them. +The SVG should be a computed property in order to improve performance, note that computed properties are cached based on their dependencies. + +```javascript +// bad +import svg from 'svg.svg'; +data() { + return { + myIcon: svg, + }; +}; + +// good +import svg from 'svg.svg'; +computed: { + myIcon() { + return svg; + } +} +``` + ### A folder for the Store The Store is a class that allows us to manage the state in a single @@ -430,11 +507,23 @@ describe('Todos App', () => { }); }); ``` +#### `mountComponent` helper +There is an helper in `spec/javascripts/helpers/vue_mount_component_helper.js` that allows you to mount a component with the given props: + +```javascript +import Vue from 'vue'; +import mountComponent from 'helpers/vue_mount_component_helper.js' +import component from 'component.vue' + +const Component = Vue.extend(component); +const data = {prop: 'foo'}; +const vm = mountComponent(Component, data); +``` + #### Test the component's output The main return value of a Vue component is the rendered output. In order to test the component we need to test the rendered output. [Vue][vue-test] guide's to unit test show us exactly that: - ### Stubbing API responses [Vue Resource Interceptors][vue-resource-interceptor] allow us to add a interceptor with the response we need: @@ -481,6 +570,198 @@ new Component({ new Component().$mount(); ``` +## Vuex +To manage the state of an application you may use [Vuex][vuex-docs]. + +_Note:_ All of the below is explained in more detail in the official [Vuex documentation][vuex-docs]. + +### Separation of concerns +Vuex is composed of State, Getters, Mutations, Actions and Modules. + +When a user clicks on an action, we need to `dispatch` it. This action will `commit` a mutation that will change the state. +_Note:_ The action itself will not update the state, only a mutation should update the state. + +#### File structure +When using Vuex at GitLab, separate this concerns into different files to improve readability. If you can, separate the Mutation Types as well: + +``` +└── store + ├── index.js # where we assemble modules and export the store + ├── actions.js # actions + ├── mutations.js # mutations + ├── getters.js # getters + └── mutation_types.js # mutation types +``` +The following examples show an application that lists and adds users to the state. + +##### `index.js` +This is the entry point for our store. You can use the following as a guide: + +```javascript +import Vue from 'vue'; +import Vuex from 'vuex'; +import * as actions from './actions'; +import * as mutations from './mutations'; + +Vue.use(Vuex); + +export default new Vuex.Store({ + actions, + getters, + state: { + users: [], + }, +}); +``` +_Note:_ If the state of the application is too complex, an individual file for the state may be better. + +#### `actions.js` +An action commits a mutatation. In this file, we will write the actions that will call the respective mutation: + +```javascript + import * as types from './mutation-types' + + export const addUser = ({ commit }, user) => { + commit(types.ADD_USER, user); + }; +``` + +To dispatch an action from a component, use the `mapActions` helper: +```javascript +import { mapActions } from 'vuex'; + +{ + methods: { + ...mapActions([ + 'addUser', + ]), + onClickUser(user) { + this.addUser(user); + }, + }, +}; +``` + +#### `getters.js` +Sometimes we may need to get derived state based on store state, like filtering for a specific prop. This can be done through the `getters`: + +```javascript +// get all the users with pets +export getUsersWithPets = (state, getters) => { + return state.users.filter(user => user.pet !== undefined); +}; +``` + +To access a getter from a component, use the `mapGetters` helper: +```javascript +import { mapGetters } from 'vuex'; + +{ + computed: { + ...mapGetters([ + 'getUsersWithPets', + ]), + }, +}; +``` + +#### `mutations.js` +The only way to actually change state in a Vuex store is by committing a mutation. + +```javascript + import * as types from './mutation-types' + export default { + [types.ADD_USER](state, user) { + state.users.push(user); + }, + }; +``` + +#### `mutations_types.js` +From [vuex mutations docs][vuex-mutations]: +> It is a commonly seen pattern to use constants for mutation types in various Flux implementations. This allows the code to take advantage of tooling like linters, and putting all constants in a single file allows your collaborators to get an at-a-glance view of what mutations are possible in the entire application. + +```javascript +export const ADD_USER = 'ADD_USER'; +``` + +### How to include the store in your application +The store should be included in the main component of your application: +```javascript + // app.vue + import store from 'store'; // it will include the index.js file + + export default { + name: 'application', + store, + ... + }; +``` + +### Vuex Gotchas +1. Avoid calling a mutation directly. Always use an action to commit a mutation. Doing so will keep consistency through out the application. From Vuex docs: + + > why don't we just call store.commit('action') directly? Well, remember that mutations must be synchronous? Actions aren't. We can perform asynchronous operations inside an action. + + ```javascript + // component.vue + + // bad + created() { + this.$store.commit('mutation'); + } + + // good + created() { + this.$store.dispatch('action'); + } + ``` +1. When possible, use mutation types instead of hardcoding strings. It will be less error prone. +1. The State will be accessible in all components descending from the use where the store is instantiated. + +### Testing Vuex +#### Testing Vuex concerns +Refer to [vuex docs][vuex-testing] regarding testing Actions, Getters and Mutations. + +#### Testing components that need a store +Smaller components might use `store` properties to access the data. +In order to write unit tests for those components, we need to include the store and provide the correct state: + +```javascript +//component_spec.js +import Vue from 'vue'; +import store from './store'; +import component from './component.vue' + +describe('component', () => { + let vm; + let Component; + + beforeEach(() => { + Component = Vue.extend(issueActions); + }); + + afterEach(() => { + vm.$destroy(); + }); + + it('should show a user', () => { + const user = { + name: 'Foo', + age: '30', + }; + + // populate the store + store.dipatch('addUser', user); + + vm = new Component({ + store, + propsData: props, + }).$mount(); + }); +}); +``` + [vue-docs]: http://vuejs.org/guide/index.html [issue-boards]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/assets/javascripts/boards [environments-table]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/assets/javascripts/environments @@ -493,3 +774,7 @@ new Component().$mount(); [vue-test]: https://vuejs.org/v2/guide/unit-testing.html [issue-boards-service]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/boards/services/board_service.js.es6 [flux]: https://facebook.github.io/flux +[vuex-docs]: https://vuex.vuejs.org +[vuex-structure]: https://vuex.vuejs.org/en/structure.html +[vuex-mutations]: https://vuex.vuejs.org/en/mutations.html +[vuex-testing]: https://vuex.vuejs.org/en/testing.html diff --git a/doc/development/hash_indexes.md b/doc/development/hash_indexes.md new file mode 100644 index 00000000000..e6c1b3590b1 --- /dev/null +++ b/doc/development/hash_indexes.md @@ -0,0 +1,20 @@ +# Hash Indexes + +Both PostgreSQL and MySQL support hash indexes besides the regular btree +indexes. Hash indexes however are to be avoided at all costs. While they may +_sometimes_ provide better performance the cost of rehashing can be very high. +More importantly: at least until PostgreSQL 10.0 hash indexes are not +WAL-logged, meaning they are not replicated to any replicas. From the PostgreSQL +documentation: + +> Hash index operations are not presently WAL-logged, so hash indexes might need +> to be rebuilt with REINDEX after a database crash if there were unwritten +> changes. Also, changes to hash indexes are not replicated over streaming or +> file-based replication after the initial base backup, so they give wrong +> answers to queries that subsequently use them. For these reasons, hash index +> use is presently discouraged. + +RuboCop is configured to register an offence when it detects the use of a hash +index. + +Instead of using hash indexes you should use regular btree indexes. diff --git a/doc/development/i18n_guide.md b/doc/development/i18n_guide.md index 756535e28bc..bd0ef39ca62 100644 --- a/doc/development/i18n_guide.md +++ b/doc/development/i18n_guide.md @@ -138,6 +138,47 @@ translations. There's no need to generate `.po` files. Translations that aren't used in the source code anymore will be marked with `~#`; these can be removed to keep our translation files clutter-free. +### Validating PO files + +To make sure we keep our translation files up to date, there's a linter that is +running on CI as part of the `static-analysis` job. + +To lint the adjustments in PO files locally you can run `rake gettext:lint`. + +The linter will take the following into account: + +- Valid PO-file syntax +- Variable usage + - Only one unnamed (`%d`) variable, since the order of variables might change + in different languages + - All variables used in the message-id are used in the translation + - There should be no variables used in a translation that aren't in the + message-id +- Errors during translation. + +The errors are grouped per file, and per message ID: + +``` +Errors in `locale/zh_HK/gitlab.po`: + PO-syntax errors + SimplePoParser::ParserErrorSyntax error in lines + Syntax error in msgctxt + Syntax error in msgid + Syntax error in msgstr + Syntax error in message_line + There should be only whitespace until the end of line after the double quote character of a message text. + Parseing result before error: '{:msgid=>["", "You are going to remove %{project_name_with_namespace}.\\n", "Removed project CANNOT be restored!\\n", "Are you ABSOLUTELY sure?"]}' + SimplePoParser filtered backtrace: SimplePoParser::ParserError +Errors in `locale/zh_TW/gitlab.po`: + 1 pipeline + <%d 條流水線> is using unknown variables: [%d] + Failure translating to zh_TW with []: too few arguments +``` + +In this output the `locale/zh_HK/gitlab.po` has syntax errors. +The `locale/zh_TW/gitlab.po` has variables that are used in the translation that +aren't in the message with id `1 pipeline`. + ## Working with special content ### Interpolation diff --git a/doc/development/img/manual_build_docs.png b/doc/development/img/manual_build_docs.png Binary files differnew file mode 100644 index 00000000000..fef767c2a79 --- /dev/null +++ b/doc/development/img/manual_build_docs.png diff --git a/doc/development/licensing.md b/doc/development/licensing.md index 1f115059fb8..9a5811d8474 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -55,6 +55,7 @@ Libraries with the following licenses are acceptable for use: - [BSD 3-Clause License][BSD-3-Clause] (also known as New BSD or Modified BSD): A permissive (non-copyleft) license as defined by the Open Source Initiative - [ISC License][ISC] (also known as the OpenBSD License): A permissive (non-copyleft) license as defined by the Open Source Initiative. - [Creative Commons Zero (CC0)][CC0]: A public domain dedication, recommended as a way to disclaim copyright on your work to the maximum extent possible. +- [Unlicense][UNLICENSE]: Another public domain dedication. ## Unacceptable Licenses @@ -63,10 +64,11 @@ Libraries with the following licenses are unacceptable for use: - [GNU GPL][GPL] (version 1, [version 2][GPLv2], [version 3][GPLv3], or any future versions): GPL-licensed libraries cannot be linked to from non-GPL projects. - [GNU AGPLv3][AGPLv3]: AGPL-licensed libraries cannot be linked to from non-GPL projects. - [Open Software License (OSL)][OSL]: is a copyleft license. In addition, the FSF [recommend against its use][OSL-GNU]. +- [Facebook BSD + PATENTS][Facebook]: is a 3-clause BSD license with a patent grant that has been deemed [Category X][x-list] by the Apache foundation. ## Requesting Approval for Licenses -Libraries that are not listed in the [Acceptable Licenses][Acceptable-Licenses] or [Unacceptable Licenses][Unacceptable-Licenses] list can be submitted to the legal team for review. Please create an issue in the [Organization Repository][Org-Repo] and cc `@gl-legal`. After a decision has been made, the original requestor is responsible for updating this document. +Libraries that are not listed in the [Acceptable Licenses][Acceptable-Licenses] or [Unacceptable Licenses][Unacceptable-Licenses] list can be submitted to the legal team for review. Please email `legal@gitlab.com` with the details. After a decision has been made, the original requestor is responsible for updating this document. ## Notes @@ -101,5 +103,8 @@ Gems which are included only in the "development" or "test" groups by Bundler ar [OSL]: https://opensource.org/licenses/OSL-3.0 [OSL-GNU]: https://www.gnu.org/licenses/license-list.en.html#OSL [Org-Repo]: https://gitlab.com/gitlab-com/organization +[UNLICENSE]: https://unlicense.org +[Facebook]: https://code.facebook.com/pages/850928938376556 +[x-list]: https://www.apache.org/legal/resolved.html#category-x [Acceptable-Licenses]: #acceptable-licenses [Unacceptable-Licenses]: #unacceptable-licenses diff --git a/doc/development/ordering_table_columns.md b/doc/development/ordering_table_columns.md new file mode 100644 index 00000000000..249e70c7b0e --- /dev/null +++ b/doc/development/ordering_table_columns.md @@ -0,0 +1,127 @@ +# Ordering Table Columns + +Similar to C structures the space of a table is influenced by the order of +columns. This is because the size of columns is aligned depending on the type of +the column. Take the following column order for example: + +* id (integer, 4 bytes) +* name (text, variable) +* user_id (integer, 4 bytes) + +Integers are aligned to the word size. This means that on a 64 bit platform the +actual size of each column would be: 8 bytes, variable, 8 bytes. This means that +each row will require at least 16 bytes for the two integers, and a variable +amount for the text field. If a table has a few rows this is not an issue, but +once you start storing millions of rows you can save space by using a different +order. For the above example a more ideal column order would be the following: + +* id (integer, 4 bytes) +* user_id (integer, 4 bytes) +* name (text, variable) + +In this setup the `id` and `user_id` columns can be packed together, which means +we only need 8 bytes to store _both_ of them. This in turn each row will require +8 bytes less of space. + +For GitLab we require that columns of new tables are ordered based to use the +least amount of space. An easy way of doing this is to order them based on the +type size in descending order with variable sizes (string and text columns for +example) at the end. + +## Type Sizes + +While the PostgreSQL docuemntation +(https://www.postgresql.org/docs/current/static/datatype.html) contains plenty +of information we will list the sizes of common types here so it's easier to +look them up. Here "word" refers to the word size, which is 4 bytes for a 32 +bits platform and 8 bytes for a 64 bits platform. + +| Type | Size | Aligned To | +|:-----------------|:-------------------------------------|:-----------| +| smallint | 2 bytes | 1 word | +| integer | 4 bytes | 1 word | +| bigint | 8 bytes | 8 bytes | +| real | 4 bytes | 1 word | +| double precision | 8 bytes | 8 bytes | +| boolean | 1 byte | not needed | +| text / string | variable, 1 byte plus the data | 1 word | +| bytea | variable, 1 or 4 bytes plus the data | 1 word | +| timestamp | 8 bytes | 8 bytes | +| timestamptz | 8 bytes | 8 bytes | +| date | 4 bytes | 1 word | + +A "variable" size means the actual size depends on the value being stored. If +PostgreSQL determines this can be embedded directly into a row it may do so, but +for very large values it will store the data externally and store a pointer (of +1 word in size) in the column. Because of this variable sized columns should +always be at the end of a table. + +## Real Example + +Let's use the "events" table as an example, which currently has the following +layout: + +| Column | Type | Size | +|:------------|:----------------------------|:---------| +| id | integer | 4 bytes | +| target_type | character varying | variable | +| target_id | integer | 4 bytes | +| title | character varying | variable | +| data | text | variable | +| project_id | integer | 4 bytes | +| created_at | timestamp without time zone | 8 bytes | +| updated_at | timestamp without time zone | 8 bytes | +| action | integer | 4 bytes | +| author_id | integer | 4 bytes | + +After adding padding to align the columns this would translate to columns being +divided into fixed size chunks as follows: + +| Chunk Size | Columns | +|:-----------|:------------------| +| 8 bytes | id | +| variable | target_type | +| 8 bytes | target_id | +| variable | title | +| variable | data | +| 8 bytes | project_id | +| 8 bytes | created_at | +| 8 bytes | updated_at | +| 8 bytes | action, author_id | + +This means that excluding the variable sized data we need at least 48 bytes per +row. + +We can optimise this by using the following column order instead: + +| Column | Type | Size | +|:------------|:----------------------------|:---------| +| created_at | timestamp without time zone | 8 bytes | +| updated_at | timestamp without time zone | 8 bytes | +| id | integer | 4 bytes | +| target_id | integer | 4 bytes | +| project_id | integer | 4 bytes | +| action | integer | 4 bytes | +| author_id | integer | 4 bytes | +| target_type | character varying | variable | +| title | character varying | variable | +| data | text | variable | + +This would produce the following chunks: + +| Chunk Size | Columns | +|:-----------|:-------------------| +| 8 bytes | created_at | +| 8 bytes | updated_at | +| 8 bytes | id, target_id | +| 8 bytes | project_id, action | +| 8 bytes | author_id | +| variable | target_type | +| variable | title | +| variable | data | + +Here we only need 40 bytes per row excluding the variable sized data. 8 bytes +being saved may not sound like much, but for tables as large as the "events" +table it does begin to matter. For example, when storing 80 000 000 rows this +translates to a space saving of at least 610 MB: all by just changing the order +of a few columns. diff --git a/doc/development/serializing_data.md b/doc/development/serializing_data.md index 2b56f48bc44..37332c20147 100644 --- a/doc/development/serializing_data.md +++ b/doc/development/serializing_data.md @@ -1,7 +1,8 @@ # Serializing Data **Summary:** don't store serialized data in the database, use separate columns -and/or tables instead. +and/or tables instead. This includes storing of comma separated values as a +string. Rails makes it possible to store serialized data in JSON, YAML or other formats. Such a field can be defined as follows: diff --git a/doc/development/sql.md b/doc/development/sql.md index 23fd7604957..974b1d99dff 100644 --- a/doc/development/sql.md +++ b/doc/development/sql.md @@ -216,4 +216,30 @@ exact same results. This also means there's no need to add an index on `created_at` to ensure consistent performance as `id` is already indexed by default. +## Use WHERE EXISTS instead of WHERE IN + +While `WHERE IN` and `WHERE EXISTS` can be used to produce the same data it is +recommended to use `WHERE EXISTS` whenever possible. While in many cases +PostgreSQL can optimise `WHERE IN` quite well there are also many cases where +`WHERE EXISTS` will perform (much) better. + +In Rails you have to use this by creating SQL fragments: + +```ruby +Project.where('EXISTS (?)', User.select(1).where('projects.creator_id = users.id AND users.foo = X')) +``` + +This would then produce a query along the lines of the following: + +```sql +SELECT * +FROM projects +WHERE EXISTS ( + SELECT 1 + FROM users + WHERE projects.creator_id = users.id + AND users.foo = X +) +``` + [gin-index]: http://www.postgresql.org/docs/current/static/gin.html diff --git a/doc/development/testing.md b/doc/development/testing.md index 3d5aa3d45e9..83269303005 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -157,8 +157,9 @@ trade-off: - Unit tests are usually cheap, and you should consider them like the basement of your house: you need them to be confident that your code is behaving - correctly. However if you run only unit tests without integration / system tests, you might [miss] the [big] [picture]! -- Integration tests are a bit more expensive, but don't abuse them. A feature test + correctly. However if you run only unit tests without integration / system + tests, you might [miss] the [big] [picture]! +- Integration tests are a bit more expensive, but don't abuse them. A system test is often better than an integration test that is stubbing a lot of internals. - System tests are expensive (compared to unit tests), even more if they require a JavaScript driver. Make sure to follow the guidelines in the [Speed](#test-speed) @@ -188,24 +189,34 @@ Please consult the [dedicated "Frontend testing" guide](./fe_guide/testing.md). ### General Guidelines - Use a single, top-level `describe ClassName` block. -- Use `described_class` instead of repeating the class name being described - (_this is enforced by RuboCop_). - Use `.method` to describe class methods and `#method` to describe instance methods. - Use `context` to test branching logic. -- Use multi-line `do...end` blocks for `before` and `after`, even when it would - fit on a single line. - Don't assert against the absolute value of a sequence-generated attribute (see [Gotchas](gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). -- Don't supply the `:each` argument to hooks since it's the default. -- Prefer `not_to` to `to_not` (_this is enforced by RuboCop_). - Try to match the ordering of tests to the ordering within the class. - Try to follow the [Four-Phase Test][four-phase-test] pattern, using newlines to separate phases. -- Try to use `Gitlab.config.gitlab.host` rather than hard coding `'localhost'` +- Use `Gitlab.config.gitlab.host` rather than hard coding `'localhost'` +- Don't assert against the absolute value of a sequence-generated attribute (see + [Gotchas](gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). +- Don't supply the `:each` argument to hooks since it's the default. - On `before` and `after` hooks, prefer it scoped to `:context` over `:all` [four-phase-test]: https://robots.thoughtbot.com/four-phase-test +### Automatic retries and flaky tests detection + +On our CI, we use [rspec-retry] to automatically retry a failing example a few +times (see [`spec/spec_helper.rb`] for the precise retries count). + +We also use a home-made `RspecFlaky::Listener` listener which records flaky +examples in a JSON report file on `master` (`retrieve-tests-metadata` and `update-tests-metadata` jobs), and warns when a new flaky example +is detected in any other branch (`flaky-examples-check` job). In the future, the +`flaky-examples-check` job will not be allowed to fail. + +[rspec-retry]: https://github.com/NoRedInk/rspec-retry +[`spec/spec_helper.rb`]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/spec_helper.rb + ### `let` variables GitLab's RSpec suite has made extensive use of `let` variables to reduce @@ -268,6 +279,43 @@ end - Avoid scenario titles that add no information, such as "successfully". - Avoid scenario titles that repeat the feature title. +### Table-based / Parameterized tests + +This style of testing is used to exercise one piece of code with a comprehensive +range of inputs. By specifying the test case once, alongside a table of inputs +and the expected output for each, your tests can be made easier to read and more +compact. + +We use the [rspec-parameterized](https://github.com/tomykaira/rspec-parameterized) +gem. A short example, using the table syntax and checking Ruby equality for a +range of inputs, might look like this: + +```ruby +describe "#==" do + using Rspec::Parameterized::TableSyntax + + let(:project1) { create(:project) } + let(:project2) { create(:project) } + where(:a, :b, :result) do + 1 | 1 | true + 1 | 2 | false + true | true | true + true | false | false + project1 | project1 | true + project2 | project2 | true + project 1 | project2 | false + end + + with_them do + it { expect(a == b).to eq(result) } + + it 'is isomorphic' do + expect(b == a).to eq(result) + end + end +end +``` + ### Matchers Custom matchers should be created to clarify the intent and/or hide the @@ -276,6 +324,15 @@ complexity of RSpec expectations.They should be placed under a certain type of specs only (e.g. features, requests etc.) but shouldn't be if they apply to multiple type of specs. +#### have_gitlab_http_status + +Prefer `have_gitlab_http_status` over `have_http_status` because the former +could also show the response body whenever the status mismatched. This would +be very useful whenever some tests start breaking and we would love to know +why without editing the source and rerun the tests. + +This is especially useful whenever it's showing 500 internal server error. + ### Shared contexts All shared contexts should be be placed under `spec/support/shared_contexts/`. @@ -472,10 +529,7 @@ slowest test files and try to improve them. ## CI setup -- On CE, the test suite only runs against PostgreSQL by default. We additionally - run the suite against MySQL for tags, `master`, and any branch that includes - `mysql` in the name. -- On EE, the test suite always runs both PostgreSQL and MySQL. +- On CE and EE, the test suite runs both PostgreSQL and MySQL. - Rails logging to `log/test.log` is disabled by default in CI [for performance reasons][logging]. To override this setting, provide the `RAILS_ENABLE_TEST_LOG` environment variable. diff --git a/doc/development/verifying_database_capabilities.md b/doc/development/verifying_database_capabilities.md new file mode 100644 index 00000000000..cc6d62957e3 --- /dev/null +++ b/doc/development/verifying_database_capabilities.md @@ -0,0 +1,26 @@ +# Verifying Database Capabilities + +Sometimes certain bits of code may only work on a certain database and/or +version. While we try to avoid such code as much as possible sometimes it is +necessary to add database (version) specific behaviour. + +To facilitate this we have the following methods that you can use: + +* `Gitlab::Database.postgresql?`: returns `true` if PostgreSQL is being used +* `Gitlab::Database.mysql?`: returns `true` if MySQL is being used +* `Gitlab::Database.version`: returns the PostgreSQL version number as a string + in the format `X.Y.Z`. This method does not work for MySQL + +This allows you to write code such as: + +```ruby +if Gitlab::Database.postgresql? + if Gitlab::Database.version.to_f >= 9.6 + run_really_fast_query + else + run_fast_query + end +else + run_query +end +``` diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index eac9ec2a470..479258f743e 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -103,3 +103,24 @@ If that job fails, read the instructions in the job log for what to do next. Contributors do not need to submit their changes to EE, GitLab Inc. employees on the other hand need to make sure that their changes apply cleanly to both CE and EE. + +## Previewing the changes live + +If you want to preview your changes live, you can use the manual `build-docs` +job in your merge request. + + + +This job will: + +1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) + project named after the scheme: `<CE/EE-branch-slug>-built-from-ce-ee` +1. Trigger a pipeline and build the docs site with your changes + +Look for the docs URL at the output of the `build-docs` job. + +>**Note:** +Make sure that you always delete the branch of the merge request you were +working on. If you don't, the remote docs branch won't be removed either, +and the server where the Review Apps are hosted will eventually be out of +disk space. diff --git a/doc/gitlab-basics/create-project.md b/doc/gitlab-basics/create-project.md index 2513f4b420a..67ef189fee9 100644 --- a/doc/gitlab-basics/create-project.md +++ b/doc/gitlab-basics/create-project.md @@ -1,5 +1,9 @@ # How to create a project in GitLab +>**Notes:** +- For a list of words that are not allowed to be used as project names see the + [reserved names][reserved]. + 1. In your dashboard, click the green **New project** button or use the plus icon in the upper right corner of the navigation bar. @@ -9,13 +13,17 @@  +1. Choose if you want start a blank project, or with one of the predefined + [Project Templates](https://gitlab.com/gitlab-org/project-templates): + this will kickstart your repository code and CI automatically. + Otherwise, if you have a project in a different repository, you can [import it] by + clicking an **Import project from** button provided this is enabled in + your GitLab instance. Ask your administrator if not. + 1. Provide the following information: - Enter the name of your project in the **Project name** field. You can't use special characters, but you can use spaces, hyphens, underscores or even emoji. - - If you have a project in a different repository, you can [import it] by - clicking an **Import project from** button provided this is enabled in - your GitLab instance. Ask your administrator if not. - The **Project description (optional)** field enables you to enter a description for your project's dashboard, which will help others understand what your project is about. Though it's not required, it's a good @@ -26,3 +34,4 @@ 1. Click **Create project**. [import it]: ../workflow/importing/README.md +[reserved]: ../user/reserved_names.md diff --git a/doc/gitlab-basics/img/create_new_project_info.png b/doc/gitlab-basics/img/create_new_project_info.png Binary files differindex fcfbca87b91..ef8753e224b 100644 --- a/doc/gitlab-basics/img/create_new_project_info.png +++ b/doc/gitlab-basics/img/create_new_project_info.png diff --git a/doc/install/README.md b/doc/install/README.md index bc831a37735..656f8720361 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -17,10 +17,18 @@ the hardware requirements. - [Installation from source](installation.md) - Install GitLab from source. Useful for unsupported systems like *BSD. For an overview of the directory structure, read the [structure documentation](structure.md). -- [Docker](https://docs.gitlab.com/omnibus/docker/) - Install GitLab using Docker. +- [Docker](docker.md) - Install GitLab using Docker. + +## Install GitLab on cloud providers + - [Installing in Kubernetes](kubernetes/index.md) - Install GitLab into a Kubernetes Cluster using our official Helm Chart Repository. -- Testing only! [DigitalOcean and Docker Machine](digitaloceandocker.md) - +- [Install GitLab on OpenShift](../articles/openshift_and_gitlab/index.md) +- [Install GitLab on DC/OS](https://mesosphere.com/blog/gitlab-dcos/) via [GitLab-Mesosphere integration](https://about.gitlab.com/2016/09/16/announcing-gitlab-and-mesosphere/) +- [Install GitLab on Azure](azure/index.md) +- [Install GitLab on Google Cloud Platform](google_cloud_platform/index.md) +- [Install on AWS](https://about.gitlab.com/aws/) +- _Testing only!_ [DigitalOcean and Docker Machine](digitaloceandocker.md) - Quickly test any version of GitLab on DigitalOcean using Docker Machine. ## Database diff --git a/doc/install/azure/img/azure-add-inbound-sec-rule-http.png b/doc/install/azure/img/azure-add-inbound-sec-rule-http.png Binary files differnew file mode 100644 index 00000000000..abf500cb63a --- /dev/null +++ b/doc/install/azure/img/azure-add-inbound-sec-rule-http.png diff --git a/doc/install/azure/img/azure-add-inbound-sec-rule-ssh.png b/doc/install/azure/img/azure-add-inbound-sec-rule-ssh.png Binary files differnew file mode 100644 index 00000000000..f7a8a04dfa7 --- /dev/null +++ b/doc/install/azure/img/azure-add-inbound-sec-rule-ssh.png diff --git a/doc/install/azure/img/azure-create-virtual-machine-basics-password.png b/doc/install/azure/img/azure-create-virtual-machine-basics-password.png Binary files differnew file mode 100644 index 00000000000..80bf39ecb48 --- /dev/null +++ b/doc/install/azure/img/azure-create-virtual-machine-basics-password.png diff --git a/doc/install/azure/img/azure-create-virtual-machine-basics.png b/doc/install/azure/img/azure-create-virtual-machine-basics.png Binary files differnew file mode 100644 index 00000000000..229c073fe17 --- /dev/null +++ b/doc/install/azure/img/azure-create-virtual-machine-basics.png diff --git a/doc/install/azure/img/azure-create-virtual-machine-deployment.png b/doc/install/azure/img/azure-create-virtual-machine-deployment.png Binary files differnew file mode 100644 index 00000000000..5cfdd973a58 --- /dev/null +++ b/doc/install/azure/img/azure-create-virtual-machine-deployment.png diff --git a/doc/install/azure/img/azure-create-virtual-machine-purchase.png b/doc/install/azure/img/azure-create-virtual-machine-purchase.png Binary files differnew file mode 100644 index 00000000000..f4de62299f1 --- /dev/null +++ b/doc/install/azure/img/azure-create-virtual-machine-purchase.png diff --git a/doc/install/azure/img/azure-create-virtual-machine-settings.png b/doc/install/azure/img/azure-create-virtual-machine-settings.png Binary files differnew file mode 100644 index 00000000000..20097921660 --- /dev/null +++ b/doc/install/azure/img/azure-create-virtual-machine-settings.png diff --git a/doc/install/azure/img/azure-create-virtual-machine-size.png b/doc/install/azure/img/azure-create-virtual-machine-size.png Binary files differnew file mode 100644 index 00000000000..a408394151f --- /dev/null +++ b/doc/install/azure/img/azure-create-virtual-machine-size.png diff --git a/doc/install/azure/img/azure-dashboard-highlight-nsg.png b/doc/install/azure/img/azure-dashboard-highlight-nsg.png Binary files differnew file mode 100644 index 00000000000..025efd33977 --- /dev/null +++ b/doc/install/azure/img/azure-dashboard-highlight-nsg.png diff --git a/doc/install/azure/img/azure-dashboard-running-resources.png b/doc/install/azure/img/azure-dashboard-running-resources.png Binary files differnew file mode 100644 index 00000000000..7e661a6aa65 --- /dev/null +++ b/doc/install/azure/img/azure-dashboard-running-resources.png diff --git a/doc/install/azure/img/azure-dashboard.png b/doc/install/azure/img/azure-dashboard.png Binary files differnew file mode 100644 index 00000000000..375ec8622b8 --- /dev/null +++ b/doc/install/azure/img/azure-dashboard.png diff --git a/doc/install/azure/img/azure-inbound-sec-rules-list.png b/doc/install/azure/img/azure-inbound-sec-rules-list.png Binary files differnew file mode 100644 index 00000000000..1667671b21d --- /dev/null +++ b/doc/install/azure/img/azure-inbound-sec-rules-list.png diff --git a/doc/install/azure/img/azure-new-gitlab-ce.png b/doc/install/azure/img/azure-new-gitlab-ce.png Binary files differnew file mode 100644 index 00000000000..88949f69a94 --- /dev/null +++ b/doc/install/azure/img/azure-new-gitlab-ce.png diff --git a/doc/install/azure/img/azure-new-search-gitlab.png b/doc/install/azure/img/azure-new-search-gitlab.png Binary files differnew file mode 100644 index 00000000000..f96ed577d62 --- /dev/null +++ b/doc/install/azure/img/azure-new-search-gitlab.png diff --git a/doc/install/azure/img/azure-nsg-inbound-sec-rules-add-highlight.png b/doc/install/azure/img/azure-nsg-inbound-sec-rules-add-highlight.png Binary files differnew file mode 100644 index 00000000000..c9a08b87ce6 --- /dev/null +++ b/doc/install/azure/img/azure-nsg-inbound-sec-rules-add-highlight.png diff --git a/doc/install/azure/img/azure-nsg-inbound-sec-rules-highlight.png b/doc/install/azure/img/azure-nsg-inbound-sec-rules-highlight.png Binary files differnew file mode 100644 index 00000000000..6423625ca8b --- /dev/null +++ b/doc/install/azure/img/azure-nsg-inbound-sec-rules-highlight.png diff --git a/doc/install/azure/img/azure-vm-domain-name.png b/doc/install/azure/img/azure-vm-domain-name.png Binary files differnew file mode 100644 index 00000000000..01c03004b24 --- /dev/null +++ b/doc/install/azure/img/azure-vm-domain-name.png diff --git a/doc/install/azure/img/azure-vm-management-public-ip.png b/doc/install/azure/img/azure-vm-management-public-ip.png Binary files differnew file mode 100644 index 00000000000..ef313641db8 --- /dev/null +++ b/doc/install/azure/img/azure-vm-management-public-ip.png diff --git a/doc/install/azure/img/azure-vm-management-settings-network-interfaces.png b/doc/install/azure/img/azure-vm-management-settings-network-interfaces.png Binary files differnew file mode 100644 index 00000000000..4ff10718059 --- /dev/null +++ b/doc/install/azure/img/azure-vm-management-settings-network-interfaces.png diff --git a/doc/install/azure/img/azure-vm-management.png b/doc/install/azure/img/azure-vm-management.png Binary files differnew file mode 100644 index 00000000000..a0e0067258c --- /dev/null +++ b/doc/install/azure/img/azure-vm-management.png diff --git a/doc/install/azure/img/gitlab-admin-area-9.4.0.png b/doc/install/azure/img/gitlab-admin-area-9.4.0.png Binary files differnew file mode 100644 index 00000000000..b7ee4598731 --- /dev/null +++ b/doc/install/azure/img/gitlab-admin-area-9.4.0.png diff --git a/doc/install/azure/img/gitlab-admin-area.png b/doc/install/azure/img/gitlab-admin-area.png Binary files differnew file mode 100644 index 00000000000..028f0b0a0eb --- /dev/null +++ b/doc/install/azure/img/gitlab-admin-area.png diff --git a/doc/install/azure/img/gitlab-change-password.png b/doc/install/azure/img/gitlab-change-password.png Binary files differnew file mode 100644 index 00000000000..7350d604d56 --- /dev/null +++ b/doc/install/azure/img/gitlab-change-password.png diff --git a/doc/install/azure/img/gitlab-home.png b/doc/install/azure/img/gitlab-home.png Binary files differnew file mode 100644 index 00000000000..7c935805ea2 --- /dev/null +++ b/doc/install/azure/img/gitlab-home.png diff --git a/doc/install/azure/img/gitlab-login.png b/doc/install/azure/img/gitlab-login.png Binary files differnew file mode 100644 index 00000000000..95fe5aec1e0 --- /dev/null +++ b/doc/install/azure/img/gitlab-login.png diff --git a/doc/install/azure/img/gitlab-new-project.png b/doc/install/azure/img/gitlab-new-project.png Binary files differnew file mode 100644 index 00000000000..3b6b9a81682 --- /dev/null +++ b/doc/install/azure/img/gitlab-new-project.png diff --git a/doc/install/azure/img/gitlab-project-home-empty.png b/doc/install/azure/img/gitlab-project-home-empty.png Binary files differnew file mode 100644 index 00000000000..54d0b36a251 --- /dev/null +++ b/doc/install/azure/img/gitlab-project-home-empty.png diff --git a/doc/install/azure/img/gitlab-project-home-instructions.png b/doc/install/azure/img/gitlab-project-home-instructions.png Binary files differnew file mode 100644 index 00000000000..fd040d33e95 --- /dev/null +++ b/doc/install/azure/img/gitlab-project-home-instructions.png diff --git a/doc/install/azure/img/gitlab-ssh-update-in-progress.png b/doc/install/azure/img/gitlab-ssh-update-in-progress.png Binary files differnew file mode 100644 index 00000000000..8a686ebde26 --- /dev/null +++ b/doc/install/azure/img/gitlab-ssh-update-in-progress.png diff --git a/doc/install/azure/index.md b/doc/install/azure/index.md new file mode 100644 index 00000000000..9cc4b56c932 --- /dev/null +++ b/doc/install/azure/index.md @@ -0,0 +1,439 @@ +# Install GitLab on Microsoft Azure + +> _This article was originally written by Dave Wentzel and [published on the GitLab Blog][Original-Blog-Post]._ +> +> _Ported to the GitLab documentation and updated on 2017-08-24 by [Ian Scorer](https://gitlab.com/iscorer)._ + +Azure is Microsoft's business cloud and GitLab is a pre-configured offering on the Azure Marketplace. +Hopefully, you aren't surprised to hear that Microsoft and Azure have embraced open source software +like Ubuntu, Red Hat Enterprise Linux, and of course - GitLab! This means that you can spin up a +pre-configured GitLab VM and have your very own private GitLab up and running in around 30 minutes. +Let's get started. + +## Getting started + +First, you'll need an account on Azure. There are three ways to do this: + +- If your company (or you) already has an account, then you are ready to go! +- You can also open your own Azure account for free. _At time of writing_, you get $200 +of credit to spend on Azure services for 30 days. You can use this credit to try out paid Azure +services, exploring Microsoft's cloud for free. Even after the first 30 days, you never have to pay +anything unless you decide to transition to paid services with a Pay-As-You-Go Azure subscription. +This is a great way to try out Azure and cloud computing, and you can +[read more in their comprehensive FAQ][Azure-Free-Account-FAQ]. +- If you have an MSDN subscription, you can activate your Azure subscriber benefits. Your MSDN +subscription gives you recurring Azure credits every month, so why not put those credits to use and +try out GitLab right now? + +## Working with Azure + +Once you have an Azure account, you can get started. Login to Azure using +[portal.azure.com](https://portal.azure.com) and the first thing you will see is the Dashboard: + + + +The Dashboard gives you a quick overview of Azure resources, and from here you you can build VMs, +create SQL Databases, author websites, and perform lots of other cloud tasks. + +## Create New VM + +The [Azure Marketplace][Azure-Marketplace] is an online store for pre-configured applications and +services which have been optimized for the cloud by software vendors like GitLab, and both +the [Community Edition ("CE")][CE] and the [Enterprise Edition ("EE")][EE] versions of GitLab are +available on the Azure Marketplace as pre-configured solutions. + +To begin creating a new GitLab VM, click on the **+ New** icon, type "GitLab" into the search +box, and then click the **"GitLab Community Edition"** search result: + + + +A new "blade" window will pop-out, where you can read more about the **"GitLab Community Edition"** +offering which is freely available under the MIT Expat License: + + + +Click **"Create"** and you will be presented with the "Create virtual machine" blade: + + + +## Basics + +The first items we need to configure are the basic settings of the underlying virtual machine: + +1. Enter a `Name` for the VM - e.g. **"GitLab-CE"** +1. Select a `VM disk type` - either **HDD** _(slower, lower cost)_ or **SSD** _(faster, higher cost)_ +1. Enter a `User name` - e.g. **"gitlab-admin"** +1. Select an `Authentication type`, either **SSH public key** or **Password**: + + >**Note:** if you're unsure which authentication type to use, select **Password** + + 1. If you chose **SSH public key** - enter your `SSH public key` into the field provided + _(read the [SSH documentation][GitLab-Docs-SSH] to learn more about how to setup SSH + public keys)_ + 1. If you chose **Password** - enter the password you wish to use _(this is the password that you + will use later in this tutorial to [SSH] into the VM, so make sure it's a strong password/passphrase)_ +1. Choose the appropriate `Subscription` tier for your Azure account +1. Choose an existing `Resource Group` or create a new one - e.g. **"GitLab-CE-Azure"** +>**Note:** a "Resource group" is a way to group related resources together for easier administration. +We chose "GitLab-CE-Azure", but your resource group can have the same name as your VM. +1. Choose a `Location` - if you're unsure, select the default location + +Here are the settings we've used: + + + +Check the settings you have entered, and then click **"OK"** when you're ready to proceed. + +## Size + +Next, you need to choose the size of your VM - selecting features such as the number of CPU cores, +the amount of RAM, the size of storage (and its speed), etc. + +>**Note:** in common with other cloud vendors, Azure operates a resource/usage pricing model, i.e. +the more resources your VM consumes the more it will cost you to run, so make your selection +carefully. You'll see that Azure provides an _estimated_ monthly cost beneath each VM Size to help +guide your selection. + +The default size - the lowest cost **"DS1_V2 Standard"** VM - meets the minimum system requirements +to run a small GitLab environment for testing and evaluation purposes, and so we're going to go +ahead and select this one, but please choose the size which best meets your own requirements: + + + +>**Note:** be aware that whilst your VM is active (known as "allocated"), it will incur +"compute charges" which, ultimately, you will be billed for. So, even if you're using the +free trial credits, you'll likely want to learn +[how to properly shutdown an Azure VM to save money][Azure-Properly-Shutdown-VM]. + +Go ahead and click your chosen size, then click **"Select"** when you're ready to proceed to the +next step. + +## Settings + +On the next blade, you're asked to configure the Storage, Network and Extension settings. +We've gone with the default settings as they're sufficient for test-driving GitLab, but please +choose the settings which best meet your own requirements: + + + +Review the settings and then click **"OK"** when you're ready to proceed to the last step. + +## Purchase + +The Purchase page is the last step and here you will be presented with the price per hour for your +new VM. You'll be billed only for the VM itself (e.g. "Standard DS1 v2") because the +**"GitLab Community Edition"** marketplace solution is free to use at 0 USD/hr: + + + +>**Note:** at this stage, you can review and modify the any of the settings you have made during all +previous steps, just click on any of the four steps to re-open them. + +When you have read and agreed to the terms of use and are ready to proceed, click **"Purchase"**. + +## Deployment + +At this point, Azure will begin deploying your new VM. The deployment process will take a few +minutes to complete, with progress displayed on the **"Deployment"** blade: + + + +Once the deployment process is complete, the new VM and its associated resources will be displayed +on the Azure Dashboard (you may need to refresh the page): + + + +The new VM can also be accessed by clicking the `All resources` or `Virtual machines` icons in the +Azure Portal sidebar navigation menu. + +## Setup a domain name + +The VM will have a public IP address (static by default), but Azure allows us to assign a friendly +DNS name to the VM, so let's go ahead and do that. + +From the Dashboard, click on the **"GitLab-CE"** tile to open the management blade for the new VM. +The public IP address that the VM uses is shown in the 'Essentials' section: + + + +Click on the public IP address - which should open the **"Public IP address - Configuration"** blade, +then click on **"Configuration"** (under "Settings"). Now enter a friendly DNS name for your instance +in the `DNS name label` field: + + + +In the screenshot above, you'll see that we've set the `DNS name label` to **"gitlab-ce-test"**. +This will make our VM accessible at `gitlab-ce-test.centralus.cloudapp.azure.com` +_(the full domain name of your own VM will be different, of course)_. + +Click **"Save"** for the changes to take effect. + +>**Note:** if you want to use your own domain name, you will need to add a DNS `A` record at your +domain registrar which points to the public IP address of your Azure VM. If you do this, you'll need +to make sure your VM is configured to use a _static_ public IP address (i.e. not a _dynamic_ one) +or you will have to reconfigure the DNS `A` record each time Azure reassigns your VM a new public IP +address. Read [IP address types and allocation methods in Azure][Azure-IP-Address-Types] to learn more. + +## Let's open some ports! + +At this stage you should have a running and fully operational VM. However, none of the services on +your VM (e.g. GitLab) will be publicly accessible via the internet until you have opened up the +neccessary ports to enable access to those services. + +Ports are opened by adding _security rules_ to the **"Network security group"** (NSG) which our VM +has been assigned to. If you followed the process above, then Azure will have automatically created +an NSG named `GitLab-CE-nsg` and assigned the `GitLab-CE` VM to it. + +>**Note:** if you gave your VM a different name then the NSG automatically created by Azure will +also have a different name - the name you have your VM, with `-nsg` appended to it. + +You can navigate to the NSG settings via many different routes in the Azure Portal, but one of the +simplest ways is to go to the Azure Dashboard, and then click on the Network Security Group listed +in the **"All resources"** tile: + + + +With the **"Network security group"** blade open, click on **"Inbound security rules"** under +**"Settings"**: + + + +Next, click **"Add"**: + + + +### Which ports to open? + +Like all servers, our VM will be running many services. However, we want to open up the correct +ports to enable public internet access to two services in particular: + +1. **HTTP** (port 80) - opening port 80 will enable our VM to respond to HTTP requests, allowing +public access to the instance of GitLab running on our VM. +1. **SSH** (port 22) - opening port 22 will enable our VM to respond to SSH connection requests, +allowing public access (with authentication) to remote terminal sessions +_(you'll see why we need [SSH] access to our VM [later on in this tutorial](#maintaining-your-gitlab-instance))_ + +### Open HTTP on Port 80 + +In the **"Add inbound security rule"** blade, let's open port 80 so that our VM will accept HTTP +connections: + + + +1. Enter **"HTTP"** in the `Name` field +1. Select **HTTP** from the options in the `Service` drop-down +1. Make sure the `Action` is set to **Allow** +1. Click **"OK"** + +### Open SSH on Port 22 + +Repeat the above process, adding a second Inbound security rule to open port 22, enabling our VM to +accept [SSH] connections: + + + +1. Enter **"SSH"** in the `Name` field +1. Select **SSH** from the options in the `Service` drop-down +1. Make sure the `Action` is set to **Allow** +1. Click **"OK"** + + +It will take a moment for Azure to add each new Inbound Security Rule (and you may need to click on +**"Inbound security rules"** to refresh the list), but once completed, you should see the two new +rules in the list: + + + +## Connecting to GitLab +Use the domain name you set up earlier (or the public IP address) to visit your new GitLab instance +in your browser. If everything has gone according to plan you should be presented with the +following page, asking you to set a _new_ password for the administrator account automatically +created by GitLab: + + + +Enter your _new_ password into both form fields, and then click **"Change your password"**. + +Once you have changed the password you will be redirected to the GitLab login page. Use `root` as +the username, enter the new password you set in the previous step, and then click **"Sign in"**: + + + +### Success? + +After signing in successfully, you should see the GitLab Projects page displaying a +**"Welcome to GitLab!"** message: + + + +If so, you now have a working GitLab instance on your own private Azure VM. **Congratulations!** + +## Creating your first GitLab project + +You can skip this section if you are familiar with Git and GitLab. Otherwise, let's create our first +project. From the Welcome page, click **"New Project"**. + +Let's give our project a name and a description, and then accept the default values for everything +else: + +1. Enter **"demo"** into the `Project path` project name field +1. Enter a `description`, e.g. **"My awesome demo project!"** +1. Click **"Create project"** + + + +Once the new project has been created (which should only take a moment), you'll be redirected to +homepage for the project: + + + +If you scroll further down the project's home page, you'll see some basic instructions on how to +setup a local clone of your new repository and push and pull from it: + + + +**That's it! You now have your own private GitLab environment installed and running in the cloud!** + +## Maintaining your GitLab instance + +It's important to keep your GitLab environment up-to-date. The GitLab team is constantly making +enhancements and occasionally you may need to update for security reasons. So let's review how to +update GitLab. + +### Checking our current version + +To check which version of GitLab we're currently running, click on the "Admin Area" link - it's the +the wrench icon displayed in the top-right, next to the search box. + +In the following screenshot you can see an **"update asap"** notification message in the top-right. +This particular message indicates that there is a newer version of GitLab available which contains +one or more security fixes: + + + +Under the **"Components"** section, we can see that our VM is currently running version `8.6.5` of +GitLab. This is the version of GitLab which was contained in the Azure Marketplace +**"GitLab Community Edition"** offering we used to build the VM when we wrote this tutorial. + +>**Note:** The version of GitLab in your own VM instance may well be different, but the update +process will still be the same. + +### Connect via SSH + +To perform an update, we need to connect directly to our Azure VM instance and run some commands +from the terminal. Our Azure VM is actually a server running Linux (Ubuntu), so we'll need to +connect to it using SSH ([Secure Shell][SSH]). + +If you're running Windows, you'll need to connect using [PuTTY] or an equivalent Windows SSH client. +If you're running Linux or macOS, then you already have an SSH client installed. + +>**Note:** +- Remember that you will need to login with the username and password you specified +[when you created](#basics) your Azure VM +- If you need to reset your VM password, read +[how to reset SSH credentials for a user on an Azure VM][Azure-Troubleshoot-SSH-Connection]. + +#### SSH from the command-line + +If you're running [SSH] from the command-line (terminal), then type in the following command to +connect to your VM, substituting `username` and `your-azure-domain-name.com` for the correct values. + +Again, remember that your Azure VM domain name will be the one you +[setup previously in the tutorial](#set-up-a-domain-name). If you didn't setup a domain name for +your VM, you can use the IP address in its place in the following command: + +```bash +ssh username@your-azure-domain-name.com +``` +Provide your password at the prompt to authenticate. + +#### SSH from Windows (PuTTY) + +If you're using [PuTTY] in Windows as your [SSH] client, then you might want to take a quick +read on [using PuTTY in Windows][Using-SSH-In-Putty]. + +### Updating GitLab + +Once you've logged in via SSH, enter the following command to update GitLab to the latest +version: + +```bash +sudo apt-get update && sudo apt-get install gitlab-ce +``` + +This command will update GitLab and its associated components to the latest versions, so it will +take a little time to complete. You'll see various update tasks being completed in your SSH +terminal window: + + + +Once the update process has completed, you'll see a message like this: + +``` +Upgrade complete! If your GitLab server is misbehaving try running + + sudo gitlab-ctl restart + +before anything else. +``` + +#### Check out your updated GitLab + +Refresh your GitLab instance in the browser and navigate to the Admin Area. You should now have an +up-to-date GitLab instance. + +When we wrote this tutorial our Azure VM GitLab instance was updated to the latest version at time +of writing (`9.4.0`). You can see that the message which was previously displaying **"update asap"** +is now showing **"up-to-date"**: + + + +## Conclusion + +Naturally, we believe that GitLab is a great git repository tool. However, GitLab is a whole lot +more than that too. GitLab unifies issues, code review, CI and CD into a single UI, helping you to +move faster from idea to production, and in this tutorial we showed you how quick and easy it is to +setup and run your own instance of GitLab on Azure, Microsoft's cloud service. + +Azure is a great way to experiment with GitLab, and if you decide (as we hope) that GitLab is for +you, you can continue to use Azure as your secure, scalable cloud provider or of course run GitLab +on any cloud service you choose. + +## Where to next? + +Check out our other [Technical Articles][GitLab-Technical-Articles] or browse the [GitLab Documentation][GitLab-Docs] to learn more about GitLab. + +### Useful links + +- [GitLab Community Edition][CE] +- [GitLab Enterprise Edition][EE] +- [Microsoft Azure][Azure] + - [Azure - Free Account FAQ][Azure-Free-Account-FAQ] + - [Azure - Marketplace][Azure-Marketplace] + - [Azure Portal][Azure-Portal] + - [Azure - Pricing Calculator][Azure-Pricing-Calculator] + - [Azure - Troubleshoot SSH Connections to an Azure Linux VM][Azure-Troubleshoot-SSH-Connection] + - [Azure - Properly Shutdown an Azure VM][Azure-Properly-Shutdown-VM] +- [SSH], [PuTTY] and [Using SSH in PuTTY][Using-SSH-In-Putty] + +[Original-Blog-Post]: https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/ "How to Setup a GitLab Instance on Microsoft Azure" +[GitLab-Docs]: https://docs.gitlab.com/ce/README.html "GitLab Documentation" +[GitLab-Technical-Articles]: https://docs.gitlab.com/ce/articles/index.html "GitLab Technical Articles" +[GitLab-Docs-SSH]: https://docs.gitlab.com/ce/ssh/README.html "GitLab Documentation: SSH" +[CE]: https://about.gitlab.com/features/ +[EE]: https://about.gitlab.com/features/#ee-starter + +[Azure-Troubleshoot-Linux-VM]: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/troubleshoot-app-connection "Troubleshoot application connectivity issues on a Linux virtual machine in Azure" +[Azure-IP-Address-Types]: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-ip-addresses-overview-arm "IP address types and allocation methods in Azure" +[Azure-How-To-Open-Ports]: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/nsg-quickstart-portal "How to open ports to a virtual machine with the Azure portal" +[Azure-Troubleshoot-SSH-Connection]: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/troubleshoot-ssh-connection "Troubleshoot SSH connections to an Azure Linux VM" +[Azure]: https://azure.microsoft.com/en-us/ +[Azure-Free-Account-FAQ]: https://azure.microsoft.com/en-us/free/free-account-faq/ +[Azure-Marketplace]: https://azure.microsoft.com/en-us/marketplace/ +[Azure-Portal]: https://portal.azure.com +[Azure-Pricing-Calculator]: https://azure.microsoft.com/en-us/pricing/calculator/ +[Azure-Properly-Shutdown-VM]: https://buildazure.com/2017/03/16/properly-shutdown-azure-vm-to-save-money/ "Properly Shutdown an Azure VM to Save Money" + +[SSH]: https://en.wikipedia.org/wiki/Secure_Shell +[PuTTY]: http://www.putty.org/ +[Using-SSH-In-Putty]: https://mediatemple.net/community/products/dv/204404604/using-ssh-in-putty-
\ No newline at end of file diff --git a/doc/install/docker.md b/doc/install/docker.md new file mode 100644 index 00000000000..933756072ff --- /dev/null +++ b/doc/install/docker.md @@ -0,0 +1,18 @@ +# GitLab Docker images + +[Docker](https://www.docker.com) and container technology have been revolutionizing the software world for the past few years. They combine the performance and efficiency of native execution with the abstraction, security, and immutability of virtualization. + +GitLab provides official Docker images to allowing you to easily take advantage of the benefits of containerization while operating your GitLab instance. + +## Omnibus GitLab based images + +GitLab maintains a set of [official Docker images](https://hub.docker.com/r/gitlab) based on our [Omnibus GitLab package](https://docs.gitlab.com/omnibus/README.html). These images include: +* [GitLab Community Edition](https://hub.docker.com/r/gitlab/gitlab-ce/) +* [GitLab Enterprise Edition](https://hub.docker.com/r/gitlab/gitlab-ee/) +* [GitLab Runner](https://hub.docker.com/r/gitlab/gitlab-runner/) + +A [complete usage guide](https://docs.gitlab.com/omnibus/docker/) to these images is available, as well as the [Dockerfile used for building the images](https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master/docker). + +## Cloud native images + +GitLab is also working towards a [cloud native set of containers](https://gitlab.com/charts/helm.gitlab.io#docker-container-images), with a single image for each component service. We intend for these images to eventually replace the [Omnibus GitLab based images](#omnibus-gitlab-based-images). diff --git a/doc/install/installation.md b/doc/install/installation.md index 22aedb5403e..66eb7675896 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -66,6 +66,9 @@ Install the required packages (needed to compile Ruby and native extensions to R sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake +Ubuntu 14.04 (Trusty Tahr) doesn't have the `libre2-dev` package available, but +you can [install re2 manually](https://github.com/google/re2/wiki/Install). + If you want to use Kerberos for user authentication, then install libkrb5-dev: sudo apt-get install libkrb5-dev @@ -77,7 +80,7 @@ Make sure you have the right version of Git installed # Install Git sudo apt-get install -y git-core - # Make sure Git is version 2.8.4 or higher + # Make sure Git is version 2.13.0 or higher git --version Is the system packaged Git too old? Remove it and compile from source. @@ -296,9 +299,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 9-4-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 9-5-stable gitlab -**Note:** You can change `9-4-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `9-5-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It @@ -507,15 +510,17 @@ Check if GitLab and its environment are configured correctly: sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +### Compile GetText PO files + + sudo -u git -H bundle exec rake gettext:pack RAILS_ENV=production + sudo -u git -H bundle exec rake gettext:po_to_json RAILS_ENV=production + ### Compile Assets sudo -u git -H yarn install --production --pure-lockfile sudo -u git -H bundle exec rake gitlab:assets:compile RAILS_ENV=production NODE_ENV=production -### Compile GetText PO files - - sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production - ### Start Your GitLab Instance sudo service gitlab start diff --git a/doc/install/kubernetes/gitlab_chart.md b/doc/install/kubernetes/gitlab_chart.md index d2442a4fbde..a339bc23809 100644 --- a/doc/install/kubernetes/gitlab_chart.md +++ b/doc/install/kubernetes/gitlab_chart.md @@ -1,9 +1,9 @@ # GitLab Helm Chart -> Officially supported cloud providers are Google Container Service and Azure Container Service. - -> Officially supported schedulers are Kubernetes and Terraform. +> **Note:** +* > **Note**: This chart will be replaced by the [gitlab-omnibus](gitlab_omnibus.md) chart, once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). +* Officially supported cloud providers are Google Container Service and Azure Container Service. -The `gitlab` Helm chart deploys GitLab into your Kubernetes cluster. +The `gitlab` Helm chart deploys just GitLab into your Kubernetes cluster, and offers extensive configuration options. For most deployments we recommended the [gitlab-omnibus](gitlab_omnibus.md) chart, This chart includes the following: @@ -22,9 +22,7 @@ This chart includes the following: - [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) provisioner support in the underlying infrastructure - The ability to point a DNS entry or URL at your GitLab install - The `kubectl` CLI installed locally and authenticated for the cluster -- The Helm Client installed locally -- The Helm Server (Tiller) already installed and running in the cluster, by running `helm init` -- The GitLab Helm Repo [added to your Helm Client](index.md#add-the-gitlab-helm-repository) +- The [Helm client](https://github.com/kubernetes/helm/blob/master/docs/quickstart.md) installed locally on your machine ## Configuring GitLab @@ -207,7 +205,7 @@ its class in an annotation. >**Note:** The Ingress alone doesn't expose GitLab externally. You need to have a Ingress controller setup to do that. Setting up an Ingress controller can be done by installing the `nginx-ingress` helm chart. But be sure -to read the [documentation](https://github.com/kubernetes/charts/blob/master/stable/nginx-ingress/README.md). +to read the [documentation](https://github.com/kubernetes/charts/blob/master/stable/nginx-ingress/README.md). >**Note:** If you would like to use the Registry, you will also need to ensure your Ingress supports a [sufficiently large request size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size). @@ -428,6 +426,13 @@ ingress: ## Installing GitLab using the Helm Chart > You may see a temporary error message `SchedulerPredicates failed due to PersistentVolumeClaim is not bound` while storage provisions. Once the storage provisions, the pods will automatically restart. This may take a couple minutes depending on your cloud provider. If the error persists, please review the [prerequisites](#prerequisites) to ensure you have enough RAM, CPU, and storage. +Add the GitLab Helm repository and initialize Helm: + +```bash +helm repo add gitlab https://charts.gitlab.io +helm init +``` + Once you [have configured](#configuration) GitLab in your `values.yml` file, run the following: diff --git a/doc/install/kubernetes/gitlab_omnibus.md b/doc/install/kubernetes/gitlab_omnibus.md new file mode 100644 index 00000000000..d7fd8613633 --- /dev/null +++ b/doc/install/kubernetes/gitlab_omnibus.md @@ -0,0 +1,171 @@ +# GitLab-Omnibus Helm Chart +> **Note:** +* This Helm chart is in beta, while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being worked on. +* GitLab is working on a [cloud native set of Charts](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md) which will eventually replace these. +* Officially supported cloud providers are Google Container Service and Azure Container Service. + +This work is based partially on: https://github.com/lwolf/kubernetes-gitlab/. GitLab would like to thank Sergey Nuzhdin for his work. + +## Introduction + +This chart provides an easy way to get started with GitLab, provisioning an installation with nearly all functionality enabled. SSL is automatically provisioned as well via [Let's Encrypt](https://letsencrypt.org/). + +The deployment includes: + +- A [GitLab Omnibus](https://docs.gitlab.com/omnibus/) Pod, including Mattermost, Container Registry, and Prometheus +- An auto-scaling [GitLab Runner](https://docs.gitlab.com/runner/) using the Kubernetes executor +- [Redis](https://github.com/kubernetes/charts/tree/master/stable/redis) +- [PostgreSQL](https://github.com/kubernetes/charts/tree/master/stable/postgresql) +- [NGINX Ingress](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) +- Persistent Volume Claims for Data, Registry, Postgres, and Redis + +A video demonstration of GitLab utilizing this chart [is available](https://about.gitlab.com/handbook/sales/demo/). + +## Prerequisites + +- _At least_ 4 GB of RAM available on your cluster. 41GB of storage and 2 CPU are also required. +- Kubernetes 1.4+ with Beta APIs enabled +- [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) provisioner support in the underlying infrastructure +- A [wildcard DNS entry](#networking-prerequisites), which resolves to the external IP address +- The `kubectl` CLI installed locally and authenticated for the cluster +- The [Helm client](https://github.com/kubernetes/helm/blob/master/docs/quickstart.md) installed locally on your machine + +### Networking Prerequisites + +This chart configures a GitLab server and Kubernetes cluster which can support dynamic [Review Apps](https://docs.gitlab.com/ee/ci/review_apps/index.html), as well as services like the integrated [Container Registry](https://docs.gitlab.com/ee/user/project/container_registry.html) and [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/). + +To support the GitLab services and dynamic environments, a wildcard DNS entry is required which resolves to the [Load Balancer](#load-balancer-ip) or [External IP](#external-ip). Configuration of the DNS entry will depend upon the DNS service being used. + +#### External IP (Recommended) + +To provision an external IP on GCP and Azure, simply request a new address from the Networking section. Ensure that the region matches the region your container cluster is created in. Note, it is important that the IP is not assigned at this point in time. It will be automatically assigned once the Helm chart is installed, and assigned to the Load Balancer. + +Now that an external IP address has been allocated, ensure that the wildcard DNS entry you would like to use resolves to this IP. Please consult the documentation for your DNS service for more information on creating DNS records. + +Finally, set the `baseIP` setting to this IP address when [deploying GitLab](#configuring-and-installing-gitlab). + +#### Load Balancer IP + +If you do not specify a `baseIP`, an ephemeral IP will be assigned to the Load Balancer or Ingress. You can retrieve this IP by running the following command *after* deploying GitLab: + +`kubectl get svc -w --namespace nginx-ingress nginx` + +The IP address will be displayed in the `EXTERNAL-IP` field, and should be used to configure the Wildcard DNS entry. For more information on creating a wildcard DNS entry, consult the documentation for the DNS server you are using. + +For production deployments of GitLab, we strongly recommend using an [External IP](#external-ip). + +## Configuring and Installing GitLab + +For most installations, only two parameters are required: +- `baseDomain`: the [base domain](#networking-prerequisites) of the wildcard host entry. For example, `mycompany.io` if the wild card entry is `*.mycompany.io`. +- `legoEmail`: Email address to use when requesting new SSL certificates from Let's Encrypt. + +Other common configuration options: +- `baseIP`: the desired [external IP address](#external-ip-recommended) +- `gitlab`: Choose the [desired edition](https://about.gitlab.com/products), either `ee` or `ce`. `ce` is the default. +- `gitlabEELicense`: For Enterprise Edition, the [license](https://docs.gitlab.com/ee/user/admin_area/license.html) can be installed directly via the Chart +- `provider`: Optimizes the deployment for a cloud provider. The default is `gke` for [Google Container Engine](https://cloud.google.com/container-engine/), with `acs` also supported for the [Azure Container Service](https://azure.microsoft.com/en-us/services/container-service/). + +For additional configuration options, consult the [values.yaml](https://gitlab.com/charts/charts.gitlab.io/blob/master/charts/gitlab-omnibus/values.yaml). + +### Choosing a different GitLab release version + +The version of GitLab installed is based on the `gitlab` setting (see [section](#choosing-gitlab-edition) above), and +the value of the corresponding helm setting: `gitlabCEImage` or `gitabEEImage`. + +```yaml +gitlab: CE +gitlabCEImage: gitlab/gitlab-ce:9.5.2-ce.0 +gitlabEEImage: gitlab/gitlab-ee:9.5.2-ee.0 +``` + +The different images can be found in the [gitlab-ce](https://hub.docker.com/r/gitlab/gitlab-ce/tags/) and [gitlab-ee](https://hub.docker.com/r/gitlab/gitlab-ee/tags/) +repositories on Docker Hub. + +### Persistent storage +> **Note:** +If you are using a machine type with support for less than 4 attached disks, like an Azure trial, you should disable dedicated storage for Postgres and Redis. + +By default, persistent storage is enabled for GitLab and the charts it depends +on (Redis and PostgreSQL). + +Components can have their claim size set from your `values.yaml`, along with whether to provision separate storage for Postgres and Redis. + +Basic configuration: + +```yaml +redisImage: redis:3.2.10 +redisDedicatedStorage: true +redisStorageSize: 5Gi +postgresImage: postgres:9.6.3 +# If you disable postgresDedicatedStorage, you should consider bumping up gitlabRailsStorageSize +postgresDedicatedStorage: true +postgresStorageSize: 30Gi +gitlabRailsStorageSize: 30Gi +gitlabRegistryStorageSize: 30Gi +gitlabConfigStorageSize: 1Gi +``` + +### Routing and SSL + +Ingress routing and SSL are automatically configured within this Chart. An NGINX ingress is provisioned and configured, and will route traffic to any service. SSL certificates are automatically created and configured by [kube-lego](https://github.com/kubernetes/charts/tree/master/stable/kube-lego). + +> **Note:** +Let's Encrypt limits a single TLD to five certificate requests within a single week. This means that common DNS wildcard services like [xip.io](http://xip.io) and [nip.io](http://nip.io) are unlikely to work. + +## Installing GitLab using the Helm Chart +> **Note:** +You may see a temporary error message `SchedulerPredicates failed due to PersistentVolumeClaim is not bound` while storage provisions. Once the storage provisions, the pods will automatically start. This may take a couple minutes depending on your cloud provider. If the error persists, please review the [prerequisites](#prerequisites) to ensure you have enough RAM, CPU, and storage. + +Add the GitLab Helm repository and initialize Helm: + +```bash +helm repo add gitlab https://charts.gitlab.io +helm init +``` + +Once you have reviewed the [configuration settings](#configuring-and-installing-gitlab) you can install the chart. We recommending saving your configuration options in a `values.yaml` file for easier upgrades in the future. + +For example: + +```bash +helm install --name gitlab -f values.yaml gitlab/gitlab-omnibus +``` + +or passing them on the command line: + +```bash +helm install --name gitlab --set baseDomain=gitlab.io,baseIP=1.1.1.1,gitlab=ee,gitlabEELicense=$LICENSE,legoEmail=email@gitlab.com gitlab/gitlab-omnibus +``` + +## Updating GitLab using the Helm Chart + +Once your GitLab Chart is installed, configuration changes and chart updates +should we done using `helm upgrade` + +```bash +helm upgrade -f <CONFIG_VALUES_FILE> <RELEASE-NAME> gitlab/gitlab +``` + +where: + +- `<CONFIG_VALUES_FILE>` is the path to values file containing your custom + [configuration] (#configuring-and-installing-gitlab). +- `<RELEASE-NAME>` is the name you gave the chart when installing it. + In the [Install section](#installing-gitlab-using-the-helm-chart) we called it `gitlab`. + +## Uninstalling GitLab using the Helm Chart + +To uninstall the GitLab Chart, run the following: + +```bash +helm delete <RELEASE-NAME> +``` + +where: + +- `<RELEASE-NAME>` is the name you gave the chart when installing it. + In the [Install section](#installing) we called it `gitlab`. + +[kube-srv]: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types +[storageclass]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#storageclasses diff --git a/doc/install/kubernetes/gitlab_runner_chart.md b/doc/install/kubernetes/gitlab_runner_chart.md index 515b2841d08..d31c763ed64 100644 --- a/doc/install/kubernetes/gitlab_runner_chart.md +++ b/doc/install/kubernetes/gitlab_runner_chart.md @@ -1,7 +1,6 @@ # GitLab Runner Helm Chart -> Officially supported cloud providers are Google Container Service and Azure Container Service. - -> Officially supported schedulers are Kubernetes and Terraform. +> **Note:** +Officially supported cloud providers are Google Container Service and Azure Container Service. The `gitlab-runner` Helm chart deploys a GitLab Runner instance into your Kubernetes cluster. @@ -17,9 +16,7 @@ This chart configures the Runner to: - Your GitLab Server's API is reachable from the cluster - Kubernetes 1.4+ with Beta APIs enabled - The `kubectl` CLI installed locally and authenticated for the cluster -- The Helm Client installed locally -- The Helm Server (Tiller) already installed and running in the cluster, by running `helm init` -- The GitLab Helm Repo added to your Helm Client. See [Adding GitLab Helm Repo](index.md#add-the-gitlab-helm-repository) +- The [Helm client](https://github.com/kubernetes/helm/blob/master/docs/quickstart.md) installed locally on your machine ## Configuring GitLab Runner using the Helm Chart @@ -36,6 +33,8 @@ In order for GitLab Runner to function, your config file **must** specify the fo - `runnerRegistrationToken` - The Registration Token for adding new Runners to the GitLab Server. This must be retrieved from your GitLab Instance. See the [GitLab Runner Documentation](../../ci/runners/README.md#creating-and-registering-a-runner) for more information. +Unless you need to specify additional configuration, you are [ready to install](#installing-gitlab-runner-using-the-helm-chart). + ### Other configuration The rest of the configuration is [documented in the `values.yaml`](https://gitlab.com/charts/charts.gitlab.io/blob/master/charts/gitlab-runner/values.yaml) in the chart repository. @@ -115,6 +114,17 @@ runners: ``` +### Controlling maximum Runner concurrency + +A single GitLab Runner deployed on Kubernetes is able to execute multiple jobs in parallel by automatically starting additional Runner pods. The [`concurrent` setting](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) controls the maximum number of pods allowed at a single time, and defaults to `10`. + +```yaml +## Configure the maximum number of concurrent jobs +## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section +## +concurrent: 10 +``` + ### Running Docker-in-Docker containers with GitLab Runners See [Running Privileged Containers for the Runners](#running-privileged-containers-for-the-runners) for how to enable it, @@ -190,6 +200,13 @@ certsSecretName: <SECRET NAME> ## Installing GitLab Runner using the Helm Chart +Add the GitLab Helm repository and initialize Helm: + +```bash +helm repo add gitlab https://charts.gitlab.io +helm init +``` + Once you [have configured](#configuration) GitLab Runner in your `values.yml` file, run the following: diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index 5ea08869a9b..fb6c0c2d263 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -1,47 +1,58 @@ # Installing GitLab on Kubernetes > Officially supported cloud providers are Google Container Service and Azure Container Service. -> Officially supported schedulers are Kubernetes, Terraform and Tectonic. - The easiest method to deploy GitLab in [Kubernetes](https://kubernetes.io/) is -to take advantage of the official GitLab Helm charts. [Helm] is a package +to take advantage of GitLab's Helm charts. [Helm] is a package management tool for Kubernetes, allowing apps to be easily managed via their Charts. A [Chart] is a detailed description of the application including how it should be deployed, upgraded, and configured. -The GitLab Helm repository is located at https://charts.gitlab.io. -You can report any issues related to GitLab's Helm Charts at +GitLab provides [official Helm Charts](#official-gitlab-helm-charts-recommended) which are the recommended way to run GitLab within Kubernetes. + +There are also two other sets of charts: +* Our [upcoming cloud native Charts](#upcoming-cloud-native-helm-charts), which are in development but will eventually replace the current official charts. +* [Community contributed charts](#community-contributed-helm-charts). These charts should be considered deprecated, in favor of the official charts. + +## Official GitLab Helm Charts + +These charts utilize our [GitLab Omnibus Docker images](https://docs.gitlab.com/omnibus/docker/README.html). You can report any issues and feedback related to these charts at https://gitlab.com/charts/charts.gitlab.io/issues. -Contributions and improvements are also very welcome. -## Prerequisites +### Deploying GitLab on Kubernetes +> **Note**: This chart will eventually be replaced by the [cloud native charts](#upcoming-cloud-native-helm-charts), which are presently in development. + +The best way to deploy GitLab on Kubernetes is to use the [gitlab-omnibus](gitlab_omnibus.md) chart. + +It includes everything needed to run GitLab, including: a [Runner](https://docs.gitlab.com/runner/), [Container Registry](https://docs.gitlab.com/ee/user/project/container_registry.html#gitlab-container-registry), [automatic SSL](https://github.com/kubernetes/charts/tree/master/stable/kube-lego), and an [Ingress](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). This chart is in beta while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being completed. + +### Deploying just the GitLab Runner + +To deploy just the [GitLab Runner](https://docs.gitlab.com/runner/), utilize the [gitlab-runner](gitlab_runner_chart.md) chart. + +It offers a quick way to configure and deploy the Runner on Kubernetes, regardless of where your GitLab server may be running. -To use the charts, the Helm tool must be installed and initialized. The best -place to start is by reviewing the [Helm Quick Start Guide][helm-quick]. +### Advanced deployment of GitLab +> **Note**: This chart will be replaced by the [gitlab-omnibus](gitlab_omnibus.md) chart, once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). -## Add the GitLab Helm repository +If advanced configuration of GitLab is required, the beta [gitlab](gitlab_chart.md) chart can be used which deploys the GitLab service along with optional Postgres and Redis. It offers extensive configuration, but requires deep knowledge of Kubernetes and Helm to use. -Once Helm has been installed, the GitLab chart repository must be added: +For most deployments we recommend using our [gitlab-omnibus](gitlab_omnibus.md) chart. -```bash -helm repo add gitlab https://charts.gitlab.io -``` +## Upcoming Cloud Native Helm Charts -After adding the repository, Helm must be re-initialized: +GitLab is working towards a building a [cloud native deployment method](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). A key part of this effort is to isolate each service into it's [own Docker container and Helm chart](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420), rather than utilizing the all-in-one container image of the [current charts](#official-gitlab-helm-charts-recommended). -```bash -helm init -``` +By offering individual containers and charts, we will be able to provide a number of benefits: +* Easier horizontal scaling of each service +* Smaller more efficient images +* Potential for rolling updates and canaries within a service +* and plenty more. -## Using the GitLab Helm Charts +This is a large project and will be worked on over the span of multiple releases. For the most up to date status and release information, please see our [tracking issue](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420). -GitLab makes available two Helm Charts, one for the GitLab server and another -for the Runner. More detailed information on installing and configuring each -Chart can be found below: +## Community Contributed Helm Charts -- [Install GitLab](gitlab_chart.md) -- [Install GitLab Runner](gitlab_runner_chart.md) +The community has also [contributed GitLab charts](https://github.com/kubernetes/charts/tree/master/stable/gitlab-ce) to the [Helm Stable Repository](https://github.com/kubernetes/charts#repository-structure). These charts should be considered [deprecated](https://github.com/kubernetes/charts/issues/1138) in favor of the [official Charts](#official-gitlab-helm-charts-recommended). [chart]: https://github.com/kubernetes/charts -[helm-quick]: https://github.com/kubernetes/helm/blob/master/docs/quickstart.md [helm]: https://github.com/kubernetes/helm/blob/master/README.md diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 141df55f6bc..f672b358096 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -82,7 +82,9 @@ errors during usage. We recommend having at least 2GB of swap on your server, even if you currently have enough available RAM. Having swap will help reduce the chance of errors occurring -if your available memory changes. +if your available memory changes. We also recommend [configuring the kernels swappiness setting](https://askubuntu.com/a/103916) +to a low value like `10` to make the most of your RAM while still having the swap +available when needed. Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about many you need of those. @@ -104,6 +106,10 @@ features of GitLab work with MySQL/MariaDB: See [issue #30472][30472] for more information. 1. GitLab Geo does [not support MySQL](https://docs.gitlab.com/ee/gitlab-geo/database.html#mysql-replication). 1. [Zero downtime migrations][zero] do not work with MySQL +1. GitLab [optimizes the loading of dashboard events](https://gitlab.com/gitlab-org/gitlab-ce/issues/31806) using [PostgreSQL LATERAL JOINs](https://blog.heapanalytics.com/postgresqls-powerful-new-join-type-lateral/). +1. In general, SQL optimized for PostgreSQL may run much slower in MySQL due to + differences in query planners. For example, subqueries that work well in PostgreSQL + may not be [performant in MySQL](https://dev.mysql.com/doc/refman/5.7/en/optimizing-subqueries.html) 1. We expect this list to grow over time. Existing users using GitLab with MySQL/MariaDB are advised to diff --git a/doc/integration/README.md b/doc/integration/README.md index d70b9a7f54b..09d96bdd338 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -13,7 +13,6 @@ Bitbucket.org account - [External issue tracker](external-issue-tracker.md) Redmine, JIRA, etc. - [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages - [JIRA](../user/project/integrations/jira.md) Integrate with the JIRA issue tracker -- [Koding](../administration/integration/koding.md) Configure Koding to use IDE integration - [LDAP](ldap.md) Set up sign in via LDAP - [OAuth2 provider](oauth_provider.md) OAuth2 application creation - [OmniAuth](omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google, Bitbucket, Facebook, Shibboleth, SAML, Crowd, Azure and Authentiq ID diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index 6c11f46a70a..0e20b8096e9 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -224,3 +224,21 @@ By default Sign In is enabled via all the OAuth Providers that have been configu In order to enable/disable an OmniAuth provider, go to Admin Area -> Settings -> Sign-in Restrictions section -> Enabled OAuth Sign-In sources and select the providers you want to enable or disable.  + + +## Keep OmniAuth user profiles up to date + +You can enable profile syncing from selected OmniAuth providers and for all or for specific user information. + + ```ruby + gitlab_rails['sync_profile_from_provider'] = ['twitter', 'google_oauth2'] + gitlab_rails['sync_profile_attributes'] = ['name', 'email', 'location'] + ``` + + **For installations from source** + + ```yaml + omniauth: + sync_profile_from_provider: ['twitter', 'google_oauth2'] + sync_profile_claims_from_provider: ['email', 'location'] + ```
\ No newline at end of file diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 10f5ab3370d..ae69d7f92f2 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -144,9 +144,8 @@ gitlab_rails['backup_upload_connection'] = { 'region' => 'eu-west-1', 'aws_access_key_id' => 'AKIAKIAKI', 'aws_secret_access_key' => 'secret123' - # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty - # ie. 'aws_access_key_id' => '', - # 'use_iam_profile' => 'true' + # If using an IAM Profile, don't configure aws_access_key_id & aws_secret_access_key + # 'use_iam_profile' => true } gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' ``` diff --git a/doc/security/README.md b/doc/security/README.md index 38706e48ec5..0fea6be8b55 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -1,6 +1,7 @@ # Security - [Password length limits](password_length_limits.md) +- [Restrict SSH key technologies and minimum length](ssh_keys_restrictions.md) - [Rack attack](rack_attack.md) - [Webhooks and insecure internal web services](webhooks.md) - [Information exclusivity](information_exclusivity.md) diff --git a/doc/security/img/ssh_keys_restrictions_settings.png b/doc/security/img/ssh_keys_restrictions_settings.png Binary files differnew file mode 100644 index 00000000000..2e918fd4b3f --- /dev/null +++ b/doc/security/img/ssh_keys_restrictions_settings.png diff --git a/doc/security/ssh_keys_restrictions.md b/doc/security/ssh_keys_restrictions.md new file mode 100644 index 00000000000..213fa5bfef5 --- /dev/null +++ b/doc/security/ssh_keys_restrictions.md @@ -0,0 +1,19 @@ +# Restrict allowed SSH key technologies and minimum length + +`ssh-keygen` allows users to create RSA keys with as few as 768 bits, which +falls well below recommendations from certain standards groups (such as the US +NIST). Some organizations deploying GitLab will need to enforce minimum key +strength, either to satisfy internal security policy or for regulatory +compliance. + +Similarly, certain standards groups recommend using RSA, ECDSA, or ED25519 over +the older DSA, and administrators may need to limit the allowed SSH key +algorithms. + +GitLab allows you to restrict the allowed SSH key technology as well as specify +the minimum key length for each technology. + +In the Admin area under **Settings** (`/admin/application_settings`), look for +the "Visibility and Access Controls" area: + + diff --git a/doc/ssh/README.md b/doc/ssh/README.md index cf28f1a2eca..793de9d777c 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -193,6 +193,38 @@ How to add your SSH key to Eclipse: https://wiki.eclipse.org/EGit/User_Guide#Ecl [winputty]: https://the.earth.li/~sgtatham/putty/0.67/htmldoc/Chapter8.html#pubkey-puttygen +## SSH on the GitLab server + +GitLab integrates with the system-installed SSH daemon, designating a user +(typically named `git`) through which all access requests are handled. Users +connecting to the GitLab server over SSH are identified by their SSH key instead +of their username. + +SSH *client* operations performed on the GitLab server wil be executed as this +user. Although it is possible to modify the SSH configuration for this user to, +e.g., provide a private SSH key to authenticate these requests by, this practice +is **not supported** and is strongly discouraged as it presents significant +security risks. + +The GitLab check process includes a check for this condition, and will direct you +to this section if your server is configured like this, e.g.: + +``` +$ gitlab-rake gitlab:check +# ... +Git user has default SSH configuration? ... no + Try fixing it: + mkdir ~/gitlab-check-backup-1504540051 + sudo mv /var/lib/git/.ssh/id_rsa ~/gitlab-check-backup-1504540051 + sudo mv /var/lib/git/.ssh/id_rsa.pub ~/gitlab-check-backup-1504540051 + For more information see: + doc/ssh/README.md in section "SSH on the GitLab server" + Please fix the error above and rerun the checks. +``` + +Remove the custom configuration as soon as you're able to. These customizations +are *explicitly not supported* and may stop working at any time. + ## Troubleshooting If on Git clone you are prompted for a password like `git@gitlab.com's password:` diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md index 583ec5522fd..0399ebec86a 100644 --- a/doc/system_hooks/system_hooks.md +++ b/doc/system_hooks/system_hooks.md @@ -31,7 +31,7 @@ X-Gitlab-Event: System Hook "path": "storecloud", "path_with_namespace": "jsmith/storecloud", "project_id": 74, - "project_visibility": "private", + "project_visibility": "private" } ``` @@ -48,7 +48,7 @@ X-Gitlab-Event: System Hook "path": "underscore", "path_with_namespace": "jsmith/underscore", "project_id": 73, - "project_visibility": "internal", + "project_visibility": "internal" } ``` @@ -66,7 +66,7 @@ X-Gitlab-Event: System Hook "owner_name": "John Smith", "owner_email": "johnsmith@gmail.com", "project_visibility": "internal", - "old_path_with_namespace": "jsmith/overscore", + "old_path_with_namespace": "jsmith/overscore" } ``` @@ -84,7 +84,7 @@ X-Gitlab-Event: System Hook "owner_name": "John Smith", "owner_email": "johnsmith@gmail.com", "project_visibility": "internal", - "old_path_with_namespace": "jsmith/overscore", + "old_path_with_namespace": "jsmith/overscore" } ``` @@ -101,7 +101,7 @@ X-Gitlab-Event: System Hook "path": "storecloud", "path_with_namespace": "jsmith/storecloud", "project_id": 74, - "project_visibility": "private", + "project_visibility": "private" } ``` @@ -121,7 +121,7 @@ X-Gitlab-Event: System Hook "user_name": "John Smith", "user_username": "johnsmith", "user_id": 41, - "project_visibility": "private", + "project_visibility": "private" } ``` @@ -141,7 +141,7 @@ X-Gitlab-Event: System Hook "user_name": "John Smith", "user_username": "johnsmith", "user_id": 41, - "project_visibility": "private", + "project_visibility": "private" } ``` diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index 0c0d482499a..fac91935a45 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -37,7 +37,6 @@ This page gathers all the resources for the topic **Authentication** within GitL - [Private Tokens](../../api/README.md#private-tokens) - [Impersonation tokens](../../api/README.md#impersonation-tokens) - [GitLab as an OAuth2 provider](../../api/oauth2.md#gitlab-as-an-oauth2-provider) -- [GitLab Runner API - Authentication](../../api/ci/runners.md#authentication) ## Third-party resources diff --git a/doc/topics/autodevops/img/auto_devops_settings.png b/doc/topics/autodevops/img/auto_devops_settings.png Binary files differnew file mode 100644 index 00000000000..57bd7650a30 --- /dev/null +++ b/doc/topics/autodevops/img/auto_devops_settings.png diff --git a/doc/topics/autodevops/img/auto_monitoring.png b/doc/topics/autodevops/img/auto_monitoring.png Binary files differnew file mode 100644 index 00000000000..5661b50841b --- /dev/null +++ b/doc/topics/autodevops/img/auto_monitoring.png diff --git a/doc/topics/autodevops/img/guide_connect_cluster.png b/doc/topics/autodevops/img/guide_connect_cluster.png Binary files differnew file mode 100644 index 00000000000..b856b81a1d0 --- /dev/null +++ b/doc/topics/autodevops/img/guide_connect_cluster.png diff --git a/doc/topics/autodevops/img/guide_integration.png b/doc/topics/autodevops/img/guide_integration.png Binary files differnew file mode 100644 index 00000000000..723b2619ea2 --- /dev/null +++ b/doc/topics/autodevops/img/guide_integration.png diff --git a/doc/topics/autodevops/img/guide_secret.png b/doc/topics/autodevops/img/guide_secret.png Binary files differnew file mode 100644 index 00000000000..01f5aa49908 --- /dev/null +++ b/doc/topics/autodevops/img/guide_secret.png diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md new file mode 100644 index 00000000000..babf44d2665 --- /dev/null +++ b/doc/topics/autodevops/index.md @@ -0,0 +1,342 @@ +# Auto DevOps + +> [Introduced][ce-37115] in GitLab 10.0. Auto DevOps is currently in Beta and +**not recommended for production use**. Access to the Container Registry is only +available while the pipeline is running. Restarting a pod, scaling a service, or +other actions which require on-going access **will fail** even for public +projects. On-going secure access is planned for a subsequent release. + +Auto DevOps brings best practices to your project in an easy and default way. A +typical web project starts with Continuous Integration (CI), then adds automated +deployment to production, and maybe some time in the future adds some kind of +monitoring. With Auto DevOps, every project has a complete workflow, with +no configuration, including: + +- [Auto Build](#auto-build) +- [Auto Test](#auto-test) +- [Auto Code Quality](#auto-code-quality) +- [Auto Review Apps](#auto-review-apps) +- [Auto Deploy](#auto-deploy) +- [Auto Monitoring](#-auto-monitoring) + +## Overview + +You will need [Kubernetes](https://kubernetes.io/) and +[Prometheus](https://prometheus.io/) to make full use of Auto DevOps, but +even projects using only [GitLab Runners](https://docs.gitlab.com/runner/) will +be able to make use of Auto Build, Auto Test, and Auto Code Quality. + +Auto DevOps makes use of an open source tool called +[Herokuish](https://github.com/gliderlabs/herokuish) which uses [Heroku +buildpacks](https://devcenter.heroku.com/articles/buildpacks) to automatically +detect, build, and test applications. Auto DevOps supports all of the languages +and frameworks that are [supported by +Herokuish](https://github.com/gliderlabs/herokuish#buildpacks) such as Ruby, +Rails, Node, PHP, Python, and Java, and [custom buildpacks can be +specified](#using-custom-buildpacks). *GitLab is in no way affiliated with Heroku +or Glider Labs.* + +Projects can [customize](#customizing) the process by specifying [custom +buildpacks](#custom-buildpack), [custom `Dockerfile`s](#custom-dockerfile), +[custom Helm charts](#custom-helm-chart), or even copying the complete CI/CD +configuration into your project to enable staging and canary deployments, and +more. + +## Quick start + +If you are using GitLab.com, see our [quick start guide](quick_start_guide.md) +for using Auto DevOps with GitLab.com and an external Kubernetes cluster on +Google Cloud. + +For self-hosted installations, the easiest way to make use of Auto DevOps is to +install GitLab inside a Kubernetes cluster using the [GitLab-Omnibus Helm +Chart](../../install/kubernetes/gitlab_omnibus.md) which automatically installs +and configures everything you need. + +## Prerequisites + +You will need one or more GitLab Runners, a Kubernetes cluster, and Prometheus +installed in the cluster to make full use of Auto DevOps. If you do not have +Kubernetes or Prometheus installed then Auto Review Apps, Auto Deploy, and Auto +Monitoring will be silently skipped. + +If you are using GitLab outside of Kubernetes, for example with GitLab.com, then +you should take these prerequisites into account: + +1. **Base domain** - You will need a base domain configured with wildcard DNS to + be used by all of your Auto DevOps applications. + +1. **GitLab Runner** - Your Runner needs to be configured to be able to run Docker. + Generally this means using the + [Docker](https://docs.gitlab.com/runner/executors/docker.html) or [Kubernetes + executor](https://docs.gitlab.com/runner/executors/kubernetes.html), with + [privileged mode enabled](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). + The Runners do not need to be installed in the Kubernetes cluster, but the + Kubernetes executor is easy to use and is automatically autoscaling. + Docker-based Runners can be configured to autoscale as well, using [Docker + Machine](https://docs.gitlab.com/runner/install/autoscaling.html). Runners + should be registered as [shared Runners](../../ci/runners/README.md#registering-a-shared-runner) + for the entire GitLab instance, or [specific Runners](../../ci/runners/README.md#registering-a-specific-runner) + that are assigned to specific projects. + +1. **Kubernetes** - To enable deploys, you will need Kubernetes 1.5+, with NGINX + ingress and wildcard SSL termination, for example using the + [`nginx-ingress`](https://github.com/kubernetes/charts/tree/master/stable/nginx-ingress) + and [`kube-lego`](https://github.com/kubernetes/charts/tree/master/stable/kube-lego) + Helm charts respectively. The [Kubernetes service][kubernetes-service] + integration will need to be enabled for the project, or enabled as a + [default service template](../../user/project/integrations/services_templates.md) + for the entire GitLab installation. + +1. **Prometheus** - To enable Auto Monitoring, you will need Prometheus installed + somewhere (inside or outside your cluster) and configured to scrape your + Kubernetes cluster. To get response metrics (in addition to system metrics), + you need to [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-prometheus-to-monitor-for-nginx-ingress-metrics). + The [Prometheus service](../../user/project/integrations/prometheus.md) + integration needs to be enabled for the project, or enabled as a + [default service template](../../user/project/integrations/services_templates.md) + for the entire GitLab installation. + +## Enabling Auto DevOps + +In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps +section. Select "Enable Auto DevOps", add in your base domain, and save. + + + +## Stages of Auto DevOps + +The following sections describe the stages of Auto DevOps. + +### Auto Build + +Auto Build creates a build of the application in one of two ways: + +- If there is a `Dockerfile`, it will use `docker build` to create a Docker image. +- Otherwise, it will use [Herokuish](https://github.com/gliderlabs/herokuish) + and [Heroku buildpacks](https://devcenter.heroku.com/articles/buildpacks) + to automatically detect and build the application into a Docker image. + +Either way, the resulting Docker image is automatically pushed to the +[Container Registry][container-registry], tagged with the commit SHA. + +### Auto Test + +Auto Test automatically tests your application using +[Herokuish](https://github.com/gliderlabs/herokuish) and [Heroku +buildpacks](https://devcenter.heroku.com/articles/buildpacks). Auto Test will +analyze your project to detect the language and framework, and run appropriate +tests. Several languages and frameworks are detected automatically, but if your +language is not detected, you may succeed with a [custom +buildpack](#custom-buildpack). + +Auto Test uses tests you already have in your application. If there are no +tests, it's up to you to add them. + +### Auto Code Quality + +Auto Code Quality uses the open source +[`codeclimate` image](https://hub.docker.com/r/codeclimate/codeclimate/) to run +static analysis and other code checks on the current code, creating a report +that is uploaded as an artifact. In GitLab EE, differences between the source +and target branches are shown in the merge request widget. *GitLab is in no way +affiliated with Code Climate.* + +### Auto Review Apps + +Auto Review Apps create a [Review App][review-app] for each branch. Review Apps +are temporary application environments based on the branch's code so developers, +designers, QA, product managers, and other reviewers can actually see and +interact with code changes as part of the review process. + +The review app will have a unique URL based on the project name, the branch +name, and a unique number, combined with the Auto DevOps base domain. For +example, `user-project-branch-1234.example.com`. A link to the Review App shows +up in the merge request widget for easy discovery. When the branch is deleted, +for example after the merge request is merged, the Review App will automatically +be deleted. + +This is an optional step, since many projects do not have a Kubernetes cluster +available. If the Kubernetes service is not configured, or if the variable +`AUTO_DEVOPS_DOMAIN` is not available (usually set automatically by the Auto +DevOps setting), the job will silently be skipped. + +### Auto Deploy + +After a branch or merge request is merged into `master`, Auto Deploy deploys the +application to a `production` environment in the Kubernetes cluster, with a +namespace based on the project name and unique project ID. e.g. `project-4321`. +This is an optional step, since many projects do not have a Kubernetes cluster +available. If the Kubernetes service is not configured, or if the variable +`AUTO_DEVOPS_DOMAIN` is not available (usually set automatically by the Auto +DevOps setting), the job will silently be skipped. + +Auto Deploy doesn't include deployments to staging or canary by default, but the +Auto DevOps template contains job definitions for these tasks if you want to +enable them. + +### Auto Monitoring + +Once your application is deployed, Auto Monitoring makes it possible to monitor +your application's server and response metrics right out of the box. Auto +Monitoring uses [Prometheus](../../user/project/integrations/prometheus.md) to +get system metrics such as CPU and memory usage directly from +[Kubernetes](../../user/project/integrations/prometheus_library/kubernetes.md), +and response metrics such as HTTP error rates, latency, and throughput from the +[NGINX +server](../../user/project/integrations/prometheus_library/nginx_ingress.md). + +* Response Metrics: latency, throughput, error rate +* System Metrics: CPU utilization, memory utilization + +To view the metrics, open the [Monitoring dashboard for a deployed environment](../../ci/environments.md#monitoring-environments). + + + +### Configuring Auto Monitoring + +If GitLab has been deployed using the +[omnibus-gitlab](../../install/kubernetes/gitlab_omnibus.md) Helm chart, no +configuration is required. + +If you have installed GitLab using a different method: + +1. [Deploy Prometheus](../../user/project/integrations/prometheus.md#configuring-your-own-prometheus-server-within-kubernetes) into your Kubernetes cluster +1. If you would like response metrics, ensure you are running at least version 0.9.0 of NGINX Ingress and [enable Prometheus metrics](https://github.com/kubernetes/ingress/blob/master/examples/customization/custom-vts-metrics/nginx/nginx-vts-metrics-conf.yaml). +1. Finally, [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) the NGINX Ingress deployment to be scraped by Prometheus using `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. + +## Customizing + +### PostgreSQL Database Support + +In order to support applications that require a database, +[PostgreSQL][postgresql] is provisioned by default. Credentials to access the +database are preconfigured, but can be customized by setting the associated +[variables](#postgresql-variables). These credentials can be used for defining a +`DATABASE_URL` of the format: +`postgres://user:password@postgres-host:postgres-port/postgres-database`. + +PostgreSQL provisioning can be disabled by creating a project variable +`POSTGRES_ENABLED` set to `false`. + +#### PostgreSQL Variables + +Any variables set at the project or group level will override variables set in +the CI/CD configuration. + +1. `POSTGRES_ENABLED: "false"`: disable automatic deployment of PostgreSQL +1. `POSTGRES_USER: "my-user"`: use custom username for PostgreSQL +1. `POSTGRES_PASSWORD: "password"`: use custom password for PostgreSQL +1. `POSTGRES_DB: "my-database"`: use custom database name for PostgreSQL + +### Custom buildpack + +If the automatic buildpack detection fails for your project, or if you want to +use a custom buildpack, you can override the buildpack using a project variable +or a `.buildpack` file in your project: + +- **Project variable** - Create a project variable `BUILDPACK_URL` with the URL + of the buildpack to use. + +- **`.buildpack` file** - Add a file in your project's repo called `.buildpack` + and add the URL of the buildpack to use on a line in the file. If you want to + use multiple buildpacks, you can enter them in, one on each line + + >**Note:** Using multiple buildpacks may break Auto Test. + +### Custom `Dockerfile` + +If your project has a `Dockerfile` in the root of the project repo, Auto DevOps +will build a Docker image based on the Dockerfile rather than using buildpacks. +This can be much faster and result in smaller images, especially if your +Dockerfile is based on [Alpine](https://hub.docker.com/_/alpine/). + +### Custom Helm Chart + +Auto DevOps uses Helm to deploy your application to Kubernetes. You can override +the Helm chart used by bundling up a chart into your project repo or by +specifying a project variable. + +**Bundled chart** - If your project has a `chart` directory with a `Chart.yaml` +file in it, Auto DevOps will detect the chart and use it instead of the default +chart. This can be a great way to control exactly how your application is +deployed. + +**Project variable** - Create a project variable `AUTO_DEVOPS_CHART` with the +URL of a custom chart to use. + +### Enable staging, canaries, and more with custom `.gitlab-ci.yml` + +If you want to modify the CI/CD pipeline used by Auto DevOps, you can copy the +Auto DevOps template into your project's repo and edit as you see fit. + +From your project home page, click on the `Set up CI` button, or click on the `+` +button and `New file` and pick `.gitlab-ci.yml` as the template type, or view an +existing `.gitlab-ci.yml` file. Then select "Auto DevOps" from the template +dropdown. You will then be able to edit or add any jobs needed. + +For example, if you want deploys to go to a staging environment instead of +directly to a production environment, you can enable the `staging` job by +renaming `.staging` to `staging`. Then make sure to uncomment the `when` key of +the `production` job to turn it into a manual action instead of deploying +automatically. + +## Currently supported languages + +>**Note:** +Not all buildpacks support Auto Test yet, as it's a relatively new +enhancement. All of Heroku's [officially supported +languages](https://devcenter.heroku.com/articles/heroku-ci#currently-supported-languages) +support it, and some third-party buildpacks as well e.g., Go, Node, Java, PHP, +Python, Ruby, Gradle, Scala, and Elixir all support Auto Test, but notably the +multi-buildpack does not. + +As of GitLab 10.0, the supported buildpacks are: + +``` +* heroku-buildpack-multi v1.0.0 +* heroku-buildpack-ruby v168 +* heroku-buildpack-nodejs v99 +* heroku-buildpack-clojure v77 +* heroku-buildpack-python v99 +* heroku-buildpack-java v53 +* heroku-buildpack-gradle v23 +* heroku-buildpack-scala v78 +* heroku-buildpack-play v26 +* heroku-buildpack-php v122 +* heroku-buildpack-go v72 +* heroku-buildpack-erlang fa17af9 +* buildpack-nginx v8 +``` + +## Private Project Support - Experimental + +When a project has been marked as private, GitLab's [Container +Registry][container-registry] requires authentication when downloading +containers. Auto DevOps will automatically provide the required authentication +information to Kubernetes, allowing temporary access to the registry. +Authentication credentials will be valid while the pipeline is running, allowing +for a successful initial deployment. + +After the pipeline completes, Kubernetes will no longer be able to access the +container registry. **Restarting a pod, scaling a service, or other actions which +require on-going access to the registry will fail**. On-going secure access is +planned for a subsequent release. + +## Troubleshooting + +- Auto Build and Auto Test may fail in detecting your language/framework. There + may be no buildpack for your application, or your application may be missing the + key files the buildpack is looking for. For example, for ruby apps, you must + have a `Gemfile` to be properly detected, even though it is possible to write a + Ruby app without a `Gemfile`. Try specifying a [custom + buildpack](#custom-buildpack). +- Auto Test may fail because of a mismatch between testing frameworks. In this + case, you may need to customize your `.gitlab-ci.yml` with your test commands. + +[ce-37115]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37115 +[kubernetes-service]: ../../user/project/integrations/kubernetes.md +[docker-in-docker]: ../../docker/using_docker_build.md#use-docker-in-docker-executor +[review-app]: ../../ci/review_apps/index.md +[container-registry]: ../../user/project/container_registry.md +[postgresql]: https://www.postgresql.org/ diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md new file mode 100644 index 00000000000..f23c9d794b4 --- /dev/null +++ b/doc/topics/autodevops/quick_start_guide.md @@ -0,0 +1,95 @@ +# Auto DevOps: quick start guide + +> [Introduced][ce-37115] in GitLab 10.0. Auto DevOps is currently in Beta and +**not recommended for production use**. + +This is a step-by-step guide to deploying a project hosted on GitLab.com to +Google Cloud, using Auto DevOps. + +We made a minimal [Ruby +application](https://gitlab.com/gitlab-examples/minimal-ruby-app) to use as an +example for this guide. It contains two files: + +* `server.rb` - our application. It will start an HTTP server on port 5000 and + render "Hello, world!" +* `Dockerfile` - to build our app into a container image. It will use a ruby + base image and run `server.rb` + +## Fork sample project on GitLab.com + +Let’s start by forking our sample application. Go to [the project +page](https://gitlab.com/gitlab-examples/minimal-ruby-app) and press the `Fork` +button. Soon you should have a project under your namespace with the necessary +files. + +## Setup your own cluster on Google Container Engine + +If you do not already have a Google Cloud account, create one at https://console.cloud.google.com. + +Visit the [`Container Engine`](https://console.cloud.google.com/kubernetes/list) tab and create a new cluster. You can change the name and leave the rest of the default settings. Once you have your cluster running, you need to connect to the cluster by following the Google interface. + +## Connect to Kubernetes cluster + +You need to have the Google Cloud SDK installed. e.g. +On OSX, install [homebrew](https://brew.sh): + +1. Install Brew Caskroom: `brew install caskroom/cask/brew-cask` +2. Install Google Cloud SDK: `brew cask install google-cloud-sdk` +3. Add `kubectl`: `gcloud components install kubectl` +4. Log in: `gcloud auth login` + +Now go back to the Google interface, find your cluster, and follow the instructions under `Connect to the cluster` and open the Kubernetes Dashboard. It will look something like `gcloud container clusters get-credentials ruby-autodeploy \ --zone europe-west2-c --project api-project-XXXXXXX` and then `kubectl proxy`. + + + +## Copy credentials to GitLab.com project + +Once you have the Kubernetes Dashboard interface running, you should visit `Secrets` under the `Config` section. There you should find the settings we need for GitLab integration: ca.crt and token. + + + +You need to copy-paste the ca.crt and token into your project on GitLab.com in the Kubernetes integration page under project **Settings > Integrations > Project services > Kubernetes**. Don't actually copy the namespace though. Each project should have a unique namespace, and by leaving it blank, GitLab will create one for you. + + + +For API URL, you should use the `Endpoint` IP from your cluster page on Google Cloud Platform. + +## Expose application to the world + +In order to be able to visit your application, you need to install an NGINX ingress controller and point your domain name to its external IP address. + +### Set up Ingress controller + +You’ll need to make sure you have an ingress controller. If you don’t have one, do: + +```sh +brew install kubernetes-helm +helm init +helm install --name ruby-app stable/nginx-ingress +``` + +This should create several services including `ruby-app-nginx-ingress-controller`. You can list your services by running `kubectl get svc` to confirm that. + +### Point DNS at Cluster IP + +Find out the external IP address of the `ruby-app-nginx-ingress-controller` by running: + +```sh +kubectl get svc ruby-app-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' +``` + +Use this IP address to configure your DNS. This part heavily depends on your preferences and domain provider. But in case you are not sure, just create an A record with a wildcard host like `*.<your-domain>`. + +Use `nslookup minimal-ruby-app-staging.<yourdomain>` to confirm that domain is assigned to the cluster IP. + +## Set up Auto DevOps + +In your GitLab.com project, go to **Settings > CI/CD** and find the Auto DevOps section. Select "Enable Auto DevOps", add in your base domain, and save. + + + +Then trigger your first pipeline run. This will create a new pipeline with several jobs: `build`, `test`, `codequality`, and `production`. The `build` job will create a docker image with your new change and push it to the GitLab Container Registry. The `test` job will test your change. The `codequality` job will run static analysis on your change. The `production` job will deploy your change to a production application. Once the deploy job succeeds you should be able to see your application by visiting the Kubernetes dashboard. Select the namespace of your project, which will look like `minimal-ruby-app-23`, but with a unique ID for your project, and your app will be listed as "production" under the Deployment tab. + +Once its ready - just visit http://minimal-ruby-app.example.com to see “Hello, world!” + +[ce-37115]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37115 diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 604f9375714..df56f031970 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -22,6 +22,7 @@ We've gathered some resources to help you to get the best from Git with GitLab. - [Cherry-picking a commit](../../user/project/merge_requests/cherry_pick_changes.md#cherry-picking-a-commit) - [Squashing commits](../../workflow/gitlab_flow.md#squashing-commits-with-rebase) - **Articles:** + - [Numerous _undo_ possibilities in Git](../../articles/numerous_undo_possibilities_in_git/index.md) - [How to install Git](../../articles/how_to_install_git/index.md) - [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) - [Eight Tips to help you work better with Git](https://about.gitlab.com/2015/02/19/8-tips-to-help-you-work-better-with-git/) diff --git a/doc/topics/index.md b/doc/topics/index.md index ad388dff822..b51f24b02e4 100644 --- a/doc/topics/index.md +++ b/doc/topics/index.md @@ -7,6 +7,7 @@ you through better understanding GitLab's concepts through our regular docs, and, when available, through articles (guides, tutorials, technical overviews, blog posts) and videos. +- [Auto DevOps](autodevops/index.md) - [Authentication](authentication/index.md) - [Continuous Integration (GitLab CI)](../ci/README.md) - [Git](git/index.md) diff --git a/doc/update/8.17-to-9.0.md b/doc/update/8.17-to-9.0.md index 4d3ababaa41..2abc57da1a0 100644 --- a/doc/update/8.17-to-9.0.md +++ b/doc/update/8.17-to-9.0.md @@ -264,6 +264,16 @@ sudo systemctl daemon-reload ### 9. Install libs, migrations, etc. +GitLab 9.0.11 [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/24570) +a dependency on on the `re2` regular expression library. To install this dependency: + +```bash +sudo apt-get install libre2-dev +``` + +Ubuntu 14.04 (Trusty Tahr) doesn't have the `libre2-dev` package available, but +you can [install re2 manually](https://github.com/google/re2/wiki/Install). + ```bash cd /home/git/gitlab diff --git a/doc/update/9.0-to-9.1.md b/doc/update/9.0-to-9.1.md index 2b4a7bed27f..3fd1d023d2a 100644 --- a/doc/update/9.0-to-9.1.md +++ b/doc/update/9.0-to-9.1.md @@ -264,6 +264,16 @@ sudo systemctl daemon-reload ### 9. Install libs, migrations, etc. +GitLab 9.1.8 [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/24570) +a dependency on on the `re2` regular expression library. To install this dependency: + +```bash +sudo apt-get install libre2-dev +``` + +Ubuntu 14.04 (Trusty Tahr) doesn't have the `libre2-dev` package available, but +you can [install re2 manually](https://github.com/google/re2/wiki/Install). + ```bash cd /home/git/gitlab diff --git a/doc/update/9.1-to-9.2.md b/doc/update/9.1-to-9.2.md index f38547bba1a..5f7a616cc7d 100644 --- a/doc/update/9.1-to-9.2.md +++ b/doc/update/9.1-to-9.2.md @@ -100,6 +100,7 @@ cd /home/git/gitlab sudo -u git -H git fetch --all sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale ``` For GitLab Community Edition: @@ -221,6 +222,16 @@ sudo systemctl daemon-reload ### 10. Install libs, migrations, etc. +GitLab 9.2.8 [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/24570) +a dependency on on the `re2` regular expression library. To install this dependency: + +```bash +sudo apt-get install libre2-dev +``` + +Ubuntu 14.04 (Trusty Tahr) doesn't have the `libre2-dev` package available, but +you can [install re2 manually](https://github.com/google/re2/wiki/Install). + ```bash cd /home/git/gitlab @@ -236,6 +247,11 @@ sudo -u git -H bundle clean # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:pack RAILS_ENV=production +sudo -u git -H bundle exec rake gettext:po_to_json RAILS_ENV=production + # Update node dependencies and recompile assets sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md index 373f43eb3bb..9d0b0da7edb 100644 --- a/doc/update/9.2-to-9.3.md +++ b/doc/update/9.2-to-9.3.md @@ -100,6 +100,7 @@ cd /home/git/gitlab sudo -u git -H git fetch --all sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale ``` For GitLab Community Edition: @@ -257,6 +258,16 @@ sudo systemctl daemon-reload ### 12. Install libs, migrations, etc. +GitLab 9.3.8 [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/24570) +a dependency on on the `re2` regular expression library. To install this dependency: + +```bash +sudo apt-get install libre2-dev +``` + +Ubuntu 14.04 (Trusty Tahr) doesn't have the `libre2-dev` package available, but +you can [install re2 manually](https://github.com/google/re2/wiki/Install). + ```bash cd /home/git/gitlab @@ -272,6 +283,10 @@ sudo -u git -H bundle clean # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + # Update node dependencies and recompile assets sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production diff --git a/doc/update/9.3-to-9.4.md b/doc/update/9.3-to-9.4.md index b167f0737aa..9ee01bc9c51 100644 --- a/doc/update/9.3-to-9.4.md +++ b/doc/update/9.3-to-9.4.md @@ -100,6 +100,7 @@ cd /home/git/gitlab sudo -u git -H git fetch --all sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale ``` For GitLab Community Edition: @@ -270,6 +271,16 @@ sudo systemctl daemon-reload ### 12. Install libs, migrations, etc. +GitLab 9.4 [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/24570) +a dependency on on the `re2` regular expression library. To install this dependency: + +```bash +sudo apt-get install libre2-dev +``` + +Ubuntu 14.04 (Trusty Tahr) doesn't have the `libre2-dev` package available, but +you can [install re2 manually](https://github.com/google/re2/wiki/Install). + ```bash cd /home/git/gitlab @@ -285,6 +296,10 @@ sudo -u git -H bundle clean # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + # Update node dependencies and recompile assets sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production diff --git a/doc/update/9.4-to-9.5.md b/doc/update/9.4-to-9.5.md index fc87b2d0f1e..1b5a15589af 100644 --- a/doc/update/9.4-to-9.5.md +++ b/doc/update/9.4-to-9.5.md @@ -295,6 +295,10 @@ sudo -u git -H bundle clean # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + # Update node dependencies and recompile assets sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index ac1bcb8f241..30107360446 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -35,7 +35,7 @@ current version with `cat VERSION`). cd /home/git/gitlab sudo -u git -H git fetch --all -sudo -u git -H git checkout -- Gemfile.lock db/schema.rb +sudo -u git -H git checkout -- Gemfile.lock db/schema.rb locale sudo -u git -H git checkout LATEST_TAG -b LATEST_TAG ``` @@ -56,6 +56,12 @@ sudo -u git -H bundle clean # Run database migrations sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production +# Compile GetText PO files +# Internationalization was added in `v9.2.0` so these commands are only +# required for versions equal or major to it. +sudo -u git -H bundle exec rake gettext:pack RAILS_ENV=production +sudo -u git -H bundle exec rake gettext:po_to_json RAILS_ENV=production + # Clean up assets and cache sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile cache:clear RAILS_ENV=production NODE_ENV=production ``` diff --git a/doc/user/discussions/img/automatically_resolve_outdated_discussions.png b/doc/user/discussions/img/automatically_resolve_outdated_discussions.png Binary files differnew file mode 100644 index 00000000000..9a798ddd178 --- /dev/null +++ b/doc/user/discussions/img/automatically_resolve_outdated_discussions.png diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 8b1d299484c..efea99eb120 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -116,6 +116,23 @@ are resolved.  +### Automatically resolve merge request diff discussions when they become outdated + +> [Introduced][ce-14053] in GitLab 10.0. + +You can automatically resolve merge request diff discussions on lines modified +with a new push. + +Navigate to your project's settings page, select the **Automatically resolve +merge request diffs discussions on lines changed with a push** check box and hit +**Save** for the changes to take effect. + + + +From now on, any discussions on a diff will be resolved by default if a push +makes that diff section outdated. Discussions on lines that don't change and +top-level resolvable discussions are not automatically resolved. + ## Threaded discussions > [Introduced][ce-7527] in GitLab 9.1. @@ -141,6 +158,7 @@ comments in greater detail. [ce-7527]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7527 [ce-7180]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7180 [ce-8266]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8266 +[ce-14053]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14053 [resolve-discussion-button]: img/resolve_discussion_button.png [resolve-comment-button]: img/resolve_comment_button.png [discussion-view]: img/discussion_view.png diff --git a/doc/user/group/img/share_with_group_lock.png b/doc/user/group/img/share_with_group_lock.png Binary files differindex 8df41bf9465..c0f25389eaf 100644 --- a/doc/user/group/img/share_with_group_lock.png +++ b/doc/user/group/img/share_with_group_lock.png diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 08da721c71d..db0242f1324 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -55,8 +55,18 @@ By doing so: - John mentions everyone from his team with `@john-team` - John mentions only his marketing team with `@john-team/marketing` +## Issues and merge requests within a group + +Issues and merge requests are part of projects. For a given group, view all the +[issues](../project/issues/index.md#issues-per-group) and [merge requests](../project/merge_requests/index.md#merge-requests-per-group) across all the projects in that group, +together in a single list view. + ## Create a new group +> **Notes:** +- For a list of words that are not allowed to be used as group names see the + [reserved names][reserved]. + You can create a group in GitLab from: 1. The Groups page: expand the left menu, click **Groups**, and click the green button **New group**: @@ -153,6 +163,13 @@ Find this option under your project's settings. GitLab administrators can use the admin interface to move any project to any namespace if needed. +## Sharing a project with a group + +You can [share your projects with a group](../project/members/share_project_with_groups.md) +and give your group members access to the project all at once. + +Alternatively, you can [lock the sharing with group feature](#share-with-group-lock). + ## Manage group memberships via LDAP In GitLab Enterprise Edition it is possible to manage GitLab group memberships using LDAP groups. @@ -173,10 +190,28 @@ access further configurations for your group. #### Enforce 2FA to group members -Add a secury layer to your group by +Add a security layer to your group by [enforcing two-factor authentication (2FA)](../../security/two_factor_authentication.md#enforcing-2fa-for-all-users-in-a-group) to all group members. +#### Share with group lock + +Prevent projects in a group from [sharing +a project with another group](../project/members/share_project_with_groups.md). +This allows for tighter control over project access. + +For example, consider you have two distinct teams (Group A and Group B) +working together in a project. +To inherit the group membership, you share the project between the +two groups A and B. **Share with group lock** prevents any project within +the group from being shared with another group. By doing so, you +guarantee only the right group members have access to that projects. + +To enable this feature, navigate to the group settings page. Select +**Share with group lock** and **Save the group**. + + + #### Member Lock (EES/EEP) Available in [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/), @@ -185,15 +220,6 @@ level of members in group. Learn more about [Member Lock](https://docs.gitlab.com/ee/user/group/index.html#member-lock-ees-eep). -#### Share with group lock (EES/EEP) - -In [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/) -it is possible to prevent projects in a group from [sharing -a project with another group](../../workflow/share_projects_with_other_groups.md). -This allows for tighter control over project access. - -Learn more about [Share with group lock](https://docs.gitlab.com/ee/user/group/index.html#share-with-group-lock-ees-eep). - ### Advanced settings - **Projects**: view all projects within that group, add members to each project, @@ -205,4 +231,5 @@ for the group (GitLab admins only, available in [GitLab Enterprise Edition Start - **Pipelines quota**: keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group [permissions]: ../permissions.md#permissions -[ee]: https://about.gitlab.com/products/
\ No newline at end of file +[ee]: https://about.gitlab.com/products/ +[reserved]: ../reserved_names.md diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 5724dcfab48..d2478aea4bd 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -83,10 +83,7 @@ structure. - You need to be an Owner of a group in order to be able to create a subgroup. For more information check the [permissions table][permissions]. - For a list of words that are not allowed to be used as group names see the - [`path_regex.rb` file][reserved] under the `TOP_LEVEL_ROUTES`, `PROJECT_WILDCARD_ROUTES` and `GROUP_ROUTES` lists: - - `TOP_LEVEL_ROUTES`: are names that are reserved as usernames or top level groups - - `PROJECT_WILDCARD_ROUTES`: are names that are reserved for child groups or projects. - - `GROUP_ROUTES`: are names that are reserved for all groups or projects. + [reserved names][reserved]. To create a subgroup: @@ -175,5 +172,5 @@ Here's a list of what you can't do with subgroups: [ce-2772]: https://gitlab.com/gitlab-org/gitlab-ce/issues/2772 [permissions]: ../../permissions.md#group -[reserved]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/path_regex.rb +[reserved]: ../../reserved_names.md [issue]: https://gitlab.com/gitlab-org/gitlab-ce/issues/30472#note_27747600 diff --git a/doc/user/index.md b/doc/user/index.md index 1281cc6e4f0..f239a15d441 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -71,80 +71,44 @@ your code, use it as an issue tracker, collaborate on code, and continuously build, test, and deploy your app with built-in GitLab CI/CD. Or, you can do it all at once, from one single project. -### Repository - -Host your codebase in [GitLab repositories](project/repository/index.md) with version control -and as part of a fully integrated platform. - -### Issues - -Explore the best of GitLab [Issues](project/issues/index.md). - -### Merge Requests - -Collanorate on code, gather reviews, live preview changes per branch, and -request approvals with [Merge Requests](project/merge_requests/index.md). - -### Milestones - -Work on multiple issues and merge requests towards the same target date -with [Milestones](project/milestones/index.md). - -### GitLab Pages - -Publish your static site directly from GitLab with [GitLab Pages](project/pages/index.md). You -can [build, test, and deploy any Static Site Generator](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) with Pages. - -### Container Registry - -Build and deploy Docker images with [GitLab Container Registry](project/container_registry.md). +- [Repositories](project/repository/index.md): Host your codebase in +repositories with version control and as part of a fully integrated platform. +- [Issues](project/issues/index.md): Explore the best of GitLab Issues' features. +- [Merge Requests](project/merge_requests/index.md): Collaborate on code, +reviews, live preview changes per branch, and request approvals with Merge Requests. +- [Milestones](project/milestones/index.md): Work on multiple issues and merge +requests towards the same target date with Milestones. ## GitLab CI/CD Use built-in [GitLab CI/CD](../ci/README.md) to test, build, and deploy your applications directly from GitLab. No third-party integrations needed. -### Auto Deploy - -Deploy your application out-of-the-box with [GitLab Auto Deploy](../ci/autodeploy/index.md). - -### Review Apps - -Live-preview the changes introduced by a merge request with [Review Apps](../ci/review_apps/index.md). - -## Groups - -With GitLab [Groups](group/index.md) you can assemble related projects together -and grant members access to several projects at once. - -### Subgroups - -Groups can also be nested in [subgroups](group/subgroups/index.md). +- [GitLab Auto Deploy](../ci/autodeploy/index.md): Deploy your application out-of-the-box with GitLab Auto Deploy. +- [Review Apps](../ci/review_apps/index.md): Live-preview the changes introduced by a merge request with Review Apps. +- [GitLab Pages](project/pages/index.md): Publish your static site directly from +GitLab with Gitlab Pages. You can build, test, and deploy any Static Site Generator with Pages. +- [GitLab Container Registry](project/container_registry.md): Build and deploy Docker +images with Container Registry. ## Account There is a lot you can customize and configure to enjoy the best of GitLab. -[Manage your user settings](profile/index.md) to change your personal info, +- [Settings](profile/index.md): Manage your user settings to change your personal info, personal access tokens, authorized applications, etc. +- [Authentication](../topics/authentication/index.md): Read through the authentication +methods available in GitLab. +- [Permissions](permissions.md): Learn the different set of permissions levels for each +user type (guest, reporter, developer, master, owner). -### Authentication - -Read through the [authentication](../topics/authentication/index.md) methods available in GitLab. - -### Permissions - -Learn the different set of [permissions](permissions.md) for user type (guest, reporter, developer, master, owner). - -## Integrations - -[Integrate GitLab](../integration/README.md) with your preferred tool, -such as Trello, JIRA, etc. +## Groups -## Git and GitLab +With GitLab [Groups](group/index.md) you can assemble related projects together +and grant members access to several projects at once. -Learn what is [Git](../topics/git/index.md) and its best practices. +Groups can also be nested in [subgroups](group/subgroups/index.md). ## Discussions @@ -155,6 +119,13 @@ When performing inline reviews to implementations to your codebase through merge requests you can gather feedback through [resolvable discussions](discussions/index.md#resolvable-discussions). +### GitLab Flavored Markdown (GFM) + +Read through the [GFM documentation](markdown.md) to learn how to apply +the best of GitLab Flavored Markdown in your discussions, comments, +issues and merge requests descriptions, and everywhere else GMF is +supported. + ## Todos Never forget to reply to your collaborators. [GitLab Todos](../workflow/todos.md) @@ -162,12 +133,21 @@ are a tool for working faster and more effectively with your team, by listing all user or group mentions, as well as issues and merge requests you're assigned to. +## Search + +[Search and filter](search/index.md) through groups, projects, issues, merge requests, files, code, and more. + ## Snippets [Snippets](snippets.md) are code blocks that you want to store in GitLab, from which you have quick access to. You can also gather feedback on them through [discussions](#discussions). +## Integrations + +[Integrate GitLab](../integration/README.md) with your preferred tool, +such as Trello, JIRA, etc. + ## Webhooks Configure [webhooks](project/integrations/webhooks.html) to listen for @@ -178,3 +158,6 @@ POST request with data to the webhook URL. Automate GitLab via [API](../api/README.html). +## Git and GitLab + +Learn what is [Git](../topics/git/index.md) and its best practices. diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 3d47e644ad2..0c17905aa8c 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -5,32 +5,32 @@ particular group or project. If a user is both in a group's project and the project itself, the highest permission level is used. On public and internal projects the Guest role is not enforced. All users will -be able to create issues, leave comments, and pull or download the project code. +be able to create issues, leave comments, and clone or download the project code. -When a member leaves the team the all assigned Issues and Merge Requests +When a member leaves the team all the assigned [Issues](project/issues/index.md) and [Merge Requests](project/merge_requests/index.md) will be unassigned automatically. -GitLab administrators receive all permissions. +GitLab [administrators](../README.md#administrator-documentation) receive all permissions. -To add or import a user, you can follow the [project users and members -documentation](../workflow/add-user/add-user.md). +To add or import a user, you can follow the +[project members documentation](../user/project/members/index.md). -## Project +## Project members permissions The following table depicts the various user permission levels in a project. | Action | Guest | Reporter | Developer | Master | Owner | |---------------------------------------|---------|------------|-------------|----------|--------| -| Create new issue | ✓ | ✓ | ✓ | ✓ | ✓ | -| Create confidential issue | ✓ | ✓ | ✓ | ✓ | ✓ | -| View confidential issues | (✓) [^1] | ✓ | ✓ | ✓ | ✓ | -| Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | -| See a list of jobs | ✓ [^2] | ✓ | ✓ | ✓ | ✓ | -| See a job log | ✓ [^2] | ✓ | ✓ | ✓ | ✓ | -| Download and browse job artifacts | ✓ [^2] | ✓ | ✓ | ✓ | ✓ | -| View wiki pages | ✓ | ✓ | ✓ | ✓ | ✓ | -| Pull project code | | ✓ | ✓ | ✓ | ✓ | -| Download project | | ✓ | ✓ | ✓ | ✓ | +| Create new issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| View confidential issues | (✓) [^2] | ✓ | ✓ | ✓ | ✓ | +| Leave comments | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| See a list of jobs | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | +| See a job log | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | +| Download and browse job artifacts | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | +| View wiki pages | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| Pull project code | [^1] | ✓ | ✓ | ✓ | ✓ | +| Download project | [^1] | ✓ | ✓ | ✓ | ✓ | | Create code snippets | | ✓ | ✓ | ✓ | ✓ | | Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | | Manage labels | | ✓ | ✓ | ✓ | ✓ | @@ -71,11 +71,62 @@ The following table depicts the various user permission levels in a project. | Switch visibility level | | | | | ✓ | | Transfer project to another namespace | | | | | ✓ | | Remove project | | | | | ✓ | -| Force push to protected branches [^3] | | | | | | -| Remove protected branches [^3] | | | | | | +| Force push to protected branches [^4] | | | | | | +| Remove protected branches [^4] | | | | | | | Remove pages | | | | | ✓ | -## Group +## Project features permissions + +### Wiki and issues + +Project features like wiki and issues can be hidden from users depending on +which visibility level you select on project settings. + +- Disabled: disabled for everyone +- Only team members: only team members will see even if your project is public or internal +- Everyone with access: everyone can see depending on your project visibility level + +### Protected branches + +To prevent people from messing with history or pushing code without +review, we've created protected branches. Read through the documentation on +[protected branches](project/protected_branches.md) +to learn more. + +Additionally, you can allow or forbid users with Master and/or +Developer permissions to push to a protected branch. Read through the documentation on +[Allowed to Merge and Allowed to Push settings](project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings) +to learn more. + +### Cycle Analytics permissions + +Find the current permissions on the Cycle Analytics dashboard on +the [documentation on Cycle Analytics permissions](project/cycle_analytics.md#permissions). + +### Issue Board permissions + +Developers and users with higher permission level can use all +the functionality of the Issue Board, that is create/delete lists +and drag issues around. Read though the +[documentation on Issue Boards permissions](project/issue_board.md#permissions) +to learn more. + +### File Locking permissions (EEP) + +The user that locks a file or directory is the only one that can edit and push their changes back to the repository where the locked objects are located. + +Read through the documentation on [permissions for File Locking](https://docs.gitlab.com/ee/user/project/file_lock.html#permissions-on-file-locking) to learn more. + +File Locking is available in +[GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/) only. + +### Confidential Issues permissions + +Confidential issues can be accessed by reporters and higher permission levels, +as well as by guest users that create a confidential issue. To learn more, +read through the documentation on [permissions and access to confidential issues](project/issues/confidential_issues.md#permissions-and-access-to-confidential-issues). + +## Group members permissions Any user can remove themselves from a group, unless they are the last Owner of the group. The following table depicts the various user permission levels in a @@ -91,7 +142,16 @@ group. | Remove group | | | | | ✓ | | Manage group labels | | ✓ | ✓ | ✓ | ✓ | -## External Users +### Subgroup permissions + +When you add a member to a subgroup, they inherit the membership and +permission level from the parent group. This model allows access to +nested groups if you have membership in one of its parents. + +To learn more, read through the documentation on +[subgroups memberships](group/subgroups/index.md#membership). + +## External users permissions In cases where it is desired that a user has access only to some internal or private projects, there is the option of creating **External Users**. This @@ -115,18 +175,9 @@ will find the option to flag the user as external. By default new users are not set as external users. This behavior can be changed by an administrator under **Admin > Application Settings**. -## Project features - -Project features like wiki and issues can be hidden from users depending on -which visibility level you select on project settings. - -- Disabled: disabled for everyone -- Only team members: only team members will see even if your project is public or internal -- Everyone with access: everyone can see depending on your project visibility level - -## GitLab CI +## GitLab CI/CD permissions -GitLab CI permissions rely on the role the user has in GitLab. There are four +GitLab CI/CD permissions rely on the role the user has in GitLab. There are four permission levels in total: - admin @@ -134,7 +185,7 @@ permission levels in total: - developer - guest/reporter -The admin user can perform any action on GitLab CI in scope of the GitLab +The admin user can perform any action on GitLab CI/CD in scope of the GitLab instance and project. In addition, all admins can use the admin interface under `/admin/runners`. @@ -150,7 +201,7 @@ instance and project. In addition, all admins can use the admin interface under | See events in the system | | | | ✓ | | Admin interface | | | | ✓ | -### Jobs permissions +### Job permissions >**Note:** GitLab 8.12 has a completely redesigned job permissions system. @@ -164,20 +215,47 @@ users: | Run CI job | | ✓ | ✓ | ✓ | | Clone source and LFS from current project | | ✓ | ✓ | ✓ | | Clone source and LFS from public projects | | ✓ | ✓ | ✓ | -| Clone source and LFS from internal projects | | ✓ [^4] | ✓ [^4] | ✓ | -| Clone source and LFS from private projects | | ✓ [^5] | ✓ [^5] | ✓ [^5] | +| Clone source and LFS from internal projects | | ✓ [^5] | ✓ [^5] | ✓ | +| Clone source and LFS from private projects | | ✓ [^6] | ✓ [^6] | ✓ [^6] | | Push source and LFS | | | | | | Pull container images from current project | | ✓ | ✓ | ✓ | | Pull container images from public projects | | ✓ | ✓ | ✓ | -| Pull container images from internal projects| | ✓ [^4] | ✓ [^4] | ✓ | -| Pull container images from private projects | | ✓ [^5] | ✓ [^5] | ✓ [^5] | +| Pull container images from internal projects| | ✓ [^5] | ✓ [^5] | ✓ | +| Pull container images from private projects | | ✓ [^6] | ✓ [^6] | ✓ [^6] | | Push container images to current project | | ✓ | ✓ | ✓ | | Push container images to other projects | | | | | -[^1]: Guest users can only view the confidential issues they created themselves -[^2]: If **Public pipelines** is enabled in **Project Settings > Pipelines** -[^3]: Not allowed for Guest, Reporter, Developer, Master, or Owner -[^4]: Only if user is not external one. -[^5]: Only if user is a member of the project. +### New CI job permissions model + +GitLab 8.12 has a completely redesigned job permissions system. To learn more, +read through the documentation on the [new CI/CD permissions model](project/new_ci_build_permissions_model.md#new-ci-job-permissions-model). + +## Running pipelines on protected branches + +The permission to merge or push to protected branches is used to define if a user can +run CI/CD pipelines and execute actions on jobs that are related to those branches. + +See [Security on protected branches](../ci/pipelines.md#security-on-protected-branches) +for details about the pipelines security model. + +## LDAP users permissions + +Since GitLab 8.15, LDAP user permissions can now be manually overridden by an admin user. +Read through the documentation on [LDAP users permissions](https://docs.gitlab.com/ee/articles/how_to_configure_ldap_gitlab_ee/index.html#updating-user-permissions-new-feature) to learn more. + +## Auditor users permissions (EEP) + +An Auditor user should be able to access all projects and groups of a GitLab instance +with the permissions described on the documentation on [auditor users permissions](https://docs.gitlab.com/ee/administration/auditor_users.html#permissions-and-restrictions-of-an-auditor-user). + +Auditor users are available in [GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/) +only. + +[^1]: On public and internal projects, all users are able to perform this action. +[^2]: Guest users can only view the confidential issues they created themselves +[^3]: If **Public pipelines** is enabled in **Project Settings > Pipelines** +[^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner +[^5]: Only if user is not external one. +[^6]: Only if user is a member of the project. [ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 [new-mod]: project/new_ci_build_permissions_model.md diff --git a/doc/user/profile/img/profile_settings_dropdown.png b/doc/user/profile/img/profile_settings_dropdown.png Binary files differnew file mode 100644 index 00000000000..a2c620642e2 --- /dev/null +++ b/doc/user/profile/img/profile_settings_dropdown.png diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 629d69d8aea..5c615daf464 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -120,6 +120,11 @@ If a project is private, credentials will need to be provided for authorization. The preferred way to do this, is by using [personal access tokens][pat]. The minimal scope needed is `read_registry`. +Example of using a personal access token: +``` +docker login registry.example.com -u <your_username> -p <your_personal_access_token> +``` + ## Troubleshooting the GitLab Container Registry ### Basic Troubleshooting diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index ea7496af089..7c94f4ef4d8 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -39,4 +39,58 @@ changes you made after picking the template and return it to its initial status.  +## Description template example + +We make use of Description Templates for Issues and Merge Requests within the GitLab Community Edition project. Please refer to the [`.gitlab` folder][gitlab-ce-templates] for some examples. + +> **Tip:** +It is possible to use [quick actions](./quick_actions.md) within description templates to quickly add labels, assignees, and milestones. The quick actions will only be executed if the user submitting the Issue or Merge Request has the permissions perform the relevant actions. + +Here is an example for a Bug report template: + +``` +Summary + +(Summarize the bug encountered concisely) + + +Steps to reproduce + +(How one can reproduce the issue - this is very important) + + +Example Project + +(If possible, please create an example project here on GitLab.com that exhibits the problematic behaviour, and link to it here in the bug report) + +(If you are using an older version of GitLab, this will also determine whether the bug has been fixed in a more recent version) + + +What is the current bug behavior? + +(What actually happens) + + +What is the expected correct behavior? + +(What you should see instead) + + +Relevant logs and/or screenshots + +(Paste any relevant logs - please use code blocks (```) to format console output, +logs, and code as it's very hard to read otherwise.) + + +Possible fixes + +(If you can, link to the line of code that might be responsible for the problem) + +/label ~bug ~reproduced ~needs-investigation +/cc @project-manager +/assign @qa-tester +``` + [ce-4981]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4981 +[gitlab-ce-templates]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/.gitlab + diff --git a/doc/user/project/gpg_signed_commits/index.md b/doc/user/project/gpg_signed_commits/index.md new file mode 100644 index 00000000000..261eedeb412 --- /dev/null +++ b/doc/user/project/gpg_signed_commits/index.md @@ -0,0 +1 @@ +This document was moved to [another location](../repository/gpg_signed_commits/index.md). diff --git a/doc/user/project/import/bitbucket.md b/doc/user/project/import/bitbucket.md new file mode 100644 index 00000000000..b22c7db0047 --- /dev/null +++ b/doc/user/project/import/bitbucket.md @@ -0,0 +1,62 @@ +# Import your project from Bitbucket to GitLab + +Import your projects from Bitbucket to GitLab with minimal effort. + +## Overview + +>**Note:** +The [Bitbucket integration][bb-import] must be first enabled in order to be +able to import your projects from Bitbucket. Ask your GitLab administrator +to enable this if not already. + +- At its current state, the Bitbucket importer can import: + - the repository description (GitLab 7.7+) + - the Git repository data (GitLab 7.7+) + - the issues (GitLab 7.7+) + - the issue comments (GitLab 8.15+) + - the pull requests (GitLab 8.4+) + - the pull request comments (GitLab 8.15+) + - the milestones (GitLab 8.15+) + - the wiki (GitLab 8.15+) +- References to pull requests and issues are preserved (GitLab 8.7+) +- Repository public access is retained. If a repository is private in Bitbucket + it will be created as private in GitLab as well. + + +## How it works + +When issues/pull requests are being imported, the Bitbucket importer tries to find +the Bitbucket author/assignee in GitLab's database using the Bitbucket ID. For this +to work, the Bitbucket author/assignee should have signed in beforehand in GitLab +and **associated their Bitbucket account**. If the user is not +found in GitLab's database, the project creator (most of the times the current +user that started the import process) is set as the author, but a reference on +the issue about the original Bitbucket author is kept. + +The importer will create any new namespaces (groups) if they don't exist or in +the case the namespace is taken, the repository will be imported under the user's +namespace that started the import process. + +## Importing your Bitbucket repositories + +1. Sign in to GitLab and go to your dashboard. +1. Click on **New project**. + +  + +1. Click on the "Bitbucket" button + +  + +1. Grant GitLab access to your Bitbucket account + +  + +1. Click on the projects that you'd like to import or **Import all projects**. + You can also select the namespace under which each project will be + imported. + +  + +[bb-import]: ../../../integration/bitbucket.md +[social sign-in]: ../../profile/account/social_sign_in.md diff --git a/doc/user/project/import/clearcase.md b/doc/user/project/import/clearcase.md new file mode 100644 index 00000000000..f23623ed485 --- /dev/null +++ b/doc/user/project/import/clearcase.md @@ -0,0 +1,47 @@ +# Migrating from ClearCase + +[ClearCase](https://www-03.ibm.com/software/products/en/clearcase/) is a set of +tools developed by IBM which also include a centralized version control system +similar to Git. + +A good read of ClearCase's basic concepts is can be found in this [StackOverflow +post](https://stackoverflow.com/a/645771/974710). + +The following table illustrates the main differences between ClearCase and Git: + +| Aspect | ClearCase | Git | +| ------ | --------- | --- | +| Repository model | Client-server | Distributed | +| Revision IDs | Branch + number | Global alphanumeric ID | +| Scope of Change | File | Directory tree snapshot | +| Concurrency model | Merge | Merge | +| Storage Method | Deltas | Full content | +| Client | CLI, Eclipse, CC Client | CLI, Eclipse, Git client/GUIs | +| Server | UNIX, Windows legacy systems | UNIX, macOS | +| License | Proprietary | GPL | + +_Taken from the slides [ClearCase and the journey to Git](https://www.open.collab.net/media/pdfs/ClearCase-and-the-journey-to-Git.pdf) provided by collab.net_ + +## Why migrate + +ClearCase can be difficult to manage both from a user and an admin perspective. +Migrating to Git/GitLab there is: + +- **No licensing costs**, Git is GPL while ClearCase is proprietary. +- **Shorter learning curve**, Git has a big community and a vast number of + tutorials to get you started. +- **Integration with modern tools**, migrating to Git and GitLab you can have + an open source end-to-end software development platform with built-in version + control, issue tracking, code review, CI/CD, and more. + +## How to migrate + +While there doesn't exist a tool to fully migrate from ClearCase to Git, here +are some useful links to get you started: + +- [Bridge for Git and ClearCase](https://github.com/charleso/git-cc) +- [Slides "ClearCase and the journey to Git"](https://www.open.collab.net/media/pdfs/ClearCase-and-the-journey-to-Git.pdf) +- [ClearCase to Git](https://therub.org/2013/07/19/clearcase-to-git/) +- [Dual syncing ClearCase to Git](https://therub.org/2013/10/22/dual-syncing-clearcase-and-git/) +- [Moving to Git from ClearCase](https://sateeshkumarb.wordpress.com/2011/01/15/moving-to-git-from-clearcase/) +- [ClearCase to Git webinar](https://www.brighttalk.com/webcast/11817/162473/clearcase-to-git) diff --git a/doc/user/project/import/cvs.md b/doc/user/project/import/cvs.md new file mode 100644 index 00000000000..cabd0eef8d6 --- /dev/null +++ b/doc/user/project/import/cvs.md @@ -0,0 +1,68 @@ +# Migrating from CVS + +[CVS](https://savannah.nongnu.org/projects/cvs) is an old centralized version +control system similar to [SVN](svn.md). + +## CVS vs Git + +The following list illustrates the main differences between CVS and Git: + +- **Git is distributed.** On the other hand, CVS is centralized using a client-server + architecture. This translates to Git having a more flexible workflow since + your working area is a copy of the entire repository. This decreases the + overhead when switching branches or merging for example, since you don't have + to communicate with a remote server. +- **Atomic operations.** In Git all operations are + [atomic](https://en.wikipedia.org/wiki/Atomic_commit), either they succeed as + whole, or they fail without any changes. In CVS, commits (and other operations) + are not atomic. If an operation on the repository is interrupted in the middle, + the repository can be left in an inconsistent state. +- **Storage method.** Changes in CVS are per file (changeset), while in Git + a committed file(s) is stored in its entirety (snapshot). That means that's + very easy in Git to revert or undo a whole change. +- **Revision IDs.** The fact that in CVS changes are per files, the revision ID + is depicted by version numbers, for example `1.4` reflects how many time a + given file has been changed. In Git, each version of a project as a whole + (each commit) has its unique name given by SHA-1. +- **Merge tracking.** Git uses a commit-before-merge approach rather than + merge-before-commit (or update-then-commit) like CVS. If while you were + preparing to create a new commit (new revision) somebody created a + new commit on the same branch and pushed to the central repository, CVS would + force you to first update your working directory and resolve conflicts before + allowing you to commit. This is not the case with Git. You first commit, save + your state in version control, then you merge the other developer's changes. + You can also ask the other developer to do the merge and resolve any conflicts + themselves. +- **Signed commits.** Git supports signing your commits with GPG for additional + security and verification that the commit indeed came from its original author. + GitLab can [integrate with GPG](../repository/gpg_signed_commits/index.md) + and show whether a signed commit is correctly verified. + +_Some of the items above were taken from this great +[Stack Overflow post](https://stackoverflow.com/a/824241/974710). For a more +complete list of differences, consult the +Wikipedia article on [comparing the different version control software](https://en.wikipedia.org/wiki/Comparison_of_version_control_software)._ + +## Why migrate + +CVS is old with no new release since 2008. Git provides more tools to work +with (`git bisect` for one) which makes for a more productive workflow. +Migrating to Git/GitLab there is: + +- **Shorter learning curve**, Git has a big community and a vast number of + tutorials to get you started (see our [Git topic](../../../topics/git/index.md)). +- **Integration with modern tools**, migrating to Git and GitLab you can have + an open source end-to-end software development platform with built-in version + control, issue tracking, code review, CI/CD, and more. +- **Support for many network protocols**. Git supports SSH, HTTP/HTTPS and rsync + among others, whereas CVS supports only SSH and its own insecure pserver + protocol with no user authentication. + +## How to migrate + +Here's a few links to get you started with the migration: + +- [Migrate using the `cvs-fast-export` tool](http://www.catb.org/~esr/reposurgeon/dvcs-migration-guide.html) ([_source code_](https://gitlab.com/esr/cvs-fast-export)) +- [Stack Overflow post on importing the CVS repo](https://stackoverflow.com/a/11490134/974710) +- [Convert a CVS repository to Git](http://www.techrepublic.com/blog/linux-and-open-source/convert-cvs-repositories-to-git/) +- [Man page of the `git-cvsimport` tool](https://www.kernel.org/pub/software/scm/git/docs/git-cvsimport.html) diff --git a/doc/user/project/import/fogbugz.md b/doc/user/project/import/fogbugz.md new file mode 100644 index 00000000000..17222c53675 --- /dev/null +++ b/doc/user/project/import/fogbugz.md @@ -0,0 +1,28 @@ +# Import your project from FogBugz to GitLab + +It only takes a few simple steps to import your project from FogBugz. +The importer will import all of your cases and comments with original case +numbers and timestamps. You will also have the opportunity to map FogBugz +users to GitLab users. + +1. From your GitLab dashboard click 'New project' +1. Click on the 'FogBugz' button + +  + +1. Enter your FogBugz URL, email address, and password. + +  + +1. Create mapping from FogBugz users to GitLab users. + +  + +1. Select the projects you wish to import by clicking the Import buttons + +  + +1. Once the import has finished click the link to take you to the project +dashboard. Follow the directions to push your existing repository. + +  diff --git a/doc/user/project/import/gitea.md b/doc/user/project/import/gitea.md new file mode 100644 index 00000000000..f5746a0fb31 --- /dev/null +++ b/doc/user/project/import/gitea.md @@ -0,0 +1,77 @@ +# Import your project from Gitea to GitLab + +Import your projects from Gitea to GitLab with minimal effort. + +## Overview + +>**Note:** +This requires Gitea `v1.0.0` or newer. + +- At its current state, Gitea importer can import: + - the repository description (GitLab 8.15+) + - the Git repository data (GitLab 8.15+) + - the issues (GitLab 8.15+) + - the pull requests (GitLab 8.15+) + - the milestones (GitLab 8.15+) + - the labels (GitLab 8.15+) +- Repository public access is retained. If a repository is private in Gitea + it will be created as private in GitLab as well. + +## How it works + +Since Gitea is currently not an OAuth provider, author/assignee cannot be mapped +to users in your GitLab's instance. This means that the project creator (most of +the times the current user that started the import process) is set as the author, +but a reference on the issue about the original Gitea author is kept. + +The importer will create any new namespaces (groups) if they don't exist or in +the case the namespace is taken, the repository will be imported under the user's +namespace that started the import process. + +## Importing your Gitea repositories + +The importer page is visible when you create a new project. + + + +Click on the **Gitea** link and the import authorization process will start. + + + +### Authorize access to your repositories using a personal access token + +With this method, you will perform a one-off authorization with Gitea to grant +GitLab access your repositories: + +1. Go to <https://you-gitea-instance/user/settings/applications> (replace + `you-gitea-instance` with the host of your Gitea instance). +1. Click **Generate New Token**. +1. Enter a token description. +1. Click **Generate Token**. +1. Copy the token hash. +1. Go back to GitLab and provide the token to the Gitea importer. +1. Hit the **List Your Gitea Repositories** button and wait while GitLab reads + your repositories' information. Once done, you'll be taken to the importer + page to select the repositories to import. + +### Select which repositories to import + +After you've authorized access to your Gitea repositories, you will be +redirected to the Gitea importer page. + +From there, you can see the import statuses of your Gitea repositories. + +- Those that are being imported will show a _started_ status, +- those already successfully imported will be green with a _done_ status, +- whereas those that are not yet imported will have an **Import** button on the + right side of the table. + +If you want, you can import all your Gitea projects in one go by hitting +**Import all projects** in the upper left corner. + + + +--- + +You can also choose a different name for the project and a different namespace, +if you have the privileges to do so. diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md new file mode 100644 index 00000000000..016f98966e3 --- /dev/null +++ b/doc/user/project/import/github.md @@ -0,0 +1,122 @@ +# Import your project from GitHub to GitLab + +Import your projects from GitHub to GitLab with minimal effort. + +## Overview + +>**Note:** +If you are an administrator you can enable the [GitHub integration][gh-import] +in your GitLab instance sitewide. This configuration is optional, users will +still be able to import their GitHub repositories with a +[personal access token][gh-token]. + +>**Note:** +Administrators of a GitLab instance (Community or Enterprise Edition) can also +use the [GitHub rake task][gh-rake] to import projects from GitHub without the +constrains of a Sidekiq worker. + +- At its current state, GitHub importer can import: + - the repository description (GitLab 7.7+) + - the Git repository data (GitLab 7.7+) + - the issues (GitLab 7.7+) + - the pull requests (GitLab 8.4+) + - the wiki pages (GitLab 8.4+) + - the milestones (GitLab 8.7+) + - the labels (GitLab 8.7+) + - the release note descriptions (GitLab 8.12+) +- References to pull requests and issues are preserved (GitLab 8.7+) +- Repository public access is retained. If a repository is private in GitHub + it will be created as private in GitLab as well. + +## How it works + +When issues/pull requests are being imported, the GitHub importer tries to find +the GitHub author/assignee in GitLab's database using the GitHub ID. For this +to work, the GitHub author/assignee should have signed in beforehand in GitLab +and **associated their GitHub account**. If the user is not +found in GitLab's database, the project creator (most of the times the current +user that started the import process) is set as the author, but a reference on +the issue about the original GitHub author is kept. + +The importer will create any new namespaces (groups) if they don't exist or in +the case the namespace is taken, the repository will be imported under the user's +namespace that started the import process. + +## Importing your GitHub repositories + +The importer page is visible when you create a new project. + + + +Click on the **GitHub** link and the import authorization process will start. +There are two ways to authorize access to your GitHub repositories: + +1. [Using the GitHub integration][gh-integration] (if it's enabled by your + GitLab administrator). This is the preferred way as it's possible to + preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works) + section. +1. [Using a personal access token][gh-token] provided by GitHub. + + + +### Authorize access to your repositories using the GitHub integration + +If the [GitHub integration][gh-import] is enabled by your GitLab administrator, +you can use it instead of the personal access token. + +1. First you may want to connect your GitHub account to GitLab in order for + the username mapping to be correct. +1. Once you connect GitHub, click the **List your GitHub repositories** button + and you will be redirected to GitHub for permission to access your projects. +1. After accepting, you'll be automatically redirected to the importer. + +You can now go on and [select which repositories to import](#select-which-repositories-to-import). + +### Authorize access to your repositories using a personal access token + +>**Note:** +For a proper author/assignee mapping for issues and pull requests, the +[GitHub integration][gh-integration] should be used instead of the +[personal access token][gh-token]. If the GitHub integration is enabled by your +GitLab administrator, it should be the preferred method to import your repositories. +Read more in the [How it works](#how-it-works) section. + +If you are not using the GitHub integration, you can still perform a one-off +authorization with GitHub to grant GitLab access your repositories: + +1. Go to <https://github.com/settings/tokens/new>. +1. Enter a token description. +1. Check the `repo` scope. +1. Click **Generate token**. +1. Copy the token hash. +1. Go back to GitLab and provide the token to the GitHub importer. +1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads + your repositories' information. Once done, you'll be taken to the importer + page to select the repositories to import. + +### Select which repositories to import + +After you've authorized access to your GitHub repositories, you will be +redirected to the GitHub importer page. + +From there, you can see the import statuses of your GitHub repositories. + +- Those that are being imported will show a _started_ status, +- those already successfully imported will be green with a _done_ status, +- whereas those that are not yet imported will have an **Import** button on the + right side of the table. + +If you want, you can import all your GitHub projects in one go by hitting +**Import all projects** in the upper left corner. + + + +--- + +You can also choose a different name for the project and a different namespace, +if you have the privileges to do so. + +[gh-import]: ../../../integration/github.md "GitHub integration" +[gh-rake]: ../../../administration/raketasks/github_import.md "GitHub rake task" +[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration +[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token diff --git a/doc/user/project/import/gitlab_com.md b/doc/user/project/import/gitlab_com.md new file mode 100644 index 00000000000..3b37da67a5b --- /dev/null +++ b/doc/user/project/import/gitlab_com.md @@ -0,0 +1,20 @@ +# Project importing from GitLab.com to your private GitLab instance + +You can import your existing GitLab.com projects to your GitLab instance. But keep in mind that it is possible only if +GitLab support is enabled on your GitLab instance. +You can read more about GitLab support [here](http://docs.gitlab.com/ce/integration/gitlab.html) +To get to the importer page you need to go to "New project" page. + +>**Note:** +If you are interested in importing Wiki and Merge Request data to your new +instance, you'll need to follow the instructions for [project export](../settings/import_export.md) + + + +Click on the "Import projects from GitLab.com" link and you will be redirected to GitLab.com +for permission to access your projects. After accepting, you'll be automatically redirected to the importer. + + + +To import a project, you can simple click "Import". The importer will import your repository and issues. +Once the importer is done, a new GitLab project will be created with your imported data. diff --git a/doc/workflow/importing/img/bitbucket_import_grant_access.png b/doc/user/project/import/img/bitbucket_import_grant_access.png Binary files differindex 429904e621d..429904e621d 100644 --- a/doc/workflow/importing/img/bitbucket_import_grant_access.png +++ b/doc/user/project/import/img/bitbucket_import_grant_access.png diff --git a/doc/workflow/importing/img/bitbucket_import_new_project.png b/doc/user/project/import/img/bitbucket_import_new_project.png Binary files differindex 8ed528c2f09..8ed528c2f09 100644 --- a/doc/workflow/importing/img/bitbucket_import_new_project.png +++ b/doc/user/project/import/img/bitbucket_import_new_project.png diff --git a/doc/workflow/importing/img/bitbucket_import_select_project.png b/doc/user/project/import/img/bitbucket_import_select_project.png Binary files differindex 1bca6166ec8..1bca6166ec8 100644 --- a/doc/workflow/importing/img/bitbucket_import_select_project.png +++ b/doc/user/project/import/img/bitbucket_import_select_project.png diff --git a/doc/workflow/importing/fogbugz_importer/fogbugz_import_finished.png b/doc/user/project/import/img/fogbugz_import_finished.png Binary files differindex 62c5c86c9b3..62c5c86c9b3 100644 --- a/doc/workflow/importing/fogbugz_importer/fogbugz_import_finished.png +++ b/doc/user/project/import/img/fogbugz_import_finished.png diff --git a/doc/workflow/importing/fogbugz_importer/fogbugz_import_login.png b/doc/user/project/import/img/fogbugz_import_login.png Binary files differindex 96bce70b74d..96bce70b74d 100644 --- a/doc/workflow/importing/fogbugz_importer/fogbugz_import_login.png +++ b/doc/user/project/import/img/fogbugz_import_login.png diff --git a/doc/workflow/importing/fogbugz_importer/fogbugz_import_select_fogbogz.png b/doc/user/project/import/img/fogbugz_import_select_fogbogz.png Binary files differindex b26c652e382..b26c652e382 100644 --- a/doc/workflow/importing/fogbugz_importer/fogbugz_import_select_fogbogz.png +++ b/doc/user/project/import/img/fogbugz_import_select_fogbogz.png diff --git a/doc/workflow/importing/fogbugz_importer/fogbugz_import_select_project.png b/doc/user/project/import/img/fogbugz_import_select_project.png Binary files differindex ccc82f9d4cd..ccc82f9d4cd 100644 --- a/doc/workflow/importing/fogbugz_importer/fogbugz_import_select_project.png +++ b/doc/user/project/import/img/fogbugz_import_select_project.png diff --git a/doc/workflow/importing/fogbugz_importer/fogbugz_import_user_map.png b/doc/user/project/import/img/fogbugz_import_user_map.png Binary files differindex 28ff55a8d89..28ff55a8d89 100644 --- a/doc/workflow/importing/fogbugz_importer/fogbugz_import_user_map.png +++ b/doc/user/project/import/img/fogbugz_import_user_map.png diff --git a/doc/workflow/importing/gitlab_importer/importer.png b/doc/user/project/import/img/gitlab_importer.png Binary files differindex 27d42eb492e..27d42eb492e 100644 --- a/doc/workflow/importing/gitlab_importer/importer.png +++ b/doc/user/project/import/img/gitlab_importer.png diff --git a/doc/workflow/importing/gitlab_importer/new_project_page.png b/doc/user/project/import/img/gitlab_new_project_page.png Binary files differindex c673724f436..c673724f436 100644 --- a/doc/workflow/importing/gitlab_importer/new_project_page.png +++ b/doc/user/project/import/img/gitlab_new_project_page.png diff --git a/doc/workflow/importing/img/import_projects_from_gitea_new_import.png b/doc/user/project/import/img/import_projects_from_gitea_new_import.png Binary files differindex a3f603cbd0a..a3f603cbd0a 100644 --- a/doc/workflow/importing/img/import_projects_from_gitea_new_import.png +++ b/doc/user/project/import/img/import_projects_from_gitea_new_import.png diff --git a/doc/workflow/importing/img/import_projects_from_github_importer.png b/doc/user/project/import/img/import_projects_from_github_importer.png Binary files differindex d8effaf6075..d8effaf6075 100644 --- a/doc/workflow/importing/img/import_projects_from_github_importer.png +++ b/doc/user/project/import/img/import_projects_from_github_importer.png diff --git a/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png b/doc/user/project/import/img/import_projects_from_github_select_auth_method.png Binary files differindex 1ccb38a815e..1ccb38a815e 100644 --- a/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png +++ b/doc/user/project/import/img/import_projects_from_github_select_auth_method.png diff --git a/doc/workflow/importing/img/import_projects_from_new_project_page.png b/doc/user/project/import/img/import_projects_from_new_project_page.png Binary files differindex 97ca30b2087..97ca30b2087 100644 --- a/doc/workflow/importing/img/import_projects_from_new_project_page.png +++ b/doc/user/project/import/img/import_projects_from_new_project_page.png diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md new file mode 100644 index 00000000000..8da6e2a8207 --- /dev/null +++ b/doc/user/project/import/index.md @@ -0,0 +1,24 @@ +# Migrating projects to a GitLab instance + +1. [From Bitbucket.org](bitbucket.md) +1. [From ClearCase](clearcase.md) +1. [From CVS](cvs.md) +1. [From FogBugz](fogbugz.md) +1. [From GitHub.com of GitHub Enterprise](github.md) +1. [From GitLab.com](gitlab_com.md) +1. [From Gitea](gitea.md) +1. [From Perforce](perforce.md) +1. [From SVN](svn.md) +1. [From TFS](tfs.md) + +In addition to the specific migration documentation above, you can import any +Git repository via HTTP from the New Project page. Be aware that if the +repository is too large the import can timeout. + +## Migrating from self-hosted GitLab to GitLab.com + +You can copy your repos by changing the remote and pushing to the new server, +but issues and merge requests can't be imported. + +If you want to retain all metadata like issues and merge requests, you can use +the [import/export feature](../settings/import_export.md). diff --git a/doc/user/project/import/perforce.md b/doc/user/project/import/perforce.md new file mode 100644 index 00000000000..aa7508e1e8e --- /dev/null +++ b/doc/user/project/import/perforce.md @@ -0,0 +1,50 @@ +# Migrating from Perforce Helix + +[Perforce Helix](https://www.perforce.com/) provides a set of tools which also +include a centralized, proprietary version control system similar to Git. + +## Perforce vs Git + +The following list illustrates the main differences between Perforce Helix and +Git: + +1. In general the biggest difference is that Perforce branching is heavyweight + compared to Git's lightweight branching. When you create a branch in Perforce, + it creates an integration record in their proprietary database for every file + in the branch, regardless how many were actually changed. Whereas Git was + implemented with a different architecture so that a single SHA acts as a pointer + to the state of the whole repo after the changes, making it very easy to branch. + This is what made feature branching workflows so easy to adopt with Git. +1. Also, context switching between branches is much easier in Git. If your manager + said 'You need to stop work on that new feature and fix this security + vulnerability' you can do so very easily in Git. +1. Having a complete copy of the project and its history on your local machine + means every transaction is superfast and Git provides that. You can branch/merge + and experiment in isolation, then clean up your mess before sharing your new + cool stuff with everyone. +1. Git also made code review simple because you could share your changes without + merging them to master, whereas Perforce had to implement a Shelving feature on + the server so others could review changes before merging. + +## Why migrate + +Perforce Helix can be difficult to manage both from a user and an admin +perspective. Migrating to Git/GitLab there is: + +- **No licensing costs**, Git is GPL while Perforce Helix is proprietary. +- **Shorter learning curve**, Git has a big community and a vast number of + tutorials to get you started. +- **Integration with modern tools**, migrating to Git and GitLab you can have + an open source end-to-end software development platform with built-in version + control, issue tracking, code review, CI/CD, and more. + +## How to migrate + +Git includes a built-in mechanism (`git p4`) to pull code from Perforce and to +submit back from Git to Perforce. + +Here's a few links to get you started: + +- [git-p4 manual page](https://www.kernel.org/pub/software/scm/git/docs/git-p4.html) +- [git-p4 example usage](https://git.wiki.kernel.org/index.php/Git-p4_Usage) +- [Git book migration guide](https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git#_perforce_import) diff --git a/doc/user/project/import/svn.md b/doc/user/project/import/svn.md new file mode 100644 index 00000000000..7a3628a39d7 --- /dev/null +++ b/doc/user/project/import/svn.md @@ -0,0 +1,183 @@ +# Migrating from SVN to GitLab + +Subversion (SVN) is a central version control system (VCS) while +Git is a distributed version control system. There are some major differences +between the two, for more information consult your favorite search engine. + +## Overview + +There are two approaches to SVN to Git migration: + +1. [Git/SVN Mirror](#smooth-migration-with-a-gitsvn-mirror-using-subgit) which: + - Makes the GitLab repository to mirror the SVN project. + - Git and SVN repositories are kept in sync; you can use either one. + - Smoothens the migration process and allows to manage migration risks. + +1. [Cut over migration](#cut-over-migration-with-svn2git) which: + - Translates and imports the existing data and history from SVN to Git. + - Is a fire and forget approach, good for smaller teams. + +## Smooth migration with a Git/SVN mirror using SubGit + +[SubGit](https://subgit.com) is a tool for a smooth, stress-free SVN to Git +migration. It creates a writable Git mirror of a local or remote Subversion +repository and that way you can use both Subversion and Git as long as you like. +It requires access to your GitLab server as it talks with the Git repositories +directly in a filesystem level. + +### SubGit prerequisites + +1. Install Oracle JRE 1.8 or newer. On Debian-based Linux distributions you can + follow [this article](http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html). +1. Download SubGit from https://subgit.com/download/. +1. Unpack the downloaded SubGit zip archive to the `/opt` directory. The `subgit` + command will be available at `/opt/subgit-VERSION/bin/subgit`. + +### SubGit configuration + +The first step to mirror you SVN repository in GitLab is to create a new empty +project which will be used as a mirror. For Omnibus installations the path to +the repository will be located at +`/var/opt/gitlab/git-data/repositories/USER/REPO.git` by default. For +installations from source, the default repository directory will be +`/home/git/repositories/USER/REPO.git`. For convenience, assign this path to a +variable: + +``` +GIT_REPO_PATH=/var/opt/gitlab/git-data/repositories/USER/REPOS.git +``` + +SubGit will keep this repository in sync with a remote SVN project. For +convenience, assign your remote SVN project URL to a variable: + +``` +SVN_PROJECT_URL=http://svn.company.com/repos/project +``` + +Next you need to run SubGit to set up a Git/SVN mirror. Make sure the following +`subgit` command is ran on behalf of the same user that keeps ownership of +GitLab Git repositories (by default `git`): + +``` +subgit configure --layout auto $SVN_PROJECT_URL $GIT_REPO_PATH +``` + +Adjust authors and branches mappings, if necessary. Open with your favorite +text editor: + +``` +edit $GIT_REPO_PATH/subgit/authors.txt +edit $GIT_REPO_PATH/subgit/config +``` + +For more information regarding the SubGit configuration options, refer to +[SubGit's documentation](https://subgit.com/documentation.html) website. + +### Initial translation + +Now that SubGit has configured the Git/SVN repos, run `subgit` to perform the +initial translation of existing SVN revisions into the Git repository: + +``` +subgit install $GIT_REPO_PATH +``` + +After the initial translation is completed, the Git repository and the SVN +project will be kept in sync by `subgit` - new Git commits will be translated to +SVN revisions and new SVN revisions will be translated to Git commits. Mirror +works transparently and does not require any special commands. + +If you would prefer to perform one-time cut over migration with `subgit`, use +the `import` command instead of `install`: + +``` +subgit import $GIT_REPO_PATH +``` + +### SubGit licensing + +Running SubGit in a mirror mode requires a +[registration](https://subgit.com/pricing.html). Registration is free for open +source, academic and startup projects. + +We're currently working on deeper GitLab/SubGit integration. You may track our +progress at [this issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/990). + +### SubGit support + +For any questions related to SVN to GitLab migration with SubGit, you can +contact the SubGit team directly at [support@subgit.com](mailto:support@subgit.com). + +## Cut over migration with svn2git + +If you are currently using an SVN repository, you can migrate the repository +to Git and GitLab. We recommend a hard cut over - run the migration command once +and then have all developers start using the new GitLab repository immediately. +Otherwise, it's hard to keep changing in sync in both directions. The conversion +process should be run on a local workstation. + +Install `svn2git`. On all systems you can install as a Ruby gem if you already +have Ruby and Git installed. + +```bash +sudo gem install svn2git +``` + +On Debian-based Linux distributions you can install the native packages: + +```bash +sudo apt-get install git-core git-svn ruby +``` + +Optionally, prepare an authors file so `svn2git` can map SVN authors to Git authors. +If you choose not to create the authors file then commits will not be attributed +to the correct GitLab user. Some users may not consider this a big issue while +others will want to ensure they complete this step. If you choose to map authors +you will be required to map every author that is present on changes in the SVN +repository. If you don't, the conversion will fail and you will have to update +the author file accordingly. The following command will search through the +repository and output a list of authors. + +```bash +svn log --quiet | grep -E "r[0-9]+ \| .+ \|" | cut -d'|' -f2 | sed 's/ //g' | sort | uniq +``` + +Use the output from the last command to construct the authors file. +Create a file called `authors.txt` and add one mapping per line. + +``` +janedoe = Jane Doe <janedoe@example.com> +johndoe = John Doe <johndoe@example.com> +``` + +If your SVN repository is in the standard format (trunk, branches, tags, +not nested) the conversion is simple. For a non-standard repository see +[svn2git documentation](https://github.com/nirvdrum/svn2git). The following +command will checkout the repository and do the conversion in the current +working directory. Be sure to create a new directory for each repository before +running the `svn2git` command. The conversion process will take some time. + +```bash +svn2git https://svn.example.com/path/to/repo --authors /path/to/authors.txt +``` + +If your SVN repository requires a username and password add the +`--username <username>` and `--password <password` flags to the above command. +`svn2git` also supports excluding certain file paths, branches, tags, etc. See +[svn2git documentation](https://github.com/nirvdrum/svn2git) or run +`svn2git --help` for full documentation on all of the available options. + +Create a new GitLab project, where you will eventually push your converted code. +Copy the SSH or HTTP(S) repository URL from the project page. Add the GitLab +repository as a Git remote and push all the changes. This will push all commits, +branches and tags. + +```bash +git remote add origin git@gitlab.com:<group>/<project>.git +git push --all origin +git push --tags origin +``` + +## Contribute to this guide +We welcome all contributions that would expand this guide with instructions on +how to migrate from SVN and other version control systems. diff --git a/doc/user/project/import/tfs.md b/doc/user/project/import/tfs.md new file mode 100644 index 00000000000..8727c2ff6c3 --- /dev/null +++ b/doc/user/project/import/tfs.md @@ -0,0 +1,42 @@ +# Migrating from TFS + +[TFS](https://www.visualstudio.com/tfs/) is a set of tools developed by Microsoft +which also includes a centralized version control system (TFVC) similar to Git. + +In this document, we emphasize on the TFVC to Git migration. + +## TFVC vs Git + +The following list illustrates the main differences between TFVC and Git: + +- **Git is distributed** whereas TFVC is centralized using a client-server + architecture. This translates to Git having a more flexible workflow since + your working area is a copy of the entire repository. This decreases the + overhead when switching branches or merging for example, since you don't have + to communicate with a remote server. +- **Storage method.** Changes in CVS are per file (changeset), while in Git + a committed file(s) is stored in its entirety (snapshot). That means that's + very easy in Git to revert or undo a whole change. + +_Check also Microsoft's documentation on the +[comparison of Git and TFVC](https://www.visualstudio.com/en-us/docs/tfvc/comparison-git-tfvc) +and the Wikipedia article on +[comparing the different version control software](https://en.wikipedia.org/wiki/Comparison_of_version_control_software)._ + +## Why migrate + +Migrating to Git/GitLab there is: + +- **No licensing costs**, Git is GPL while TFVC is proprietary. +- **Shorter learning curve**, Git has a big community and a vast number of + tutorials to get you started (see our [Git topic](../../../topics/git/index.md)). +- **Integration with modern tools**, migrating to Git and GitLab you can have + an open source end-to-end software development platform with built-in version + control, issue tracking, code review, CI/CD, and more. + +## How to migrate + +The best option to migrate from TFVC to Git is to use the +[`git-tfs`](https://github.com/git-tfs/git-tfs) tool. A specific guide for the +migration exists: +[Migrate TFS to Git](https://github.com/git-tfs/git-tfs/blob/master/doc/usecases/migrate_tfs_to_git.md). diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 91a19600951..d6b3d59d407 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -24,6 +24,7 @@ integrated platform from messing with history or pushing code without review - [Protected tags](protected_tags.md): Control over who has permission to create tags, and prevent accidental update or deletion + - [Signing commits](gpg_signed_commits/index.md): use GPG to sign your commits - [Merge Requests](merge_requests/index.md): Apply your branching strategy and get reviewed by your team - [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) (**EES/EEP**): Ask for approval before @@ -66,8 +67,6 @@ website with GitLab Pages **Other features:** - [Cycle Analytics](cycle_analytics.md): Review your development lifecycle -- [Koding integration](koding.md) (not available on GitLab.com): Integrate -with Koding to have access to a web terminal right from the GitLab UI - [Syntax highlighting](highlighting.md): An alternative to customize your code blocks, overriding GitLab's default choice of language @@ -90,15 +89,19 @@ from your fork to the upstream project ## Import or export a project -- Import a project from: - - [GitHub to GitLab](../../workflow/importing/import_projects_from_github.md) - - [BitBucket to GitLab](../../workflow/importing/import_projects_from_bitbucket.md) - - [Gitea to GitLab](../../workflow/importing/import_projects_from_gitea.md) - - [FogBugz to GitLab](../../workflow/importing/import_projects_from_fogbugz.md) +- [Import a project](import/index.md) from: + - [GitHub to GitLab](import/github.md) + - [BitBucket to GitLab](import/bitbucket.md) + - [Gitea to GitLab](import/gitea.md) + - [FogBugz to GitLab](import/fogbugz.md) - [Export a project from GitLab](settings/import_export.md#exporting-a-project-and-its-data) - [Importing and exporting projects between GitLab instances](settings/import_export.md) -## Leave a project +## Project's members + +Learn how to [add members to your projects](members/index.md). + +### Leave a project **Leave project** will only display on the project's dashboard when a project is part of a group (under a diff --git a/doc/user/project/integrations/img/jira_service_page.png b/doc/user/project/integrations/img/jira_service_page.png Binary files differindex e69376f74c4..63aa0e99a50 100644 --- a/doc/user/project/integrations/img/jira_service_page.png +++ b/doc/user/project/integrations/img/jira_service_page.png diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 4f583879a4e..93aec56f8dc 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -10,7 +10,12 @@ JIRA](https://www.programmableweb.com/news/how-and-why-to-integrate-gitlab-jira/ ## Configuration -Each GitLab project can be configured to connect to a different JIRA instance. +Each GitLab project can be configured to connect to a different JIRA instance. That +means one GitLab project maps to _all_ JIRA projects in that JIRA instance once +the configuration is set up. Therefore, you don't have to explicitly associate +one GitLab project to any JIRA project. Once the configuration is set up, any JIRA +projects in the JIRA instance are already mapped to the GitLab project. + If you have one JIRA instance you can pre-fill the settings page with a default template, see the [Services Templates][services-templates] docs. @@ -103,7 +108,6 @@ in the table below. | ----- | ----------- | | `Web URL` | The base URL to the JIRA instance web interface which is being linked to this GitLab project. E.g., `https://jira.example.com`. | | `JIRA API URL` | The base URL to the JIRA instance API. Web URL value will be used if not set. E.g., `https://jira-api.example.com`. | -| `Project key` | Put a JIRA project key (in uppercase), e.g. `MARS` in this field. This is only for testing the configuration settings. JIRA integration in GitLab works with _all_ JIRA projects in your JIRA instance. This field will be removed in a future release. | | `Username` | The user name created in [configuring JIRA step](#configuring-jira). | | `Password` |The password of the user created in [configuring JIRA step](#configuring-jira). | | `Transition ID` | This is the ID of a transition that moves issues to a closed state. You can find this number under JIRA workflow administration ([see screenshot](img/jira_workflow_screenshot.png)). **Closing JIRA issues via commits or Merge Requests won't work if you don't set the ID correctly.** | diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 6f15765751c..5fefb3b69c4 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -40,7 +40,7 @@ Installing and configuring Prometheus to monitor applications is fairly straight ### Configuring Omnibus GitLab Prometheus to monitor Kubernetes deployments With Omnibus GitLab running inside of Kubernetes, you can leverage the bundled -version of Prometheus to collect the supported metrics. Once enabled, Prometheus will automatically begin monitoring Kubernetes Nodes and any [annotated Pods](https://prometheus.io/docs/operating/configuration/#<kubernetes_sd_config>). +version of Prometheus to collect the supported metrics. Once enabled, Prometheus will automatically begin monitoring Kubernetes Nodes and any [annotated Pods](https://prometheus.io/docs/operating/configuration/#<kubernetes_sd_config>). 1. Read how to configure the bundled Prometheus server in the [Administration guide][gitlab-prometheus-k8s-monitor]. @@ -133,6 +133,8 @@ to integrate with. Once configured, GitLab will attempt to retrieve performance metrics for any environment which has had a successful deployment. +GitLab will automatically scan the Prometheus server for known metrics and attempt to identify the metrics for a particular environment. The supported metrics and scan process is detailed in our [Prometheus Metric Library documentation](prometheus_library/metrics.html). + [Learn more about monitoring environments.](../../../ci/environments.md#monitoring-environments) ## Determining the performance impact of a merge @@ -174,7 +176,7 @@ If the "Attempting to load performance data" screen continues to appear, it coul [prometheus-docker-image]: https://hub.docker.com/r/prom/prometheus/ [prometheus-yml]:samples/prometheus.yml [gitlab.com-ip-range]: https://gitlab.com/gitlab-com/infrastructure/issues/434 -[ci-environment-slug]: https://docs.gitlab.com/ce/ci/variables/#predefined-variables-environment-variables +[ci-environment-slug]: ../../../ci/variables/#predefined-variables-environment-variables [ce-8935]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8935 [ce-10408]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10408 [promgldocs]: ../../../administration/monitoring/prometheus/index.md diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index eb8cd821ddc..9f0308d8111 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -23,4 +23,4 @@ Prometheus server up and running. You have two options here: In order to isolate and only display relevant metrics for a given environment however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). -If you are using [GitLab Auto-Deploy][autodeploy] and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. +If you are using [GitLab Auto-Deploy][../../../ci/autodeploy/index.md] and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. diff --git a/doc/user/project/integrations/prometheus_library/metrics.md b/doc/user/project/integrations/prometheus_library/metrics.md index 6bdffce9c55..f09ecf9ff2d 100644 --- a/doc/user/project/integrations/prometheus_library/metrics.md +++ b/doc/user/project/integrations/prometheus_library/metrics.md @@ -4,6 +4,7 @@ GitLab offers automatic detection of select [Prometheus exporters](https://prometheus.io/docs/instrumenting/exporters/). Currently supported exporters are: * [Kubernetes](kubernetes.md) * [NGINX](nginx.md) +* [NGINX Ingress Controller](nginx_ingress.md) * [HAProxy](haproxy.md) * [Amazon Cloud Watch](cloudwatch.md) @@ -14,10 +15,7 @@ We have tried to surface the most important metrics for each exporter, and will GitLab retrieves performance data from the configured Prometheus server, and attempts to identifying the presence of known metrics. Once identified, GitLab then needs to be able to map the data to a particular environment. In order to isolate and only display relevant metrics for a given environment, GitLab needs a method to detect which labels are associated. To do that, -GitLab will look for the required metrics which have a label that -matches the [$CI_ENVIRONMENT_SLUG][ci-environment-slug]. - -For example if you are deploying to an environment named `production`, there must be a label for the metric with the value of `production`. +GitLab uses the defined queries and fills in the environment specific variables. Typically this involves looking for the [$CI_ENVIRONMENT_SLUG](https://docs.gitlab.com/ee/ci/variables/#predefined-variables-environment-variables), but may also include other information such as the project's Kubernetes namespace. Each search query is defined in the [exporter specific documentation](#prometheus-metrics-library). ## Adding to the library diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index b3470773996..12e3321f5f3 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -8,8 +8,8 @@ GitLab has support for automatically detecting and monitoring NGINX. This is pro | Name | Query | | ---- | ----- | | Throughput (req/sec) | sum(rate(nginx_requests_total{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) | -| Latency (ms) | avg(nginx_upstream_response_msecs_avg{%{environment_filter}}) * 1000 | -| HTTP Error Rate (%) | sum(rate(haproxy_frontend_http_responses_total{code="5xx",%{environment_filter}}[2m])) / sum(rate(haproxy_frontend_http_responses_total{%{environment_filter}}[2m])) | +| Latency (ms) | avg(nginx_upstream_response_msecs_avg{%{environment_filter}}) | +| HTTP Error Rate (HTTP Errors / sec) | rate(nginx_responses_total{status_code="5xx", %{environment_filter}}[2m])) | ## Configuring Prometheus to monitor for NGINX metrics diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md new file mode 100644 index 00000000000..84ee8bc45e5 --- /dev/null +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -0,0 +1,25 @@ +# Monitoring NGINX Ingress Controller +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13438) in GitLab 9.5 + +GitLab has support for automatically detecting and monitoring the Kubernetes NGINX ingress controller. This is provided by leveraging the built in Prometheus metrics included in [version 0.9.0](https://github.com/kubernetes/ingress/blob/master/controllers/nginx/Changelog.md#09-beta1) of the ingress. + +## Metrics supported + +| Name | Query | +| ---- | ----- | +| Throughput (req/sec) | sum(rate(nginx_upstream_requests_total{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) | +| Latency (ms) | avg(nginx_upstream_response_msecs_avg{upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}) | +| HTTP Error Rate (HTTP Errors / sec) | sum(rate(nginx_upstream_responses_total{status_code="5xx", upstream=~"%{kube_namespace}-%{ci_environment_slug}-.*"}[2m])) | + +## Configuring Prometheus to monitor for NGINX ingress metrics + +The easiest way to get started is to use at least version 0.9.0 of [NGINX ingress](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). If you are using NGINX as your Kubernetes ingress, there is [direct support](https://github.com/kubernetes/ingress/pull/423) for enabling Prometheus monitoring in the 0.9.0 release. + +If you have deployed with the [gitlab-omnibus](https://docs.gitlab.com/ee/install/kubernetes/gitlab_omnibus.md) Helm chart, these metrics will be automatically enabled and annotated for Prometheus monitoring. + +## Specifying the Environment label + +In order to isolate and only display relevant metrics for a given environment +however, GitLab needs a method to detect which labels are associated. To do this, GitLab will search metrics with appropriate labels. In this case, the `upstream` label must be of the form `<Kubernetes Namespace>-<CI_ENVIRONMENT_SLUG>-*`. + +If you have used [Auto Deploy](https://docs.gitlab.com/ee/ci/autodeploy/index.html) to deploy your app, this format will be used automatically and metrics will be detected with no action on your part. diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 1760b182114..0bf1f396f9d 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -9,7 +9,7 @@ keep security vulnerabilities private or prevent surprises from leaking out. ## Making an issue confidential -You can make an issue confidential either by creating a new issue or editing +You can make an issue confidential during issue creation or by editing an existing one. When you create a new issue, a checkbox right below the text area is available @@ -19,11 +19,19 @@ confidential checkbox and hit **Save changes**.  -## Making an issue non-confidential +## Modifying issue confidentiality -To make an issue non-confidential, all you have to do is edit it and unmark -the confidential checkbox. Once you save the issue, it will gain the default -visibility level you have chosen for your project. +There are two ways to change an issue's confidentiality. + +The first way is to edit the issue and mark/unmark the confidential checkbox. +Once you save the issue, it will change the confidentiality of the issue. + +The second way is to locate the Confidentiality section in the sidebar and click +**Edit**. A popup should appear and give you the option to turn on or turn off confidentiality. + +| Turn off confidentiality | Turn on confidentiality | +| :-----------: | :----------: | +|  |  | Every change from regular to confidential and vice versa, is indicated by a system note in the issue's comments. @@ -49,6 +57,12 @@ issue you are commenting on is confidential.  +There is also an indicator on the sidebar denoting confidentiality. + +| Confidential issue | Not confidential issue | +| :-----------: | :----------: | +|  |  | + ## Permissions and access to confidential issues There are two kinds of level access for confidential issues. The general rule diff --git a/doc/user/project/issues/img/confidential_issues_index_page.png b/doc/user/project/issues/img/confidential_issues_index_page.png Binary files differindex e4b492a2769..f3efe0ce04e 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_index_page.png +++ b/doc/user/project/issues/img/confidential_issues_index_page.png diff --git a/doc/user/project/issues/img/confidential_issues_issue_page.png b/doc/user/project/issues/img/confidential_issues_issue_page.png Binary files differindex f04ec8ff32b..0f5c774d258 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_issue_page.png +++ b/doc/user/project/issues/img/confidential_issues_issue_page.png diff --git a/doc/user/project/issues/img/confidential_issues_system_notes.png b/doc/user/project/issues/img/confidential_issues_system_notes.png Binary files differindex 82e0dd8e85e..355be80ecb6 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_system_notes.png +++ b/doc/user/project/issues/img/confidential_issues_system_notes.png diff --git a/doc/user/project/issues/img/group_issues_list_view.png b/doc/user/project/issues/img/group_issues_list_view.png Binary files differnew file mode 100644 index 00000000000..5d20e8cbc89 --- /dev/null +++ b/doc/user/project/issues/img/group_issues_list_view.png diff --git a/doc/user/project/issues/img/issue_tracker.png b/doc/user/project/issues/img/issue_tracker.png Binary files differdeleted file mode 100755 index ab25cb64d13..00000000000 --- a/doc/user/project/issues/img/issue_tracker.png +++ /dev/null diff --git a/doc/user/project/issues/img/project_issues_list_view.png b/doc/user/project/issues/img/project_issues_list_view.png Binary files differnew file mode 100644 index 00000000000..2fcc9e8d9da --- /dev/null +++ b/doc/user/project/issues/img/project_issues_list_view.png diff --git a/doc/user/project/issues/img/sidebar_confidential_issue.png b/doc/user/project/issues/img/sidebar_confidential_issue.png Binary files differnew file mode 100755 index 00000000000..d99a1ca756e --- /dev/null +++ b/doc/user/project/issues/img/sidebar_confidential_issue.png diff --git a/doc/user/project/issues/img/sidebar_move_issue.png b/doc/user/project/issues/img/sidebar_move_issue.png Binary files differnew file mode 100644 index 00000000000..111f7861364 --- /dev/null +++ b/doc/user/project/issues/img/sidebar_move_issue.png diff --git a/doc/user/project/issues/img/sidebar_not_confidential_issue.png b/doc/user/project/issues/img/sidebar_not_confidential_issue.png Binary files differnew file mode 100755 index 00000000000..2e6cbbc5b3a --- /dev/null +++ b/doc/user/project/issues/img/sidebar_not_confidential_issue.png diff --git a/doc/user/project/issues/img/turn_off_confidentiality.png b/doc/user/project/issues/img/turn_off_confidentiality.png Binary files differnew file mode 100644 index 00000000000..248ae6522d6 --- /dev/null +++ b/doc/user/project/issues/img/turn_off_confidentiality.png diff --git a/doc/user/project/issues/img/turn_on_confidentiality.png b/doc/user/project/issues/img/turn_on_confidentiality.png Binary files differnew file mode 100644 index 00000000000..fac4c833699 --- /dev/null +++ b/doc/user/project/issues/img/turn_on_confidentiality.png diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 1f78849a92c..0f187946a4a 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -7,7 +7,7 @@ of solving a problem. It allows you, your team, and your collaborators to share and discuss proposals before and while implementing them. -Issues and the GitLab Issue Tracker are available in all +GitLab Issues and the GitLab Issue Tracker are available in all [GitLab Products](https://about.gitlab.com/products/) as part of the [GitLab Workflow](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/). @@ -48,11 +48,27 @@ for feature proposals and another one for bug reports. ## Issue Tracker -The issue tracker is the collection of opened and closed issues created in a project. +The Issue Tracker is the collection of opened and closed issues created in a project. +It is available for all projects, from the moment the project is created. - +Find the issue tracker by navigating to your **Project's homepage** > **Issues**. -Find the issue tracker by navigating to your **Project's Dashboard** > **Issues**. +### Issues per project + +When you access your project's issues, GitLab will present them in a list, +and you can use the tabs available to quickly filter by open and closed issues. + + + +You can also [search and filter](../../search/index.md#issues-and-merge-requests-per-project) the results more deeply with GitLab's search capacities. + +### Issues per group + +View all the issues in a group (that is, all the issues across all projects in that +group) by navigating to **Group > Issues**. This view also has the open and closed +issue tabs. + + ## GitLab Issues Functionalities @@ -70,6 +86,10 @@ Read through the [documentation on creating issues](create_new_issue.md). Learn distinct ways to [close issues](closing_issues.md) in GitLab. +## Moving issues + +Read through the [documentation on moving issues](moving_issues.md). + ## Create a merge request from an issue Learn more about it on the [GitLab Issues Functionalities documentation](issues_functionalities.md#18-new-merge-request). @@ -120,6 +140,12 @@ to find out more about this feature. With [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/), you can also create various boards per project with [Multiple Issue Boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards). +### External Issue Tracker + +Alternatively to GitLab's built-in Issue Tracker, you can also use an [external +tracker](../../../integration/external-issue-tracker.md) such as Jira, Redmine, +or Bugzilla. + ### Issue's API Read through the [API documentation](../../../api/issues.md). diff --git a/doc/user/project/issues/moving_issues.md b/doc/user/project/issues/moving_issues.md new file mode 100644 index 00000000000..211a651b89e --- /dev/null +++ b/doc/user/project/issues/moving_issues.md @@ -0,0 +1,10 @@ +# Moving Issues + +Please read through the [GitLab Issue Documentation](index.md) for an overview on GitLab Issues. + +Moving an issue will close it and duplicate it on the specified project. +There will also be a system note added to both issues indicating where it came from or went to. + +You can move an issue with the "Move issue" button at the bottom of the right-sidebar when viewing the issue. + + diff --git a/doc/user/project/koding.md b/doc/user/project/koding.md index 455e2ee47b4..86e06a39e59 100644 --- a/doc/user/project/koding.md +++ b/doc/user/project/koding.md @@ -1,6 +1,9 @@ # Koding integration -> [Introduced][ce-5909] in GitLab 8.11. +>**Notes:** +- **As of GitLab 10.0, the Koding integration is deprecated and will be removed + in a future version.** +- [Introduced][ce-5909] in GitLab 8.11. This document will guide you through using Koding integration on GitLab in detail. For configuring and installing please follow the diff --git a/doc/workflow/add-user/img/access_requests_management.png b/doc/user/project/members/img/access_requests_management.png Binary files differindex 3693bed869b..3693bed869b 100644 --- a/doc/workflow/add-user/img/access_requests_management.png +++ b/doc/user/project/members/img/access_requests_management.png diff --git a/doc/workflow/add-user/img/add_new_user_to_project_settings.png b/doc/user/project/members/img/add_new_user_to_project_settings.png Binary files differindex 40db600455f..40db600455f 100644 --- a/doc/workflow/add-user/img/add_new_user_to_project_settings.png +++ b/doc/user/project/members/img/add_new_user_to_project_settings.png diff --git a/doc/workflow/add-user/img/add_user_email_accept.png b/doc/user/project/members/img/add_user_email_accept.png Binary files differindex 763b3ff463d..763b3ff463d 100644 --- a/doc/workflow/add-user/img/add_user_email_accept.png +++ b/doc/user/project/members/img/add_user_email_accept.png diff --git a/doc/workflow/add-user/img/add_user_email_ready.png b/doc/user/project/members/img/add_user_email_ready.png Binary files differindex 0066eb3427b..0066eb3427b 100644 --- a/doc/workflow/add-user/img/add_user_email_ready.png +++ b/doc/user/project/members/img/add_user_email_ready.png diff --git a/doc/workflow/add-user/img/add_user_email_search.png b/doc/user/project/members/img/add_user_email_search.png Binary files differindex 66bcd6aad80..66bcd6aad80 100644 --- a/doc/workflow/add-user/img/add_user_email_search.png +++ b/doc/user/project/members/img/add_user_email_search.png diff --git a/doc/workflow/add-user/img/add_user_give_permissions.png b/doc/user/project/members/img/add_user_give_permissions.png Binary files differindex 376a3eefccc..376a3eefccc 100644 --- a/doc/workflow/add-user/img/add_user_give_permissions.png +++ b/doc/user/project/members/img/add_user_give_permissions.png diff --git a/doc/workflow/add-user/img/add_user_import_members_from_another_project.png b/doc/user/project/members/img/add_user_import_members_from_another_project.png Binary files differindex 0c32001098e..0c32001098e 100644 --- a/doc/workflow/add-user/img/add_user_import_members_from_another_project.png +++ b/doc/user/project/members/img/add_user_import_members_from_another_project.png diff --git a/doc/workflow/add-user/img/add_user_imported_members.png b/doc/user/project/members/img/add_user_imported_members.png Binary files differindex 51fd7688890..51fd7688890 100644 --- a/doc/workflow/add-user/img/add_user_imported_members.png +++ b/doc/user/project/members/img/add_user_imported_members.png diff --git a/doc/workflow/add-user/img/add_user_list_members.png b/doc/user/project/members/img/add_user_list_members.png Binary files differindex e0fa404288d..e0fa404288d 100644 --- a/doc/workflow/add-user/img/add_user_list_members.png +++ b/doc/user/project/members/img/add_user_list_members.png diff --git a/doc/workflow/add-user/img/add_user_members_menu.png b/doc/user/project/members/img/add_user_members_menu.png Binary files differindex 8e61d15fe65..8e61d15fe65 100644 --- a/doc/workflow/add-user/img/add_user_members_menu.png +++ b/doc/user/project/members/img/add_user_members_menu.png diff --git a/doc/workflow/add-user/img/add_user_search_people.png b/doc/user/project/members/img/add_user_search_people.png Binary files differindex 41767a9167c..41767a9167c 100644 --- a/doc/workflow/add-user/img/add_user_search_people.png +++ b/doc/user/project/members/img/add_user_search_people.png diff --git a/doc/workflow/groups/max_access_level.png b/doc/user/project/members/img/max_access_level.png Binary files differindex 63f33f9d91d..63f33f9d91d 100644 --- a/doc/workflow/groups/max_access_level.png +++ b/doc/user/project/members/img/max_access_level.png diff --git a/doc/user/project/members/img/other_group_sees_shared_project.png b/doc/user/project/members/img/other_group_sees_shared_project.png Binary files differnew file mode 100644 index 00000000000..e4c93a13abb --- /dev/null +++ b/doc/user/project/members/img/other_group_sees_shared_project.png diff --git a/doc/workflow/add-user/img/request_access_button.png b/doc/user/project/members/img/request_access_button.png Binary files differindex 608baccb0ca..608baccb0ca 100644 --- a/doc/workflow/add-user/img/request_access_button.png +++ b/doc/user/project/members/img/request_access_button.png diff --git a/doc/user/project/members/img/share_project_with_groups.png b/doc/user/project/members/img/share_project_with_groups.png Binary files differnew file mode 100644 index 00000000000..0907438cb84 --- /dev/null +++ b/doc/user/project/members/img/share_project_with_groups.png diff --git a/doc/user/project/members/img/share_project_with_groups_tab.png b/doc/user/project/members/img/share_project_with_groups_tab.png Binary files differnew file mode 100644 index 00000000000..fc489aae003 --- /dev/null +++ b/doc/user/project/members/img/share_project_with_groups_tab.png diff --git a/doc/workflow/add-user/img/withdraw_access_request_button.png b/doc/user/project/members/img/withdraw_access_request_button.png Binary files differindex 6edd786b151..6edd786b151 100644 --- a/doc/workflow/add-user/img/withdraw_access_request_button.png +++ b/doc/user/project/members/img/withdraw_access_request_button.png diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md new file mode 100644 index 00000000000..b8dd96087f1 --- /dev/null +++ b/doc/user/project/members/index.md @@ -0,0 +1,116 @@ +# Project's members + +You can manage the groups and users and their access levels in all of your +projects. You can also personalize the access level you give each user, +per-project. + +You should have `master` or `owner` [permissions](../../permissions.md) to add +or import a new user to your project. + +To view, edit, add, and remove project's members, go to your +project's **Settings > Members**. + +--- + +## Add a user + +Right next to **People**, start typing the name or username of the user you +want to add. + + + +--- + +Select the user and the [permission level](../../user/permissions.md) +that you'd like to give the user. Note that you can select more than one user. + + + +--- + +Once done, hit **Add users to project** and they will be immediately added to +your project with the permissions you gave them above. + + + +--- + +From there on, you can either remove an existing user or change their access +level to the project. + +## Import users from another project + +You can import another project's users in your own project by hitting the +**Import members** button on the upper right corner of the **Members** menu. + +In the dropdown menu, you can see only the projects you are Master on. + + + +--- + +Select the one you want and hit **Import project members**. A flash message +notifying you that the import was successful will appear, and the new members +are now in the project's members list. Notice that the permissions that they +had on the project you imported from are retained. + + + +--- + +## Invite people using their e-mail address + +If a user you want to give access to doesn't have an account on your GitLab +instance, you can invite them just by typing their e-mail address in the +user search field. + + + +--- + +As you can imagine, you can mix inviting multiple people and adding existing +GitLab users to the project. + + + +--- + +Once done, hit **Add users to project** and watch that there is a new member +with the e-mail address we used above. From there on, you can resend the +invitation, change their access level or even delete them. + + + +--- + +Once the user accepts the invitation, they will be prompted to create a new +GitLab account using the same e-mail address the invitation was sent to. + +## Request access to a project + +As a project owner you can enable or disable non members to request access to +your project. Go to the project settings and click on **Allow users to request access**. + +As a user, you can request to be a member of a project. Go to the project you'd +like to be a member of, and click the **Request Access** button on the right +side of your screen. + + + +--- + +Project owners & masters will be notified of your request and will be able to approve or +decline it on the members page. + + + +--- + +If you change your mind before your request is approved, just click the +**Withdraw Access Request** button. + + + +## Share project with group + +Alternatively, you can [share a project with an entire group](share_project_with_groups.md) instead of adding users one by one. diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md new file mode 100644 index 00000000000..f5c748a03b3 --- /dev/null +++ b/doc/user/project/members/share_project_with_groups.md @@ -0,0 +1,43 @@ +# Share Projects with other Groups + +You can share projects with other [groups](../../group/index.md). This makes it +possible to add a group of users to a project with a single action. + +## Groups as collections of users + +Groups are used primarily to [create collections of projects](../../group/index.md), but you can also +take advantage of the fact that groups define collections of _users_, namely the group +members. + +## Sharing a project with a group of users + +The primary mechanism to give a group of users, say 'Engineering', access to a project, +say 'Project Acme', in GitLab is to make the 'Engineering' group the owner of 'Project +Acme'. But what if 'Project Acme' already belongs to another group, say 'Open Source'? +This is where the group sharing feature can be of use. + +To share 'Project Acme' with the 'Engineering' group, go to the project settings page for 'Project Acme' and use the left navigation menu to go to the **Settings > Members** section. + + + +Then select the 'Share with group' tab by clicking it. + +Now you can add the 'Engineering' group with the maximum access level of your choice. Click 'Share' to share it. + + + +After sharing 'Project Acme' with 'Engineering', the project will be listed on the group dashboard. + + + +## Maximum access level + +In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Master' or 'Owner') will only have 'Developer' access to 'Project Acme'. + +## Share project with group lock + +It is possible to prevent projects in a group from [sharing +a project with another group](../members/share_project_with_groups.md). +This allows for tighter control over project access. + +Learn more about [Share with group lock](../../group/index.html#share-with-group-lock). diff --git a/doc/user/project/merge_requests/img/group_merge_requests_list_view.png b/doc/user/project/merge_requests/img/group_merge_requests_list_view.png Binary files differnew file mode 100644 index 00000000000..02a88d0112f --- /dev/null +++ b/doc/user/project/merge_requests/img/group_merge_requests_list_view.png diff --git a/doc/user/project/merge_requests/img/merge_request_diff_file_navigation.png b/doc/user/project/merge_requests/img/merge_request_diff_file_navigation.png Binary files differnew file mode 100644 index 00000000000..9b8aee47411 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_request_diff_file_navigation.png diff --git a/doc/user/project/merge_requests/img/project_merge_requests_list_view.png b/doc/user/project/merge_requests/img/project_merge_requests_list_view.png Binary files differnew file mode 100644 index 00000000000..702ec1a2949 --- /dev/null +++ b/doc/user/project/merge_requests/img/project_merge_requests_list_view.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 9bdf2a998d3..26c6277d33a 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -56,6 +56,23 @@ B. Consider you're a web developer writing a webpage for your company's: 1. Once approved, your merge request is [squashed and merged](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html), and [deployed to staging with GitLab Pages](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) (Squash and Merge is available in GitLab Enterprise Edition Starter) 1. Your production team [cherry picks](#cherry-pick-changes) the merge commit into production +## Merge requests per project + +View all the merge requests within a project by navigating to **Project > Merge Requests**. + +When you access your project's merge requests, GitLab will present them in a list, +and you can use the tabs available to quickly filter by open and closed. You can also [search and filter the results](../../search/index.md#issues-and-merge-requests-per-project). + + + +## Merge requests per group + +View all the merge requests in a group (that is, all the merge requests across all projects in that +group) by navigating to **Group > Merge Requests**. This view also has the open, merged, and closed +merge request tabs, from which you can [search and filter the results](../../search/index.md#issues-and-merge-requests-per-group). + + + ## Authorization for merge requests There are two main ways to have a merge request flow with GitLab: @@ -120,6 +137,14 @@ have been marked as a **Work In Progress**. [Learn more about settings a merge request as "Work In Progress".](work_in_progress_merge_requests.md) +## Merge request diff file navigation + +The diff view has a persistent dropdown for file navigation. As you scroll through +diffs with a large number of files and/or many changes in those files, you can +easily jump to any changed file through the dropdown navigation. + + + ## Ignore whitespace changes in Merge Request diff view If you click the **Hide whitespace changes** button, you can see the diff @@ -141,7 +166,6 @@ all your changes will be available to preview by anyone with the Review Apps lin [Read more about Review Apps.](../../../ci/review_apps/index.md) - ## Tips Here are some tips that will help you be more efficient with merge requests in @@ -230,4 +254,4 @@ git checkout origin/merge-requests/1 ``` [protected branches]: ../protected_branches.md -[ee]: https://about.gitlab.com/gitlab-ee/ "GitLab Enterprise Edition" +[ee]: https://about.gitlab.com/gitlab-ee/ "GitLab Enterprise Edition"
\ No newline at end of file diff --git a/doc/user/project/pipelines/job_artifacts.md b/doc/user/project/pipelines/job_artifacts.md index e853bfff444..4e93e680fd2 100644 --- a/doc/user/project/pipelines/job_artifacts.md +++ b/doc/user/project/pipelines/job_artifacts.md @@ -100,16 +100,21 @@ inside GitLab that make that possible. It is possible to download the latest artifacts of a job via a well known URL so you can use it for scripting purposes. +>**Note:** +The latest artifacts are considered as the artifacts created by jobs in the +latest pipeline that succeeded for the specific ref. +Artifacts for other pipelines can be accessed with direct access to them. + The structure of the URL to download the whole artifacts archive is the following: ``` -https://example.com/<namespace>/<project>/builds/artifacts/<ref>/download?job=<job_name> +https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/download?job=<job_name> ``` To download a single file from the artifacts use the following URL: ``` -https://example.com/<namespace>/<project>/builds/artifacts/<ref>/raw/<path_to_file>?job=<job_name> +https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/raw/<path_to_file>?job=<job_name> ``` For example, to download the latest artifacts of the job named `coverage` of @@ -117,26 +122,26 @@ the `master` branch of the `gitlab-ce` project that belongs to the `gitlab-org` namespace, the URL would be: ``` -https://gitlab.com/gitlab-org/gitlab-ce/builds/artifacts/master/download?job=coverage +https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/artifacts/master/download?job=coverage ``` To download the file `coverage/index.html` from the same artifacts use the following URL: ``` -https://gitlab.com/gitlab-org/gitlab-ce/builds/artifacts/master/raw/coverage/index.html?job=coverage +https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/artifacts/master/raw/coverage/index.html?job=coverage ``` There is also a URL to browse the latest job artifacts: ``` -https://example.com/<namespace>/<project>/builds/artifacts/<ref>/browse?job=<job_name> +https://example.com/<namespace>/<project>/-/jobs/artifacts/<ref>/browse?job=<job_name> ``` For example: ``` -https://gitlab.com/gitlab-org/gitlab-ce/builds/artifacts/master/browse?job=coverage +https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/artifacts/master/browse?job=coverage ``` The latest builds are also exposed in the UI in various places. Specifically, diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index 3ff5a08d72c..dbc1305101f 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -66,10 +66,30 @@ in the pipelines settings page. ## Visibility of pipelines -For public and internal projects, the pipelines page can be accessed by -anyone and those logged in respectively. If you wish to hide it so that only -the members of the project or group have access to it, uncheck the **Public -pipelines** checkbox and save the changes. +Access to pipelines and job details (including output of logs and artifacts) +is checked against your current user access level and the **Public pipelines** +project setting. + +If **Public pipelines** is enabled (default): + +- for **public** projects, anyone can view the pipelines and access the job details + (output logs and artifacts) +- for **internal** projects, any logged in user can view the pipelines + and access the job details + (output logs and artifacts) +- for **private** projects, any member (guest or higher) can view the pipelines + and access the job details + (output logs and artifacts) + +If **Public pipelines** is disabled: + +- for **public** projects, anyone can view the pipelines, but only members + (reporter or higher) can access the job details (output logs and artifacts) +- for **internal** projects, any logged in user can view the pipelines, + but only members (reporter or higher) can access the job details (output logs + and artifacts) +- for **private** projects, only members (reporter or higher) + can view the pipelines and access the job details (output logs and artifacts) ## Auto-cancel pending pipelines diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 0570d9f471f..0cbb0c878c2 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -115,6 +115,14 @@ Deleting a protected branch is only allowed via the web interface, not via Git. This means that you can't accidentally delete a protected branch from your command line or a Git client application. +## Running pipelines on protected branches + +The permission to merge or push to protected branches is used to define if a user can +run CI/CD pipelines and execute actions on jobs that are related to those branches. + +See [Security on protected branches](../../ci/pipelines.md#security-on-protected-branches) +for details about the pipelines security model. + ## Changelog **9.2** diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index ce4dd4e99d5..6a5d2d40927 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -38,3 +38,4 @@ do. | `/award :emoji:` | Toggle award for :emoji: | | `/board_move ~column` | Move issue to column on the board | | `/duplicate #issue` | Closes this issue and marks it as a duplicate of another issue | +| `/move path/to/project` | Moves issue to another project | diff --git a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png b/doc/user/project/repository/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png Binary files differindex 8e26d98f1b0..8e26d98f1b0 100644 --- a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png +++ b/doc/user/project/repository/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png diff --git a/doc/user/project/repository/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png b/doc/user/project/repository/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png Binary files differnew file mode 100644 index 00000000000..5c14df36d73 --- /dev/null +++ b/doc/user/project/repository/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png diff --git a/doc/user/project/repository/gpg_signed_commits/img/project_signed_and_unsigned_commits.png b/doc/user/project/repository/gpg_signed_commits/img/project_signed_and_unsigned_commits.png Binary files differnew file mode 100644 index 00000000000..088ecfa6d89 --- /dev/null +++ b/doc/user/project/repository/gpg_signed_commits/img/project_signed_and_unsigned_commits.png diff --git a/doc/user/project/repository/gpg_signed_commits/img/project_signed_commit_unverified_signature.png b/doc/user/project/repository/gpg_signed_commits/img/project_signed_commit_unverified_signature.png Binary files differnew file mode 100644 index 00000000000..4e3392406b1 --- /dev/null +++ b/doc/user/project/repository/gpg_signed_commits/img/project_signed_commit_unverified_signature.png diff --git a/doc/user/project/repository/gpg_signed_commits/img/project_signed_commit_verified_signature.png b/doc/user/project/repository/gpg_signed_commits/img/project_signed_commit_verified_signature.png Binary files differnew file mode 100644 index 00000000000..766970dee81 --- /dev/null +++ b/doc/user/project/repository/gpg_signed_commits/img/project_signed_commit_verified_signature.png diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md new file mode 100644 index 00000000000..20aadb8f7ff --- /dev/null +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -0,0 +1,256 @@ +# Signing commits with GPG + +> [Introduced][ce-9546] in GitLab 9.5. + +GitLab can show whether a commit is verified or not when signed with a GPG key. +All you need to do is upload the public GPG key in your profile settings. + +GPG verified tags are not supported yet. + +## Getting started with GPG + +Here are a few guides to get you started with GPG: + +- [Git Tools - Signing Your Work](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) +- [Managing OpenPGP Keys](https://riseup.net/en/security/message-security/openpgp/gpg-keys) +- [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices) +- [Creating a new GPG key with subkeys](https://www.void.gr/kargig/blog/2013/12/02/creating-a-new-gpg-key-with-subkeys/) (advanced) + +## How GitLab handles GPG + +GitLab uses its own keyring to verify the GPG signature. It does not access any +public key server. + +In order to have a commit verified on GitLab the corresponding public key needs +to be uploaded to GitLab. For a signature to be verified three conditions need +to be met: + +1. The public key needs to be added your GitLab account +1. One of the emails in the GPG key matches your **primary** email +1. The committer's email matches the verified email from the gpg key + +## Generating a GPG key + +>**Notes:** +- If your Operating System has `gpg2` installed, replace `gpg` with `gpg2` in + the following commands. +- If Git is using `gpg` and you get errors like `secret key not available` or + `gpg: signing failed: secret key not available`, run the following command to + change to `gpg2`: + ``` + git config --global gpg.program gpg2 + ``` + +If you don't already have a GPG key, the following steps will help you get +started: + +1. [Install GPG](https://www.gnupg.org/download/index.html) for your operating system +1. Generate the private/public key pair with the following command: + + ```sh + gpg --full-gen-key + ``` + + This will spawn a series of questions. + +1. The first question is which algorithm can be used. Select the kind you want + or press <kbd>Enter</kbd> to choose the default (RSA and RSA): + + ``` + Please select what kind of key you want: + (1) RSA and RSA (default) + (2) DSA and Elgamal + (3) DSA (sign only) + (4) RSA (sign only) + Your selection? 1 + ``` + +1. The next question is key length. We recommend to choose the highest value + which is `4096`: + + ``` + RSA keys may be between 1024 and 4096 bits long. + What keysize do you want? (2048) 4096 + Requested keysize is 4096 bits + ``` +1. Next, you need to specify the validity period of your key. This is something + subjective, and you can use the default value which is to never expire: + + ``` + Please specify how long the key should be valid. + 0 = key does not expire + <n> = key expires in n days + <n>w = key expires in n weeks + <n>m = key expires in n months + <n>y = key expires in n years + Key is valid for? (0) 0 + Key does not expire at all + ``` + +1. Confirm that the answers you gave were correct by typing `y`: + + ``` + Is this correct? (y/N) y + ``` + +1. Enter you real name, the email address to be associated with this key (should + match the primary email address you use in GitLab) and an optional comment + (press <kbd>Enter</kbd> to skip): + + ``` + GnuPG needs to construct a user ID to identify your key. + + Real name: Mr. Robot + Email address: mr@robot.sh + Comment: + You selected this USER-ID: + "Mr. Robot <mr@robot.sh>" + + Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O + ``` + +1. Pick a strong password when asked and type it twice to confirm. +1. Use the following command to list the private GPG key you just created: + + ``` + gpg --list-secret-keys mr@robot.sh + ``` + + Replace `mr@robot.sh` with the email address you entered above. + +1. Copy the GPG key ID that starts with `sec`. In the following example, that's + `0x30F2B65B9246B6CA`: + + ``` + sec rsa4096/0x30F2B65B9246B6CA 2017-08-18 [SC] + D5E4F29F3275DC0CDA8FFC8730F2B65B9246B6CA + uid [ultimate] Mr. Robot <mr@robot.sh> + ssb rsa4096/0xB7ABC0813E4028C0 2017-08-18 [E] + ``` + +1. Export the public key of that ID (replace your key ID from the previous step): + + ``` + gpg --armor --export 0x30F2B65B9246B6CA + ``` + +1. Finally, copy the public key and [add it in your profile settings](#adding-a-gpg-key-to-your-account) + +## Adding a GPG key to your account + +>**Note:** +Once you add a key, you cannot edit it, only remove it. In case the paste +didn't work, you'll have to remove the offending key and re-add it. + +You can add a GPG key in your profile's settings: + +1. On the upper right corner, click on your avatar and go to your **Settings**. + +  + +1. Navigate to the **GPG keys** tab and paste your _public_ key in the 'Key' + box. + +  + +1. Finally, click on **Add key** to add it to GitLab. You will be able to see + its fingerprint, the corresponding email address and creation date. + +  + +## Associating your GPG key with Git + +After you have [created your GPG key](#generating-a-gpg-key) and [added it to +your account](#adding-a-gpg-key-to-your-account), it's time to tell Git which +key to use. + +1. Use the following command to list the private GPG key you just created: + + ``` + gpg --list-secret-keys mr@robot.sh + ``` + + Replace `mr@robot.sh` with the email address you entered above. + +1. Copy the GPG key ID that starts with `sec`. In the following example, that's + `0x30F2B65B9246B6CA`: + + ``` + sec rsa4096/0x30F2B65B9246B6CA 2017-08-18 [SC] + D5E4F29F3275DC0CDA8FFC8730F2B65B9246B6CA + uid [ultimate] Mr. Robot <mr@robot.sh> + ssb rsa4096/0xB7ABC0813E4028C0 2017-08-18 [E] + ``` + +1. Tell Git to use that key to sign the commits: + + ``` + git config --global user.signingkey 0x30F2B65B9246B6CA + ``` + + Replace `0x30F2B65B9246B6CA` with your GPG key ID. + +## Signing commits + +After you have [created your GPG key](#generating-a-gpg-key) and [added it to +your account](#adding-a-gpg-key-to-your-account), you can start signing your +commits: + +1. Commit like you used to, the only difference is the addition of the `-S` flag: + + ``` + git commit -S -m "My commit msg" + ``` + +1. Enter the passphrase of your GPG key when asked. +1. Push to GitLab and check that your commits [are verified](#verifying-commits). + +If you don't want to type the `-S` flag every time you commit, you can tell Git +to sign your commits automatically: + +``` +git config --global commit.gpgsign true +``` + +## Verifying commits + +1. Within a project or [merge request](../../merge_requests/index.md), navigate to + the **Commits** tab. Signed commits will show a badge containing either + "Verified" or "Unverified", depending on the verification status of the GPG + signature. + +  + +1. By clicking on the GPG badge, details of the signature are displayed. + +  + +  + +## Revoking a GPG key + +Revoking a key **unverifies** already signed commits. Commits that were +verified by using this key will change to an unverified state. Future commits +will also stay unverified once you revoke this key. This action should be used +in case your key has been compromised. + +To revoke a GPG key: + +1. On the upper right corner, click on your avatar and go to your **Settings**. +1. Navigate to the **GPG keys** tab. +1. Click on **Revoke** besides the GPG key you want to delete. + +## Removing a GPG key + +Removing a key **does not unverify** already signed commits. Commits that were +verified by using this key will stay verified. Only unpushed commits will stay +unverified once you remove this key. To unverify already signed commits, you need +to [revoke the associated GPG key](#revoking-a-gpg-key) from your account. + +To remove a GPG key from your account: + +1. On the upper right corner, click on your avatar and go to your **Settings**. +1. Navigate to the **GPG keys** tab. +1. Click on the trash icon besides the GPG key you want to delete. + +[ce-9546]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9546 diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 4b2c435a120..235af83353d 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -20,7 +20,9 @@ documentation. For security reasons, when using the command line, we strongly recommend you to [connect with GitLab via SSH](../../../ssh/README.md). -## Create and edit files +## Files + +### Create and edit files Host your codebase in GitLab repositories by pushing your files to GitLab. You can either use the user interface (UI), or connect your local computer @@ -47,6 +49,10 @@ it's easier to do so [via GitLab UI](web_editor.md): To get started with the command line, please read through the [command line basics documentation](../../../gitlab-basics/command-line-commands.md). +### Find files + +Use GitLab's [file finder](../../../workflow/file_finder.md) to search for files in a repository. + ## Branches When you submit changes in a new branch, you create a new version @@ -105,6 +111,8 @@ right from the UI. - **Revert a commit:** Easily [revert a commit](../merge_requests/revert_changes.md#reverting-a-commit) from the UI to a selected branch. +- **Sign a commit:** +Use GPG to [sign your commits](gpg_signed_commits/index.md). ## Repository size diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 35960ade3d4..23b1c61cd16 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -9,6 +9,9 @@ > application settings (`/admin/application_settings`) under 'Import sources'. > Ask your administrator if you don't see the **GitLab export** button when > creating a new project. +> - Starting with GitLab 10.0, administrators can disable the project export option +> on the GitLab instance in application settings (`/admin/application_settings`) +> under 'Visibility and Access Controls'. > - You can find some useful raketasks if you are an administrator in the > [import_export](../../../administration/raketasks/project_import_export.md) > raketask. @@ -25,17 +28,18 @@ with all their related data and be moved into a new GitLab instance. ## Version history -| GitLab version | Import/Export version | -| -------- | -------- | -| 9.4.0 to current | 0.1.8 | -| 9.2.0 | 0.1.7 | -| 8.17.0 | 0.1.6 | -| 8.13.0 | 0.1.5 | -| 8.12.0 | 0.1.4 | -| 8.10.3 | 0.1.3 | -| 8.10.0 | 0.1.2 | -| 8.9.5 | 0.1.1 | -| 8.9.0 | 0.1.0 | +| GitLab version | Import/Export version | +| ---------------- | --------------------- | +| 10.0 to current | 0.2.0 | +| 9.4.0 | 0.1.8 | +| 9.2.0 | 0.1.7 | +| 8.17.0 | 0.1.6 | +| 8.13.0 | 0.1.5 | +| 8.12.0 | 0.1.4 | +| 8.10.3 | 0.1.3 | +| 8.10.0 | 0.1.2 | +| 8.9.5 | 0.1.1 | +| 8.9.0 | 0.1.0 | > The table reflects what GitLab version we updated the Import/Export version at. > For instance, 8.10.3 and 8.11 will have the same Import/Export version (0.1.3) diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index e9ee1abc6c1..c0b8a87f038 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -4,7 +4,7 @@ A separate system for documentation called Wiki, is built right into each GitLab project. It is enabled by default on all new projects and you can find it under **Wiki** in your project. -Wikis are very convenient if you don't want to keep you documentation in your +Wikis are very convenient if you don't want to keep your documentation in your repository, but you do want to keep it in the same project where your code resides. diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md new file mode 100644 index 00000000000..50ec99be48b --- /dev/null +++ b/doc/user/reserved_names.md @@ -0,0 +1,109 @@ +# Reserved project and group names + +Not all project & group names are allowed because they would conflict with +existing routes used by GitLab. + +For a list of words that are not allowed to be used as group or project names, see the +[`path_regex.rb` file][reserved] under the `TOP_LEVEL_ROUTES`, `PROJECT_WILDCARD_ROUTES` and `GROUP_ROUTES` lists: +- `TOP_LEVEL_ROUTES`: are names that are reserved as usernames or top level groups +- `PROJECT_WILDCARD_ROUTES`: are names that are reserved for child groups or projects. +- `GROUP_ROUTES`: are names that are reserved for all groups or projects. + +## Reserved project names + +It is currently not possible to create a project with the following names: + +- - +- badges +- blame +- blob +- builds +- commits +- create +- create_dir +- edit +- environments/folders +- files +- find_file +- gitlab-lfs/objects +- info/lfs/objects +- new +- preview +- raw +- refs +- tree +- update +- wikis + +## Reserved group names + +Currently the following names are reserved as top level groups: + +- 503.html +- - +- .well-known +- 404.html +- 422.html +- 500.html +- 502.html +- abuse_reports +- admin +- api +- apple-touch-icon-precomposed.png +- apple-touch-icon.png +- files +- assets +- autocomplete +- ci +- dashboard +- deploy.html +- explore +- favicon.ico +- groups +- header_logo_dark.png +- header_logo_light.png +- health_check +- help +- import +- invites +- jwt +- koding +- notification_settings +- oauth +- profile +- projects +- public +- robots.txt +- s +- search +- sent_notifications +- slash-command-logo.png +- snippets +- u +- unicorn_test +- unsubscribes +- uploads +- users + +These group names are unavailable as subgroup names: + +- - +- activity +- analytics +- audit_events +- avatar +- edit +- group_members +- hooks +- issues +- labels +- ldap +- ldap_group_links +- merge_requests +- milestones +- notification_setting +- pipeline_quota +- projects +- subgroups + +[reserved]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/path_regex.rb diff --git a/doc/user/search/img/group_issues_filter.png b/doc/user/search/img/group_issues_filter.png Binary files differnew file mode 100644 index 00000000000..45eced79b99 --- /dev/null +++ b/doc/user/search/img/group_issues_filter.png diff --git a/doc/user/search/img/issue_search_by_term.png b/doc/user/search/img/issue_search_by_term.png Binary files differnew file mode 100644 index 00000000000..3cefa3adb8b --- /dev/null +++ b/doc/user/search/img/issue_search_by_term.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 6d59dcc6c75..21e96d8b11c 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -27,7 +27,7 @@ on the search field on the top-right of your screen:  -## Issues and merge requests per project +### Issues and merge requests per project If you want to search for issues present in a specific project, navigate to a project's **Issues** tab, and click on the field **Search or filter results...**. It will @@ -40,6 +40,32 @@ The same process is valid for merge requests. Navigate to your project's **Merge and click **Search or filter results...**. Merge requests can be filtered by author, assignee, milestone, and label. +### Searching for specific terms + +You can filter issues and merge requests by specific terms included in titles or descriptions. + +* Syntax + * Searches look for all the words in a query, in any order. E.g.: searching + issues for `display bug` will return all issues matching both those words, in any order. + * To find the exact term, use double quotes: `"display bug"` +* Limitation + * For performance reasons, terms shorter than 3 chars are ignored. E.g.: searching + issues for `included in titles` is same as `included titles` + + + +### Issues and merge requests per group + +Similar to **Issues and merge requests per project**, you can also search for issues +within a group. Navigate to a group's **Issues** tab and query search results in +the same way as you do for projects. + + + +The same process is valid for merge requests. Navigate to your project's **Merge Requests** tab. +The search and filter UI currently uses dropdowns. In a future release, the same +dynamic UI as above will be carried over here. + ## Search history You can view recent searches by clicking on the little arrow-clock icon, which is to the left of the search input. Click the search entry to run that search again. This feature is available for issues and merge requests. Searches are stored locally in your browser. diff --git a/doc/user/snippets.md b/doc/user/snippets.md index 78861625f8a..2170b079f62 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -16,7 +16,7 @@ Comments on snippets was [introduced](https://gitlab.com/gitlab-org/gitlab-ce/is ## Project snippets -Project snippets are always related to a specific project - see [Project features](../workflow/project_features.md) for more information. +Project snippets are always related to a specific project - see [Project's features](project/index.md#project-39-s-features) for more information. ## Personal snippets diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 925bbf76d49..673e08287a3 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -16,14 +16,13 @@ - [File finder](file_finder.md) - [Labels](../user/project/labels.md) - [Notification emails](notifications.md) -- [Project Features](project_features.md) +- [Projects](../user/project/index.md) - [Project forking workflow](forking_workflow.md) -- [Project users](add-user/add-user.md) +- [Project users](../user/project/members/index.md) - [Protected branches](../user/project/protected_branches.md) - [Protected tags](../user/project/protected_tags.md) - [Quick Actions](../user/project/quick_actions.md) -- [Sharing a project with a group](share_with_group.md) -- [Share projects with other groups](share_projects_with_other_groups.md) +- [Sharing projects with groups](../user/project/members/share_project_with_groups.md) - [Time tracking](time_tracking.md) - [Web Editor](../user/project/repository/web_editor.md) - [Releases](releases.md) diff --git a/doc/workflow/add-user/add-user.md b/doc/workflow/add-user/add-user.md index e541111d7b3..35cc080d2b7 100644 --- a/doc/workflow/add-user/add-user.md +++ b/doc/workflow/add-user/add-user.md @@ -1,114 +1 @@ -# Project users - -You can manage the groups and users and their access levels in all of your -projects. You can also personalize the access level you give each user, -per-project. - -You should have `master` or `owner` permissions to add or import a new user -to your project. - -The first step to add or import a user, go to your project and click on -**Members** in the drop-down menu on the right side of your screen. - - - ---- - -## Add a user - -Right next to **People**, start typing the name or username of the user you -want to add. - - - ---- - -Select the user and the [permission level](../../user/permissions.md) -that you'd like to give the user. Note that you can select more than one user. - - - ---- - -Once done, hit **Add users to project** and they will be immediately added to -your project with the permissions you gave them above. - - - ---- - -From there on, you can either remove an existing user or change their access -level to the project. - -## Import users from another project - -You can import another project's users in your own project by hitting the -**Import members** button on the upper right corner of the **Members** menu. - -In the dropdown menu, you can see only the projects you are Master on. - - - ---- - -Select the one you want and hit **Import project members**. A flash message -notifying you that the import was successful will appear, and the new members -are now in the project's members list. Notice that the permissions that they -had on the project you imported from are retained. - - - ---- - -## Invite people using their e-mail address - -If a user you want to give access to doesn't have an account on your GitLab -instance, you can invite them just by typing their e-mail address in the -user search field. - - - ---- - -As you can imagine, you can mix inviting multiple people and adding existing -GitLab users to the project. - - - ---- - -Once done, hit **Add users to project** and watch that there is a new member -with the e-mail address we used above. From there on, you can resend the -invitation, change their access level or even delete them. - - - ---- - -Once the user accepts the invitation, they will be prompted to create a new -GitLab account using the same e-mail address the invitation was sent to. - -## Request access to a project - -As a project owner you can enable or disable non members to request access to -your project. Go to the project settings and click on **Allow users to request access**. - -As a user, you can request to be a member of a project. Go to the project you'd -like to be a member of, and click the **Request Access** button on the right -side of your screen. - - - ---- - -Project owners & masters will be notified of your request and will be able to approve or -decline it on the members page. - - - ---- - -If you change your mind before your request is approved, just click the -**Withdraw Access Request** button. - - +This document was moved to [../../user/project/members/index.md](../../user/project/members/index.md) diff --git a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys.png b/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys.png Binary files differdeleted file mode 100644 index e525083918b..00000000000 --- a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys.png +++ /dev/null diff --git a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png b/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png Binary files differdeleted file mode 100644 index f715c46adc3..00000000000 --- a/doc/workflow/gpg_signed_commits/img/profile_settings_gpg_keys_single_key.png +++ /dev/null diff --git a/doc/workflow/gpg_signed_commits/img/project_signed_and_unsigned_commits.png b/doc/workflow/gpg_signed_commits/img/project_signed_and_unsigned_commits.png Binary files differdeleted file mode 100644 index 16ec2d031ae..00000000000 --- a/doc/workflow/gpg_signed_commits/img/project_signed_and_unsigned_commits.png +++ /dev/null diff --git a/doc/workflow/gpg_signed_commits/img/project_signed_commit_unverified_signature.png b/doc/workflow/gpg_signed_commits/img/project_signed_commit_unverified_signature.png Binary files differdeleted file mode 100644 index 22565cf7c7e..00000000000 --- a/doc/workflow/gpg_signed_commits/img/project_signed_commit_unverified_signature.png +++ /dev/null diff --git a/doc/workflow/gpg_signed_commits/img/project_signed_commit_verified_signature.png b/doc/workflow/gpg_signed_commits/img/project_signed_commit_verified_signature.png Binary files differdeleted file mode 100644 index 1778b2ddf2b..00000000000 --- a/doc/workflow/gpg_signed_commits/img/project_signed_commit_verified_signature.png +++ /dev/null diff --git a/doc/workflow/gpg_signed_commits/index.md b/doc/workflow/gpg_signed_commits/index.md deleted file mode 100644 index 7d5762d2b9d..00000000000 --- a/doc/workflow/gpg_signed_commits/index.md +++ /dev/null @@ -1,84 +0,0 @@ -# Signing commits with GPG - -## Getting started - -- [Git Tools - Signing Your Work](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) -- [Git Tools - Signing Your Work: GPG introduction](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work#_gpg_introduction) -- [Git Tools - Signing Your Work: Signing commits](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work#_signing_commits) - -## How GitLab handles GPG - -GitLab uses its own keyring to verify the GPG signature. It does not access any -public key server. - -In order to have a commit verified on GitLab the corresponding public key needs -to be uploaded to GitLab. - -For a signature to be verified two prerequisites need to be met: - -1. The public key needs to be added to GitLab -1. One of the emails in the GPG key matches your **primary** email - -## Add a GPG key - -1. On the upper right corner, click on your avatar and go to your **Settings**. - -  - -1. Navigate to the **GPG keys** tab. - -  - -1. Paste your **public** key in the 'Key' box. - -  - -1. Finally, click on **Add key** to add it to GitLab. You will be able to see - its fingerprint, the corresponding email address and creation date. - -  - ->**Note:** -Once you add a key, you cannot edit it, only remove it. In case the paste -didn't work, you will have to remove the offending key and re-add it. - -## Remove a GPG key - -1. On the upper right corner, click on your avatar and go to your **Settings**. - -1. Navigate to the **GPG keys** tab. - -1. Click on the trash icon besides the GPG key you want to delete. - ->**Note:** -Removing a key **does not unverify** already signed commits. Commits that were -verified by using this key will stay verified. Only unpushed commits will stay -unverified once you remove this key. - -## Revoke a GPG key - -1. On the upper right corner, click on your avatar and go to your **Settings**. - -1. Navigate to the **GPG keys** tab. - -1. Click on **Revoke** besides the GPG key you want to delete. - ->**Note:** -Revoking a key **unverifies** already signed commits. Commits that were -verified by using this key will change to an unverified state. Future commits -will also stay unverified once you revoke this key. This action should be used -in case your key has been compromised. - -## Verifying commits - -1. Within a project navigate to the **Commits** tag. Signed commits will show a - badge containing either "Verified" or "Unverified", depending on the - verification status of the GPG signature. - -  - -1. By clicking on the GPG badge details of the signature are displayed. - -  - -  diff --git a/doc/workflow/groups/other_group_sees_shared_project.png b/doc/workflow/groups/other_group_sees_shared_project.png Binary files differdeleted file mode 100644 index 67af27043eb..00000000000 --- a/doc/workflow/groups/other_group_sees_shared_project.png +++ /dev/null diff --git a/doc/workflow/groups/share_project_with_groups.png b/doc/workflow/groups/share_project_with_groups.png Binary files differdeleted file mode 100644 index 3cb4796f9f7..00000000000 --- a/doc/workflow/groups/share_project_with_groups.png +++ /dev/null diff --git a/doc/workflow/importing/README.md b/doc/workflow/importing/README.md index 2d91bee0e94..f753708ad89 100644 --- a/doc/workflow/importing/README.md +++ b/doc/workflow/importing/README.md @@ -1,17 +1 @@ -# Migrating projects to a GitLab instance
-
-1. [Bitbucket](import_projects_from_bitbucket.md)
-1. [GitHub](import_projects_from_github.md)
-1. [GitLab.com](import_projects_from_gitlab_com.md)
-1. [FogBugz](import_projects_from_fogbugz.md)
-1. [Gitea](import_projects_from_gitea.md)
-1. [SVN](migrating_from_svn.md)
-
-In addition to the specific migration documentation above, you can import any
-Git repository via HTTP from the New Project page. Be aware that if the
-repository is too large the import can timeout.
-
-### Migrating from self-hosted GitLab to GitLab.com
-
-You can copy your repos by changing the remote and pushing to the new server;
-but issues and merge requests can't be imported.
+This document was moved to a [new location](../../user/project/import/index.md).
diff --git a/doc/workflow/importing/import_projects_from_bitbucket.md b/doc/workflow/importing/import_projects_from_bitbucket.md index f3c636ed1d5..248c3990372 100644 --- a/doc/workflow/importing/import_projects_from_bitbucket.md +++ b/doc/workflow/importing/import_projects_from_bitbucket.md @@ -1,62 +1 @@ -# Import your project from Bitbucket to GitLab
-
-Import your projects from Bitbucket to GitLab with minimal effort.
-
-## Overview
-
->**Note:**
-The [Bitbucket integration][bb-import] must be first enabled in order to be
-able to import your projects from Bitbucket. Ask your GitLab administrator
-to enable this if not already.
-
-- At its current state, the Bitbucket importer can import:
- - the repository description (GitLab 7.7+)
- - the Git repository data (GitLab 7.7+)
- - the issues (GitLab 7.7+)
- - the issue comments (GitLab 8.15+)
- - the pull requests (GitLab 8.4+)
- - the pull request comments (GitLab 8.15+)
- - the milestones (GitLab 8.15+)
- - the wiki (GitLab 8.15+)
-- References to pull requests and issues are preserved (GitLab 8.7+)
-- Repository public access is retained. If a repository is private in Bitbucket
- it will be created as private in GitLab as well.
-
-
-## How it works
-
-When issues/pull requests are being imported, the Bitbucket importer tries to find
-the Bitbucket author/assignee in GitLab's database using the Bitbucket ID. For this
-to work, the Bitbucket author/assignee should have signed in beforehand in GitLab
-and **associated their Bitbucket account**. If the user is not
-found in GitLab's database, the project creator (most of the times the current
-user that started the import process) is set as the author, but a reference on
-the issue about the original Bitbucket author is kept.
-
-The importer will create any new namespaces (groups) if they don't exist or in
-the case the namespace is taken, the repository will be imported under the user's
-namespace that started the import process.
-
-## Importing your Bitbucket repositories
-
-1. Sign in to GitLab and go to your dashboard.
-1. Click on **New project**.
-
- 
-
-1. Click on the "Bitbucket" button
-
- 
-
-1. Grant GitLab access to your Bitbucket account
-
- 
-
-1. Click on the projects that you'd like to import or **Import all projects**.
- You can also select the namespace under which each project will be
- imported.
-
- 
-
-[bb-import]: ../../integration/bitbucket.md
-[social sign-in]: ../../user/profile/account/social_sign_in.md
+This document was moved to a [new location](../../user/project/import/bitbucket.md).
diff --git a/doc/workflow/importing/import_projects_from_fogbugz.md b/doc/workflow/importing/import_projects_from_fogbugz.md index 71af0f9ea44..050746e2b4d 100644 --- a/doc/workflow/importing/import_projects_from_fogbugz.md +++ b/doc/workflow/importing/import_projects_from_fogbugz.md @@ -1,29 +1 @@ -# Import your project from FogBugz to GitLab - -It only takes a few simple steps to import your project from FogBugz. -The importer will import all of your cases and comments with original case -numbers and timestamps. You will also have the opportunity to map FogBugz -users to GitLab users. - -* From your GitLab dashboard click 'New project' - -* Click on the 'FogBugz' button - - - -* Enter your FogBugz URL, email address, and password. - - - -* Create mapping from FogBugz users to GitLab users. - - - -* Select the projects you wish to import by clicking the Import buttons - - - -* Once the import has finished click the link to take you to the project -dashboard. Follow the directions to push your existing repository. - - +This document was moved to a [new location](../../user/project/import/fogbugz.md). diff --git a/doc/workflow/importing/import_projects_from_gitea.md b/doc/workflow/importing/import_projects_from_gitea.md index f5746a0fb31..cb90c490b0f 100644 --- a/doc/workflow/importing/import_projects_from_gitea.md +++ b/doc/workflow/importing/import_projects_from_gitea.md @@ -1,77 +1 @@ -# Import your project from Gitea to GitLab - -Import your projects from Gitea to GitLab with minimal effort. - -## Overview - ->**Note:** -This requires Gitea `v1.0.0` or newer. - -- At its current state, Gitea importer can import: - - the repository description (GitLab 8.15+) - - the Git repository data (GitLab 8.15+) - - the issues (GitLab 8.15+) - - the pull requests (GitLab 8.15+) - - the milestones (GitLab 8.15+) - - the labels (GitLab 8.15+) -- Repository public access is retained. If a repository is private in Gitea - it will be created as private in GitLab as well. - -## How it works - -Since Gitea is currently not an OAuth provider, author/assignee cannot be mapped -to users in your GitLab's instance. This means that the project creator (most of -the times the current user that started the import process) is set as the author, -but a reference on the issue about the original Gitea author is kept. - -The importer will create any new namespaces (groups) if they don't exist or in -the case the namespace is taken, the repository will be imported under the user's -namespace that started the import process. - -## Importing your Gitea repositories - -The importer page is visible when you create a new project. - - - -Click on the **Gitea** link and the import authorization process will start. - - - -### Authorize access to your repositories using a personal access token - -With this method, you will perform a one-off authorization with Gitea to grant -GitLab access your repositories: - -1. Go to <https://you-gitea-instance/user/settings/applications> (replace - `you-gitea-instance` with the host of your Gitea instance). -1. Click **Generate New Token**. -1. Enter a token description. -1. Click **Generate Token**. -1. Copy the token hash. -1. Go back to GitLab and provide the token to the Gitea importer. -1. Hit the **List Your Gitea Repositories** button and wait while GitLab reads - your repositories' information. Once done, you'll be taken to the importer - page to select the repositories to import. - -### Select which repositories to import - -After you've authorized access to your Gitea repositories, you will be -redirected to the Gitea importer page. - -From there, you can see the import statuses of your Gitea repositories. - -- Those that are being imported will show a _started_ status, -- those already successfully imported will be green with a _done_ status, -- whereas those that are not yet imported will have an **Import** button on the - right side of the table. - -If you want, you can import all your Gitea projects in one go by hitting -**Import all projects** in the upper left corner. - - - ---- - -You can also choose a different name for the project and a different namespace, -if you have the privileges to do so. +This document was moved to a [new location](../../user/project/import/gitea.md). diff --git a/doc/workflow/importing/import_projects_from_github.md b/doc/workflow/importing/import_projects_from_github.md index 8ed1d98d05b..13639feaa04 100644 --- a/doc/workflow/importing/import_projects_from_github.md +++ b/doc/workflow/importing/import_projects_from_github.md @@ -1,122 +1 @@ -# Import your project from GitHub to GitLab
-
-Import your projects from GitHub to GitLab with minimal effort.
-
-## Overview
-
->**Note:**
-If you are an administrator you can enable the [GitHub integration][gh-import]
-in your GitLab instance sitewide. This configuration is optional, users will
-still be able to import their GitHub repositories with a
-[personal access token][gh-token].
-
->**Note:**
-Administrators of a GitLab instance (Community or Enterprise Edition) can also
-use the [GitHub rake task][gh-rake] to import projects from GitHub without the
-constrains of a Sidekiq worker.
-
-- At its current state, GitHub importer can import:
- - the repository description (GitLab 7.7+)
- - the Git repository data (GitLab 7.7+)
- - the issues (GitLab 7.7+)
- - the pull requests (GitLab 8.4+)
- - the wiki pages (GitLab 8.4+)
- - the milestones (GitLab 8.7+)
- - the labels (GitLab 8.7+)
- - the release note descriptions (GitLab 8.12+)
-- References to pull requests and issues are preserved (GitLab 8.7+)
-- Repository public access is retained. If a repository is private in GitHub
- it will be created as private in GitLab as well.
-
-## How it works
-
-When issues/pull requests are being imported, the GitHub importer tries to find
-the GitHub author/assignee in GitLab's database using the GitHub ID. For this
-to work, the GitHub author/assignee should have signed in beforehand in GitLab
-and **associated their GitHub account**. If the user is not
-found in GitLab's database, the project creator (most of the times the current
-user that started the import process) is set as the author, but a reference on
-the issue about the original GitHub author is kept.
-
-The importer will create any new namespaces (groups) if they don't exist or in
-the case the namespace is taken, the repository will be imported under the user's
-namespace that started the import process.
-
-## Importing your GitHub repositories
-
-The importer page is visible when you create a new project.
-
-
-
-Click on the **GitHub** link and the import authorization process will start.
-There are two ways to authorize access to your GitHub repositories:
-
-1. [Using the GitHub integration][gh-integration] (if it's enabled by your
- GitLab administrator). This is the preferred way as it's possible to
- preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works)
- section.
-1. [Using a personal access token][gh-token] provided by GitHub.
-
-
-
-### Authorize access to your repositories using the GitHub integration
-
-If the [GitHub integration][gh-import] is enabled by your GitLab administrator,
-you can use it instead of the personal access token.
-
-1. First you may want to connect your GitHub account to GitLab in order for
- the username mapping to be correct.
-1. Once you connect GitHub, click the **List your GitHub repositories** button
- and you will be redirected to GitHub for permission to access your projects.
-1. After accepting, you'll be automatically redirected to the importer.
-
-You can now go on and [select which repositories to import](#select-which-repositories-to-import).
-
-### Authorize access to your repositories using a personal access token
-
->**Note:**
-For a proper author/assignee mapping for issues and pull requests, the
-[GitHub integration][gh-integration] should be used instead of the
-[personal access token][gh-token]. If the GitHub integration is enabled by your
-GitLab administrator, it should be the preferred method to import your repositories.
-Read more in the [How it works](#how-it-works) section.
-
-If you are not using the GitHub integration, you can still perform a one-off
-authorization with GitHub to grant GitLab access your repositories:
-
-1. Go to <https://github.com/settings/tokens/new>.
-1. Enter a token description.
-1. Check the `repo` scope.
-1. Click **Generate token**.
-1. Copy the token hash.
-1. Go back to GitLab and provide the token to the GitHub importer.
-1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads
- your repositories' information. Once done, you'll be taken to the importer
- page to select the repositories to import.
-
-### Select which repositories to import
-
-After you've authorized access to your GitHub repositories, you will be
-redirected to the GitHub importer page.
-
-From there, you can see the import statuses of your GitHub repositories.
-
-- Those that are being imported will show a _started_ status,
-- those already successfully imported will be green with a _done_ status,
-- whereas those that are not yet imported will have an **Import** button on the
- right side of the table.
-
-If you want, you can import all your GitHub projects in one go by hitting
-**Import all projects** in the upper left corner.
-
-
-
----
-
-You can also choose a different name for the project and a different namespace,
-if you have the privileges to do so.
-
-[gh-import]: ../../integration/github.md "GitHub integration"
-[gh-rake]: ../../administration/raketasks/github_import.md "GitHub rake task"
-[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration
-[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token
+This document was moved to a [new location](../../user/project/import/github.md).
diff --git a/doc/workflow/importing/import_projects_from_gitlab_com.md b/doc/workflow/importing/import_projects_from_gitlab_com.md index b27125a44de..df4c180919a 100644 --- a/doc/workflow/importing/import_projects_from_gitlab_com.md +++ b/doc/workflow/importing/import_projects_from_gitlab_com.md @@ -1,21 +1 @@ -# Project importing from GitLab.com to your private GitLab instance - -You can import your existing GitLab.com projects to your GitLab instance. But keep in mind that it is possible only if -GitLab support is enabled on your GitLab instance. -You can read more about GitLab support [here](http://docs.gitlab.com/ce/integration/gitlab.html) -To get to the importer page you need to go to "New project" page. - ->**Note:** -If you are interested in importing Wiki and Merge Request data to your new instance, you'll need to follow the instructions for [project export](../../user/project/settings/import_export.md) - - - -Click on the "Import projects from GitLab.com" link and you will be redirected to GitLab.com -for permission to access your projects. After accepting, you'll be automatically redirected to the importer. - - - - - -To import a project, you can simple click "Import". The importer will import your repository and issues. -Once the importer is done, a new GitLab project will be created with your imported data.
\ No newline at end of file +This document was moved to a [new location](../../user/project/import/gitlab_com.md). diff --git a/doc/workflow/importing/migrating_from_svn.md b/doc/workflow/importing/migrating_from_svn.md index 7a3628a39d7..81df3fbcdbb 100644 --- a/doc/workflow/importing/migrating_from_svn.md +++ b/doc/workflow/importing/migrating_from_svn.md @@ -1,183 +1 @@ -# Migrating from SVN to GitLab - -Subversion (SVN) is a central version control system (VCS) while -Git is a distributed version control system. There are some major differences -between the two, for more information consult your favorite search engine. - -## Overview - -There are two approaches to SVN to Git migration: - -1. [Git/SVN Mirror](#smooth-migration-with-a-gitsvn-mirror-using-subgit) which: - - Makes the GitLab repository to mirror the SVN project. - - Git and SVN repositories are kept in sync; you can use either one. - - Smoothens the migration process and allows to manage migration risks. - -1. [Cut over migration](#cut-over-migration-with-svn2git) which: - - Translates and imports the existing data and history from SVN to Git. - - Is a fire and forget approach, good for smaller teams. - -## Smooth migration with a Git/SVN mirror using SubGit - -[SubGit](https://subgit.com) is a tool for a smooth, stress-free SVN to Git -migration. It creates a writable Git mirror of a local or remote Subversion -repository and that way you can use both Subversion and Git as long as you like. -It requires access to your GitLab server as it talks with the Git repositories -directly in a filesystem level. - -### SubGit prerequisites - -1. Install Oracle JRE 1.8 or newer. On Debian-based Linux distributions you can - follow [this article](http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html). -1. Download SubGit from https://subgit.com/download/. -1. Unpack the downloaded SubGit zip archive to the `/opt` directory. The `subgit` - command will be available at `/opt/subgit-VERSION/bin/subgit`. - -### SubGit configuration - -The first step to mirror you SVN repository in GitLab is to create a new empty -project which will be used as a mirror. For Omnibus installations the path to -the repository will be located at -`/var/opt/gitlab/git-data/repositories/USER/REPO.git` by default. For -installations from source, the default repository directory will be -`/home/git/repositories/USER/REPO.git`. For convenience, assign this path to a -variable: - -``` -GIT_REPO_PATH=/var/opt/gitlab/git-data/repositories/USER/REPOS.git -``` - -SubGit will keep this repository in sync with a remote SVN project. For -convenience, assign your remote SVN project URL to a variable: - -``` -SVN_PROJECT_URL=http://svn.company.com/repos/project -``` - -Next you need to run SubGit to set up a Git/SVN mirror. Make sure the following -`subgit` command is ran on behalf of the same user that keeps ownership of -GitLab Git repositories (by default `git`): - -``` -subgit configure --layout auto $SVN_PROJECT_URL $GIT_REPO_PATH -``` - -Adjust authors and branches mappings, if necessary. Open with your favorite -text editor: - -``` -edit $GIT_REPO_PATH/subgit/authors.txt -edit $GIT_REPO_PATH/subgit/config -``` - -For more information regarding the SubGit configuration options, refer to -[SubGit's documentation](https://subgit.com/documentation.html) website. - -### Initial translation - -Now that SubGit has configured the Git/SVN repos, run `subgit` to perform the -initial translation of existing SVN revisions into the Git repository: - -``` -subgit install $GIT_REPO_PATH -``` - -After the initial translation is completed, the Git repository and the SVN -project will be kept in sync by `subgit` - new Git commits will be translated to -SVN revisions and new SVN revisions will be translated to Git commits. Mirror -works transparently and does not require any special commands. - -If you would prefer to perform one-time cut over migration with `subgit`, use -the `import` command instead of `install`: - -``` -subgit import $GIT_REPO_PATH -``` - -### SubGit licensing - -Running SubGit in a mirror mode requires a -[registration](https://subgit.com/pricing.html). Registration is free for open -source, academic and startup projects. - -We're currently working on deeper GitLab/SubGit integration. You may track our -progress at [this issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/990). - -### SubGit support - -For any questions related to SVN to GitLab migration with SubGit, you can -contact the SubGit team directly at [support@subgit.com](mailto:support@subgit.com). - -## Cut over migration with svn2git - -If you are currently using an SVN repository, you can migrate the repository -to Git and GitLab. We recommend a hard cut over - run the migration command once -and then have all developers start using the new GitLab repository immediately. -Otherwise, it's hard to keep changing in sync in both directions. The conversion -process should be run on a local workstation. - -Install `svn2git`. On all systems you can install as a Ruby gem if you already -have Ruby and Git installed. - -```bash -sudo gem install svn2git -``` - -On Debian-based Linux distributions you can install the native packages: - -```bash -sudo apt-get install git-core git-svn ruby -``` - -Optionally, prepare an authors file so `svn2git` can map SVN authors to Git authors. -If you choose not to create the authors file then commits will not be attributed -to the correct GitLab user. Some users may not consider this a big issue while -others will want to ensure they complete this step. If you choose to map authors -you will be required to map every author that is present on changes in the SVN -repository. If you don't, the conversion will fail and you will have to update -the author file accordingly. The following command will search through the -repository and output a list of authors. - -```bash -svn log --quiet | grep -E "r[0-9]+ \| .+ \|" | cut -d'|' -f2 | sed 's/ //g' | sort | uniq -``` - -Use the output from the last command to construct the authors file. -Create a file called `authors.txt` and add one mapping per line. - -``` -janedoe = Jane Doe <janedoe@example.com> -johndoe = John Doe <johndoe@example.com> -``` - -If your SVN repository is in the standard format (trunk, branches, tags, -not nested) the conversion is simple. For a non-standard repository see -[svn2git documentation](https://github.com/nirvdrum/svn2git). The following -command will checkout the repository and do the conversion in the current -working directory. Be sure to create a new directory for each repository before -running the `svn2git` command. The conversion process will take some time. - -```bash -svn2git https://svn.example.com/path/to/repo --authors /path/to/authors.txt -``` - -If your SVN repository requires a username and password add the -`--username <username>` and `--password <password` flags to the above command. -`svn2git` also supports excluding certain file paths, branches, tags, etc. See -[svn2git documentation](https://github.com/nirvdrum/svn2git) or run -`svn2git --help` for full documentation on all of the available options. - -Create a new GitLab project, where you will eventually push your converted code. -Copy the SSH or HTTP(S) repository URL from the project page. Add the GitLab -repository as a Git remote and push all the changes. This will push all commits, -branches and tags. - -```bash -git remote add origin git@gitlab.com:<group>/<project>.git -git push --all origin -git push --tags origin -``` - -## Contribute to this guide -We welcome all contributions that would expand this guide with instructions on -how to migrate from SVN and other version control systems. +This document was moved to a [new location](../../user/project/import/svn.md). diff --git a/doc/workflow/project_features.md b/doc/workflow/project_features.md index 3f5de2bd4b1..feb88712f5a 100644 --- a/doc/workflow/project_features.md +++ b/doc/workflow/project_features.md @@ -1,45 +1 @@ -# Project features - -When in a Project -> Settings, you will find Features on the bottom of the page that you can toggle. - -Below you will find a more elaborate explanation of each of these. - -## Issues - -Issues is a really powerful, but lightweight issue tracking system. - -You can make tickets, assign them to people, file them under milestones, order them with labels and have discussion in them. - -They integrate deeply into GitLab and are easily referenced from anywhere by using `#` and the issue number. - -## Merge Requests - -Using a merge request, you can review and discuss code before it is merged in the branch of your code. - -As with issues, it can be assigned; people, issues, etc. can be referenced; milestones attached. - -We see it as an integral part of working together on code and couldn't work without it. - -## Wiki - -This is a separate system for documentation, built right into GitLab. - -It is source controlled and is very convenient if you don't want to keep you documentation in your source code, but you do want to keep it in your GitLab project. - -[Read more about Wikis.](../user/project/wiki/index.md) - -## Snippets - -Snippets are little bits of code or text. - -This is a nice place to put code or text that is used semi-regularly within the project, but does not belong in source control. - -For example, a specific config file that is used by the team that is only valid for the people that work on the code. - -## Git LFS - ->**Note:** Project-specific LFS setting was added on 8.12 and is available only to admins. - -Git Large File Storage allows you to easily manage large binary files with Git. -With this setting admins can better control which projects are allowed to use -LFS. +This document was moved to [../user/project/index.md](../user/project/index.md) diff --git a/doc/workflow/share_projects_with_other_groups.md b/doc/workflow/share_projects_with_other_groups.md index 40d756bc199..2eb4d24958a 100644 --- a/doc/workflow/share_projects_with_other_groups.md +++ b/doc/workflow/share_projects_with_other_groups.md @@ -1,32 +1 @@ -# Share Projects with other Groups - -You can share projects with other groups. This makes it possible to add a group of users -to a project with a single action. - -## Groups as collections of users - -Groups are used primarily to [create collections of projects](../user/group/index.md), but you can also -take advantage of the fact that groups define collections of _users_, namely the group -members. - -## Sharing a project with a group of users - -The primary mechanism to give a group of users, say 'Engineering', access to a project, -say 'Project Acme', in GitLab is to make the 'Engineering' group the owner of 'Project -Acme'. But what if 'Project Acme' already belongs to another group, say 'Open Source'? -This is where the group sharing feature can be of use. - -To share 'Project Acme' with the 'Engineering' group, go to the project settings page for 'Project Acme' and use the left navigation menu to go to the 'Groups' section. - - - -Now you can add the 'Engineering' group with the maximum access level of your choice. -After sharing 'Project Acme' with 'Engineering', the project is listed on the group dashboard. - - - -## Maximum access level - - - -In the screenshot above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Master' or 'Owner') will only have 'Developer' access to 'Project Acme'. +This document was moved to [../user/project/members/share_project_with_groups.md](../user/project/members/share_project_with_groups.md) diff --git a/doc/workflow/share_with_group.md b/doc/workflow/share_with_group.md index 3b7690973cb..2eb4d24958a 100644 --- a/doc/workflow/share_with_group.md +++ b/doc/workflow/share_with_group.md @@ -1,13 +1 @@ -# Sharing a project with a group - -If you want to share a single project in a group with another group, -you can do so easily. By setting the permission you can quickly -give a select group of users access to a project in a restricted manner. - -In a project go to the project settings -> groups. - -Now you can select a group that you want to share this project with and with -which maximum access level. Users in that group are able to access this project -with their set group access level, up to the maximum level that you've set. - - +This document was moved to [../user/project/members/share_project_with_groups.md](../user/project/members/share_project_with_groups.md) |
