From fd88b0ca56b3a4230902f76a7b049228e53e6bb0 Mon Sep 17 00:00:00 2001 From: Tony Rom Date: Thu, 16 Nov 2017 21:10:15 +0300 Subject: Add `pipelines` endpoint to merge requests API --- doc/api/merge_requests.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) (limited to 'doc') diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 880b0ed2c65..a3261cf7ee4 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -431,6 +431,32 @@ Parameters: } ``` +## List MR pipelines + +Get a list of merge request pipelines. + +``` +GET /projects/:id/merge_requests/:merge_request_iid/pipelines +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `merge_request_iid` (required) - The internal ID of the merge request + +Example of response + +```json +[ + { + "id": 77, + "sha": "959e04d7c7a30600c894bd3c0cd0e1ce7f42c11d", + "ref": "master", + "status": "success" + } +] +``` + ## Create MR Creates a new merge request. -- cgit v1.2.1 From b570b34787b9a83e6af4e39a1e7445f86ff20911 Mon Sep 17 00:00:00 2001 From: Vincent Lae Date: Thu, 21 Dec 2017 13:07:54 +0000 Subject: fix example in ci ssh_keys documentation --- doc/ci/ssh_keys/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index df0e1521150..b8df0bfba20 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -181,7 +181,7 @@ before_script: ## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the ## following two lines. ## - - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts' + - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts - chmod 644 ~/.ssh/known_hosts ## -- cgit v1.2.1 From 3deb4e694d4dc68cf9d9548f9c101876fea4fdad Mon Sep 17 00:00:00 2001 From: Joshua Lambert Date: Thu, 21 Dec 2017 10:16:53 -0500 Subject: Add docs for AutoDevOps --- doc/topics/autodevops/index.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) (limited to 'doc') diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 0b48596006d..4056469e6c4 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -20,6 +20,7 @@ project in an easy and automatic way: 1. [Auto Test](#auto-test) 1. [Auto Code Quality](#auto-code-quality) 1. [Auto SAST (Static Application Security Testing)](#auto-sast) +1. [Auto Browser Performance Testing](#auto-browser-performance-testing) 1. [Auto Review Apps](#auto-review-apps) 1. [Auto Deploy](#auto-deploy) 1. [Auto Monitoring](#auto-monitoring) @@ -215,6 +216,20 @@ check out. Any security warnings are also [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html). +### Auto Browser Performance Testing + +> Introduced in [GitLab Enterprise Edition Premium][ee] 10.3. + +Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://hub.docker.com/r/sitespeedio/sitespeed.io/) to measure the performance of a web page. A JSON report is created and uploaded as an artifact, which includes the overall performance score for each page. By default, the root page of Review and Production environments will be tested. If you would like to add additional URL's to test, simply add the paths to a file named `.gitlab-urls.txt` in the root directory, one per line. For example: + +``` +/ +/features +/direction +``` + +In GitLab Enterprise Edition Premium, performance differences between the source and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). + ### Auto Review Apps NOTE: **Note:** -- cgit v1.2.1 From 5be9a521df57d8dba6c4520220fe5f8a6a001dfa Mon Sep 17 00:00:00 2001 From: Joshua Lambert Date: Thu, 21 Dec 2017 19:51:53 +0000 Subject: Set Auto Browser Performance Testing to 10.4 --- doc/topics/autodevops/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 4056469e6c4..7863252dc17 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -218,7 +218,7 @@ Any security warnings are also [shown in the merge request widget](https://docs. ### Auto Browser Performance Testing -> Introduced in [GitLab Enterprise Edition Premium][ee] 10.3. +> Introduced in [GitLab Enterprise Edition Premium][ee] 10.4. Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://hub.docker.com/r/sitespeedio/sitespeed.io/) to measure the performance of a web page. A JSON report is created and uploaded as an artifact, which includes the overall performance score for each page. By default, the root page of Review and Production environments will be tested. If you would like to add additional URL's to test, simply add the paths to a file named `.gitlab-urls.txt` in the root directory, one per line. For example: -- cgit v1.2.1 From e5b5d27deaf4855bd9cab67aed521c2dd0db00e4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Elan=20Ruusam=C3=A4e?= Date: Tue, 19 Dec 2017 21:20:12 +0000 Subject: plantuml.md: use html tags so code fence example would work --- doc/administration/integration/plantuml.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) (limited to 'doc') diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 93c3642a1f1..65f59b72690 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -58,30 +58,32 @@ our AsciiDoc snippets, wikis and repos using delimited blocks: - **Markdown** - ```plantuml - Bob -> Alice : hello - Alice -> Bob : Go Away - ``` +
+    ```plantuml
+    Bob -> Alice : hello
+    Alice -> Bob : Go Away
+    ```
+    
- **AsciiDoc** - ``` +
     [plantuml, format="png", id="myDiagram", width="200px"]
     --
     Bob->Alice : hello
     Alice -> Bob : Go Away
     --
-    ```
+    
- **reStructuredText** - ``` +
     .. plantuml::
        :caption: Caption with **bold** and *italic*
 
        Bob -> Alice: hello
        Alice -> Bob: Go Away
-    ```
+    
You can also use the `uml::` directive for compatibility with [sphinxcontrib-plantuml](https://pypi.python.org/pypi/sphinxcontrib-plantuml), but please note that we currently only support the `caption` option. -- cgit v1.2.1 From e5eb6710cb07ddc9dc7ac1731663349ee699d7db Mon Sep 17 00:00:00 2001 From: Drew Blessing Date: Wed, 27 Dec 2017 20:06:50 +0000 Subject: Add note about only 20 commits being included in push payload --- doc/user/project/integrations/webhooks.md | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'doc') diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index eafdd28071d..82175c70e49 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -54,6 +54,12 @@ Below are described the supported events. Triggered when you push to the repository except when pushing tags. +> **Note:** When more than 20 commits are pushed at once, the `commits` web hook + attribute will only contain the first 20 for performance reasons. Loading + detailed commit data is expensive. Note that despite only 20 commits being + present in the `commits` attribute, the `total_commits_count` attribute will + contain the actual total. + **Request header**: ``` -- cgit v1.2.1 From 3d650717b670e255c02c242e2e9de739054a689e Mon Sep 17 00:00:00 2001 From: Victor Wu Date: Thu, 28 Dec 2017 10:17:30 +0000 Subject: Table flip quick action docs --- doc/user/project/quick_actions.md | 2 ++ 1 file changed, 2 insertions(+) (limited to 'doc') diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index e81e935e37d..442fc978284 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -39,3 +39,5 @@ do. | `/board_move ~column` | Move issue to column on the board | | `/duplicate #issue` | Closes this issue and marks it as a duplicate of another issue | | `/move path/to/project` | Moves issue to another project | +| `/tableflip` | Append the comment with `(╯°□°)╯︵ ┻━┻` | +| `/shrug` | Append the comment with `¯\_(ツ)_/¯` | \ No newline at end of file -- cgit v1.2.1 From c44d72a8cef70ad26ff5e5cc149f6a00331f184e Mon Sep 17 00:00:00 2001 From: Mayra Cabrera Date: Thu, 28 Dec 2017 11:18:22 -0600 Subject: Updates documentation for prometheus and kubernetes integration --- doc/user/project/clusters/index.md | 1 + doc/user/project/integrations/kubernetes.md | 5 ++++- doc/user/project/integrations/project_services.md | 2 +- 3 files changed, 6 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index e2924c66e70..d5619c7b563 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -83,6 +83,7 @@ added directly to your configured cluster. Those applications are needed for | ----------- | :------------: | ----------- | | [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | | [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | +| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications | ## Enabling or disabling the Cluster integration diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index e9738b683f9..079a840a56a 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -1,7 +1,10 @@ --- -last_updated: 2017-09-25 +last_updated: 2017-12-28 --- +CAUTION: **Warning:** +Kubernetes service integration has been deprecated on GitLab 10.4. Fields on Kubernetes integration page are now uneditable, you can configure your clusters using the new [Clusters](../clusters/index.md) page. + # GitLab Kubernetes / OpenShift integration GitLab can be configured to interact with Kubernetes, or other systems using the diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index a0405161495..6bda07f103e 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -39,7 +39,7 @@ Click on the service links to see further configuration instructions and details | [Irker (IRC gateway)](irker.md) | Send IRC messages, on update, to a list of recipients through an Irker gateway | | [JIRA](jira.md) | JIRA issue tracker | | JetBrains TeamCity CI | A continuous integration and build server | -| [Kubernetes](kubernetes.md) | A containerized deployment service | +| [Kubernetes](kubernetes.md) _(Has been deprecated in GitLab 10.4)_ | A containerized deployment service | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | | [Microsoft teams](microsoft_teams.md) | Receive notifications for actions that happen on GitLab into a room on Microsoft Teams using Office 365 Connectors | -- cgit v1.2.1 From ea7f6fa80bb9699585a330f084331ea0f74500de Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Thu, 28 Dec 2017 16:46:10 -0600 Subject: Fix the header level for the Changelog Arguments documentation --- doc/development/changelog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/changelog.md b/doc/development/changelog.md index 48cffc0dd18..18f4177a5e5 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -127,7 +127,7 @@ type: If you're working on the GitLab EE repository, the entry will be added to `changelogs/unreleased-ee/` instead. -#### Arguments +### Arguments | Argument | Shorthand | Purpose | | ----------------- | --------- | ---------------------------------------------------------------------------------------------------------- | -- cgit v1.2.1 From c630bced675a30cff568ccc2c0eb71ef0e61651a Mon Sep 17 00:00:00 2001 From: Edouard Date: Fri, 29 Dec 2017 11:09:49 +0000 Subject: Documenting that resolved JIRA tickets are not automatically transitioned --- doc/user/project/integrations/jira.md | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'doc') diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 7dc234a9759..411547e9800 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -173,6 +173,7 @@ where `PROJECT-1` is the issue ID of the JIRA project. - Only commits and merges into the project's default branch (usually **master**) will close an issue in Jira. You can change your projects default branch under [project settings](img/jira_project_settings.png). +- The JIRA issue will not be transitionned if it has a resolution. ### JIRA issue closing example @@ -222,6 +223,10 @@ JIRA issue references and update comments will not work if the GitLab issue trac Make sure the `Transition ID` you set within the JIRA settings matches the one your project needs to close a ticket. +Make sure that the JIRA issue is not already marked as resolved, in other word that +the JIRA issue resolution field is not set. (It should not be striked through in +JIRA lists). + [services-templates]: services_templates.md [jira-repo-old-docs]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-13-stable/doc/project_services/jira.md [jira]: https://www.atlassian.com/software/jira -- cgit v1.2.1 From 8a75e49eec130942f5b0f9ee1f43158ef623a7e5 Mon Sep 17 00:00:00 2001 From: Edouard Date: Fri, 29 Dec 2017 12:58:03 +0000 Subject: Typos correction on the advise of @smcgivern --- doc/user/project/integrations/jira.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 411547e9800..f77569e4886 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -173,7 +173,7 @@ where `PROJECT-1` is the issue ID of the JIRA project. - Only commits and merges into the project's default branch (usually **master**) will close an issue in Jira. You can change your projects default branch under [project settings](img/jira_project_settings.png). -- The JIRA issue will not be transitionned if it has a resolution. +- The JIRA issue will not be transitioned if it has a resolution. ### JIRA issue closing example @@ -223,9 +223,9 @@ JIRA issue references and update comments will not work if the GitLab issue trac Make sure the `Transition ID` you set within the JIRA settings matches the one your project needs to close a ticket. -Make sure that the JIRA issue is not already marked as resolved, in other word that -the JIRA issue resolution field is not set. (It should not be striked through in -JIRA lists). +Make sure that the JIRA issue is not already marked as resolved, in other words that +the JIRA issue resolution field is not set. (It should not be struck through in +JIRA lists.) [services-templates]: services_templates.md [jira-repo-old-docs]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-13-stable/doc/project_services/jira.md -- cgit v1.2.1 From 1f50eb4f57ae14336a677f53d9a2c9501033f966 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Wed, 20 Dec 2017 14:05:02 +0100 Subject: Add docs about end-to-end testing / GitLab QA tests --- doc/development/testing_guide/end_to_end_tests.md | 42 +++++++++++++++++++++++ doc/development/testing_guide/index.md | 8 +++++ 2 files changed, 50 insertions(+) create mode 100644 doc/development/testing_guide/end_to_end_tests.md (limited to 'doc') diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md new file mode 100644 index 00000000000..87a21b9f7fc --- /dev/null +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -0,0 +1,42 @@ +# End-to-End Testing + +## What is End-to-End testing? + +End-to-End testing is a strategy used to check whether your application works +as expected across entire software stack and architecture, including +integration of all microservices and components that are supposed to work +together. + +## How do we test GitLab? + +We use [Omnibus GitLab][omnibus-gitlab] to build GitLab packages and then we +test these packages using [GitLab QA][gitlab-qa] project, which is entirely +black-box, click-driven testing framework. + +### Testing nightly builds + +We run scheduled pipeline each night to test nightly builds created by Omnibus. +You can find these nightly pipelines at [GitLab QA pipelines page][gitlab-qa-pipelines]. + +### Testing code in merge requests + +It is also possible to trigger packages build and [GitLab QA pipeline][gitlab-qa-pipelines] +using a manual action that should be present in the merge request widget on +your merge request. Look for `package-qa` manual action. + +Below you can read more about how to use it and how does it work. + +## How does it work? + +We are using _multi-project pipelines_ to run end-to-end tests. + +## How do I test my code? + +## How do I contribute? + +## Where can I ask for help? + + +[omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab +[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa +[gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 65386f231a0..4ca192aee7e 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -65,6 +65,13 @@ Everything you should know about how to test Rake tasks. --- +## [End-to-end tests](end_to_end_tests.md) + +Everything you should know about how to run end-to-end tests, also known as +[GitLab QA][gitlab-qa] tests. + +--- + ## Spinach (feature) tests GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426) @@ -89,3 +96,4 @@ test should be re-implemented using RSpec instead. [Capybara]: https://github.com/teamcapybara/capybara [Karma]: http://karma-runner.github.io/ [Jasmine]: https://jasmine.github.io/ +[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa -- cgit v1.2.1 From 629d966c4807174ddb91e753b9aa15ec2697cd08 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Tue, 2 Jan 2018 14:22:48 +0100 Subject: Extend documentation on end-to-end integration tests --- doc/development/testing_guide/end_to_end_tests.md | 47 +++++++++++++++++++---- doc/development/testing_guide/index.md | 4 +- 2 files changed, 42 insertions(+), 9 deletions(-) (limited to 'doc') diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md index 87a21b9f7fc..6be0decd75d 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -20,23 +20,56 @@ You can find these nightly pipelines at [GitLab QA pipelines page][gitlab-qa-pip ### Testing code in merge requests -It is also possible to trigger packages build and [GitLab QA pipeline][gitlab-qa-pipelines] -using a manual action that should be present in the merge request widget on -your merge request. Look for `package-qa` manual action. +It is also possible to trigger build of GitLab packages and then pass these +package to GitLab QA to run tests in a [pipeline][gitlab-qa-pipelines]. + +Developers can trigger a `package-qa` manual action, that should be present in +the merge request widget in your merge request. + +It is possible to trigger Gitlab QA pipeline from merge requests in GitLab CE +and GitLab EE, but QA triggering manual action is also available in the Omnibus +GitLab project as well. Below you can read more about how to use it and how does it work. -## How does it work? +#### How does it work? + +Currently, we are _multi-project pipeline_-like approach to run QA pipelines. + +1. Developer triggers manual action in the CE or EE merge request, that starts +a chain of pipelines. +1. Triggering this action enqueues a new CI job that is going to be picked by a +runner. +1. The script, that is being executed, triggers a pipeline in GitLab Omnibus +projects, and waits for the resulting status. We call it _status attribution_. +1. GitLab packages are being built in the pipeline started in Omnibus. Packages +are going to be sent to Container Registry. +1. When packages are ready, and available in the registry, a final step in the +pipeline that is now running in Omnibus triggers a new pipeline in the GitLab +QA project. It also waits for the resulting status. +1. GitLab QA pulls images from the registry, spins-up containers and runs tests +against test environment that has been just orchestrated. +1. The result of GitLab QA pipeline is being propagated upstream, through +Omnibus, to CE / EE merge request. -We are using _multi-project pipelines_ to run end-to-end tests. +#### How do I write tests? -## How do I test my code? +In order to write new tests, you first need to learn more about GitLab QA +architecture. There is some documentation about it in GitLab QA project +[here][gitlab-qa-architecture]. -## How do I contribute? +Once you decided we to put test environment orchestration scenarios and +instance specs, take a looks at [relevant documentation][instance-qa-readme] ## Where can I ask for help? +You can ask question in `#qa` channel on Slack (GitLab internal) or you can +find an issue you would like to work on in [the issue tracker][gitlab-qa-issues] +and start a new discussion there. [omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab [gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa [gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines +[instance-qa-readme]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/README.md +[gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md +[gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 4ca192aee7e..74d09eb91ff 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -67,8 +67,8 @@ Everything you should know about how to test Rake tasks. ## [End-to-end tests](end_to_end_tests.md) -Everything you should know about how to run end-to-end tests, also known as -[GitLab QA][gitlab-qa] tests. +Everything you should know about how to run end-to-end tests using +[GitLab QA][gitlab-qa] testing framework. --- -- cgit v1.2.1 From a265ac25cbf86ffcaffac2e097611625c05976bf Mon Sep 17 00:00:00 2001 From: Lin Jen-Shin Date: Fri, 29 Dec 2017 14:29:53 +0800 Subject: Add documents for GitLab utilities --- doc/development/README.md | 1 + doc/development/utilities.md | 92 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+) create mode 100644 doc/development/utilities.md (limited to 'doc') diff --git a/doc/development/README.md b/doc/development/README.md index b624aa37c70..12cca9f84b7 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -27,6 +27,7 @@ comments: false ## Backend guides +- [GitLab utilities](utilities.md) - [API styleguide](api_styleguide.md) Use this styleguide if you are contributing to the API. - [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers diff --git a/doc/development/utilities.md b/doc/development/utilities.md new file mode 100644 index 00000000000..951c3ef85ce --- /dev/null +++ b/doc/development/utilities.md @@ -0,0 +1,92 @@ +# GitLab utilities + +We developed a number of utilities to ease development. + +## [`MergeHash`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/utils/merge_hash.rb) + +* Deep merges an array of hashes: + + ``` ruby + Gitlab::Utils::MergeHash.merge( + [{ hello: ["world"] }, + { hello: "Everyone" }, + { hello: { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } }, + "Goodbye", "Hallo"] + ) + ``` + + Gives: + + ``` ruby + [ + { + hello: + [ + "world", + "Everyone", + { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } + ] + }, + "Goodbye" + ] + ``` + +* Extracts all keys and values from a hash into an array: + + ``` ruby + Gitlab::Utils::MergeHash.crush( + { hello: "world", this: { crushes: ["an entire", "hash"] } } + ) + ``` + + Gives: + + ``` ruby + [:hello, "world", :this, :crushes, "an entire", "hash"] + ``` + +## [`StrongMemoize`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/utils/strong_memoize.rb) + +* Memoize the value even if it is `nil` or `false`. + + We often do `@value ||= compute`, however this doesn't work well if + `compute` might eventually give `nil` and we don't want to compute again. + Instead we could use `defined?` to check if the value is set or not. + However it's tedious to write such pattern, and `StrongMemoize` would + help us use such pattern. + + Instead of writing patterns like this: + + ``` ruby + class Find + def result + return @result if defined?(@result) + + @result = search + end + end + ``` + + We could write it like: + + ``` ruby + class Find + include Gitlab::Utils::StrongMemoize + + def result + strong_memoize(:result) do + search + end + end + end + ``` + +* Clear memoization + + ``` ruby + class Find + include Gitlab::Utils::StrongMemoize + end + + Find.new.clear_memoization(:result) + ``` -- cgit v1.2.1 From 51bb5abe269c8639b4f9ebbb6cb5e917d2710210 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Tue, 2 Jan 2018 14:30:29 +0100 Subject: Link to end to end test guideline from test pyramid --- doc/development/testing_guide/testing_levels.md | 3 +++ 1 file changed, 3 insertions(+) (limited to 'doc') diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 1cbd4350284..dd0e9a77164 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -121,6 +121,9 @@ running feature tests (i.e. using Capybara) against it. The actual test scenarios and steps are [part of GitLab Rails] so that they're always in-sync with the codebase. +Read a separate document about [end-to-end tests](../end_to_end_tests.md) to +learn more. + [multiple pieces]: ../architecture.md#components [GitLab Shell]: https://gitlab.com/gitlab-org/gitlab-shell [GitLab Workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse -- cgit v1.2.1 From 1aa25a3fe3b7167c2d9c73930cf9a05794be0dd2 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Tue, 2 Jan 2018 14:37:01 +0100 Subject: Copy-edit end-to-end testing guidelines --- doc/development/testing_guide/end_to_end_tests.md | 31 +++++++++++++---------- 1 file changed, 18 insertions(+), 13 deletions(-) (limited to 'doc') diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md index 6be0decd75d..561547e1581 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -36,21 +36,24 @@ Below you can read more about how to use it and how does it work. Currently, we are _multi-project pipeline_-like approach to run QA pipelines. -1. Developer triggers manual action in the CE or EE merge request, that starts -a chain of pipelines. -1. Triggering this action enqueues a new CI job that is going to be picked by a -runner. +1. Developer triggers a manual action, that can be found in CE and EE merge +requests, what starts a chain of pipelines. + 1. The script, that is being executed, triggers a pipeline in GitLab Omnibus -projects, and waits for the resulting status. We call it _status attribution_. -1. GitLab packages are being built in the pipeline started in Omnibus. Packages -are going to be sent to Container Registry. +projects, and waits for the resulting status. We call this a _status attribution_. + +1. GitLab packages are being built in Omnibus pipeline. Packages are going to be +pushed to Container Registry. + 1. When packages are ready, and available in the registry, a final step in the -pipeline that is now running in Omnibus triggers a new pipeline in the GitLab -QA project. It also waits for the resulting status. +pipeline, that is now running in Omnibus, triggers a new pipeline in the GitLab +QA project. It also waits for a resulting status. + 1. GitLab QA pulls images from the registry, spins-up containers and runs tests -against test environment that has been just orchestrated. +against a test environment that has been just orchestrated by `gitlab-qa` tool. + 1. The result of GitLab QA pipeline is being propagated upstream, through -Omnibus, to CE / EE merge request. +Omnibus, back to CE / EE merge request. #### How do I write tests? @@ -58,8 +61,9 @@ In order to write new tests, you first need to learn more about GitLab QA architecture. There is some documentation about it in GitLab QA project [here][gitlab-qa-architecture]. -Once you decided we to put test environment orchestration scenarios and +Once you decided were to put test environment orchestration scenarios and instance specs, take a looks at [relevant documentation][instance-qa-readme] +and examples in [the `qa/` directory][instance-qa-examples]. ## Where can I ask for help? @@ -70,6 +74,7 @@ and start a new discussion there. [omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab [gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa [gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines -[instance-qa-readme]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/README.md [gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md [gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues +[instance-qa-readme]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/README.md +[instance-qa-examples]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa -- cgit v1.2.1 From 124ffb2134be85575ddc75bfa34903fb738c6930 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Tue, 2 Jan 2018 14:42:28 +0100 Subject: Fix link to end-to-end testing docs from test pyramid --- doc/development/testing_guide/testing_levels.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index dd0e9a77164..4adf0dc7c7a 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -121,7 +121,7 @@ running feature tests (i.e. using Capybara) against it. The actual test scenarios and steps are [part of GitLab Rails] so that they're always in-sync with the codebase. -Read a separate document about [end-to-end tests](../end_to_end_tests.md) to +Read a separate document about [end-to-end tests](end_to_end_tests.md) to learn more. [multiple pieces]: ../architecture.md#components -- cgit v1.2.1 From 3d5e4e34ca30f083e0f6ea0bbfe1a28c2f8f4b63 Mon Sep 17 00:00:00 2001 From: Onuwa Nnachi Isaac Date: Tue, 2 Jan 2018 19:06:11 +0100 Subject: Fix typo in writing_documentation.md --- doc/development/writing_documentation.md | 4 ++-- doc/doc | 1 + 2 files changed, 3 insertions(+), 2 deletions(-) create mode 120000 doc/doc (limited to 'doc') diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 43a79ffcaa5..133ac0234cf 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -15,7 +15,7 @@ request introducing these changes must be accompanied by the documentation (either updating existing ones or creating new ones). This is also valid when changes are introduced to the UI. -The one resposible for writing the first piece of documentation is the developer who +The one responsible for writing the first piece of documentation is the developer who wrote the code. It's the job of the Product Manager to ensure all features are shipped with its docs, whether is a small or big change. At the pace GitLab evolves, this is the only way to keep the docs up-to-date. If you have any questions about it, @@ -31,7 +31,7 @@ Every major feature (regardless if present in GitLab Community or Enterprise edi should present, at the beginning of the document, two main sections: **overview** and **use cases**. Every GitLab EE-only feature should also contain these sections. -**Overview**: at the name suggests, the goal here is to provide an overview of the feature. +**Overview**: as the name suggests, the goal here is to provide an overview of the feature. Describe what is it, what it does, why it is important/cool/nice-to-have, what problem it solves, and what you can do with this feature that you couldn't do before. diff --git a/doc/doc b/doc/doc new file mode 120000 index 00000000000..e78a24e7fe6 --- /dev/null +++ b/doc/doc @@ -0,0 +1 @@ +/home/isaac/apps/opensource/GitLab/gitlab-ce/doc \ No newline at end of file -- cgit v1.2.1 From 2a2376096d09a3e61974bf7ed1bb22e379d85f3c Mon Sep 17 00:00:00 2001 From: Joshua Lambert Date: Wed, 3 Jan 2018 01:14:43 -0500 Subject: Align chart docs with readme --- doc/install/kubernetes/index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index 0932e1eee3a..cd889e74487 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -9,11 +9,11 @@ should be deployed, upgraded, and configured. ## Chart Overview -* **[GitLab-Omnibus](gitlab_omnibus.md)**: The best way to run GitLab on Kubernetes today, suited for small to medium deployments. The chart is in beta and will be deprecated by the [cloud native GitLab chart](#cloud-native-gitlab-chart). +* **[GitLab-Omnibus](gitlab_omnibus.md)**: The best way to run GitLab on Kubernetes today, suited for small deployments. The chart is in beta and will be deprecated by the [cloud native GitLab chart](#cloud-native-gitlab-chart). * **[Cloud Native GitLab Chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md)**: The next generation GitLab chart, currently in development. Will support large deployments with horizontal scaling of individual GitLab components. * Other Charts * [GitLab Runner Chart](gitlab_runner_chart.md): For deploying just the GitLab Runner. - * [Advanced GitLab Installation](gitlab_chart.md): Deprecated, being replaced by the [cloud native GitLab chart](#cloud-native-gitlab-chart). Provides additional deployment options, but provides less functionality out-of-the-box. + * [Advanced GitLab Installation](gitlab_chart.md): Deprecated, being replaced by the [cloud native GitLab chart](#cloud-native-gitlab-chart). Provides additional deployment options, but provides less functionality out-of-the-box. * [Community Contributed Charts](#community-contributed-charts): Community contributed charts, deprecated by the official GitLab chart. ## GitLab-Omnibus Chart (Recommended) -- cgit v1.2.1 From 78d22fb20db14c90861318b9f316466fbf002114 Mon Sep 17 00:00:00 2001 From: Yorick Peterse Date: Thu, 21 Dec 2017 16:44:07 +0100 Subject: Use a background migration for issues.closed_at In a previous attempt (rolled back in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/16021) we tried to migrate `issues.closed_at` from timestamp to timestamptz using a regular migration. This has a bad impact on GitLab.com and as such was rolled back. This commit re-implements the original migrations using generic background migrations, allowing us to still migrate the data in a single release but without a negative impact on availability. To ensure the database schema is up to date the background migrations are performed inline in development and test environments. We also make sure to not migrate that that doesn't need migrating in the first place or has already been migrated. --- doc/development/what_requires_downtime.md | 57 +++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) (limited to 'doc') diff --git a/doc/development/what_requires_downtime.md b/doc/development/what_requires_downtime.md index 05e0a64af18..9d0c62ecc35 100644 --- a/doc/development/what_requires_downtime.md +++ b/doc/development/what_requires_downtime.md @@ -195,6 +195,63 @@ end And that's it, we're done! +## Changing Column Types For Large Tables + +While `change_column_type_concurrently` can be used for changing the type of a +column without downtime it doesn't work very well for large tables. Because all +of the work happens in sequence the migration can take a very long time to +complete, preventing a deployment from proceeding. +`change_column_type_concurrently` can also produce a lot of pressure on the +database due to it rapidly updating many rows in sequence. + +To reduce database pressure you should instead use +`change_column_type_using_background_migration` when migrating a column in a +large table (e.g. `issues`). This method works similar to +`change_column_type_concurrently` but uses background migration to spread the +work / load over a longer time period, without slowing down deployments. + +Usage of this method is fairly simple: + +```ruby +class ExampleMigration < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + disable_ddl_transaction! + + class Issue < ActiveRecord::Base + self.table_name = 'issues' + + include EachBatch + + def self.to_migrate + where('closed_at IS NOT NULL') + end + end + + def up + change_column_type_using_background_migration( + Issue.to_migrate, + :closed_at, + :datetime_with_timezone + ) + end + + def down + change_column_type_using_background_migration( + Issue.to_migrate, + :closed_at, + :datetime + ) + end +end +``` + +This would change the type of `issues.closed_at` to `timestamp with time zone`. + +Keep in mind that the relation passed to +`change_column_type_using_background_migration` _must_ include `EachBatch`, +otherwise it will raise a `TypeError`. + ## Adding Indexes Adding indexes is an expensive process that blocks INSERT and UPDATE queries for -- cgit v1.2.1 From 5e0143a84bca7fd8b2dccd175e0f50c87dea4b98 Mon Sep 17 00:00:00 2001 From: Alessio Caiazza Date: Sat, 27 May 2017 15:23:27 +0200 Subject: Add online attribute to runner api entity --- doc/api/runners.md | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) (limited to 'doc') diff --git a/doc/api/runners.md b/doc/api/runners.md index 015b09a745e..50981ed96bc 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -30,14 +30,16 @@ Example response: "description": "test-1-20150125", "id": 6, "is_shared": false, - "name": null + "name": null, + "online": true }, { "active": true, "description": "test-2-20150125", "id": 8, "is_shared": false, - "name": null + "name": null, + "online": false } ] ``` @@ -69,28 +71,32 @@ Example response: "description": "shared-runner-1", "id": 1, "is_shared": true, - "name": null + "name": null, + "online": true }, { "active": true, "description": "shared-runner-2", "id": 3, "is_shared": true, - "name": null + "name": null, + "online": false }, { "active": true, "description": "test-1-20150125", "id": 6, "is_shared": false, - "name": null + "name": null, + "online": true }, { "active": true, "description": "test-2-20150125", "id": 8, "is_shared": false, - "name": null + "name": null, + "online": false } ] ``` @@ -122,6 +128,7 @@ Example response: "is_shared": false, "contacted_at": "2016-01-25T16:39:48.066Z", "name": null, + "online": true, "platform": null, "projects": [ { @@ -176,6 +183,7 @@ Example response: "is_shared": false, "contacted_at": "2016-01-25T16:39:48.066Z", "name": null, + "online": true, "platform": null, "projects": [ { @@ -327,14 +335,16 @@ Example response: "description": "test-2-20150125", "id": 8, "is_shared": false, - "name": null + "name": null, + "online": false }, { "active": true, "description": "development_runner", "id": 5, "is_shared": true, - "name": null + "name": null, + "online": true } ] ``` @@ -364,7 +374,8 @@ Example response: "description": "test-2016-02-01", "id": 9, "is_shared": false, - "name": null + "name": null, + "online": true } ``` -- cgit v1.2.1 From 78cdac8401375cc85be54ae68e5d94d02a90233c Mon Sep 17 00:00:00 2001 From: Luc Didry Date: Wed, 3 Jan 2018 17:32:34 +0100 Subject: Expose project_id on /api/v4/pages/domains --- doc/api/pages_domains.md | 1 + 1 file changed, 1 insertion(+) (limited to 'doc') diff --git a/doc/api/pages_domains.md b/doc/api/pages_domains.md index 50685f335f7..20275b902c6 100644 --- a/doc/api/pages_domains.md +++ b/doc/api/pages_domains.md @@ -21,6 +21,7 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/a { "domain": "ssl.domain.example", "url": "https://ssl.domain.example", + "project_id": 1337, "certificate": { "expired": false, "expiration": "2020-04-12T14:32:00.000Z" -- cgit v1.2.1 From b299198e1eb5e1f26d5267f4a64944e600086d6b Mon Sep 17 00:00:00 2001 From: Filipa Lacerda Date: Wed, 3 Jan 2018 23:14:55 +0000 Subject: Adds `eslint-plugin-vue`, fixes linter errors and adds docs --- doc/development/fe_guide/style_guide_js.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 1cd66f27492..3c5d69e1f71 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -101,16 +101,16 @@ followed by any global declarations, then a blank newline prior to any imports o ``` Import statements are following usual naming guidelines, for example object literals use camel case: - + ```javascript // some_object file export default { key: 'value', }; - + // bad import ObjectLiteral from 'some_object'; - + // good import objectLiteral from 'some_object'; ``` @@ -255,6 +255,10 @@ A forEach will cause side effects, it will be mutating the array being iterated. ### Vue.js +#### `eslint-vue-plugin` +We default to [eslint-vue-plugin][eslint-plugin-vue], with the `plugin:vue/recommended`. +Please check this [rules][eslint-plugin-vue-rules] for more documentation. + #### Basic Rules 1. The service has it's own file 1. The store has it's own file @@ -513,8 +517,8 @@ On those a default key should not be provided. 1. `props` 1. `mixins` 1. `directives` - 1. `data` 1. `components` + 1. `data` 1. `computedProps` 1. `methods` 1. `beforeCreate` @@ -582,3 +586,5 @@ The goal of this accord is to make sure we are all on the same page. [eslintrc]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.eslintrc [eslint-this]: http://eslint.org/docs/rules/class-methods-use-this [eslint-new]: http://eslint.org/docs/rules/no-new +[eslint-plugin-vue]: https://github.com/vuejs/eslint-plugin-vue +[eslint-plugin-vue-rules]: https://github.com/vuejs/eslint-plugin-vue#bulb-rules -- cgit v1.2.1 From 127cedd0203a7ad16808391b6b768a52e8cb89f8 Mon Sep 17 00:00:00 2001 From: Mayra Cabrera Date: Wed, 3 Jan 2018 17:59:23 -0600 Subject: Update Kubernetes service documentation States this service was deprecated on 10.3 and his behavior depends on the kubernetes service status. Also includes a link for the release post for more information. --- doc/api/services.md | 3 +++ doc/user/project/integrations/kubernetes.md | 2 +- doc/user/project/integrations/project_services.md | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/api/services.md b/doc/api/services.md index 7e2afc71f9a..2928ab6cc75 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -562,6 +562,9 @@ DELETE /projects/:id/services/jira Kubernetes / Openshift integration +CAUTION: **Warning:** +Kubernetes service integration has been deprecated in GitLab 10.3. API service endpoints will continue to work as long as the Kubernetes service is active, however if the service is inactive API endpoints will automatically return a `400 Bad Request`. Read [GitLab 10.3 release post](https://about.gitlab.com/2017/12/22/gitlab-10-3-released/#kubernetes-integration-service) for more information. + ### Create/Edit Kubernetes service Set Kubernetes service for a project. diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index 079a840a56a..710cf78e84f 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -3,7 +3,7 @@ last_updated: 2017-12-28 --- CAUTION: **Warning:** -Kubernetes service integration has been deprecated on GitLab 10.4. Fields on Kubernetes integration page are now uneditable, you can configure your clusters using the new [Clusters](../clusters/index.md) page. +Kubernetes service integration has been deprecated in GitLab 10.3. If the service is active the cluster information still be editable, however we advised to disable and reconfigure the clusters using the new [Clusters](../clusters/index.md) page. If the service is inactive the fields will be uneditable. Read [GitLab 10.3 release post](https://about.gitlab.com/2017/12/22/gitlab-10-3-released/#kubernetes-integration-service) for more information. # GitLab Kubernetes / OpenShift integration diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index 6bda07f103e..9496d6f2ce0 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -39,7 +39,7 @@ Click on the service links to see further configuration instructions and details | [Irker (IRC gateway)](irker.md) | Send IRC messages, on update, to a list of recipients through an Irker gateway | | [JIRA](jira.md) | JIRA issue tracker | | JetBrains TeamCity CI | A continuous integration and build server | -| [Kubernetes](kubernetes.md) _(Has been deprecated in GitLab 10.4)_ | A containerized deployment service | +| [Kubernetes](kubernetes.md) _(Has been deprecated in GitLab 10.3)_ | A containerized deployment service | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | | [Microsoft teams](microsoft_teams.md) | Receive notifications for actions that happen on GitLab into a room on Microsoft Teams using Office 365 Connectors | -- cgit v1.2.1 From de36a8e27961d4c2af43d0ac2d700a391c245353 Mon Sep 17 00:00:00 2001 From: James Lopez Date: Thu, 4 Jan 2018 11:02:43 +0100 Subject: refactor spec, add docs --- doc/administration/raketasks/check.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index c8b5434c068..7dabc014bad 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -34,13 +34,13 @@ This task loops through all repositories on the GitLab server and runs the **Omnibus Installation** ``` -sudo gitlab-rake gitlab:repo:check +sudo gitlab-rake gitlab:git:fsck ``` **Source Installation** ```bash -sudo -u git -H bundle exec rake gitlab:repo:check RAILS_ENV=production +sudo -u git -H bundle exec rake gitlab:git:fsck RAILS_ENV=production ``` ### Check repositories for a specific user -- cgit v1.2.1 From e8acb3f11755811fca28d38bb0cbba44add7b0af Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Thu, 4 Jan 2018 12:09:14 +0100 Subject: Copy-edit end-to-end testing guide documentation --- doc/development/testing_guide/end_to_end_tests.md | 34 ++++++++++++----------- 1 file changed, 18 insertions(+), 16 deletions(-) (limited to 'doc') diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md index 561547e1581..30efe3e3b76 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -23,24 +23,25 @@ You can find these nightly pipelines at [GitLab QA pipelines page][gitlab-qa-pip It is also possible to trigger build of GitLab packages and then pass these package to GitLab QA to run tests in a [pipeline][gitlab-qa-pipelines]. -Developers can trigger a `package-qa` manual action, that should be present in -the merge request widget in your merge request. +Developers can trigger the `package-qa` manual action, that should be present in +the merge request widget. -It is possible to trigger Gitlab QA pipeline from merge requests in GitLab CE -and GitLab EE, but QA triggering manual action is also available in the Omnibus -GitLab project as well. +It is also possible to trigger Gitlab QA pipeline from merge requests in +Omnibus GitLab project. You can find a manual action that is similar to +`package-qa`, mentioned above, in your Omnibus-related merge requests as well. Below you can read more about how to use it and how does it work. #### How does it work? -Currently, we are _multi-project pipeline_-like approach to run QA pipelines. +Currently, we are using _multi-project pipeline_-like approach to run QA +pipelines. 1. Developer triggers a manual action, that can be found in CE and EE merge -requests, what starts a chain of pipelines. +requests. This starts a chain of pipelines in multiple projects. -1. The script, that is being executed, triggers a pipeline in GitLab Omnibus -projects, and waits for the resulting status. We call this a _status attribution_. +1. The script being executed triggers a pipeline in GitLab Omnibus and waits +for the resulting status. We call this a _status attribution_. 1. GitLab packages are being built in Omnibus pipeline. Packages are going to be pushed to Container Registry. @@ -50,24 +51,25 @@ pipeline, that is now running in Omnibus, triggers a new pipeline in the GitLab QA project. It also waits for a resulting status. 1. GitLab QA pulls images from the registry, spins-up containers and runs tests -against a test environment that has been just orchestrated by `gitlab-qa` tool. +against a test environment that has been just orchestrated by the `gitlab-qa` +tool. -1. The result of GitLab QA pipeline is being propagated upstream, through +1. The result of the GitLab QA pipeline is being propagated upstream, through Omnibus, back to CE / EE merge request. #### How do I write tests? In order to write new tests, you first need to learn more about GitLab QA -architecture. There is some documentation about it in GitLab QA project -[here][gitlab-qa-architecture]. +architecture. See the [documentation about it][gitlab-qa-architecture] in +GitLab QA project. -Once you decided were to put test environment orchestration scenarios and -instance specs, take a looks at [relevant documentation][instance-qa-readme] +Once you decided where to put test environment orchestration scenarios and +instance specs, take a look at the [relevant documentation][instance-qa-readme] and examples in [the `qa/` directory][instance-qa-examples]. ## Where can I ask for help? -You can ask question in `#qa` channel on Slack (GitLab internal) or you can +You can ask question in the `#qa` channel on Slack (GitLab internal) or you can find an issue you would like to work on in [the issue tracker][gitlab-qa-issues] and start a new discussion there. -- cgit v1.2.1 From d6bd608b354928196142f5bec65f29b80960cd30 Mon Sep 17 00:00:00 2001 From: Mark Fletcher Date: Thu, 4 Jan 2018 12:50:34 +0000 Subject: Specifies the accepted refs for downloading an archive via the API --- doc/api/repositories.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/api/repositories.md b/doc/api/repositories.md index 03b32577872..5fb25e40ed7 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -113,7 +113,7 @@ GET /projects/:id/repository/archive Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user -- `sha` (optional) - The commit SHA to download defaults to the tip of the default branch +- `sha` (optional) - The commit SHA to download. A tag, branch reference or sha can be used. This defaults to the tip of the default branch if not specified ## Compare branches, tags or commits -- cgit v1.2.1 From 0a02b0d8eada41b72d19bee3f963d7cee5b19c57 Mon Sep 17 00:00:00 2001 From: Gauthier Wallet Date: Thu, 4 Jan 2018 16:26:14 +0000 Subject: Update settings.md --- doc/api/settings.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/api/settings.md b/doc/api/settings.md index 0e4758cda2d..0b5b1f0c134 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -69,7 +69,7 @@ PUT /application/settings | `after_sign_up_text` | string | no | Text shown to the user after signing up | | `akismet_api_key` | string | no | API key for akismet spam protection | | `akismet_enabled` | boolean | no | Enable or disable akismet spam protection | -| `circuitbreaker_access_retries | integer | no | The number of attempts GitLab will make to access a storage. | +| `circuitbreaker_access_retries` | integer | no | The number of attempts GitLab will make to access a storage. | | `circuitbreaker_check_interval` | integer | no | Number of seconds in between storage checks. | | `circuitbreaker_failure_count_threshold` | integer | no | The number of failures of after which GitLab will completely prevent access to the storage. | | `circuitbreaker_failure_reset_time` | integer | no | Time in seconds GitLab will keep storage failure information. When no failures occur during this time, the failure information is reset. | -- cgit v1.2.1 From 5e148d4e931792733400f59864e1aa886ef55953 Mon Sep 17 00:00:00 2001 From: Felipe Artur Date: Wed, 6 Dec 2017 17:07:47 -0200 Subject: EE-BACKPORT group boards --- doc/api/boards.md | 200 +++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 183 insertions(+), 17 deletions(-) (limited to 'doc') diff --git a/doc/api/boards.md b/doc/api/boards.md index 69c47abc806..a5f455e1c43 100644 --- a/doc/api/boards.md +++ b/doc/api/boards.md @@ -15,10 +15,10 @@ GET /projects/:id/boards | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | ```bash -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/:id/boards +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards ``` Example response: @@ -27,6 +27,19 @@ Example response: [ { "id" : 1, + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site" + }, + "milestone": { + "id": 12 + "title": "10.0" + }, "lists" : [ { "id" : 1, @@ -60,6 +73,159 @@ Example response: ] ``` +## Single board + +Get a single board. + +``` +GET /projects/:id/boards/:board_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1 +``` + +Example response: + +```json + { + "id": 1, + "name:": "project issue board", + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site" + }, + "milestone": { + "id": 12 + "title": "10.0" + }, + "lists" : [ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } + ] + } +``` + +## Create a board (EES-Only) + +Creates a board. + +``` +POST /projects/:id/boards +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `name` | string | yes | The name of the new board | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards?name=newboard +``` + +Example response: + +```json + { + "id": 1, + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site" + }, + "name": "newboard", + "milestone": { + "id": 12 + "title": "10.0" + }, + "lists" : [ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } + ] + } +``` + +## Delete a board (EES-Only) + +Deletes a board. + +``` +DELETE /projects/:id/boards/:board_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1 +``` + ## List board lists Get a list of the board's lists. @@ -71,8 +237,8 @@ GET /projects/:id/boards/:board_id/lists | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists @@ -122,9 +288,9 @@ GET /projects/:id/boards/:board_id/lists/:list_id | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | -| `list_id`| integer | yes | The ID of a board's list | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id`| integer | yes | The ID of a board's list | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists/1 @@ -154,9 +320,9 @@ POST /projects/:id/boards/:board_id/lists | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | -| `label_id` | integer | yes | The ID of a label | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `label_id` | integer | yes | The ID of a label | ```bash curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists?label_id=5 @@ -186,10 +352,10 @@ PUT /projects/:id/boards/:board_id/lists/:list_id | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | -| `list_id` | integer | yes | The ID of a board's list | -| `position` | integer | yes | The position of the list | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | +| `position` | integer | yes | The position of the list | ```bash curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists/1?position=2 @@ -219,9 +385,9 @@ DELETE /projects/:id/boards/:board_id/lists/:list_id | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | -| `list_id` | integer | yes | The ID of a board's list | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | ```bash curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists/1 -- cgit v1.2.1 From 27a75ea1757d1c1b67bf501ec333221ed5e92d04 Mon Sep 17 00:00:00 2001 From: Jan Provaznik Date: Wed, 20 Dec 2017 10:01:21 +0100 Subject: Backport 'Rebase' feature from EE to CE When a project uses fast-forward merging strategy user has to rebase MRs to target branch before it can be merged. Now user can do rebase in UI by clicking 'Rebase' button instead of doing rebase locally. This feature was already present in EE, this is only backport of the feature to CE. Couple of changes: * removed rebase license check * renamed migration (changed timestamp) Closes #40301 --- doc/user/project/merge_requests/fast_forward_merge.md | 4 ++-- doc/user/project/merge_requests/img/ff_merge_mr.png | Bin 21380 -> 0 bytes doc/user/project/merge_requests/img/ff_merge_rebase.png | Bin 0 -> 26945 bytes 3 files changed, 2 insertions(+), 2 deletions(-) delete mode 100644 doc/user/project/merge_requests/img/ff_merge_mr.png create mode 100644 doc/user/project/merge_requests/img/ff_merge_rebase.png (limited to 'doc') diff --git a/doc/user/project/merge_requests/fast_forward_merge.md b/doc/user/project/merge_requests/fast_forward_merge.md index 085170d9f03..3cd91a185e3 100644 --- a/doc/user/project/merge_requests/fast_forward_merge.md +++ b/doc/user/project/merge_requests/fast_forward_merge.md @@ -9,7 +9,7 @@ When the fast-forward merge ([`--ff-only`][ffonly]) setting is enabled, no merge commits will be created and all merges are fast-forwarded, which means that merging is only allowed if the branch could be fast-forwarded. -When a fast-forward merge is not possible, the user must rebase the branch manually. +When a fast-forward merge is not possible, the user is given the option to rebase. ## Use cases @@ -25,7 +25,7 @@ merge commits. In such cases, the fast-forward merge is the perfect candidate. Now, when you visit the merge request page, you will be able to accept it **only if a fast-forward merge is possible**. -![Fast forward merge request](img/ff_merge_mr.png) +![Fast forward merge request](img/ff_merge_rebase.png) If the target branch is ahead of the source branch, you need to rebase the source branch locally before you will be able to do a fast-forward merge. diff --git a/doc/user/project/merge_requests/img/ff_merge_mr.png b/doc/user/project/merge_requests/img/ff_merge_mr.png deleted file mode 100644 index 241cc990343..00000000000 Binary files a/doc/user/project/merge_requests/img/ff_merge_mr.png and /dev/null differ diff --git a/doc/user/project/merge_requests/img/ff_merge_rebase.png b/doc/user/project/merge_requests/img/ff_merge_rebase.png new file mode 100644 index 00000000000..f6139f189ce Binary files /dev/null and b/doc/user/project/merge_requests/img/ff_merge_rebase.png differ -- cgit v1.2.1 From 3514b7248cf00bcee8a6b3133e4e157f656d30c6 Mon Sep 17 00:00:00 2001 From: Alessio Caiazza Date: Tue, 13 Jun 2017 22:03:34 +0200 Subject: Add status attribute to runner api entity --- doc/api/runners.md | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) (limited to 'doc') diff --git a/doc/api/runners.md b/doc/api/runners.md index 50981ed96bc..7495c6cdedb 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -31,7 +31,8 @@ Example response: "id": 6, "is_shared": false, "name": null, - "online": true + "online": true, + "status": "online" }, { "active": true, @@ -39,7 +40,8 @@ Example response: "id": 8, "is_shared": false, "name": null, - "online": false + "online": false, + "status": "offline" } ] ``` @@ -72,7 +74,8 @@ Example response: "id": 1, "is_shared": true, "name": null, - "online": true + "online": true, + "status": "online" }, { "active": true, @@ -81,6 +84,7 @@ Example response: "is_shared": true, "name": null, "online": false + "status": "offline" }, { "active": true, @@ -89,6 +93,7 @@ Example response: "is_shared": false, "name": null, "online": true + "status": "paused" }, { "active": true, @@ -96,7 +101,8 @@ Example response: "id": 8, "is_shared": false, "name": null, - "online": false + "online": false, + "status": "offline" } ] ``` @@ -129,6 +135,7 @@ Example response: "contacted_at": "2016-01-25T16:39:48.066Z", "name": null, "online": true, + "status": "online", "platform": null, "projects": [ { @@ -184,6 +191,7 @@ Example response: "contacted_at": "2016-01-25T16:39:48.066Z", "name": null, "online": true, + "status": "online", "platform": null, "projects": [ { @@ -336,7 +344,8 @@ Example response: "id": 8, "is_shared": false, "name": null, - "online": false + "online": false, + "status": "offline" }, { "active": true, @@ -345,6 +354,7 @@ Example response: "is_shared": true, "name": null, "online": true + "status": "paused" } ] ``` @@ -375,7 +385,8 @@ Example response: "id": 9, "is_shared": false, "name": null, - "online": true + "online": true, + "status": "online" } ``` -- cgit v1.2.1 From 13926246020fb41599e1a7b908912bb2e61f114f Mon Sep 17 00:00:00 2001 From: James Lopez Date: Fri, 5 Jan 2018 11:16:18 +0100 Subject: add deprecation and removal issue to docs --- doc/administration/raketasks/check.md | 9 +++++++++ 1 file changed, 9 insertions(+) (limited to 'doc') diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index 7dabc014bad..831b73237b6 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -28,6 +28,12 @@ exactly which repositories are causing the trouble. ### Check all GitLab repositories +>**Note:** +> +> - `gitlab:repo:check` has been deprecated in favour of `gitlab:git:fsck` +> - [Deprecated][ce-15931] in GitLab 10.4. +> - `gitlab:repo:check` will be removed in the future. [Removal issue][ce-41699] + This task loops through all repositories on the GitLab server and runs the 3 integrity checks described previously. @@ -76,3 +82,6 @@ The LDAP check Rake task will test the bind_dn and password credentials (if configured) and will list a sample of LDAP users. This task is also executed as part of the `gitlab:check` task, but can run independently. See [LDAP Rake Tasks - LDAP Check](ldap.md#check) for details. + +[ce-15931]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15931 +[ce-41699]: https://gitlab.com/gitlab-org/gitlab-ce/issues/41699 -- cgit v1.2.1 From 0feb2437e1318fa1b3157bf322d6759bb32bc01a Mon Sep 17 00:00:00 2001 From: James Lopez Date: Fri, 5 Jan 2018 10:30:57 +0000 Subject: Update check.md --- doc/administration/raketasks/check.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index 831b73237b6..c39cb49b1c6 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -30,7 +30,7 @@ exactly which repositories are causing the trouble. >**Note:** > -> - `gitlab:repo:check` has been deprecated in favour of `gitlab:git:fsck` +> - `gitlab:repo:check` has been deprecated in favor of `gitlab:git:fsck` > - [Deprecated][ce-15931] in GitLab 10.4. > - `gitlab:repo:check` will be removed in the future. [Removal issue][ce-41699] -- cgit v1.2.1 From c5e2c0665fe7e4937689cfedaa064aa64f538c8b Mon Sep 17 00:00:00 2001 From: "Jacob Vosmaer (GitLab)" Date: Fri, 5 Jan 2018 11:31:12 +0000 Subject: Allow local tests to use a modified Gitaly --- doc/development/gitaly.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) (limited to 'doc') diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index ca2048c7019..26abf967dcf 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -97,6 +97,29 @@ describe 'Gitaly Request count tests' do end ``` +## Running tests with a locally modified version of Gitaly + +Normally, gitlab-ce/ee tests use a local clone of Gitaly in `tmp/tests/gitaly` +pinned at the version specified in GITALY_SERVER_VERSION. If you want +to run tests locally against a modified version of Gitaly you can +replace `tmp/tests/gitaly` with a symlink. + +```shell +rm -rf tmp/tests/gitaly +ln -s /path/to/gitaly tmp/tests/gitaly +``` + +Make sure you run `make` in your local Gitaly directory before running +tests. Otherwise, Gitaly will fail to boot. + +If you make changes to your local Gitaly in between test runs you need +to manually run `make` again. + +Note that CI tests will not use your locally modified version of +Gitaly. To use a custom Gitaly version in CI you need to update +GITALY_SERVER_VERSION. You can use the format `=revision` to use a +non-tagged commit from https://gitlab.com/gitlab-org/gitaly in CI. + --- [Return to Development documentation](README.md) -- cgit v1.2.1 From 55980c7eca8e82e306fb3b8ade1f4a5b68a60e9f Mon Sep 17 00:00:00 2001 From: Felipe Artur Date: Fri, 5 Jan 2018 11:24:24 -0200 Subject: Remove EE only sections from docs --- doc/api/boards.md | 85 ------------------------------------------------------- 1 file changed, 85 deletions(-) (limited to 'doc') diff --git a/doc/api/boards.md b/doc/api/boards.md index a5f455e1c43..246de50323e 100644 --- a/doc/api/boards.md +++ b/doc/api/boards.md @@ -141,91 +141,6 @@ Example response: } ``` -## Create a board (EES-Only) - -Creates a board. - -``` -POST /projects/:id/boards -``` - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `name` | string | yes | The name of the new board | - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards?name=newboard -``` - -Example response: - -```json - { - "id": 1, - "project": { - "id": 5, - "name": "Diaspora Project Site", - "name_with_namespace": "Diaspora / Diaspora Project Site", - "path": "diaspora-project-site", - "path_with_namespace": "diaspora/diaspora-project-site", - "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", - "web_url": "http://example.com/diaspora/diaspora-project-site" - }, - "name": "newboard", - "milestone": { - "id": 12 - "title": "10.0" - }, - "lists" : [ - { - "id" : 1, - "label" : { - "name" : "Testing", - "color" : "#F0AD4E", - "description" : null - }, - "position" : 1 - }, - { - "id" : 2, - "label" : { - "name" : "Ready", - "color" : "#FF0000", - "description" : null - }, - "position" : 2 - }, - { - "id" : 3, - "label" : { - "name" : "Production", - "color" : "#FF5F00", - "description" : null - }, - "position" : 3 - } - ] - } -``` - -## Delete a board (EES-Only) - -Deletes a board. - -``` -DELETE /projects/:id/boards/:board_id -``` - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | - -```bash -curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1 -``` - ## List board lists Get a list of the board's lists. -- cgit v1.2.1 From bf8c5643a9cb665467deea0106fa35b91b2fa938 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Fri, 5 Jan 2018 15:18:17 +0100 Subject: Bump import/export version to 2.2.0 We need to bump import/export version because we introduced a new object's hierarchy that is not backwards compatible. --- doc/user/project/settings/import_export.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 1b8a84c9599..b8f865679a2 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -30,7 +30,8 @@ with all their related data and be moved into a new GitLab instance. | GitLab version | Import/Export version | | ---------------- | --------------------- | -| 10.3 to current | 0.2.1 | +| 10.4 to current | 0.2.2 | +| 10.3 | 0.2.1 | | 10.0 | 0.2.0 | | 9.4.0 | 0.1.8 | | 9.2.0 | 0.1.7 | -- cgit v1.2.1 From 8bdc6c74e82445048d66f6bf4be9dd0db7dc4737 Mon Sep 17 00:00:00 2001 From: Grzegorz Bizon Date: Fri, 5 Jan 2018 15:32:41 +0100 Subject: Rephrase paragraph about e2e tests in merge requests in docs --- doc/development/testing_guide/end_to_end_tests.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) (limited to 'doc') diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md index 30efe3e3b76..abe5b06e0f0 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -20,15 +20,13 @@ You can find these nightly pipelines at [GitLab QA pipelines page][gitlab-qa-pip ### Testing code in merge requests -It is also possible to trigger build of GitLab packages and then pass these -package to GitLab QA to run tests in a [pipeline][gitlab-qa-pipelines]. +It is possible to run end-to-end tests (eventually being run within a +[GitLab QA pipeline][gitlab-qa-pipelines]) for a merge request by triggering +the `package-qa` manual action, that should be present in a merge request +widget. -Developers can trigger the `package-qa` manual action, that should be present in -the merge request widget. - -It is also possible to trigger Gitlab QA pipeline from merge requests in -Omnibus GitLab project. You can find a manual action that is similar to -`package-qa`, mentioned above, in your Omnibus-related merge requests as well. +Mmanual action that starts end-to-end tests is also available in merge requests +in Omnibus GitLab project. Below you can read more about how to use it and how does it work. -- cgit v1.2.1 From 34b9cc9674554155af49c9a7fe60aaeba72bb23d Mon Sep 17 00:00:00 2001 From: Brent Greeff Date: Fri, 5 Jan 2018 15:21:53 +0000 Subject: API: get participants from merge_requests & issues --- doc/api/issues.md | 39 +++++++++++++++++++++++++++++++++++++++ doc/api/merge_requests.md | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+) (limited to 'doc') diff --git a/doc/api/issues.md b/doc/api/issues.md index d2fefbe68aa..da89db17cd9 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -1124,6 +1124,45 @@ Example response: ``` +## Participants on issues + +``` +GET /projects/:id/issues/:issue_iid/participants +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `issue_iid` | integer | yes | The internal ID of a project's issue | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/93/participants +``` + +Example response: + +```json +[ + { + "id": 1, + "name": "John Doe1", + "username": "user1", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/c922747a93b40d1ea88262bf1aebee62?s=80&d=identicon", + "web_url": "http://localhost/user1" + }, + { + "id": 5, + "name": "John Doe5", + "username": "user5", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/4aea8cf834ed91844a2da4ff7ae6b491?s=80&d=identicon", + "web_url": "http://localhost/user5" + } +] +``` + + ## Comments on issues Comments are done via the [notes](notes.md) resource. diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 4d3592e8f71..24afcef9a31 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -308,6 +308,41 @@ Parameters: } ``` +## Get single MR participants + +Get a list of merge request participants. + +``` +GET /projects/:id/merge_requests/:merge_request_iid/participants +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `merge_request_iid` (required) - The internal ID of the merge request + + +```json +[ + { + "id": 1, + "name": "John Doe1", + "username": "user1", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/c922747a93b40d1ea88262bf1aebee62?s=80&d=identicon", + "web_url": "http://localhost/user1" + }, + { + "id": 2, + "name": "John Doe2", + "username": "user2", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/10fc7f102be8de7657fb4d80898bbfe3?s=80&d=identicon", + "web_url": "http://localhost/user2" + }, +] +``` + ## Get single MR commits Get a list of merge request commits. -- cgit v1.2.1 From 6d972724d426938a0bfd6744dc6464ec5f7e17f9 Mon Sep 17 00:00:00 2001 From: LUKE BENNETT Date: Sat, 6 Jan 2018 03:05:22 +0000 Subject: Update installation.md for 10.4.0.rc1 --- doc/install/installation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 67b89d608cc..2b7352d3561 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -299,9 +299,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-3-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-4-stable gitlab -**Note:** You can change `10-3-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `10-4-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It -- cgit v1.2.1 From 13be4f49f81109e612c73645f991e5e1b07b6d3f Mon Sep 17 00:00:00 2001 From: Takuya Noguchi Date: Sun, 7 Jan 2018 13:35:03 +0900 Subject: Substitute deprecated ui_charcoal with new default ui_indigo --- doc/development/fe_guide/style_guide_scss.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md index 77b308c4a43..86a8b4135af 100644 --- a/doc/development/fe_guide/style_guide_scss.md +++ b/doc/development/fe_guide/style_guide_scss.md @@ -216,7 +216,7 @@ If you want a line or set of lines to be ignored by the linter, you can use ```scss // This lint rule is disabled because the class name comes from a gem. // scss-lint:disable SelectorFormat -.ui_charcoal { +.ui_indigo { background-color: #333; } // scss-lint:enable SelectorFormat -- cgit v1.2.1 From a32fcf33ee56ebd7e2d07e377610cfcce53d5a88 Mon Sep 17 00:00:00 2001 From: Onuwa Nnachi Isaac Date: Mon, 8 Jan 2018 15:18:13 +0100 Subject: chore: remove symbolic link --- doc/doc | 1 - 1 file changed, 1 deletion(-) delete mode 120000 doc/doc (limited to 'doc') diff --git a/doc/doc b/doc/doc deleted file mode 120000 index e78a24e7fe6..00000000000 --- a/doc/doc +++ /dev/null @@ -1 +0,0 @@ -/home/isaac/apps/opensource/GitLab/gitlab-ce/doc \ No newline at end of file -- cgit v1.2.1 From d0b8f536a1865af3741fc3255325b7e211ed1d42 Mon Sep 17 00:00:00 2001 From: Yorick Peterse Date: Tue, 2 Jan 2018 17:21:28 +0100 Subject: Remove soft removals related code This removes all usage of soft removals except for the "pending delete" system implemented for projects. This in turn simplifies all the query plans of the models that used soft removals. Since we don't really use soft removals for anything useful there's no point in keeping it around. This _does_ mean that hard removals of issues (which only admins can do if I'm not mistaken) can influence the "iid" values, but that code is broken to begin with. More on this (and how to fix it) can be found in https://gitlab.com/gitlab-org/gitlab-ce/issues/31114. Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/37447 --- doc/api/pipeline_triggers.md | 5 ----- 1 file changed, 5 deletions(-) (limited to 'doc') diff --git a/doc/api/pipeline_triggers.md b/doc/api/pipeline_triggers.md index 9030ae32d17..e881e61d4ef 100644 --- a/doc/api/pipeline_triggers.md +++ b/doc/api/pipeline_triggers.md @@ -24,7 +24,6 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/ "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", @@ -55,7 +54,6 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/ "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", @@ -85,7 +83,6 @@ curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form descri "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", @@ -116,7 +113,6 @@ curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form descrip "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", @@ -146,7 +142,6 @@ curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitl "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", -- cgit v1.2.1 From be8b9021f2578ced36295557a4476119270463fc Mon Sep 17 00:00:00 2001 From: Mattia Rizzolo Date: Sun, 7 Jan 2018 14:58:34 +0000 Subject: Update irker documentation to mention an irker bug with key-protected channels. Signed-off-by: Mattia Rizzolo --- doc/user/project/integrations/irker.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/user/project/integrations/irker.md b/doc/user/project/integrations/irker.md index c63ea1316fe..ecdd83ce8f0 100644 --- a/doc/user/project/integrations/irker.md +++ b/doc/user/project/integrations/irker.md @@ -47,4 +47,8 @@ Irker accepts channel names of the form `chan` and `#chan`, both for the case, `Aorimn` is treated as a nick and no more as a channel name. Irker can also join password-protected channels. Users need to append -`?key=thesecretpassword` to the chan name. +`?key=thesecretpassword` to the chan name. When using this feature remember to +**not** put the `#` sign in front of the channel name; failing to do so will +result on irker joining a channel literally named `#chan?key=password` henceforth +leaking the channel key through the `/whois` IRC command (depending on IRC server +configuration). This is due to a long standing irker bug. -- cgit v1.2.1 From 97311688b5d1b33b1fa0b3958d88af19bc8089b8 Mon Sep 17 00:00:00 2001 From: Onuwa Nnachi Isaac Date: Mon, 8 Jan 2018 18:11:25 +0100 Subject: Fix: remove unnecessary line --- doc/development/architecture.md | 2 -- 1 file changed, 2 deletions(-) (limited to 'doc') diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 54029e00507..d1ba7d3dfc3 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -133,8 +133,6 @@ Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [v ### Log locations of the services -Note: `/home/git/` is shorthand for `/home/git`. - gitlabhq (includes Unicorn and Sidekiq logs) - `/home/git/gitlab/log/` contains `application.log`, `production.log`, `sidekiq.log`, `unicorn.stdout.log`, `githost.log` and `unicorn.stderr.log` normally. -- cgit v1.2.1 From 255a0f85e3b62845b58f5a4aa189e57f36992c77 Mon Sep 17 00:00:00 2001 From: Michael Kozono Date: Tue, 30 May 2017 16:24:45 -0700 Subject: Backport option to disable writing to `authorized_keys` file MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Originally branch 'mk-toggle-writing-to-auth-keys-1631' See merge request !2004 Squashed commits: Add authorized_keys_enabled to Application Settings Ensure default settings are exposed in UI Without this change, `authorized_keys_enabled` is unchecked when it is nil, even if it should be checked by default. Add “Speed up SSH operations” documentation Clarify the reasons for disabling writes Add "How to go back" section Tweak copy Update Application Setting screenshot --- .../img/write_to_authorized_keys_setting.png | Bin 0 -> 94218 bytes doc/administration/operations/index.md | 3 +- doc/administration/operations/speed_up_ssh.md | 69 +++++++++++++++++++++ 3 files changed, 71 insertions(+), 1 deletion(-) create mode 100644 doc/administration/operations/img/write_to_authorized_keys_setting.png create mode 100644 doc/administration/operations/speed_up_ssh.md (limited to 'doc') diff --git a/doc/administration/operations/img/write_to_authorized_keys_setting.png b/doc/administration/operations/img/write_to_authorized_keys_setting.png new file mode 100644 index 00000000000..232765f1917 Binary files /dev/null and b/doc/administration/operations/img/write_to_authorized_keys_setting.png differ diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index 320d71a9527..f96a084c853 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -13,4 +13,5 @@ by GitLab to another file system or another server. that to prioritize important jobs. - [Sidekiq MemoryKiller](sidekiq_memory_killer.md): Configure Sidekiq MemoryKiller to restart Sidekiq. -- [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer. \ No newline at end of file +- [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer. +- [Speed up SSH operations](speed_up_ssh.md) diff --git a/doc/administration/operations/speed_up_ssh.md b/doc/administration/operations/speed_up_ssh.md new file mode 100644 index 00000000000..9b260beb34f --- /dev/null +++ b/doc/administration/operations/speed_up_ssh.md @@ -0,0 +1,69 @@ +# Speed up SSH operations + +## The problem + +SSH operations become slow as the number of users grows. + +## The reason + +OpenSSH searches for a key to authorize a user via a linear search. In the worst case, such as when the user is not authorized to access GitLab, OpenSSH will scan the entire file to search for a key. This can take significant time and disk I/O, which will delay users attempting to push or pull to a repository. Making matters worse, if users add or remove keys frequently, the operating system may not be able to cache the authorized_keys file, which causes the disk to be accessed repeatedly. + +## The solution + +GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to check whether the user is authorized to access GitLab. + +> **Warning:** OpenSSH version 6.9+ is required because `AuthorizedKeysCommand` must be able to accept a fingerprint. These instructions will break installations using older versions of OpenSSH, such as those included with CentOS as of May 2017. + +Create this file at `/opt/gitlab-shell/authorized_keys`: + +``` +#!/bin/bash + +if [[ "$1" == "git" ]]; then + /opt/gitlab/embedded/service/gitlab-shell/bin/authorized_keys $2 +fi +``` + +Set appropriate ownership and permissions: + +``` +sudo chown root:git /opt/gitlab-shell/authorized_keys +sudo chmod 0650 /opt/gitlab-shell/authorized_keys +``` + +Add the following to `/etc/ssh/sshd_config`: + +``` +AuthorizedKeysCommand /opt/gitlab-shell/authorized_keys %u %k +AuthorizedKeysCommandUser git +``` + +Reload the sshd service: + +``` +sudo service sshd reload +``` + +Confirm that SSH is working by removing your user's SSH key in the UI, adding a new one, and attempting to pull a repo. + +> **Warning:** Do not disable writes until SSH is confirmed to be working perfectly because the file will quickly become out-of-date. + +In the case of lookup failures (which are not uncommon), the `authorized_keys` file will still be scanned. So git SSH performance will still be slow for many users as long as a large file exists. + +You can disable any more writes to the `authorized_keys` file by unchecking `Write to "authorized_keys" file` in the Application Settings of your GitLab installation. + +![Write to authorized keys setting](img/write_to_authorized_keys_setting.png) + +Again, confirm that SSH is working by removing your user's SSH key in the UI, adding a new one, and attempting to pull a repo. + +Then you can backup and delete your `authorized_keys` file for best performance. + +## How to go back to using the `authorized_keys` file + +This is a brief overview. Please refer to the above instructions for more context. + +1. Rebuild the `authorized_keys` file. See https://docs.gitlab.com/ce/administration/raketasks/maintenance.html#rebuild-authorized_keys-file +1. Enable writes to the `authorized_keys` file +1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` +1. Reload the sshd service +1. Remove the `/opt/gitlab-shell/authorized_keys` file -- cgit v1.2.1 From 07bd79cd721a13fc012b77b09e56cd709e5a22f3 Mon Sep 17 00:00:00 2001 From: Ernst van Nierop Date: Fri, 15 Dec 2017 12:39:44 +0000 Subject: Combine ssh docs and rename the doc Backport to CE, originally branch 'evn-ssh-clarify-docs See merge request gitlab-org/gitlab-ee!3753 --- .../operations/fast_ssh_key_lookup.md | 192 +++++++++++++++++++++ doc/administration/operations/index.md | 2 +- doc/administration/operations/speed_up_ssh.md | 70 +------- 3 files changed, 194 insertions(+), 70 deletions(-) create mode 100644 doc/administration/operations/fast_ssh_key_lookup.md (limited to 'doc') diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md new file mode 100644 index 00000000000..b86168f935a --- /dev/null +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -0,0 +1,192 @@ +# Fast lookup of authorized SSH keys in the database + +Regular SSH operations become slow as the number of users grows because OpenSSH +searches for a key to authorize a user via a linear search. In the worst case, +such as when the user is not authorized to access GitLab, OpenSSH will scan the +entire file to search for a key. This can take significant time and disk I/O, +which will delay users attempting to push or pull to a repository. Making +matters worse, if users add or remove keys frequently, the operating system may +not be able to cache the `authorized_keys` file, which causes the disk to be +accessed repeatedly. + +GitLab Shell solves this by providing a way to authorize SSH users via a fast, +indexed lookup in the GitLab database. This page describes how to enable the fast +lookup of authorized SSH keys. + +> **Warning:** OpenSSH version 6.9+ is required because +`AuthorizedKeysCommand` must be able to accept a fingerprint. These +instructions will break installations using older versions of OpenSSH, such as +those included with CentOS 6 as of September 2017. If you want to use this +feature for CentOS 6, follow [the instructions on how to build and install a custom OpenSSH package](#compiling-a-custom-version-of-openssh-for-centos-6) before continuing. + +## Setting up fast lookup via GitLab Shell + +GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup +to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to +check whether the user is authorized to access GitLab. + +Create the directory `/opt/gitlab-shell` first: + +```bash +sudo mkdir -p /opt/gitlab-shell +``` + +Create this file at `/opt/gitlab-shell/authorized_keys`: + +``` +#!/bin/bash + +if [[ "$1" == "git" ]]; then + /opt/gitlab/embedded/service/gitlab-shell/bin/authorized_keys $2 +fi +``` + +Set appropriate ownership and permissions: + +``` +sudo chown root:git /opt/gitlab-shell/authorized_keys +sudo chmod 0650 /opt/gitlab-shell/authorized_keys +``` + +Add the following to `/etc/ssh/sshd_config` or to `/assets/sshd_config` if you +are using Omnibus Docker: + +``` +AuthorizedKeysCommand /opt/gitlab-shell/authorized_keys %u %k +AuthorizedKeysCommandUser git +``` + +Reload OpenSSH: + +```bash +# Debian or Ubuntu installations +sudo service ssh reload + +# CentOS installations +sudo service sshd reload +``` + +Confirm that SSH is working by removing your user's SSH key in the UI, adding a +new one, and attempting to pull a repo. + +> **Warning:** Do not disable writes until SSH is confirmed to be working +perfectly because the file will quickly become out-of-date. + +In the case of lookup failures (which are not uncommon), the `authorized_keys` +file will still be scanned. So git SSH performance will still be slow for many +users as long as a large file exists. + +You can disable any more writes to the `authorized_keys` file by unchecking +`Write to "authorized_keys" file` in the Application Settings of your GitLab +installation. + +![Write to authorized keys setting](img/write_to_authorized_keys_setting.png) + +Again, confirm that SSH is working by removing your user's SSH key in the UI, +adding a new one, and attempting to pull a repo. + +Then you can backup and delete your `authorized_keys` file for best performance. + +## How to go back to using the `authorized_keys` file + +This is a brief overview. Please refer to the above instructions for more context. + +1. [Rebuild the `authorized_keys` file](../raketasks/maintenance.md#rebuild-authorized_keys-file) +1. Enable writes to the `authorized_keys` file in Application Settings +1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` or from `/assets/sshd_config` if you are using Omnibus Docker. +1. Reload sshd: `sudo service sshd reload` +1. Remove the `/opt/gitlab-shell/authorized_keys` file + +## Compiling a custom version of OpenSSH for CentOS 6 + +Building a custom version of OpenSSH is not necessary for Ubuntu 16.04 users, +since Ubuntu 16.04 ships with OpenSSH 7.2. + +It is also unnecessary for CentOS 7.4 users, as that version ships with +OpenSSH 7.4. If you are using CentOS 7.0 - 7.3, we strongly recommend that you +upgrade to CentOS 7.4 instead of following this procedure. This should be as +simple as running `yum update`. + +CentOS 6 users must build their own OpenSSH package to enable SSH lookups via +the database. The following instructions can be used to build OpenSSH 7.5: + +1. First, download the package and install the required packages: + + ``` + sudo su - + cd /tmp + curl --remote-name https://mirrors.evowise.com/pub/OpenBSD/OpenSSH/portable/openssh-7.5p1.tar.gz + tar xzvf openssh-7.5p1.tar.gz + yum install rpm-build gcc make wget openssl-devel krb5-devel pam-devel libX11-devel xmkmf libXt-devel + ``` + +3. Prepare the build by copying files to the right place: + + ``` + mkdir -p /root/rpmbuild/{SOURCES,SPECS} + cp ./openssh-7.5p1/contrib/redhat/openssh.spec /root/rpmbuild/SPECS/ + cp openssh-7.5p1.tar.gz /root/rpmbuild/SOURCES/ + cd /root/rpmbuild/SPECS + ``` + +3. Next, set the spec settings properly: + + ``` + sed -i -e "s/%define no_gnome_askpass 0/%define no_gnome_askpass 1/g" openssh.spec + sed -i -e "s/%define no_x11_askpass 0/%define no_x11_askpass 1/g" openssh.spec + sed -i -e "s/BuildPreReq/BuildRequires/g" openssh.spec + ``` + +3. Build the RPMs: + + ``` + rpmbuild -bb openssh.spec + ``` + +4. Ensure the RPMs were built: + + ``` + ls -al /root/rpmbuild/RPMS/x86_64/ + ``` + + You should see something as the following: + + ``` + total 1324 + drwxr-xr-x. 2 root root 4096 Jun 20 19:37 . + drwxr-xr-x. 3 root root 19 Jun 20 19:37 .. + -rw-r--r--. 1 root root 470828 Jun 20 19:37 openssh-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 490716 Jun 20 19:37 openssh-clients-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 17020 Jun 20 19:37 openssh-debuginfo-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 367516 Jun 20 19:37 openssh-server-7.5p1-1.x86_64.rpm + ``` + +5. Install the packages. OpenSSH packages will replace `/etc/pam.d/sshd` + with its own version, which may prevent users from logging in, so be sure + that the file is backed up and restored after installation: + + ``` + timestamp=$(date +%s) + cp /etc/pam.d/sshd pam-ssh-conf-$timestamp + rpm -Uvh /root/rpmbuild/RPMS/x86_64/*.rpm + yes | cp pam-ssh-conf-$timestamp /etc/pam.d/sshd + ``` + +6. Verify the installed version. In another window, attempt to login to the server: + + ``` + ssh -v + ``` + + You should see a line that reads: "debug1: Remote protocol version 2.0, remote software version OpenSSH_7.5" + + If not, you may need to restart sshd (e.g. `systemctl restart sshd.service`). + +7. *IMPORTANT!* Open a new SSH session to your server before exiting to make + sure everything is working! If you need to downgrade, simple install the + older package: + + ``` + # Only run this if you run into a problem logging in + yum downgrade openssh-server openssh openssh-clients + ``` diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index f96a084c853..5655b7efec6 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -14,4 +14,4 @@ that to prioritize important jobs. - [Sidekiq MemoryKiller](sidekiq_memory_killer.md): Configure Sidekiq MemoryKiller to restart Sidekiq. - [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer. -- [Speed up SSH operations](speed_up_ssh.md) +- [Speed up SSH operations](fast_ssh_key_lookup.md): Authorize SSH users via a fast, indexed lookup to the GitLab database. diff --git a/doc/administration/operations/speed_up_ssh.md b/doc/administration/operations/speed_up_ssh.md index 9b260beb34f..89265b3018b 100644 --- a/doc/administration/operations/speed_up_ssh.md +++ b/doc/administration/operations/speed_up_ssh.md @@ -1,69 +1 @@ -# Speed up SSH operations - -## The problem - -SSH operations become slow as the number of users grows. - -## The reason - -OpenSSH searches for a key to authorize a user via a linear search. In the worst case, such as when the user is not authorized to access GitLab, OpenSSH will scan the entire file to search for a key. This can take significant time and disk I/O, which will delay users attempting to push or pull to a repository. Making matters worse, if users add or remove keys frequently, the operating system may not be able to cache the authorized_keys file, which causes the disk to be accessed repeatedly. - -## The solution - -GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to check whether the user is authorized to access GitLab. - -> **Warning:** OpenSSH version 6.9+ is required because `AuthorizedKeysCommand` must be able to accept a fingerprint. These instructions will break installations using older versions of OpenSSH, such as those included with CentOS as of May 2017. - -Create this file at `/opt/gitlab-shell/authorized_keys`: - -``` -#!/bin/bash - -if [[ "$1" == "git" ]]; then - /opt/gitlab/embedded/service/gitlab-shell/bin/authorized_keys $2 -fi -``` - -Set appropriate ownership and permissions: - -``` -sudo chown root:git /opt/gitlab-shell/authorized_keys -sudo chmod 0650 /opt/gitlab-shell/authorized_keys -``` - -Add the following to `/etc/ssh/sshd_config`: - -``` -AuthorizedKeysCommand /opt/gitlab-shell/authorized_keys %u %k -AuthorizedKeysCommandUser git -``` - -Reload the sshd service: - -``` -sudo service sshd reload -``` - -Confirm that SSH is working by removing your user's SSH key in the UI, adding a new one, and attempting to pull a repo. - -> **Warning:** Do not disable writes until SSH is confirmed to be working perfectly because the file will quickly become out-of-date. - -In the case of lookup failures (which are not uncommon), the `authorized_keys` file will still be scanned. So git SSH performance will still be slow for many users as long as a large file exists. - -You can disable any more writes to the `authorized_keys` file by unchecking `Write to "authorized_keys" file` in the Application Settings of your GitLab installation. - -![Write to authorized keys setting](img/write_to_authorized_keys_setting.png) - -Again, confirm that SSH is working by removing your user's SSH key in the UI, adding a new one, and attempting to pull a repo. - -Then you can backup and delete your `authorized_keys` file for best performance. - -## How to go back to using the `authorized_keys` file - -This is a brief overview. Please refer to the above instructions for more context. - -1. Rebuild the `authorized_keys` file. See https://docs.gitlab.com/ce/administration/raketasks/maintenance.html#rebuild-authorized_keys-file -1. Enable writes to the `authorized_keys` file -1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` -1. Reload the sshd service -1. Remove the `/opt/gitlab-shell/authorized_keys` file +This document was moved to [another location](fast_ssh_key_lookup.md). -- cgit v1.2.1 From e1008da2a81a1c77c63b4f28ea8cb2aa498bc372 Mon Sep 17 00:00:00 2001 From: Douglas Barbosa Alexandre Date: Mon, 8 Jan 2018 19:22:17 -0200 Subject: Add docs for the gitlab:uploads:check rake task --- doc/administration/raketasks/check.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) (limited to 'doc') diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index c39cb49b1c6..7e452ae503d 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -76,6 +76,26 @@ Example output: ![gitlab:user:check_repos output](../img/raketasks/check_repos_output.png) +## Uploaded Files Integrity + +The uploads check Rake task will loop through all uploads in the database +and runs two checks to determine the integrity of each file: + +1. Check if the file exist in the file system. +1. Check if the checksum of the file in the file system matches the checksum in the database. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:uploads:check +``` + +**Source Installation** + +```bash +sudo -u git -H bundle exec rake gitlab:uploads:check RAILS_ENV=production +``` + ## LDAP Check The LDAP check Rake task will test the bind_dn and password credentials -- cgit v1.2.1 From f6c1d382591fd837e96e99ff115b7beb0e6b3f43 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Francisco=20Javier=20L=C3=B3pez?= Date: Tue, 9 Jan 2018 11:36:12 +0000 Subject: Add option to disable commit stats to commit API --- doc/api/commits.md | 1 + 1 file changed, 1 insertion(+) (limited to 'doc') diff --git a/doc/api/commits.md b/doc/api/commits.md index c9b72d4a1dd..63554c63057 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -159,6 +159,7 @@ Parameters: | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | `sha` | string | yes | The commit hash or name of a repository branch or tag | +| `stats` | boolean | no | Include commit stats. Default is true | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/5/repository/commits/master -- cgit v1.2.1 From 601dc0d8b3ae08f688653df1019f2f31e020b598 Mon Sep 17 00:00:00 2001 From: Filipa Lacerda Date: Tue, 9 Jan 2018 11:58:34 +0000 Subject: Fix broken specs --- doc/development/fe_guide/style_guide_js.md | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-) (limited to 'doc') diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 3c5d69e1f71..02773162801 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -364,6 +364,10 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. + + // bad + ``` #### Quotes @@ -513,25 +517,7 @@ On those a default key should not be provided. ``` 1. Properties in a Vue Component: - 1. `name` - 1. `props` - 1. `mixins` - 1. `directives` - 1. `components` - 1. `data` - 1. `computedProps` - 1. `methods` - 1. `beforeCreate` - 1. `created` - 1. `beforeMount` - 1. `mounted` - 1. `beforeUpdate` - 1. `updated` - 1. `activated` - 1. `deactivated` - 1. `beforeDestroy` - 1. `destroyed` - + Check [order of properties in components rule][vue-order]. #### Vue and Bootstrap @@ -588,3 +574,4 @@ The goal of this accord is to make sure we are all on the same page. [eslint-new]: http://eslint.org/docs/rules/no-new [eslint-plugin-vue]: https://github.com/vuejs/eslint-plugin-vue [eslint-plugin-vue-rules]: https://github.com/vuejs/eslint-plugin-vue#bulb-rules +[vue-order]: https://github.com/vuejs/eslint-plugin-vue/blob/master/docs/rules/order-in-components.md -- cgit v1.2.1 From 3c7fbb84c2929f93a5eef13dc2bbe6d3c6bb46bf Mon Sep 17 00:00:00 2001 From: Dmitriy Zaporozhets Date: Tue, 9 Jan 2018 14:18:13 +0000 Subject: Use 0.69 version of codeclimate image for CI code quality docs --- doc/ci/examples/code_climate.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md index 6a5821762cc..f919ed3c797 100644 --- a/doc/ci/examples/code_climate.md +++ b/doc/ci/examples/code_climate.md @@ -16,7 +16,8 @@ codequality: - docker:dind script: - docker pull codeclimate/codeclimate - - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate analyze -f json > codeclimate.json || true + - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 init + - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 analyze -f json > codeclimate.json || true artifacts: paths: [codeclimate.json] ``` -- cgit v1.2.1 From c73eb55d93f067c8e55632d38df57fdb51f08220 Mon Sep 17 00:00:00 2001 From: Nick Thomas Date: Wed, 20 Dec 2017 15:41:36 +0000 Subject: Use the new gitlab-shell authorized-keys helper in documentation --- .../operations/fast_ssh_key_lookup.md | 32 ++++------------------ 1 file changed, 5 insertions(+), 27 deletions(-) (limited to 'doc') diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index b86168f935a..835ed8c8006 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -25,34 +25,12 @@ GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to check whether the user is authorized to access GitLab. -Create the directory `/opt/gitlab-shell` first: - -```bash -sudo mkdir -p /opt/gitlab-shell -``` - -Create this file at `/opt/gitlab-shell/authorized_keys`: - -``` -#!/bin/bash - -if [[ "$1" == "git" ]]; then - /opt/gitlab/embedded/service/gitlab-shell/bin/authorized_keys $2 -fi -``` - -Set appropriate ownership and permissions: - -``` -sudo chown root:git /opt/gitlab-shell/authorized_keys -sudo chmod 0650 /opt/gitlab-shell/authorized_keys -``` - -Add the following to `/etc/ssh/sshd_config` or to `/assets/sshd_config` if you -are using Omnibus Docker: +Add the following to your `sshd_config` file. This is usuaully located at +`/etc/ssh/sshd_config`, but it will be `/assets/sshd_config` if you're using +Omnibus Docker: ``` -AuthorizedKeysCommand /opt/gitlab-shell/authorized_keys %u %k +AuthorizedKeysCommand /opt/embedded/gitlab-shell/bin/gitlab-shell-authorized-keys-check git %u %k AuthorizedKeysCommandUser git ``` @@ -70,7 +48,7 @@ Confirm that SSH is working by removing your user's SSH key in the UI, adding a new one, and attempting to pull a repo. > **Warning:** Do not disable writes until SSH is confirmed to be working -perfectly because the file will quickly become out-of-date. +perfectly, because the file will quickly become out-of-date. In the case of lookup failures (which are not uncommon), the `authorized_keys` file will still be scanned. So git SSH performance will still be slow for many -- cgit v1.2.1 From 80b8fe24a18151eae7262589432a3158354c7273 Mon Sep 17 00:00:00 2001 From: Douglas Barbosa Alexandre Date: Tue, 9 Jan 2018 18:34:44 -0200 Subject: Fix small typos --- doc/administration/raketasks/check.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index 7e452ae503d..94d71e32ef9 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -79,10 +79,10 @@ Example output: ## Uploaded Files Integrity The uploads check Rake task will loop through all uploads in the database -and runs two checks to determine the integrity of each file: +and run two checks to determine the integrity of each file: -1. Check if the file exist in the file system. -1. Check if the checksum of the file in the file system matches the checksum in the database. +1. Check if the file exist on the file system. +1. Check if the checksum of the file on the file system matches the checksum in the database. **Omnibus Installation** -- cgit v1.2.1 From d270693c86beda5c44ec4e5d842d81961a8809e6 Mon Sep 17 00:00:00 2001 From: Douglas Barbosa Alexandre Date: Tue, 9 Jan 2018 22:02:44 -0200 Subject: Add docs for ENV variables take gitlab:uploads:check rake task supports --- doc/administration/raketasks/check.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'doc') diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index 94d71e32ef9..d1ed152b58c 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -96,6 +96,19 @@ sudo gitlab-rake gitlab:uploads:check sudo -u git -H bundle exec rake gitlab:uploads:check RAILS_ENV=production ``` +This task also accepts some environment variables which you can use to override +certain values: + +Variable | Type | Description +-------- | ---- | ----------- +`BATCH` | integer | Specifies the size of the batch. Defaults to 200. +`ID_FROM` | integer | Specifies the ID to start from, inclusive of the value. +`ID_TO` | integer | Specifies the ID value to end at, inclusive of the value. + +```bash +sudo gitlab-rake gitlab:uploads:check BATCH=100 ID_FROM=50 ID_TO=250 +``` + ## LDAP Check The LDAP check Rake task will test the bind_dn and password credentials -- cgit v1.2.1 From 8cce56b044a3718e20e01dfa2b1871d15f04bef3 Mon Sep 17 00:00:00 2001 From: John Doe Date: Wed, 10 Jan 2018 07:53:50 +0100 Subject: Change order of reference notes in permissions docs Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/41137 --- doc/user/permissions.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) (limited to 'doc') diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 4fa83388d0c..708d07fcec9 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -200,7 +200,7 @@ instance and project. In addition, all admins can use the admin interface under |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | -| Erase job artifacts and trace | | ✓ [^7] | ✓ | ✓ | +| Erase job artifacts and trace | | ✓ [^5] | ✓ | ✓ | | Remove project | | | ✓ | ✓ | | Create project | | | ✓ | ✓ | | Change project configuration | | | ✓ | ✓ | @@ -223,13 +223,13 @@ users: | Run CI job | | ✓ | ✓ | ✓ | | Clone source and LFS from current project | | ✓ | ✓ | ✓ | | Clone source and LFS from public projects | | ✓ | ✓ | ✓ | -| Clone source and LFS from internal projects | | ✓ [^5] | ✓ [^5] | ✓ | -| Clone source and LFS from private projects | | ✓ [^6] | ✓ [^6] | ✓ [^6] | +| Clone source and LFS from internal projects | | ✓ [^6] | ✓ [^6] | ✓ | +| Clone source and LFS from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | | Push source and LFS | | | | | | Pull container images from current project | | ✓ | ✓ | ✓ | | Pull container images from public projects | | ✓ | ✓ | ✓ | -| Pull container images from internal projects| | ✓ [^5] | ✓ [^5] | ✓ | -| Pull container images from private projects | | ✓ [^6] | ✓ [^6] | ✓ [^6] | +| Pull container images from internal projects| | ✓ [^6] | ✓ [^6] | ✓ | +| Pull container images from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | | Push container images to current project | | ✓ | ✓ | ✓ | | Push container images to other projects | | | | | @@ -259,12 +259,13 @@ with the permissions described on the documentation on [auditor users permission Auditor users are available in [GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/) only. -[^1]: On public and internal projects, all users are able to perform this action. +[^1]: On public and internal projects, all users are able to perform this action [^2]: Guest users can only view the confidential issues they created themselves [^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** [^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner -[^5]: Only if user is not external one. -[^6]: Only if user is a member of the project. -[^7]: Only if the build was triggered by the user +[^5]: Only if the job was triggered by the user +[^6]: Only if user is not external one +[^7]: Only if user is a member of the project + [ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 [new-mod]: project/new_ci_build_permissions_model.md -- cgit v1.2.1 From f3281f1a0c171869ab085b27a80982c47ea4f198 Mon Sep 17 00:00:00 2001 From: Fabio Busatto Date: Wed, 10 Jan 2018 13:12:00 +0000 Subject: Remove Beta notice for Clusters --- doc/user/project/clusters/index.md | 3 --- 1 file changed, 3 deletions(-) (limited to 'doc') diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index d5619c7b563..5f14d232cb1 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -2,9 +2,6 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) in 10.1. -CAUTION: **Warning:** -The Cluster integration is currently in **Beta**. - With a cluster associated to your project, you can use Review Apps, deploy your applications, run your pipelines, and much more, in an easy way. -- cgit v1.2.1 From 7ed08fcd5db058500b3092fd94791e9bd8a29bb7 Mon Sep 17 00:00:00 2001 From: Tony Rom Date: Wed, 10 Jan 2018 20:05:34 +0300 Subject: Fix style --- doc/api/merge_requests.md | 2 -- 1 file changed, 2 deletions(-) (limited to 'doc') diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index a3261cf7ee4..f9cbdffc254 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -444,8 +444,6 @@ Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user - `merge_request_iid` (required) - The internal ID of the merge request -Example of response - ```json [ { -- cgit v1.2.1 From c94d6ad199b0904ce64164a68429fa2df8d66a6a Mon Sep 17 00:00:00 2001 From: Marcia Ramos Date: Wed, 10 Jan 2018 17:10:38 +0000 Subject: Docs: move article "installing Git" to its topic-related dir --- doc/articles/how_to_install_git/index.md | 67 +----------------------------- doc/articles/index.md | 12 ------ doc/topics/git/how_to_install_git/index.md | 66 +++++++++++++++++++++++++++++ doc/topics/git/index.md | 65 +++++++++++++++-------------- 4 files changed, 101 insertions(+), 109 deletions(-) create mode 100644 doc/topics/git/how_to_install_git/index.md (limited to 'doc') diff --git a/doc/articles/how_to_install_git/index.md b/doc/articles/how_to_install_git/index.md index 37b60501ce2..3e6003a33b7 100644 --- a/doc/articles/how_to_install_git/index.md +++ b/doc/articles/how_to_install_git/index.md @@ -1,66 +1 @@ -# Installing Git - -> **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** user guide || -> **Level:** beginner || -> **Author:** [Sean Packham](https://gitlab.com/SeanPackham) || -> **Publication date:** 2017-05-15 - -To begin contributing to GitLab projects -you will need to install the Git client on your computer. -This article will show you how to install Git on macOS, Ubuntu Linux and Windows. - -## Install Git on macOS using the Homebrew package manager - -Although it is easy to use the version of Git shipped with macOS -or install the latest version of Git on macOS by downloading it from the project website, -we recommend installing it via Homebrew to get access to -an extensive selection of dependancy managed libraries and applications. - -If you are sure you don't need access to any additional development libraries -or don't have approximately 15gb of available disk space for Xcode and Homebrew -use one of the the aforementioned methods. - -### Installing Xcode - -Xcode is needed by Homebrew to build dependencies. -You can install [XCode](https://developer.apple.com/xcode/) -through the macOS App Store. - -### Installing Homebrew - -Once Xcode is installed browse to the [Homebrew website](http://brew.sh/index.html) -for the official Homebrew installation instructions. - -### Installing Git via Homebrew - -With Homebrew installed you are now ready to install Git. -Open a Terminal and enter in the following command: - -```bash -brew install git -``` - -Congratulations you should now have Git installed via Homebrew. -Next read our article on [adding an SSH key to GitLab](../../ssh/README.md). - -## Install Git on Ubuntu Linux - -On Ubuntu and other Linux operating systems -it is recommended to use the built in package manager to install Git. - -Open a Terminal and enter in the following commands -to install the latest Git from the official Git maintained package archives: - -```bash -sudo apt-add-repository ppa:git-core/ppa -sudo apt-get update -sudo apt-get install git -``` - -Congratulations you should now have Git installed via the Ubuntu package manager. -Next read our article on [adding an SSH key to GitLab](../../ssh/README.md). - -## Installing Git on Windows from the Git website - -Browse to the [Git website](https://git-scm.com/) and download and install Git for Windows. -Next read our article on [adding an SSH key to GitLab](../../ssh/README.md). +This document was moved to [another location](../../topics/git/how_to_install_git/index.md). diff --git a/doc/articles/index.md b/doc/articles/index.md index 06675e15d76..d8a7874b96d 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -30,18 +30,6 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM | [How to use GitLab CI and MacStadium to build your macOS or iOS projects](https://about.gitlab.com/2017/05/15/how-to-use-macstadium-and-gitlab-ci-to-build-your-macos-or-ios-projects/) | Technical overview | 2017-05-15 | | [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) | Tutorial | 2016-03-10 | -## Git - -Learn how to use [Git with GitLab](../topics/git/index.md): - -| Article title | Category | Publishing date | -| :------------ | :------: | --------------: | -| [Numerous _undo_ possibilities in Git](numerous_undo_possibilities_in_git/index.md) | Tutorial | 2017-08-17 | -| [Why Git is Worth the Learning Curve](https://about.gitlab.com/2017/05/17/learning-curve-is-the-biggest-challenge-developers-face-with-git/) | Concepts | 2017-05-17 | -| [How to install Git](how_to_install_git/index.md) | Tutorial | 2017-05-15 | -| [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) | Tutorial | 2017-01-30 | -| [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) | Technical overview | 2016-12-08 | - ## GitLab Pages Learn how to deploy a static website with [GitLab Pages](../user/project/pages/index.md#getting-started): diff --git a/doc/topics/git/how_to_install_git/index.md b/doc/topics/git/how_to_install_git/index.md new file mode 100644 index 00000000000..cdf61057449 --- /dev/null +++ b/doc/topics/git/how_to_install_git/index.md @@ -0,0 +1,66 @@ +# Installing Git + +> **[Article Type](../../../development/writing_documentation.html#types-of-technical-articles):** user guide || +> **Level:** beginner || +> **Author:** [Sean Packham](https://gitlab.com/SeanPackham) || +> **Publication date:** 2017-05-15 + +To begin contributing to GitLab projects +you will need to install the Git client on your computer. +This article will show you how to install Git on macOS, Ubuntu Linux and Windows. + +## Install Git on macOS using the Homebrew package manager + +Although it is easy to use the version of Git shipped with macOS +or install the latest version of Git on macOS by downloading it from the project website, +we recommend installing it via Homebrew to get access to +an extensive selection of dependancy managed libraries and applications. + +If you are sure you don't need access to any additional development libraries +or don't have approximately 15gb of available disk space for Xcode and Homebrew +use one of the the aforementioned methods. + +### Installing Xcode + +Xcode is needed by Homebrew to build dependencies. +You can install [XCode](https://developer.apple.com/xcode/) +through the macOS App Store. + +### Installing Homebrew + +Once Xcode is installed browse to the [Homebrew website](http://brew.sh/index.html) +for the official Homebrew installation instructions. + +### Installing Git via Homebrew + +With Homebrew installed you are now ready to install Git. +Open a Terminal and enter in the following command: + +```bash +brew install git +``` + +Congratulations you should now have Git installed via Homebrew. +Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). + +## Install Git on Ubuntu Linux + +On Ubuntu and other Linux operating systems +it is recommended to use the built in package manager to install Git. + +Open a Terminal and enter in the following commands +to install the latest Git from the official Git maintained package archives: + +```bash +sudo apt-add-repository ppa:git-core/ppa +sudo apt-get update +sudo apt-get install git +``` + +Congratulations you should now have Git installed via the Ubuntu package manager. +Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). + +## Installing Git on Windows from the Git website + +Browse to the [Git website](https://git-scm.com/) and download and install Git for Windows. +Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 588f4fa369f..f69e2e49f0c 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -14,6 +14,7 @@ We've gathered some resources to help you to get the best from Git with GitLab. ## Getting started - [Git concepts](../../university/training/user_training.md#git-concepts) +- [How to install Git](how_to_install_git/index.md) - [Start using Git on the command line](../../gitlab-basics/start-using-git.md) - [Command Line basic commands](../../gitlab-basics/command-line-commands.md) - [GitLab Git Cheat Sheet (download)](https://gitlab.com/gitlab-com/marketing/raw/master/design/print/git-cheatsheet/print-pdf/git-cheatsheet.pdf) @@ -21,27 +22,39 @@ We've gathered some resources to help you to get the best from Git with GitLab. - [Revert a commit](../../user/project/merge_requests/revert_changes.md#reverting-a-commit) - [Cherry-picking a commit](../../user/project/merge_requests/cherry_pick_changes.md#cherry-picking-a-commit) - [Squashing commits](../../workflow/gitlab_flow.md#squashing-commits-with-rebase) -- **Articles:** - - [Numerous _undo_ possibilities in Git](../../articles/numerous_undo_possibilities_in_git/index.md) - - [How to install Git](../../articles/how_to_install_git/index.md) - - [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) - - [Eight Tips to help you work better with Git](https://about.gitlab.com/2015/02/19/8-tips-to-help-you-work-better-with-git/) -- **Presentations:** - - [GLU Course: About Version Control](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit?usp=sharing) -- **Third-party resources:** - - What is [Git](https://git-scm.com) - - [Version control](https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control) - - [Getting Started - Git Basics](https://git-scm.com/book/en/v2/Getting-Started-Git-Basics) - - [Getting Started - Installing Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - - [Git on the Server - GitLab](https://git-scm.com/book/en/v2/Git-on-the-Server-GitLab) + +**Third-party references:** + +- [Getting Started - Git website](https://git-scm.com) +- [Getting Started - Version control](https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control) +- [Getting Started - Git Basics](https://git-scm.com/book/en/v2/Getting-Started-Git-Basics) +- [Getting Started - Installing Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +- [Git on the Server - GitLab](https://git-scm.com/book/en/v2/Git-on-the-Server-GitLab) + +### Concepts + +- Article (2017-05-17): [Why Git is Worth the Learning Curve](https://about.gitlab.com/2017/05/17/learning-curve-is-the-biggest-challenge-developers-face-with-git/) +- Article (2016-05-11): [The future of SaaS hosted Git repository pricing](https://about.gitlab.com/2016/05/11/git-repository-pricing/) +- GLU Course (Presentation): [About Version Control](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit?usp=sharing) + +## Exploring Git + +- [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) +- [Eight Tips to help you work better with Git](https://about.gitlab.com/2015/02/19/8-tips-to-help-you-work-better-with-git/) + +## Troubleshooting Git + +- [Numerous _undo_ possibilities in Git](../../articles/numerous_undo_possibilities_in_git/index.md) +- Learn a few [Git troubleshooting](troubleshooting_git.md) techniques to help you out. ## Branching strategies -- **Articles:** - - [GitLab Flow](https://about.gitlab.com/2014/09/29/gitlab-flow/) -- **Third-party resources:** - - [Git Branching - Branches in a Nutshell](https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell) - - [Git Branching - Branching Workflows](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows) +- [GitLab Flow](https://about.gitlab.com/2014/09/29/gitlab-flow/) + +**Third-party references:** + +- [Git Branching - Branches in a Nutshell](https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell) +- [Git Branching - Branching Workflows](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows) ## Advanced use @@ -55,17 +68,7 @@ We've gathered some resources to help you to get the best from Git with GitLab. ## Git LFS -- [Git LFS](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) +- [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) +- [GitLab Git LFS documentation](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) - [Git-Annex to Git-LFS migration guide](https://docs.gitlab.com/ee/workflow/lfs/migrate_from_git_annex_to_git_lfs.html) -- **Articles:** - - [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) - - [Towards a production quality open source Git LFS server](https://about.gitlab.com/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) - -## Troubleshooting - -- Learn a few [Git troubleshooting](troubleshooting_git.md) techniques to help you out. - -## General information - -- **Articles:** - - [The future of SaaS hosted Git repository pricing](https://about.gitlab.com/2016/05/11/git-repository-pricing/) +- Article (2015-08-13): [Towards a production quality open source Git LFS server](https://about.gitlab.com/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) -- cgit v1.2.1 From 7e4d640d3ee18e9f7050f77a26fbfa54b5c680bc Mon Sep 17 00:00:00 2001 From: Xiaogang Wen Date: Wed, 10 Jan 2018 18:18:39 +0000 Subject: Resolve "Include the Crowd application permission information on the "Crowd OmniAuth Provider" page" --- doc/administration/auth/crowd.md | 12 ++++++++++++ .../auth/img/crowd_application_authorisation.png | Bin 0 -> 126994 bytes 2 files changed, 12 insertions(+) create mode 100644 doc/administration/auth/img/crowd_application_authorisation.png (limited to 'doc') diff --git a/doc/administration/auth/crowd.md b/doc/administration/auth/crowd.md index 2c289c67a6d..6db74958d5a 100644 --- a/doc/administration/auth/crowd.md +++ b/doc/administration/auth/crowd.md @@ -66,3 +66,15 @@ On the sign in page there should now be a Crowd tab in the sign in form. [reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure [restart]: ../restart_gitlab.md#installations-from-source + +## Troubleshooting + +If you see an error message like the one below when you sign in after Crowd authentication is configured, you may want to consult the Crowd administrator for the Crowd log file to know the exact cause: + +``` +could not authorize you from Crowd because invalid credentials +``` + +Please make sure the Crowd users who need to login to GitLab are authorized to [the application](#configure-a-new-crowd-application) in the step of **Authorisation**. This could be verified by try "Authentication test" for Crowd as of 2.11. + +![Example Crowd application authorisation configuration](img/crowd_application_authorisation.png) \ No newline at end of file diff --git a/doc/administration/auth/img/crowd_application_authorisation.png b/doc/administration/auth/img/crowd_application_authorisation.png new file mode 100644 index 00000000000..70339891b34 Binary files /dev/null and b/doc/administration/auth/img/crowd_application_authorisation.png differ -- cgit v1.2.1 From 188b6e007dbcc6da91e46b4ae00d2e714f29e62d Mon Sep 17 00:00:00 2001 From: Marcia Ramos Date: Wed, 10 Jan 2018 16:37:24 -0200 Subject: move article to its topic-related dir, update links --- doc/articles/index.md | 1 - .../img/add-gitlab-to-project.png | Bin 37386 -> 0 bytes .../openshift_and_gitlab/img/add-to-project.png | Bin 21672 -> 0 bytes .../openshift_and_gitlab/img/create-project-ui.png | Bin 22290 -> 0 bytes .../openshift_and_gitlab/img/gitlab-logs.png | Bin 70858 -> 0 bytes .../openshift_and_gitlab/img/gitlab-overview.png | Bin 106432 -> 0 bytes .../openshift_and_gitlab/img/gitlab-running.png | Bin 107993 -> 0 bytes .../openshift_and_gitlab/img/gitlab-scale.png | Bin 36628 -> 0 bytes .../openshift_and_gitlab/img/gitlab-settings.png | Bin 111366 -> 0 bytes .../openshift_and_gitlab/img/no-resources.png | Bin 34669 -> 0 bytes .../img/openshift-infra-project.png | Bin 95725 -> 0 bytes .../openshift_and_gitlab/img/pods-overview.png | Bin 106861 -> 0 bytes doc/articles/openshift_and_gitlab/img/rc-name.png | Bin 51390 -> 0 bytes .../openshift_and_gitlab/img/running-pods.png | Bin 29818 -> 0 bytes .../openshift_and_gitlab/img/storage-volumes.png | Bin 49584 -> 0 bytes .../openshift_and_gitlab/img/web-console.png | Bin 34774 -> 0 bytes doc/articles/openshift_and_gitlab/index.md | 511 +-------------------- doc/install/README.md | 2 +- .../img/add-gitlab-to-project.png | Bin 0 -> 37386 bytes .../openshift_and_gitlab/img/add-to-project.png | Bin 0 -> 21672 bytes .../openshift_and_gitlab/img/create-project-ui.png | Bin 0 -> 22290 bytes .../openshift_and_gitlab/img/gitlab-logs.png | Bin 0 -> 70858 bytes .../openshift_and_gitlab/img/gitlab-overview.png | Bin 0 -> 106432 bytes .../openshift_and_gitlab/img/gitlab-running.png | Bin 0 -> 107993 bytes .../openshift_and_gitlab/img/gitlab-scale.png | Bin 0 -> 36628 bytes .../openshift_and_gitlab/img/gitlab-settings.png | Bin 0 -> 111366 bytes .../openshift_and_gitlab/img/no-resources.png | Bin 0 -> 34669 bytes .../img/openshift-infra-project.png | Bin 0 -> 95725 bytes .../openshift_and_gitlab/img/pods-overview.png | Bin 0 -> 106861 bytes doc/install/openshift_and_gitlab/img/rc-name.png | Bin 0 -> 51390 bytes .../openshift_and_gitlab/img/running-pods.png | Bin 0 -> 29818 bytes .../openshift_and_gitlab/img/storage-volumes.png | Bin 0 -> 49584 bytes .../openshift_and_gitlab/img/web-console.png | Bin 0 -> 34774 bytes doc/install/openshift_and_gitlab/index.md | 510 ++++++++++++++++++++ 34 files changed, 512 insertions(+), 512 deletions(-) delete mode 100644 doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png delete mode 100644 doc/articles/openshift_and_gitlab/img/add-to-project.png delete mode 100644 doc/articles/openshift_and_gitlab/img/create-project-ui.png delete mode 100644 doc/articles/openshift_and_gitlab/img/gitlab-logs.png delete mode 100644 doc/articles/openshift_and_gitlab/img/gitlab-overview.png delete mode 100644 doc/articles/openshift_and_gitlab/img/gitlab-running.png delete mode 100644 doc/articles/openshift_and_gitlab/img/gitlab-scale.png delete mode 100644 doc/articles/openshift_and_gitlab/img/gitlab-settings.png delete mode 100644 doc/articles/openshift_and_gitlab/img/no-resources.png delete mode 100644 doc/articles/openshift_and_gitlab/img/openshift-infra-project.png delete mode 100644 doc/articles/openshift_and_gitlab/img/pods-overview.png delete mode 100644 doc/articles/openshift_and_gitlab/img/rc-name.png delete mode 100644 doc/articles/openshift_and_gitlab/img/running-pods.png delete mode 100644 doc/articles/openshift_and_gitlab/img/storage-volumes.png delete mode 100644 doc/articles/openshift_and_gitlab/img/web-console.png create mode 100644 doc/install/openshift_and_gitlab/img/add-gitlab-to-project.png create mode 100644 doc/install/openshift_and_gitlab/img/add-to-project.png create mode 100644 doc/install/openshift_and_gitlab/img/create-project-ui.png create mode 100644 doc/install/openshift_and_gitlab/img/gitlab-logs.png create mode 100644 doc/install/openshift_and_gitlab/img/gitlab-overview.png create mode 100644 doc/install/openshift_and_gitlab/img/gitlab-running.png create mode 100644 doc/install/openshift_and_gitlab/img/gitlab-scale.png create mode 100644 doc/install/openshift_and_gitlab/img/gitlab-settings.png create mode 100644 doc/install/openshift_and_gitlab/img/no-resources.png create mode 100644 doc/install/openshift_and_gitlab/img/openshift-infra-project.png create mode 100644 doc/install/openshift_and_gitlab/img/pods-overview.png create mode 100644 doc/install/openshift_and_gitlab/img/rc-name.png create mode 100644 doc/install/openshift_and_gitlab/img/running-pods.png create mode 100644 doc/install/openshift_and_gitlab/img/storage-volumes.png create mode 100644 doc/install/openshift_and_gitlab/img/web-console.png create mode 100644 doc/install/openshift_and_gitlab/index.md (limited to 'doc') diff --git a/doc/articles/index.md b/doc/articles/index.md index d8a7874b96d..ca605bb47ed 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -62,7 +62,6 @@ upgrade, integrate, migrate to GitLab: | :------------ | :------: | --------------: | | [Video Tutorial: Idea to Production on Google Container Engine (GKE)](https://about.gitlab.com/2017/01/23/video-tutorial-idea-to-production-on-google-container-engine-gke/) | Tutorial | 2017-01-23 | | [How to Setup a GitLab Instance on Microsoft Azure](https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/) | Tutorial | 2016-07-13 | -| [Get started with OpenShift Origin 3 and GitLab](openshift_and_gitlab/index.md) | Tutorial | 2016-06-28 | | [Getting started with GitLab and DigitalOcean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/) | Tutorial | 2016-04-27 | ## Software development diff --git a/doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png b/doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png deleted file mode 100644 index fcad4e59ae3..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/add-to-project.png b/doc/articles/openshift_and_gitlab/img/add-to-project.png deleted file mode 100644 index bd915a229f6..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/add-to-project.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/create-project-ui.png b/doc/articles/openshift_and_gitlab/img/create-project-ui.png deleted file mode 100644 index e72866f252a..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/create-project-ui.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-logs.png b/doc/articles/openshift_and_gitlab/img/gitlab-logs.png deleted file mode 100644 index 1e24080c7df..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/gitlab-logs.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-overview.png b/doc/articles/openshift_and_gitlab/img/gitlab-overview.png deleted file mode 100644 index 3c5df0ea101..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/gitlab-overview.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-running.png b/doc/articles/openshift_and_gitlab/img/gitlab-running.png deleted file mode 100644 index c7db691cb30..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/gitlab-running.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-scale.png b/doc/articles/openshift_and_gitlab/img/gitlab-scale.png deleted file mode 100644 index 4903c7d7498..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/gitlab-scale.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-settings.png b/doc/articles/openshift_and_gitlab/img/gitlab-settings.png deleted file mode 100644 index db4360ffef0..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/gitlab-settings.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/no-resources.png b/doc/articles/openshift_and_gitlab/img/no-resources.png deleted file mode 100644 index 480fb766468..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/no-resources.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/openshift-infra-project.png b/doc/articles/openshift_and_gitlab/img/openshift-infra-project.png deleted file mode 100644 index 8b9f85aa341..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/openshift-infra-project.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/pods-overview.png b/doc/articles/openshift_and_gitlab/img/pods-overview.png deleted file mode 100644 index e1cf08bd217..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/pods-overview.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/rc-name.png b/doc/articles/openshift_and_gitlab/img/rc-name.png deleted file mode 100644 index 889e34adbec..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/rc-name.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/running-pods.png b/doc/articles/openshift_and_gitlab/img/running-pods.png deleted file mode 100644 index 3fd4e56662f..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/running-pods.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/storage-volumes.png b/doc/articles/openshift_and_gitlab/img/storage-volumes.png deleted file mode 100644 index ae1e5381faa..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/storage-volumes.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/img/web-console.png b/doc/articles/openshift_and_gitlab/img/web-console.png deleted file mode 100644 index aa1425d4f94..00000000000 Binary files a/doc/articles/openshift_and_gitlab/img/web-console.png and /dev/null differ diff --git a/doc/articles/openshift_and_gitlab/index.md b/doc/articles/openshift_and_gitlab/index.md index c0bbcfe2a8a..b7594cfef7f 100644 --- a/doc/articles/openshift_and_gitlab/index.md +++ b/doc/articles/openshift_and_gitlab/index.md @@ -1,510 +1 @@ -# Getting started with OpenShift Origin 3 and GitLab - -> **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** tutorial || -> **Level:** intermediary || -> **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) || -> **Publication date:** 2016-06-28 - -## Introduction - -[OpenShift Origin][openshift] is an open source container application -platform created by [RedHat], based on [kubernetes] and [Docker]. That means -you can host your own PaaS for free and almost with no hassle. - -In this tutorial, we will see how to deploy GitLab in OpenShift using GitLab's -official Docker image while getting familiar with the web interface and CLI -tools that will help us achieve our goal. - ---- - -## Prerequisites - -OpenShift 3 is not yet deployed on RedHat's offered Online platform ([openshift.com]), -so in order to test it, we will use an [all-in-one Virtualbox image][vm] that is -offered by the OpenShift developers and managed by Vagrant. If you haven't done -already, go ahead and install the following components as they are essential to -test OpenShift easily: - -- [VirtualBox] -- [Vagrant] -- [OpenShift Client][oc] (`oc` for short) - -It is also important to mention that for the purposes of this tutorial, the -latest Origin release is used: - -- **oc** `v1.3.0` (must be [installed][oc-gh] locally on your computer) -- **openshift** `v1.3.0` (is pre-installed in the [VM image][vm-new]) -- **kubernetes** `v1.3.0` (is pre-installed in the [VM image][vm-new]) - ->**Note:** -If you intend to deploy GitLab on a production OpenShift cluster, there are some -limitations to bare in mind. Read on the [limitations](#current-limitations) -section for more information and follow the linked links for the relevant -discussions. - -Now that you have all batteries, let's see how easy it is to test OpenShift -on your computer. - -## Getting familiar with OpenShift Origin - -The environment we are about to use is based on CentOS 7 which comes with all -the tools needed pre-installed: Docker, kubernetes, OpenShift, etcd. - -### Test OpenShift using Vagrant - -As of this writing, the all-in-one VM is at version 1.3, and that's -what we will use in this tutorial. - -In short: - -1. Open a terminal and in a new directory run: - ```sh - vagrant init openshift/origin-all-in-one - ``` -1. This will generate a Vagrantfile based on the all-in-one VM image -1. In the same directory where you generated the Vagrantfile - enter: - - ```sh - vagrant up - ``` - -This will download the VirtualBox image and fire up the VM with some preconfigured -values as you can see in the Vagrantfile. As you may have noticed, you need -plenty of RAM (5GB in our example), so make sure you have enough. - -Now that OpenShift is setup, let's see how the web console looks like. - -### Explore the OpenShift web console - -Once Vagrant finishes its thing with the VM, you will be presented with a -message which has some important information. One of them is the IP address -of the deployed OpenShift platform and in particular . -Open this link with your browser and accept the self-signed certificate in -order to proceed. - -Let's login as admin with username/password `admin/admin`. This is what the -landing page looks like: - -![openshift web console](img/web-console.png) - -You can see that a number of [projects] are already created for testing purposes. - -If you head over the `openshift-infra` project, a number of services with their -respective pods are there to explore. - -![openshift web console](img/openshift-infra-project.png) - -We are not going to explore the whole interface, but if you want to learn about -the key concepts of OpenShift, read the [core concepts reference][core] in the -official documentation. - -### Explore the OpenShift CLI - -OpenShift Client (`oc`), is a powerful CLI tool that talks to the OpenShift API -and performs pretty much everything you can do from the web UI and much more. - -Assuming you have [installed][oc] it, let's explore some of its main -functionalities. - -Let's first see the version of `oc`: - -```sh -$ oc version - -oc v1.3.0 -kubernetes v1.3.0+52492b4 -``` - -With `oc help` you can see the top level arguments you can run with `oc` and -interact with your cluster, kubernetes, run applications, create projects and -much more. - -Let's login to the all-in-one VM and see how to achieve the same results like -when we visited the web console earlier. The username/password for the -administrator user is `admin/admin`. There is also a test user with username/ -password `user/user`, with limited access. Let's login as admin for the moment: - -```sh -$ oc login https://10.2.2.2:8443 - -Authentication required for https://10.2.2.2:8443 (openshift) -Username: admin -Password: -Login successful. - -You have access to the following projects and can switch between them with 'oc project ': - - * cockpit - * default (current) - * delete - * openshift - * openshift-infra - * sample - -Using project "default". -``` - -Switch to the `openshift-infra` project with: - -```sh -oc project openshift-infra -``` - -And finally, see its status: - -```sh -oc status -``` - -The last command should spit a bunch of information about the statuses of the -pods and the services, which if you look closely is what we encountered in the -second image when we explored the web console. - -You can always read more about `oc` in the [OpenShift CLI documentation][oc]. - -### Troubleshooting the all-in-one VM - -Using the all-in-one VM gives you the ability to test OpenShift whenever you -want. That means you get to play with it, shutdown the VM, and pick up where -you left off. - -Sometimes though, you may encounter some issues, like OpenShift not running -when booting up the VM. The web UI may not responding or you may see issues -when trying to login with `oc`, like: - -``` -The connection to the server 10.2.2.2:8443 was refused - did you specify the right host or port? -``` - -In that case, the OpenShift service might not be running, so in order to fix it: - -1. SSH into the VM by going to the directory where the Vagrantfile is and then - run: - - ```sh - vagrant ssh - ``` - -1. Run `systemctl` and verify by the output that the `openshift` service is not - running (it will be in red color). If that's the case start the service with: - - ```sh - sudo systemctl start openshift - ``` - -1. Verify the service is up with: - - ```sh - systemctl status openshift -l - ``` - -Now you will be able to login using `oc` (like we did before) and visit the web -console. - -## Deploy GitLab - -Now that you got a taste of what OpenShift looks like, let's deploy GitLab! - -### Create a new project - -First, we will create a new project to host our application. You can do this -either by running the CLI client: - -```bash -$ oc new-project gitlab -``` - -or by using the web interface: - -![Create a new project from the UI](img/create-project-ui.png) - -If you used the command line, `oc` automatically uses the new project and you -can see its status with: - -```sh -$ oc status - -In project gitlab on server https://10.2.2.2:8443 - -You have no services, deployment configs, or build configs. -Run 'oc new-app' to create an application. -``` - -If you visit the web console, you can now see `gitlab` listed in the projects list. - -The next step is to import the OpenShift template for GitLab. - -### Import the template - -The [template][templates] is basically a JSON file which describes a set of -related object definitions to be created together, as well as a set of -parameters for those objects. - -The template for GitLab resides in the Omnibus GitLab repository under the -docker directory. Let's download it locally with `wget`: - -```bash -wget https://gitlab.com/gitlab-org/omnibus-gitlab/raw/master/docker/openshift-template.json -``` - -And then let's import it in OpenShift: - -```bash -oc create -f openshift-template.json -n openshift -``` - ->**Note:** -The `-n openshift` namespace flag is a trick to make the template available to all -projects. If you recall from when we created the `gitlab` project, `oc` switched -to it automatically, and that can be verified by the `oc status` command. If -you omit the namespace flag, the application will be available only to the -current project, in our case `gitlab`. The `openshift` namespace is a global -one that the administrators should use if they want the application to be -available to all users. - -We are now ready to finally deploy GitLab! - -### Create a new application - -The next step is to use the template we previously imported. Head over to the -`gitlab` project and hit the **Add to Project** button. - -![Add to project](img/add-to-project.png) - -This will bring you to the catalog where you can find all the pre-defined -applications ready to deploy with the click of a button. Search for `gitlab` -and you will see the previously imported template: - -![Add GitLab to project](img/add-gitlab-to-project.png) - -Select it, and in the following screen you will be presented with the predefined -values used with the GitLab template: - -![GitLab settings](img/gitlab-settings.png) - -Notice at the top that there are three resources to be created with this -template: - -- `gitlab-ce` -- `gitlab-ce-redis` -- `gitlab-ce-postgresql` - -While PostgreSQL and Redis are bundled in Omnibus GitLab, the template is using -separate images as you can see from [this line][line] in the template. - -The predefined values have been calculated for the purposes of testing out -GitLab in the all-in-one VM. You don't need to change anything here, hit -**Create** to start the deployment. - -If you are deploying to production you will want to change the **GitLab instance -hostname** and use greater values for the volume sizes. If you don't provide a -password for PostgreSQL, it will be created automatically. - ->**Note:** -The `gitlab.apps.10.2.2.2.xip.io` hostname that is used by default will -resolve to the host with IP `10.2.2.2` which is the IP our VM uses. It is a -trick to have distinct FQDNs pointing to services that are on our local network. -Read more on how this works in . - -Now that we configured this, let's see how to manage and scale GitLab. - -## Manage and scale GitLab - -Setting up GitLab for the first time might take a while depending on your -internet connection and the resources you have attached to the all-in-one VM. -GitLab's docker image is quite big (~500MB), so you'll have to wait until -it's downloaded and configured before you use it. - -### Watch while GitLab gets deployed - -Navigate to the `gitlab` project at **Overview**. You can notice that the -deployment is in progress by the orange color. The Docker images are being -downloaded and soon they will be up and running. - -![GitLab overview](img/gitlab-overview.png) - -Switch to the **Browse > Pods** and you will eventually see all 3 pods in a -running status. Remember the 3 resources that were to be created when we first -created the GitLab app? This is where you can see them in action. - -![Running pods](img/running-pods.png) - -You can see GitLab being reconfigured by taking look at the logs in realtime. -Click on `gitlab-ce-2-j7ioe` (your ID will be different) and go to the **Logs** -tab. - -![GitLab logs](img/gitlab-logs.png) - -At a point you should see a _**gitlab Reconfigured!**_ message in the logs. -Navigate back to the **Overview** and hopefully all pods will be up and running. - -![GitLab running](img/gitlab-running.png) - -Congratulations! You can now navigate to your new shinny GitLab instance by -visiting where you will be asked to -change the root user password. Login using `root` as username and providing the -password you just set, and start using GitLab! - -### Scale GitLab with the push of a button - -If you reach to a point where your GitLab instance could benefit from a boost -of resources, you'd be happy to know that you can scale up with the push of a -button. - -In the **Overview** page just click the up arrow button in the pod where -GitLab is. The change is instant and you can see the number of [replicas] now -running scaled to 2. - -![GitLab scale](img/gitlab-scale.png) - -Upping the GitLab pods is actually like adding new application servers to your -cluster. You can see how that would work if you didn't use GitLab with -OpenShift by following the [HA documentation][ha] for the application servers. - -Bare in mind that you may need more resources (CPU, RAM, disk space) when you -scale up. If a pod is in pending state for too long, you can navigate to -**Browse > Events** and see the reason and message of the state. - -![No resources](img/no-resources.png) - -### Scale GitLab using the `oc` CLI - -Using `oc` is super easy to scale up the replicas of a pod. You may want to -skim through the [basic CLI operations][basic-cli] to get a taste how the CLI -commands are used. Pay extra attention to the object types as we will use some -of them and their abbreviated versions below. - -In order to scale up, we need to find out the name of the replication controller. -Let's see how to do that using the following steps. - -1. Make sure you are in the `gitlab` project: - - ```sh - oc project gitlab - ``` - -1. See what services are used for this project: - - ```sh - oc get svc - ``` - - The output will be similar to: - - ``` - NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE - gitlab-ce 172.30.243.177 22/TCP,80/TCP 5d - gitlab-ce-postgresql 172.30.116.75 5432/TCP 5d - gitlab-ce-redis 172.30.105.88 6379/TCP 5d - ``` - -1. We need to see the replication controllers of the `gitlab-ce` service. - Get a detailed view of the current ones: - - ```sh - oc describe rc gitlab-ce - ``` - - This will return a large detailed list of the current replication controllers. - Search for the name of the GitLab controller, usually `gitlab-ce-1` or if - that failed at some point and you spawned another one, it will be named - `gitlab-ce-2`. - -1. Scale GitLab using the previous information: - - ```sh - oc scale --replicas=2 replicationcontrollers gitlab-ce-2 - ``` - -1. Get the new replicas number to make sure scaling worked: - - ```sh - oc get rc gitlab-ce-2 - ``` - - which will return something like: - - ``` - NAME DESIRED CURRENT AGE - gitlab-ce-2 2 2 5d - ``` - -And that's it! We successfully scaled the replicas to 2 using the CLI. - -As always, you can find the name of the controller using the web console. Just -click on the service you are interested in and you will see the details in the -right sidebar. - -![Replication controller name](img/rc-name.png) - -### Autoscaling GitLab - -In case you were wondering whether there is an option to autoscale a pod based -on the resources of your server, the answer is yes, of course there is. - -We will not expand on this matter, but feel free to read the documentation on -OpenShift's website about [autoscaling]. - -## Current limitations - -As stated in the [all-in-one VM][vm] page: - -> By default, OpenShift will not allow a container to run as root or even a -non-random container assigned userid. Most Docker images in the Dockerhub do not -follow this best practice and instead run as root. - -The all-in-one VM we are using has this security turned off so it will not -bother us. In any case, it is something to keep in mind when deploying GitLab -on a production cluster. - -In order to deploy GitLab on a production cluster, you will need to assign the -GitLab service account to the `anyuid` Security Context. - -1. Edit the Security Context: - ```sh - oc edit scc anyuid - ``` - -1. Add `system:serviceaccount::gitlab-ce-user` to the `users` section. - If you changed the Application Name from the default the user will - will be `-user` instead of `gitlab-ce-user` - -1. Save and exit the editor - -## Conclusion - -By now, you should have an understanding of the basic OpenShift Origin concepts -and a sense of how things work using the web console or the CLI. - -GitLab was hard to install in previous versions of OpenShift, -but now that belongs to the past. Upload a template, create a project, add an -application and you are done. You are ready to login to your new GitLab instance. - -And remember that in this tutorial we just scratched the surface of what Origin -is capable of. As always, you can refer to the detailed -[documentation][openshift-docs] to learn more about deploying your own OpenShift -PaaS and managing your applications with the ease of containers. - -[RedHat]: https://www.redhat.com/en "RedHat website" -[openshift]: https://www.openshift.org "OpenShift Origin website" -[vm]: https://www.openshift.org/vm/ "OpenShift All-in-one VM" -[vm-new]: https://atlas.hashicorp.com/openshift/boxes/origin-all-in-one "Official OpenShift Vagrant box on Atlas" -[template]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/docker/openshift-template.json "OpenShift template for GitLab" -[openshift.com]: https://openshift.com "OpenShift Online" -[kubernetes]: http://kubernetes.io/ "Kubernetes website" -[Docker]: https://www.docker.com "Docker website" -[oc]: https://docs.openshift.org/latest/cli_reference/get_started_cli.html "Documentation - oc CLI documentation" -[VirtualBox]: https://www.virtualbox.org/wiki/Downloads "VirtualBox downloads" -[Vagrant]: https://www.vagrantup.com/downloads.html "Vagrant downloads" -[projects]: https://docs.openshift.org/latest/dev_guide/projects.html "Documentation - Projects overview" -[core]: https://docs.openshift.org/latest/architecture/core_concepts/index.html "Documentation - Core concepts of OpenShift Origin" -[templates]: https://docs.openshift.org/latest/architecture/core_concepts/templates.html "Documentation - OpenShift templates" -[old-post]: https://blog.openshift.com/deploy-gitlab-openshift/ "Old post - Deploy GitLab on OpenShift" -[line]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/658c065c8d022ce858dd63eaeeadb0b2ddc8deea/docker/openshift-template.json#L239 "GitLab - OpenShift template" -[oc-gh]: https://github.com/openshift/origin/releases/tag/v1.3.0 "Openshift 1.3.0 release on GitHub" -[ha]: http://docs.gitlab.com/ce/administration/high_availability/gitlab.html "Documentation - GitLab High Availability" -[replicas]: https://docs.openshift.org/latest/architecture/core_concepts/deployments.html#replication-controllers "Documentation - Replication controller" -[autoscaling]: https://docs.openshift.org/latest/dev_guide/pod_autoscaling.html "Documentation - Autoscale" -[basic-cli]: https://docs.openshift.org/latest/cli_reference/basic_cli_operations.html "Documentation - Basic CLI operations" -[openshift-docs]: https://docs.openshift.org "OpenShift documentation" +This document was moved to [another location](../../install/openshift_and_gitlab/index.html). diff --git a/doc/install/README.md b/doc/install/README.md index 540cb0d3f38..43197351db3 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -27,7 +27,7 @@ the hardware requirements. - [Installing in Kubernetes](kubernetes/index.md) - Install GitLab into a Kubernetes Cluster using our official Helm Chart Repository. -- [Install GitLab on OpenShift](../articles/openshift_and_gitlab/index.md) +- [Install GitLab on OpenShift](openshift_and_gitlab/index.md) - [Install GitLab on DC/OS](https://mesosphere.com/blog/gitlab-dcos/) via [GitLab-Mesosphere integration](https://about.gitlab.com/2016/09/16/announcing-gitlab-and-mesosphere/) - [Install GitLab on Azure](azure/index.md) - [Install GitLab on Google Cloud Platform](google_cloud_platform/index.md) diff --git a/doc/install/openshift_and_gitlab/img/add-gitlab-to-project.png b/doc/install/openshift_and_gitlab/img/add-gitlab-to-project.png new file mode 100644 index 00000000000..fcad4e59ae3 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/add-gitlab-to-project.png differ diff --git a/doc/install/openshift_and_gitlab/img/add-to-project.png b/doc/install/openshift_and_gitlab/img/add-to-project.png new file mode 100644 index 00000000000..bd915a229f6 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/add-to-project.png differ diff --git a/doc/install/openshift_and_gitlab/img/create-project-ui.png b/doc/install/openshift_and_gitlab/img/create-project-ui.png new file mode 100644 index 00000000000..e72866f252a Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/create-project-ui.png differ diff --git a/doc/install/openshift_and_gitlab/img/gitlab-logs.png b/doc/install/openshift_and_gitlab/img/gitlab-logs.png new file mode 100644 index 00000000000..1e24080c7df Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/gitlab-logs.png differ diff --git a/doc/install/openshift_and_gitlab/img/gitlab-overview.png b/doc/install/openshift_and_gitlab/img/gitlab-overview.png new file mode 100644 index 00000000000..3c5df0ea101 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/gitlab-overview.png differ diff --git a/doc/install/openshift_and_gitlab/img/gitlab-running.png b/doc/install/openshift_and_gitlab/img/gitlab-running.png new file mode 100644 index 00000000000..c7db691cb30 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/gitlab-running.png differ diff --git a/doc/install/openshift_and_gitlab/img/gitlab-scale.png b/doc/install/openshift_and_gitlab/img/gitlab-scale.png new file mode 100644 index 00000000000..4903c7d7498 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/gitlab-scale.png differ diff --git a/doc/install/openshift_and_gitlab/img/gitlab-settings.png b/doc/install/openshift_and_gitlab/img/gitlab-settings.png new file mode 100644 index 00000000000..db4360ffef0 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/gitlab-settings.png differ diff --git a/doc/install/openshift_and_gitlab/img/no-resources.png b/doc/install/openshift_and_gitlab/img/no-resources.png new file mode 100644 index 00000000000..480fb766468 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/no-resources.png differ diff --git a/doc/install/openshift_and_gitlab/img/openshift-infra-project.png b/doc/install/openshift_and_gitlab/img/openshift-infra-project.png new file mode 100644 index 00000000000..8b9f85aa341 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/openshift-infra-project.png differ diff --git a/doc/install/openshift_and_gitlab/img/pods-overview.png b/doc/install/openshift_and_gitlab/img/pods-overview.png new file mode 100644 index 00000000000..e1cf08bd217 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/pods-overview.png differ diff --git a/doc/install/openshift_and_gitlab/img/rc-name.png b/doc/install/openshift_and_gitlab/img/rc-name.png new file mode 100644 index 00000000000..889e34adbec Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/rc-name.png differ diff --git a/doc/install/openshift_and_gitlab/img/running-pods.png b/doc/install/openshift_and_gitlab/img/running-pods.png new file mode 100644 index 00000000000..3fd4e56662f Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/running-pods.png differ diff --git a/doc/install/openshift_and_gitlab/img/storage-volumes.png b/doc/install/openshift_and_gitlab/img/storage-volumes.png new file mode 100644 index 00000000000..ae1e5381faa Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/storage-volumes.png differ diff --git a/doc/install/openshift_and_gitlab/img/web-console.png b/doc/install/openshift_and_gitlab/img/web-console.png new file mode 100644 index 00000000000..aa1425d4f94 Binary files /dev/null and b/doc/install/openshift_and_gitlab/img/web-console.png differ diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md new file mode 100644 index 00000000000..8fba44aea02 --- /dev/null +++ b/doc/install/openshift_and_gitlab/index.md @@ -0,0 +1,510 @@ +# Getting started with OpenShift Origin 3 and GitLab + +> **[Article Type](../../development/writing_documentation.html#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) || +> **Publication date:** 2016-06-28 + +## Introduction + +[OpenShift Origin][openshift] is an open source container application +platform created by [RedHat], based on [kubernetes] and [Docker]. That means +you can host your own PaaS for free and almost with no hassle. + +In this tutorial, we will see how to deploy GitLab in OpenShift using GitLab's +official Docker image while getting familiar with the web interface and CLI +tools that will help us achieve our goal. + +--- + +## Prerequisites + +OpenShift 3 is not yet deployed on RedHat's offered Online platform ([openshift.com]), +so in order to test it, we will use an [all-in-one Virtualbox image][vm] that is +offered by the OpenShift developers and managed by Vagrant. If you haven't done +already, go ahead and install the following components as they are essential to +test OpenShift easily: + +- [VirtualBox] +- [Vagrant] +- [OpenShift Client][oc] (`oc` for short) + +It is also important to mention that for the purposes of this tutorial, the +latest Origin release is used: + +- **oc** `v1.3.0` (must be [installed][oc-gh] locally on your computer) +- **openshift** `v1.3.0` (is pre-installed in the [VM image][vm-new]) +- **kubernetes** `v1.3.0` (is pre-installed in the [VM image][vm-new]) + +>**Note:** +If you intend to deploy GitLab on a production OpenShift cluster, there are some +limitations to bare in mind. Read on the [limitations](#current-limitations) +section for more information and follow the linked links for the relevant +discussions. + +Now that you have all batteries, let's see how easy it is to test OpenShift +on your computer. + +## Getting familiar with OpenShift Origin + +The environment we are about to use is based on CentOS 7 which comes with all +the tools needed pre-installed: Docker, kubernetes, OpenShift, etcd. + +### Test OpenShift using Vagrant + +As of this writing, the all-in-one VM is at version 1.3, and that's +what we will use in this tutorial. + +In short: + +1. Open a terminal and in a new directory run: + ```sh + vagrant init openshift/origin-all-in-one + ``` +1. This will generate a Vagrantfile based on the all-in-one VM image +1. In the same directory where you generated the Vagrantfile + enter: + + ```sh + vagrant up + ``` + +This will download the VirtualBox image and fire up the VM with some preconfigured +values as you can see in the Vagrantfile. As you may have noticed, you need +plenty of RAM (5GB in our example), so make sure you have enough. + +Now that OpenShift is setup, let's see how the web console looks like. + +### Explore the OpenShift web console + +Once Vagrant finishes its thing with the VM, you will be presented with a +message which has some important information. One of them is the IP address +of the deployed OpenShift platform and in particular . +Open this link with your browser and accept the self-signed certificate in +order to proceed. + +Let's login as admin with username/password `admin/admin`. This is what the +landing page looks like: + +![openshift web console](img/web-console.png) + +You can see that a number of [projects] are already created for testing purposes. + +If you head over the `openshift-infra` project, a number of services with their +respective pods are there to explore. + +![openshift web console](img/openshift-infra-project.png) + +We are not going to explore the whole interface, but if you want to learn about +the key concepts of OpenShift, read the [core concepts reference][core] in the +official documentation. + +### Explore the OpenShift CLI + +OpenShift Client (`oc`), is a powerful CLI tool that talks to the OpenShift API +and performs pretty much everything you can do from the web UI and much more. + +Assuming you have [installed][oc] it, let's explore some of its main +functionalities. + +Let's first see the version of `oc`: + +```sh +$ oc version + +oc v1.3.0 +kubernetes v1.3.0+52492b4 +``` + +With `oc help` you can see the top level arguments you can run with `oc` and +interact with your cluster, kubernetes, run applications, create projects and +much more. + +Let's login to the all-in-one VM and see how to achieve the same results like +when we visited the web console earlier. The username/password for the +administrator user is `admin/admin`. There is also a test user with username/ +password `user/user`, with limited access. Let's login as admin for the moment: + +```sh +$ oc login https://10.2.2.2:8443 + +Authentication required for https://10.2.2.2:8443 (openshift) +Username: admin +Password: +Login successful. + +You have access to the following projects and can switch between them with 'oc project ': + + * cockpit + * default (current) + * delete + * openshift + * openshift-infra + * sample + +Using project "default". +``` + +Switch to the `openshift-infra` project with: + +```sh +oc project openshift-infra +``` + +And finally, see its status: + +```sh +oc status +``` + +The last command should spit a bunch of information about the statuses of the +pods and the services, which if you look closely is what we encountered in the +second image when we explored the web console. + +You can always read more about `oc` in the [OpenShift CLI documentation][oc]. + +### Troubleshooting the all-in-one VM + +Using the all-in-one VM gives you the ability to test OpenShift whenever you +want. That means you get to play with it, shutdown the VM, and pick up where +you left off. + +Sometimes though, you may encounter some issues, like OpenShift not running +when booting up the VM. The web UI may not responding or you may see issues +when trying to login with `oc`, like: + +``` +The connection to the server 10.2.2.2:8443 was refused - did you specify the right host or port? +``` + +In that case, the OpenShift service might not be running, so in order to fix it: + +1. SSH into the VM by going to the directory where the Vagrantfile is and then + run: + + ```sh + vagrant ssh + ``` + +1. Run `systemctl` and verify by the output that the `openshift` service is not + running (it will be in red color). If that's the case start the service with: + + ```sh + sudo systemctl start openshift + ``` + +1. Verify the service is up with: + + ```sh + systemctl status openshift -l + ``` + +Now you will be able to login using `oc` (like we did before) and visit the web +console. + +## Deploy GitLab + +Now that you got a taste of what OpenShift looks like, let's deploy GitLab! + +### Create a new project + +First, we will create a new project to host our application. You can do this +either by running the CLI client: + +```bash +$ oc new-project gitlab +``` + +or by using the web interface: + +![Create a new project from the UI](img/create-project-ui.png) + +If you used the command line, `oc` automatically uses the new project and you +can see its status with: + +```sh +$ oc status + +In project gitlab on server https://10.2.2.2:8443 + +You have no services, deployment configs, or build configs. +Run 'oc new-app' to create an application. +``` + +If you visit the web console, you can now see `gitlab` listed in the projects list. + +The next step is to import the OpenShift template for GitLab. + +### Import the template + +The [template][templates] is basically a JSON file which describes a set of +related object definitions to be created together, as well as a set of +parameters for those objects. + +The template for GitLab resides in the Omnibus GitLab repository under the +docker directory. Let's download it locally with `wget`: + +```bash +wget https://gitlab.com/gitlab-org/omnibus-gitlab/raw/master/docker/openshift-template.json +``` + +And then let's import it in OpenShift: + +```bash +oc create -f openshift-template.json -n openshift +``` + +>**Note:** +The `-n openshift` namespace flag is a trick to make the template available to all +projects. If you recall from when we created the `gitlab` project, `oc` switched +to it automatically, and that can be verified by the `oc status` command. If +you omit the namespace flag, the application will be available only to the +current project, in our case `gitlab`. The `openshift` namespace is a global +one that the administrators should use if they want the application to be +available to all users. + +We are now ready to finally deploy GitLab! + +### Create a new application + +The next step is to use the template we previously imported. Head over to the +`gitlab` project and hit the **Add to Project** button. + +![Add to project](img/add-to-project.png) + +This will bring you to the catalog where you can find all the pre-defined +applications ready to deploy with the click of a button. Search for `gitlab` +and you will see the previously imported template: + +![Add GitLab to project](img/add-gitlab-to-project.png) + +Select it, and in the following screen you will be presented with the predefined +values used with the GitLab template: + +![GitLab settings](img/gitlab-settings.png) + +Notice at the top that there are three resources to be created with this +template: + +- `gitlab-ce` +- `gitlab-ce-redis` +- `gitlab-ce-postgresql` + +While PostgreSQL and Redis are bundled in Omnibus GitLab, the template is using +separate images as you can see from [this line][line] in the template. + +The predefined values have been calculated for the purposes of testing out +GitLab in the all-in-one VM. You don't need to change anything here, hit +**Create** to start the deployment. + +If you are deploying to production you will want to change the **GitLab instance +hostname** and use greater values for the volume sizes. If you don't provide a +password for PostgreSQL, it will be created automatically. + +>**Note:** +The `gitlab.apps.10.2.2.2.xip.io` hostname that is used by default will +resolve to the host with IP `10.2.2.2` which is the IP our VM uses. It is a +trick to have distinct FQDNs pointing to services that are on our local network. +Read more on how this works in . + +Now that we configured this, let's see how to manage and scale GitLab. + +## Manage and scale GitLab + +Setting up GitLab for the first time might take a while depending on your +internet connection and the resources you have attached to the all-in-one VM. +GitLab's docker image is quite big (~500MB), so you'll have to wait until +it's downloaded and configured before you use it. + +### Watch while GitLab gets deployed + +Navigate to the `gitlab` project at **Overview**. You can notice that the +deployment is in progress by the orange color. The Docker images are being +downloaded and soon they will be up and running. + +![GitLab overview](img/gitlab-overview.png) + +Switch to the **Browse > Pods** and you will eventually see all 3 pods in a +running status. Remember the 3 resources that were to be created when we first +created the GitLab app? This is where you can see them in action. + +![Running pods](img/running-pods.png) + +You can see GitLab being reconfigured by taking look at the logs in realtime. +Click on `gitlab-ce-2-j7ioe` (your ID will be different) and go to the **Logs** +tab. + +![GitLab logs](img/gitlab-logs.png) + +At a point you should see a _**gitlab Reconfigured!**_ message in the logs. +Navigate back to the **Overview** and hopefully all pods will be up and running. + +![GitLab running](img/gitlab-running.png) + +Congratulations! You can now navigate to your new shinny GitLab instance by +visiting where you will be asked to +change the root user password. Login using `root` as username and providing the +password you just set, and start using GitLab! + +### Scale GitLab with the push of a button + +If you reach to a point where your GitLab instance could benefit from a boost +of resources, you'd be happy to know that you can scale up with the push of a +button. + +In the **Overview** page just click the up arrow button in the pod where +GitLab is. The change is instant and you can see the number of [replicas] now +running scaled to 2. + +![GitLab scale](img/gitlab-scale.png) + +Upping the GitLab pods is actually like adding new application servers to your +cluster. You can see how that would work if you didn't use GitLab with +OpenShift by following the [HA documentation][ha] for the application servers. + +Bare in mind that you may need more resources (CPU, RAM, disk space) when you +scale up. If a pod is in pending state for too long, you can navigate to +**Browse > Events** and see the reason and message of the state. + +![No resources](img/no-resources.png) + +### Scale GitLab using the `oc` CLI + +Using `oc` is super easy to scale up the replicas of a pod. You may want to +skim through the [basic CLI operations][basic-cli] to get a taste how the CLI +commands are used. Pay extra attention to the object types as we will use some +of them and their abbreviated versions below. + +In order to scale up, we need to find out the name of the replication controller. +Let's see how to do that using the following steps. + +1. Make sure you are in the `gitlab` project: + + ```sh + oc project gitlab + ``` + +1. See what services are used for this project: + + ```sh + oc get svc + ``` + + The output will be similar to: + + ``` + NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE + gitlab-ce 172.30.243.177 22/TCP,80/TCP 5d + gitlab-ce-postgresql 172.30.116.75 5432/TCP 5d + gitlab-ce-redis 172.30.105.88 6379/TCP 5d + ``` + +1. We need to see the replication controllers of the `gitlab-ce` service. + Get a detailed view of the current ones: + + ```sh + oc describe rc gitlab-ce + ``` + + This will return a large detailed list of the current replication controllers. + Search for the name of the GitLab controller, usually `gitlab-ce-1` or if + that failed at some point and you spawned another one, it will be named + `gitlab-ce-2`. + +1. Scale GitLab using the previous information: + + ```sh + oc scale --replicas=2 replicationcontrollers gitlab-ce-2 + ``` + +1. Get the new replicas number to make sure scaling worked: + + ```sh + oc get rc gitlab-ce-2 + ``` + + which will return something like: + + ``` + NAME DESIRED CURRENT AGE + gitlab-ce-2 2 2 5d + ``` + +And that's it! We successfully scaled the replicas to 2 using the CLI. + +As always, you can find the name of the controller using the web console. Just +click on the service you are interested in and you will see the details in the +right sidebar. + +![Replication controller name](img/rc-name.png) + +### Autoscaling GitLab + +In case you were wondering whether there is an option to autoscale a pod based +on the resources of your server, the answer is yes, of course there is. + +We will not expand on this matter, but feel free to read the documentation on +OpenShift's website about [autoscaling]. + +## Current limitations + +As stated in the [all-in-one VM][vm] page: + +> By default, OpenShift will not allow a container to run as root or even a +non-random container assigned userid. Most Docker images in the Dockerhub do not +follow this best practice and instead run as root. + +The all-in-one VM we are using has this security turned off so it will not +bother us. In any case, it is something to keep in mind when deploying GitLab +on a production cluster. + +In order to deploy GitLab on a production cluster, you will need to assign the +GitLab service account to the `anyuid` Security Context. + +1. Edit the Security Context: + ```sh + oc edit scc anyuid + ``` + +1. Add `system:serviceaccount::gitlab-ce-user` to the `users` section. + If you changed the Application Name from the default the user will + will be `-user` instead of `gitlab-ce-user` + +1. Save and exit the editor + +## Conclusion + +By now, you should have an understanding of the basic OpenShift Origin concepts +and a sense of how things work using the web console or the CLI. + +GitLab was hard to install in previous versions of OpenShift, +but now that belongs to the past. Upload a template, create a project, add an +application and you are done. You are ready to login to your new GitLab instance. + +And remember that in this tutorial we just scratched the surface of what Origin +is capable of. As always, you can refer to the detailed +[documentation][openshift-docs] to learn more about deploying your own OpenShift +PaaS and managing your applications with the ease of containers. + +[RedHat]: https://www.redhat.com/en "RedHat website" +[openshift]: https://www.openshift.org "OpenShift Origin website" +[vm]: https://www.openshift.org/vm/ "OpenShift All-in-one VM" +[vm-new]: https://atlas.hashicorp.com/openshift/boxes/origin-all-in-one "Official OpenShift Vagrant box on Atlas" +[template]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/docker/openshift-template.json "OpenShift template for GitLab" +[openshift.com]: https://openshift.com "OpenShift Online" +[kubernetes]: http://kubernetes.io/ "Kubernetes website" +[Docker]: https://www.docker.com "Docker website" +[oc]: https://docs.openshift.org/latest/cli_reference/get_started_cli.html "Documentation - oc CLI documentation" +[VirtualBox]: https://www.virtualbox.org/wiki/Downloads "VirtualBox downloads" +[Vagrant]: https://www.vagrantup.com/downloads.html "Vagrant downloads" +[projects]: https://docs.openshift.org/latest/dev_guide/projects.html "Documentation - Projects overview" +[core]: https://docs.openshift.org/latest/architecture/core_concepts/index.html "Documentation - Core concepts of OpenShift Origin" +[templates]: https://docs.openshift.org/latest/architecture/core_concepts/templates.html "Documentation - OpenShift templates" +[old-post]: https://blog.openshift.com/deploy-gitlab-openshift/ "Old post - Deploy GitLab on OpenShift" +[line]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/658c065c8d022ce858dd63eaeeadb0b2ddc8deea/docker/openshift-template.json#L239 "GitLab - OpenShift template" +[oc-gh]: https://github.com/openshift/origin/releases/tag/v1.3.0 "Openshift 1.3.0 release on GitHub" +[ha]: ../../administration/high_availability/gitlab.html "Documentation - GitLab High Availability" +[replicas]: https://docs.openshift.org/latest/architecture/core_concepts/deployments.html#replication-controllers "Documentation - Replication controller" +[autoscaling]: https://docs.openshift.org/latest/dev_guide/pod_autoscaling.html "Documentation - Autoscale" +[basic-cli]: https://docs.openshift.org/latest/cli_reference/basic_cli_operations.html "Documentation - Basic CLI operations" +[openshift-docs]: https://docs.openshift.org "OpenShift documentation" -- cgit v1.2.1 From 01492fd06058cd580e236194285850787ff76790 Mon Sep 17 00:00:00 2001 From: Brendan O'Leary Date: Wed, 10 Jan 2018 15:34:08 -0500 Subject: Port gitlab-ee MR 3981 to CE --- doc/university/high-availability/aws/README.md | 4 ++++ .../high-availability/aws/img/reference-arch.png | Bin 0 -> 183997 bytes 2 files changed, 4 insertions(+) create mode 100644 doc/university/high-availability/aws/img/reference-arch.png (limited to 'doc') diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 54625996dff..ddc853afded 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -26,6 +26,10 @@ Login to your AWS account through the `My Account` dropdown on Amazon Web Services console from where we can choose all of the services we'll be using to configure our cloud infrastructure. +### Reference Architecture + +![Reference Architecture](img/reference-arch.png) + *** ## Network diff --git a/doc/university/high-availability/aws/img/reference-arch.png b/doc/university/high-availability/aws/img/reference-arch.png new file mode 100644 index 00000000000..271ee5bc614 Binary files /dev/null and b/doc/university/high-availability/aws/img/reference-arch.png differ -- cgit v1.2.1 From 778b231f3a5dd42ebe195d4719a26bf675093350 Mon Sep 17 00:00:00 2001 From: Victor Wu Date: Wed, 10 Jan 2018 20:42:16 +0000 Subject: Update Redmine docs to let user know that `new_issue_url` is not being used. --- doc/user/project/integrations/redmine.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) (limited to 'doc') diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index f530b6cb649..cc3218fbfd1 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -10,12 +10,7 @@ in the table below. | `description` | A name for the issue tracker (to differentiate between instances, for example) | | `project_url` | The URL to the project in Redmine which is being linked to this GitLab project | | `issues_url` | The URL to the issue in Redmine project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | - | `new_issue_url` | This is the URL to create a new issue in Redmine for the project linked to this GitLab project | - - Once you have configured and enabled Redmine: - - the **Issues** link on the GitLab project pages takes you to the appropriate - Redmine issue index - - clicking **New issue** on the project dashboard creates a new Redmine issue + | `new_issue_url` | This is the URL to create a new issue in Redmine for the project linked to this GitLab project. **This is currently not being used and will be removed in a future release.** | As an example, below is a configuration for a project named gitlab-ci. -- cgit v1.2.1 From 6ef28ace53abb12e1475746882237cecf8f95d7e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matija=20=C4=8Cupi=C4=87?= Date: Thu, 11 Jan 2018 05:31:55 +0100 Subject: Add API requirements to docs --- doc/user/project/clusters/index.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index d5619c7b563..218b9dee41d 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -25,11 +25,14 @@ prerequisites must be met: be enabled in GitLab at the instance level. If that's not the case, ask your administrator to enable it. - Your associated Google account must have the right privileges to manage - clusters on GKE. That would mean that a - [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account) - must be set up. -- You must have Master [permissions] in order to be able to access the **Cluster** - page. + clusters on GKE. That would mean that a [billing + account](https://cloud.google.com/billing/docs/how-to/manage-billing-account) + must be set up and that you have to have permissions to access it. +- You must have Master [permissions] in order to be able to access the + **Cluster** page. +- You must have [Cloud Billing API](https://cloud.google.com/billing/) enabled +- You must have [Resource Manager + API](https://cloud.google.com/resource-manager/) If all of the above requirements are met, you can proceed to add a new GKE cluster. -- cgit v1.2.1 From 3545ceb02c08b71422b51209a2cca21155236cfc Mon Sep 17 00:00:00 2001 From: Sascha Szott Date: Thu, 11 Jan 2018 08:04:50 +0000 Subject: apply documentation changes from EE to CE --- doc/api/snippets.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/api/snippets.md b/doc/api/snippets.md index fdafbfb5b9e..e57143e4215 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -84,7 +84,11 @@ Parameters: ``` bash -curl --request POST --data '{"title": "This is a snippet", "content": "Hello world", "description": "Hello World snippet", "file_name": "test.txt", "visibility": "internal" }' --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/snippets +curl --request POST \ + --data '{"title": "This is a snippet", "content": "Hello world", "description": "Hello World snippet", "file_name": "test.txt", "visibility": "internal" }' \ + --header 'Content-Type: application/json' \ + --header "PRIVATE-TOKEN: valid_api_token" \ + https://gitlab.example.com/api/v4/snippets ``` Example response: @@ -131,7 +135,11 @@ Parameters: ``` bash -curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data '{"title": "foo", "content": "bar"}' https://gitlab.example.com/api/v4/snippets/1 +curl --request PUT \ + --data '{"title": "foo", "content": "bar"}' \ + --header 'Content-Type: application/json' \ + --header "PRIVATE-TOKEN: valid_api_token" \ + https://gitlab.example.com/api/v4/snippets/1 ``` Example response: -- cgit v1.2.1 From d48df55177533922ad17913995203fb57c00289a Mon Sep 17 00:00:00 2001 From: Mark Fletcher Date: Thu, 11 Jan 2018 11:23:42 +0000 Subject: Fix external links in Snippets API documentation --- doc/api/snippets.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/api/snippets.md b/doc/api/snippets.md index fdafbfb5b9e..b6a2833bb42 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -2,7 +2,7 @@ > [Introduced][ce-6373] in GitLab 8.15. -### Snippet visibility level +## Snippet visibility level Snippets in GitLab can be either private, internal, or public. You can set it with the `visibility` field in the snippet. @@ -265,4 +265,5 @@ Example response: } ``` -[ce-[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508 +[ce-6373]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6373 +[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12655 -- cgit v1.2.1 From 0cc25f0f1b5e672e76fa93be8bf5ec553f10699a Mon Sep 17 00:00:00 2001 From: Andrew Newdigate Date: Thu, 11 Jan 2018 12:05:55 +0000 Subject: Update end_to_end_tests.md --- doc/development/testing_guide/end_to_end_tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md index abe5b06e0f0..5b4f6511f04 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -25,7 +25,7 @@ It is possible to run end-to-end tests (eventually being run within a the `package-qa` manual action, that should be present in a merge request widget. -Mmanual action that starts end-to-end tests is also available in merge requests +Manual action that starts end-to-end tests is also available in merge requests in Omnibus GitLab project. Below you can read more about how to use it and how does it work. -- cgit v1.2.1 From 3b6fb6b8679abc7915be669bd937cb0178721c8a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Cindy=20Pallares=20=F0=9F=A6=89?= Date: Fri, 29 Dec 2017 17:39:27 +0000 Subject: Recommend against the use of EFS --- doc/administration/high_availability/nfs.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index e09ccaba08c..d8928a7fe4c 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -32,7 +32,9 @@ options: ## AWS Elastic File System -GitLab does not recommend using AWS Elastic File System (EFS). +GitLab strongly recommends against using AWS Elastic File System (EFS). +Our support team will not be able to assist on performance issues related to +file system access. Customers and users have reported that AWS EFS does not perform well for GitLab's use-case. There are several issues that can cause problems. For these reasons -- cgit v1.2.1 From 7a01d667ce406d880225259bfc8595e7add836d9 Mon Sep 17 00:00:00 2001 From: Marcia Ramos Date: Thu, 11 Jan 2018 17:12:02 +0000 Subject: Docs: move article "Laravel and Envoy w/ CI/CD" --- doc/articles/index.md | 1 - .../img/container_registry_checkbox.png | Bin 4730 -> 0 bytes .../img/container_registry_page_empty_image.png | Bin 56091 -> 0 bytes .../img/container_registry_page_with_image.jpg | Bin 93531 -> 0 bytes .../img/deploy_keys_page.png | Bin 339666 -> 0 bytes .../img/environment_page.png | Bin 185393 -> 0 bytes .../img/environments_page.png | Bin 134742 -> 0 bytes .../img/laravel_welcome_page.png | Bin 5785 -> 0 bytes .../img/laravel_with_gitlab_and_envoy.png | Bin 177704 -> 0 bytes .../img/pipeline_page.png | Bin 172664 -> 0 bytes .../img/pipelines_page.png | Bin 119955 -> 0 bytes .../img/pipelines_page_deploy_button.png | Bin 141393 -> 0 bytes .../img/production_server_app_directory.png | Bin 11082 -> 0 bytes .../img/production_server_current_directory.png | Bin 21993 -> 0 bytes .../img/secret_variables_page.png | Bin 233764 -> 0 bytes .../laravel_with_gitlab_and_envoy/index.md | 681 +------------------- doc/ci/README.md | 2 +- doc/ci/examples/README.md | 1 + .../img/container_registry_checkbox.png | Bin 0 -> 4730 bytes .../img/container_registry_page_empty_image.png | Bin 0 -> 56091 bytes .../img/container_registry_page_with_image.jpg | Bin 0 -> 93531 bytes .../img/deploy_keys_page.png | Bin 0 -> 339666 bytes .../img/environment_page.png | Bin 0 -> 185393 bytes .../img/environments_page.png | Bin 0 -> 134742 bytes .../img/laravel_welcome_page.png | Bin 0 -> 5785 bytes .../img/laravel_with_gitlab_and_envoy.png | Bin 0 -> 177704 bytes .../img/pipeline_page.png | Bin 0 -> 172664 bytes .../img/pipelines_page.png | Bin 0 -> 119955 bytes .../img/pipelines_page_deploy_button.png | Bin 0 -> 141393 bytes .../img/production_server_app_directory.png | Bin 0 -> 11082 bytes .../img/production_server_current_directory.png | Bin 0 -> 21993 bytes .../img/secret_variables_page.png | Bin 0 -> 233764 bytes .../laravel_with_gitlab_and_envoy/index.md | 684 +++++++++++++++++++++ 33 files changed, 687 insertions(+), 682 deletions(-) delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png delete mode 100644 doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/environment_page.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/environments_page.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipeline_page.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/img/secret_variables_page.png create mode 100644 doc/ci/examples/laravel_with_gitlab_and_envoy/index.md (limited to 'doc') diff --git a/doc/articles/index.md b/doc/articles/index.md index d8a7874b96d..41fe8005d1f 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -17,7 +17,6 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM | Article title | Category | Publishing date | | :------------ | :------: | --------------: | | [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Admin guide | 2017-11-24 | -| [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 | | [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 | | [Dockerizing GitLab Review Apps](https://about.gitlab.com/2017/07/11/dockerizing-review-apps/) | Concepts | 2017-07-11 | | [Continuous Integration: From Jenkins to GitLab Using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/) | Concepts | 2017-07-27 | diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png deleted file mode 100644 index a56c07a0da7..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png deleted file mode 100644 index b1406fed6b8..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg b/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg deleted file mode 100644 index d1f0cbc08ab..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png deleted file mode 100644 index 9aae11b8679..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png deleted file mode 100644 index a06b6d417cd..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png deleted file mode 100644 index d357ecda7d2..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png deleted file mode 100644 index 3bb21fd12b4..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png b/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png deleted file mode 100644 index bc188f83fb1..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png deleted file mode 100644 index baf8dec499c..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png deleted file mode 100644 index d96c43bcf16..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png b/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png deleted file mode 100644 index 997db10189f..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png b/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png deleted file mode 100644 index 6dbc29fc25c..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png b/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png deleted file mode 100644 index 8a6dcccfa38..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png b/doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png deleted file mode 100644 index 658c0b5bcac..00000000000 Binary files a/doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png and /dev/null differ diff --git a/doc/articles/laravel_with_gitlab_and_envoy/index.md b/doc/articles/laravel_with_gitlab_and_envoy/index.md index b20bd8c247a..b092cdb0f7a 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/index.md +++ b/doc/articles/laravel_with_gitlab_and_envoy/index.md @@ -1,680 +1 @@ -# Test and deploy Laravel applications with GitLab CI/CD and Envoy - -> **[Article Type](../../development/writing_documentation.md#types-of-technical-articles):** tutorial || -> **Level:** intermediary || -> **Author:** [Mehran Rasulian](https://gitlab.com/mehranrasulian) || -> **Publication date:** 2017-08-31 - -## Introduction - -GitLab features our applications with Continuous Integration, and it is possible to easily deploy the new code changes to the production server whenever we want. - -In this tutorial, we'll show you how to initialize a [Laravel](http://laravel.com/) application and setup our [Envoy](https://laravel.com/docs/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../../ci/README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/). - -We assume you have a basic experience with Laravel, Linux servers, -and you know how to use GitLab. - -Laravel is a high quality web framework written in PHP. -It has a great community with a [fantastic documentation](https://laravel.com/docs). -Aside from the usual routing, controllers, requests, responses, views, and (blade) templates, out of the box Laravel provides plenty of additional services such as cache, events, localization, authentication and many others. - -We will use [Envoy](https://laravel.com/docs/master/envoy) as an SSH task runner based on PHP. -It uses a clean, minimal [Blade syntax](https://laravel.com/docs/blade) to setup tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/artisan). - -## Initialize our Laravel app on GitLab - -We assume [you have installed a new laravel project](https://laravel.com/docs/installation#installation), so let's start with a unit test, and initialize Git for the project. - -### Unit Test - -Every new installation of Laravel (currently 5.4) comes with two type of tests, 'Feature' and 'Unit', placed in the tests directory. -Here's a unit test from `test/Unit/ExampleTest.php`: - -```php -assertTrue(true); - } -} -``` - -This test is as simple as asserting that the given value is true. - -Laravel uses `PHPUnit` for tests by default. -If we run `vendor/bin/phpunit` we should see the green output: - -```bash -vendor/bin/phpunit -OK (1 test, 1 assertions) -``` - -This test will be used later for continuously testing our app with GitLab CI/CD. - -### Push to GitLab - -Since we have our app up and running locally, it's time to push the codebase to our remote repository. -Let's create [a new project](../../gitlab-basics/create-project.md) in GitLab named `laravel-sample`. -After that, follow the command line instructions displayed on the project's homepage to initiate the repository on our machine and push the first commit. - - -```bash -cd laravel-sample -git init -git remote add origin git@gitlab.example.com:/laravel-sample.git -git add . -git commit -m 'Initial Commit' -git push -u origin master -``` - -## Configure the production server - -Before we begin setting up Envoy and GitLab CI/CD, let's quickly make sure the production server is ready for deployment. -We have installed LEMP stack which stands for Linux, Nginx, MySQL and PHP on our Ubuntu 16.04. - -### Create a new user - -Let's now create a new user that will be used to deploy our website and give it -the needed permissions using [Linux ACL](https://serversforhackers.com/video/linux-acls): - -```bash -# Create user deployer -sudo adduser deployer -# Give the read-write-execute permissions to deployer user for directory /var/www -sudo setfacl -R -m u:deployer:rwx /var/www -``` - -If you don't have ACL installed on your Ubuntu server, use this command to install it: - -```bash -sudo apt install acl -``` - -### Add SSH key - -Let's suppose we want to deploy our app to the production server from a private repository on GitLab. First, we need to [generate a new SSH key pair **with no passphrase**](../../ssh/README.md) for the deployer user. - -After that, we need to copy the private key, which will be used to connect to our server as the deployer user with SSH, to be able to automate our deployment process: - -```bash -# As the deployer user on server -# -# Copy the content of public key to authorized_keys -cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys -# Copy the private key text block -cat ~/.ssh/id_rsa -``` - -Now, let's add it to your GitLab project as a [secret variable](../../ci/variables/README.md#secret-variables). -Secret variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. -They can be added per project by navigating to the project's **Settings** > **CI/CD**. - -![secret variables page](img/secret_variables_page.png) - -To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. -We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our remote server as the deployer user without entering its password. - -We also need to add the public key to **Project** > **Settings** > **Repository** as [Deploy Keys](../../ssh/README.md/#deploy-keys), which gives us the ability to access our repository from the server through [SSH protocol](../../gitlab-basics/command-line-commands.md/#start-working-on-your-project). - - -```bash -# As the deployer user on the server -# -# Copy the public key -cat ~/.ssh/id_rsa.pub -``` - -![deploy keys page](img/deploy_keys_page.png) - -To the field **Title**, add any name you want, and paste the public key into the **Key** field. - -Now, let's clone our repository on the server just to make sure the `deployer` user has access to the repository. - -```bash -# As the deployer user on server -# -git clone git@gitlab.example.com:/laravel-sample.git -``` - ->**Note:** -Answer **yes** if asked `Are you sure you want to continue connecting (yes/no)?`. -It adds GitLab.com to the known hosts. - -### Configuring Nginx - -Now, let's make sure our web server configuration points to the `current/public` rather than `public`. - -Open the default Nginx server block configuration file by typing: - -```bash -sudo nano /etc/nginx/sites-available/default -``` - -The configuration should be like this. - -``` -server { - root /var/www/app/current/public; - server_name example.com; - # Rest of the configuration -} -``` - ->**Note:** -You may replace the app's name in `/var/www/app/current/public` with the folder name of your application. - -## Setting up Envoy - -So we have our Laravel app ready for production. -The next thing is to use Envoy to perform the deploy. - -To use Envoy, we should first install it on our local machine [using the given instructions by Laravel](https://laravel.com/docs/envoy/#introduction). - -### How Envoy works - -The pros of Envoy is that it doesn't require Blade engine, it just uses Blade syntax to define tasks. -To start, we create an `Envoy.blade.php` in the root of our app with a simple task to test Envoy. - - -```php -@servers(['web' => 'remote_username@remote_host']) - -@task('list', [on => 'web']) - ls -l -@endtask -``` - -As you may expect, we have an array within `@servers` directive at the top of the file, which contains a key named `web` with a value of the server's address (e.g. `deployer@192.168.1.1`). -Then within our `@task` directive we define the bash commands that should be run on the server when the task is executed. - -On the local machine use the `run` command to run Envoy tasks. - -```bash -envoy run list -``` - -It should execute the `list` task we defined earlier, which connects to the server and lists directory contents. - -Envoy is not a dependency of Laravel, therefore you can use it for any PHP application. - -### Zero downtime deployment - -Every time we deploy to the production server, Envoy downloads the latest release of our app from GitLab repository and replace it with preview's release. -Envoy does this without any [downtime](https://en.wikipedia.org/wiki/Downtime), -so we don't have to worry during the deployment while someone might be reviewing the site. -Our deployment plan is to clone the latest release from GitLab repository, install the Composer dependencies and finally, activate the new release. - -#### @setup directive - -The first step of our deployment process is to define a set of variables within [@setup](https://laravel.com/docs/envoy/#setup) directive. -You may change the `app` to your application's name: - - -```php -... - -@setup - $repository = 'git@gitlab.example.com:/laravel-sample.git'; - $releases_dir = '/var/www/app/releases'; - $app_dir = '/var/www/app'; - $release = date('YmdHis'); - $new_release_dir = $releases_dir .'/'. $release; -@endsetup - -... -``` - -- `$repository` is the address of our repository -- `$releases_dir` directory is where we deploy the app -- `$app_dir` is the actual location of the app that is live on the server -- `$release` contains a date, so every time that we deploy a new release of our app, we get a new folder with the current date as name -- `$new_release_dir` is the full path of the new release which is used just to make the tasks cleaner - -#### @story directive - -The [@story](https://laravel.com/docs/envoy/#stories) directive allows us define a list of tasks that can be run as a single task. -Here we have three tasks called `clone_repository`, `run_composer`, `update_symlinks`. These variables are usable to making our task's codes more cleaner: - - -```php -... - -@story('deploy') - clone_repository - run_composer - update_symlinks -@endstory - -... -``` - -Let's create these three tasks one by one. - -#### Clone the repository - -The first task will create the `releases` directory (if it doesn't exist), and then clone the `master` branch of the repository (by default) into the new release directory, given by the `$new_release_dir` variable. -The `releases` directory will hold all our deployments: - -```php -... - -@task('clone_repository') - echo 'Cloning repository' - [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} - git clone --depth 1 {{ $repository }} {{ $new_release_dir }} -@endtask - -... -``` - -While our project grows, its Git history will be very very long over time. -Since we are creating a directory per release, it might not be necessary to have the history of the project downloaded for each release. -The `--depth 1` option is a great solution which saves systems time and disk space as well. - -#### Installing dependencies with Composer - -As you may know, this task just navigates to the new release directory and runs Composer to install the application dependencies: - -```php -... - -@task('run_composer') - echo "Starting deployment ({{ $release }})" - cd {{ $new_release_dir }} - composer install --prefer-dist --no-scripts -q -o -@endtask - -... -``` - -#### Activate new release - -Next thing to do after preparing the requirements of our new release, is to remove the storage directory from it and to create two symbolic links to point the application's `storage` directory and `.env` file to the new release. -Then, we need to create another symbolic link to the new release with the name of `current` placed in the app directory. -The `current` symbolic link always points to the latest release of our app: - -```php -... - -@task('update_symlinks') - echo "Linking storage directory" - rm -rf {{ $new_release_dir }}/storage - ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage - - echo 'Linking .env file' - ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env - - echo 'Linking current release' - ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current -@endtask -``` - -As you see, we use `-nfs` as an option for `ln` command, which says that the `storage`, `.env` and `current` no longer points to the preview's release and will point them to the new release by force (`f` from `-nfs` means force), which is the case when we are doing multiple deployments. - -### Full script - -The script is ready, but make sure to change the `deployer@192.168.1.1` to your server and also change `/var/www/app` with the directory you want to deploy your app. - -At the end, our `Envoy.blade.php` file will look like this: - -```php -@servers(['web' => 'deployer@192.168.1.1']) - -@setup - $repository = 'git@gitlab.example.com:/laravel-sample.git'; - $releases_dir = '/var/www/app/releases'; - $app_dir = '/var/www/app'; - $release = date('YmdHis'); - $new_release_dir = $releases_dir .'/'. $release; -@endsetup - -@story('deploy') - clone_repository - run_composer - update_symlinks -@endstory - -@task('clone_repository') - echo 'Cloning repository' - [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} - git clone --depth 1 {{ $repository }} {{ $new_release_dir }} -@endtask - -@task('run_composer') - echo "Starting deployment ({{ $release }})" - cd {{ $new_release_dir }} - composer install --prefer-dist --no-scripts -q -o -@endtask - -@task('update_symlinks') - echo "Linking storage directory" - rm -rf {{ $new_release_dir }}/storage - ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage - - echo 'Linking .env file' - ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env - - echo 'Linking current release' - ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current -@endtask -``` - -One more thing we should do before any deployment is to manually copy our application `storage` folder to the `/var/www/app` directory on the server for the first time. -You might want to create another Envoy task to do that for you. -We also create the `.env` file in the same path to setup our production environment variables for Laravel. -These are persistent data and will be shared to every new release. - -Now, we would need to deploy our app by running `envoy run deploy`, but it won't be necessary since GitLab can handle that for us with CI's [environments](../../ci/environments.md), which will be described [later](#setting-up-gitlab-ci-cd) in this tutorial. - -Now it's time to commit [Envoy.blade.php](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Envoy.blade.php) and push it to the `master` branch. -To keep things simple, we commit directly to `master`, without using [feature-branches](../../workflow/gitlab_flow.md/#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial. -In a real world project, teams may use [Issue Tracker](../../user/project/issues/index.md) and [Merge Requests](../../user/project/merge_requests/index.md) to move their code across branches: - -```bash -git add Envoy.blade.php -git commit -m 'Add Envoy' -git push origin master -``` - -## Continuous Integration with GitLab - -We have our app ready on GitLab, and we also can deploy it manually. -But let's take a step forward to do it automatically with [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) method. -We need to check every commit with a set of automated tests to become aware of issues at the earliest, and then, we can deploy to the target environment if we are happy with the result of the tests. - -[GitLab CI/CD](../../ci/README.md) allows us to use [Docker](https://docker.com/) engine to handle the process of testing and deploying our app. -In the case you're not familiar with Docker, refer to [How to Automate Docker Deployments](http://paislee.io/how-to-automate-docker-deployments/). - -To be able to build, test, and deploy our app with GitLab CI/CD, we need to prepare our work environment. -To do that, we'll use a Docker image which has the minimum requirements that a Laravel app needs to run. -[There are other ways](../../ci/examples/php.md/#test-php-projects-using-the-docker-executor) to do that as well, but they may lead our builds run slowly, which is not what we want when there are faster options to use. - -With Docker images our builds run incredibly faster! - -### Create a Container Image - -Let's create a [Dockerfile](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Dockerfile) in the root directory of our app with the following content: - -```bash -# Set the base image for subsequent instructions -FROM php:7.1 - -# Update packages -RUN apt-get update - -# Install PHP and composer dependencies -RUN apt-get install -qq git curl libmcrypt-dev libjpeg-dev libpng-dev libfreetype6-dev libbz2-dev - -# Clear out the local repository of retrieved package files -RUN apt-get clean - -# Install needed extensions -# Here you can install any other extension that you need during the test and deployment process -RUN docker-php-ext-install mcrypt pdo_mysql zip - -# Install Composer -RUN curl --silent --show-error https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer - -# Install Laravel Envoy -RUN composer global require "laravel/envoy=~1.0" -``` - -We added the [official PHP 7.1 Docker image](https://hub.docker.com/r/_/php/), which consist of a minimum installation of Debian Jessie with PHP pre-installed, and works perfectly for our use case. - -We used `docker-php-ext-install` (provided by the official PHP Docker image) to install the PHP extensions we need. - -#### Setting Up GitLab Container Registry - -Now that we have our `Dockerfile` let's build and push it to our [GitLab Container Registry](../../user/project/container_registry.md). - -> The registry is the place to store and tag images for later use. Developers may want to maintain their own registry for private, company images, or for throw-away images used only in testing. Using GitLab Container Registry means you don't need to set up and administer yet another service or use a public registry. - -On your GitLab project repository navigate to the **Registry** tab. - -![container registry page empty image](img/container_registry_page_empty_image.png) - -You may need to [enable Container Registry](../../user/project/container_registry.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Sharing and permissions**. - -![container registry checkbox](img/container_registry_checkbox.png) - -To start using Container Registry on our machine, we first need to login to the GitLab registry using our GitLab username and password: - -```bash -docker login registry.gitlab.com -``` -Then we can build and push our image to GitLab: - -```bash -docker build -t registry.gitlab.com//laravel-sample . - -docker push registry.gitlab.com//laravel-sample -``` - ->**Note:** -To run the above commands, we first need to have [Docker](https://docs.docker.com/engine/installation/) installed on our machine. - -Congratulations! You just pushed the first Docker image to the GitLab Registry, and if you refresh the page you should be able to see it: - -![container registry page with image](img/container_registry_page_with_image.jpg) - ->**Note:** -You can also [use GitLab CI/CD](https://about.gitlab.com/2016/05/23/gitlab-container-registry/#use-with-gitlab-ci) to build and push your Docker images, rather than doing that on your machine. - -We'll use this image further down in the `.gitlab-ci.yml` configuration file to handle the process of testing and deploying our app. - -Let's commit the `Dockerfile` file. - -```bash -git add Dockerfile -git commit -m 'Add Dockerfile' -git push origin master -``` - -### Setting up GitLab CI/CD - -In order to build and test our app with GitLab CI/CD, we need a file called `.gitlab-ci.yml` in our repository's root. It is similar to Circle CI and Travis CI, but built-in GitLab. - -Our `.gitlab-ci.yml` file will look like this: - -```yaml -image: registry.gitlab.com//laravel-sample:latest - -services: - - mysql:5.7 - -variables: - MYSQL_DATABASE: homestead - MYSQL_ROOT_PASSWORD: secret - DB_HOST: mysql - DB_USERNAME: root - -stages: - - test - - deploy - -unit_test: - stage: test - script: - - cp .env.example .env - - composer install - - php artisan key:generate - - php artisan migrate - - vendor/bin/phpunit - -deploy_production: - stage: deploy - script: - - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - - eval $(ssh-agent -s) - - ssh-add <(echo "$SSH_PRIVATE_KEY") - - mkdir -p ~/.ssh - - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - - - ~/.composer/vendor/bin/envoy run deploy - environment: - name: production - url: http://192.168.1.1 - when: manual - only: - - master -``` - -That's a lot to take in, isn't it? Let's run through it step by step. - -#### Image and Services - -[GitLab Runners](../../ci/runners/README.md) run the script defined by `.gitlab-ci.yml`. -The `image` keyword tells the Runners which image to use. -The `services` keyword defines additional images [that are linked to the main image](../../ci/docker/using_docker_images.md/#what-is-a-service). -Here we use the container image we created before as our main image and also use MySQL 5.7 as a service. - -```yaml -image: registry.gitlab.com//laravel-sample:latest - -services: - - mysql:5.7 - -... -``` - ->**Note:** -If you wish to test your app with different PHP versions and [database management systems](../../ci/services/README.md), you can define different `image` and `services` keywords for each test job. - -#### Variables - -GitLab CI/CD allows us to use [environment variables](../../ci/yaml/README.md#variables) in our jobs. -We defined MySQL as our database management system, which comes with a superuser root created by default. - -So we should adjust the configuration of MySQL instance by defining `MYSQL_DATABASE` variable as our database name and `MYSQL_ROOT_PASSWORD` variable as the password of `root`. -Find out more about MySQL variables at the [official MySQL Docker Image](https://hub.docker.com/r/_/mysql/). - -Also set the variables `DB_HOST` to `mysql` and `DB_USERNAME` to `root`, which are Laravel specific variables. -We define `DB_HOST` as `mysql` instead of `127.0.0.1`, as we use MySQL Docker image as a service which [is linked to the main Docker image](../../ci/docker/using_docker_images.md/#how-services-are-linked-to-the-build). - -```yaml -... - -variables: - MYSQL_DATABASE: homestead - MYSQL_ROOT_PASSWORD: secret - DB_HOST: mysql - DB_USERNAME: root - -... -``` - -#### Unit Test as the first job - -We defined the required shell scripts as an array of the [script](../../ci/yaml/README.md#script) variable to be executed when running `unit_test` job. - -These scripts are some Artisan commands to prepare the Laravel, and, at the end of the script, we'll run the tests by `PHPUnit`. - -```yaml -... - -unit_test: - script: - # Install app dependencies - - composer install - # Setup .env - - cp .env.example .env - # Generate an environment key - - php artisan key:generate - # Run migrations - - php artisan migrate - # Run tests - - vendor/bin/phpunit - -... -``` - -#### Deploy to production - -The job `deploy_production` will deploy the app to the production server. -To deploy our app with Envoy, we had to set up the `$SSH_PRIVATE_KEY` variable as an [SSH private key](../../ci/ssh_keys/README.md/#ssh-keys-when-using-the-docker-executor). -If the SSH keys have added successfully, we can run Envoy. - -As mentioned before, GitLab supports [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) methods as well. -The [environment](../../ci/yaml/README.md#environment) keyword tells GitLab that this job deploys to the `production` environment. -The `url` keyword is used to generate a link to our application on the GitLab Environments page. -The `only` keyword tells GitLab CI that the job should be executed only when the pipeline is building the `master` branch. -Lastly, `when: manual` is used to turn the job from running automatically to a manual action. - -```yaml -... - -deploy_production: - script: - # Add the private SSH key to the build environment - - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - - eval $(ssh-agent -s) - - ssh-add <(echo "$SSH_PRIVATE_KEY") - - mkdir -p ~/.ssh - - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - - # Run Envoy - - ~/.composer/vendor/bin/envoy run deploy - - environment: - name: production - url: http://192.168.1.1 - when: manual - only: - - master -``` - -You may also want to add another job for [staging environment](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments), to final test your application before deploying to production. - -### Turn on GitLab CI/CD - -We have prepared everything we need to test and deploy our app with GitLab CI/CD. -To do that, commit and push `.gitlab-ci.yml` to the `master` branch. It will trigger a pipeline, which you can watch live under your project's **Pipelines**. - -![pipelines page](img/pipelines_page.png) - -Here we see our **Test** and **Deploy** stages. -The **Test** stage has the `unit_test` build running. -click on it to see the Runner's output. - -![pipeline page](img/pipeline_page.png) - -After our code passed through the pipeline successfully, we can deploy to our production server by clicking the **play** button on the right side. - -![pipelines page deploy button](img/pipelines_page_deploy_button.png) - -Once the deploy pipeline passed successfully, navigate to **Pipelines > Environments**. - -![environments page](img/environments_page.png) - -If something doesn't work as expected, you can roll back to the latest working version of your app. - -![environment page](img/environment_page.png) - -By clicking on the external link icon specified on the right side, GitLab opens the production website. -Our deployment successfully was done and we can see the application is live. - -![laravel welcome page](img/laravel_welcome_page.png) - -In the case that you're interested to know how is the application directory structure on the production server after deployment, here are three directories named `current`, `releases` and `storage`. -As you know, the `current` directory is a symbolic link that points to the latest release. -The `.env` file consists of our Laravel environment variables. - -![production server app directory](img/production_server_app_directory.png) - -If you navigate to the `current` directory, you should see the application's content. -As you see, the `.env` is pointing to the `/var/www/app/.env` file and also `storage` is pointing to the `/var/www/app/storage/` directory. - -![production server current directory](img/production_server_current_directory.png) - -## Conclusion - -We configured GitLab CI to perform automated tests and used the method of [Continuous Delivery](https://continuousdelivery.com/) to deploy to production a Laravel application with Envoy, directly from the codebase. - -Envoy also was a great match to help us deploy the application without writing our custom bash script and doing Linux magics. +This document was moved to [another location](../../ci/examples/laravel_with_gitlab_and_envoy/index.md). diff --git a/doc/ci/README.md b/doc/ci/README.md index 05d792dea0f..5829aaee9c9 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -120,7 +120,7 @@ Here is an collection of tutorials and guides on setting up your CI pipeline. - [Run PHP Composer & NPM scripts then deploy them to a staging server](examples/deployment/composer-npm-deploy.md) - [Analyze code quality with the Code Climate CLI](examples/code_climate.md) - **Articles** - - [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](../articles/laravel_with_gitlab_and_envoy/index.md) + - [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](examples/laravel_with_gitlab_and_envoy/index.md) - [How to deploy Maven projects to Artifactory with GitLab CI/CD](examples/artifactory_and_gitlab/index.md) - [Automated Debian packaging](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) - [Spring boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 25a0c5dcff5..d4590d0f495 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -16,6 +16,7 @@ Apart from those, here is an collection of tutorials and guides on setting up yo - [Testing a PHP application](php.md) - [Run PHP Composer & NPM scripts then deploy them to a staging server](deployment/composer-npm-deploy.md) +- [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) ### Ruby diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png new file mode 100644 index 00000000000..a56c07a0da7 Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png new file mode 100644 index 00000000000..b1406fed6b8 Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg new file mode 100644 index 00000000000..d1f0cbc08ab Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png new file mode 100644 index 00000000000..9aae11b8679 Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environment_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environment_page.png new file mode 100644 index 00000000000..a06b6d417cd Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environment_page.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environments_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environments_page.png new file mode 100644 index 00000000000..d357ecda7d2 Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environments_page.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png new file mode 100644 index 00000000000..3bb21fd12b4 Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png new file mode 100644 index 00000000000..bc188f83fb1 Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipeline_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipeline_page.png new file mode 100644 index 00000000000..baf8dec499c Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipeline_page.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page.png new file mode 100644 index 00000000000..d96c43bcf16 Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png new file mode 100644 index 00000000000..997db10189f Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png new file mode 100644 index 00000000000..6dbc29fc25c Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png new file mode 100644 index 00000000000..8a6dcccfa38 Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/secret_variables_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/secret_variables_page.png new file mode 100644 index 00000000000..658c0b5bcac Binary files /dev/null and b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/secret_variables_page.png differ diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md new file mode 100644 index 00000000000..e1aff6fdf36 --- /dev/null +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -0,0 +1,684 @@ +--- +redirect_from: 'https://docs.gitlab.com/ee/articles/laravel_with_gitlab_and_envoy/index.html' +--- + +# Test and deploy Laravel applications with GitLab CI/CD and Envoy + +> **[Article Type](../../../development/writing_documentation.md#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Mehran Rasulian](https://gitlab.com/mehranrasulian) || +> **Publication date:** 2017-08-31 + +## Introduction + +GitLab features our applications with Continuous Integration, and it is possible to easily deploy the new code changes to the production server whenever we want. + +In this tutorial, we'll show you how to initialize a [Laravel](http://laravel.com/) application and setup our [Envoy](https://laravel.com/docs/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/). + +We assume you have a basic experience with Laravel, Linux servers, +and you know how to use GitLab. + +Laravel is a high quality web framework written in PHP. +It has a great community with a [fantastic documentation](https://laravel.com/docs). +Aside from the usual routing, controllers, requests, responses, views, and (blade) templates, out of the box Laravel provides plenty of additional services such as cache, events, localization, authentication and many others. + +We will use [Envoy](https://laravel.com/docs/master/envoy) as an SSH task runner based on PHP. +It uses a clean, minimal [Blade syntax](https://laravel.com/docs/blade) to setup tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/artisan). + +## Initialize our Laravel app on GitLab + +We assume [you have installed a new laravel project](https://laravel.com/docs/installation#installation), so let's start with a unit test, and initialize Git for the project. + +### Unit Test + +Every new installation of Laravel (currently 5.4) comes with two type of tests, 'Feature' and 'Unit', placed in the tests directory. +Here's a unit test from `test/Unit/ExampleTest.php`: + +```php +assertTrue(true); + } +} +``` + +This test is as simple as asserting that the given value is true. + +Laravel uses `PHPUnit` for tests by default. +If we run `vendor/bin/phpunit` we should see the green output: + +```bash +vendor/bin/phpunit +OK (1 test, 1 assertions) +``` + +This test will be used later for continuously testing our app with GitLab CI/CD. + +### Push to GitLab + +Since we have our app up and running locally, it's time to push the codebase to our remote repository. +Let's create [a new project](../../../gitlab-basics/create-project.md) in GitLab named `laravel-sample`. +After that, follow the command line instructions displayed on the project's homepage to initiate the repository on our machine and push the first commit. + + +```bash +cd laravel-sample +git init +git remote add origin git@gitlab.example.com:/laravel-sample.git +git add . +git commit -m 'Initial Commit' +git push -u origin master +``` + +## Configure the production server + +Before we begin setting up Envoy and GitLab CI/CD, let's quickly make sure the production server is ready for deployment. +We have installed LEMP stack which stands for Linux, Nginx, MySQL and PHP on our Ubuntu 16.04. + +### Create a new user + +Let's now create a new user that will be used to deploy our website and give it +the needed permissions using [Linux ACL](https://serversforhackers.com/video/linux-acls): + +```bash +# Create user deployer +sudo adduser deployer +# Give the read-write-execute permissions to deployer user for directory /var/www +sudo setfacl -R -m u:deployer:rwx /var/www +``` + +If you don't have ACL installed on your Ubuntu server, use this command to install it: + +```bash +sudo apt install acl +``` + +### Add SSH key + +Let's suppose we want to deploy our app to the production server from a private repository on GitLab. First, we need to [generate a new SSH key pair **with no passphrase**](../../../ssh/README.md) for the deployer user. + +After that, we need to copy the private key, which will be used to connect to our server as the deployer user with SSH, to be able to automate our deployment process: + +```bash +# As the deployer user on server +# +# Copy the content of public key to authorized_keys +cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys +# Copy the private key text block +cat ~/.ssh/id_rsa +``` + +Now, let's add it to your GitLab project as a [secret variable](../../variables/README.md#secret-variables). +Secret variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. +They can be added per project by navigating to the project's **Settings** > **CI/CD**. + +![secret variables page](img/secret_variables_page.png) + +To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. +We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our remote server as the deployer user without entering its password. + +We also need to add the public key to **Project** > **Settings** > **Repository** as [Deploy Keys](../../../ssh/README.md/#deploy-keys), which gives us the ability to access our repository from the server through [SSH protocol](../../../gitlab-basics/command-line-commands.md/#start-working-on-your-project). + + +```bash +# As the deployer user on the server +# +# Copy the public key +cat ~/.ssh/id_rsa.pub +``` + +![deploy keys page](img/deploy_keys_page.png) + +To the field **Title**, add any name you want, and paste the public key into the **Key** field. + +Now, let's clone our repository on the server just to make sure the `deployer` user has access to the repository. + +```bash +# As the deployer user on server +# +git clone git@gitlab.example.com:/laravel-sample.git +``` + +>**Note:** +Answer **yes** if asked `Are you sure you want to continue connecting (yes/no)?`. +It adds GitLab.com to the known hosts. + +### Configuring Nginx + +Now, let's make sure our web server configuration points to the `current/public` rather than `public`. + +Open the default Nginx server block configuration file by typing: + +```bash +sudo nano /etc/nginx/sites-available/default +``` + +The configuration should be like this. + +``` +server { + root /var/www/app/current/public; + server_name example.com; + # Rest of the configuration +} +``` + +>**Note:** +You may replace the app's name in `/var/www/app/current/public` with the folder name of your application. + +## Setting up Envoy + +So we have our Laravel app ready for production. +The next thing is to use Envoy to perform the deploy. + +To use Envoy, we should first install it on our local machine [using the given instructions by Laravel](https://laravel.com/docs/envoy/#introduction). + +### How Envoy works + +The pros of Envoy is that it doesn't require Blade engine, it just uses Blade syntax to define tasks. +To start, we create an `Envoy.blade.php` in the root of our app with a simple task to test Envoy. + + +```php +@servers(['web' => 'remote_username@remote_host']) + +@task('list', [on => 'web']) + ls -l +@endtask +``` + +As you may expect, we have an array within `@servers` directive at the top of the file, which contains a key named `web` with a value of the server's address (e.g. `deployer@192.168.1.1`). +Then within our `@task` directive we define the bash commands that should be run on the server when the task is executed. + +On the local machine use the `run` command to run Envoy tasks. + +```bash +envoy run list +``` + +It should execute the `list` task we defined earlier, which connects to the server and lists directory contents. + +Envoy is not a dependency of Laravel, therefore you can use it for any PHP application. + +### Zero downtime deployment + +Every time we deploy to the production server, Envoy downloads the latest release of our app from GitLab repository and replace it with preview's release. +Envoy does this without any [downtime](https://en.wikipedia.org/wiki/Downtime), +so we don't have to worry during the deployment while someone might be reviewing the site. +Our deployment plan is to clone the latest release from GitLab repository, install the Composer dependencies and finally, activate the new release. + +#### @setup directive + +The first step of our deployment process is to define a set of variables within [@setup](https://laravel.com/docs/envoy/#setup) directive. +You may change the `app` to your application's name: + + +```php +... + +@setup + $repository = 'git@gitlab.example.com:/laravel-sample.git'; + $releases_dir = '/var/www/app/releases'; + $app_dir = '/var/www/app'; + $release = date('YmdHis'); + $new_release_dir = $releases_dir .'/'. $release; +@endsetup + +... +``` + +- `$repository` is the address of our repository +- `$releases_dir` directory is where we deploy the app +- `$app_dir` is the actual location of the app that is live on the server +- `$release` contains a date, so every time that we deploy a new release of our app, we get a new folder with the current date as name +- `$new_release_dir` is the full path of the new release which is used just to make the tasks cleaner + +#### @story directive + +The [@story](https://laravel.com/docs/envoy/#stories) directive allows us define a list of tasks that can be run as a single task. +Here we have three tasks called `clone_repository`, `run_composer`, `update_symlinks`. These variables are usable to making our task's codes more cleaner: + + +```php +... + +@story('deploy') + clone_repository + run_composer + update_symlinks +@endstory + +... +``` + +Let's create these three tasks one by one. + +#### Clone the repository + +The first task will create the `releases` directory (if it doesn't exist), and then clone the `master` branch of the repository (by default) into the new release directory, given by the `$new_release_dir` variable. +The `releases` directory will hold all our deployments: + +```php +... + +@task('clone_repository') + echo 'Cloning repository' + [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} + git clone --depth 1 {{ $repository }} {{ $new_release_dir }} +@endtask + +... +``` + +While our project grows, its Git history will be very very long over time. +Since we are creating a directory per release, it might not be necessary to have the history of the project downloaded for each release. +The `--depth 1` option is a great solution which saves systems time and disk space as well. + +#### Installing dependencies with Composer + +As you may know, this task just navigates to the new release directory and runs Composer to install the application dependencies: + +```php +... + +@task('run_composer') + echo "Starting deployment ({{ $release }})" + cd {{ $new_release_dir }} + composer install --prefer-dist --no-scripts -q -o +@endtask + +... +``` + +#### Activate new release + +Next thing to do after preparing the requirements of our new release, is to remove the storage directory from it and to create two symbolic links to point the application's `storage` directory and `.env` file to the new release. +Then, we need to create another symbolic link to the new release with the name of `current` placed in the app directory. +The `current` symbolic link always points to the latest release of our app: + +```php +... + +@task('update_symlinks') + echo "Linking storage directory" + rm -rf {{ $new_release_dir }}/storage + ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage + + echo 'Linking .env file' + ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env + + echo 'Linking current release' + ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current +@endtask +``` + +As you see, we use `-nfs` as an option for `ln` command, which says that the `storage`, `.env` and `current` no longer points to the preview's release and will point them to the new release by force (`f` from `-nfs` means force), which is the case when we are doing multiple deployments. + +### Full script + +The script is ready, but make sure to change the `deployer@192.168.1.1` to your server and also change `/var/www/app` with the directory you want to deploy your app. + +At the end, our `Envoy.blade.php` file will look like this: + +```php +@servers(['web' => 'deployer@192.168.1.1']) + +@setup + $repository = 'git@gitlab.example.com:/laravel-sample.git'; + $releases_dir = '/var/www/app/releases'; + $app_dir = '/var/www/app'; + $release = date('YmdHis'); + $new_release_dir = $releases_dir .'/'. $release; +@endsetup + +@story('deploy') + clone_repository + run_composer + update_symlinks +@endstory + +@task('clone_repository') + echo 'Cloning repository' + [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} + git clone --depth 1 {{ $repository }} {{ $new_release_dir }} +@endtask + +@task('run_composer') + echo "Starting deployment ({{ $release }})" + cd {{ $new_release_dir }} + composer install --prefer-dist --no-scripts -q -o +@endtask + +@task('update_symlinks') + echo "Linking storage directory" + rm -rf {{ $new_release_dir }}/storage + ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage + + echo 'Linking .env file' + ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env + + echo 'Linking current release' + ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current +@endtask +``` + +One more thing we should do before any deployment is to manually copy our application `storage` folder to the `/var/www/app` directory on the server for the first time. +You might want to create another Envoy task to do that for you. +We also create the `.env` file in the same path to setup our production environment variables for Laravel. +These are persistent data and will be shared to every new release. + +Now, we would need to deploy our app by running `envoy run deploy`, but it won't be necessary since GitLab can handle that for us with CI's [environments](../../environments.md), which will be described [later](#setting-up-gitlab-ci-cd) in this tutorial. + +Now it's time to commit [Envoy.blade.php](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Envoy.blade.php) and push it to the `master` branch. +To keep things simple, we commit directly to `master`, without using [feature-branches](../../../workflow/gitlab_flow.md/#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial. +In a real world project, teams may use [Issue Tracker](../../../user/project/issues/index.md) and [Merge Requests](../../../user/project/merge_requests/index.md) to move their code across branches: + +```bash +git add Envoy.blade.php +git commit -m 'Add Envoy' +git push origin master +``` + +## Continuous Integration with GitLab + +We have our app ready on GitLab, and we also can deploy it manually. +But let's take a step forward to do it automatically with [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) method. +We need to check every commit with a set of automated tests to become aware of issues at the earliest, and then, we can deploy to the target environment if we are happy with the result of the tests. + +[GitLab CI/CD](../../README.md) allows us to use [Docker](https://docker.com/) engine to handle the process of testing and deploying our app. +In the case you're not familiar with Docker, refer to [How to Automate Docker Deployments](http://paislee.io/how-to-automate-docker-deployments/). + +To be able to build, test, and deploy our app with GitLab CI/CD, we need to prepare our work environment. +To do that, we'll use a Docker image which has the minimum requirements that a Laravel app needs to run. +[There are other ways](../php.md/#test-php-projects-using-the-docker-executor) to do that as well, but they may lead our builds run slowly, which is not what we want when there are faster options to use. + +With Docker images our builds run incredibly faster! + +### Create a Container Image + +Let's create a [Dockerfile](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Dockerfile) in the root directory of our app with the following content: + +```bash +# Set the base image for subsequent instructions +FROM php:7.1 + +# Update packages +RUN apt-get update + +# Install PHP and composer dependencies +RUN apt-get install -qq git curl libmcrypt-dev libjpeg-dev libpng-dev libfreetype6-dev libbz2-dev + +# Clear out the local repository of retrieved package files +RUN apt-get clean + +# Install needed extensions +# Here you can install any other extension that you need during the test and deployment process +RUN docker-php-ext-install mcrypt pdo_mysql zip + +# Install Composer +RUN curl --silent --show-error https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer + +# Install Laravel Envoy +RUN composer global require "laravel/envoy=~1.0" +``` + +We added the [official PHP 7.1 Docker image](https://hub.docker.com/r/_/php/), which consist of a minimum installation of Debian Jessie with PHP pre-installed, and works perfectly for our use case. + +We used `docker-php-ext-install` (provided by the official PHP Docker image) to install the PHP extensions we need. + +#### Setting Up GitLab Container Registry + +Now that we have our `Dockerfile` let's build and push it to our [GitLab Container Registry](../../../user/project/container_registry.md). + +> The registry is the place to store and tag images for later use. Developers may want to maintain their own registry for private, company images, or for throw-away images used only in testing. Using GitLab Container Registry means you don't need to set up and administer yet another service or use a public registry. + +On your GitLab project repository navigate to the **Registry** tab. + +![container registry page empty image](img/container_registry_page_empty_image.png) + +You may need to [enable Container Registry](../../../user/project/container_registry.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Sharing and permissions**. + +![container registry checkbox](img/container_registry_checkbox.png) + +To start using Container Registry on our machine, we first need to login to the GitLab registry using our GitLab username and password: + +```bash +docker login registry.gitlab.com +``` +Then we can build and push our image to GitLab: + +```bash +docker build -t registry.gitlab.com//laravel-sample . + +docker push registry.gitlab.com//laravel-sample +``` + +>**Note:** +To run the above commands, we first need to have [Docker](https://docs.docker.com/engine/installation/) installed on our machine. + +Congratulations! You just pushed the first Docker image to the GitLab Registry, and if you refresh the page you should be able to see it: + +![container registry page with image](img/container_registry_page_with_image.jpg) + +>**Note:** +You can also [use GitLab CI/CD](https://about.gitlab.com/2016/05/23/gitlab-container-registry/#use-with-gitlab-ci) to build and push your Docker images, rather than doing that on your machine. + +We'll use this image further down in the `.gitlab-ci.yml` configuration file to handle the process of testing and deploying our app. + +Let's commit the `Dockerfile` file. + +```bash +git add Dockerfile +git commit -m 'Add Dockerfile' +git push origin master +``` + +### Setting up GitLab CI/CD + +In order to build and test our app with GitLab CI/CD, we need a file called `.gitlab-ci.yml` in our repository's root. It is similar to Circle CI and Travis CI, but built-in GitLab. + +Our `.gitlab-ci.yml` file will look like this: + +```yaml +image: registry.gitlab.com//laravel-sample:latest + +services: + - mysql:5.7 + +variables: + MYSQL_DATABASE: homestead + MYSQL_ROOT_PASSWORD: secret + DB_HOST: mysql + DB_USERNAME: root + +stages: + - test + - deploy + +unit_test: + stage: test + script: + - cp .env.example .env + - composer install + - php artisan key:generate + - php artisan migrate + - vendor/bin/phpunit + +deploy_production: + stage: deploy + script: + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + - eval $(ssh-agent -s) + - ssh-add <(echo "$SSH_PRIVATE_KEY") + - mkdir -p ~/.ssh + - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' + + - ~/.composer/vendor/bin/envoy run deploy + environment: + name: production + url: http://192.168.1.1 + when: manual + only: + - master +``` + +That's a lot to take in, isn't it? Let's run through it step by step. + +#### Image and Services + +[GitLab Runners](../../runners/README.md) run the script defined by `.gitlab-ci.yml`. +The `image` keyword tells the Runners which image to use. +The `services` keyword defines additional images [that are linked to the main image](../../docker/using_docker_images.md/#what-is-a-service). +Here we use the container image we created before as our main image and also use MySQL 5.7 as a service. + +```yaml +image: registry.gitlab.com//laravel-sample:latest + +services: + - mysql:5.7 + +... +``` + +>**Note:** +If you wish to test your app with different PHP versions and [database management systems](../../services/README.md), you can define different `image` and `services` keywords for each test job. + +#### Variables + +GitLab CI/CD allows us to use [environment variables](../../yaml/README.md#variables) in our jobs. +We defined MySQL as our database management system, which comes with a superuser root created by default. + +So we should adjust the configuration of MySQL instance by defining `MYSQL_DATABASE` variable as our database name and `MYSQL_ROOT_PASSWORD` variable as the password of `root`. +Find out more about MySQL variables at the [official MySQL Docker Image](https://hub.docker.com/r/_/mysql/). + +Also set the variables `DB_HOST` to `mysql` and `DB_USERNAME` to `root`, which are Laravel specific variables. +We define `DB_HOST` as `mysql` instead of `127.0.0.1`, as we use MySQL Docker image as a service which [is linked to the main Docker image](../../docker/using_docker_images.md/#how-services-are-linked-to-the-build). + +```yaml +... + +variables: + MYSQL_DATABASE: homestead + MYSQL_ROOT_PASSWORD: secret + DB_HOST: mysql + DB_USERNAME: root + +... +``` + +#### Unit Test as the first job + +We defined the required shell scripts as an array of the [script](../../yaml/README.md#script) variable to be executed when running `unit_test` job. + +These scripts are some Artisan commands to prepare the Laravel, and, at the end of the script, we'll run the tests by `PHPUnit`. + +```yaml +... + +unit_test: + script: + # Install app dependencies + - composer install + # Setup .env + - cp .env.example .env + # Generate an environment key + - php artisan key:generate + # Run migrations + - php artisan migrate + # Run tests + - vendor/bin/phpunit + +... +``` + +#### Deploy to production + +The job `deploy_production` will deploy the app to the production server. +To deploy our app with Envoy, we had to set up the `$SSH_PRIVATE_KEY` variable as an [SSH private key](../../ssh_keys/README.md/#ssh-keys-when-using-the-docker-executor). +If the SSH keys have added successfully, we can run Envoy. + +As mentioned before, GitLab supports [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) methods as well. +The [environment](../../yaml/README.md#environment) keyword tells GitLab that this job deploys to the `production` environment. +The `url` keyword is used to generate a link to our application on the GitLab Environments page. +The `only` keyword tells GitLab CI that the job should be executed only when the pipeline is building the `master` branch. +Lastly, `when: manual` is used to turn the job from running automatically to a manual action. + +```yaml +... + +deploy_production: + script: + # Add the private SSH key to the build environment + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + - eval $(ssh-agent -s) + - ssh-add <(echo "$SSH_PRIVATE_KEY") + - mkdir -p ~/.ssh + - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' + + # Run Envoy + - ~/.composer/vendor/bin/envoy run deploy + + environment: + name: production + url: http://192.168.1.1 + when: manual + only: + - master +``` + +You may also want to add another job for [staging environment](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments), to final test your application before deploying to production. + +### Turn on GitLab CI/CD + +We have prepared everything we need to test and deploy our app with GitLab CI/CD. +To do that, commit and push `.gitlab-ci.yml` to the `master` branch. It will trigger a pipeline, which you can watch live under your project's **Pipelines**. + +![pipelines page](img/pipelines_page.png) + +Here we see our **Test** and **Deploy** stages. +The **Test** stage has the `unit_test` build running. +click on it to see the Runner's output. + +![pipeline page](img/pipeline_page.png) + +After our code passed through the pipeline successfully, we can deploy to our production server by clicking the **play** button on the right side. + +![pipelines page deploy button](img/pipelines_page_deploy_button.png) + +Once the deploy pipeline passed successfully, navigate to **Pipelines > Environments**. + +![environments page](img/environments_page.png) + +If something doesn't work as expected, you can roll back to the latest working version of your app. + +![environment page](img/environment_page.png) + +By clicking on the external link icon specified on the right side, GitLab opens the production website. +Our deployment successfully was done and we can see the application is live. + +![laravel welcome page](img/laravel_welcome_page.png) + +In the case that you're interested to know how is the application directory structure on the production server after deployment, here are three directories named `current`, `releases` and `storage`. +As you know, the `current` directory is a symbolic link that points to the latest release. +The `.env` file consists of our Laravel environment variables. + +![production server app directory](img/production_server_app_directory.png) + +If you navigate to the `current` directory, you should see the application's content. +As you see, the `.env` is pointing to the `/var/www/app/.env` file and also `storage` is pointing to the `/var/www/app/storage/` directory. + +![production server current directory](img/production_server_current_directory.png) + +## Conclusion + +We configured GitLab CI to perform automated tests and used the method of [Continuous Delivery](https://continuousdelivery.com/) to deploy to production a Laravel application with Envoy, directly from the codebase. + +Envoy also was a great match to help us deploy the application without writing our custom bash script and doing Linux magics. -- cgit v1.2.1 From 8d569f9fe9bddaa09cb05c1c42b1c39b7bf3b3aa Mon Sep 17 00:00:00 2001 From: Lin Jen-Shin Date: Fri, 5 Jan 2018 23:21:47 +0800 Subject: Document that we need rsync for backing up --- doc/install/installation.md | 2 +- doc/raketasks/backup_restore.md | 21 +++++++++++++++++---- doc/update/10.3-to-10.4.md | 2 ++ 3 files changed, 20 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/install/installation.md b/doc/install/installation.md index 2b7352d3561..b2acd5e78b5 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -64,7 +64,7 @@ up-to-date and install it. Install the required packages (needed to compile Ruby and native extensions to Ruby gems): - sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake + sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate rsync python-docutils pkg-config cmake Ubuntu 14.04 (Trusty Tahr) doesn't have the `libre2-dev` package available, but you can [install re2 manually](https://github.com/google/re2/wiki/Install). diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 54c3e20d61d..50bb665216e 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -5,8 +5,8 @@ An application data backup creates an archive file that contains the database, all repositories and all attachments. -You can only restore a backup to **exactly the same version and type (CE/EE)** -of GitLab on which it was created. The best way to migrate your repositories +You can only restore a backup to **exactly the same version and type (CE/EE)** +of GitLab on which it was created. The best way to migrate your repositories from one server to another is through backup restore. ## Backup @@ -14,6 +14,19 @@ from one server to another is through backup restore. GitLab provides a simple command line interface to backup your whole installation, and is flexible enough to fit your needs. +### Requirements + +If you're using GitLab with the Omnibus package, you're all set. If you +installed GitLab from source, make sure the following packages are installed: + +* rsync + +If you're using Ubuntu, you could run: + +``` +sudo apt-get install -y rsync +``` + ### Backup timestamp >**Note:** @@ -431,7 +444,7 @@ The [restore prerequisites section](#restore-prerequisites) includes crucial information. Make sure to read and test the whole restore process at least once before attempting to perform it in a production environment. -You can only restore a backup to **exactly the same version and type (CE/EE)** of +You can only restore a backup to **exactly the same version and type (CE/EE)** of GitLab that you created it on, for example CE 9.1.0. ### Restore prerequisites @@ -511,7 +524,7 @@ sudo service gitlab restart This procedure assumes that: -- You have installed the **exact same version and type (CE/EE)** of GitLab +- You have installed the **exact same version and type (CE/EE)** of GitLab Omnibus with which the backup was created. - You have run `sudo gitlab-ctl reconfigure` at least once. - GitLab is running. If not, start it using `sudo gitlab-ctl start`. diff --git a/doc/update/10.3-to-10.4.md b/doc/update/10.3-to-10.4.md index 850cb3103f4..67b7e634c94 100644 --- a/doc/update/10.3-to-10.4.md +++ b/doc/update/10.3-to-10.4.md @@ -21,6 +21,8 @@ sudo service gitlab stop ### 2. Backup +NOTE: If you installed GitLab from source, make sure `rsync` is installed. + ```bash cd /home/git/gitlab -- cgit v1.2.1