diff options
Diffstat (limited to 'doc')
41 files changed, 1184 insertions, 63 deletions
diff --git a/doc/administration/img/failing_storage.png b/doc/administration/img/failing_storage.png Binary files differnew file mode 100644 index 00000000000..82b393a58b2 --- /dev/null +++ b/doc/administration/img/failing_storage.png diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index 55a45119525..624a908b3a3 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -60,7 +60,7 @@ respectively. path: /mnt/cephfs/repositories ``` -1. [Restart GitLab] for the changes to take effect. +1. [Restart GitLab][restart-gitlab] for the changes to take effect. >**Note:** The [`gitlab_shell: repos_path` entry][repospath] in `gitlab.yml` will be @@ -97,9 +97,80 @@ be stored via the **Application Settings** in the Admin area. Beginning with GitLab 8.13.4, multiple paths can be chosen. New projects will be randomly placed on one of the selected paths. +## Handling failing repository storage + +> [Introduced][ce-11449] in GitLab 9.5. + +When GitLab detects access to the repositories storage fails repeatedly, it can +gracefully prevent attempts to access the storage. This might be useful when +the repositories are stored somewhere on the network. + +The configuration could look as follows: + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + git_data_dirs({ + "default" => { + "path" => "/mnt/nfs-01/git-data", + "failure_count_threshold" => 10, + "failure_wait_time" => 30, + "failure_reset_time" => 1800, + "storage_timeout" => 5 + } + }) + ``` + +1. Save the file and [reconfigure GitLab][reconfigure-gitlab] for the changes to take effect. + +--- + +**For installations from source** + +1. Edit `config/gitlab.yml`: + + ```yaml + repositories: + storages: # You must have at least a `default` storage path. + default: + path: /home/git/repositories/ + failure_count_threshold: 10 # number of failures before stopping attempts + failure_wait_time: 30 # Seconds after last access failure before trying again + failure_reset_time: 1800 # Time in seconds to expire failures + storage_timeout: 5 # Time in seconds to wait before aborting a storage access attempt + ``` + +1. Save the file and [restart GitLab][restart-gitlab] for the changes to take effect. + + +**`failure_count_threshold`:** The number of failures of after which GitLab will +completely prevent access to the storage. The number of failures can be reset in +the admin interface: `https://gitlab.example.com/admin/health_check` or using the +[api](../api/repository_storage_health.md) to allow access to the storage again. + +**`failure_wait_time`:** When access to a storage fails. GitLab will prevent +access to the storage for the time specified here. This allows the filesystem to +recover without. + +**`failure_reset_time`:** The time in seconds GitLab will keep failure +information. When no failures occur during this time, information about the +mount is reset. + +**`storage_timeout`:** The time in seconds GitLab will try to access storage. +After this time a timeout error will be raised. + +When storage failures occur, this will be visible in the admin interface like this: + + + +To allow access to all storages, click the `Reset git storage health information` button. + [ce-4578]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4578 -[restart gitlab]: restart_gitlab.md#installations-from-source -[reconfigure gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure +[restart-gitlab]: restart_gitlab.md#installations-from-source +[reconfigure-gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure [backups]: ../raketasks/backup_restore.md [raketask]: https://gitlab.com/gitlab-org/gitlab-ce/blob/033e5423a2594e08a7ebcd2379bd2331f4c39032/lib/backup/repository.rb#L54-56 [repospath]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-9-stable/config/gitlab.yml.example#L457 +[ce-11449]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11449 diff --git a/doc/api/commits.md b/doc/api/commits.md index c91f9ecbdaf..2a78553782f 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -69,8 +69,9 @@ POST /projects/:id/repository/commits | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | -| `branch` | string | yes | The name of a branch | +| `branch` | string | yes | Name of the branch to commit into. To create a new branch, also provide `start_branch`. | | `commit_message` | string | yes | Commit message | +| `start_branch` | string | no | Name of the branch to start the new commit from | | `actions[]` | array | yes | An array of action hashes to commit as a batch. See the next table for what attributes it can take. | | `author_email` | string | no | Specify the commit author's email address | | `author_name` | string | no | Specify the commit author's name | diff --git a/doc/api/group_milestones.md b/doc/api/group_milestones.md index 086fba7e91d..dbfc7529125 100644 --- a/doc/api/group_milestones.md +++ b/doc/api/group_milestones.md @@ -6,7 +6,7 @@ Returns a list of group milestones. ``` GET /groups/:id/milestones -GET /groups/:id/milestones?iids=42 +GET /groups/:id/milestones?iids[]=42 GET /groups/:id/milestones?iids[]=42&iids[]=43 GET /groups/:id/milestones?state=active GET /groups/:id/milestones?state=closed @@ -18,7 +18,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | -| `iids` | Array[integer] | optional | Return only the milestones having the given `iids` | +| `iids[]` | Array[integer] | optional | Return only the milestones having the given `iid` | | `state` | string | optional | Return only `active` or `closed` milestones` | | `search` | string | optional | Return only milestones with a title or description matching the provided string | diff --git a/doc/api/issues.md b/doc/api/issues.md index 6bac2927339..f30ed08d0fa 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -40,7 +40,7 @@ GET /issues?assignee_id=5 | `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all`. Defaults to `created-by-me` _([Introduced][ce-13004] in GitLab 9.5)_ | | `author_id` | integer | no | Return issues created by the given user `id`. Combine with `scope=all` or `scope=assigned-to-me`. _([Introduced][ce-13004] in GitLab 9.5)_ | | `assignee_id` | integer | no | Return issues assigned to the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `iids` | Array[integer] | no | Return only the issues having the given `iid` | +| `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | | `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | | `search` | string | no | Search issues against their `title` and `description` | @@ -132,7 +132,7 @@ GET /groups/:id/issues?assignee_id=5 | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | | `state` | string | no | Return all issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | -| `iids` | Array[integer] | no | Return only the issues having the given `iid` | +| `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | | `milestone` | string | no | The milestone title | | `scope` | string | no | Return issues for the given scope: `created-by-me`, `assigned-to-me` or `all` _([Introduced][ce-13004] in GitLab 9.5)_ | | `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | @@ -227,7 +227,7 @@ GET /projects/:id/issues?assignee_id=5 | Attribute | Type | Required | Description | |-------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------| | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `iids` | Array[integer] | no | Return only the milestone having the given `iid` | +| `iids[]` | Array[integer] | no | Return only the milestone having the given `iid` | | `state` | string | no | Return all issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `No+Label` lists all issues with no labels | | `milestone` | string | no | The milestone title | diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index d0725b5e06e..802e5362d70 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -117,7 +117,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer | yes | The ID of a project | -| `iids` | Array[integer] | no | Return the request having the given `iid` | +| `iids[]` | Array[integer] | no | Return the request having the given `iid` | | `state` | string | no | Return all merge requests or just those that are `opened`, `closed`, or `merged`| | `order_by`| string | no | Return requests ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | diff --git a/doc/api/milestones.md b/doc/api/milestones.md index a082d548499..84930f0bdc9 100644 --- a/doc/api/milestones.md +++ b/doc/api/milestones.md @@ -6,7 +6,7 @@ Returns a list of project milestones. ``` GET /projects/:id/milestones -GET /projects/:id/milestones?iids=42 +GET /projects/:id/milestones?iids[]=42 GET /projects/:id/milestones?iids[]=42&iids[]=43 GET /projects/:id/milestones?state=active GET /projects/:id/milestones?state=closed @@ -18,7 +18,7 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `iids` | Array[integer] | optional | Return only the milestones having the given `iids` | +| `iids[]` | Array[integer] | optional | Return only the milestones having the given `iid` | | `state` | string | optional | Return only `active` or `closed` milestones` | | `search` | string | optional | Return only milestones with a title or description matching the provided string | diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index 1fc577561a0..c517a38a8ba 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -76,7 +76,8 @@ Example response: Parameters: - `file_path` (required) - Url encoded full path to new file. Ex. lib%2Fclass%2Erb -- `branch` (required) - The name of branch +- `branch` (required) - Name of the branch +- `start_branch` (optional) - Name of the branch to start the new commit from - `encoding` (optional) - Change encoding to 'base64'. Default is text. - `author_email` (optional) - Specify the commit author's email address - `author_name` (optional) - Specify the commit author's name @@ -105,7 +106,8 @@ Example response: Parameters: - `file_path` (required) - Url encoded full path to new file. Ex. lib%2Fclass%2Erb -- `branch` (required) - The name of branch +- `branch` (required) - Name of the branch +- `start_branch` (optional) - Name of the branch to start the new commit from - `encoding` (optional) - Change encoding to 'base64'. Default is text. - `author_email` (optional) - Specify the commit author's email address - `author_name` (optional) - Specify the commit author's name @@ -144,7 +146,8 @@ Example response: Parameters: - `file_path` (required) - Url encoded full path to new file. Ex. lib%2Fclass%2Erb -- `branch` (required) - The name of branch +- `branch` (required) - Name of the branch +- `start_branch` (optional) - Name of the branch to start the new commit from - `author_email` (optional) - Specify the commit author's email address - `author_name` (optional) - Specify the commit author's name - `commit_message` (required) - Commit message diff --git a/doc/api/repository_storage_health.md b/doc/api/repository_storage_health.md new file mode 100644 index 00000000000..e0c0315c2d7 --- /dev/null +++ b/doc/api/repository_storage_health.md @@ -0,0 +1,74 @@ +# Circuitbreaker API + +> [Introduced][ce-11449] in GitLab 9.5. + +The Circuitbreaker API is only accessible to administrators. All requests by +guests will respond with `401 Unauthorized`, and all requests by normal users +will respond with `403 Forbidden`. + +## Repository Storages + +### Get all storage information + +Returns of all currently configured storages and their health information. + +``` +GET /circuit_breakers/repository_storage +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/circuit_breakers/repository_storage +``` + +```json +[ + { + "storage_name": "default", + "failing_on_hosts": [], + "total_failures": 0 + }, + { + "storage_name": "broken", + "failing_on_hosts": [ + "web01", "worker01" + ], + "total_failures": 1 + } +] +``` + +### Get failing storages + +This returns a list of all currently failing storages. + +``` +GET /circuit_breakers/repository_storage/failing +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/circuit_breakers/repository_storage/failing +``` + +```json +[ + { + "storage_name":"broken", + "failing_on_hosts":["web01", "worker01"], + "total_failures":2 + } +] +``` + +## Reset failing storage information + +Use this remove all failing storage information and allow access to the storage again. + +``` +DELETE /circuit_breakers/repository_storage +``` + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/circuit_breakers/repository_storage +``` + +[ce-11449]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11449 diff --git a/doc/articles/index.md b/doc/articles/index.md index 9d2e5956029..558c624fe39 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -80,7 +80,7 @@ Install, upgrade, integrate, migrate to GitLab: | :------------ | :------: | --------------: | | [Video Tutorial: Idea to Production on Google Container Engine (GKE)](https://about.gitlab.com/2017/01/23/video-tutorial-idea-to-production-on-google-container-engine-gke/) | Tutorial | 2017/01/23 | | [How to Setup a GitLab Instance on Microsoft Azure](https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/) | Tutorial | 2016/07/13 | -| [Get started with OpenShift Origin 3 and GitLab](https://about.gitlab.com/2016/06/28/get-started-with-openshift-origin-3-and-gitlab/) | Tutorial | 2016/06/28 | +| [Get started with OpenShift Origin 3 and GitLab](openshift_and_gitlab/index.md) | Tutorial | 2016/06/28 | | [Getting started with GitLab and DigitalOcean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/) | Tutorial | 2016/04/27 | ## Software development diff --git a/doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png b/doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png Binary files differnew file mode 100644 index 00000000000..fcad4e59ae3 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png diff --git a/doc/articles/openshift_and_gitlab/img/add-to-project.png b/doc/articles/openshift_and_gitlab/img/add-to-project.png Binary files differnew file mode 100644 index 00000000000..bd915a229f6 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/add-to-project.png diff --git a/doc/articles/openshift_and_gitlab/img/create-project-ui.png b/doc/articles/openshift_and_gitlab/img/create-project-ui.png Binary files differnew file mode 100644 index 00000000000..e72866f252a --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/create-project-ui.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-logs.png b/doc/articles/openshift_and_gitlab/img/gitlab-logs.png Binary files differnew file mode 100644 index 00000000000..1e24080c7df --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/gitlab-logs.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-overview.png b/doc/articles/openshift_and_gitlab/img/gitlab-overview.png Binary files differnew file mode 100644 index 00000000000..3c5df0ea101 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/gitlab-overview.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-running.png b/doc/articles/openshift_and_gitlab/img/gitlab-running.png Binary files differnew file mode 100644 index 00000000000..c7db691cb30 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/gitlab-running.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-scale.png b/doc/articles/openshift_and_gitlab/img/gitlab-scale.png Binary files differnew file mode 100644 index 00000000000..4903c7d7498 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/gitlab-scale.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-settings.png b/doc/articles/openshift_and_gitlab/img/gitlab-settings.png Binary files differnew file mode 100644 index 00000000000..db4360ffef0 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/gitlab-settings.png diff --git a/doc/articles/openshift_and_gitlab/img/no-resources.png b/doc/articles/openshift_and_gitlab/img/no-resources.png Binary files differnew file mode 100644 index 00000000000..480fb766468 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/no-resources.png diff --git a/doc/articles/openshift_and_gitlab/img/openshift-infra-project.png b/doc/articles/openshift_and_gitlab/img/openshift-infra-project.png Binary files differnew file mode 100644 index 00000000000..8b9f85aa341 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/openshift-infra-project.png diff --git a/doc/articles/openshift_and_gitlab/img/pods-overview.png b/doc/articles/openshift_and_gitlab/img/pods-overview.png Binary files differnew file mode 100644 index 00000000000..e1cf08bd217 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/pods-overview.png diff --git a/doc/articles/openshift_and_gitlab/img/rc-name.png b/doc/articles/openshift_and_gitlab/img/rc-name.png Binary files differnew file mode 100644 index 00000000000..889e34adbec --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/rc-name.png diff --git a/doc/articles/openshift_and_gitlab/img/running-pods.png b/doc/articles/openshift_and_gitlab/img/running-pods.png Binary files differnew file mode 100644 index 00000000000..3fd4e56662f --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/running-pods.png diff --git a/doc/articles/openshift_and_gitlab/img/storage-volumes.png b/doc/articles/openshift_and_gitlab/img/storage-volumes.png Binary files differnew file mode 100644 index 00000000000..ae1e5381faa --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/storage-volumes.png diff --git a/doc/articles/openshift_and_gitlab/img/web-console.png b/doc/articles/openshift_and_gitlab/img/web-console.png Binary files differnew file mode 100644 index 00000000000..aa1425d4f94 --- /dev/null +++ b/doc/articles/openshift_and_gitlab/img/web-console.png diff --git a/doc/articles/openshift_and_gitlab/index.md b/doc/articles/openshift_and_gitlab/index.md new file mode 100644 index 00000000000..7f76e577efa --- /dev/null +++ b/doc/articles/openshift_and_gitlab/index.md @@ -0,0 +1,510 @@ +# Getting started with OpenShift Origin 3 and GitLab + +> **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) || +> **Publication date:** 2016/06/28 + +## Introduction + +[OpenShift Origin][openshift] is an open source container application +platform created by [RedHat], based on [kubernetes] and [Docker]. That means +you can host your own PaaS for free and almost with no hassle. + +In this tutorial, we will see how to deploy GitLab in OpenShift using GitLab's +official Docker image while getting familiar with the web interface and CLI +tools that will help us achieve our goal. + +--- + +## Prerequisites + +OpenShift 3 is not yet deployed on RedHat's offered Online platform ([openshift.com]), +so in order to test it, we will use an [all-in-one Virtualbox image][vm] that is +offered by the OpenShift developers and managed by Vagrant. If you haven't done +already, go ahead and install the following components as they are essential to +test OpenShift easily: + +- [VirtualBox] +- [Vagrant] +- [OpenShift Client][oc] (`oc` for short) + +It is also important to mention that for the purposes of this tutorial, the +latest Origin release is used: + +- **oc** `v1.3.0` (must be [installed][oc-gh] locally on your computer) +- **openshift** `v1.3.0` (is pre-installed in the [VM image][vm-new]) +- **kubernetes** `v1.3.0` (is pre-installed in the [VM image][vm-new]) + +>**Note:** +If you intend to deploy GitLab on a production OpenShift cluster, there are some +limitations to bare in mind. Read on the [limitations](#current-limitations) +section for more information and follow the linked links for the relevant +discussions. + +Now that you have all batteries, let's see how easy it is to test OpenShift +on your computer. + +## Getting familiar with OpenShift Origin + +The environment we are about to use is based on CentOS 7 which comes with all +the tools needed pre-installed: Docker, kubernetes, OpenShift, etcd. + +### Test OpenShift using Vagrant + +As of this writing, the all-in-one VM is at version 1.3, and that's +what we will use in this tutorial. + +In short: + +1. Open a terminal and in a new directory run: + ```sh + vagrant init openshift/origin-all-in-one + ``` +1. This will generate a Vagrantfile based on the all-in-one VM image +1. In the same directory where you generated the Vagrantfile + enter: + + ```sh + vagrant up + ``` + +This will download the VirtualBox image and fire up the VM with some preconfigured +values as you can see in the Vagrantfile. As you may have noticed, you need +plenty of RAM (5GB in our example), so make sure you have enough. + +Now that OpenShift is setup, let's see how the web console looks like. + +### Explore the OpenShift web console + +Once Vagrant finishes its thing with the VM, you will be presented with a +message which has some important information. One of them is the IP address +of the deployed OpenShift platform and in particular <https://10.2.2.2:8443/console/>. +Open this link with your browser and accept the self-signed certificate in +order to proceed. + +Let's login as admin with username/password `admin/admin`. This is what the +landing page looks like: + + + +You can see that a number of [projects] are already created for testing purposes. + +If you head over the `openshift-infra` project, a number of services with their +respective pods are there to explore. + + + +We are not going to explore the whole interface, but if you want to learn about +the key concepts of OpenShift, read the [core concepts reference][core] in the +official documentation. + +### Explore the OpenShift CLI + +OpenShift Client (`oc`), is a powerful CLI tool that talks to the OpenShift API +and performs pretty much everything you can do from the web UI and much more. + +Assuming you have [installed][oc] it, let's explore some of its main +functionalities. + +Let's first see the version of `oc`: + +```sh +$ oc version + +oc v1.3.0 +kubernetes v1.3.0+52492b4 +``` + +With `oc help` you can see the top level arguments you can run with `oc` and +interact with your cluster, kubernetes, run applications, create projects and +much more. + +Let's login to the all-in-one VM and see how to achieve the same results like +when we visited the web console earlier. The username/password for the +administrator user is `admin/admin`. There is also a test user with username/ +password `user/user`, with limited access. Let's login as admin for the moment: + +```sh +$ oc login https://10.2.2.2:8443 + +Authentication required for https://10.2.2.2:8443 (openshift) +Username: admin +Password: +Login successful. + +You have access to the following projects and can switch between them with 'oc project <projectname>': + + * cockpit + * default (current) + * delete + * openshift + * openshift-infra + * sample + +Using project "default". +``` + +Switch to the `openshift-infra` project with: + +```sh +oc project openshift-infra +``` + +And finally, see its status: + +```sh +oc status +``` + +The last command should spit a bunch of information about the statuses of the +pods and the services, which if you look closely is what we encountered in the +second image when we explored the web console. + +You can always read more about `oc` in the [OpenShift CLI documentation][oc]. + +### Troubleshooting the all-in-one VM + +Using the all-in-one VM gives you the ability to test OpenShift whenever you +want. That means you get to play with it, shutdown the VM, and pick up where +you left off. + +Sometimes though, you may encounter some issues, like OpenShift not running +when booting up the VM. The web UI may not responding or you may see issues +when trying to login with `oc`, like: + +``` +The connection to the server 10.2.2.2:8443 was refused - did you specify the right host or port? +``` + +In that case, the OpenShift service might not be running, so in order to fix it: + +1. SSH into the VM by going to the directory where the Vagrantfile is and then + run: + + ```sh + vagrant ssh + ``` + +1. Run `systemctl` and verify by the output that the `openshift` service is not + running (it will be in red color). If that's the case start the service with: + + ```sh + sudo systemctl start openshift + ``` + +1. Verify the service is up with: + + ```sh + systemctl status openshift -l + ``` + +Now you will be able to login using `oc` (like we did before) and visit the web +console. + +## Deploy GitLab + +Now that you got a taste of what OpenShift looks like, let's deploy GitLab! + +### Create a new project + +First, we will create a new project to host our application. You can do this +either by running the CLI client: + +```bash +$ oc new-project gitlab +``` + +or by using the web interface: + + + +If you used the command line, `oc` automatically uses the new project and you +can see its status with: + +```sh +$ oc status + +In project gitlab on server https://10.2.2.2:8443 + +You have no services, deployment configs, or build configs. +Run 'oc new-app' to create an application. +``` + +If you visit the web console, you can now see `gitlab` listed in the projects list. + +The next step is to import the OpenShift template for GitLab. + +### Import the template + +The [template][templates] is basically a JSON file which describes a set of +related object definitions to be created together, as well as a set of +parameters for those objects. + +The template for GitLab resides in the Omnibus GitLab repository under the +docker directory. Let's download it locally with `wget`: + +```bash +wget https://gitlab.com/gitlab-org/omnibus-gitlab/raw/master/docker/openshift-template.json +``` + +And then let's import it in OpenShift: + +```bash +oc create -f openshift-template.json -n openshift +``` + +>**Note:** +The `-n openshift` namespace flag is a trick to make the template available to all +projects. If you recall from when we created the `gitlab` project, `oc` switched +to it automatically, and that can be verified by the `oc status` command. If +you omit the namespace flag, the application will be available only to the +current project, in our case `gitlab`. The `openshift` namespace is a global +one that the administrators should use if they want the application to be +available to all users. + +We are now ready to finally deploy GitLab! + +### Create a new application + +The next step is to use the template we previously imported. Head over to the +`gitlab` project and hit the **Add to Project** button. + + + +This will bring you to the catalog where you can find all the pre-defined +applications ready to deploy with the click of a button. Search for `gitlab` +and you will see the previously imported template: + + + +Select it, and in the following screen you will be presented with the predefined +values used with the GitLab template: + + + +Notice at the top that there are three resources to be created with this +template: + +- `gitlab-ce` +- `gitlab-ce-redis` +- `gitlab-ce-postgresql` + +While PostgreSQL and Redis are bundled in Omnibus GitLab, the template is using +separate images as you can see from [this line][line] in the template. + +The predefined values have been calculated for the purposes of testing out +GitLab in the all-in-one VM. You don't need to change anything here, hit +**Create** to start the deployment. + +If you are deploying to production you will want to change the **GitLab instance +hostname** and use greater values for the volume sizes. If you don't provide a +password for PostgreSQL, it will be created automatically. + +>**Note:** +The `gitlab.apps.10.2.2.2.xip.io` hostname that is used by default will +resolve to the host with IP `10.2.2.2` which is the IP our VM uses. It is a +trick to have distinct FQDNs pointing to services that are on our local network. +Read more on how this works in <http://xip.io>. + +Now that we configured this, let's see how to manage and scale GitLab. + +## Manage and scale GitLab + +Setting up GitLab for the first time might take a while depending on your +internet connection and the resources you have attached to the all-in-one VM. +GitLab's docker image is quite big (~500MB), so you'll have to wait until +it's downloaded and configured before you use it. + +### Watch while GitLab gets deployed + +Navigate to the `gitlab` project at **Overview**. You can notice that the +deployment is in progress by the orange color. The Docker images are being +downloaded and soon they will be up and running. + + + +Switch to the **Browse > Pods** and you will eventually see all 3 pods in a +running status. Remember the 3 resources that were to be created when we first +created the GitLab app? This is where you can see them in action. + + + +You can see GitLab being reconfigured by taking look at the logs in realtime. +Click on `gitlab-ce-2-j7ioe` (your ID will be different) and go to the **Logs** +tab. + + + +At a point you should see a _**gitlab Reconfigured!**_ message in the logs. +Navigate back to the **Overview** and hopefully all pods will be up and running. + + + +Congratulations! You can now navigate to your new shinny GitLab instance by +visiting <http://gitlab.apps.10.2.2.2.xip.io> where you will be asked to +change the root user password. Login using `root` as username and providing the +password you just set, and start using GitLab! + +### Scale GitLab with the push of a button + +If you reach to a point where your GitLab instance could benefit from a boost +of resources, you'd be happy to know that you can scale up with the push of a +button. + +In the **Overview** page just click the up arrow button in the pod where +GitLab is. The change is instant and you can see the number of [replicas] now +running scaled to 2. + + + +Upping the GitLab pods is actually like adding new application servers to your +cluster. You can see how that would work if you didn't use GitLab with +OpenShift by following the [HA documentation][ha] for the application servers. + +Bare in mind that you may need more resources (CPU, RAM, disk space) when you +scale up. If a pod is in pending state for too long, you can navigate to +**Browse > Events** and see the reason and message of the state. + + + +### Scale GitLab using the `oc` CLI + +Using `oc` is super easy to scale up the replicas of a pod. You may want to +skim through the [basic CLI operations][basic-cli] to get a taste how the CLI +commands are used. Pay extra attention to the object types as we will use some +of them and their abbreviated versions below. + +In order to scale up, we need to find out the name of the replication controller. +Let's see how to do that using the following steps. + +1. Make sure you are in the `gitlab` project: + + ```sh + oc project gitlab + ``` + +1. See what services are used for this project: + + ```sh + oc get svc + ``` + + The output will be similar to: + + ``` + NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE + gitlab-ce 172.30.243.177 <none> 22/TCP,80/TCP 5d + gitlab-ce-postgresql 172.30.116.75 <none> 5432/TCP 5d + gitlab-ce-redis 172.30.105.88 <none> 6379/TCP 5d + ``` + +1. We need to see the replication controllers of the `gitlab-ce` service. + Get a detailed view of the current ones: + + ```sh + oc describe rc gitlab-ce + ``` + + This will return a large detailed list of the current replication controllers. + Search for the name of the GitLab controller, usually `gitlab-ce-1` or if + that failed at some point and you spawned another one, it will be named + `gitlab-ce-2`. + +1. Scale GitLab using the previous information: + + ```sh + oc scale --replicas=2 replicationcontrollers gitlab-ce-2 + ``` + +1. Get the new replicas number to make sure scaling worked: + + ```sh + oc get rc gitlab-ce-2 + ``` + + which will return something like: + + ``` + NAME DESIRED CURRENT AGE + gitlab-ce-2 2 2 5d + ``` + +And that's it! We successfully scaled the replicas to 2 using the CLI. + +As always, you can find the name of the controller using the web console. Just +click on the service you are interested in and you will see the details in the +right sidebar. + + + +### Autoscaling GitLab + +In case you were wondering whether there is an option to autoscale a pod based +on the resources of your server, the answer is yes, of course there is. + +We will not expand on this matter, but feel free to read the documentation on +OpenShift's website about [autoscaling]. + +## Current limitations + +As stated in the [all-in-one VM][vm] page: + +> By default, OpenShift will not allow a container to run as root or even a +non-random container assigned userid. Most Docker images in the Dockerhub do not +follow this best practice and instead run as root. + +The all-in-one VM we are using has this security turned off so it will not +bother us. In any case, it is something to keep in mind when deploying GitLab +on a production cluster. + +In order to deploy GitLab on a production cluster, you will need to assign the +GitLab service account to the `anyuid` Security Context. + +1. Edit the Security Context: + ```sh + oc edit scc anyuid + ``` + +1. Add `system:serviceaccount:<project>:gitlab-ce-user` to the `users` section. + If you changed the Application Name from the default the user will + will be `<app-name>-user` instead of `gitlab-ce-user` + +1. Save and exit the editor + +## Conclusion + +By now, you should have an understanding of the basic OpenShift Origin concepts +and a sense of how things work using the web console or the CLI. + +GitLab was hard to install in previous versions of OpenShift, +but now that belongs to the past. Upload a template, create a project, add an +application and you are done. You are ready to login to your new GitLab instance. + +And remember that in this tutorial we just scratched the surface of what Origin +is capable of. As always, you can refer to the detailed +[documentation][openshift-docs] to learn more about deploying your own OpenShift +PaaS and managing your applications with the ease of containers. + +[RedHat]: https://www.redhat.com/en "RedHat website" +[openshift]: https://www.openshift.org "OpenShift Origin website" +[vm]: https://www.openshift.org/vm/ "OpenShift All-in-one VM" +[vm-new]: https://atlas.hashicorp.com/openshift/boxes/origin-all-in-one "Official OpenShift Vagrant box on Atlas" +[template]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/docker/openshift-template.json "OpenShift template for GitLab" +[openshift.com]: https://openshift.com "OpenShift Online" +[kubernetes]: http://kubernetes.io/ "Kubernetes website" +[Docker]: https://www.docker.com "Docker website" +[oc]: https://docs.openshift.org/latest/cli_reference/get_started_cli.html "Documentation - oc CLI documentation" +[VirtualBox]: https://www.virtualbox.org/wiki/Downloads "VirtualBox downloads" +[Vagrant]: https://www.vagrantup.com/downloads.html "Vagrant downloads" +[projects]: https://docs.openshift.org/latest/dev_guide/projects.html "Documentation - Projects overview" +[core]: https://docs.openshift.org/latest/architecture/core_concepts/index.html "Documentation - Core concepts of OpenShift Origin" +[templates]: https://docs.openshift.org/latest/architecture/core_concepts/templates.html "Documentation - OpenShift templates" +[old-post]: https://blog.openshift.com/deploy-gitlab-openshift/ "Old post - Deploy GitLab on OpenShift" +[line]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/658c065c8d022ce858dd63eaeeadb0b2ddc8deea/docker/openshift-template.json#L239 "GitLab - OpenShift template" +[oc-gh]: https://github.com/openshift/origin/releases/tag/v1.3.0 "Openshift 1.3.0 release on GitHub" +[ha]: http://docs.gitlab.com/ce/administration/high_availability/gitlab.html "Documentation - GitLab High Availability" +[replicas]: https://docs.openshift.org/latest/architecture/core_concepts/deployments.html#replication-controllers "Documentation - Replication controller" +[autoscaling]: https://docs.openshift.org/latest/dev_guide/pod_autoscaling.html "Documentation - Autoscale" +[basic-cli]: https://docs.openshift.org/latest/cli_reference/basic_cli_operations.html "Documentation - Basic CLI operations" +[openshift-docs]: https://docs.openshift.org "OpenShift documentation" diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index e67db9ff142..f83a60e49e8 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -7,6 +7,11 @@ storing data in a single JSON column the data is stored in a separate table. ## When To Use Background Migrations +>**Note:** +When adding background migrations _you must_ make sure they are announced in the +monthly release post along with an estimate of how long it will take to complete +the migrations. + In the vast majority of cases you will want to use a regular Rails migration instead. Background migrations should _only_ be used when migrating _data_ in tables that have so many rows this process would take hours when performed in a @@ -91,6 +96,10 @@ BackgroundMigrationWorker.perform_bulk_in(5.minutes, jobs) ## Cleaning Up +>**Note:** +Cleaning up any remaining background migrations _must_ be done in either a major +or minor release, you _must not_ do this in a patch release. + Because background migrations can take a long time you can't immediately clean things up after scheduling them. For example, you can't drop a column that's used in the migration process as this would cause jobs to fail. This means that diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 149a0159680..6ade3231fac 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -11,7 +11,7 @@ See [our current .eslintrc][eslintrc] for specific rules and patterns. #### ESlint -1. **Never** disable eslint rules unless you have a good reason. +1. **Never** disable eslint rules unless you have a good reason. You may see a lot of legacy files with `/* eslint-disable some-rule, some-other-rule */` at the top, but legacy files are a special case. Any time you develop a new feature or refactor an existing one, you should abide by the eslint rules. @@ -100,26 +100,44 @@ followed by any global declarations, then a blank newline prior to any imports o export default Foo; ``` -1. Relative paths: Unless you are writing a test, always reference other scripts using -relative paths instead of `~` - * In **app/assets/javascripts**: +1. Relative paths: when importing a module in the same directory, a child +directory, or an immediate parent directory prefer relative paths. When +importing a module which is two or more levels up, prefer either `~/` or `ee/` +. - ```javascript - // bad - import Foo from '~/foo' +In **app/assets/javascripts/my-feature/subdir**: - // good - import Foo from '../foo'; - ``` - * In **spec/javascripts**: +``` javascript +// bad +import Foo from '~/my-feature/foo'; +import Bar from '~/my-feature/subdir/bar'; +import Bin from '~/my-feature/subdir/lib/bin'; - ```javascript - // bad - import Foo from '../../app/assets/javascripts/foo' +// good +import Foo from '../foo'; +import Bar from './bar'; +import Bin from './lib/bin'; +``` - // good - import Foo from '~/foo'; - ``` +In **spec/javascripts**: + +``` javascript +// bad +import Foo from '../../app/assets/javascripts/my-feature/foo'; + +// good +import Foo from '~/my-feature/foo'; +``` + +When referencing an **EE component**: + +``` javascript +// bad +import Foo from '../../../../../ee/app/assets/javascripts/my-feature/ee-foo'; + +// good +import Foo from 'ee/my-feature/foo'; +``` 1. Avoid using IIFE. Although we have a lot of examples of files which wrap their contents in IIFEs (immediately-invoked function expressions), @@ -465,7 +483,7 @@ A forEach will cause side effects, it will be mutating the array being iterated. #### Vue and Boostrap 1. Tooltips: Do not rely on `has-tooltip` class name for Vue components - ``` javascript + ```javascript // bad <span class="has-tooltip" diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 161d2544169..9b8ab5da74e 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -35,12 +35,11 @@ Please don't depend on GitLab-specific code since it can change in future versions. If needed copy-paste GitLab code into the migration to make it forward compatible. -## Commit Guidelines +## Schema Changes -Each migration **must** be added in its own commit with a descriptive commit -message. If a commit adds a migration it _should only_ include the migration and -any corresponding changes to `db/schema.rb`. This makes it easy to revert a -database migration without accidentally reverting other changes. +Migrations that make changes to the database schema (e.g. adding a column) can +only be added in the monthly release, patch releases may only contain data +migrations _unless_ schema changes are absolutely required to solve a problem. ## Downtime Tagging @@ -224,9 +223,9 @@ add_column(:projects, :foo, :integer, default: 10, limit: 8) ## Timestamp column type -By default, Rails uses the `timestamp` data type that stores timestamp data without timezone information. -The `timestamp` data type is used by calling either the `add_timestamps` or the `timestamps` method. -Also Rails converts the `:datetime` data type to the `timestamp` one. +By default, Rails uses the `timestamp` data type that stores timestamp data without timezone information. +The `timestamp` data type is used by calling either the `add_timestamps` or the `timestamps` method. +Also Rails converts the `:datetime` data type to the `timestamp` one. Example: diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 42bb5e8619c..bfd80aab6a4 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -146,3 +146,20 @@ If new emoji are added, the spritesheet may change size. To compensate for such changes, first generate the `emoji.png` spritesheet with the above Rake task, then check the dimensions of the new spritesheet and update the `SPRITESHEET_WIDTH` and `SPRITESHEET_HEIGHT` constants accordingly. + +## Updating project templates + +Starting a project from a template needs this project to be exported. On a +up to date master branch with run: + +``` +gdk run db +# In a new terminal window +bundle exec rake gitlab:update_project_templates +git checkout -b update-project-templates +git add vendor/project_templates +git commit +git push -u origin update-project-templates +``` + +Now create a merge request and merge that to master. diff --git a/doc/install/installation.md b/doc/install/installation.md index 8ded607bcab..22aedb5403e 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -168,8 +168,10 @@ are out of date, so we'll need to install through the following commands: curl --location https://deb.nodesource.com/setup_7.x | sudo bash - sudo apt-get install -y nodejs - # install yarn - curl --location https://yarnpkg.com/install.sh | bash - + curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - + echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list + sudo apt-get update + sudo apt-get install yarn Visit the official websites for [node](https://nodejs.org/en/download/package-manager/) and [yarn](https://yarnpkg.com/en/docs/install/) if you have any trouble with these steps. diff --git a/doc/update/8.17-to-9.0.md b/doc/update/8.17-to-9.0.md index 6308317b1f2..4d3ababaa41 100644 --- a/doc/update/8.17-to-9.0.md +++ b/doc/update/8.17-to-9.0.md @@ -65,7 +65,10 @@ Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage JavaScript dependencies. ```bash -curl --location https://yarnpkg.com/install.sh | bash - +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn ``` More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). diff --git a/doc/update/9.0-to-9.1.md b/doc/update/9.0-to-9.1.md index 2d597894517..2b4a7bed27f 100644 --- a/doc/update/9.0-to-9.1.md +++ b/doc/update/9.0-to-9.1.md @@ -65,7 +65,10 @@ Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage JavaScript dependencies. ```bash -curl --location https://yarnpkg.com/install.sh | bash - +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn ``` More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). diff --git a/doc/update/9.1-to-9.2.md b/doc/update/9.1-to-9.2.md index 225a4dcc924..f38547bba1a 100644 --- a/doc/update/9.1-to-9.2.md +++ b/doc/update/9.1-to-9.2.md @@ -65,7 +65,10 @@ Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage JavaScript dependencies. ```bash -curl --location https://yarnpkg.com/install.sh | bash - +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn ``` More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md index 910539acc70..373f43eb3bb 100644 --- a/doc/update/9.2-to-9.3.md +++ b/doc/update/9.2-to-9.3.md @@ -65,7 +65,10 @@ Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage JavaScript dependencies. ```bash -curl --location https://yarnpkg.com/install.sh | bash - +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn ``` More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). diff --git a/doc/update/9.3-to-9.4.md b/doc/update/9.3-to-9.4.md index 9540c36e7d0..b167f0737aa 100644 --- a/doc/update/9.3-to-9.4.md +++ b/doc/update/9.3-to-9.4.md @@ -65,7 +65,10 @@ Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage JavaScript dependencies. ```bash -curl --location https://yarnpkg.com/install.sh | bash - +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn ``` More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). diff --git a/doc/update/9.4-to-9.5.md b/doc/update/9.4-to-9.5.md new file mode 100644 index 00000000000..fc87b2d0f1e --- /dev/null +++ b/doc/update/9.4-to-9.5.md @@ -0,0 +1,352 @@ +# From 9.4 to 9.5 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.3.tar.gz +echo '1014ee699071aa2ddd501907d18cbe15399c997d ruby-2.3.3.tar.gz' | shasum -c - && tar xzf ruby-2.3.3.tar.gz +cd ruby-2.3.3 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets and +it has a minimum requirement of node v4.3.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v4.3.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 9-5-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 9-5-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell'. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-9.5 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/9-4-stable:config/gitlab.yml.example origin/9-5-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/9-4-stable:lib/support/nginx/gitlab-ssl origin/9-5-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/9-4-stable:lib/support/nginx/gitlab origin/9-5-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-5-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-4-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/9-4-stable:lib/support/init.d/gitlab.default.example origin/9-5-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (9.4) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 9.3 to 9.4](9.3-to-9.4.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-5-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-5-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/update/README.md b/doc/update/README.md index 22dbc7c750f..c98e20686e0 100644 --- a/doc/update/README.md +++ b/doc/update/README.md @@ -34,17 +34,67 @@ update them are in [a separate document][omnidocker]. ## Upgrading without downtime -Starting with GitLab 9.1.0 it's possible to upgrade to a newer major, minor, or patch version of GitLab -without having to take your GitLab instance offline. However, for this to work -there are the following requirements: - -1. You can only upgrade 1 minor release at a time. So from 9.1 to 9.2, not to 9.3. -2. You have to be on the most recent patch release. For example, if 9.1.15 is the last - release of 9.1 then you can safely upgrade from that version to any 9.2.x version. - However, if you are running 9.1.14 you first need to upgrade to 9.1.15. +Starting with GitLab 9.1.0 it's possible to upgrade to a newer major, minor, or +patch version of GitLab without having to take your GitLab instance offline. +However, for this to work there are the following requirements: + +1. You can only upgrade 1 minor release at a time. So from 9.1 to 9.2, not to + 9.3. 2. You have to use [post-deployment migrations](../development/post_deployment_migrations.md). -3. You are using PostgreSQL. If you are using MySQL please look at the release post to see if downtime is required. +3. You are using PostgreSQL. If you are using MySQL please look at the release + post to see if downtime is required. + +Most of the time you can safely upgrade from a patch release to the next minor +release if the patch release is not the latest. For example, upgrading from +9.1.1 to 9.2.0 should be safe even if 9.1.2 has been released. We do recommend +you check the release posts of any releases between your current and target +version just in case they include any migrations that may require you to upgrade +1 release at a time. + +Some releases may also include so called "background migrations". These +migrations are performed in the background by Sidekiq and are often used for +migrating data. Background migrations are only added in the monthly releases. + +Certain major/minor releases may require a set of background migrations to be +finished. To guarantee this such a release will process any remaining jobs +before continuing the upgrading procedure. While this won't require downtime +(if the above conditions are met) we recommend users to keep at least 1 week +between upgrading major/minor releases, allowing the background migrations to +finish. The time necessary to complete these migrations can be reduced by +increasing the number of Sidekiq workers that can process jobs in the +`background_migration` queue. + +As a rule of thumb, any database smaller than 10 GB won't take too much time to +upgrade; perhaps an hour at most per minor release. Larger databases however may +require more time, but this is highly dependent on the size of the database and +the migrations that are being performed. + +### Examples + +To help explain this, let's look at some examples. + +**Example 1:** You are running a large GitLab installation using version 9.4.2, +which is the latest patch release of 9.4. When GitLab 9.5.0 is released this +installation can be safely upgraded to 9.5.0 without requiring downtime if the +requirements mentioned above are met. You can also skip 9.5.0 and upgrade to +9.5.1 once it's released, but you **can not** upgrade straight to 9.6.0; you +_have_ to first upgrade to a 9.5.x release. + +**Example 2:** You are running a large GitLab installation using version 9.4.2, +which is the latest patch release of 9.4. GitLab 9.5 includes some background +migrations, and 10.0 will require these to be completed (processing any +remaining jobs for you). Skipping 9.5 is not possible without downtime, and due +to the background migrations would require potentially hours of downtime +depending on how long it takes for the background migrations to complete. To +work around this you will have to upgrade to 9.5.x first, then wait at least a +week before upgrading to 10.0. + +**Example 3:** You use MySQL as the database for GitLab. Any upgrade to a new +major/minor release will require downtime. If a release includes any background +migrations this could potentially lead to hours of downtime, depending on the +size of your database. To work around this you will have to use PostgreSQL and +meet the other online upgrade requirements mentioned above. ## Upgrading between editions diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 0d29b471d52..b42b8f0a525 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -248,7 +248,7 @@ GFM will recognize the following: | `~123` | label by ID | | `~bug` | one-word label by name | | `~"feature request"` | multi-word label by name | -| `%123` | milestone by ID | +| `%123` | project milestone by ID | | `%v1.23` | one-word milestone by name | | `%"release candidate"` | multi-word milestone by name | | `9ba12248` | specific commit | @@ -262,7 +262,7 @@ GFM also recognizes certain cross-project references: |:----------------------------------------|:------------------------| | `namespace/project#123` | issue | | `namespace/project!123` | merge request | -| `namespace/project%123` | milestone | +| `namespace/project%123` | project milestone | | `namespace/project$123` | snippet | | `namespace/project@9ba12248` | specific commit | | `namespace/project@9ba12248...b19a04f5` | commit range comparison | @@ -274,7 +274,7 @@ It also has a shorthand version to reference other projects from the same namesp |:------------------------------|:------------------------| | `project#123` | issue | | `project!123` | merge request | -| `project%123` | milestone | +| `project%123` | project milestone | | `project$123` | snippet | | `project@9ba12248` | specific commit | | `project@9ba12248...b19a04f5` | commit range comparison | diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index c03a2df9a72..47eb0b34f66 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -438,7 +438,6 @@ X-Gitlab-Event: Note Hook "iid": 1, "description": "Et voluptas corrupti assumenda temporibus. Architecto cum animi eveniet amet asperiores. Vitae numquam voluptate est natus sit et ad id.", "position": 0, - "locked_at": null, "source":{ "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 23ffde4e8bd..876b98a4dc5 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -56,4 +56,5 @@ total merge requests and issues. ## Quick actions -[Quick actions](../quick_actions.md) are available for assigning and removing project milestones only. [In the future](https://gitlab.com/gitlab-org/gitlab-ce/issues/34778), this will also apply to group milestones. +[Quick actions](../quick_actions.md) are available for assigning and removing +project and group milestones. |
