diff options
Diffstat (limited to 'doc')
32 files changed, 1201 insertions, 272 deletions
diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index f9bc4f59345..f532a106bc6 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -30,38 +30,6 @@ Omnibus GitLab packages. [Available configuration setups](#available-configuration-setups) section below. -<!-- START doctoc generated TOC please keep comment here to allow auto update --> -<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> -**Table of Contents** - -- [Overview](#overview) - - [High Availability with Sentinel](#high-availability-with-sentinel) - - [Recommended setup](#recommended-setup) - - [Redis setup overview](#redis-setup-overview) - - [Sentinel setup overview](#sentinel-setup-overview) - - [Available configuration setups](#available-configuration-setups) -- [Configuring Redis HA](#configuring-redis-ha) - - [Prerequisites](#prerequisites) - - [Step 1. Configuring the master Redis instance](#step-1-configuring-the-master-redis-instance) - - [Step 2. Configuring the slave Redis instances](#step-2-configuring-the-slave-redis-instances) - - [Step 3. Configuring the Redis Sentinel instances](#step-3-configuring-the-redis-sentinel-instances) - - [Step 4. Configuring the GitLab application](#step-4-configuring-the-gitlab-application) -- [Switching from an existing single-machine installation to Redis HA](#switching-from-an-existing-single-machine-installation-to-redis-ha) -- [Example of a minimal configuration with 1 master, 2 slaves and 3 Sentinels](#example-of-a-minimal-configuration-with-1-master-2-slaves-and-3-sentinels) - - [Example configuration for Redis master and Sentinel 1](#example-configuration-for-redis-master-and-sentinel-1) - - [Example configuration for Redis slave 1 and Sentinel 2](#example-configuration-for-redis-slave-1-and-sentinel-2) - - [Example configuration for Redis slave 2 and Sentinel 3](#example-configuration-for-redis-slave-2-and-sentinel-3) - - [Example configuration for the GitLab application](#example-configuration-for-the-gitlab-application) -- [Advanced configuration](#advanced-configuration) - - [Control running services](#control-running-services) -- [Troubleshooting](#troubleshooting) - - [Troubleshooting Redis replication](#troubleshooting-redis-replication) - - [Troubleshooting Sentinel](#troubleshooting-sentinel) -- [Changelog](#changelog) -- [Further reading](#further-reading) - -<!-- END doctoc generated TOC please keep comment here to allow auto update --> - ## Overview Before diving into the details of setting up Redis and Redis Sentinel for HA, @@ -107,10 +75,12 @@ to help keep servers online with minimal to no downtime. Redis Sentinel: - Promotes a **Slave** to **Master** when the **Master** fails - Demotes a **Master** to **Slave** when the failed **Master** comes back online (to prevent data-partitioning) -- Can be queried by clients to always connect to the current **Master** server +- Can be queried by the application to always connect to the current **Master** + server -When a **Master** fails to respond, it's the client's responsibility to handle -timeout and reconnect (querying a **Sentinel** for a new **Master**). +When a **Master** fails to respond, it's the application's responsibility +(in our case GitLab) to handle timeout and reconnect (querying a **Sentinel** +for a new **Master**). To get a better understanding on how to correctly setup Sentinel, please read the [Redis Sentinel documentation](http://redis.io/topics/sentinel) first, as @@ -289,12 +259,7 @@ The prerequisites for a HA Redis setup are the following: ### Step 1. Configuring the master Redis instance -1. SSH into the **master** Redis server and login as root: - - ``` - sudo -i - ``` - +1. SSH into the **master** Redis server. 1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab package you want using **steps 1 and 2** from the GitLab downloads page. - Make sure you select the correct Omnibus package, with the same version @@ -334,12 +299,7 @@ The prerequisites for a HA Redis setup are the following: ### Step 2. Configuring the slave Redis instances -1. SSH into the **slave** Redis server and login as root: - - ``` - sudo -i - ``` - +1. SSH into the **slave** Redis server. 1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab package you want using **steps 1 and 2** from the GitLab downloads page. - Make sure you select the correct Omnibus package, with the same version @@ -417,12 +377,7 @@ multiple machines with the Sentinel daemon. --- -1. SSH into the server that will host Redis Sentinel and login as root: - - ``` - sudo -i - ``` - +1. SSH into the server that will host Redis Sentinel. 1. **You can omit this step if the Sentinels will be hosted in the same node as the other Redis instances.** @@ -437,7 +392,6 @@ multiple machines with the Sentinel daemon. Sentinels in the same node as the other Redis instances, some values might be duplicate below): - ```ruby redis_sentinel_role['enable'] = true @@ -530,6 +484,7 @@ it needs to access at least one of the listed. The following steps should be performed in the [GitLab application server](gitlab.md) which ideally should not have Redis or Sentinels on it for a HA setup. +1. SSH into the server where the GitLab application is installed. 1. Edit `/etc/gitlab/gitlab.rb` and add/change the following lines: ``` diff --git a/doc/administration/high_availability/redis_source.md b/doc/administration/high_availability/redis_source.md index 8558ba82d63..3629772b8af 100644 --- a/doc/administration/high_availability/redis_source.md +++ b/doc/administration/high_availability/redis_source.md @@ -17,27 +17,6 @@ If you're not sure whether this guide is for you, please refer to [Available configuration setups](redis.md#available-configuration-setups) in the Omnibus Redis HA documentation. ---- - -<!-- START doctoc generated TOC please keep comment here to allow auto update --> -<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> -**Table of Contents** - -- [Configuring your own Redis server](#configuring-your-own-redis-server) - - [Prerequisites](#prerequisites) - - [Step 1. Configuring the master Redis instance](#step-1-configuring-the-master-redis-instance) - - [Step 2. Configuring the slave Redis instances](#step-2-configuring-the-slave-redis-instances) - - [Step 3. Configuring the Redis Sentinel instances](#step-3-configuring-the-redis-sentinel-instances) - - [Step 4. Configuring the GitLab application](#step-4-configuring-the-gitlab-application) -- [Example of minimal configuration with 1 master, 2 slaves and 3 Sentinels](#example-of-minimal-configuration-with-1-master-2-slaves-and-3-sentinels) - - [Example configuration for Redis master and Sentinel 1](#example-configuration-for-redis-master-and-sentinel-1) - - [Example configuration for Redis slave 1 and Sentinel 2](#example-configuration-for-redis-slave-1-and-sentinel-2) - - [Example configuration for Redis slave 2 and Sentinel 3](#example-configuration-for-redis-slave-2-and-sentinel-3) - - [Example configuration of the GitLab application](#example-configuration-of-the-gitlab-application) -- [Troubleshooting](#troubleshooting) - -<!-- END doctoc generated TOC please keep comment here to allow auto update --> - ## Configuring your own Redis server This is the section where we install and setup the new Redis instances. diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md index 5a9a1582877..14cd7a03826 100644 --- a/doc/administration/reply_by_email.md +++ b/doc/administration/reply_by_email.md @@ -105,6 +105,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". gitlab_rails['incoming_email_mailbox_name'] = "inbox" + # The IDLE command timeout. + gitlab_rails['incoming_email_idle_timeout'] = 60 ``` ```ruby @@ -133,6 +135,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". gitlab_rails['incoming_email_mailbox_name'] = "inbox" + # The IDLE command timeout. + gitlab_rails['incoming_email_idle_timeout'] = 60 ``` 1. Reconfigure GitLab and restart mailroom for the changes to take effect: @@ -192,6 +196,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". mailbox: "inbox" + # The IDLE command timeout. + idle_timeout: 60 ``` ```yaml @@ -221,6 +227,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". mailbox: "inbox" + # The IDLE command timeout. + idle_timeout: 60 ``` 1. Enable `mail_room` in the init script at `/etc/default/gitlab`: @@ -277,6 +285,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". mailbox: "inbox" + # The IDLE command timeout. + idle_timeout: 60 ``` As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. diff --git a/doc/api/groups.md b/doc/api/groups.md index 45a3118f27a..5e6f498c365 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -6,8 +6,13 @@ Get a list of groups. (As user: my groups or all available, as admin: all groups Parameters: -- `all_available` (optional) - if passed, show all groups you have access to -- `skip_groups` (optional)(array of group IDs) - if passed, skip groups +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `skip_groups` | array of integers | no | Skip the group IDs passes | +| `all_available` | boolean | no | Show all the groups you have access to | +| `search` | string | no | Return list of authorized groups matching the search criteria | +| `order_by` | string | no | Order groups by `name` or `path`. Default is `name` | +| `sort` | string | no | Order groups in `asc` or `desc` order. Default is `asc` | ``` GET /groups diff --git a/doc/api/projects.md b/doc/api/projects.md index bbb3bfb4995..467a880ac13 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -850,7 +850,7 @@ POST /projects/:id/archive | `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/archive" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/archive" ``` Example response: @@ -939,7 +939,7 @@ POST /projects/:id/unarchive | `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/unarchive" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/unarchive" ``` Example response: diff --git a/doc/ci/README.md b/doc/ci/README.md index 6b90940c047..545cc72682d 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,6 +1,6 @@ -## GitLab CI Documentation +# GitLab CI Documentation -### CI User documentation +## CI User documentation - [Get started with GitLab CI](quick_start/README.md) - [CI examples for various languages](examples/README.md) @@ -20,4 +20,8 @@ - [API](../api/ci/README.md) - [CI services (linked docker containers)](services/README.md) - [CI/CD pipelines settings](../user/project/pipelines/settings.md) -- [**New CI build permissions model**](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds. +- [Review Apps](review_apps/index.md) + +## Breaking changes + +- [New CI build permissions model](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds. diff --git a/doc/ci/environments.md b/doc/ci/environments.md index e070302fb82..096c567c992 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -3,69 +3,523 @@ >**Note:** Introduced in GitLab 8.9. -## Environments +During the development of software, there can be many stages until it's ready +for public consumption. You sure want to first test your code and then deploy it +in a testing or staging environment before you release it to the public. That +way you can prevent bugs not only in your software, but in the deployment +process as well. -Environments are places where code gets deployed, such as staging or production. -CI/CD [Pipelines] usually have one or more [jobs] that deploy to an environment. -Defining environments in a project's `.gitlab-ci.yml` lets developers track -[deployments] to these environments. +GitLab CI is capable of not only testing or building your projects, but also +deploying them in your infrastructure, with the added benefit of giving you a +way to track your deployments. In other words, you can always know what is +currently being deployed or has been deployed on your servers. -## Deployments +## Overview -Deployments are created when [jobs] deploy versions of code to [environments]. +With environments, you can control the Continuous Deployment of your software +all within GitLab. All you need to do is define them in your project's +[`.gitlab-ci.yml`][yaml] as we will explore below. GitLab provides a full +history of your deployments per every environment. -### Checkout deployments locally +Environments are like tags for your CI jobs, describing where code gets deployed. +Deployments are created when [jobs] deploy versions of code to environments, +so every environment can have one or more deployments. GitLab keeps track of +your deployments, so you always know what is currently being deployed on your +servers. -Since 8.13, a reference in the git repository is saved for each deployment. So -knowing what the state is of your current environments is only a `git fetch` -away. +To better understand how environments and deployments work, let's consider an +example. We assume that you have already created a project in GitLab and set up +a Runner. The example will cover the following: -In your git config, append the `[remote "<your-remote>"]` block with an extra -fetch line: +- We are developing an application +- We want to run tests and build our app on all branches +- Our default branch is `master` +- We deploy the app only when a pipeline on `master` branch is run + +Let's see how it all ties together. + +## Defining environments +Let's consider the following `.gitlab-ci.yml` example: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master ``` -fetch = +refs/environments/*:refs/remotes/origin/environments/* + +We have defined 3 [stages](yaml/README.md#stages): + +- test +- build +- deploy + +The jobs assigned to these stages will run in this order. If a job fails, then +the builds that are assigned to the next stage won't run, rendering the pipeline +as failed. In our case, the `test` job will run first, then the `build` and +lastly the `deploy_staging`. With this, we ensure that first the tests pass, +then our app is able to be built successfully, and lastly we deploy to the +staging server. + +The `environment` keyword is just a hint for GitLab that this job actually +deploys to this environment's `name`. It can also have a `url` which, as we +will later see, is exposed in various places within GitLab. Each time a job that +has an environment specified and succeeds, a deployment is recorded, remembering +the Git SHA and environment name. + +To sum up, with the above `.gitlab-ci.yml` we have achieved that: + +- All branches will run the `test` and `build` jobs. +- The `deploy_staging` job will run [only](yaml/README.md#only) on the `master` + branch which means all merge requests that are created from branches don't + get to deploy to the staging server +- When a merge request is merged, all jobs will run and the `deploy_staging` + in particular will deploy our code to a staging server while the deployment + will be recorded in an environment named `staging`. + +Let's now see how that information is exposed within GitLab. + +## Viewing the current status of an environment + +The environment list under your project's **Pipelines ➔ Environments**, is +where you can find information of the last deployment status of an environment. + +Here's how the Environments page looks so far. + + + +There's a bunch of information there, specifically you can see: + +- The environment's name with a link to its deployments +- The last deployment ID number and who performed it +- The build ID of the last deployment with its respective job name +- The commit information of the last deployment such as who committed, to what + branch and the Git SHA of the commit +- The exact time the last deployment was performed +- A button that takes you to the URL that you have defined under the + `environment` keyword in `.gitlab-ci.yml` +- A button that re-deploys the latest deployment, meaning it runs the job + defined by the environment name for that specific commit + +>**Notes:** +- While you can create environments manually in the web interface, we recommend + that you define your environments in `.gitlab-ci.yml` first. They will + be automatically created for you after the first deploy. +- The environments page can only be viewed by Reporters and above. For more + information on the permissions, see the [permissions documentation][permissions]. +- Only deploys that happen after your `.gitlab-ci.yml` is properly configured + will show up in the "Environment" and "Last deployment" lists. + +The information shown in the Environments page is limited to the latest +deployments, but as you may have guessed an environment can have multiple +deployments. + +## Viewing the deployment history of an environment + +GitLab keeps track of your deployments, so you always know what is currently +being deployed on your servers. That way you can have the full history of your +deployments per every environment right in your browser. Clicking on an +environment will show the history of its deployments. Assuming you have deployed +multiple times already, here's how a specific environment's page looks like. + + + +We can see the same information as when in the Environments page, but this time +all deployments are shown. As you may have noticed, apart from the **Re-deploy** +button there are now **Rollback** buttons for each deployment. Let's see how +that works. + +## Rolling back changes + +You can't control everything, so sometimes things go wrong. When that unfortunate +time comes GitLab has you covered. Simply by clicking the **Rollback** button +that can be found in the deployments page +(**Pipelines ➔ Environments ➔ `environment name`**) you can relaunch the +job with the commit associated with it. + +>**Note:** +Bare in mind that your mileage will vary and it's entirely up to how you define +the deployment process in the job's `script` whether the rollback succeeds or not. +GitLab CI is just following orders. + +Thankfully that was the staging server that we had to rollback, and since we +learn from our mistakes, we decided to not make the same again when we deploy +to the production server. Enter manual actions for deployments. + +## Manually deploying to environments + +Turning a job from running automatically to a manual action is as simple as +adding `when: manual` to it. To expand on our previous example, let's add +another job that this time deploys our app to a production server and is +tracked by a `production` environment. The `.gitlab-ci.yml` looks like this +so far: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master + +deploy_prod: + stage: deploy + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + only: + - master ``` -## Defining environments +The `when: manual` action exposes a play button in GitLab's UI and the +`deploy_prod` job will only be triggered if and when we click that play button. +You can find it in the pipeline, build, environment, and deployment views. + +| Pipelines | Single pipeline | Environments | Deployments | Builds | +| --------- | ----------------| ------------ | ----------- | -------| +|  |  |  |  |  | + +Clicking on the play button in either of these places will trigger the +`deploy_prod` job, and the deployment will be recorded under a new +environment named `production`. + +>**Note:** +Remember that if your environment's name is `production` (all lowercase), then +it will get recorded in [Cycle Analytics](../user/project/cycle_analytics.md). +Double the benefit! + +While this is fine for deploying to some stable environments like staging or +production, what happens for branches? So far we haven't defined anything +regarding deployments for branches other than `master`. Dynamic environments +will help us achieve that. + +## Dynamic environments -You can create and delete environments manually in the web interface, but we -recommend that you define your environments in `.gitlab-ci.yml` first, which -will automatically create environments for you after the first deploy. +As the name suggests, it is possible to create environments on the fly by just +declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is +the basis of [Review apps](review_apps.md). -The `environment` is just a hint for GitLab that this job actually deploys to -this environment. Each time the job succeeds, a deployment is recorded, -remembering the git SHA and environment. +GitLab Runner exposes various [environment variables][variables] when a job runs, +and as such, you can use them as environment names. Let's add another job in +our example which will deploy to all branches except `master`: -Add something like this to your `.gitlab-ci.yml`: +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + only: + - branches + except: + - master ``` -production: + +Let's break it down in pieces. The job's name is `deploy_review` and it runs +on the `deploy` stage. The `script` at this point is fictional, you'd have to +use your own based on your deployment. Then, we set the `environment` with the +`environment:name` being `review/$CI_BUILD_REF_NAME`. Now that's an interesting +one. Since the [environment name][env-name] can contain also slashes (`/`), we +can use this pattern to distinguish between dynamic environments and the regular +ones. + +So, the first part is `review`, followed by a `/` and then `$CI_BUILD_REF_NAME` +which takes the value of the branch name. We also use the same +`$CI_BUILD_REF_NAME` value in the `environment:url` so that the environment +can get a specific and distinct URL for each branch. Again, the way you set up +the webserver to serve these requests is based on your setup. + +Last but not least, we tell the job to run [`only`][only] on branches +[`except`][only] master. + +>**Note:** +You are not bound to use the same prefix or only slashes in the dynamic +environments' names (`/`), but as we will see later, this will enable the +[grouping similar environments](#grouping-similar-environments) feature. + +The whole `.gitlab-ci.yml` looks like this so far: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + only: + - branches + except: + - master + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master + +deploy_prod: stage: deploy - script: dpl... - environment: production + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + only: + - master ``` -See full [documentation](yaml/README.md#environment). +A more realistic example would include copying files to a location where a +webserver (NGINX) could then read and serve. The example below will copy the +`public` directory to `/srv/nginx/$CI_BUILD_REF_NAME/public`: -## Seeing environment status +```yaml +review_app: + stage: deploy + script: + - rsync -av --delete public /srv/nginx/$CI_BUILD_REF_NAME + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com +``` -You can find the environment list under **Pipelines > Environments** for your -project. You'll see the git SHA and date of the last deployment to each -environment defined. +It is assumed that the user has already setup NGINX and GitLab Runner in the +server this job will run on. >**Note:** -Only deploys that happen after your `.gitlab-ci.yml` is properly configured will -show up in the environments and deployments lists. +Be sure to check out the [limitations](#limitations) section for some edge +cases regarding naming of you branches and Review Apps. + +--- + +The development workflow would now be: + +- Developer creates a branch locally +- Developer makes changes, commits and pushes the branch to GitLab +- Developer creates a merge request + +Behind the scenes: + +- GitLab Runner picks up the changes and starts running the jobs +- The jobs run sequentially as defined in `stages` + - First, the tests pass + - Then, the build begins and successfully also passes + - Lastly, the app is deployed to an environment with a name specific to the + branch + +So now, every branch gets its own environment and is deployed to its own place +with the added benefit of having a [history of deployments](#viewing-the-deployment-history-of-an-environment) +and also being able to [rollback changes](#rolling-back-changes) if needed. +Let's briefly see where URL that's defined in the environments is exposed. + +## Making use of the environment URL + +The environment URL is exposed in a few places within GitLab. + +| In a merge request widget as a link | In the Environments view as a button | In the Deployments view as a button | +| -------------------- | ------------ | ----------- | +|  |  |  | + +If a merge request is eventually merged to the default branch (in our case +`master`) and that branch also deploys to an environment (in our case `staging` +and/or `production`) you can see this information in the merge request itself. + + + +--- + +We now have a full development cycle, where our app is tested, built, deployed +as a Review app, deployed to a staging server once the merge request is merged, +and finally manually deployed to the production server. What we just described +is a single workflow, but imagine tens of developers working on a project +at the same time. They each push to their branches, and dynamic environments are +created all the time. In that case, we probably need to do some clean up. Read +next how environments can be stopped. + +## Stopping an environment + +By stopping an environment, you are effectively terminating its recording of the +deployments that happen in it. + +A branch is associated with an environment when the CI pipeline that is created +for this branch, was recently deployed to this environment. You can think of +the CI pipeline as the glue between the branch and the environment: +`branch ➔ CI pipeline ➔ environment`. + +There is a special case where environments can be manually stopped. That can +happen if you provide another job for that matter. The syntax is a little +tricky since a job calls another job to do the job. + +Consider the following example where the `deploy_review` calls the `stop_review` +to clean up and stop the environment: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + on_stop: stop_review + only: + - branches + except: + - master -## Seeing deployment history +stop_review: + variables: + GIT_STRATEGY: none + script: + - echo "Remove review app" + when: manual + environment: + name: review/$CI_BUILD_REF_NAME + action: stop +``` -Clicking on an environment will show the history of deployments. +Setting the [`GIT_STRATEGY`][git-strategy] to `none` is necessary on the +`stop_review` job so that the [GitLab Runner] won't try to checkout the code +after the branch is deleted. >**Note:** -Only deploys that happen after your `.gitlab-ci.yml` is properly configured will -show up in the environments and deployments lists. +Starting with GitLab 8.14, dynamic environments will be stopped automatically +when their associated branch is deleted. + +When you have an environment that has a stop action defined (typically when +the environment describes a review app), GitLab will automatically trigger a +stop action when the associated branch is deleted. + +You can read more in the [`.gitlab-ci.yml` reference][onstop]. + +## Grouping similar environments + +> [Introduced][ce-7015] in GitLab 8.14. + +As we've seen in the [dynamic environments](#dynamic-environments), you can +prepend their name with a word, then followed by a `/` and finally the branch +name which is automatically defined by the `CI_BUILD_REF_NAME` variable. + +In short, environments that are named like `type/foo` are presented under a +group named `type`. + +In our minimal example, we name the environments `review/$CI_BUILD_REF_NAME` +where `$CI_BUILD_REF_NAME` is the branch name: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME +``` + +In that case, if you visit the Environments page, and provided the branches +exist, you should see something like: + + + +## Checkout deployments locally + +Since 8.13, a reference in the git repository is saved for each deployment. So +knowing what the state is of your current environments is only a `git fetch` +away. + +In your git config, append the `[remote "<your-remote>"]` block with an extra +fetch line: + +``` +fetch = +refs/environments/*:refs/remotes/origin/environments/* +``` + +## Limitations + +1. If the branch name contains special characters (`/`), and you use the + `$CI_BUILD_REF_NAME` variable to dynamically create environments, there might + be complications during your Review Apps deployment. Follow the + [issue 22849][ce-22849] for more information. +1. You are limited to use only the [CI predefined variables][variables] in the + `environment: name`. If you try to re-use variables defined inside `script` + as part of the environment name, it will not work. + +## Further reading + +Below are some links you may find interesting: + +- [The `.gitlab-ci.yml` definition of environments](yaml/README.md#environment) +- [A blog post on Deployments & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) +- [Review Apps - Use dynamic environments to deploy your code for every branch](review_apps/index.md) [Pipelines]: pipelines.md [jobs]: yaml/README.md#jobs +[yaml]: yaml/README.md [environments]: #environments [deployments]: #deployments +[permissions]: ../user/permissions.md +[variables]: variables/README.md +[env-name]: yaml/README.md#environment-name +[only]: yaml/README.md#only-and-except +[onstop]: yaml/README.md#environment-on_stop +[ce-7015]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7015 +[ce-22849]: https://gitlab.com/gitlab-org/gitlab-ce/issues/22849 +[gitlab runner]: https://docs.gitlab.com/runner/ +[git-strategy]: yaml/README.md#git-strategy diff --git a/doc/ci/img/deployments_view.png b/doc/ci/img/deployments_view.png Binary files differnew file mode 100644 index 00000000000..ca6097cbea4 --- /dev/null +++ b/doc/ci/img/deployments_view.png diff --git a/doc/ci/img/environments_available_staging.png b/doc/ci/img/environments_available_staging.png Binary files differnew file mode 100644 index 00000000000..784c4fd944c --- /dev/null +++ b/doc/ci/img/environments_available_staging.png diff --git a/doc/ci/img/environments_dynamic_groups.png b/doc/ci/img/environments_dynamic_groups.png Binary files differnew file mode 100644 index 00000000000..e89b66c502c --- /dev/null +++ b/doc/ci/img/environments_dynamic_groups.png diff --git a/doc/ci/img/environments_link_url.png b/doc/ci/img/environments_link_url.png Binary files differnew file mode 100644 index 00000000000..224c21adfb5 --- /dev/null +++ b/doc/ci/img/environments_link_url.png diff --git a/doc/ci/img/environments_link_url_deployments.png b/doc/ci/img/environments_link_url_deployments.png Binary files differnew file mode 100644 index 00000000000..9419668a9bd --- /dev/null +++ b/doc/ci/img/environments_link_url_deployments.png diff --git a/doc/ci/img/environments_link_url_mr.png b/doc/ci/img/environments_link_url_mr.png Binary files differnew file mode 100644 index 00000000000..3276dfb6096 --- /dev/null +++ b/doc/ci/img/environments_link_url_mr.png diff --git a/doc/ci/img/environments_manual_action_builds.png b/doc/ci/img/environments_manual_action_builds.png Binary files differnew file mode 100644 index 00000000000..d4bb7ccdbae --- /dev/null +++ b/doc/ci/img/environments_manual_action_builds.png diff --git a/doc/ci/img/environments_manual_action_deployments.png b/doc/ci/img/environments_manual_action_deployments.png Binary files differnew file mode 100644 index 00000000000..c2477381c80 --- /dev/null +++ b/doc/ci/img/environments_manual_action_deployments.png diff --git a/doc/ci/img/environments_manual_action_environments.png b/doc/ci/img/environments_manual_action_environments.png Binary files differnew file mode 100644 index 00000000000..56601c0db2d --- /dev/null +++ b/doc/ci/img/environments_manual_action_environments.png diff --git a/doc/ci/img/environments_manual_action_pipelines.png b/doc/ci/img/environments_manual_action_pipelines.png Binary files differnew file mode 100644 index 00000000000..eb6e87cd956 --- /dev/null +++ b/doc/ci/img/environments_manual_action_pipelines.png diff --git a/doc/ci/img/environments_manual_action_single_pipeline.png b/doc/ci/img/environments_manual_action_single_pipeline.png Binary files differnew file mode 100644 index 00000000000..9713ad212e2 --- /dev/null +++ b/doc/ci/img/environments_manual_action_single_pipeline.png diff --git a/doc/ci/img/environments_mr_review_app.png b/doc/ci/img/environments_mr_review_app.png Binary files differnew file mode 100644 index 00000000000..a2ae25d62fa --- /dev/null +++ b/doc/ci/img/environments_mr_review_app.png diff --git a/doc/ci/img/environments_view.png b/doc/ci/img/environments_view.png Binary files differnew file mode 100644 index 00000000000..131a9718cc4 --- /dev/null +++ b/doc/ci/img/environments_view.png diff --git a/doc/ci/review_apps/img/review_apps_preview_in_mr.png b/doc/ci/review_apps/img/review_apps_preview_in_mr.png Binary files differnew file mode 100644 index 00000000000..15bcb90518c --- /dev/null +++ b/doc/ci/review_apps/img/review_apps_preview_in_mr.png diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md new file mode 100644 index 00000000000..a66165dc973 --- /dev/null +++ b/doc/ci/review_apps/index.md @@ -0,0 +1,124 @@ +# Getting started with Review Apps + +> +- [Introduced][ce-21971] in GitLab 8.12. Further additions were made in GitLab + 8.13 and 8.14. +- Inspired by [Heroku's Review Apps][heroku-apps] which itself was inspired by + [Fourchette]. + +The basis of Review Apps is the [dynamic environments] which allow you to create +a new environment (dynamically) for each one of your branches. + +A Review App can then be visible as a link when you visit the [merge request] +relevant to the branch. That way, you are able to see live all changes introduced +by the merge request changes. Reviewing anything, from performance to interface +changes, becomes much easier with a live environment and as such, Review Apps +can make a huge impact on your development flow. + +They mostly make sense to be used with web applications, but you can use them +any way you'd like. + +## Overview + +Simply put, a Review App is a mapping of a branch with an environment as there +is a 1:1 relation between them. + +Here's an example of what it looks like when viewing a merge request with a +dynamically set environment. + + + +In the image above you can see that the `add-new-line` branch was successfully +built and deployed under a dynamic environment and can be previewed with an +also dynamically URL. + +The details of the Review Apps implementation depend widely on your real +technology stack and on your deployment process. The simplest case it to +deploy a simple static HTML website, but it will not be that straightforward +when your app is using a database for example. To make a branch be deployed +on a temporary instance and booting up this instance with all required software +and services automatically on the fly is not a trivial task. However, it is +doable, especially if you use Docker, or at least a configuration management +tool like Chef, Puppet, Ansible or Salt. + +## Prerequisites + +To get a better understanding of Review Apps, you must first learn how +environments and deployments work. The following docs will help you grasp that +knowledge: + +1. First, learn about [environments][] and their role in the development workflow. +1. Then make a small stop to learn about [CI variables][variables] and how they + can be used in your CI jobs. +1. Next, explore the [`environment` syntax][yaml-env] as defined in `.gitlab-ci.yml`. + This will be your primary reference when you are finally comfortable with + how environments work. +1. Additionally, find out about [manual actions][] and how you can use them to + deploy to critical environments like production with the push of a button. +1. And as a last step, follow the [example tutorials](#examples) which will + guide you step by step to set up the infrastructure and make use of + Review Apps. + +## Configuration + +The configuration of Review apps depends on your technology stack and your +infrastructure. Read the [dynamic environments] documentation to understand +how to define and create them. + +## Creating and destroying Review Apps + +The creation and destruction of a Review App is defined in `.gitlab-ci.yml` +at a job level under the `environment` keyword. + +Check the [environments] documentation how to do so. + +## A simple workflow + +The process of adding Review Apps in your workflow would look like: + +1. Set up the infrastructure to host and deploy the Review Apps. +1. [Install][install-runner] and [configure][conf-runner] a Runner that does + the deployment. +1. Set up a job in `.gitlab-ci.yml` that uses the predefined + [predefined CI environment variable][variables] `${CI_BUILD_REF_NAME}` to + create dynamic environments and restrict it to run only on branches. +1. Optionally set a job that [manually stops][manual-env] the Review Apps. + +From there on, you would follow the branched Git flow: + +1. Push a branch and let the Runner deploy the Review App based on the `script` + definition of the dynamic environment job. +1. Wait for the Runner to build and/or deploy your web app. +1. Click on the link that's present in the MR related to the branch and see the + changes live. + +## Limitations + +Check the [environments limitations](../environments.md#limitations). + +## Examples + +A list of examples used with Review Apps can be found below: + +- [Use with NGINX][app-nginx] - Use NGINX and the shell executor of GitLab Runner + to deploy a simple HTML website. + +And below is a soon to be added examples list: + +- Use with Amazon S3 +- Use on Heroku with dpl +- Use with OpenShift/kubernetes + +[app-nginx]: https://gitlab.com/gitlab-examples/review-apps-nginx +[ce-21971]: https://gitlab.com/gitlab-org/gitlab-ce/issues/21971 +[dynamic environments]: ../environments.md#dynamic-environments +[environments]: ../environments.md +[fourchette]: https://github.com/rainforestapp/fourchette +[heroku-apps]: https://devcenter.heroku.com/articles/github-integration-review-apps +[manual actions]: ../environments.md#manual-actions +[merge request]: ../../user/project/merge_requests.md +[variables]: ../variables/README.md +[yaml-env]: ../yaml/README.md#environment +[install-runner]: https://docs.gitlab.com/runner/install/ +[conf-runner]: https://docs.gitlab.com/runner/commands/ +[manual-env]: ../environments.md#stopping-an-environment diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 5c0e1c44e3f..6fee750c709 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -541,6 +541,8 @@ same manual action multiple times. An example usage of manual actions is deployment to production. +Read more at the [environments documentation][env-manual]. + ### environment > Introduced in GitLab 8.9. @@ -552,28 +554,14 @@ An example usage of manual actions is deployment to production. If `environment` is specified and no environment under that name exists, a new one will be created automatically. -The `environment` name can contain: - -- letters -- digits -- spaces -- `-` -- `_` -- `/` -- `$` -- `{` -- `}` - -Common names are `qa`, `staging`, and `production`, but you can use whatever -name works with your workflow. - In its simplest form, the `environment` keyword can be defined like: ``` deploy to production: stage: deploy script: git push production HEAD:master - environment: production + environment: + name: production ``` In the above example, the `deploy to production` job will be marked as doing a @@ -588,6 +576,21 @@ Before GitLab 8.11, the name of an environment could be defined as a string like `environment: production`. The recommended way now is to define it under the `name` keyword. +The `environment` name can contain: + +- letters +- digits +- spaces +- `-` +- `_` +- `/` +- `$` +- `{` +- `}` + +Common names are `qa`, `staging`, and `production`, but you can use whatever +name works with your workflow. + Instead of defining the name of the environment right after the `environment` keyword, it is also possible to define it as a separate value. For that, use the `name` keyword under `environment`: @@ -626,7 +629,12 @@ deploy to production: #### environment:on_stop -> [Introduced][ce-6669] in GitLab 8.13. +> +**Notes:** +- [Introduced][ce-6669] in GitLab 8.13. +- Starting with GitLab 8.14, when you have an environment that has a stop action + defined, GitLab will automatically trigger a stop action when the associated + branch is deleted. Closing (stoping) environments can be achieved with the `on_stop` keyword defined under `environment`. It declares a different job that runs in order to close @@ -681,6 +689,13 @@ The `stop_review_app` job is **required** to have the following keywords defined These parameters can use any of the defined [CI variables](#variables) (including predefined, secure variables and `.gitlab-ci.yml` variables). +>**Note:** +Be aware than if the branch name contains special characters and you use the +`$CI_BUILD_REF_NAME` variable to dynamically create environments, there might +be complications during deployment. Follow the +[issue 22849](https://gitlab.com/gitlab-org/gitlab-ce/issues/22849) for more +information. + For example: ``` @@ -1210,6 +1225,7 @@ capitalization, the commit will be created but the builds will be skipped. Visit the [examples README][examples] to see a list of examples using GitLab CI with various languages. +[env-manual]: ../environments.md#manually-deploying-to-environments [examples]: ../examples/README.md [ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323 [environment]: ../environments.md diff --git a/doc/development/README.md b/doc/development/README.md index f88456a7a7a..371bb55c127 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -22,6 +22,7 @@ ## Process - [Generate a changelog entry with `bin/changelog`](changelog.md) +- [Limit conflicts with EE when developing on CE](limit_ee_conflicts.md) - [Code review guidelines](code_review.md) for reviewing code and having code reviewed. - [Merge request performance guidelines](merge_request_performance_guidelines.md) for ensuring merge requests do not negatively impact GitLab performance diff --git a/doc/development/frontend.md b/doc/development/frontend.md index ec8f2d6531c..9e782ab977f 100644 --- a/doc/development/frontend.md +++ b/doc/development/frontend.md @@ -205,6 +205,57 @@ command line. Please note: Not all of the frontend fixtures are generated. Some are still static files. These will not be touched by `rake teaspoon:fixtures`. +## Design Patterns + +### Singletons + +When exactly one object is needed for a given task, prefer to define it as a +`class` rather than as an object literal. Prefer also to explicitly restrict +instantiation, unless flexibility is important (e.g. for testing). + +``` +// bad + +gl.MyThing = { + prop1: 'hello', + method1: () => {} +}; + +// good + +class MyThing { + constructor() { + this.prop1 = 'hello'; + } + method1() {} +} + +gl.MyThing = new MyThing(); + +// best + +let singleton; + +class MyThing { + constructor() { + if (!singleton) { + singleton = this; + singleton.init(); + } + return singleton; + } + + init() { + this.prop1 = 'hello'; + } + + method1() {} +} + +gl.MyThing = MyThing; + +``` + ## Supported browsers For our currently-supported browsers, see our [requirements][requirements]. diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index b25ce79e89f..7bfc9cb361f 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -32,6 +32,95 @@ spec/models/user_spec.rb|6 error| Failure/Error: u = described_class.new NoMeth Except for the top-level `describe` block, always provide a String argument to `describe`. +## Don't assert against the absolute value of a sequence-generated attribute + +Consider the following factory: + +```ruby +FactoryGirl.define do + factory :label do + sequence(:title) { |n| "label#{n}" } + end +end +``` + +Consider the following API spec: + +```ruby +require 'rails_helper' + +describe API::Labels do + it 'creates a first label' do + create(:label) + + get api("/projects/#{project.id}/labels", user) + + expect(response).to have_http_status(200) + expect(json_response.first['name']).to eq('label1') + end + + it 'creates a second label' do + create(:label) + + get api("/projects/#{project.id}/labels", user) + + expect(response).to have_http_status(200) + expect(json_response.first['name']).to eq('label1') + end +end +``` + +When run, this spec doesn't do what we might expect: + +```sh +1) API::API reproduce sequence issue creates a second label + Failure/Error: expect(json_response.first['name']).to eq('label1') + + expected: "label1" + got: "label2" + + (compared using ==) +``` + +That's because FactoryGirl sequences are not reseted for each example. + +Please remember that sequence-generated values exist only to avoid having to +explicitly set attributes that have a uniqueness constraint when using a factory. + +### Solution + +If you assert against a sequence-generated attribute's value, you should set it +explicitly. Also, the value you set shouldn't match the sequence pattern. + +For instance, using our `:label` factory, writing `create(:label, title: 'foo')` +is ok, but `create(:label, title: 'label1')` is not. + +Following is the fixed API spec: + +```ruby +require 'rails_helper' + +describe API::Labels do + it 'creates a first label' do + create(:label, title: 'foo') + + get api("/projects/#{project.id}/labels", user) + + expect(response).to have_http_status(200) + expect(json_response.first['name']).to eq('foo') + end + + it 'creates a second label' do + create(:label, title: 'bar') + + get api("/projects/#{project.id}/labels", user) + + expect(response).to have_http_status(200) + expect(json_response.first['name']).to eq('bar') + end +end +``` + ## Don't `rescue Exception` See ["Why is it bad style to `rescue Exception => e` in Ruby?"][Exception]. diff --git a/doc/development/limit_ee_conflicts.md b/doc/development/limit_ee_conflicts.md new file mode 100644 index 00000000000..b7e6387838e --- /dev/null +++ b/doc/development/limit_ee_conflicts.md @@ -0,0 +1,272 @@ +# Limit conflicts with EE when developing on CE + +This guide contains best-practices for avoiding conflicts between CE and EE. + +## Context + +Usually, GitLab Community Edition is merged into the Enterprise Edition once a +week. During these merges, it's very common to get conflicts when some changes +in CE do not apply cleanly to EE. + +There are a few things that can help you as a developer to: + +- know when your merge request to CE will conflict when merged to EE +- avoid such conflicts in the first place +- ease future conflict resolutions if conflict is inevitable + +## Check the `rake ee_compat_check` in your merge requests + +For each commit (except on `master`), the `rake ee_compat_check` CI job tries to +detect if the current branch's changes will conflict during the CE->EE merge. + +The job reports what files are conflicting and how to setup a merge request +against EE. Here is roughly how it works: + +1. Generates the diff between your branch and current CE `master` +1. Tries to apply it to current EE `master` +1. If it applies cleanly, the job succeeds, otherwise... +1. Detects a branch with the `-ee` suffix in EE +1. If it exists, generate the diff between this branch and current EE `master` +1. Tries to apply it to current EE `master` +1. If it applies cleanly, the job succeeds + +In the case where the job fails, it means you should create a `<ce_branch>-ee` +branch, push it to EE and open a merge request against EE `master`. At this +point if you retry the failing job in your CE merge request, it should now pass. + +Notes: + +- This task is not a silver-bullet, its current goal is to bring awareness to + developers that their work needs to be ported to EE. +- Community contributors shouldn't submit merge requests against EE, but + reviewers should take actions by either creating such EE merge request or + asking a GitLab developer to do it once the merge request is merged. +- If you branch is more than 500 commits behind `master`, the job will fail and + you should rebase your branch upon latest `master`. + +## Possible type of conflicts + +### Controllers + +#### List or arrays are augmented in EE + +In controllers, the most common type of conflict is with `before_action` that +has a list of actions in CE but EE adds some actions to that list. + +The same problem often occurs for `params.require` / `params.permit` calls. + +##### Mitigations + +Separate CE and EE actions/keywords. For instance for `params.require` in +`ProjectsController`: + +```ruby +def project_params + params.require(:project).permit(project_params_ce) + # On EE, this is always: + # params.require(:project).permit(project_params_ce << project_params_ee) +end + +# Always returns an array of symbols, created however best fits the use case. +# It _should_ be sorted alphabetically. +def project_params_ce + %i[ + description + name + path + ] +end + +# (On EE) +def project_params_ee + %i[ + approvals_before_merge + approver_group_ids + approver_ids + ... + ] +end +``` + +#### Additional condition(s) in EE + +For instance for LDAP: + +```diff + def destroy + @key = current_user.keys.find(params[:id]) + - @key.destroy + + @key.destroy unless @key.is_a? LDAPKey + + respond_to do |format| +``` + +Or for Geo: + +```diff +def after_sign_out_path_for(resource) +- current_application_settings.after_sign_out_path.presence || new_user_session_path ++ if Gitlab::Geo.secondary? ++ Gitlab::Geo.primary_node.oauth_logout_url(@geo_logout_state) ++ else ++ current_application_settings.after_sign_out_path.presence || new_user_session_path ++ end +end +``` + +Or even for audit log: + +```diff +def approve_access_request +- Members::ApproveAccessRequestService.new(membershipable, current_user, params).execute ++ member = Members::ApproveAccessRequestService.new(membershipable, current_user, params).execute ++ ++ log_audit_event(member, action: :create) + + redirect_to polymorphic_url([membershipable, :members]) +end +``` + +### Views + +#### Additional view code in EE + +A block of code added in CE conflicts because there is already another block +at the same place in EE + +##### Mitigations + +Blocks of code that are EE-specific should be moved to partials as much as +possible to avoid conflicts with big chunks of HAML code that that are not fun +to resolve when you add the indentation to the equation. + +For instance this kind of thing: + +```haml +- if can?(current_user, :"admin_#{issuable.to_ability_name}", issuable.project) + - has_due_date = issuable.has_attribute?(:due_date) + %hr + .row + %div{ class: (has_due_date ? "col-lg-6" : "col-sm-12") } + .form-group.issue-assignee + = f.label :assignee_id, "Assignee", class: "control-label #{"col-lg-4" if has_due_date}" + .col-sm-10{ class: ("col-lg-8" if has_due_date) } + .issuable-form-select-holder + - if issuable.assignee_id + = f.hidden_field :assignee_id + = dropdown_tag(user_dropdown_label(issuable.assignee_id, "Assignee"), options: { toggle_class: "js-dropdown-keep-input js-user-search js-issuable-form-dropdown js-assignee-search", title: "Select assignee", filter: true, dropdown_class: "dropdown-menu-user dropdown-menu-selectable dropdown-menu-assignee js-filter-submit", + placeholder: "Search assignee", data: { first_user: current_user.try(:username), null_user: true, current_user: true, project_id: project.try(:id), selected: issuable.assignee_id, field_name: "#{issuable.class.model_name.param_key}[assignee_id]", default_label: "Assignee"} }) + .form-group.issue-milestone + = f.label :milestone_id, "Milestone", class: "control-label #{"col-lg-4" if has_due_date}" + .col-sm-10{ class: ("col-lg-8" if has_due_date) } + .issuable-form-select-holder + = render "shared/issuable/milestone_dropdown", selected: issuable.milestone, name: "#{issuable.class.model_name.param_key}[milestone_id]", show_any: false, show_upcoming: false, extra_class: "js-issuable-form-dropdown js-dropdown-keep-input", dropdown_title: "Select milestone" + .form-group + - has_labels = @labels && @labels.any? + = f.label :label_ids, "Labels", class: "control-label #{"col-lg-4" if has_due_date}" + = f.hidden_field :label_ids, multiple: true, value: '' + .col-sm-10{ class: "#{"col-lg-8" if has_due_date} #{'issuable-form-padding-top' if !has_labels}" } + .issuable-form-select-holder + = render "shared/issuable/label_dropdown", classes: ["js-issuable-form-dropdown"], selected: issuable.labels, data_options: { field_name: "#{issuable.class.model_name.param_key}[label_ids][]", show_any: false, show_menu_above: 'true' }, dropdown_title: "Select label" + + - if issuable.respond_to?(:weight) + .form-group + = f.label :label_ids, class: "control-label #{"col-lg-4" if has_due_date}" do + Weight + .col-sm-10{ class: ("col-lg-8" if has_due_date) } + = f.select :weight, issues_weight_options(issuable.weight, edit: true), { include_blank: true }, + { class: 'select2 js-select2', data: { placeholder: "Select weight" }} + + - if has_due_date + .col-lg-6 + .form-group + = f.label :due_date, "Due date", class: "control-label" + .col-sm-10 + .issuable-form-select-holder + = f.text_field :due_date, id: "issuable-due-date", class: "datepicker form-control", placeholder: "Select due date" +``` + +could be simplified by using partials: + +```haml += render 'metadata_form', issuable: issuable +``` + +and then the `_metadata_form.html.haml` could be as follows: + +```haml +- return unless can?(current_user, :"admin_#{issuable.to_ability_name}", issuable.project) + +- has_due_date = issuable.has_attribute?(:due_date) +%hr +.row + %div{ class: (has_due_date ? "col-lg-6" : "col-sm-12") } + .form-group.issue-assignee + = f.label :assignee_id, "Assignee", class: "control-label #{"col-lg-4" if has_due_date}" + .col-sm-10{ class: ("col-lg-8" if has_due_date) } + .issuable-form-select-holder + - if issuable.assignee_id + = f.hidden_field :assignee_id + = dropdown_tag(user_dropdown_label(issuable.assignee_id, "Assignee"), options: { toggle_class: "js-dropdown-keep-input js-user-search js-issuable-form-dropdown js-assignee-search", title: "Select assignee", filter: true, dropdown_class: "dropdown-menu-user dropdown-menu-selectable dropdown-menu-assignee js-filter-submit", + placeholder: "Search assignee", data: { first_user: current_user.try(:username), null_user: true, current_user: true, project_id: project.try(:id), selected: issuable.assignee_id, field_name: "#{issuable.class.model_name.param_key}[assignee_id]", default_label: "Assignee"} }) + .form-group.issue-milestone + = f.label :milestone_id, "Milestone", class: "control-label #{"col-lg-4" if has_due_date}" + .col-sm-10{ class: ("col-lg-8" if has_due_date) } + .issuable-form-select-holder + = render "shared/issuable/milestone_dropdown", selected: issuable.milestone, name: "#{issuable.class.model_name.param_key}[milestone_id]", show_any: false, show_upcoming: false, extra_class: "js-issuable-form-dropdown js-dropdown-keep-input", dropdown_title: "Select milestone" + .form-group + - has_labels = @labels && @labels.any? + = f.label :label_ids, "Labels", class: "control-label #{"col-lg-4" if has_due_date}" + = f.hidden_field :label_ids, multiple: true, value: '' + .col-sm-10{ class: "#{"col-lg-8" if has_due_date} #{'issuable-form-padding-top' if !has_labels}" } + .issuable-form-select-holder + = render "shared/issuable/label_dropdown", classes: ["js-issuable-form-dropdown"], selected: issuable.labels, data_options: { field_name: "#{issuable.class.model_name.param_key}[label_ids][]", show_any: false, show_menu_above: 'true' }, dropdown_title: "Select label" + + = render 'weight_form', issuable: issuable, has_due_date: has_due_date + + - if has_due_date + .col-lg-6 + .form-group + = f.label :due_date, "Due date", class: "control-label" + .col-sm-10 + .issuable-form-select-holder + = f.text_field :due_date, id: "issuable-due-date", class: "datepicker form-control", placeholder: "Select due date" +``` + +and then the `_weight_form.html.haml` could be as follows: + +```haml +- return unless issuable.respond_to?(:weight) + +- has_due_date = issuable.has_attribute?(:due_date) + +.form-group + = f.label :label_ids, class: "control-label #{"col-lg-4" if has_due_date}" do + Weight + .col-sm-10{ class: ("col-lg-8" if has_due_date) } + = f.select :weight, issues_weight_options(issuable.weight, edit: true), { include_blank: true }, + { class: 'select2 js-select2', data: { placeholder: "Select weight" }} +``` + +Note: + +- The safeguards at the top allow to get rid of an unneccessary indentation level +- Here we only moved the 'Weight' code to a partial since this is the only + EE-specific code in that view, so it's the most likely to conflict, but you + are encouraged to use partials even for code that's in CE to logically split + big views into several smaller files. + +#### Indentation issue + +Sometimes a code block is indented more or less in EE because there's an +additional condition. + +##### Mitigations + +Blocks of code that are EE-specific should be moved to partials as much as +possible to avoid conflicts with big chunks of HAML code that that are not fun +to resolve when you add the indentation in the equation. + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/testing.md b/doc/development/testing.md index b0b26ccf57a..4dc535fb359 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -64,6 +64,7 @@ the command line via `bundle exec teaspoon`, or via a web browser at methods. - Use `context` to test branching logic. - Don't `describe` symbols (see [Gotchas](gotchas.md#dont-describe-symbols)). +- Don't assert against the absolute value of a sequence-generated attribute (see [Gotchas](gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). - Don't supply the `:each` argument to hooks since it's the default. - Prefer `not_to` to `to_not` (_this is enforced by Rubocop_). - Try to match the ordering of tests to the ordering within the class. diff --git a/doc/development/ux_guide/copy.md b/doc/development/ux_guide/copy.md index 03392a003ee..227b4fd3451 100644 --- a/doc/development/ux_guide/copy.md +++ b/doc/development/ux_guide/copy.md @@ -7,7 +7,7 @@ We are currently inconsistent with this guidance. Images below are created to il ## Contents * [Brevity](#brevity) -* [Forms](#forms) +* [Sentence case](#sentence-case) * [Terminology](#terminology) --- @@ -27,52 +27,73 @@ Preferrably use context and placement of controls to make it obvious what clicki --- -## Forms +## Sentence case +Use sentence case for all titles, headings, labels, menu items, and buttons. -### Adding items +--- + +## Terminology +Only use the terms in the tables below. + +### Issues + +#### Adjectives (states) + +| Term | +| ---- | +| Open | +| Closed | +| Deleted | + +>**Example:** +Use `5 open issues` and don't use `5 pending issues`. + +#### Verbs (actions) + +| Term | Use | Don't | +| ---- | --- | --- | +| Add | Add an issue | Don't use `create` or `new` | +| View | View an open or closed issue || +| Edit | Edit an open or closed issue | Don't use `update` | +| Close | Close an open issue || +| Re-open | Re-open a closed issue | There should never be a need to use `open` as a verb | +| Delete | Delete an open or closed issue || + +#### Add issue When viewing a list of issues, there is a button that is labeled `Add`. Given the context in the example, it is clearly referring to issues. If the context were not clear enough, the label could be `Add issue`. Clicking the button will bring you to the `Add issue` form. Other add flows should be similar.  -The form should be titled `Add issue`. The submit button should be labeled `Save` or `Submit`. Do not use `Add`, `Create`, `New`, or `Save Changes`. The cancel button should be labeled `Cancel`. Do not use `Back`. +The form should be titled `Add issue`. The submit button should be labeled `Submit`. Don't use `Add`, `Create`, `New`, or `Save changes`. The cancel button should be labeled `Cancel`. Don't use `Back`.  -### Editing items +#### Edit issue When in context of an issue, the affordance to edit it is labeled `Edit`. If the context is not clear enough, `Edit issue` could be considered. Other edit flows should be similar.  -The form should be titled `Edit Issue`. The submit button should be labeled `Save`. Do not use `Edit`, `Update`, `New`, or `Save Changes`. The cancel button should be labeled `Cancel`. Do not use `Back`. +The form should be titled `Edit issue`. The submit button should be labeled `Save`. Don't use `Edit`, `Update`, `Submit`, or `Save changes`. The cancel button should be labeled `Cancel`. Don't use `Back`.  ---- - -## Terminology -### Issues +### Merge requests #### Adjectives (states) -| Term | Use | -| ---- | --- | -| Open | Issue is active | -| Closed | Issue is no longer active | - ->**Example:** -Use `5 open issues` and do not use `5 pending issues`. -Only use the adjectives in the table above. +| Term | +| ---- | +| Open | +| Merged | #### Verbs (actions) -| Term | Use | -| ---- | --- | -| Add | For adding an issue. Do not use `create` or `new` | -| View | View an issue | -| Edit | Edit an issue. Do not use `update` | -| Close | Closing an issue | -| Re-open | Re-open an issue. There should never be a need to use `open` as a verb | -| Delete | Deleting an issue. Do not use `remove` |
\ No newline at end of file +| Term | Use | Don't | +| ---- | --- | --- | +| Add | Add a merge request | Do not use `create` or `new` | +| View | View an open or merged merge request || +| Edit | Edit an open or merged merge request| Do not use `update` | +| Merge | Merge an open merge request ||
\ No newline at end of file diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 766a7119943..42b515761e0 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -143,9 +143,6 @@ On a very active server (10,000 active users) the Sidekiq process can use 1GB+ o ## Supported web browsers -- Chrome (Latest stable version) -- Firefox (Latest released version and [latest ESR version](https://www.mozilla.org/en-US/firefox/organizations/)) -- Safari 7+ (known problem: required fields in html5 do not work) -- Opera (Latest released version) -- Internet Explorer (IE) 11+ but please make sure that you have the `Compatibility View` mode disabled. -- Edge (Latest stable version) +We support the current and the previous major release of Firefox, Safari and Microsoft browsers (Microsoft Edge and Internet Explorer 11). + +Each time a new browser version is released, we begin supporting that version and stop supporting the third most recent version. diff --git a/doc/user/markdown.md b/doc/user/markdown.md index dbc7e0f14e3..162d1bd7ed4 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -1,43 +1,5 @@ # Markdown -## Table of Contents - -**[GitLab Flavored Markdown](#gitlab-flavored-markdown-gfm)** - -* [Newlines](#newlines) -* [Multiple underscores in words](#multiple-underscores-in-words) -* [URL auto-linking](#url-auto-linking) -* [Multiline Blockquote](#multiline-blockquote) -* [Code and Syntax Highlighting](#code-and-syntax-highlighting) -* [Inline Diff](#inline-diff) -* [Emoji](#emoji) -* [Special GitLab references](#special-gitlab-references) -* [Task Lists](#task-lists) -* [Videos](#videos) - -**[Standard Markdown](#standard-markdown)** - -* [Headers](#headers) -* [Emphasis](#emphasis) -* [Lists](#lists) -* [Links](#links) -* [Images](#images) -* [Blockquotes](#blockquotes) -* [Inline HTML](#inline-html) -* [Horizontal Rule](#horizontal-rule) -* [Line Breaks](#line-breaks) -* [Tables](#tables) -* [Footnotes](#footnotes) - -**[Wiki-Specific Markdown](#wiki-specific-markdown)** - -* [Wiki - Direct page link](#wiki-direct-page-link) -* [Wiki - Direct file link](#wiki-direct-file-link) -* [Wiki - Hierarchical link](#wiki-hierarchical-link) -* [Wiki - Root link](#wiki-root-link) - -**[References](#references)** - ## GitLab Flavored Markdown (GFM) > **Note:** @@ -64,7 +26,7 @@ You can use GFM in the following areas: You can also use other rich text files in GitLab. You might have to install a dependency to do so. Please see the [github-markup gem readme](https://github.com/gitlabhq/markup#markups) for more information. -## Newlines +### Newlines > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#newlines @@ -84,7 +46,7 @@ Violets are blue Sugar is sweet -## Multiple underscores in words +### Multiple underscores in words > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiple-underscores-in-words @@ -99,7 +61,7 @@ perform_complicated_task do_this_and_do_that_and_another_thing -## URL auto-linking +### URL auto-linking > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#url-auto-linking @@ -120,7 +82,7 @@ GFM will autolink almost any URL you copy and paste into your text: * irc://irc.freenode.net/gitlab * http://localhost:3000 -## Multiline Blockquote +### Multiline Blockquote > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiline-blockquote @@ -154,7 +116,7 @@ multiple lines, you can quote that without having to manually prepend `>` to every line! >>> -## Code and Syntax Highlighting +### Code and Syntax Highlighting > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting @@ -224,7 +186,7 @@ s = "There is no highlighting for this." But let's throw in a <b>tag</b>. ``` -## Inline Diff +### Inline Diff > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-diff @@ -240,7 +202,7 @@ However the wrapping tags cannot be mixed as such: - {- deletions -] - [- deletions -} -## Emoji +### Emoji > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#emoji @@ -265,7 +227,7 @@ If you are new to this, don't be :fearful:. You can easily join the emoji :famil Consult the [Emoji Cheat Sheet](http://emoji.codes) for a list of all supported emoji codes. :thumbsup: -## Special GitLab References +### Special GitLab References GFM recognizes special references. @@ -305,7 +267,7 @@ GFM also recognizes certain cross-project references: | `namespace/project@9ba12248...b19a04f5` | commit range comparison | | `namespace/project~"Some label"` | issues with given label | -## Task Lists +### Task Lists > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#task-lists @@ -328,7 +290,7 @@ You can add task lists to issues, merge requests and comments. To create a task Task lists can only be created in descriptions, not in titles. Task item state can be managed by editing the description's Markdown or by toggling the rendered check boxes. -## Videos +### Videos > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#videos @@ -345,9 +307,9 @@ Here's a sample video:  -# Standard Markdown +## Standard Markdown -## Headers +### Headers ```no-highlight # H1 @@ -366,21 +328,6 @@ Alt-H2 ------ ``` -# H1 -## H2 -### H3 -#### H4 -##### H5 -###### H6 - -Alternatively, for H1 and H2, an underline-ish style: - -Alt-H1 -====== - -Alt-H2 ------- - ### Header IDs and links All Markdown-rendered headers automatically get IDs, except in comments. @@ -416,7 +363,7 @@ Would generate the following link IDs: Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. -## Emphasis +### Emphasis ```no-highlight Emphasis, aka italics, with *asterisks* or _underscores_. @@ -436,7 +383,7 @@ Combined emphasis with **asterisks and _underscores_**. Strikethrough uses two tildes. ~~Scratch this.~~ -## Lists +### Lists ```no-highlight 1. First ordered list item @@ -492,7 +439,7 @@ the second list item will be incorrectly labeled as `1`. Second paragraph of first item. 2. Another item -## Links +### Links There are two ways to create links, inline-style and reference-style. @@ -501,9 +448,9 @@ There are two ways to create links, inline-style and reference-style. [I'm a reference-style link][Arbitrary case-insensitive reference text] [I'm a relative reference to a repository file](LICENSE) - + [I am an absolute reference within the repository](/doc/user/markdown.md) - + [I link to the Milestones page](/../milestones) [You can use numbers for reference-style link definitions][1] @@ -523,9 +470,9 @@ There are two ways to create links, inline-style and reference-style. [I'm a relative reference to a repository file](LICENSE)[^1] [I am an absolute reference within the repository](/doc/user/markdown.md) - + [I link to the Milestones page](/../milestones) - + [You can use numbers for reference-style link definitions][1] Or leave it empty and use the [link text itself][] @@ -544,7 +491,8 @@ Relative links do not allow referencing project files in a wiki page or wiki pag will point the link to `wikis/style` when the link is inside of a wiki markdown file. -## Images + +### Images Here's our logo (hover to see the title text): @@ -568,7 +516,7 @@ Reference-style: [logo]: img/markdown_logo.png -## Blockquotes +### Blockquotes ```no-highlight > Blockquotes are very handy in email to emulate reply text. @@ -586,7 +534,7 @@ Quote break. > This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. -## Inline HTML +### Inline HTML You can also use raw HTML in your Markdown, and it'll mostly work pretty well. @@ -610,7 +558,7 @@ See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubyd <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> </dl> -## Horizontal Rule +### Horizontal Rule ``` Three or more... @@ -642,7 +590,7 @@ ___ Underscores -## Line Breaks +### Line Breaks My basic recommendation for learning how line breaks work is to experiment and discover -- hit <Enter> once (i.e., insert one newline), then hit it twice (i.e., insert two newlines), see what happens. You'll soon learn to get what you want. "Markdown Toggle" is your friend. @@ -672,7 +620,7 @@ This line is also a separate paragraph, and... This line is on its own line, because the previous line ends with two spaces. -## Tables +### Tables Tables aren't part of the core Markdown spec, but they are part of GFM and Markdown Here supports them. @@ -708,16 +656,15 @@ By including colons in the header row, you can align the text within that column | Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | | Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | -## Footnotes - -You can add footnotes to your text as follows.[^1] -[^1]: This is my awesome footnote. +### Footnotes ``` -You can add footnotes to your text as follows.[^1] -[^1]: This is my awesome footnote. +You can add footnotes to your text as follows.[^2] +[^2]: This is my awesome footnote. ``` +You can add footnotes to your text as follows.[^2] + ## Wiki-specific Markdown The following examples show how links inside wikis behave. @@ -752,30 +699,30 @@ A link can be constructed relative to the current wiki page using `./<page>`, - If this snippet was placed on a page at `<your_wiki>/documentation/main`, it would link to `<your_wiki>/documentation/related`: - ```markdown - [Link to Related Page](./related) - ``` + ```markdown + [Link to Related Page](./related) + ``` - If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, it would link to `<your_wiki>/documentation/main`: - ```markdown - [Link to Related Page](../main) - ``` + ```markdown + [Link to Related Page](../main) + ``` - If this snippet was placed on a page at `<your_wiki>/documentation/main`, it would link to `<your_wiki>/documentation/related.md`: - ```markdown - [Link to Related Page](./related.md) - ``` + ```markdown + [Link to Related Page](./related.md) + ``` - If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, it would link to `<your_wiki>/documentation/main.md`: - ```markdown - [Link to Related Page](../main.md) - ``` + ```markdown + [Link to Related Page](../main.md) + ``` ### Wiki - Root link @@ -783,22 +730,25 @@ A link starting with a `/` is relative to the wiki root. - This snippet links to `<wiki_root>/documentation`: - ```markdown - [Link to Related Page](/documentation) - ``` + ```markdown + [Link to Related Page](/documentation) + ``` - This snippet links to `<wiki_root>/miscellaneous.md`: - ```markdown - [Link to Related Page](/miscellaneous.md) - ``` + ```markdown + [Link to Related Page](/miscellaneous.md) + ``` + ## References - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). - The [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. - [Dillinger.io](http://dillinger.io) is a handy tool for testing standard markdown. +[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com +[^2]: This is my awesome footnote. + [markdown.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md [rouge]: http://rouge.jneen.net/ "Rouge website" [redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" -[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com diff --git a/doc/user/permissions.md b/doc/user/permissions.md index d6216a8dd50..cea78864df2 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -32,6 +32,8 @@ The following table depicts the various user permission levels in a project. | See a commit status | | ✓ | ✓ | ✓ | ✓ | | See a container registry | | ✓ | ✓ | ✓ | ✓ | | See environments | | ✓ | ✓ | ✓ | ✓ | +| Create new environments | | | ✓ | ✓ | ✓ | +| Stop environments | | | ✓ | ✓ | ✓ | | See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | | Manage/Accept merge requests | | | ✓ | ✓ | ✓ | | Create new merge request | | | ✓ | ✓ | ✓ | @@ -45,7 +47,6 @@ The following table depicts the various user permission levels in a project. | Create or update commit status | | | ✓ | ✓ | ✓ | | Update a container registry | | | ✓ | ✓ | ✓ | | Remove a container registry image | | | ✓ | ✓ | ✓ | -| Create new environments | | | ✓ | ✓ | ✓ | | Create new milestones | | | | ✓ | ✓ | | Add new team members | | | | ✓ | ✓ | | Push to protected branches | | | | ✓ | ✓ | @@ -58,7 +59,6 @@ The following table depicts the various user permission levels in a project. | Manage runners | | | | ✓ | ✓ | | Manage build triggers | | | | ✓ | ✓ | | Manage variables | | | | ✓ | ✓ | -| Delete environments | | | | ✓ | ✓ | | Switch visibility level | | | | | ✓ | | Transfer project to another namespace | | | | | ✓ | | Remove project | | | | | ✓ | |
