diff options
Diffstat (limited to 'doc/ci')
| -rw-r--r-- | doc/ci/autodeploy/quick_start_guide.md | 6 | ||||
| -rw-r--r-- | doc/ci/docker/using_docker_build.md | 8 | ||||
| -rw-r--r-- | doc/ci/examples/browser_performance.md | 73 | ||||
| -rw-r--r-- | doc/ci/examples/code_quality.md | 82 | ||||
| -rw-r--r-- | doc/ci/examples/container_scanning.md | 83 | ||||
| -rw-r--r-- | doc/ci/examples/dast.md | 67 | ||||
| -rw-r--r-- | doc/ci/examples/deployment/composer-npm-deploy.md | 24 | ||||
| -rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png | bin | 4730 -> 0 bytes | |||
| -rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/index.md | 4 | ||||
| -rw-r--r-- | doc/ci/examples/test-and-deploy-python-application-to-heroku.md | 12 | ||||
| -rw-r--r-- | doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md | 11 | ||||
| -rw-r--r-- | doc/ci/quick_start/README.md | 2 | ||||
| -rw-r--r-- | doc/ci/runners/README.md | 6 | ||||
| -rw-r--r-- | doc/ci/variables/where_variables_can_be_used.md | 2 | ||||
| -rw-r--r-- | doc/ci/yaml/README.md | 81 |
15 files changed, 359 insertions, 102 deletions
diff --git a/doc/ci/autodeploy/quick_start_guide.md b/doc/ci/autodeploy/quick_start_guide.md index 3836216e951..1473703542d 100644 --- a/doc/ci/autodeploy/quick_start_guide.md +++ b/doc/ci/autodeploy/quick_start_guide.md @@ -23,9 +23,9 @@ You need to have the Google Cloud SDK installed. e.g. On OSX, install [homebrew](https://brew.sh): 1. Install Brew Caskroom: `brew install caskroom/cask/brew-cask` -2. Install Google Cloud SDK: `brew cask install google-cloud-sdk` -3. Add `kubectl`: `gcloud components install kubectl` -4. Log in: `gcloud auth login` +1. Install Google Cloud SDK: `brew cask install google-cloud-sdk` +1. Add `kubectl`: `gcloud components install kubectl` +1. Log in: `gcloud auth login` Now go back to the Google interface, find your cluster, and follow the instructions under `Connect to the cluster` and open the Kubernetes Dashboard. It will look something like `gcloud container clusters get-credentials ruby-autodeploy \ --zone europe-west2-c --project api-project-XXXXXXX` and then `kubectl proxy`. diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 3b41036cd14..fef367051bf 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -46,18 +46,18 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. --description "My Runner" ``` -2. Install Docker Engine on server. +1. Install Docker Engine on server. For more information how to install Docker Engine on different systems checkout the [Supported installations](https://docs.docker.com/engine/installation/). -3. Add `gitlab-runner` user to `docker` group: +1. Add `gitlab-runner` user to `docker` group: ```bash sudo usermod -aG docker gitlab-runner ``` -4. Verify that `gitlab-runner` has access to Docker: +1. Verify that `gitlab-runner` has access to Docker: ```bash sudo -u gitlab-runner -H docker info @@ -75,7 +75,7 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. - docker run my-docker-image /script/to/run/tests ``` -5. You can now use `docker` command and install `docker-compose` if needed. +1. You can now use `docker` command and install `docker-compose` if needed. NOTE: **Note:** By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. diff --git a/doc/ci/examples/browser_performance.md b/doc/ci/examples/browser_performance.md index d36e97ebfd3..7c3b3a65675 100644 --- a/doc/ci/examples/browser_performance.md +++ b/doc/ci/examples/browser_performance.md @@ -1,14 +1,20 @@ # Browser Performance Testing with the Sitespeed.io container +CAUTION: **Caution:** +The job definition shown below is supported on GitLab 11.5 and later versions. +It also requires the GitLab Runner 11.5 or later. +For earlier versions, use the [previous job definitions](#previous-job-definitions). + This example shows how to run the [Sitespeed.io container](https://hub.docker.com/r/sitespeedio/sitespeed.io/) on your code by using GitLab CI/CD and [Sitespeed.io](https://www.sitespeed.io) using Docker-in-Docker. -First, you need a GitLab Runner with the +First, you need GitLab Runner with [docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). -Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called -`performance`: + +Once you set up the Runner, add a new job to `.gitlab-ci.yml` that +generates the expected report: ```yaml performance: @@ -26,19 +32,22 @@ performance: - mv sitespeed-results/data/performance.json performance.json artifacts: paths: - - performance.json - - sitespeed-results/ + - sitespeed-results/ + reports: + performance: performance.json ``` -The above example will: +The above example will create a `performance` job in your CI/CD pipeline and will run +Sitespeed.io against the webpage you defined in `URL` to gather key metrics. +The [GitLab plugin](https://gitlab.com/gitlab-org/gl-performance) for +Sitespeed.io is downloaded in order to save the report as a +[Performance report artifact](https://docs.gitlab.com/ee//ci/yaml/README.html#artifactsreportsperformance) +that you can later download and analyze. +Due to implementation limitations we always take the latest Performance artifact available. -1. Create a `performance` job in your CI/CD pipeline and will run - Sitespeed.io against the webpage you defined in `URL`. -1. The [GitLab plugin](https://gitlab.com/gitlab-org/gl-performance) for - Sitespeed.io is downloaded in order to export key metrics to JSON. The full - HTML Sitespeed.io report will also be saved as an artifact, and if you have - [GitLab Pages](../../user/project/pages/index.md) enabled, it can be viewed - directly in your browser. +The full HTML Sitespeed.io report will also be saved as an artifact, and if you have +[GitLab Pages](../../user/project/pages/index.md) enabled, it can be viewed +directly in your browser. For further customization options of Sitespeed.io, including the ability to provide a list of URLs to test, please consult @@ -46,8 +55,8 @@ provide a list of URLs to test, please consult TIP: **Tip:** For [GitLab Premium](https://about.gitlab.com/pricing/) users, key metrics are automatically -extracted and shown right in the merge request widget. Learn more about -[Browser Performance Testing](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). +extracted and shown right in the merge request widget. +[Learn more on Browser Performance Testing in merge requests](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html). ## Performance testing on Review Apps @@ -106,8 +115,40 @@ performance: - mv sitespeed-results/data/performance.json performance.json artifacts: paths: - - performance.json - sitespeed-results/ + reports: + performance: performance.json ``` A complete example can be found in our [Auto DevOps CI YML](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml). + +## Previous job definitions + +CAUTION: **Caution:** +Before GitLab 11.5, Performance job and artifact had to be named specifically +to automatically extract report data and show it in the merge request widget. +While these old job definitions are still maintained they have been deprecated +and may be removed in next major release, GitLab 12.0. +You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change. + +For GitLab 11.4 and earlier, the job should look like: + +```yaml +performance: + stage: performance + image: docker:git + variables: + URL: https://example.com + services: + - docker:stable-dind + script: + - mkdir gitlab-exporter + - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js + - mkdir sitespeed-results + - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL + - mv sitespeed-results/data/performance.json performance.json + artifacts: + paths: + - performance.json + - sitespeed-results/ +```
\ No newline at end of file diff --git a/doc/ci/examples/code_quality.md b/doc/ci/examples/code_quality.md index 2a7040ecdeb..ae000b9d30d 100644 --- a/doc/ci/examples/code_quality.md +++ b/doc/ci/examples/code_quality.md @@ -1,11 +1,18 @@ # Analyze your project's Code Quality +CAUTION: **Caution:** +The job definition shown below is supported on GitLab 11.5 and later versions. +It also requires the GitLab Runner 11.5 or later. +For earlier versions, use the [previous job definitions](#previous-job-definitions). + This example shows how to run Code Quality on your code by using GitLab CI/CD and Docker. -First, you need GitLab Runner with [docker-in-docker executor][dind]. +First, you need GitLab Runner with +[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). -Once you set up the Runner, add a new job to `.gitlab-ci.yml`, called `code_quality`: +Once you set up the Runner, add a new job to `.gitlab-ci.yml` that +generates the expected report: ```yaml code_quality: @@ -23,27 +30,72 @@ code_quality: --volume /var/run/docker.sock:/var/run/docker.sock "registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code artifacts: - paths: [gl-code-quality-report.json] + reports: + codequality: gl-code-quality-report.json ``` The above example will create a `code_quality` job in your CI/CD pipeline which -will scan your source code for code quality issues. The report will be saved -as an artifact that you can later download and analyze. +will scan your source code for code quality issues. The report will be saved as a +[Code Quality report artifact](../../ci/yaml/README.md#artifactsreportscodequality) +that you can later download and analyze. +Due to implementation limitations we always take the latest Code Quality artifact available. TIP: **Tip:** -Starting with [GitLab Starter][ee] 9.3, this information will -be automatically extracted and shown right in the merge request widget. To do -so, the CI/CD job must be named `code_quality` and the artifact path must be -`gl-code-quality-report.json`. +For [GitLab Starter][ee] users, this information will be automatically +extracted and shown right in the merge request widget. [Learn more on Code Quality in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html). +## Previous job definitions + CAUTION: **Caution:** -Code Quality was previously using `codeclimate` and `codequality` for job name and -`codeclimate.json` for the artifact name. While these old names -are still maintained they have been deprecated with GitLab 11.0 and may be removed -in next major release, GitLab 12.0. You are advised to update your current `.gitlab-ci.yml` -configuration to reflect that change. +Before GitLab 11.5, Code Quality job and artifact had to be named specifically +to automatically extract report data and show it in the merge request widget. +While these old job definitions are still maintained they have been deprecated +and may be removed in next major release, GitLab 12.0. +You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change. + +For GitLab 11.4 and earlier, the job should look like: + +```yaml +code_quality: + image: docker:stable + variables: + DOCKER_DRIVER: overlay2 + allow_failure: true + services: + - docker:stable-dind + script: + - export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') + - docker run + --env SOURCE_CODE="$PWD" + --volume "$PWD":/code + --volume /var/run/docker.sock:/var/run/docker.sock + "registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code + artifacts: + paths: [gl-code-quality-report.json] +``` + +Alternatively the job name could be `codeclimate` or `codequality` +and the artifact name could be `codeclimate.json`. +These names have been deprecated with GitLab 11.0 +and may be removed in next major release, GitLab 12.0. + +For GitLab 10.3 and earlier, the job should look like: + +```yaml +codequality: + image: docker:latest + variables: + DOCKER_DRIVER: overlay + services: + - docker:dind + script: + - docker pull codeclimate/codeclimate:0.69.0 + - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 init + - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 analyze -f json > codeclimate.json || true + artifacts: + paths: [codeclimate.json] +``` [cli]: https://github.com/codeclimate/codeclimate -[dind]: ../docker/using_docker_build.md#use-docker-in-docker-executor [ee]: https://about.gitlab.com/pricing/ diff --git a/doc/ci/examples/container_scanning.md b/doc/ci/examples/container_scanning.md index bc948dc6ea9..68330261910 100644 --- a/doc/ci/examples/container_scanning.md +++ b/doc/ci/examples/container_scanning.md @@ -1,13 +1,20 @@ # Container Scanning with GitLab CI/CD +CAUTION: **Caution:** +The job definition shown below is supported on GitLab 11.5 and later versions. +It also requires the GitLab Runner 11.5 or later. +For earlier versions, use the [previous job definitions](#previous-job-definitions). + You can check your Docker images (or more precisely the containers) for known vulnerabilities by using [Clair](https://github.com/coreos/clair) and [clair-scanner](https://github.com/arminc/clair-scanner), two open source tools for Vulnerability Static Analysis for containers. -All you need is a GitLab Runner with the Docker executor (the shared Runners on -GitLab.com will work fine). You can then add a new job to `.gitlab-ci.yml`, -called `container_scanning`: +First, you need GitLab Runner with +[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). + +Once you set up the Runner, add a new job to `.gitlab-ci.yml` that +generates the expected report: ```yaml container_scanning: @@ -36,32 +43,26 @@ container_scanning: - while( ! wget -T 10 -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; echo -n "." ; if [ $retries -eq 10 ] ; then echo " Timeout, aborting." ; exit 1 ; fi ; retries=$(($retries+1)) ; done - ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-container-scanning-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true artifacts: - paths: [gl-container-scanning-report.json] + reports: + container_scanning: gl-container-scanning-report.json ``` The above example will create a `container_scanning` job in your CI/CD pipeline, pull the image from the [Container Registry](../../user/project/container_registry.md) (whose name is defined from the two `CI_APPLICATION_` variables) and scan it -for possible vulnerabilities. The report will be saved as an artifact that you -can later download and analyze. +for possible vulnerabilities. The report will be saved as a +[Container Scanning report artifact](https://docs.gitlab.com/ee//ci/yaml/README.html#artifactsreportscontainer_scanning) +that you can later download and analyze. +Due to implementation limitations we always take the latest Container Scanning artifact available. If you want to whitelist some specific vulnerabilities, you can do so by defining them in a [YAML file](https://github.com/arminc/clair-scanner/blob/master/README.md#example-whitelist-yaml-file), in our case its named `clair-whitelist.yml`. TIP: **Tip:** -Starting with [GitLab Ultimate][ee] 10.4, this information will -be automatically extracted and shown right in the merge request widget. To do -so, the CI/CD job must be named `container_scanning` and the artifact path must be -`gl-container-scanning-report.json`. -[Learn more on container scanning results shown in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html). - -CAUTION: **Caution:** -Before GitLab 11.0, Container Scanning was previously using `sast:container` for job name and -`gl-sast-container-report.json` for the artifact name. While these old names -are still maintained, they have been deprecated with GitLab 11.0 and may be removed -in next major release, GitLab 12.0. You are advised to update your current `.gitlab-ci.yml` -configuration to reflect that change. +For [GitLab Ultimate][ee] users, this information will +be automatically extracted and shown right in the merge request widget. +[Learn more on Container Scanning in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html). CAUTION: **Caution:** Starting with GitLab 11.5, Container Scanning feature is licensed under the name `container_scanning`. @@ -69,4 +70,50 @@ While the old name `sast_container` is still maintained, it has been deprecated may be removed in next major release, GitLab 12.0. You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change if you are using the `$GITLAB_FEATURES` environment variable. +## Previous job definitions + +CAUTION: **Caution:** +Before GitLab 11.5, Container Scanning job and artifact had to be named specifically +to automatically extract report data and show it in the merge request widget. +While these old job definitions are still maintained they have been deprecated +and may be removed in next major release, GitLab 12.0. +You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change. + +For GitLab 11.4 and earlier, the job should look like: + +```yaml +container_scanning: + image: docker:stable + variables: + DOCKER_DRIVER: overlay2 + ## Define two new variables based on GitLab's CI/CD predefined variables + ## https://docs.gitlab.com/ee/ci/variables/#predefined-variables-environment-variables + CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG + CI_APPLICATION_TAG: $CI_COMMIT_SHA + allow_failure: true + services: + - docker:stable-dind + script: + - docker run -d --name db arminc/clair-db:latest + - docker run -p 6060:6060 --link db:postgres -d --name clair --restart on-failure arminc/clair-local-scan:v2.0.1 + - apk add -U wget ca-certificates + - docker pull ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} + - wget https://github.com/arminc/clair-scanner/releases/download/v8/clair-scanner_linux_amd64 + - mv clair-scanner_linux_amd64 clair-scanner + - chmod +x clair-scanner + - touch clair-whitelist.yml + - while( ! wget -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; done + - retries=0 + - echo "Waiting for clair daemon to start" + - while( ! wget -T 10 -q -O /dev/null http://docker:6060/v1/namespaces ) ; do sleep 1 ; echo -n "." ; if [ $retries -eq 10 ] ; then echo " Timeout, aborting." ; exit 1 ; fi ; retries=$(($retries+1)) ; done + - ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-container-scanning-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true + artifacts: + paths: [gl-container-scanning-report.json] +``` + +Alternatively the job name could be `sast:container` +and the artifact name could be `gl-sast-container-report.json`. +These names have been deprecated with GitLab 11.0 +and may be removed in next major release, GitLab 12.0. + [ee]: https://about.gitlab.com/pricing/ diff --git a/doc/ci/examples/dast.md b/doc/ci/examples/dast.md index ff20f0b3b5e..0ca89eb6700 100644 --- a/doc/ci/examples/dast.md +++ b/doc/ci/examples/dast.md @@ -1,16 +1,26 @@ # Dynamic Application Security Testing with GitLab CI/CD +CAUTION: **Caution:** +The job definition shown below is supported on GitLab 11.5 and later versions. +It also requires the GitLab Runner 11.5 or later. +For earlier versions, use the [previous job definitions](#previous-job-definitions). + [Dynamic Application Security Testing (DAST)](https://en.wikipedia.org/wiki/Dynamic_program_analysis) is using the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy) to perform an analysis on your running web application. +Since it is based on [ZAP Baseline](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan) +DAST will perform passive scanning only; +it will not actively attack your application. It can be very useful combined with [Review Apps](../review_apps/index.md). ## Example -All you need is a GitLab Runner with the Docker executor (the shared Runners on -GitLab.com will work fine). You can then add a new job to `.gitlab-ci.yml`, -called `dast`: +First, you need GitLab Runner with +[docker-in-docker executor](../docker/using_docker_build.md#use-docker-in-docker-executor). + +Once you set up the Runner, add a new job to `.gitlab-ci.yml` that +generates the expected report: ```yaml dast: @@ -23,13 +33,16 @@ dast: - /zap/zap-baseline.py -J gl-dast-report.json -t $website || true - cp /zap/wrk/gl-dast-report.json . artifacts: - paths: [gl-dast-report.json] + reports: + dast: gl-dast-report.json ``` The above example will create a `dast` job in your CI/CD pipeline which will run the tests on the URL defined in the `website` variable (change it to use your -own) and finally write the results in the `gl-dast-report.json` file. You can -then download and analyze the report artifact in JSON format. +own) and scan it for possible vulnerabilities. The report will be saved as a +[DAST report artifact](https://docs.gitlab.com/ee//ci/yaml/README.html#artifactsreportsdast) +that you can later download and analyze. +Due to implementation limitations we always take the latest DAST artifact available. It's also possible to authenticate the user before performing DAST checks: @@ -39,25 +52,51 @@ dast: variables: website: "https://example.com" login_url: "https://example.com/sign-in" + username: "john.doe@example.com" + password: "john-doe-password" allow_failure: true script: - mkdir /zap/wrk/ - /zap/zap-baseline.py -J gl-dast-report.json -t $website --auth-url $login_url - --auth-username "john.doe@example.com" - --auth-password "john-doe-password" || true + --auth-username $username + --auth-password $password || true - cp /zap/wrk/gl-dast-report.json . artifacts: - paths: [gl-dast-report.json] + reports: + dast: gl-dast-report.json ``` See [zaproxy documentation](https://gitlab.com/gitlab-org/security-products/zaproxy) to learn more about authentication settings. TIP: **Tip:** -Starting with [GitLab Ultimate][ee] 10.4, this information will -be automatically extracted and shown right in the merge request widget. To do -so, the CI job must be named `dast` and the artifact path must be -`gl-dast-report.json`. -[Learn more about DAST results shown in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html). +For [GitLab Ultimate][ee] users, this information will +be automatically extracted and shown right in the merge request widget. +[Learn more on DAST in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html). + +## Previous job definitions + +CAUTION: **Caution:** +Before GitLab 11.5, DAST job and artifact had to be named specifically +to automatically extract report data and show it in the merge request widget. +While these old job definitions are still maintained they have been deprecated +and may be removed in next major release, GitLab 12.0. +You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change. + +For GitLab 11.4 and earlier, the job should look like: + +```yaml +dast: + image: registry.gitlab.com/gitlab-org/security-products/zaproxy + variables: + website: "https://example.com" + allow_failure: true + script: + - mkdir /zap/wrk/ + - /zap/zap-baseline.py -J gl-dast-report.json -t $website || true + - cp /zap/wrk/gl-dast-report.json . + artifacts: + paths: [gl-dast-report.json] +``` [ee]: https://about.gitlab.com/pricing/ diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 55ff131efaa..36358515b84 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -33,9 +33,9 @@ before_script: In this particular case, the `npm deploy` script is a Gulp script that does the following: 1. Compile CSS & JS -2. Create sprites -3. Copy various assets (images, fonts) around -4. Replace some strings +1. Create sprites +1. Copy various assets (images, fonts) around +1. Replace some strings All these operations will put all files into a `build` folder, which is ready to be deployed to a live server. @@ -62,10 +62,10 @@ before_script: In order, this means that: -1. We check if the `ssh-agent` is available and we install it if it's not; -2. We create the `~/.ssh` folder; -3. We make sure we're running bash; -4. We disable host checking (we don't ask for user accept when we first connect to a server; and since every job will equal a first connect, we kind of need this) +1. We check if the `ssh-agent` is available and we install it if it's not. +1. We create the `~/.ssh` folder. +1. We make sure we're running bash. +1. We disable host checking (we don't ask for user accept when we first connect to a server and since every job will equal a first connect, we kind of need this). And this is basically all you need in the `before_script` section. @@ -91,11 +91,11 @@ stage_deploy: Here's the breakdown: 1. `only:dev` means that this build will run only when something is pushed to the `dev` branch. You can remove this block completely and have everything be ran on every push (but probably this is something you don't want) -2. `ssh-add ...` we will add that private key you added on the web UI to the docker container -3. We will connect via `ssh` and create a new `_tmp` folder -4. We will connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder -5. We will connect again to `ssh` and move the `live` folder to an `_old` folder, then move `_tmp` to `live`. -6. We connect to ssh and remove the `_old` folder +1. `ssh-add ...` we will add that private key you added on the web UI to the docker container +1. We will connect via `ssh` and create a new `_tmp` folder +1. We will connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder +1. We will connect again to `ssh` and move the `live` folder to an `_old` folder, then move `_tmp` to `live`. +1. We connect to ssh and remove the `_old` folder What's the deal with the artifacts? We just tell GitLab CI to keep the `build` directory (later on, you can download that as needed). diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png Binary files differdeleted file mode 100644 index a56c07a0da7..00000000000 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png +++ /dev/null diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index b6989d229d1..b090ea014dc 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -444,9 +444,7 @@ On your GitLab project repository navigate to the **Registry** tab.  -You may need to [enable Container Registry](../../../user/project/container_registry.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Sharing and permissions**. - - +You may need to [enable Container Registry](../../../user/project/container_registry.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Permissions**. To start using Container Registry on our machine, we first need to login to the GitLab registry using our GitLab username and password: diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 087b317ab73..ec0b5aaed09 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -40,15 +40,17 @@ production: ``` This project has three jobs: -1. `test` - used to test Django application, -2. `staging` - used to automatically deploy staging environment every push to `master` branch -3. `production` - used to automatically deploy production environment for every created tag + +- `test` - used to test Django application, +- `staging` - used to automatically deploy staging environment every push to `master` branch +- `production` - used to automatically deploy production environment for every created tag ## Store API keys You'll need to create two variables in `Settings > CI/CD > Variables` on your GitLab project settings: -1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, -2. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. + +- `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app. +- `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. Find your Heroku API key in [Manage Account](https://dashboard.heroku.com/account). diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 7f9ab1f3a5e..33a353f17f5 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -36,16 +36,17 @@ production: ``` This project has three jobs: -1. `test` - used to test Rails application, -2. `staging` - used to automatically deploy staging environment every push to `master` branch -3. `production` - used to automatically deploy production environment for every created tag + +- `test` - used to test Rails application. +- `staging` - used to automatically deploy staging environment every push to `master` branch. +- `production` - used to automatically deploy production environment for every created tag. ## Store API keys You'll need to create two variables in your project's **Settings > CI/CD > Variables**: -1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, -2. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. +- `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app. +- `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. Find your Heroku API key in [Manage Account](https://dashboard.heroku.com/account). diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 636117536a2..bdc593493ea 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -168,7 +168,7 @@ can be found at <https://docs.gitlab.com/runner/>. In order to have a functional Runner you need to follow two steps: 1. [Install it][runner-install] -2. [Configure it](../runners/README.md#registering-a-specific-runner) +1. [Configure it](../runners/README.md#registering-a-specific-runner) Follow the links above to set up your own Runner or use a Shared Runner as described in the next section. diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 2a179bfbbf0..9c9ea651678 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -138,9 +138,9 @@ project without requiring your authorization, so use it with caution. An admin can enable/disable a specific Runner for projects: 1. Navigate to **Admin > Runners** -2. Find the Runner you wish to enable/disable -3. Click edit on the Runner -4. Click **Enable** or **Disable** on the project +1. Find the Runner you wish to enable/disable +1. Click edit on the Runner +1. Click **Enable** or **Disable** on the project ## Protected Runners diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index 4e8ce10c9cb..1d98e8426fe 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -17,7 +17,7 @@ There are two places defined variables can be used. On the: | Definition | Can be expanded? | Expansion place | Description | |--------------------------------------|-------------------|-----------------|--------------| -| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>Supported: all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>Not suported: variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> | +| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>Supported: all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>Not supported: variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> | | `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support: <ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> | | `variables` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | | `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 5deeb2b0133..8b8bd6ec795 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -103,7 +103,7 @@ rspec: - $RSPEC ``` -In the example above, the `rspec` job inherits from the `.tests` template job. +In the example above, the `rspec` job inherits from the `.tests` template job. GitLab will perform a reverse deep merge based on the keys. GitLab will: - Merge the `rspec` contents into `.tests` recursively. @@ -476,6 +476,7 @@ docker build: - Dockerfile - docker/scripts/* - dockerfiles/**/* + - more_scripts/*.{rb,py,sh} ``` In the scenario above, if you are pushing multiple commits to GitLab to an @@ -485,6 +486,7 @@ one of the commits contains changes to either: - The `Dockerfile` file. - Any of the files inside `docker/scripts/` directory. - Any of the files and subfolders inside `dockerfiles` directory. +- Any of the files with `rb`, `py`, `sh` extensions inside `more_scripts` directory. CAUTION: **Warning:** There are some caveats when using this feature with new branches and tags. See @@ -1337,6 +1339,81 @@ concatenated into a single file. Use a filename pattern (`junit: rspec-*.xml`), an array of filenames (`junit: [rspec-1.xml, rspec-2.xml, rspec-3.xml]`), or a combination thereof (`junit: [rspec.xml, test-results/TEST-*.xml]`). +#### `artifacts:reports:codequality` **[STARTER]** + +> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above. + +The `codequality` report collects [CodeQuality issues](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html) +as artifacts. + +The collected Code Quality report will be uploaded to GitLab as an artifact and will +be automatically shown in merge requests. + +#### `artifacts:reports:sast` **[ULTIMATE]** + +> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above. + +The `sast` report collects [SAST vulnerabilities](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html) +as artifacts. + +The collected SAST report will be uploaded to GitLab as an artifact and will +be automatically shown in merge requests, pipeline view and provide data for security +dashboards. + +#### `artifacts:reports:dependency_scanning` **[ULTIMATE]** + +> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above. + +The `dependency_scanning` report collects [Dependency Scanning vulnerabilities](https://docs.gitlab.com/ee/user/project/merge_requests/dependency_scanning.html) +as artifacts. + +The collected Dependency Scanning report will be uploaded to GitLab as an artifact and will +be automatically shown in merge requests, pipeline view and provide data for security +dashboards. + +#### `artifacts:reports:container_scanning` **[ULTIMATE]** + +> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above. + +The `container_scanning` report collects [Container Scanning vulnerabilities](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html) +as artifacts. + +The collected Container Scanning report will be uploaded to GitLab as an artifact and will +be automatically shown in merge requests, pipeline view and provide data for security +dashboards. + +#### `artifacts:reports:dast` **[ULTIMATE]** + +> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above. + +The `dast` report collects [DAST vulnerabilities](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html) +as artifacts. + +The collected DAST report will be uploaded to GitLab as an artifact and will +be automatically shown in merge requests, pipeline view and provide data for security +dashboards. + +#### `artifacts:reports:license_management` **[ULTIMATE]** + +> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above. + +The `license_management` report collects [Licenses](https://docs.gitlab.com/ee/user/project/merge_requests/license_management.html) +as artifacts. + +The collected License Management report will be uploaded to GitLab as an artifact and will +be automatically shown in merge requests, pipeline view and provide data for security +dashboards. + +#### `artifacts:reports:performance` **[PREMIUM]** + +> Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above. + +The `performance` report collects [Performance metrics](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html) +as artifacts. + +The collected Performance report will be uploaded to GitLab as an artifact and will +be automatically shown in merge requests. + ## `dependencies` > Introduced in GitLab 8.6 and GitLab Runner v1.1.1. @@ -1692,7 +1769,7 @@ stages: production: script: - - install_depedencies + - install_dependencies - deploy - notify_owner ``` |
