diff options
Diffstat (limited to 'doc/ci/examples')
17 files changed, 113 insertions, 477 deletions
diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index e728941dd9d..9fa0bb080ac 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -20,32 +20,31 @@ Examples are available in several forms. As a collection of: ## CI/CD examples -The following table lists examples with step-by-step tutorials that are contained in this section. +The following table lists examples with step-by-step tutorials that are contained in this section: -| Use case | Resource | -|:------------------------------|:---------------------------------------------------------------------------------------------------------------------------| +| Use case | Resource | +|:------------------------------|:---------| | Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | -| Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | -| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | -| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | -| Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | -| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | -| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | +| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | +| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | | GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | -| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | -| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | -| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | -| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | -| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | -| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | -| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | +| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | +| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | +| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | +| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | +| Load performance testing | [Load Performance Testing with the k6 container](../../user/project/merge_requests/load_performance_testing.md). | +| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | +| NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). | +| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | +| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | +| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | | Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | -| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | -| Multi project pipeline | [Build, test deploy using multi project pipeline](https://gitlab.com/gitlab-examples/upstream-project). | -| NPM with semantic-release | [Publish NPM packages to the GitLab Package Registry using semantic-release](semantic-release.md). | +| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | +| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | +| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | +| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | -### Contributing examples +### How to contributing examples Contributions are welcome! You can help your favorite programming language users and GitLab by sending a merge request with a guide for that language. @@ -72,6 +71,7 @@ choose one of these templates: - [dotNET (`dotNET.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET.gitlab-ci.yml) - [dotNET Core (`dotNET-Core.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/dotNET-Core.yml) - [Elixir (`Elixir.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Elixir.gitlab-ci.yml) +- [Flutter (`Flutter.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Flutter.gitlab-ci.yml) - [goLang (`Go.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Go.gitlab-ci.yml) - [Gradle (`Gradle.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Gradle.gitlab-ci.yml) - [Grails (`Grails.gitlab-ci.yml`)](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Grails.gitlab-ci.yml) diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index e37bdcc9407..c1df21297e3 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -3,9 +3,14 @@ stage: Verify group: Continuous Integration info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/articles/artifactory_and_gitlab/index.html' +author: Fabio Busatto +author_gitlab: bikebilly type: tutorial +date: 2017-08-15 --- +<!-- vale off --> + # How to deploy Maven projects to Artifactory with GitLab CI/CD ## Introduction diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index b7f59761889..fccc62a4ca0 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -19,14 +19,14 @@ To learn more, read [Using external secrets in CI](../../secrets/index.md). This tutorial assumes you are familiar with GitLab CI/CD and Vault. -To follow along, you will need: +To follow along, you must have: - An account on GitLab. - A running Vault server and access to it is required to configure authentication and create roles and policies. For HashiCorp Vaults, this can be the Open Source or Enterprise version. NOTE: -You will need to replace the `vault.example.com` URL below with the URL of your Vault server and `gitlab.example.com` with the URL of your GitLab instance. +You must replace the `vault.example.com` URL below with the URL of your Vault server, and `gitlab.example.com` with the URL of your GitLab instance. ## How it works @@ -47,6 +47,7 @@ The JWT's payload looks like this: "project_id": "22", # "project_path": "mygroup/myproject", # "user_id": "42", # Id of the user executing the job + "user_login": "myuser" # GitLab @username "user_email": "myuser@example.com", # Email of the user executing the job "pipeline_id": "1212", # "job_id": "1212", # @@ -56,7 +57,7 @@ The JWT's payload looks like this: } ``` -The JWT is encoded by using RS256 and signed with a dedicated private key. The expire time for the token will be set to job's timeout, if specified, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job will generate new JWT using the current signing key. +The JWT is encoded by using RS256 and signed with a dedicated private key. The expire time for the token is set to job's timeout, if specified, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job generates new JWT using the current signing key. You can use this JWT and your instance's JWKS endpoint (`https://gitlab.example.com/-/jwks`) to authenticate with a Vault server that is configured to allow the JWT Authentication method for authentication. @@ -110,7 +111,7 @@ EOF Success! Uploaded policy: myproject-production ``` -You'll also need roles that will link the JWT with these policies. +You also need roles that link the JWT with these policies. One for staging named `myproject-staging`: @@ -150,7 +151,7 @@ $ vault write auth/jwt/role/myproject-production - <<EOF EOF ``` -This example uses [bound_claims](https://www.vaultproject.io/api/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims will be allowed to authenticate. +This example uses [bound_claims](https://www.vaultproject.io/api/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims is allowed to authenticate. Combined with [protected branches](../../../user/project/protected_branches.md), you can restrict who is able to authenticate and read the secrets. @@ -158,7 +159,7 @@ Combined with [protected branches](../../../user/project/protected_branches.md), [user_claim](https://www.vaultproject.io/api/auth/jwt#user_claim) specifies the name for the Identity alias created by Vault upon a successful login. -[bound_claims_type](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values will be interpreted as globs, with `*` matching any number of characters. +[bound_claims_type](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values are interpreted as globs, with `*` matching any number of characters. For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). @@ -177,7 +178,7 @@ $ vault write auth/jwt/config \ For the full list of available configuration options, see Vault's [API documentation](https://www.vaultproject.io/api/auth/jwt#configure). -The following job, when run for the `master` branch, will be able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`: +The following job, when run for the `master` branch, is able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`: ```yaml read_secrets: @@ -201,7 +202,7 @@ read_secrets:  -The following job will be able to authenticate using the `myproject-production` role and read secrets under `/secret/myproject/production/`: +The following job is able to authenticate using the `myproject-production` role and read secrets under `/secret/myproject/production/`: ```yaml read_secrets: diff --git a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md index f4f3bf306ef..9c145677f6e 100644 --- a/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md +++ b/doc/ci/examples/deploy_spring_boot_to_cloud_foundry/index.md @@ -2,18 +2,23 @@ stage: Release group: Release info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +author: Dylan Griffith +author_gitlab: DylanGriffith type: tutorial +date: 2018-06-07 +description: "Continuous Deployment of a Spring Boot application to Cloud Foundry with GitLab CI/CD" --- +<!-- vale off --> + # Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD ## Introduction -In this article, we'll demonstrate how to deploy a [Spring -Boot](https://projects.spring.io/spring-boot/) application to [Cloud -Foundry (CF)](https://www.cloudfoundry.org/) with GitLab CI/CD using the [Continuous -Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-deployment) -method. +This article demonstrates how to use the [Continuous Deployment](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-deployment) +method to deploy a [Spring Boot](https://projects.spring.io/spring-boot/) application to +[Cloud Foundry (CF)](https://www.cloudfoundry.org/) +with GitLab CI/CD. All the code for this project can be found in this [GitLab repository](https://gitlab.com/gitlab-examples/spring-gitlab-cf-deploy-demo). @@ -25,17 +30,16 @@ using GitLab CI/CD, read through the blog post [Continuous Delivery of a Spring This tutorial assumes you are familiar with Java, GitLab, Cloud Foundry, and GitLab CI/CD. -To follow along, you will need: +To follow along, you need: - An account on [Pivotal Web Services (PWS)](https://run.pivotal.io/) or any other Cloud Foundry (CF) instance. - An account on GitLab. NOTE: -You will need to replace the `api.run.pivotal.io` URL in the all below -commands with the [API -URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html) of your CF -instance if you're not deploying to PWS. +If you're not deploying to PWS, you must replace the `api.run.pivotal.io` URL in all the below +commands with the [API URL](https://docs.cloudfoundry.org/running/cf-api-endpoint.html) +of your CF instance. ## Create your project @@ -46,9 +50,9 @@ GitLab when creating a new project: ## Configure the deployment to Cloud Foundry -To deploy to Cloud Foundry we need to add a `manifest.yml` file. This -is the configuration for the CF CLI we will use to deploy the application. We -will create this in the root directory of our project with the following +To deploy to Cloud Foundry you must add a `manifest.yml` file. This +is the configuration for the CF CLI you must use to deploy the application. +Create this in the root directory of your project with the following content: ```yaml @@ -62,12 +66,12 @@ applications: ## Configure GitLab CI/CD to deploy your application -Now we need to add the GitLab CI/CD configuration file -([`.gitlab-ci.yml`](../../yaml/README.md)) to our -project's root. This is how GitLab figures out what commands need to be run whenever -code is pushed to our repository. We will add the following `.gitlab-ci.yml` -file to the root directory of the repository, GitLab will detect it -automatically and run the steps defined once we push our code: +Now you must add the GitLab CI/CD configuration file +([`.gitlab-ci.yml`](../../yaml/README.md)) +to your project's root. This is how GitLab figures out what commands must run whenever +code is pushed to your repository. Add the following `.gitlab-ci.yml` +file to the root directory of the repository. GitLab detects it +automatically and runs the defined steps once you push your code: ```yaml image: java:8 @@ -96,15 +100,13 @@ production: - master ``` -We've used the `java:8` [Docker -image](../../docker/using_docker_images.md) to build -our application as it provides the up-to-date Java 8 JDK on [Docker -Hub](https://hub.docker.com/). We've also added the [`only` -clause](../../yaml/README.md#onlyexcept-basic) -to ensure our deployments only happen when we push to the master branch. +This uses the `java:8` [Docker image](../../docker/using_docker_images.md) +to build your application, as it provides the up-to-date Java 8 JDK on [Docker Hub](https://hub.docker.com/). +You also added the [`only` clause](../../yaml/README.md#onlyexcept-basic) +to ensure your deployments only happen when you push to the master branch. Because the steps defined in `.gitlab-ci.yml` require credentials to sign in to -CF, you'll need to add your CF credentials as +CF, you must add your CF credentials as [environment variables](../../variables/README.md#predefined-environment-variables) in GitLab CI/CD. To set the environment variables, navigate to your project's **Settings > CI/CD**, and then expand **Variables**. Name the variables @@ -122,8 +124,8 @@ your application and add its credentials to GitLab instead of using a developer's credentials. To start a manual deployment in GitLab go to **CI/CD > Pipelines** then click -on **Run Pipeline**. After the app is finished deploying, it will display the -URL of your application in the logs for the `production` job like: +**Run Pipeline**. After the app is finished deploying, it displays the +URL of your application in the logs for the `production` job: ```shell requested state: started diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 386512af38b..958093364af 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -55,10 +55,10 @@ To use different provider take a look at long list of [Supported Providers](http ## Using Dpl with Docker -In most cases, you will have configured [GitLab Runner](https://docs.gitlab.com/runner/) to use your server's shell commands. +In most cases, you configured [GitLab Runner](https://docs.gitlab.com/runner/) to use your server's shell commands. This means that all commands are run in the context of local user (e.g. `gitlab_runner` or `gitlab_ci_multi_runner`). It also means that most probably in your Docker container you don't have the Ruby runtime installed. -You will have to install it: +You must install it: ```yaml staging: @@ -115,7 +115,7 @@ We also use two secure variables: ## Storing API keys -Secure Variables can added by going to your project's +To add secure variables, navigate to your project's **Settings > CI / CD > Variables**. The variables that are defined in the project settings are sent along with the build script to the runner. The secure variables are stored out of the repository. Never store secrets in diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 24990264f19..6bc96ae6c30 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -9,13 +9,13 @@ type: tutorial This guide covers the building of dependencies of a PHP project while compiling assets via an NPM script using [GitLab CI/CD](../../README.md). -While it is possible to create your own image with custom PHP and Node.js versions, for brevity, we will use an existing [Docker image](https://hub.docker.com/r/tetraweb/php/) that contains both PHP and Node.js installed. +While it is possible to create your own image with custom PHP and Node.js versions, for brevity we use an existing [Docker image](https://hub.docker.com/r/tetraweb/php/) that contains both PHP and Node.js installed. ```yaml image: tetraweb/php ``` -The next step is to install zip/unzip packages and make composer available. We will place these in the `before_script` section: +The next step is to install zip/unzip packages and make composer available. We place these in the `before_script` section: ```yaml before_script: @@ -26,7 +26,7 @@ before_script: - php -r "unlink('composer-setup.php');" ``` -This will make sure we have all requirements ready. Next, we want to run `composer install` to fetch all PHP dependencies and `npm install` to load Node.js packages, then run the `npm` script. We need to append them into `before_script` section: +This makes sure we have all requirements ready. Next, run `composer install` to fetch all PHP dependencies and `npm install` to load Node.js packages. Then run the `npm` script. We need to append them into `before_script` section: ```yaml before_script: @@ -43,19 +43,19 @@ In this particular case, the `npm deploy` script is a Gulp script that does the 1. Copy various assets (images, fonts) around 1. Replace some strings -All these operations will put all files into a `build` folder, which is ready to be deployed to a live server. +All these operations put all files into a `build` folder, which is ready to be deployed to a live server. ## How to transfer files to a live server -You have multiple options: rsync, SCP, SFTP, and so on. For now, we will use SCP. +You have multiple options: rsync, SCP, SFTP, and so on. For now, use SCP. -To make this work, you need to add a GitLab CI/CD Variable (accessible on `gitlab.example/your-project-name/variables`). That variable will be called `STAGING_PRIVATE_KEY` and it's the **private** SSH key of your server. +To make this work, you must add a GitLab CI/CD Variable (accessible on `gitlab.example/your-project-name/variables`). Name this variable `STAGING_PRIVATE_KEY` and set it to the **private** SSH key of your server. ### Security tip Create a user that has access **only** to the folder that needs to be updated. -After you create that variable, you need to make sure that key will be added to the Docker container on run: +After you create that variable, make sure that key is added to the Docker container on run: ```yaml before_script: @@ -71,7 +71,7 @@ In order, this means that: 1. We check if the `ssh-agent` is available and we install it if it's not. 1. We create the `~/.ssh` folder. 1. We make sure we're running bash. -1. We disable host checking (we don't ask for user accept when we first connect to a server and since every job will equal a first connect, we kind of need this). +1. We disable host checking (we don't ask for user accept when we first connect to a server, and since every job equals a first connect, we need this). And this is basically all you need in the `before_script` section. @@ -96,14 +96,14 @@ stage_deploy: Here's the breakdown: -1. `only:dev` means that this build will run only when something is pushed to the `dev` branch. You can remove this block completely and have everything be ran on every push (but probably this is something you don't want) -1. `ssh-add ...` we will add that private key you added on the web UI to the Docker container -1. We will connect via `ssh` and create a new `_tmp` folder -1. We will connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder -1. We will connect again via `ssh` and move the `live` folder to an `_old` folder, then move `_tmp` to `live`. -1. We connect to SSH and remove the `_old` folder +1. `only:dev` means that this build runs only when something is pushed to the `dev` branch. You can remove this block completely and have everything run on every push (but probably this is something you don't want). +1. `ssh-add ...` we add that private key you added on the web UI to the Docker container. +1. We connect via `ssh` and create a new `_tmp` folder. +1. We connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder. +1. We connect again via `ssh` and move the `live` folder to an `_old` folder, then move `_tmp` to `live`. +1. We connect to SSH and remove the `_old` folder. -What's the deal with the artifacts? We just tell GitLab CI/CD to keep the `build` directory (later on, you can download that as needed). +What's the deal with the artifacts? We tell GitLab CI/CD to keep the `build` directory (later on, you can download that as needed). ### Why we do it this way @@ -114,7 +114,7 @@ If you're using this only for stage server, you could do this in two steps: - scp -P22 -r build/* server_user@server_host:htdocs/wp-content/themes/live ``` -The problem is that there will be a small period of time when you won't have the app on your server. +The problem is that there's a small period of time when you don't have the app on your server. Therefore, for a production environment we use additional steps to ensure that at any given time, a functional app is in place. @@ -122,13 +122,13 @@ Therefore, for a production environment we use additional steps to ensure that a Since this was a WordPress project, I gave real life code snippets. Some further ideas you can pursue: -- Having a slightly different script for `master` branch will allow you to deploy to a production server from that branch and to a stage server from any other branches. +- Having a slightly different script for `master` branch allows you to deploy to a production server from that branch and to a stage server from any other branches. - Instead of pushing it live, you can push it to WordPress official repository (with creating a SVN commit, etc.). - You could generate i18n text domains on the fly. --- -Our final `.gitlab-ci.yml` will look like this: +Our final `.gitlab-ci.yml` looks like this: ```yaml image: tetraweb/php diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 7abdcf1f9be..298ffff568a 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -2,9 +2,14 @@ stage: Verify group: Continuous Integration info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +author: Ryan Hall +author_gitlab: blitzgren type: tutorial +date: 2018-03-07 --- +<!-- vale off --> + # DevOps and Game Dev with GitLab CI/CD With advances in WebGL and WebSockets, browsers are extremely viable as game development diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 96183b040a2..4521c2ed52e 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -2,9 +2,15 @@ stage: Verify group: Testing info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +author: Vincent Tunru +author_gitlab: Vinnl type: tutorial +date: 2019-02-18 +description: 'Confidence checking your entire app every time a new feature is added can quickly become repetitive. Learn how to automate it with GitLab CI/CD.' --- +<!-- vale off --> + # End-to-end testing with GitLab CI/CD and WebdriverIO [Review Apps](../../review_apps/index.md) are great: for every merge request diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 490fb857942..c6ddeefb916 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -2,9 +2,15 @@ stage: Verify group: Continuous Integration info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +disqus_identifier: 'https://docs.gitlab.com/ee/articles/laravel_with_gitlab_and_envoy/index.html' +author: Mehran Rasulian +author_gitlab: mehranrasulian type: tutorial +date: 2017-08-31 --- +<!-- vale off --> + # Test and deploy Laravel applications with GitLab CI/CD and Envoy ## Introduction diff --git a/doc/ci/examples/semantic-release.md b/doc/ci/examples/semantic-release.md index 70d29b739b1..037faaf66a2 100644 --- a/doc/ci/examples/semantic-release.md +++ b/doc/ci/examples/semantic-release.md @@ -35,7 +35,7 @@ You can also view or fork the complete [example source](https://gitlab.com/gitla } ``` -1. Update the `files` key with glob pattern(s) that selects all files that should be included in the published module. More information about `files` can be found [in NPM's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json#files). +1. Update the `files` key with glob pattern(s) that selects all files that should be included in the published module. More information about `files` can be found [in NPM's documentation](https://docs.npmjs.com/cli/v6/configuring-npm/package-json/#files). 1. Add a `.gitignore` file to the project to avoid committing `node_modules`: diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 089d72852bb..1204a1ae837 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -28,16 +28,16 @@ test: staging: stage: deploy script: - - gem install dpl - - dpl --provider=heroku --app=gitlab-ci-ruby-test-staging --api-key=$HEROKU_STAGING_API_KEY + - gem install dpl --pre + - dpl heroku api --app=gitlab-ci-ruby-test-staging --api-key=$HEROKU_STAGING_API_KEY only: - master production: stage: deploy script: - - gem install dpl - - dpl --provider=heroku --app=gitlab-ci-ruby-test-prod --api-key=$HEROKU_PRODUCTION_API_KEY + - gem install dpl --pre + - dpl heroku api --app=gitlab-ci-ruby-test-prod --api-key=$HEROKU_PRODUCTION_API_KEY only: - tags ``` @@ -50,7 +50,7 @@ This project has three jobs: ## Store API keys -You'll need to create two variables in your project's **Settings > CI/CD > Environment variables**: +You'll need to create two variables in your project's **Settings > CI/CD > Environment variables** and do not check **Protect variable** and **Mask variable**: - `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app. - `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-new.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-new.png Binary files differdeleted file mode 100644 index 04d3dc40fa5..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-new.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-server.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-server.png Binary files differdeleted file mode 100644 index 63812b41c2c..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/mix-phoenix-server.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/pipelines.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/pipelines.png Binary files differdeleted file mode 100644 index c0daa1a6a91..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/pipelines.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/select_template_v12_6.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/select_template_v12_6.png Binary files differdeleted file mode 100644 index c8c5e152a13..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/select_template_v12_6.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/set_up_ci_v12_6.png b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/set_up_ci_v12_6.png Binary files differdeleted file mode 100644 index fafabb27bac..00000000000 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/img/set_up_ci_v12_6.png +++ /dev/null diff --git a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md index 90f04fb3615..057b6ec126f 100644 --- a/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/test_phoenix_app_with_gitlab_ci_cd/index.md @@ -1,397 +1,8 @@ --- -stage: Verify -group: Continuous Integration -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: tutorial +redirect_to: '../README.md' --- -# Testing a Phoenix application with GitLab CI/CD +This example is no longer available. [View other examples](../README.md). -[Phoenix](https://www.phoenixframework.org/) is a web development framework written in [Elixir](https://elixir-lang.org), which is a -functional language designed for productivity and maintainability that runs on the -[Erlang VM](https://www.erlang.org). Erlang VM is really fast and can handle very large numbers of -simultaneous users. - -That's why we're hearing so much about Phoenix today. - -In this tutorial, we'll teach you how to set up [GitLab CI/CD](../../README.md) to build and test a Phoenix -application. - -The tutorial assumes that you know how to create a Phoenix app, run tests locally, and how to work with Git -and the GitLab UI. - -## Introduction - -### What is Phoenix? - -[Phoenix](https://www.phoenixframework.org/) is a web development framework written in [Elixir](https://elixir-lang.org). It's useful - for building fast, reliable, and high-performance applications, as it uses [Erlang VM](https://www.erlang.org). - -Many components and concepts are similar to Ruby on Rails or Python's Django. High developer -productivity and high application performance are only a few advantages on learning how to use it. -Working on the MVC pattern, it's was designed to be modular and flexible. Easy to maintain a growing -app is a plus. - -Phoenix can run in any OS where Erlang is supported: - -- Ubuntu -- CentOS -- Mac OS X -- Debian -- Windows -- Fedora -- Raspberry Pi OS - -Check the [Phoenix learning guide](https://hexdocs.pm/phoenix/overview.html) for more information. - -### What is Elixir? - -[Elixir](https://elixir-lang.org) is a dynamic, functional language created to use all the maturity of Erlang -(30 years old!) in these days, in an easy way. It has similarities with Ruby, specially on syntax, -so Ruby developers are quite excited with the rapid growing of Elixir. A full-stack Ruby developer -can learn how to use Elixir and Phoenix in just a few weeks! - -In Elixir we have a command called `mix`, which is a helper to create projects, testing, run -migrations and [much more](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix). We'll use it later on in this tutorial. - -Check the [Elixir documentation](https://elixir-lang.org/getting-started/introduction) for more information. - -## Requirements - -To follow this tutorial, you'll need to have installed: - -- Elixir [installation instructions](https://elixir-lang.org/install) -- Phoenix Framework [installation instructions](https://hexdocs.pm/phoenix/installation.html) -- PostgreSQL (if you need to use MySQL server, check [Phoenix instructions](https://hexdocs.pm/phoenix/ecto.html#using-mysql)) - -### Create a new Phoenix project - -Open your terminal and go to the directory you wish to create your project. -You don't need to create an empty directory for the project's files, because the `mix` command will -do it for us. - -When we call `mix` command, we'll pass two arguments: - -- The task we want it to run: `phoenix.new` -- And the parameter `phoenix.new` requires, which is the name of the new project. In this case, - we're calling it `hello_gitlab_ci`, but you're free to set your own name: - -```shell -mix phoenix.new hello_gitlab_ci -``` - -When asked, answer `Y` to fetch and install dependencies. - -If everything went fine, you'll get an output like this: - - - -Now, our project is located inside the directory with the same name we pass to `mix` command, for -example, `~/GitLab/hello_gitlab_ci`. -If we take a look at the directory, we'll see the Phoenix files and the dependencies needed to run. - -### Initialize the PostgreSQL database - -By default, Phoenix requires a PostgreSQL database to store whatever we need to store in our app. In -this case, we'll only create an empty database. - -First, we need to navigate to our recently created project's directory, and then execute again -`mix`. This time, `mix` will receive the parameter `ecto.create`, which is the task to create our -new database. [Ecto](https://hexdocs.pm/ecto/Ecto.html) is the database wrapper for Elixir. - -When we do run `mix` the first time after creating our project, it will compile our files to -bytecode, which will be interpreted by Erlang VM. In the next times, it will only compile our -changes. - -Run the commands below to create our empty database: - -```shell -cd hello_gitlab_ci -mix ecto.create -``` - -We expect to see this output at the end of the command: - -```plaintext -Generated hello_gitlab_ci app -The database for HelloGitlabCi.Repo has been created -``` - -Phoenix assumes that our PostgreSQL database will have a `postgres` user account with the correct -permissions and a password of `postgres`. If it's not your case, check -[Ecto's instructions](https://hexdocs.pm/ecto/Ecto.html#module-repositories). - -### Start Phoenix server - -Now, it's time to see if everything we did until now went well. We'll call `mix` again, this time -with `phoenix.server` parameter, which will start Phoenix's HTTP Server. - -```shell -mix phoenix.server -``` - -This will be the output to this command: - -```plaintext -[info] Running HelloGitlabCi.Endpoint with Cowboy using http://localhost:4000 -23 May 11:44:35 - info: compiling -23 May 11:44:37 - info: compiled 6 files into 2 files, copied 3 in 9.8 sec -``` - -Now, we have our app running locally. We can preview it directly on our browser. Let's open -[`localhost:4000`](http://localhost:4000) to see our Phoenix Framework welcome page. If the link do -not work, open [`127.0.0.1:4000`](http://127.0.0.1:4000) instead and later, configure your OS to -point `localhost` to `127.0.0.1`. - - - -Great, now we have a local Phoenix Server running our app. - -Locally, our application is running in an [`iex`](https://elixir-lang.org/getting-started/introduction.html#interactive-mode) session, which stands for Interactive Elixir. -In this interactive mode, we can type any Elixir expression and get its result. To exit `iex`, we -need to press `Ctrl+C` twice. So, when we need to stop the Phoenix server, we have to hit `Ctrl+C` -twice. - -## Introducing GitLab CI/CD - -With GitLab, we can manage our development workflow, improve our productivity, track issues, -perform code review, and much more from a single platform. With GitLab CI/CD, we can be much more -productive, because every time we, or our co-workers push any code, GitLab CI/CD will build and -test the changes, telling us in real time if anything goes wrong. - -Certainly, when our application starts to grow, we'll need more developers working on the same -project and this process of building and testing can easily become a mess without proper management. -That's also why GitLab CI/CD is so important to our application. Every time someone pushes its code to -GitLab, we'll quickly know if their changes broke something or not. We don't need to stop everything -we're doing to test manually and locally every change our team does. - -Let's see this in practice. - -## Adjusting Phoenix configuration - -Now, we need to adjust our Phoenix configuration before configuring GitLab CI/CD. -There is a directory (`config`) in your Phoenix project that contains a configuration file for every -environment it can run. Since we will work with a single environment, we'll edit just the test -configuration file (`test.exs`). - -But, why do we need to adjust our configuration? Well, GitLab CI/CD builds and tests our code in one -isolated virtual machine, called a [runner](../../runners/README.md), using Docker technology. In this runner, -GitLab CI/CD has access to everything our Phoenix application need to run, exactly as we have in our -`localhost`, but we have to tell GitLab CI/CD where to create and find this database using system -variables. This way, GitLab CI/CD will create our test database inside the runner, just like we do -when running our Phoenix in our `localhost`. - -- Open `hello_gitlab_ci/config/test.exs` on your favorite code editor -- Go to **Configure your database** session and edit the block to include `System.get_env`: - - ```elixir - # Configure your database - config :hello_gitlab_ci, HelloGitlabCi.Repo, - adapter: Ecto.Adapters.Postgres, - username: System.get_env("POSTGRES_USER") || "postgres", - password: System.get_env("POSTGRES_PASSWORD") || "postgres", - database: System.get_env("POSTGRES_DB") || "hello_gitlab_ci_test", - hostname: System.get_env("POSTGRES_HOST") || "localhost", - pool: Ecto.Adapters.SQL.Sandbox - ``` - - We'll need these system variables later on. - -- Create an empty file named `.gitkeep` into `hello_gitlab_ci/priv/repo/migrations` - - As our project is still fresh, we don't have any data on our database, so, the `migrations` - directory will be empty. - Without `.gitkeep`, Git will not upload this empty directory and we'll got an error when running our - test on GitLab. - - If we add a folder via the GitLab UI, GitLab itself will add the `.gitkeep` to that new dir. - -Now, let's run a local test and see if everything we did didn't break anything. - -## Testing - -Earlier, when we created our project, we ran `mix phoenix.new`. -This task created everything a Phoenix application needed, including some unit tests into -`hello_gitlab_ci/test` directory. - -Let's run a new task with `mix` to run those tests for us. This time, the parameter expected is -`test`. We can add `--trace` parameter for debugging purposes. - -In your terminal, navigate to the directory `hello_gitlab_ci` and run: - -```shell -mix test -``` - -Our expected result is this: - -```plaintext -.... - -Finished in 0.7 seconds -4 tests, 0 failures - -Randomized with seed 610000 -``` - -Our test was successful. It's time to push our files to GitLab. - -## Configuring CI/CD Pipeline - -The first step is to create a new file called `.gitlab-ci.yml` in `hello_gitlab_ci` directory of our -project. - -- The easiest way to do this is to click on **Set up CI/CD** on project's main page: - -  - -- On the next screen, we can use a template with Elixir tests already included. Click on **Apply a template** and select **Elixir**: - -  - - This template file tells GitLab CI/CD about what we wish to do every time a new commit is made. - However, we have to adapt it slightly to run a Phoenix app. - -- The first line tells GitLab what Docker image will be used. - - Remember when we learned about runners, the isolated virtual machine where GitLab CI/CD builds and tests - our application? This virtual machine must have all dependencies to run our application. This is - where a Docker image is needed. The correct image will provide the entire system for us. - - As we are focusing on testing (not deploying), you can use the [elixir:latest](https://hub.docker.com/_/elixir) Docker image, which already has the - dependencies for running Phoenix tests installed, such as Elixir and Erlang: - - ```yaml - image: elixir:latest - ``` - -- We'll only use `postgres`, so we can delete the `mysql` and `redis` lines from the `services` section: - - ```yaml - services: - - postgres:latest - ``` - -- Now, we'll create a new section called `variables`, before the `before_script` section: - - ```yaml - variables: - POSTGRES_DB: hello_gitlab_ci_test - POSTGRES_HOST: postgres - POSTGRES_USER: postgres - POSTGRES_PASSWORD: "postgres" - MIX_ENV: "test" - ``` - - Above, we set up the values for GitLab CI/CD to authenticate into PostgreSQL, like we did in - `config/test.exs` earlier. The `POSTGRES_USER` and `POSTGRES_PASSWORD` values - are used by the `postgres` service to create a user with those credentials. - -- In the `before_script` section, we'll add some commands to prepare everything for the test: - - ```yaml - before_script: - - mix local.rebar --force - - mix local.hex --force - - mix deps.get --only test - - mix ecto.create - - mix ecto.migrate - ``` - - This ensures that [rebar3](https://rebar3.org) and [hex](https://hex.pm) are both installed - before attempting to fetch the dependencies that are required to run the tests. Next, the `postgres` db - is created and migrated with `ecto`, to ensure it's up-to-date. - -- Finally, we'll leave the `mix` section unchanged. - -Let's take a look at the updated file after the changes: - -```yaml -image: elixir:latest - -services: - - postgres:latest - -variables: - POSTGRES_DB: hello_gitlab_ci_test - POSTGRES_HOST: postgres - POSTGRES_USER: postgres - POSTGRES_PASSWORD: "postgres" - MIX_ENV: "test" - -before_script: - - mix local.rebar --force - - mix local.hex --force - - mix deps.get --only test - - mix ecto.create - - mix ecto.migrate - -mix: - script: - - mix test -``` - -For safety, we can check if we get any syntax errors before submitting this file to GitLab. Copy the -contents of `.gitlab-ci.yml` and paste it on [GitLab CI/CD Lint tool](https://gitlab.com/ci/lint). Please note that -this link will only work for logged in users. - -## Watching the build - -I don't know about you, but I love to watch that black screen being filled with compilation output. -With this, I can feel the happiness of something I made working correctly. On `localhost` it's easy -to watch our build, but on GitLab, is it possible? Yes! - -Let's go to **Pipelines** and see GitLab doing the job. Just click on **Pipelines** to find the -actual running build job. - - - -Click on build's ID to watch the entire process. If everything went as expected, we can wait for the -**Build succeeded** at the end of the process! :) - -```shell -$ mix test -.... - -Finished in 0.3 seconds -4 tests, 0 failures - -Randomized with seed 206909 -Build succeeded -``` - -If we take a look at the project's main page on the GitLab UI, we can see the status of the last -build made by GitLab CI/CD. - -Time to show the world our green build badge! Navigate to your project's **Settings > CI/CD** and -expand **General pipelines settings**. Scroll down to **Pipeline status** and copy the Markdown code -for your badge. Paste it on the top of your `README.md` file, to let people outside of our project -see if our latest code is running without errors. - -When we finish this edition, GitLab will start another build and show a **build running** badge. It -is expected, after all we just configured GitLab CI/CD to do this for every push! But you may think -"Why run build and tests for simple things like editing README.md?" and it is a good question. -For changes that don't affect your application, you can add the keyword [`[ci skip]`](../../yaml/README.md#skip-pipeline) -to commit message and the build related to that commit will be skipped. - -In the end, we finally got our pretty green build succeeded badge! By outputting the result on the -README file, it shows to whoever lands on your project's page that your code is up-to-date and -working properly. - -## Conclusion - -When we have a growing application with many developers working on it, or when we have an open -source project being watched and contributed by the community, it is really important to have our -code permanently working. GitLab CI/CD is a time saving powerful tool to help us maintain our code -organized and working. - -As we could see in this post, GitLab CI/CD is really easy to configure and use. We have [many -other reasons](https://about.gitlab.com/blog/2015/02/03/7-reasons-why-you-should-be-using-ci/) to keep -using GitLab CI/CD. The benefits to our teams will be huge! - -## References - -- [GitLab CI/CD introductory guide](https://about.gitlab.com/blog/2015/12/14/getting-started-with-gitlab-and-gitlab-ci/) -- [GitLab CI/CD full Documentation](../../README.md) -- [GitLab Runner documentation](../../runners/README.md) -- [Using Docker images documentation](../../docker/using_docker_images.md) +<!-- This redirect file can be deleted after <2021-04-05>. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> |
