diff options
Diffstat (limited to 'doc/ci/runners')
| -rw-r--r-- | doc/ci/runners/README.md | 289 | ||||
| -rw-r--r-- | doc/ci/runners/build_cloud/linux_build_cloud.md | 127 | ||||
| -rw-r--r-- | doc/ci/runners/build_cloud/windows_build_cloud.md | 156 | ||||
| -rw-r--r-- | doc/ci/runners/configure_runners.md | 36 | ||||
| -rw-r--r-- | doc/ci/runners/img/protected_runners_check_box.png | bin | 3963 -> 0 bytes | |||
| -rw-r--r-- | doc/ci/runners/img/protected_runners_check_box_v14_1.png | bin | 0 -> 3500 bytes | |||
| -rw-r--r-- | doc/ci/runners/img/shared_runner_ip_address.png | bin | 20276 -> 0 bytes | |||
| -rw-r--r-- | doc/ci/runners/img/shared_runner_ip_address_14_1.png | bin | 0 -> 8577 bytes | |||
| -rw-r--r-- | doc/ci/runners/index.md | 18 |
9 files changed, 323 insertions, 303 deletions
diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 45113c480c7..5ab8653dc35 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -1,289 +1,8 @@ --- -stage: Verify -group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference +redirect_to: 'index.md' --- -# GitLab SaaS runners +This document was moved to [another location](index.md). -If you are using self-managed GitLab or you want to use your own runners on GitLab.com, you can -[install and configure your own runners](https://docs.gitlab.com/runner/install/). - -If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on shared runners. No configuration is required. -Your jobs can run on [Linux](#linux-shared-runners) or [Windows](#windows-shared-runners-beta). - -The number of minutes you can use on these shared runners depends on your -[quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota), -which depends on your [subscription plan](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes). - -## Linux shared runners - -Linux shared runners on GitLab.com run in autoscale mode and are powered by Google Cloud Platform. - -Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available for users and customers on GitLab.com. - -GitLab offers Ultimate tier capabilities and included CI/CD minutes per group per month for our [Open Source](https://about.gitlab.com/solutions/open-source/join/), [Education](https://about.gitlab.com/solutions/education/), and [Startups](https://about.gitlab.com/solutions/startups/) programs. For private projects, GitLab offers various [plans](https://about.gitlab.com/pricing/), starting with a Free tier. - -All your CI/CD jobs run on [n1-standard-1 instances](https://cloud.google.com/compute/docs/machine-types) with 3.75GB of RAM, CoreOS and the latest Docker Engine -installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default -region of the VMs is US East1. -Each instance is used only for one job, this ensures any sensitive data left on the system can't be accessed by other people their CI jobs. - -The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don't run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times. - -Jobs handled by the shared runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), -**time out after 3 hours**, regardless of the timeout configured in a -project. Check the issues [4010](https://gitlab.com/gitlab-com/infrastructure/-/issues/4010) and [4070](https://gitlab.com/gitlab-com/infrastructure/-/issues/4070) for the reference. - -Below are the shared runners settings. - -| Setting | GitLab.com | Default | -| ----------- | ----------------- | ---------- | -| [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) | [Runner versions dashboard](https://dashboards.gitlab.com/d/000000159/ci?from=now-1h&to=now&refresh=5m&orgId=1&panelId=12&fullscreen&theme=light) | - | -| Executor | `docker+machine` | - | -| Default Docker image | `ruby:2.5` | - | -| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | - -### Pre-clone script - -Linux shared runners on GitLab.com provide a way to run commands in a CI -job before the runner attempts to run `git init` and `git fetch` to -download a GitLab repository. The -[`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) -can be used for: - -- Seeding the build directory with repository data -- Sending a request to a server -- Downloading assets from a CDN -- Any other commands that must run before the `git init` - -To use this feature, define a [CI/CD variable](../../ci/variables/README.md#custom-cicd-variables) called -`CI_PRE_CLONE_SCRIPT` that contains a bash script. - -[This example](../../development/pipelines.md#pre-clone-step) -demonstrates how you might use a pre-clone step to seed the build -directory. - -### `config.toml` - -The full contents of our `config.toml` are: - -NOTE: -Settings that are not public are shown as `X`. - -**Google Cloud Platform** - -```toml -concurrent = X -check_interval = 1 -metrics_server = "X" -sentry_dsn = "X" - -[[runners]] - name = "docker-auto-scale" - request_concurrency = X - url = "https://gitlab.com/" - token = "SHARED_RUNNER_TOKEN" - pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\"" - executor = "docker+machine" - environment = [ - "DOCKER_DRIVER=overlay2", - "DOCKER_TLS_CERTDIR=" - ] - limit = X - [runners.docker] - image = "ruby:2.5" - privileged = true - volumes = [ - "/certs/client", - "/dummy-sys-class-dmi-id:/sys/class/dmi/id:ro" # Make kaniko builds work on GCP. - ] - [runners.machine] - IdleCount = 50 - IdleTime = 3600 - MaxBuilds = 1 # For security reasons we delete the VM after job has finished so it's not reused. - MachineName = "srm-%s" - MachineDriver = "google" - MachineOptions = [ - "google-project=PROJECT", - "google-disk-size=25", - "google-machine-type=n1-standard-1", - "google-username=core", - "google-tags=gitlab-com,srm", - "google-use-internal-ip", - "google-zone=us-east1-d", - "engine-opt=mtu=1460", # Set MTU for container interface, for more information check https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3214#note_82892928 - "google-machine-image=PROJECT/global/images/IMAGE", - "engine-opt=ipv6", # This will create IPv6 interfaces in the containers. - "engine-opt=fixed-cidr-v6=fc00::/7", - "google-operation-backoff-initial-interval=2" # Custom flag from forked docker-machine, for more information check https://github.com/docker/machine/pull/4600 - ] - [[runners.machine.autoscaling]] - Periods = ["* * * * * sat,sun *"] - Timezone = "UTC" - IdleCount = 70 - IdleTime = 3600 - [[runners.machine.autoscaling]] - Periods = ["* 30-59 3 * * * *", "* 0-30 4 * * * *"] - Timezone = "UTC" - IdleCount = 700 - IdleTime = 3600 - [runners.cache] - Type = "gcs" - Shared = true - [runners.cache.gcs] - CredentialsFile = "/path/to/file" - BucketName = "bucket-name" -``` - -## Windows shared runners (beta) - -The Windows shared runners are in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) -and shouldn't be used for production workloads. - -During this beta period, the [shared runner pipeline quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota) -applies for groups and projects in the same manner as Linux runners. This may -change when the beta period ends, as discussed in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30834). - -Windows shared runners on GitLab.com autoscale by launching virtual machines on -the Google Cloud Platform. This solution uses an -[autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/tree/master/docs/readme.md) -developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). -Windows shared runners execute your CI/CD jobs on `n1-standard-2` instances with -2 vCPUs and 7.5 GB RAM. You can find a full list of available Windows packages in -the [package documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/master/cookbooks/preinstalled-software/README.md). - -We want to keep iterating to get Windows shared runners in a stable state and -[generally available](https://about.gitlab.com/handbook/product/gitlab-the-product/#generally-available-ga). -You can follow our work towards this goal in the -[related epic](https://gitlab.com/groups/gitlab-org/-/epics/2162). - -### Configuration - -The full contents of our `config.toml` are: - -NOTE: -Settings that aren't public are shown as `X`. - -```toml -concurrent = X -check_interval = 3 - -[[runners]] - name = "windows-runner" - url = "https://gitlab.com/" - token = "TOKEN" - executor = "custom" - builds_dir = "C:\\GitLab-Runner\\builds" - cache_dir = "C:\\GitLab-Runner\\cache" - shell = "powershell" - [runners.custom] - config_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - config_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "config"] - prepare_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - prepare_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "prepare"] - run_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - run_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "run"] - cleanup_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" - cleanup_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "cleanup"] -``` - -The full contents of our `autoscaler/config.toml` are: - -```toml -Provider = "gcp" -Executor = "winrm" -OS = "windows" -LogLevel = "info" -LogFormat = "text" -LogFile = "C:\\GitLab-Runner\\autoscaler\\autoscaler.log" -VMTag = "windows" - -[GCP] - ServiceAccountFile = "PATH" - Project = "some-project-df9323" - Zone = "us-east1-c" - MachineType = "n1-standard-2" - Image = "IMAGE" - DiskSize = 50 - DiskType = "pd-standard" - Subnetwork = "default" - Network = "default" - Tags = ["TAGS"] - Username = "gitlab_runner" - -[WinRM] - MaximumTimeout = 3600 - ExecutionMaxRetries = 0 - -[ProviderCache] - Enabled = true - Directory = "C:\\GitLab-Runner\\autoscaler\\machines" -``` - -### Example - -Below is a simple `.gitlab-ci.yml` file to show how to start using the -Windows shared runners: - -```yaml -.shared_windows_runners: - tags: - - shared-windows - - windows - - windows-1809 - -stages: - - build - - test - -before_script: - - Set-Variable -Name "time" -Value (date -Format "%H:%m") - - echo ${time} - - echo "started by ${GITLAB_USER_NAME}" - -build: - extends: - - .shared_windows_runners - stage: build - script: - - echo "running scripts in the build job" - -test: - extends: - - .shared_windows_runners - stage: test - script: - - echo "running scripts in the test job" -``` - -### Limitations and known issues - -- All the limitations mentioned in our [beta - definition](https://about.gitlab.com/handbook/product/#beta). -- The average provisioning time for a new Windows VM is 5 minutes. - This means that you may notice slower build start times - on the Windows shared runner fleet during the beta. In a future - release we intend to update the autoscaler to enable - the pre-provisioning of virtual machines. This is intended to significantly reduce - the time it takes to provision a VM on the Windows fleet. You can - follow along in the [related issue](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/issues/32). -- The Windows shared runner fleet may be unavailable occasionally - for maintenance or updates. -- The Windows shared runner virtual machine instances do not use the - GitLab Docker executor. This means that you can't specify - [`image`](../../ci/yaml/README.md#image) or [`services`](../../ci/yaml/README.md#services) in - your pipeline configuration. -- For the beta release, we have included a set of software packages in - the base VM image. If your CI job requires additional software that's - not included in this list, then you must add installation - commands to [`before_script`](../../ci/yaml/README.md#before_script) or [`script`](../../ci/yaml/README.md#script) to install the required - software. Note that each job runs on a new VM instance, so the - installation of additional software packages needs to be repeated for - each job in your pipeline. -- The job may stay in a pending state for longer than the - Linux shared runners. -- There is the possibility that we introduce breaking changes which will - require updates to pipelines that are using the Windows shared runner - fleet. +<!-- This redirect file can be deleted after 2021-09-28. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/build_cloud/linux_build_cloud.md b/doc/ci/runners/build_cloud/linux_build_cloud.md new file mode 100644 index 00000000000..710054921ef --- /dev/null +++ b/doc/ci/runners/build_cloud/linux_build_cloud.md @@ -0,0 +1,127 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Linux shared runners + +Linux shared runners on GitLab.com run in autoscale mode and are powered by Google Cloud Platform. + +Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available for users and customers on GitLab.com. + +GitLab offers Ultimate tier capabilities and included CI/CD minutes per group per month for our [Open Source](https://about.gitlab.com/solutions/open-source/join/), [Education](https://about.gitlab.com/solutions/education/), and [Startups](https://about.gitlab.com/solutions/startups/) programs. For private projects, GitLab offers various [plans](https://about.gitlab.com/pricing/), starting with a Free tier. + +All your CI/CD jobs run on [n1-standard-1 instances](https://cloud.google.com/compute/docs/machine-types) with 3.75GB of RAM, CoreOS and the latest Docker Engine +installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default +region of the VMs is US East1. +Each instance is used only for one job, this ensures any sensitive data left on the system can't be accessed by other people their CI jobs. + +The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don't run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times. + +Jobs handled by the shared runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), +**time out after 3 hours**, regardless of the timeout configured in a +project. Check the issues [4010](https://gitlab.com/gitlab-com/infrastructure/-/issues/4010) and [4070](https://gitlab.com/gitlab-com/infrastructure/-/issues/4070) for the reference. + +Below are the shared runners settings. + +| Setting | GitLab.com | Default | +| ----------- | ----------------- | ---------- | +| Executor | `docker+machine` | - | +| Default Docker image | `ruby:2.5` | - | +| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | + +## Pre-clone script + +Linux shared runners on GitLab.com provide a way to run commands in a CI +job before the runner attempts to run `git init` and `git fetch` to +download a GitLab repository. The +[`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) +can be used for: + +- Seeding the build directory with repository data +- Sending a request to a server +- Downloading assets from a CDN +- Any other commands that must run before the `git init` + +To use this feature, define a [CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables) called +`CI_PRE_CLONE_SCRIPT` that contains a bash script. + +[This example](../../../development/pipelines.md#pre-clone-step) +demonstrates how you might use a pre-clone step to seed the build +directory. + +NOTE: +The `CI_PRE_CLONE_SCRIPT` variable does not work on Windows runners. + +## `config.toml` + +The full contents of our `config.toml` are: + +NOTE: +Settings that are not public are shown as `X`. + +**Google Cloud Platform** + +```toml +concurrent = X +check_interval = 1 +metrics_server = "X" +sentry_dsn = "X" + +[[runners]] + name = "docker-auto-scale" + request_concurrency = X + url = "https://gitlab.com/" + token = "SHARED_RUNNER_TOKEN" + pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\"" + executor = "docker+machine" + environment = [ + "DOCKER_DRIVER=overlay2", + "DOCKER_TLS_CERTDIR=" + ] + limit = X + [runners.docker] + image = "ruby:2.5" + privileged = true + volumes = [ + "/certs/client", + "/dummy-sys-class-dmi-id:/sys/class/dmi/id:ro" # Make kaniko builds work on GCP. + ] + [runners.machine] + IdleCount = 50 + IdleTime = 3600 + MaxBuilds = 1 # For security reasons we delete the VM after job has finished so it's not reused. + MachineName = "srm-%s" + MachineDriver = "google" + MachineOptions = [ + "google-project=PROJECT", + "google-disk-size=25", + "google-machine-type=n1-standard-1", + "google-username=core", + "google-tags=gitlab-com,srm", + "google-use-internal-ip", + "google-zone=us-east1-d", + "engine-opt=mtu=1460", # Set MTU for container interface, for more information check https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3214#note_82892928 + "google-machine-image=PROJECT/global/images/IMAGE", + "engine-opt=ipv6", # This will create IPv6 interfaces in the containers. + "engine-opt=fixed-cidr-v6=fc00::/7", + "google-operation-backoff-initial-interval=2" # Custom flag from forked docker-machine, for more information check https://github.com/docker/machine/pull/4600 + ] + [[runners.machine.autoscaling]] + Periods = ["* * * * * sat,sun *"] + Timezone = "UTC" + IdleCount = 70 + IdleTime = 3600 + [[runners.machine.autoscaling]] + Periods = ["* 30-59 3 * * * *", "* 0-30 4 * * * *"] + Timezone = "UTC" + IdleCount = 700 + IdleTime = 3600 + [runners.cache] + Type = "gcs" + Shared = true + [runners.cache.gcs] + CredentialsFile = "/path/to/file" + BucketName = "bucket-name" +``` diff --git a/doc/ci/runners/build_cloud/windows_build_cloud.md b/doc/ci/runners/build_cloud/windows_build_cloud.md new file mode 100644 index 00000000000..5a85f28e4b9 --- /dev/null +++ b/doc/ci/runners/build_cloud/windows_build_cloud.md @@ -0,0 +1,156 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Windows shared runners (beta) + +The Windows shared runners are in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) +and shouldn't be used for production workloads. + +During this beta period, the [shared runner pipeline quota](../../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota) +applies for groups and projects in the same manner as Linux runners. This may +change when the beta period ends, as discussed in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30834). + +Windows shared runners on GitLab.com autoscale by launching virtual machines on +the Google Cloud Platform. This solution uses an +[autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/tree/master/docs/readme.md) +developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). +Windows shared runners execute your CI/CD jobs on `n1-standard-2` instances with +2 vCPUs and 7.5 GB RAM. You can find a full list of available Windows packages in +the [package documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/master/cookbooks/preinstalled-software/README.md). + +We want to keep iterating to get Windows shared runners in a stable state and +[generally available](https://about.gitlab.com/handbook/product/gitlab-the-product/#generally-available-ga). +You can follow our work towards this goal in the +[related epic](https://gitlab.com/groups/gitlab-org/-/epics/2162). + +## Configuration + +The full contents of our `config.toml` are: + +NOTE: +Settings that aren't public are shown as `X`. + +```toml +concurrent = X +check_interval = 3 + +[[runners]] + name = "windows-runner" + url = "https://gitlab.com/" + token = "TOKEN" + executor = "custom" + builds_dir = "C:\\GitLab-Runner\\builds" + cache_dir = "C:\\GitLab-Runner\\cache" + shell = "powershell" + [runners.custom] + config_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" + config_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "config"] + prepare_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" + prepare_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "prepare"] + run_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" + run_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "run"] + cleanup_exec = "C:\\GitLab-Runner\\autoscaler\\autoscaler.exe" + cleanup_args = ["--config", "C:\\GitLab-Runner\\autoscaler\\config.toml", "custom", "cleanup"] +``` + +The full contents of our `autoscaler/config.toml` are: + +```toml +Provider = "gcp" +Executor = "winrm" +OS = "windows" +LogLevel = "info" +LogFormat = "text" +LogFile = "C:\\GitLab-Runner\\autoscaler\\autoscaler.log" +VMTag = "windows" + +[GCP] + ServiceAccountFile = "PATH" + Project = "some-project-df9323" + Zone = "us-east1-c" + MachineType = "n1-standard-2" + Image = "IMAGE" + DiskSize = 50 + DiskType = "pd-standard" + Subnetwork = "default" + Network = "default" + Tags = ["TAGS"] + Username = "gitlab_runner" + +[WinRM] + MaximumTimeout = 3600 + ExecutionMaxRetries = 0 + +[ProviderCache] + Enabled = true + Directory = "C:\\GitLab-Runner\\autoscaler\\machines" +``` + +## Example + +Below is a simple `.gitlab-ci.yml` file to show how to start using the +Windows shared runners: + +```yaml +.shared_windows_runners: + tags: + - shared-windows + - windows + - windows-1809 + +stages: + - build + - test + +before_script: + - Set-Variable -Name "time" -Value (date -Format "%H:%m") + - echo ${time} + - echo "started by ${GITLAB_USER_NAME}" + +build: + extends: + - .shared_windows_runners + stage: build + script: + - echo "running scripts in the build job" + +test: + extends: + - .shared_windows_runners + stage: test + script: + - echo "running scripts in the test job" +``` + +## Limitations and known issues + +- All the limitations mentioned in our [beta + definition](https://about.gitlab.com/handbook/product/#beta). +- The average provisioning time for a new Windows VM is 5 minutes. + This means that you may notice slower build start times + on the Windows shared runner fleet during the beta. In a future + release we intend to update the autoscaler to enable + the pre-provisioning of virtual machines. This is intended to significantly reduce + the time it takes to provision a VM on the Windows fleet. You can + follow along in the [related issue](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/issues/32). +- The Windows shared runner fleet may be unavailable occasionally + for maintenance or updates. +- The Windows shared runner virtual machine instances do not use the + GitLab Docker executor. This means that you can't specify + [`image`](../../../ci/yaml/index.md#image) or [`services`](../../../ci/yaml/index.md#services) in + your pipeline configuration. +- For the beta release, we have included a set of software packages in + the base VM image. If your CI job requires additional software that's + not included in this list, then you must add installation + commands to [`before_script`](../../../ci/yaml/index.md#before_script) or [`script`](../../../ci/yaml/index.md#script) to install the required + software. Note that each job runs on a new VM instance, so the + installation of additional software packages needs to be repeated for + each job in your pipeline. +- The job may stay in a pending state for longer than the + Linux shared runners. +- There is the possibility that we introduce breaking changes which will + require updates to pipelines that are using the Windows shared runner + fleet. diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 775de26b772..2f845a05a4e 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -10,7 +10,7 @@ type: reference If you have installed your own runners, you can configure and secure them in GitLab. If you need to configure runners on the machine where you installed GitLab Runner, see -[the GitLab Runner documentation](https://docs.gitlab.com/runner/configuration). +[the GitLab Runner documentation](https://docs.gitlab.com/runner/configuration/). ## Manually clear the runner cache @@ -19,14 +19,14 @@ Read [clearing the cache](../caching/index.md#clearing-the-cache). ## Set maximum job timeout for a runner For each runner, you can specify a *maximum job timeout*. This timeout, -if smaller than the [project defined timeout](../pipelines/settings.md#timeout), takes precedence. +if smaller than the [project defined timeout](../pipelines/settings.md#set-a-limit-for-how-long-jobs-can-run), takes precedence. This feature can be used to prevent your shared runner from being overwhelmed by a project that has jobs with a long timeout (for example, one week). When not configured, runners do not override the project timeout. -On GitLab.com, you cannot override the job timeout for shared runners and must use the [project defined timeout](../pipelines/settings.md#timeout). +On GitLab.com, you cannot override the job timeout for shared runners and must use the [project defined timeout](../pipelines/settings.md#set-a-limit-for-how-long-jobs-can-run). To set the maximum job timeout: @@ -90,7 +90,7 @@ To protect or unprotect a runner: 1. Check the **Protected** option. 1. Click **Save changes**. - + ### Forks @@ -146,7 +146,7 @@ the GitLab instance. To determine this: 1. On the left sidebar, select **Overview > Runners**. 1. Find the runner in the table and view the **IP Address** column. - + ### Determine the IP address of a specific runner @@ -169,13 +169,13 @@ GitLab CI tags are not the same as Git tags. GitLab CI tags are associated with Git tags are associated with commits. By tagging a runner for the types of jobs it can handle, you can make sure -shared runners will [only run the jobs they are equipped to run](../yaml/README.md#tags). +shared runners will [only run the jobs they are equipped to run](../yaml/index.md#tags). For instance, at GitLab we have runners tagged with `rails` if they contain the appropriate dependencies to run Rails test suites. When you [register a runner](https://docs.gitlab.com/runner/register/), its default behavior is to **only pick** -[tagged jobs](../yaml/README.md#tags). +[tagged jobs](../yaml/index.md#tags). To change this, you must have the [Owner role](../../user/permissions.md#project-members-permissions) for the project. To make a runner pick untagged jobs: @@ -230,7 +230,7 @@ Example 2: ## Configure runner behavior with variables -You can use [CI/CD variables](../variables/README.md) to configure runner Git behavior +You can use [CI/CD variables](../variables/index.md) to configure runner Git behavior globally or for individual jobs: - [`GIT_STRATEGY`](#git-strategy) @@ -250,7 +250,7 @@ You can also use variables to configure how many times a runner > - `GIT_STRATEGY=none` requires GitLab Runner v1.7+. You can set the `GIT_STRATEGY` used to fetch the repository content, either -globally or per-job in the [`variables`](../yaml/README.md#variables) section: +globally or per-job in the [`variables`](../yaml/index.md#variables) section: ```yaml variables: @@ -258,7 +258,7 @@ variables: ``` There are three possible values: `clone`, `fetch`, and `none`. If left unspecified, -jobs use the [project's pipeline setting](../pipelines/settings.md#git-strategy). +jobs use the [project's pipeline setting](../pipelines/settings.md#choose-the-default-git-strategy). `clone` is the slowest option. It clones the repository from scratch for every job, ensuring that the local working copy is always pristine. @@ -281,7 +281,7 @@ The `kubernetes` executor always clones into an temporary directory. A Git strategy of `none` also re-uses the local working copy, but skips all Git operations normally done by GitLab. GitLab Runner pre-clone scripts are also skipped, if present. This strategy could mean you need to add `fetch` and `checkout` commands -to [your `.gitlab-ci.yml` script](../yaml/README.md#script). +to [your `.gitlab-ci.yml` script](../yaml/index.md#script). It can be used for jobs that operate exclusively on artifacts, like a deployment job. Git repository data may be present, but it's likely out of date. You should only @@ -293,7 +293,7 @@ rely on files brought into the local working copy from cache or artifacts. The `GIT_SUBMODULE_STRATEGY` variable is used to control if / how Git submodules are included when fetching the code before a build. You can set them -globally or per-job in the [`variables`](../yaml/README.md#variables) section. +globally or per-job in the [`variables`](../yaml/index.md#variables) section. There are three possible values: `none`, `normal`, and `recursive`: @@ -332,7 +332,7 @@ For this feature to work correctly, the submodules must be configured The `GIT_CHECKOUT` variable can be used when the `GIT_STRATEGY` is set to either `clone` or `fetch` to specify whether a `git checkout` should be run. If not specified, it defaults to true. You can set them globally or per-job in the -[`variables`](../yaml/README.md#variables) section. +[`variables`](../yaml/index.md#variables) section. If set to `false`, the runner: @@ -360,7 +360,7 @@ script: The `GIT_CLEAN_FLAGS` variable is used to control the default behavior of `git clean` after checking out the sources. You can set it globally or per-job in the -[`variables`](../yaml/README.md#variables) section. +[`variables`](../yaml/index.md#variables) section. `GIT_CLEAN_FLAGS` accepts all possible options of the [`git clean`](https://git-scm.com/docs/git-clean) command. @@ -386,7 +386,7 @@ script: > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4142) in GitLab Runner 13.1. The `GIT_FETCH_EXTRA_FLAGS` variable is used to control the behavior of -`git fetch`. You can set it globally or per-job in the [`variables`](../yaml/README.md#variables) section. +`git fetch`. You can set it globally or per-job in the [`variables`](../yaml/index.md#variables) section. `GIT_FETCH_EXTRA_FLAGS` accepts all options of the [`git fetch`](https://git-scm.com/docs/git-fetch) command. However, `GIT_FETCH_EXTRA_FLAGS` flags are appended after the default flags that can't be modified. @@ -428,7 +428,7 @@ It can be helpful for repositories with a large number of commits or old, large passed to `git fetch` and `git clone`. In GitLab 12.0 and later, newly-created projects automatically have a -[default `git depth` value of `50`](../pipelines/settings.md#git-shallow-clone). +[default `git depth` value of `50`](../pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone). If you use a depth of `1` and have a queue of jobs or retry jobs, jobs may fail. @@ -450,7 +450,7 @@ variables: GIT_DEPTH: "3" ``` -You can set it globally or per-job in the [`variables`](../yaml/README.md#variables) section. +You can set it globally or per-job in the [`variables`](../yaml/index.md#variables) section. ### Custom build directories @@ -559,7 +559,7 @@ variables: GET_SOURCES_ATTEMPTS: 3 ``` -You can set them globally or per-job in the [`variables`](../yaml/README.md#variables) section. +You can set them globally or per-job in the [`variables`](../yaml/index.md#variables) section. ## System calls not available on GitLab.com shared runners diff --git a/doc/ci/runners/img/protected_runners_check_box.png b/doc/ci/runners/img/protected_runners_check_box.png Binary files differdeleted file mode 100644 index 3c47ebdec29..00000000000 --- a/doc/ci/runners/img/protected_runners_check_box.png +++ /dev/null diff --git a/doc/ci/runners/img/protected_runners_check_box_v14_1.png b/doc/ci/runners/img/protected_runners_check_box_v14_1.png Binary files differnew file mode 100644 index 00000000000..d67085d83f9 --- /dev/null +++ b/doc/ci/runners/img/protected_runners_check_box_v14_1.png diff --git a/doc/ci/runners/img/shared_runner_ip_address.png b/doc/ci/runners/img/shared_runner_ip_address.png Binary files differdeleted file mode 100644 index 527b4f4043d..00000000000 --- a/doc/ci/runners/img/shared_runner_ip_address.png +++ /dev/null diff --git a/doc/ci/runners/img/shared_runner_ip_address_14_1.png b/doc/ci/runners/img/shared_runner_ip_address_14_1.png Binary files differnew file mode 100644 index 00000000000..d7eeeae7a3c --- /dev/null +++ b/doc/ci/runners/img/shared_runner_ip_address_14_1.png diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md new file mode 100644 index 00000000000..da7289a0ebd --- /dev/null +++ b/doc/ci/runners/index.md @@ -0,0 +1,18 @@ +--- +stage: Verify +group: Runner +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference +--- + +# GitLab SaaS runners + +If you are using self-managed GitLab or you want to use your own runners on GitLab.com, you can +[install and configure your own runners](https://docs.gitlab.com/runner/install/). + +If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on shared runners. No configuration is required. +Your jobs can run on [Linux](build_cloud/linux_build_cloud.md) or [Windows](build_cloud/windows_build_cloud.md). + +The number of minutes you can use on these shared runners depends on your +[quota](../../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota), +which depends on your [subscription plan](../../subscriptions/gitlab_com/index.md#ci-pipeline-minutes). |
