diff options
Diffstat (limited to 'doc')
22 files changed, 452 insertions, 201 deletions
diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index d7c45e7d91d..dc6a71e2ebd 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -49,25 +49,6 @@ Starting with GitLab 11.4, Gitaly is a replacement for NFS except when the [Elastic Search indexer](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer) is used. -### Network architecture - -- gitlab-rails shards repositories into "repository storages" -- gitlab-rails/config/gitlab.yml contains a map from storage names to - (Gitaly address, Gitaly token) pairs -- the `storage name` -\> `(Gitaly address, Gitaly token)` map in - gitlab.yml is the single source of truth for the Gitaly network - topology -- a (Gitaly address, Gitaly token) corresponds to a Gitaly server -- a Gitaly server hosts one or more storages -- Gitaly addresses must be specified in such a way that they resolve - correctly for ALL Gitaly clients -- Gitaly clients are: unicorn, sidekiq, gitlab-workhorse, - gitlab-shell, and Gitaly itself -- special case: a Gitaly server must be able to make RPC calls **to - itself** via its own (Gitaly address, Gitaly token) pair as - specified in gitlab-rails/config/gitlab.yml -- Gitaly servers must not be exposed to the public internet - Gitaly network traffic is unencrypted so you should use a firewall to restrict access to your Gitaly server. diff --git a/doc/administration/operations/filesystem_benchmarking.md b/doc/administration/operations/filesystem_benchmarking.md index 44018e966e0..0397452e650 100644 --- a/doc/administration/operations/filesystem_benchmarking.md +++ b/doc/administration/operations/filesystem_benchmarking.md @@ -44,12 +44,10 @@ user 0m0.025s sys 0m0.091s ``` -From experience with multiple customers, the following are ranges that indicate -whether your filesystem performance is satisfactory or less than ideal: - -| Rating | Benchmark result | -|:----------|:------------------------| -| Best | Less than 10 seconds | -| OK | 10-18 seconds | -| Poor | 18-25 seconds | -| Very poor | Greater than 25 seconds | +From experience with multiple customers, this task should take under 10 +seconds to indicate good filesystem performance. + +NOTE: **Note:** +This test is naive and only evaluates write performance. It's possible to +receive good results on this test but still have poor performance due to read +speed and various other factors.
\ No newline at end of file diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 6c77d82e37e..cc1423c335c 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1731,13 +1731,17 @@ include: --- -Since GitLab 10.8 we are now recursively merging the files defined in `include` +Since GitLab 10.8 we are now deep merging the files defined in `include` with those in `.gitlab-ci.yml`. Files defined by `include` are always -evaluated first and recursively merged with the content of `.gitlab-ci.yml`, no +evaluated first and merged with the content of `.gitlab-ci.yml`, no matter the position of the `include` keyword. You can take advantage of -recursive merging to customize and override details in included CI +merging to customize and override details in included CI configurations with local definitions. +NOTE: **Note:** +The recursive includes are not supported, meaning your external files +should not use the `include` keyword, as it will be ignored. + The following example shows specific YAML-defined variables and details of the `production` job from an include file being customized in `.gitlab-ci.yml`. @@ -1787,11 +1791,7 @@ with the environment url of the `production` job defined in `autodevops-template.yml` have been overridden by new values defined in `.gitlab-ci.yml`. -NOTE: **Note:** -Recursive includes are not supported meaning your external files -should not use the `include` keyword, as it will be ignored. - -Recursive merging lets you extend and override dictionary mappings, but +The merging lets you extend and override dictionary mappings, but you cannot add or modify items to an included array. For example, to add an additional item to the production job script, you must repeat the existing script items. diff --git a/doc/development/README.md b/doc/development/README.md index bcf57a223f5..f22dde32de9 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -52,7 +52,6 @@ description: 'Learn how to contribute to GitLab.' - [Prometheus metrics](prometheus_metrics.md) - [Guidelines for reusing abstractions](reusing_abstractions.md) - [DeclarativePolicy framework](policies.md) -- [Switching to Rails 5](switching_to_rails5.md) ## Performance guides diff --git a/doc/development/automatic_ce_ee_merge.md b/doc/development/automatic_ce_ee_merge.md index 8208e9007ea..0cc083cefc0 100644 --- a/doc/development/automatic_ce_ee_merge.md +++ b/doc/development/automatic_ce_ee_merge.md @@ -1,12 +1,13 @@ # Automatic CE->EE merge Commits pushed to CE `master` are automatically merged into EE `master` roughly -every 5 minutes. Changes are merged using the `ours` merge strategy in the -context of EE. This means that any merge conflicts are resolved by taking the EE -changes and discarding the CE changes. This removes the need for resolving -conflicts or reverting changes, at the cost of **absolutely requiring** EE merge -requests to be created whenever a CE merge request causes merge conflicts. -Failing to do so can result in changes not making their way into EE. +every 5 minutes. Changes are merged using the `recursive=ours` merge strategy in +the context of EE. This means that any merge conflicts are resolved by taking +the EE changes and discarding the CE changes. This removes the need for +resolving conflicts or reverting changes, at the cost of **absolutely +requiring** EE merge requests to be created whenever a CE merge request causes +merge conflicts. Failing to do so can result in changes not making their way +into EE. ## Always create an EE merge request if there are conflicts @@ -169,10 +170,6 @@ you are not required to submit the EE-equivalent MR, but it's still recommended. job failed, you are required to submit the EE MR so that you can fix the conflicts in EE before merging your changes into CE. ---- - -[Return to Development documentation](README.md) - ## FAQ ### How does automatic merging work? @@ -180,7 +177,8 @@ before merging your changes into CE. The automatic merging is performed using a project called [Merge Train](https://gitlab.com/gitlab-org/merge-train/). This project will clone CE and EE master, and merge CE master into EE master using `git merge ---strategy=ours`. This process runs roughly every 5 minutes. +--strategy=recursive --strategy-option=ours`. This process runs multiple times +per hour. For more information on the exact implementation you can refer to the source code. @@ -225,3 +223,7 @@ so that the EE specific changes are not intertwined with CE code. For Ruby code you can do this by moving the EE code to a separate module, which can then be injected into the appropriate classes or modules. See [Guidelines for implementing Enterprise Edition features](ee_features.md) for more information. + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index 8f1317e235d..ac910e80a89 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -12,7 +12,7 @@ are very appreciative of the work done by translators and proofreaders! - Bulgarian - Lyubomir Vasilev - [Crowdin](https://crowdin.com/profile/lyubomirv) - Catalan - - Proofreaders needed. + - David Planella - [GitLab](https://gitlab.com/dplanella), [Crowdin](https://crowdin.com/profile/dplanella) - Chinese Simplified - Huang Tao - [GitLab](https://gitlab.com/htve), [Crowdin](https://crowdin.com/profile/htve) - Chinese Traditional diff --git a/doc/development/new_fe_guide/style/prettier.md b/doc/development/new_fe_guide/style/prettier.md index baaea67d38b..4495f38f262 100644 --- a/doc/development/new_fe_guide/style/prettier.md +++ b/doc/development/new_fe_guide/style/prettier.md @@ -57,3 +57,38 @@ node ./scripts/frontend/prettier.js save-all ./vendor/ ``` This will go over all files in a specific folder and save it. + +## VSCode Settings + +### Format on Save + +To automatically format your files with Prettier, add the following properties to your User or Workspace Settings: + +```javascript +{ + "[javascript]": { + "editor.formatOnSave": true + }, + "[vue]": { + "editor.formatOnSave": true + }, +} +``` + +### Conflicts with Vetur Extension + +There are some [runtime issues](https://github.com/vuejs/vetur/issues/950) with [Prettier](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode) and [the Vetur extension](https://marketplace.visualstudio.com/items?itemName=octref.vetur) for VSCode. To fix this, try adding the following properties to your User or Workspace Settings: + +```javascript +{ + "prettier.disableLanguages": [], + "vetur.format.defaultFormatter.html": "none", + "vetur.format.defaultFormatter.js": "none", + "vetur.format.defaultFormatter.css": "none", + "vetur.format.defaultFormatter.less": "none", + "vetur.format.defaultFormatter.postcss": "none", + "vetur.format.defaultFormatter.scss": "none", + "vetur.format.defaultFormatter.stylus": "none", + "vetur.format.defaultFormatter.ts": "none", +} +``` diff --git a/doc/development/switching_to_rails5.md b/doc/development/switching_to_rails5.md deleted file mode 100644 index c9a4ce1a1d1..00000000000 --- a/doc/development/switching_to_rails5.md +++ /dev/null @@ -1,27 +0,0 @@ -# Switching to Rails 5 - -GitLab switched recently to Rails 5. This is a big change (especially for backend development) and it introduces couple of temporary inconveniences. - -## After the switch, I found a broken feature. What do I do? - -Many fixes and tweaks were done to make our codebase compatible with Rails 5, but it's possible that not all issues were found. If you find an bug, please create an issue and assign it the ~rails5 label. - -## It takes much longer to run CI pipelines that build GitLab. Why? - -We are temporarily running CI pipelines with Rails 4 and 5 so that we ensure we remain compatible with Rails 4 in case we must revert back to Rails 4 from Rails 5 (this can double the duration of CI pipelines). - -We might revert back to Rails 4 if we found a major issue we were unable to quickly fix. - -Once we are sure we can stay with Rails 5, we will stop running CI pipelines with Rails 4. - -## Can I skip running Rails 4 tests? - -If you are sure that your merge request doesn't introduce any incompatibility, you can just include `norails4` anywhere in your branch name and Rails 4 tests will be skipped. - -## CI is failing on my test with Rails 4. How can I debug it? - -You can run specs locally with Rails 4 using the following command: - -```sh -BUNDLE_GEMFILE=Gemfile.rails4 RAILS5=0 bundle exec rspec ... -``` diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index a63656fafef..57bc71d2903 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -657,6 +657,7 @@ Restoring database tables: - Loading fixture wikis...[SKIPPING] Restoring repositories: - Restoring repository abcd... [DONE] +- Object pool 1 ... Deleting tmp directories...[DONE] ``` diff --git a/doc/user/project/clusters/eks_and_gitlab/index.md b/doc/user/project/clusters/eks_and_gitlab/index.md index fa2ed21f980..0e6d4bce153 100644 --- a/doc/user/project/clusters/eks_and_gitlab/index.md +++ b/doc/user/project/clusters/eks_and_gitlab/index.md @@ -49,7 +49,7 @@ A few details from the EKS cluster will be required to connect it to GitLab: - Get the certificate with: ```sh - kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 -D + kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 --decode ``` 1. **Create admin token**: A `cluster-admin` token is required to install and diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 95bd5d1cf4e..db48388e90d 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -140,13 +140,14 @@ To add an existing Kubernetes cluster to your project: to grant access. - **Project namespace** (optional) - You don't have to fill it in; by leaving it blank, GitLab will create one for you. Also: - - Each project should have a unique namespace. - - The project namespace is not necessarily the namespace of the secret, if - you're using a secret with broader permissions, like the secret from `default`. - - You should **not** use `default` as the project namespace. - - If you or someone created a secret specifically for the project, usually - with limited permissions, the secret's namespace and project namespace may - be the same. + - Each project should have a unique namespace. + - The project namespace is not necessarily the namespace of the secret, if + you're using a secret with broader permissions, like the secret from `default`. + - You should **not** use `default` as the project namespace. + - If you or someone created a secret specifically for the project, usually + with limited permissions, the secret's namespace and project namespace may + be the same. + 1. Finally, click the **Create Kubernetes cluster** button. After a couple of minutes, your cluster will be ready to go. You can now proceed @@ -157,8 +158,8 @@ To determine the: - API URL, run `kubectl cluster-info | grep 'Kubernetes master' | awk '/http/ {print $NF}'`. - Token: 1. List the secrets by running: `kubectl get secrets`. Note the name of the secret you need the token for. - 1. Get the token for the appropriate secret by running: `kubectl get secret <SECRET_NAME> -o jsonpath="{['data']['token']}" | base64 -D`. -- CA certificate, run `kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 -D`. + 1. Get the token for the appropriate secret by running: `kubectl get secret <SECRET_NAME> -o jsonpath="{['data']['token']}" | base64 --decode`. +- CA certificate, run `kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 --decode`. ## Security implications @@ -168,7 +169,7 @@ are trusted, so **only trusted users should be allowed to control your clusters* The default cluster configuration grants access to a wide set of functionalities needed to successfully build and deploy a containerized -application. Bare in mind that the same credentials are used for all the +application. Bear in mind that the same credentials are used for all the applications running on the cluster. ## Access controls @@ -354,6 +355,18 @@ to reach your apps. This heavily depends on your domain provider, but in case you aren't sure, just create an A record with a wildcard host like `*.example.com.`. +## Multiple Kubernetes clusters **[PREMIUM]** + +> Introduced in [GitLab Premium][ee] 10.3. + +With GitLab Premium, you can associate more than one Kubernetes clusters to your +project. That way you can have different clusters for different environments, +like dev, staging, production, etc. + +Simply add another cluster, like you did the first time, and make sure to +[set an environment scope](#setting-the-environment-scope) that will +differentiate the new cluster with the rest. + ## Setting the environment scope **[PREMIUM]** When adding more than one Kubernetes clusters to your project, you need @@ -372,11 +385,11 @@ Also, jobs that don't have an environment keyword set will not be able to access For example, let's say the following Kubernetes clusters exist in a project: -| Cluster | Environment scope | -| ---------- | ------------------- | -| Development| `*` | -| Staging | `staging/*` | -| Production | `production/*` | +| Cluster | Environment scope | +| ----------- | ----------------- | +| Development | `*` | +| Staging | `staging` | +| Production | `production` | And the following environments are set in [`.gitlab-ci.yml`](../../../ci/yaml/README.md): @@ -393,14 +406,14 @@ deploy to staging: stage: deploy script: make deploy environment: - name: staging/$CI_COMMIT_REF_NAME + name: staging url: https://staging.example.com/ deploy to production: stage: deploy script: make deploy environment: - name: production/$CI_COMMIT_REF_NAME + name: production url: https://example.com/ ``` @@ -410,18 +423,6 @@ The result will then be: - The staging cluster will be used for the "deploy to staging" job. - The production cluster will be used for the "deploy to production" job. -## Multiple Kubernetes clusters - -> Introduced in [GitLab Premium][ee] 10.3. - -With GitLab Premium, you can associate more than one Kubernetes clusters to your -project. That way you can have different clusters for different environments, -like dev, staging, production, etc. - -Simply add another cluster, like you did the first time, and make sure to -[set an environment scope](#setting-the-environment-scope) that will -differentiate the new cluster with the rest. - ## Deployment variables The Kubernetes cluster integration exposes the following @@ -463,6 +464,14 @@ builds is that they must have a matching your build has no `environment:name` set, it will not be passed the Kubernetes credentials. +## Monitoring your Kubernetes cluster **[ULTIMATE]** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4701) in [GitLab Ultimate][ee] 10.6. + +When [Prometheus is deployed](#installing-applications), GitLab will automatically monitor the cluster's health. At the top of the cluster settings page, CPU and Memory utilization is displayed, along with the total amount available. Keeping an eye on cluster resources can be important, if the cluster runs out of memory pods may be shutdown or fail to start. + + + ## Enabling or disabling the Kubernetes cluster integration After you have successfully added your cluster information, you can enable the @@ -489,13 +498,16 @@ To remove the Kubernetes cluster integration from your project, simply click the **Remove integration** button. You will then be able to follow the procedure and add a Kubernetes cluster again. +## View Kubernetes pod logs from GitLab **[ULTIMATE]** + +Learn how to easily +[view the logs of running pods in connected Kubernetes clusters](https://docs.gitlab.com/ee/user/project/clusters/kubernetes_pod_logs.html). + ## What you can get with the Kubernetes integration Here's what you can do with GitLab if you enable the Kubernetes integration. -### Deploy Boards - -> Available in [GitLab Premium][ee]. +### Deploy Boards **[PREMIUM]** GitLab's Deploy Boards offer a consolidated view of the current health and status of each CI [environment](../../../ci/environments.md) running on Kubernetes, @@ -503,24 +515,22 @@ displaying the status of the pods in the deployment. Developers and other teammates can view the progress and status of a rollout, pod by pod, in the workflow they already use without any need to access Kubernetes. -[> Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) +[Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) -### Canary Deployments - -> Available in [GitLab Premium][ee]. +### Canary Deployments **[PREMIUM]** Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) and visualize your canary deployments right inside the Deploy Board, without the need to leave GitLab. -[> Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) +[Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) ### Kubernetes monitoring Automatically detect and monitor Kubernetes metrics. Automatic monitoring of [NGINX ingress](../integrations/prometheus_library/nginx.md) is also supported. -[> Read more about Kubernetes monitoring](../integrations/prometheus_library/kubernetes.md) +[Read more about Kubernetes monitoring](../integrations/prometheus_library/kubernetes.md) ### Auto DevOps @@ -530,7 +540,7 @@ applications. To make full use of Auto DevOps(Auto Deploy, Auto Review Apps, and Auto Monitoring) you will need the Kubernetes project integration enabled. -[> Read more about Auto DevOps](../../../topics/autodevops/index.md) +[Read more about Auto DevOps](../../../topics/autodevops/index.md) ### Web terminals @@ -546,8 +556,6 @@ containers. To use this integration, you should deploy to Kubernetes using the deployment variables above, ensuring any pods you create are labelled with `app=$CI_ENVIRONMENT_SLUG`. GitLab will do the rest! -## Read more - ### Integrating Amazon EKS cluster with GitLab - Learn how to [connect and deploy to an Amazon EKS cluster](eks_and_gitlab/index.md). diff --git a/doc/user/project/clusters/serverless/img/deploy-stage.png b/doc/user/project/clusters/serverless/img/deploy-stage.png Binary files differindex dc2f8af9c63..a1e0095bf29 100644 --- a/doc/user/project/clusters/serverless/img/deploy-stage.png +++ b/doc/user/project/clusters/serverless/img/deploy-stage.png diff --git a/doc/user/project/clusters/serverless/img/dns-entry.png b/doc/user/project/clusters/serverless/img/dns-entry.png Binary files differindex 2e7655c6041..678743f256e 100644 --- a/doc/user/project/clusters/serverless/img/dns-entry.png +++ b/doc/user/project/clusters/serverless/img/dns-entry.png diff --git a/doc/user/project/clusters/serverless/img/install-knative.png b/doc/user/project/clusters/serverless/img/install-knative.png Binary files differindex a9fcc127240..94f4c84181f 100644 --- a/doc/user/project/clusters/serverless/img/install-knative.png +++ b/doc/user/project/clusters/serverless/img/install-knative.png diff --git a/doc/user/project/clusters/serverless/img/knative-app.png b/doc/user/project/clusters/serverless/img/knative-app.png Binary files differindex 54301e1786f..a5b2945f6f4 100644 --- a/doc/user/project/clusters/serverless/img/knative-app.png +++ b/doc/user/project/clusters/serverless/img/knative-app.png diff --git a/doc/user/project/clusters/serverless/img/serverless-page.png b/doc/user/project/clusters/serverless/img/serverless-page.png Binary files differindex 473ee801f10..5d808fc2d4f 100644 --- a/doc/user/project/clusters/serverless/img/serverless-page.png +++ b/doc/user/project/clusters/serverless/img/serverless-page.png diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index bdbc4f7f09d..c1f2e844362 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -2,109 +2,204 @@ > Introduced in GitLab 11.5. +CAUTION: **Caution:** +Serverless is currently in [alpha](https://about.gitlab.com/handbook/product/#alpha). + Run serverless workloads on Kubernetes using [Knative](https://cloud.google.com/knative/). ## Overview Knative extends Kubernetes to provide a set of middleware components that are useful to build modern, source-centric, container-based applications. Knative brings some significant benefits out of the box through its main components: -- [Build:](https://github.com/knative/build) Source-to-container build orchestration -- [Eventing:](https://github.com/knative/eventing) Management and delivery of events -- [Serving:](https://github.com/knative/serving) Request-driven compute that can scale to zero +- [Build](https://github.com/knative/build): Source-to-container build orchestration. +- [Eventing](https://github.com/knative/eventing): Management and delivery of events. +- [Serving](https://github.com/knative/serving): Request-driven compute that can scale to zero. For more information on Knative, visit the [Knative docs repo](https://github.com/knative/docs). +With GitLab serverless, you can deploy both functions-as-a-service (FaaS) and serverless applications. + ## Requirements To run Knative on Gitlab, you will need: -1. **Kubernetes:** An RBAC-enabled Kubernetes cluster is required to deploy Knative. - The simplest way to get started is to add a cluster using [GitLab's GKE integration](https://docs.gitlab.com/ee/user/project/clusters/#adding-and-creating-a-new-gke-cluster-via-gitlab). - GitLab recommends -1. **Helm Tiller:** Helm is a package manager for Kubernetes and is required to install - all the other applications. -1. **Domain Name:** Knative will provide its own load balancer using Istio. It will provide an - external IP address for all the applications served by Knative. You will be prompted to enter a - wildcard domain where your applications will be served. Configure your DNS server to use the +1. **Kubernetes Cluster:** An RBAC-enabled Kubernetes cluster is required to deploy Knative. + The simplest way to get started is to add a cluster using [GitLab's GKE integration](../index.md#adding-and-creating-a-new-gke-cluster-via-gitlab). +1. **Helm Tiller:** Helm is a package manager for Kubernetes and is required to install + Knative. +1. **Domain Name:** Knative will provide its own load balancer using Istio. It will provide an + external IP address for all the applications served by Knative. You will be prompted to enter a + wildcard domain where your applications will be served. Configure your DNS server to use the external IP address for that domain. -1. **Serverless `gitlab-ci.yml` Template:** GitLab uses [Kaniko](https://github.com/GoogleContainerTools/kaniko) - to build the application and the [TriggerMesh CLI](https://github.com/triggermesh/tm), to simplify the +1. **`gitlab-ci.yml`:** GitLab uses [Kaniko](https://github.com/GoogleContainerTools/kaniko) + to build the application and the [TriggerMesh CLI](https://github.com/triggermesh/tm) to simplify the deployment of knative services and functions. - - Add the following `.gitlab-ci.yml` to the root of your repository (you may skip this step if using the sample - [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) mentioned below). - - ```yaml - stages: - - build - - deploy - - build: - stage: build - image: - name: gcr.io/kaniko-project/executor:debug - entrypoint: [""] - only: - - master - script: - - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json - - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE - - deploy: - stage: deploy - image: gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5 - only: - - master - environment: production - script: - - echo "$CI_REGISTRY_IMAGE" - - tm -n "$KUBE_NAMESPACE" --config "$KUBECONFIG" deploy service "$CI_PROJECT_NAME" --from-image "$CI_REGISTRY_IMAGE" --wait - ``` - -1. **Dockerfile:** Knative requires a Dockerfile in order to build your application. It should be included - at the root of your project's repo and expose port 8080. +1. **`serverless.yml`** (for [functions only](#deploying-functions)): When using serverless to deploy functions, the `serverless.yml` file + will contain the information for all the functions being hosted in the repository as well as a reference to the + runtime being used. +1. **`Dockerfile`:** Knative requires a `Dockerfile` in order to build your application. It should be included + at the root of your project's repo and expose port `8080`. ## Installing Knative via GitLab's Kubernetes integration NOTE: **Note:** -Minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.50 GB memory. RBAC must be enabled. - -You may download the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get started. - -1. [Add a Kubernetes cluster](https://docs.gitlab.com/ce/user/project/clusters/) and install Helm. +The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.50 GB memory. **RBAC must be enabled.** -1. Once Helm has been successfully installed, on the Knative app section, enter the domain to be used with +1. [Add a Kubernetes cluster](../index.md) and install Helm. +1. Once Helm has been successfully installed, on the Knative app section, enter the domain to be used with your application and click "Install".  -1. After the Knative installation has finished, retrieve the Istio Ingress IP address by running the following command: +1. After the Knative installation has finished, you can wait for the IP address to be displayed in the + **Knative IP Address** field or retrieve the Istio Ingress IP address by running the following command: - ```bash - kubectl get svc --namespace=istio-system knative-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} ' - ``` + ```bash + kubectl get svc --namespace=istio-system knative-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} ' + ``` - Output: + Output: - ```bash - 35.161.143.124 my-machine-name:~ my-user$ - ``` + ```bash + 35.161.143.124 my-machine-name:~ my-user$ + ``` -1. The ingress is now available at this address and will route incoming requests to the proper service based on the DNS - name in the request. To support this, a wildcard DNS A record should be created for the desired domain name. For example, - if your Knative base domain is `knative.example.com` then you need to create an A record with domain `*.knative.example.com` - pointing the ip address of the ingress. +1. The ingress is now available at this address and will route incoming requests to the proper service based on the DNS + name in the request. To support this, a wildcard DNS A record should be created for the desired domain name. For example, + if your Knative base domain is `knative.info` then you need to create an A record with domain `*.knative.info` + pointing the ip address of the ingress.  +## Deploying Functions + +> Introduced in GitLab 11.6. + +Using functions is useful for initiating, responding, or triggering independent +events without needing to maintain a complex unified infrastructure. This allows +you to focus on a single task that can be executed/scaled automatically and independently. + +In order to deploy functions to your Knative instance, the following templates must be present: + +1. `gitlab-ci.yml`: This template allows to define the stage, environment, and + image to be used for your functions. It must be included at the root of your repository: + + ```yaml + stages: + - deploy + + functions: + stage: deploy + environment: test + image: gcr.io/triggermesh/tm:v0.0.6 + script: + - tm -n "$KUBE_NAMESPACE" set registry-auth gitlab-registry --registry "$CI_REGISTRY" --username "$CI_REGISTRY_USER" --password "$CI_JOB_TOKEN" + - tm -n "$KUBE_NAMESPACE" --registry-host "$CI_REGISTRY_IMAGE" deploy --wait + ``` + +2. `serverless.yml`: This template contains the metadata for your functions, + such as name, runtime, and environment. It must be included at the root of your repository: + + ```yaml + service: knative-test + description: "Deploying functions from GitLab using Knative" + + provider: + name: triggermesh + registry-secret: gitlab-registry + environment: + FOO: BAR + + functions: + container: + handler: simple + description: "knative canonical sample" + runtime: https://gitlab.com/triggermesh/runtimes/raw/master/kaniko.yaml + buildargs: + - DIRECTORY=simple + environment: + SIMPLE_MSG: Hello + + nodejs: + handler: nodejs + runtime: https://gitlab.com/triggermesh/runtimes/raw/master/nodejs.yaml + description: "nodejs fragment" + buildargs: + - DIRECTORY=nodejs + environment: + FUNCTION: nodejs + ``` + +After the templates have been created, each function must be defined as a single +file in your repository. Committing a function to your project will result in a +CI pipeline being executed which will deploy each function as a Knative service. +Once the deploy stage has finished, additional details for the function will +appear under **Operations > Serverless**. + + + +This page contains all functions available for the project, the URL for +accessing the function, and if available, the function's runtime information. +The details are derived from the Knative installation inside each of the project's +Kubernetes cluster. + +The function details can be retrieved directly from Knative on the cluster: + +```bash +kubectl -n "$KUBE_NAMESPACE" get services.serving.knative.dev +``` + +Currently, the Serverless page presents all functions available in all clusters registered for the project with Knative installed. + +## Deploying Serverless applications + +> Introduced in GitLab 11.5. + +NOTE: **Note:** +You can reference the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get started. + +Add the following `.gitlab-ci.yml` to the root of your repository +(you may skip this step if using the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) mentioned above): + +```yaml +stages: + - build + - deploy + +build: + stage: build + image: + name: gcr.io/kaniko-project/executor:debug + entrypoint: [""] + only: + - master + script: + - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json + - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $CI_REGISTRY_IMAGE + +deploy: + stage: deploy + image: gcr.io/triggermesh/tm@sha256:e3ee74db94d215bd297738d93577481f3e4db38013326c90d57f873df7ab41d5 + only: + - master + environment: production + script: + - echo "$CI_REGISTRY_IMAGE" + - tm -n "$KUBE_NAMESPACE" --config "$KUBECONFIG" deploy service "$CI_PROJECT_NAME" --from-image "$CI_REGISTRY_IMAGE" --wait +``` + ## Deploy the application with Knative -With all the pieces in place, you can simply create a new CI pipeline to deploy the Knative application. Navigate to -**CI/CD >> Pipelines** and click the **Run Pipeline** button at the upper-right part of the screen. Then, on the +With all the pieces in place, you can create a new CI pipeline to deploy the Knative application. Navigate to +**CI/CD > Pipelines** and click the **Run Pipeline** button at the upper-right part of the screen. Then, on the Pipelines page, click **Create pipeline**. ## Obtain the URL for the Knative deployment +There are two ways to obtain the URL for the Knative deployment. + +### Using the CI/CD deployment job output + Once all the stages of the pipeline finish, click the **deploy** stage.  @@ -131,7 +226,7 @@ Service domain: knative.knative-4342902.knative.info Job succeeded ``` -The second to last line, labeled **Service domain** contains the URL for the deployment. Copy and paste the domain into your +The second to last line, labeled **Service domain** contains the URL for the deployment. Copy and paste the domain into your browser to see the app live. -
\ No newline at end of file + diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 7d0e567cae7..d9a2eeb32ae 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -41,6 +41,7 @@ Once you have a connected Kubernetes cluster with Helm installed, deploying a ma Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/kubernetes/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/). The Prometheus server will [automatically detect and monitor](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Ckubernetes_sd_config%3E) nodes, pods, and endpoints. To configure a resource to be monitored by Prometheus, simply set the following [Kubernetes annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/): + * `prometheus.io/scrape` to `true` to enable monitoring of the resource. * `prometheus.io/port` to define the port of the metrics endpoint. * `prometheus.io/path` to define the path of the metrics endpoint. Defaults to `/metrics`. diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index 247e8d2a6a0..cea9628966d 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -234,26 +234,25 @@ reiterating the importance of HTTPS. ## Issuing Certificates -GitLab Pages accepts [PEM](https://support.quovadisglobal.com/kb/a37/what-is-pem-format.aspx) certificates issued by -[Certificate Authorities (CA)](https://en.wikipedia.org/wiki/Certificate_authority) -and self-signed certificates. Of course, -[you'd rather issue a certificate than generate a self-signed](https://en.wikipedia.org/wiki/Self-signed_certificate), -for security reasons and for having browsers trusting your -site's certificate. - -There are several different kinds of certificates, each one -with certain security level. A static personal website will +GitLab Pages accepts certificates provided in the [PEM](https://support.quovadisglobal.com/kb/a37/what-is-pem-format.aspx) format, issued by +[Certificate Authorities (CAs)](https://en.wikipedia.org/wiki/Certificate_authority) or as +[self-signed certificates](https://en.wikipedia.org/wiki/Self-signed_certificate). Note that [self-signed certificates are typically not used](https://securingtomorrow.mcafee.com/other-blogs/mcafee-labs/self-signed-certificates-secure-so-why-ban/) +for public websites for security reasons and to ensure that browsers trust your site's certificate. + +There are various kinds of certificates, each one +with a certain security level. A static personal website will not require the same security level as an online banking web app, -for instance. There are a couple Certificate Authorities that +for instance. + +There are some certificate authorities that offer free certificates, aiming to make the internet more secure to everyone. The most popular is [Let's Encrypt](https://letsencrypt.org/), which issues certificates trusted by most of browsers, it's open -source, and free to use. Please read through this tutorial to -understand [how to secure your GitLab Pages website with Let's Encrypt](https://about.gitlab.com/2016/04/11/tutorial-securing-your-gitlab-pages-with-tls-and-letsencrypt/). +source, and free to use. See our tutorial on [how to secure your GitLab Pages website with Let's Encrypt](lets_encrypt_for_gitlab_pages.md). -With the same popularity, there are [certificates issued by CloudFlare](https://www.cloudflare.com/ssl/), +Similarly popular are [certificates issued by CloudFlare](https://www.cloudflare.com/ssl/), which also offers a [free CDN service](https://blog.cloudflare.com/cloudflares-free-cdn-and-you/). -Their certs are valid up to 15 years. Read through the tutorial on +Their certs are valid up to 15 years. See the tutorial on [how to add a CloudFlare Certificate to your GitLab Pages website](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/). ### Adding certificates to your project diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 7de9ae0caea..ce4fccdaff3 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -170,7 +170,7 @@ optionally secure it with SSL/TLS certificates. You can read the following tutorials to learn how to use these third-party certificates with GitLab Pages: - [CloudFlare](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/) -- [Let's Encrypt](https://about.gitlab.com/2016/04/11/tutorial-securing-your-gitlab-pages-with-tls-and-letsencrypt/) (mind that although this article is out-of-date, it can still be useful to guide you through the basic steps) +- [Let's Encrypt](lets_encrypt_for_gitlab_pages.md) ## Advanced use diff --git a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md new file mode 100644 index 00000000000..f639188684b --- /dev/null +++ b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md @@ -0,0 +1,158 @@ +--- +description: "How to secure GitLab Pages websites with Let's Encrypt." +--- + +# Let's Encrypt for GitLab Pages + +If you have a GitLab Pages website served under your own domain, +you might want to secure it with a SSL/TSL certificate. + +[Let's Encrypt](https://letsencrypt.org) is a free, automated, and +open source Certificate Authority. + +## Requirements + +To follow along with this tutorial, we assume you already have: + +- Created a [project](getting_started_part_two.md) in GitLab which + contains your website's source code. +- Acquired a domain (`example.com`) and added a [DNS entry](getting_started_part_three.md#dns-records) + pointing it to your Pages website. +- [Added your domain to your Pages project](getting_started_part_three.md#add-your-custom-domain-to-gitlab-pages-settings) + and verified your ownership. +- Cloned your project into your computer. +- Your website up and running, served under HTTP protocol at `http://example.com`. + +## Obtaining a Let's Encrypt certificate + +Once you have the requirements addressed, follow the instructions +below to learn how to obtain the certificate. + +NOTE: **Note:** +The instructions below were tested on macOS Mojave. For other +operating systems the steps might be slightly different. Follow the +[CertBot instructions](https://certbot.eff.org/) according to your OS. + +1. On your computer, open a terminal and navigate to your repository's + root directory: + + ```bash + cd path/to/dir + ``` + +1. Install CertBot (the tool Let's Encrypt uses to issue certificates): + + ```bash + brew install certbot + ``` + +1. Request a certificate for your domain (`example.com`) and + provide an email account (`your@email.com`) to receive notifications: + + ```bash + sudo certbot certonly -a manual -d example.com --email your@email.com + ``` + + Alternatively, you can register without adding an e-mail account, + but you won't be notified about the certificate expiration's date: + + ```bash + sudo certbot certonly -a manual -d example.com --register-unsafely-without-email + ``` + + TIP: **Tip:** + Read through CertBot's documentation on their + [command line options](https://certbot.eff.org/docs/using.html#certbot-command-line-options). + +1. You'll be prompted with a message to agree with their terms. + Press `A` to agree and `Y` to let they log your IP. + + CertBot will then prompt you with the following message: + + ```bash + Create a file containing just this data: + + Rxnv6WKo95hsuLVX3osmT6LgmzsJKSaK9htlPToohOP.HUGNKk82jlsmOOfphlt8Jy69iuglsn095nxOMH9j3Yb + + And make it available on your web server at this URL: + + http://example.com/.well-known/acme-challenge/Rxnv6WKo95hsuLVX3osmT6LgmzsJKSaK9htlPToohOP + + Press Enter to Continue + ``` + +1. **Do not press Enter yet.** Let's Encrypt will need to verify your + domain ownership before issuing the certificate. To do so, create 3 + consecutive directories under your website's root: + `/.well-known/acme-challenge/Rxnv6WKo95hsuLVX3osmT6LgmzsJKSaK9htlPToohOP/` + and add to the last folder an `index.html` file containing the content + referred on the previous prompt message: + + ```bash + Rxnv6WKo95hsuLVX3osmT6LgmzsJKSaK9htlPToohOP.HUGNKk82jlsmOOfphlt8Jy69iuglsn095nxOMH9j3Yb + ``` + + Note that this file needs to be accessed under + `http://example.com/.well-known/acme-challenge/Rxnv6WKo95hsuLVX3osmT6LgmzsJKSaK9htlPToohOP` + to allow Let's Encrypt to verify the ownership of your domain, + therefore, it needs to be part of the website content under the + repo's [`public`](index.md#how-it-works) folder. + +1. Add, commit, and push the file into your repo in GitLab. Once the pipeline + passes, press **Enter** on your terminal to continue issuing your + certificate. CertBot will then prompt you with the following message: + + ```bash + Waiting for verification... + Cleaning up challenges + + IMPORTANT NOTES: + - Congratulations! Your certificate and chain have been saved at: + /etc/letsencrypt/live/example.com/fullchain.pem + Your key file has been saved at: + /etc/letsencrypt/live/example.com/privkey.pem + Your cert will expire on 2019-03-12. To obtain a new or tweaked + version of this certificate in the future, simply run certbot + again. To non-interactively renew *all* of your certificates, run + "certbot renew" + - If you like Certbot, please consider supporting our work by: + + Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate + Donating to EFF: https://eff.org/donate-le + ``` + +## Add your certificate to GitLab Pages + +Now that your certificate has been issued, let's add it to your Pages site: + +1. Back at GitLab, navigate to your project's **Settings > Pages**, + find your domain and click **Details** and **Edit** to add your certificate. +1. From your terminal, copy and paste the certificate into the first field + **Certificate (PEM)**: + + ```bash + sudo cat /etc/letsencrypt/live/example.com/fullchain.pem | pbcopy + ``` + +1. Copy and paste the public key into the second field **Key (PEM)**: + + ```bash + sudo cat /etc/letsencrypt/live/example.com/privkey.pem | pbcopy + ``` + +1. Click **Save changes** to apply them to your website. +1. Wait a few minutes for DNS propagation. +1. Visit your website at `https://example.com`. + +To force `https` connections on your site, navigate to your +project's **Settings > Pages** and check **Force domains with SSL +certificates to use HTTPS**. + +## Renewal + +Let's Encrypt certificates expire every 90 days and you'll have to +renew them periodically. To renew all your certificates at once, run: + +```bash +sudo certbot renew +``` diff --git a/doc/workflow/notifications.md b/doc/workflow/notifications.md index 020aa73f809..6ce789998a4 100644 --- a/doc/workflow/notifications.md +++ b/doc/workflow/notifications.md @@ -135,6 +135,7 @@ Notification emails include headers that provide extra content about the notific | X-GitLab-Pipeline-Id | Only in pipeline emails, the ID of the pipeline the notification is for | | X-GitLab-Reply-Key | A unique token to support reply by email | | X-GitLab-NotificationReason | The reason for being notified. "mentioned", "assigned", etc | +| List-Id | The path of the project in a RFC 2919 mailing list identifier useful for email organization, for example, with GMail filters | #### X-GitLab-NotificationReason |
