diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-05-19 07:33:21 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-05-19 07:33:21 +0000 |
| commit | 36a59d088eca61b834191dacea009677a96c052f (patch) | |
| tree | e4f33972dab5d8ef79e3944a9f403035fceea43f /doc/user/project/clusters | |
| parent | a1761f15ec2cae7c7f7bbda39a75494add0dfd6f (diff) | |
| download | gitlab-ce-36a59d088eca61b834191dacea009677a96c052f.tar.gz | |
Add latest changes from gitlab-org/gitlab@15-0-stable-eev15.0.0-rc42
Diffstat (limited to 'doc/user/project/clusters')
22 files changed, 50 insertions, 1873 deletions
diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index 82106c9d1a9..935bc01bae7 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -226,7 +226,7 @@ on the running pod. If you are using a self-managed GitLab instance, you need to configure Amazon credentials. GitLab uses these credentials to assume an Amazon IAM role to create your cluster. -Create an IAM user and ensure it has permissions to assume the role(s) that +Create an IAM user and ensure it has permissions to assume the roles that your users need to create EKS clusters. For example, the following policy document allows assuming a role whose name starts with diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md index f2d537513b7..c55c11151ce 100644 --- a/doc/user/project/clusters/add_existing_cluster.md +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -27,7 +27,7 @@ To add any cluster to GitLab, you need: - Either a GitLab.com account or an account for a self-managed installation running GitLab 12.5 or later. - The Maintainer role for group-level and project-level clusters. -- Access to the Admin area for instance-level clusters. +- Access to the Admin Area for instance-level clusters. - A Kubernetes cluster. - Cluster administration access to the cluster with `kubectl`. @@ -230,3 +230,22 @@ kubectl create clusterrolebinding permissive-binding \ --user=kubelet \ --group=system:serviceaccounts ``` + +## Troubleshooting + +### `There was a problem authenticating with your cluster. Please ensure your CA Certificate and Token are valid` + +If you encounter this error while connecting a Kubernetes cluster, ensure you're +properly pasting the service token. Some shells may add a line break to the +service token, making it invalid. Ensure that there are no line breaks by +pasting your token into an editor and removing any additional spaces. + +You may also experience this error if your certificate is not valid. To check that your certificate's +subject alternative names contain the correct domain for your cluster's API, run this command: + +```shell +echo | openssl s_client -showcerts -connect kubernetes.example.com:443 2>/dev/null | +openssl x509 -inform pem -noout -text +``` + +The `-connect` argument expects a `host:port` combination. For example, `https://kubernetes.example.com` would be `kubernetes.example.com:443`. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index a4f6dc325c8..8cdd1792e7f 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -10,60 +10,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. -To create a new cluster use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac). - -NOTE: -Every new Google Cloud Platform (GCP) account receives -[$300 in credit upon sign up](https://console.cloud.google.com/freetrial). -In partnership with Google, GitLab is able to offer an additional $200 for new GCP -accounts to get started with the GitLab integration with Google Kubernetes Engine. -[Follow this link](https://cloud.google.com/partners/partnercredit/?pcn_code=0014M00001h35gDQAQ#contact-form) -to apply for credit. - -NOTE: -Watch the webcast [Scalable app deployment with GitLab and Google Cloud Platform](https://about.gitlab.com/webcast/scalable-app-deploy/) -and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) -in a few clicks. - -## Create new cluster - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. - -As of GitLab 14.0, use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac) -to **safely create new clusters from GitLab**. - -Creating clusters from GitLab using cluster certificates is still available on the -GitLab UI but was **deprecated** in GitLab 14.0 and is scheduled for removal in -GitLab 15.0. We don't recommend using this method. - -You can create a new cluster hosted in EKS, GKE, on premises, and with other -providers using cluster certificates: - -- [New cluster hosted on Google Kubernetes Engine (GKE)](add_gke_clusters.md). -- [New cluster hosted on Amazon Elastic Kubernetes Service (EKS)](add_eks_clusters.md). - -To host them on premises and with other providers, you can use Terraform -or your preferred tool of choice to create and connect a cluster with GitLab. -The [GitLab Terraform provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs/resources/project_cluster) -supports connecting existing clusters using the certificate-based connection method. - -## Add existing cluster - -As of GitLab 14.0, use the [GitLab agent](../../clusters/agent/index.md) -to connect your cluster to GitLab. - -Alternatively, you can [add an existing cluster](add_existing_cluster.md) -through the certificate-based method, but we don't recommend using this method for [security implications](../../infrastructure/clusters/connect/index.md#security-implications-for-clusters-connected-with-certificates). - -## Configure your cluster - -As of GitLab 14.0, use the [GitLab agent](../../clusters/agent/index.md) -to configure your cluster. +To create and manage a new cluster use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac). ## Disable a cluster -When you successfully create a new Kubernetes cluster or add an existing -one to GitLab, the cluster connection to GitLab becomes enabled. To disable it: +When you successfully connect an existing cluster using cluster certificates, the cluster connection to GitLab becomes enabled. To disable it: 1. Go to your: - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. @@ -95,26 +46,3 @@ To remove the Kubernetes cluster integration: 1. Go to your cluster details page. 1. Select the **Advanced Settings** tab. 1. Select either **Remove integration** or **Remove integration and resources**. - -## Access controls - -See [cluster access controls (RBAC or ABAC)](cluster_access.md). - -## Troubleshooting - -### There was a problem authenticating with your cluster. Please ensure your CA Certificate and Token are valid - -If you encounter this error while adding a Kubernetes cluster, ensure you're -properly pasting the service token. Some shells may add a line break to the -service token, making it invalid. Ensure that there are no line breaks by -pasting your token into an editor and removing any additional spaces. - -You may also experience this error if your certificate is not valid. To check that your certificate's -subject alternative names contain the correct domain for your cluster's API, run this: - -```shell -echo | openssl s_client -showcerts -connect kubernetes.example.com:443 2>/dev/null | -openssl x509 -inform pem -noout -text -``` - -Note that the `-connect` argument expects a `host:port` combination. For example, `https://kubernetes.example.com` would be `kubernetes.example.com:443`. diff --git a/doc/user/project/clusters/cluster_access.md b/doc/user/project/clusters/cluster_access.md index 8ff0a275649..43ceb3673d8 100644 --- a/doc/user/project/clusters/cluster_access.md +++ b/doc/user/project/clusters/cluster_access.md @@ -28,7 +28,7 @@ Helm also creates additional service accounts and other resources for each installed application. Consult the documentation of the Helm charts for each application for details. -If you are [adding an existing Kubernetes cluster](add_remove_clusters.md#add-existing-cluster), +If you are [adding an existing Kubernetes cluster](add_existing_cluster.md), ensure the token of the account has administrator privileges for the cluster. The resources created by GitLab differ depending on the type of cluster. diff --git a/doc/user/project/clusters/deploy_to_cluster.md b/doc/user/project/clusters/deploy_to_cluster.md index c1cdf754c11..fc41533b17c 100644 --- a/doc/user/project/clusters/deploy_to_cluster.md +++ b/doc/user/project/clusters/deploy_to_cluster.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect your cluster to GitLab, use the [GitLab agent](../../clusters/agent/index.md). -To deploy with the agent, use the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). +To deploy with the agent, use the [CI/CD workflow](../../clusters/agent/ci_cd_workflow.md). A Kubernetes cluster can be the destination for a deployment job. If diff --git a/doc/user/project/clusters/gitlab_managed_clusters.md b/doc/user/project/clusters/gitlab_managed_clusters.md index 9c5cc14f720..e295abf8d31 100644 --- a/doc/user/project/clusters/gitlab_managed_clusters.md +++ b/doc/user/project/clusters/gitlab_managed_clusters.md @@ -9,15 +9,19 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 11.5. > - Became [optional](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26565) in GitLab 11.11. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect your cluster to GitLab, use the [GitLab agent](../../../user/clusters/agent/index.md). To manage applications, use the [Cluster Project Management Template](../../../user/clusters/management_project_template.md). +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `certificate_based_clusters`. + You can choose to allow GitLab to manage your cluster for you. If your cluster is managed by GitLab, resources for your projects are automatically created. See -the [Access controls](add_remove_clusters.md#access-controls) section for +the [Access controls](cluster_access.md) section for details about the created resources. If you choose to manage your own cluster, project-specific resources aren't created diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index b5e2a1bad51..b1158be9fb6 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -4,14 +4,23 @@ group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Kubernetes Logs (DEPRECATED) **(FREE)** +# Kubernetes Logs (DEPRECATED) **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4752) in GitLab 11.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26383) from GitLab Ultimate to GitLab Free 12.9. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/360182) behind a [feature flag](../../../administration/feature_flags.md) named `monitor_logging` in GitLab 15.0. Disabled by default. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: +This feature is in its end-of-life process. This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +It will be [removed completely](https://gitlab.com/gitlab-org/gitlab/-/issues/346485) in GitLab 15.2. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `monitor_logging` and the one named `certificate_based_clusters`. +On GitLab.com, this feature is not available. +This feature is not recommended for production use. GitLab makes it easy to view the logs of running pods in [connected Kubernetes clusters](index.md). By displaying the logs directly in GitLab diff --git a/doc/user/project/clusters/protect/container_host_security/index.md b/doc/user/project/clusters/protect/container_host_security/index.md deleted file mode 100644 index c897100f14e..00000000000 --- a/doc/user/project/clusters/protect/container_host_security/index.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Container Host Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Host Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -Container Host Security in GitLab provides Intrusion Detection and Prevention capabilities that can -monitor and (optionally) block activity inside the containers themselves. This is done by leveraging -an integration with Falco to provide the monitoring capabilities and an integration with Pod -Security Policies and AppArmor to provide blocking capabilities. - -## Overview - -Container Host Security can be used to monitor and block activity inside a container as well as to -enforce security policies across the entire Kubernetes cluster. Falco profiles allow for users to -define the activity they want to monitor for and detect. Among other things, this can include system -log entries, process starts, file activity, and network ports opened. AppArmor is used to block any -undesired activity via AppArmor profiles. These profiles are loaded into the cluster when -referenced by Pod Security Policies. - -By default, Container Host Security is deployed into the cluster in monitor mode only, with no -default profiles or rules running out-of-the-box. Activity monitoring and blocking begins only when -users define profiles for these technologies. - -## Installation - -See the [installation guide](quick_start_guide.md) for the recommended steps to install the -Container Host Security capabilities. This guide shows the recommended way of installing Container -Host Security through the Cluster Management Project. However, it's also possible to do a manual -installation through our Helm chart. - -## Features - -- Prevent containers from starting as root. -- Limit the privileges and system calls available to containers. -- Monitor system logs, process starts, files read/written/deleted, and network ports opened. -- Optionally block processes from starting or files from being read/written/deleted. - -## Supported container orchestrators - -Kubernetes v1.14+ is the only supported container orchestrator. OpenShift and other container -orchestrators aren't supported. - -## Supported Kubernetes providers - -The following cloud providers are supported: - -- Amazon EKS -- Google GKE - -Although Container Host Security may function on Azure or self-managed Kubernetes instances, it isn't -officially tested and supported on those providers. - -## Roadmap - -See the [Category Direction page](https://about.gitlab.com/direction/protect/container_host_security/) -for more information on the product direction of Container Host Security. diff --git a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md deleted file mode 100644 index af3128e3006..00000000000 --- a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Getting started with Container Host Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Host Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The following steps are recommended for installing Container Host Security. - -## Installation steps - -The following steps are recommended to install and use Container Host Security through GitLab: - -1. [Install at least one runner and connect it to GitLab](https://docs.gitlab.com/runner/). -1. [Create a group](../../../../group/#create-a-group). -1. [Connect a Kubernetes cluster to the group](../../add_remove_clusters.md). -1. [Create a cluster management project and associate it with the Kubernetes cluster](../../../../clusters/management_project.md). - -1. Install and configure an Ingress node: - - - [Install the Ingress node via CI/CD (Cluster Management Project)](../../../../clusters/applications.md#install-ingress-using-gitlab-cicd). - - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../gitlab_managed_clusters.md#base-domain) - into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes - cluster. - -1. [Install and configure Falco](../../../../clusters/applications.md#install-falco-using-gitlab-cicd) - for activity monitoring. -1. [Install and configure AppArmor](../../../../clusters/applications.md#install-apparmor-using-gitlab-cicd) - for activity blocking. -1. [Configure Pod Security Policies](../../../../clusters/applications.md#using-podsecuritypolicy-in-your-deployments) - (required to be able to load AppArmor profiles). - -It's possible to install and manage Falco and AppArmor in other ways, such as installing them -manually in a Kubernetes cluster and then connecting it back to GitLab. These methods aren't -supported or documented. - -## Viewing the logs - -Falco logs can be viewed by running the following command in your Kubernetes cluster: - -```shell -kubectl -n gitlab-managed-apps logs -l app=falco -``` - -## Troubleshooting - -### Trouble connecting to the cluster - -Your CI/CD pipeline may occasionally fail or have trouble connecting to the cluster. Here are some -initial troubleshooting steps that resolve the most common problems: - -1. [Clear the cluster cache](../../gitlab_managed_clusters.md#clearing-the-cluster-cache) -1. If things still aren't working, a more assertive set of actions may help get things back to a - good state: - - - Stop and [delete the problematic environment](../../../../../ci/environments/#delete-a-stopped-environment) - in GitLab. - - Delete the relevant namespace in Kubernetes by running - `kubectl delete namespaces <insert-some-namespace-name>` in your Kubernetes cluster. - - Rerun the application project pipeline to redeploy the application. - -**Related documentation links:** - -- [Cluster Management Project](../../../../clusters/management_project.md) diff --git a/doc/user/project/clusters/protect/container_network_security/index.md b/doc/user/project/clusters/protect/container_network_security/index.md deleted file mode 100644 index b294859c660..00000000000 --- a/doc/user/project/clusters/protect/container_network_security/index.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Container Network Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Network Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -Container Network Security in GitLab provides basic firewall functionality by leveraging Cilium -NetworkPolicies to filter traffic going in and out of the cluster as well as traffic between pods -inside the cluster. Container Network Security can be used to enforce L3, L4, and L7 policies and -can prevent an attacker with control over one pod from spreading laterally to access other pods in -the same cluster. Both Ingress and Egress rules are supported. - -By default, Cilium is deployed in Detection-only mode and only logs attack attempts. GitLab provides -a set of out-of-the-box policies as examples and to help users get started. These policies are -disabled by default, as they must usually be customized to match application-specific needs. - -## Installation - -See the [installation guide](quick_start_guide.md) for the recommended steps to install GitLab -Container Network Security. This guide shows the recommended way of installing Container Network -Security through the Cluster Management Project. However, it's also possible to install Cilium -manually through our Helm chart. - -## Features - -- GitLab managed installation of Cilium. -- Support for L3, L4, and L7 policies. -- Ability to export logs to a SIEM. -- Statistics page showing volume of packets processed and dropped over time (Ultimate users only). -- Management of NetworkPolicies through code in a project (Available for auto DevOps users only). -- Management of CiliumNetworkPolicies through a UI policy manager (Ultimate users only). - -## Supported container orchestrators - -Kubernetes v1.14+ is the only supported container orchestrator. OpenShift and other container -orchestrators aren't supported. - -## Supported Kubernetes providers - -The following cloud providers are supported: - -- Amazon EKS -- Google GKE - -Although Container Network Security may function on Azure or self-managed Kubernetes instances, it -isn't officially tested and supported on those providers. - -## Supported NetworkPolicies - -GitLab only supports the use of CiliumNetworkPolicies. Although generic Kubernetes NetworkPolicies -or other kinds of NetworkPolicies may work, GitLab doesn't test or support them. - -## Managing NetworkPolicies through GitLab vs your cloud provider - -Some cloud providers offer integrations with Cilium or offer other ways to manage NetworkPolicies in -Kubernetes. GitLab Container Network Security doesn't support deployments that have NetworkPolicies -managed by an external provider. By choosing to manage NetworkPolicies through GitLab, you can take -advantage of the following benefits: - -- Support for handling NetworkPolicy infrastructure as code. -- Full revision history and audit log of all changes made. -- Ability to revert back to a previous version at any time. - -## Roadmap - -See the [Category Direction page](https://about.gitlab.com/direction/protect/container_network_security/) -for more information on the product direction of Container Network Security. diff --git a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md deleted file mode 100644 index 7671ed7eb73..00000000000 --- a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md +++ /dev/null @@ -1,230 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Getting started with Container Network Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Network Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The following steps are recommended for installing Container Network Security. - -## Installation steps - -The following steps are recommended to install and use Container Network Security through GitLab: - -1. [Install at least one runner and connect it to GitLab](https://docs.gitlab.com/runner/). -1. [Create a group](../../../../group/#create-a-group). -1. [Connect a Kubernetes cluster to the group](../../add_remove_clusters.md). -1. [Create a cluster management project and associate it with the Kubernetes cluster](../../../../clusters/management_project.md). - -1. Install and configure an Ingress node: - - - [Install the Ingress node via CI/CD (Cluster Management Project)](../../../../clusters/applications.md#install-ingress-using-gitlab-cicd). - - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../gitlab_managed_clusters.md#base-domain) - into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes - cluster. - -1. [Install and configure Cilium](#use-the-cluster-management-template-to-install-cilium). -1. Be sure to restart all pods that were running before Cilium was installed by running this command - in your cluster: - - `kubectl get pods --all-namespaces -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HOSTNETWORK:.spec.hostNetwork --no-headers=true | grep '<none>' | awk '{print "-n "$1" "$2}' | xargs -L 1 -r kubectl delete pod` - - You can skip this step if `nodeinit.restartPods` is set to `true` on your Helm chart. - -It's possible to install and manage Cilium in other ways. For example, you could use the GitLab Helm -chart to install Cilium manually in a Kubernetes cluster, and then connect it back to GitLab. -However, such methods aren't documented or officially supported by GitLab. - -### Use the Cluster Management template to install Cilium - -[Cilium](https://cilium.io/) is a networking plug-in for Kubernetes that you can use to implement -support for [`NetworkPolicy`](https://kubernetes.io/docs/concepts/services-networking/network-policies/) -resources. For more information, see [Network Policies](../../../../../topics/autodevops/stages.md#network-policy). - -You can use the [Cluster Management Project Template](../../../../clusters/management_project_template.md) -to install Cilium in your Kubernetes cluster. - -1. In your cluster management project, go to `helmfile.yaml` and uncomment `- path: applications/cilium/helmfile.yaml`. -1. In `applications/cilium/helmfile.yaml`, set `clusterType` to either `gke` or `eks` based on which Kubernetes provider your are using. - - ```yaml - environments: - default: - values: - # Set to "gke" or "eks" based on your cluster type - - clusterType: "" - ``` - -1. Merge or push these changes to the default branch of your cluster management project, -and [GitLab CI/CD](../../../../../ci/index.md) will automatically install Cilium. - -WARNING: -Installation and removal of the Cilium requires a **manual** -[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods) -of all affected pods in all namespaces to ensure that they are -[managed](https://docs.cilium.io/en/stable/operations/troubleshooting/#ensure-managed-pod) -by the correct networking plug-in. When Hubble is enabled, its related pod might require a -restart depending on whether it started prior to Cilium. For more information, see -[Failed Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment) -in the Kubernetes docs. - -NOTE: -Major upgrades might require additional setup steps. For more information, see -the official [upgrade guide](https://docs.cilium.io/en/stable/operations/upgrade/). - -Support for installing the Cilium application is provided by the -GitLab Container Security group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). - -### Configure the Cilium Helm chart - -You can customize Cilium's Helm variables by editing the `applications/cilium/values.yaml` -file in your cluster management project. Refer to the [Cilium Helm reference](https://docs.cilium.io/en/stable/helm-reference/) -for the available configuration options. - -By default, Cilium's -[audit mode](https://docs.cilium.io/en/stable/gettingstarted/policy-creation/#enable-policy-audit-mode) -is enabled. In audit mode, Cilium doesn't drop disallowed packets. You -can use `policy-verdict` log to observe policy-related decisions. You -can disable audit mode by setting `policyAuditMode: false` in -`applications/cilium/values.yaml`. - -The Cilium monitor log for traffic is logged out by the -`cilium-monitor` sidecar container. You can check these logs with the following command: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -You can disable the monitor log in `application/cilium/values.yaml`: - -```yaml -monitor: - enabled: false -``` - -The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default -and it's set to collect per namespace flow metrics. This metrics are accessible on the -[Threat Monitoring](../../../../application_security/threat_monitoring/index.md) -dashboard. You can disable Hubble by adding the following to -`applications/cilium/values.yaml`: - -```yaml -hubble: - enabled: false -``` - -You can also adjust Helm values for Hubble by using -`applications/cilium/values.yaml`: - -```yaml -hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -## Managing Network Policies - -Managing NetworkPolicies through GitLab is advantageous over managing the policies in Kubernetes -directly. Kubernetes doesn't provide a GUI editor, a change control process, or a revision history. -Network Policies can be managed through GitLab in one of two ways: - -- Management through a YAML file in each application's project (for projects using Auto DevOps). For - more information, see the [Network Policy documentation](../../../../../topics/autodevops/stages.md#network-policy). -- Management through the GitLab Policy management UI (for projects not using Auto DevOps). For more - information, see the [Container Network Policy documentation](../../../../application_security/policies/index.md#container-network-policy) (Ultimate only). - -Each method has benefits and drawbacks: - -| | YAML method | UI method (Ultimate only) | -|--|:------------|:-------------------------------| -| **Benefits** | A change control process is possible by requiring [MR Approvals](../../../merge_requests/approvals/index.md). All changes are fully tracked and audited in the same way that Git tracks the history of any file in its repository. | The UI provides a simple rules editor for users who are less familiar with the YAML syntax of NetworkPolicies. This view is a live representation of the policies currently deployed in the Kubernetes cluster. The UI also allows for multiple network policies to be created per environment. | -| **Drawbacks** | Only one network policy can be deployed per environment (although that policy can be as detailed as needed). Also, if changes were made in Kubernetes directly rather than through the `auto-deploy-values.yaml` file, the YAML file's contents don't represent the actual state of policies deployed in Kubernetes. | Policy changes aren't audited and a change control process isn't available. | - -Users are encouraged to choose one of the two methods to manage their policies. If users attempt to -use both methods simultaneously, when the application project pipeline runs the contents of the -NetworkPolicy in the `auto-deploy-values.yaml` file may override policies configured in the UI -editor. - -## Monitoring throughput **(ULTIMATE)** - -To view statistics for Container Network Security, you must follow the installation steps above and -configure GitLab integration with Prometheus. Also, if you use custom Helm values for Cilium, you -must enable Hubble with flow metrics for each namespace by adding the following lines to -your [Cilium values](#use-the-cluster-management-template-to-install-cilium): - -```yaml -hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -Additional information about the statistics page is available in the -[documentation that describes the Threat Management UI](../../../../application_security/policies/index.md#container-network-policy). - -## Forwarding logs to a SIEM - -Cilium logs can be forwarded to a SIEM or an external logging system through syslog protocol by -installing and configuring Fluentd. Fluentd can be installed through the GitLab -[Cluster Management Project](../../../../clusters/applications.md#install-fluentd-using-gitlab-cicd). - -## Viewing the logs - -Cilium logs can be viewed by running the following command in your Kubernetes cluster: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -## Troubleshooting - -### Traffic is not being blocked as expected - -By default, Cilium is installed in Audit mode only, meaning that NetworkPolicies log policy -violations but don't block any traffic. To set Cilium to Blocking mode, you must add the following -lines to the `applications/cilium/values.yaml` file in your cluster management project: - -```yaml -policyEnforcementMode: "always" - -monitor: - eventTypes: ["drop", "policy-verdict"] -``` - -### Traffic is not being allowed as expected - -Keep in mind that when Cilium is set to blocking mode (rather than Audit mode), NetworkPolicies -operate on an allow-list basis. If one or more NetworkPolicies apply to a node, then all traffic -that doesn't match at least one Policy is blocked. To resolve, add NetworkPolicies defining the -traffic that you want to allow in the node. - -### Trouble connecting to the cluster - -Occasionally, your CI/CD pipeline may fail or have trouble connecting to the cluster. Here are some -initial troubleshooting steps that resolve the most common problems: - -1. [Clear the cluster cache](../../gitlab_managed_clusters.md#clearing-the-cluster-cache). -1. If things still aren't working, a more assertive set of actions may help get things back into a - good state: - - - Stop and [delete the problematic environment](../../../../../ci/environments/index.md#delete-a-stopped-environment) in GitLab. - - Delete the relevant namespace in Kubernetes by running `kubectl delete namespaces <insert-some-namespace-name>` in your Kubernetes cluster. - - Rerun the application project pipeline to redeploy the application. - -**Related documentation links:** - -- [Cluster Management Project](../../../../clusters/management_project.md) diff --git a/doc/user/project/clusters/protect/index.md b/doc/user/project/clusters/protect/index.md deleted file mode 100644 index 6b89f7f1557..00000000000 --- a/doc/user/project/clusters/protect/index.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Protecting your deployed applications **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -The Container Network Security and Container Host Security features are in their end-of-life -processes. They're -[deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -GitLab makes it straightforward to protect applications deployed in [connected Kubernetes clusters](index.md). -These protections are available in the Kubernetes network layer and in the container itself. At -the network layer, the Container Network Security capabilities in GitLab provide basic firewall -functionality by leveraging Cilium NetworkPolicies to filter traffic going in and out of the cluster -and traffic between pods inside the cluster. Inside the container, Container Host Security provides -Intrusion Detection and Prevention capabilities that can monitor and block activity inside the -containers themselves. - -## Capabilities - -The following capabilities are available to protect deployed applications in Kubernetes: - -- Container Network Security - - [Overview](container_network_security/index.md) - - [Installation guide](container_network_security/quick_start_guide.md) -- Container Host Security - - [Overview](container_host_security/index.md) - - [Installation guide](container_host_security/quick_start_guide.md) diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 9d623518f72..086e1fccf6c 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -40,8 +40,8 @@ for an overview of how this is accomplished in GitLab! To create an executable runbook, you need: - **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the - applications. The simplest way to get started is to add a cluster using one - of the [GitLab integrations](../add_remove_clusters.md#create-new-cluster). + applications. The simplest way to get started is to connect a cluster using the + [GitLab agent](../../../clusters/agent/index.md). - **Ingress** - Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications. - **JupyterHub** - [JupyterHub](https://jupyterhub.readthedocs.io/) is a multi-user diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index cf571abbf8a..93bc41dc24c 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -2,506 +2,10 @@ stage: Configure group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../../../update/removals.md#gitlab-serverless' --- -# Deploying AWS Lambda function using GitLab CI/CD **(FREE)** +# Deploying AWS Lambda function using GitLab CI/CD (removed) **(FREE)** -GitLab allows users to easily deploy AWS Lambda functions and create rich serverless applications. - -GitLab supports deployment of AWS Lambda functions through GitLab CI/CD using the following Serverless frameworks: - -- [Serverless Framework with AWS](#serverless-framework) -- [AWS' Serverless Application Model (SAM)](#aws-serverless-application-model) - -## Serverless Framework - -The [Serverless Framework can deploy to AWS](https://www.serverless.com/framework/docs/providers/aws/). - -We have prepared an example with a step-by-step guide to create a simple function and deploy it on AWS. - -Additionally, in the [How To section](#how-to), you can read about different use cases like: - -- Running a function locally. -- Working with secrets. -- Setting up CORS. - -Alternatively, you can quickly [create a new project with a template](../../working_with_projects.md#create-a-project). The [`Serverless Framework/JS` template](https://gitlab.com/gitlab-org/project-templates/serverless-framework/) already includes all parts described below. - -### Example - -This example shows you how to: - -1. Create a basic AWS Lambda Node.js function. -1. Link the function to an API Gateway `GET` endpoint. - -#### Steps - -The example consists of the following steps: - -1. Creating a Lambda handler function. -1. Creating a `serverless.yml` file. -1. Crafting the `.gitlab-ci.yml` file. -1. Setting up your AWS credentials with your GitLab account. -1. Deploying your function. -1. Testing the deployed function. - -Lets take it step by step. - -#### Creating a Lambda handler function - -Your Lambda function is the primary handler of requests. In this case, create a very simple Node.js `hello` function: - -```javascript -'use strict'; - -module.exports.hello = async event => { - return { - statusCode: 200, - body: JSON.stringify( - { - message: 'Your function executed successfully!' - }, - null, - 2 - ), - }; -}; -``` - -Place this code in the file `src/handler.js`. - -`src` is the standard location for serverless functions, but is customizable should you desire that. - -In our case, `module.exports.hello` defines the `hello` handler to reference later in the `serverless.yml`. - -You can learn more about the [AWS Lambda Node.js function handler](https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html) and all its various options in its documentation. - -#### Creating a `serverless.yml` file - -In the root of your project, create a `serverless.yml` file containing configuration specifics for the Serverless Framework. - -Put the following code in the file: - -```yaml -service: gitlab-example -provider: - name: aws - runtime: nodejs14.x - -functions: - hello: - handler: src/handler.hello - events: - - http: GET hello -``` - -Our function contains a handler and a event. - -The handler definition provisions the Lambda function using the source code located `src/handler.hello`. - -The `events` declaration creates an AWS API Gateway `GET` endpoint to receive external requests and hand them over to the Lambda function via a service integration. - -You can read more about the [available properties and additional configuration possibilities](https://www.serverless.com/framework/docs/providers/aws/guide/serverless.yml/) of the Serverless Framework. - -#### Crafting the `.gitlab-ci.yml` file - -In a `.gitlab-ci.yml` file in the root of your project, place the following code: - -```yaml -image: node:latest - -stages: - - deploy - -production: - stage: deploy - before_script: - - npm config set prefix /usr/local - - npm install -g serverless - script: - - serverless deploy --stage production --verbose - environment: production -``` - -This example code does the following: - -1. Uses the `node:latest` image for all GitLab CI/CD builds -1. The `deploy` stage: - - Installs the Serverless Framework. - - Deploys the serverless function to your AWS account using the AWS credentials - defined above. - -#### Setting up your AWS credentials with your GitLab account - -In order to interact with your AWS account, the GitLab CI/CD pipelines require both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be defined in your GitLab settings under **Settings > CI/CD > Variables**. -For more information please see [Create a custom variable in the UI](../../../../ci/variables/index.md#custom-variables-validated-by-gitlab). - - The AWS credentials you provide must include IAM policies that provision correct - access control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. - -#### Deploying your function - -`git push` the changes to your GitLab repository and the GitLab build pipeline deploys your function. - -Your GitLab deploy stage log contains output containing your AWS Lambda endpoint URL, -with log lines similar to this: - -```plaintext -endpoints: - GET - https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello -``` - -#### Manually testing your function - -Running the following `curl` command should trigger your function. -Your URL should be the one retrieved from the GitLab deploy stage log: - -```shell -curl "https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello" -``` - -That should output: - -```json -{ - "message": "Your function executed successfully!" -} -``` - -Hooray! You now have a AWS Lambda function deployed via GitLab CI/CD. - -Nice work! - -### How To - -In this section, we show you how to build on the basic example to: - -- Run the function locally. -- Set up secret variables. -- Set up CORS. - -#### Running function locally - -The `serverless-offline` plugin allows to run your code locally. To run your code locally: - -1. Add the following to your `serverless.yml`: - - ```yaml - plugins: - - serverless-offline - ``` - -1. Start the service by running the following command: - - ```shell - serverless offline - ``` - -Running the following `curl` command should trigger your function. - -```shell -curl "http://localhost:3000/hello" -``` - -It should output: - -```json -{ - "message": "Your function executed successfully!" -} -``` - -#### Secret variables - -Secrets are injected into your functions using environment variables. - -By defining variables in the provider section of the `serverless.yml`, you add them to -the environment of the deployed function: - -```yaml -provider: - # Other configuration omitted - # ... - environment: - A_VARIABLE: ${env:A_VARIABLE} -``` - -From there, you can reference them in your functions as well. -Remember to add `A_VARIABLE` to your GitLab CI/CD variables under **Settings > CI/CD > Variables** to be picked up and deployed with your function. - -NOTE: -Anyone with access to the AWS environment may be able to see the values of those -variables persisted in the lambda definition. - -#### Setting up CORS - -If you want to set up a web page that makes calls to your function, like we have done in the [template](https://gitlab.com/gitlab-org/project-templates/serverless-framework/), you need to deal with the Cross-Origin Resource Sharing (CORS). - -The quick way to do that is to add the `cors: true` flag to the HTTP endpoint in your `serverless.yml`: - -```yaml -functions: - hello: - handler: src/handler.hello - events: - - http: # Rewrite this part to enable CORS - path: hello - method: get - cors: true # <-- CORS here -``` - -You also need to return CORS specific headers in your function response: - -```javascript -'use strict'; - -module.exports.hello = async event => { - return { - statusCode: 200, - headers: { - // Uncomment the line below if you need access to cookies or authentication - // 'Access-Control-Allow-Credentials': true, - 'Access-Control-Allow-Origin': '*' - }, - body: JSON.stringify( - { - message: 'Your function executed successfully!' - }, - null, - 2 - ), - }; -}; -``` - -For more information, see the [Your CORS and API Gateway survival guide](https://www.serverless.com/blog/cors-api-gateway-survival-guide/) -blog post written by the Serverless Framework team. - -#### Writing automated tests - -The [Serverless Framework](https://gitlab.com/gitlab-org/project-templates/serverless-framework/) -example project shows how to use Jest, Axios, and `serverless-offline` plugin to do -automated testing of both local and deployed serverless function. - -### Examples and template - -The example code is available: - -- As a [clonable repository](https://gitlab.com/gitlab-org/serverless/examples/serverless-framework-js). -- In a version with [tests and secret variables](https://gitlab.com/gitlab-org/project-templates/serverless-framework/). - -You can also use a [template](../../working_with_projects.md#create-a-project) -(based on the version with tests and secret variables) from within the GitLab UI (see -the `Serverless Framework/JS` template). - -## AWS Serverless Application Model - -AWS Serverless Application Model is an open source framework for building serverless -applications. It makes it easier to build and deploy serverless applications. For more -details, please take a look at AWS documentation on [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/). - -### Deploying AWS Lambda function using AWS SAM and GitLab CI/CD - -GitLab allows developers to build and deploy serverless applications using the combination of: - -- [AWS Serverless Application Model (AWS SAM)](https://aws.amazon.com/serverless/sam/). -- GitLab CI/CD. - -### Example - -This example shows you how to: - -- Install SAM CLI. -- Create a sample SAM application including a Lambda function and API Gateway. -- Build and deploy the application to your AWS account using GitLab CI/CD. - -### Steps - -The example consists of the following steps: - -1. Installing SAM CLI. -1. Creating an AWS SAM application using SAM CLI. -1. Crafting the `.gitlab-ci.yml` file. -1. Setting up your AWS credentials with your GitLab account. -1. Deploying your application. -1. Testing the deployed function. - -### Installing SAM CLI - -AWS SAM provides a CLI called AWS SAM CLI to make it easier to create and manage -applications. - -Some steps in this documentation use SAM CLI. Follow the instructions for -[installing SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) -to install and configure SAM CLI. - -If you use [AWS Cloud9](https://aws.amazon.com/cloud9/) as your integrated development -environment (IDE), the following are installed for you: - -- [AWS Command Line Interface](https://docs.aws.amazon.com/en_pv/cli/latest/userguide/cli-chap-install.html) -- [SAM CLI](https://docs.aws.amazon.com/en_pv/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) -- [Docker](https://docs.docker.com/install/) and necessary Docker images. - -### Creating an AWS SAM application using SAM CLI - -To create a new AWS SAM application: - -1. Create a new GitLab project. -1. `git clone` the project into your local environment. -1. Change to the newly cloned project and create a new SAM app using the following command: - - ```shell - sam init -r python3.8 -n gitlabpoc --app-template "hello-world" - ``` - -1. `git push` the application back to the GitLab project. - -This creates a SAM app named `gitlabpoc` using the default configuration, a single -Python 3.8 function invoked by an [Amazon API Gateway](https://aws.amazon.com/api-gateway/) -endpoint. To see additional runtimes supported by SAM and options for `sam init`, run: - -```shell -sam init -h -``` - -### Setting up your AWS credentials with your GitLab account - -In order to interact with your AWS account, the GitLab CI/CD pipelines require both -`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be set in the project's CI/CD variables. - -To set these: - -1. Navigate to the project's **Settings > CI/CD**. -1. Expand the **Variables** section and create entries for `AWS_ACCESS_KEY_ID` and - `AWS_SECRET_ACCESS_KEY`. -1. Mask the credentials so they do not show in logs using the **Masked** toggle. - -The AWS credentials you provide must include IAM policies that provision correct access -control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. - -### Crafting the `.gitlab-ci.yml` file - -In a [`.gitlab-ci.yml`](../../../../ci/yaml/index.md) file in the root of your project, -add the following and replace `<S3_bucket_name>` with the name of the S3 bucket where you -want to store your package: - -```yaml -image: python:latest - -stages: - - deploy - -production: - stage: deploy - before_script: - - apt-get update - - apt-get install -y python3-pip - - pip3 install awscli --upgrade - - pip3 install aws-sam-cli --upgrade - script: - - sam build - - sam package --output-template-file packaged.yaml --s3-bucket <S3_bucket_name> - - sam deploy --template-file packaged.yaml --stack-name gitlabpoc --s3-bucket <S3_bucket_name> --capabilities CAPABILITY_IAM --region us-east-1 - environment: production -``` - -Let's examine the configuration file more closely: - -- `image` specifies the Docker image to use for this build. This is the latest Python - image since the sample application is written in Python. -- AWS CLI and AWS SAM CLI are installed in the `before_script` section. -- SAM build, package, and deploy commands are used to build, package, and deploy the - application. - -### Deploying your application - -Push changes to your GitLab repository and the GitLab build pipeline -deploys your application. If your: - -- Build and deploy are successful, [test your deployed application](#testing-the-deployed-application). -- Build fails, look at the build log to see why the build failed. Some common reasons - the build might fail are: - - - Incompatible versions of software. For example, Python runtime version might be - different from the Python on the build machine. Address this by installing the - required versions of the software. - - You may not be able to access your AWS account from GitLab. Check the CI/CD variables - you set up with AWS credentials. - - You may not have permission to deploy a serverless application. Make sure you - provide all required permissions to deploy a serverless application. - -### Testing the deployed application - -To test the application you deployed, please go to the build log and follow the following steps: - -1. Click on "Show complete raw" on the upper right-hand corner: - -  - -1. Look for HelloWorldApi – API Gateway endpoint similar to shown below: - -  - -1. Use curl to test the API. For example: - - ```shell - curl "https://py4rg7qtlg.execute-api.us-east-1.amazonaws.com/Prod/hello/" - ``` - -Output should be: - -```json -{"message": "hello world"} -``` - -### Testing Locally - -AWS SAM provides functionality to test your applications locally. You must have AWS SAM -CLI installed locally for you to test locally. - -First, test the function. - -SAM provides a default event in `events/event.json` that includes a message body of: - -```plaintext -{\"message\": \"hello world\"} -``` - -If you pass that event into the `HelloWorldFunction`, it should respond with the same -body. - -Invoke the function by running: - -```shell -sam local invoke HelloWorldFunction -e events/event.json -``` - -Output should be: - -```json -{"message": "hello world"} -``` - -After you confirm that Lambda function is working as expected, test the API Gateway -using following steps. - -Start the API locally by running: - -```shell -sam local start-api -``` - -SAM again launches a Docker container, this time with a mocked Amazon API Gateway -listening on `localhost:3000`. - -Call the `hello` API by running: - -```shell -curl "http://127.0.0.1:3000/hello" -``` - -Output again should be: - -```json -{"message": "hello world"} -``` +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86267) in GitLab 15.0. diff --git a/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png b/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png Binary files differdeleted file mode 100644 index a19d236fc39..00000000000 --- a/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-endpoint.png b/doc/user/project/clusters/serverless/img/function-endpoint.png Binary files differdeleted file mode 100644 index a38fe2cb6c2..00000000000 --- a/doc/user/project/clusters/serverless/img/function-endpoint.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-execution.png b/doc/user/project/clusters/serverless/img/function-execution.png Binary files differdeleted file mode 100644 index f60dd277081..00000000000 --- a/doc/user/project/clusters/serverless/img/function-execution.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-list_v12_7.png b/doc/user/project/clusters/serverless/img/function-list_v12_7.png Binary files differdeleted file mode 100644 index f2a27ce7b0f..00000000000 --- a/doc/user/project/clusters/serverless/img/function-list_v12_7.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/sam-api-endpoint.png b/doc/user/project/clusters/serverless/img/sam-api-endpoint.png Binary files differdeleted file mode 100644 index 3407b2684fd..00000000000 --- a/doc/user/project/clusters/serverless/img/sam-api-endpoint.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/sam-complete-raw.png b/doc/user/project/clusters/serverless/img/sam-complete-raw.png Binary files differdeleted file mode 100644 index 1130cd29d56..00000000000 --- a/doc/user/project/clusters/serverless/img/sam-complete-raw.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png b/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png Binary files differdeleted file mode 100644 index f88eb4bdcd2..00000000000 --- a/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 29164da307b..432caa8476f 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -2,818 +2,10 @@ stage: Configure group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../../../update/removals.md#gitlab-serverless' --- -# Serverless (DEPRECATED) **(FREE)** +# Serverless (removed) **(FREE)** -> - Introduced in GitLab 11.5. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3. - -WARNING: -Serverless is currently in [alpha](../../../../policy/alpha-beta-support.md#alpha-features). - -## Overview - -Serverless architectures offer Operators and Developers the ability write highly scalable applications without provisioning a single server. - -GitLab supports several ways deploy Serverless applications in both Kubernetes Environments and also major cloud Function as a Service (FaaS) environments. - -Currently we support: - -- [Knative](#knative): Build Knative applications with Knative and `gitlabktl` on GKE and EKS. -- [AWS Lambda](aws.md): Create serverless applications via the Serverless Framework and GitLab CI/CD. - -## Knative - -Run serverless workloads on Kubernetes using [Knative](https://cloud.google.com/knative/). - -Knative extends Kubernetes to provide a set of middleware components that are useful to build -modern, source-centric, container-based applications. Knative brings some significant benefits out -of the box through its main components: - -- [Serving](https://github.com/knative/serving): Request-driven compute that can scale to zero. -- [Eventing](https://github.com/knative/eventing): Management and delivery of events. - -For more information on Knative, visit the [Knative docs repository](https://github.com/knative/docs). - -With GitLab Serverless, you can deploy both FaaS and serverless applications. - -## Prerequisites - -To run Knative on GitLab, you need: - -1. **Existing GitLab project:** You need a GitLab project to associate all resources. The simplest way to get started: - - If you are planning on [deploying functions](#deploying-functions), - clone the [functions example project](https://gitlab.com/knative-examples/functions) to get - started. - - If you are planning on [deploying a serverless application](#deploying-serverless-applications), - clone the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get - started. -1. **Kubernetes Cluster:** An RBAC-enabled Kubernetes cluster is required to deploy Knative. - The simplest way to get started is to add a cluster using the GitLab [GKE integration](../add_remove_clusters.md). - The set of minimum recommended cluster specifications to run Knative is 3 nodes, 6 vCPUs, and 22.50 GB memory. -1. **GitLab Runner:** A runner is required to run the CI jobs that deploy serverless - applications or functions onto your cluster. You can install GitLab Runner - onto the [existing Kubernetes cluster](https://docs.gitlab.com/runner/install/kubernetes.html). -1. **Domain Name:** Knative provides its own load balancer using Istio, and an - external IP address or hostname for all the applications served by Knative. Enter a - wildcard domain to serve your applications. Configure your DNS server to use the - external IP address or hostname for that domain. -1. **`.gitlab-ci.yml`:** GitLab uses [Kaniko](https://github.com/GoogleContainerTools/kaniko) - to build the application. We also use [GitLab Knative tool](https://gitlab.com/gitlab-org/gitlabktl) - CLI to simplify the deployment of services and functions to Knative. -1. **`serverless.yml`** (for [functions only](#deploying-functions)): When using serverless to deploy functions, the `serverless.yml` file - contains the information for all the functions being hosted in the repository as well as a reference - to the runtime being used. -1. **`Dockerfile`** (for [applications only](#deploying-serverless-applications)): Knative requires a - `Dockerfile` in order to build your applications. It should be included at the root of your - project's repository and expose port `8080`. `Dockerfile` is not require if you plan to build serverless functions - using our [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes). -1. **Prometheus** (optional): The [Prometheus cluster integration](../../../clusters/integrations.md#prometheus-cluster-integration) - allows you to monitor the scale and traffic of your serverless function/application. -1. **Logging** (optional): Configuring logging allows you to view and search request logs for your serverless function/application. - See [Configuring logging](#configuring-logging) for more information. - -## Configuring Knative - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/58941) in GitLab 12.0. - -1. Follow the steps to - [add a Kubernetes - cluster](../add_remove_clusters.md). - -1. Ensure GitLab can manage Knative: - - For a non-GitLab managed cluster, ensure that the service account for the token - provided can manage resources in the `serving.knative.dev` API group. - - For a GitLab managed cluster, if you added the cluster in [GitLab 12.1 or later](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30235), - then GitLab already has the required access and you can proceed to the next step. - - Otherwise, you need to manually grant the GitLab service account the ability to manage - resources in the `serving.knative.dev` API group. Since every GitLab service account - has the `edit` cluster role, the simplest way to do this is with an - [aggregated ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#aggregated-clusterroles) - adding rules to the default `edit` cluster role: First, save the following YAML as - `knative-serving-only-role.yaml`: - - ```yaml - apiVersion: rbac.authorization.k8s.io/v1 - kind: ClusterRole - metadata: - name: knative-serving-only-role - labels: - rbac.authorization.k8s.io/aggregate-to-edit: "true" - rules: - - apiGroups: - - serving.knative.dev - resources: - - configurations - - configurationgenerations - - routes - - revisions - - revisionuids - - autoscalers - - services - verbs: - - get - - list - - create - - update - - delete - - patch - - watch - ``` - - Then run the following command: - - ```shell - kubectl apply -f knative-serving-only-role.yaml - ``` - - Alternatively, permissions can be granted on a per-service account basis - using `Role`s and `RoleBinding`s (see the [Kubernetes RBAC - documentation](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) - for more information). - -1. Follow the steps to deploy [functions](#deploying-functions) - or [serverless applications](#deploying-serverless-applications) onto your - cluster. - -1. **Optional:** For invocation metrics to show in GitLab, additional Istio metrics need to be configured in your cluster. For example, with Knative v0.9.0, you can use [this manifest](https://gitlab.com/gitlab-org/charts/knative/-/raw/v0.10.0/vendor/istio-metrics.yml). - -## Supported runtimes - -Serverless functions for GitLab can be run using: - -- [GitLab-managed](#gitlab-managed-runtimes) runtimes. -- [OpenFaaS](#openfaas-runtimes) runtimes. - -If a runtime is not available for the required programming language, consider deploying a -[serverless application](#deploying-serverless-applications). - -### GitLab-managed runtimes - -The following GitLab-managed [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes) -are available: - -- `go` (proof of concept) -- `nodejs` -- `ruby` - -You must provide a `Dockerfile` to run serverless functions if no runtime is specified. - -### OpenFaaS runtimes - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29253) in GitLab 12.5. - -[OpenFaaS classic runtimes](https://github.com/openfaas/templates#templates-in-store) can be used with GitLab serverless. - -OpenFaas runtimes are available for the following languages: - -- C# -- Go -- NodeJS -- PHP -- Python -- Ruby - -Runtimes are specified using the pattern: `openfaas/classic/<template_name>`. The following -example shows how to define a function in `serverless.yml` using an OpenFaaS runtime: - -```yaml -hello: - source: ./hello - runtime: openfaas/classic/ruby - description: "Ruby function using OpenFaaS classic runtime" -``` - -`handler` is not needed for OpenFaaS functions. The location of the handler is defined -by the conventions of the runtime. - -See the [`ruby-openfaas-function`](https://gitlab.com/knative-examples/ruby-openfaas-function) -project for an example of a function using an OpenFaaS runtime. - -## Deploying functions - -> Introduced in GitLab 11.6. - -You can find and import all the files referenced in this doc in the -**[functions example project](https://gitlab.com/knative-examples/functions)**. - -Follow these steps to deploy a function using the Node.js runtime to your -Knative instance (you can skip these steps if you've cloned the example -project): - -1. Create a directory to house the function. In this example we will - create a directory called `echo` at the root of the project. - -1. Create the file to contain the function code. In this example, our file is called `echo.js` and is located inside the `echo` directory. If your project is: - - Public, continue to the next step. - - Private, you must [create a GitLab deploy token](../../deploy_tokens/index.md#creating-a-deploy-token) with `gitlab-deploy-token` as the name and the `read_registry` scope. - -1. `.gitlab-ci.yml`: this defines a pipeline used to deploy your functions. - It must be included at the root of your repository: - - ```yaml - include: - - template: Serverless.gitlab-ci.yml - - functions:build: - extends: .serverless:build:functions - environment: production - - functions:deploy: - extends: .serverless:deploy:functions - environment: production - ``` - - This `.gitlab-ci.yml` creates jobs that invoke some predefined commands to - build and deploy your functions to your cluster. - - `Serverless.gitlab-ci.yml` is a template that allows customization. - You can either import it with `include` parameter and use `extends` to - customize your jobs, or you can inline the entire template by choosing it - from **Apply a template** dropdown when editing the `.gitlab-ci.yml` file through - the user interface. - -1. `serverless.yml`: this file contains the metadata for your functions, - such as name, runtime, and environment. - - It must be included at the root of your repository. - The following is a sample `echo` function which shows the required structure - for the file. - - You can find the relevant files for this project in the [functions example project](https://gitlab.com/knative-examples/functions). - - ```yaml - service: functions - description: "GitLab Serverless functions using Knative" - - provider: - name: triggermesh - envs: - FOO: value - secrets: - - my-secrets - - functions: - echo-js: - handler: echo-js - source: ./echo-js - runtime: gitlab/runtimes/nodejs - description: "node.js runtime function" - envs: - MY_FUNCTION: echo-js - secrets: - - my-secrets - ``` - -Explanation of the fields used above: - -### `service` - -| Parameter | Description | -|-----------|-------------| -| `service` | Name for the Knative service which serves the function. | -| `description` | A short description of the `service`. | - -### `provider` - -| Parameter | Description | -|-----------|-------------| -| `name` | Indicates which provider is used to execute the `serverless.yml` file. In this case, the TriggerMesh middleware. | -| `envs` | Includes the environment variables to be passed as part of function execution for **all** functions in the file, where `FOO` is the variable name and `BAR` are the variable contents. You may replace this with your own variables. | -| `secrets` | Includes the contents of the Kubernetes secret as environment variables accessible to be passed as part of function execution for **all** functions in the file. The secrets are expected in `INI` format. | - -### `functions` - -In the `serverless.yml` example above, the function name is `echo` and the -subsequent lines contain the function attributes. - -| Parameter | Description | -|-----------|-------------| -| `handler` | The function's name. | -| `source` | Directory with sources of a functions. | -| `runtime` (optional)| The runtime to be used to execute the function. This can be a runtime alias (see [Runtime aliases](#runtime-aliases)), or it can be a full URL to a custom runtime repository. When the runtime is not specified, we assume that `Dockerfile` is present in the function directory specified by `source`. | -| `description` | A short description of the function. | -| `envs` | Sets an environment variable for the specific function only. | -| `secrets` | Includes the contents of the Kubernetes secret as environment variables accessible to be passed as part of function execution for the specific function only. The secrets are expected in `INI` format. | - -### Deployment - -#### Runtime aliases - -The optional `runtime` parameter can refer to one of the following runtime aliases (also see [Supported runtimes](#supported-runtimes)): - -| Runtime alias | Maintained by | -|-------------|---------------| -| `gitlab/runtimes/go` | GitLab | -| `gitlab/runtimes/nodejs` | GitLab | -| `gitlab/runtimes/ruby` | GitLab | -| `openfaas/classic/csharp` | OpenFaaS | -| `openfaas/classic/go` | OpenFaaS | -| `openfaas/classic/node` | OpenFaaS | -| `openfaas/classic/php7` | OpenFaaS | -| `openfaas/classic/python` | OpenFaaS | -| `openfaas/classic/python3` | OpenFaaS | -| `openfaas/classic/ruby` | OpenFaaS | - -After the `.gitlab-ci.yml` template has been added and the `serverless.yml` file -has been created, pushing a commit to your project results in a CI pipeline -being executed which deploys each function as a Knative service. After the -deploy stage has finished, additional details for the function display -under **Infrastructure > Serverless platform**. - - - -This page contains all functions available for the project, the description for -accessing the function, and, if available, the function's runtime information. -The details are derived from the Knative installation inside each of the project's -Kubernetes cluster. Click on each function to obtain detailed scale and invocation data. - -The function details can be retrieved directly from Knative on the cluster: - -```shell -kubectl -n "$KUBE_NAMESPACE" get services.serving.knative.dev -``` - -The sample function can now be triggered from any HTTP client using a simple `POST` call: - - 1. Using curl (replace the URL on the last line with the URL of your application): - - ```shell - curl \ - --header "Content-Type: application/json" \ - --request POST \ - --data '{"GitLab":"FaaS"}' \ - "http://functions-echo.functions-1.functions.example.com/" - ``` - - 1. Using a web-based tool (such as Postman or Restlet) - -  - -### Secrets - -To access your Kubernetes secrets from within your function, the secrets should be created under the namespace of your serverless deployment and specified in your `serverless.yml` file as above. -You can create secrets in several ways. The following sections show some examples. - -#### CLI example - -```shell -kubectl create secret generic my-secrets -n "$KUBE_NAMESPACE" --from-literal MY_SECRET=imverysecure -``` - -#### Part of deployment job - -You can extend your `.gitlab-ci.yml` to create the secrets during deployment using the [CI/CD variables](../../../../ci/variables/index.md) -stored securely under your GitLab project. - -```yaml -deploy:function: - stage: deploy - environment: production - extends: .serverless:deploy:functions - before_script: - - kubectl create secret generic my-secret - --from-literal MY_SECRET="$GITLAB_SECRET_VARIABLE" - --namespace "$KUBE_NAMESPACE" - --dry-run -o yaml | kubectl apply -f - -``` - -### Running functions locally - -Running a function locally is a good way to quickly verify behavior during development. - -Running functions locally requires: - -- Go 1.12 or newer installed. -- Docker Engine installed and running. -- `gitlabktl` installed using the Go package manager: - - ```shell - GO111MODULE=on go get gitlab.com/gitlab-org/gitlabktl - ``` - -To run a function locally: - -1. Navigate to the root of your GitLab serverless project. -1. Build your function into a Docker image: - - ```shell - gitlabktl serverless build - ``` - -1. Run your function in Docker: - - ```shell - docker run -itp 8080:8080 <your_function_name> - ``` - -1. Invoke your function: - - ```shell - curl "http://localhost:8080" - ``` - -## Deploying Serverless applications - -> Introduced in GitLab 11.5. - -Serverless applications are an alternative to [serverless functions](#deploying-functions). -They're useful in scenarios where an existing runtime does not meet the needs of -an application, such as one written in a language that has no runtime available. -Note though that serverless applications should be stateless. - -You can reference and import the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) -to get started. Add the following `.gitlab-ci.yml` to the root of your repository -(you may skip this step if you've previously cloned the previously mentioned, -sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app)): - -```yaml -include: - - template: Serverless.gitlab-ci.yml - -build: - extends: .serverless:build:image - -deploy: - extends: .serverless:deploy:image -``` - -`Serverless.gitlab-ci.yml` is a template that allows customization. -You can either import it with `include` parameter and use `extends` to -customize your jobs, or you can inline the entire template by choosing it -from **Apply a template** dropdown when editing the `.gitlab-ci.yml` file through -the user interface. - -A `serverless.yml` file is not required when deploying serverless applications. - -### Deploy the application with Knative - -With all the pieces in place, the next time a CI pipeline runs the Knative application deploys. Navigate to -**CI/CD > Pipelines** and click the most recent pipeline. - -### Function details - -Go to the **Infrastructure > Serverless platform** page to see the final URL of your functions. - - - -### Invocation metrics - -On the same page as above, click on one of the function -rows to bring up the function details page. - - - -The pod count gives you the number of pods running the serverless function instances on a given cluster. - -For the Knative function invocations to appear, the -[Prometheus cluster integration must be enabled](../../../clusters/integrations.md#prometheus-cluster-integration). - -Once Prometheus is enabled, a message may appear indicating that the metrics data _is -loading or is not available at this time._ It appears upon the first access of the -page, but should go away after a few seconds. If the message does not disappear, then it -is possible that GitLab is unable to connect to the Prometheus instance running on the -cluster. - -## Configuring logging - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33330) in GitLab 12.5. - -### Prerequisites - -- A GitLab-managed cluster. -- `kubectl` installed and working. - -Running `kubectl` commands on your cluster requires setting up access to the -cluster first. For clusters created on: - -- GKE, see [GKE Cluster Access](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl) -- Other platforms, see [Install and Set Up kubectl](https://kubernetes.io/docs/tasks/tools/). - -### Enable request log template - -Run the following command to enable request logs: - -```shell -kubectl edit cm -n knative-serving config-observability -``` - -Copy the `logging.request-log-template` from the `data._example` field to the data field one level up in the hierarchy. - -### Enable request logs - -Run the following commands to install Elasticsearch, Kibana, and Filebeat into a `kube-logging` namespace and configure all nodes to forward logs using Filebeat: - -```shell -kubectl apply -f https://gitlab.com/gitlab-org/serverless/configurations/knative/raw/v0.7.0/kube-logging-filebeat.yaml -kubectl label nodes --all beta.kubernetes.io/filebeat-ready="true" -``` - -### Viewing request logs - -To view request logs: - -1. Run `kubectl proxy`. -1. Navigate to [Kibana UI](http://localhost:8001/api/v1/namespaces/kube-logging/services/kibana/proxy/app/kibana). - -Or: - -1. Open the [Kibana UI](http://localhost:8001/api/v1/namespaces/kube-logging/services/kibana/proxy/app/kibana). -1. Click on **Discover**, then select `filebeat-*` from the dropdown on the left. -1. Enter `kubernetes.container.name:"queue-proxy" AND message:/httpRequest/` into the search box. - -## Enabling TLS for Knative services - -By default, a GitLab serverless deployment is served over `http`. To serve -over `https`, you must manually obtain and install TLS certificates. - -The simplest way to accomplish this is to use Certbot to -[manually obtain Let's Encrypt certificates](https://knative.dev/docs/serving/using-a-tls-cert/#using-certbot-to-manually-obtain-let-s-encrypt-certificates). -Certbot is a free, open source software tool for automatically using Let's Encrypt -certificates on manually-administered websites to enable HTTPS. - -The following instructions relate to installing and running Certbot on a Linux -server that has Python 3 installed, and may not work on other operating systems -or with other versions of Python. - -1. Install Certbot by running the - [`certbot-auto` wrapper script](https://eff-certbot.readthedocs.io/install.html#certbot-auto). - On the command line of your server, run the following commands: - - ```shell - wget https://dl.eff.org/certbot-auto - sudo mv certbot-auto /usr/local/bin/certbot-auto - sudo chown root /usr/local/bin/certbot-auto - sudo chmod 0755 /usr/local/bin/certbot-auto - /usr/local/bin/certbot-auto --help - ``` - - To check the integrity of the `certbot-auto` script, run: - - ```shell - wget -N https://dl.eff.org/certbot-auto.asc - gpg2 --keyserver ipv4.pool.sks-keyservers.net --recv-key A2CFB51FA275A7286234E7B24D17C995CD9775F2 - gpg2 --trusted-key 4D17C995CD9775F2 --verify certbot-auto.asc /usr/local/bin/certbot-auto - ``` - - The output of the last command should look something like: - - ```shell - gpg: Signature made Mon 10 Jun 2019 06:24:40 PM EDT - gpg: using RSA key A2CFB51FA275A7286234E7B24D17C995CD9775F2 - gpg: key 4D17C995CD9775F2 marked as ultimately trusted - gpg: checking the trustdb - gpg: marginals needed: 3 completes needed: 1 trust model: pgp - gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u - gpg: next trustdb check due at 2027-11-22 - gpg: Good signature from "Let's Encrypt Client Team <letsencrypt-client@eff.org>" [ultimate] - ``` - -1. Run the following command to use Certbot to request a certificate - using DNS challenge during authorization: - - ```shell - /usr/local/bin/certbot-auto certonly --manual --preferred-challenges dns -d '*.<namespace>.example.com' - ``` - - Where `<namespace>` is the namespace created by GitLab for your serverless project (composed of `<project_name>-<project_id>-<environment>`) and - `example.com` is the domain being used for your project. If you are unsure what the namespace of your project is, navigate - to the **Infrastructure > Serverless platform** page of your project and inspect - the endpoint provided for your function/app. - -  - - In the above image, the namespace for the project is `node-function-11909507` and the domain is `knative.info`, thus - certificate request line would look like this: - - ```shell - ./certbot-auto certonly --manual --preferred-challenges dns -d '*.node-function-11909507.knative.info' - ``` - - The Certbot tool walks you through the steps of validating that you own each domain that you specify by creating TXT records in those domains. - After this process is complete, the output should look something like this: - - ```shell - IMPORTANT NOTES: - - Congratulations! Your certificate and chain have been saved at: - /etc/letsencrypt/live/namespace.example.com/fullchain.pem - Your key file has been saved at: - /etc/letsencrypt/live/namespace.example/privkey.pem - Your cert will expire on 2019-09-19. To obtain a new or tweaked - version of this certificate in the future, simply run certbot-auto - again. To non-interactively renew *all* of your certificates, run - "certbot-auto renew" - -----BEGIN PRIVATE KEY----- - - Your account credentials have been saved in your Certbot - configuration directory at /etc/letsencrypt. You should make a - secure backup of this folder now. This configuration directory will - also contain certificates and private keys obtained by Certbot so - making regular backups of this folder is ideal. - ``` - -1. Create certificate and private key files. Using the contents of the files - returned by Certbot, create two files in order to create the - Kubernetes secret: - - Run the following command to see the contents of `fullchain.pem`: - - ```shell - sudo cat /etc/letsencrypt/live/node-function-11909507.knative.info/fullchain.pem - ``` - - Output should look like this: - - ```shell - -----BEGIN CERTIFICATE----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b4ag== - -----END CERTIFICATE----- - -----BEGIN CERTIFICATE----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - K2fcb195768c39e9a94cec2c2e30Qg== - -----END CERTIFICATE----- - ``` - - Create a file with the name `cert.pem` with the contents of the entire output. - - Once `cert.pem` is created, run the following command to see the contents of `privkey.pem`: - - ```shell - sudo cat /etc/letsencrypt/live/namespace.example/privkey.pem - ``` - - Output should look like this: - - ```shell - -----BEGIN PRIVATE KEY----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - -----BEGIN CERTIFICATE----- - fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 4f294d1eaca42b8692017b4262== - -----END PRIVATE KEY----- - ``` - - Create a new file with the name `cert.pk` with the contents of the entire output. - -1. Create a Kubernetes secret to hold your TLS certificate, `cert.pem`, and - the private key `cert.pk`: - - NOTE: - Running `kubectl` commands on your cluster requires setting up access to the cluster first. - For clusters created on GKE, see - [GKE Cluster Access](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl). - For other platforms, [install `kubectl`](https://kubernetes.io/docs/tasks/tools/). - - ```shell - kubectl create --namespace istio-system secret tls istio-ingressgateway-certs \ - --key cert.pk \ - --cert cert.pem - ``` - - Where `cert.pem` and `cert.pk` are your certificate and private key files. Note that the `istio-ingressgateway-certs` secret name is required. - -1. Configure Knative to use the new secret that you created for HTTPS - connections. Run the - following command to open the Knative shared `gateway` in edit mode: - - ```shell - kubectl edit gateway knative-ingress-gateway --namespace knative-serving - ``` - - Update the gateway to include the following `tls:` section and configuration: - - ```shell - tls: - mode: SIMPLE - privateKey: /etc/istio/ingressgateway-certs/tls.key - serverCertificate: /etc/istio/ingressgateway-certs/tls.crt - ``` - - Example: - - ```shell - apiVersion: networking.istio.io/v1alpha3 - kind: Gateway - metadata: - # ... skipped ... - spec: - selector: - istio: ingressgateway - servers: - - hosts: - - "*" - port: - name: http - number: 80 - protocol: HTTP - - hosts: - - "*" - port: - name: https - number: 443 - protocol: HTTPS - tls: - mode: SIMPLE - privateKey: /etc/istio/ingressgateway-certs/tls.key - serverCertificate: /etc/istio/ingressgateway-certs/tls.crt - ``` - - After your changes are running on your Knative cluster, you can begin using the HTTPS protocol for secure access your deployed Knative services. - In the event a mistake is made during this process and you need to update the cert, you must edit the gateway `knative-ingress-gateway` - to switch back to `PASSTHROUGH` mode. Once corrections are made, edit the file again so the gateway uses the new certificates. - -## Using an older version of `gitlabktl` - -There may be situations where you want to run an older version of `gitlabktl`. This -requires setting an older version of the `gitlabktl` image in the `.gitlab-ci.yml` file. - -To set an older version, add `image:` to the `functions:deploy` block. For example: - -```yaml -functions:deploy: - extends: .serverless:deploy:functions - environment: production - image: registry.gitlab.com/gitlab-org/gitlabktl:0.5.0 -``` - -Different versions are available by changing the version tag at the end of the registry URL in the -format `registry.gitlab.com/gitlab-org/gitlabktl:<version>`. - -For a full inventory of available `gitlabktl` versions, see the `gitlabktl` project's -[container registry](https://gitlab.com/gitlab-org/gitlabktl/container_registry). +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86267) in GitLab 15.0. |
