diff options
Diffstat (limited to 'doc/user/clusters/agent/index.md')
| -rw-r--r-- | doc/user/clusters/agent/index.md | 329 |
1 files changed, 329 insertions, 0 deletions
diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md new file mode 100644 index 00000000000..7b745577cc4 --- /dev/null +++ b/doc/user/clusters/agent/index.md @@ -0,0 +1,329 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# GitLab Kubernetes Agent **(PREMIUM ONLY)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223061) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. + +## Goals + +The [GitLab Kubernetes Agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) is an active in-cluster component for solving GitLab and Kubernetes integration tasks in a secure and cloud native way. + +Features: + +1. Makes it possible to integrate GitLab with a Kubernetes cluster behind a firewall or NAT +1. Enables pull-based GitOps deployments by leveraging the [GitOps Engine](https://github.com/argoproj/gitops-engine) +1. Allows for real-time access to API endpoints within a cluster. +1. Many more features are planned. Please [review our roadmap](https://gitlab.com/groups/gitlab-org/-/epics/3329). + +## Architecture + +### GitLab Agent GitOps workflow + +```mermaid +sequenceDiagram + participant D as Developer + participant A as Application code repository + participant M as Manifest repository + participant K as Kubernetes agent + participant C as Agent configuration repository + K->C: Grab the configuration + D->>+A: Pushing code changes + A->>M: Updating manifest + loop Regularly + K-->>M: Watching changes + M-->>K: Pulling and applying changes + end +``` + +Please refer to our [full architecture documentation in the Agent project](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). + +## Getting started with GitOps using the GitLab Agent and the GitLab Cloud Native Helm chart + +There are several components that work in concert for the Agent to accomplish GitOps deployments: + +1. A Kubernetes cluster that is properly configured +1. A configuration repository that contains a `config.yaml` file. This `config.yaml` tells the Agent which repositories to synchronize with. +1. A manifest repository that contains a `manifest.yaml`. This `manifest.yaml` (which can be autogenerated) is tracked by the Agent and any changes to the file are automatically applied to the cluster. + +The setup process involves a few steps that, once completed, will enable GitOps deployments to work + +1. Installing the Agent server via GitLab Helm chart +1. Defining a configuration directory +1. Creating an Agent record in GitLab +1. Generating and copying a Secret token used to connect to the Agent +1. Installing the Agent into the cluster +1. Creating a `manifest.yaml` + +### Installing the Agent server via Helm + +Currently the GitLab Kubernetes Agent can only be deployed via our [Helm chart](https://gitlab.com/gitlab-org/charts/gitlab). + +NOTE: We are working quickly to [include the Agent in Official Linux Package](https://gitlab.com/gitlab-org/gitlab/-/issues/223060). + +If you don't already have GitLab installed via Helm please refer to our [installation documentation](https://docs.gitlab.com/charts/installation/) + +When installing/upgrading the GitLab Helm chart please consider the following Helm 2 example (if using Helm 3 please modify): + +```shell +helm upgrade --force --install gitlab . \ + --timeout 600 \ + --set global.hosts.domain=<YOUR_DOMAIN> \ + --set global.hosts.externalIP=<YOUR_IP> \ + --set certmanager-issuer.email=<YOUR_EMAIL> \ + --set name=gitlab-instance \ + --set global.kas.enabled=true +``` + +`global.kas.enabled=true` must be set in order for the Agent to be properly installed and configured. + +### Defining a configuration repository + +Next you will need a GitLab repository that will contain your Agent configuration. + +The minimal repository layout looks like this: + +`.gitlab/agents/<agent-name>/config.yaml` + +The `config.yaml` file contents should look like this: + +```yaml +gitops: + manifest_projects: + - id: "path-to/your-awesome-project" +``` + +### Creating an Agent record in GitLab + +Next you will need to create an GitLab Rails Agent record so that your GitLab project so that the Agent itself can associate with a GitLab project. This process will also yield a Secret that you will use to configure the Agent in subsequent steps. + +There are two ways to accomplish this: + +1. Via the Rails console +1. Via GraphQL + +To do this you could either run `rails c` or via GraphQL. From `rails c`: + +```ruby + project = ::Project.find_by_full_path("path-to/your-awesome-project") + agent = ::Clusters::Agent.create(name: "<agent-name>", project: project) + token = ::Clusters::AgentToken.create(agent: agent) + token.token # this will print out the token you need to use on the next step +``` + +or using GraphQL: + +with this approach, you'll need a premium license to use this feature. + +If you are new to using the GitLab GraphQL API please refer to the [Getting started with the GraphQL API page](../../../api/graphql/getting_started.md) or check out the [GraphQL Explorer](https://gitlab.com/-/graphql-explorer). + +```json + mutation createAgent { + createClusterAgent(input: { projectPath: "path-to/your-awesome-project", name: "<agent-name>" }) { + clusterAgent { + id + name + } + errors + } + } + + mutation createToken { + clusterAgentTokenCreate(input: { clusterAgentId: <cluster-agent-id-taken-from-the-previous-mutation> }) { + secret # This is the value you need to use on the next step + token { + createdAt + id + } + errors + } + } +``` + +Note that GraphQL will only show you the token once, after you've created it. + +### Creating the Kubernetes secret + +Once the token has been generated it needs to be applied to the Kubernetes cluster. + +If you didn't previously define or create a namespace you need to do that first: + +```shell +kubectl create namespace <YOUR-DESIRED-NAMESPACE> +``` + +Run the following command to create your Secret: + +```shell +kubectl create secret generic -n <YOUR-DESIRED-NAMESPACE> gitlab-agent-token --from-literal=token='YOUR_AGENT_TOKEN' +``` + +### Installing the Agent into the cluster + +Next you are now ready to install the in-cluster component of the Agent. The below is an example YAML file of the Kubernetes resources required for the Agent to be installed. + +Let's highlight a few of the details in the example below: + +1. You can replace `gitlab-agent` with <YOUR-DESIRED-NAMESPACE> +1. For the `kas-address` (Kubernetes Agent Server), you can replace `grpc://host.docker.internal:5005` with the address of the kas agent that was initialized via your Helm install. +1. If you defined your own secret name, then replace `gitlab-agent-token` with your secret name. + +`./resources.yml` + +```yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: gitlab-agent +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: gitlab-agent +spec: + replicas: 1 + selector: + matchLabels: + app: gitlab-agent + template: + metadata: + labels: + app: gitlab-agent + spec: + serviceAccountName: gitlab-agent + containers: + - name: agent + image: "registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/agentk:latest" + args: + - --token-file=/config/token + - --kas-address + - grpc://host.docker.internal:5005 # {"$openapi":"kas-address"} + volumeMounts: + - name: token-volume + mountPath: /config + volumes: + - name: token-volume + secret: + secretName: gitlab-agent-token + strategy: + type: RollingUpdate + rollingUpdate: + maxSurge: 0 + maxUnavailable: 1 +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: gitlab-agent-write +rules: +- resources: + - '*' + apiGroups: + - '*' + verbs: + - create + - update + - delete + - patch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: gitlab-agent-write-binding +roleRef: + name: gitlab-agent-write + kind: ClusterRole + apiGroup: rbac.authorization.k8s.io +subjects: +- name: gitlab-agent + kind: ServiceAccount + namespace: gitlab-agent +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: gitlab-agent-read +rules: +- resources: + - '*' + apiGroups: + - '*' + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: gitlab-agent-read-binding +roleRef: + name: gitlab-agent-read + kind: ClusterRole + apiGroup: rbac.authorization.k8s.io +subjects: +- name: gitlab-agent + kind: ServiceAccount + namespace: gitlab-agent + +``` + +```shell +kubectl apply -n gitlab-agent -f ./resources.yml +``` + +```plaintext +$ kubectl get pods --all-namespaces +NAMESPACE NAME READY STATUS RESTARTS AGE +gitlab-agent gitlab-agent-77689f7dcb-5skqk 1/1 Running 0 51s +kube-system coredns-f9fd979d6-n6wcw 1/1 Running 0 14m +kube-system etcd-minikube 1/1 Running 0 14m +kube-system kube-apiserver-minikube 1/1 Running 0 14m +kube-system kube-controller-manager-minikube 1/1 Running 0 14m +kube-system kube-proxy-j6zdh 1/1 Running 0 14m +kube-system kube-scheduler-minikube 1/1 Running 0 14m +kube-system storage-provisioner 1/1 Running 0 14m +``` + +### Creating a `manifest.yaml` + +In the above step, you configured a `config.yaml` to point to which GitLab projects the Agent should synchronize. Within each one of those projects, you need to create a `manifest.yaml` file which the Agent will monitor. This `manifest.yaml` can be autogenerated by a templating engine or other means. + +Example `manifest.yaml`: + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: nginx-deployment +spec: + selector: + matchLabels: + app: nginx + replicas: 2 + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 +``` + +The above file creates a simple NGINX deployment. + +Each time you commit and push a change to the `manifest.yaml` the Agent will observe the change. Example log: + +```plaintext +2020-09-15_14:09:04.87946 gitlab-k8s-agent : time="2020-09-15T10:09:04-04:00" level=info msg="Config: new commit" agent_id=1 commit_id=e6a3651f1faa2e928fe6120e254c122451be4eea +``` + +## Example projects + +Basic GitOps example deploying NGINX: [Configuration repository](https://gitlab.com/gitlab-org/configure/examples/kubernetes-agent), [Manifest repository](https://gitlab.com/gitlab-org/configure/examples/gitops-project) |