diff options
Diffstat (limited to 'doc/user/project')
95 files changed, 1041 insertions, 2858 deletions
diff --git a/doc/user/project/canary_deployments.md b/doc/user/project/canary_deployments.md index 344caed7449..a76517a7341 100644 --- a/doc/user/project/canary_deployments.md +++ b/doc/user/project/canary_deployments.md @@ -25,9 +25,6 @@ If there is a problem with the new version of the application, only a small percentage of users are affected and the change can either be fixed or quickly reverted. -Leveraging [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments), visualize your canary -deployments right inside the [deploy board](deploy_boards.md), without the need to leave GitLab. - ## Use cases Canary deployments can be used when you want to ship features to only a portion of @@ -45,38 +42,7 @@ may want to consider [setting `service.spec.sessionAffinity` to `ClientIP` in your Kubernetes service definitions](https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies), but that is beyond the scope of this document. -## Enabling Canary Deployments - -Canary deployments require that you properly configure deploy boards: - -1. Follow the steps to [enable deploy boards](deploy_boards.md#enabling-deploy-boards). -1. To track canary deployments you must label your Kubernetes deployments and - pods with `track: canary`. To get started quickly, you can use the [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy) - template for canary deployments that GitLab provides. - -Depending on the deploy, the label should be either `stable` or `canary`. -GitLab assumes the track label is `stable` if the label is blank or missing. -Any other track label is considered `canary` (temporary). -This allows GitLab to discover whether a deployment is stable or canary (temporary). - -Once all of the above are set up and the pipeline has run at least once, -Go to the environments page under **Pipelines > Environments**. -As the pipeline executes, deploy boards clearly mark canary pods, enabling -quick and clear insight into the status of each environment and deployment. - -Canary deployments are marked with a yellow dot in the deploy board so that you -can quickly notice them. - - - -### Advanced traffic control with Canary Ingress (DEPRECATED) - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215501) in GitLab 13.6. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -WARNING: -This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +## Advanced traffic control with Canary Ingress Canary deployments can be more strategic with [Canary Ingress](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary), which is an advanced traffic routing service that controls incoming HTTP @@ -84,7 +50,7 @@ requests between stable and canary deployments based on factors such as weight, and others. GitLab uses this service in its [Auto Deploy architecture](../../topics/autodevops/upgrading_auto_deploy_dependencies.md#v2-chart-resource-architecture) to let users quickly and safely roll out their new deployments. -#### How to set up a Canary Ingress in a canary deployment +### How to set up a Canary Ingress in a canary deployment A Canary Ingress is installed by default if your Auto DevOps pipeline uses [`v2.0.0+` of `auto-deploy-image`](../../topics/autodevops/upgrading_auto_deploy_dependencies.md#verify-dependency-versions). @@ -106,14 +72,51 @@ Here's an example setup flow from scratch: 1. [Run a new Auto DevOps pipeline](../../ci/pipelines/index.md#run-a-pipeline-manually) and make sure that the `canary` job succeeds and creates a canary deployment with Canary Ingress. -#### How to check the current traffic weight on a Canary Ingress +### Show Canary Ingress deployments on deploy boards (deprecated) + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215501) in GitLab 13.6. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. +> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. + +To view canary deployments you must properly configure deploy boards: + +1. Follow the steps to [enable deploy boards](deploy_boards.md#enabling-deploy-boards). +1. To track canary deployments you must label your Kubernetes deployments and + pods with `track: canary`. To get started quickly, you can use the [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy) + template for canary deployments that GitLab provides. + +Depending on the deploy, the label should be either `stable` or `canary`. +GitLab assumes the track label is `stable` if the label is blank or missing. +Any other track label is considered `canary` (temporary). +This allows GitLab to discover whether a deployment is stable or canary (temporary). + +Once all of the above are set up and the pipeline has run at least once, +Go to the environments page under **Pipelines > Environments**. +As the pipeline executes, deploy boards clearly mark canary pods, enabling +quick and clear insight into the status of each environment and deployment. + +Canary deployments are marked with a yellow dot in the deploy board so that you +can quickly notice them. + + + +#### How to check the current traffic weight on a Canary Ingress (deprecated) + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. 1. Visit the [deploy board](../../user/project/deploy_boards.md). 1. View the current weights on the right.  -#### How to change the traffic weight on a Canary Ingress +#### How to change the traffic weight on a Canary Ingress (deprecated) + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. You can change the traffic weight in your environment's deploy board by using [GraphiQL](../../api/graphql/getting_started.md#graphiql), or by sending requests to the [GraphQL API](../../api/graphql/getting_started.md#command-line). diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index 82106c9d1a9..935bc01bae7 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -226,7 +226,7 @@ on the running pod. If you are using a self-managed GitLab instance, you need to configure Amazon credentials. GitLab uses these credentials to assume an Amazon IAM role to create your cluster. -Create an IAM user and ensure it has permissions to assume the role(s) that +Create an IAM user and ensure it has permissions to assume the roles that your users need to create EKS clusters. For example, the following policy document allows assuming a role whose name starts with diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md index f2d537513b7..c55c11151ce 100644 --- a/doc/user/project/clusters/add_existing_cluster.md +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -27,7 +27,7 @@ To add any cluster to GitLab, you need: - Either a GitLab.com account or an account for a self-managed installation running GitLab 12.5 or later. - The Maintainer role for group-level and project-level clusters. -- Access to the Admin area for instance-level clusters. +- Access to the Admin Area for instance-level clusters. - A Kubernetes cluster. - Cluster administration access to the cluster with `kubectl`. @@ -230,3 +230,22 @@ kubectl create clusterrolebinding permissive-binding \ --user=kubelet \ --group=system:serviceaccounts ``` + +## Troubleshooting + +### `There was a problem authenticating with your cluster. Please ensure your CA Certificate and Token are valid` + +If you encounter this error while connecting a Kubernetes cluster, ensure you're +properly pasting the service token. Some shells may add a line break to the +service token, making it invalid. Ensure that there are no line breaks by +pasting your token into an editor and removing any additional spaces. + +You may also experience this error if your certificate is not valid. To check that your certificate's +subject alternative names contain the correct domain for your cluster's API, run this command: + +```shell +echo | openssl s_client -showcerts -connect kubernetes.example.com:443 2>/dev/null | +openssl x509 -inform pem -noout -text +``` + +The `-connect` argument expects a `host:port` combination. For example, `https://kubernetes.example.com` would be `kubernetes.example.com:443`. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index a4f6dc325c8..8cdd1792e7f 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -10,60 +10,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. -To create a new cluster use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac). - -NOTE: -Every new Google Cloud Platform (GCP) account receives -[$300 in credit upon sign up](https://console.cloud.google.com/freetrial). -In partnership with Google, GitLab is able to offer an additional $200 for new GCP -accounts to get started with the GitLab integration with Google Kubernetes Engine. -[Follow this link](https://cloud.google.com/partners/partnercredit/?pcn_code=0014M00001h35gDQAQ#contact-form) -to apply for credit. - -NOTE: -Watch the webcast [Scalable app deployment with GitLab and Google Cloud Platform](https://about.gitlab.com/webcast/scalable-app-deploy/) -and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) -in a few clicks. - -## Create new cluster - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. - -As of GitLab 14.0, use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac) -to **safely create new clusters from GitLab**. - -Creating clusters from GitLab using cluster certificates is still available on the -GitLab UI but was **deprecated** in GitLab 14.0 and is scheduled for removal in -GitLab 15.0. We don't recommend using this method. - -You can create a new cluster hosted in EKS, GKE, on premises, and with other -providers using cluster certificates: - -- [New cluster hosted on Google Kubernetes Engine (GKE)](add_gke_clusters.md). -- [New cluster hosted on Amazon Elastic Kubernetes Service (EKS)](add_eks_clusters.md). - -To host them on premises and with other providers, you can use Terraform -or your preferred tool of choice to create and connect a cluster with GitLab. -The [GitLab Terraform provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs/resources/project_cluster) -supports connecting existing clusters using the certificate-based connection method. - -## Add existing cluster - -As of GitLab 14.0, use the [GitLab agent](../../clusters/agent/index.md) -to connect your cluster to GitLab. - -Alternatively, you can [add an existing cluster](add_existing_cluster.md) -through the certificate-based method, but we don't recommend using this method for [security implications](../../infrastructure/clusters/connect/index.md#security-implications-for-clusters-connected-with-certificates). - -## Configure your cluster - -As of GitLab 14.0, use the [GitLab agent](../../clusters/agent/index.md) -to configure your cluster. +To create and manage a new cluster use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac). ## Disable a cluster -When you successfully create a new Kubernetes cluster or add an existing -one to GitLab, the cluster connection to GitLab becomes enabled. To disable it: +When you successfully connect an existing cluster using cluster certificates, the cluster connection to GitLab becomes enabled. To disable it: 1. Go to your: - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. @@ -95,26 +46,3 @@ To remove the Kubernetes cluster integration: 1. Go to your cluster details page. 1. Select the **Advanced Settings** tab. 1. Select either **Remove integration** or **Remove integration and resources**. - -## Access controls - -See [cluster access controls (RBAC or ABAC)](cluster_access.md). - -## Troubleshooting - -### There was a problem authenticating with your cluster. Please ensure your CA Certificate and Token are valid - -If you encounter this error while adding a Kubernetes cluster, ensure you're -properly pasting the service token. Some shells may add a line break to the -service token, making it invalid. Ensure that there are no line breaks by -pasting your token into an editor and removing any additional spaces. - -You may also experience this error if your certificate is not valid. To check that your certificate's -subject alternative names contain the correct domain for your cluster's API, run this: - -```shell -echo | openssl s_client -showcerts -connect kubernetes.example.com:443 2>/dev/null | -openssl x509 -inform pem -noout -text -``` - -Note that the `-connect` argument expects a `host:port` combination. For example, `https://kubernetes.example.com` would be `kubernetes.example.com:443`. diff --git a/doc/user/project/clusters/cluster_access.md b/doc/user/project/clusters/cluster_access.md index 8ff0a275649..43ceb3673d8 100644 --- a/doc/user/project/clusters/cluster_access.md +++ b/doc/user/project/clusters/cluster_access.md @@ -28,7 +28,7 @@ Helm also creates additional service accounts and other resources for each installed application. Consult the documentation of the Helm charts for each application for details. -If you are [adding an existing Kubernetes cluster](add_remove_clusters.md#add-existing-cluster), +If you are [adding an existing Kubernetes cluster](add_existing_cluster.md), ensure the token of the account has administrator privileges for the cluster. The resources created by GitLab differ depending on the type of cluster. diff --git a/doc/user/project/clusters/deploy_to_cluster.md b/doc/user/project/clusters/deploy_to_cluster.md index c1cdf754c11..fc41533b17c 100644 --- a/doc/user/project/clusters/deploy_to_cluster.md +++ b/doc/user/project/clusters/deploy_to_cluster.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect your cluster to GitLab, use the [GitLab agent](../../clusters/agent/index.md). -To deploy with the agent, use the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). +To deploy with the agent, use the [CI/CD workflow](../../clusters/agent/ci_cd_workflow.md). A Kubernetes cluster can be the destination for a deployment job. If diff --git a/doc/user/project/clusters/gitlab_managed_clusters.md b/doc/user/project/clusters/gitlab_managed_clusters.md index 9c5cc14f720..e295abf8d31 100644 --- a/doc/user/project/clusters/gitlab_managed_clusters.md +++ b/doc/user/project/clusters/gitlab_managed_clusters.md @@ -9,15 +9,19 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 11.5. > - Became [optional](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26565) in GitLab 11.11. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect your cluster to GitLab, use the [GitLab agent](../../../user/clusters/agent/index.md). To manage applications, use the [Cluster Project Management Template](../../../user/clusters/management_project_template.md). +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `certificate_based_clusters`. + You can choose to allow GitLab to manage your cluster for you. If your cluster is managed by GitLab, resources for your projects are automatically created. See -the [Access controls](add_remove_clusters.md#access-controls) section for +the [Access controls](cluster_access.md) section for details about the created resources. If you choose to manage your own cluster, project-specific resources aren't created diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index b5e2a1bad51..b1158be9fb6 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -4,14 +4,23 @@ group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Kubernetes Logs (DEPRECATED) **(FREE)** +# Kubernetes Logs (DEPRECATED) **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4752) in GitLab 11.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26383) from GitLab Ultimate to GitLab Free 12.9. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/360182) behind a [feature flag](../../../administration/feature_flags.md) named `monitor_logging` in GitLab 15.0. Disabled by default. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: +This feature is in its end-of-life process. This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +It will be [removed completely](https://gitlab.com/gitlab-org/gitlab/-/issues/346485) in GitLab 15.2. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `monitor_logging` and the one named `certificate_based_clusters`. +On GitLab.com, this feature is not available. +This feature is not recommended for production use. GitLab makes it easy to view the logs of running pods in [connected Kubernetes clusters](index.md). By displaying the logs directly in GitLab diff --git a/doc/user/project/clusters/protect/container_host_security/index.md b/doc/user/project/clusters/protect/container_host_security/index.md deleted file mode 100644 index c897100f14e..00000000000 --- a/doc/user/project/clusters/protect/container_host_security/index.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Container Host Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Host Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -Container Host Security in GitLab provides Intrusion Detection and Prevention capabilities that can -monitor and (optionally) block activity inside the containers themselves. This is done by leveraging -an integration with Falco to provide the monitoring capabilities and an integration with Pod -Security Policies and AppArmor to provide blocking capabilities. - -## Overview - -Container Host Security can be used to monitor and block activity inside a container as well as to -enforce security policies across the entire Kubernetes cluster. Falco profiles allow for users to -define the activity they want to monitor for and detect. Among other things, this can include system -log entries, process starts, file activity, and network ports opened. AppArmor is used to block any -undesired activity via AppArmor profiles. These profiles are loaded into the cluster when -referenced by Pod Security Policies. - -By default, Container Host Security is deployed into the cluster in monitor mode only, with no -default profiles or rules running out-of-the-box. Activity monitoring and blocking begins only when -users define profiles for these technologies. - -## Installation - -See the [installation guide](quick_start_guide.md) for the recommended steps to install the -Container Host Security capabilities. This guide shows the recommended way of installing Container -Host Security through the Cluster Management Project. However, it's also possible to do a manual -installation through our Helm chart. - -## Features - -- Prevent containers from starting as root. -- Limit the privileges and system calls available to containers. -- Monitor system logs, process starts, files read/written/deleted, and network ports opened. -- Optionally block processes from starting or files from being read/written/deleted. - -## Supported container orchestrators - -Kubernetes v1.14+ is the only supported container orchestrator. OpenShift and other container -orchestrators aren't supported. - -## Supported Kubernetes providers - -The following cloud providers are supported: - -- Amazon EKS -- Google GKE - -Although Container Host Security may function on Azure or self-managed Kubernetes instances, it isn't -officially tested and supported on those providers. - -## Roadmap - -See the [Category Direction page](https://about.gitlab.com/direction/protect/container_host_security/) -for more information on the product direction of Container Host Security. diff --git a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md deleted file mode 100644 index af3128e3006..00000000000 --- a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Getting started with Container Host Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Host Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The following steps are recommended for installing Container Host Security. - -## Installation steps - -The following steps are recommended to install and use Container Host Security through GitLab: - -1. [Install at least one runner and connect it to GitLab](https://docs.gitlab.com/runner/). -1. [Create a group](../../../../group/#create-a-group). -1. [Connect a Kubernetes cluster to the group](../../add_remove_clusters.md). -1. [Create a cluster management project and associate it with the Kubernetes cluster](../../../../clusters/management_project.md). - -1. Install and configure an Ingress node: - - - [Install the Ingress node via CI/CD (Cluster Management Project)](../../../../clusters/applications.md#install-ingress-using-gitlab-cicd). - - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../gitlab_managed_clusters.md#base-domain) - into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes - cluster. - -1. [Install and configure Falco](../../../../clusters/applications.md#install-falco-using-gitlab-cicd) - for activity monitoring. -1. [Install and configure AppArmor](../../../../clusters/applications.md#install-apparmor-using-gitlab-cicd) - for activity blocking. -1. [Configure Pod Security Policies](../../../../clusters/applications.md#using-podsecuritypolicy-in-your-deployments) - (required to be able to load AppArmor profiles). - -It's possible to install and manage Falco and AppArmor in other ways, such as installing them -manually in a Kubernetes cluster and then connecting it back to GitLab. These methods aren't -supported or documented. - -## Viewing the logs - -Falco logs can be viewed by running the following command in your Kubernetes cluster: - -```shell -kubectl -n gitlab-managed-apps logs -l app=falco -``` - -## Troubleshooting - -### Trouble connecting to the cluster - -Your CI/CD pipeline may occasionally fail or have trouble connecting to the cluster. Here are some -initial troubleshooting steps that resolve the most common problems: - -1. [Clear the cluster cache](../../gitlab_managed_clusters.md#clearing-the-cluster-cache) -1. If things still aren't working, a more assertive set of actions may help get things back to a - good state: - - - Stop and [delete the problematic environment](../../../../../ci/environments/#delete-a-stopped-environment) - in GitLab. - - Delete the relevant namespace in Kubernetes by running - `kubectl delete namespaces <insert-some-namespace-name>` in your Kubernetes cluster. - - Rerun the application project pipeline to redeploy the application. - -**Related documentation links:** - -- [Cluster Management Project](../../../../clusters/management_project.md) diff --git a/doc/user/project/clusters/protect/container_network_security/index.md b/doc/user/project/clusters/protect/container_network_security/index.md deleted file mode 100644 index b294859c660..00000000000 --- a/doc/user/project/clusters/protect/container_network_security/index.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Container Network Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Network Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -Container Network Security in GitLab provides basic firewall functionality by leveraging Cilium -NetworkPolicies to filter traffic going in and out of the cluster as well as traffic between pods -inside the cluster. Container Network Security can be used to enforce L3, L4, and L7 policies and -can prevent an attacker with control over one pod from spreading laterally to access other pods in -the same cluster. Both Ingress and Egress rules are supported. - -By default, Cilium is deployed in Detection-only mode and only logs attack attempts. GitLab provides -a set of out-of-the-box policies as examples and to help users get started. These policies are -disabled by default, as they must usually be customized to match application-specific needs. - -## Installation - -See the [installation guide](quick_start_guide.md) for the recommended steps to install GitLab -Container Network Security. This guide shows the recommended way of installing Container Network -Security through the Cluster Management Project. However, it's also possible to install Cilium -manually through our Helm chart. - -## Features - -- GitLab managed installation of Cilium. -- Support for L3, L4, and L7 policies. -- Ability to export logs to a SIEM. -- Statistics page showing volume of packets processed and dropped over time (Ultimate users only). -- Management of NetworkPolicies through code in a project (Available for auto DevOps users only). -- Management of CiliumNetworkPolicies through a UI policy manager (Ultimate users only). - -## Supported container orchestrators - -Kubernetes v1.14+ is the only supported container orchestrator. OpenShift and other container -orchestrators aren't supported. - -## Supported Kubernetes providers - -The following cloud providers are supported: - -- Amazon EKS -- Google GKE - -Although Container Network Security may function on Azure or self-managed Kubernetes instances, it -isn't officially tested and supported on those providers. - -## Supported NetworkPolicies - -GitLab only supports the use of CiliumNetworkPolicies. Although generic Kubernetes NetworkPolicies -or other kinds of NetworkPolicies may work, GitLab doesn't test or support them. - -## Managing NetworkPolicies through GitLab vs your cloud provider - -Some cloud providers offer integrations with Cilium or offer other ways to manage NetworkPolicies in -Kubernetes. GitLab Container Network Security doesn't support deployments that have NetworkPolicies -managed by an external provider. By choosing to manage NetworkPolicies through GitLab, you can take -advantage of the following benefits: - -- Support for handling NetworkPolicy infrastructure as code. -- Full revision history and audit log of all changes made. -- Ability to revert back to a previous version at any time. - -## Roadmap - -See the [Category Direction page](https://about.gitlab.com/direction/protect/container_network_security/) -for more information on the product direction of Container Network Security. diff --git a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md deleted file mode 100644 index 7671ed7eb73..00000000000 --- a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md +++ /dev/null @@ -1,230 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Getting started with Container Network Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Network Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The following steps are recommended for installing Container Network Security. - -## Installation steps - -The following steps are recommended to install and use Container Network Security through GitLab: - -1. [Install at least one runner and connect it to GitLab](https://docs.gitlab.com/runner/). -1. [Create a group](../../../../group/#create-a-group). -1. [Connect a Kubernetes cluster to the group](../../add_remove_clusters.md). -1. [Create a cluster management project and associate it with the Kubernetes cluster](../../../../clusters/management_project.md). - -1. Install and configure an Ingress node: - - - [Install the Ingress node via CI/CD (Cluster Management Project)](../../../../clusters/applications.md#install-ingress-using-gitlab-cicd). - - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../gitlab_managed_clusters.md#base-domain) - into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes - cluster. - -1. [Install and configure Cilium](#use-the-cluster-management-template-to-install-cilium). -1. Be sure to restart all pods that were running before Cilium was installed by running this command - in your cluster: - - `kubectl get pods --all-namespaces -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HOSTNETWORK:.spec.hostNetwork --no-headers=true | grep '<none>' | awk '{print "-n "$1" "$2}' | xargs -L 1 -r kubectl delete pod` - - You can skip this step if `nodeinit.restartPods` is set to `true` on your Helm chart. - -It's possible to install and manage Cilium in other ways. For example, you could use the GitLab Helm -chart to install Cilium manually in a Kubernetes cluster, and then connect it back to GitLab. -However, such methods aren't documented or officially supported by GitLab. - -### Use the Cluster Management template to install Cilium - -[Cilium](https://cilium.io/) is a networking plug-in for Kubernetes that you can use to implement -support for [`NetworkPolicy`](https://kubernetes.io/docs/concepts/services-networking/network-policies/) -resources. For more information, see [Network Policies](../../../../../topics/autodevops/stages.md#network-policy). - -You can use the [Cluster Management Project Template](../../../../clusters/management_project_template.md) -to install Cilium in your Kubernetes cluster. - -1. In your cluster management project, go to `helmfile.yaml` and uncomment `- path: applications/cilium/helmfile.yaml`. -1. In `applications/cilium/helmfile.yaml`, set `clusterType` to either `gke` or `eks` based on which Kubernetes provider your are using. - - ```yaml - environments: - default: - values: - # Set to "gke" or "eks" based on your cluster type - - clusterType: "" - ``` - -1. Merge or push these changes to the default branch of your cluster management project, -and [GitLab CI/CD](../../../../../ci/index.md) will automatically install Cilium. - -WARNING: -Installation and removal of the Cilium requires a **manual** -[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods) -of all affected pods in all namespaces to ensure that they are -[managed](https://docs.cilium.io/en/stable/operations/troubleshooting/#ensure-managed-pod) -by the correct networking plug-in. When Hubble is enabled, its related pod might require a -restart depending on whether it started prior to Cilium. For more information, see -[Failed Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment) -in the Kubernetes docs. - -NOTE: -Major upgrades might require additional setup steps. For more information, see -the official [upgrade guide](https://docs.cilium.io/en/stable/operations/upgrade/). - -Support for installing the Cilium application is provided by the -GitLab Container Security group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). - -### Configure the Cilium Helm chart - -You can customize Cilium's Helm variables by editing the `applications/cilium/values.yaml` -file in your cluster management project. Refer to the [Cilium Helm reference](https://docs.cilium.io/en/stable/helm-reference/) -for the available configuration options. - -By default, Cilium's -[audit mode](https://docs.cilium.io/en/stable/gettingstarted/policy-creation/#enable-policy-audit-mode) -is enabled. In audit mode, Cilium doesn't drop disallowed packets. You -can use `policy-verdict` log to observe policy-related decisions. You -can disable audit mode by setting `policyAuditMode: false` in -`applications/cilium/values.yaml`. - -The Cilium monitor log for traffic is logged out by the -`cilium-monitor` sidecar container. You can check these logs with the following command: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -You can disable the monitor log in `application/cilium/values.yaml`: - -```yaml -monitor: - enabled: false -``` - -The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default -and it's set to collect per namespace flow metrics. This metrics are accessible on the -[Threat Monitoring](../../../../application_security/threat_monitoring/index.md) -dashboard. You can disable Hubble by adding the following to -`applications/cilium/values.yaml`: - -```yaml -hubble: - enabled: false -``` - -You can also adjust Helm values for Hubble by using -`applications/cilium/values.yaml`: - -```yaml -hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -## Managing Network Policies - -Managing NetworkPolicies through GitLab is advantageous over managing the policies in Kubernetes -directly. Kubernetes doesn't provide a GUI editor, a change control process, or a revision history. -Network Policies can be managed through GitLab in one of two ways: - -- Management through a YAML file in each application's project (for projects using Auto DevOps). For - more information, see the [Network Policy documentation](../../../../../topics/autodevops/stages.md#network-policy). -- Management through the GitLab Policy management UI (for projects not using Auto DevOps). For more - information, see the [Container Network Policy documentation](../../../../application_security/policies/index.md#container-network-policy) (Ultimate only). - -Each method has benefits and drawbacks: - -| | YAML method | UI method (Ultimate only) | -|--|:------------|:-------------------------------| -| **Benefits** | A change control process is possible by requiring [MR Approvals](../../../merge_requests/approvals/index.md). All changes are fully tracked and audited in the same way that Git tracks the history of any file in its repository. | The UI provides a simple rules editor for users who are less familiar with the YAML syntax of NetworkPolicies. This view is a live representation of the policies currently deployed in the Kubernetes cluster. The UI also allows for multiple network policies to be created per environment. | -| **Drawbacks** | Only one network policy can be deployed per environment (although that policy can be as detailed as needed). Also, if changes were made in Kubernetes directly rather than through the `auto-deploy-values.yaml` file, the YAML file's contents don't represent the actual state of policies deployed in Kubernetes. | Policy changes aren't audited and a change control process isn't available. | - -Users are encouraged to choose one of the two methods to manage their policies. If users attempt to -use both methods simultaneously, when the application project pipeline runs the contents of the -NetworkPolicy in the `auto-deploy-values.yaml` file may override policies configured in the UI -editor. - -## Monitoring throughput **(ULTIMATE)** - -To view statistics for Container Network Security, you must follow the installation steps above and -configure GitLab integration with Prometheus. Also, if you use custom Helm values for Cilium, you -must enable Hubble with flow metrics for each namespace by adding the following lines to -your [Cilium values](#use-the-cluster-management-template-to-install-cilium): - -```yaml -hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -Additional information about the statistics page is available in the -[documentation that describes the Threat Management UI](../../../../application_security/policies/index.md#container-network-policy). - -## Forwarding logs to a SIEM - -Cilium logs can be forwarded to a SIEM or an external logging system through syslog protocol by -installing and configuring Fluentd. Fluentd can be installed through the GitLab -[Cluster Management Project](../../../../clusters/applications.md#install-fluentd-using-gitlab-cicd). - -## Viewing the logs - -Cilium logs can be viewed by running the following command in your Kubernetes cluster: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -## Troubleshooting - -### Traffic is not being blocked as expected - -By default, Cilium is installed in Audit mode only, meaning that NetworkPolicies log policy -violations but don't block any traffic. To set Cilium to Blocking mode, you must add the following -lines to the `applications/cilium/values.yaml` file in your cluster management project: - -```yaml -policyEnforcementMode: "always" - -monitor: - eventTypes: ["drop", "policy-verdict"] -``` - -### Traffic is not being allowed as expected - -Keep in mind that when Cilium is set to blocking mode (rather than Audit mode), NetworkPolicies -operate on an allow-list basis. If one or more NetworkPolicies apply to a node, then all traffic -that doesn't match at least one Policy is blocked. To resolve, add NetworkPolicies defining the -traffic that you want to allow in the node. - -### Trouble connecting to the cluster - -Occasionally, your CI/CD pipeline may fail or have trouble connecting to the cluster. Here are some -initial troubleshooting steps that resolve the most common problems: - -1. [Clear the cluster cache](../../gitlab_managed_clusters.md#clearing-the-cluster-cache). -1. If things still aren't working, a more assertive set of actions may help get things back into a - good state: - - - Stop and [delete the problematic environment](../../../../../ci/environments/index.md#delete-a-stopped-environment) in GitLab. - - Delete the relevant namespace in Kubernetes by running `kubectl delete namespaces <insert-some-namespace-name>` in your Kubernetes cluster. - - Rerun the application project pipeline to redeploy the application. - -**Related documentation links:** - -- [Cluster Management Project](../../../../clusters/management_project.md) diff --git a/doc/user/project/clusters/protect/index.md b/doc/user/project/clusters/protect/index.md deleted file mode 100644 index 6b89f7f1557..00000000000 --- a/doc/user/project/clusters/protect/index.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Protecting your deployed applications **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -The Container Network Security and Container Host Security features are in their end-of-life -processes. They're -[deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -GitLab makes it straightforward to protect applications deployed in [connected Kubernetes clusters](index.md). -These protections are available in the Kubernetes network layer and in the container itself. At -the network layer, the Container Network Security capabilities in GitLab provide basic firewall -functionality by leveraging Cilium NetworkPolicies to filter traffic going in and out of the cluster -and traffic between pods inside the cluster. Inside the container, Container Host Security provides -Intrusion Detection and Prevention capabilities that can monitor and block activity inside the -containers themselves. - -## Capabilities - -The following capabilities are available to protect deployed applications in Kubernetes: - -- Container Network Security - - [Overview](container_network_security/index.md) - - [Installation guide](container_network_security/quick_start_guide.md) -- Container Host Security - - [Overview](container_host_security/index.md) - - [Installation guide](container_host_security/quick_start_guide.md) diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 9d623518f72..086e1fccf6c 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -40,8 +40,8 @@ for an overview of how this is accomplished in GitLab! To create an executable runbook, you need: - **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the - applications. The simplest way to get started is to add a cluster using one - of the [GitLab integrations](../add_remove_clusters.md#create-new-cluster). + applications. The simplest way to get started is to connect a cluster using the + [GitLab agent](../../../clusters/agent/index.md). - **Ingress** - Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications. - **JupyterHub** - [JupyterHub](https://jupyterhub.readthedocs.io/) is a multi-user diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index cf571abbf8a..93bc41dc24c 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -2,506 +2,10 @@ stage: Configure group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../../../update/removals.md#gitlab-serverless' --- -# Deploying AWS Lambda function using GitLab CI/CD **(FREE)** +# Deploying AWS Lambda function using GitLab CI/CD (removed) **(FREE)** -GitLab allows users to easily deploy AWS Lambda functions and create rich serverless applications. - -GitLab supports deployment of AWS Lambda functions through GitLab CI/CD using the following Serverless frameworks: - -- [Serverless Framework with AWS](#serverless-framework) -- [AWS' Serverless Application Model (SAM)](#aws-serverless-application-model) - -## Serverless Framework - -The [Serverless Framework can deploy to AWS](https://www.serverless.com/framework/docs/providers/aws/). - -We have prepared an example with a step-by-step guide to create a simple function and deploy it on AWS. - -Additionally, in the [How To section](#how-to), you can read about different use cases like: - -- Running a function locally. -- Working with secrets. -- Setting up CORS. - -Alternatively, you can quickly [create a new project with a template](../../working_with_projects.md#create-a-project). The [`Serverless Framework/JS` template](https://gitlab.com/gitlab-org/project-templates/serverless-framework/) already includes all parts described below. - -### Example - -This example shows you how to: - -1. Create a basic AWS Lambda Node.js function. -1. Link the function to an API Gateway `GET` endpoint. - -#### Steps - -The example consists of the following steps: - -1. Creating a Lambda handler function. -1. Creating a `serverless.yml` file. -1. Crafting the `.gitlab-ci.yml` file. -1. Setting up your AWS credentials with your GitLab account. -1. Deploying your function. -1. Testing the deployed function. - -Lets take it step by step. - -#### Creating a Lambda handler function - -Your Lambda function is the primary handler of requests. In this case, create a very simple Node.js `hello` function: - -```javascript -'use strict'; - -module.exports.hello = async event => { - return { - statusCode: 200, - body: JSON.stringify( - { - message: 'Your function executed successfully!' - }, - null, - 2 - ), - }; -}; -``` - -Place this code in the file `src/handler.js`. - -`src` is the standard location for serverless functions, but is customizable should you desire that. - -In our case, `module.exports.hello` defines the `hello` handler to reference later in the `serverless.yml`. - -You can learn more about the [AWS Lambda Node.js function handler](https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html) and all its various options in its documentation. - -#### Creating a `serverless.yml` file - -In the root of your project, create a `serverless.yml` file containing configuration specifics for the Serverless Framework. - -Put the following code in the file: - -```yaml -service: gitlab-example -provider: - name: aws - runtime: nodejs14.x - -functions: - hello: - handler: src/handler.hello - events: - - http: GET hello -``` - -Our function contains a handler and a event. - -The handler definition provisions the Lambda function using the source code located `src/handler.hello`. - -The `events` declaration creates an AWS API Gateway `GET` endpoint to receive external requests and hand them over to the Lambda function via a service integration. - -You can read more about the [available properties and additional configuration possibilities](https://www.serverless.com/framework/docs/providers/aws/guide/serverless.yml/) of the Serverless Framework. - -#### Crafting the `.gitlab-ci.yml` file - -In a `.gitlab-ci.yml` file in the root of your project, place the following code: - -```yaml -image: node:latest - -stages: - - deploy - -production: - stage: deploy - before_script: - - npm config set prefix /usr/local - - npm install -g serverless - script: - - serverless deploy --stage production --verbose - environment: production -``` - -This example code does the following: - -1. Uses the `node:latest` image for all GitLab CI/CD builds -1. The `deploy` stage: - - Installs the Serverless Framework. - - Deploys the serverless function to your AWS account using the AWS credentials - defined above. - -#### Setting up your AWS credentials with your GitLab account - -In order to interact with your AWS account, the GitLab CI/CD pipelines require both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be defined in your GitLab settings under **Settings > CI/CD > Variables**. -For more information please see [Create a custom variable in the UI](../../../../ci/variables/index.md#custom-variables-validated-by-gitlab). - - The AWS credentials you provide must include IAM policies that provision correct - access control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. - -#### Deploying your function - -`git push` the changes to your GitLab repository and the GitLab build pipeline deploys your function. - -Your GitLab deploy stage log contains output containing your AWS Lambda endpoint URL, -with log lines similar to this: - -```plaintext -endpoints: - GET - https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello -``` - -#### Manually testing your function - -Running the following `curl` command should trigger your function. -Your URL should be the one retrieved from the GitLab deploy stage log: - -```shell -curl "https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello" -``` - -That should output: - -```json -{ - "message": "Your function executed successfully!" -} -``` - -Hooray! You now have a AWS Lambda function deployed via GitLab CI/CD. - -Nice work! - -### How To - -In this section, we show you how to build on the basic example to: - -- Run the function locally. -- Set up secret variables. -- Set up CORS. - -#### Running function locally - -The `serverless-offline` plugin allows to run your code locally. To run your code locally: - -1. Add the following to your `serverless.yml`: - - ```yaml - plugins: - - serverless-offline - ``` - -1. Start the service by running the following command: - - ```shell - serverless offline - ``` - -Running the following `curl` command should trigger your function. - -```shell -curl "http://localhost:3000/hello" -``` - -It should output: - -```json -{ - "message": "Your function executed successfully!" -} -``` - -#### Secret variables - -Secrets are injected into your functions using environment variables. - -By defining variables in the provider section of the `serverless.yml`, you add them to -the environment of the deployed function: - -```yaml -provider: - # Other configuration omitted - # ... - environment: - A_VARIABLE: ${env:A_VARIABLE} -``` - -From there, you can reference them in your functions as well. -Remember to add `A_VARIABLE` to your GitLab CI/CD variables under **Settings > CI/CD > Variables** to be picked up and deployed with your function. - -NOTE: -Anyone with access to the AWS environment may be able to see the values of those -variables persisted in the lambda definition. - -#### Setting up CORS - -If you want to set up a web page that makes calls to your function, like we have done in the [template](https://gitlab.com/gitlab-org/project-templates/serverless-framework/), you need to deal with the Cross-Origin Resource Sharing (CORS). - -The quick way to do that is to add the `cors: true` flag to the HTTP endpoint in your `serverless.yml`: - -```yaml -functions: - hello: - handler: src/handler.hello - events: - - http: # Rewrite this part to enable CORS - path: hello - method: get - cors: true # <-- CORS here -``` - -You also need to return CORS specific headers in your function response: - -```javascript -'use strict'; - -module.exports.hello = async event => { - return { - statusCode: 200, - headers: { - // Uncomment the line below if you need access to cookies or authentication - // 'Access-Control-Allow-Credentials': true, - 'Access-Control-Allow-Origin': '*' - }, - body: JSON.stringify( - { - message: 'Your function executed successfully!' - }, - null, - 2 - ), - }; -}; -``` - -For more information, see the [Your CORS and API Gateway survival guide](https://www.serverless.com/blog/cors-api-gateway-survival-guide/) -blog post written by the Serverless Framework team. - -#### Writing automated tests - -The [Serverless Framework](https://gitlab.com/gitlab-org/project-templates/serverless-framework/) -example project shows how to use Jest, Axios, and `serverless-offline` plugin to do -automated testing of both local and deployed serverless function. - -### Examples and template - -The example code is available: - -- As a [clonable repository](https://gitlab.com/gitlab-org/serverless/examples/serverless-framework-js). -- In a version with [tests and secret variables](https://gitlab.com/gitlab-org/project-templates/serverless-framework/). - -You can also use a [template](../../working_with_projects.md#create-a-project) -(based on the version with tests and secret variables) from within the GitLab UI (see -the `Serverless Framework/JS` template). - -## AWS Serverless Application Model - -AWS Serverless Application Model is an open source framework for building serverless -applications. It makes it easier to build and deploy serverless applications. For more -details, please take a look at AWS documentation on [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/). - -### Deploying AWS Lambda function using AWS SAM and GitLab CI/CD - -GitLab allows developers to build and deploy serverless applications using the combination of: - -- [AWS Serverless Application Model (AWS SAM)](https://aws.amazon.com/serverless/sam/). -- GitLab CI/CD. - -### Example - -This example shows you how to: - -- Install SAM CLI. -- Create a sample SAM application including a Lambda function and API Gateway. -- Build and deploy the application to your AWS account using GitLab CI/CD. - -### Steps - -The example consists of the following steps: - -1. Installing SAM CLI. -1. Creating an AWS SAM application using SAM CLI. -1. Crafting the `.gitlab-ci.yml` file. -1. Setting up your AWS credentials with your GitLab account. -1. Deploying your application. -1. Testing the deployed function. - -### Installing SAM CLI - -AWS SAM provides a CLI called AWS SAM CLI to make it easier to create and manage -applications. - -Some steps in this documentation use SAM CLI. Follow the instructions for -[installing SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) -to install and configure SAM CLI. - -If you use [AWS Cloud9](https://aws.amazon.com/cloud9/) as your integrated development -environment (IDE), the following are installed for you: - -- [AWS Command Line Interface](https://docs.aws.amazon.com/en_pv/cli/latest/userguide/cli-chap-install.html) -- [SAM CLI](https://docs.aws.amazon.com/en_pv/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) -- [Docker](https://docs.docker.com/install/) and necessary Docker images. - -### Creating an AWS SAM application using SAM CLI - -To create a new AWS SAM application: - -1. Create a new GitLab project. -1. `git clone` the project into your local environment. -1. Change to the newly cloned project and create a new SAM app using the following command: - - ```shell - sam init -r python3.8 -n gitlabpoc --app-template "hello-world" - ``` - -1. `git push` the application back to the GitLab project. - -This creates a SAM app named `gitlabpoc` using the default configuration, a single -Python 3.8 function invoked by an [Amazon API Gateway](https://aws.amazon.com/api-gateway/) -endpoint. To see additional runtimes supported by SAM and options for `sam init`, run: - -```shell -sam init -h -``` - -### Setting up your AWS credentials with your GitLab account - -In order to interact with your AWS account, the GitLab CI/CD pipelines require both -`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be set in the project's CI/CD variables. - -To set these: - -1. Navigate to the project's **Settings > CI/CD**. -1. Expand the **Variables** section and create entries for `AWS_ACCESS_KEY_ID` and - `AWS_SECRET_ACCESS_KEY`. -1. Mask the credentials so they do not show in logs using the **Masked** toggle. - -The AWS credentials you provide must include IAM policies that provision correct access -control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. - -### Crafting the `.gitlab-ci.yml` file - -In a [`.gitlab-ci.yml`](../../../../ci/yaml/index.md) file in the root of your project, -add the following and replace `<S3_bucket_name>` with the name of the S3 bucket where you -want to store your package: - -```yaml -image: python:latest - -stages: - - deploy - -production: - stage: deploy - before_script: - - apt-get update - - apt-get install -y python3-pip - - pip3 install awscli --upgrade - - pip3 install aws-sam-cli --upgrade - script: - - sam build - - sam package --output-template-file packaged.yaml --s3-bucket <S3_bucket_name> - - sam deploy --template-file packaged.yaml --stack-name gitlabpoc --s3-bucket <S3_bucket_name> --capabilities CAPABILITY_IAM --region us-east-1 - environment: production -``` - -Let's examine the configuration file more closely: - -- `image` specifies the Docker image to use for this build. This is the latest Python - image since the sample application is written in Python. -- AWS CLI and AWS SAM CLI are installed in the `before_script` section. -- SAM build, package, and deploy commands are used to build, package, and deploy the - application. - -### Deploying your application - -Push changes to your GitLab repository and the GitLab build pipeline -deploys your application. If your: - -- Build and deploy are successful, [test your deployed application](#testing-the-deployed-application). -- Build fails, look at the build log to see why the build failed. Some common reasons - the build might fail are: - - - Incompatible versions of software. For example, Python runtime version might be - different from the Python on the build machine. Address this by installing the - required versions of the software. - - You may not be able to access your AWS account from GitLab. Check the CI/CD variables - you set up with AWS credentials. - - You may not have permission to deploy a serverless application. Make sure you - provide all required permissions to deploy a serverless application. - -### Testing the deployed application - -To test the application you deployed, please go to the build log and follow the following steps: - -1. Click on "Show complete raw" on the upper right-hand corner: - -  - -1. Look for HelloWorldApi – API Gateway endpoint similar to shown below: - -  - -1. Use curl to test the API. For example: - - ```shell - curl "https://py4rg7qtlg.execute-api.us-east-1.amazonaws.com/Prod/hello/" - ``` - -Output should be: - -```json -{"message": "hello world"} -``` - -### Testing Locally - -AWS SAM provides functionality to test your applications locally. You must have AWS SAM -CLI installed locally for you to test locally. - -First, test the function. - -SAM provides a default event in `events/event.json` that includes a message body of: - -```plaintext -{\"message\": \"hello world\"} -``` - -If you pass that event into the `HelloWorldFunction`, it should respond with the same -body. - -Invoke the function by running: - -```shell -sam local invoke HelloWorldFunction -e events/event.json -``` - -Output should be: - -```json -{"message": "hello world"} -``` - -After you confirm that Lambda function is working as expected, test the API Gateway -using following steps. - -Start the API locally by running: - -```shell -sam local start-api -``` - -SAM again launches a Docker container, this time with a mocked Amazon API Gateway -listening on `localhost:3000`. - -Call the `hello` API by running: - -```shell -curl "http://127.0.0.1:3000/hello" -``` - -Output again should be: - -```json -{"message": "hello world"} -``` +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86267) in GitLab 15.0. diff --git a/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png b/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png Binary files differdeleted file mode 100644 index a19d236fc39..00000000000 --- a/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-endpoint.png b/doc/user/project/clusters/serverless/img/function-endpoint.png Binary files differdeleted file mode 100644 index a38fe2cb6c2..00000000000 --- a/doc/user/project/clusters/serverless/img/function-endpoint.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-execution.png b/doc/user/project/clusters/serverless/img/function-execution.png Binary files differdeleted file mode 100644 index f60dd277081..00000000000 --- a/doc/user/project/clusters/serverless/img/function-execution.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-list_v12_7.png b/doc/user/project/clusters/serverless/img/function-list_v12_7.png Binary files differdeleted file mode 100644 index f2a27ce7b0f..00000000000 --- a/doc/user/project/clusters/serverless/img/function-list_v12_7.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/sam-api-endpoint.png b/doc/user/project/clusters/serverless/img/sam-api-endpoint.png Binary files differdeleted file mode 100644 index 3407b2684fd..00000000000 --- a/doc/user/project/clusters/serverless/img/sam-api-endpoint.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/sam-complete-raw.png b/doc/user/project/clusters/serverless/img/sam-complete-raw.png Binary files differdeleted file mode 100644 index 1130cd29d56..00000000000 --- a/doc/user/project/clusters/serverless/img/sam-complete-raw.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png b/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png Binary files differdeleted file mode 100644 index f88eb4bdcd2..00000000000 --- a/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 29164da307b..432caa8476f 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -2,818 +2,10 @@ stage: Configure group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../../../update/removals.md#gitlab-serverless' --- -# Serverless (DEPRECATED) **(FREE)** +# Serverless (removed) **(FREE)** -> - Introduced in GitLab 11.5. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3. - -WARNING: -Serverless is currently in [alpha](../../../../policy/alpha-beta-support.md#alpha-features). - -## Overview - -Serverless architectures offer Operators and Developers the ability write highly scalable applications without provisioning a single server. - -GitLab supports several ways deploy Serverless applications in both Kubernetes Environments and also major cloud Function as a Service (FaaS) environments. - -Currently we support: - -- [Knative](#knative): Build Knative applications with Knative and `gitlabktl` on GKE and EKS. -- [AWS Lambda](aws.md): Create serverless applications via the Serverless Framework and GitLab CI/CD. - -## Knative - -Run serverless workloads on Kubernetes using [Knative](https://cloud.google.com/knative/). - -Knative extends Kubernetes to provide a set of middleware components that are useful to build -modern, source-centric, container-based applications. Knative brings some significant benefits out -of the box through its main components: - -- [Serving](https://github.com/knative/serving): Request-driven compute that can scale to zero. -- [Eventing](https://github.com/knative/eventing): Management and delivery of events. - -For more information on Knative, visit the [Knative docs repository](https://github.com/knative/docs). - -With GitLab Serverless, you can deploy both FaaS and serverless applications. - -## Prerequisites - -To run Knative on GitLab, you need: - -1. **Existing GitLab project:** You need a GitLab project to associate all resources. The simplest way to get started: - - If you are planning on [deploying functions](#deploying-functions), - clone the [functions example project](https://gitlab.com/knative-examples/functions) to get - started. - - If you are planning on [deploying a serverless application](#deploying-serverless-applications), - clone the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get - started. -1. **Kubernetes Cluster:** An RBAC-enabled Kubernetes cluster is required to deploy Knative. - The simplest way to get started is to add a cluster using the GitLab [GKE integration](../add_remove_clusters.md). - The set of minimum recommended cluster specifications to run Knative is 3 nodes, 6 vCPUs, and 22.50 GB memory. -1. **GitLab Runner:** A runner is required to run the CI jobs that deploy serverless - applications or functions onto your cluster. You can install GitLab Runner - onto the [existing Kubernetes cluster](https://docs.gitlab.com/runner/install/kubernetes.html). -1. **Domain Name:** Knative provides its own load balancer using Istio, and an - external IP address or hostname for all the applications served by Knative. Enter a - wildcard domain to serve your applications. Configure your DNS server to use the - external IP address or hostname for that domain. -1. **`.gitlab-ci.yml`:** GitLab uses [Kaniko](https://github.com/GoogleContainerTools/kaniko) - to build the application. We also use [GitLab Knative tool](https://gitlab.com/gitlab-org/gitlabktl) - CLI to simplify the deployment of services and functions to Knative. -1. **`serverless.yml`** (for [functions only](#deploying-functions)): When using serverless to deploy functions, the `serverless.yml` file - contains the information for all the functions being hosted in the repository as well as a reference - to the runtime being used. -1. **`Dockerfile`** (for [applications only](#deploying-serverless-applications)): Knative requires a - `Dockerfile` in order to build your applications. It should be included at the root of your - project's repository and expose port `8080`. `Dockerfile` is not require if you plan to build serverless functions - using our [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes). -1. **Prometheus** (optional): The [Prometheus cluster integration](../../../clusters/integrations.md#prometheus-cluster-integration) - allows you to monitor the scale and traffic of your serverless function/application. -1. **Logging** (optional): Configuring logging allows you to view and search request logs for your serverless function/application. - See [Configuring logging](#configuring-logging) for more information. - -## Configuring Knative - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/58941) in GitLab 12.0. - -1. Follow the steps to - [add a Kubernetes - cluster](../add_remove_clusters.md). - -1. Ensure GitLab can manage Knative: - - For a non-GitLab managed cluster, ensure that the service account for the token - provided can manage resources in the `serving.knative.dev` API group. - - For a GitLab managed cluster, if you added the cluster in [GitLab 12.1 or later](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30235), - then GitLab already has the required access and you can proceed to the next step. - - Otherwise, you need to manually grant the GitLab service account the ability to manage - resources in the `serving.knative.dev` API group. Since every GitLab service account - has the `edit` cluster role, the simplest way to do this is with an - [aggregated ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#aggregated-clusterroles) - adding rules to the default `edit` cluster role: First, save the following YAML as - `knative-serving-only-role.yaml`: - - ```yaml - apiVersion: rbac.authorization.k8s.io/v1 - kind: ClusterRole - metadata: - name: knative-serving-only-role - labels: - rbac.authorization.k8s.io/aggregate-to-edit: "true" - rules: - - apiGroups: - - serving.knative.dev - resources: - - configurations - - configurationgenerations - - routes - - revisions - - revisionuids - - autoscalers - - services - verbs: - - get - - list - - create - - update - - delete - - patch - - watch - ``` - - Then run the following command: - - ```shell - kubectl apply -f knative-serving-only-role.yaml - ``` - - Alternatively, permissions can be granted on a per-service account basis - using `Role`s and `RoleBinding`s (see the [Kubernetes RBAC - documentation](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) - for more information). - -1. Follow the steps to deploy [functions](#deploying-functions) - or [serverless applications](#deploying-serverless-applications) onto your - cluster. - -1. **Optional:** For invocation metrics to show in GitLab, additional Istio metrics need to be configured in your cluster. For example, with Knative v0.9.0, you can use [this manifest](https://gitlab.com/gitlab-org/charts/knative/-/raw/v0.10.0/vendor/istio-metrics.yml). - -## Supported runtimes - -Serverless functions for GitLab can be run using: - -- [GitLab-managed](#gitlab-managed-runtimes) runtimes. -- [OpenFaaS](#openfaas-runtimes) runtimes. - -If a runtime is not available for the required programming language, consider deploying a -[serverless application](#deploying-serverless-applications). - -### GitLab-managed runtimes - -The following GitLab-managed [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes) -are available: - -- `go` (proof of concept) -- `nodejs` -- `ruby` - -You must provide a `Dockerfile` to run serverless functions if no runtime is specified. - -### OpenFaaS runtimes - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29253) in GitLab 12.5. - -[OpenFaaS classic runtimes](https://github.com/openfaas/templates#templates-in-store) can be used with GitLab serverless. - -OpenFaas runtimes are available for the following languages: - -- C# -- Go -- NodeJS -- PHP -- Python -- Ruby - -Runtimes are specified using the pattern: `openfaas/classic/<template_name>`. The following -example shows how to define a function in `serverless.yml` using an OpenFaaS runtime: - -```yaml -hello: - source: ./hello - runtime: openfaas/classic/ruby - description: "Ruby function using OpenFaaS classic runtime" -``` - -`handler` is not needed for OpenFaaS functions. The location of the handler is defined -by the conventions of the runtime. - -See the [`ruby-openfaas-function`](https://gitlab.com/knative-examples/ruby-openfaas-function) -project for an example of a function using an OpenFaaS runtime. - -## Deploying functions - -> Introduced in GitLab 11.6. - -You can find and import all the files referenced in this doc in the -**[functions example project](https://gitlab.com/knative-examples/functions)**. - -Follow these steps to deploy a function using the Node.js runtime to your -Knative instance (you can skip these steps if you've cloned the example -project): - -1. Create a directory to house the function. In this example we will - create a directory called `echo` at the root of the project. - -1. Create the file to contain the function code. In this example, our file is called `echo.js` and is located inside the `echo` directory. If your project is: - - Public, continue to the next step. - - Private, you must [create a GitLab deploy token](../../deploy_tokens/index.md#creating-a-deploy-token) with `gitlab-deploy-token` as the name and the `read_registry` scope. - -1. `.gitlab-ci.yml`: this defines a pipeline used to deploy your functions. - It must be included at the root of your repository: - - ```yaml - include: - - template: Serverless.gitlab-ci.yml - - functions:build: - extends: .serverless:build:functions - environment: production - - functions:deploy: - extends: .serverless:deploy:functions - environment: production - ``` - - This `.gitlab-ci.yml` creates jobs that invoke some predefined commands to - build and deploy your functions to your cluster. - - `Serverless.gitlab-ci.yml` is a template that allows customization. - You can either import it with `include` parameter and use `extends` to - customize your jobs, or you can inline the entire template by choosing it - from **Apply a template** dropdown when editing the `.gitlab-ci.yml` file through - the user interface. - -1. `serverless.yml`: this file contains the metadata for your functions, - such as name, runtime, and environment. - - It must be included at the root of your repository. - The following is a sample `echo` function which shows the required structure - for the file. - - You can find the relevant files for this project in the [functions example project](https://gitlab.com/knative-examples/functions). - - ```yaml - service: functions - description: "GitLab Serverless functions using Knative" - - provider: - name: triggermesh - envs: - FOO: value - secrets: - - my-secrets - - functions: - echo-js: - handler: echo-js - source: ./echo-js - runtime: gitlab/runtimes/nodejs - description: "node.js runtime function" - envs: - MY_FUNCTION: echo-js - secrets: - - my-secrets - ``` - -Explanation of the fields used above: - -### `service` - -| Parameter | Description | -|-----------|-------------| -| `service` | Name for the Knative service which serves the function. | -| `description` | A short description of the `service`. | - -### `provider` - -| Parameter | Description | -|-----------|-------------| -| `name` | Indicates which provider is used to execute the `serverless.yml` file. In this case, the TriggerMesh middleware. | -| `envs` | Includes the environment variables to be passed as part of function execution for **all** functions in the file, where `FOO` is the variable name and `BAR` are the variable contents. You may replace this with your own variables. | -| `secrets` | Includes the contents of the Kubernetes secret as environment variables accessible to be passed as part of function execution for **all** functions in the file. The secrets are expected in `INI` format. | - -### `functions` - -In the `serverless.yml` example above, the function name is `echo` and the -subsequent lines contain the function attributes. - -| Parameter | Description | -|-----------|-------------| -| `handler` | The function's name. | -| `source` | Directory with sources of a functions. | -| `runtime` (optional)| The runtime to be used to execute the function. This can be a runtime alias (see [Runtime aliases](#runtime-aliases)), or it can be a full URL to a custom runtime repository. When the runtime is not specified, we assume that `Dockerfile` is present in the function directory specified by `source`. | -| `description` | A short description of the function. | -| `envs` | Sets an environment variable for the specific function only. | -| `secrets` | Includes the contents of the Kubernetes secret as environment variables accessible to be passed as part of function execution for the specific function only. The secrets are expected in `INI` format. | - -### Deployment - -#### Runtime aliases - -The optional `runtime` parameter can refer to one of the following runtime aliases (also see [Supported runtimes](#supported-runtimes)): - -| Runtime alias | Maintained by | -|-------------|---------------| -| `gitlab/runtimes/go` | GitLab | -| `gitlab/runtimes/nodejs` | GitLab | -| `gitlab/runtimes/ruby` | GitLab | -| `openfaas/classic/csharp` | OpenFaaS | -| `openfaas/classic/go` | OpenFaaS | -| `openfaas/classic/node` | OpenFaaS | -| `openfaas/classic/php7` | OpenFaaS | -| `openfaas/classic/python` | OpenFaaS | -| `openfaas/classic/python3` | OpenFaaS | -| `openfaas/classic/ruby` | OpenFaaS | - -After the `.gitlab-ci.yml` template has been added and the `serverless.yml` file -has been created, pushing a commit to your project results in a CI pipeline -being executed which deploys each function as a Knative service. After the -deploy stage has finished, additional details for the function display -under **Infrastructure > Serverless platform**. - - - -This page contains all functions available for the project, the description for -accessing the function, and, if available, the function's runtime information. -The details are derived from the Knative installation inside each of the project's -Kubernetes cluster. Click on each function to obtain detailed scale and invocation data. - -The function details can be retrieved directly from Knative on the cluster: - -```shell -kubectl -n "$KUBE_NAMESPACE" get services.serving.knative.dev -``` - -The sample function can now be triggered from any HTTP client using a simple `POST` call: - - 1. Using curl (replace the URL on the last line with the URL of your application): - - ```shell - curl \ - --header "Content-Type: application/json" \ - --request POST \ - --data '{"GitLab":"FaaS"}' \ - "http://functions-echo.functions-1.functions.example.com/" - ``` - - 1. Using a web-based tool (such as Postman or Restlet) - -  - -### Secrets - -To access your Kubernetes secrets from within your function, the secrets should be created under the namespace of your serverless deployment and specified in your `serverless.yml` file as above. -You can create secrets in several ways. The following sections show some examples. - -#### CLI example - -```shell -kubectl create secret generic my-secrets -n "$KUBE_NAMESPACE" --from-literal MY_SECRET=imverysecure -``` - -#### Part of deployment job - -You can extend your `.gitlab-ci.yml` to create the secrets during deployment using the [CI/CD variables](../../../../ci/variables/index.md) -stored securely under your GitLab project. - -```yaml -deploy:function: - stage: deploy - environment: production - extends: .serverless:deploy:functions - before_script: - - kubectl create secret generic my-secret - --from-literal MY_SECRET="$GITLAB_SECRET_VARIABLE" - --namespace "$KUBE_NAMESPACE" - --dry-run -o yaml | kubectl apply -f - -``` - -### Running functions locally - -Running a function locally is a good way to quickly verify behavior during development. - -Running functions locally requires: - -- Go 1.12 or newer installed. -- Docker Engine installed and running. -- `gitlabktl` installed using the Go package manager: - - ```shell - GO111MODULE=on go get gitlab.com/gitlab-org/gitlabktl - ``` - -To run a function locally: - -1. Navigate to the root of your GitLab serverless project. -1. Build your function into a Docker image: - - ```shell - gitlabktl serverless build - ``` - -1. Run your function in Docker: - - ```shell - docker run -itp 8080:8080 <your_function_name> - ``` - -1. Invoke your function: - - ```shell - curl "http://localhost:8080" - ``` - -## Deploying Serverless applications - -> Introduced in GitLab 11.5. - -Serverless applications are an alternative to [serverless functions](#deploying-functions). -They're useful in scenarios where an existing runtime does not meet the needs of -an application, such as one written in a language that has no runtime available. -Note though that serverless applications should be stateless. - -You can reference and import the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) -to get started. Add the following `.gitlab-ci.yml` to the root of your repository -(you may skip this step if you've previously cloned the previously mentioned, -sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app)): - -```yaml -include: - - template: Serverless.gitlab-ci.yml - -build: - extends: .serverless:build:image - -deploy: - extends: .serverless:deploy:image -``` - -`Serverless.gitlab-ci.yml` is a template that allows customization. -You can either import it with `include` parameter and use `extends` to -customize your jobs, or you can inline the entire template by choosing it -from **Apply a template** dropdown when editing the `.gitlab-ci.yml` file through -the user interface. - -A `serverless.yml` file is not required when deploying serverless applications. - -### Deploy the application with Knative - -With all the pieces in place, the next time a CI pipeline runs the Knative application deploys. Navigate to -**CI/CD > Pipelines** and click the most recent pipeline. - -### Function details - -Go to the **Infrastructure > Serverless platform** page to see the final URL of your functions. - - - -### Invocation metrics - -On the same page as above, click on one of the function -rows to bring up the function details page. - - - -The pod count gives you the number of pods running the serverless function instances on a given cluster. - -For the Knative function invocations to appear, the -[Prometheus cluster integration must be enabled](../../../clusters/integrations.md#prometheus-cluster-integration). - -Once Prometheus is enabled, a message may appear indicating that the metrics data _is -loading or is not available at this time._ It appears upon the first access of the -page, but should go away after a few seconds. If the message does not disappear, then it -is possible that GitLab is unable to connect to the Prometheus instance running on the -cluster. - -## Configuring logging - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33330) in GitLab 12.5. - -### Prerequisites - -- A GitLab-managed cluster. -- `kubectl` installed and working. - -Running `kubectl` commands on your cluster requires setting up access to the -cluster first. For clusters created on: - -- GKE, see [GKE Cluster Access](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl) -- Other platforms, see [Install and Set Up kubectl](https://kubernetes.io/docs/tasks/tools/). - -### Enable request log template - -Run the following command to enable request logs: - -```shell -kubectl edit cm -n knative-serving config-observability -``` - -Copy the `logging.request-log-template` from the `data._example` field to the data field one level up in the hierarchy. - -### Enable request logs - -Run the following commands to install Elasticsearch, Kibana, and Filebeat into a `kube-logging` namespace and configure all nodes to forward logs using Filebeat: - -```shell -kubectl apply -f https://gitlab.com/gitlab-org/serverless/configurations/knative/raw/v0.7.0/kube-logging-filebeat.yaml -kubectl label nodes --all beta.kubernetes.io/filebeat-ready="true" -``` - -### Viewing request logs - -To view request logs: - -1. Run `kubectl proxy`. -1. Navigate to [Kibana UI](http://localhost:8001/api/v1/namespaces/kube-logging/services/kibana/proxy/app/kibana). - -Or: - -1. Open the [Kibana UI](http://localhost:8001/api/v1/namespaces/kube-logging/services/kibana/proxy/app/kibana). -1. Click on **Discover**, then select `filebeat-*` from the dropdown on the left. -1. Enter `kubernetes.container.name:"queue-proxy" AND message:/httpRequest/` into the search box. - -## Enabling TLS for Knative services - -By default, a GitLab serverless deployment is served over `http`. To serve -over `https`, you must manually obtain and install TLS certificates. - -The simplest way to accomplish this is to use Certbot to -[manually obtain Let's Encrypt certificates](https://knative.dev/docs/serving/using-a-tls-cert/#using-certbot-to-manually-obtain-let-s-encrypt-certificates). -Certbot is a free, open source software tool for automatically using Let's Encrypt -certificates on manually-administered websites to enable HTTPS. - -The following instructions relate to installing and running Certbot on a Linux -server that has Python 3 installed, and may not work on other operating systems -or with other versions of Python. - -1. Install Certbot by running the - [`certbot-auto` wrapper script](https://eff-certbot.readthedocs.io/install.html#certbot-auto). - On the command line of your server, run the following commands: - - ```shell - wget https://dl.eff.org/certbot-auto - sudo mv certbot-auto /usr/local/bin/certbot-auto - sudo chown root /usr/local/bin/certbot-auto - sudo chmod 0755 /usr/local/bin/certbot-auto - /usr/local/bin/certbot-auto --help - ``` - - To check the integrity of the `certbot-auto` script, run: - - ```shell - wget -N https://dl.eff.org/certbot-auto.asc - gpg2 --keyserver ipv4.pool.sks-keyservers.net --recv-key A2CFB51FA275A7286234E7B24D17C995CD9775F2 - gpg2 --trusted-key 4D17C995CD9775F2 --verify certbot-auto.asc /usr/local/bin/certbot-auto - ``` - - The output of the last command should look something like: - - ```shell - gpg: Signature made Mon 10 Jun 2019 06:24:40 PM EDT - gpg: using RSA key A2CFB51FA275A7286234E7B24D17C995CD9775F2 - gpg: key 4D17C995CD9775F2 marked as ultimately trusted - gpg: checking the trustdb - gpg: marginals needed: 3 completes needed: 1 trust model: pgp - gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u - gpg: next trustdb check due at 2027-11-22 - gpg: Good signature from "Let's Encrypt Client Team <letsencrypt-client@eff.org>" [ultimate] - ``` - -1. Run the following command to use Certbot to request a certificate - using DNS challenge during authorization: - - ```shell - /usr/local/bin/certbot-auto certonly --manual --preferred-challenges dns -d '*.<namespace>.example.com' - ``` - - Where `<namespace>` is the namespace created by GitLab for your serverless project (composed of `<project_name>-<project_id>-<environment>`) and - `example.com` is the domain being used for your project. If you are unsure what the namespace of your project is, navigate - to the **Infrastructure > Serverless platform** page of your project and inspect - the endpoint provided for your function/app. - -  - - In the above image, the namespace for the project is `node-function-11909507` and the domain is `knative.info`, thus - certificate request line would look like this: - - ```shell - ./certbot-auto certonly --manual --preferred-challenges dns -d '*.node-function-11909507.knative.info' - ``` - - The Certbot tool walks you through the steps of validating that you own each domain that you specify by creating TXT records in those domains. - After this process is complete, the output should look something like this: - - ```shell - IMPORTANT NOTES: - - Congratulations! Your certificate and chain have been saved at: - /etc/letsencrypt/live/namespace.example.com/fullchain.pem - Your key file has been saved at: - /etc/letsencrypt/live/namespace.example/privkey.pem - Your cert will expire on 2019-09-19. To obtain a new or tweaked - version of this certificate in the future, simply run certbot-auto - again. To non-interactively renew *all* of your certificates, run - "certbot-auto renew" - -----BEGIN PRIVATE KEY----- - - Your account credentials have been saved in your Certbot - configuration directory at /etc/letsencrypt. You should make a - secure backup of this folder now. This configuration directory will - also contain certificates and private keys obtained by Certbot so - making regular backups of this folder is ideal. - ``` - -1. Create certificate and private key files. Using the contents of the files - returned by Certbot, create two files in order to create the - Kubernetes secret: - - Run the following command to see the contents of `fullchain.pem`: - - ```shell - sudo cat /etc/letsencrypt/live/node-function-11909507.knative.info/fullchain.pem - ``` - - Output should look like this: - - ```shell - -----BEGIN CERTIFICATE----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b4ag== - -----END CERTIFICATE----- - -----BEGIN CERTIFICATE----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - K2fcb195768c39e9a94cec2c2e30Qg== - -----END CERTIFICATE----- - ``` - - Create a file with the name `cert.pem` with the contents of the entire output. - - Once `cert.pem` is created, run the following command to see the contents of `privkey.pem`: - - ```shell - sudo cat /etc/letsencrypt/live/namespace.example/privkey.pem - ``` - - Output should look like this: - - ```shell - -----BEGIN PRIVATE KEY----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - -----BEGIN CERTIFICATE----- - fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 4f294d1eaca42b8692017b4262== - -----END PRIVATE KEY----- - ``` - - Create a new file with the name `cert.pk` with the contents of the entire output. - -1. Create a Kubernetes secret to hold your TLS certificate, `cert.pem`, and - the private key `cert.pk`: - - NOTE: - Running `kubectl` commands on your cluster requires setting up access to the cluster first. - For clusters created on GKE, see - [GKE Cluster Access](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl). - For other platforms, [install `kubectl`](https://kubernetes.io/docs/tasks/tools/). - - ```shell - kubectl create --namespace istio-system secret tls istio-ingressgateway-certs \ - --key cert.pk \ - --cert cert.pem - ``` - - Where `cert.pem` and `cert.pk` are your certificate and private key files. Note that the `istio-ingressgateway-certs` secret name is required. - -1. Configure Knative to use the new secret that you created for HTTPS - connections. Run the - following command to open the Knative shared `gateway` in edit mode: - - ```shell - kubectl edit gateway knative-ingress-gateway --namespace knative-serving - ``` - - Update the gateway to include the following `tls:` section and configuration: - - ```shell - tls: - mode: SIMPLE - privateKey: /etc/istio/ingressgateway-certs/tls.key - serverCertificate: /etc/istio/ingressgateway-certs/tls.crt - ``` - - Example: - - ```shell - apiVersion: networking.istio.io/v1alpha3 - kind: Gateway - metadata: - # ... skipped ... - spec: - selector: - istio: ingressgateway - servers: - - hosts: - - "*" - port: - name: http - number: 80 - protocol: HTTP - - hosts: - - "*" - port: - name: https - number: 443 - protocol: HTTPS - tls: - mode: SIMPLE - privateKey: /etc/istio/ingressgateway-certs/tls.key - serverCertificate: /etc/istio/ingressgateway-certs/tls.crt - ``` - - After your changes are running on your Knative cluster, you can begin using the HTTPS protocol for secure access your deployed Knative services. - In the event a mistake is made during this process and you need to update the cert, you must edit the gateway `knative-ingress-gateway` - to switch back to `PASSTHROUGH` mode. Once corrections are made, edit the file again so the gateway uses the new certificates. - -## Using an older version of `gitlabktl` - -There may be situations where you want to run an older version of `gitlabktl`. This -requires setting an older version of the `gitlabktl` image in the `.gitlab-ci.yml` file. - -To set an older version, add `image:` to the `functions:deploy` block. For example: - -```yaml -functions:deploy: - extends: .serverless:deploy:functions - environment: production - image: registry.gitlab.com/gitlab-org/gitlabktl:0.5.0 -``` - -Different versions are available by changing the version tag at the end of the registry URL in the -format `registry.gitlab.com/gitlab-org/gitlabktl:<version>`. - -For a full inventory of available `gitlabktl` versions, see the `gitlabktl` project's -[container registry](https://gitlab.com/gitlab-org/gitlabktl/container_registry). +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86267) in GitLab 15.0. diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 567c15c7d5f..42c1b8b0a62 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -16,12 +16,16 @@ type: howto, reference > This is [fixed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60525) in > GitLab 13.12. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. [An epic exists](https://gitlab.com/groups/gitlab-org/-/epics/2493) to add this functionality to the [agent](../index.md). +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. + GitLab deploy boards offer a consolidated view of the current health and status of each CI [environment](../../ci/environments/index.md) running on [Kubernetes](https://kubernetes.io), displaying the status of the pods in the deployment. Developers and other teammates can view the diff --git a/doc/user/project/import/cvs.md b/doc/user/project/import/cvs.md index 8bb716d8122..17d717bdc61 100644 --- a/doc/user/project/import/cvs.md +++ b/doc/user/project/import/cvs.md @@ -24,8 +24,8 @@ The following list illustrates the main differences between CVS and Git: whole, or they fail without any changes. In CVS, commits (and other operations) are not atomic. If an operation on the repository is interrupted in the middle, the repository can be left in an inconsistent state. -- **Storage method.** Changes in CVS are per file (changeset), while in Git - a committed file(s) is stored in its entirety (snapshot). That means it's +- **Storage method.** Changes in CVS are per file (changeset), while in Git, + committed files are stored in their entirety (snapshot). This means it is very easy in Git to revert or undo a whole change. - **Revision IDs.** The fact that in CVS changes are per files, the revision ID is depicted by version numbers, for example `1.4` reflects how many times a diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 329d91916e8..a190edb179d 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -7,66 +7,56 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Import your project from GitHub to GitLab **(FREE)** -Using the importer, you can import your GitHub repositories to GitLab.com or to -your self-managed GitLab instance. - -The following aspects of a project are imported: - -- Repository description (GitLab.com & 7.7+) -- Git repository data (GitLab.com & 7.7+) -- Issues (GitLab.com & 7.7+) -- Pull requests (GitLab.com & 8.4+) -- Wiki pages (GitLab.com & 8.4+) -- Milestones (GitLab.com & 8.7+) -- Labels (GitLab.com & 8.7+) -- Release note descriptions (GitLab.com & 8.12+) -- Pull request review comments (GitLab.com & 10.2+) -- Pull request reviews (GitLab.com & 13.7+) -- Pull request "merged by" information (GitLab.com & 13.7+) -- Regular issue and pull request comments -- [Git Large File Storage (LFS) Objects](../../../topics/git/lfs/index.md) -- Pull request comments replies in discussions ([GitLab.com & 14.5+](https://gitlab.com/gitlab-org/gitlab/-/issues/336596)) -- Diff Notes suggestions ([GitLab.com & 14.7+](https://gitlab.com/gitlab-org/gitlab/-/issues/340624)) - -References to pull requests and issues are preserved (GitLab.com & 8.7+), and -each imported repository maintains visibility level unless that [visibility -level is restricted](../../public_access.md#restrict-use-of-public-or-internal-projects), -in which case it defaults to the default project visibility. - -The namespace is a user or group in GitLab, such as `gitlab.com/janedoe` or -`gitlab.com/customer-success`. You can do some bulk actions to move projects to -different namespaces in the rails console. - -This process does not migrate or import any types of groups or organizations -from GitHub to GitLab. - -## Use cases - -The steps you take depend on whether you are importing from GitHub.com or -GitHub Enterprise. The steps also depend on whether you are importing to GitLab.com or -self-managed GitLab instance. - -- If you're importing to GitLab.com, you can alternatively import GitHub repositories - using a [personal access token](#use-a-github-token). We do not recommend - this method, as it does not associate all user activity (such as issues and - pull requests) with matching GitLab users. -- If you're importing to a self-managed GitLab instance, you can alternatively use the - [GitHub Rake task](../../../administration/raketasks/github_import.md) to import - projects without the constraints of a [Sidekiq](../../../development/sidekiq_style_guide.md) worker. -- If you're importing from GitHub Enterprise to your self-managed GitLab instance: - - You must first enable [GitHub integration](../../../integration/github.md). - - To import projects from GitHub Enterprise to GitLab.com, use the [Import API](../../../api/import.md). - - If GitLab is behind a HTTP/HTTPS proxy you must populate the [allowlist for local requests](../../../security/webhooks.md#allowlist-for-local-requests) - with `github.com` and `api.github.com` to solve the hostname. For more information, read the issue - [Importing a GitHub project requires DNS resolution even when behind a proxy](https://gitlab.com/gitlab-org/gitlab/-/issues/37941) -- If you're importing from GitHub.com to your self-managed GitLab instance, - setting up GitHub integration is not required. You can use the [Import API](../../../api/import.md). - -## How it works +You can import your GitHub repositories: + +- From either GitHub.com or GitHub Enterprise. +- To either GitLab.com or a self-managed GitLab instance. + +This process does not migrate or import any types of groups or organizations from GitHub to GitLab. + +The namespace is a user or group in GitLab, such as `gitlab.com/sidney-jones` or +`gitlab.com/customer-success`. You can use bulk actions in the rails console to move projects to +different namespaces. + +If you are importing to a self-managed GitLab instance, you can use the +[GitHub Rake task](../../../administration/raketasks/github_import.md) instead. This allows you to import projects +without the constraints of a [Sidekiq](../../../development/sidekiq/index.md) worker. + +If you are importing from GitHub Enterprise to a self-managed GitLab instance: + +- You must first enable [GitHub integration](../../../integration/github.md). +- To import projects from GitHub Enterprise to GitLab.com, use the [Import API](../../../api/import.md). +- If GitLab is behind a HTTP/HTTPS proxy, you must populate the [allowlist for local requests](../../../security/webhooks.md#allowlist-for-local-requests) + with `github.com` and `api.github.com` to solve the hostname. For more information, read the issue + [Importing a GitHub project requires DNS resolution even when behind a proxy](https://gitlab.com/gitlab-org/gitlab/-/issues/37941). + +If you are importing from GitHub.com to a self-managed GitLab instance: + +- Setting up GitHub integration is not required. +- You can use the [Import API](../../../api/import.md). + +When importing projects: + +- If a user referenced in the project is not found in the GitLab database, the project creator is set as the author and + assignee. The project creator is usually the user that initiated the import process. A note on the issue mentioning the + original GitHub author is added. +- The importer creates any new namespaces (or groups) if they do not exist, or, if the namespace is taken, the + repository is imported under the namespace of the user who initiated the import process. The namespace or repository + name can also be edited, with the proper permissions. +- The importer also imports branches on forks of projects related to open pull requests. These branches are + imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to + a discrepancy in branches compared to those of the GitHub repository. + +For additional technical details, refer to the [GitHub Importer](../../../development/github_importer.md) +developer documentation. + +For an overview of the import process, see the video [Migrating from GitHub to GitLab](https://youtu.be/VYOXuOg9tQI). + +## Prerequisites When issues and pull requests are being imported, the importer attempts to find -their GitHub authors and assignees in the database of the GitLab instance (note -that pull requests are called "merge requests" in GitLab). +their GitHub authors and assignees in the database of the GitLab instance. Pull requests are called _merge requests_ in +GitLab. For this association to succeed, each GitHub author and assignee in the repository must meet one of the following conditions prior to the import: @@ -75,32 +65,9 @@ must meet one of the following conditions prior to the import: - Have a GitHub account with a [public-facing email address](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-github-user-account/managing-email-preferences/setting-your-commit-email-address) that matches their GitLab account's email address. - NOTE: - GitLab content imports that use GitHub accounts require that the GitHub - public-facing email address is populated so that all comments and - contributions are properly mapped to the same user in GitLab. GitHub - Enterprise (on premise) does not require this field to be populated to use the - product, so you may have to add it on existing accounts for the imported - content to be properly mapped to the user in the new system. Refer to GitHub - documentation for instructions on how to add this email address. - -If a user referenced in the project is not found in the GitLab database, the project creator (typically the user -that initiated the import process) is set as the author/assignee, but a note on the issue mentioning the original -GitHub author is added. - -The importer creates any new namespaces (groups) if they do not exist, or, if the namespace is taken, the -repository is imported under the namespace of the user who initiated the import process. The namespace/repository -name can also be edited, with the proper permissions. - -The importer will also import branches on forks of projects related to open pull requests. These branches will be -imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to -a discrepancy in branches compared to those of the GitHub repository. - -For additional technical details, you can refer to the -[GitHub Importer](../../../development/github_importer.md "Working with the GitHub importer") -developer documentation. - -For an overview of the import process, see the video [Migrating from GitHub to GitLab](https://youtu.be/VYOXuOg9tQI). +GitLab content imports that use GitHub accounts require that the GitHub public-facing email address is populated. This means +all comments and contributions are properly mapped to the same user in GitLab. GitHub Enterprise does not require this +field to be populated so you may have to add it on existing accounts. ## Import your GitHub repository into GitLab @@ -108,10 +75,12 @@ For an overview of the import process, see the video [Migrating from GitHub to G Before you begin, ensure that any GitHub users who you want to map to GitLab users have either: -- A GitLab account that has logged in using the GitHub icon - \- or - +- A GitLab account that has logged in using the GitHub icon. - A GitLab account with an email address that matches the [publicly visible email address](https://docs.github.com/en/rest/reference/users#get-a-user) in the profile of the GitHub user +If you are importing to GitLab.com, you can alternatively import GitHub repositories using a [personal access token](#use-a-github-token). +We do not recommend this method, as it does not associate all user activity (such as issues and pull requests) with matching GitLab users. + User-matching attempts occur in that order, and if a user is not identified either way, the activity is associated with the user account that is performing the import. @@ -119,10 +88,10 @@ NOTE: If you are using a self-managed GitLab instance or if you are importing from GitHub Enterprise, this process requires that you have configured [GitHub integration](../../../integration/github.md). -1. From the top navigation bar, click **+** and select **New project**. +1. From the top navigation bar, select **+** and select **New project**. 1. Select the **Import project** tab and then select **GitHub**. 1. Select the first button to **List your GitHub repositories**. You are redirected to a page on [GitHub](https://github.com) to authorize the GitLab application. -1. Click **Authorize GitlabHQ**. You are redirected back to the GitLab Import page and all of your GitHub repositories are listed. +1. Select **Authorize GitlabHQ**. You are redirected back to the GitLab Import page and all of your GitHub repositories are listed. 1. Continue on to [selecting which repositories to import](#select-which-repositories-to-import). ### Use a GitHub token @@ -134,14 +103,13 @@ associate all user activity (such as issues and pull requests) with matching Git If you are an administrator of a self-managed GitLab instance or if you are importing from GitHub Enterprise, you cannot use a personal access token. The [GitHub integration method (above)](#use-the-github-integration) is recommended for all users. -Read more in the [How it works](#how-it-works) section. If you are not using the GitHub integration, you can still perform an authorization with GitHub to grant GitLab access your repositories: 1. Go to <https://github.com/settings/tokens/new> 1. Enter a token description. 1. Select the repository scope. -1. Click **Generate token**. +1. Select **Generate token**. 1. Copy the token hash. 1. Go back to GitLab and provide the token to the GitHub importer. 1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads your repositories' information. @@ -161,7 +129,7 @@ your GitHub repositories are listed. you can filter projects by name. If filter is applied, **Import all repositories** only imports matched repositories. 1. The **Status** column shows the import status of each repository. You can choose to leave the page open and it will update in real-time or you can return to it later. -1. Once a repository has been imported, click its GitLab path to open its GitLab URL. +1. Once a repository has been imported, select its GitLab path to open its GitLab URL.  @@ -197,6 +165,30 @@ Reducing the time spent in cloning a repository can be done by increasing networ performance (by using high performance SSDs, for example) of the disks that store the Git repositories (for your GitLab instance). Increasing the number of Sidekiq workers does *not* reduce the time spent cloning repositories. +## Imported data + +The following items of a project are imported: + +- Repository description. +- Git repository data. +- Issues. +- Pull requests. +- Wiki pages. +- Milestones. +- Labels. +- Release note descriptions. +- Pull request review comments. +- Regular issue and pull request comments. +- [Git Large File Storage (LFS) Objects](../../../topics/git/lfs/index.md). +- Pull request reviews (GitLab.com and GitLab 13.7 and later). +- Pull request "merged by" information (GitLab.com and GitLab 13.7 and later). +- Pull request comments replies in discussions ([GitLab.com and GitLab 14.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/336596)). +- Diff Notes suggestions ([GitLab.com and GitLab 14.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/340624)). + +References to pull requests and issues are preserved. Each imported repository maintains visibility level unless that +[visibility level is restricted](../../public_access.md#restrict-use-of-public-or-internal-projects), in which case it +defaults to the default project visibility. + ## Alternative way to import notes and diff notes When GitHub Importer runs on extremely large projects not all notes & diff notes can be imported due to GitHub API `issues_comments` & `pull_requests_comments` endpoints limitation. diff --git a/doc/user/project/import/img/gitlab_import_history_page_v14_10.png b/doc/user/project/import/img/gitlab_import_history_page_v14_10.png Binary files differindex c93b5ed2b27..812696a8faa 100644 --- a/doc/user/project/import/img/gitlab_import_history_page_v14_10.png +++ b/doc/user/project/import/img/gitlab_import_history_page_v14_10.png diff --git a/doc/user/project/integrations/bamboo.md b/doc/user/project/integrations/bamboo.md index bf343078634..22e6d45dd96 100644 --- a/doc/user/project/integrations/bamboo.md +++ b/doc/user/project/integrations/bamboo.md @@ -46,7 +46,7 @@ integration in GitLab. 1. Select **Atlassian Bamboo**. 1. Ensure the **Active** checkbox is selected. 1. Enter the base URL of your Bamboo server. For example, `https://bamboo.example.com`. -1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](overview.md#ssl-verification). +1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](index.md#manage-ssl-verification). 1. Enter the [build key](#identify-the-bamboo-build-plan-build-key) from your Bamboo build plan. 1. If necessary, enter a username and password for a Bamboo user that has @@ -71,7 +71,7 @@ Bamboo. For example, `https://bamboo.example.com/browse/PROJ-PLAN`. ### Builds not triggered If builds are not triggered, ensure you entered the right GitLab IP address in -Bamboo under **Trigger IP addresses**. Also check [service hook logs](overview.md#troubleshooting-integrations) for request failures. +Bamboo under **Trigger IP addresses**. Also check [service hook logs](index.md#troubleshooting-integrations) for request failures. ### Advanced Atlassian Bamboo features not available in GitLab UI diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index 4a9a8d62098..0f7ce182e1a 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -57,4 +57,4 @@ internal issue tracker, the internal issue is linked. ## Troubleshooting -To see recent service hook deliveries, check [service hook logs](overview.md#troubleshooting-integrations). +To see recent service hook deliveries, check [service hook logs](index.md#troubleshooting-integrations). diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index 2dae02dc093..dc56c2669f8 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w NOTE: The GitLab Slack application is only configurable for GitLab.com. It will **not** work for on-premises installations where you can configure the -[Slack slash commands](slack_slash_commands.md) service instead. We're planning +[Slack slash commands](slack_slash_commands.md) integration instead. We're planning to make this configurable for all GitLab installations, but there's no ETA - see [#28164](https://gitlab.com/gitlab-org/gitlab/-/issues/28164). @@ -31,17 +31,21 @@ Alternatively, you can configure the Slack application with a project's integration settings. Keep in mind that you must have the appropriate permissions for your Slack -team to be able to install a new application, read more in Slack's +workspace to be able to install a new application. Read more in Slack's documentation on [Adding an app to your workspace](https://slack.com/help/articles/202035138-Add-apps-to-your-Slack-workspace). -To enable the GitLab service for your Slack team: +To enable the GitLab integration for your Slack workspace: 1. Go to your project's **Settings > Integration > Slack application** (only visible on GitLab.com). -1. Select **Add to Slack**. +1. Select **Install Slack app**. +1. Select **Allow** on Slack's confirmation screen. That's all! You can now start using the Slack slash commands. +You can also select **Reinstall Slack app** to update the app in your Slack workspace +to the latest version. See the [Version history](#version-history) for details. + ## Create a project alias for Slack To create a project alias on GitLab.com for Slack integration: @@ -62,7 +66,7 @@ GitLab error: project or alias not found ## Usage -After confirming the installation, you, and everyone else in your Slack team, +After confirming the installation, you, and everyone else in your Slack workspace, can use all the [slash commands](../../../integration/slash_commands.md). When you perform your first slash command, you are asked to authorize your @@ -78,3 +82,11 @@ project, you would do: ```plaintext /gitlab gitlab-org/gitlab issue show 1001 ``` + +## Version history + +### 15.0+ + +In GitLab 15.0 the Slack app is updated to [Slack's new granular permissions app model](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3). + +There is no change in functionality. A reinstall is not required but recommended. diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index 9764c4d44a0..7af2e431157 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -6,26 +6,114 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Project integrations **(FREE)** -You can find the available integrations under your project's -**Settings > Integrations** page. You need to have at least -the Maintainer role on the project. +You can integrate your GitLab projects with other applications. Integrations are +like plugins, and give you the freedom to add +functionality to GitLab. -## Integrations +## View project integrations -Like plugins, integrations allow you to integrate GitLab with other applications, adding additional features. -For more information, read the -[overview of integrations](overview.md) or learn how to manage your integrations: +Prerequisites: -- *For GitLab 13.3 and later,* read [Project integration management](../../admin_area/settings/project_integration_management.md). -- *For GitLab 13.2 and earlier,* read [Integration Management](../../admin_area/settings/project_integration_management.md), - which replaced the deprecated Service Templates [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268032) - in GitLab 14.0. +- You must have at least the Maintainer role for the project. -## Project webhooks +To view the available integrations for your project: -Project webhooks allow you to trigger a URL if for example new code is pushed or -a new issue is created. You can configure webhooks to listen for specific events -like pushes, issues or merge requests. GitLab sends a POST request with data -to the webhook URL. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Integrations**. + +You can also view and manage integration settings across [all projects in an instance or group](../../admin_area/settings/project_integration_management.md). +For a single project, you can choose to inherit the instance or group configuration, +or provide custom settings. + +NOTE: +Instance and group-based integration management replaces service templates, which +were [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268032) in GitLab 14.0. + +## Manage SSL verification + +By default, the SSL certificate for outgoing HTTP requests is verified based on +an internal list of Certificate Authorities. This means the certificate cannot +be self-signed. + +You can turn off SSL verification in the configuration settings for [webhooks](webhooks.md#configure-a-webhook-in-gitlab) +and some integrations. + +## Available integrations + +You can configure the following integrations. + +| Integration | Description | Integration hooks | +|-----------------------------------------------------------------------------|-----------------------------------------------------------------------|------------------------| +| [Asana](asana.md) | Add commit messages as comments to Asana tasks. | **{dotted-circle}** No | +| Assembla | Manage projects. | **{dotted-circle}** No | +| [Atlassian Bamboo CI](bamboo.md) | Run CI/CD pipelines with Atlassian Bamboo. | **{check-circle}** Yes | +| [Bugzilla](bugzilla.md) | Use Bugzilla as the issue tracker. | **{dotted-circle}** No | +| Buildkite | Run CI/CD pipelines with Buildkite. | **{check-circle}** Yes | +| Campfire | Connect to chat. | **{dotted-circle}** No | +| [Confluence Workspace](../../../api/integrations.md#confluence-integration) | Use Confluence Cloud Workspace as an internal wiki. | **{dotted-circle}** No | +| [Custom issue tracker](custom_issue_tracker.md) | Use a custom issue tracker. | **{dotted-circle}** No | +| [Datadog](../../../integration/datadog.md) | Trace your GitLab pipelines with Datadog. | **{check-circle}** Yes | +| [Discord Notifications](discord_notifications.md) | Send notifications about project events to a Discord channel. | **{dotted-circle}** No | +| Drone CI | Run CI/CD pipelines with Drone. | **{check-circle}** Yes | +| [Emails on push](emails_on_push.md) | Send commits and diff of each push by email. | **{dotted-circle}** No | +| [EWM](ewm.md) | Use IBM Engineering Workflow Management as the issue tracker. | **{dotted-circle}** No | +| [External wiki](../wiki/index.md#link-an-external-wiki) | Link an external wiki. | **{dotted-circle}** No | +| [Flowdock](../../../api/integrations.md#flowdock) | Send notifications from GitLab to Flowdock flows. | **{dotted-circle}** No | +| [GitHub](github.md) | Obtain statuses for commits and pull requests. | **{dotted-circle}** No | +| [Google Chat](hangouts_chat.md) | Send notifications from your GitLab project to a room in Google Chat. | **{dotted-circle}** No | +| [Harbor](harbor.md) | Use Harbor as the container registry. | **{dotted-circle}** No | +| [irker (IRC gateway)](irker.md) | Send IRC messages. | **{dotted-circle}** No | +| [Jenkins](../../../integration/jenkins.md) | Run CI/CD pipelines with Jenkins. | **{check-circle}** Yes | +| JetBrains TeamCity CI | Run CI/CD pipelines with TeamCity. | **{check-circle}** Yes | +| [Jira](../../../integration/jira/index.md) | Use Jira as the issue tracker. | **{dotted-circle}** No | +| [Mattermost notifications](mattermost.md) | Send notifications about project events to Mattermost channels. | **{dotted-circle}** No | +| [Mattermost slash commands](mattermost_slash_commands.md) | Perform common tasks with slash commands. | **{dotted-circle}** No | +| [Microsoft Teams notifications](microsoft_teams.md) | Receive event notifications. | **{dotted-circle}** No | +| Packagist | Keep your PHP dependencies updated on Packagist. | **{check-circle}** Yes | +| [Pipelines emails](pipeline_status_emails.md) | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | +| [Pivotal Tracker](pivotal_tracker.md) | Add commit messages as comments to Pivotal Tracker stories. | **{dotted-circle}** No | +| [Prometheus](prometheus.md) | Monitor application metrics. | **{dotted-circle}** No | +| Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | +| [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | +| [Slack application](gitlab_slack_application.md) | Use Slack's official GitLab application. | **{dotted-circle}** No | +| [Slack notifications](slack.md) | Send notifications about project events to Slack. | **{dotted-circle}** No | +| [Slack slash commands](slack_slash_commands.md) | Enable slash commands in a workspace. | **{dotted-circle}** No | +| [Unify Circuit](unify_circuit.md) | Send notifications about project events to Unify Circuit. | **{dotted-circle}** No | +| [Webex Teams](webex_teams.md) | Receive events notifications. | **{dotted-circle}** No | +| [YouTrack](youtrack.md) | Use YouTrack as the issue tracker. | **{dotted-circle}** No | +| [ZenTao](zentao.md) | Use ZenTao as the issue tracker. | **{dotted-circle}** No | + +### Project webhooks + +You can configure a project webhook to listen for specific events +like pushes, issues, or merge requests. When the webhook is triggered, GitLab +sends a POST request with data to a specified webhook URL. Learn more [about webhooks](webhooks.md). + +## Push hooks limit + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17874) in GitLab 12.4. + +If a single push includes changes to more than three branches or tags, integrations +supported by `push_hooks` and `tag_push_hooks` events aren't executed. + +You can change the number of supported branches or tags by changing the +[`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). + +## Troubleshooting integrations + +Some integrations use hooks to integrate with external applications. To confirm which ones use integration hooks, see the [available integrations](#available-integrations). Learn more about [troubleshooting integration hooks](webhooks.md#troubleshoot-webhooks). + +### `Test Failed. Save Anyway` error + +Some integrations fail with an error `Test Failed. Save Anyway` when you set them +up on uninitialized repositories. This error occurs because the integration uses +push data to build the test payload, and there are no push events in the project. + +To resolve this error, initialize the repository by pushing a test file to the project +and set up the integration again. + +## Contribute to integrations + +To add a new integration, see the [Integrations development guide](../../../development/integrations/index.md). diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index 081780e6277..9625edcd8f9 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -1,117 +1,11 @@ --- -stage: Ecosystem -group: Integrations -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'index.md' +remove_date: '2022-07-20' --- -# Integrations **(FREE)** +This document was moved to [another location](index.md). -Integrations allow you to integrate GitLab with other applications. They -are a bit like plugins in that they allow a lot of freedom in adding -functionality to GitLab. - -## Accessing integrations - -To find the available integrations for your project: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Integrations**. - -There are more than 20 integrations to integrate with. Select the one that you -want to configure. - -## Integrations listing - -Click on the integration links to see further configuration instructions and details. - -| Integration | Description | Integration hooks | -| --------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------------------- | -| [Asana](asana.md) | Add commit messages as comments to Asana tasks. | **{dotted-circle}** No | -| Assembla | Manage projects. | **{dotted-circle}** No | -| [Atlassian Bamboo CI](bamboo.md) | Run CI/CD pipelines with Atlassian Bamboo. | **{check-circle}** Yes | -| [Bugzilla](bugzilla.md) | Use Bugzilla as the issue tracker. | **{dotted-circle}** No | -| Buildkite | Run CI/CD pipelines with Buildkite. | **{check-circle}** Yes | -| Campfire | Connect to chat. | **{dotted-circle}** No | -| [Confluence Workspace](../../../api/integrations.md#confluence-integration) | Replace the link to the internal wiki with a link to a Confluence Cloud Workspace. | **{dotted-circle}** No | -| [Custom issue tracker](custom_issue_tracker.md) | Use a custom issue tracker. | **{dotted-circle}** No | -| [Datadog](../../../integration/datadog.md) | Trace your GitLab pipelines with Datadog. | **{check-circle}** Yes | -| [Discord Notifications](discord_notifications.md) | Send notifications about project events to a Discord channel. | **{dotted-circle}** No | -| Drone CI | Run CI/CD pipelines with Drone. | **{check-circle}** Yes | -| [Emails on push](emails_on_push.md) | Send commits and diff of each push by email. | **{dotted-circle}** No | -| [EWM](ewm.md) | Use IBM Engineering Workflow Management as the issue tracker. | **{dotted-circle}** No | -| [External wiki](../wiki/index.md#link-an-external-wiki) | Link an external wiki. | **{dotted-circle}** No | -| [Flowdock](../../../api/integrations.md#flowdock) | Send notifications from GitLab to Flowdock flows. | **{dotted-circle}** No | -| [GitHub](github.md) | Obtain statuses for commits and pull requests. | **{dotted-circle}** No | -| [Google Chat](hangouts_chat.md) | Send notifications from your GitLab project to a room in Google Chat.| **{dotted-circle}** No | -| [Harbor](harbor.md) | Use Harbor as the container registry. | **{dotted-circle}** No | -| [irker (IRC gateway)](irker.md) | Send IRC messages. | **{dotted-circle}** No | -| [Jenkins](../../../integration/jenkins.md) | Run CI/CD pipelines with Jenkins. | **{check-circle}** Yes | -| JetBrains TeamCity CI | Run CI/CD pipelines with TeamCity. | **{check-circle}** Yes | -| [Jira](../../../integration/jira/index.md) | Use Jira as the issue tracker. | **{dotted-circle}** No | -| [Mattermost notifications](mattermost.md) | Send notifications about project events to Mattermost channels. | **{dotted-circle}** No | -| [Mattermost slash commands](mattermost_slash_commands.md) | Perform common tasks with slash commands. | **{dotted-circle}** No | -| [Microsoft Teams notifications](microsoft_teams.md) | Receive event notifications. | **{dotted-circle}** No | -| Packagist | Keep your PHP dependencies updated on Packagist. | **{check-circle}** Yes | -| [Pipelines emails](pipeline_status_emails.md) | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | -| [Pivotal Tracker](pivotal_tracker.md) | Add commit messages as comments to Pivotal Tracker stories. | **{dotted-circle}** No | -| [Prometheus](prometheus.md) | Monitor application metrics. | **{dotted-circle}** No | -| Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | -| [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | -| [Slack application](gitlab_slack_application.md) | Use Slack's official GitLab application. | **{dotted-circle}** No | -| [Slack notifications](slack.md) | Send notifications about project events to Slack. | **{dotted-circle}** No | -| [Slack slash commands](slack_slash_commands.md) | Enable slash commands in workspace. | **{dotted-circle}** No | -| [Unify Circuit](unify_circuit.md) | Send notifications about project events to Unify Circuit. | **{dotted-circle}** No | -| [Webex Teams](webex_teams.md) | Receive events notifications. | **{dotted-circle}** No | -| [YouTrack](youtrack.md) | Use YouTrack as the issue tracker. | **{dotted-circle}** No | -| [ZenTao](zentao.md) | Use ZenTao as the issue tracker. | **{dotted-circle}** No | - -## Push hooks limit - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17874) in GitLab 12.4. - -If a single push includes changes to more than three branches or tags, integrations -supported by `push_hooks` and `tag_push_hooks` events aren't executed. - -The number of branches or tags supported can be changed via -[`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). - -## Project integration management - -Project integration management lets you control integration settings across all projects -of an instance. On the project level, administrators you can choose whether to inherit the -instance configuration or provide custom settings. - -Read more about [Project integration management](../../admin_area/settings/project_integration_management.md). - -## SSL verification - -By default, the SSL certificate for outgoing HTTP requests is verified based on -an internal list of Certificate Authorities. This means the certificate cannot -be self-signed. - -You can turn off SSL verification in the configuration settings for [webhooks](webhooks.md#configure-a-webhook-in-gitlab) -and some integrations. - -## Troubleshooting integrations - -Some integrations use hooks for integration with external applications. To confirm which ones use integration hooks, see the [integrations listing](#integrations-listing) above. Learn more about [troubleshooting integration hooks](webhooks.md#troubleshoot-webhooks). - -### Uninitialized repositories - -Some integrations fail with an error `Test Failed. Save Anyway` when you attempt to set them up on -uninitialized repositories. Some integrations use push data to build the test payload, -and this error occurs when no push events exist in the project yet. - -To resolve this error, initialize the repository by pushing a test file to the project and set up -the integration again. - -## Contributing to integrations - -Because GitLab is open source we can ship with the code and tests for all -plugins. This allows the community to keep the plugins up to date so that they -always work in newer GitLab versions. - -For an overview of what integrations are available, please see the -[integrations source directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/app/models/integrations). - -Contributions are welcome! +<!-- This redirect file can be deleted after 2022-07-20. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index f4f5b3f545b..ac7d447961c 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -57,7 +57,7 @@ You can configure a webhook for a group or a project. The URL must be percent-encoded if it contains one or more special characters. 1. In **Secret token**, enter the [secret token](#validate-payloads-by-using-a-secret-token) to validate payloads. 1. In the **Trigger** section, select the [events](webhook_events.md) to trigger the webhook. -1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](overview.md#ssl-verification). +1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](index.md#manage-ssl-verification). 1. Select **Add webhook**. ## Configure your webhook receiver endpoint @@ -244,7 +244,7 @@ To view the table: - **Failed to connect** if it is misconfigured, and needs manual intervention to re-enable it. - **Fails to connect** if it is temporarily disabled and will retry later. - +  1. Select **Edit** for the webhook you want to view. diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 15130523da6..9d23a63b940 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -36,7 +36,7 @@ The second way is to locate the Confidentiality section in the sidebar and click | Turn off confidentiality | Turn on confidentiality | | :-----------: | :----------: | -|  |  | +|  |  | Every change from regular to confidential and vice versa, is indicated by a system note in the issue's comments. @@ -97,5 +97,5 @@ sees in the project's search results respectively. - [Merge requests for confidential issues](../merge_requests/confidential.md) - [Make an epic confidential](../../group/epics/manage_epics.md#make-an-epic-confidential) -- [Mark a comment as confidential](../../discussions/index.md#mark-a-comment-as-confidential) +- [Add an internal note](../../discussions/index.md#add-an-internal-note) - [Security practices for confidential merge requests](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) at GitLab diff --git a/doc/user/project/issues/csv_import.md b/doc/user/project/issues/csv_import.md index e4b8efd27ed..a3f6ee5e61e 100644 --- a/doc/user/project/issues/csv_import.md +++ b/doc/user/project/issues/csv_import.md @@ -6,8 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Importing issues from CSV **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/23532) in GitLab 11.7. - Issues can be imported to a project by uploading a CSV file with the columns `title` and `description`. Other columns are **not** imported. If you want to retain columns such as labels and milestones, consider the [Move Issue feature](managing_issues.md#move-an-issue). @@ -25,33 +23,29 @@ You must have at least the Developer role for a project to import issues. To import issues: -1. Navigate to a project's Issues list page. -1. If existing issues are present, click the import icon at the top right, next to the **Edit issues** button. -1. For a project without any issues, click the button labeled **Import CSV** in the middle of the page. -1. Select the file and click the **Import issues** button. +1. Go to your project's Issues list page. +1. Open the import feature, depending if the project has issues: + - Existing issues are present: Select the import icon at the top right, next to **Edit issues**. + - Project has no issues: Select **Import CSV** in the middle of the page. +1. Select the file you want to import, and then select **Import issues**. -The file is processed in the background and a notification email is sent -to you once the import is complete. +The file is processed in the background, and a notification email is sent +to you after the import is complete. ## CSV file format -When importing issues from a CSV file, it must be formatted in a certain way: +To import issues, GitLab requires CSV files have a specific format: -- **header row:** CSV files must include the following headers: -`title` and `description`. The case of the headers does not matter. -- **columns:** Data from columns beyond `title` and `description` are not imported. -- **separators:** The column separator is automatically detected from the header row. - Supported separator characters are: commas (`,`), semicolons (`;`), and tabs (`\t`). - The row separator can be either `CRLF` or `LF`. -- **double-quote character:** The double-quote (`"`) character is used to quote fields, - enabling the use of the column separator within a field (see the third line in the - sample CSV data below). To insert a double-quote (`"`) within a quoted - field, use two double-quote characters in succession (`""`). -- **data rows:** After the header row, succeeding rows must follow the same column - order. The issue title is required while the description is optional. +| Element | Format | +|------------------------|--------| +| header row | CSV files must include the following headers: `title` and `description`. The case of the headers does not matter. | +| columns | Data from columns beyond `title` and `description` are not imported. | +| separators | The column separator is detected from the header row. Supported separator characters are commas (`,`), semicolons (`;`), and tabs (`\t`). The row separator can be either `CRLF` or `LF`. | +| double-quote character | The double-quote (`"`) character is used to quote fields, enabling the use of the column separator in a field (see the third line in the sample CSV data below). To insert a double-quote (`"`) in a quoted field use two double-quote characters in succession (`""`). | +| data rows | After the header row, following rows must use the same column order. The issue title is required, but the description is optional. | -If you have special characters _within_ a field, (such as `\n` or `,`), -wrap the characters in double quotes. +If you have special characters in a field, (such as `\n` or `,`), surround the +characters with double quotes (`"`). Sample CSV data: @@ -64,6 +58,7 @@ Another Title,"A description, with a comma" ### File size -The limit depends on the configuration value of Max Attachment Size for the GitLab instance. +The limit depends on how your GitLab instance is hosted: -For GitLab.com, it is set to 10 MB. +- Self-managed: Set by the configuration value of `Max Attachment Size` for the GitLab instance. +- GitLab SaaS: On GitLab.com, it's set to 10 MB. diff --git a/doc/user/project/issues/img/confidential_issues_issue_page.png b/doc/user/project/issues/img/confidential_issues_issue_page.png Binary files differindex 0f5c774d258..b349149aa98 100644 --- a/doc/user/project/issues/img/confidential_issues_issue_page.png +++ b/doc/user/project/issues/img/confidential_issues_issue_page.png diff --git a/doc/user/project/issues/img/design_management_v14_10.png b/doc/user/project/issues/img/design_management_v14_10.png Binary files differindex a10be15eafd..133c0a52d6b 100644 --- a/doc/user/project/issues/img/design_management_v14_10.png +++ b/doc/user/project/issues/img/design_management_v14_10.png diff --git a/doc/user/project/issues/img/sidebar_confidential_issue.png b/doc/user/project/issues/img/sidebar_confidential_issue.png Binary files differindex a320f4dcfe5..0ef61c7f1b0 100644 --- a/doc/user/project/issues/img/sidebar_confidential_issue.png +++ b/doc/user/project/issues/img/sidebar_confidential_issue.png diff --git a/doc/user/project/issues/img/turn_off_confidentiality.png b/doc/user/project/issues/img/turn_off_confidentiality.png Binary files differdeleted file mode 100644 index 04a85933071..00000000000 --- a/doc/user/project/issues/img/turn_off_confidentiality.png +++ /dev/null diff --git a/doc/user/project/issues/img/turn_off_confidentiality_v15_0.png b/doc/user/project/issues/img/turn_off_confidentiality_v15_0.png Binary files differnew file mode 100644 index 00000000000..37cbe0f4fd9 --- /dev/null +++ b/doc/user/project/issues/img/turn_off_confidentiality_v15_0.png diff --git a/doc/user/project/issues/img/turn_on_confidentiality.png b/doc/user/project/issues/img/turn_on_confidentiality.png Binary files differdeleted file mode 100644 index fac360ca6dc..00000000000 --- a/doc/user/project/issues/img/turn_on_confidentiality.png +++ /dev/null diff --git a/doc/user/project/issues/img/turn_on_confidentiality_v15_0.png b/doc/user/project/issues/img/turn_on_confidentiality_v15_0.png Binary files differnew file mode 100644 index 00000000000..498867d1933 --- /dev/null +++ b/doc/user/project/issues/img/turn_on_confidentiality_v15_0.png diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 4508ef30ac5..7db66dd013b 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -225,6 +225,8 @@ When you're creating a new issue, you can complete the following fields: ## Edit an issue +> Reordering list items [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15260) in GitLab 15.0. + You can edit an issue's title and description. Prerequisites: @@ -237,6 +239,14 @@ To edit an issue: 1. Edit the available fields. 1. Select **Save changes**. +You can also reorder list items, which include bullet, numerical, and task list items. +To reorder list items: + +1. Hover over the list item row to make the drag icon visible. +1. Click and hold the drag icon. +1. Drag the row to the new position in the list. +1. Release the drag icon. + ### Bulk edit issues from a project > - Assigning epic [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210470) in GitLab 13.2. diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index ff4677eddde..28bd353d8cc 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -189,7 +189,7 @@ You can filter and sort members in a project. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. In the **Filter members** box, select `Membership` `=` `Inherited`. -1. Press Enter. +1. Press <kbd>Enter</kbd>.  @@ -198,7 +198,7 @@ You can filter and sort members in a project. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. In the **Filter members** box, select `Membership` `=` `Direct`. -1. Press Enter. +1. Press <kbd>Enter</kbd>.  diff --git a/doc/user/project/merge_requests/allow_collaboration.md b/doc/user/project/merge_requests/allow_collaboration.md index 5826ebcab49..d06c8182e22 100644 --- a/doc/user/project/merge_requests/allow_collaboration.md +++ b/doc/user/project/merge_requests/allow_collaboration.md @@ -60,10 +60,10 @@ In the following example: To change or add a commit to the contributor's merge request: -1. Open the merge request page, and select the **Overview** tab. -1. Scroll to the merge request widget, and select **Check out branch**. +1. Go to the merge request. +1. In the upper right corner, select **Code**, then select **Check out branch**. 1. In the modal window, select **Copy** (**{copy-to-clipboard}**). -1. In your terminal, navigate to your cloned version of the repository, and +1. In your terminal, go to your cloned version of the repository, and paste the commands. For example: ```shell diff --git a/doc/user/project/merge_requests/approvals/img/security_approvals_v15_0.png b/doc/user/project/merge_requests/approvals/img/security_approvals_v15_0.png Binary files differnew file mode 100644 index 00000000000..b28d216f180 --- /dev/null +++ b/doc/user/project/merge_requests/approvals/img/security_approvals_v15_0.png diff --git a/doc/user/project/merge_requests/approvals/index.md b/doc/user/project/merge_requests/approvals/index.md index e940426dc67..f0ab4d606ad 100644 --- a/doc/user/project/merge_requests/approvals/index.md +++ b/doc/user/project/merge_requests/approvals/index.md @@ -102,8 +102,8 @@ Without the approvals, the work cannot merge. Required approvals enable multiple - Use the [code owners of changed files](rules.md#code-owners-as-eligible-approvers), to determine who should review the work. - Require an [approval before merging code that causes test coverage to decline](../../../../ci/pipelines/settings.md#coverage-check-approval-rule) -- [Require approval from a security team](../../../application_security/index.md#security-approvals-in-merge-requests) - before merging code that could introduce a vulnerability. +- Users on GitLab Ultimate can also [require approval from a security team](../../../application_security/index.md#security-approvals-in-merge-requests) + before merging code that could introduce a vulnerability. ## Related topics diff --git a/doc/user/project/merge_requests/approvals/rules.md b/doc/user/project/merge_requests/approvals/rules.md index 01772e59127..17a42e1b540 100644 --- a/doc/user/project/merge_requests/approvals/rules.md +++ b/doc/user/project/merge_requests/approvals/rules.md @@ -242,3 +242,14 @@ the API. For more information about this validation error, read [issue 285129](https://gitlab.com/gitlab-org/gitlab/-/issues/285129). + +## Security Approvals **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357021) in GitLab 15.0. + +You can use [scan result policies](../../../application_security/policies/scan-result-policies.md#scan-result-policy-editor) to define security approvals based on the status of vulnerabilities in the merge request and the default branch. +Details for each security policy is shown in the Security Approvals section of your Merge Request configuration. + + + +These policies are both created and edited in the [security policy editor](../../../application_security/policies/index.md#policy-editor). diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index 0ede9310393..9c2b54888fb 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -119,16 +119,9 @@ when more changes are added to it: 1. Select the **Remove all approvals when commits are added to the source branch** checkbox. 1. Select **Save changes**. -Approvals aren't reset when a merge request is [rebased from the UI](../fast_forward_merge.md) +Approvals aren't reset when a merge request is [rebased from the UI](../methods/index.md#rebasing-in-semi-linear-merge-methods) However, approvals are reset if the target branch is changed. -## Security approvals in merge requests **(ULTIMATE)** - -You can require that a member of your security team approves a merge request if a -merge request could introduce a vulnerability. - -To learn more, see [Security approvals in merge requests](../../../application_security/index.md#security-approvals-in-merge-requests). - ## Code coverage check approvals You can require specific approvals if a merge request would result in a decline in code test diff --git a/doc/user/project/merge_requests/changes.md b/doc/user/project/merge_requests/changes.md index 8796ea0635b..5016a33ed28 100644 --- a/doc/user/project/merge_requests/changes.md +++ b/doc/user/project/merge_requests/changes.md @@ -14,7 +14,7 @@ changes. By default, the diff view compares the versions of files in the merge request source branch to the files in the target branch, and shows only the parts of a file that have changed. - + ## Show all changes in a merge request diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index 10fc778d5ae..7e8ef9272d4 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -72,7 +72,7 @@ This example shows how to run Code Quality on your code by using GitLab CI/CD an In either configuration, the runner must have enough disk space to handle generated Code Quality files. For example on the [GitLab project](https://gitlab.com/gitlab-org/gitlab) the files are approximately 7 GB. -Once you set up GitLab Runner, include the Code Quality template in your CI configuration: +Once you set up GitLab Runner, include the [Code Quality template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Code-Quality.gitlab-ci.yml) in your CI configuration: ```yaml include: @@ -257,9 +257,9 @@ The template has these [`rules`](../../../ci/yaml/index.md#rules) for the `code ```yaml code_quality: rules: - - if: '$CODE_QUALITY_DISABLED' + - if: $CODE_QUALITY_DISABLED when: never - - if: '$CI_COMMIT_TAG || $CI_COMMIT_BRANCH' + - if: $CI_COMMIT_TAG || $CI_COMMIT_BRANCH ``` If you are using merge request pipelines, your `rules` (or [`workflow: rules`](../../../ci/yaml/index.md#workflow)) @@ -268,9 +268,9 @@ might look like this example: ```yaml job1: rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' # Run job1 in merge request pipelines - - if: '$CI_COMMIT_BRANCH == "main"' # Run job1 in pipelines on the main branch (but not in other branch pipelines) - - if: '$CI_COMMIT_TAG' # Run job1 in pipelines for tags + - if: $CI_PIPELINE_SOURCE == "merge_request_event" # Run job1 in merge request pipelines + - if: $CI_COMMIT_BRANCH == "main" # Run job1 in pipelines on the main branch (but not in other branch pipelines) + - if: $CI_COMMIT_TAG # Run job1 in pipelines for tags ``` To make these work together, you need to overwrite the code quality `rules` @@ -282,11 +282,11 @@ include: code_quality: rules: - - if: '$CODE_QUALITY_DISABLED' + - if: $CODE_QUALITY_DISABLED when: never - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' # Run code quality job in merge request pipelines - - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH' # Run code quality job in pipelines on the default branch (but not in other branch pipelines) - - if: '$CI_COMMIT_TAG' # Run code quality job in pipelines for tags + - if: $CI_PIPELINE_SOURCE == "merge_request_event" # Run code quality job in merge request pipelines + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run code quality job in pipelines on the default branch (but not in other branch pipelines) + - if: $CI_COMMIT_TAG # Run code quality job in pipelines for tags ``` ### Configure Code Quality to use a private container image registry @@ -541,7 +541,7 @@ This can be due to multiple reasons: - If no [degradation or error is detected](https://docs.codeclimate.com/docs/maintainability#section-checks), nothing is displayed. - The [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) CI/CD - setting can cause the Code Quality artifact(s) to expire faster than desired. + setting can cause the Code Quality artifacts to expire faster than desired. - The widgets use the pipeline of the latest commit to the target branch. If commits are made to the default branch that do not run the code quality job, this may cause the merge request widget to have no base report for comparison. - If you use the [`REPORT_STDOUT` environment variable](https://gitlab.com/gitlab-org/ci-cd/codequality#environment-variables), no report file is generated and nothing displays in the merge request. - Large `gl-code-quality-report.json` files (esp. >10 MB) are [known to prevent the report from being displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/2737). diff --git a/doc/user/project/merge_requests/confidential.md b/doc/user/project/merge_requests/confidential.md index 6900880417f..5b17ec009e4 100644 --- a/doc/user/project/merge_requests/confidential.md +++ b/doc/user/project/merge_requests/confidential.md @@ -74,5 +74,5 @@ Open a merge request - [Confidential issues](../issues/confidential_issues.md) - [Make an epic confidential](../../group/epics/manage_epics.md#make-an-epic-confidential) -- [Mark a comment as confidential](../../discussions/index.md#mark-a-comment-as-confidential) +- [Add an internal note](../../discussions/index.md#add-an-internal-note) - [Security practices for confidential merge requests](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) at GitLab diff --git a/doc/user/project/merge_requests/fast_forward_merge.md b/doc/user/project/merge_requests/fast_forward_merge.md index 77162aa0b83..048421a3a5b 100644 --- a/doc/user/project/merge_requests/fast_forward_merge.md +++ b/doc/user/project/merge_requests/fast_forward_merge.md @@ -1,74 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, concepts +redirect_to: 'methods/index.md' +remove_date: '2022-08-09' --- -# Fast-forward merge requests **(FREE)** +This document was moved to [another location](methods/index.md). -Sometimes, a workflow policy might mandate a clean commit history without -merge commits. In such cases, the fast-forward merge is the perfect candidate. - -With fast-forward merge requests, you can retain a linear Git history and a way -to accept merge requests without creating merge commits. - -When the fast-forward merge -([`--ff-only`](https://git-scm.com/docs/git-merge#git-merge---ff-only)) setting -is enabled, no merge commits are created and all merges are fast-forwarded, -which means that merging is only allowed if the branch can be fast-forwarded. - -When a fast-forward merge is not possible, the user is given the option to rebase. - -NOTE: -Projects using the fast-forward merge strategy can't filter merge requests -[by deployment date](../../search/index.md#filtering-merge-requests-by-environment-or-deployment-date), -because no merge commit is created. - -## Enabling fast-forward merges - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. -1. In the **Merge method** section, select **Fast-forward merge**. -1. Select **Save changes**. - -Now, when you visit the merge request page, you can accept it -**only if a fast-forward merge is possible**. - - - -If a fast-forward merge is not possible but a conflict free rebase is possible, -a rebase button is offered. - -You can also rebase without running a CI/CD pipeline. -[Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/118825) GitLab 14.7. - -The rebase action is also available as a [quick action command: `/rebase`](../../../topics/git/git_rebase.md#rebase-from-the-gitlab-ui). - - - -If the target branch is ahead of the source branch and a conflict free rebase is -not possible, you need to rebase the -source branch locally before you can do a fast-forward merge. - - - -## Fast-forward merges prevent squashing commits - -If your project has enabled fast-forward merges, to merge cleanly, the code in a -merge request cannot use [squashing during merge](squash_and_merge.md). Squashing -is available only when accepting a merge request. Rebasing may be required before -squashing, even though squashing can itself be considered equivalent to rebasing. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2022-08-09>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md index fd1751585d5..c1986a80ca0 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/project/merge_requests/getting_started.md @@ -51,9 +51,9 @@ Learn the various ways to [create a merge request](creating_merge_requests.md). When you start a new merge request, you can immediately include the following options. You can also add them later by either selecting **Edit** on the merge request's page at the top-right side, or by using -[keyboard shortcuts for merge requests](../../shortcuts.md#issues-and-merge-requests): +[keyboard shortcuts for merge requests](../../shortcuts.md#merge-requests): -- [Assign](#assignee) the merge request to a colleague for review. With [multiple assignees](#multiple-assignees), you can assign it to more than one person at a time. +- [Assign](index.md#assign-a-user-to-a-merge-request) the merge request to a colleague for review. With [multiple assignees](index.md#assign-multiple-users), you can assign it to more than one person at a time. - Set a [milestone](../milestones/index.md) to track time-sensitive changes. - Add [labels](../labels.md) to help contextualize and filter your merge requests over time. - [Require approval](approvals/index.md#required-approvals) from your team. @@ -76,43 +76,11 @@ After you have created the merge request, you can also: Many of these options can be set: -- From the merge request page, with [keyboard shortcuts](../../shortcuts.md#issues-and-merge-requests). +- From the merge request page, with [keyboard shortcuts](../../shortcuts.md#merge-requests). - When pushing changes from the command line, with [Git push options](../push_options.md). See also other [features associated to merge requests](reviews/index.md#associated-features). -### Assignee - -Choose an assignee to designate someone as the person responsible -for the first [review of the merge request](reviews/index.md). -Open the drop down box to search for the user you wish to assign, -and the merge request is added to their -[assigned merge request list](../../search/index.md#search-issues-and-merge-requests). - -#### Multiple assignees **(PREMIUM)** - -> Moved to GitLab Premium in 13.9 - -Multiple people often review merge requests at the same time. -GitLab allows you to have multiple assignees for merge requests -to indicate everyone that is reviewing or accountable for it. - - - -To assign multiple assignees to a merge request: - -1. From a merge request, expand the right sidebar and locate the **Assignees** section. -1. Click on **Edit** and from the dropdown menu, select as many users as you want - to assign the merge request to. - -Similarly, assignees are removed by deselecting them from the same -dropdown menu. - -It is also possible to manage multiple assignees: - -- When creating a merge request. -- Using [quick actions](../quick_actions.md#issues-merge-requests-and-epics). - ### Reviewer WARNING: diff --git a/doc/user/project/merge_requests/img/merge_method_ff_v15_0.png b/doc/user/project/merge_requests/img/merge_method_ff_v15_0.png Binary files differnew file mode 100644 index 00000000000..323fd03ffa2 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_method_ff_v15_0.png diff --git a/doc/user/project/merge_requests/img/merge_method_merge_commit_v15_0.png b/doc/user/project/merge_requests/img/merge_method_merge_commit_v15_0.png Binary files differnew file mode 100644 index 00000000000..b880c2c0e04 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_method_merge_commit_v15_0.png diff --git a/doc/user/project/merge_requests/img/merge_method_merge_commit_with_semi_linear_history_v15_0.png b/doc/user/project/merge_requests/img/merge_method_merge_commit_with_semi_linear_history_v15_0.png Binary files differnew file mode 100644 index 00000000000..9eab71e9d3c --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_method_merge_commit_with_semi_linear_history_v15_0.png diff --git a/doc/user/project/merge_requests/img/mr-diff-example_v14_8.png b/doc/user/project/merge_requests/img/mr-diff-example_v14_8.png Binary files differdeleted file mode 100644 index 1984defde9a..00000000000 --- a/doc/user/project/merge_requests/img/mr-diff-example_v14_8.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/mr-diff-example_v15.png b/doc/user/project/merge_requests/img/mr-diff-example_v15.png Binary files differnew file mode 100644 index 00000000000..8fdf3935906 --- /dev/null +++ b/doc/user/project/merge_requests/img/mr-diff-example_v15.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index a3b9fb52f0d..510dcd82907 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -70,6 +70,72 @@ change and whether you need access to a development environment: - [Push changes from the command line](../../../gitlab-basics/start-using-git.md), if you are familiar with Git and the command line. +## Assign a user to a merge request + +When a merge request is created, it's assigned by default to the person who created it. +This person owns the merge request, but isn't responsible for [reviewing it](reviews/index.md). +To assign the merge request to someone else, use the `/assign @user` +[quick action](../quick_actions.md#issues-merge-requests-and-epics) in a text area in +a merge request, or: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. On the right sidebar, expand the right sidebar and locate the **Assignees** section. +1. Select **Edit**. +1. Search for the user you want to assign, and select the user. + +The merge request is added to the user's +[assigned merge request list](../../search/index.md#search-issues-and-merge-requests). + +### Assign multiple users **(PREMIUM)** + +> Moved to GitLab Premium in 13.9. + +GitLab enables multiple assignees for merge requests, if multiple people are +accountable for it: + + + +To assign multiple assignees to a merge request, use the `/assign @user` +[quick action](../quick_actions.md#issues-merge-requests-and-epics) in a text area, or: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. On the right sidebar, expand the right sidebar and locate the **Assignees** section. +1. Select **Edit** and, from the dropdown list, select all users you want + to assign the merge request to. + +To remove an assignee, clear the user from the same dropdown list. + +## Close a merge request + +If you decide to permanently stop work on a merge request, +GitLab recommends you close the merge request rather than +[delete it](#delete-a-merge-request). The author and assignees of a merge request, and users with +Developer, Maintainer, or Owner [roles](../../permissions.md) in a project +can close merge requests in the project: + +1. Go to the merge request you want to close. +1. Scroll to the comment box at the bottom of the page. +1. Following the comment box, select **Close merge request**. + +GitLab closes the merge request, but preserves records of the merge request, +its comments, and any associated pipelines. + +### Delete a merge request + +GitLab recommends you close, rather than delete, merge requests. + +WARNING: +You cannot undo the deletion of a merge request. + +To delete a merge request: + +1. Sign in to GitLab as a user with the project Owner role. + Only users with this role can delete merge requests in a project. +1. Go to the merge request you want to delete, and select **Edit**. +1. Scroll to the bottom of the page, and select **Delete merge request**. + ## Request attention to a merge request > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343528) in GitLab 14.10 [with a flag](../../../administration/feature_flags.md) named `mr_attention_requests`. Disabled by default. @@ -117,35 +183,6 @@ only one attention request, which is synced across both duties. If the attention request is removed from you, either as an assignee or a reviewer, it is removed from both your duties. -## Close a merge request - -If you decide to permanently stop work on a merge request, -GitLab recommends you close the merge request rather than -[delete it](#delete-a-merge-request). The author and assignees of a merge request, and users with -Developer, Maintainer, or Owner [roles](../../permissions.md) in a project -can close merge requests in the project: - -1. Go to the merge request you want to close. -1. Scroll to the comment box at the bottom of the page. -1. Following the comment box, select **Close merge request**. - -GitLab closes the merge request, but preserves records of the merge request, -its comments, and any associated pipelines. - -### Delete a merge request - -GitLab recommends you close, rather than delete, merge requests. - -WARNING: -You cannot undo the deletion of a merge request. - -To delete a merge request: - -1. Sign in to GitLab as a user with the project Owner role. - Only users with this role can delete merge requests in a project. -1. Go to the merge request you want to delete, and select **Edit**. -1. Scroll to the bottom of the page, and select **Delete merge request**. - ## Merge request workflows For a software developer working in a team: diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md index 7861e1e28fc..a5fff4a38be 100644 --- a/doc/user/project/merge_requests/load_performance_testing.md +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -189,7 +189,7 @@ review: paths: - review.env rules: - - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. + - if: $CI_COMMIT_BRANCH # Modify to match your pipeline rules, or use `only/except` if needed. load_performance: dependencies: @@ -197,5 +197,5 @@ load_performance: variables: K6_DOCKER_OPTIONS: '--env-file review.env' rules: - - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. + - if: $CI_COMMIT_BRANCH # Modify to match your pipeline rules, or use `only/except` if needed. ``` diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index 256dde4fa17..ac1c61f2e72 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -50,7 +50,7 @@ You can prevent merge requests from being merged if: This works for both: - GitLab CI/CD pipelines -- Pipelines run from an [external CI integration](../integrations/overview.md#integrations-listing) +- Pipelines run from an [external CI integration](../integrations/index.md#available-integrations) As a result, [disabling GitLab CI/CD pipelines](../../../ci/enable_or_disable_ci.md) does not disable this feature, as it is possible to use pipelines from external @@ -81,13 +81,13 @@ it could allow code that fails tests to be merged: ```yaml branch-pipeline-job: rules: - - if: '$CI_PIPELINE_SOURCE == "push"' + - if: $CI_PIPELINE_SOURCE == "push" script: - echo "Code testing scripts here, for example." merge-request-pipeline-job: rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CI_PIPELINE_SOURCE == "merge_request_event" script: - echo "No tests run, but this pipeline always succeeds and enables merge." - echo true diff --git a/doc/user/project/merge_requests/methods/index.md b/doc/user/project/merge_requests/methods/index.md new file mode 100644 index 00000000000..adfa5288f81 --- /dev/null +++ b/doc/user/project/merge_requests/methods/index.md @@ -0,0 +1,116 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference, concepts +--- + +# Merge methods **(FREE)** + +The merge method you select for your project determines how the changes in your +merge requests are merged into an existing branch. + +## Configure a project's merge method + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Merge requests**. +1. In the **Merge method** section, select your desired merge method. +1. Select **Save changes**. + +## Merge commit + +This setting is the default. It always creates a separate merge commit, +even when using [squash](../squash_and_merge.md). An example commit graph generated using this merge method: + + + +- For regular merges, it is equivalent to the command `git merge --no-ff <source-branch>`. +- For squash merges, it squashes all commits in the source branch before merging it normally. It performs actions similar to: + + ```shell + git checkout `git merge-base <source-branch> <target-branch>` + git merge --squash <source-branch> + SOURCE_SHA=`git rev-parse HEAD` + git checkout <target-branch> + git merge --no-ff $SOURCE_SHA + ``` + +## Merge commit with semi-linear history + +A merge commit is created for every merge, but the branch is only merged if +a fast-forward merge is possible. This ensures that if the merge request build +succeeded, the target branch build also succeeds after the merge. An example commit graph generated using this merge method: + + + +When you visit the merge request page with `Merge commit with semi-linear history` +method selected, you can accept it **only if a fast-forward merge is possible**. +When a fast-forward merge is not possible, the user is given the option to rebase, see +[Rebasing in (semi-)linear merge methods](#rebasing-in-semi-linear-merge-methods). + +This method is equivalent to the same Git commands as in the **Merge commit** method. However, +if your source branch is based on an out-of-date version of the target branch (such as `main`), +you must rebase your source branch. +This merge method creates a cleaner-looking history, while still enabling you to +see where every branch began and was merged. + +## Fast-forward merge + +Sometimes, a workflow policy might mandate a clean commit history without +merge commits. In such cases, the fast-forward merge is appropriate. With +fast-forward merge requests, you can retain a linear Git history and a way +to accept merge requests without creating merge commits. An example commit graph +generated using this merge method: + + + +This method is equivalent to `git merge --ff <source-branch>` for regular merges, and to +`git merge -squash <source-branch>` for squash merges. + +When the fast-forward merge +([`--ff-only`](https://git-scm.com/docs/git-merge#git-merge---ff-only)) setting +is enabled, no merge commits are created and all merges are fast-forwarded, +which means that merging is only allowed if the branch can be fast-forwarded. +When a fast-forward merge is not possible, the user is given the option to rebase, see +[Rebasing in (semi-)linear merge methods](#rebasing-in-semi-linear-merge-methods). + +NOTE: +Projects using the fast-forward merge strategy can't filter merge requests +[by deployment date](../../../search/index.md#filtering-merge-requests-by-environment-or-deployment-date), +because no merge commit is created. + +When you visit the merge request page with `Fast-forward merge` +method selected, you can accept it **only if a fast-forward merge is possible**. + + + +## Rebasing in (semi-)linear merge methods + +> Rebasing without running a CI/CD pipeline [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118825) in GitLab 14.7. + +In these merge methods, you can merge only when your source branch is up-to-date with the target branch: + +- Merge commit with semi-linear history. +- Fast-forward merge. + +If a fast-forward merge is not possible but a conflict-free rebase is possible, +GitLab offers you the [`/rebase` quick action](../../../../topics/git/git_rebase.md#rebase-from-the-gitlab-ui), +and the ability to **Rebase** from the user interface: + + + +In [GitLab 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/118825) and later, you can also rebase without running a CI/CD pipeline. + +If the target branch is ahead of the source branch and a conflict-free rebase is +not possible, you must rebase the source branch locally before you can do a fast-forward merge. + + + +Rebasing may be required before squashing, even though squashing can itself be +considered equivalent to rebasing. + +## Related topics + +- [Commits history](../commits.md) +- [Squash and merge](../squash_and_merge.md) diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index 6441ccb73fe..7b4a41f9339 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -14,7 +14,7 @@ by clicking the **Revert** button in merge requests and commit details. NOTE: The **Revert** button is shown only for projects that use the merge method "Merge Commit", which can be set under the project's -**Settings > General > Merge request**. [Fast-forward commits](fast_forward_merge.md) +**Settings > General > Merge request**. [Fast-forward commits](methods/index.md#fast-forward-merge) can not be reverted by using the merge request view. After the merge request has been merged, use the **Revert** button diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index 512faae82a9..eb5a54e6119 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -131,17 +131,6 @@ the author of the merge request can request a new review from the reviewer: GitLab creates a new [to-do item](../../../todos.md) for the reviewer, and sends them a notification email. -## Semi-linear history merge requests - -A merge commit is created for every merge, but the branch is only merged if -a fast-forward merge is possible. This ensures that if the merge request build -succeeded, the target branch build also succeeds after the merge. - -1. Go to your project and select **Settings > General**. -1. Expand **Merge requests**. -1. In the **Merge method** section, select **Merge commit with semi-linear history**. -1. Select **Save changes**. - ## Comment on multiple lines > - [Introduced](https://gitlab.com/gitlab-org/ux-research/-/issues/870) in GitLab 13.2. @@ -211,17 +200,17 @@ These features are associated with merge requests: - [Cherry-pick changes](../cherry_pick_changes.md): Cherry-pick any commit in the UI by selecting the **Cherry-pick** button in a merged merge requests or a commit. -- [Fast-forward merge requests](../fast_forward_merge.md): +- [Fast-forward merge requests](../methods/index.md#fast-forward-merge): For a linear Git history and a way to accept merge requests without creating merge commits - [Find the merge request that introduced a change](../versions.md): - When viewing the commit details page, GitLab links to the merge request(s) containing that commit. + When viewing the commit details page, GitLab links to the merge requests containing that commit. - [Merge requests versions](../versions.md): Select and compare the different versions of merge request diffs - [Resolve conflicts](../conflicts.md): GitLab can provide the option to resolve certain merge request conflicts in the GitLab UI. - [Revert changes](../revert_changes.md): Revert changes from any commit from a merge request. -- [Keyboard shortcuts](../../../shortcuts.md#issues-and-merge-requests): +- [Keyboard shortcuts](../../../shortcuts.md#merge-requests): Access and modify specific parts of a merge request with keyboard commands. ## Troubleshooting @@ -365,3 +354,7 @@ All the above can be done with the [`git-mr`](https://gitlab.com/glensc/git-mr) In a group, the sidebar displays the total count of open merge requests. This value is cached if it's greater than than 1000. The cached value is rounded to thousands (or millions) and updated every 24 hours. + +## Related topics + +- [Merge methods](../methods/index.md) diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index a1d6959b75e..7e37990b9bf 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -18,8 +18,8 @@ in your Git repository by using the _squash and merge_ strategy. Each time a branch merges into your base branch, up to two commits are added: - The single commit created by squashing the commits from the branch. -- A merge commit, unless you have [enabled fast-forward merges](fast_forward_merge.md#enabling-fast-forward-merges) - in your project. Fast-forward merges disable both merge commits and squashing. +- A merge commit, unless you have enabled [fast-forward merges](methods/index.md#fast-forward-merge) + in your project. Fast-forward merges disable merge commits. By default, squashed commits contain the following metadata: @@ -74,7 +74,7 @@ To configure the default squashing behavior for all merge requests in your proje ## Related topics - [Commit message templates](commit_templates.md) -- [Fast-forward merges](fast_forward_merge.md) +- [Merge methods](methods/index.md) <!-- ## Troubleshooting diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index 9f1e5ae7046..85b5bbea284 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -28,7 +28,7 @@ between pipeline completion and the visualization loading on the page. For the coverage analysis to work, you have to provide a properly formatted [Cobertura XML](https://cobertura.github.io/cobertura/) report to -[`artifacts:reports:cobertura`](../../../ci/yaml/artifacts_reports.md#artifactsreportscobertura-deprecated). +[`artifacts:reports:coverage_report`](../../../ci/yaml/artifacts_reports.md#artifactsreportscoverage_report). This format was originally developed for Java, but most coverage analysis frameworks for other languages have plugins to add support for it, like: @@ -196,7 +196,9 @@ coverage-jdk11: needs: ["test-jdk11"] artifacts: reports: - cobertura: target/site/cobertura.xml + coverage_report: + coverage_format: cobertura + path: target/site/cobertura.xml ``` #### Gradle example @@ -232,7 +234,9 @@ coverage-jdk11: needs: ["test-jdk11"] artifacts: reports: - cobertura: build/cobertura.xml + coverage_report: + coverage_format: cobertura + path: build/cobertura.xml ``` ### Python example @@ -251,9 +255,12 @@ run tests: - coverage run -m pytest - coverage report - coverage xml + coverage: '/TOTAL.*\s([.\d]+)%/' artifacts: reports: - cobertura: coverage.xml + coverage_report: + coverage_format: cobertura + path: coverage.xml ``` ### PHP example @@ -263,7 +270,7 @@ to collect test coverage data and generate the report. With a minimal [`phpunit.xml`](https://phpunit.readthedocs.io/en/9.5/configuration.html) file (you may reference [this example repository](https://gitlab.com/yookoala/code-coverage-visualization-with-php/)), you can run the test and -generate the coverage xml: +generate the `coverage.xml`: ```yaml run tests: @@ -283,7 +290,9 @@ run tests: - php ./vendor/bin/phpunit --coverage-text --coverage-cobertura=coverage.cobertura.xml artifacts: reports: - cobertura: coverage.cobertura.xml + coverage_report: + coverage_format: cobertura + path: coverage.cobertura.xml ``` [Codeception](https://codeception.com/), through PHPUnit, also supports generating Cobertura report with @@ -318,7 +327,9 @@ run tests: name: ${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHA} expire_in: 2 days reports: - cobertura: build/coverage.xml + coverage_report: + coverage_format: cobertura + path: build/coverage.xml ``` ### Go example @@ -345,7 +356,9 @@ run tests: - go run github.com/boumenot/gocover-cobertura < coverage.txt > coverage.xml artifacts: reports: - cobertura: coverage.xml + coverage_report: + coverage_format: cobertura + path: coverage.xml ``` ### Ruby example @@ -372,5 +385,7 @@ run tests: - bundle exec rspec artifacts: reports: - cobertura: coverage/coverage.xml + coverage_report: + coverage_format: cobertura + path: coverage/coverage.xml ``` diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 4501cf500b0..c2b85a2183c 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -53,10 +53,14 @@ If you're in a project and select **Issues > Milestones**, GitLab displays only ## Creating milestones -Users with at least the Developer role can create milestones. +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. Milestones can be created either at project or group level. +Prerequisites: + +- You must have at least the Reporter role for a group. + To create a milestone: 1. On the top bar, select **Menu > Projects** and find your project or **Menu > Groups** and find your group. @@ -69,7 +73,13 @@ To create a milestone: ## Editing milestones -Users with at least the Developer role can edit milestones. +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. + +Users with at least the Reporter role can edit milestones. + +Prerequisites: + +- You must have at least the Reporter role for a group. To edit a milestone: diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md index f08afc924f3..5fd17b5c07e 100644 --- a/doc/user/project/pages/getting_started/pages_from_scratch.md +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -202,7 +202,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH pages: script: @@ -222,7 +222,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH pages: script: @@ -233,7 +233,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" ``` ### Specify a stage to deploy @@ -253,7 +253,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH pages: stage: deploy @@ -265,7 +265,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" ``` Now add another job to the CI file, telling it to @@ -276,7 +276,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH pages: stage: deploy @@ -288,7 +288,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" test: stage: test @@ -300,7 +300,7 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "main"' + - if: $CI_COMMIT_BRANCH != "main" ``` When the `test` job runs in the `test` stage, Jekyll @@ -327,7 +327,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH before_script: - gem install bundler @@ -341,7 +341,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" test: stage: test @@ -351,7 +351,7 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "main"' + - if: $CI_COMMIT_BRANCH != "main" ``` ### Build faster with cached dependencies @@ -367,7 +367,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH cache: paths: @@ -385,7 +385,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" test: stage: test @@ -395,7 +395,7 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "main"' + - if: $CI_COMMIT_BRANCH != "main" ``` In this case, you need to exclude the `/vendor` diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index f274338c2fd..5846ceeb1b6 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -304,7 +304,7 @@ Find more details in the [Pages administration documentation](../../../administr Safari requires the web server to support the [Range request header](https://developer.apple.com/library/archive/documentation/AppleApplications/Reference/SafariWebContent/CreatingVideoforSafarioniPhone/CreatingVideoforSafarioniPhone.html#//apple_ref/doc/uid/TP40006514-SW6) in order to play your media content. For GitLab Pages to serve -HTTP Range requests, you should use the following two variables in your `.gitlab-ci.yaml` file: +HTTP Range requests, you should use the following two variables in your `.gitlab-ci.yml` file: ```yaml pages: diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 06396b5cd62..56f5a2d24ff 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -10,7 +10,7 @@ In GitLab, [permissions](../permissions.md) are fundamentally defined around the idea of having read or write permission to the repository and branches. To impose further restrictions on certain branches, they can be protected. -The default branch for your repository is protected by default. +The [default branch](repository/branches/default.md) for your repository is protected by default. ## Who can modify a protected branch @@ -39,7 +39,8 @@ Prerequisite: To protect a branch: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to merge** list, select a role, or group that can merge into this branch. In GitLab Premium, you can also add users. @@ -50,13 +51,18 @@ The protected branch displays in the list of protected branches. ## Configure multiple protected branches by using a wildcard +If both a specific rule and a wildcard rule apply to the same branch, the most +permissive rule controls how the branch behaves. For merge controls to work properly, +set **Allowed to push** to a broader set of users than **Allowed to merge**. + Prerequisite: - You must have at least the Maintainer role. To protect multiple branches at the same time: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, type the branch name and a wildcard. For example: @@ -88,7 +94,8 @@ from the command line or from a Git client application. To create a new branch through the user interface: -1. Go to **Repository > Branches**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Branches**. 1. Select **New branch**. 1. Fill in the branch name and select an existing branch, tag, or commit to base the new branch on. Only existing protected branches and commits @@ -96,21 +103,28 @@ To create a new branch through the user interface: ## Require everyone to submit merge requests for a protected branch -You can force everyone to submit a merge request, rather than allowing them to check in directly -to a protected branch. This is compatible with workflows like the [GitLab workflow](../../topics/gitlab_flow.md). +You can force everyone to submit a merge request, rather than allowing them to +check in directly to a protected branch. This setting is compatible with workflows +like the [GitLab workflow](../../topics/gitlab_flow.md). -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to merge** list, select **Developers + Maintainers**. 1. From the **Allowed to push** list, select **No one**. + + NOTE: + Setting a role, group or user as **Allowed to push** also allows those users to merge. + 1. Select **Protect**. ## Allow everyone to push directly to a protected branch You can allow everyone with write access to push to the protected branch. -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** list, select **Developers + Maintainers**. @@ -137,7 +151,8 @@ Prerequisites: To allow a deploy key to push to a protected branch: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** list, select the deploy key. @@ -155,7 +170,8 @@ protected branches. To protect a new branch and enable force push: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. @@ -166,7 +182,8 @@ To protect a new branch and enable force push: To enable force pushes on branches that are already protected: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. In the list of protected branches, next to the branch, turn on the **Allowed to force push** toggle. @@ -181,7 +198,8 @@ For a protected branch, you can require at least one approval by a [Code Owner]( To protect a new branch and enable Code Owner's approval: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. @@ -190,7 +208,8 @@ To protect a new branch and enable Code Owner's approval: To enable Code Owner's approval on branches that are already protected: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. In the list of protected branches, next to the branch, turn on the **Code owner approval** toggle. @@ -221,9 +240,11 @@ for details about the pipelines security model. Users with at least the Maintainer role can manually delete protected branches by using the GitLab web interface: -1. Go to **Repository > Branches**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Branches**. 1. Next to the branch you want to delete, select **Delete** (**{remove}**). -1. On the confirmation dialog, type the branch name and select **Delete protected branch**. +1. On the confirmation dialog, type the branch name. +1. Select **Yes, delete protected branch**. Protected branches can only be deleted by using GitLab either from the UI or API. This prevents accidentally deleting a branch through local Git commands or diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index d04f79d11c7..6ef8477b6b6 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -63,6 +63,7 @@ time as pushing changes: | `merge_request.remove_source_branch` | Set the merge request to remove the source branch when it's merged. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | | `merge_request.title="<title>"` | Set the title of the merge request. Ex: `git push -o merge_request.title="The title I want"`. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | | `merge_request.description="<description>"` | Set the description of the merge request. Ex: `git push -o merge_request.description="The description I want"`. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | +| `merge_request.draft` | Mark the merge request as a draft. Ex: `git push -o merge_request.draft`. | [15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/296673) | | `merge_request.milestone="<milestone>"` | Set the milestone of the merge request. Ex: `git push -o merge_request.milestone="3.0"`. | [14.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63960) | | `merge_request.label="<label>"` | Add labels to the merge request. If the label does not exist, it is created. For example, for two labels: `git push -o merge_request.label="label1" -o merge_request.label="label2"`. | [12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31831) | | `merge_request.unlabel="<label>"` | Remove labels from the merge request. For example, for two labels: `git push -o merge_request.unlabel="label1" -o merge_request.unlabel="label2"`. | [12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31831) | diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 26b36198bde..c0a6fa9c301 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -91,6 +91,7 @@ To create a release in the Releases page: - [Title](#title). - [Milestones](#associate-milestones-with-a-release). - [Release notes](#release-notes-description). + - Whether or not to include the [Tag message](../../../topics/git/tags.md). - [Asset links](#links). 1. Select **Create release**. @@ -439,8 +440,11 @@ Every release has a description. You can add any text you like, but we recommend including a changelog to describe the content of your release. This helps users quickly scan the differences between each release you publish. -[Git's tagging messages](https://git-scm.com/book/en/v2/Git-Basics-Tagging) and -Release note descriptions are unrelated. Description supports [Markdown](../../markdown.md). +[Git's tagging messages](https://git-scm.com/book/en/v2/Git-Basics-Tagging) can +be included in Release note descriptions by selecting **Include tag message in +the release notes**. + +Description supports [Markdown](../../markdown.md). ### Release assets @@ -529,7 +533,7 @@ The physical location of the asset can change at any time and the direct link re > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16821) in GitLab 14.9. -The `filepath` from [permanent links to release assets](#permanent-links-to-release-assets) can be used in combination with [permanent link to the latest release](#permanent-link-to-latest-release). It is useful when we want to link a permanant URL to download an asset from the *latest release*. +The `filepath` from [permanent links to release assets](#permanent-links-to-release-assets) can be used in combination with [permanent link to the latest release](#permanent-link-to-latest-release). It is useful when we want to link a permanent URL to download an asset from the *latest release*. The format of the URL is: diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index 00998c9a227..b2a6c8848ce 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -11,7 +11,7 @@ GPG ([GNU Privacy Guard](https://gnupg.org/)) key. When you add a cryptographic signature to your commit, you provide extra assurance that a commit originated from you, rather than an impersonator. If GitLab can verify a commit author's identity with a public GPG key, the commit is marked **Verified** in the -GitLab UI. You can then configure [push rules](../push_rules.md#enabling-push-rules) +GitLab UI. You can then configure [push rules](../push_rules.md) for your project to reject individual commits not signed with GPG, or reject all commits from unverified users. diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 6ece6e3e4e0..02b5639cae8 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -244,6 +244,11 @@ When you [rename a user](../../profile/index.md#change-your-username), - The redirects are available as long as the original path is not claimed by another group, user, or project. +WARNING: +The [CI/CD `includes` keyword](../../../ci/yaml/includes.md) can't follow project +redirects. Pipelines fail with a syntax error when configured to use `includes` +to fetch configuration from a project that is renamed or moved. + ## Related topics - [GitLab Workflow VS Code extension](vscode.md). diff --git a/doc/user/project/repository/jupyter_notebooks/index.md b/doc/user/project/repository/jupyter_notebooks/index.md index 39b57f89ceb..d013d2802bf 100644 --- a/doc/user/project/repository/jupyter_notebooks/index.md +++ b/doc/user/project/repository/jupyter_notebooks/index.md @@ -23,21 +23,25 @@ it's rendered into HTML when you view it: Interactive features, including JavaScript plots, don't work when viewed in GitLab. -## Cleaner diffs +## Cleaner diffs and raw diffs > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 as an [Alpha](../../../../policy/alpha-beta-support.md#alpha-features) release [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75500) in GitLab 14.9. Feature flag `jupyter_clean_diffs` removed. > - [Reintroduced toggle](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85079) in GitLab 15.0 [with a flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. Enabled by default. +> - Selecting between raw and cleaner diffs [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85203) in GitLab 15.0 [with a flag](../../../../administration/feature_flags.md) named `rendered_diffs_viewer`. Enabled by default. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. +On self-managed GitLab, by default semantic diffs are available. To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. On GitLab.com, this feature is available. -This feature is ready for production use. + +FLAG: +On self-managed GitLab, by default the ability to switch between raw and rendered diffs is available. To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md)named `rendered_diffs_viewer`. On GitLab.com, this feature is available. When commits include changes to Jupyter Notebook files, GitLab: - Transforms the machine-readable `.ipynb` file into a human-readable Markdown file. - Displays a cleaner version of the diff that includes syntax highlighting. +- Enables switching between raw and rendered diffs on the Commit and Compare pages. (Not available on merge request pages.) Code suggestions are not available on diffs and merge requests for `.ipynb` files. diff --git a/doc/user/project/repository/managing_large_repositories.md b/doc/user/project/repository/managing_large_repositories.md new file mode 100644 index 00000000000..d6f88697c54 --- /dev/null +++ b/doc/user/project/repository/managing_large_repositories.md @@ -0,0 +1,51 @@ +--- +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +description: "Documentation on large repositories." +--- + +# Managing large repositories **(FREE SELF)** + +GitLab, like any Git based system, is subject to similar performance restraints when it comes to large +repositories that size into the gigabytes. + +On this page we detail several best practices to improve performance with these large repositories on GitLab. + +## Large File System (LFS) + +It's *strongly* recommended in any Git system that binary or blob files (for example, packages, audio, video, graphics, etc.) are stored as Large File Storage (LFS) objects. In such setup, the Objects are stored elsewhere, such as in Object Storage, and this can reduce the repository size significantly, thus improving performance. + +Refer to the [Git LFS docs for more info](../../../topics/git/lfs/index.md). + +## Gitaly Pack Objects Cache + +Gitaly, the service that provides storage for Git repositories, can be configured to cache a short rolling window of Git fetch responses. This is recommended for large repositories as it can notably reduce server load when your server receives lots of fetch traffic. + +Refer to the [Gitaly Pack Objects Cache for more info](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache). + +## Reference Architectures + +Large repositories tend to be found in larger organisations with many users. The GitLab Quality and Support teams provide several [Reference Architectures](../../../administration/reference_architectures/index.md) that are the recommended way to deploy GitLab at scale. + +In these types of setups it's recommended that the GitLab environment used matches a Reference Architecture to improve performance. + +## Gitaly Cluster + +Gitaly Cluster can notably improve large repository performance as it holds multiple replicas of the repository across several nodes. As a result, Gitaly Cluster can load balance read requests against those repositories and is also fault tolerant. + +It's recommended for large repositories, however, Gitaly Cluster is a large solution with additional complexity of setup and management. Refer to the [Gitaly Cluster docs for more info](../../../administration/gitaly/index.md), specifically the [Before deploying Gitaly Cluster](../../../administration/gitaly/index.md#before-deploying-gitaly-cluster) section. + +## Keep GitLab up to date + +Performance improvements and fixes are added continuously in GitLab. As such, it's recommended you keep GitLab updated to the latest version where possible to benefit from these. + +## Reduce concurrent clones in CI/CD + +Large repositories tend to be monorepos. This in turn typically means that these repositories get a lot of traffic not only from users, but from CI/CD. + +CI/CD loads tend to be concurrent as pipelines are scheduled during set times. As a result, the Git requests against the repositories can spike notably during these times and lead to reduced performance for both CI and users alike. + +When designing CI/CD pipelines, it's advisable to reduce their concurrency by staggering them to run at different times, for example, a set running at one time and then another set running several minutes later. + +There's several other actions that can be explored to improve CI/CD performance with large repositories. Refer to the [Runner docs for more info](../../../ci/large_repositories/index.md). diff --git a/doc/user/project/repository/mirror/index.md b/doc/user/project/repository/mirror/index.md index e1017e78437..1645cf7244e 100644 --- a/doc/user/project/repository/mirror/index.md +++ b/doc/user/project/repository/mirror/index.md @@ -228,10 +228,19 @@ This error can occur when a firewall performs a `Deep SSH Inspection` on outgoin If you receive this error after creating a new project using [GitLab CI/CD for external repositories](../../../../ci/ci_cd_for_external_repos/): -```plaintext -"2:fetch remote: "fatal: could not read Username for 'https://bitbucket.org': -terminal prompts disabled\n": exit status 128." -``` +- In Bitbucket Cloud: + + ```plaintext + "2:fetch remote: "fatal: could not read Username for 'https://bitbucket.org': + terminal prompts disabled\n": exit status 128." + ``` + +- In Bitbucket Server (self-managed): + + ```plaintext + "2:fetch remote: "fatal: could not read Username for 'https://lab.example.com': + terminal prompts disabled\n": exit status 128. + ``` Check if the repository owner is specified in the URL of your mirrored repository: @@ -239,13 +248,21 @@ Check if the repository owner is specified in the URL of your mirrored repositor 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Mirroring repositories**. 1. If no repository owner is specified, delete and add the URL again in this format, - replacing `OWNER`, `ACCOUNTNAME`, and `REPONAME` with your values: + replacing `OWNER`, `ACCOUNTNAME`, `PATH_TO_REPO`, and `REPONAME` with your values: - ```plaintext - https://OWNER@bitbucket.org/ACCOUNTNAME/REPONAME.git - ``` + - In Bitbucket Cloud: + + ```plaintext + https://OWNER@bitbucket.org/ACCOUNTNAME/REPONAME.git + ``` + + - In Bitbucket Server (self-managed): + + ```plaintext + https://OWNER@lab.example.com/PATH_TO_REPO/REPONAME.git + ``` -When connecting to the repository for mirroring, Bitbucket requires the repository owner in the string. +When connecting to the Cloud or self-managed Bitbucket repository for mirroring, the repository owner is required in the string. ### Pull mirror is missing LFS files @@ -257,3 +274,31 @@ In some cases, pull mirroring does not transfer LFS files. This issue occurs whe [Fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/335123) in GitLab 14.0.6. - You mirror an external repository using object storage. An issue exists [to fix this problem](https://gitlab.com/gitlab-org/gitlab/-/issues/335495). + +### `The repository is being updated`, but neither fails nor succeeds visibly + +In rare cases, mirroring slots on Redis can become exhausted, +possibly because Sidekiq workers are reaped due to out-of-memory (OoM) events. +When this occurs, mirroring jobs start and complete quickly, but they neither +fail nor succeed. They also do not leave a clear log. To check for this problem: + +1. Enter the [Rails console](../../../../administration/operations/rails_console.md) + and check Redis' mirroring capacity: + + ```ruby + current = Gitlab::Redis::SharedState.with { |redis| redis.scard('MIRROR_PULL_CAPACITY') }.to_i + maximum = Gitlab::CurrentSettings.mirror_max_capacity + available = maximum - current + ``` + +1. If the mirroring capacity is `0` or very low, you can drain all stuck jobs with: + + ```ruby + Gitlab::Redis::SharedState.with { |redis| redis.smembers('MIRROR_PULL_CAPACITY') }.each do |pid| + Gitlab::Redis::SharedState.with { |redis| redis.srem('MIRROR_PULL_CAPACITY', pid) } + end + ``` + +1. After you run the command, the [background jobs page](../../../admin_area/index.md#background-jobs) + should show new mirroring jobs being scheduled, especially when + [triggered manually](#update-a-mirror). diff --git a/doc/user/project/repository/mirror/pull.md b/doc/user/project/repository/mirror/pull.md index 4c437d0f8c8..73103a9af3d 100644 --- a/doc/user/project/repository/mirror/pull.md +++ b/doc/user/project/repository/mirror/pull.md @@ -62,7 +62,8 @@ Prerequisite: 1. Enter the **Git repository URL**. Include the username in the URL, if required: `https://MYUSERNAME@github.com/GROUPNAME/PROJECTNAME.git` 1. In **Mirror direction**, select **Pull**. -1. In **Authentication method**, select your authentication method. +1. In **Authentication method**, select your authentication method. To learn more, read + [Authentication methods for mirrors](index.md#authentication-methods-for-mirrors). 1. Select any of the options you need: - [**Overwrite diverged branches**](#overwrite-diverged-branches) - [**Trigger pipelines for mirror updates**](#trigger-pipelines-for-mirror-updates) @@ -114,6 +115,21 @@ and mirroring attempts stop. This failure is visible in either the: To resume project mirroring, [force an update](index.md#force-an-update). +If many projects are affected by this problem, such as after a long network or +server outage, you can use the [Rails console](../../../../administration/operations/rails_console.md) +to identify and update all affected projects with this command: + +```ruby +Project.find_each do |p| + if p.import_state && p.import_state.retry_count >= 14 + puts "Resetting mirroring operation for #{p.full_path}" + p.import_state.reset_retry_count + p.import_state.set_next_execution_to_now(prioritized: true) + p.import_state.save! + end +end +``` + ## Related topics - Configure [pull mirroring intervals](../../../../administration/instance_limits.md#pull-mirroring-interval) diff --git a/doc/user/project/repository/mirror/push.md b/doc/user/project/repository/mirror/push.md index ebc4430ac53..c00ebf415c9 100644 --- a/doc/user/project/repository/mirror/push.md +++ b/doc/user/project/repository/mirror/push.md @@ -38,8 +38,8 @@ To set up push mirroring for an existing project: 1. Expand **Mirroring repositories**. 1. Enter a repository URL. 1. In the **Mirror direction** dropdown list, select **Push**. -1. Select an **Authentication method**. - You can authenticate with either a password or an [SSH key](index.md#ssh-authentication). +1. Select an **Authentication method**. To learn more, read + [Authentication methods for mirrors](index.md#authentication-methods-for-mirrors). 1. Select **Only mirror protected branches**, if necessary. 1. Select **Keep divergent refs**, if desired. 1. To save the configuration, select **Mirror repository**. diff --git a/doc/user/project/repository/push_rules.md b/doc/user/project/repository/push_rules.md index bb473a2830b..994cf78d98a 100644 --- a/doc/user/project/repository/push_rules.md +++ b/doc/user/project/repository/push_rules.md @@ -6,148 +6,136 @@ info: "To determine the technical writer assigned to the Stage/Group associated # Push rules **(PREMIUM)** -Gain additional control over what can and can't be pushed to your repository by using -regular expressions to reject pushes based on commit contents, branch names or file details. - -GitLab already offers [protected branches](../protected_branches.md), but there are -cases when you need some specific rules. Some common scenarios: preventing Git tag removal, or -enforcing a special format for commit messages. - Push rules are [pre-receive Git hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) you -can enable in a user-friendly interface. They are defined either: +can enable in a user-friendly interface. Push rules give you more control over what +can and can't be pushed to your repository. While GitLab offers +[protected branches](../protected_branches.md), you may need more specific rules, such as: -- Globally if you are an administrator. -- Per project, so you can have different rules applied to different - projects depending on your needs. +- Evaluating the contents of a commit. +- Confirming commit messages match expected formats. +- Enforcing branch name rules. +- Evaluating the details of files. +- Preventing Git tag removal. -## Use cases +GitLab uses [RE2 syntax](https://github.com/google/re2/wiki/Syntax) for regular expressions +in push rules. You can test them at the [regex101 regex tester](https://regex101.com/). -Every push rule could have its own use case, but let's consider some examples. +For custom push rules use [server hooks](../../../administration/server_hooks.md). -### Commit messages with a specific reference +## Enable global push rules -Let's assume you have the following requirements for your workflow: +You can create push rules for all new projects to inherit, but they can be overridden +at the project level or the [group level](../../group/index.md#group-push-rules). -- every commit should reference a Jira issue, for example: `Refactored css. Fixes JIRA-123.` -- users should not be able to remove Git tags with `git push` +Prerequisite: -Write a regular expression that requires the mention -of a Jira issue in the commit message, like `JIRA\-\d+`. +- You must be an administrator. -Now when a user tries to push a commit with a message `Bugfix`, their push is -declined. Only pushing commits with messages like `Bugfix according to JIRA-123` -is accepted. +To create global push rules: -### Restrict branch names +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Push Rules**. +1. Expand **Push rules**. +1. Set the rule you want. +1. Select **Save push rules**. -If your company has a strict policy for branch names, you may want the branches to start -with a certain name. This approach enables different -GitLab CI/CD jobs (such as `feature`, `hotfix`, `docker`, `android`) that rely on the -branch name. +## Override global push rules per project -Your developers may not remember that policy, so they might push to -various branches, and CI pipelines might not work as expected. By restricting the -branch names globally in Push Rules, such mistakes are prevented. -All branch names that don't match your push rule are rejected. +The push rule of an individual project overrides the global push rule. +To override global push rules for a specific project: -Note that the name of your default branch is always allowed, regardless of the branch naming -regular expression (regex) specified. GitLab is configured this way -because merges typically have the default branch as their target. -If you have other target branches, include them in your regex. (See [Enabling push rules](#enabling-push-rules)). +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Push rules**. +1. Set the rule you want. +1. Select **Save push rules**. -The default branch also defaults to being a [protected branch](../protected_branches.md), -which already limits users from pushing directly. +## Verify users -Some example regular expressions you can use in push rules: +Use these rules to validate users who make commits. -- `^JIRA-` Branches must start with `JIRA-`. -- `-JIRA$` Branches must end with `-JIRA`. -- `^[a-z0-9\\-]{4,15}$` Branches must be between `4` and `15` characters long, - accepting only lowercase letters, numbers and dashes. +- **Reject unverified users**: Users must have a [confirmed email address](../../../security/user_email_confirmation.md). +- **Check whether the commit author is a GitLab user**: The commit author and committer must have an email address that's been verified by GitLab. +- **Commit author's email**: Both the author's and committer's email addresses must match the regular expression. + To allow any email address, leave empty. -#### Default restricted branch names +## Validate commit messages -> Introduced in GitLab 12.10. +Use these rules for your commit messages. -By default, GitLab restricts certain formats of branch names for security purposes. -40-character hexadecimal names, similar to Git commit hashes, are prohibited. +- **Require expression in commit messages**: Messages must match the + expression. To allow any commit message, leave empty. + Uses multiline mode, which can be disabled by using `(?-m)`. -### Custom Push Rules **(PREMIUM SELF)** + For example, if every commit should reference a Jira issue + (like `Refactored css. Fixes JIRA-123.`), the regular expression would be + `JIRA\-\d+`. +- **Reject expression in commit messages**: Commit messages must not match + the expression. To allow any commit message, leave empty. + Uses multiline mode, which can be disabled by using `(?-m)`. -It's possible to create custom push rules rather than the push rules available in -**Admin Area > Push Rules** by using more advanced server hooks. +## Validate branch names -See [server hooks](../../../administration/server_hooks.md) for more information. +To validate your branch names, enter a regular expression for **Branch name**. +To allow any branch name, leave empty. Your +[default branch](branches/default.md) is always allowed. -## Enabling push rules +Examples: -You can create push rules for all new projects to inherit, but they can be overridden -at the project level or the [group level](../../group/index.md#group-push-rules). +- Branches must start with `JIRA-`. -To create global push rules: + ```plaintext + `^JIRA-` + ``` -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Push Rules**. +- Branches must end with `-JIRA`. -To override global push rules in a project's settings: + ```plaintext + `-JIRA$` + ``` -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Repository**. -1. Expand **Push rules**. -1. Set the rule you want. -1. Select **Save push rules**. +- Branches must be between `4` and `15` characters long, + accepting only lowercase letters, numbers and dashes. -The following options are available: + ```plaintext + `^[a-z0-9\\-]{4,15}$` + ``` -| Push rule | Description | -|---------------------------------|-------------| -| Removal of tags with `git push` | Forbid users to remove Git tags with `git push`. Tags can be deleted through the web UI. | -| Check whether the commit author is a GitLab user | Restrict commits to existing GitLab users (checked against their emails). <sup>1</sup> | -| Reject unverified users | GitLab rejects any commit that was not committed by the same user as the user who pushed it, or where the committer's email address is not [confirmed](../../../security/user_email_confirmation.md). | -| Check whether commit is signed through GPG | Reject commit when it is not signed through GPG. Read [signing commits with GPG](gpg_signed_commits/index.md). | -| Prevent pushing secret files | GitLab rejects any files that are likely to contain secrets. See the [forbidden file names](#prevent-pushing-secrets-to-the-repository). | -| Require expression in commit messages | Only commit messages that match this regular expression are allowed to be pushed. <sup>2</sup> Leave empty to allow any commit message. Uses multiline mode, which can be disabled using `(?-m)`. | -| Reject expression in commit messages | Only commit messages that do not match this regular expression are allowed to be pushed. <sup>2</sup> Leave empty to allow any commit message. Uses multiline mode, which can be disabled using `(?-m)`. | -| Restrict by branch name | Only branch names that match this regular expression are allowed to be pushed. <sup>2</sup> Leave empty to allow all branch names. | -| Restrict by commit author's email | Only commit author's email that match this regular expression are allowed to be pushed. <sup>1</sup> <sup>2</sup> Leave empty to allow any email. | -| Prohibited file names | Any committed filenames that match this regular expression and do not already exist in the repository are not allowed to be pushed. <sup>2</sup> Leave empty to allow any filenames. See [common examples](#prohibited-file-names). | -| Maximum file size | Pushes that contain added or updated files that exceed this file size (in MB) are rejected. Set to 0 to allow files of any size. Files tracked by Git LFS are exempted. | +NOTE: +In GitLab 12.10 and later, certain formats of branch names are restricted by +default for security purposes. 40-character hexadecimal names, similar to Git +commit hashes, are prohibited. -1. Checks both the commit author and committer. -1. GitLab uses [RE2 syntax](https://github.com/google/re2/wiki/Syntax) for regular expressions in push rules, and you can test them at the [regex101 regex tester](https://regex101.com/). +## Prevent unintended consequences -### Caveat to "Reject unsigned commits" push rule +Use these rules to prevent unintended consequences. -This push rule ignores commits that are authenticated and created by GitLab -(either through the UI or API). When the **Reject unsigned commits** push rule is -enabled, unsigned commits may still show up in the commit history if a commit was -created **in** GitLab itself. As expected, commits created outside GitLab and -pushed to the repository are rejected. For more information about how GitLab -plans to fix this issue, read [issue #19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185). +- **Reject unsigned commits**: Commit must be signed through [GPG](gpg_signed_commits/index.md). This rule + can block some legitimate commits [created in the Web IDE](#reject-unsigned-commits-push-rule-disables-web-ide), + and allow [unsigned commits created in the GitLab UI](#unsigned-commits-created-in-the-gitlab-ui). +- **Do not allow users to remove Git tags with `git push`**: Users cannot use `git push` to remove Git tags. + Users can still delete tags in the UI. -#### "Reject unsigned commits" push rule disables Web IDE +## Validate files -In 13.10, if a project has the "Reject unsigned commits" push rule, the user is not allowed to -commit through GitLab Web IDE. +Use these rules to validate files contained in the commit. -To allow committing through the Web IDE on a project with this push rule, a GitLab administrator -must disable the feature flag `reject_unsigned_commits_by_gitlab`. This can be done through a -[rails console](../../../administration/operations/rails_console.md) and running: +- **Prevent pushing secret files**: Files must not contain [secrets](#prevent-pushing-secrets-to-the-repository). +- **Prohibited file names**: Files that do not exist in the repository + must not match the regular expression. To allow all file names, leave empty. See [common examples](#prohibit-files-by-name). +- **Maximum file size**: Added or updated files must not exceed this + file size (in MB). To allow files of any size, set to `0`. Files tracked by Git LFS are exempted. -```ruby -Feature.disable(:reject_unsigned_commits_by_gitlab) -``` - -## Prevent pushing secrets to the repository +### Prevent pushing secrets to the repository > Moved to GitLab Premium in 13.9. -Secrets, such as credential files and SSH private keys, should never be committed to a version control +Never commit secrets, such as credential files and SSH private keys, to a version control system. In GitLab, you can use a predefined list of files to block those files from a -repository. Any merge request containing a file matching the list is blocked from being merged. -Files already committed to the repository are not restricted by this push rule. +repository. Merge requests that contain a file that matches the list are blocked. +This push rule does not restrict files already committed to the repository. -Files blocked by this rule are listed below. For a complete list of criteria, see +Files blocked by this rule are listed below. For a complete list of criteria, refer to [`files_denylist.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/gitlab/checks/files_denylist.yml). - AWS CLI credential blobs: @@ -211,78 +199,85 @@ Files blocked by this rule are listed below. For a complete list of criteria, se - `*.history` - `*_history` -### Prevent pushing secrets to all projects +### Prohibit files by name -To set a global push rule to prevent pushing secrets to all projects: +> Moved to GitLab Premium in 13.9. -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Push Rules**. -1. Expand **Push rules**. -1. Select **Prevent pushing secret files**. -1. Select **Save push rules**. +In Git, filenames include both the file's name, and all directories preceding the name. +When you `git push`, each filename in the push is compared to the regular expression +in **Prohibited file names**. -### Prevent pushing secrets to a project +The regular expression in your **Prohibited file names** push rule can contain multiple, +independent matches to exclude. You can match file names broadly to any location in +your repository, or restrict only in certain locations. Filename matches can also +be partial, and exclude file types by extension. -The push rule of a project overrides the global push rule. +These examples use regex (regular expressions) string boundary characters to match +the beginning of a string (`^`), and its end (`$`). They also include instances +where either the directory path or the filename can include `.` or `/`. Both of +these special regex characters must be escaped with a backslash `\\` if you want +to use them as normal characters in a match condition. -To prevent pushing secrets to a project: +- **Prevent pushing `.exe` files to any location in the repository** - This regex + matches any filename that contains `.exe` at the end: -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Repository**. -1. Expand **Push rules**. -1. Select **Prevent pushing secret files**. -1. Select **Save push rules**. + ```plaintext + \.exe$ + ``` -## Prohibited file names +- **Prevent pushing a specific configuration file in the repository root** -> Moved to GitLab Premium in 13.9. + ```plaintext + ^config\.yml$ + ``` -Each filename contained in a Git push is compared to the regular expression in this field. Filenames in Git consist of both the file's name and any directory that may precede it. A singular regular expression can contain multiple independent matches used as exclusions. File names can be broadly matched to any location in the repository, or restricted to specific locations. Filenames can also be partial matches used to exclude file types by extension. +- **Prevent pushing a specific configuration file in a known directory** -The following examples make use of regex string boundary characters which match the beginning of a string (`^`), and the end (`$`). They also include instances where either the directory path or the filename can include `.` or `/`. Both of these special regex characters have to be escaped with a backslash `\\` to be used as normal characters in a match condition. + ```plaintext + ^directory-name\/config\.yml$ + ``` -Example: prevent pushing any `.exe` files to any location in the repository. This uses a partial match, which matches any filename that contains `.exe` at the end: +- **Prevent pushing a specific file to any location in the repository** - This example tests + for any file named `install.exe`. The parenthesized expression `(^|\/)` matches either + a file following a directory separator, or a file in the root directory of the repository: -```plaintext -\.exe$ -``` + ```plaintext + (^|\/)install\.exe$ + ``` -Example: prevent a specific configuration file in the repository root from being pushed: +- **Combine all previous expressions into one expression** - The preceding expressions rely + on the end-of-string character `$`. We can move that part of each expression to the + end of the grouped collection of match conditions, where it is appended to all matches: -```plaintext -^config\.yml$ -``` + ```plaintext + (\.exe|^config\.yml|^directory-name\/config\.yml|(^|\/)install\.exe)$ + ``` -Example: prevent a specific configuration file in a known directory from being pushed: +## Related topics -```plaintext -^directory-name\/config\.yml$ -``` +- [Server hooks](../../../administration/server_hooks.md), to create complex custom push rules +- [Signing commits with GPG](gpg_signed_commits/index.md) +- [Protected branches](../protected_branches.md) -Example: prevent the specific file named `install.exe` from being pushed to any -location in the repository. The parenthesized expression `(^|\/)` matches either -a file following a directory separator or a file in the root directory of the repository: +## Troubleshooting -```plaintext -(^|\/)install\.exe$ -``` +### Reject unsigned commits push rule disables Web IDE -Example: combining all of the above in a single expression. The preceding expressions rely -on the end-of-string character `$`. We can move that part of each expression to the -end of the grouped collection of match conditions where it is appended to all matches: +In GitLab 13.10, if a project has the **Reject unsigned commits** push rule, the user cannot +create commits through the GitLab Web IDE. -```plaintext -(\.exe|^config\.yml|^directory-name\/config\.yml|(^|\/)install\.exe)$ -``` +To allow committing through the Web IDE on a project with this push rule, a GitLab administrator +must disable the feature flag `reject_unsigned_commits_by_gitlab`. [with a flag](../../../administration/feature_flags.md) -<!-- ## Troubleshooting +```ruby +Feature.disable(:reject_unsigned_commits_by_gitlab) +``` -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### Unsigned commits created in the GitLab UI -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +The **Reject unsigned commits** push rule ignores commits that are authenticated +and created by GitLab (either through the UI or API). When this push rule is +enabled, unsigned commits may still appear in the commit history if a commit was +created in GitLab itself. As expected, commits created outside GitLab and +pushed to the repository are rejected. For more information about this issue, +read [issue #19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185). diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index 6eed1717507..747bd690911 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -159,7 +159,7 @@ repository project, GitLab performs these actions: - Creates a default branch. - Commits a blank `README.md` file to it. - Creates and redirects you to a new branch based on the issue title. -- _If your project is [configured with a deployment service](../integrations/overview.md) like Kubernetes,_ +- _If your project is [configured with a deployment service](../integrations/index.md) like Kubernetes,_ GitLab prompts you to set up [auto deploy](../../../topics/autodevops/stages.md#auto-deploy) by helping you create a `.gitlab-ci.yml` file. diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 30261ed5082..9b37e147a52 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -41,7 +41,7 @@ Prerequisites: To export a project and its data, follow these steps: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings**. +1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. Select **Export project**. 1. After the export is generated, you should receive an email with a link to download the file. @@ -82,6 +82,7 @@ The following items are **not** exported: - Merge Request Approvers and [the number of required approvals](https://gitlab.com/gitlab-org/gitlab/-/issues/221088) - Repository size limits - Deploy keys allowed to push to protected branches +- Secure Files These content rules also apply to creating projects from templates on the [group](../../group/custom_project_templates.md) diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 31cda756a78..1e63472763d 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -88,11 +88,11 @@ read-only view to discourage this behavior. Compliance framework pipelines allow group owners to define a compliance pipeline in a separate repository that gets -executed in place of the local project's `gitlab-ci.yml` file. As part of this pipeline, an -`include` statement can reference the local project's `gitlab-ci.yml` file. This way, the compliance +executed in place of the local project's `.gitlab-ci.yml` file. As part of this pipeline, an +`include` statement can reference the local project's `.gitlab-ci.yml` file. This way, the compliance pipeline jobs can run alongside the project-specific jobs any time the pipeline runs. Jobs and variables defined in the compliance -pipeline can't be changed by variables in the local project's `gitlab-ci.yml` file. +pipeline can't be changed by variables in the local project's `.gitlab-ci.yml` file. When you set up the compliance framework, use the **Compliance pipeline configuration** box to link the compliance framework to specific CI/CD configuration. Use the @@ -314,7 +314,7 @@ related to the project by selecting the **Disable email notifications** checkbox Set up your project's merge request settings: -- Set up the merge request method (merge commit, [fast-forward merge](../merge_requests/fast_forward_merge.md)). +- Set up the [merge request method](../merge_requests/methods/index.md) (merge commit, fast-forward merge). - Add merge request [description templates](../description_templates.md#description-templates). - Enable [merge request approvals](../merge_requests/approvals/index.md). - Enable [status checks](../merge_requests/status_checks.md). @@ -409,10 +409,13 @@ NOTE: Only project owners and administrators have the [permissions](../../permissions.md#project-members-permissions) to transfer a project. -You can transfer an existing project into a [group](../../group/index.md). +You can transfer an existing project to another [group](../../group/index.md), +or you can transfer a [personal project](../working_with_projects.md#view-personal-projects) to a group. Prerequisites: +- A group for your project. You can [view your existing groups](../../group/index.md#view-groups) + to find a suitable group. If you don't have a group, [create one](../../group/index.md#create-a-group). - You must have at least the Maintainer role in that group. - You must be the Owner of that project. - The group to which the project is being transferred to must allow creation of new projects. @@ -423,15 +426,15 @@ Prerequisites: To transfer a project: -1. Navigate to your project's **Settings > General**. -1. Under **Advanced**, click **Expand**. -1. Under "Transfer project", choose the namespace you want to transfer the - project to. -1. Confirm the transfer by typing the project's path as instructed. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Advanced**. +1. Under **Transfer project**, choose the namespace to transfer the project to. +1. Select **Transfer project**. +1. Enter the project's name and select **Confirm**. -Once done, you are redirected to the new project's namespace. At this point, -read what happens with the -[redirects from the old project to the new one](../repository/index.md#what-happens-when-a-repository-path-changes). +You are redirected to the project's new URL. Read what happens with the +[redirects from the old URL to the new one](../repository/index.md#what-happens-when-a-repository-path-changes). NOTE: GitLab administrators can use the [administration interface](../../admin_area/index.md#administering-projects) @@ -520,7 +523,8 @@ If you want to use the fork for yourself and don't need to send you can safely remove the fork relationship. WARNING: -Once removed, the fork relationship cannot be restored. You can't send merge requests to the source, and if anyone has forked your project, their fork also loses the relationship. +Once removed, you can't send merge requests to the source, and if anyone has forked your project, their fork also loses the relationship. +To restore the fork relationship, [use the API](../../../api/projects.md#create-a-forked-fromto-relation-between-existing-projects). To do so: @@ -556,10 +560,6 @@ Automatically [create](../../../operations/incident_management/incidents.md#crea Configure Error Tracking to discover and view [Sentry errors within GitLab](../../../operations/error_tracking.md). -### Jaeger tracing - -Add the URL of a Jaeger server to allow your users to [easily access the Jaeger UI from within GitLab](../../../operations/tracing.md). - ### Status Page **(ULTIMATE)** [Add Storage credentials](../../../operations/incident_management/status_page.md#sync-incidents-to-the-status-page) diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index b66913b7223..e332b74f908 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -25,7 +25,7 @@ Use a project access token to authenticate: Project access tokens are similar to [group access tokens](../../group/settings/group_access_tokens.md) and [personal access tokens](../../profile/personal_access_tokens.md). -In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) as personal access tokens if the limit is set. +In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. You can use project access tokens: @@ -48,7 +48,7 @@ To create a project access token: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Enter a name. The token name is visible to any user with permissions to view the project. -1. Optional. Enter an expiry date for the token. The token expires on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. +1. Optional. Enter an expiry date for the token. The token expires on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-project-access-token). diff --git a/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png b/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png Binary files differdeleted file mode 100644 index 380d96f1db9..00000000000 --- a/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png +++ /dev/null diff --git a/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png b/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png Binary files differdeleted file mode 100644 index 89864858ed3..00000000000 --- a/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png +++ /dev/null diff --git a/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_3.png b/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_3.png Binary files differdeleted file mode 100644 index 52776c6a290..00000000000 --- a/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_3.png +++ /dev/null diff --git a/doc/user/project/static_site_editor/index.md b/doc/user/project/static_site_editor/index.md index 220623d0372..343482757f5 100644 --- a/doc/user/project/static_site_editor/index.md +++ b/doc/user/project/static_site_editor/index.md @@ -2,35 +2,15 @@ stage: Create group: Editor info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" -type: reference, how-to -description: "The static site editor enables users to edit content on static websites without prior knowledge of the underlying templating language, site architecture or Git commands." +remove_date: '2022-08-03' +redirect_to: '../web_ide/index.md' --- -# Static Site Editor **(FREE)** +# Static Site Editor (removed) **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28758) in GitLab 12.10. -> - WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214559) in GitLab 13.0. -> - Non-Markdown content blocks not editable on the WYSIWYG mode [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216836) in GitLab 13.3. -> - Formatting Markdown [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49052) in GitLab 13.7. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77246) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77246) -in GitLab 14.7, and is planned for -[removal](https://gitlab.com/groups/gitlab-org/-/epics/7351) in GitLab 14.10. -Users should instead use the [Web Editor](../repository/web_editor.md) or [Web IDE](../web_ide/index.md). [Removal instructions](#remove-the-static-site-editor) for existing projects are included on this page. - -Static Site Editor (SSE) enables users to edit content on static websites without -prior knowledge of the underlying templating language, site architecture, or -Git commands. A contributor to your project can quickly edit a Markdown page -and submit the changes for review. For example: - -- Non-technical collaborators can edit a page directly from the browser. - They don't need to know Git and the details of your project to contribute. -- Recently hired team members can quickly edit content. -- Temporary collaborators can jump from project to project and quickly edit pages instead - of having to clone or fork every single project they need to submit changes to. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77246) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/352505) in 15.0. +Use the [Web Editor](../repository/web_editor.md) or [Web IDE](../web_ide/index.md) instead. ## Remove the Static Site Editor @@ -68,235 +48,3 @@ from an existing project, remove links that point back to the editor: `/helpers/custom_helpers.rb` entirely. 1. Clean up any extraneous configuration files. 1. Commit and push your changes. - -## Requirements - -- In order use the Static Site Editor feature, your project needs to be - pre-configured with the [Static Site Editor Middleman template](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman). -- You need to be logged into GitLab and be a member of the - project (with Developer or higher permission levels). - -## How it works - -The Static Site Editor is in an early stage of development and only supports -Middleman sites for now. You have to use a specific site template to start -using it. The project template is configured to deploy a [Middleman](https://middlemanapp.com/) -static website with [GitLab Pages](../pages/index.md). - -Once your website is up and running, an **Edit this page** button displays on -the bottom-left corner of its pages: - - - -When you click it, GitLab opens up an editor window from which the content -can be directly edited. When you're ready, you can submit your changes in a -click of a button: - - - -When an editor submits their changes, these are the following actions that GitLab -performs automatically in the background: - -1. Creates a new branch. -1. Commits their changes. - 1. Fixes formatting according to the [Handbook Markdown Style Guide](https://about.gitlab.com/handbook/markdown-guide/) - style guide and add them through another commit. -1. Opens a merge request against the default branch. - -The editor can then navigate to the merge request to assign it to a colleague for review. - -## Set up your project - -First, set up the project. Once done, you can use the Static Site Editor to -[edit your content](#edit-content). - -1. To get started, create a new project from the [Static Site Editor - Middleman](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman) - template. You can either [fork it](../repository/forking_workflow.md#creating-a-fork) - or [create a new project from a template](../working_with_projects.md#create-a-project-from-a-built-in-template). -1. Edit the [`data/config.yml`](#static-site-generator-configuration) configuration file - to replace `<username>` and `<project-name>` with the proper values for - your project's path. -1. Optional. Edit the [`.gitlab/static-site-editor.yml`](#static-site-editor-configuration-file) file - to customize the behavior of the Static Site Editor. -1. When you submit your changes, GitLab triggers a CI/CD pipeline to deploy your project with GitLab Pages. -1. When the pipeline finishes, from your project's left-side menu, go to **Settings > Pages** to find the URL of your new website. -1. Visit your website and look at the bottom-left corner of the screen to see the new **Edit this page** button. - -Anyone satisfying the [requirements](#requirements) can edit the -content of the pages without prior knowledge of Git or of your site's -codebase. - -## Edit content - -> - Support for modifying the default merge request title and description [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216861) in GitLab 13.5. -> - Support for selecting a merge request template [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263252) in GitLab 13.6. - -After setting up your project, you can start editing content directly from the Static Site Editor. - -To edit a file: - -1. Visit the page you want to edit. -1. Select **Edit this page**. -1. The file is opened in the Static Site Editor in **WYSIWYG** mode. If you - wish to edit the raw Markdown instead, you can toggle the **Markdown** mode - in the bottom-right corner. -1. When you're done, click **Submit changes...**. -1. Optional. Adjust the default title and description of the merge request, to submit - with your changes. Alternatively, select a [merge request template](../../../user/project/description_templates.md#create-a-merge-request-template) - from the dropdown menu and edit it accordingly. -1. Select **Submit changes**. -1. A new merge request is automatically created and you can assign a colleague for review. - -### Text - -> Support for `*.md.erb` files [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223171) in GitLab 13.2. - -The Static Site Editors supports Markdown files (`.md`, `.md.erb`) for editing text. - -### Images - -> - Support for adding images through the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216640) in GitLab 13.1. -> - Support for uploading images via the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218529) in GitLab 13.6. - -#### Upload an image - -You can upload image files via the WYSIWYG editor directly to the repository to default upload directory -`source/images`. To do so: - -1. Select the image icon (**{doc-image}**). -1. Select the **Upload file** tab. -1. To select a file from your computer, select **Choose file**. -1. Optional. Add a description to the image for SEO and accessibility ([ALT text](https://moz.com/learn/seo/alt-text)). -1. Select **Insert image**. - -The selected file can be any supported image file (`.png`, `.jpg`, `.jpeg`, `.gif`). The editor renders -thumbnail previews so you can verify the correct image is included and there aren't any references to -missing images. - -#### Link to an image - -You can also link to an image if you'd like: - -1. Select the image icon (**{doc-image}**). -1. Select the **Link to an image** tab. -1. Add the link to the image into the **Image URL** field (use the full path; relative paths are not supported yet). -1. Optional. Add a description to the image for SEO and accessibility ([ALT text](https://moz.com/learn/seo/alt-text)). -1. Select **Insert image**. - -The link can reference images already hosted in your project, an asset hosted -externally on a content delivery network, or any other external URL. The editor renders thumbnail previews -so you can verify the correct image is included and there aren't any references to missing images. - -### Videos - -> - Support for embedding YouTube videos through the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216642) in GitLab 13.5. - -You can embed YouTube videos on the WYSIWYG mode by clicking the video icon (**{live-preview}**). -The following URL/ID formats are supported: - -- **YouTube watch URLs**: `https://www.youtube.com/watch?v=0t1DgySidms` -- **YouTube embed URLs**: `https://www.youtube.com/embed/0t1DgySidms` -- **YouTube video IDs**: `0t1DgySidms` - -### Front matter - -> - Markdown front matter hidden on the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216834) in GitLab 13.1. -> - Ability to edit page front matter [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235921) in GitLab 13.5. - -Front matter is a flexible and convenient way to define page-specific variables in data files -intended to be parsed by a static site generator. Use it to set a page's -title, layout template, or author. You can also pass any kind of metadata to the -generator as the page renders out to HTML. Included at the very top of each data file, the -front matter is often formatted as YAML or JSON, and requires consistent and accurate syntax. - -To edit the front matter from the Static Site Editor you can use the GitLab regular file editor, -the Web IDE, or update the data directly from the WYSIWYG editor: - -1. Click the **Page settings** button on the bottom-right to reveal a web form with the data you - have on the page's front matter. The form is populated with the current data: - -  - -1. Update the values as you wish and close the panel. -1. When you're done, click **Submit changes...**. -1. Describe your changes (add a commit message). -1. Click **Submit changes**. -1. Click **View merge request** to view it. - -Adding new attributes to the page's front matter from the form is not supported. -To add new attributes: - -- Edit the file locally -- Edit the file with the GitLab regular file editor. -- Edit the file with the Web IDE. - -After adding an attribute, the form loads the new fields. - -## Configuration files - -You can customize the behavior of a project which uses the Static Site Editor with -the following configuration files: - -- The [`.gitlab/static-site-editor.yml`](#static-site-editor-configuration-file), which customizes the - behavior of the Static Site Editor. -- [Static Site Generator configuration files](#static-site-generator-configuration), - such as `data/config.yml`, which configures the Static Site Generator itself. - It also controls the **Edit this page** button when the site is generated. - -### Static Site Editor configuration file - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4267) in GitLab 13.6. - -The `.gitlab/static-site-editor.yml` configuration file contains entries you can -use to customize behavior of the Static Site Editor (SSE). If the file does not exist, -default values which support a default Middleman project configuration are used. -The [Static Site Editor - Middleman](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman) project template generates a file pre-populated with these defaults. - -To customize the behavior of the SSE, edit `.gitlab/static-site-editor.yml`'s entries, -according to your project's needs. Make sure to respect YAML syntax. - -After the table, see an [example of the SSE configuration file](#gitlabstatic-site-editoryml-example). - -| Entry | GitLab version | Type | Default value | Description | -|---|---|---|---|---| -| `image_upload_path` | [13.6](https://gitlab.com/gitlab-org/gitlab/-/issues/216641) | String | `source/images` | Directory for images uploaded from the WYSIWYG editor. | - -#### `.gitlab/static-site-editor.yml` example - -```yaml -image_upload_path: 'source/images' # Relative path to the project's root. Don't include leading or trailing slashes. -``` - -### Static Site Generator configuration - -The Static Site Editor uses Middleman's configuration file, `data/config.yml` -to customize the behavior of the project itself. This file also controls the -**Edit this page** button, rendered through the file -[`layout.erb`](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman/-/blob/master/source/layouts/layout.erb). - -To [configure the project template to your own project](#set-up-your-project), -you must replace the `<username>` and `<project-name>` in the `data/config.yml` -file with the proper values for your project's path. - -[Other Static Site Generators](#using-other-static-site-generators) used with -the Static Site Editor may use different configuration files or approaches. - -## Using Other Static Site Generators - -Although Middleman is the only Static Site Generator officially supported -by the Static Site Editor, you can configure your project's build and deployment -to use a different Static Site Generator. In this case, use the Middleman layout -as an example, and follow a similar approach to properly render an **Edit this page** -button in your Static Site Generator's layout. - -## Upgrade from GitLab 12.10 to 13.0 - -In GitLab 13.0, we [introduced breaking changes](https://gitlab.com/gitlab-org/gitlab/-/issues/213282) -to the URL structure of the Static Site Editor. Follow the instructions in this -[snippet](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman/snippets/1976539) -to update your project with the 13.0 changes. - -## Limitations - -- The Static Site Editor still cannot be quickly added to existing Middleman sites. - Follow this [epic](https://gitlab.com/groups/gitlab-org/-/epics/2784) for updates. diff --git a/doc/user/project/time_tracking.md b/doc/user/project/time_tracking.md index 5f747d99ce7..0891e02f3f7 100644 --- a/doc/user/project/time_tracking.md +++ b/doc/user/project/time_tracking.md @@ -114,6 +114,18 @@ so if you remove more time than already entered, GitLab ignores the subtraction. To remove all the time spent at once, use the `/remove_time_spent` [quick action](quick_actions.md). +### Delete time spent + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356796) in GitLab 15.0. + +A timelog is a single entry of time spent, either positive or negative. + +Prerequisites: + +- You must be the author of the timelog or have at least the Maintainer role for the project. + +You can [delete timelogs](../../api/graphql/reference/index.md#mutationtimelogdelete) using the GraphQL API. + ## View a time tracking report > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271409) in GitLab 13.12. diff --git a/doc/user/project/web_ide/img/command_palette_v13_6.png b/doc/user/project/web_ide/img/command_palette_v13_6.png Binary files differdeleted file mode 100644 index 54580a79ebd..00000000000 --- a/doc/user/project/web_ide/img/command_palette_v13_6.png +++ /dev/null diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 8f9486633d5..9db30ee2ab6 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -23,13 +23,8 @@ and from merge requests: 1. Select **Open in Web IDE** from the list to display it as the editing option. 1. Select **Open in Web IDE** to open the editor. - *When viewing a merge request* - - 1. Go to your merge request, and select the **Overview** tab. - 1. Scroll to the widgets section, after the merge request description. - 1. Select **Open in Web IDE** if it is visible. - 1. If **Open in Web IDE** is not visible: - 1. Select the **(angle-down)** next to **Open in Gitpod**. - 1. Select **Open in Web IDE** from the list to display it as the editing option. - 1. Select **Open in Web IDE** to open the editor. + 1. Go to your merge request. + 1. In the upper right corner, select **Code**, then select **Open in Gitpod**. ## File finder @@ -52,7 +47,8 @@ Some commands have a keyboard shortcut assigned to them. The command palette displays this shortcut next to each command. You can use this shortcut to invoke the command without having to select it in the command palette. - +For a full list of keyboard shortcuts in the Web IDE, refer to the +[Keyboard shortcuts](../../shortcuts.md#web-ide) list. ## Syntax highlighting diff --git a/doc/user/project/wiki/group.md b/doc/user/project/wiki/group.md index 37f2ef8fc6a..dc448fed970 100644 --- a/doc/user/project/wiki/group.md +++ b/doc/user/project/wiki/group.md @@ -1,8 +1,7 @@ --- stage: Create group: Editor -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" -type: reference, how-to +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Group wikis **(PREMIUM)** @@ -61,6 +60,24 @@ available, you have to: All files in the wiki are available in this Git repository. +## Configure group wiki visibility + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208412) in GitLab 15.0. + +Wikis are enabled by default in GitLab. Group [administrators](../../permissions.md) +can enable or disable a group wiki through the group settings. + +To open group settings: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Expand **Permissions and group features**. +1. Scroll to **Wiki** and select one of these options: + - **Enabled**: Everyone who can access the group can access the wiki. + - **Private**: Only group members can access the wiki. + - **Disabled**: The wiki isn't accessible, and cannot be downloaded. +1. Select **Save changes**. + ## Related topics - [Wiki settings for administrators](../../../administration/wikis/index.md) diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 7d155ea9b06..fe6c7ae62b8 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -275,7 +275,7 @@ can enable or disable a project wiki by following the instructions in Administrators for self-managed GitLab installs can [configure additional wiki settings](../../../administration/wikis/index.md). -You can't disable [group wikis](group.md) from the GitLab user interface. +You can disable group wikis from the [group settings](group.md#configure-group-wiki-visibility) ## Link an external wiki diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md index 03530b59e9b..bfc83aa22f5 100644 --- a/doc/user/project/working_with_projects.md +++ b/doc/user/project/working_with_projects.md @@ -275,6 +275,18 @@ To add a star to a project: - Number of open merge requests. - Number of open issues. +## View personal projects + +Personal projects are projects created under your personal namespace. + +For example, if you create an account with the username `alex`, and create a project +called `my-project` under your username, the project is created at `https://gitlab.example.com/alex/my-project`. + +To view your personal projects: + +1. On the top bar, select **Menu > Projects > Your Projects**. +1. Under **Your projects**, select **Personal**. + ## Delete a project After you delete a project, projects in personal namespaces are deleted immediately. To delay deletion of projects in a group |
