diff options
Diffstat (limited to 'doc/user/packages')
| -rw-r--r-- | doc/user/packages/composer_repository/index.md | 19 | ||||
| -rw-r--r-- | doc/user/packages/conan_repository/index.md | 18 | ||||
| -rw-r--r-- | doc/user/packages/container_registry/index.md | 45 | ||||
| -rw-r--r-- | doc/user/packages/dependency_proxy/index.md | 214 | ||||
| -rw-r--r-- | doc/user/packages/generic_packages/index.md | 14 | ||||
| -rw-r--r-- | doc/user/packages/go_proxy/index.md | 6 | ||||
| -rw-r--r-- | doc/user/packages/index.md | 6 | ||||
| -rw-r--r-- | doc/user/packages/maven_repository/index.md | 16 | ||||
| -rw-r--r-- | doc/user/packages/npm_registry/index.md | 26 | ||||
| -rw-r--r-- | doc/user/packages/nuget_repository/index.md | 8 | ||||
| -rw-r--r-- | doc/user/packages/package_registry/index.md | 12 | ||||
| -rw-r--r-- | doc/user/packages/pypi_repository/index.md | 6 | ||||
| -rw-r--r-- | doc/user/packages/workflows/monorepo.md | 121 | ||||
| -rw-r--r-- | doc/user/packages/workflows/project_registry.md | 120 |
14 files changed, 362 insertions, 269 deletions
diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index 6e5563d0d4e..751915f84a0 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Composer packages in the Package Registry @@ -70,13 +70,16 @@ so that anyone who can access the project can use the package as a dependency. Prerequisites: -- A package in a GitLab repository. +- A package in a GitLab repository. Composer packages should be versioned based on + the [Composer specification](https://getcomposer.org/doc/04-schema.md#version). + If the version is not valid, for example, it has three dots (`1.0.0.0`), an + error (`Validation failed: Version is invalid`) occurs when you publish. - A valid `composer.json` file. - The Packages feature is enabled in a GitLab repository. - The project ID, which is on the project's home page. - A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. - NOTE: **Note:** + NOTE: [Deploy tokens](../../project/deploy_tokens/index.md) are [not yet supported](https://gitlab.com/gitlab-org/gitlab/-/issues/240897) for use with Composer. @@ -116,7 +119,7 @@ You can publish a Composer package to the Package Registry as part of your CI/CD 1. Run the pipeline. -You can view the published package by going to **Packages & Registries > Package Registry** and selecting the **Composer** tab. +To view the published package, go to **Packages & Registries > Package Registry** and select the **Composer** tab. ### Use a CI/CD template @@ -126,7 +129,7 @@ A more detailed Composer CI/CD file is also available as a `.gitlab-ci.yml` temp 1. Above the file list, click **Set up CI/CD**. If this button is not available, select **CI/CD Configuration** and then **Edit**. 1. From the **Apply a template** list, select **Composer**. -CAUTION: **Warning:** +WARNING: Do not save unless you want to overwrite the existing CI/CD file. ## Install a Composer package @@ -139,7 +142,7 @@ Prerequisites: - The group ID, which is on the group's home page. - A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to, at minimum, `read_api`. - NOTE: **Note:** + NOTE: [Deploy tokens](../../project/deploy_tokens/index.md) are [not yet supported](https://gitlab.com/gitlab-org/gitlab/-/issues/240897) for use with Composer. @@ -248,14 +251,14 @@ To install a package: composer config --unset gitlab-domains ``` - NOTE: **Note:** + NOTE: On GitLab.com, Composer uses the GitLab token from `auth.json` as a private token by default. Without the `gitlab-domains` definition in `composer.json`, Composer uses the GitLab token as basic-auth, with the token as a username and a blank password. This results in a 401 error. Output indicates that the package has been successfully installed. -CAUTION: **Important:** +WARNING: Never commit the `auth.json` file to your repository. To install packages from a CI/CD job, consider using the [`composer config`](https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md#authentication) tool with your personal access token stored in a [GitLab CI/CD environment variable](../../../ci/variables/README.md) or in diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index f6f8d6d61b6..73798d363af 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Conan packages in the Package Registry @@ -51,7 +51,7 @@ compilers. This example uses the CMake compiler. To install CMake: -- For Mac, use [homebrew](https://brew.sh/) and run `brew install cmake`. +- For Mac, use [Homebrew](https://brew.sh/) and run `brew install cmake`. - For other operating systems, follow the instructions at [cmake.org](https://cmake.org/install/). When installation is complete, verify you can use CMake in your terminal by @@ -86,7 +86,7 @@ To build a package: conan create . mycompany/beta ``` - NOTE: **Note:** + NOTE: If you use an [instance remote](#add-a-remote-for-your-instance), you must follow a specific [naming convention](#package-recipe-naming-convention-for-instance-remotes). @@ -205,7 +205,7 @@ For example: CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> conan upload Hello/0.1@mycompany/beta --all --remote=gitlab ``` -NOTE: **Note:** +NOTE: Because your authentication with GitLab expires on a regular basis, you may occasionally need to re-enter your personal access token. @@ -221,7 +221,7 @@ In a terminal, run this command: conan remote add_ref Hello/0.1@mycompany/beta gitlab ``` -NOTE: **Note:** +NOTE: The package recipe includes the version, so the default remote for `Hello/0.1@user/channel` doesn't work for `Hello/0.2@user/channel`. @@ -292,7 +292,7 @@ Prerequisites: - [Authentication](#authenticate-to-the-package-registry) with the Package Registry must be configured. -1. In the project where you want to install the package as a dependency, open +1. In the project where you want to install the package as a dependency, open `conanfile.txt`. Or, in the root of your project, create a file called `conanfile.txt`. @@ -316,10 +316,10 @@ Prerequisites: 1. Install the dependencies listed in `conanfile.txt`: ```shell - conan install <options> + conan install .. <options> ``` -NOTE: **Note:** +NOTE: If you try to install the package you just created in this tutorial, the package already exists on your local computer, so this command has no effect. @@ -336,7 +336,7 @@ There are two ways to remove a Conan package from the GitLab Package Registry. You must explicitly include the remote in this command, otherwise the package is removed only from your local system cache. - NOTE: **Note:** + NOTE: This command removes all recipe and binary package files from the Package Registry. diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index 1cb6c951bd9..4e8d105adfa 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # GitLab Container Registry @@ -16,6 +16,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - The group-level Container Registry was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23315) in GitLab 12.10. > - Searching by image repository name was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31322) in GitLab 13.0. +NOTE: +If you pull container images from Docker Hub, you can also use the [GitLab Dependency Proxy](../dependency_proxy/index.md#use-the-dependency-proxy-for-docker-images) to avoid running into rate limits and speed up your pipelines. + With the Docker Container Registry integrated into GitLab, every GitLab project can have its own space to store its Docker images. @@ -127,7 +130,7 @@ To build and push to the Container Registry: docker push registry.example.com/group/project/image ``` -You can also view these commands by going to your project's **Packages & Registries > Container Registry**. +To view these commands, go to your project's **Packages & Registries > Container Registry**. ## Build and push by using GitLab CI/CD @@ -291,7 +294,7 @@ deploy: - master ``` -NOTE: **Note:** +NOTE: This example explicitly calls `docker pull`. If you prefer to implicitly pull the built image using `image:`, and use either the [Docker](https://docs.gitlab.com/runner/executors/docker.html) or [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html) executor, @@ -332,11 +335,11 @@ error during connect: Get http://docker:2376/v1.39/info: dial tcp: lookup docker You can delete images from your Container Registry in multiple ways. -CAUTION: **Warning:** +WARNING: Deleting images is a destructive action and can't be undone. To restore a deleted image, you must rebuild and re-upload it. -NOTE: **Note:** +NOTE: Administrators should review how to [garbage collect](../../../administration/packages/container_registry.md#container-registry-garbage-collection) the deleted images. @@ -368,7 +371,7 @@ information, see the following endpoints: ### Delete images using GitLab CI/CD -CAUTION: **Warning:** +WARNING: GitLab CI/CD doesn't provide a built-in way to remove your images, but this example uses a third-party tool called [reg](https://github.com/genuinetools/reg) that talks to the GitLab Registry API. You are responsible for your own actions. @@ -426,7 +429,7 @@ delete_image: - master ``` -TIP: **Tip:** +NOTE: You can download the latest `reg` release from [the releases page](https://github.com/genuinetools/reg/releases), then update the code example by changing the `REG_SHA256` and `REG_VERSION` variables @@ -489,6 +492,8 @@ Cleanup policies can be run on all projects, with these exceptions: The cleanup policy collects all tags in the Container Registry and excludes tags until only the tags to be deleted remain. +The cleanup policy searches for images based on the tag name. Support for the full path [has not yet been implemented](https://gitlab.com/gitlab-org/gitlab/-/issues/281071), but would allow you to clean up dynamically-named tags. + The cleanup policy: 1. Collects all tags for a given repository in a list. @@ -501,7 +506,7 @@ The cleanup policy: 1. Excludes from the list any tags matching the `name_regex_keep` value (tags to preserve). 1. Finally, the remaining tags in the list are deleted from the Container Registry. -CAUTION: **Warning:** +WARNING: On GitLab.com, the execution time for the cleanup policy is limited, and some of the tags may remain in the Container Registry after the policy runs. The next time the policy runs, the remaining tags are included, so it may take multiple runs for all tags to be deleted. @@ -513,29 +518,31 @@ You can create a cleanup policy in [the API](#use-the-cleanup-policy-api) or the To create a cleanup policy in the UI: 1. For your project, go to **Settings > CI/CD**. -1. Expand the **Cleanup policy for tags** section. +1. Expand the **Clean up image tags** section. 1. Complete the fields. | Field | Description | |---------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------| - | **Cleanup policy** | Turn the policy on or off. | - | **Expiration interval** | How long tags are exempt from being deleted. | - | **Expiration schedule** | How often the policy should run. | - | **Number of tags to retain** | How many tags to _always_ keep for each image. | - | **Tags with names matching this regex pattern expire:** | The regex pattern that determines which tags to remove. This value cannot be blank. For all tags, use `.*`. See other [regex pattern examples](#regex-pattern-examples). | - | **Tags with names matching this regex pattern are preserved:** | The regex pattern that determines which tags to preserve. The `latest` tag is always preserved. For all tags, use `.*`. See other [regex pattern examples](#regex-pattern-examples). | + | **Toggle** | Turn the policy on or off. | + | **Run cleanup** | How often the policy should run. | + | **Keep the most recent** | How many tags to _always_ keep for each image. | + | **Keep tags matching** | The regex pattern that determines which tags to preserve. The `latest` tag is always preserved. For all tags, use `.*`. See other [regex pattern examples](#regex-pattern-examples). | + | **Remove tags older than** | Remove only tags older than X days. | + | **Remove tags matching** | The regex pattern that determines which tags to remove. This value cannot be blank. For all tags, use `.*`. See other [regex pattern examples](#regex-pattern-examples). | -1. Click **Set cleanup policy**. +1. Click **Save**. Depending on the interval you chose, the policy is scheduled to run. -NOTE: **Note:** -If you edit the policy and click **Set cleanup policy** again, the interval is reset. +NOTE: +If you edit the policy and click **Save** again, the interval is reset. ### Regex pattern examples Cleanup policies use regex patterns to determine which tags should be preserved or removed, both in the UI and the API. +Regex patterns are automatically surrounded with `\A` and `\Z` anchors. Do not include any `\A`, `\Z`, `^` or `$` token in the regex patterns as they are not necessary. + Here are examples of regex patterns you may want to use: - Match all tags: @@ -573,7 +580,7 @@ Examples: - Select all tags, keep at least 1 tag per image, clean up any tag older than 14 days, run once a month, preserve any images with the name `master` and the policy is enabled: ```shell - curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":"","name_regex_delete":".*","name_regex_keep":".*-master"}}' 'https://gitlab.example.com/api/v4/projects/2' + curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":"","name_regex_delete":".*","name_regex_keep":".*-master"}}' "https://gitlab.example.com/api/v4/projects/2" ``` See the API documentation for further details: [Edit project](../../../api/projects.md#edit-project). diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index e1fae770a5d..fbede6d13b7 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -1,13 +1,15 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Dependency Proxy > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7934) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.11. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/273655) to [GitLab Core](https://about.gitlab.com/pricing/) in GitLab 13.6. +> - [Support for private groups](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) in [GitLab Core](https://about.gitlab.com/pricing/) 13.7. +> - Anonymous access to images in public groups is no longer available starting in [GitLab Core](https://about.gitlab.com/pricing/) 13.7. The GitLab Dependency Proxy is a local proxy you can use for your frequently-accessed upstream images. @@ -15,11 +17,14 @@ upstream images. In the case of CI/CD, the Dependency Proxy receives a request and returns the upstream image from a registry, acting as a pull-through cache. -## Prerequisites +NOTE: +The Dependency Proxy is not compatible with Docker version 20.x and later. +If you are using the Dependency Proxy, Docker version 19.x.x is recommended until +[issue #290944](https://gitlab.com/gitlab-org/gitlab/-/issues/290944) is resolved. -To use the Dependency Proxy: +## Prerequisites -- Your group must be public. Authentication for private groups is [not supported yet](https://gitlab.com/gitlab-org/gitlab/-/issues/11582). +The Dependency Proxy must be [enabled by an administrator](../../../administration/packages/dependency_proxy.md). ### Supported images and packages @@ -32,6 +37,11 @@ The following images and packages are supported. For a list of planned additions, view the [direction page](https://about.gitlab.com/direction/package/dependency_proxy/#top-vision-items). +## Enable the Dependency Proxy + +The Dependency Proxy is disabled by default. +[Learn how an administrator can enable it](../../../administration/packages/dependency_proxy.md). + ## View the Dependency Proxy To view the Dependency Proxy: @@ -42,16 +52,136 @@ The Dependency Proxy is not available for projects. ## Use the Dependency Proxy for Docker images -CAUTION: **Important:** -In some specific storage configurations, an issue occurs and container images are not pulled correctly from the cache. The problem occurs when an image is located in object storage. The proxy looks for it locally and fails to find it. View [issue #208080](https://gitlab.com/gitlab-org/gitlab/-/issues/208080) for details. - You can use GitLab as a source for your Docker images. Prerequisites: - Your images must be stored on [Docker Hub](https://hub.docker.com/). -- Docker Hub must be available. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/241639) - for progress on accessing images when Docker Hub is down. + +### Authenticate with the Dependency Proxy + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) in [GitLab Core](https://about.gitlab.com/pricing/) 13.7. +> - It's [deployed behind a feature flag](../../feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](../../../administration/packages/dependency_proxy.md#disabling-authentication). **(CORE ONLY)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. +The requirement to authenticate is a breaking change added in 13.7. An [administrator can temporarily +disable it](../../../administration/packages/dependency_proxy.md#disabling-authentication) if it +has disrupted your existing Dependency Proxy usage. + +Because the Dependency Proxy is storing Docker images in a space associated with your group, +you must authenticate against the Dependency Proxy. + +Follow the [instructions for using images from a private registry](../../../ci/docker/using_docker_images.md#define-an-image-from-a-private-container-registry), +but instead of using `registry.example.com:5000`, use your GitLab domain with no port `gitlab.example.com`. + +For example, to manually log in: + +```shell +docker login gitlab.example.com --username my_username --password my_password +``` + +You can authenticate using: + +- Your GitLab username and password. +- A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `read_registry` and `write_registry`. + +#### Authenticate within CI/CD + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/280582) in GitLab 13.7. + +To work with the Dependency Proxy in [GitLab CI/CD](../../../ci/README.md), you can use: + +- `CI_DEPENDENCY_PROXY_USER`: A CI user for logging in to the Dependency Proxy. +- `CI_DEPENDENCY_PROXY_PASSWORD`: A CI password for logging in to the Dependency Proxy. +- `CI_DEPENDENCY_PROXY_SERVER`: The server for logging in to the Dependency Proxy. +- `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX`: The image prefix for pulling images through the Dependency Proxy. + +This script shows how to use these variables to log in and pull an image from the Dependency Proxy: + +```yaml +# .gitlab-ci.yml + +dependency-proxy-pull-master: + # Official docker image. + image: docker:latest + stage: build + services: + - docker:dind + before_script: + - docker login -u "$CI_DEPENDENCY_PROXY_USER" -p "$CI_DEPENDENCY_PROXY_PASSWORD" "$CI_DEPENDENCY_PROXY_SERVER" + script: + - docker pull "$CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX"/alpine:latest +``` + +`CI_DEPENDENCY_PROXY_SERVER` and `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` include the server port. So if you use `CI_DEPENDENCY_PROXY_SERVER` to log in, for example, you must explicitly include the port in your pull command and vice-versa: + +```shell +docker pull gitlab.example.com:443/my-group/dependency_proxy/containers/alpine:latest +``` + +You can also use [custom environment variables](../../../ci/variables/README.md#custom-environment-variables) to store and access your personal access token or other valid credentials. + +##### Authenticate with `DOCKER_AUTH_CONFIG` + +You can use the Dependency Proxy to pull your base image. + +1. [Create a `DOCKER_AUTH_CONFIG` environment variable](../../../ci/docker/using_docker_images.md#define-an-image-from-a-private-container-registry). +1. Get credentials that allow you to log into the Dependency Proxy. +1. Generate the version of these credentials that will be used by Docker: + + ```shell + # The use of "-n" - prevents encoding a newline in the password. + echo -n "my_username:my_password" | base64 + + # Example output to copy + bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= + ``` + + This can also be other credentials such as: + + ```shell + echo -n "my_username:personal_access_token" | base64 + echo -n "deploy_token_username:deploy_token" | base64 + ``` + +1. Create a [custom environment variables](../../../ci/variables/README.md#custom-environment-variables) +named `DOCKER_AUTH_CONFIG` with a value of: + + ```json + { + "auths": { + "https://gitlab.example.com": { + "auth": "(Base64 content from above)" + } + } + } + ``` + + To use `$CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` when referencing images, you must explicitly include the port in your `DOCKER_AUTH_CONFIG` value: + + ```json + { + "auths": { + "https://gitlab.example.com:443": { + "auth": "(Base64 content from above)" + } + } + } + ``` + +1. Now reference the Dependency Proxy in your base image: + + ```yaml + # .gitlab-ci.yml + image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/node:latest + ... + ``` + +### Store a Docker image in Dependency Proxy cache To store a Docker image in Dependency Proxy storage: @@ -89,3 +219,69 @@ stored. To reclaim disk space used by image blobs that are no longer needed, use the [Dependency Proxy API](../../../api/dependency_proxy.md). + +## Docker Hub rate limits and the Dependency Proxy + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241639) in [GitLab Core](https://about.gitlab.com/pricing/) 13.7. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +Watch how to [use the Dependency Proxy to help avoid Docker Hub rate limits](https://youtu.be/Nc4nUo7Pq08). + +In November 2020, Docker introduced +[rate limits on pull requests from Docker Hub](https://docs.docker.com/docker-hub/download-rate-limit/). +If your GitLab [CI/CD configuration](../../../ci/README.md) uses +an image from Docker Hub, each time a job runs, it may count as a pull request. +To help get around this limit, you can pull your image from the Dependency Proxy cache instead. + +When you pull an image (by using a command like `docker pull` or, in a `.gitlab-ci.yml` +file, `image: foo:latest`), the Docker client makes a collection of requests: + +1. The image manifest is requested. The manifest contains information about + how to build the image. +1. Using the manifest, the Docker client requests a collection of layers, also + known as blobs, one at a time. + +The Docker Hub rate limit is based on the number of GET requests for the manifest. The Dependency Proxy +caches both the manifest and blobs for a given image, so when you request it again, +Docker Hub does not have to be contacted. + +### How does GitLab know if a cached tagged image is stale? + +If you are using an image tag like `alpine:latest`, the image changes +over time. Each time it changes, the manifest contains different information about which +blobs to request. The Dependency Proxy does not pull a new image each time the +manifest changes; it checks only when the manifest becomes stale. + +Docker does not count HEAD requests for the image manifest towards the rate limit. +You can make a HEAD request for `alpine:latest`, view the digest (checksum) +value returned in the header, and determine if a manifest has changed. + +The Dependency Proxy starts all requests with a HEAD request. If the manifest +has become stale, only then is a new image pulled. + +For example, if your pipeline pulls `node:latest` every five +minutes, the Dependency Proxy caches the entire image and only updates it if +`node:latest` changes. So instead of having 360 requests for the image in six hours +(which exceeds the Docker Hub rate limit), you only have one pull request, unless +the manifest changed during that time. + +### Check your Docker Hub rate limit + +If you are curious about how many requests to Docker Hub you have made and how +many remain, you can run these commands from your runner, or even in a CI/CD +script: + +```shell +# Note, you must have jq installed to run this command +TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq --raw-output .token) && curl --head --header "Authorization: Bearer $TOKEN" "https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest" 2>&1 | grep RateLimit +... +``` + +The output is something like: + +```shell +RateLimit-Limit: 100;w=21600 +RateLimit-Remaining: 98;w=21600 +``` + +This example shows the total limit of 100 pulls in six hours, with 98 pulls remaining. diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index f489c7c800f..c9859840c9c 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # GitLab Generic Packages Repository **(CORE)** @@ -13,7 +13,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - It's recommended for production use. > - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-generic-packages-in-the-package-registry). -CAUTION: **Warning:** +WARNING: This feature might not be available to you. Check the **version history** note above for details. Publish generic files, like release binaries, in your project’s Package Registry. Then, install the packages whenever you need to use them as a dependency. @@ -39,7 +39,7 @@ PUT /projects/:id/packages/generic/:package_name/:package_version/:file_name | -------------------| --------------- | ---------| -------------------------------------------------------------------------------------------------------------------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/README.md#namespaced-path-encoding). | | `package_name` | string | yes | The package name. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), dots (`.`), hyphens (`-`), or underscores (`_`). -| `package_version` | string | yes | The package version. It can contain only numbers (`0-9`), and dots (`.`). Must be in the format of `X.Y.Z`, i.e. should match `/\A\d+\.\d+\.\d+\z/` regular expresion. +| `package_version` | string | yes | The package version. It can contain only numbers (`0-9`), and dots (`.`). Must be in the format of `X.Y.Z`, i.e. should match `/\A\d+\.\d+\.\d+\z/` regular expression. | `file_name` | string | yes | The file name. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), dots (`.`), hyphens (`-`), or underscores (`_`). Provide the file context in the request body. @@ -49,7 +49,7 @@ Example request: ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" \ --upload-file path/to/file.txt \ - https://gitlab.example.com/api/v4/projects/24/packages/generic/my_package/0.0.1/file.txt + "https://gitlab.example.com/api/v4/projects/24/packages/generic/my_package/0.0.1/file.txt" ``` Example response: @@ -79,13 +79,13 @@ GET /projects/:id/packages/generic/:package_name/:package_version/:file_name | `package_version` | string | yes | The package version. | | `file_name` | string | yes | The file name. | -The file context is served in the response body. The response content type will be `application/octet-stream`. +The file context is served in the response body. The response content type is `application/octet-stream`. Example request that uses a personal access token: ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" \ - https://gitlab.example.com/api/v4/projects/24/packages/generic/my_package/0.0.1/file.txt + "https://gitlab.example.com/api/v4/projects/24/packages/generic/my_package/0.0.1/file.txt" ``` ## Publish a generic package by using CI/CD @@ -105,7 +105,7 @@ stages: upload: stage: upload script: - - 'curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file path/to/file.txt ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/my_package/0.0.1/file.txt' + - 'curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file path/to/file.txt "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/my_package/0.0.1/file.txt"' download: stage: download diff --git a/doc/user/packages/go_proxy/index.md b/doc/user/packages/go_proxy/index.md index 81edc3afcfd..87cd4a4a9a6 100644 --- a/doc/user/packages/go_proxy/index.md +++ b/doc/user/packages/go_proxy/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Go proxy for GitLab @@ -45,7 +45,7 @@ Feature.enable(:go_proxy, Project.find(1)) Feature.disable(:go_proxy, Project.find(2)) ``` -NOTE: **Note:** +NOTE: Even if it's enabled, GitLab doesn't display Go modules in the **Package Registry**. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/213770) for details. @@ -159,7 +159,7 @@ later, can be written with `go env -w <var>=<value>`. For example, Go modules and module versions are defined by source repositories, such as Git, SVN, and Mercurial. A module is a repository that contains `go.mod` and Go -files. Module versions are defined by VCS tags. +files. Module versions are defined by version control system (VCS) tags. To publish a module, push `go.mod` and source files to a VCS repository. To publish a module version, push a VCS tag. diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 83d179cbc9c..9da14842845 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Packages & Registries @@ -32,8 +32,8 @@ You can also use the [API](../../api/packages.md) to administer the Package Regi ## Accepting contributions -The below table lists formats that are not supported, but are accepting Community contributions for. Consider contributing to GitLab. This [development documentation](../../development/packages.md) will -guide you through the process. +The below table lists formats that are not supported, but are accepting Community contributions for. Consider contributing to GitLab. This [development documentation](../../development/packages.md) +guides you through the process. | Format | Status | | ------ | ------ | diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 5d0d64b310d..e0f5a400977 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Maven packages in the Package Repository @@ -361,7 +361,7 @@ To use the GitLab endpoint for Maven packages, choose an option: - **Instance-level**: Use when you have many Maven packages in different GitLab groups or in their own namespace. -The option you choose determines the settings you'll add to your `pom.xml` file. +The option you choose determines the settings you add to your `pom.xml` file. In all cases, to publish a package, you need: @@ -653,7 +653,7 @@ The message should show that the package is downloading from the Package Registr Downloading from gitlab-maven: http://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.pom ``` -TIP: **Tip:** +NOTE: In the GitLab UI, on the Package Registry page for Maven, you can view and copy these commands. ### Use Gradle @@ -707,23 +707,23 @@ You can create a new package each time the `master` branch is updated. 1. Make sure your `pom.xml` file includes the following. You can either let Maven use the CI environment variables, as shown in this example, - or you can hard code your project's ID. + or you can hard code your server's hostname and project's ID. ```xml <repositories> <repository> <id>gitlab-maven</id> - <url>https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> + <url>${env.CI_SERVER_URL}/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> </repository> </repositories> <distributionManagement> <repository> <id>gitlab-maven</id> - <url>https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> + <url>${env.CI_SERVER_URL}/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> </repository> <snapshotRepository> <id>gitlab-maven</id> - <url>https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> + <url>${env.CI_SERVER_URL}/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> </snapshotRepository> </distributionManagement> ``` @@ -793,7 +793,7 @@ mvn deploy \ -Dorg.slf4j.simpleLogger.log.org.apache.maven.wagon.providers.http.httpclient.wire=trace ``` -CAUTION: **Caution:** +WARNING: When you set these options, all network requests are logged and a large amount of output is generated. ### Useful Maven command-line options diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index feb797240f4..51b41b842fa 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # NPM packages in the Package Registry @@ -12,7 +12,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Publish NPM packages in your project's Package Registry. Then install the packages whenever you need to use them as a dependency. -Only [scoped](https://docs.npmjs.com/misc/scope) packages are supported. +Only [scoped](https://docs.npmjs.com/misc/scope/) packages are supported. ## Build an NPM package @@ -25,7 +25,7 @@ the [next section](#authenticate-to-the-package-registry). ### Install NPM Install Node.js and NPM in your local development environment by following -the instructions at [npmjs.com](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm). +the instructions at [npmjs.com](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm/). When installation is complete, verify you can use NPM in your terminal by running: @@ -102,10 +102,11 @@ To authenticate to the Package Registry, you must use one of the following: - It's not recommended, but you can use [OAuth tokens](../../../api/oauth2.md#resource-owner-password-credentials-flow). Standard OAuth tokens cannot authenticate to the GitLab NPM Registry. You must use a personal access token with OAuth headers. - A [CI job token](#authenticate-with-a-ci-job-token). +- Your NPM package name must be in the format of [@scope:package-name](#package-naming-convention). It must match exactly, including the case. ### Authenticate with a personal access token or deploy token -To authenticate with the Package Registry, you will need a [personal access token](../../profile/personal_access_tokens.md) or [deploy token](../../project/deploy_tokens/index.md). +To authenticate with the Package Registry, you need a [personal access token](../../profile/personal_access_tokens.md) or [deploy token](../../project/deploy_tokens/index.md). #### Project-level NPM endpoint @@ -228,7 +229,7 @@ This regex allows almost all of the characters that NPM allows, with a few excep The regex also allows for capital letters, while NPM does not. Capital letters are needed because the scope must be identical to the root namespace of the project. -CAUTION: **Caution:** +WARNING: When you update the path of a user or group, or transfer a subgroup or project, you must remove any NPM packages first. You cannot update the root namespace of a project with NPM packages. Make sure you update your `.npmrc` files to follow @@ -240,6 +241,7 @@ Prerequisites: - [Authenticate](#authenticate-to-the-package-registry) to the Package Registry. - Set a [project-level NPM endpoint](#use-the-gitlab-endpoint-for-npm-packages). +- Your NPM package name must be in the format of [@scope:package-name](#package-naming-convention). It must match exactly, including the case. To upload an NPM package to your project, run this command: @@ -277,6 +279,10 @@ deploy: - npm publish ``` +See the +[Publish NPM packages to the GitLab Package Registry using semantic-release](../../../ci/examples/semantic-release.md) +step-by-step guide and demo project for a complete example. + ## Publishing packages with the same name or version You cannot publish a package if a package of the same name and version already exists. @@ -356,7 +362,7 @@ In GitLab 12.6 and later, packages published to the Package Registry expose the > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9425) in GitLab Premium 12.8. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -You can add [distribution tags](https://docs.npmjs.com/cli/dist-tag) to newly-published packages. +You can add [distribution tags](https://docs.npmjs.com/cli/dist-tag/) to newly-published packages. Tags are optional and can be assigned to only one package at a time. When you publish a package without a tag, the `latest` tag is added by default. @@ -451,3 +457,11 @@ If you get this error, ensure that: - The scoped packages URL includes a trailing slash: - Correct: `//gitlab.example.com/api/v4/packages/npm/` - Incorrect: `//gitlab.example.com/api/v4/packages/npm` + +### `npm publish` returns `npm ERR! 400 Bad Request` + +If you get this error, your package name may not meet the +[@scope:package-name package naming convention](#package-naming-convention). + +Ensure the name meets the convention exactly, including the case. +Then try to publish again. diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 2b61c4a6c28..bdf50ecef0b 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # NuGet packages in the Package Registry @@ -181,7 +181,7 @@ nuget push <package_file> -Source <source_name> Prerequisite: -[A NuGet package created with .NET CLI](https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package-dotnet-cli). +- [A NuGet package created with .NET CLI](https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package-dotnet-cli). Publish a package by running this command: @@ -233,7 +233,7 @@ updated: ### Install a package with the NuGet CLI -CAUTION: **Warning:** +WARNING: By default, `nuget` checks the official source at `nuget.org` first. If you have a NuGet package in the Package Registry with the same name as a package at `nuget.org`, you must specify the source name to install the correct package. @@ -253,7 +253,7 @@ nuget install <package_id> -OutputDirectory <output_directory> \ ### Install a package with the .NET CLI -CAUTION: **Warning:** +WARNING: If you have a package in the Package Registry with the same name as a package at a different source, verify the order in which `dotnet` checks sources during install. This is defined in the `nuget.config` file. diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index 56fd4a02ba0..5876ef19ad9 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Package Registry @@ -33,20 +33,19 @@ CI/CD templates, which you can use to get started, are in [this repo](https://gi Learn more about using CI/CD to build: -- [Maven packages](../maven_repository/index.md#create-maven-packages-with-gitlab-cicd) -- [NPM packages](../npm_registry/index.md#publish-an-npm-package-by-using-cicd) - [Composer packages](../composer_repository/index.md#publish-a-composer-package-by-using-cicd) -- [NuGet packages](../nuget_repository/index.md#publish-a-nuget-package-by-using-cicd) - [Conan packages](../conan_repository/index.md#publish-a-conan-package-by-using-cicd) - [Generic packages](../generic_packages/index.md#publish-a-generic-package-by-using-cicd) +- [Maven packages](../maven_repository/index.md#create-maven-packages-with-gitlab-cicd) +- [NPM packages](../npm_registry/index.md#publish-an-npm-package-by-using-cicd) +- [NuGet packages](../nuget_repository/index.md#publish-a-nuget-package-by-using-cicd) If you use CI/CD to build a package, extended activity information is displayed when you view the package details:  -When using Maven and NPM, you can view which pipeline published the package, and -the commit and user who triggered it. +You can view which pipeline published the package, and the commit and user who triggered it. However, the history is limited to five updates of a given package. ## Download a package @@ -95,4 +94,3 @@ The **Packages & Registries > Package Registry** entry is removed from the sideb Learn how to use the GitLab Package Registry to build your own custom package workflow. - [Use a project as a package registry](../workflows/project_registry.md) to publish all of your packages to one project. -- Publish multiple different packages from one [monorepo project](../workflows/monorepo.md). diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 83b29d5f53a..e78224f89d1 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # PyPI packages in the Package Registry @@ -237,11 +237,11 @@ When publishing packages, note that: - The maximum allowed size is 50 MB. - You can't upload the same version of a package multiple times. If you try, - you'll receive the error `Validation failed: File name has already been taken`. + you receive the error `Validation failed: File name has already been taken`. ### Ensure your version string is valid -If your version string (for example, `0.0.1`) isn't valid, it will be rejected. +If your version string (for example, `0.0.1`) isn't valid, it gets rejected. GitLab uses the following regex to validate the version string. ```ruby diff --git a/doc/user/packages/workflows/monorepo.md b/doc/user/packages/workflows/monorepo.md index 1e375dffe7e..abba9df6ec2 100644 --- a/doc/user/packages/workflows/monorepo.md +++ b/doc/user/packages/workflows/monorepo.md @@ -1,120 +1,9 @@ --- -stage: Package -group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: '../npm_registry/index.md' +disqus_identifier: 'https://docs.gitlab.com/ee/user/packages/workflows/monorepo.html' --- -# Monorepo package management workflows +This document was moved to [another location](../npm_registry/index.md). -Oftentimes, one project or Git repository may contain multiple different -sub-projects or submodules that all get packaged and published individually. - -## Publishing different packages to the parent project - -The number and name of packages you can publish to one project is not limited. -You can accomplish this by setting up different configuration files for each -package. See the documentation for the package manager of your choice since -each has its own specific files and instructions to follow to publish -a given package. - -Here, we take a walk through how to do this with [NPM](../npm_registry/index.md). - -Let us say we have a project structure like so: - -```plaintext -MyProject/ - |- src/ - | |- components/ - | |- Foo/ - |- package.json -``` - -`MyProject` is the parent project, which contains a sub-project `Foo` in the -`components` directory. We would like to publish packages for both `MyProject` -as well as `Foo`. - -Following the instructions in the -[GitLab NPM registry documentation](../npm_registry/index.md), -publishing `MyProject` consists of modifying the `package.json` file with a -`publishConfig` section, as well as either modifying your local NPM configuration with -CLI commands like `npm config set`, or saving a `.npmrc` file in the root of the -project specifying these configuration settings. - -If you follow the instructions you can publish `MyProject` by running -`npm publish` from the root directory. - -Publishing `Foo` is almost exactly the same, you simply have to follow the steps -while in the `Foo` directory. `Foo` needs its own `package.json` file, -which can be added manually or using `npm init`. It also needs its own -configuration settings. Since you are publishing to the same place, if you -used `npm config set` to set the registry for the parent project, then no -additional setup is necessary. If you used a `.npmrc` file, you need an -additional `.npmrc` file in the `Foo` directory (be sure to add `.npmrc` files -to the `.gitignore` file or use environment variables in place of your access -tokens to prevent them from being exposed). It can be identical to the -one you used in `MyProject`. You can now run `npm publish` from the `Foo` -directory and you can publish `Foo` separately from `MyProject` - -A similar process could be followed for Conan packages, instead of dealing with -`.npmrc` and `package.json`, you just deal with `conanfile.py` in -multiple locations within the project. - -## Publishing to other projects - -A package is associated with a project on GitLab, but the package does not -need to be associated with the code in that project. Notice when configuring -NPM or Maven, you only use the `Project ID` to set the registry URL that the -package is to be uploaded to. If you set this to any project that you have -access to and update any other configuration similarly depending on the package type, -your packages are published to that project. This means you can publish -multiple packages to one project, even if their code does not exist in the same -place. See the [project registry workflow documentation](project_registry.md) -for more details. - -## CI workflows for automating packaging - -CI pipelines open an entire world of possibilities for dealing with the patterns -described in the previous sections. A common desire would be to publish -specific packages only if changes were made to those directories. - -Using the example project above, this `gitlab-ci.yml` file publishes -`Foo` anytime changes are made to the `Foo` directory on the `master` branch, -and publish `MyPackage` anytime changes are made to anywhere _except_ the `Foo` -directory on the `master` branch. - -```yaml -image: node:latest - -stages: - - build - -build-foo-package: - stage: build - variables: - PACKAGE: "Foo" - script: - - cd src/components/Foo - - echo "Building $PACKAGE" - - npm publish - only: - refs: - - master - - merge_requests - changes: - - "src/components/Foo/**/*" - -build-my-project-package: - stage: build - variables: - PACKAGE: "MyPackage" - script: - - echo "Building $PACKAGE" - - npm publish - only: - refs: - - master - - merge_requests - except: - changes: - - "src/components/Foo/**/*" -``` +<!-- This redirect file can be deleted after <2021-02-14>. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/user/packages/workflows/project_registry.md b/doc/user/packages/workflows/project_registry.md index a8972f05acd..aea1238b9da 100644 --- a/doc/user/packages/workflows/project_registry.md +++ b/doc/user/packages/workflows/project_registry.md @@ -1,98 +1,84 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Project as a package registry +# Store all of your packages in one GitLab project -Using the features of the package registry, it is possible to use one project to store all of your packages. +You can store all of your packages in one project's Package Registry. Rather than using +a GitLab repository to store code, you can use the repository to store all your packages. +Then you can configure your remote repositories to point to the project in GitLab. -This guide mirrors the creation of [this package registry](https://gitlab.com/sabrams/my-package-registry). +You might want to do this because: -For the video version, see [Single Project Package Registry Demo](https://youtu.be/ui2nNBwN35c). - -## How does this work? - -You might be wondering "how is it possible to upload two packages from different codebases to the same project on GitLab?". - -It is easy to forget that a package on GitLab belongs to a project, but a project does not have to be a code repository. -The code used to build your packages can be stored anywhere - maybe it is another project on GitLab, or maybe a completely -different system altogether. All that matters is that when you configure your remote repositories for those packages, you -point them at the same project on GitLab. - -## Why would I do this? - -There are a few reasons you might want to publish all your packages to one project on GitLab: - -1. You want to publish your packages on GitLab, but to a project that is different from where your code is stored. -1. You would like to group packages together in ways that make sense for your usage (such as all NPM packages in one project, - all packages being used by a specific department in one project, or all private packages in one project) -1. You want to use one remote for all of your packages when installing them into other projects. -1. You would like to migrate your packages to a single place on GitLab from a third-party package registry and do not - want to worry about setting up separate projects for each package. -1. You want to have your CI pipelines build all of your packages to one project so the individual responsible for -validating packages can manage them all in one place. +- You want to publish your packages in GitLab, but to a different project from where your code is stored. +- You want to group packages together in one project. For example, you might want to put all NPM packages, + or all packages for a specific department, or all private packages in the same project. +- When you install packages for other projects, you want to use one remote. +- You want to migrate your packages from a third-party package registry to a single place in GitLab and do not + want to worry about setting up separate projects for each package. +- You want to have your CI/CD pipelines build all of your packages to one project, so the person responsible for + validating packages can manage them all in one place. ## Example walkthrough -There is no functionality specific to this feature. All we are doing is taking advantage of functionality available in each -of the package management systems to publish packages of different types to the same place. - -Let's take a look at how you might create a public place to hold all of your public packages. - -### Create a project - -First, create a new project on GitLab. It does not have to have any code or content. Make note of the project ID -displayed on the project overview page for use later in this process. - -### Create an access token +No functionality is specific to this feature. Instead, we're taking advantage of the functionality +of each package management system to publish different package types to the same place. -All of the package repositories available on the GitLab package registry are accessible using [GitLab personal access -tokens](../../profile/personal_access_tokens.md). +- <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> + Watch a video of how to add Maven, NPM, and Conan packages to [the same project](https://youtu.be/ui2nNBwN35c). +- [View an example project](https://gitlab.com/sabrams/my-package-registry/-/packages). -While using CI, you can alternatively use CI job tokens (`CI_JOB_TOKEN`) to authenticate. +## Store different package types in one GitLab project -### Configure your local project for the GitLab registry and upload +Let's take a look at how you might create a public place to hold all of your public packages. -There are many ways to use this feature. You can upload all types of packages to the same project, -split things up based on package type, or package visibility level. +1. Create a new project in GitLab. The project doesn't require any code or content. Note the project ID + that's displayed on the project overview page. +1. Create an access token. All package types in the Package Registry are accessible by using + [GitLab personal access tokens](../../profile/personal_access_tokens.md). + If you're using CI/CD, you can use CI job tokens (`CI_JOB_TOKEN`) to authenticate. +1. Configure your local project and publish the package. -The purpose of this tutorial is to demonstrate the root idea that one project can hold many unrelated -packages, and to allow you to discover the best way to use this functionality yourself. +You can upload all types of packages to the same project, or +split things up based on package type or package visibility level. -#### NPM +### NPM -If you are using NPM, this involves creating an `.npmrc` file and adding the appropriate URL for uploading packages -to your project using your project ID, then adding a section to your `package.json` file with a similar URL. +If you're using NPM, create an `.npmrc` file. Add the appropriate URL for publishing +packages to your project. Finally, add a section to your `package.json` file. -Follow -the instructions in the [GitLab NPM Registry documentation](../npm_registry/index.md#authenticate-to-the-package-registry). After -you do this, you can push your NPM package to your project using `npm publish`, as described in the -[publishing packages](../npm_registry/index.md#publish-an-npm-package) section of the docs. +Follow the instructions in the +[GitLab NPM Registry documentation](../npm_registry/index.md#authenticate-to-the-package-registry). After +you do this, you can publish your NPM package to your project using `npm publish`, as described in the +[publishing packages](../npm_registry/index.md#publish-an-npm-package) section. -#### Maven +### Maven -If you are using Maven, this involves updating your `pom.xml` file with distribution sections, including the +If you are using Maven, you update your `pom.xml` file with distribution sections. These updates include the appropriate URL for your project, as described in the [GitLab Maven Repository documentation](../maven_repository/index.md#project-level-maven-endpoint). Then, you need to add a `settings.xml` file and [include your access token](../maven_repository/index.md#authenticate-with-a-personal-access-token-in-maven). -Now you can [deploy Maven packages](../maven_repository/index.md#publish-a-package) to your project. +Now you can [publish Maven packages](../maven_repository/index.md#publish-a-package) to your project. + +### Conan -#### Conan +For Conan, you need to add GitLab as a Conan registry remote. Follow the instructions in the +[GitLab Conan Repository docs](../conan_repository/index.md#add-the-package-registry-as-a-conan-remote). +Then, create your package using the plus-separated (`+`) project path as your Conan user. For example, +if your project is located at `https://gitlab.com/foo/bar/my-proj`, +[create your Conan package](../conan_repository/index.md) using `conan create . foo+bar+my-proj/channel`. +`channel` is your package channel (such as `stable` or `beta`). -For Conan, first you need to add GitLab as a Conan registry remote. Follow the instructions in the [GitLab Conan Repository docs](../conan_repository/index.md#add-the-package-registry-as-a-conan-remote) -to do so. Then, create your package using the plus-separated (`+`) project path as your Conan user. For example, -if your project is located at `https://gitlab.com/foo/bar/my-proj`, then you can [create your Conan package](../conan_repository/index.md) -using `conan create . foo+bar+my-proj/channel`, where `channel` is your package channel (such as `stable` or `beta`). After your package -is created, you are ready to [upload your package](../conan_repository/index.md#publish-a-conan-package) depending on your final package recipe. For example: +After you create your package, you're ready to [publish your package](../conan_repository/index.md#publish-a-conan-package), +depending on your final package recipe. For example: ```shell CONAN_LOGIN_USERNAME=<gitlab-username> CONAN_PASSWORD=<personal_access_token> conan upload MyPackage/1.0.0@foo+bar+my-proj/channel --all --remote=gitlab ``` -#### Composer - -It is currently not possible to publish a Composer package to a project that is different from where its code resides. +### All other package types -If you attempt to publish a Composer package to a different project, you get a `404 Branch Not Found` -or `404 Tag Not Found` error. +[All package types supported by GitLab](../index.md) can be published in +the same GitLab project. In previous releases, not all package types could +be published in the same project. |