diff options
Diffstat (limited to 'doc/user/packages')
17 files changed, 1142 insertions, 1124 deletions
diff --git a/doc/user/packages/composer_repository/img/project_id_v13_5.png b/doc/user/packages/composer_repository/img/project_id_v13_5.png Binary files differdeleted file mode 100644 index 45861b6ff1b..00000000000 --- a/doc/user/packages/composer_repository/img/project_id_v13_5.png +++ /dev/null diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index d4fb662c3a6..6e5563d0d4e 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -77,7 +77,7 @@ Prerequisites: - A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. NOTE: **Note:** - [Deploy tokens](./../../project/deploy_tokens/index.md) are + [Deploy tokens](../../project/deploy_tokens/index.md) are [not yet supported](https://gitlab.com/gitlab-org/gitlab/-/issues/240897) for use with Composer. To publish the package: @@ -140,7 +140,7 @@ Prerequisites: - A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to, at minimum, `read_api`. NOTE: **Note:** - [Deploy tokens](./../../project/deploy_tokens/index.md) are + [Deploy tokens](../../project/deploy_tokens/index.md) are [not yet supported](https://gitlab.com/gitlab-org/gitlab/-/issues/240897) for use with Composer. To install a package: diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index 10c820a86be..f6f8d6d61b6 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -12,23 +12,27 @@ info: To determine the technical writer assigned to the Stage/Group associated w Publish Conan packages in your project’s Package Registry. Then install the packages whenever you need to use them as a dependency. -To publish Conan packages to the Package Registry, add the -Package Registry as a remote and authenticate with it. +To publish Conan packages to the Package Registry, add the Package Registry as a +remote and authenticate with it. -Then you can run `conan` commands and publish your package to the Package Registry. +Then you can run `conan` commands and publish your package to the +Package Registry. ## Build a Conan package -This section explains how to install Conan and build a package for your C/C++ project. +This section explains how to install Conan and build a package for your C/C++ +project. -If you already use Conan and know how to build your own packages, go to the [next section](#add-the-package-registry-as-a-conan-remote). +If you already use Conan and know how to build your own packages, go to the +[next section](#add-the-package-registry-as-a-conan-remote). ### Install Conan -Download the Conan package manager to your local development environment by following -the instructions at [conan.io](https://conan.io/downloads.html). +Download the Conan package manager to your local development environment by +following the instructions at [conan.io](https://conan.io/downloads.html). -When installation is complete, verify you can use Conan in your terminal by running: +When installation is complete, verify you can use Conan in your terminal by +running: ```shell conan --version @@ -42,15 +46,16 @@ Conan version 1.20.5 ### Install CMake -When you develop with C++ and Conan, you have a range of compilers to choose from. -This example uses the CMake compiler. +When you develop with C++ and Conan, you can select from many available +compilers. This example uses the CMake compiler. To install CMake: - For Mac, use [homebrew](https://brew.sh/) and run `brew install cmake`. - For other operating systems, follow the instructions at [cmake.org](https://cmake.org/install/). -When installation is complete, verify you can use CMake in your terminal by running: +When installation is complete, verify you can use CMake in your terminal by +running: ```shell cmake --version @@ -60,8 +65,8 @@ The CMake version is printed in the output. ### Create a project -To test the Package Registry, you need a C++ project. If you don't already have one, you can clone the -Conan [hello world starter project](https://github.com/conan-io/hello). +To test the Package Registry, you need a C++ project. If you don't already have +one, you can clone the Conan [hello world starter project](https://github.com/conan-io/hello). ### Build a package @@ -74,22 +79,26 @@ To build a package: conan new Hello/0.1 -t ``` -1. Create a package for the recipe by running `conan create` with the Conan user and channel: +1. Create a package for the recipe by running `conan create` with the Conan user + and channel: ```shell conan create . mycompany/beta ``` NOTE: **Note:** - If you use an [instance remote](#add-a-remote-for-your-instance), you must follow a specific [naming convention](#package-recipe-naming-convention-for-instance-remotes). + If you use an [instance remote](#add-a-remote-for-your-instance), you must + follow a specific [naming convention](#package-recipe-naming-convention-for-instance-remotes). A package with the recipe `Hello/0.1@mycompany/beta` is created. -For more details on creating and managing Conan packages, see the [Conan docs](https://docs.conan.io/en/latest/creating_packages.html). +For more details about creating and managing Conan packages, see the +[Conan documentation](https://docs.conan.io/en/latest/creating_packages.html). ## Add the Package Registry as a Conan remote -To run `conan` commands, you must add the Package Registry as a Conan remote for your project or instance. +To run `conan` commands, you must add the Package Registry as a Conan remote for +your project or instance. ### Add a remote for your project @@ -143,9 +152,9 @@ To add the remote: #### Package recipe naming convention for instance remotes -The standard Conan recipe convention is `package_name/version@user/channel`, -but if you're using an [instance remote](#add-a-remote-for-your-instance), the recipe -`user` must be the plus sign (`+`) separated project path. +The standard Conan recipe convention is `package_name/version@user/channel`, but +if you're using an [instance remote](#add-a-remote-for-your-instance), the +recipe `user` must be the plus sign (`+`) separated project path. Example recipe names: @@ -156,58 +165,68 @@ Example recipe names: | `gitlab-org/gitlab-ce` | `my-package/1.0.0@gitlab-org+gitlab-ce/stable` | Yes | | `gitlab-org/gitlab-ce` | `my-package/1.0.0@foo/stable` | No | -[Project remotes](#add-a-remote-for-your-project) have a more flexible naming convention. +[Project remotes](#add-a-remote-for-your-project) have a more flexible naming +convention. ## Authenticate to the Package Registry -To authenticate to the Package Registry, you need either a personal access token or deploy token. +To authenticate to the Package Registry, you need either a personal access token +or deploy token. -- If you use a [personal access token](../../../user/profile/personal_access_tokens.md), set the scope to `api`. -- If you use a [deploy token](./../../project/deploy_tokens/index.md), set the scope to `read_package_registry`, `write_package_registry`, or both. +- If you use a [personal access token](../../../user/profile/personal_access_tokens.md), + set the scope to `api`. +- If you use a [deploy token](../../project/deploy_tokens/index.md), set the + scope to `read_package_registry`, `write_package_registry`, or both. ### Add your credentials to the GitLab remote -Associate your token with the GitLab remote, so that you don't have to explicitly -add a token to every Conan command. +Associate your token with the GitLab remote, so that you don't have to +explicitly add a token to every Conan command. Prerequisites: - You must have an authentication token. -- The Conan remote [must be set](#add-the-package-registry-as-a-conan-remote). +- The Conan remote [must be configured](#add-the-package-registry-as-a-conan-remote). -In a terminal, run this command. In this example, the remote name is `gitlab`. Use the name of your remote. +In a terminal, run this command. In this example, the remote name is `gitlab`. +Use the name of your remote. ```shell conan user <gitlab_username or deploy_token_username> -r gitlab -p <personal_access_token or deploy_token> ``` -Now when you run commands with `--remote=gitlab`, your username and password are automatically included in the requests. +Now when you run commands with `--remote=gitlab`, your username and password are +included in the requests. -Alternately, you can explicitly include your credentials in any given command. For example: +Alternatively, you can explicitly include your credentials in any given command. +For example: ```shell CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> conan upload Hello/0.1@mycompany/beta --all --remote=gitlab ``` NOTE: **Note:** -Your authentication with GitLab expires on a regular basis, -so occasionally you may need to re-enter your personal access token. +Because your authentication with GitLab expires on a regular basis, you may +occasionally need to re-enter your personal access token. ### Set a default remote for your project (optional) -If you want to interact with the GitLab Package Registry without having to specify a remote, -you can tell Conan to always use the Package Registry for your packages. +If you want to interact with the GitLab Package Registry without having to +specify a remote, you can tell Conan to always use the Package Registry for your +packages. -In a terminal, run this command. +In a terminal, run this command: ```shell conan remote add_ref Hello/0.1@mycompany/beta gitlab ``` NOTE: **Note:** -The package recipe includes the version, so the default remote for `Hello/0.1@user/channel` does not work for `Hello/0.2@user/channel`. +The package recipe includes the version, so the default remote for +`Hello/0.1@user/channel` doesn't work for `Hello/0.2@user/channel`. -If you do not set a default user or remote, you can still include the user and remote in your commands: +If you don't set a default user or remote, you can still include the user and +remote in your commands: ```shell `CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> <conan command> --remote=gitlab @@ -215,17 +234,18 @@ If you do not set a default user or remote, you can still include the user and r ## Publish a Conan package -Publish a Conan package to the Package Registry, so that anyone who can access the project can use the package as a dependency. +Publish a Conan package to the Package Registry, so that anyone who can access +the project can use the package as a dependency. Prerequisites: -To publish a Conan package, you need: - -- The Package Registry [set as a remote](#add-the-package-registry-as-a-conan-remote). -- [Authentication](#authenticate-to-the-package-registry) set up with the Package Registry. -- A local [Conan package](https://docs.conan.io/en/latest/creating_packages/getting_started.html). +- The Conan remote [must be configured](#add-the-package-registry-as-a-conan-remote). +- [Authentication](#authenticate-to-the-package-registry) with the + Package Registry must be configured. +- A local [Conan package](https://docs.conan.io/en/latest/creating_packages/getting_started.html) + must exist. - For an instance remote, the package must meet the [naming convention](#package-recipe-naming-convention-for-instance-remotes). -- A project ID, which is on the project's homepage. +- You must have the project ID, which is on the project's homepage. To publish the package, use the `conan upload` command: @@ -237,11 +257,11 @@ conan upload Hello/0.1@mycompany/beta --all > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11678) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.7. -To work with Conan commands in [GitLab CI/CD](./../../../ci/README.md), you can use -`CI_JOB_TOKEN` in place of the personal access token in your commands. +To work with Conan commands in [GitLab CI/CD](../../../ci/README.md), you can +use `CI_JOB_TOKEN` in place of the personal access token in your commands. -You can provide the `CONAN_LOGIN_USERNAME` and `CONAN_PASSWORD` with each -Conan command in your `.gitlab-ci.yml` file. For example: +You can provide the `CONAN_LOGIN_USERNAME` and `CONAN_PASSWORD` with each Conan +command in your `.gitlab-ci.yml` file. For example: ```yaml image: conanio/gcc7 @@ -255,24 +275,26 @@ create_package: - CONAN_LOGIN_USERNAME=ci_user CONAN_PASSWORD=${CI_JOB_TOKEN} conan upload <package-name>/0.1@<group-name>+<project-name>/stable --all --remote=gitlab ``` -Additional Conan images to use as the basis of your CI file are available -in the [Conan docs](https://docs.conan.io/en/latest/howtos/run_conan_in_docker.html#available-docker-images). +Additional Conan images to use as the basis of your CI file are available in the +[Conan docs](https://docs.conan.io/en/latest/howtos/run_conan_in_docker.html#available-docker-images). ## Install a Conan package -Install a Conan package from the Package Registry so you can use it as a dependency. +Install a Conan package from the Package Registry so you can use it as a +dependency. -Conan packages are often installed as dependencies by using the `conanfile.txt` file. +Conan packages are often installed as dependencies by using the `conanfile.txt` +file. Prerequisites: -To install a Conan package, you need: - -- The Package Registry [set as a remote](#add-the-package-registry-as-a-conan-remote). -- [Authentication](#authenticate-to-the-package-registry) set up with the Package Registry. +- The Conan remote [must be configured](#add-the-package-registry-as-a-conan-remote). +- [Authentication](#authenticate-to-the-package-registry) with the + Package Registry must be configured. -1. In the project where you want to install the package as a dependency, open `conanfile.txt`. - Or, in the root of your project, create a file called `conanfile.txt`. +1. In the project where you want to install the package as a dependency, open + `conanfile.txt`. Or, in the root of your project, create a file called + `conanfile.txt`. 1. Add the Conan recipe to the `[requires]` section of the file: @@ -284,7 +306,8 @@ To install a Conan package, you need: cmake ``` -1. At the root of your project, create a `build` directory and change to that directory: +1. At the root of your project, create a `build` directory and change to that + directory: ```shell mkdir build && cd build @@ -298,7 +321,7 @@ To install a Conan package, you need: NOTE: **Note:** If you try to install the package you just created in this tutorial, the package -already exists on your local machine, so this command has no effect. +already exists on your local computer, so this command has no effect. ## Remove a Conan package @@ -310,19 +333,22 @@ There are two ways to remove a Conan package from the GitLab Package Registry. conan remove Hello/0.2@user/channel --remote=gitlab ``` - You must explicitly include the remote in this command, otherwise the package is only removed from your - local system cache. + You must explicitly include the remote in this command, otherwise the package + is removed only from your local system cache. NOTE: **Note:** - This command removes all recipe and binary package files from the Package Registry. + This command removes all recipe and binary package files from the + Package Registry. - From the GitLab user interface: - Go to your project's **Packages & Registries > Package Registry**. Remove the package by clicking the red trash icon. + Go to your project's **Packages & Registries > Package Registry**. Remove the + package by clicking the red trash icon. ## Search for Conan packages in the Package Registry -To search by full or partial package name, or by exact recipe, run the `conan search` command. +To search by full or partial package name, or by exact recipe, run the +`conan search` command. - To search for all packages with a specific package name: @@ -336,7 +362,8 @@ To search by full or partial package name, or by exact recipe, run the `conan se conan search He* --remote=gitlab ``` -The scope of your search includes all projects you have permission to access. This includes your private projects as well as all public projects. +The scope of your search includes all projects you have permission to access. +This includes your private projects as well as all public projects. ## Fetch Conan package information from the Package Registry @@ -351,7 +378,9 @@ conan info Hello/0.1@mycompany/beta The GitLab Conan repository supports the following Conan CLI commands: - `conan upload`: Upload your recipe and package files to the Package Registry. -- `conan install`: Install a Conan package from the Package Registry, this includes using the `conanfile.txt` file. -- `conan search`: Search the Package Registry for public packages, and private packages you have permission to view. +- `conan install`: Install a Conan package from the Package Registry, which + includes using the `conanfile.txt` file. +- `conan search`: Search the Package Registry for public packages, and private + packages you have permission to view. - `conan info`: View the information on a given package from the Package Registry. - `conan remove`: Delete the package from the Package Registry. diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index baadd3c91a7..1cb6c951bd9 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -447,7 +447,7 @@ For the project where it's defined, tags matching the regex pattern are removed. The underlying layers and images remain. To delete the underlying layers and images that aren't associated with any tags, administrators can use -[garbage collection](../../../administration/packages/container_registry.md#removing-unused-layers-not-referenced-by-manifests) with the `-m` switch. +[garbage collection](../../../administration/packages/container_registry.md#removing-untagged-manifests-and-unreferenced-layers) with the `-m` switch. ### Enable the cleanup policy @@ -468,7 +468,7 @@ Cleanup policies can be run on all projects, with these exceptions: ``` There are performance risks with enabling it for all projects, especially if you - are using an [external registry](./index.md#use-with-external-container-registries). + are using an [external registry](index.md#use-with-external-container-registries). - For self-managed GitLab instances, you can enable or disable the cleanup policy for a specific project. @@ -522,7 +522,7 @@ To create a cleanup policy in the UI: | **Expiration interval** | How long tags are exempt from being deleted. | | **Expiration schedule** | How often the policy should run. | | **Number of tags to retain** | How many tags to _always_ keep for each image. | - | **Tags with names matching this regex pattern expire:** | The regex pattern that determines which tags to remove. For all tags, use `.*`. See other [regex pattern examples](#regex-pattern-examples). | + | **Tags with names matching this regex pattern expire:** | The regex pattern that determines which tags to remove. This value cannot be blank. For all tags, use `.*`. See other [regex pattern examples](#regex-pattern-examples). | | **Tags with names matching this regex pattern are preserved:** | The regex pattern that determines which tags to preserve. The `latest` tag is always preserved. For all tags, use `.*`. See other [regex pattern examples](#regex-pattern-examples). | 1. Click **Set cleanup policy**. @@ -544,6 +544,8 @@ Here are examples of regex patterns you may want to use: .* ``` + This is the default value for the expiration regex. + - Match tags that start with `v`: ```plaintext @@ -578,7 +580,7 @@ See the API documentation for further details: [Edit project](../../../api/proje ### Use with external container registries -When using an [external container registry](./../../../administration/packages/container_registry.md#use-an-external-container-registry-with-gitlab-as-an-auth-endpoint), +When using an [external container registry](../../../administration/packages/container_registry.md#use-an-external-container-registry-with-gitlab-as-an-auth-endpoint), running a cleanup policy on a project may have some performance risks. If a project runs a policy to remove thousands of tags the GitLab background jobs may get backed up or fail completely. diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index 3fa21ef486b..e1fae770a5d 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -4,9 +4,10 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# Dependency Proxy **(PREMIUM)** +# Dependency Proxy -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7934) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.11. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7934) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.11. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/273655) to [GitLab Core](https://about.gitlab.com/pricing/) in GitLab 13.6. The GitLab Dependency Proxy is a local proxy you can use for your frequently-accessed upstream images. @@ -41,6 +42,9 @@ The Dependency Proxy is not available for projects. ## Use the Dependency Proxy for Docker images +CAUTION: **Important:** +In some specific storage configurations, an issue occurs and container images are not pulled correctly from the cache. The problem occurs when an image is located in object storage. The proxy looks for it locally and fails to find it. View [issue #208080](https://gitlab.com/gitlab-org/gitlab/-/issues/208080) for details. + You can use GitLab as a source for your Docker images. Prerequisites: diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index 0c961b5c86f..f489c7c800f 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -49,7 +49,7 @@ Example request: ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" \ --upload-file path/to/file.txt \ - https://gitlab.example.com/api/v4/projects/24/generic/my_package/0.0.1/file.txt + https://gitlab.example.com/api/v4/projects/24/packages/generic/my_package/0.0.1/file.txt ``` Example response: @@ -85,12 +85,12 @@ Example request that uses a personal access token: ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" \ - https://gitlab.example.com/api/v4/projects/24/generic/my_package/0.0.1/file.txt + https://gitlab.example.com/api/v4/projects/24/packages/generic/my_package/0.0.1/file.txt ``` ## Publish a generic package by using CI/CD -To work with generic packages in [GitLab CI/CD](./../../../ci/README.md), you can use +To work with generic packages in [GitLab CI/CD](../../../ci/README.md), you can use `CI_JOB_TOKEN` in place of the personal access token in your commands. For example: @@ -137,3 +137,9 @@ Feature.disable(:generic_packages) # For a single project Feature.disable(:generic_packages, Project.find(<project id>)) ``` + +### Generic package sample project + +The [Write CI-CD Variables in Pipeline](https://gitlab.com/guided-explorations/cfg-data/write-ci-cd-variables-in-pipeline) project contains a working example you can use to create, upload, and download generic packages in GitLab CI/CD. + +It also demonstrates how to manage a semantic version for the generic package: storing it in a CI/CD variable, retrieving it, incrementing it, and writing it back to the CI/CD variable when tests for the download work correctly. diff --git a/doc/user/packages/go_proxy/index.md b/doc/user/packages/go_proxy/index.md index bd3b5b49ebd..81edc3afcfd 100644 --- a/doc/user/packages/go_proxy/index.md +++ b/doc/user/packages/go_proxy/index.md @@ -4,11 +4,11 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab Go Proxy +# Go proxy for GitLab > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27376) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.1. > - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. +> - It's disabled for GitLab.com. > - It's not recommended for production use. > - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-the-go-proxy). > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. @@ -16,14 +16,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w With the Go proxy for GitLab, every project in GitLab can be fetched with the [Go proxy protocol](https://proxy.golang.org/). -## Prerequisites +## Enable the Go proxy -### Enable the Go proxy +The Go proxy for GitLab is under development, and isn't ready for production use +due to [potential performance issues with large repositories](https://gitlab.com/gitlab-org/gitlab/-/issues/218083). -The Go proxy for GitLab is under development and not ready for production use, due to -[potential performance issues with large repositories](https://gitlab.com/gitlab-org/gitlab/-/issues/218083). - -It is deployed behind a feature flag that is **disabled by default**. +It's deployed behind a feature flag that is _disabled by default_. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can enable it for your instance. @@ -47,57 +45,46 @@ Feature.enable(:go_proxy, Project.find(1)) Feature.disable(:go_proxy, Project.find(2)) ``` -### Enable the Package Registry - -The Package Registry is enabled for new projects by default. If you cannot find -the **Packages > List** entry under your project's sidebar, verify -the following: - -1. Your GitLab administrator has - [enabled support for the Package Registry](../../../administration/packages/index.md). -1. The Package Registry is [enabled for your project](../index.md). - NOTE: **Note:** -GitLab does not currently display Go modules in the **Packages Registry** of a project. -Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/213770) for details. +Even if it's enabled, GitLab doesn't display Go modules in the **Package Registry**. +Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/213770) for +details. ## Add GitLab as a Go proxy -NOTE: **Note:** -To use a Go proxy, you must be using Go 1.13 or later. - -The available proxy endpoints are: +To use GitLab as a Go proxy, you must be using Go 1.13 or later. -- Project - can fetch modules defined by a project - `/api/v4/projects/:id/packages/go` +The available proxy endpoint is for fetching modules by project: `/api/v4/projects/:id/packages/go` -To use the Go proxy for GitLab to fetch Go modules from GitLab, add the -appropriate proxy endpoint to `GOPROXY`. For details on setting Go environment -variables, see [Set environment variables](#set-environment-variables). For -details on configuring `GOPROXY`, see [Dependency Management in Go > -Proxies](../../../development/go_guide/dependencies.md#proxies). +To fetch Go modules from GitLab, add the project-specific endpoint to `GOPROXY`. -For example, adding the project-specific endpoint to `GOPROXY` will tell Go -to initially query that endpoint and fall back to the default behavior: +Go queries the endpoint and falls back to the default behavior: ```shell -go env -w GOPROXY='https://gitlab.com/api/v4/projects/1234/packages/go,https://proxy.golang.org,direct' +go env -w GOPROXY='https://gitlab.example.com/api/v4/projects/1234/packages/go,https://proxy.golang.org,direct' ``` -With this configuration, Go fetches dependencies as follows: +With this configuration, Go fetches dependencies in this order: -1. Attempt to fetch from the project-specific Go proxy. -1. Attempt to fetch from [proxy.golang.org](https://proxy.golang.org). -1. Fetch directly with version control system operations (such as `git clone`, +1. Go attempts to fetch from the project-specific Go proxy. +1. Go attempts to fetch from [proxy.golang.org](https://proxy.golang.org). +1. Go fetches directly with version control system operations (like `git clone`, `svn checkout`, and so on). -If `GOPROXY` is not specified, Go follows steps 2 and 3, which corresponds to -setting `GOPROXY` to `https://proxy.golang.org,direct`. If `GOPROXY` only -contains the project-specific endpoint, Go will only query that endpoint. +If `GOPROXY` isn't specified, Go follows steps 2 and 3, which corresponds to +setting `GOPROXY` to `https://proxy.golang.org,direct`. If `GOPROXY` +contains only the project-specific endpoint, Go queries only that endpoint. + +For details about how to set Go environment variables, see +[Set environment variables](#set-environment-variables). + +For details about configuring `GOPROXY`, see +[Dependency Management in Go > Proxies](../../../development/go_guide/dependencies.md#proxies). ## Fetch modules from private projects -`go` does not support transmitting credentials over insecure connections. The -steps below work only if GitLab is configured for HTTPS. +`go` doesn't support transmitting credentials over insecure connections. The +following steps work only if GitLab is configured for HTTPS: 1. Configure Go to include HTTP basic authentication credentials when fetching from the Go proxy for GitLab. @@ -107,31 +94,38 @@ steps below work only if GitLab is configured for HTTPS. ### Enable request authentication Create a [personal access token](../../profile/personal_access_tokens.md) with -the `api` or `read_api` scope and add it to -[`~/.netrc`](https://ec.haxx.se/usingcurl/usingcurl-netrc): +the scope set to `api` or `read_api`. -```netrc +Open your [`~/.netrc`](https://ec.haxx.se/usingcurl/usingcurl-netrc) file +and add the following text. Replace the variables in `< >` with your values. + +```plaintext machine <url> login <username> password <token> ``` -`<url>` should be the URL of the GitLab instance, for example `gitlab.com`. -`<username>` and `<token>` should be your username and the personal access -token, respectively. +- `<url>`: The GitLab URL, for example `gitlab.com`. +- `<username>`: Your username. +- `<token>`: Your personal access token. ### Disable checksum database queries -When downloading dependencies, by default Go 1.13 and later validate fetched -sources against the checksum database `sum.golang.org`. If the checksum of the -fetched sources does not match the checksum from the database, Go will not build -the dependency. This causes private modules to fail to build, as -`sum.golang.org` cannot fetch the source of private modules and thus cannot -provide a checksum. To resolve this issue, `GONOSUMDB` should be set to a -comma-separated list of private projects. For details on setting Go environment -variables, see [Set environment variables](#set-environment-variables). For more -details on disabling this feature of Go, see [Dependency Management in Go > -Checksums](../../../development/go_guide/dependencies.md#checksums). +When downloading dependencies with Go 1.13 and later, fetched sources are +validated against the checksum database `sum.golang.org`. + +If the checksum of the fetched sources doesn't match the checksum from the +database, Go doesn't build the dependency. + +Private modules fail to build because `sum.golang.org` can't fetch the source +of private modules, and so it cannot provide a checksum. + +To resolve this issue, set `GONOSUMDB` to a comma-separated list of private +projects. For details about setting Go environment variables, see +[Set environment variables](#set-environment-variables). For more details about +disabling this feature of Go, see +[Dependency Management in Go > Checksums](../../../development/go_guide/dependencies.md#checksums). -For example, to disable checksum queries for `gitlab.com/my/project`, set `GONOSUMDB`: +For example, to disable checksum queries for `gitlab.com/my/project`, set +`GONOSUMDB`: ```shell go env -w GONOSUMDB='gitlab.com/my/project,<previous value>' @@ -139,32 +133,36 @@ go env -w GONOSUMDB='gitlab.com/my/project,<previous value>' ## Working with Go -If you are unfamiliar with managing dependencies in Go, or Go in general, -consider reviewing the following documentation: +If you're unfamiliar with managing dependencies in Go, or Go in general, review +the following documentation: - [Dependency Management in Go](../../../development/go_guide/dependencies.md) - [Go Modules Reference](https://golang.org/ref/mod) -- [Documentation (golang.org)](https://golang.org/doc/) -- [Learn (learn.go.dev)](https://learn.go.dev/) +- [Documentation (`golang.org`)](https://golang.org/doc/) +- [Learn (`learn.go.dev`)](https://learn.go.dev/) ### Set environment variables -Go uses environment variables to control various features. These can be managed -in all the usual ways, but Go 1.14 will read and write Go environment variables -from and to a special Go environment file, `~/.go/env` by default. If `GOENV` is -set to a file, Go will read and write that file instead. If `GOENV` is not set -but `GOPATH` is set, Go will read and write `$GOPATH/env`. +Go uses environment variables to control various features. You can manage these +variables in all the usual ways. However, Go 1.14 reads and writes Go +environment variables to and from a special Go environment file, `~/.go/env` by +default. + +- If `GOENV` is set to a file, Go reads and writes to and from that file instead. +- If `GOENV` is not set but `GOPATH` is set, Go reads and writes `$GOPATH/env`. Go environment variables can be read with `go env <var>` and, in Go 1.14 and -later, can be written with `go env -w <var>=<value>`. For example, `go env -GOPATH` or `go env -w GOPATH=/go`. +later, can be written with `go env -w <var>=<value>`. For example, +`go env GOPATH` or `go env -w GOPATH=/go`. ### Release a module Go modules and module versions are defined by source repositories, such as Git, -SVN, Mercurial, and so on. A module is a repository containing `go.mod` and Go -files. Module versions are defined by VCS tags. To publish a module, push -`go.mod` and source files to a VCS repository. To publish a module version, push -a VCS tag. See [Dependency Management in Go > -Versioning](../../../development/go_guide/dependencies.md#versioning) for more -details on what constitutes a valid module or module version. +SVN, and Mercurial. A module is a repository that contains `go.mod` and Go +files. Module versions are defined by VCS tags. + +To publish a module, push `go.mod` and source files to a VCS repository. To +publish a module version, push a VCS tag. + +See [Dependency Management in Go > Versioning](../../../development/go_guide/dependencies.md#versioning) +for more details about what constitutes a valid module or module version. diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 53ba3d01b3e..83d179cbc9c 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -30,31 +30,31 @@ The Package Registry supports the following formats: You can also use the [API](../../api/packages.md) to administer the Package Registry. -The GitLab [Container Registry](container_registry/index.md) is a secure and private registry for container images. -It's built on open source software and completely integrated within GitLab. -Use GitLab CI/CD to create and publish images. Use the GitLab [API](../../api/container_registry.md) to -manage the registry across groups and projects. +## Accepting contributions -The [Dependency Proxy](dependency_proxy/index.md) is a local proxy for frequently-used upstream images and packages. - -## Suggested contributions - -Consider contributing to GitLab. This [development documentation](../../development/packages.md) will -guide you through the process. Or check out how other members of the community -are adding support for [PHP](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17417) or [Terraform](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18834). +The below table lists formats that are not supported, but are accepting Community contributions for. Consider contributing to GitLab. This [development documentation](../../development/packages.md) will +guide you through the process. -| Format | Use case | +| Format | Status | | ------ | ------ | -| [Cargo](https://gitlab.com/gitlab-org/gitlab/-/issues/33060) | Cargo is the Rust package manager. Build, publish and share Rust packages | -| [Chef](https://gitlab.com/gitlab-org/gitlab/-/issues/36889) | Configuration management with Chef using all the benefits of a repository manager. | -| [CocoaPods](https://gitlab.com/gitlab-org/gitlab/-/issues/36890) | Speed up development with Xcode and CocoaPods. | -| [Conda](https://gitlab.com/gitlab-org/gitlab/-/issues/36891) | Secure and private local Conda repositories. | -| [CRAN](https://gitlab.com/gitlab-org/gitlab/-/issues/36892) | Deploy and resolve CRAN packages for the R language. | -| [Debian](https://gitlab.com/gitlab-org/gitlab/-/issues/5835) | Host and provision Debian packages. | -| [Opkg](https://gitlab.com/gitlab-org/gitlab/-/issues/36894) | Optimize your work with OpenWrt using Opkg repositories. | -| [P2](https://gitlab.com/gitlab-org/gitlab/-/issues/36895) | Host all your Eclipse plugins in your own GitLab P2 repository. | -| [Puppet](https://gitlab.com/gitlab-org/gitlab/-/issues/36897) | Configuration management meets repository management with Puppet repositories. | -| [RPM](https://gitlab.com/gitlab-org/gitlab/-/issues/5932) | Distribute RPMs directly from GitLab. | -| [RubyGems](https://gitlab.com/gitlab-org/gitlab/-/issues/803) | Use GitLab to host your own gems. | -| [SBT](https://gitlab.com/gitlab-org/gitlab/-/issues/36898) | Resolve dependencies from and deploy build output to SBT repositories when running SBT builds. | -| [Vagrant](https://gitlab.com/gitlab-org/gitlab/-/issues/36899) | Securely host your Vagrant boxes in local repositories. | +| Chef | [#36889](https://gitlab.com/gitlab-org/gitlab/-/issues/36889) | +| CocoaPods | [#36890](https://gitlab.com/gitlab-org/gitlab/-/issues/36890) | +| Conda | [#36891](https://gitlab.com/gitlab-org/gitlab/-/issues/36891) | +| CRAN | [#36892](https://gitlab.com/gitlab-org/gitlab/-/issues/36892) | +| Debian | [WIP: Merge Request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/44746) | +| Opkg | [#36894](https://gitlab.com/gitlab-org/gitlab/-/issues/36894) | +| P2 | [#36895](https://gitlab.com/gitlab-org/gitlab/-/issues/36895) | +| Puppet | [#36897](https://gitlab.com/gitlab-org/gitlab/-/issues/36897) | +| RPM | [#5932](https://gitlab.com/gitlab-org/gitlab/-/issues/5932) | +| RubyGems | [#803](https://gitlab.com/gitlab-org/gitlab/-/issues/803) | +| SBT | [#36898](https://gitlab.com/gitlab-org/gitlab/-/issues/36898) | +| Terraform | [WIP: Merge Request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18834) | +| Vagrant | [#36899](https://gitlab.com/gitlab-org/gitlab/-/issues/36899) | + +## Container Registry + +The GitLab [Container Registry](container_registry/index.md) is a secure and private registry for container images. It's built on open source software and completely integrated within GitLab. Use GitLab CI/CD to create and publish images. Use the GitLab [API](../../api/container_registry.md) to manage the registry across groups and projects. + +## Dependency Proxy + +The [Dependency Proxy](dependency_proxy/index.md) is a local proxy for frequently-used upstream images and packages. diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index d4a8ff0cdb4..5d0d64b310d 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -17,9 +17,9 @@ Then, install the packages whenever you need to use them as a dependency. This section explains how to install Maven and build a package. If you already use Maven and know how to build your own packages, go to the -[next section](#add-the-package-registry-as-a-maven-remote). +[next section](#authenticate-to-the-package-registry-with-maven). -Maven repositories work well with Gradle, too. To set up a Gradle project, see [get started with Gradle](#use-gradle-to-create-a-java-project). +Maven repositories work well with Gradle, too. To set up a Gradle project, see [get started with Gradle](#build-a-java-project-with-gradle). ### Install Maven @@ -29,14 +29,14 @@ The required minimum versions are: - Maven 3.6+ Follow the instructions at [maven.apache.org](https://maven.apache.org/install.html) -to download and install Maven for your local development environment. Once +to download and install Maven for your local development environment. After installation is complete, verify you can use Maven in your terminal by running: ```shell mvn --version ``` -You should see something similar to the below printed in the output: +The output should be similar to: ```shell Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T20:00:29+01:00) @@ -48,29 +48,26 @@ OS name: "mac os x", version: "10.15.2", arch: "x86_64", family: "mac" ### Create a project -Understanding how to create a full Java project is outside the scope of this -guide but you can follow the steps below to create a new project that can be +Follow these steps to create a Maven project that can be published to the GitLab Package Registry. -Start by opening your terminal and creating a directory where you would like to -store the project in your environment. From inside the directory, you can run -the following Maven command to initialize a new package: +1. Open your terminal and create a directory to store the project. +1. From the new directory, run this Maven command to initialize a new package: -```shell -mvn archetype:generate -DgroupId=com.mycompany.mydepartment -DartifactId=my-project -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false -``` + ```shell + mvn archetype:generate -DgroupId=com.mycompany.mydepartment -DartifactId=my-project -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false + ``` -The arguments are as follows: + The arguments are: -- `DgroupId`: A unique string that identifies your package. You should follow -the [Maven naming conventions](https://maven.apache.org/guides/mini/guide-naming-conventions.html). -- `DartifactId`: The name of the JAR, appended to the end of the `DgroupId`. -- `DarchetypeArtifactId`: The archetype used to create the initial structure of -the project. -- `DinteractiveMode`: Create the project using batch mode (optional). + - `DgroupId`: A unique string that identifies your package. Follow + the [Maven naming conventions](https://maven.apache.org/guides/mini/guide-naming-conventions.html). + - `DartifactId`: The name of the `JAR`, appended to the end of the `DgroupId`. + - `DarchetypeArtifactId`: The archetype used to create the initial structure of + the project. + - `DinteractiveMode`: Create the project using batch mode (optional). -After running the command, you should see the following message, indicating that -your project has been set up successfully: +This message indicates that the project was set up successfully: ```shell ... @@ -82,33 +79,33 @@ your project has been set up successfully: [INFO] ------------------------------------------------------------------------ ``` -You should see a new directory where you ran this command matching your -`DartifactId` parameter (in this case it should be `my-project`). +In the folder where you ran the command, a new directory should be displayed. +The directory name should match the `DartifactId` parameter, which in this case, +is `my-project`. -## Use Gradle to create a Java project +## Build a Java project with Gradle This section explains how to install Gradle and initialize a Java project. If you already use Gradle and know how to build your own packages, go to the -[next section](#add-the-package-registry-as-a-maven-remote). +[next section](#authenticate-to-the-package-registry-with-maven). ### Install Gradle -Installation is needed only if you want to create a new Gradle project. Follow +If you want to create a new Gradle project, you must install Gradle. Follow instructions at [gradle.org](https://gradle.org/install/) to download and install Gradle for your local development environment. -Verify you can use Gradle in your terminal by running: +In your terminal, verify you can use Gradle by running: ```shell gradle -version ``` -If you want to use an existing Gradle project, installation is not necessary. -Simply execute `gradlew` (on Linux) or `gradlew.bat` (on Windows) in the project -directory instead. +To use an existing Gradle project, in the project directory, +on Linux execute `gradlew`, or on Windows execute `gradlew.bat`. -You should see something similar to the below printed in the output: +The output should be similar to: ```plaintext ------------------------------------------------------------ @@ -127,88 +124,79 @@ OS: Windows 10 10.0 amd64 ### Create a Java project -Understanding how to create a full Java project in Gradle is outside the scope of this -guide, but you can follow the steps below to create a new project that can be +Follow these steps to create a Maven project that can be published to the GitLab Package Registry. -Start by opening your terminal and creating a directory where you would like to -store the project in your environment. From inside the directory, you can run -the following Maven command to initialize a new package: +1. Open your terminal and create a directory to store the project. +1. From this new directory, run this Maven command to initialize a new package: -```shell -gradle init -``` + ```shell + gradle init + ``` -The output should be + The output should be: -```plaintext -Select type of project to generate: - 1: basic - 2: application - 3: library - 4: Gradle plugin -Enter selection (default: basic) [1..4] -``` + ```plaintext + Select type of project to generate: + 1: basic + 2: application + 3: library + 4: Gradle plugin + Enter selection (default: basic) [1..4] + ``` -Enter `3` to create a new Library project. The output should be: +1. Enter `3` to create a new Library project. The output should be: -```plaintext -Select implementation language: - 1: C++ - 2: Groovy - 3: Java - 4: Kotlin - 5: Scala - 6: Swift -``` + ```plaintext + Select implementation language: + 1: C++ + 2: Groovy + 3: Java + 4: Kotlin + 5: Scala + 6: Swift + ``` -Enter `3` to create a new Java Library project. The output should be: +1. Enter `3` to create a new Java Library project. The output should be: -```plaintext -Select build script DSL: - 1: Groovy - 2: Kotlin -Enter selection (default: Groovy) [1..2] -``` + ```plaintext + Select build script DSL: + 1: Groovy + 2: Kotlin + Enter selection (default: Groovy) [1..2] + ``` -Choose `1` to create a new Java Library project which is described in Groovy DSL. The output should be: +1. Enter `1` to create a new Java Library project that is described in Groovy DSL. The output should be: -```plaintext -Select test framework: - 1: JUnit 4 - 2: TestNG - 3: Spock - 4: JUnit Jupiter -``` + ```plaintext + Select test framework: + 1: JUnit 4 + 2: TestNG + 3: Spock + 4: JUnit Jupiter + ``` -Choose `1` to initialize the project with JUnit 4 testing libraries. The output should be: +1. Enter `1` to initialize the project with JUnit 4 testing libraries. The output should be: -```plaintext -Project name (default: test): -``` + ```plaintext + Project name (default: test): + ``` -Enter a project name or hit enter to use the directory name as project name. +1. Enter a project name or press Enter to use the directory name as project name. -## Add the Package Registry as a Maven remote +## Authenticate to the Package Registry with Maven -The next step is to add the GitLab Package Registry as a Maven remote. If a -project is private or you want to upload Maven artifacts to GitLab, -credentials must be provided for authorization too. Support is available -for [personal access tokens](#authenticate-with-a-personal-access-token), -[CI job tokens](#authenticate-with-a-ci-job-token), and -[deploy tokens](../../project/deploy_tokens/index.md) only. Regular username/password -credentials do not work. +To authenticate to the Package Registry, you need either a personal access token or deploy token. -### Authenticate with a personal access token +- If you use a [personal access token](../../../user/profile/personal_access_tokens.md), set the scope to `api`. +- If you use a [deploy token](../../project/deploy_tokens/index.md), set the scope to `read_package_registry`, `write_package_registry`, or both. -To authenticate with a [personal access token](../../profile/personal_access_tokens.md), -set the scope to `api` when creating one, and add it to your Maven or Gradle configuration -files. +### Authenticate with a personal access token in Maven -#### Authenticate with a personal access token in Maven +To use a personal access token, add this section to your +[`settings.xml`](https://maven.apache.org/settings.html) file. -Add a corresponding section to your -[`settings.xml`](https://maven.apache.org/settings.html) file: +The `name` must be `Private-Token`. ```xml <settings> @@ -228,45 +216,40 @@ Add a corresponding section to your </settings> ``` -#### Authenticate with a personal access token in Gradle +### Authenticate with a deploy token in Maven -Create a file `~/.gradle/gradle.properties` with the following content: +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) deploy token authentication in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -```groovy -gitLabPrivateToken=REPLACE_WITH_YOUR_PERSONAL_ACCESS_TOKEN -``` +To use a deploy token, add this section to your +[`settings.xml`](https://maven.apache.org/settings.html) file. -Add a repositories section to your -[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) -file: +The `name` must be `Deploy-Token`. -```groovy -repositories { - maven { - url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" - name "GitLab" - credentials(HttpHeaderCredentials) { - name = 'Private-Token' - value = gitLabPrivateToken - } - authentication { - header(HttpHeaderAuthentication) - } - } -} +```xml +<settings> + <servers> + <server> + <id>gitlab-maven</id> + <configuration> + <httpHeaders> + <property> + <name>Deploy-Token</name> + <value>REPLACE_WITH_YOUR_DEPLOY_TOKEN</value> + </property> + </httpHeaders> + </configuration> + </server> + </servers> +</settings> ``` -You should now be able to upload Maven artifacts to your project. - -### Authenticate with a CI job token +### Authenticate with a CI job token in Maven -If you're using GitLab CI/CD, a CI job token can be used instead -of a personal access token. +To authenticate with a CI job token, add this section to your +[`settings.xml`](https://maven.apache.org/settings.html) file. -#### Authenticate with a CI job token in Maven - -To authenticate with a CI job token, add a corresponding section to your -[`settings.xml`](https://maven.apache.org/settings.html) file: +The `name` must be `Job-Token`. ```xml <settings> @@ -286,23 +269,35 @@ To authenticate with a CI job token, add a corresponding section to your </settings> ``` -You can read more on -[how to create Maven packages using GitLab CI/CD](#creating-maven-packages-with-gitlab-cicd). +Read more about [how to create Maven packages using GitLab CI/CD](#create-maven-packages-with-gitlab-cicd). + +## Authenticate to the Package Registry with Gradle -#### Authenticate with a CI job token in Gradle +To authenticate to the Package Registry, you need either a personal access token or deploy token. -To authenticate with a CI job token, add a repositories section to your +- If you use a [personal access token](../../../user/profile/personal_access_tokens.md), set the scope to `api`. +- If you use a [deploy token](../../project/deploy_tokens/index.md), set the scope to `read_package_registry`, `write_package_registry`, or both. + +### Authenticate with a personal access token in Gradle + +Create a file `~/.gradle/gradle.properties` with the following content: + +```groovy +gitLabPrivateToken=REPLACE_WITH_YOUR_PERSONAL_ACCESS_TOKEN +``` + +Add a `repositories` section to your [`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) file: ```groovy repositories { maven { - url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" + url "https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven" name "GitLab" credentials(HttpHeaderCredentials) { - name = 'Job-Token' - value = System.getenv("CI_JOB_TOKEN") + name = 'Private-Token' + value = gitLabPrivateToken } authentication { header(HttpHeaderAuthentication) @@ -311,51 +306,42 @@ repositories { } ``` -### Authenticate with a deploy token - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0. - -To authenticate with a [deploy token](./../../project/deploy_tokens/index.md), -set the scope to `api` when creating one, and add it to your Maven or Gradle configuration -files. - -#### Authenticate with a deploy token in Maven +### Authenticate with a deploy token in Gradle -Add a corresponding section to your -[`settings.xml`](https://maven.apache.org/settings.html) file: +To authenticate with a deploy token, add a `repositories` section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) +file: -```xml -<settings> - <servers> - <server> - <id>gitlab-maven</id> - <configuration> - <httpHeaders> - <property> - <name>Deploy-Token</name> - <value>REPLACE_WITH_YOUR_DEPLOY_TOKEN</value> - </property> - </httpHeaders> - </configuration> - </server> - </servers> -</settings> +```groovy +repositories { + maven { + url "https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(HttpHeaderCredentials) { + name = 'Deploy-Token' + value = '<deploy-token>' + } + authentication { + header(HttpHeaderAuthentication) + } + } +} ``` -#### Authenticate with a deploy token in Gradle +### Authenticate with a CI job token in Gradle -To authenticate with a deploy token, add a repositories section to your +To authenticate with a CI job token, add a `repositories` section to your [`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) file: ```groovy repositories { maven { - url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" + url "https://gitlab.example.com/api/v4/groups/<group>/-/packages/maven" name "GitLab" credentials(HttpHeaderCredentials) { - name = 'Deploy-Token' - value = '<deploy-token>' + name = 'Job-Token' + value = System.getenv("CI_JOB_TOKEN") } authentication { header(HttpHeaderAuthentication) @@ -364,177 +350,162 @@ repositories { } ``` -## Configuring your project to use the GitLab Maven repository URL - -To download and upload packages from GitLab, you need a `repository` and -`distributionManagement` section in your `pom.xml` file. If you're following the -steps from above, then you must add the following information to your -`my-project/pom.xml` file. +## Use the GitLab endpoint for Maven packages -Depending on your workflow and the amount of Maven packages you have, there are -3 ways you can configure your project to use the GitLab endpoint for Maven packages: +To use the GitLab endpoint for Maven packages, choose an option: -- **Project level**: Useful when you have few Maven packages which are not under +- **Project-level**: Use when you have few Maven packages and they are not in the same GitLab group. -- **Group level**: Useful when you have many Maven packages under the same GitLab +- **Group-level**: Use when you have many Maven packages in the same GitLab group. -- **Instance level**: Useful when you have many Maven packages under different - GitLab groups or on their own namespace. +- **Instance-level**: Use when you have many Maven packages in different + GitLab groups or in their own namespace. + +The option you choose determines the settings you'll add to your `pom.xml` file. -NOTE: **Note:** -In all cases, you need a project specific URL for uploading a package in -the `distributionManagement` section. +In all cases, to publish a package, you need: -### Project level Maven endpoint +- A project-specific URL in the `distributionManagement` section. +- A `repository` and `distributionManagement` section. -The example below shows how the relevant `repository` section of your `pom.xml` -would look like in Maven: +### Project-level Maven endpoint + +The relevant `repository` section of your `pom.xml` +in Maven should look like this: ```xml <repositories> <repository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven</url> </repository> </repositories> <distributionManagement> <repository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven</url> </repository> <snapshotRepository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven</url> </snapshotRepository> </distributionManagement> ``` -The corresponding section in Gradle would look like this: +The corresponding section in Gradle would be: ```groovy repositories { maven { - url "https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven" + url "https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven" name "GitLab" } } ``` -The `id` must be the same with what you -[defined in `settings.xml`](#add-the-package-registry-as-a-maven-remote). - -Replace `PROJECT_ID` with your project ID which can be found on the home page -of your project. +- The `id` is what you [defined in `settings.xml`](#authenticate-to-the-package-registry-with-maven). +- The `PROJECT_ID` is your project ID, which you can view on your project's home page. +- Replace `gitlab.example.com` with your domain name. +- For retrieving artifacts, use either the + [URL-encoded](../../../api/README.md#namespaced-path-encoding) path of the project + (like `group%2Fproject`) or the project's ID (like `42`). However, only the + project's ID can be used for publishing. -If you have a self-managed GitLab installation, replace `gitlab.com` with your -domain name. +### Group-level Maven endpoint -NOTE: **Note:** -For retrieving artifacts, you can use either the -[URL encoded](../../../api/README.md#namespaced-path-encoding) path of the project -(such as `group%2Fproject`) or the project's ID (such as `42`). However, only the -project's ID can be used for uploading. - -### Group level Maven endpoint - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8798) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8798) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. If you rely on many packages, it might be inefficient to include the `repository` section -with a unique URL for each package. Instead, you can use the group level endpoint for -all your Maven packages stored within one GitLab group. Only packages you have access to +with a unique URL for each package. Instead, you can use the group-level endpoint for +all the Maven packages stored within one GitLab group. Only packages you have access to are available for download. -The group level endpoint works with any package names, which means the you -have the flexibility of naming compared to [instance level endpoint](#instance-level-maven-endpoint). -However, GitLab does not guarantee the uniqueness of the package names within +The group-level endpoint works with any package names, so you +have more flexibility in naming, compared to the [instance-level endpoint](#instance-level-maven-endpoint). +However, GitLab does not guarantee the uniqueness of package names within the group. You can have two projects with the same package name and package version. As a result, GitLab serves whichever one is more recent. -The example below shows how the relevant `repository` section of your `pom.xml` -would look like. You still need a project specific URL for uploading a package in +This example shows the relevant `repository` section of your `pom.xml` file. +You still need a project-specific URL for publishing a package in the `distributionManagement` section: ```xml <repositories> <repository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/groups/GROUP_ID/-/packages/maven</url> + <url>https://gitlab.example.com/api/v4/groups/GROUP_ID/-/packages/maven</url> </repository> </repositories> <distributionManagement> <repository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven</url> </repository> <snapshotRepository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven</url> </snapshotRepository> </distributionManagement> ``` -For Gradle, the corresponding repositories section would look like: +For Gradle, the corresponding `repositories` section would look like: ```groovy repositories { maven { - url "https://gitlab.com/api/v4/groups/GROUP_ID/-/packages/maven" + url "https://gitlab.example.com/api/v4/groups/GROUP_ID/-/packages/maven" name "GitLab" } } ``` -The `id` must be the same with what you -[defined in `settings.xml`](#add-the-package-registry-as-a-maven-remote). - -Replace `my-group` with your group name and `PROJECT_ID` with your project ID -which can be found on the home page of your project. - -If you have a self-managed GitLab installation, replace `gitlab.com` with your -domain name. +- For the `id`, use what you [defined in `settings.xml`](#authenticate-to-the-package-registry-with-maven). +- For `my-group`, use your group name. +- For `PROJECT_ID`, use your project ID, which you can view on your project's home page. +- Replace `gitlab.example.com` with your domain name. +- For retrieving artifacts, use either the + [URL-encoded](../../../api/README.md#namespaced-path-encoding) path of the group + (like `group%2Fsubgroup`) or the group's ID (like `12`). -NOTE: **Note:** -For retrieving artifacts, you can use either the -[URL encoded](../../../api/README.md#namespaced-path-encoding) path of the group -(such as `group%2Fsubgroup`) or the group's ID (such as `12`). +### Instance-level Maven endpoint -### Instance level Maven endpoint - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8274) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8274) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. If you rely on many packages, it might be inefficient to include the `repository` section -with a unique URL for each package. Instead, you can use the instance level endpoint for -all maven packages stored in GitLab and the packages you have access to are available +with a unique URL for each package. Instead, you can use the instance-level endpoint for +all Maven packages stored in GitLab. All packages you have access to are available for download. -Note that **only packages that have the same path as the project** are exposed via -the instance level endpoint. +**Only packages that have the same path as the project** are exposed by +the instance-level endpoint. -| Project | Package | Instance level endpoint available | +| Project | Package | Instance-level endpoint available | | ------- | ------- | --------------------------------- | | `foo/bar` | `foo/bar/1.0-SNAPSHOT` | Yes | | `gitlab-org/gitlab` | `foo/bar/1.0-SNAPSHOT` | No | | `gitlab-org/gitlab` | `gitlab-org/gitlab/1.0-SNAPSHOT` | Yes | -The example below shows how the relevant `repository` section of your `pom.xml` -would look like. You still need a project specific URL for uploading a package in -the `distributionManagement` section: +This example shows how relevant `repository` section of your `pom.xml`. +You still need a project-specific URL in the `distributionManagement` section. ```xml <repositories> <repository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/packages/maven</url> + <url>https://gitlab.example.com/api/v4/packages/maven</url> </repository> </repositories> <distributionManagement> <repository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven</url> </repository> <snapshotRepository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven</url> </snapshotRepository> </distributionManagement> ``` @@ -544,40 +515,35 @@ The corresponding repositories section in Gradle would look like: ```groovy repositories { maven { - url "https://gitlab.com/api/v4/packages/maven" + url "https://gitlab.example.com/api/v4/packages/maven" name "GitLab" } } ``` -The `id` must be the same with what you -[defined in `settings.xml`](#add-the-package-registry-as-a-maven-remote). - -Replace `PROJECT_ID` with your project ID which can be found on the home page -of your project. - -If you have a self-managed GitLab installation, replace `gitlab.com` with your -domain name. +- The `id` is what you [defined in `settings.xml`](#authenticate-to-the-package-registry-with-maven). +- The `PROJECT_ID` is your project ID, which you can view on your project's home page. +- Replace `gitlab.example.com` with your domain name. +- For retrieving artifacts, use either the + [URL-encoded](../../../api/README.md#namespaced-path-encoding) path of the project + (like `group%2Fproject`) or the project's ID (like `42`). However, only the + project's ID can be used for publishing. -NOTE: **Note:** -For retrieving artifacts, you can use either the -[URL encoded](../../../api/README.md#namespaced-path-encoding) path of the project -(such as `group%2Fproject`) or the project's ID (such as `42`). However, only the -project's ID can be used for uploading. +## Publish a package -## Uploading packages +After you have set up the [remote and authentication](#authenticate-to-the-package-registry-with-maven) +and [configured your project](#use-the-gitlab-endpoint-for-maven-packages), +publish a Maven artifact from your project. -Once you have set up the [remote and authentication](#add-the-package-registry-as-a-maven-remote) -and [configured your project](#configuring-your-project-to-use-the-gitlab-maven-repository-url), -test to upload a Maven artifact from a project of yours. +### Publish by using Maven -### Upload using Maven +To publish a package by using Maven: ```shell mvn deploy ``` -If the deploy is successful, you should see the build success message again: +If the deploy is successful, the build success message should be displayed: ```shell ... @@ -585,108 +551,112 @@ If the deploy is successful, you should see the build success message again: ... ``` -You should also see that the upload was uploaded to the correct registry: +The message should also show that the package was published to the correct location: ```shell -Uploading to gitlab-maven: https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.jar +Uploading to gitlab-maven: https://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.jar ``` -### Upload using Gradle +### Publish by using Gradle -Add the Gradle plugin [`maven-publish`](https://docs.gradle.org/current/userguide/publishing_maven.html) to the plugins section: +To publish a package by using Gradle: -```groovy -plugins { - id 'java' - id 'maven-publish' -} -``` +1. Add the Gradle plugin [`maven-publish`](https://docs.gradle.org/current/userguide/publishing_maven.html) to the plugins section: -Add a `publishing` section: + ```groovy + plugins { + id 'java' + id 'maven-publish' + } + ``` -```groovy -publishing { - publications { - library(MavenPublication) { - from components.java - } - } - repositories { - maven { - url "https://gitlab.com/api/v4/projects/<PROJECT_ID>/packages/maven" - credentials(HttpHeaderCredentials) { - name = "Private-Token" - value = gitLabPrivateToken // the variable resides in ~/.gradle/gradle.properties - } - authentication { - header(HttpHeaderAuthentication) - } - } - } -} -``` +1. Add a `publishing` section: + + ```groovy + publishing { + publications { + library(MavenPublication) { + from components.java + } + } + repositories { + maven { + url "https://gitlab.example.com/api/v4/projects/<PROJECT_ID>/packages/maven" + credentials(HttpHeaderCredentials) { + name = "Private-Token" + value = gitLabPrivateToken // the variable resides in ~/.gradle/gradle.properties + } + authentication { + header(HttpHeaderAuthentication) + } + } + } + } + ``` -Replace `PROJECT_ID` with your project ID which can be found on the home page -of your project. +1. Replace `PROJECT_ID` with your project ID, which can be found on your project's home page. -Run the publish task: +1. Run the publish task: -```shell -gradle publish -``` + ```shell + gradle publish + ``` -You can then navigate to your project's **Packages & Registries** page and see the uploaded -artifacts or even delete them. +Now navigate to your project's **Packages & Registries** page and view the published artifacts. -## Installing a package +## Install a package -Installing a package from the GitLab Package Registry requires that you set up -the [remote and authentication](#add-the-package-registry-as-a-maven-remote) -as above. Once this is completed, there are two ways to install a package. +To install a package from the GitLab Package Registry, you must configure +the [remote and authenticate](#authenticate-to-the-package-registry-with-maven). +When this is completed, there are two ways to install a package. -### Install using Maven with `mvn install` +### Use Maven with `mvn install` -Add the dependency manually to your project `pom.xml` file. To add the example -created above, the XML would look like: +To install a package by using `mvn install`: -```xml -<dependency> - <groupId>com.mycompany.mydepartment</groupId> - <artifactId>my-project</artifactId> - <version>1.0-SNAPSHOT</version> -</dependency> -``` +1. Add the dependency manually to your project `pom.xml` file. + To add the example created earlier, the XML would be: -Then, inside your project, run the following: + ```xml + <dependency> + <groupId>com.mycompany.mydepartment</groupId> + <artifactId>my-project</artifactId> + <version>1.0-SNAPSHOT</version> + </dependency> + ``` -```shell -mvn install -``` +1. In your project, run the following: + + ```shell + mvn install + ``` -Provided everything is set up correctly, you should see the dependency -downloaded from the GitLab Package Registry: +The message should show that the package is downloading from the Package Registry: ```shell -Downloading from gitlab-maven: http://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.pom +Downloading from gitlab-maven: http://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.pom ``` -### Install using Maven with `mvn dependency:get` +### Use Maven with `mvn dependency:get` -The second way to install packages is to use the Maven commands directly. -Inside your project directory, run: +You can install packages by using the Maven commands directly. + +1. In your project directory, run: + + ```shell + mvn dependency:get -Dartifact=com.nickkipling.app:nick-test-app:1.1-SNAPSHOT + ``` + +The message should show that the package is downloading from the Package Registry: ```shell -mvn dependency:get -Dartifact=com.nickkipling.app:nick-test-app:1.1-SNAPSHOT +Downloading from gitlab-maven: http://gitlab.example.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.pom ``` -You should see the same downloading message confirming that the project was -retrieved from the GitLab Package Registry. - TIP: **Tip:** -Both the XML block and Maven command are readily copy and pastable from the -Package details page, allowing for quick and easy installation. +In the GitLab UI, on the Package Registry page for Maven, you can view and copy these commands. -### Install using Gradle +### Use Gradle Add a [dependency](https://docs.gradle.org/current/userguide/declaring_dependencies.html) to `build.gradle` in the dependencies section: @@ -696,25 +666,25 @@ dependencies { } ``` -## Removing a package +## Remove a package -In the packages view of your project page, you can delete packages by clicking -the red trash icons or by clicking the **Delete** button on the package details -page. +For your project, go to **Packages & Registries > Package Registry**. -## Creating Maven packages with GitLab CI/CD +To remove a package, click the red trash icon or, from the package details, the **Delete** button. -Once you have your repository configured to use the GitLab Maven Repository, +## Create Maven packages with GitLab CI/CD + +After you have configured your repository to use the Package Repository for Maven, you can configure GitLab CI/CD to build new packages automatically. -### Creating Maven packages with GitLab CI/CD using Maven +### Create Maven packages with GitLab CI/CD by using Maven -The example below shows how to create a new package each time the `master` branch -is updated: +You can create a new package each time the `master` branch is updated. 1. Create a `ci_settings.xml` file that serves as Maven's `settings.xml` file. - Add the server section with the same ID you defined in your `pom.xml` file. - For example, in our case it's `gitlab-maven`: + +1. Add the `server` section with the same ID you defined in your `pom.xml` file. + For example, use `gitlab-maven` as the ID: ```xml <settings xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" @@ -735,30 +705,29 @@ is updated: </settings> ``` -1. Make sure your `pom.xml` file includes the following: +1. Make sure your `pom.xml` file includes the following. + You can either let Maven use the CI environment variables, as shown in this example, + or you can hard code your project's ID. ```xml <repositories> <repository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> </repository> </repositories> <distributionManagement> <repository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> </repository> <snapshotRepository> <id>gitlab-maven</id> - <url>https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> + <url>https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/maven</url> </snapshotRepository> </distributionManagement> ``` - TIP: **Tip:** - You can either let Maven utilize the CI environment variables or hardcode your project's ID. - 1. Add a `deploy` job to your `.gitlab-ci.yml` file: ```yaml @@ -773,16 +742,17 @@ is updated: 1. Push those files to your repository. The next time the `deploy` job runs, it copies `ci_settings.xml` to the -user's home location (in this case the user is `root` since it runs in a -Docker container), and Maven uses the configured CI -[environment variables](../../../ci/variables/README.md#predefined-environment-variables). +user's home location. In this example: + +- The user is `root`, because the job runs in a Docker container. +- Maven uses the configured CI [environment variables](../../../ci/variables/README.md#predefined-environment-variables). -### Creating Maven packages with GitLab CI/CD using Gradle +### Create Maven packages with GitLab CI/CD by using Gradle -The example below shows how to create a new package each time the `master` branch -is updated: +You can create a package each time the `master` branch +is updated. -1. Make sure you use the Job-Token authentication as described in ["Authenticating with a CI job token in Gradle"](#authenticate-with-a-ci-job-token-in-gradle). +1. Authenticate with [a CI job token in Gradle](#authenticate-with-a-ci-job-token-in-gradle). 1. Add a `deploy` job to your `.gitlab-ci.yml` file: @@ -795,11 +765,13 @@ is updated: - master ``` -1. Push those files to your repository. +1. Commit files to your repository. + +When the pipeline is successful, the package is created. ### Version validation -The version string is validated using the following regex. +The version string is validated by using the following regex. ```ruby \A(\.?[\w\+-]+\.?)+\z @@ -827,7 +799,7 @@ When you set these options, all network requests are logged and a large amount o ### Useful Maven command-line options There are some [Maven command-line options](https://maven.apache.org/ref/current/maven-embedder/cli.html) -that may be useful when performing tasks with GitLab CI/CD. +that you can use when performing tasks with GitLab CI/CD. - File transfer progress can make the CI logs hard to read. Option `-ntp,--no-transfer-progress` was added in @@ -835,7 +807,7 @@ that may be useful when performing tasks with GitLab CI/CD. Alternatively, look at `-B,--batch-mode` [or lower level logging changes.](https://stackoverflow.com/questions/21638697/disable-maven-download-progress-indication) -- Specify where to find the POM file (`-f,--file`): +- Specify where to find the `pom.xml` file (`-f,--file`): ```yaml package: @@ -852,11 +824,10 @@ that may be useful when performing tasks with GitLab CI/CD. - 'mvn -s settings/ci.xml package' ``` -### Verifying your Maven settings +### Verify your Maven settings -If you encounter issues within CI that relate to the `settings.xml` file, it might be useful -to add an additional script task or job to -[verify the effective settings](https://maven.apache.org/plugins/maven-help-plugin/effective-settings-mojo.html). +If you encounter issues within CI/CD that relate to the `settings.xml` file, try adding +an additional script task or job to [verify the effective settings](https://maven.apache.org/plugins/maven-help-plugin/effective-settings-mojo.html). The help plugin can also provide [system properties](https://maven.apache.org/plugins/maven-help-plugin/system-mojo.html), including environment variables: diff --git a/doc/user/packages/npm_registry/img/npm_package_view_v12_5.png b/doc/user/packages/npm_registry/img/npm_package_view_v12_5.png Binary files differdeleted file mode 100644 index a6f823011eb..00000000000 --- a/doc/user/packages/npm_registry/img/npm_package_view_v12_5.png +++ /dev/null diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index f15b31d8b67..feb797240f4 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -4,331 +4,378 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab NPM Registry +# NPM packages in the Package Registry > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5934) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -With the GitLab NPM Registry, every -project can have its own space to store NPM packages. +Publish NPM packages in your project's Package Registry. Then install the +packages whenever you need to use them as a dependency. - - -NOTE: **Note:** Only [scoped](https://docs.npmjs.com/misc/scope) packages are supported. -## Enabling the NPM Registry - -NOTE: **Note:** -This option is available only if your GitLab administrator has -[enabled support for the NPM registry](../../../administration/packages/index.md). - -Enabling the NPM registry makes it available for all new projects -by default. To enable it for existing projects, or if you want to disable it: - -1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. -1. Find the Packages feature and enable or disable it. -1. Click on **Save changes** for the changes to take effect. - -You should then be able to see the **Packages & Registries** section on the left sidebar. +## Build an NPM package -Before proceeding to authenticating with the GitLab NPM Registry, you should -get familiar with the package naming convention. +This section covers how to install NPM or Yarn and build a package for your +JavaScript project. -## Getting started +If you already use NPM and know how to build your own packages, go to +the [next section](#authenticate-to-the-package-registry). -This section covers how to install NPM (or Yarn) and build a package for your -JavaScript project. This is a quickstart if you are new to NPM packages. If you -are already using NPM and understand how to build your own packages, move on to -the [next section](#authenticating-to-the-gitlab-npm-registry). +### Install NPM -### Installing NPM +Install Node.js and NPM in your local development environment by following +the instructions at [npmjs.com](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm). -Follow the instructions at [npmjs.com](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) to download and install Node.js and -NPM to your local development environment. - -Once installation is complete, verify you can use NPM in your terminal by +When installation is complete, verify you can use NPM in your terminal by running: ```shell npm --version ``` -You should see the NPM version printed in the output: +The NPM version is shown in the output: ```plaintext 6.10.3 ``` -### Installing Yarn +### Install Yarn -You may want to install and use Yarn as an alternative to NPM. Follow the -instructions at [yarnpkg.com](https://classic.yarnpkg.com/en/docs/install) to install on -your development environment. +As an alternative to NPM, you can install Yarn in your local environment by following the +instructions at [yarnpkg.com](https://classic.yarnpkg.com/en/docs/install). -Once installed, you can verify that Yarn is available with the following command: +When installation is complete, verify you can use Yarn in your terminal by +running: ```shell yarn --version ``` -You should see the version printed like so: +The Yarn version is shown in the output: ```plaintext 1.19.1 ``` -### Creating a project +### Create a project -Understanding how to create a full JavaScript project is outside the scope of -this guide but you can initialize a new empty package by creating and navigating -to an empty directory and using the following command: +To create a project: -```shell -npm init -``` +1. Create an empty directory. +1. Go to the directory and initialize an empty package by running: -Or if you're using Yarn: + ```shell + npm init + ``` -```shell -yarn init -``` + Or if you're using Yarn: + + ```shell + yarn init + ``` + +1. Enter responses to the questions. Ensure the **package name** follows + the [naming convention](#package-naming-convention) and is scoped to the + project or group where the registry exists. + +A `package.json` file is created. + +## Use the GitLab endpoint for NPM packages + +To use the GitLab endpoint for NPM packages, choose an option: + +- **Project-level**: Use when you have few NPM packages and they are not in + the same GitLab group. +- **Instance-level**: Use when you have many NPM packages in different + GitLab groups or in their own namespace. Be sure to comply with the [package naming convention](#package-naming-convention). -This takes you through a series of questions to produce a `package.json` -file, which is required for all NPM packages. The most important question is the -package name. NPM packages must [follow the naming convention](#package-naming-convention) -and be scoped to the project or group where the registry exists. +Some features such as [publishing](#publish-an-npm-package) a package is only available on the project-level endpoint. -Once you have completed the setup, you are now ready to upload your package to -the GitLab registry. To get started, you need to set up authentication and then -configure GitLab as a remote registry. +## Authenticate to the Package Registry -## Authenticating to the GitLab NPM Registry +To authenticate to the Package Registry, you must use one of the following: -If a project is private or you want to upload an NPM package to GitLab, -you need to provide credentials for authentication. [Personal access tokens](../../profile/personal_access_tokens.md) -and [deploy tokens](../../project/deploy_tokens/index.md) -are preferred, but support is available for [OAuth tokens](../../../api/oauth2.md#resource-owner-password-credentials-flow). +- A [personal access token](../../../user/profile/personal_access_tokens.md) + (required for two-factor authentication (2FA)), with the scope set to `api`. +- A [deploy token](../../project/deploy_tokens/index.md), with the scope set to `read_package_registry`, `write_package_registry`, or both. +- It's not recommended, but you can use [OAuth tokens](../../../api/oauth2.md#resource-owner-password-credentials-flow). + Standard OAuth tokens cannot authenticate to the GitLab NPM Registry. You must use a personal access token with OAuth headers. +- A [CI job token](#authenticate-with-a-ci-job-token). -CAUTION: **Two-factor authentication (2FA) is only supported with personal access tokens:** -If you have 2FA enabled, you need to use a [personal access token](../../profile/personal_access_tokens.md) with OAuth headers with the scope set to `api` or a [deploy token](../../project/deploy_tokens/index.md) with `read_package_registry` or `write_package_registry` scopes. Standard OAuth tokens cannot authenticate to the GitLab NPM Registry. +### Authenticate with a personal access token or deploy token -### Authenticating with a personal access token or deploy token +To authenticate with the Package Registry, you will need a [personal access token](../../profile/personal_access_tokens.md) or [deploy token](../../project/deploy_tokens/index.md). -To authenticate with a [personal access token](../../profile/personal_access_tokens.md) or [deploy token](../../project/deploy_tokens/index.md), -set your NPM configuration: +#### Project-level NPM endpoint + +To use the [project-level](#use-the-gitlab-endpoint-for-npm-packages) NPM endpoint, set your NPM configuration: ```shell # Set URL for your scoped packages. # For example package with name `@foo/bar` will use this URL for download -npm config set @foo:registry https://gitlab.com/api/v4/packages/npm/ +npm config set @foo:registry https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/ -# Add the token for the scoped packages URL. This will allow you to download -# `@foo/` packages from private projects. -npm config set '//gitlab.com/api/v4/packages/npm/:_authToken' "<your_token>" - -# Add token for uploading to the registry. Replace <your_project_id> -# with the project you want your package to be uploaded to. -npm config set '//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "<your_token>" +# Add the token for the scoped packages URL. Replace <your_project_id> +# with the project where your package is located. +npm config set '//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "<your_token>" ``` -Replace `<your_project_id>` with your project ID which can be found on the home page -of your project and `<your_token>` with your personal access token or deploy token. - -If you have a self-managed GitLab installation, replace `gitlab.com` with your -domain name. +- `<your_project_id>` is your project ID, found on the project's home page. +- `<your_token>` is your personal access token or deploy token. +- Replace `gitlab.example.com` with your domain name. -You should now be able to download and upload NPM packages to your project. +You should now be able to publish and install NPM packages in your project. -NOTE: **Note:** -If you encounter an error message with [Yarn](https://classic.yarnpkg.com/en/), see the -[troubleshooting section](#troubleshooting). +If you encounter an error with [Yarn](https://classic.yarnpkg.com/en/), view +[troubleshooting steps](#troubleshooting). -### Using variables to avoid hard-coding auth token values +#### Instance-level NPM endpoint -To avoid hard-coding the `authToken` value, you may use a variables in its place: +To use the [instance-level](#use-the-gitlab-endpoint-for-npm-packages) NPM endpoint, set your NPM configuration: ```shell -npm config set '//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "${NPM_TOKEN}" -npm config set '//gitlab.com/api/v4/packages/npm/:_authToken' "${NPM_TOKEN}" -``` +# Set URL for your scoped packages. +# For example package with name `@foo/bar` will use this URL for download +npm config set @foo:registry https://gitlab.example.com/api/v4/packages/npm/ -Then, you could run `npm publish` either locally or via GitLab CI/CD: +# Add the token for the scoped packages URL. This will allow you to download +# `@foo/` packages from private projects. +npm config set '//gitlab.example.com/api/v4/packages/npm/:_authToken' "<your_token>" +``` -- **Locally:** Export `NPM_TOKEN` before publishing: +- `<your_token>` is your personal access token or deploy token. +- Replace `gitlab.example.com` with your domain name. - ```shell - NPM_TOKEN=<your_token> npm publish - ``` +You should now be able to publish and install NPM packages in your project. -- **GitLab CI/CD:** Set an `NPM_TOKEN` [variable](../../../ci/variables/README.md) - under your project's **Settings > CI/CD > Variables**. +If you encounter an error with [Yarn](https://classic.yarnpkg.com/en/), view +[troubleshooting steps](#troubleshooting). -### Authenticating with a CI job token +### Authenticate with a CI job token -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9104) in GitLab Premium 12.5. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9104) in GitLab Premium 12.5. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -If you’re using NPM with GitLab CI/CD, a CI job token can be used instead of a personal access token or deploy token. +If you're using NPM with GitLab CI/CD, a CI job token can be used instead of a personal access token or deploy token. The token inherits the permissions of the user that generates the pipeline. -Add a corresponding section to your `.npmrc` file: +#### Project-level NPM endpoint + +To use the [project-level](#use-the-gitlab-endpoint-for-npm-packages) NPM endpoint, add a corresponding section to your `.npmrc` file: ```ini -@foo:registry=https://gitlab.com/api/v4/packages/npm/ -//gitlab.com/api/v4/packages/npm/:_authToken=${CI_JOB_TOKEN} -//gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN} +@foo:registry=https://gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/ +//gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN} ``` -## Uploading packages +#### Instance-level NPM endpoint -Before you can upload a package, you need to specify the registry -for NPM. To do this, add the following section to the bottom of `package.json`: +To use the [instance-level](#use-the-gitlab-endpoint-for-npm-packages) NPM endpoint, add a corresponding section to your `.npmrc` file: -```json -"publishConfig": { - "@foo:registry":"https://gitlab.com/api/v4/projects/<your_project_id>/packages/npm/" -} +```ini +@foo:registry=https://gitlab.example.com/api/v4/packages/npm/ +//gitlab.example.com/api/v4/packages/npm/:_authToken=${CI_JOB_TOKEN} ``` -Replace `<your_project_id>` with your project ID, which can be found on the home -page of your project, and replace `@foo` with your own scope. - -If you have a self-managed GitLab installation, replace `gitlab.com` with your -domain name. +#### Use variables to avoid hard-coding auth token values -Once you have enabled it and set up [authentication](#authenticating-to-the-gitlab-npm-registry), -you can upload an NPM package to your project: +To avoid hard-coding the `authToken` value, you may use a variable in its place: ```shell -npm publish +npm config set '//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "${NPM_TOKEN}" +npm config set '//gitlab.example.com/api/v4/packages/npm/:_authToken' "${NPM_TOKEN}" ``` -You can then navigate to your project's **Packages & Registries** page and see the uploaded -packages or even delete them. +Then, you can run `npm publish` either locally or by using GitLab CI/CD. -Attempting to publish a package with a name that already exists within -a given scope causes a `403 Forbidden!` error. +- **Locally:** Export `NPM_TOKEN` before publishing: -## Uploading a package with the same version twice + ```shell + NPM_TOKEN=<your_token> npm publish + ``` -You cannot upload a package with the same name and version twice, unless you -delete the existing package first. This aligns with npmjs.org's behavior, with -the exception that npmjs.org does not allow users to ever publish the same version -more than once, even if it has been deleted. +- **GitLab CI/CD:** Set an `NPM_TOKEN` [variable](../../../ci/variables/README.md) + under your project's **Settings > CI/CD > Variables**. ## Package naming convention -**Packages must be scoped in the root namespace of the project**. The package -name may be anything but it is preferred that the project name be used unless -it is not possible due to a naming collision. For example: +Your NPM package name must be in the format of `@scope:package-name`. + +- The `@scope` is the root namespace of the GitLab project. It must match exactly, including the case. +- The `package-name` can be whatever you want. + +For example, if your project is `https://gitlab.example.com/my-org/engineering-group/team-amazing/analytics`, +the root namespace is `my-org`. When you publish a package, it must have `my-org` as the scope. | Project | Package | Supported | | ---------------------- | ----------------------- | --------- | -| `foo/bar` | `@foo/bar` | Yes | -| `foo/bar/baz` | `@foo/baz` | Yes | -| `foo/bar/buz` | `@foo/anything` | Yes | +| `my-org/bar` | `@my-org/bar` | Yes | +| `my-org/bar/baz` | `@my-org/baz` | Yes | +| `My-org/Bar/baz` | `@My-org/Baz` | Yes | +| `my-org/bar/buz` | `@my-org/anything` | Yes | | `gitlab-org/gitlab` | `@gitlab-org/gitlab` | Yes | | `gitlab-org/gitlab` | `@foo/bar` | No | -The regex that is used for naming is validating all package names from all package managers: +In GitLab, this regex validates all package names from all package managers: ```plaintext /\A\@?(([\w\-\.\+]*)\/)*([\w\-\.]+)@?(([\w\-\.\+]*)\/)*([\w\-\.]*)\z/ ``` -It allows for capital letters, while NPM does not, and allows for almost all of the -characters NPM allows with a few exceptions (`~` is not allowed). +This regex allows almost all of the characters that NPM allows, with a few exceptions (for example, `~` is not allowed). + +The regex also allows for capital letters, while NPM does not. Capital letters are needed because the scope must be +identical to the root namespace of the project. -NOTE: **Note:** -Capital letters are needed because the scope is required to be -identical to the top level namespace of the project. So, for example, if your -project path is `My-Group/project-foo`, your package must be named `@My-Group/any-package-name`. -`@my-group/any-package-name` will not work. +CAUTION: **Caution:** +When you update the path of a user or group, or transfer a subgroup or project, +you must remove any NPM packages first. You cannot update the root namespace +of a project with NPM packages. Make sure you update your `.npmrc` files to follow +the naming convention and run `npm publish` if necessary. -CAUTION: **When updating the path of a user/group or transferring a (sub)group/project:** -Make sure to remove any NPM packages first. You cannot update the root namespace of a project with NPM packages. Don't forget to update your `.npmrc` files to follow the above naming convention and run `npm publish` if necessary. +## Publish an NPM package -Now, you can configure your project to authenticate with the GitLab NPM -Registry. +Prerequisites: -## Installing a package +- [Authenticate](#authenticate-to-the-package-registry) to the Package Registry. +- Set a [project-level NPM endpoint](#use-the-gitlab-endpoint-for-npm-packages). -NPM packages are commonly installed using the `npm` or `yarn` commands -inside a JavaScript project. If you haven't already, set the -URL for scoped packages. You can do this with the following command: +To upload an NPM package to your project, run this command: ```shell -npm config set @foo:registry https://gitlab.com/api/v4/packages/npm/ +npm publish ``` -Replace `@foo` with your scope. +To view the package, go to your project's **Packages & Registries**. -Next, you need to ensure [authentication](#authenticating-to-the-gitlab-npm-registry) -is setup so you can successfully install the package. Once this has been -completed, you can run the following command inside your project to install a -package: +If you try to publish a package [with a name that already exists](#publishing-packages-with-the-same-name-or-version) within +a given scope, you get a `403 Forbidden!` error. -```shell -npm install @my-project-scope/my-package -``` +## Publish an NPM package by using CI/CD -Or if you're using Yarn: +Prerequisites: -```shell -yarn add @my-project-scope/my-package +- [Authenticate](#authenticate-to-the-package-registry) to the Package Registry. +- Set a [project-level NPM endpoint](#use-the-gitlab-endpoint-for-npm-packages). + +To work with NPM commands within [GitLab CI/CD](../../../ci/README.md), you can use +`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands. + +An example `.gitlab-ci.yml` file for publishing NPM packages: + +```yaml +image: node:latest + +stages: + - deploy + +deploy: + stage: deploy + script: + - echo "//gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc + - npm publish ``` -### Forwarding requests to npmjs.org +## Publishing packages with the same name or version + +You cannot publish a package if a package of the same name and version already exists. +You must delete the existing package first. + +This aligns with npmjs.org's behavior. However, npmjs.org does not ever let you publish +the same version more than once, even if it has been deleted. + +## Install a package + +NPM packages are commonly-installed by using the `npm` or `yarn` commands +in a JavaScript project. + +1. Set the URL for scoped packages by running: + + ```shell + npm config set @foo:registry https://gitlab.example.com/api/v4/packages/npm/ + ``` + + Replace `@foo` with your scope. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/55344) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.9. +1. Ensure [authentication](#authenticate-to-the-package-registry) is configured. -By default, when an NPM package is not found in the GitLab NPM Registry, the request is forwarded to [npmjs.com](https://www.npmjs.com/). +1. In your project, to install a package, run: + + ```shell + npm install @my-project-scope/my-package + ``` + + Or if you're using Yarn: + + ```shell + yarn add @my-project-scope/my-package + ``` + +In [GitLab 12.9 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/55344), +when an NPM package is not found in the Package Registry, the request is forwarded to [npmjs.com](https://www.npmjs.com/). Administrators can disable this behavior in the [Continuous Integration settings](../../admin_area/settings/continuous_integration.md). -### Installing packages from other organizations +### Install NPM packages from other organizations You can route package requests to organizations and users outside of GitLab. -To do this, add lines to your `.npmrc` file, replacing `my-org` with the namespace or group that owns your project's repository. The name is case-sensitive and must match the name of your group or namespace exactly. +To do this, add lines to your `.npmrc` file. Replace `my-org` with the namespace or group that owns your project's repository, +and use your organization's URL. The name is case-sensitive and must match the name of your group or namespace exactly. ```shell @foo:registry=https://gitlab.example.com/api/v4/packages/npm/ -//gitlab.com/api/v4/packages/npm/:_authToken= "<your_token>" -//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken= "<your_token>" +//gitlab.example.com/api/v4/packages/npm/:_authToken= "<your_token>" +//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken= "<your_token>" @my-other-org:registry=https://gitlab.example.com/api/v4/packages/npm/ -//gitlab.com/api/v4/packages/npm/:_authToken= "<your_token>" -//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken= "<your_token>" +//gitlab.example.com/api/v4/packages/npm/:_authToken= "<your_token>" +//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken= "<your_token>" ``` -## Removing a package +### NPM dependencies metadata -In the packages view of your project page, you can delete packages by clicking -the red trash icons or by clicking the **Delete** button on the package details -page. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11867) in GitLab Premium 12.6. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -## Publishing a package with CI/CD +In GitLab 12.6 and later, packages published to the Package Registry expose the following attributes to the NPM client: -To work with NPM commands within [GitLab CI/CD](./../../../ci/README.md), you can use -`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands. +- name +- version +- dist-tags +- dependencies + - dependencies + - devDependencies + - bundleDependencies + - peerDependencies + - deprecated -A simple example `.gitlab-ci.yml` file for publishing NPM packages: +## Add NPM distribution tags -```yaml -image: node:latest +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9425) in GitLab Premium 12.8. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -stages: - - deploy +You can add [distribution tags](https://docs.npmjs.com/cli/dist-tag) to newly-published packages. +Tags are optional and can be assigned to only one package at a time. -deploy: - stage: deploy - script: - - echo "//gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc - - npm publish +When you publish a package without a tag, the `latest` tag is added by default. +When you install a package without specifying the tag or version, the `latest` tag is used. + +Examples of the supported `dist-tag` commands: + +```shell +npm publish @scope/package --tag # Publish a package with new tag +npm dist-tag add @scope/package@version my-tag # Add a tag to an existing package +npm dist-tag ls @scope/package # List all tags under the package +npm dist-tag rm @scope/package@version my-tag # Delete a tag from the package +npm install @scope/package@my-tag # Install a specific tag ``` -Learn more about [using `CI_JOB_TOKEN` to authenticate to the GitLab NPM registry](#authenticating-with-a-ci-job-token). +You cannot use your `CI_JOB_TOKEN` or deploy token with the `npm dist-tag` commands. +View [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/258835) for details. + +Due to a bug in NPM 6.9.0, deleting distribution tags fails. Make sure your NPM version is 6.9.1 or later. ## Troubleshooting @@ -344,7 +391,7 @@ info No lockfile found. warning XXX: No license field [1/4] 🔍 Resolving packages... [2/4] 🚚 Fetching packages... -error An unexpected error occurred: "https://gitlab.com/api/v4/projects/XXX/packages/npm/XXX/XXX/-/XXX/XXX-X.X.X.tgz: Request failed \"404 Not Found\"". +error An unexpected error occurred: "https://gitlab.example.com/api/v4/projects/XXX/packages/npm/XXX/XXX/-/XXX/XXX-X.X.X.tgz: Request failed \"404 Not Found\"". info If you think this is a bug, please open a bug report with the information provided in "/Users/XXX/gitlab-migration/module-util/yarn-error.log". info Visit https://classic.yarnpkg.com/en/docs/cli/install for documentation about this command ``` @@ -353,14 +400,14 @@ In this case, try adding this to your `.npmrc` file (and replace `<your_token>` with your personal access token or deploy token): ```plaintext -//gitlab.com/api/v4/projects/:_authToken=<your_token> +//gitlab.example.com/api/v4/projects/:_authToken=<your_token> ``` You can also use `yarn config` instead of `npm config` when setting your auth-token dynamically: ```shell -yarn config set '//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "<your_token>" -yarn config set '//gitlab.com/api/v4/packages/npm/:_authToken' "<your_token>" +yarn config set '//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "<your_token>" +yarn config set '//gitlab.example.com/api/v4/packages/npm/:_authToken' "<your_token>" ``` ### `npm publish` targets default NPM registry (`registry.npmjs.org`) @@ -375,23 +422,20 @@ should look like: "name": "@foo/my-package", "version": "1.0.0", "description": "Example package for GitLab NPM registry", - "publishConfig": { - "@foo:registry":"https://gitlab.com/api/v4/projects/<your_project_id>/packages/npm/" - } } ``` And the `.npmrc` file should look like: ```ini -//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken=<your_token> -//gitlab.com/api/v4/packages/npm/:_authToken=<your_token> -@foo:registry=https://gitlab.com/api/v4/packages/npm/ +//gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken=<your_token> +//gitlab.example.com/api/v4/packages/npm/:_authToken=<your_token> +@foo:registry=https://gitlab.example.com/api/v4/packages/npm/ ``` ### `npm install` returns `Error: Failed to replace env in config: ${NPM_TOKEN}` -You do not need a token to run `npm install` unless your project is private (the token is only required to publish). If the `.npmrc` file was checked in with a reference to `$NPM_TOKEN`, you can remove it. If you prefer to leave the reference in, you need to set a value prior to running `npm install` or set the value using [GitLab environment variables](./../../../ci/variables/README.md): +You do not need a token to run `npm install` unless your project is private. The token is only required to publish. If the `.npmrc` file was checked in with a reference to `$NPM_TOKEN`, you can remove it. If you prefer to leave the reference in, you must set a value prior to running `npm install` or set the value by using [GitLab environment variables](../../../ci/variables/README.md): ```shell NPM_TOKEN=<your_token> npm install @@ -399,50 +443,11 @@ NPM_TOKEN=<your_token> npm install ### `npm install` returns `npm ERR! 403 Forbidden` -- Check that your token is not expired and has appropriate permissions. -- Check that [your token does not begin with `-`](https://gitlab.com/gitlab-org/gitlab/-/issues/235473). -- Check if you have attempted to publish a package with a name that already exists within a given scope. -- Ensure the scoped packages URL includes a trailing slash: - - Correct: `//gitlab.com/api/v4/packages/npm/` - - Incorrect: `//gitlab.com/api/v4/packages/npm` - -## NPM dependencies metadata - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11867) in GitLab Premium 12.6. - -Starting from GitLab 12.6, new packages published to the GitLab NPM Registry expose the following attributes to the NPM client: - -- name -- version -- dist-tags -- dependencies - - dependencies - - devDependencies - - bundleDependencies - - peerDependencies - - deprecated - -## NPM distribution tags - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9425) in GitLab Premium 12.8. - -You can add [distribution tags](https://docs.npmjs.com/cli/dist-tag) for newly published packages. -They follow NPM's convention where they are optional, and each tag can only be assigned to one -package at a time. The `latest` tag is added by default when a package is published without a tag. -The same applies to installing a package without specifying the tag or version. - -Examples of the supported `dist-tag` commands and using tags in general: - -```shell -npm publish @scope/package --tag # Publish new package with new tag -npm dist-tag add @scope/package@version my-tag # Add a tag to an existing package -npm dist-tag ls @scope/package # List all tags under the package -npm dist-tag rm @scope/package@version my-tag # Delete a tag from the package -npm install @scope/package@my-tag # Install a specific tag -``` - -NOTE: **Note:** -You cannot use your `CI_JOB_TOKEN` or deploy token with the `npm dist-tag` commands. View [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/258835) for details. +If you get this error, ensure that: -CAUTION: **Warning:** -Due to a bug in NPM 6.9.0, deleting dist tags fails. Make sure your NPM version is greater than 6.9.1. +- Your token is not expired and has appropriate permissions. +- [Your token does not begin with `-`](https://gitlab.com/gitlab-org/gitlab/-/issues/235473). +- A package with the same name doesn't already exist within the given scope. +- The scoped packages URL includes a trailing slash: + - Correct: `//gitlab.example.com/api/v4/packages/npm/` + - Incorrect: `//gitlab.example.com/api/v4/packages/npm` diff --git a/doc/user/packages/nuget_repository/img/visual_studio_nuget_source_added.png b/doc/user/packages/nuget_repository/img/visual_studio_nuget_source_added.png Binary files differindex e4f6068f28c..8c14a14e304 100644 --- a/doc/user/packages/nuget_repository/img/visual_studio_nuget_source_added.png +++ b/doc/user/packages/nuget_repository/img/visual_studio_nuget_source_added.png diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 22e1a95649d..2b61c4a6c28 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -4,34 +4,38 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab NuGet Repository +# NuGet packages in the Package Registry > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20050) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.8. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -With the GitLab NuGet Repository, every project can have its own space to store NuGet packages. +Publish NuGet packages in your project’s Package Registry. Then, install the +packages whenever you need to use them as a dependency. -The GitLab NuGet Repository works with: +The Package Registry works with: - [NuGet CLI](https://docs.microsoft.com/en-us/nuget/reference/nuget-exe-cli-reference) - [.NET Core CLI](https://docs.microsoft.com/en-us/dotnet/core/tools/) - [Visual Studio](https://visualstudio.microsoft.com/vs/) -## Setting up your development environment +## Install NuGet -[NuGet CLI 5.1 or later](https://www.nuget.org/downloads) is required. Earlier versions have not been tested -against the GitLab NuGet Repository and might not work. If you have [Visual Studio](https://visualstudio.microsoft.com/vs/), -NuGet CLI is probably already installed. +The required minimum versions are: -Alternatively, you can use [.NET SDK 3.0 or later](https://dotnet.microsoft.com/download/dotnet-core/3.0), which installs NuGet CLI. +- [NuGet CLI 5.1 or later](https://www.nuget.org/downloads). If you have + [Visual Studio](https://visualstudio.microsoft.com/vs/), the NuGet CLI is + probably already installed. +- Alternatively, you can use [.NET SDK 3.0 or later](https://dotnet.microsoft.com/download/dotnet-core/3.0), + which installs the NuGet CLI. +- NuGet protocol version 3 or later. -You can confirm that [NuGet CLI](https://www.nuget.org/) is properly installed with: +Verify that the [NuGet CLI](https://www.nuget.org/) is installed by running: ```shell nuget help ``` -You should see something similar to: +The output should be similar to: ```plaintext NuGet Version: 5.1.0.6013 @@ -43,103 +47,98 @@ Available commands: [output truncated] ``` -NOTE: **Note:** -GitLab currently only supports NuGet's protocol version 3. Earlier versions are not supported. +### Install NuGet on macOS -### macOS support +For macOS, you can use [Mono](https://www.mono-project.com/) to run the +NuGet CLI. -For macOS, you can also use [Mono](https://www.mono-project.com/) to run -the NuGet CLI. For Homebrew users, run `brew install mono` to install -Mono. Then you should be able to download the Windows C# binary -`nuget.exe` from the [NuGet CLI page](https://www.nuget.org/downloads) -and run: +1. If you use Homebrew, to install Mono, run `brew install mono`. +1. Download the Windows C# binary `nuget.exe` from the [NuGet CLI page](https://www.nuget.org/downloads). +1. Run this command: -```shell -mono nuget.exe -``` - -## Enabling the NuGet Repository - -NOTE: **Note:** -This option is available only if your GitLab administrator has -[enabled support for the Package Registry](../../../administration/packages/index.md). - -When the NuGet Repository is enabled, it is available for all new projects -by default. To enable it for existing projects, or if you want to disable it: - -1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. -1. Find the Packages feature and enable or disable it. -1. Click on **Save changes** for the changes to take effect. + ```shell + mono nuget.exe + ``` -You should then be able to see the **Packages & Registries** section on the left sidebar. +## Add the Package Registry as a source for NuGet packages -## Adding the GitLab NuGet Repository as a source to NuGet +To publish and install packages to the Package Registry, you must add the +Package Registry as a source for your packages. -You need the following: +Prerequisites: - Your GitLab username. - A personal access token or deploy token. For repository authentication: - - You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. - - You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. -- A suitable name for your source. -- Your project ID which can be found on the home page of your project. + - You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) + with the scope set to `api`. + - You can generate a [deploy token](../../project/deploy_tokens/index.md) + with the scope set to `read_package_registry`, `write_package_registry`, or + both. +- A name for your source. +- Your project ID, which is found on your project's home page. You can now add a new source to NuGet with: -- [NuGet CLI](#add-nuget-repository-source-with-nuget-cli) -- [Visual Studio](#add-nuget-repository-source-with-visual-studio). -- [.NET CLI](#add-nuget-repository-source-with-net-cli) +- [NuGet CLI](#add-a-source-with-the-nuget-cli) +- [Visual Studio](#add-a-source-with-visual-studio) +- [.NET CLI](#add-a-source-with-the-net-cli) -### Add NuGet Repository source with NuGet CLI +### Add a source with the NuGet CLI -To add the GitLab NuGet Repository as a source with `nuget`: +To add the Package Registry as a source with `nuget`: ```shell -nuget source Add -Name <source_name> -Source "https://gitlab-instance.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json" -UserName <gitlab_username or deploy_token_username> -Password <gitlab_personal_access_token or deploy_token> +nuget source Add -Name <source_name> -Source "https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json" -UserName <gitlab_username or deploy_token_username> -Password <gitlab_personal_access_token or deploy_token> ``` -Where: - -- `<source_name>` is your desired source name. +- `<source_name>` is the desired source name. For example: ```shell -nuget source Add -Name "GitLab" -Source "https://gitlab.example/api/v4/projects/10/packages/nuget/index.json" -UserName carol -Password 12345678asdf +nuget source Add -Name "GitLab" -Source "https://gitlab.example.com/api/v4/projects/10/packages/nuget/index.json" -UserName carol -Password 12345678asdf ``` -### Add NuGet Repository source with Visual Studio +### Add a source with Visual Studio + +To add the Package Registry as a source with Visual Studio: 1. Open [Visual Studio](https://visualstudio.microsoft.com/vs/). -1. Open the **FILE > OPTIONS** (Windows) or **Visual Studio > Preferences** (Mac OS). -1. In the **NuGet** section, open **Sources** to see a list of all your NuGet sources. -1. Click **Add**. -1. Fill the fields with: - - **Name**: Desired name for the source - - **Location**: `https://gitlab.com/api/v4/projects/<your_project_id>/packages/nuget/index.json` - - Replace `<your_project_id>` with your project ID. - - If you have a self-managed GitLab installation, replace `gitlab.com` with your domain name. - - **Username**: Your GitLab username or deploy token username - - **Password**: Your personal access token or deploy token +1. In Windows, select **File > Options**. On macOS, select **Visual Studio > Preferences**. +1. In the **NuGet** section, select **Sources** to view a list of all your NuGet sources. +1. Select **Add**. +1. Complete the following fields: + - **Name**: Name for the source. + - **Location**: `https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json`, + where `<your_project_id>` is your project ID, and `gitlab.example.com` is + your domain name. + - **Username**: Your GitLab username or deploy token username. + - **Password**: Your personal access token or deploy token.  1. Click **Save**. -  +The source is displayed in your list. -In case of any warning, please make sure that the **Location**, **Username**, and **Password** are correct. + -### Add NuGet Repository source with .NET CLI +If you get a warning, ensure that the **Location**, **Username**, and +**Password** are correct. -To add the GitLab NuGet Repository as a source for .NET, create a file named `nuget.config` in the root of your project with the following content: +### Add a source with the .NET CLI -```xml -<?xml version="1.0" encoding="utf-8"?> -<configuration> +To add the Package Registry as a source for .NET: + +1. In the root of your project, create a file named `nuget.config`. +1. Add this content: + + ```xml + <?xml version="1.0" encoding="utf-8"?> + <configuration> <packageSources> <clear /> - <add key="gitlab" value="https://gitlab-instance.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json" /> + <add key="gitlab" value="https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json" /> </packageSources> <packageSourceCredentials> <gitlab> @@ -147,46 +146,51 @@ To add the GitLab NuGet Repository as a source for .NET, create a file named `nu <add key="ClearTextPassword" value="<gitlab_personal_access_token or deploy_token>" /> </gitlab> </packageSourceCredentials> -</configuration> -``` + </configuration> + ``` + +## Publish a NuGet package -## Uploading packages +When publishing packages: -When uploading packages, note that: +- The Package Registry on GitLab.com can store up to 500 MB of content. + This limit is [configurable for self-managed GitLab instances](../../../administration/instance_limits.md#package-registry-limits). +- If you publish the same package with the same version multiple times, each + consecutive upload is saved as a separate file. When installing a package, + GitLab serves the most recent file. +- When publishing packages to GitLab, they aren't displayed in the packages user + interface of your project immediately. It can take up to 10 minutes to process + a package. -- The Package Registry on GitLab.com can store up to 500 MB of content. This limit is [configurable for self-managed GitLab instances](../../../administration/instance_limits.md#package-registry-limits). -- If you upload the same package with the same version multiple times, each consecutive upload - is saved as a separate file. When installing a package, GitLab serves the most recent file. -- When uploading packages to GitLab, they are not displayed in the packages UI of your project - immediately. It can take up to 10 minutes to process a package. +### Publish a package with the NuGet CLI -### Upload packages with NuGet CLI +Prerequisite: -This section assumes that your project is properly built and you already [created a NuGet package with NuGet CLI](https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package). -Upload your package using the following command: +- [A NuGet package created with NuGet CLI](https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package). + +Publish a package by running this command: ```shell nuget push <package_file> -Source <source_name> ``` -Where: - - `<package_file>` is your package filename, ending in `.nupkg`. -- `<source_name>` is the [source name used during setup](#adding-the-gitlab-nuget-repository-as-a-source-to-nuget). +- `<source_name>` is the [source name used during setup](#add-a-source-with-the-nuget-cli). + +### Publish a package with the .NET CLI + +Prerequisite: -### Upload packages with .NET CLI +[A NuGet package created with .NET CLI](https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package-dotnet-cli). -This section assumes that your project is properly built and you already [created a NuGet package with .NET CLI](https://docs.microsoft.com/en-us/nuget/create-packages/creating-a-package-dotnet-cli). -Upload your package using the following command: +Publish a package by running this command: ```shell dotnet nuget push <package_file> --source <source_name> ``` -Where: - - `<package_file>` is your package filename, ending in `.nupkg`. -- `<source_name>` is the [source name used during setup](#adding-the-gitlab-nuget-repository-as-a-source-to-nuget). +- `<source_name>` is the [source name used during setup](#add-a-source-with-the-net-cli). For example: @@ -194,16 +198,47 @@ For example: dotnet nuget push MyPackage.1.0.0.nupkg --source gitlab ``` +### Publish a NuGet package by using CI/CD + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36424) in GitLab 13.3. + +If you’re using NuGet with GitLab CI/CD, a CI job token can be used instead of a +personal access token or deploy token. The token inherits the permissions of the +user that generates the pipeline. + +This example shows how to create a new package each time the `master` branch is +updated: + +1. Add a `deploy` job to your `.gitlab-ci.yml` file: + + ```yaml + image: mcr.microsoft.com/dotnet/core/sdk:3.1 + + stages: + - deploy + + deploy: + stage: deploy + script: + - dotnet pack -c Release + - dotnet nuget add source "$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text + - dotnet nuget push "bin/Release/*.nupkg" --source gitlab + only: + - master + ``` + +1. Commit the changes and push it to your GitLab repository to trigger a new CI/CD build. + ## Install packages -### Install a package with NuGet CLI +### Install a package with the NuGet CLI CAUTION: **Warning:** -By default, `nuget` checks the official source at `nuget.org` first. If you have a package in the -GitLab NuGet Repository with the same name as a package at `nuget.org`, you must specify the source -name to install the correct package. +By default, `nuget` checks the official source at `nuget.org` first. If you have +a NuGet package in the Package Registry with the same name as a package at +`nuget.org`, you must specify the source name to install the correct package. -Install the latest version of a package using the following command: +Install the latest version of a package by running this command: ```shell nuget install <package_id> -OutputDirectory <output_directory> \ @@ -211,60 +246,24 @@ nuget install <package_id> -OutputDirectory <output_directory> \ -Source <source_name> ``` -Where: - - `<package_id>` is the package ID. - `<output_directory>` is the output directory, where the package is installed. -- `<package_version>` (Optional) is the package version. -- `<source_name>` (Optional) is the source name. +- `<package_version>` The package version. Optional. +- `<source_name>` The source name. Optional. -### Install a package with .NET CLI +### Install a package with the .NET CLI CAUTION: **Warning:** -If you have a package in the GitLab NuGet Repository with the same name as a package at a different source, -you should verify the order in which `dotnet` checks sources during install. This is defined in the -`nuget.config` file. +If you have a package in the Package Registry with the same name as a package at +a different source, verify the order in which `dotnet` checks sources during +install. This is defined in the `nuget.config` file. -Install the latest version of a package using the following command: +Install the latest version of a package by running this command: ```shell dotnet add package <package_id> \ -v <package_version> ``` -Where: - - `<package_id>` is the package ID. -- `<package_version>` (Optional) is the package version. - -## Publishing a NuGet package with CI/CD - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36424) in GitLab 13.3. - -If you’re using NuGet with GitLab CI/CD, a CI job token can be used instead of a personal access token or deploy token. -The token inherits the permissions of the user that generates the pipeline. - -This example shows how to create a new package each time the `master` branch -is updated: - -1. Add a `deploy` job to your `.gitlab-ci.yml` file: - - ```yaml - image: mcr.microsoft.com/dotnet/core/sdk:3.1 - - stages: - - deploy - - deploy: - stage: deploy - script: - - dotnet restore -p:Configuration=Release - - dotnet build -c Release - - dotnet pack -c Release - - dotnet nuget add source "$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text - - dotnet nuget push "bin/Release/*.nupkg" --source gitlab - only: - - master - ``` - -1. Commit the changes and push it to your GitLab repository to trigger a new CI build. +- `<package_version>` is the package version. Optional. diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index ae9ca5b2333..56fd4a02ba0 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -31,23 +31,30 @@ authenticate with GitLab by using the `CI_JOB_TOKEN`. CI/CD templates, which you can use to get started, are in [this repo](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). -Learn more about [using CI/CD to build Maven packages](../maven_repository/index.md#creating-maven-packages-with-gitlab-cicd), [NPM packages](../npm_registry/index.md#publishing-a-package-with-cicd), [Composer packages](../composer_repository/index.md#publish-a-composer-package-by-using-cicd), [NuGet Packages](../nuget_repository/index.md#publishing-a-nuget-package-with-cicd), [Conan Packages](../conan_repository/index.md#publish-a-conan-package-by-using-cicd), [PyPI packages](../pypi_repository/index.md#using-gitlab-ci-with-pypi-packages), and [generic packages](../generic_packages/index.md#publish-a-generic-package-by-using-cicd). +Learn more about using CI/CD to build: -If you use CI/CD to build a package, extended activity -information is displayed when you view the package details: +- [Maven packages](../maven_repository/index.md#create-maven-packages-with-gitlab-cicd) +- [NPM packages](../npm_registry/index.md#publish-an-npm-package-by-using-cicd) +- [Composer packages](../composer_repository/index.md#publish-a-composer-package-by-using-cicd) +- [NuGet packages](../nuget_repository/index.md#publish-a-nuget-package-by-using-cicd) +- [Conan packages](../conan_repository/index.md#publish-a-conan-package-by-using-cicd) +- [Generic packages](../generic_packages/index.md#publish-a-generic-package-by-using-cicd) + +If you use CI/CD to build a package, extended activity information is displayed +when you view the package details:  -When using Maven and NPM, you can view which pipeline published the package, as well as the commit and -user who triggered it. +When using Maven and NPM, you can view which pipeline published the package, and +the commit and user who triggered it. ## Download a package To download a package: 1. Go to **Packages & Registries > Package Registry**. -1. Click the name of the package you want to download. -1. In the **Activity** section, click the name of the package you want to download. +1. Select the name of the package you want to download. +1. In the **Activity** section, select the name of the package you want to download. ## Delete a package diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 33c37ee6a6c..83b29d5f53a 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -4,89 +4,83 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab PyPI Repository +# PyPI packages in the Package Registry > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -With the GitLab PyPI Repository, every project can have its own space to store PyPI packages. +Publish PyPI packages in your project’s Package Registry. Then install the +packages whenever you need to use them as a dependency. -The GitLab PyPI Repository works with: +The Package Registry works with: - [pip](https://pypi.org/project/pip/) - [twine](https://pypi.org/project/twine/) -## Setting up your development environment +## Build a PyPI package -You need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/). +This section explains how to create a PyPI package. -## Enabling the PyPI Repository +If you already use PyPI and know how to build your own packages, go to the +[next section](#authenticate-with-the-package-registry). -NOTE: **Note:** -This option is available only if your GitLab administrator has -[enabled support for the Package Registry](../../../administration/packages/index.md). +### Install pip and twine -After the PyPI Repository is enabled, it is available for all new projects -by default. To enable it for existing projects, or if you want to disable it: +Install a recent version of [pip](https://pypi.org/project/pip/) and +[twine](https://pypi.org/project/twine/). -1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. -1. Find the Packages feature and enable or disable it. -1. Click on **Save changes** for the changes to take effect. +### Create a project -You should then be able to see the **Packages & Registries** section on the left sidebar. +Create a test project. -## Getting started +1. Open your terminal. +1. Create a directory called `MyPyPiPackage`, and then go to that directory: -This section covers creating a new example PyPI package to upload. This is a -quickstart to test out the **GitLab PyPI Registry**. If you already understand how -to build and publish your own packages, move on to the [next section](#adding-the-gitlab-pypi-repository-as-a-source). + ```shell + mkdir MyPyPiPackage && cd MyPyPiPackage + ``` -### Create a project +1. Create another directory and go to it: -Understanding how to create a full Python project is outside the scope of this -guide, but you can create a small package to test out the registry. Start by -creating a new directory called `MyPyPiPackage`: + ```shell + mkdir mypypipackage && cd mypypipackage + ``` -```shell -mkdir MyPyPiPackage && cd MyPyPiPackage -``` +1. Create the required files in this directory: -After creating this, create another directory inside: + ```shell + touch __init__.py + touch greet.py + ``` -```shell -mkdir mypypipackage && cd mypypipackage -``` +1. Open the `greet.py` file, and then add: -Create two new files inside this directory to set up the basic project: + ```python + def SayHello(): + print("Hello from MyPyPiPackage") + return + ``` -```shell -touch __init__.py -touch greet.py -``` +1. Open the `__init__.py` file, and then add: -Inside `greet.py`, add the following code: + ```python + from .greet import SayHello + ``` -```python -def SayHello(): - print("Hello from MyPyPiPackage") - return -``` +1. To test the code, in your `MyPyPiPackage` directory, start the Python prompt. -Inside the `__init__.py` file, add the following: + ```shell + python + ``` -```python -from .greet import SayHello -``` +1. Run this command: -Now that the basics of our project is completed, we can test that the code runs. -Start the Python prompt inside your top `MyPyPiPackage` directory. Then run: + ```python + >>> from mypypipackage import SayHello + >>> SayHello() + ``` -```python ->>> from mypypipackage import SayHello ->>> SayHello() -``` - -You should see an output similar to the following: +A message indicates that the project was set up successfully: ```plaintext Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18) @@ -97,80 +91,81 @@ Type "help", "copyright", "credits" or "license" for more information. Hello from MyPyPiPackage ``` -Once we've verified that the sample project is working as above, we can next -work on creating a package. - ### Create a package -Inside your `MyPyPiPackage` directory, we need to create a `setup.py` file. Run -the following: +After you create a project, you can create a package. -```shell -touch setup.py -``` +1. In your terminal, go to the `MyPyPiPackage` directory. +1. Create a `setup.py` file: -This file contains all the information about our package. For more information -about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). -GitLab identifies packages based on -[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), -so ensure your package name meets these requirements. -See the [installation section](#install-packages) for more details. - -For this guide, we don't need to extensively fill out this file, simply add the -below to your `setup.py`: - -```python -import setuptools - -setuptools.setup( - name="mypypipackage", - version="0.0.1", - author="Example Author", - author_email="author@example.com", - description="A small example package", - packages=setuptools.find_packages(), - classifiers=[ - "Programming Language :: Python :: 3", - "License :: OSI Approved :: MIT License", - "Operating System :: OS Independent", - ], - python_requires='>=3.6', -) -``` + ```shell + touch setup.py + ``` -Save the file, then execute the setup like so: + This file contains all the information about the package. For more information + about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). + Because GitLab identifies packages based on + [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), + ensure your package name meets these requirements. See the [installation section](#authenticate-with-a-ci-job-token) + for details. -```shell -python3 setup.py sdist bdist_wheel -``` +1. Open the `setup.py` file, and then add basic information: + + ```python + import setuptools -If successful, you should be able to see the output in a newly created `dist` -folder. Run: + setuptools.setup( + name="mypypipackage", + version="0.0.1", + author="Example Author", + author_email="author@example.com", + description="A small example package", + packages=setuptools.find_packages(), + classifiers=[ + "Programming Language :: Python :: 3", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + ], + python_requires='>=3.6', + ) + ``` + +1. Save the file. +1. Execute the setup: + + ```shell + python3 setup.py sdist bdist_wheel + ``` + +The output should be visible in a newly-created `dist` folder: ```shell ls dist ``` -And confirm your output matches the below: +The output should appear similar to the following: ```plaintext mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz ``` -Our package is now all set up and ready to be uploaded to the **GitLab PyPI -Package Registry**. Before we do so, we next need to set up authentication. +The package is now ready to be published to the Package Registry. -## Adding the GitLab PyPI Repository as a source +## Authenticate with the Package Registry -### Authenticating with a personal access token +Before you can publish to the Package Registry, you must authenticate. -You need the following: +To do this, you can use: -- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. -- A suitable name for your source. -- Your project ID which can be found on the home page of your project. +- A [personal access token](../../../user/profile/personal_access_tokens.md) + with the scope set to `api`. +- A [deploy token](../../project/deploy_tokens/index.md) with the scope set to + `read_package_registry`, `write_package_registry`, or both. +- A [CI job token](#authenticate-with-a-ci-job-token). -Edit your `~/.pypirc` file and add the following: +### Authenticate with a personal access token + +To authenticate with a personal access token, edit the `~/.pypirc` file and add: ```ini [distutils] @@ -178,20 +173,17 @@ index-servers = gitlab [gitlab] -repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi +repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi username = __token__ password = <your personal access token> ``` -### Authenticating with a deploy token - -You need the following: +- `username` must be `__token__` exactly. +- Your project ID is on your project's home page. -- A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the `read_package_registry` and/or `write_package_registry` scopes for repository authentication. -- A suitable name for your source. -- Your project ID which can be found on the home page of your project. +### Authenticate with a deploy token -Edit your `~/.pypirc` file and add the following: +To authenticate with a deploy token, edit your `~/.pypirc` file and add: ```ini [distutils] @@ -199,21 +191,58 @@ index-servers = gitlab [gitlab] -repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi +repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi username = <deploy token username> password = <deploy token> ``` -## Uploading packages +Your project ID is on your project's home page. -When uploading packages, note that: +### Authenticate with a CI job token -- The maximum allowed size is 50 Megabytes. -- You cannot upload the same version of a package multiple times. If you try, you receive the error `Validation failed: File name has already been taken`. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4. + +To work with PyPI commands within [GitLab CI/CD](../../../ci/README.md), you +can use `CI_JOB_TOKEN` instead of a personal access token or deploy token. + +For example: + +```yaml +image: python:latest + +run: + script: + - pip install twine + - python setup.py sdist bdist_wheel + - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/* +``` + +You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check in to +GitLab: + +```ini +[distutils] +index-servers = + gitlab + +[gitlab] +repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi +username = gitlab-ci-token +password = ${env.CI_JOB_TOKEN} +``` + +## Publish a PyPI package + +When publishing packages, note that: + +- The maximum allowed size is 50 MB. +- You can't upload the same version of a package multiple times. If you try, + you'll receive the error `Validation failed: File name has already been taken`. ### Ensure your version string is valid -If your version string (for example, `0.0.1`) is invalid, it will be rejected. GitLab uses the following regex to validate the version string. +If your version string (for example, `0.0.1`) isn't valid, it will be rejected. +GitLab uses the following regex to validate the version string. ```ruby \A(?: @@ -227,118 +256,86 @@ If your version string (for example, `0.0.1`) is invalid, it will be rejected. G )\z}xi ``` -You can play around with the regex and try your version strings on [this regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV). +You can experiment with the regex and try your version strings by using this +[regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV). -For more details about the regex used, please check the [documentation here](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions)) +For more details about the regex, review this [documentation](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions). -### Upload packages with Twine +### Publish a PyPI package by using twine -If you were following the guide above, then the `MyPyPiPackage` package should -be ready to be uploaded. Run the following command: +To publish a PyPI package, run a command like: ```shell python3 -m twine upload --repository gitlab dist/* ``` -If successful, you should see the following: +This message indicates that the package was published successfully: ```plaintext -Uploading distributions to https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi +Uploading distributions to https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi Uploading mypypipackage-0.0.1-py3-none-any.whl 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s] Uploading mypypipackage-0.0.1.tar.gz 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s] ``` -This indicates that the package was uploaded successfully. You can then navigate -to your project's **Packages & Registries** page and see the uploaded packages. +To view the published package, go to your project's **Packages & Registries** +page. -If you would rather not use a `.pypirc` file to define your repository source, -you can upload to the repository with the authentication inline: +If you didn't use a `.pypirc` file to define your repository source, you can +publish to the repository with the authentication inline: ```shell -TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.com/api/v4/projects/<project_id>/packages/pypi dist/* +TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/* ``` -If you did not follow the guide above, then you need to ensure your package -has been properly built and you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). +If you didn't follow the steps on this page, ensure your package was properly +built, and that you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). -You can then upload your package using the following command: +You can then upload your package by using the following command: ```shell python -m twine upload --repository <source_name> dist/<package_file> ``` -Where: - - `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`. -- `<source_name>` is the [source name used during setup](#adding-the-gitlab-pypi-repository-as-a-source). +- `<source_name>` is the [source name used during setup](#authenticate-with-the-package-registry). -## Install packages +## Install a PyPI package -Install the latest version of a package using the following command: +To install the latest version of a package, use the following command: ```shell -pip install --extra-index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name> +pip install --extra-index-url https://__token__:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name> ``` -Where: - - `<package_name>` is the package name. - `<personal_access_token>` is a personal access token with the `read_api` scope. - `<project_id>` is the project ID. -If you were following the guide above and want to test installing the -`MyPyPiPackage` package, you can run the following: +If you were following the guide and want to install the +`MyPyPiPackage` package, you can run: ```shell -pip install mypypipackage --no-deps --extra-index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +pip install mypypipackage --no-deps --extra-index-url https://__token__:<personal_access_token>@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple ``` -This should result in the following: +This message indicates that the package was installed successfully: ```plaintext -Looking in indexes: https://__token__:****@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +Looking in indexes: https://__token__:****@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple Collecting mypypipackage - Downloading https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB) + Downloading https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB) Installing collected packages: mypypipackage Successfully installed mypypipackage-0.0.1 ``` -GitLab looks for packages using -[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), -so the characters `-`, `_`, and `.` are all treated the same and repeated characters are removed. -A `pip install` request for `my.package` looks for packages that match any of -the three characters, such as `my-package`, `my_package`, and `my....package`. - -## Using GitLab CI with PyPI packages +### Package names -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4. - -To work with PyPI commands within [GitLab CI/CD](./../../../ci/README.md), you can use -`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands. +GitLab looks for packages that use +[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names). +The characters `-`, `_`, and `.` are all treated the same, and repeated +characters are removed. -For example: - -```yaml -image: python:latest - -run: - script: - - pip install twine - - python setup.py sdist bdist_wheel - - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/* -``` - -You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check into GitLab: - -```ini -[distutils] -index-servers = - gitlab - -[gitlab] -repository = https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi -username = gitlab-ci-token -password = ${env.CI_JOB_TOKEN} -``` +A `pip install` request for `my.package` looks for packages that match any of +the three characters, such as `my-package`, `my_package`, and `my....package`. diff --git a/doc/user/packages/workflows/monorepo.md b/doc/user/packages/workflows/monorepo.md index f20f3427ac5..1e375dffe7e 100644 --- a/doc/user/packages/workflows/monorepo.md +++ b/doc/user/packages/workflows/monorepo.md @@ -68,7 +68,7 @@ package is to be uploaded to. If you set this to any project that you have access to and update any other configuration similarly depending on the package type, your packages are published to that project. This means you can publish multiple packages to one project, even if their code does not exist in the same -place. See the [project registry workflow documentation](./project_registry.md) +place. See the [project registry workflow documentation](project_registry.md) for more details. ## CI workflows for automating packaging diff --git a/doc/user/packages/workflows/project_registry.md b/doc/user/packages/workflows/project_registry.md index 24437e6dfac..a8972f05acd 100644 --- a/doc/user/packages/workflows/project_registry.md +++ b/doc/user/packages/workflows/project_registry.md @@ -67,16 +67,16 @@ If you are using NPM, this involves creating an `.npmrc` file and adding the app to your project using your project ID, then adding a section to your `package.json` file with a similar URL. Follow -the instructions in the [GitLab NPM Registry documentation](../npm_registry/index.md#authenticating-to-the-gitlab-npm-registry). After +the instructions in the [GitLab NPM Registry documentation](../npm_registry/index.md#authenticate-to-the-package-registry). After you do this, you can push your NPM package to your project using `npm publish`, as described in the -[uploading packages](../npm_registry/index.md#uploading-packages) section of the docs. +[publishing packages](../npm_registry/index.md#publish-an-npm-package) section of the docs. #### Maven If you are using Maven, this involves updating your `pom.xml` file with distribution sections, including the appropriate URL for your project, as described in the [GitLab Maven Repository documentation](../maven_repository/index.md#project-level-maven-endpoint). -Then, you need to add a `settings.xml` file and [include your access token](../maven_repository/index.md#authenticate-with-a-personal-access-token). -Now you can [deploy Maven packages](../maven_repository/index.md#uploading-packages) to your project. +Then, you need to add a `settings.xml` file and [include your access token](../maven_repository/index.md#authenticate-with-a-personal-access-token-in-maven). +Now you can [deploy Maven packages](../maven_repository/index.md#publish-a-package) to your project. #### Conan |
