diff options
Diffstat (limited to 'doc/user/packages')
-rw-r--r-- | doc/user/packages/conan_repository/index.md | 4 | ||||
-rw-r--r-- | doc/user/packages/container_registry/index.md | 87 | ||||
-rw-r--r-- | doc/user/packages/generic_packages/index.md | 16 | ||||
-rw-r--r-- | doc/user/packages/index.md | 27 | ||||
-rw-r--r-- | doc/user/packages/maven_repository/index.md | 2 | ||||
-rw-r--r-- | doc/user/packages/npm_registry/index.md | 45 | ||||
-rw-r--r-- | doc/user/packages/rubygems_registry/index.md | 13 |
7 files changed, 147 insertions, 47 deletions
diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index 9df4aeb404a..53d191cbcfe 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -186,6 +186,10 @@ To authenticate to the Package Registry, you need one of the following: scope set to `read_package_registry`, `write_package_registry`, or both. - A [CI job token](#publish-a-conan-package-by-using-cicd). +NOTE: +Packages from private and internal projects are hidden if you are not +authenticated. If you try to search or download a package from a private or internal project without authenticating, you will receive the error `unable to find the package in remote` in the Conan client. + ### Add your credentials to the GitLab remote Associate your token with the GitLab remote, so that you don't have to diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index bc96d3c937c..6d7b009bb09 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -698,6 +698,13 @@ You can, however, remove the Container Registry for a project: The **Packages & Registries > Container Registry** entry is removed from the project's sidebar. +## Manifest lists and garbage collection + +Manifest lists are commonly used for creating multi-architecture images. If you rely on manifest +lists, you should tag all the individual manifests referenced by a list in their respective +repositories, and not just the manifest list itself. This ensures that those manifests aren't +garbage collected, as long as they have at least one tag pointing to them. + ## Troubleshooting the GitLab Container Registry ### Docker connection error @@ -729,20 +736,78 @@ As a workaround, you should include the architecture in the tag name of individu ### The cleanup policy doesn't delete any tags -In GitLab 13.6 and earlier, when you run the cleanup policy, -you may expect it to delete tags and it does not. +There can be different reasons behind this: + +- In GitLab 13.6 and earlier, when you run the cleanup policy you may expect it to delete tags and + it does not. This occurs when the cleanup policy is saved without editing the value in the + **Remove tags matching** field. This field has a grayed out `.*` value as a placeholder. Unless + `.*` (or another regex pattern) is entered explicitly into the field, a `nil` value is submitted. + This value prevents the saved cleanup policy from matching any tags. As a workaround, edit the + cleanup policy. In the **Remove tags matching** field, enter `.*` and save. This value indicates + that all tags should be removed. + +- If you are on GitLab self-managed instances and you have 1000+ tags in a container repository, you + might run into a [Container Registry token expiration issue](https://gitlab.com/gitlab-org/gitlab/-/issues/288814), + with `error authorizing context: invalid token` in the logs. + + To fix this, there are two workarounds: + + - If you are on GitLab 13.9 or later, you can [set limits for the cleanup policy](#set-cleanup-limits-to-conserve-resources). + This limits the cleanup execution in time, and avoids the expired token error. + + - Extend the expiration delay of the Container Registry authentication tokens. This defaults to 5 + minutes. You can set a custom value by running + `ApplicationSetting.last.update(container_registry_token_expire_delay: <integer>)` in the Rails + console, where `<integer>` is the desired number of minutes. For reference, 15 minutes is the + value currently in use for GitLab.com. Be aware that by extending this value you increase the + time required to revoke permissions. -This issue occurs when the cleanup policy was saved without -editing the value in the **Remove tags matching** field. +If the previous fixes didn't work or you are on earlier versions of GitLab, you can generate a list +of the tags that you want to delete, and then use that list to delete the tags. To do this, follow +these steps: -This field had a grayed out `.*` value as a placeholder. -Unless `.*` (or other regex pattern) was entered explicitly into the -field, a `nil` value was submitted. This value prevents the -saved cleanup policy from matching any tags. +1. Run the following shell script. The command just before the `for` loop ensures that + `list_o_tags.out` is always reinitialized when starting the loop. After running this command, all + the tags' names will be in the `list_o_tags.out` file: + + ```shell + # Get a list of all tags in a certain container repository while considering [pagination](../../../api/README.md#pagination) + echo -n "" > list_o_tags.out; for i in {1..N}; do curl --header 'PRIVATE-TOKEN: <PAT>' "https://gitlab.example.com/api/v4/projects/<Project_id>/registry/repositories/<container_repo_id>/tags?per_page=100&page=${i}" | jq '.[].name' | sed 's:^.\(.*\).$:\1:' >> list_o_tags.out; done + ``` + +1. Remove from the `list_o_tags.out` file any tags that you want to keep. Here are some example + `sed` commands for this. Note that these commands are simply examples. You may change them to + best suit your needs: + + ```shell + # Remove the `latest` tag from the file + sed -i '/latest/d' list_o_tags.out -As a workaround, edit the cleanup policy. In the **Remove tags matching** -field, enter `.*` and save. This value indicates that all tags should -be removed. + # Remove the first N tags from the file + sed -i '1,Nd' list_o_tags.out + + # Remove the tags starting with `Av` from the file + sed -i '/^Av/d' list_o_tags.out + + # Remove the tags ending with `_v3` from the file + sed -i '/_v3$/d' list_o_tags.out + ``` + + If you are running macOS, you must add `.bak` to the commands. For example: + + ```shell + sed -i .bak '/latest/d' list_o_tags.out + ``` + +1. Double-check the `list_o_tags.out` file to make sure it contains only the tags that you want to + delete. + +1. Run this shell script to delete the tags in the `list_o_tags.out` file: + + ```shell + # loop over list_o_tags.out to delete a single tag at a time + while read -r LINE || [[ -n $LINE ]]; do echo ${LINE}; curl --request DELETE --header 'PRIVATE-TOKEN: <PAT>' "https://gitlab.example.com/api/v4/projects/<Project_id>/registry/repositories/<container_repo_id>/tags/${LINE}"; sleep 0.1; echo; done < list_o_tags.out > delete.logs + ``` ### Troubleshoot as a GitLab server admin diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index 57d6245dd96..e20ac57e64f 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -69,22 +69,6 @@ Example response: } ``` -Example request using a deploy token: - -```shell -curl --header "DEPLOY-TOKEN: <deploy_token>" \ - --upload-file path/to/file.txt \ - "https://gitlab.example.com/api/v4/projects/24/packages/generic/my_package/0.0.1/file.txt?status=hidden" -``` - -Example response: - -```json -{ - "message":"201 Created" -} -``` - ## Download package file Download a package file. diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 74072aa95e1..b871a08c133 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -12,22 +12,17 @@ packages, which can be easily consumed as a dependency in downstream projects. The Package Registry supports the following formats: -<div class="row"> -<div class="col-md-9"> -<table align="left" style="width:50%"> -<tr style="background:#dfdfdf"><th>Package type</th><th>GitLab version</th></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/composer_repository/index.html">Composer</a></td><td>13.2+</td></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/conan_repository/index.html">Conan</a></td><td>12.6+</td></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/go_proxy/index.html">Go</a></td><td>13.1+</td></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/maven_repository/index.html">Maven</a></td><td>11.3+</td></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/npm_registry/index.html">npm</a></td><td>11.7+</td></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/nuget_repository/index.html">NuGet</a></td><td>12.8+</td></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/pypi_repository/index.html">PyPI</a></td><td>12.10+</td></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/generic_packages/index.html">Generic packages</a></td><td>13.5+</td></tr> -<tr><td><a href="https://docs.gitlab.com/ee/user/packages/rubygems_registry/index.html">RubyGems</a></td><td>13.10+</td></tr> -</table> -</div> -</div> +| Package type | GitLab version | +| ------------ | -------------- | +| [Composer](composer_repository/index.md) | 13.2+ | +| [Conan](conan_repository/index.md) | 12.6+ | +| [Go](go_proxy/index.md) | 13.1+ | +| [Maven](maven_repository/index.md) | 11.3+ | +| [npm](npm_registry/index.md) | 11.7+ | +| [NuGet](nuget_repository/index.md) | 12.8+ | +| [PyPI](pypi_repository/index.md) | 12.10+ | +| [Generic packages](generic_packages/index.md) | 13.5+ | +| [Ruby gems](rubygems_registry/index.md) | 13.10+ | You can also use the [API](../../api/packages.md) to administer the Package Registry. diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index d4dc9f0ae78..ba7b55dc47d 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -625,7 +625,7 @@ In the UI: 1. For your group, go to **Settings > Packages & Registries**. 1. Expand the **Package Registry** section. 1. Turn on the **Reject duplicates** toggle. -1. Optional. To allow some duplicate packages, in the **Exceptions** box, enter a regex pattern that matches the names of packages you want to allow. +1. Optional. To allow some duplicate packages, in the **Exceptions** box, enter a regex pattern that matches the names and/or versions of packages you want to allow. Your changes are automatically saved. diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index b6312002184..ace432b305f 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -14,6 +14,9 @@ packages whenever you need to use them as a dependency. Only [scoped](https://docs.npmjs.com/misc/scope/) packages are supported. +For documentation of the specific API endpoints that the npm package manager +client uses, see the [npm API documentation](../../../api/packages/npm.md). + ## Build an npm package This section covers how to install npm or Yarn and build a package for your @@ -43,7 +46,7 @@ The npm version is shown in the output: ### Install Yarn As an alternative to npm, you can install Yarn in your local environment by following the -instructions at [yarnpkg.com](https://classic.yarnpkg.com/en/docs/install). +instructions at [classic.yarnpkg.com](https://classic.yarnpkg.com/en/docs/install). When installation is complete, verify you can use Yarn in your terminal by running: @@ -305,6 +308,46 @@ See the [Publish npm packages to the GitLab Package Registry using semantic-release](../../../ci/examples/semantic-release.md) step-by-step guide and demo project for a complete example. +## Configure the GitLab npm registry with Yarn 2 + +You can get started with Yarn 2 by following the documentation at +[https://yarnpkg.com/getting-started/install](https://yarnpkg.com/getting-started/install). + +To publish and install with the project-level npm endpoint, set the following configuration in +`.yarnrc.yml`: + +```yaml +npmScopes: + foo: + npmRegistryServer: "https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/" + npmPublishRegistry: "https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/" + +npmRegistries: + //gitlab.example.com/api/v4/projects/<your_project_id>/packages/npm/: + npmAlwaysAuth: true + npmAuthToken: "<your_token>" +``` + +For the instance-level npm endpoint, use this Yarn 2 configuration in `.yarnrc.yml`: + +```yaml +npmScopes: + foo: + npmRegistryServer: "https://gitlab.example.com/api/v4/packages/npm/" + +npmRegistries: + //gitlab.example.com/api/v4/packages/npm/: + npmAlwaysAuth: true + npmAuthToken: "<your_token>" +``` + +In this configuration: + +- Replace `<your_token>` with your personal access token or deploy token. +- Replace `<your_project_id>` with your project's ID, which you can find on the project's home page. +- Replace `gitlab.example.com` with your domain name. +- Your scope is `foo`, without `@`. + ## Publishing packages with the same name or version You cannot publish a package if a package of the same name and version already exists. diff --git a/doc/user/packages/rubygems_registry/index.md b/doc/user/packages/rubygems_registry/index.md index aa50bc6c2bc..e4d297ac1d8 100644 --- a/doc/user/packages/rubygems_registry/index.md +++ b/doc/user/packages/rubygems_registry/index.md @@ -44,7 +44,7 @@ Feature.enable(:rubygem_packages, Project.find(1)) Feature.disable(:rubygem_packages, Project.find(2)) ``` -## Create a Ruby Gem +## Create a Ruby gem If you need help creating a Ruby gem, see the [RubyGems documentation](https://guides.rubygems.org/make-your-own-gem/). @@ -80,10 +80,19 @@ you can use `CI_JOB_TOKEN` instead of a personal access token or deploy token. For example: ```yaml -image: ruby:latest +# assuming a my_gem.gemspec file is present in the repository with the version currently set to 0.0.1 +image: ruby run: + before_script: + - mkdir ~/.gem + - echo "---" > ~/.gem/credentials + - | + echo "https://gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/rubygems: '${CI_JOB_TOKEN}'" >> ~/.gem/credentials + - chmod 0600 ~/.gem/credentials # rubygems requires 0600 permissions on the credentials file script: + - gem build my_gem + - gem push my_gem-0.0.1.gem --host https://gitlab.example.com/api/v4/projects/${CI_PROJECT_ID}/packages/rubygems ``` You can also use `CI_JOB_TOKEN` in a `~/.gem/credentials` file that you check in to |