diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-07-20 15:40:28 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-07-20 15:40:28 +0000 |
commit | b595cb0c1dec83de5bdee18284abe86614bed33b (patch) | |
tree | 8c3d4540f193c5ff98019352f554e921b3a41a72 /doc/user/packages | |
parent | 2f9104a328fc8a4bddeaa4627b595166d24671d0 (diff) | |
download | gitlab-ce-b595cb0c1dec83de5bdee18284abe86614bed33b.tar.gz |
Add latest changes from gitlab-org/gitlab@15-2-stable-eev15.2.0-rc42
Diffstat (limited to 'doc/user/packages')
-rw-r--r-- | doc/user/packages/conan_repository/index.md | 5 | ||||
-rw-r--r-- | doc/user/packages/container_registry/index.md | 32 | ||||
-rw-r--r-- | doc/user/packages/debian_repository/index.md | 3 | ||||
-rw-r--r-- | doc/user/packages/dependency_proxy/index.md | 8 | ||||
-rw-r--r-- | doc/user/packages/generic_packages/index.md | 22 | ||||
-rw-r--r-- | doc/user/packages/helm_repository/index.md | 5 | ||||
-rw-r--r-- | doc/user/packages/infrastructure_registry/index.md | 16 | ||||
-rw-r--r-- | doc/user/packages/npm_registry/index.md | 9 | ||||
-rw-r--r-- | doc/user/packages/package_registry/reduce_package_registry_storage.md | 36 | ||||
-rw-r--r-- | doc/user/packages/pypi_repository/index.md | 61 | ||||
-rw-r--r-- | doc/user/packages/terraform_module_registry/index.md | 13 |
11 files changed, 158 insertions, 52 deletions
diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index b3eadc13772..7260dbb616c 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -14,6 +14,9 @@ The Conan package registry for GitLab is under development and isn't ready for p limited functionality. This [epic](https://gitlab.com/groups/gitlab-org/-/epics/6816) details the remaining work and timelines to make it production ready. +NOTE: +The Conan registry is not FIPS compliant and is disabled when [FIPS mode](../../../development/fips_compliance.md) is enabled. + Publish Conan packages in your project's Package Registry. Then install the packages whenever you need to use them as a dependency. @@ -105,7 +108,7 @@ For more details about creating and managing Conan packages, see the #### Package without a username and a channel -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345055) in GitLab 14.6. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345055) in GitLab 14.6. Even though they are [recommended](https://docs.conan.io/en/latest/reference/conanfile/attributes.html#user-channel) to distinguish your package from a similarly named existing package, diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index ae64c419632..d0c771ecc41 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -208,10 +208,10 @@ should look: ```yaml build: - image: docker:19.03.12 + image: docker:20.10.16 stage: build services: - - docker:19.03.12-dind + - docker:20.10.16-dind script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - docker build -t $CI_REGISTRY/group/project/image:latest . @@ -222,10 +222,10 @@ You can also make use of [other CI/CD variables](../../../ci/variables/index.md) ```yaml build: - image: docker:19.03.12 + image: docker:20.10.16 stage: build services: - - docker:19.03.12-dind + - docker:20.10.16-dind variables: IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG script: @@ -248,9 +248,9 @@ when needed. Changes to `main` also get tagged as `latest` and deployed using an application-specific deploy script: ```yaml -image: docker:19.03.12 +image: docker:20.10.16 services: - - docker:19.03.12-dind + - docker:20.10.16-dind stages: - build @@ -323,9 +323,9 @@ Below is an example of what your `.gitlab-ci.yml` should look like: ```yaml build: - image: $CI_REGISTRY/group/project/docker:19.03.12 + image: $CI_REGISTRY/group/project/docker:20.10.16 services: - - name: $CI_REGISTRY/group/project/docker:19.03.12-dind + - name: $CI_REGISTRY/group/project/docker:20.10.16-dind alias: docker stage: build script: @@ -333,7 +333,7 @@ build: - docker run my-docker-image /script/to/run/tests ``` -If you forget to set the service alias, the `docker:19.03.12` image is unable to find the +If you forget to set the service alias, the `docker:20.10.16` image is unable to find the `dind` service, and an error like the following is thrown: ```plaintext @@ -353,7 +353,7 @@ Below is an example of what your `.gitlab-ci.yml` should look like: ```yaml build: - image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker:19.03.12 + image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker:20.10.16 services: - name: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker:18.09.7-dind alias: docker @@ -363,7 +363,7 @@ build: - docker run my-docker-image /script/to/run/tests ``` -If you forget to set the service alias, the `docker:19.03.12` image is unable to find the +If you forget to set the service alias, the `docker:20.10.16` image is unable to find the `dind` service, and an error like the following is thrown: ```plaintext @@ -438,10 +438,10 @@ stages: - clean build_image: - image: docker:19.03.12 + image: docker:20.10.16 stage: build services: - - docker:19.03.12-dind + - docker:20.10.16-dind variables: IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG script: @@ -454,10 +454,10 @@ build_image: - main delete_image: - image: docker:19.03.12 + image: docker:20.10.16 stage: clean services: - - docker:19.03.12-dind + - docker:20.10.16-dind variables: IMAGE_TAG: $CI_PROJECT_PATH:$CI_COMMIT_REF_SLUG REG_SHA256: ade837fc5224acd8c34732bf54a94f579b47851cc6a7fd5899a98386b782e228 @@ -564,7 +564,7 @@ project or branch name. Special characters can include: - Trailing hyphen/dash To get around this, you can [change the group path](../../group/index.md#change-a-groups-path), -[change the project path](../../project/settings/index.md#renaming-a-repository) or change the branch +[change the project path](../../project/settings/index.md#rename-a-repository) or change the branch name. You may also get a `404 Not Found` or `Unknown Manifest` message if you are using diff --git a/doc/user/packages/debian_repository/index.md b/doc/user/packages/debian_repository/index.md index a8f0672e376..1d846a60281 100644 --- a/doc/user/packages/debian_repository/index.md +++ b/doc/user/packages/debian_repository/index.md @@ -15,6 +15,9 @@ The Debian package registry for GitLab is under development and isn't ready for limited functionality. This [epic](https://gitlab.com/groups/gitlab-org/-/epics/6057) details the remaining work and timelines to make it production ready. +NOTE: +The Debian registry is not FIPS compliant and is disabled when [FIPS mode](../../../development/fips_compliance.md) is enabled. + Publish Debian packages in your project's Package Registry. Then install the packages whenever you need to use them as a dependency. diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index af54d928bec..4770057e4ea 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -152,17 +152,17 @@ FROM gitlab.example.com:443/my-group/dependency_proxy/containers/alpine:latest ```yaml # .gitlab-ci.yml -image: docker:19.03.12 +image: docker:20.10.16 variables: DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" services: - - docker:19.03.12-dind + - docker:20.10.16-dind build: - image: docker:19.03.12 + image: docker:20.10.16 before_script: - docker login -u $CI_DEPENDENCY_PROXY_USER -p $CI_DEPENDENCY_PROXY_PASSWORD $CI_DEPENDENCY_PROXY_SERVER script: @@ -301,7 +301,7 @@ hub_docker_quota_check: ### Dependency Proxy Connection Failure -If a service alias is not set the `docker:19.03.12` image is unable to find the +If a service alias is not set the `docker:20.10.16` image is unable to find the `dind` service, and an error like the following is thrown: ```plaintext diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index 22b792e443f..8d5fc73ad4e 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -209,3 +209,25 @@ upload: The [Write CI-CD Variables in Pipeline](https://gitlab.com/guided-explorations/cfg-data/write-ci-cd-variables-in-pipeline) project contains a working example you can use to create, upload, and download generic packages in GitLab CI/CD. It also demonstrates how to manage a semantic version for the generic package: storing it in a CI/CD variable, retrieving it, incrementing it, and writing it back to the CI/CD variable when tests for the download work correctly. + +## Troubleshooting + +### Internal Server error on large file uploads to S3 + +S3-compatible object storage [limits the size of a single PUT request to 5GB](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html). If the `aws_signature_version` is set to `2` in the [object storage connection settings](../../../administration/object_storage.md), attempting to publish a package file larger than the 5GB limit can result in a `HTTP 500: Internal Server Error` response. + +If you are receiving `HTTP 500: Internal Server Error` responses when publishing large files to S3, set the `aws_signature_version` to `4`: + +```ruby +# Consolidated Object Storage settings +gitlab_rails['object_store']['connection'] = { + # Other connection settings + `aws_signature_version` => '4' +} +# OR +# Storage-specific form settings +gitlab_rails['packages_object_store_connection'] = { + # Other connection settings + `aws_signature_version` => '4' +} +``` diff --git a/doc/user/packages/helm_repository/index.md b/doc/user/packages/helm_repository/index.md index 88ea5afad3c..07e853fa18c 100644 --- a/doc/user/packages/helm_repository/index.md +++ b/doc/user/packages/helm_repository/index.md @@ -69,6 +69,11 @@ Once built, a chart can be uploaded to the desired channel with `curl` or `helm - `<project_id>`: the project ID (like `42`). - `<channel>`: the name of the channel (like `stable`). +### Release channels + +You can publish Helm charts to channels in GitLab. Channels are a method you can use to differentiate Helm chart repositories. +For example, you can use `stable` and `devel` as channels to allow users to add the `stable` repo while `devel` charts are isolated. + ## Use CI/CD to publish a Helm package To publish a Helm package automated through [GitLab CI/CD](../../../ci/index.md), you can use diff --git a/doc/user/packages/infrastructure_registry/index.md b/doc/user/packages/infrastructure_registry/index.md index 551289a575a..e6a179c9d12 100644 --- a/doc/user/packages/infrastructure_registry/index.md +++ b/doc/user/packages/infrastructure_registry/index.md @@ -85,3 +85,19 @@ You can also remove the Infrastructure Registry for a specific project: 1. Select **Save changes**. To enable it back, follow the same steps above and toggle it on (in blue). + +## How module resolution works + +When you upload a new module, GitLab generates a path for the module, for example, `https://gitlab.example.com/parent-group/my-infra-package`. + +- This path conforms with [the Terraform spec](https://www.terraform.io/internals/module-registry-protocol). +- The name of the path must be unique within the namespace. + +For projects in subgroups, GitLab checks that the module name does not already exist anywhere in the namespace, including all subgroups and the parent group. + +For example, if: + +- The project is `gitlab.example.com/parent-group/sub-group/my-project`. +- The infrastructure package is `my-infra-package`. + +The project name must be unique in all projects in all groups under `parent-group`. diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index bdcbea68568..7ea3c1aa0c8 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -145,6 +145,9 @@ If you encounter an error with [Yarn](https://classic.yarnpkg.com/en/), view #### Instance-level npm endpoint +NOTE: +Note: Using `CI_JOB_TOKEN` to install npm packages with dependencies in another project will give you 404 errors. You can use a [personal access token](../../profile/personal_access_tokens.md) as a workaround. [GitLab-#352962](https://gitlab.com/gitlab-org/gitlab/-/issues/352962) proposes a fix to this bug. + To use the [instance-level](#use-the-gitlab-endpoint-for-npm-packages) npm endpoint, set your npm configuration: ```shell @@ -232,6 +235,12 @@ When you use the [instance-level endpoint](#use-the-gitlab-endpoint-for-npm-pack example, a package `@MyScope/package-name` in GitLab becomes `@myscope/package-name` for npm. - The `package-name` can be whatever you want. +NOTE: +The value used for the `@scope` is the root of the project that will end up hosting the packages and not the root +of the project with the source code of the package itself. For example, assume your package source code is located +at `source-code-group/package-code` and deployed to a package registry inside `registries-group/registry-project`. +In this case, the `@scope` needs to be `@registries-group` and not `@source-code-group`. + For example, if your project is `https://gitlab.example.com/my-org/engineering-group/team-amazing/analytics`, the root namespace is `my-org`. When you publish a package, it must have `my-org` as the scope. diff --git a/doc/user/packages/package_registry/reduce_package_registry_storage.md b/doc/user/packages/package_registry/reduce_package_registry_storage.md index ed4ef1665bc..4a03bd9e8a0 100644 --- a/doc/user/packages/package_registry/reduce_package_registry_storage.md +++ b/doc/user/packages/package_registry/reduce_package_registry_storage.md @@ -50,3 +50,39 @@ To delete package files in the UI, from your group or project: 1. Expand the ellipsis and select **Delete file**. The package files are permanently deleted. + +## Cleanup policy + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346153) in GitLab 15.2. + +Depending on the number of packages to remove, the process of manually deleting the packages can take a long time to finish. +A cleanup policy defines a set of rules that, applied to a project, defines which package files you can automatically delete. + +### Enable the cleanup policy + +By default, the packages cleanup policy is disabled. To enable it: + +1. Go to your project **Settings > Packages & Registries**. +1. Expand **Manage storage used by package assets**. +1. Set the rules appropriately. + +NOTE: +To access these project settings, you must be at least a maintainer on the related project. + +### Available rules + +- `Number of duplicated assets to keep`. The number of duplicated assets to keep. Some package formats allow you + to upload more than one copy of an asset. You can limit the number of duplicated assets to keep and automatically + delete the oldest files once the limit is reached. + +### Set cleanup limits to conserve resources + +A background process executes the package-cleanup policies. This process can take a long time to finish and consumes +server resources while it is running. + +You can use the following setting to limit the number of cleanup workers: + +- `package_registry_cleanup_policies_worker_capacity`: the maximum number of cleanup workers running concurrently. + This number must be greater than or equal to `0`. + We recommend starting with a low number and increasing it after monitoring the resources used by the background workers. + To remove all workers and not execute the cleanup policies, set this to `0`. The default value is `2`. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index eee6d55a3ce..b8996dc2963 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -99,45 +99,54 @@ Hello from MyPyPiPackage After you create a project, you can create a package. 1. In your terminal, go to the `MyPyPiPackage` directory. -1. Create a `setup.py` file: +1. Create a `pyproject.toml` file: ```shell - touch setup.py + touch pyproject.toml ``` This file contains all the information about the package. For more information - about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). + about this file, see [creating `pyproject.toml`](https://packaging.python.org/en/latest/tutorials/packaging-projects/#creating-pyproject-toml). Because GitLab identifies packages based on [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), ensure your package name meets these requirements. See the [installation section](#authenticate-with-a-ci-job-token) for details. -1. Open the `setup.py` file, and then add basic information: - - ```python - import setuptools - - setuptools.setup( - name="mypypipackage", - version="0.0.1", - author="Example Author", - author_email="author@example.com", - description="A small example package", - packages=setuptools.find_packages(), - classifiers=[ - "Programming Language :: Python :: 3", - "License :: OSI Approved :: MIT License", - "Operating System :: OS Independent", - ], - python_requires='>=3.6', - ) +1. Open the `pyproject.toml` file, and then add basic information: + + ```toml + [build-system] + requires = ["setuptools>=61.0"] + build-backend = "setuptools.build_meta" + + [project] + name = "mypypipackage" + version = "0.0.1" + authors = [ + { name="Example Author", email="author@example.com" }, + ] + description = "A small example package" + requires-python = ">=3.7" + classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: OS Independent", + ] + + [tool.setuptools.packages] + find = {} ``` 1. Save the file. -1. Execute the setup: +1. Install the package build library: + + ```shell + pip install build + ``` + +1. Build the package: ```shell - python3 setup.py sdist bdist_wheel + python -m build ``` The output should be visible in a newly-created `dist` folder: @@ -218,8 +227,8 @@ image: python:latest run: script: - - pip install twine - - python setup.py sdist bdist_wheel + - pip install build twine + - python -m build - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/* ``` diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md index 42c85ae9d41..2668b8b35ac 100644 --- a/doc/user/packages/terraform_module_registry/index.md +++ b/doc/user/packages/terraform_module_registry/index.md @@ -108,7 +108,7 @@ Where `<namespace>` is the [namespace](../../../user/group/index.md#namespaces) To work with Terraform modules in [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` in place of the personal access token in your commands. -For example: +For example, this job uploads a new module for the `local` [system provider](https://registry.terraform.io/browse/providers) and uses the module version from the Git commit tag: ```yaml stages: @@ -121,15 +121,18 @@ upload: TERRAFORM_MODULE_DIR: ${CI_PROJECT_DIR} # The path to your Terraform module TERRAFORM_MODULE_NAME: ${CI_PROJECT_NAME} # The name of your Terraform module TERRAFORM_MODULE_SYSTEM: local # The system or provider your Terraform module targets (ex. local, aws, google) - TERRAFORM_MODULE_VERSION: ${CI_COMMIT_TAG} # The version of your Terraform module to be published to your project's registry + TERRAFORM_MODULE_VERSION: ${CI_COMMIT_TAG} # Tag commits with SemVer for the version of your Terraform module to be published script: - - tar -cvzf ${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz -C ${TERRAFORM_MODULE_DIR} --exclude=./.git . - - 'curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file ${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/terraform/modules/${TERRAFORM_MODULE_NAME}/${TERRAFORM_MODULE_SYSTEM}/${TERRAFORM_MODULE_VERSION}/file' + - TERRAFORM_MODULE_NAME=$(echo "${TERRAFORM_MODULE_NAME}" | tr " _" -) # module-name must not have spaces or underscores, so translate them to hyphens + - tar -vczf ${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz -C ${TERRAFORM_MODULE_DIR} --exclude=./.git . + - 'curl --location --header "JOB-TOKEN: ${CI_JOB_TOKEN}" + --upload-file ${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz + ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/terraform/modules/${TERRAFORM_MODULE_NAME}/${TERRAFORM_MODULE_SYSTEM}/${TERRAFORM_MODULE_VERSION}/file' rules: - if: $CI_COMMIT_TAG ``` -To trigger this upload job, add a Git tag to your commit. The `rules:if: $CI_COMMIT_TAG` defines this so that not every commit to your repo triggers the upload. +To trigger this upload job, add a Git tag to your commit. Ensure the tag follows the [Semantic Versioning Specification](https://semver.org/) that Terraform requires. The `rules:if: $CI_COMMIT_TAG` ensures that only tagged commits to your repo trigger the module upload job. For other ways to control jobs in your CI/CD pipeline, refer to the [`.gitlab-ci.yml`](../../../ci/yaml/index.md) keyword reference. ## Example projects |