diff options
Diffstat (limited to 'doc/user/packages')
26 files changed, 676 insertions, 97 deletions
diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index ffbd8a848b5..a56a67d4959 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -20,7 +20,7 @@ by default. To enable it for existing projects, or if you want to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. ## Getting started @@ -108,14 +108,19 @@ conan search Hello* --all --remote=gitlab ## Authenticating to the GitLab Conan Repository -You will need to generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. +You will need a personal access token or deploy token. + +For repository authentication: + +- You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. +- You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. ### Adding a Conan user to the GitLab remote Once you have a personal access token and have [set your Conan remote](#adding-the-gitlab-package-registry-as-a-conan-remote), you can associate the token with the remote so you do not have to explicitly add them to each Conan command you run: ```shell -conan user <gitlab-username> -r gitlab -p <personal_access_token> +conan user <gitlab_username or deploy_token_username> -r gitlab -p <personal_access_token or deploy_token> ``` Note: **Note** @@ -130,7 +135,7 @@ Alternatively, you could explicitly include your credentials in any given comman For example: ```shell -CONAN_LOGIN_USERNAME=<gitlab-username> CONAN_PASSWORD=<personal_access_token> conan upload Hello/0.1@my-group+my-project/beta --all --remote=gitlab +CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> conan upload Hello/0.1@my-group+my-project/beta --all --remote=gitlab ``` ### Setting a default remote to your project (optional) @@ -148,7 +153,7 @@ This functionality is best suited for when you want to consume or install packag The rest of the example commands in this documentation assume that you have added a Conan user with your credentials to the `gitlab` remote and will not include the explicit credentials or remote option, but be aware that any of the commands could be run without having added a user or default remote: ```shell -`CONAN_LOGIN_USERNAME=<gitlab-username> CONAN_PASSWORD=<personal_access_token> <conan command> --remote=gitlab +`CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> <conan command> --remote=gitlab ``` ## Uploading a package @@ -274,7 +279,7 @@ To work with Conan commands within [GitLab CI/CD](./../../../ci/README.md), you It is easiest to provide the `CONAN_LOGIN_USERNAME` and `CONAN_PASSWORD` with each Conan command in your `.gitlab-ci.yml` file: -```yml +```yaml image: conanio/gcc7 create_package: diff --git a/doc/user/packages/container_registry/img/container_registry_group_repositories_v12_10.png b/doc/user/packages/container_registry/img/container_registry_group_repositories_v12_10.png Binary files differdeleted file mode 100644 index e2b606d024f..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_group_repositories_v12_10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_0.png b/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_0.png Binary files differnew file mode 100644 index 00000000000..14119abd56a --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_0.png diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_v12_10.png b/doc/user/packages/container_registry/img/container_registry_repositories_v12_10.png Binary files differdeleted file mode 100644 index 9e113be0a26..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repositories_v12_10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_v13_0.png b/doc/user/packages/container_registry/img/container_registry_repositories_v13_0.png Binary files differnew file mode 100644 index 00000000000..7286007b953 --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_repositories_v13_0.png diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v12_10.png b/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v12_10.png Binary files differdeleted file mode 100644 index e94aab58a1d..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v12_10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_0.png b/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_0.png Binary files differnew file mode 100644 index 00000000000..f7c3aafcc8e --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_0.png diff --git a/doc/user/packages/container_registry/img/container_registry_repository_details_v12.10.png b/doc/user/packages/container_registry/img/container_registry_repository_details_v12.10.png Binary files differdeleted file mode 100644 index b911ffea935..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repository_details_v12.10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png b/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png Binary files differnew file mode 100644 index 00000000000..088e80221de --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png diff --git a/doc/user/packages/container_registry/img/container_registry_tags_v12_10.png b/doc/user/packages/container_registry/img/container_registry_tags_v12_10.png Binary files differdeleted file mode 100644 index 40abc7eda9b..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_tags_v12_10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/expiration-policy-app.png b/doc/user/packages/container_registry/img/expiration-policy-app.png Binary files differdeleted file mode 100644 index e2d3d668e38..00000000000 --- a/doc/user/packages/container_registry/img/expiration-policy-app.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png b/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png Binary files differnew file mode 100644 index 00000000000..93c9e00a8f5 --- /dev/null +++ b/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index 5505a4503ca..5e642e1e21c 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -8,6 +8,7 @@ > login to GitLab's Container Registry. > - Multiple level image names support was added in GitLab 9.1. > - The group level Container Registry was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23315) in GitLab 12.10. +> - Searching by image repository name was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31322) in GitLab 13.0. NOTE: **Note:** This document is the user guide. To learn how to enable GitLab Container @@ -19,14 +20,14 @@ have its own space to store its Docker images. You can read more about Docker Registry at <https://docs.docker.com/registry/introduction/>. - + ## Enable the Container Registry for your project CAUTION: **Warning:** The Container Registry follows the visibility settings of the project. If the project is public, so is the Container Registry. -If you cannot find the **Packages > Container Registry** entry under your +If you cannot find the **Packages & Registries > Container Registry** entry under your project's sidebar, it is not enabled in your GitLab instance. Ask your administrator to enable GitLab Container Registry following the [administration documentation](../../../administration/packages/container_registry.md). @@ -44,7 +45,7 @@ project: projects this might be enabled by default. For existing projects (prior GitLab 8.8), you will have to explicitly enable it. 1. Press **Save changes** for the changes to take effect. You should now be able - to see the **Packages > Container Registry** link in the sidebar. + to see the **Packages & Registries > Container Registry** link in the sidebar. ## Control Container Registry from within GitLab @@ -53,13 +54,14 @@ for both projects and groups. ### Control Container Registry for your project -Navigate to your project's **{package}** **Packages > Container Registry**. +Navigate to your project's **{package}** **Packages & Registries > Container Registry**. - + This view will: - Show all the image repositories that belong to the project. +- Allow you to filter image repositories by their name. - Allow you to [delete](#delete-images-from-within-gitlab) one or more image repository. - Allow you to navigate to the image repository details page. - Show a **Quick start** dropdown with the most common commands to log in, build and push @@ -67,9 +69,9 @@ This view will: ### Control Container Registry for your group -Navigate to your groups's **{package}** **Packages > Container Registry**. +Navigate to your groups's **{package}** **Packages & Registries > Container Registry**. - + This view will: @@ -81,7 +83,7 @@ This view will: Clicking on the name of any image repository will navigate to the details. - + NOTE: **Note:** The following page has the same functionalities both in the **Group level container registry** @@ -108,7 +110,7 @@ For more information on running Docker containers, visit the ## Authenticating to the GitLab Container Registry -If you visit the **Packages > Container Registry** link under your project's +If you visit the **Packages & Registries > Container Registry** link under your project's menu, you can see the explicit instructions to login to the Container Registry using your GitLab credentials. @@ -135,7 +137,7 @@ The minimum scope needed for both of them is `read_registry`. Example of using a token: -```sh +```shell docker login registry.example.com -u <username> -p <token> ``` @@ -152,14 +154,14 @@ docker push registry.example.com/group/project/image Your image will be named after the following scheme: -```text +```plaintext <registry URL>/<namespace>/<project>/<image> ``` GitLab supports up to three levels of image repository names. The following examples of image tags are valid: -```text +```plaintext registry.example.com/group/project:some-tag registry.example.com/group/project/image:latest registry.example.com/group/project/my/image:rc1 @@ -204,7 +206,7 @@ Available for all projects, though more suitable for public ones: your Docker images and has read/write access to the Registry. This is ephemeral, so it's only valid for one job. You can use the following example as-is: - ```sh + ```shell docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY ``` @@ -219,7 +221,7 @@ For private and internal projects: Replace the `<username>` and `<access_token>` in the following example: - ```sh + ```shell docker login -u <username> -p <access_token> $CI_REGISTRY ``` @@ -229,7 +231,7 @@ For private and internal projects: Once created, you can use the special environment variables, and GitLab CI/CD will fill them in for you. You can use the following example as-is: - ```sh + ```shell docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY ``` @@ -389,7 +391,7 @@ the deleted images. To delete images from within GitLab: -1. Navigate to your project's or group's **{package}** **Packages > Container Registry**. +1. Navigate to your project's or group's **{package}** **Packages & Registries > Container Registry**. 1. From the **Container Registry** page, you can select what you want to delete, by either: @@ -401,7 +403,7 @@ To delete images from within GitLab: 1. In the dialog box, click **Remove tag**. -  +  ### Delete images using the API @@ -490,7 +492,7 @@ older tags and images are regularly removed from the Container Registry. NOTE: **Note:** Expiration policies for projects created before GitLab 12.8 may be enabled by an admin in the [CI/CD Package Registry settings](./../../admin_area/settings/index.md#cicd). -Note the inherant [risks involved](./index.md#use-with-external-container-registries). +Note the inherent [risks involved](./index.md#use-with-external-container-registries). It is possible to create a per-project expiration policy, so that you can make sure that older tags and images are regularly removed from the Container Registry. @@ -503,23 +505,25 @@ then goes through a process of excluding tags from it until only the ones to be 1. Evaluates the `name_regex`, excluding non-matching names from the list. 1. Excludes any tags that do not have a manifest (not part of the options). 1. Orders the remaining tags by `created_date`. -1. Excludes from the list the N tags based on the `keep_n` value (Expiration latest). +1. Excludes from the list the N tags based on the `keep_n` value (Number of tags to retain). 1. Excludes from the list the tags older than the `older_than` value (Expiration interval). +1. Excludes from the list any tags matching the `name_regex_keep` value (Images to preserve). 1. Finally, the remaining tags in the list are deleted from the Container Registry. ### Managing project expiration policy through the UI To manage project expiration policy, navigate to **{settings}** **Settings > CI/CD > Container Registry tag expiration policy**. - + The UI allows you to configure the following: - **Expiration policy:** enable or disable the expiration policy. - **Expiration interval:** how long tags are exempt from being deleted. - **Expiration schedule:** how often the cron job checking the tags should run. -- **Expiration latest:** how many tags to _always_ keep for each image. +- **Number of tags to retain:** how many tags to _always_ keep for each image. - **Docker tags with names matching this regex pattern will expire:** the regex used to determine what tags should be expired. To qualify all tags for expiration, use the default value of `.*`. +- **Docker tags with names matching this regex pattern will be preserved:** the regex used to determine what tags should be preserved. To preserve all tags, use the default value of `.*`. ### Managing project expiration policy through the API @@ -527,16 +531,10 @@ You can set, update, and disable the expiration policies using the GitLab API. Examples: -- Select all tags, keep at least 1 tag per image, expire any tag older than 14 days, run once a month, and the policy is enabled: +- Select all tags, keep at least 1 tag per image, expire any tag older than 14 days, run once a month, preserve any images with the name `master` and the policy is enabled: ```shell - curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":".*"}}' 'https://gitlab.example.com/api/v4/projects/2' - ``` - -- Select only tags with a name that contains `stable`, keep at least 50 tag per image, expire any tag older than 7 days, run every day, and the policy is enabled: - - ```shell - curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1day","enabled":true,"keep_n":50"older_than":"7d","name_regex":"*stable"}}' 'https://gitlab.example.com/api/v4/projects/2' + curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":"","name_regex_delete":".*","name_regex_keep":".*-master"}}' 'https://gitlab.example.com/api/v4/projects/2' ``` See the API documentation for further details: [Edit project](../../../api/projects.md#edit-project). @@ -544,7 +542,7 @@ See the API documentation for further details: [Edit project](../../../api/proje ### Use with external container registries When using an [external container registry](./../../../administration/packages/container_registry.md#use-an-external-container-registry-with-gitlab-as-an-auth-endpoint), -running an experation policy on a project may have some performance risks. If a project is going to run +running an expiration policy on a project may have some performance risks. If a project is going to run a policy that will remove large quantities of tags (in the thousands), the GitLab background jobs that run the policy may get backed up or fail completely. It is recommended you only enable container expiration policies for projects that were created before GitLab 12.8 if you are confident the amount of tags @@ -552,10 +550,12 @@ being cleaned up will be minimal. ## Limitations -Moving or renaming existing Container Registry repositories is not supported +- Moving or renaming existing Container Registry repositories is not supported once you have pushed images, because the images are signed, and the signature includes the repository name. To move or rename a repository with a Container Registry, you will have to delete all existing images. +- Prior to GitLab 12.10, any tags that use the same image ID as the `latest` tag +will not be deleted by the expiration policy. ## Troubleshooting the GitLab Container Registry @@ -577,3 +577,47 @@ Troubleshooting the GitLab Container Registry, most of the times, requires administration access to the GitLab server. [Read how to troubleshoot the Container Registry](../../../administration/packages/container_registry.md#troubleshooting). + +### Unable to change path or transfer a project + +If you try to change a project's path or transfer a project to a new namespace, +you may receive one of the following errors: + +- "Project cannot be transferred, because tags are present in its container registry." +- "Namespace cannot be moved because at least one project has tags in container registry." + +This issue occurs when the project has images in the Container Registry. +You must delete or move these images before you can change the path or transfer +the project. + +The following procedure uses these sample project names: + +- For the current project: `example.gitlab.com/org/build/sample_project/cr:v2.9.1` +- For the new project: `example.gitlab.com/new_org/build/new_sample_project/cr:v2.9.1` + +Use your own URLs to complete the following steps: + +1. Download the Docker images on your computer: + + ```shell + docker login example.gitlab.com + docker pull example.gitlab.com/org/build/sample_project/cr:v2.9.1 + ``` + +1. Rename the images to match the new project name: + + ```shell + docker tag example.gitlab.com/org/build/sample_project/cr:v2.9.1 example.gitlab.com/new_org/build/new_sample_project/cr:v2.9.1 + ``` + +1. Delete the images in both projects by using the [UI](#delete-images) or [API](../../../api/packages.md#delete-a-project-package). + There may be a delay while the images are queued and deleted. +1. Change the path or transfer the project by going to **Settings > General** + and expanding **Advanced**. +1. Restore the images: + + ```shell + docker push example.gitlab.com/new_org/build/new_sample_project/cr:v2.9.1 + ``` + +Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/18383) for details. diff --git a/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png b/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png Binary files differindex 0b94efdd83e..e550d296d5a 100644 --- a/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png +++ b/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index cfdcd9821fb..be9710053dd 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -12,7 +12,7 @@ receiving a request and returning the upstream image from a registry, acting as a pull-through cache. The dependency proxy is available in the group level. To access it, navigate to -a group's **Packages > Dependency Proxy**. +a group's **Packages & Registries > Dependency Proxy**.  @@ -33,7 +33,7 @@ The following dependency proxies are supported. With the Docker dependency proxy, you can use GitLab as a source for a Docker image. To get a Docker image into the dependency proxy: -1. Find the proxy URL on your group's page under **Packages > Dependency Proxy**, +1. Find the proxy URL on your group's page under **Packages & Registries > Dependency Proxy**, for example `gitlab.com/groupname/dependency_proxy/containers`. 1. Trigger GitLab to pull the Docker image you want (e.g., `alpine:latest` or `linuxserver/nextcloud:latest`) and store it in the proxy storage by using diff --git a/doc/user/packages/img/group_packages_list_v12_10.png b/doc/user/packages/img/group_packages_list_v12_10.png Binary files differdeleted file mode 100644 index ba9f2892961..00000000000 --- a/doc/user/packages/img/group_packages_list_v12_10.png +++ /dev/null diff --git a/doc/user/packages/img/group_packages_list_v13_0.png b/doc/user/packages/img/group_packages_list_v13_0.png Binary files differnew file mode 100644 index 00000000000..8cf3fd1a131 --- /dev/null +++ b/doc/user/packages/img/group_packages_list_v13_0.png diff --git a/doc/user/packages/img/package_detail_v12_10.png b/doc/user/packages/img/package_detail_v12_10.png Binary files differdeleted file mode 100644 index b2cd8e31955..00000000000 --- a/doc/user/packages/img/package_detail_v12_10.png +++ /dev/null diff --git a/doc/user/packages/img/package_detail_v13_0.png b/doc/user/packages/img/package_detail_v13_0.png Binary files differnew file mode 100644 index 00000000000..dcfbc0a4787 --- /dev/null +++ b/doc/user/packages/img/package_detail_v13_0.png diff --git a/doc/user/packages/img/project_packages_list_v12_10.png b/doc/user/packages/img/project_packages_list_v12_10.png Binary files differdeleted file mode 100644 index 2dfb92fa796..00000000000 --- a/doc/user/packages/img/project_packages_list_v12_10.png +++ /dev/null diff --git a/doc/user/packages/img/project_packages_list_v13_0.png b/doc/user/packages/img/project_packages_list_v13_0.png Binary files differnew file mode 100644 index 00000000000..468a6fe6467 --- /dev/null +++ b/doc/user/packages/img/project_packages_list_v13_0.png diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 8e98dd70346..132a64d99a3 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -18,26 +18,28 @@ The Packages feature allows GitLab to act as a repository for the following: ## Enable the Package Registry for your project -If you cannot find the **{package}** **Packages > List** entry under your -project's sidebar, it is not enabled in your GitLab instance. Ask your -administrator to enable GitLab Package Registry following the administration -documentation. +If you cannot find the **{package}** **Packages & Registries > Package +Registry** entry under your project's sidebar, ensure that: -Once enabled for your GitLab instance, to enable Package Registry for your -project: +1. The GitLab Package Registry has been enabled by your administrator (following + [this documentation](../../administration/packages/index.md)); and +1. The Package Registry has been enabled for your project. + +Once an administrator has enabled the GitLab Package Registry for your GitLab +instance, to enable Package Registry for your project: 1. Go to your project's **Settings > General** page. 1. Expand the **Visibility, project features, permissions** section and enable the **Packages** feature on your project. 1. Press **Save changes** for the changes to take effect. You should now be able to -see the **Packages > List** link in the sidebar. +see the **Packages & Registries > Package Registry** link in the sidebar. ### View Packages for your project -Navigating to your project's **{package}** **Packages > List** will show a list +Navigating to your project's **{package}** **Packages & Registries > Package Registry** will show a list of all packages that have been added to your project. - + On this page, you can: @@ -51,9 +53,9 @@ On this page, you can: ### View Packages for your group You can view all packages belonging to a group by navigating to **{package}** -**Packages > List** from the group sidebar. +**Packages & Registries > Package Registry** from the group sidebar. - + On this page, you can: @@ -68,7 +70,7 @@ On this page, you can: Additional package information can be viewed by browsing to the package details page from the either the project or group list. - + On this page you can: @@ -112,7 +114,6 @@ are adding support for [PHP](https://gitlab.com/gitlab-org/gitlab/-/merge_reques | [Opkg](https://gitlab.com/gitlab-org/gitlab/issues/36894) | Optimize your work with OpenWrt using Opkg repositories. | | [P2](https://gitlab.com/gitlab-org/gitlab/issues/36895) | Host all your Eclipse plugins in your own GitLab P2 repository. | | [Puppet](https://gitlab.com/gitlab-org/gitlab/issues/36897) | Configuration management meets repository management with Puppet repositories. | -| [PyPi](https://gitlab.com/gitlab-org/gitlab/issues/10483) | Host PyPi distributions. | | [RPM](https://gitlab.com/gitlab-org/gitlab/issues/5932) | Distribute RPMs directly from GitLab. | | [RubyGems](https://gitlab.com/gitlab-org/gitlab/issues/803) | Use GitLab to host your own gems. | | [SBT](https://gitlab.com/gitlab-org/gitlab/issues/36898) | Resolve dependencies from and deploy build output to SBT repositories when running SBT builds. | diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index b4aec11d029..51e62dc871e 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -21,17 +21,19 @@ to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. Next, you must configure your project to authorize with the GitLab Maven repository. -## Getting Started +## Getting Started with Maven This section will cover installing Maven and building a package. This is a quickstart to help if you're new to building Maven packages. If you're already using Maven and understand how to build your own packages, move onto the [next section](#adding-the-gitlab-package-registry-as-a-maven-remote). +Maven repositories work well with Gradle, too. Move onto [getting started with Gradle](#getting-started-with-gradle) if you want to setup a Gradle project. + ### Installing Maven The required minimum versions are: @@ -96,20 +98,129 @@ your project has been set up successfully: You should see a new directory where you ran this command matching your `DartifactId` parameter (in this case it should be `my-project`). +## Getting started with Gradle + +This section will cover installing Gradle and initializing a Java project. This is a +quickstart to help if you're new to Gradle. If you're already +using Gradle and understand how to build your own packages, move onto the +[next section](#adding-the-gitlab-package-registry-as-a-maven-remote). + +### Installing Gradle + +Installation is needed only if you want to create a new Gradle project. Follow +instructions at [gradle.org](https://gradle.org/install/) to download and install +Gradle for your local development environment. + +Verify you can use Gradle in your terminal by running: + +```shell +gradle -version +``` + +If you want to use an existing Gradle project, installation is not necessary. +Simply execute `gradlew` (on Linux) or `gradlew.bat` (on Windows) in the project +directory instead. + +You should see something imilar to the below printed in the output: + +```plaintext +------------------------------------------------------------ +Gradle 6.0.1 +------------------------------------------------------------ + +Build time: 2019-11-18 20:25:01 UTC +Revision: fad121066a68c4701acd362daf4287a7c309a0f5 + +Kotlin: 1.3.50 +Groovy: 2.5.8 +Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 +JVM: 11.0.5 (Oracle Corporation 11.0.5+10) +OS: Windows 10 10.0 amd64 +``` + +### Creating a project in Gradle + +Understanding how to create a full Java project in Gradle is outside the scope of this +guide, but you can follow the steps below to create a new project that can be +published to the GitLab Package Registry. + +Start by opening your terminal and creating a directory where you would like to +store the project in your environment. From inside the directory, you can run +the following Maven command to initialize a new package: + +```shell +gradle init +``` + +The output should be + +```plaintext +Select type of project to generate: + 1: basic + 2: application + 3: library + 4: Gradle plugin +Enter selection (default: basic) [1..4] +``` + +Enter `3` to create a new Library project. The output should be: + +```plaintext +Select implementation language: + 1: C++ + 2: Groovy + 3: Java + 4: Kotlin + 5: Scala + 6: Swift +``` + +Enter `3` to create a new Java Library project. The output should be: + +```plaintext +Select build script DSL: + 1: Groovy + 2: Kotlin +Enter selection (default: Groovy) [1..2] +``` + +Choose `1` to create a new Java Library project which will be described in Groovy DSL. The output should be: + +```plaintext +Select test framework: + 1: JUnit 4 + 2: TestNG + 3: Spock + 4: JUnit Jupiter +``` + +Choose `1` to initialize the project with JUnit 4 testing libraries. The output should be: + +```plaintext +Project name (default: test): +``` + +Enter a project name or hit enter to use the directory name as project name. + ## Adding the GitLab Package Registry as a Maven remote The next step is to add the GitLab Package Registry as a Maven remote. If a project is private or you want to upload Maven artifacts to GitLab, credentials will need to be provided for authorization too. Support is available -for [personal access tokens](#authenticating-with-a-personal-access-token) and -[CI job tokens](#authenticating-with-a-ci-job-token) only. -[Deploy tokens](../../project/deploy_tokens/index.md) and regular username/password +for [personal access tokens](#authenticating-with-a-personal-access-token), +[CI job tokens](#authenticating-with-a-ci-job-token), and +[deploy tokens](../../project/deploy_tokens/index.md) only. Regular username/password credentials do not work. ### Authenticating with a personal access token To authenticate with a [personal access token](../../profile/personal_access_tokens.md), -set the scope to `api` and add a corresponding section to your +set the scope to `api` when creating one, and add it to your Maven or Gradle configuration +files. + +#### Authenticating with a personal access token in Maven + +Add a corresponding section to your [`settings.xml`](https://maven.apache.org/settings.html) file: ```xml @@ -130,13 +241,43 @@ set the scope to `api` and add a corresponding section to your </settings> ``` +#### Authenticating with a personal access token in Gradle + +Create a file `~/.gradle/gradle.properties` with the following content: + +```groovy +gitLabPrivateToken=REPLACE_WITH_YOUR_PERSONAL_ACCESS_TOKEN +``` + +Add a repositories section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) +file: + +```groovy +repositories { + maven { + url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(HttpHeaderCredentials) { + name = 'Private-Token' + value = gitLabPrivateToken + } + authentication { + header(HttpHeaderAuthentication) + } + } +} +``` + You should now be able to upload Maven artifacts to your project. ### Authenticating with a CI job token -If you're using Maven with GitLab CI/CD, a CI job token can be used instead +If you're using GitLab CI/CD, a CI job token can be used instead of a personal access token. +#### Authenticating with a CI job token in Maven + To authenticate with a CI job token, add a corresponding section to your [`settings.xml`](https://maven.apache.org/settings.html) file: @@ -161,6 +302,81 @@ To authenticate with a CI job token, add a corresponding section to your You can read more on [how to create Maven packages using GitLab CI/CD](#creating-maven-packages-with-gitlab-cicd). +#### Authenticating with a CI job token in Gradle + +To authenticate with a CI job token, add a repositories section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) +file: + +```groovy +repositories { + maven { + url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(HttpHeaderCredentials) { + name = 'Job-Token' + value = '${CI_JOB_TOKEN}' + } + authentication { + header(HttpHeaderAuthentication) + } + } +} +``` + +### Authenticating with a deploy token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0. + +To authenticate with a [deploy token](./../../project/deploy_tokens/index.md), +set the scope to `api` when creating one, and add it to your Maven or Gradle configuration +files. + +#### Authenticating with a deploy token in Maven + +Add a corresponding section to your +[`settings.xml`](https://maven.apache.org/settings.html) file: + +```xml +<settings> + <servers> + <server> + <id>gitlab-maven</id> + <configuration> + <httpHeaders> + <property> + <name>Deploy-Token</name> + <value>REPLACE_WITH_YOUR_DEPLOY_TOKEN</value> + </property> + </httpHeaders> + </configuration> + </server> + </servers> +</settings> +``` + +#### Authenticating with a deploy token in Gradle + +To authenticate with a deploy token, add a repositories section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) +file: + +```groovy +repositories { + maven { + url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(HttpHeaderCredentials) { + name = 'Deploy-Token' + value = '<deploy-token>' + } + authentication { + header(HttpHeaderAuthentication) + } + } +} +``` + ## Configuring your project to use the GitLab Maven repository URL To download and upload packages from GitLab, you need a `repository` and @@ -185,7 +401,7 @@ the `distributionManagement` section. ### Project level Maven endpoint The example below shows how the relevant `repository` section of your `pom.xml` -would look like: +would look like in Maven: ```xml <repositories> @@ -206,6 +422,17 @@ would look like: </distributionManagement> ``` +The corresponding section in Gradle would look like this: + +```groovy +repositories { + maven { + url "https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven" + name "GitLab" + } +} +``` + The `id` must be the same with what you [defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). @@ -223,7 +450,7 @@ project's ID can be used for uploading. ### Group level Maven endpoint -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8798) in GitLab Premium 11.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8798) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. If you rely on many packages, it might be inefficient to include the `repository` section with a unique URL for each package. Instead, you can use the group level endpoint for @@ -259,6 +486,17 @@ the `distributionManagement` section: </distributionManagement> ``` +For Gradle, the corresponding repositories section would look like: + +```groovy +repositories { + maven { + url "https://gitlab.com/api/v4/groups/GROUP_ID/-/packages/maven" + name "GitLab" + } +} +``` + The `id` must be the same with what you [defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). @@ -275,7 +513,7 @@ For retrieving artifacts, you can use either the ### Instance level Maven endpoint -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8274) in GitLab Premium 11.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8274) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. If you rely on many packages, it might be inefficient to include the `repository` section with a unique URL for each package. Instead, you can use the instance level endpoint for @@ -314,6 +552,17 @@ the `distributionManagement` section: </distributionManagement> ``` +The corresponding repositories section in Gradle would look like: + +```groovy +repositories { + maven { + url "https://gitlab.com/api/v4/packages/maven" + name "GitLab" + } +} +``` + The `id` must be the same with what you [defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). @@ -333,7 +582,9 @@ project's ID can be used for uploading. Once you have set up the [remote and authentication](#adding-the-gitlab-package-registry-as-a-maven-remote) and [configured your project](#configuring-your-project-to-use-the-gitlab-maven-repository-url), -test to upload a Maven artifact from a project of yours: +test to upload a Maven artifact from a project of yours. + +### Upload using Maven ```shell mvn deploy @@ -353,7 +604,51 @@ You should also see that the upload was uploaded to the correct registry: Uploading to gitlab-maven: https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.jar ``` -You can then navigate to your project's **Packages** page and see the uploaded +### Upload using Gradle + +Add the Gradle plugin [`maven-publish`](https://docs.gradle.org/current/userguide/publishing_maven.html) to the plugins section: + +```groovy +plugins { + id 'java' + id 'maven-publish' +} +``` + +Add a `publishing` section: + +```groovy +publishing { + publications { + library(MavenPublication) { + from components.java + } + } + repositories { + maven { + url "https://gitlab.com/api/v4/projects/<PROJECT_ID>/packages/maven" + credentials(HttpHeaderCredentials) { + name = "Private-Token" + value = gitLabPrivateToken // the variable resides in ~/.gradle/gradle.properties + } + authentication { + header(HttpHeaderAuthentication) + } + } + } +} +``` + +Replace `PROJECT_ID` with your project ID which can be found on the home page +of your project. + +Run the publish task: + +```shell +gradle publish +``` + +You can then navigate to your project's **Packages & Registries** page and see the uploaded artifacts or even delete them. ## Installing a package @@ -362,7 +657,7 @@ Installing a package from the GitLab Package Registry requires that you set up the [remote and authentication](#adding-the-gitlab-package-registry-as-a-maven-remote) as above. Once this is completed, there are two ways for installaing a package. -### Install with `mvn install` +### Install using Maven with `mvn install` Add the dependency manually to your project `pom.xml` file. To add the example created above, the XML would look like: @@ -388,7 +683,7 @@ downloaded from the GitLab Package Registry: Downloading from gitlab-maven: http://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.pom ``` -### Install with `mvn dependency:get` +#### Install with `mvn dependency:get` The second way to install packages is to use the Maven commands directly. Inside your project directory, run: @@ -404,6 +699,16 @@ TIP: **Tip:** Both the XML block and Maven command are readily copy and pastable from the Package details page, allowing for quick and easy installation. +### Install using Gradle + +Add a [dependency](https://docs.gradle.org/current/userguide/declaring_dependencies.html) to build.gradle in the dependencies section: + +```groovy +dependencies { + implementation 'com.mycompany.mydepartment:my-project:1.0-SNAPSHOT' +} +``` + ## Removing a package In the packages view of your project page, you can delete packages by clicking @@ -413,11 +718,15 @@ page. ## Creating Maven packages with GitLab CI/CD Once you have your repository configured to use the GitLab Maven Repository, -you can configure GitLab CI/CD to build new packages automatically. The example below -shows how to create a new package each time the `master` branch is updated: +you can configure GitLab CI/CD to build new packages automatically. + +### Creating Maven packages with GitLab CI/CD using Maven + +The example below shows how to create a new package each time the `master` branch +is updated: 1. Create a `ci_settings.xml` file that will serve as Maven's `settings.xml` file. - Add the server section with the same id you defined in your `pom.xml` file. + Add the server section with the same ID you defined in your `pom.xml` file. For example, in our case it's `gitlab-maven`: ```xml @@ -481,6 +790,31 @@ user's home location (in this case the user is `root` since it runs in a Docker container), and Maven will utilize the configured CI [environment variables](../../../ci/variables/README.md#predefined-environment-variables). +### Creating Maven packages with GitLab CI/CD using Gradle + +The example below shows how to create a new package each time the `master` branch +is updated: + +1. Make sure you use the Job-Token authentication as described in ["Authenticating with a CI job token in Gradle"](#authenticating-with-a-ci-job-token-in-gradle). + +1. Add a `deploy` job to your `.gitlab-ci.yml` file: + + ```yaml + deploy: + image: gradle:latest + script: + - 'gradle publish' + only: + - master + ``` + +1. Push those files to your repository. + +The next time the `deploy` job runs, it will copy `ci_settings.xml` to the +user's home location (in this case the user is `root` since it runs in a +Docker container), and Maven will use the configured CI +[environment variables](../../../ci/variables/README.md#predefined-environment-variables). + ## Troubleshooting ### Useful Maven command line options diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index e66b3d1ac63..b909646431b 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -23,7 +23,7 @@ by default. To enable it for existing projects, or if you want to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. Before proceeding to authenticating with the GitLab NPM Registry, you should get familiar with the package naming convention. @@ -100,14 +100,15 @@ configure GitLab as a remote registry. If a project is private or you want to upload an NPM package to GitLab, credentials will need to be provided for authentication. [Personal access tokens](../../profile/personal_access_tokens.md) +and [deploy tokens](../../project/deploy_tokens/index.md) are preferred, but support is available for [OAuth tokens](../../../api/oauth2.md#resource-owner-password-credentials-flow). -CAUTION: **2FA is only supported with personal access tokens:** -If you have 2FA enabled, you need to use a [personal access token](../../profile/personal_access_tokens.md) with OAuth headers with the scope set to `api`. Standard OAuth tokens won't be able to authenticate to the GitLab NPM Registry. +CAUTION: **Two-factor authentication (2FA) is only supported with personal access tokens:** +If you have 2FA enabled, you need to use a [personal access token](../../profile/personal_access_tokens.md) with OAuth headers with the scope set to `api` or a [deploy token](../../project/deploy_tokens/index.md) with `read_package_registry` or `write_package_registry` scopes. Standard OAuth tokens won't be able to authenticate to the GitLab NPM Registry. -### Authenticating with a personal access token +### Authenticating with a personal access token or deploy token -To authenticate with a [personal access token](../../profile/personal_access_tokens.md), +To authenticate with a [personal access token](../../profile/personal_access_tokens.md) or [deploy token](../../project/deploy_tokens/index.md), set your NPM configuration: ```shell @@ -125,7 +126,7 @@ npm config set '//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_au ``` Replace `<your_project_id>` with your project ID which can be found on the home page -of your project and `<your_token>` with your personal access token. +of your project and `<your_token>` with your personal access token or deploy token. If you have a self-managed GitLab installation, replace `gitlab.com` with your domain name. @@ -160,7 +161,7 @@ Then, you could run `npm publish` either locally or via GitLab CI/CD: > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9104) in GitLab Premium 12.5. -If you’re using NPM with GitLab CI/CD, a CI job token can be used instead of a personal access token. +If you’re using NPM with GitLab CI/CD, a CI job token can be used instead of a personal access token or deploy token. The token will inherit the permissions of the user that generates the pipeline. Add a corresponding section to your `.npmrc` file: @@ -195,7 +196,7 @@ you can upload an NPM package to your project: npm publish ``` -You can then navigate to your project's **Packages** page and see the uploaded +You can then navigate to your project's **Packages & Registries** page and see the uploaded packages or even delete them. If you attempt to publish a package with a name that already exists within @@ -286,11 +287,11 @@ page. ## Publishing a package with CI/CD To work with NPM commands within [GitLab CI/CD](./../../../ci/README.md), you can use -`CI_JOB_TOKEN` in place of the personal access token in your commands. +`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands. A simple example `.gitlab-ci.yml` file for publishing NPM packages: -```yml +```yaml image: node:latest stages: @@ -323,9 +324,9 @@ info Visit https://classic.yarnpkg.com/en/docs/cli/install for documentation abo ``` In this case, try adding this to your `.npmrc` file (and replace `<your_token>` -with your personal access token): +with your personal access token or deploy token): -```text +```plaintext //gitlab.com/api/v4/projects/:_authToken=<your_token> ``` @@ -363,6 +364,14 @@ You do not need a token to run `npm install` unless your project is private (the NPM_TOKEN=<your_token> npm install ``` +### `npm install` returns `npm ERR! 403 Forbidden` + +- Check that your token is not expired and has appropriate permissions. +- Check if you have attempted to publish a package with a name that already exists within a given scope. +- Ensure the scoped packages URL includes a trailing slash: + - Correct: `//gitlab.com/api/v4/packages/npm/` + - Incorrect: `//gitlab.com/api/v4/packages/npm` + ## NPM dependencies metadata > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/11867) in GitLab Premium 12.6. diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index ed936b546d2..d9efb3239a8 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -61,14 +61,16 @@ by default. To enable it for existing projects, or if you want to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. ## Adding the GitLab NuGet Repository as a source to NuGet You will need the following: - Your GitLab username. -- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. +- A personal access token or deploy token. For repository authentication: + - You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. + - You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. - A suitable name for your source. - Your project ID which can be found on the home page of your project. @@ -83,7 +85,7 @@ You can now add a new source to NuGet with: To add the GitLab NuGet Repository as a source with `nuget`: ```shell -nuget source Add -Name <source_name> -Source "https://gitlab-instance.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json" -UserName <gitlab_username> -Password <gitlab_personal_access_token> +nuget source Add -Name <source_name> -Source "https://gitlab-instance.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json" -UserName <gitlab_username or deploy_token_username> -Password <gitlab_personal_access_token or deploy_token> ``` Where: @@ -107,8 +109,8 @@ nuget source Add -Name "GitLab" -Source "https//gitlab.example/api/v4/projects/1 - **Location**: `https://gitlab.com/api/v4/projects/<your_project_id>/packages/nuget/index.json` - Replace `<your_project_id>` with your project ID. - If you have a self-managed GitLab installation, replace `gitlab.com` with your domain name. - - **Username**: Your GitLab username - - **Password**: Your personal access token + - **Username**: Your GitLab username or deploy token username + - **Password**: Your personal access token or deploy token  @@ -131,8 +133,8 @@ To add the GitLab NuGet Repository as a source for .NET, create a file named `nu </packageSources> <packageSourceCredentials> <gitlab> - <add key="Username" value="<gitlab_username>" /> - <add key="ClearTextPassword" value="<gitlab_personal_access_token>" /> + <add key="Username" value="<gitlab_username or deploy_token_username>" /> + <add key="ClearTextPassword" value="<gitlab_personal_access_token or deploy_token>" /> </gitlab> </packageSourceCredentials> </configuration> @@ -201,7 +203,7 @@ nuget install <package_id> -OutputDirectory <output_directory> \ Where: -- `<package_id>` is the package id. +- `<package_id>` is the package ID. - `<output_directory>` is the output directory, where the package will be installed. - `<package_version>` (Optional) is the package version. - `<source_name>` (Optional) is the source name. @@ -222,5 +224,5 @@ dotnet add package <package_id> \ Where: -- `<package_id>` is the package id. +- `<package_id>` is the package ID. - `<package_version>` (Optional) is the package version. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 11d7b828813..979f7a3c966 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -26,10 +26,132 @@ by default. To enable it for existing projects, or if you want to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. + +## Getting started + +This section will cover creating a new example PyPi package to upload. This is a +quickstart to test out the **GitLab PyPi Registry**. If you already understand how +to build and publish your own packages, move on to the [next section](#adding-the-gitlab-pypi-repository-as-a-source). + +### Create a project + +Understanding how to create a full Python project is outside the scope of this +guide, but you can create a small package to test out the registry. Start by +creating a new directory called `MyPyPiPackage`: + +```shell +mkdir MyPyPiPackage && cd MyPyPiPackage +``` + +After creating this, create another directory inside: + +```shell +mkdir mypypipackage && cd mypypipackage +``` + +Create two new files inside this directory to set up the basic project: + +```shell +touch __init__.py +touch greet.py +``` + +Inside `greet.py`, add the following code: + +```python +def SayHello(): + print("Hello from MyPyPiPackage") + return +``` + +Inside the `__init__.py` file, add the following: + +```python +from .greet import SayHello +``` + +Now that the basics of our project is completed, we can test that the code runs. +Start the Python prompt inside your top `MyPyPiPackage` directory. Then run: + +```python +>>> from mypypipackage import SayHello +>>> SayHello() +``` + +You should see an output similar to the following: + +```plaintext +Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18) +[Clang 6.0 (clang-600.0.57)] on darwin +Type "help", "copyright", "credits" or "license" for more information. +>>> from mypypipackage import SayHello +>>> SayHello() +Hello from MyPyPiPackage +``` + +Once we've verified that the sample project is working as above, we can next +work on creating a package. + +### Create a package + +Inside your `MyPyPiPackage` directory, we need to create a `setup.py` file. Run +the following: + +```shell +touch setup.py +``` + +This file contains all the information about our package. For more information +about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). +For this guide, we don't need to extensively fill out this file, simply add the +below to your `setup.py`: + +```python +import setuptools + +setuptools.setup( + name="mypypipackage", + version="0.0.1", + author="Example Author", + author_email="author@example.com", + description="A small example package", + packages=setuptools.find_packages(), + classifiers=[ + "Programming Language :: Python :: 3", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + ], + python_requires='>=3.6', +) +``` + +Save the file, then execute the setup like so: + +```shell +python3 setup.py sdist bdist_wheel +``` + +If successful, you should be able to see the output in a newly created `dist` +folder. Run: + +```shell +ls dist +``` + +And confirm your output matches the below: + +```plaintext +mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz +``` + +Our package is now all set up and ready to be uploaded to the **GitLab PyPi +Package Registry**. Before we do so, we next need to set up authentication. ## Adding the GitLab PyPi Repository as a source +### Authenticating with a personal access token + You will need the following: - A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. @@ -39,12 +161,37 @@ You will need the following: Edit your `~/.pypirc` file and add the following: ```ini +[distutils] +index-servers = + gitlab + [gitlab] repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi username = __token__ password = <your personal access token> ``` +### Authenticating with a deploy token + +You will need the following: + +- A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the `read_package_registry` and/or `write_package_registry` scopes for repository authentication. +- A suitable name for your source. +- Your project ID which can be found on the home page of your project. + +Edit your `~/.pypirc` file and add the following: + +```ini +[distutils] +index-servers = + gitlab + +[gitlab] +repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi +username = <deploy token username> +password = <deploy token> +``` + ## Uploading packages When uploading packages, note that: @@ -52,13 +199,33 @@ When uploading packages, note that: - The maximum allowed size is 50 Megabytes. - If you upload the same package with the same version multiple times, each consecutive upload is saved as a separate file. When installing a package, GitLab will serve the most recent file. -- When uploading packages to GitLab, they will not be displayed in the packages UI of your project - immediately. It can take up to 10 minutes to process a package. ### Upload packages with Twine -This section assumes that your project is properly built and you already [created a PyPi package with setuptools](https://packaging.python.org/tutorials/packaging-projects/). -Upload your package using the following command: +If you were following the guide above, then the `MyPyPiPackage` package should +be ready to be uploaded. Run the following command: + +```shell +python3 -m twine upload --repository gitlab dist/* +``` + +If successful, you should see the following: + +```plaintext +Uploading distributions to https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi +Uploading mypypipackage-0.0.1-py3-none-any.whl +100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s] +Uploading mypypipackage-0.0.1.tar.gz +100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s] +``` + +This indicates that the package was uploaded successfully. You can then navigate +to your project's **Packages & Registries** page and see the uploaded packages. + +If you did not follow the guide above, the you'll need to ensure your package +has been properly built and you [created a PyPi package with setuptools](https://packaging.python.org/tutorials/packaging-projects/). + +You can then upload your package using the following command: ```shell python -m twine upload --repository <source_name> dist/<package_file> @@ -80,5 +247,22 @@ pip install --index-url https://__token__:<personal_access_token>@gitlab.com/api Where: - `<package_name>` is the package name. -- `<personal_access_token>` is your personal access token. -- `<project_id>` is your project id number. +- `<personal_access_token>` is a personal access token with the `read_api` scope. +- `<project_id>` is the project ID. + +If you were following the guide above and want to test installing the +`MyPyPiPackage` package, you can run the following: + +```shell +pip install mypypipackage --no-deps --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +``` + +This should result in the following: + +```plaintext +Looking in indexes: https://__token__:****@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +Collecting mypypipackage + Downloading https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB) +Installing collected packages: mypypipackage +Successfully installed mypypipackage-0.0.1 +``` |
