--- stage: Package group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # PyPI packages in the Package Registry **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in GitLab 12.10. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) from GitLab Premium to GitLab Free in 13.3. Publish PyPI packages in your project's Package Registry. Then install the packages whenever you need to use them as a dependency. The Package Registry works with: - [pip](https://pypi.org/project/pip/) - [twine](https://pypi.org/project/twine/) For documentation of the specific API endpoints that the `pip` and `twine` clients use, see the [PyPI API documentation](../../../api/packages/pypi.md). ## Build a PyPI package This section explains how to create a PyPI package. If you already use PyPI and know how to build your own packages, go to the [next section](#authenticate-with-the-package-registry). ### Install pip and twine Install a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/). ### Create a project Create a test project. 1. Open your terminal. 1. Create a directory called `MyPyPiPackage`, and then go to that directory: ```shell mkdir MyPyPiPackage && cd MyPyPiPackage ``` 1. Create another directory and go to it: ```shell mkdir mypypipackage && cd mypypipackage ``` 1. Create the required files in this directory: ```shell touch __init__.py touch greet.py ``` 1. Open the `greet.py` file, and then add: ```python def SayHello(): print("Hello from MyPyPiPackage") return ``` 1. Open the `__init__.py` file, and then add: ```python from .greet import SayHello ``` 1. To test the code, in your `MyPyPiPackage` directory, start the Python prompt. ```shell python ``` 1. Run this command: ```python >>> from mypypipackage import SayHello >>> SayHello() ``` A message indicates that the project was set up successfully: ```plaintext Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from mypypipackage import SayHello >>> SayHello() Hello from MyPyPiPackage ``` ### Create a package After you create a project, you can create a package. 1. In your terminal, go to the `MyPyPiPackage` directory. 1. Create a `pyproject.toml` file: ```shell touch pyproject.toml ``` This file contains all the information about the package. For more information about this file, see [creating `pyproject.toml`](https://packaging.python.org/en/latest/tutorials/packaging-projects/#creating-pyproject-toml). Because GitLab identifies packages based on [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), ensure your package name meets these requirements. See the [installation section](#authenticate-with-a-ci-job-token) for details. 1. Open the `pyproject.toml` file, and then add basic information: ```toml [build-system] requires = ["setuptools>=61.0"] build-backend = "setuptools.build_meta" [project] name = "mypypipackage" version = "0.0.1" authors = [ { name="Example Author", email="author@example.com" }, ] description = "A small example package" requires-python = ">=3.7" classifiers = [ "Programming Language :: Python :: 3", "Operating System :: OS Independent", ] [tool.setuptools.packages] find = {} ``` 1. Save the file. 1. Install the package build library: ```shell pip install build ``` 1. Build the package: ```shell python -m build ``` The output should be visible in a newly-created `dist` folder: ```shell ls dist ``` The output should appear similar to the following: ```plaintext mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz ``` The package is now ready to be published to the Package Registry. ## Authenticate with the Package Registry Before you can publish to the Package Registry, you must authenticate. To do this, you can use: - A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. - A [deploy token](../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. - A [CI job token](#authenticate-with-a-ci-job-token). ### Authenticate with a personal access token To authenticate with a personal access token, edit the `~/.pypirc` file and add: ```ini [distutils] index-servers = gitlab [gitlab] repository = https://gitlab.example.com/api/v4/projects//packages/pypi username = password = ``` The `` is either the project's [URL-encoded](../../../api/index.md#namespaced-path-encoding) path (for example, `group%2Fproject`), or the project's ID (for example `42`). ### Authenticate with a deploy token To authenticate with a deploy token, edit your `~/.pypirc` file and add: ```ini [distutils] index-servers = gitlab [gitlab] repository = https://gitlab.example.com/api/v4/projects//packages/pypi username = password = ``` The `` is either the project's [URL-encoded](../../../api/index.md#namespaced-path-encoding) path (for example, `group%2Fproject`), or the project's ID (for example `42`). ### Authenticate with a CI job token > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4. To work with PyPI commands within [GitLab CI/CD](../../../ci/index.md), you can use `CI_JOB_TOKEN` instead of a personal access token or deploy token. For example: ```yaml image: python:latest run: script: - pip install build twine - python -m build - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/* ``` You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check in to GitLab: ```ini [distutils] index-servers = gitlab [gitlab] repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi username = gitlab-ci-token password = ${env.CI_JOB_TOKEN} ``` ### Authenticate to access packages within a group Follow the instructions above for the token type, but use the group URL in place of the project URL: ```shell https://gitlab.example.com/api/v4/groups//-/packages/pypi ``` ## Publish a PyPI package Prerequisites: - You must [authenticate with the Package Registry](#authenticate-with-the-package-registry). - Your [version string must be valid](#ensure-your-version-string-is-valid). - The maximum allowed package size is 5 GB. - You can't upload the same version of a package multiple times. If you try, you receive the error `400 Bad Request`. - PyPI packages are published using your projectID. - If your project is in a group, PyPI packages published to your project registry are also available at the group-level registry (see [Install from the group level](#install-from-the-group-level)). You can then [publish a package by using twine](#publish-a-pypi-package-by-using-twine). ### Ensure your version string is valid If your version string (for example, `0.0.1`) isn't valid, it gets rejected. GitLab uses the following regex to validate the version string. ```ruby \A(?: v? (?:([0-9]+)!)? (?# epoch) ([0-9]+(?:\.[0-9]+)*) (?# release segment) ([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)? (?# pre-release) ((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))? (?# post release) ([-_\.]?(dev)[-_\.]?([0-9]+)?)? (?# dev release) (?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))? (?# local version) )\z}xi ``` You can experiment with the regex and try your version strings by using this [regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV). For more details about the regex, review this [documentation](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions). ### Publish a PyPI package by using twine To publish a PyPI package, run a command like: ```shell python3 -m twine upload --repository gitlab dist/* ``` This message indicates that the package was published successfully: ```plaintext Uploading distributions to https://gitlab.example.com/api/v4/projects//packages/pypi Uploading mypypipackage-0.0.1-py3-none-any.whl 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s] Uploading mypypipackage-0.0.1.tar.gz 100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s] ``` To view the published package, go to your project's **Packages and registries** page. If you didn't use a `.pypirc` file to define your repository source, you can publish to the repository with the authentication inline: ```shell TWINE_PASSWORD= TWINE_USERNAME= python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects//packages/pypi dist/* ``` If you didn't follow the steps on this page, ensure your package was properly built, and that you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). You can then upload your package by using the following command: ```shell python -m twine upload --repository dist/ ``` - `` is your package filename, ending in `.tar.gz` or `.whl`. - `` is the [source name used during setup](#authenticate-with-the-package-registry). ### Publishing packages with the same name or version You cannot publish a package if a package of the same name and version already exists. You must [delete the existing package](../../packages/package_registry/reduce_package_registry_storage.md#delete-a-package) first. If you attempt to publish the same package more than once, a `400 Bad Request` error occurs. ## Install a PyPI package In [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/233413), when a PyPI package is not found in the Package Registry, the request is forwarded to [pypi.org](https://pypi.org/). Administrators can disable this behavior in the [Continuous Integration settings](../../admin_area/settings/continuous_integration.md). WARNING: When you use the `--index-url` option, do not specify the port if it is a default port, such as `80` for a URL starting with `http` or `443` for a URL starting with `https`. ### Install from the project level To install the latest version of a package, use the following command: ```shell pip install --index-url https://:@gitlab.example.com/api/v4/projects//packages/pypi/simple --no-deps ``` - `` is the package name. - `` is a personal access token name with the `read_api` scope. - `` is a personal access token with the `read_api` scope. - `` is either the project's [URL-encoded](../../../api/index.md#namespaced-path-encoding) path (for example, `group%2Fproject`), or the project's ID (for example `42`). In these commands, you can use `--extra-index-url` instead of `--index-url`. However, using `--extra-index-url` makes you vulnerable to dependency confusion attacks because it checks the PyPi repository for the package before it checks the custom repository. `--extra-index-url` adds the provided URL as an additional registry which the client checks if the package is present. `--index-url` tells the client to check for the package on the provided URL only. If you were following the guide and want to install the `MyPyPiPackage` package, you can run: ```shell pip install mypypipackage --no-deps --index-url https://:@gitlab.example.com/api/v4/projects//packages/pypi/simple ``` This message indicates that the package was installed successfully: ```plaintext Looking in indexes: https://:****@gitlab.example.com/api/v4/projects//packages/pypi/simple Collecting mypypipackage Downloading https://gitlab.example.com/api/v4/projects//packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB) Installing collected packages: mypypipackage Successfully installed mypypipackage-0.0.1 ``` ### Install from the group level To install the latest version of a package from a group, use the following command: ```shell pip install --index-url https://:@gitlab.example.com/api/v4/groups//-/packages/pypi/simple --no-deps ``` In this command: - `` is the package name. - `` is a personal access token name with the `read_api` scope. - `` is a personal access token with the `read_api` scope. - `` is the group ID. In these commands, you can use `--extra-index-url` instead of `--index-url`. However, using `--extra-index-url` makes you vulnerable to dependency confusion attacks because it checks the PyPi repository for the package before it checks the custom repository. `--extra-index-url` adds the provided URL as an additional registry which the client checks if the package is present. `--index-url` tells the client to check for the package at the provided URL only. If you're following the guide and want to install the `MyPyPiPackage` package, you can run: ```shell pip install mypypipackage --no-deps --index-url https://:@gitlab.example.com/api/v4/groups//-/packages/pypi/simple ``` ### Package names GitLab looks for packages that use [Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names). The characters `-`, `_`, and `.` are all treated the same, and repeated characters are removed. A `pip install` request for `my.package` looks for packages that match any of the three characters, such as `my-package`, `my_package`, and `my....package`. ## Troubleshooting To improve performance, PyPI caches files related to a package. Note that PyPI doesn't remove data by itself. The cache grows as new packages are installed. If you encounter issues, clear the cache with this command: ```shell pip cache purge ``` ## Supported CLI commands The GitLab PyPI repository supports the following CLI commands: - `twine upload`: Upload a package to the registry. - `pip install`: Install a PyPI package from the registry.