From 40c40a9653fc498a1b1ea80fa67317527e8b9784 Mon Sep 17 00:00:00 2001 From: Sandra McCann Date: Mon, 30 Sep 2019 14:48:15 -0400 Subject: Backport/2.9/docs2 (#62805) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Update pip module docs (#62359) * Clarifying pip module requirements in reference to #47361 * Further clarifying message with link to ansible_python_interpreter (cherry picked from commit d3ec5ca80f630ed8c78ea3fab73e0d2927a89b5e) * chore/doc-module: sts_assume_role.py (#62475) Update `Example` section with with the correct module usage. (cherry picked from commit a4a216640fc7efe436f3f09808500b52ff6a63cd) * Improve dconf documentation to include conversion problems (#62316) (cherry picked from commit 864928365ef000c187871e6874b520f0bd2bc97c) * Add examples for various inventory setups to the documentation (#62323) * Updates docs/docsite/rst/user_guide/intro_inventory.rst, closes #12480. * Use code-block and rename groups in inventory setup examples * Fix group name in inventory setup example Co-Authored-By: Sandra McCann (cherry picked from commit 7047b66d345fc98c9ed163bbd47f3ee7c092f55b) * added networking porting guide info (#61999) * Update docs/docsite/rst/porting_guides/porting_guide_2.9.rst Co-Authored-By: Nathaniel Case (cherry picked from commit 6d35f9026f521b3191a0ad5e5abbdeb1c46ec29e) * [docs] split collections into user and dev guide sections (#62363) (cherry picked from commit 7badeb6df0e644419adf2b5daf9545d448defe60) * fixed options (#62605) (cherry picked from commit 170b4e63ffa6115275cb78ebc0f68f8aa79f99a3) * ec2_vpc_subnet: Rename resource_tags > tags (#62663) Most of the AWS module documentation refers to `tags` and not `resource_tags`. This patch updates the documentation to match other AWS module documentation. 😉 Signed-off-by: Major Hayden (cherry picked from commit cced1a3cd1e696cf5927ed248215d442dd6c1244) * [Docs] Document the resource module builder (#62222) (cherry picked from commit b17581a3075f571ed5b48126282e086a6efa30cc) * Fix link syntax and a typo in dev collections doc (#62650) (cherry picked from commit 2969614c2cb89151c3bf2487e0e1a1afe77cde96) * hcloud_volume: clarify volume size units (#62771) (cherry picked from commit 190b8fcd1cc413a8b3a6e5c9f3f74353990400dd) * ovirt_host update force doc (#62491) (cherry picked from commit 3b2b418aadc6ff2da4bae7f3d14ab8a51b207f77) * update example document for zabbix_action module (#62667) (cherry picked from commit 3299f29f7c7c4866baef78e8ae1d5272cd0b265d) --- .../rst/dev_guide/collections_tech_preview.rst | 537 --------------------- .../rst/dev_guide/developing_collections.rst | 322 ++++++++++++ docs/docsite/rst/dev_guide/index.rst | 3 +- .../developing_resource_modules_network.rst | 377 +++++++++++++++ docs/docsite/rst/network/dev_guide/index.rst | 2 + .../rst/porting_guides/porting_guide_2.9.rst | 26 +- docs/docsite/rst/user_guide/collections_using.rst | 240 +++++++++ docs/docsite/rst/user_guide/index.rst | 8 +- docs/docsite/rst/user_guide/intro_inventory.rst | 95 ++++ docs/templates/collections_galaxy_meta.rst.j2 | 19 +- lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py | 2 +- .../modules/cloud/amazon/sts_assume_role.py | 24 +- lib/ansible/modules/cloud/hcloud/hcloud_volume.py | 4 +- lib/ansible/modules/cloud/ovirt/ovirt_host.py | 1 + .../modules/monitoring/zabbix/zabbix_action.py | 5 + .../modules/network/netconf/netconf_config.py | 10 +- lib/ansible/modules/packaging/language/pip.py | 22 +- lib/ansible/modules/system/dconf.py | 4 + 18 files changed, 1125 insertions(+), 576 deletions(-) delete mode 100644 docs/docsite/rst/dev_guide/collections_tech_preview.rst create mode 100644 docs/docsite/rst/dev_guide/developing_collections.rst create mode 100644 docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst create mode 100644 docs/docsite/rst/user_guide/collections_using.rst diff --git a/docs/docsite/rst/dev_guide/collections_tech_preview.rst b/docs/docsite/rst/dev_guide/collections_tech_preview.rst deleted file mode 100644 index 0a317ca380..0000000000 --- a/docs/docsite/rst/dev_guide/collections_tech_preview.rst +++ /dev/null @@ -1,537 +0,0 @@ -:orphan: - -.. _collections: - -*********** -Collections -*********** - - -Collections are a distribution format for Ansible content. They can be used to -package and distribute playbooks, roles, modules, and plugins. -You can publish and use collections through `Ansible Galaxy `_. - -.. important:: - This feature is available in Ansible 2.8 as a *Technology Preview* and therefore is not fully supported. It should only be used for testing and should not be deployed in a production environment. - Future Galaxy or Ansible releases may introduce breaking changes. - - -.. contents:: - :local: - :depth: 2 - -Collection structure -==================== - -Collections follow a simple data structure. None of the directories are required unless you have specific content that belongs in one of them. A collection does require a ``galaxy.yml`` file at the root level of the collection. This file contains all of the metadata that Galaxy -and other tools need in order to package, build and publish the collection.:: - - collection/ - ├── docs/ - ├── galaxy.yml - ├── plugins/ - │ ├── modules/ - │ │ └── module1.py - │ ├── inventory/ - │ └── .../ - ├── README.md - ├── roles/ - │ ├── role1/ - │ ├── role2/ - │ └── .../ - ├── playbooks/ - │ ├── files/ - │ ├── vars/ - │ ├── templates/ - │ └── tasks/ - └── tests/ - - -.. note:: - * Ansible only accepts ``.yml`` extensions for galaxy.yml. - * See the `draft collection `_ for an example of a full collection structure. - * Not all directories are currently in use. Those are placeholders for future features. - - -galaxy.yml ----------- - -A collection must have a ``galaxy.yml`` file that contains the necessary information to build a collection artifact. -See :ref:`collections_galaxy_meta` for details. - - -docs directory ---------------- - -Keep general documentation for the collection here. Plugins and modules still keep their specific documentation embedded as Python docstrings. Use the ``docs`` folder to describe how to use the roles and plugins the collection provides, role requirements, and so on. Currently we are looking at Markdown as the standard format for documentation files, but this is subject to change. - -Use ``ansible-doc`` to view documentation for plugins inside a collection: - -.. code-block:: bash - - ansible-doc -t lookup my_namespace.my_collection.lookup1 - -The ``ansible-doc`` command requires the fully qualified collection name (FQCN) to display specific plugin documentation. In this example, ``my_namespace`` is the namespace and ``my_collection`` is the collection name within that namespace. - -.. note:: The Ansible collection namespace is defined in the ``galaxy.yml`` file and is not equivalent to the GitHub repository name. - - -plugins directory ------------------- - -Add a 'per plugin type' specific subdirectory here, including ``module_utils`` which is usable not only by modules, but by any other plugin by using their FQCN. This is a way to distribute modules, lookups, filters, and so on, without having to import a role in every play. - -module_utils -^^^^^^^^^^^^ - -When coding with ``module_utils`` in a collection, the Python ``import`` statement needs to take into account the FQCN along with the ``ansible_collections`` convention. The resulting Python import will look like ``from ansible_collections.{namespace}.{collection}.plugins.module_utils.{util} import {something}`` - -The following example snippets show a Python and PowerShell module using both default Ansible ``module_utils`` and -those provided by a collection. In this example the namespace is ``ansible_example``, the collection is ``community``. -In the Python example the ``module_util`` in question is called ``qradar`` such that the FQCN is -``ansible_example.community.plugins.module_utils.qradar``: - -.. code-block:: python - - from ansible.module_utils.basic import AnsibleModule - from ansible.module_utils._text import to_text - - from ansible.module_utils.six.moves.urllib.parse import urlencode, quote_plus - from ansible.module_utils.six.moves.urllib.error import HTTPError - from ansible_collections.ansible_example.community.plugins.module_utils.qradar import QRadarRequest - - argspec = dict( - name=dict(required=True, type='str'), - state=dict(choices=['present', 'absent'], required=True), - ) - - module = AnsibleModule( - argument_spec=argspec, - supports_check_mode=True - ) - - qradar_request = QRadarRequest( - module, - headers={"Content-Type": "application/json"}, - not_rest_data_keys=['state'] - ) - - -In the PowerShell example the ``module_util`` in question is called ``hyperv`` such that the FCQN is -``ansible_example.community.plugins.module_utils.hyperv``: - -.. code-block:: powershell - - #!powershell - #AnsibleRequires -CSharpUtil Ansible.Basic - #AnsibleRequires -PowerShell ansible_collections.ansible_example.community.plugins.module_utils.hyperv - - $spec = @{ - name = @{ required = $true; type = "str" } - state = @{ required = $true; choices = @("present", "absent") } - } - $module = [Ansible.Basic.AnsibleModule]::Create($args, $spec) - - Invoke-HyperVFunction -Name $module.Params.name - - $module.ExitJson() - - -roles directory ----------------- - -Collection roles are mostly the same as existing roles, but with a couple of limitations: - - - Role names are now limited to contain only lowercase alphanumeric characters, plus ``_`` and start with an alpha character. - - Roles in a collection cannot contain plugins any more. Plugins must live in the collection ``plugins`` directory tree. Each plugin is accessible to all roles in the collection. - -The directory name of the role is used as the role name. Therefore, the directory name must comply with the -above role name rules. -The collection import into Galaxy will fail if a role name does not comply with these rules. - -You can migrate 'traditional roles' into a collection but they must follow the rules above. You man need to rename roles if they don't conform. You will have to move or link any role-based plugins to the collection specific directories. - -.. note:: - - For roles imported into Galaxy directly from a GitHub repository, setting the ``role_name`` value in the role's - metadata overrides the role name used by Galaxy. For collections, that value is ignored. When importing a - collection, Galaxy uses the role directory as the name of the role and ignores the ``role_name`` metadata value. - -playbooks directory --------------------- - -TBD. - -tests directory ----------------- - -TBD. Expect tests for the collection itself to reside here. - - -.. _creating_collections: - -Creating collections -====================== - -To create a collection: - -#. Initialize a collection with :ref:`ansible-galaxy collection init` to create the skeleton directory structure. -#. Add your content to the collection. -#. Build the collection into a collection artifact with :ref:`ansible-galaxy collection build`. -#. Publish the collection artifact to Galaxy with :ref:`ansible-galaxy collection publish`. - -A user can then install your collection on their systems. - -.. note:: - Any references to ``ansible-galaxy`` below will be of a 'working version' that is in development for the 2.9 - release. As such, the command and this documentation section is subject to frequent changes. - -Currently the ``ansible-galaxy collection`` command implements the following sub commands: - -* ``init``: Create a basic collection skeleton based on the default template included with Ansible or your own template. -* ``build``: Create a collection artifact that can be uploaded to Galaxy or your own repository. -* ``publish``: Publish a built collection artifact to Galaxy. -* ``install``: Install one or more collections. - -To learn more about the ``ansible-galaxy`` cli tool, see the :ref:`ansible-galaxy` man page. - -.. _creating_collections_skeleton: - -Creating a collection skeleton ------------------------------- - -To start a new collection: - -.. code-block:: bash - - collection_dir#> ansible-galaxy collection init my_namespace.my_collection - -Then you can populate the directories with the content you want inside the collection. See -https://github.com/bcoca/collection to get a better idea of what you can place inside a collection. - - -.. _building_collections: - -Building collections --------------------- - -To build a collection, run ``ansible-galaxy collection build`` from inside the root directory of the collection: - -.. code-block:: bash - - collection_dir#> ansible-galaxy collection build - -This creates -a tarball of the built collection in the current directory which can be uploaded to Galaxy.:: - - my_collection/ - ├── galaxy.yml - ├── ... - ├── my_namespace-my_collection-1.0.0.tar.gz - └── ... - - -.. note:: - Certain files and folders are excluded when building the collection artifact. This is not currently configurable - and is a work in progress so the collection artifact may contain files you would not wish to distribute. - -This tarball is mainly intended to upload to Galaxy -as a distribution method, but you can use it directly to install the collection on target systems. - -.. _trying_collection_locally: - -Trying collection locally -------------------------- - -You can try your collection locally by installing it from the tarball. - -.. code-block:: bash - - ansible-galaxy collection install my_namespace-my_collection-1.0.0.tar.gz -p ./collections/ansible_collections - -You should use one of the values configured in :ref:`COLLECTIONS_PATHS` for your path. This is also where Ansible itself will expect to find collections when attempting to use them. - -Then try to use the local collection inside a playbook, for more details see :ref:`Using collections ` - -.. _publishing_collections: - -Publishing collections ----------------------- - -You can publish collections to Galaxy using the ``ansible-galaxy collection publish`` command or the Galaxy UI itself. - -.. note:: Once you upload a version of a collection, you cannot delete or modify that version. Ensure that everything looks okay before you upload it. - -Upload using ansible-galaxy -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To upload the collection artifact with the ``ansible-galaxy`` command: - -.. code-block:: bash - - ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz --api-key=SECRET - -The above command triggers an import process, just as if you uploaded the collection through the Galaxy website. -The command waits until the import process completes before reporting the status back. If you wish to continue -without waiting for the import result, use the ``--no-wait`` argument and manually look at the import progress in your -`My Imports `_ page. - -The API key is a secret token used by Ansible Galaxy to protect your content. You can find your API key at your -`Galaxy profile preferences `_ page. - -Upload from the Galaxy website -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To upload your collection artifact directly on Galaxy: - -#. Go to the `My Content `_ page, and click the **Add Content** button on one of your namespaces. -#. From the **Add Content** dialogue, click **Upload New Collection**, and select the collection archive file from your local filesystem. - -When uploading collections it doesn't matter which namespace you select. The collection will be uploaded to the -namespace specified in the collection metadata in the ``galaxy.yml`` file. If you're not an owner of the -namespace, the upload request will fail. - -Once Galaxy uploads and accepts a collection, you will be redirected to the **My Imports** page, which displays output from the -import process, including any errors or warnings about the metadata and content contained in the collection. - - -Collection versions -------------------- - -Once you upload a version of a collection, you cannot delete or modify that version. Ensure that everything looks okay before -uploading. The only way to change a collection is to release a new version. The latest version of a collection (by highest version number) -will be the version displayed everywhere in Galaxy; however, users will still be able to download older versions. - -Collection versions use `Sematic Versioning ` for version numbers. Please read the official documentation for details and examples. In summary: - -* Increment major (for example: x in `x.y.z`) version number for an incompatible API change. -* Increment minor (for example: y in `x.y.z`) version number for new functionality in a backwards compatible manner. -* Increment patch (for example: z in `x.y.z`) version number for backwards compatible bug fixes. - -Migrating Ansible content to a collection -========================================= - -You can experiment with migrating existing modules into a collection using the `content_collector tool `_. The ``content_collector`` is a playbook that helps you migrate content from an Ansible distribution into a collection. - -.. warning:: - - This tool is in active development and is provided only for experimentation and feedback at this point. - -See the `content_collector README `_ for full details and usage guidelines. - -Installing collections -====================== - -You can use the ``ansible-galaxy collection install`` command to install a collection on your system. You must specify an installation location using the ``-p`` option. - -To install a collection hosted in Galaxy: - -.. code-block:: bash - - ansible-galaxy collection install my_namespace.my_collection -p /collections - - -You can also directly use the tarball from your build: - -.. code-block:: bash - - ansible-galaxy collection install my_namespace-my_collection-1.0.0.tar.gz -p ./collections - -.. note:: - The install command automatically appends the path ``ansible_collections`` to the one specified with the ``-p`` option unless the - parent directory is already in a folder called ``ansible_collections``. - - -You should use one of the values configured in :ref:`COLLECTIONS_PATHS` for your path. This is also where Ansible itself will expect to find collections when attempting to use them. - -You can also keep a collection adjacent to the current playbook, under a ``collections/ansible_collections/`` directory structure. - -:: - - play.yml - ├── collections/ - │ └── ansible_collections/ - │ └── my_namespace/ - │ └── my_collection/ - - -Installing an older version of a collection -------------------------------------------- - -By default ``ansible-galaxy`` installs the latest collection that is available but you can add a version range -identifier to install a specific version. - -To install the 1.0.0 version of the collection: - -.. code-block:: bash - - ansible-galaxy collection install my_namespace.my_collection:1.0.0 - -To install the 1.0.0-beta.1 version of the collection: - -.. code-block:: bash - - ansible-galaxy collection install my_namespace.my_collection:==1.0.0-beta.1 - -To install the collections that are greater than or equal to 1.0.0 or less than 2.0.0: - -.. code-block:: bash - - ansible-galaxy collection install my_namespace.my_collection:>=1.0.0,<2.0.0 - - -You can specify multiple range identifiers which are split by ``,``. You can use the following range identifiers: - -* ``*``: Any version, this is the default used when no range specified is set. -* ``!=``: Version is not equal to the one specified. -* ``==``: Version must be the one specified. -* ``>=``: Version is greater than or equal to the one specified. -* ``>``: Version is greater than the one specified. -* ``<=``: Version is less than or equal to the one specified. -* ``<``: Version is less than the one specified. - -.. note:: - The ``ansible-galaxy`` command ignores any pre-release versions unless the ``==`` range identifier is used to - explicitly set to that pre-release version. - - -.. _collection_requirements_file: - -Install multiple collections with a requirements file ------------------------------------------------------ - -You can also setup a ``requirements.yml`` file to install multiple collections in one command. This file is a YAML file in the format: - -.. code-block:: yaml+jinja - - --- - collections: - # With just the collection name - - my_namespace.my_collection - - # With the collection name, version, and source options - - name: my_namespace.my_other_collection - version: 'version range identifiers (default: ``*``)' - source: 'The Galaxy URL to pull the collection from (default: ``--api-server`` from cmdline)' - -The ``version`` key can take in the same range identifier format documented above. - -Roles can also be specified and placed under the ``roles`` key. The values follow the same format as a requirements -file used in older Ansible releases. - -.. note:: - While both roles and collections can be specified in one requirements file, they need to be installed separately. - The ``ansible-galaxy role install -r requirements.yml`` will only install roles and - ``ansible-galaxy collection install -r requirements.yml -p ./`` will only install collections. - -.. _galaxy_server_config: - -Galaxy server configuration list --------------------------------- - -By default running ``ansible-galaxy`` will use the :ref:`galaxy_server` config value or the ``--server`` command line -argument when it performs an action against a Galaxy server. The ``ansible-galaxy collection install`` supports -installing collections from multiple servers as defined in the :ref:`ansible_configuration_settings_locations` file -using the :ref:`galaxy_server_list` configuration option. To define multiple Galaxy servers you have to create the -following entries like so: - -.. code-block:: ini - - [galaxy] - server_list = my_org_hub, release_galaxy, test_galaxy - - [galaxy_server.my_org_hub] - url=https://automation.my_org/ - username=my_user - password=my_pass - - [galaxy_server.release_galaxy] - url=https://galaxy.ansible.com/ - token=my_token - - [galaxy_server.test_galaxy] - url=https://galaxy-dev.ansible.com/ - token=my_token - -.. note:: - You can use the ``--server`` command line argument to select an explicit Galaxy server in the ``server_list`` and - the value of this arg should match the name of the server. If the value of ``--server`` is not a pre-defined server - in ``ansible.cfg`` then the value specified will be the URL used to access that server and all pre-defined servers - are ignored. Also the ``--api-key`` argument is not applied to any of the pre-defined servers, it is only applied - if no server list is defined or a URL was specified by ``--server``. - - -The :ref:`galaxy_server_list` option is a list of server identifiers in a prioritized order. When searching for a -collection, the install process will search in that order, e.g. ``my_org_hub`` first, then ``release_galaxy``, and -finally ``test_galaxy`` until the collection is found. The actual Galaxy instance is then defined under the section -``[galaxy_server.{{ id }}]`` where ``{{ id }}`` is the server identifier defined in the list. This section can then -define the following keys: - -* ``url``: The URL of the galaxy instance to connect to, this is required. -* ``token``: A token key to use for authentication against the Galaxy instance, this is mutually exclusive with ``username`` -* ``username``: The username to use for basic authentication against the Galaxy instance, this is mutually exclusive with ``token`` -* ``password``: The password to use for basic authentication - -As well as being defined in the ``ansible.cfg`` file, these server options can be defined as an environment variable. -The environment variable is in the form ``ANSIBLE_GALAXY_SERVER_{{ id }}_{{ key }}`` where ``{{ id }}`` is the upper -case form of the server identifier and ``{{ key }}`` is the key to define. For example I can define ``token`` for -``release_galaxy`` by setting ``ANSIBLE_GALAXY_SERVER_RELEASE_GALAXY_TOKEN=secret_token``. - -For operations where only one Galaxy server is used, i.e. ``publish``, ``info``, ``login`` then the first entry in the -``server_list`` is used unless an explicit server was passed in as a command line argument. - -.. note:: - Once a collection is found, any of its requirements are only searched within the same Galaxy instance as the parent - collection. The install process will not search for a collection requirement in a different Galaxy instance. - - -.. _using_collections: - -Using collections -================= - -Once installed, you can reference a collection content by its FQCN: - -.. code-block:: yaml - - - hosts: all - tasks: - - my_namespace.my_collection.mymodule: - option1: value - -This works for roles or any type of plugin distributed within the collection: - -.. code-block:: yaml - - - hosts: all - tasks: - - import_role: - name: my_namespace.my_collection.role1 - - - my_namespace.mycollection.mymodule: - option1: value - - - debug: - msg: '{{ lookup("my_namespace.my_collection.lookup1", 'param1')| my_namespace.my_collection.filter1 }}' - - -To avoid a lot of typing, you can use the ``collections`` keyword added in Ansible 2.8: - - -.. code-block:: yaml - - - hosts: all - collections: - - my_namespace.my_collection - tasks: - - import_role: - name: role1 - - - mymodule: - option1: value - - - debug: - msg: '{{ lookup("my_namespace.my_collection.lookup1", 'param1')| my_namespace.my_collection.filter1 }}' - -This keyword creates a 'search path' for non namespaced plugin references. It does not import roles or anything else. -Notice that you still need the FQCN for non-action or module plugins. diff --git a/docs/docsite/rst/dev_guide/developing_collections.rst b/docs/docsite/rst/dev_guide/developing_collections.rst new file mode 100644 index 0000000000..331c448bfb --- /dev/null +++ b/docs/docsite/rst/dev_guide/developing_collections.rst @@ -0,0 +1,322 @@ + +.. _developing_collections: + +********************** +Developing collections +********************** + + +Collections are a distribution format for Ansible content. You can use collections to package and distribute playbooks, roles, modules, and plugins. +You can publish and use collections through `Ansible Galaxy `_. + +.. contents:: + :local: + :depth: 2 + +Collection structure +==================== + +Collections follow a simple data structure. None of the directories are required unless you have specific content that belongs in one of them. A collection does require a ``galaxy.yml`` file at the root level of the collection. This file contains all of the metadata that Galaxy +and other tools need in order to package, build and publish the collection:: + + collection/ + ├── docs/ + ├── galaxy.yml + ├── plugins/ + │ ├── modules/ + │ │ └── module1.py + │ ├── inventory/ + │ └── .../ + ├── README.md + ├── roles/ + │ ├── role1/ + │ ├── role2/ + │ └── .../ + ├── playbooks/ + │ ├── files/ + │ ├── vars/ + │ ├── templates/ + │ └── tasks/ + └── tests/ + + +.. note:: + * Ansible only accepts ``.yml`` extensions for galaxy.yml. + * See the `draft collection `_ for an example of a full collection structure. + * Not all directories are currently in use. Those are placeholders for future features. + + +galaxy.yml +---------- + +A collection must have a ``galaxy.yml`` file that contains the necessary information to build a collection artifact. +See :ref:`collections_galaxy_meta` for details. + + +docs directory +--------------- + +Keep general documentation for the collection here. Plugins and modules still keep their specific documentation embedded as Python docstrings. Use the ``docs`` folder to describe how to use the roles and plugins the collection provides, role requirements, and so on. Currently we are looking at Markdown as the standard format for documentation files, but this is subject to change. + +Use ``ansible-doc`` to view documentation for plugins inside a collection: + +.. code-block:: bash + + ansible-doc -t lookup my_namespace.my_collection.lookup1 + +The ``ansible-doc`` command requires the fully qualified collection name (FQCN) to display specific plugin documentation. In this example, ``my_namespace`` is the namespace and ``my_collection`` is the collection name within that namespace. + +.. note:: The Ansible collection namespace is defined in the ``galaxy.yml`` file and is not equivalent to the GitHub repository name. + + +plugins directory +------------------ + +Add a 'per plugin type' specific subdirectory here, including ``module_utils`` which is usable not only by modules, but by any other plugin by using their FQCN. This is a way to distribute modules, lookups, filters, and so on, without having to import a role in every play. + +module_utils +^^^^^^^^^^^^ + +When coding with ``module_utils`` in a collection, the Python ``import`` statement needs to take into account the FQCN along with the ``ansible_collections`` convention. The resulting Python import will look like ``from ansible_collections.{namespace}.{collection}.plugins.module_utils.{util} import {something}`` + +The following example snippets show a Python and PowerShell module using both default Ansible ``module_utils`` and +those provided by a collection. In this example the namespace is ``ansible_example``, the collection is ``community``. +In the Python example the ``module_util`` in question is called ``qradar`` such that the FQCN is +``ansible_example.community.plugins.module_utils.qradar``: + +.. code-block:: python + + from ansible.module_utils.basic import AnsibleModule + from ansible.module_utils._text import to_text + + from ansible.module_utils.six.moves.urllib.parse import urlencode, quote_plus + from ansible.module_utils.six.moves.urllib.error import HTTPError + from ansible_collections.ansible_example.community.plugins.module_utils.qradar import QRadarRequest + + argspec = dict( + name=dict(required=True, type='str'), + state=dict(choices=['present', 'absent'], required=True), + ) + + module = AnsibleModule( + argument_spec=argspec, + supports_check_mode=True + ) + + qradar_request = QRadarRequest( + module, + headers={"Content-Type": "application/json"}, + not_rest_data_keys=['state'] + ) + + +In the PowerShell example the ``module_util`` in question is called ``hyperv`` such that the FCQN is +``ansible_example.community.plugins.module_utils.hyperv``: + +.. code-block:: powershell + + #!powershell + #AnsibleRequires -CSharpUtil Ansible.Basic + #AnsibleRequires -PowerShell ansible_collections.ansible_example.community.plugins.module_utils.hyperv + + $spec = @{ + name = @{ required = $true; type = "str" } + state = @{ required = $true; choices = @("present", "absent") } + } + $module = [Ansible.Basic.AnsibleModule]::Create($args, $spec) + + Invoke-HyperVFunction -Name $module.Params.name + + $module.ExitJson() + + +roles directory +---------------- + +Collection roles are mostly the same as existing roles, but with a couple of limitations: + + - Role names are now limited to contain only lowercase alphanumeric characters, plus ``_`` and start with an alpha character. + - Roles in a collection cannot contain plugins any more. Plugins must live in the collection ``plugins`` directory tree. Each plugin is accessible to all roles in the collection. + +The directory name of the role is used as the role name. Therefore, the directory name must comply with the +above role name rules. +The collection import into Galaxy will fail if a role name does not comply with these rules. + +You can migrate 'traditional roles' into a collection but they must follow the rules above. You man need to rename roles if they don't conform. You will have to move or link any role-based plugins to the collection specific directories. + +.. note:: + + For roles imported into Galaxy directly from a GitHub repository, setting the ``role_name`` value in the role's + metadata overrides the role name used by Galaxy. For collections, that value is ignored. When importing a + collection, Galaxy uses the role directory as the name of the role and ignores the ``role_name`` metadata value. + +playbooks directory +-------------------- + +TBD. + +tests directory +---------------- + +TBD. Expect tests for the collection itself to reside here. + + +.. _creating_collections: + +Creating collections +====================== + +To create a collection: + +#. Initialize a collection with :ref:`ansible-galaxy collection init` to create the skeleton directory structure. +#. Add your content to the collection. +#. Build the collection into a collection artifact with :ref:`ansible-galaxy collection build`. +#. Publish the collection artifact to Galaxy with :ref:`ansible-galaxy collection publish`. + +A user can then install your collection on their systems. + +Currently the ``ansible-galaxy collection`` command implements the following sub commands: + +* ``init``: Create a basic collection skeleton based on the default template included with Ansible or your own template. +* ``build``: Create a collection artifact that can be uploaded to Galaxy or your own repository. +* ``publish``: Publish a built collection artifact to Galaxy. +* ``install``: Install one or more collections. + +To learn more about the ``ansible-galaxy`` cli tool, see the :ref:`ansible-galaxy` man page. + +.. _creating_collections_skeleton: + +Creating a collection skeleton +------------------------------ + +To start a new collection: + +.. code-block:: bash + + collection_dir#> ansible-galaxy collection init my_namespace.my_collection + +Then you can populate the directories with the content you want inside the collection. See +https://github.com/bcoca/collection to get a better idea of what you can place inside a collection. + + +.. _building_collections: + +Building collections +-------------------- + +To build a collection, run ``ansible-galaxy collection build`` from inside the root directory of the collection: + +.. code-block:: bash + + collection_dir#> ansible-galaxy collection build + +This creates +a tarball of the built collection in the current directory which can be uploaded to Galaxy.:: + + my_collection/ + ├── galaxy.yml + ├── ... + ├── my_namespace-my_collection-1.0.0.tar.gz + └── ... + + +.. note:: + Certain files and folders are excluded when building the collection artifact. This is not currently configurable + and is a work in progress so the collection artifact may contain files you would not wish to distribute. + +This tarball is mainly intended to upload to Galaxy +as a distribution method, but you can use it directly to install the collection on target systems. + +.. _trying_collection_locally: + +Trying collection locally +------------------------- + +You can try your collection locally by installing it from the tarball. + +.. code-block:: bash + + ansible-galaxy collection install my_namespace-my_collection-1.0.0.tar.gz -p ./collections/ansible_collections + +You should use one of the values configured in :ref:`COLLECTIONS_PATHS` for your path. This is also where Ansible itself will expect to find collections when attempting to use them. + +Then try to use the local collection inside a playbook, for more details see :ref:`Using collections ` + +.. _publishing_collections: + +Publishing collections +---------------------- + +You can publish collections to Galaxy using the ``ansible-galaxy collection publish`` command or the Galaxy UI itself. + +.. note:: Once you upload a version of a collection, you cannot delete or modify that version. Ensure that everything looks okay before you upload it. + +Upload using ansible-galaxy +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To upload the collection artifact with the ``ansible-galaxy`` command: + +.. code-block:: bash + + ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz --api-key=SECRET + +The above command triggers an import process, just as if you uploaded the collection through the Galaxy website. +The command waits until the import process completes before reporting the status back. If you wish to continue +without waiting for the import result, use the ``--no-wait`` argument and manually look at the import progress in your +`My Imports `_ page. + +The API key is a secret token used by Ansible Galaxy to protect your content. You can find your API key at your +`Galaxy profile preferences `_ page. + +Upload from the Galaxy website +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To upload your collection artifact directly on Galaxy: + +#. Go to the `My Content `_ page, and click the **Add Content** button on one of your namespaces. +#. From the **Add Content** dialogue, click **Upload New Collection**, and select the collection archive file from your local filesystem. + +When uploading collections it doesn't matter which namespace you select. The collection will be uploaded to the +namespace specified in the collection metadata in the ``galaxy.yml`` file. If you're not an owner of the +namespace, the upload request will fail. + +Once Galaxy uploads and accepts a collection, you will be redirected to the **My Imports** page, which displays output from the +import process, including any errors or warnings about the metadata and content contained in the collection. + + +Collection versions +------------------- + +Once you upload a version of a collection, you cannot delete or modify that version. Ensure that everything looks okay before +uploading. The only way to change a collection is to release a new version. The latest version of a collection (by highest version number) +will be the version displayed everywhere in Galaxy; however, users will still be able to download older versions. + +Collection versions use `Sematic Versioning `_ for version numbers. Please read the official documentation for details and examples. In summary: + +* Increment major (for example: x in `x.y.z`) version number for an incompatible API change. +* Increment minor (for example: y in `x.y.z`) version number for new functionality in a backwards compatible manner. +* Increment patch (for example: z in `x.y.z`) version number for backwards compatible bug fixes. + +Migrating Ansible content to a collection +========================================= + +You can experiment with migrating existing modules into a collection using the `content_collector tool `_. The ``content_collector`` is a playbook that helps you migrate content from an Ansible distribution into a collection. + +.. warning:: + + This tool is in active development and is provided only for experimentation and feedback at this point. + +See the `content_collector README `_ for full details and usage guidelines. + +.. seealso:: + + :ref:`collections` + Learn how to install and use collections. + :ref:`collections_galaxy_meta` + Understand the collections metadata structure. + :ref:`developing_modules_general` + Learn about how to write Ansible modules + `Mailing List `_ + The development mailing list + `irc.freenode.net `_ + #ansible IRC chat channel diff --git a/docs/docsite/rst/dev_guide/index.rst b/docs/docsite/rst/dev_guide/index.rst index 411571145b..b74d80b3bd 100644 --- a/docs/docsite/rst/dev_guide/index.rst +++ b/docs/docsite/rst/dev_guide/index.rst @@ -16,6 +16,7 @@ Find the task that best describes what you want to do: * I want to :ref:`add a custom plugin or module locally `. * I want to figure out if :ref:`developing a module is the right approach ` for my use case. + * I want to :ref:`develop a collection `. * I've read the info above, and I'm sure I want to develop a module: @@ -81,6 +82,6 @@ If you prefer to read the entire guide, here's a list of the pages in order. developing_api developing_rebasing developing_module_utilities - collections_tech_preview + developing_collections collections_galaxy_meta overview_architecture diff --git a/docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst b/docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst new file mode 100644 index 0000000000..09227b63ac --- /dev/null +++ b/docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst @@ -0,0 +1,377 @@ + +.. _developing_resource_modules: + +*********************************** +Developing network resource modules +*********************************** + +.. contents:: + :local: + +The resource module builder is an Ansible Playbook that helps developers scaffold and maintain an Ansible network resource module. + +The resource module builder has the following capabilities: + +- Uses a defined model to scaffold a resource module directory layout and initial class files. +- Scaffolds either an Ansible role or a collection. +- Subsequent uses of the resource module builder will only replace the module arspec and file containing the module docstring. +- Allows you to store complex examples along side the model in the same directory. +- Maintains the model as the source of truth for the module and use resource module builder to update the source files as needed. +- Generates working sample modules for both ``_`` and ``_facts``. + +Accessing the resource module builder +===================================== + +To access the resource module builder: + +1. clone the github repository: + + .. code-block:: bash + + git clone https://github.com/ansible-network/resource_module_builder.git + +2. Install the requirements: + + .. code-block:: bash + + pip install -r requirements.txt + +Creating a model +================ + +You must create a model for your new resource. The resource module builder uses this model to create: + +* The scaffold for a new module +* The argspec for the new module +* The docstring for the new module + +The model is then the single source of truth for both the argspec and docstring, keeping them in sync. Use the resource module builder to generate this scaffolding. For any subsequent updates to the module, update the model first and use the resource module builder to update the module argspec and docstring. + +For example, the resource model builder includes the ``myos_interfaces.yml`` sample in the :file:`models` directory, as seen below: + +.. code-block:: yaml + + --- + GENERATOR_VERSION: '1.0' + ANSIBLE_METADATA: | + { + 'metadata_version': '1.1', + 'status': ['preview'], + 'supported_by': '' + } + NETWORK_OS: myos + RESOURCE: interfaces + COPYRIGHT: Copyright 2019 Red Hat + LICENSE: gpl-3.0.txt + + DOCUMENTATION: | + module: myos_interfaces + version_added: 2.9 + short_description: 'Manages attributes of ' + description: 'Manages attributes of .' + author: Ansible Network Engineer + notes: + - 'Tested against ' + options: + config: + description: The provided configuration + type: list + elements: dict + suboptions: + name: + type: str + description: The name of the + some_string: + type: str + description: + - The some_string_01 + choices: + - choice_a + - choice_b + - choice_c + default: choice_a + some_bool: + description: + - The some_bool. + type: bool + some_int: + description: + - The some_int. + type: int + version_added: '1.1' + some_dict: + type: dict + description: + - The some_dict. + suboptions: + property_01: + description: + - The property_01 + type: str + state: + description: + - The state of the configuration after module completion. + type: str + choices: + - merged + - replaced + - overridden + - deleted + default: merged + EXAMPLES: + - deleted_example_01.txt + - merged_example_01.txt + - overridden_example_01.txt + - replaced_example_01.txt + +Notice that you should include examples for each of the states that the resource supports. The resource module builder also includes these in the sample model. + +See `Ansible network resource models `_ for more examples. + +Using the resource module builder +================================= + +To use the resource module builder to create a collection scaffold from your resource model: + +.. code-block:: bash + + ansible-playbook -e rm_dest= \ + -e structure=collection \ + -e collection_org= \ + -e collection_name= \ + -e model= \ + site.yml + +Where the parameters are as follows: + +- ``rm_dest``: The directory where the resource module builder places the files and directories for the resource module and facts modules. +- ``structure``: The directory layout type (role or collection) + + - ``role``: Generate a role directory layout. + - ``collection``: Generate a collection directory layout. + +- ``collection_org``: The organization of the collection, required when `structure=collection`. +- ``collection_name``: The name of the collection, required when `structure=collection`. +- ``model``: The path to the model file. + +To use the resource module builder to create a role scaffold: + +.. code-block:: bash + + ansible-playbook -e rm_dest= \ + -e structure=role \ + -e model= \ + site.yml + +Examples +======== + +Collection directory layout +--------------------------- + +This example shows the directory layout for the following: + +- ``network_os``: myos +- ``resource``: interfaces + +.. code-block:: bash + + ansible-playbook -e rm_dest=~/github/rm_example \ + -e structure=collection \ + -e collection_org=cidrblock \ + -e collection_name=my_collection \ + -e model=models/myos/interfaces/myos_interfaces.yml \ + site.yml + +.. code-block:: text + + ├── docs + ├── LICENSE.txt + ├── playbooks + ├── plugins + | ├── action + | ├── filter + | ├── inventory + | ├── modules + | | ├── __init__.py + | | ├── myos_facts.py + | | └── myos_interfaces.py + | └── module_utils + | ├── __init__.py + | └── network + | ├── __init__.py + | └── myos + | ├── argspec + | | ├── facts + | | | ├── facts.py + | | | └── __init__.py + | | ├── __init__.py + | | └── interfaces + | | ├── __init__.py + | | └── interfaces.py + | ├── config + | | ├── __init__.py + | | └── interfaces + | | ├── __init__.py + | | └── interfaces.py + | ├── facts + | | ├── facts.py + | | ├── __init__.py + | | └── interfaces + | | ├── __init__.py + | | └── interfaces.py + | ├── __init__.py + | └── utils + | ├── __init__.py + | └── utils.py + ├── README.md + └── roles + + +Role directory layout +--------------------- + +This example displays the role directory layout for the following: + +- ``network_os``: myos +- ``resource``: interfaces + +.. code-block:: bash + + ansible-playbook -e rm_dest=~/github/rm_example/roles/my_role \ + -e structure=role \ + -e model=models/myos/interfaces/myos_interfaces.yml \ + site.yml + + +.. code-block:: text + + roles + └── my_role + ├── library + │ ├── __init__.py + │ ├── myos_facts.py + │ └── myos_interfaces.py + ├── LICENSE.txt + ├── module_utils + │ ├── __init__.py + │ └── network + │ ├── __init__.py + │ └── myos + │ ├── argspec + │ │ ├── facts + │ │ │ ├── facts.py + │ │ │ └── __init__.py + │ │ ├── __init__.py + │ │ └── interfaces + │ │ ├── __init__.py + │ │ └── interfaces.py + │ ├── config + │ │ ├── __init__.py + │ │ └── interfaces + │ │ ├── __init__.py + │ │ └── interfaces.py + │ ├── facts + │ │ ├── facts.py + │ │ ├── __init__.py + │ │ └── interfaces + │ │ ├── __init__.py + │ │ └── interfaces.py + │ ├── __init__.py + │ └── utils + │ ├── __init__.py + │ └── utils.py + └── README.md + + +Using the collection +-------------------- + +This example shows how to use the generated collection in a playbook: + + .. code-block:: yaml + + ---- + - hosts: myos101 + gather_facts: False + tasks: + - cidrblock.my_collection.myos_interfaces: + register: result + - debug: + var: result + - cidrblock.my_collection.myos_facts: + - debug: + var: ansible_network_resources + + +Using the role +-------------- + +This example shows how to use the generated role in a playbook: + +.. code-block:: yaml + + - hosts: myos101 + gather_facts: False + roles: + - my_role + + - hosts: myos101 + gather_facts: False + tasks: + - myos_interfaces: + register: result + - debug: + var: result + - myos_facts: + - debug: + var: ansible_network_resources + + +Resource module structure and workflow +====================================== + +The resource module structure includes the following components: + +Module + * ``library/_.py``. + * Imports the ``module_utils`` resource package and calls ``execute_module`` API + + .. code-block:: python + + def main(): + result = (module).execute_module() + +Module argspec + * ``module_utils//argspec//``. + * Argspec for the resource. + +Facts + * ``module_utils//facts//``. + * Populate facts for the resource. + * Entry in ``module_utils//facts/facts.py`` for ``get_facts`` API to keep ``_facts`` module and facts gathered for the resource module in sync for every subset. + * Entry of Resource subset in FACTS_RESOURCE_SUBSETS list in ``module_utils//facts/facts.py`` to make facts collection work. + +Module package in module_utils + * ``module_utils////``. + * Implement ``execute_module`` API that loads the configuration to device and generates the result with ``changed``, ``commands``, ``before`` and ``after`` keys. + * Call ``get_facts`` API that returns the ```` configuration facts or return the difference if the device has onbox diff support. + * Compare facts gathered and given key-values if diff is not supported. + * Generate final configuration. + +Utils + * ``module_utils//utils``. + * Utilities for the ```` platform. + +Developer notes +=============== + +The tests rely on a role generated by the resource module builder. After changes to the resource module builder, the role should be regenerated and the tests modified and run as needed. To generate the role after changes: + +.. code-block:: bash + + rm -rf rmb_tests/roles/my_role + ansible-playbook -e rm_dest=./rmb_tests/roles/my_role \ + -e structure=role \ + -e model=models/myos/interfaces/myos_interfaces.yml \ + site.yml diff --git a/docs/docsite/rst/network/dev_guide/index.rst b/docs/docsite/rst/network/dev_guide/index.rst index eb702f635b..744ed372da 100644 --- a/docs/docsite/rst/network/dev_guide/index.rst +++ b/docs/docsite/rst/network/dev_guide/index.rst @@ -18,6 +18,7 @@ If you want to extend Ansible for Network Automation by creating a module or plu Find the network developer task that best describes what you want to do: + * I want to :ref:`develop a network resource module `. * I want to :ref:`develop a network connection plugin `. * I want to :ref:`document my set of modules for a network platform `. @@ -26,5 +27,6 @@ If you prefer to read the entire guide, here's a list of the pages in order. .. toctree:: :maxdepth: 1 + developing_resource_modules_network developing_plugins_network documenting_modules_network diff --git a/docs/docsite/rst/porting_guides/porting_guide_2.9.rst b/docs/docsite/rst/porting_guides/porting_guide_2.9.rst index 6c5ec0abf6..cb2793f3e5 100644 --- a/docs/docsite/rst/porting_guides/porting_guide_2.9.rst +++ b/docs/docsite/rst/porting_guides/porting_guide_2.9.rst @@ -694,4 +694,28 @@ No notable changes Networking ========== -No notable changes +Network resource modules +------------------------ + +Ansible 2.9 introduced the first batch of network resource modules. These modules improve the usability of Ansible network modules. The older modules are deprecated in Ansible 2.9 and will be removed in Ansible 2.13. You should scan the list of deprecated modules above and replace them with the new network resource modules in your playbooks. + +Top-level connection arguments removed in 2.9 +--------------------------------------------- + +Top-level connection arguments like ``username``, ``host``, and ``password`` are removed in version 2.9. + +**OLD** In Ansible < 2.4 + +.. code-block:: yaml + + - name: example of using top-level options for connection properties + ios_command: + commands: show version + host: "{{ inventory_hostname }}" + username: cisco + password: cisco + authorize: yes + auth_pass: cisco + + +Change your playbooks to the connection types ``network_cli`` and ``netconf`` using standard Ansible connection properties, and setting those properties in inventory by group. As you update your playbooks and inventory files, you can easily make the change to ``become`` for privilege escalation (on platforms that support it). For more information, see the :ref:`using become with network modules` guide and the :ref:`platform documentation`. diff --git a/docs/docsite/rst/user_guide/collections_using.rst b/docs/docsite/rst/user_guide/collections_using.rst new file mode 100644 index 0000000000..cf6dd6447e --- /dev/null +++ b/docs/docsite/rst/user_guide/collections_using.rst @@ -0,0 +1,240 @@ + +.. _collections: + +***************** +Using collections +***************** + +Collections are a distribution format for Ansible content that can include playbooks, roles, modules, and plugins. +You can install and use collections through `Ansible Galaxy `_. + +.. contents:: + :local: + :depth: 2 + +Installing collections +====================== + +You can use the ``ansible-galaxy collection install`` command to install a collection on your system. You must specify an installation location using the ``-p`` option. + +To install a collection hosted in Galaxy: + +.. code-block:: bash + + ansible-galaxy collection install my_namespace.my_collection -p /collections + +You can also directly use the tarball from your build: + +.. code-block:: bash + + ansible-galaxy collection install my_namespace-my_collection-1.0.0.tar.gz -p ./collections + +.. note:: + The install command automatically appends the path ``ansible_collections`` to the one specified with the ``-p`` option unless the + parent directory is already in a folder called ``ansible_collections``. + + +You should use one of the values configured in :ref:`COLLECTIONS_PATHS` for your path. This is also where Ansible itself will expect to find collections when attempting to use them. + +You can also keep a collection adjacent to the current playbook, under a ``collections/ansible_collections/`` directory structure. + +:: + + play.yml + ├── collections/ + │ └── ansible_collections/ + │ └── my_namespace/ + │ └── my_collection/ + + +Installing an older version of a collection +------------------------------------------- + +By default ``ansible-galaxy`` installs the latest collection that is available but you can add a version range +identifier to install a specific version. + +To install the 1.0.0 version of the collection: + +.. code-block:: bash + + ansible-galaxy collection install my_namespace.my_collection:1.0.0 + +To install the 1.0.0-beta.1 version of the collection: + +.. code-block:: bash + + ansible-galaxy collection install my_namespace.my_collection:==1.0.0-beta.1 + +To install the collections that are greater than or equal to 1.0.0 or less than 2.0.0: + +.. code-block:: bash + + ansible-galaxy collection install my_namespace.my_collection:>=1.0.0,<2.0.0 + + +You can specify multiple range identifiers which are split by ``,``. You can use the following range identifiers: + +* ``*``: Any version, this is the default used when no range specified is set. +* ``!=``: Version is not equal to the one specified. +* ``==``: Version must be the one specified. +* ``>=``: Version is greater than or equal to the one specified. +* ``>``: Version is greater than the one specified. +* ``<=``: Version is less than or equal to the one specified. +* ``<``: Version is less than the one specified. + +.. note:: + The ``ansible-galaxy`` command ignores any pre-release versions unless the ``==`` range identifier is used to + explicitly set to that pre-release version. + + +.. _collection_requirements_file: + +Install multiple collections with a requirements file +----------------------------------------------------- + +You can also setup a ``requirements.yml`` file to install multiple collections in one command. This file is a YAML file in the format: + +.. code-block:: yaml+jinja + + --- + collections: + # With just the collection name + - my_namespace.my_collection + + # With the collection name, version, and source options + - name: my_namespace.my_other_collection + version: 'version range identifiers (default: ``*``)' + source: 'The Galaxy URL to pull the collection from (default: ``--api-server`` from cmdline)' + +The ``version`` key can take in the same range identifier format documented above. + +Roles can also be specified and placed under the ``roles`` key. The values follow the same format as a requirements +file used in older Ansible releases. + +.. note:: + While both roles and collections can be specified in one requirements file, they need to be installed separately. + The ``ansible-galaxy role install -r requirements.yml`` will only install roles and + ``ansible-galaxy collection install -r requirements.yml -p ./`` will only install collections. + +.. _galaxy_server_config: + +Galaxy server configuration list +-------------------------------- + +By default running ``ansible-galaxy`` will use the :ref:`galaxy_server` config value or the ``--server`` command line +argument when it performs an action against a Galaxy server. The ``ansible-galaxy collection install`` supports +installing collections from multiple servers as defined in the :ref:`ansible_configuration_settings_locations` file +using the :ref:`galaxy_server_list` configuration option. To define multiple Galaxy servers you have to create the +following entries like so: + +.. code-block:: ini + + [galaxy] + server_list = my_org_hub, release_galaxy, test_galaxy + + [galaxy_server.my_org_hub] + url=https://automation.my_org/ + username=my_user + password=my_pass + + [galaxy_server.release_galaxy] + url=https://galaxy.ansible.com/ + token=my_token + + [galaxy_server.test_galaxy] + url=https://galaxy-dev.ansible.com/ + token=my_token + +.. note:: + You can use the ``--server`` command line argument to select an explicit Galaxy server in the ``server_list`` and + the value of this arg should match the name of the server. If the value of ``--server`` is not a pre-defined server + in ``ansible.cfg`` then the value specified will be the URL used to access that server and all pre-defined servers + are ignored. Also the ``--api-key`` argument is not applied to any of the pre-defined servers, it is only applied + if no server list is defined or a URL was specified by ``--server``. + + +The :ref:`galaxy_server_list` option is a list of server identifiers in a prioritized order. When searching for a +collection, the install process will search in that order, e.g. ``my_org_hub`` first, then ``release_galaxy``, and +finally ``test_galaxy`` until the collection is found. The actual Galaxy instance is then defined under the section +``[galaxy_server.{{ id }}]`` where ``{{ id }}`` is the server identifier defined in the list. This section can then +define the following keys: + +* ``url``: The URL of the galaxy instance to connect to, this is required. +* ``token``: A token key to use for authentication against the Galaxy instance, this is mutually exclusive with ``username`` +* ``username``: The username to use for basic authentication against the Galaxy instance, this is mutually exclusive with ``token`` +* ``password``: The password to use for basic authentication + +As well as being defined in the ``ansible.cfg`` file, these server options can be defined as an environment variable. +The environment variable is in the form ``ANSIBLE_GALAXY_SERVER_{{ id }}_{{ key }}`` where ``{{ id }}`` is the upper +case form of the server identifier and ``{{ key }}`` is the key to define. For example I can define ``token`` for +``release_galaxy`` by setting ``ANSIBLE_GALAXY_SERVER_RELEASE_GALAXY_TOKEN=secret_token``. + +For operations where only one Galaxy server is used, i.e. ``publish``, ``info``, ``login`` then the first entry in the +``server_list`` is used unless an explicit server was passed in as a command line argument. + +.. note:: + Once a collection is found, any of its requirements are only searched within the same Galaxy instance as the parent + collection. The install process will not search for a collection requirement in a different Galaxy instance. + + +.. _using_collections: + +Using collections in a Playbook +=============================== + +Once installed, you can reference a collection content by its fully qualified collection name (FQCN): + +.. code-block:: yaml + + - hosts: all + tasks: + - my_namespace.my_collection.mymodule: + option1: value + +This works for roles or any type of plugin distributed within the collection: + +.. code-block:: yaml + + - hosts: all + tasks: + - import_role: + name: my_namespace.my_collection.role1 + + - my_namespace.mycollection.mymodule: + option1: value + + - debug: + msg: '{{ lookup("my_namespace.my_collection.lookup1", 'param1')| my_namespace.my_collection.filter1 }}' + + +To avoid a lot of typing, you can use the ``collections`` keyword added in Ansible 2.8: + + +.. code-block:: yaml + + - hosts: all + collections: + - my_namespace.my_collection + tasks: + - import_role: + name: role1 + + - mymodule: + option1: value + + - debug: + msg: '{{ lookup("my_namespace.my_collection.lookup1", 'param1')| my_namespace.my_collection.filter1 }}' + +This keyword creates a 'search path' for non namespaced plugin references. It does not import roles or anything else. +Notice that you still need the FQCN for non-action or module plugins. + +.. seealso:: + + :ref:`developing_collections` + Develop or modify a collection. + :ref:`collections_galaxy_meta` + Understand the collections metadata structure. + `Mailing List `_ + The development mailing list + `irc.freenode.net `_ + #ansible IRC chat channel diff --git a/docs/docsite/rst/user_guide/index.rst b/docs/docsite/rst/user_guide/index.rst index eeb22ec262..a20393b26f 100644 --- a/docs/docsite/rst/user_guide/index.rst +++ b/docs/docsite/rst/user_guide/index.rst @@ -4,11 +4,11 @@ User Guide Welcome to the Ansible User Guide! -This guide covers how to work with Ansible, including using the command line, working with inventory, and writing playbooks. +This guide covers how to work with Ansible, including using the command line, working with inventory, and writing playbooks. .. toctree:: :maxdepth: 2 - + quickstart intro_getting_started command_line_tools @@ -23,6 +23,4 @@ This guide covers how to work with Ansible, including using the command line, wo ../plugins/plugins intro_bsd windows - - - + collections_using diff --git a/docs/docsite/rst/user_guide/intro_inventory.rst b/docs/docsite/rst/user_guide/intro_inventory.rst index efa71055b8..74711565a7 100644 --- a/docs/docsite/rst/user_guide/intro_inventory.rst +++ b/docs/docsite/rst/user_guide/intro_inventory.rst @@ -141,6 +141,7 @@ You could also use nested groups to simplify ``prod`` and ``test`` in this inven children: west: +You can find more examples on how to organize your inventories and group your hosts in :ref:`inventory_setup_examples`. If you do have systems in multiple groups, note that variables will come from all of the groups they are a member of. Variable precedence is detailed in :ref:`ansible_variable_precedence`. @@ -649,6 +650,100 @@ For a full list with available plugins and examples, see :ref:`connection_plugin .. note:: If you're reading the docs from the beginning, this may be the first example you've seen of an Ansible playbook. This is not an inventory file. Playbooks will be covered in great detail later in the docs. +.. _inventory_setup_examples: + +Inventory setup examples +======================== + +.. _inventory_setup-per_environment: + +Example: One inventory per environment +-------------------------------------- + +If you need to manage multiple environments it's sometimes prudent to +have only hosts of a single environment defined per inventory. This +way, it is harder to, for instance, accidentally change the state of +nodes inside the "test" environment when you actually wanted to update +some "staging" servers. + +For the example mentioned above you could have an +:file:`inventory_test` file: + +.. code-block:: ini + + [dbservers] + db01.test.example.com + db02.test.example.com + + [appservers] + app01.test.example.com + app02.test.example.com + app03.test.example.com + +That file only includes hosts that are part of the "test" +environment. Define the "staging" machines in another file +called :file:`inventory_staging`: + +.. code-block:: ini + + [dbservers] + db01.staging.example.com + db02.staging.example.com + + [appservers] + app01.staging.example.com + app02.staging.example.com + app03.staging.example.com + +To apply a playbook called :file:`site.yml` +to all the app servers in the test environment, use the +following command:: + + ansible-playbook -i inventory_test site.yml -l appservers + +.. _inventory_setup-per_function: + +Example: Group by function +-------------------------- + +In the previous section you already saw an example for using groups in +order to cluster hosts that have the same function. This allows you, +for instance, to define firewall rules inside a playbook or role +without affecting database servers: + +.. code-block:: yaml + + - hosts: dbservers + tasks: + - name: allow access from 10.0.0.1 + iptables: + chain: INPUT + jump: ACCEPT + source: 10.0.0.1 + +.. _inventory_setup-per_location: + +Example: Group by location +-------------------------- + +Other tasks might be focused on where a certain host is located. Let's +say that ``db01.test.example.com`` and ``app01.test.example.com`` are +located in DC1 while ``db02.test.example.com`` is in DC2: + +.. code-block:: ini + + [dc1] + db01.test.example.com + app01.test.example.com + + [dc2] + db02.test.example.com + +In practice, you might even end up mixing all these setups as you +might need to, on one day, update all nodes in a specific data center +while, on another day, update all the application servers no matter +their location. + .. seealso:: :ref:`inventory_plugins` diff --git a/docs/templates/collections_galaxy_meta.rst.j2 b/docs/templates/collections_galaxy_meta.rst.j2 index cb3568b32a..b65faef118 100644 --- a/docs/templates/collections_galaxy_meta.rst.j2 +++ b/docs/templates/collections_galaxy_meta.rst.j2 @@ -1,13 +1,9 @@ .. _collections_galaxy_meta: ************************************ -Collection Galaxy Metadata Structure +Collection Galaxy metadata structure ************************************ -.. important:: - This feature is available in Ansible 2.8 as a *Technology Preview* and therefore is not fully supported. It should only be used for testing and should not be deployed in a production environment. - Future Galaxy or Ansible releases may introduce breaking changes. - A key component of an Ansible collection is the ``galaxy.yml`` file placed in the root directory of a collection. This file contains the metadata of the collection that is used to generate a collection artifact. @@ -78,3 +74,16 @@ Examples - demo - collection repository: "https://www.github.com/my_org/my_collection" + +.. seealso:: + + :ref:`developing_collections` + Develop or modify a collection. + :ref:`developing_modules_general` + Learn about how to write Ansible modules + :ref:`collections` + Learn how to install and use collections. + `Mailing List `_ + The development mailing list + `irc.freenode.net `_ + #ansible IRC chat channel diff --git a/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py b/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py index 5da769d763..5f3d23ec2c 100644 --- a/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py +++ b/lib/ansible/modules/cloud/amazon/ec2_vpc_subnet.py @@ -89,7 +89,7 @@ EXAMPLES = ''' state: present vpc_id: vpc-123456 cidr: 10.0.1.16/28 - resource_tags: + tags: Name: Database Subnet register: database_subnet diff --git a/lib/ansible/modules/cloud/amazon/sts_assume_role.py b/lib/ansible/modules/cloud/amazon/sts_assume_role.py index ccba39cd6b..2961d4fe90 100644 --- a/lib/ansible/modules/cloud/amazon/sts_assume_role.py +++ b/lib/ansible/modules/cloud/amazon/sts_assume_role.py @@ -86,20 +86,20 @@ EXAMPLES = ''' # Note: These examples do not set authentication details, see the AWS Guide for details. # Assume an existing role (more details: https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) -sts_assume_role: - role_arn: "arn:aws:iam::123456789012:role/someRole" - role_session_name: "someRoleSession" -register: assumed_role +- sts_assume_role: + role_arn: "arn:aws:iam::123456789012:role/someRole" + role_session_name: "someRoleSession" + register: assumed_role # Use the assumed role above to tag an instance in account 123456789012 -ec2_tag: - aws_access_key: "{{ assumed_role.sts_creds.access_key }}" - aws_secret_key: "{{ assumed_role.sts_creds.secret_key }}" - security_token: "{{ assumed_role.sts_creds.session_token }}" - resource: i-xyzxyz01 - state: present - tags: - MyNewTag: value +- ec2_tag: + aws_access_key: "{{ assumed_role.sts_creds.access_key }}" + aws_secret_key: "{{ assumed_role.sts_creds.secret_key }}" + security_token: "{{ assumed_role.sts_creds.session_token }}" + resource: i-xyzxyz01 + state: present + tags: + MyNewTag: value ''' diff --git a/lib/ansible/modules/cloud/hcloud/hcloud_volume.py b/lib/ansible/modules/cloud/hcloud/hcloud_volume.py index 5426c8cee0..0e7896991e 100644 --- a/lib/ansible/modules/cloud/hcloud/hcloud_volume.py +++ b/lib/ansible/modules/cloud/hcloud/hcloud_volume.py @@ -41,7 +41,7 @@ options: type: str size: description: - - The size of the Block Volume. + - The size of the Block Volume in GB. - Required if volume does not yet exists. type: int automount: @@ -126,7 +126,7 @@ hcloud_volume: returned: Always sample: my-volume size: - description: Size in MB of the volume + description: Size in GB of the volume type: int returned: Always sample: 1337 diff --git a/lib/ansible/modules/cloud/ovirt/ovirt_host.py b/lib/ansible/modules/cloud/ovirt/ovirt_host.py index 88e0dad390..e6186f3367 100644 --- a/lib/ansible/modules/cloud/ovirt/ovirt_host.py +++ b/lib/ansible/modules/cloud/ovirt/ovirt_host.py @@ -75,6 +75,7 @@ options: description: - "Indicates that the host should be removed even if it is non-responsive, or if it is part of a Gluster Storage cluster and has volume bricks on it." + - "WARNING: It doesn't forcibly remove the host if another host related operation is being executed on the host at the same time." default: False type: bool override_display: diff --git a/lib/ansible/modules/monitoring/zabbix/zabbix_action.py b/lib/ansible/modules/monitoring/zabbix/zabbix_action.py index 9c117228f3..db60eebe6d 100644 --- a/lib/ansible/modules/monitoring/zabbix/zabbix_action.py +++ b/lib/ansible/modules/monitoring/zabbix/zabbix_action.py @@ -359,6 +359,7 @@ EXAMPLES = ''' event_source: 'trigger' state: present status: enabled + esc_period: 60 conditions: - type: 'trigger_severity' operator: '>=' @@ -381,6 +382,7 @@ EXAMPLES = ''' event_source: 'trigger' state: present status: enabled + esc_period: 60 conditions: - type: 'trigger_name' operator: 'like' @@ -398,6 +400,8 @@ EXAMPLES = ''' - 'Admin' - type: remote_command command: 'systemctl restart zabbix-agent' + command_type: custom_script + execute_on: server run_on_hosts: - 0 @@ -411,6 +415,7 @@ EXAMPLES = ''' event_source: 'trigger' state: present status: enabled + esc_period: 60 conditions: - type: 'trigger_severity' operator: '>=' diff --git a/lib/ansible/modules/network/netconf/netconf_config.py b/lib/ansible/modules/network/netconf/netconf_config.py index ae4cc6d18a..b09d128f31 100644 --- a/lib/ansible/modules/network/netconf/netconf_config.py +++ b/lib/ansible/modules/network/netconf/netconf_config.py @@ -92,10 +92,12 @@ options: error_option: description: - This option controls the netconf server action after an error occurs while editing the configuration. - If the value is I(stop-on-error) abort the config edit on first error, if value is I(continue-on-error) - it continues to process configuration data on error, error is recorded and negative response is generated - if any errors occur. If value is C(rollback-on-error) it rollback to the original configuration in case - any error occurs, this requires the remote Netconf server to support the :rollback-on-error capability. + - If I(error_option=stop-on-error), abort the config edit on first error. + - If I(error_option=continue-on-error), continue to process configuration data on error. + The error is recorded and negative response is generated if any errors occur. + - If I(error_option=rollback-on-error), rollback to the original configuration if + any error occurs. + This requires the remote Netconf server to support the I(error_option=rollback-on-error) capability. default: stop-on-error choices: ['stop-on-error', 'continue-on-error', 'rollback-on-error'] version_added: "2.7" diff --git a/lib/ansible/modules/packaging/language/pip.py b/lib/ansible/modules/packaging/language/pip.py index 2d365c0e28..bc06b39afe 100644 --- a/lib/ansible/modules/packaging/language/pip.py +++ b/lib/ansible/modules/packaging/language/pip.py @@ -95,13 +95,14 @@ options: version_added: "1.3" executable: description: - - The explicit executable or a pathname to the executable to be used to - run pip for a specific version of Python installed in the system. For + - The explicit executable or pathname for the pip executable, + if different from the Ansible Python interpreter. For example C(pip-3.3), if there are both Python 2.7 and 3.3 installations in the system and you want to run pip for the Python 3.3 installation. - It cannot be specified together with the 'virtualenv' parameter (added in 2.1). - By default, it will take the appropriate version for the python interpreter - use by ansible, e.g. pip3 on python 3, and pip2 or pip on python 2. + - Mutually exclusive with I(virtualenv) (added in 2.1). + - Does not affect the Ansible Python interpreter. + - The setuptools package must be installed for both the Ansible Python interpreter + and for the version of Python specified by this option. type: path version_added: "1.3" umask: @@ -114,11 +115,16 @@ options: type: str version_added: "2.1" notes: - - Please note that virtualenv (U(http://www.virtualenv.org/)) must be + - The virtualenv (U(http://www.virtualenv.org/)) must be installed on the remote host if the virtualenv parameter is specified and the virtualenv needs to be created. - - By default, this module will use the appropriate version of pip for the - interpreter used by ansible (e.g. pip3 when using python 3, pip2 otherwise) + - Although it executes using the Ansible Python interpreter, the pip module shells out to + run the actual pip command, so it can use any pip version you specify with I(executable). + By default, it uses the pip version for the Ansible Python interpreter. For example, pip3 on python 3, and pip2 or pip on python 2. + - The interpreter used by Ansible + (see :ref:`ansible_python_interpreter`) + requires the setuptools package, regardless of the version of pip set with + the I(executable) option. requirements: - pip - virtualenv diff --git a/lib/ansible/modules/system/dconf.py b/lib/ansible/modules/system/dconf.py index 8183b07648..fce014b254 100644 --- a/lib/ansible/modules/system/dconf.py +++ b/lib/ansible/modules/system/dconf.py @@ -39,6 +39,10 @@ notes: wanted to provide a string value, the correct syntax would be C(value="'myvalue'") - with single quotes as part of the Ansible parameter value. + - When using loops in combination with a value like + :code:`"[('xkb', 'us'), ('xkb', 'se')]"`, you need to be aware of possible + type conversions. Applying a filter :code:`"{{ item.value | string }}"` + to the parameter variable can avoid potential conversion problems. - The easiest way to figure out exact syntax/value you need to provide for a key is by making the configuration change in application affected by the key, and then having a look at value set via commands C(dconf dump -- cgit v1.2.1