Doc fixes (#20353)

* cleanup to reflect current builds

* consolidate templating docs and minor rewording

* new templating intro page

* fixed warnings as per feedback

* Update playbooks_filters.rst

Edited for clarity.

* Update playbooks_templating.rst

Light edits

8 files changed, 109 insertions, 85 deletions

diff --git a/docs/docsite/Makefile b/docs/docsite/Makefile
index 78b3616565..84625869a5 100644
--- a/docs/docsite/Makefile
+++ b/docs/docsite/Makefile
@@ -1,4 +1,4 @@
-SITELIB = $(shell python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")
+SITELIB = $(shell python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"):
 FORMATTER=../../hacking/module_formatter.py
 DUMPER=../../hacking/dump_playbook_attributes.py
 ifeq ($(shell echo $(OS) | egrep -c 'Darwin|FreeBSD|OpenBSD|DragonFly'),1)
@@ -7,22 +7,16 @@ else
 CPUS := $(shell nproc)
 endif
 
-all: clean docs
+all: docs
 
-docs: clean directives modules htmldocs
-	-(cp *.ico htmlout/)
-	-(cp *.jpg htmlout/)
-	-(cp *.png htmlout/)
-
-variables:
-	(mkdir -p htmlout/)
-	dot variables.dot -Tpng -o htmlout/variables.png
+docs: clean htmldocs
 
 htmldocs: directives modules staticmin
 	CPUS=$(CPUS) $(MAKE) -f Makefile.sphinx html
 
-webdocs: htmldocs
+webdocs: docs
 
+#TODO: leaving htmlout removal for those having older versions, should eventually be removed also
 clean:
 	-rm -rf htmlout
 	-rm -rf _build
diff --git a/docs/docsite/rst/playbooks.rst b/docs/docsite/rst/playbooks.rst
index 8afff023e6..c6f6e5057e 100644
--- a/docs/docsite/rst/playbooks.rst
+++ b/docs/docsite/rst/playbooks.rst
@@ -16,14 +16,12 @@ ways to organize playbooks and the files they include, and we'll offer up some s
 It is recommended to look at `Example Playbooks <https://github.com/ansible/ansible-examples>`_ while reading along with the playbook documentation.  These illustrate best practices as well as how to put many of the various concepts together.
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
 
    playbooks_intro
    playbooks_roles
    playbooks_variables
-   playbooks_filters
-   playbooks_filters_ipaddr
-   playbooks_tests
+   playbooks_templating
    playbooks_conditionals
    playbooks_loops
    playbooks_blocks
diff --git a/docs/docsite/rst/playbooks_conditionals.rst b/docs/docsite/rst/playbooks_conditionals.rst
index 423bf6eb27..78959ef57e 100644
--- a/docs/docsite/rst/playbooks_conditionals.rst
+++ b/docs/docsite/rst/playbooks_conditionals.rst
@@ -4,10 +4,11 @@ Conditionals
 .. contents:: Topics
 
 
-Often the result of a play may depend on the value of a variable, fact (something learned about the remote system), 
-or previous task result.  In some cases, the values of variables may depend on other variables.  
-Further, additional groups can be created to manage hosts based on
-whether the hosts match other criteria.   There are many options to control execution flow in Ansible. More examples of supported conditionals can be located here: http://jinja.pocoo.org/docs/dev/templates/#comparisons
+Often the result of a play may depend on the value of a variable, fact (something learned about the remote system), or previous task result.
+In some cases, the values of variables may depend on other variables.
+Further, additional groups can be created to manage hosts based on whether the hosts match other criteria.
+There are many options to control execution flow in Ansible.
+More examples of supported conditionals can be located here: http://jinja.pocoo.org/docs/dev/templates/#comparisons
 
 Let's dig into what they are.
 
@@ -16,8 +17,8 @@ Let's dig into what they are.
 The When Statement
 ``````````````````
 
-Sometimes you will want to skip a particular step on a particular host.  This could be something
-as simple as not installing a certain package if the operating system is a particular version,
+Sometimes you will want to skip a particular step on a particular host.
+This could be something as simple as not installing a certain package if the operating system is a particular version,
 or it could be something like performing some cleanup steps if a filesystem is getting full.
 
 This is easy to do in Ansible with the `when` clause, which contains a raw Jinja2 expression without double curly braces (see :doc:`playbooks_variables`).
@@ -95,13 +96,12 @@ Then a conditional execution might look like::
           when: epic
 
 or::
- 
+
     tasks:
         - shell: echo "This certainly isn't epic!"
           when: not epic
 
-If a required variable has not been set, you can skip or fail using Jinja2's
-`defined` test. For example::
+If a required variable has not been set, you can skip or fail using Jinja2's `defined` test. For example::
 
     tasks:
         - shell: echo "I've got '{{ foo }}' and am not afraid to use it!"
@@ -110,8 +110,8 @@ If a required variable has not been set, you can skip or fail using Jinja2's
         - fail: msg="Bailing out. this play requires 'bar'"
           when: bar is undefined
 
-This is especially useful in combination with the conditional import of vars
-files (see below).
+This is especially useful in combination with the conditional import of vars files (see below).
+As the examples show, you don't need to use `{{ }}` to use variables inside conditionals, as these are already implied.
 
 .. _loops_and_conditionals:
 
diff --git a/docs/docsite/rst/playbooks_filters.rst b/docs/docsite/rst/playbooks_filters.rst
index 2655e9add0..1a2c37bcdf 100644
--- a/docs/docsite/rst/playbooks_filters.rst
+++ b/docs/docsite/rst/playbooks_filters.rst
@@ -1,20 +1,19 @@
-Jinja2 filters
-==============
+Filters
+-------
 
 .. contents:: Topics
 
 
-Filters in Jinja2 are a way of transforming template expressions from one kind of data into another.
-Jinja2 ships with many of these. See `builtin filters`_ in the official Jinja2 template documentation.
+Filters in Ansible are from Jinja2, and are used for transforming data inside a template expression.  Jinja2 ships with many filters. See `builtin filters`_ in the official Jinja2 template documentation.
 
-Take into account that filters always execute on the Ansible controller, **not** on the task target, as they manipulate local data.
+Take into account that templating happens on the the Ansible controller, **not** on the task's target host, so filters also execute on the controller as they manipulate local data.
 
-In addition to those, Ansible supplies many more.
+In addition the ones provided by Jinja2, Ansible ships with it's own and allows users to add their own custom filters.
 
 .. _filters_for_formatting_data:
 
 Filters For Formatting Data
----------------------------
+```````````````````````````
 
 The following filters will take a data structure in a template and render it in a slightly different format.  These
 are occasionally useful for debugging::
@@ -48,7 +47,7 @@ for example::
 .. _forcing_variables_to_be_defined:
 
 Forcing Variables To Be Defined
--------------------------------
+```````````````````````````````
 
 The default behavior from ansible and ansible.cfg is to fail if variables are undefined, but you can turn this off.
 
@@ -62,7 +61,7 @@ The variable value will be used as is, but the template evaluation will raise an
 .. _defaulting_undefined_variables:
 
 Defaulting Undefined Variables
-------------------------------
+``````````````````````````````
 
 Jinja2 provides a useful 'default' filter, that is often a better approach to failing if a variable is not defined::
 
@@ -75,7 +74,7 @@ being raised.
 .. _omitting_undefined_variables:
 
 Omitting Parameters
--------------------
+```````````````````
 
 As of Ansible 1.8, it is possible to use the default filter to omit module parameters using the special `omit` variable::
 
@@ -98,7 +97,7 @@ parameter will not be sent to the file module while the final file will receive
 .. _list_filters:
 
 List Filters
-------------
+````````````
 
 These filters all operate on list variables.
 
@@ -115,7 +114,7 @@ To get the maximum value from a list of numbers::
 .. _set_theory_filters:
 
 Set Theory Filters
-------------------
+``````````````````
 All these functions return a unique set from sets or lists.
 
 .. versionadded:: 1.4
@@ -144,7 +143,7 @@ To get the symmetric difference of 2 lists (items exclusive to each list)::
 .. _random_filter:
 
 Random Number Filter
---------------------
+````````````````````
 
 .. versionadded:: 1.6
 
@@ -179,7 +178,7 @@ As of Ansible version 2.3, it's also possible to initialize the random number ge
 
 
 Shuffle Filter
---------------
+``````````````
 
 .. versionadded:: 1.8
 
@@ -203,7 +202,8 @@ note that when used with a non 'listable' item it is a noop, otherwise it always
 .. _math_stuff:
 
 Math
---------------------
+````
+j
 .. versionadded:: 1.9
 
 
@@ -230,7 +230,8 @@ Note that jinja2 already provides some like abs() and round().
 .. json_query_filter:
 
 JSON Query Filter
------------------
+`````````````````
+
 .. versionadded:: 2.2
 
 Sometimes you end up with a complex data structure in JSON format and you need to extract only a small set of data within it. The **json_query** filter lets you query a complex JSON structure and iterate over it using a with_items structure.
@@ -296,7 +297,8 @@ In this example, we get a hash map with all ports and names of a cluster::
 .. _ipaddr_filter:
 
 IP address filter
------------------
+`````````````````
+
 .. versionadded:: 1.9
 
 To test if a string is a valid IP address::
@@ -316,10 +318,12 @@ address. For example, to get the IP address itself from a CIDR, you can use::
 More information about ``ipaddr`` filter and complete usage guide can be found
 in :doc:`playbooks_filters_ipaddr`.
 
+
 .. _hash_filters:
 
 Hashing filters
---------------------
+```````````````
+
 .. versionadded:: 1.9
 
 To get the sha1 hash of a string::
@@ -353,7 +357,7 @@ Hash types available depend on the master system running ansible,
 .. _combine_filter:
 
 Combining hashes/dictionaries
------------------------------
+`````````````````````````````
 
 .. versionadded:: 2.0
 
@@ -391,7 +395,7 @@ setting in `ansible.cfg`.
 .. _extract_filter:
 
 Extracting values from containers
----------------------------------
+`````````````````````````````````
 
 .. versionadded:: 2.1
 
@@ -424,7 +428,7 @@ This would return a list containing the value of `b['a']['x']['y']`.
 .. _comment_filter:
 
 Comment Filter
---------------
+``````````````
 
 .. versionadded:: 2.0
 
@@ -496,21 +500,11 @@ which will produce this output:
     # host: myhost
     #
 
-ipaddr() filters
-----------------
-
-A number of filters are available, including:
-
-* `ipaddr`
-* `ipv4`
-* `ipv6`
-
-See :doc:`playbooks_filters_ipaddr`
 
 .. _other_useful_filters:
 
 Other Useful Filters
---------------------
+````````````````````
 
 To add quotes for shell usage::
 
@@ -618,7 +612,7 @@ To get date object from string use the `to_datetime` filter, (new in version in
     {{ (("2016-08-04 20:00:12"|to_datetime) - ("2015-10-06"|to_datetime('%Y-%d-%m'))).seconds  }}
 
 Debugging Filters
------------------
+`````````````````
 
 .. versionadded:: 2.3
 
diff --git a/docs/docsite/rst/playbooks_filters_ipaddr.rst b/docs/docsite/rst/playbooks_filters_ipaddr.rst
index cc9baf6113..946dcb72b6 100644
--- a/docs/docsite/rst/playbooks_filters_ipaddr.rst
+++ b/docs/docsite/rst/playbooks_filters_ipaddr.rst
@@ -1,5 +1,7 @@
-Jinja2 'ipaddr()' filter
-========================
+:orphan:
+
+ipaddr filter
+`````````````
 
 .. versionadded:: 1.9
 
@@ -25,7 +27,7 @@ It can usually be installed either via your system package manager, or using
    :backlinks: top
 
 Basic tests
------------
+^^^^^^^^^^^
 
 ``ipaddr()`` is designed to return the input value if a query is True, and
 ``False`` if query is False. This way it can be very easily used in chained
@@ -86,7 +88,7 @@ And the same data filtered for IPv6 addresses::
 
 
 Filtering lists
----------------
+^^^^^^^^^^^^^^^
 
 You can filter entire lists - ``ipaddr()`` will return a list with values
 valid for a particular query::
@@ -105,7 +107,7 @@ valid for a particular query::
 
 
 Wrapping IPv6 addresses in [ ] brackets
----------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Some configuration files require IPv6 addresses to be "wrapped" in square
 brackets (``[ ]``). To accomplish that, you can use ``ipwrap()`` filter. It
@@ -124,7 +126,7 @@ chain both filters together::
 
 
 Basic queries
--------------
+^^^^^^^^^^^^^
 
 You can provide single argument to each ``ipaddr()`` filter. Filter will then
 treat it as a query and return values modified by that query. Lists will
@@ -140,7 +142,7 @@ If a query type is not recognized, Ansible will raise an error.
 
 
 Getting information about hosts and networks
---------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Here's our test list again::
 
@@ -224,7 +226,7 @@ end of the range::
 
 
 Getting information from host/prefix values
--------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Very frequently you use combination of IP addresses and subnet prefixes
 ("CIDR"), this is even more common with IPv6. ``ipaddr()`` filter can extract
@@ -288,7 +290,7 @@ If needed, you can extract subnet and prefix information from 'host/prefix' valu
     [64, 24]
 
 Converting subnet masks to CIDR notation
-----------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Given a subnet in the form of network address and subnet mask, it can be converted into CIDR notation using ``ipaddr()``.  This can be useful for converting Ansible facts gathered about network configuration from subnet masks into CIDR format::
 
@@ -320,7 +322,7 @@ This result can be canonicalised with ``ipaddr()`` to produce a subnet in CIDR f
 
 
 IP address conversion
----------------------
+^^^^^^^^^^^^^^^^^^^^^
 
 Here's our test list again::
 
@@ -361,7 +363,7 @@ You can convert IP addresses to PTR records::
 
 
 Converting IPv4 address to 6to4 address
----------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 `6to4`_ tunnel is a way to access IPv6 Internet from IPv4-only network. If you
 have a public IPv4 address, you automatically can configure it's IPv6
@@ -379,7 +381,7 @@ be automatically converted to a router address (with ``::1/48`` host address)::
 
 
 Subnet manipulation
--------------------
+^^^^^^^^^^^^^^^^^^^
 
 ``ipsubnet()`` filter can be used to manipulate network subnets in several ways.
 
@@ -462,7 +464,7 @@ difference between subnets.
 
 
 MAC address filter
-------------------
+^^^^^^^^^^^^^^^^^^
 
 You can use ``hwaddr()`` filter to check if a given string is a MAC address or
 convert it between various formats. Examples::
diff --git a/docs/docsite/rst/playbooks_lookups.rst b/docs/docsite/rst/playbooks_lookups.rst
index 97ae765a3c..5bfa3a6390 100644
--- a/docs/docsite/rst/playbooks_lookups.rst
+++ b/docs/docsite/rst/playbooks_lookups.rst
@@ -1,10 +1,9 @@
-Using Lookups
-=============
+Lookups
+-------
 
-Lookup plugins allow access of data in Ansible from outside sources.  These plugins are evaluated on the Ansible control
+Lookup plugins allow access of data in Ansible from outside sources. Like all templating, these plugins are evaluated on the Ansible control
 machine, and can include reading the filesystem but also contacting external datastores and services.
-These values are then made available using the standard templating system
-in Ansible, and are typically used to load variables or templates with information from those systems.
+These values are then made available using the standard templating system in Ansible, and are typically used to load variables or templates with information from those systems.
 
 .. note:: This is considered an advanced feature, and many users will probably not rely on these features.
 
diff --git a/docs/docsite/rst/playbooks_templating.rst b/docs/docsite/rst/playbooks_templating.rst
new file mode 100644
index 0000000000..455182c2e6
--- /dev/null
+++ b/docs/docsite/rst/playbooks_templating.rst
@@ -0,0 +1,37 @@
+Templating (Jinja2)
+===================
+
+As already referenced in the variables section, Ansible uses Jinja2 templating to enable dynamic expressions and access to variables.
+Jinja2 also greatly expands the number of filters and tests available, as well as adding a new plugin type: lookups.
+
+Please note that all templating happens on the Ansible controller before the task is sent and executed on the target machine. This is done to minimize the requirements on the target (jinja2 is only required on the controller) and to be able to pass the minimal information needed for the task, so the target machine does not need a copy of all the data that the controller has access to.
+
+.. contents:: Topics
+
+.. toctree::
+   :maxdepth: 2
+
+   playbooks_filters
+   playbooks_tests
+   playbooks_lookups
+
+
+
+.. seealso::
+
+   :doc:`playbooks`
+       An introduction to playbooks
+   :doc:`playbooks_conditionals`
+       Conditional statements in playbooks
+   :doc:`playbooks_loops`
+       Looping in playbooks
+   :doc:`playbooks_roles`
+       Playbook organization by roles
+   :doc:`playbooks_best_practices`
+       Best practices in playbooks
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel
+
+
diff --git a/docs/docsite/rst/playbooks_tests.rst b/docs/docsite/rst/playbooks_tests.rst
index 1ea7de62f7..a420e578cc 100644
--- a/docs/docsite/rst/playbooks_tests.rst
+++ b/docs/docsite/rst/playbooks_tests.rst
@@ -1,5 +1,5 @@
-Jinja2 tests
-============
+Tests
+-----
 
 .. contents:: Topics
 
@@ -8,14 +8,14 @@ Tests in Jinja2 are a way of evaluating template expressions and returning True
 Jinja2 ships with many of these. See `builtin tests`_ in the official Jinja2 template documentation.
 Tests are very similar to filters and are used mostly the same way, but they can also be used in list processing filters, like C(map()) and C(select()) to choose items in the list.
 
-Like filters, tests always execute on the Ansible controller, **not** on the target of a task, as they test local data.
+Like all templating, tests always execute on the Ansible controller, **not** on the target of a task, as they test local data.
 
 In addition to those Jinja2 tests, Ansible supplies a few more and users can easily create their own.
 
 .. _testing_strings:
 
 Testing strings
----------------
+```````````````
 
 To match strings against a substring or a regex, use the "match" or "search" filter::
 
@@ -38,7 +38,7 @@ To match strings against a substring or a regex, use the "match" or "search" fil
 .. _testing_versions:
 
 Version Comparison
-------------------
+``````````````````
 
 .. versionadded:: 1.6
 
@@ -64,7 +64,7 @@ be used.  The default is ``False``, but this setting as ``True`` uses more stric
 .. _math_tests:
 
 Group theory tests
-------------------
+``````````````````
 
 .. versionadded:: 2.1
 
@@ -84,7 +84,7 @@ To see if a list includes or is included by another list, you can use 'issubset'
 .. _path_tests:
 
 Testing paths
--------------
+`````````````
 
 The following tests can provide information about a path on the controller::
 
@@ -112,7 +112,7 @@ The following tests can provide information about a path on the controller::
 .. _test_task_results:
 
 Task results
-------------
+````````````
 
 The following tasks are illustrative of the tests meant to check the status of tasks::