Install ansible-test (#60718)

* Install ansible-test

Modify the install script to install ansible-test and its supporting
code.  Alternative to #60701 that doesn't change package_dir ansible for
fear that it might regress https://github.com/ansible/ansible/issues/10437

Also:
* No longer use package_data.  Everything in the package dirs is going
  to be installed.  Anything that shouldn't be installed needs to be
  moved elsewhere.
* modify the algorithm to store symlinks which are in the same tree
  instead of same directory

* Add ansible_test files to package-data sanity test

* MANIFEST.in cleanups

* Add lib/ansible/config/*.yml
* Make most things in code directories (lib/ansible and test/lib/ansible_test/)
  use explicit file extensions instead of wildcards for maintainability
* Exclude common file extensions that we don't want included in the code
  directories

* Change package-data test to be more complete

* Now compares the repository, sdist, and install
* Compares both that everything in the sdist is in the repo and
  everything in the install is in the sdist in addition to comparing that
  everything in the repo that we want is in the install

* Leave out test artifacts

Only include the directory structure for test/results and test/cache not
any files that may have been generated by test runs

Remove test/utils files from the sdist as these are only needed for our CI

cleanup of docs in MANIFEST.in; getting rid of build files.

* Add the ability to output sdist and snapshot to specific directory

* Add a warning about modifying the heuristic to setup.py

* Address generated files

* Use make snapshot instead of sdist to generate changelog and man pages
  and make sure they're included
* Ignore both the test/utils and generated test files (results, cache)
* Deal with Python3 __pycache__ byte code caches
* Don't check documentation, that isn't built for the sdist
* Restructure for clarity

* Add cli web docs to make clean

This was causing problems when attempting to test that the sdist didn't
have extra files

* Fix bug constructing python names from __pycache__ names

* Create a clean repo to work from

* Exclude test/legacy and be more explicit on extensions

* Exclude the legacy directory from sdist

1 files changed, 22 insertions, 11 deletions

diff --git a/MANIFEST.in b/MANIFEST.in
index 863b286037..5ec2feb94b 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -3,25 +3,36 @@ include COPYING
 include SYMLINK_CACHE.json
 include requirements.txt
 include shippable.yml
-include bin/ansible-test
+recursive-include docs *
+exclude docs/docsite/rst_warnings
+recursive-exclude docs/docsite/_build *
+recursive-exclude docs/docsite/_extensions *.pyc *.pyo
 include examples/hosts
 include examples/ansible.cfg
 include examples/scripts/ConfigureRemotingForAnsible.ps1
 include examples/scripts/upgrade_to_ps3.ps1
-recursive-include lib/ansible/executor/powershell *
-recursive-include lib/ansible/module_utils/csharp *
-recursive-include lib/ansible/module_utils/powershell *
-recursive-include lib/ansible/modules *
-recursive-include lib/ansible/galaxy/data *
-recursive-include docs *
-recursive-include licenses *
+recursive-include lib/ansible/executor/powershell *.ps1
+recursive-include lib/ansible/module_utils/csharp *.cs
+recursive-include lib/ansible/module_utils/powershell *.psm1
+recursive-include lib/ansible/modules/windows *.ps1
+recursive-include lib/ansible/galaxy/data *.yml *.j2 README.md ansible.cfg inventory .git_keep
+recursive-include lib/ansible/config *.yml
+recursive-include licenses *.txt
 recursive-include packaging *
-recursive-include test *
+recursive-include test/cache .keep
+recursive-include test/integration *
+recursive-include test/lib/ansible_test/config *.template
+recursive-include test/lib/ansible_test/_data *
+recursive-include test/lib/ansible_test/tests *
+recursive-exclude test/lib/ansible_test *.pyc *.pyo *.bak *.orig *~ *.rej
+recursive-include test/results .keep
+recursive-include test/sanity *.json *.py *.txt
+exclude test/sanity/code-smell/botmeta.*
+recursive-include test/units *
 include Makefile
 include MANIFEST.in
 include changelogs/CHANGELOG*.rst
 include contrib/README.md
-recursive-include contrib/inventory *
-exclude test/sanity/code-smell/botmeta.*
+recursive-include contrib/inventory *.py *.ini *.yml *.yaml
 recursive-include hacking/build_library *.py
 include hacking/build-ansible.py