Refactor tox & update docs

- Updated tox envlist, so just running `tox` from the CLI will now run all
voting gate tests

- Reduce duplicated definitions and commands

- Remove any reliance on run_tests within tox

- Removes all doc references to run_tests.sh, and replaces them
with their tox equivalent. Where necessary, language around the tox
commands has been altered or extended so that it makes sense and is
consistent with other parts of the docs. Also adds a new "Test Environment"
list to the docs, so that newcomers do not have to piece together CLI
commands and their cryptic extensions from tox.ini

- Move the inline shell scripting to its own file. Also fixes a bug when
passing args, since the logic assumed you were attempting a subset test
run (try `tox -e py27 -- --pdb` on master to compare)

- Moved translation tooling from run_tests to manage.py, w/ help text
and arg restrictions. This is much more flexible so that plugins can use
it without having to copy commands, but still defaults to exactly the
same parameters/behaviour from run_tests. Docs updated appropriately.

- Removed npm/karma strange reliance on either .venv or tox/py27. Now
it only uses tox/npm.

Change-Id: I883f885bd424955d39ddcfde5ba396a88cfc041e
Implements: blueprint enhance-tox
Closes-Bug: 1638672

21 files changed, 437 insertions, 187 deletions

diff --git a/.gitignore b/.gitignore
index 8a1c59f3c..78303126a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,5 +1,6 @@
 *.egg*
 *.mo
+*.pot
 *.pyc
 *.sw?
 *.sqlite3
diff --git a/doc/source/contributing.rst b/doc/source/contributing.rst
index fc6380097..05c625505 100644
--- a/doc/source/contributing.rst
+++ b/doc/source/contributing.rst
@@ -89,9 +89,8 @@ After You Write Your Patch
 
 Once you've made your changes, there are a few things to do:
 
-* Make sure the unit tests pass: ``./run_tests.sh`` for Python, and ``npm run test`` for JS.
-* Make sure the linting tasks pass: ``./run_tests.sh --pep8`` for Python, and ``npm run lint`` for JS.
-* Make sure your code is ready for translation: ``./run_tests.sh --pseudo de`` See :ref:`pseudo_translation` for more information.
+* Make sure the unit tests and linting tasks pass by running ``tox``
+* Make sure your code is ready for translation: See :ref:`pseudo_translation`.
 * Make sure your code is up-to-date with the latest master: ``git pull --rebase``
 * Finally, run ``git review`` to upload your changes to Gerrit for review.
 
@@ -132,7 +131,7 @@ Python
 ------
 
 We follow PEP8_ for all our Python code, and use ``pep8.py`` (available
-via the shortcut ``./run_tests.sh --pep8``) to validate that our code
+via the shortcut ``tox -e pep8``) to validate that our code
 meets proper Python style guidelines.
 
 .. _PEP8: http://www.python.org/dev/peps/pep-0008/
diff --git a/doc/source/quickstart.rst b/doc/source/quickstart.rst
index d5924ba54..0a5f765b1 100644
--- a/doc/source/quickstart.rst
+++ b/doc/source/quickstart.rst
@@ -22,20 +22,10 @@ On RPM-based distributions (e.g., Fedora/RHEL/CentOS/Scientific Linux)::
 Setup
 =====
 
-To setup a Horizon development environment simply clone the Horizon git
-repository from http://github.com/openstack/horizon and execute the
-``run_tests.sh`` script from the root folder (see :doc:`ref/run_tests`)::
+To begin setting up a Horizon development environment simply clone the Horizon
+git repository from https://git.openstack.org/cgit/openstack/horizon.::
 
-    > git clone https://github.com/openstack/horizon.git
-    > cd horizon
-    > ./run_tests.sh
-
-.. note::
-
-    Running ``run_tests.sh`` will build a virtualenv, ``.venv``, where all the
-    python dependencies for Horizon are installed and referenced. After the
-    dependencies are installed, the unit test suites in the Horizon repo will be
-    executed.  There should be no errors from the tests.
+  > git clone https://git.openstack.org/openstack/horizon
 
 Next you will need to setup your Django application config by copying ``openstack_dashboard/local/local_settings.py.example`` to ``openstack_dashboard/local/local_settings.py``. To do this quickly you can use the following command::
 
@@ -92,21 +82,21 @@ order to prevent Conflicts for future migrations::
     > mv openstack_dashboard/local/local_settings.diff openstack_dashboard/local/local_settings.diff.old
     > python manage.py migrate_settings --gendiff
 
-To start the Horizon development server use ``run_tests.sh``::
+To start the Horizon development server use ``tox``::
 
-    > ./run_tests.sh --runserver localhost:9000
+    > tox -e runserver
 
 .. note::
 
     The default port for runserver is 8000 which is already consumed by
-    heat-api-cfn in DevStack. If not running in DevStack
-    `./run_tests.sh --runserver` will start the test server at
-    `http://localhost:8000`.
+    heat-api-cfn in DevStack. If running in DevStack
+    `tox -e runserver -- localhost:9000` will start the test server at
+    `http://localhost:9000`.
 
 
 .. note::
 
-    The ``run_tests.sh`` script provides wrappers around ``manage.py``.
+    The ``tox`` environments provide wrappers around ``manage.py``.
     For more information on manage.py which is a django, see
     `https://docs.djangoproject.com/en/dev/ref/django-admin/`
 
diff --git a/doc/source/ref/run_tests.rst b/doc/source/ref/run_tests.rst
index 951c82e81..7f88e0d8e 100644
--- a/doc/source/ref/run_tests.rst
+++ b/doc/source/ref/run_tests.rst
@@ -2,6 +2,12 @@
 The ``run_tests.sh`` Script
 ===========================
 
+.. warning::
+
+  This script is deprecated as of Newton (11.0), and will be removed in
+  Queens (13.0), in favor of tox. The tox docs can be found at
+  https://tox.readthedocs.io/en/latest/
+
 .. contents:: Contents:
    :local:
 
diff --git a/doc/source/testing.rst b/doc/source/testing.rst
index 78ddf10c0..d56a3c1a7 100644
--- a/doc/source/testing.rst
+++ b/doc/source/testing.rst
@@ -10,34 +10,33 @@ Because Horizon is composed of both the ``horizon`` app and the
 tests. While they can be run individually without problem, there is an easier
 way:
 
-Included at the root of the repository is the ``run_tests.sh`` script
+Included at the root of the repository is the ``tox.ini`` config
 which invokes both sets of tests, and  optionally generates analyses on both
-components in the process. This script is what Jenkins uses to verify the
+components in the process. ``tox`` is what Jenkins uses to verify the
 stability of the project, so you should make sure you run it and it passes
 before you submit any pull requests/patches.
 
-To run the tests::
+To run all tests::
 
-    $ ./run_tests.sh
-
-It's also possible to :doc:`run a subset of unit tests<ref/run_tests>`.
-
-.. seealso::
-
-    :doc:`ref/run_tests`
-        Full reference for the ``run_tests.sh`` script.
+    $ tox
 
+It's also possible to run a subset of the tests. Open ``tox.ini`` in the
+Horizon root directory to see a list of test environments. You can read more
+about tox in general at https://tox.readthedocs.io/en/latest/.
 
 By default running the Selenium tests will open your Firefox browser (you have
 to install it first, else an error is raised), and you will be able to see the
-tests actions.
+tests actions::
+
+    $ tox -e selenium-headless
+
 If you want to run the suite headless, without being able to see them (as they
 are ran on Jenkins), you can run the tests::
 
-    $ ./run_tests.sh --with-selenium --selenium-headless
+    $ tox -e selenium-headless
 
 Selenium will use a virtual display in this case, instead of your own. In order
-to run the tests this way you have to install the dependency `xvfb`, like 
+to run the tests this way you have to install the dependency `xvfb`, like
 this::
 
     $ sudo apt-get install xvfb
@@ -49,12 +48,90 @@ for a Debian OS flavour, or for Fedora/Red Hat flavours::
 If you can't run a virtual display, or would prefer not to, you can use the
 PhantomJS web driver instead::
 
-    $ ./run_tests.sh --with-selenium --selenium-phantomjs
+    $ tox -e selenium-phantomjs
 
 If you need to install PhantomJS, you may do so with `npm` like this::
 
     $ npm -g install phantomjs
 
+Alternatively, many distributions have system packages for phantomjs, or
+it can be downloaded from http://phantomjs.org/download.html.
+
+tox Test Environments
+=====================
+
+This is a list of test environments available to be executed by
+``tox -e <name>``.
+
+pep8
+----
+
+Runs pep8, which is a tool that checks Python code style. You can read more
+about pep8 at https://www.python.org/dev/peps/pep-0008/
+
+py27dj18, py27dj19, py27dj110
+-----------------------------
+
+Runs the Python unit tests against Django 1.8, Django 1.9 and Django 1.10
+respectively
+
+All other dependencies are as defined by the upper-constraints file at
+https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt
+
+You can run a subset of the tests by passing the test path as an argument to
+tox::
+
+  $ tox -e py27dj18 -- openstack_dashboard.dashboards.identity.users.tests
+
+You can also pass other arguments. For example, to drop into a live debugger
+when a test fails you can use::
+
+  $ tox -e py27dj18 -- --pdb
+
+py34
+----
+
+Runs the Python unit tests with a Python 3.4 environment.
+
+py35
+----
+
+Runs the Python unit tests with a Python 3.5 environment.
+
+releasenotes
+------------
+
+Outputs Horizons release notes as HTML to ``releasenotes/build/html``.
+
+Also takes an alternative builder as an optional argument, such as
+``tox -e docs -- <builder>``, which will output to
+``releasenotes/build/<builder>``. Available builders are listed at
+http://www.sphinx-doc.org/en/latest/builders.html
+
+npm
+---
+
+Installs the npm dependencies listed in ``package.json`` and runs the
+JavaScript tests. Can also take optional arguments, which will be executed
+as an npm script following the dependency install, instead of ``test``.
+
+Example::
+
+  $ tox -e npm -- lintq
+
+docs
+----
+
+Outputs Horizons documentation as HTML to ``doc/build/html``.
+
+Also takes an alternative builder as an optional argument, such as
+``tox -e docs -- <builder>``, which will output to ``doc/build/<builder>``.
+Available builders are listed at
+http://www.sphinx-doc.org/en/latest/builders.html
+
+Example::
+
+  $ tox -e docs -- latexpdf
 
 Writing tests
 =============
diff --git a/doc/source/topics/angularjs.rst b/doc/source/topics/angularjs.rst
index cadc8e02c..939370449 100644
--- a/doc/source/topics/angularjs.rst
+++ b/doc/source/topics/angularjs.rst
@@ -38,7 +38,7 @@ ESLint
 
 ESLint is a tool for identifying and reporting on patterns in your JS code, and
 is part of the automated tests run by Jenkins. You can run ESLint from the
-horizon root directory with ``npm run lint``, or alternatively on a specific
+horizon root directory with ``tox -e npm -- lint``, or alternatively on a specific
 directory or file with ``eslint file.js``.
 
 Horizon includes a `.eslintrc` in its root directory, that is used by the
@@ -217,10 +217,13 @@ Testing
 =======
 
 1. Open <dev_server_ip:port>/jasmine in a browser. The development server can be run
-   with``./run_tests.sh --runserver`` from the horizon root directory.
-2. ``npm run test`` from the horizon root directory.
+   with ``tox -e runserver`` from the horizon root directory; by default, this will
+   run the development server at ``http://localhost:8000``.
+2. ``tox -e npm`` from the horizon root directory.
 
-The code linting job can be run with ``npm run lint``.
+The code linting job can be run with ``tox -e npm -- lint``. If there are many
+warnings, you can also use ``tox -e npm -- lintq`` to see only errors and
+ignore warnings.
 
 For more detailed information, see :doc:`javascript_testing`.
 
diff --git a/doc/source/topics/customizing.rst b/doc/source/topics/customizing.rst
index 4c5d0ef61..9115b0e7e 100644
--- a/doc/source/topics/customizing.rst
+++ b/doc/source/topics/customizing.rst
@@ -68,7 +68,7 @@ theme's ``_variables.scss``::
    @import "/themes/default/variables";
 
 Once you have made your changes you must re-generate the static files with
- ``./run_tests.sh -m collectstatic``.
+ ``tox -e manage -- collectstatic``.
 
 By default, all of the themes configured by ``AVAILABLE_THEMES`` setting are
 collected by horizon during the `collectstatic` process. By default, the themes
diff --git a/doc/source/topics/install.rst b/doc/source/topics/install.rst
index 129791943..91104afe6 100644
--- a/doc/source/topics/install.rst
+++ b/doc/source/topics/install.rst
@@ -39,7 +39,7 @@ Installation
    message catalogs::
 
     $ sudo apt-get install gettext
-    $ ./run_tests.sh --compilemessages
+    $ tox -e manage -- compilemessages
 
    This command compiles translation message catalogs within Python
    virtualenv named ``.venv``. After this step, you can remove
diff --git a/doc/source/topics/javascript_testing.rst b/doc/source/topics/javascript_testing.rst
index d0d3d0855..1fd702d34 100644
--- a/doc/source/topics/javascript_testing.rst
+++ b/doc/source/topics/javascript_testing.rst
@@ -31,12 +31,13 @@ Running Tests
 Tests can be run in two ways:
 
   1. Open <dev_server_ip:port>/jasmine in a browser. The development server can be
-     run with ``./run_tests.sh --runserver`` from the horizon root directory.
-  2. ``npm run test`` from the horizon root directory. This runs Karma,
+     run with ``tox -e runserver`` from the horizon root directory.
+  2. ``tox -e npm`` from the horizon root directory. This runs Karma,
      so it will run all the tests against PhantomJS and generate coverage
      reports.
 
-The code linting job can be run with ``npm run lint``.
+The code linting job can be run with ``tox -e npm -- lint``, or
+``tox -e npm -- lintq`` to show errors, but not warnings.
 
 Coverage Reports
 ----------------
@@ -45,7 +46,7 @@ Our Karma setup includes a plugin to generate test coverage reports. When
 developing, be sure to check the coverage reports on the master branch and
 compare your development branch; this will help identify missing tests.
 
-To generate coverage reports, run ``npm run test``. The coverage reports can be
+To generate coverage reports, run ``tox -e npm``. The coverage reports can be
 found at ``horizon/coverage-karma/`` (framework tests) and
 ``openstack_dashboard/coverage-karma/`` (dashboard tests). Load
 ``<browser>/index.html`` in a browser to view the reports.
diff --git a/doc/source/topics/translation.rst b/doc/source/topics/translation.rst
index 82da561ee..a3207d094 100644
--- a/doc/source/topics/translation.rst
+++ b/doc/source/topics/translation.rst
@@ -46,13 +46,19 @@ translated. Lets break this up into steps we can follow:
    to locate them. Refer to the guide below on how to use translation and what
    these markers look like.
 
-2. Once marked, we can then run ``./run_tests.sh --makemessages``, which
+2. Once marked, we can then run ``tox -e manage -- extract_messages``, which
    searches the codebase for these markers and extracts them into a Portable
    Object Template (POT) file. In horizon, we extract from both the ``horizon``
    folder and the ``openstack_dashboard`` folder. We use the AngularJS extractor
    for JavaScript and HTML files and the Django extractor for Python and Django
    templates; both extractors are Babel plugins.
 
+3. To update the .po files, you can run ``tox -e manage -- update_catalog`` to
+   update the .po file for every language, or you can specify a specific
+   language to update like this: ``tox -e manage -- update_catalog de``. This
+   is useful if you want to add a few extra translatabale strings for a
+   downstream customisation.
+
 .. Note ::
 
   When pushing code upstream, the only requirement is to mark the strings
@@ -242,12 +248,12 @@ translations to validate that your code is ready for translation.
 Running the pseudo translation tool
 -----------------------------------
 
-#. Make sure your English po file is up to date:
-   ``./run_tests.sh --makemessages``
+#. Make sure your .pot files are up to date:
+   ``tox -e manage -- extract_messages``
 #. Run the pseudo tool to create pseudo translations. For example, to replace
    the German translation with a pseudo translation:
-   ``./run_tests.sh --pseudo de``
-#. Compile the catalog: ``./run_tests.sh --compilemessages``
+   ``tox -e manage -- update_catalog de --pseudo``
+#. Compile the catalog: ``tox -e manage -- compilemessages``
 #. Run your development server.
 #. Log in and change to the language you pseudo translated.
 
diff --git a/doc/source/tutorials/dashboard.rst b/doc/source/tutorials/dashboard.rst
index 96071ae5f..412f85de7 100644
--- a/doc/source/tutorials/dashboard.rst
+++ b/doc/source/tutorials/dashboard.rst
@@ -30,20 +30,19 @@ The quick version
 -----------------
 
 Horizon provides a custom management command to create a typical base
-dashboard structure for you. Run the following commands at the same location
-where the ``run_tests.sh`` file resides. It generates most of the boilerplate
-code you need::
+dashboard structure for you. Run the following commands in your Horizon root
+directory. It generates most of the boilerplate code you need::
 
-    mkdir openstack_dashboard/dashboards/mydashboard
+    $ mkdir openstack_dashboard/dashboards/mydashboard
 
-    ./run_tests.sh -m startdash mydashboard \
-                  --target openstack_dashboard/dashboards/mydashboard
+    $ tox -e manage -- startdash mydashboard \
+      --target openstack_dashboard/dashboards/mydashboard
 
-    mkdir openstack_dashboard/dashboards/mydashboard/mypanel
+    $ mkdir openstack_dashboard/dashboards/mydashboard/mypanel
 
-    ./run_tests.sh -m startpanel mypanel \
-                   --dashboard=openstack_dashboard.dashboards.mydashboard \
-                   --target=openstack_dashboard/dashboards/mydashboard/mypanel
+    $ tox -e manage -- startpanel mypanel \
+      --dashboard=openstack_dashboard.dashboards.mydashboard \
+      --target=openstack_dashboard/dashboards/mydashboard/mypanel
 
 
 You will notice that the directory ``mydashboard`` gets automatically
@@ -562,10 +561,9 @@ Run and check the dashboard
 
 Everything is in place, now run ``Horizon`` on the different port::
 
-    ./run_tests.sh --runserver 0.0.0.0:8877
+    $ tox -e runserver -- 0:9000
 
-
-Go to ``http://<your server>:8877`` using a browser. After login as an admin
+Go to ``http://<your server>:9000`` using a browser. After login as an admin
 you should be able see ``My Dashboard`` shows up at the left side on horizon.
 Click it, ``My Group`` will expand with ``My Panel``. Click on ``My Panel``,
 the right side panel will display an ``Instances Tab`` which has an
diff --git a/doc/source/tutorials/table_actions.rst b/doc/source/tutorials/table_actions.rst
index 9cbb69a64..b02d74bd5 100644
--- a/doc/source/tutorials/table_actions.rst
+++ b/doc/source/tutorials/table_actions.rst
@@ -274,10 +274,10 @@ Run and check the dashboard
 
 We must once again run horizon to verify our dashboard is working::
 
-    ./run_tests.sh --runserver 0.0.0.0:8877
+    $ tox -e runserver -- 0:9000
 
 
-Go to ``http://<your server>:8877`` using a browser. After login as an admin,
+Go to ``http://<your server>:9000`` using a browser. After login as an admin,
 display ``My Panel`` to see the ``Instances`` table. For every ``ACTIVE``
 instance in the table, there will be a ``Create Snapshot`` action on the row.
 Click on ``Create Snapshot``, enter a snapshot name in the form that is shown,
diff --git a/horizon/karma.conf.js b/horizon/karma.conf.js
index b8eeed9bc..e3cc7d3ba 100644
--- a/horizon/karma.conf.js
+++ b/horizon/karma.conf.js
@@ -20,23 +20,14 @@ var fs = require('fs');
 var path = require('path');
 
 module.exports = function (config) {
-  var xstaticPath;
-  var basePaths = [
-    './.venv',
-    './.tox/py27'
-  ];
-
-  for (var i = 0; i < basePaths.length; i++) {
-    var basePath = path.resolve(basePaths[i]);
-
-    if (fs.existsSync(basePath)) {
-      xstaticPath = basePath + '/lib/python2.7/site-packages/xstatic/pkg/';
-      break;
-    }
+  var xstaticPath = path.resolve('./.tox/npm');
+
+  if (fs.existsSync(xstaticPath)) {
+    xstaticPath += '/lib/python2.7/site-packages/xstatic/pkg/';
   }
 
   if (!xstaticPath) {
-    console.error('xStatic libraries not found, please set up venv');
+    console.error('xStatic libraries not found, please run `tox -e npm`');
     process.exit(1);
   }
 
diff --git a/openstack_dashboard/karma.conf.js b/openstack_dashboard/karma.conf.js
index 0b91931f0..e0acf2c52 100644
--- a/openstack_dashboard/karma.conf.js
+++ b/openstack_dashboard/karma.conf.js
@@ -20,23 +20,14 @@ var fs = require('fs');
 var path = require('path');
 
 module.exports = function (config) {
-  var xstaticPath;
-  var basePaths = [
-    './.venv',
-    './.tox/py27'
-  ];
-
-  for (var i = 0; i < basePaths.length; i++) {
-    var basePath = path.resolve(basePaths[i]);
-
-    if (fs.existsSync(basePath)) {
-      xstaticPath = basePath + '/lib/python2.7/site-packages/xstatic/pkg/';
-      break;
-    }
+  var xstaticPath = path.resolve('./.tox/npm');
+
+  if (fs.existsSync(xstaticPath)) {
+    xstaticPath += '/lib/python2.7/site-packages/xstatic/pkg/';
   }
 
   if (!xstaticPath) {
-    console.error('xStatic libraries not found, please set up venv');
+    console.error('xStatic libraries not found, please run `tox -e npm`');
     process.exit(1);
   }
 
diff --git a/openstack_dashboard/management/commands/extract_messages.py b/openstack_dashboard/management/commands/extract_messages.py
new file mode 100644
index 000000000..f015f26b3
--- /dev/null
+++ b/openstack_dashboard/management/commands/extract_messages.py
@@ -0,0 +1,57 @@
+# Copyright 2016 Cisco Systems, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from distutils.dist import Distribution
+import os
+from subprocess import call
+
+from django.core.management.base import BaseCommand
+
+
+class Command(BaseCommand):
+    help = ('Extract strings that have been marked for translation into .POT '
+            'files.')
+
+    def add_arguments(self, parser):
+        parser.add_argument('-m', '--module', type=str, nargs='+',
+                            default=['openstack_dashboard', 'horizon'],
+                            help=("The target python module(s) to extract "
+                                  "strings from"))
+        parser.add_argument('-d', '--domain', choices=['django', 'djangojs'],
+                            nargs='+', default=['django', 'djangojs'],
+                            help="Domain(s) of the .pot file")
+        parser.add_argument('--check-only', action='store_true',
+                            help=("Checks that extraction works correctly, "
+                                  "then deletes the .pot file to avoid "
+                                  "polluting the source code"))
+
+    def handle(self, *args, **options):
+        cmd = ('python setup.py extract_messages -F babel-{domain}.cfg '
+               '-o {module}/locale/{domain}.pot')
+        distribution = Distribution()
+        distribution.parse_config_files(distribution.find_config_files())
+
+        if options['check_only']:
+            cmd += " ; rm {module}/locale/{domain}.pot"
+
+        for module in options['module']:
+            for domain in options['domain']:
+                potfile = '{module}/locale/{domain}.pot'.format(module=module,
+                                                                domain=domain)
+                if not os.path.exists(potfile):
+                    with open(potfile, 'wb') as f:
+                        f.write(b'')
+
+                call(cmd.format(module=module, domain=domain, potfile=potfile),
+                     shell=True)
diff --git a/openstack_dashboard/management/commands/update_catalog.py b/openstack_dashboard/management/commands/update_catalog.py
new file mode 100644
index 000000000..522dd7f28
--- /dev/null
+++ b/openstack_dashboard/management/commands/update_catalog.py
@@ -0,0 +1,122 @@
+# coding: utf-8
+
+# Copyright 2016 Cisco Systems, Inc.
+# Copyright 2015 IBM Corp.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import babel.messages.catalog as catalog
+import os
+from subprocess import call
+
+from django.conf import settings
+from django.core.management.base import BaseCommand
+from django.utils import translation
+
+LANGUAGE_CODES = [language[0] for language in settings.LANGUAGES
+                  if language[0] != 'en']
+POTFILE = "{module}/locale/{domain}.pot"
+POFILE = "{module}/locale/{locale}/LC_MESSAGES/{domain}.po"
+
+
+def translate(segment):
+    prefix = u""
+    # When the id starts with a newline the mo compiler enforces that
+    # the translated message must also start with a newline.  Make
+    # sure that doesn't get broken when prepending the bracket.
+    if segment.startswith('\n'):
+        prefix = u"\n"
+    orig_size = len(segment)
+    # Add extra expansion space based on recommendation from
+    # http://www-01.ibm.com/software/globalization/guidelines/a3.html
+    if orig_size < 20:
+        multiplier = 1
+    elif orig_size < 30:
+        multiplier = 0.8
+    elif orig_size < 50:
+        multiplier = 0.6
+    elif orig_size < 70:
+        multiplier = 0.4
+    else:
+        multiplier = 0.3
+    extra_length = int(max(0, (orig_size * multiplier) - 10))
+    extra_chars = "~" * extra_length
+    return u"{0}[~{1}~您好яшçあ{2}]".format(prefix, segment, extra_chars)
+
+
+class Command(BaseCommand):
+    help = 'Update a translation catalog for a specified language'
+
+    def add_arguments(self, parser):
+        parser.add_argument('-l', '--language', choices=LANGUAGE_CODES,
+                            default=LANGUAGE_CODES, nargs='+',
+                            help=("The language code(s) to pseudo translate"))
+        parser.add_argument('-m', '--module', type=str, nargs='+',
+                            default=['openstack_dashboard', 'horizon'],
+                            help=("The target python module(s) to extract "
+                                  "strings from"))
+        parser.add_argument('-d', '--domain', choices=['django', 'djangojs'],
+                            nargs='+', default=['django', 'djangojs'],
+                            help="Domain(s) of the .POT file")
+        parser.add_argument('--pseudo', action='store_true',
+                            help=("Creates a pseudo translation for the "
+                                  "specified locale, to check for "
+                                  "translatable string coverage"))
+
+    def handle(self, *args, **options):
+        for module in options['module']:
+            for domain in options['domain']:
+                potfile = POTFILE.format(module=module, domain=domain)
+
+                for language in options['language']:
+                    # Get the locale code for the language code given and
+                    # work around broken django conversion function
+                    locales = {'ko': 'ko_KR', 'pl': 'pl_PL', 'tr': 'tr_TR'}
+                    locale = locales.get(language,
+                                         translation.to_locale(language))
+                    pofile = POFILE.format(module=module, locale=locale,
+                                           domain=domain)
+
+                    # If this isn't a pseudo translation, execute pybabel
+                    if not options['pseudo']:
+                        if not os.path.exists(pofile):
+                            with open(pofile, 'wb') as fobj:
+                                fobj.write(b'')
+
+                        cmd = ('pybabel update -l {locale} -i {potfile} '
+                               '-o {pofile}').format(locale=locale,
+                                                     potfile=potfile,
+                                                     pofile=pofile)
+                        call(cmd, shell=True)
+                        continue
+
+                    # Pseudo translation logic
+                    with open(potfile, 'r') as f:
+                        pot_cat = pofile.read_po(f, ignore_obsolete=True)
+
+                    new_cat = catalog.Catalog(locale=locale,
+                                              last_translator="pseudo.py",
+                                              charset="utf-8")
+                    num_plurals = new_cat.num_plurals
+
+                    for msg in pot_cat:
+                        if msg.pluralizable:
+                            msg.string = [
+                                translate(u"{}:{}".format(i, msg.id[0]))
+                                for i in range(num_plurals)]
+                        else:
+                            msg.string = translate(msg.id)
+                        new_cat[msg.id] = msg
+
+                    with open(pofile, 'w') as f:
+                        pofile.write_po(f, new_cat, ignore_obsolete=True)
diff --git a/package.json b/package.json
index 495f8766c..135295ba8 100644
--- a/package.json
+++ b/package.json
@@ -19,7 +19,7 @@
     "karma-threshold-reporter": "0.1.15"
   },
   "scripts": {
-    "postinstall": "if [ ! -d .venv ]; then tox -epy27 --notest; fi",
+    "postinstall": "if [ ! -d .tox/npm ]; then tox -e npm --notest; fi",
     "test": "karma start horizon/karma.conf.js --single-run && karma start openstack_dashboard/karma.conf.js --single-run",
     "lint": "eslint --no-color openstack_dashboard/static horizon/static openstack_dashboard/dashboards/*/static",
     "lintq": "eslint --quiet openstack_dashboard/static horizon/static openstack_dashboard/dashboards/*/static"
diff --git a/releasenotes/notes/bp/enhance-tox-26f73a048b88df2f.yaml b/releasenotes/notes/bp/enhance-tox-26f73a048b88df2f.yaml
new file mode 100644
index 000000000..9d9f0eff0
--- /dev/null
+++ b/releasenotes/notes/bp/enhance-tox-26f73a048b88df2f.yaml
@@ -0,0 +1,11 @@
+---
+features:
+  - The hard-coded run_tests commands for extracting translatable strings and
+    updating message catalogs have been ported to django management commands
+    as extract_messages and update_catalog. These accept several parameters
+    to make them easier to use with downstream customisations and string
+    modifications, but the default behaviour is the same as before.
+deprecations:
+  - The run_tests.sh script is now deprecated and all functionality has
+    been provided by either tox or manage.py. run_tests will be removed
+    in Queens (13.0).
diff --git a/tools/pseudo.py b/tools/pseudo.py
index fbb14cb76..4a5ef1a0d 100755
--- a/tools/pseudo.py
+++ b/tools/pseudo.py
@@ -19,6 +19,10 @@ import argparse
 import babel.messages.catalog as catalog
 import babel.messages.pofile as pofile
 
+# NOTE: This implementation has been superseded by the pseudo_translate
+# management command, and will be removed in Queens (13.0) when run_tests.sh
+# is also removed.
+
 
 def translate(segment):
     prefix = u""
diff --git a/tools/unit_tests.sh b/tools/unit_tests.sh
new file mode 100755
index 000000000..edff8acfc
--- /dev/null
+++ b/tools/unit_tests.sh
@@ -0,0 +1,30 @@
+# Uses envpython and toxinidir from tox run to construct a test command
+testcommand="${1} ${2}/manage.py test"
+posargs="${@:3}"
+
+# Attempt to identify if any of the arguments passed from tox is a test subset
+if [ -n "$posargs" ]; then
+  for arg in "$posargs"
+  do
+    if [ ${arg:0:1} != "-" ]; then
+      subset=$arg
+    fi
+  done
+fi
+
+# If we are running a test subset, supply the correct settings file.
+# If not, simply run the entire test suite.
+if [ -n "$subset" ]; then
+  project="${subset%%.*}"
+
+  if [ $project == "horizon" ]; then
+    $testcommand --settings=horizon.test.settings $posargs
+  elif [ $project == "openstack_dashboard" ]; then
+    $testcommand --settings=openstack_dashboard.test.settings \
+    --exclude-dir=openstack_dashboard/test/integration_tests $posargs
+  fi
+else
+  $testcommand horizon --settings=horizon.test.settings $posargs
+  $testcommand openstack_dashboard --settings=openstack_dashboard.test.settings \
+  --exclude-dir=openstack_dashboard/test/integration_tests $posargs
+fi
diff --git a/tox.ini b/tox.ini
index 72405b110..3b30f3cd5 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,104 +1,63 @@
 [tox]
-envlist = pep8,py27dj18,py27,py34,py35,releasenotes
-minversion = 1.6
+envlist = pep8,py27dj{18,19,110},py34,py35,releasenotes,npm
+minversion = 2.3.2
 skipsdist = True
 
 [testenv]
-basepython=python2.7
 install_command = pip install -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt} {opts} {packages}
 setenv =
   VIRTUAL_ENV={envdir}
   INTEGRATION_TESTS=0
-  SELENIUM_HEADLESS=0
-  SELENIUM_PHANTOMJS=0
   NOSE_WITH_OPENSTACK=1
   NOSE_OPENSTACK_SHOW_ELAPSED=1
+whitelist_externals =
+  bash
 deps =
   -r{toxinidir}/requirements.txt
   -r{toxinidir}/test-requirements.txt
-whitelist_externals =
-  bash
-commands =
-  # Try to detect whether a limited test suite is being specified and if so
-  # direct the testing to that suite's project; otherwise run the full suite
-  # in both horizon and openstack_dashboard "projects".
-  bash -c 'project=`echo {posargs} | cut -d. -f1`; \
-           if [ -z "$project" ]; then \
-             EXIT_STATUS=0; \
-             {envpython} {toxinidir}/manage.py test horizon --settings=horizon.test.settings {posargs} || EXIT_STATUS=$?; \
-             {envpython} {toxinidir}/manage.py test openstack_dashboard --settings=openstack_dashboard.test.settings --exclude-dir=openstack_dashboard/test/integration_tests {posargs} || EXIT_STATUS=$?; \
-             exit $EXIT_STATUS; \
-           else \
-             {envpython} {toxinidir}/manage.py test {posargs} --settings=$project.test.settings --exclude-dir=openstack_dashboard/test/integration_tests; \
-           fi'
-
-# Django-1.8 is LTS
-[testenv:py27dj18]
 commands =
-  pip install django>=1.8,<1.9
-  {envpython} {toxinidir}/manage.py test horizon --settings=horizon.test.settings {posargs}
-  {envpython} {toxinidir}/manage.py test openstack_dashboard --settings=openstack_dashboard.test.settings --exclude-dir=openstack_dashboard/test/integration_tests {posargs}
+  docs: sphinx-build -W -b html doc/source doc/build/html
+  horizon: {envpython} {toxinidir}/manage.py test --settings=horizon.test.settings {posargs}
+  manage: {envpython} {toxinidir}/manage.py {posargs}
+  py27: {[unit_tests]commands}
+  py27dj18: {[unit_tests]commands}
+  py34: {[unit_tests]commands}
+  py35: {[unit_tests]commands}
+  openstack_dashboard: {envpython} {toxinidir}/manage.py test --settings=openstack_dashboard.test.settings {posargs}
+  releasenotes: sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html
+  runserver: {envpython} {toxinidir}/manage.py runserver {posargs}
+  venv: {posargs}
 
 [testenv:py34]
-basepython = python3.4
 setenv =
+  PYTHONUNBUFFERED = 1
   {[testenv]setenv}
-  PYTHONUNBUFFERED=1
+commands = {[unit_tests]commands}
 
 [testenv:py35]
-basepython = python3.5
 setenv =
+  PYTHONUNBUFFERED = 1
   {[testenv]setenv}
-  PYTHONUNBUFFERED=1
+commands = {[unit_tests]commands}
 
-[testenv:pep8]
-usedevelop = True
-whitelist_externals =
-  git
-  rm
-setenv =
-  {[testenv]setenv}
-  DJANGO_SETTINGS_MODULE=openstack_dashboard.test.settings
+[testenv:py27dj19]
 commands =
-  {[testenv:extractmessages_check]commands}
-  {[testenv:docs]commands}
-  flake8
+  pip install -U django>=1.9,<1.10
+  {[unit_tests]commands}
 
-[testenv:extractmessages]
-usedevelop = True
-setenv =
-  {[testenv]setenv}
+[testenv:py27dj110]
 commands =
- {envpython} {toxinidir}/setup.py extract_messages -F babel-django.cfg -o horizon/locale/django.pot --input-dirs horizon/
- {envpython} {toxinidir}/setup.py extract_messages -F babel-djangojs.cfg -o horizon/locale/djangojs.pot --input-dirs horizon/
- {envpython} {toxinidir}/setup.py extract_messages -F babel-django.cfg -o openstack_dashboard/locale/django.pot --input-dirs openstack_dashboard/
- {envpython} {toxinidir}/setup.py extract_messages -F babel-djangojs.cfg -o openstack_dashboard/locale/djangojs.pot --input-dirs openstack_dashboard/
+  pip install -U django>=1.10,<1.11
+  {[unit_tests]commands}
+
+[unit_tests]
+commands = bash {toxinidir}/tools/unit_tests.sh {envpython} {toxinidir} {posargs}
 
-[testenv:extractmessages_check]
-# Only checks to see if translation files can be extracted and cleans afterwards
+[testenv:pep8]
 usedevelop = True
-whitelist_externals =
-  rm
-setenv =
-  {[testenv]setenv}
-commands =
-  {[testenv:extractmessages]commands}
-  rm horizon/locale/django.pot
-  rm horizon/locale/djangojs.pot
-  rm openstack_dashboard/locale/django.pot
-  rm openstack_dashboard/locale/djangojs.pot
-
-[testenv:compilemessages]
-usedevelop = False
 commands =
- /bin/bash {toxinidir}/run_tests.sh --compilemessages -N
-
-[testenv:venv]
-commands = {posargs}
-
-[testenv:manage]
-# Env to launch manage.py commands
-commands = {envpython} {toxinidir}/manage.py {posargs}
+  {envpython} {toxinidir}/manage.py extract_messages --check-only
+  flake8
 
 [testenv:cover]
 commands =
@@ -108,13 +67,28 @@ commands =
   coverage xml
   coverage html
 
-[testenv:py27dj19]
-commands = pip install django>=1.9,<1.10
-           /bin/bash run_tests.sh -N --no-pep8 {posargs}
+[testenv:selenium]
+setenv =
+  {[testenv]setenv}
+  WITH_SELENIUM=1
+  SKIP_UNITTESTS=1
+commands = {[unit_tests]commands}
 
-[testenv:py27dj110]
-commands = pip install django>=1.10,<1.11
-           /bin/bash run_tests.sh -N --no-pep8 {posargs}
+[testenv:selenium-headless]
+setenv =
+  {[testenv]setenv}
+  SELENIUM_HEADLESS=1
+  WITH_SELENIUM=1
+  SKIP_UNITTESTS=1
+commands = {[unit_tests]commands}
+
+[testenv:selenium-phantomjs]
+setenv =
+  {[testenv]setenv}
+  SELENIUM_PHANTOMJS=1
+  WITH_SELENIUM=1
+  SKIP_UNITTESTS=1
+commands = {[unit_tests]commands}
 
 [testenv:py27integration]
 # Run integration tests only
@@ -134,16 +108,6 @@ commands =
   npm install
   npm run {posargs:test}
 
-[testenv:docs]
-setenv = DJANGO_SETTINGS_MODULE=openstack_dashboard.test.settings
-commands = sphinx-build -W -b html doc/source doc/build/html
-
-[testenv:releasenotes]
-commands = sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html
-
-[testenv:runserver]
-commands = {envpython} {toxinidir}/manage.py runserver {posargs}
-
 [testenv:tests_system_packages]
 # Provide an environment for system packagers that dont want anything from pip
 # Any extra deps needed for this env can be passed by setting TOX_EXTRA_DEPS
@@ -153,8 +117,7 @@ passenv = TOX_EXTRA_DEPS
 deps =
 commands =
   pip install -U {env:TOX_EXTRA_DEPS:}
-  {envpython} {toxinidir}/manage.py test horizon --settings=horizon.test.settings {posargs}
-  {envpython} {toxinidir}/manage.py test openstack_dashboard --settings=openstack_dashboard.test.settings {posargs}
+  {[unit_tests]commands}
 
 [flake8]
 exclude = .venv,.git,.tox,dist,*lib/python*,*egg,build,panel_template,dash_template,local_settings.py,*/local/*,*/test/test_plugins/*,.ropeproject,node_modules