Use yarn and webpack to manage zuul-web javascript

yarn drives package and dependency management. webpack handles
bundling, minification and transpiling down to browser-acceptable
javascript but allows for more modern javascript like import statements.

There are some really neat things in the webpack dev server. CSS
changes, for instance, get applied immediately without a refresh. Other
things, like the jquery plugin do need a refresh, but it's handled just
on a file changing.

As a followup, we can also consider turning the majority of the status page
into a webpack library that other people can depend on as a mechanism
for direct use. Things like that haven't been touched because allowing
folks to poke at the existing known status page without too many changes
using the tools seems like a good way for people to learn/understand the
stack.

Move things so that the built content gets put
into zuul/web/static so that the built-in static serving from zuul-web
will/can serve the files.

Update MANIFEST.in so that if npm run build:dist is run before the
python setup.py sdist, the built html/javascript content will be
included in the source tarball.

Add a pbr hook so that if yarn is installed, javascript content will be
built before the tarball.

Add a zuul job with a success url that contains a source_url
pointing to the live v3 data.

This adds a framework for verifying that we can serve the web app
urls and their dependencies for all of the various ways we want to
support folks hosting zuul-web.

It includes a very simple reverse proxy server for approximating
what we do in openstack to "white label" the Zuul service -- that
is, hide the multitenancy aspect and present the single tenant
at the site root.

We can run similar tests without the proxy to ensure the default,
multi-tenant view works as well.

Add babel transpiling enabling use of ES6 features

ECMAScript6 has a bunch of nice things, like block scoped variables,
const, template strings and classes. Babel is a javascript transpiler
which webpack can use to allow us to write using modern javascript but
the resulting code to still work on older browsers.

Use the babel-plugin-angularjs-annotate so that angular's dependency
injection doesn't get borked by babel's transpiling things (which causes
variables to otherwise be renamed in a way that causes angular to not
find them)

While we're at it, replace our use of var with let (let is the new
block-scoped version of var) and toss in some use of const and template
strings for good measure.

Add StandardJS eslint config for linting

JavaScript Standard Style is a code style similar to pep8/flake8. It's
being added here not because of the pep8 part, but because the pyflakes
equivalent can catch real errors. This uses the babel-eslint parser
since we're using Babel to transpile already.

This auto-formats the existing code with:

  npm run format

Rather than using StandardJS directly through the 'standard' package,
use the standardjs eslint plugin so that we can ignore the camelCase
rule (and any other rule that might emerge in the future)

Many of under_score/camelCase were fixed in a previous version of the patch.
Since the prevailing zuul style is camelCase methods anyway, those fixes
were left. That warning has now been disabled.

Other things, such as == vs. === and ensuring template
strings are in backticks are fixed.

Ignore indentation errors for now - we'll fix them at the end of this
stack and then remove the exclusion.

Add a 'format' npm run target that will run the eslint command with
--fix for ease of fixing reported issues.

Add a 'lint' npm run target and a 'lint' environment that runs with
linting turned to errors. The next patch makes the lint environment more
broadly useful.

When we run lint, also run the BundleAnalyzerPlugin and set the
success-url to the report.

Add an angular controller for status and stream page

Wrap the status and stream page construction with an angular controller
so that all the javascripts can be bundled in a single file.

Building the files locally is wonderful and all, but what we really want
is to make a tarball that has the built code so that it can be deployed.

Put it in the root source dir so that it can be used with the zuul
fetch-javascript-tarball role.

Also, replace the custom npm job with the new build-javascript-content
job which naturally grabs the content we want.

Make a 'main.js' file that imports the other three so that we just have
a single bundle. Then, add a 'vendor' entry in the common webpack file
and use the CommonsChunkPlugin to extract dependencies into their own
bundle. A second CommonsChunkPlugin entry pulls out a little bit of
metadata that would otherwise cause the main and vendor chunks to change
even with no source change. Then add chunkhash into the filename. This
way the files themselves can be aggressively cached.

This all follows recommendations from https://webpack.js.org/guides/caching/
https://webpack.js.org/guides/code-splitting/ and
https://webpack.js.org/guides/output-management/

Change-Id: I2e1230783fe57f1bc3b7818460463df1e659936b
Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com>
Co-Authored-By: James E. Blair <jeblair@redhat.com>

1 files changed, 256 insertions, 0 deletions

diff --git a/tools/install-js-repos-rpm.sh b/tools/install-js-repos-rpm.sh
new file mode 100755
index 000000000..6832c1444
--- /dev/null
+++ b/tools/install-js-repos-rpm.sh
@@ -0,0 +1,256 @@
+#!/bin/bash
+
+# Copyright (c) 2016 NodeSource LLC
+#
+# Permission is hereby granted, free of charge, to any person obtaining a
+# copy of this software and associated documentation files (the "Software"),
+# to deal in the Software without restriction, including without limitation
+# the rights to use, copy, modify, merge, publish, distribute, sublicense,
+# and/or sell copies of the Software, and to permit persons to whom the
+# Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+# The above license is inferred from the
+# https://github.com/nodesource/distributions source repository.
+
+# Discussion, issues and change requests at:
+#   https://github.com/nodesource/distributions
+#
+# Script to install the NodeSource Node.js 8.x repo onto an
+# Enterprise Linux or Fedora Core based system.
+#
+# Run as root or insert `sudo -E` before `bash`:
+#
+# This was downloaded from https://rpm.nodesource.com/setup_8.x
+# A few modifications have been made.
+
+print_status() {
+  local outp=$(echo "$1" | sed -r 's/\\n/\\n## /mg')
+  echo
+  echo -e "## ${outp}"
+  echo
+}
+
+bail() {
+  echo 'Error executing command, exiting'
+  exit 1
+}
+
+exec_cmd_nobail() {
+  echo "+ $1"
+  bash -c "$1"
+}
+
+exec_cmd() {
+  exec_cmd_nobail "$1" || bail
+}
+
+print_status "Installing the NodeSource Node.js 8.x repo..."
+
+print_status "Inspecting system..."
+
+if [ ! -x /bin/rpm ]; then
+  print_status "\
+You don't appear to be running an Enterprise Linux based \
+system, please contact NodeSource at \
+https://github.com/nodesource/distributions/issues if you think this \
+is incorrect or would like your distribution to be considered for \
+support.\
+"
+  exit 1
+fi
+
+## Annotated section for auto extraction in test.sh
+#-check-distro-#
+
+## Check distro and arch
+echo "+ rpm -q --whatprovides redhat-release || rpm -q --whatprovides centos-release || rpm -q --whatprovides cloudlinux-release || rpm -q --whatprovides sl-release"
+DISTRO_PKG=$(rpm -q --whatprovides redhat-release || rpm -q --whatprovides centos-release || rpm -q --whatprovides cloudlinux-release || rpm -q --whatprovides sl-release)
+echo "+ uname -m"
+UNAME_ARCH=$(uname -m)
+
+
+if [ "X${UNAME_ARCH}" == "Xi686" ]; then
+  DIST_ARCH=i386
+elif [ "X${UNAME_ARCH}" == "Xx86_64" ]; then
+  DIST_ARCH=x86_64
+else
+
+  print_status "\
+You don't appear to be running a supported machine architecture: ${UNAME_ARCH}. \
+Please contact NodeSource at \
+https://github.com/nodesource/distributions/issues if you think this is \
+incorrect or would like your architecture to be considered for support. \
+"
+  exit 1
+
+fi
+
+if [[ $DISTRO_PKG =~ ^(redhat|centos|cloudlinux|sl)- ]]; then
+    DIST_TYPE=el
+elif [[ $DISTRO_PKG =~ ^system-release- ]]; then # Amazon Linux
+    DIST_TYPE=el
+elif [[ $DISTRO_PKG =~ ^(fedora|korora)- ]]; then
+    DIST_TYPE=fc
+else
+
+  print_status "\
+You don't appear to be running a supported version of Enterprise Linux. \
+Please contact NodeSource at \
+https://github.com/nodesource/distributions/issues if you think this is \
+incorrect or would like your architecture to be considered for support. \
+Include your 'distribution package' name: ${DISTRO_PKG}. \
+"
+  exit 1
+
+fi
+
+if [[ $DISTRO_PKG =~ ^system-release-201[4-9]\. ]]; then  #NOTE: not really future-proof
+
+  # Amazon Linux, for 2014.* use el7, older versions are unknown, perhaps el6
+  DIST_VERSION=7
+
+else
+
+  ## Using the redhat-release-server-X, centos-release-X, etc. pattern
+  ## extract the major version number of the distro
+  DIST_VERSION=$(echo $DISTRO_PKG | sed -r 's/^[[:alpha:]]+-release(-server|-workstation)?-([0-9]+).*$/\2/')
+
+  if ! [[ $DIST_VERSION =~ ^[0-9][0-9]?$ ]]; then
+
+    print_status "\
+Could not determine your distribution version, you may not be running a \
+supported version of Enterprise Linux. \
+Please contact NodeSource at \
+https://github.com/nodesource/distributions/issues if you think this is \
+incorrect. Include your 'distribution package' name: ${DISTRO_PKG}. \
+"
+    exit 1
+
+  fi
+
+fi
+
+
+## Given the distro, version and arch, construct the url for
+## the appropriate nodesource-release package (it's noarch but
+## we include the arch in the directory tree anyway)
+RELEASE_URL_VERSION_STRING="${DIST_TYPE}${DIST_VERSION}"
+RELEASE_URL="\
+https://rpm.nodesource.com/pub_8.x/\
+${DIST_TYPE}/\
+${DIST_VERSION}/\
+${DIST_ARCH}/\
+nodesource-release-${RELEASE_URL_VERSION_STRING}-1.noarch.rpm"
+
+#-check-distro-#
+
+print_status "Confirming \"${DIST_TYPE}${DIST_VERSION}-${DIST_ARCH}\" is supported..."
+
+## Simple fetch & fast-fail to see if the nodesource-release
+## file exists for this distro/version/arch
+exec_cmd_nobail "curl -sLf -o /dev/null '${RELEASE_URL}'"
+RC=$?
+
+if [[ $RC != 0 ]]; then
+    print_status "\
+Your distribution, identified as \"${DISTRO_PKG}\", \
+is not currently supported, please contact NodeSource at \
+https://github.com/nodesource/distributions/issues \
+if you think this is incorrect or would like your distribution to be considered for support"
+    exit 1
+fi
+
+## EPEL is needed for EL5, we don't install it if it's missing but
+## we can give guidance
+if [ "$DIST_TYPE" == "el" ] && [ "$DIST_VERSION" == "5" ]; then
+
+  print_status "Checking if EPEL is enabled..."
+
+  echo "+ yum repolist enabled 2> /dev/null | grep epel"
+  repolist=$(yum repolist enabled 2> /dev/null | grep epel)
+
+  if [ "X${repolist}" == "X" ]; then
+    print_status "Finding current EPEL release RPM..."
+
+    ## We can scrape the html to find the latest epel-release (likely 5.4)
+    epel_url="http://dl.fedoraproject.org/pub/epel/5/${DIST_ARCH}/"
+    epel_release_view="${epel_url}repoview/epel-release.html"
+    echo "+ curl -s $epel_release_view | grep -oE 'epel-release-[0-9\-]+\.noarch\.rpm'"
+    epel=$(curl -s $epel_release_view | grep -oE 'epel-release-[0-9\-]+\.noarch\.rpm')
+    if [ "X${epel}" = "X" ]; then
+      print_status "Error: Could not find current EPEL release RPM!"
+      exit 1
+    fi
+
+    print_status "\
+The EPEL (Extra Packages for Enterprise Linux) repository is a\n\
+prerequisite for installing Node.js on your operating system. Please\n\
+add it and re-run this setup script.\n\
+\n\
+The EPEL repository RPM is available at:\n\
+  ${epel_url}${epel}\n\
+You can try installing with: \`rpm -ivh <url>\`\
+"
+
+    exit 1
+  fi
+
+fi
+
+print_status "Downloading release setup RPM..."
+
+## Two-step process to install the nodesource-release RPM,
+## Download to a tmp file then install it directly with `rpm`.
+## We don't rely on RPM's ability to fetch from HTTPS directly
+echo "+ mktemp"
+RPM_TMP=$(mktemp || bail)
+
+exec_cmd "curl -sL -o '${RPM_TMP}' '${RELEASE_URL}'"
+
+print_status "Installing release setup RPM..."
+
+## --nosignature because nodesource-release contains the signature!
+exec_cmd "rpm -i --nosignature --force '${RPM_TMP}'"
+
+print_status "Cleaning up..."
+
+exec_cmd "rm -f '${RPM_TMP}'"
+
+print_status "Checking for existing installations..."
+
+## Nasty consequences if you have an existing Node or npm package
+## installed, need to inform if they are there
+echo "+ rpm -qa 'node|npm' | grep -v nodesource"
+EXISTING_NODE=$(rpm -qa 'node|npm|iojs' | grep -v nodesource)
+
+if [ "X${EXISTING_NODE}" != "X" ]; then
+
+  # NOTE(mordred) Removed -y from the yum command below.
+  print_status "\
+Your system appears to already have Node.js installed from an alternative source.\n\
+Run \`\033[1myum remove nodejs npm\033[22m\` (as root) to remove these first.\
+"
+
+fi
+
+# NOTE(mordred) Removed -y from the yum commands below.
+print_status "\
+Run \`\033[1myum install nodejs\033[22m\` (as root) to install Node.js 8.x and npm.\n\
+You may also need development tools to build native addons:\n\
+  \`yum install gcc-c++ make\`\
+"
+
+## Alternative to install dev tools: `yum groupinstall 'Development Tools'
+
+exit 0