tests/integration: Use a minimal custom base sysroot

We have been using the Freedesktop SDK binaries to provide a base
system to run the integration tests. This works OK but it weighs in
at 985MB of content, and there is no simple way to customize it so
that we only download the bits we actually need.

This commit changes the tests to use a custom sysroot based on the
Alpine Linux distribution. The sysroot is 155MB unpacked, and packs down
to a 27MB .tar.xz.

This speeds up the integration tests significantly as we greatly reduce
the amount of network traffic required and the amount of data that gets
copied around when creating the staging area.

1 files changed, 67 insertions, 0 deletions

diff --git a/tests/integration/base/generate-base.sh b/tests/integration/base/generate-base.sh
new file mode 100755
index 000000000..433f36ea5
--- /dev/null
+++ b/tests/integration/base/generate-base.sh
@@ -0,0 +1,67 @@
+#!/bin/sh
+
+# Generate a base sysroot for running the BuildStream integration tests.
+#
+# The sysroot is based off the Alpine Linux distribution. The script downloads
+# a release of Alpine, sets up a cheap consider using `bwrap` and installs the
+# packages that are needed by the integration tests, then outputs a .tar.xz
+# file.
+
+set -eux
+
+ALPINE_ARCH=${ARCH:-x86_64}
+ALPINE_BASE=http://dl-cdn.alpinelinux.org/alpine/v3.7/releases/${ALPINE_ARCH}/alpine-minirootfs-3.7.0-${ALPINE_ARCH}.tar.gz
+
+mkdir root
+
+wget ${ALPINE_BASE} -O alpine-base.tar.gz
+
+tar -x -f ./alpine-base.tar.gz -C ./root --exclude dev/\*
+
+run() {
+    # This turns the unpacked rootfs into a container using Bubblewrap.
+    # The Alpine package manager (apk) calls `chroot` when running package
+    # triggers so we need to enable CAP_SYS_CHROOT. We also have to fake
+    # UID 0 (root) inside the container to avoid permissions errors.
+    bwrap --bind ./root / --dev /dev --proc /proc --tmpfs /tmp \
+          --ro-bind /etc/resolv.conf /etc/resolv.conf \
+          --setenv PATH "/usr/bin:/usr/sbin:/bin:/sbin" \
+          --unshare-user --uid 0 --gid 0 \
+          --cap-add CAP_SYS_CHROOT \
+          /bin/sh -c "$@"
+}
+
+# Enable testing repo for Tiny C Compiler package
+run "echo http://dl-cdn.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories"
+
+# Fetch the list of Alpine packages.
+run "apk update"
+
+# There are various random errors from `apk add` to do with ownership, probably
+# because of our hacked up `bwrap` container. The errors seem harmless so I am
+# just ignoring them.
+set +e
+
+# Install stuff needed by all integration tests that compile C code.
+#
+# Note that we use Tiny C Compiler in preference to GCC. There is a huge
+# size difference -- 600KB for TinyCC vs. 50MB to 100MB for GCC. TinyCC
+# supports most of the ISO C99 standard, but has no C++ support at all.
+run "apk add binutils libc-dev make tcc"
+run "ln -s /usr/bin/tcc /usr/bin/cc"
+
+# Install stuff for tests/integration/autotools
+run "apk add autoconf automake"
+
+# Install stuff for tests/integration/cmake
+run "apk add cmake"
+
+# Install stuff for tests/integration/pip
+run "apk add python3"
+
+set -e
+
+# Cleanup the package cache
+run "rm -R /var/cache/apk"
+
+tar -c -v -J -f integration-tests-base.tar.xz -C root .