Add OpenBSD and FreeBSD CI to Travis

This commit adds support to test all libc definitions on both OpenBSD and
FreeBSD via QEMU userspace emulation. Specially prepared images for each OS are
used which are essentially intended to run a script on startup and then exit.
Documentation has been added to the `ci/README.md` file describing this new
system.

5 files changed, 274 insertions, 59 deletions

diff --git a/ci/README.md b/ci/README.md
index 5b4c681c25..13c7c8da52 100644
--- a/ci/README.md
+++ b/ci/README.md
@@ -8,11 +8,6 @@ this project.
 
 First up, let's talk about the files in this directory:
 
-* `msys2.ps1` - a PowerShell script which is used to install MSYS2 on the
-  AppVeyor bots. As of this writing MSYS2 isn't installed by default, and this
-  script will install the right version/arch of msys2 in preparation of using
-  the contained C compiler to compile C shims.
-
 * `run-travis.sh` - a shell script run by all Travis builders, this is
   responsible for setting up the rest of the environment such as installing new
   packages, downloading Rust target libraries, etc.
@@ -30,6 +25,11 @@ First up, let's talk about the files in this directory:
 * `landing-page-*.html` - used by `dox.sh` to generate a landing page for all
   architectures' documentation.
 
+* `run-qemu.sh` - see discussion about QEMU below
+
+* `mips`, `rumprun` - instructions to build the docker image for each respective
+  CI target
+
 # CI Systems
 
 Currently this repository leverages a combination of Travis CI and AppVeyor for
@@ -38,11 +38,14 @@ running tests. The triples tested are:
 * AppVeyor
   * `{i686,x86_64}-pc-windows-{msvc,gnu}`
 * Travis
-  *  `{i686,x86_64,mips,aarch64}-unknown-linux-gnu`
-  *  `x86_64-unknown-linux-musl`
-  *  `arm-unknown-linux-gnueabihf`
-  *  `arm-linux-androideabi`
-  *  `{i686,x86_64}-apple-{darwin,ios}`
+  * `{i686,x86_64,mips,aarch64}-unknown-linux-gnu`
+  * `x86_64-unknown-linux-musl`
+  * `arm-unknown-linux-gnueabihf`
+  * `arm-linux-androideabi`
+  * `{i686,x86_64}-apple-{darwin,ios}`
+  * `x86_64-rumprun-netbsd`
+  * `x86_64-unknown-freebsd`
+  * `x86_64-unknown-openbsd`
 
 The Windows triples are all pretty standard, they just set up their environment
 then run tests, no need for downloading any extra target libs (we just download
@@ -54,15 +57,147 @@ The remaining architectures look like:
 
 * Android runs in a [docker image][android-docker] with an emulator, the NDK,
   and the SDK already set up. The entire build happens within the docker image.
-* The MIPS, ARM, and AArch64 builds all use QEMU to run the generated binary to
-  actually verify the tests pass.
+* The MIPS, ARM, and AArch64 builds all use the QEMU userspace emulator to run
+  the generated binary to actually verify the tests pass.
 * The MUSL build just has to download a MUSL compiler and target libraries and
   then otherwise runs tests normally.
 * iOS builds need an extra linker flag currently, but beyond that they're built
   as standard as everything else.
+* The rumprun target builds an entire kernel from the test suite and then runs
+  it inside QEMU using the serial console to test whether it succeeded or
+  failed.
+* The BSD builds, currently OpenBSD and FreeBSD, use QEMU to boot up a system
+  and compile/run tests. More information on that below.
 
 [android-docker]: https://github.com/rust-lang/rust-buildbot/blob/master/slaves/android/Dockerfile
 
+## QEMU
+
+Lots of the architectures tested here use QEMU in the tests, so it's worth going
+over all the crazy capabilities QEMU has and the various flavors in which we use
+it!
+
+First up, QEMU has userspace emulation where it doesn't boot a full kernel, it
+just runs a binary from another architecture (using the `qemu-<arch>` wrappers).
+We provide it the runtime path for the dynamically loaded system libraries,
+however. This strategy is used for all Linux architectures that aren't intel.
+Note that one downside of this QEMU system is that threads are barely
+implemented, so we're careful to not spawn many threads.
+
+For the rumprun target the only output is a kernel image, so we just use that
+plus the `rumpbake` command to create a full kernel image which is then run from
+within QEMU.
+
+Finally, the fun part, the BSDs. Quite a few hoops are jumped through to get CI
+working for these platforms, but the gist of it looks like:
+
+* Cross compiling from Linux to any of the BSDs seems to be quite non-standard.
+  We may be able to get it working but it might be difficult at that point to
+  ensure that the libc definitions align with what you'd get on the BSD itself.
+  As a result, we try to do compiles within the BSD distro.
+* On Travis we can't run a VM-in-a-VM, so we resort to userspace emulation
+  (QEMU).
+* Unfortunately on Travis we also can't use KVM, so the emulation is super slow.
+
+With all that in mind, the way BSD is tested looks like:
+
+1. Download a pre-prepared image for the OS being tested.
+2. Generate the tests for the OS being tested. This involves running the `ctest`
+   library over libc to generate a Rust file and a C file which will then be
+   compiled into the final test.
+3. Generate a disk image which will later be mounted by the OS being tested.
+   This image is mostly just the libc directory, but some modifications are made
+   to compile the generated files from step 2.
+4. The kernel is booted in QEMU, and it is configured to detect the libc-test
+   image being available, run the test script, and then shut down afterwards.
+5. Look for whether the tests passed in the serial console output of the kernel.
+
+There's some pretty specific instructions for setting up each image (detailed
+below), but the main gist of this is that we must avoid a vanilla `cargo run`
+inside of the `libc-test` directory (which is what it's intended for) because
+that would compile `syntex_syntax`, a large library, with userspace emulation.
+This invariably times out on Travis, so we can't do that.
+
+Once all those hoops are jumped through, however, we can be happy that we're
+testing almost everything!
+
+Below are some details of how to set up the initial OS images which are
+downloaded. Each image must be enabled have input/output over the serial
+console, log in automatically at the serial console, detect if a second drive in
+QEMU is available, and if so mount it, run a script (it'll specifically be
+`run-qemu.sh` in this folder which is copied into the generated image talked
+about above), and then shut down.
+
+### QEMU setup - FreeBSD
+
+1. Download CD installer (most minimal is fine)
+2. `qemu-img create -f qcow2 foo.qcow2 2G`
+3. `qemu -cdrom foo.iso -drive if=virtio,file=foo.qcow2 -net nic,model=virtio -net user`
+4. run installer
+5. `echo 'console="comconsole"' >> /boot/loader.conf`
+6. `echo 'autoboot_delay="0"' >> /boot/loader.conf`
+7. look at /etc/ttys, see what getty argument is for ttyu0
+8. edit /etc/gettytab, look for ttyu0 argument, prepend `:al=root` to line
+   beneath
+
+(note that the current image has a `freebsd` user, but this isn't really
+necessary)
+
+Once that's done, arrange for this script to run at login:
+
+```
+#!/bin/sh
+
+sudo kldload ext2fs
+[ -e /dev/vtbd1 ] || exit 0
+sudo mount -t ext2fs /dev/vtbd1 /mnt
+sh /mnt/run.sh /mnt
+sudo poweroff
+```
+
+Helpful links
+
+* https://en.wikibooks.org/wiki/QEMU/Images
+* https://blog.nekoconeko.nl/blog/2015/06/04/creating-an-openstack-freebsd-image.html
+* https://www.freebsd.org/doc/handbook/serialconsole-setup.html
+
+
+### QEMU setup - OpenBSD
+
+1. Download CD installer
+2. `qemu-img create -f qcow2 foo.qcow2 2G`
+3. `qemu -cdrom foo.iso -drive if=virtio,file=foo.qcow2 -net nic,model=virtio -net user`
+4. run installer
+5. `echo 'set tty com0' >> /etc/boot.conf`
+6. `echo 'boot' >> /etc/boot.conf`
+7. Modify /etc/ttys, change the `tty00` at the end from 'unknown off' to
+   'vt220 on secure'
+8. Modify same line in /etc/ttys to have `"/root/foo.sh"` as the shell
+9. Add this script to `/root/foo.sh`
+
+```
+#!/bin/sh
+exec 1>/dev/tty00
+exec 2>&1
+
+if mount -t ext2fs /dev/sd1c /mnt; then
+  sh /mnt/run.sh /mnt
+  shutdown -ph now
+fi
+
+# limited shell...
+exec /bin/sh < /dev/tty00
+```
+
+10. `chmod +x /root/foo.sh`
+
+Helpful links:
+
+* https://en.wikibooks.org/wiki/QEMU/Images
+* http://www.openbsd.org/faq/faq7.html#SerCon
+
+# Questions?
+
 Hopefully that's at least somewhat of an introduction to everything going on
 here, and feel free to ping @alexcrichton with questions!
 
diff --git a/ci/Vagrantfile b/ci/Vagrantfile
deleted file mode 100644
index 70cfcf3284..0000000000
--- a/ci/Vagrantfile
+++ /dev/null
@@ -1,38 +0,0 @@
-# A vagrant configuration file for running tests on BSD-like machines
-#
-# Note that this was originally intended to later be used to run tests on
-# Travis, but it didn't work out. Regardless this has stuck around! You can run
-# tests in FreeBSD via:
-#
-#     git clone https://github.com/alexcrichton/libc
-#     cd libc/ci
-#     vagrant up freebsd
-#     vagrant ssh freebsd
-#     ...
-#     cd /vagrant/libc-test
-#     cargo run
-#
-# And "that's it"! You look up instructions on Vagrant's website for how to
-# install vagrant.
-
-Vagrant.configure(2) do |config|
-  # For a complete reference, please see the online documentation at
-  # https://docs.vagrantup.com.
-
-  config.vm.synced_folder "..", "/vagrant"
-
-  config.vm.define :freebsd do |bsd|
-    bsd.vm.box = "arkadi/freebsd-10.1-amd64"
-    bsd.vm.provision :shell, inline: 'yes | sudo pkg install rust cargo'
-    bsd.vm.provider "virtualbox" do |vb|
-      vb.memory = "2048"
-    end
-  end
-
-  config.vm.define :openbsd do |bsd|
-    bsd.vm.box = "bodgit/openbsd-5.7-amd64"
-    bsd.vm.provider "virtualbox" do |vb|
-      vb.memory = "2048"
-    end
-  end
-end
diff --git a/ci/run-qemu.sh b/ci/run-qemu.sh
new file mode 100644
index 0000000000..78da64aa8f
--- /dev/null
+++ b/ci/run-qemu.sh
@@ -0,0 +1,38 @@
+# Initial script which is run inside of all qemu images. The first argument to
+# this script (as arranged by the qemu image itself) is the path to where the
+# libc crate is mounted.
+#
+# For qemu images we currently need to install Rust manually as this wasn't done
+# by the initial run-travis.sh script
+#
+# FIXME: feels like run-travis.sh should be responsible for downloading the
+#        compiler.
+
+set -ex
+
+ROOT=$1
+cp -r $ROOT/libc /tmp/libc
+cd /tmp/libc
+
+TARGET=$(cat $ROOT/TARGET)
+
+case $TARGET in
+  *-freebsd)
+    sudo pkg install -y rust cargo
+    ;;
+
+  *-openbsd)
+    pkg_add rust curl gcc-4.8.4p4
+    curl https://static.rust-lang.org/cargo-dist/2015-04-02/cargo-nightly-x86_64-unknown-openbsd.tar.gz | \
+      tar xzf - -C /tmp
+    export PATH=$PATH:/tmp/cargo-nightly-x86_64-unknown-openbsd/cargo/bin
+    export CC=egcc
+    ;;
+
+  *)
+    echo "Unknown target: $TARGET"
+    exit 1
+    ;;
+esac
+
+exec sh ci/run.sh $TARGET
diff --git a/ci/run-travis.sh b/ci/run-travis.sh
index 75ce8ba1d8..4e9d92f60f 100644
--- a/ci/run-travis.sh
+++ b/ci/run-travis.sh
@@ -1,6 +1,8 @@
 # Entry point for all travis builds, this will set up the Travis environment by
 # downloading any dependencies. It will then execute the `run.sh` script to
 # build and execute all tests.
+#
+# For a full description of how all tests are run, see `ci/README.md`
 
 set -ex
 
@@ -26,9 +28,84 @@ install() {
   fi
 }
 
+# If we're going to run tests inside of a qemu image, then we don't need any of
+# the scripts below. Instead, download the image, prepare a filesystem which has
+# the current state of this repository, and then run the image.
+#
+# It's assume that all images, when run with two disks, will run the `run.sh`
+# script from the second which we place inside.
+if [ "$QEMU" != "" ]; then
+  # Acquire QEMU and the base OS image
+  install qemu-kvm
+  tmpdir=/tmp/qemu-img-creation
+  mkdir -p $tmpdir
+  if [ ! -f $tmpdir/$QEMU ]; then
+    curl https://people.mozilla.org/~acrichton/libc-test/qemu/$QEMU.gz | \
+      gunzip -d > $tmpdir/$QEMU
+  fi
+
+  # Generate all.{c,rs} on the host which will be compiled inside QEMU. Do this
+  # here because compiling syntex_syntax in QEMU would time out basically
+  # everywhere.
+  rm -rf $tmpdir/generated
+  mkdir -p $tmpdir/generated
+  CARGO_TARGET_DIR=$tmpdir/generated-build \
+    cargo build --manifest-path libc-test/generate-files/Cargo.toml
+  (cd libc-test && TARGET=$TARGET OUT_DIR=$tmpdir/generated SKIP_COMPILE=1 \
+    $tmpdir/generated-build/debug/generate-files)
+
+  # Create a mount a fresh new filesystem image that we'll later pass to QEMU,
+  # this contains the checkout of libc and will be able to run all tests
+  rm -f $tmpdir/libc-test.img
+  dd if=/dev/null of=$tmpdir/libc-test.img bs=1M seek=5
+  mkfs.ext2 -F $tmpdir/libc-test.img
+  rm -rf $tmpdir/mount
+  mkdir $tmpdir/mount
+  sudo mount -t ext2 -o loop $tmpdir/libc-test.img $tmpdir/mount
+
+  # Copy this folder into the mounted image, the `run.sh` entry point, and
+  # overwrite the standard libc-test Cargo.toml with the overlay one which will
+  # assume the all.{c,rs} test files have already been generated
+  sudo mkdir $tmpdir/mount/libc
+  sudo cp -r * $tmpdir/mount/libc/
+  sudo cp ci/run-qemu.sh $tmpdir/mount/run.sh
+  echo $TARGET | sudo tee -a $tmpdir/mount/TARGET
+  sudo cp $tmpdir/generated/* $tmpdir/mount/libc/libc-test
+  sudo cp libc-test/run-generated-Cargo.toml $tmpdir/mount/libc/libc-test/Cargo.toml
+
+  sudo umount $tmpdir/mount
+
+  # If we can use kvm, prefer that, otherwise just fall back to user-space
+  # emulation.
+  if kvm-ok; then
+    program="sudo kvm"
+  else
+    program=qemu-system-x86_64
+  fi
+
+  # Pass -snapshot to prevent tampering with the disk images, this helps when
+  # running this script in development. The two drives are then passed next,
+  # first is the OS and second is the one we just made. Next the network is
+  # configured to work (I'm not entirely sure how), and then finally we turn off
+  # graphics and redirect the serial console output to out.log.
+  $program \
+    -m 1024 \
+    -snapshot \
+    -drive if=virtio,file=$tmpdir/$QEMU \
+    -drive if=virtio,file=$tmpdir/libc-test.img \
+    -net nic,model=virtio \
+    -net user \
+    -nographic \
+    -vga none 2>&1 | tee out.log
+  exec grep "^PASSED .* tests" out.log
+fi
+
 mkdir -p .cargo
 cp ci/cargo-config .cargo/config
 
+# Next up we need to install the standard library for the version of Rust that
+# we're testing. Get fancy targets from the EXTRA_TARGETS URL and otherwise get
+# all others from the official distribution.
 if [ "$TRAVIS" = "true" ]; then
   case "$TARGET" in
     *-apple-ios | *-rumprun-*)
@@ -53,10 +130,14 @@ if [ "$TRAVIS" = "true" ]; then
   esac
 fi
 
-# Pull a pre-built docker image for testing android, then run tests entirely
-# within that image. Note that this is using the same rustc installation that
-# travis has (sharing it via `-v`) and otherwise the tests run entirely within
-# the container.
+# If we're testing with a docker image, then run tests entirely within that
+# image. Note that this is using the same rustc installation that travis has
+# (sharing it via `-v`) and otherwise the tests run entirely within the
+# container.
+#
+# For the docker build we mount the entire current directory at /checkout, set
+# up some environment variables to let it run, and then run the standard run.sh
+# script.
 if [ "$DOCKER" != "" ]; then
   args=""
 
@@ -81,6 +162,8 @@ if [ "$DOCKER" != "" ]; then
     ci/run.sh $TARGET
 fi
 
+# If we're not running docker or qemu, then we may still need some packages
+# and/or tools with various configurations here and there.
 case "$TARGET" in
   x86_64-unknown-linux-musl)
     install musl-tools
@@ -111,6 +194,7 @@ case "$TARGET" in
 
 esac
 
+# Finally, if we've gotten this far, actually run the tests.
 sh ci/run.sh $TARGET
 
 if [ "$TARGET" = "x86_64-unknown-linux-gnu" ] && \
diff --git a/ci/run.sh b/ci/run.sh
index 14985be912..706bf7a1d1 100644
--- a/ci/run.sh
+++ b/ci/run.sh
@@ -47,11 +47,7 @@ case "$TARGET" in
     grep "^PASSED .* tests" /tmp/out
     ;;
 
-  *-apple-ios)
-    libc-test/target/$TARGET/debug/libc-test
-    ;;
-
   *)
-    cargo run --manifest-path libc-test/Cargo.toml --target $TARGET
+    libc-test/target/$TARGET/debug/libc-test
     ;;
 esac