diff options
author | Robert Collins <robertc@robertcollins.net> | 2012-12-19 15:28:22 +1300 |
---|---|---|
committer | Robert Collins <robertc@robertcollins.net> | 2012-12-19 15:28:22 +1300 |
commit | b4fdb43317ad5611ea3816e94a5c9b1cc838f762 (patch) | |
tree | 6cbc53ed57cd93f4624a0c66f230899d29233a68 /doc | |
parent | 8b20441776e21a089d4d6dade3a8d5f08b108d39 (diff) | |
download | testrepository-git-b4fdb43317ad5611ea3816e94a5c9b1cc838f762.tar.gz |
Document overview.
Diffstat (limited to 'doc')
-rw-r--r-- | doc/MANUAL.txt | 55 |
1 files changed, 45 insertions, 10 deletions
diff --git a/doc/MANUAL.txt b/doc/MANUAL.txt index 11f396b..5cd1fed 100644 --- a/doc/MANUAL.txt +++ b/doc/MANUAL.txt @@ -208,11 +208,44 @@ Remote or isolated test environments A common problem with parallel test running is test runners that use global resources such as well known ports, well known database names or predictable -directories on disk. Test repository has an optional interface where it can ask -for environments to run your tests in, and then enumerate and execute the tests -in that environment for you, allowing you to isolate your concurrent test -processes, or to have testr coordinate tests running on remote machines / in -virtual machines. +directories on disk. + +One way to solve this is to setup isolated environments such as chroots, +containers or even separate machines. Such environments typically require +some coordination when being used to run tests, so testr provides an explicit +model for working with them. + +The model testr has is intended to support both developers working +incrementally on a change and CI systems running tests in a one-off setup, +for both statically and dynamically provisioned environments. + +The process testr follows is: + +# The user should perform any one-time or once-per-session setup. For instance, + checking out source code, creating a template container, sourcing your cloud + credentials. + +# Execute testr run. + +# testr queries for concurrency. + +# testr will make a callout request to provision that many instances. + The provisioning callout needs to synchronise source code and do any other + per-instance setup at this stage. + +# testr will make callouts to execute tests, supplying files that should be + copied into the execution environment. Note that instances may be used for + more than one command execution. + +# testr will callout to dispose of the instances after the test run completes. + +Instances may be expensive to create and dispose of. testr does not perform +any caching, but the callout pattern is intended to facilitate external +caching - the provisioning callout can be used to pull environments out of +a cache, and the dispose to just return it to the cache. + +Configuring environment support +------------------------------- There are three callouts that testrepository depends on - configured in .testr.conf as usual. For instance:: @@ -228,11 +261,13 @@ These should operate as follows: that testr should supply to the dispose and execute commands. There should be no other output on stdout (stderr is entirely up for grabs). An exit code of non-zero will cause testr to consider the command to have failed. A - provisioned instance should be able to execute the list tests and execute - tests commands that testr will run via the instance_execute callout. Its - possible to lazy-provision things if you desire - testr doesn't care - but - to reduce latency we suggest performing any rsync or other code - synchronisation steps during the provision step. + provisioned instance should be able to execute the list tests command and + execute tests commands that testr will run via the instance_execute callout. + Its possible to lazy-provision things if you desire - testr doesn't care - + but to reduce latency we suggest performing any rsync or other code + synchronisation steps during the provision step, as testr may make multiple + calls to one environment, and re-doing costly operations on each command + execution would impair performance. * instance_dispose should take a list of instance ids and get rid of them this might mean putting them back in a pool of instances, or powering them |