testresources: extensions to python unittest to allow declarative use of resources by test cases. Copyright (C) 2005-2008 Robert Collins This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA testresources is attempting to extend unittest with a clean and simple api to provide test optimisation where expensive common resources are needed for test cases - for example sample working trees for VCS systems, reference databases for enterprise applications, or web servers ... let imagination run wild. Dependencies ============ * Python 2.4+ * testtools Note that testtools is required for *running* the tests for testresources. You can use testresources in your own app without using testtools. How testresources Works ======================= The basic idea of testresources is: * Tests declare the resources they need in a ``resources`` attribute. * When the test is run, the required resource objects are allocated (either newly constructed, or reused), and assigned to attributes of the TestCase. testresources distinguishes a 'resource' (a subclass of ``TestResource``) which acts as a kind of factory, and a 'resource' which can be any kind of object returned from the resource class's ``getResource`` method. Resources are either clean or dirty. Being clean means they have same state in all important ways as a newly constructed instance and they can therefore be safely reused. Main Classes ============ testresources.ResourcedTestCase ------------------------------- By extending or mixing-in this class, tests can have necessary resources automatically allocated and disposed or recycled. ResourceTestCase can be used as a base class for tests, and when that is done tests will have their ``resources`` attribute automatically checked for resources by both OptimisingTestSuite and their own setUp() and tearDown() methods. (This allows tests to remain functional without needing this specific TestSuite as a container). Alternatively, you can call ResourceTestCase.setUpResources(self) and ResourceTestCase.tearDownResources(self) from your own classes setUp and tearDown and the same behaviour will be activated. To declare the use of a resource, set the ``resources`` attribute to a list of tuples of ``(attribute_name, resource_factory)``. During setUp, for each declared requriment, the test gains an attribute pointing to an allocated resource, which is the result of calling ``resource_factory.getResource()``. ``finishedWith`` will be called on each resource during tearDown(). For example:: class TestLog(testresources.TestCase): resources = [('branch', BzrPopulatedBranch())] def test_log(self): show_log(self.branch, ...) testresources.TestResource -------------------------- A TestResource is an object that tests can use to create resources. It can be overridden to manage different types of resources. Normally test code doesn't need to call any methods on it, as this will be arranged by the testresources machinery. When implementing a new ``TestResource`` subclass you should consider overriding these methods: ``make`` Must be overridden in every concrete subclass. Returns a new instance of the resource object (the actual resource, not the TestResource). Doesn't need to worry about reuse, which is taken care of separately. This method is only called when a new resource is definitely needed. ``clean`` Cleans up an existing resource instance, eg by deleting a directory or closing a network connection. By default this does nothing, which may be appropriate for resources that are automatically garbage collected. ``reset`` Reset a no-longer-used dirty resource to a clean state. By default this just discards it and creates a new one, but for some resources there may be a faster way to reset them. ``isDirty`` Check whether an existing resource is dirty. By default this just reports whether ``TestResource.dirtied`` has been called. For instance:: class TemporaryDirectoryResource(TestResource): def clean(self, resource): osutils.rmtree(resource) def make(self): return tempfile.mkdtemp() def isDirty(self, resource): # Can't detect when the directory is written to, so assume it # can never be reused. We could list the directory, but that might # not catch it being open as a cwd etc. return True The ``resources`` list on the TestResource object is used to declare dependencies. For instance, a DataBaseResource that needs a TemporaryDirectory might be declared with a resources list:: class DataBaseResource(TestResource): resources = [("scratchdir", TemporaryDirectoryResource())] Most importantly, two getResources to the same TestResource with no finishedWith call in the middle, will return the same object as long as it is not dirty. When a Test has a dependency and that dependency successfully completes but returns None, the framework does *not* consider this an error: be sure to always return a valid resource, or raise an error. Error handling hasn't been heavily exercised, but any bugs in this area will be promptly dealt with. A sample TestResource can be found in the doc/ folder. See pydoc testresources.TestResource for details. testresources.GenericResource ----------------------------- Glue to adapt testresources to an existing resource-like class. testresources.OptimisingTestSuite --------------------------------- This TestSuite will introspect all the test cases it holds directly and if they declare needed resources, will run the tests in an order that attempts to minimise the number of setup and tear downs required. It attempts to achieve this by callling getResource() and finishedWith() around the sequence of tests that use a specific resource. Tests are added to an OptimisingTestSuite as normal. Any standard library TestSuite objects will be flattened, while any custom TestSuite subclasses will be distributed across their member tests. This means that any custom logic in test suites should be preserved, at the price of some level of optimisation. Because the test suite does the optimisation, you can control the amount of optimising that takes place by adding more or fewer tests to a single OptimisingTestSuite. You could add everything to a single OptimisingTestSuite, getting global optimisation or you could use several smaller OptimisingTestSuites. testresources.TestLoader ------------------------ This is a trivial TestLoader that creates OptimisingTestSuites by default. unittest.TestResult ------------------- testresources will log activity about resource creation and destruction to the result object tests are run with. 4 extension methods are looked for: ``startCleanResource``, ``stopCleanResource``, ``startMakeResource``, ``stopMakeResource``. ``testresources.tests.ResultWithResourceExtensions`` is an example of a ``TestResult`` with these methods present. Controlling Resource Reuse ========================== When or how do I mark the resource dirtied? The simplest approach is to have ``TestResource.make`` call ``self.dirtied``: the resource is always immediately dirty and will never be reused without first being reset. This is appropriate when the underlying resource is cheap to reset or recreate, or when it's hard to detect whether it's been dirtied or to trap operations that change it. Alternatively, override ``TestResource.isDirty`` and inspect the resource to see if it is safe to reuse. Finally, you can arrange for the returned resource to always call back to ``TestResource.dirtied`` on the first operation that mutates it. FAQ === * Can I dynamically request resources inside a test method? Generally, no, you shouldn't do this. The idea is that the resources are declared statically, so that testresources can "smooth" resource usage across several tests. * Isn't using the same word 'resource' for the TestResource and the resource confusing? Yes. * If the resource is held inside the TestResource object, and the TestResource is typically constructed inline in the test case ``resources`` attribute, how can they be shared across different test classes? Good question. I guess you should arrange for a single instance to be held in an appropriate module scope, then referenced by the test classes that want to share it.