[test] Cache last output and compare next time.

Compare the current output against a previous run to determine if there
has been any change since last time, and only run through imagediff if
there has been. For the vector surfaces, we can check the vector output
first and potentially skip the rasterisation. On my machine this reduces
the time for a second run from 6 minutes to 2m30s. As most of the time,
most test output will remain unchanged, so this seems to be a big win. On
unix systems, hard linking is used to reduce the amount of storage space
required - others will see about a three-fold increase in the amount of
disk used.  The directory continues to be a stress test for file selectors.

In order to reduce the changes between runs, the current time is no longer
written to the PNG files (justified by that it only exists as a debugging
aid) and the boilerplate tweaks the PS surface so that the creation date
is fixed. To fully realise the benefits here, we need to strip the
creation time from all the reference images...

The biggest problem with using the caches is that different runs of the
test suite can go through different code paths, introducing potential
Heisenbergs. If you suspect that caching is interfering with the test
results, use 'make -C test clean-caches check'.

1 files changed, 5 insertions, 13 deletions

diff --git a/boilerplate/cairo-boilerplate-pdf.c b/boilerplate/cairo-boilerplate-pdf.c
index b0d957152..fb7d1aa2f 100644
--- a/boilerplate/cairo-boilerplate-pdf.c
+++ b/boilerplate/cairo-boilerplate-pdf.c
@@ -74,6 +74,7 @@ _cairo_boilerplate_pdf_create_surface (const char		 *name,
     ptc->height = height;
 
     xasprintf (&ptc->filename, "%s-out.pdf", name);
+    xunlink (ptc->filename);
 
     surface = cairo_pdf_surface_create (ptc->filename, width, height);
     if (cairo_surface_status (surface))
@@ -107,10 +108,11 @@ _cairo_boilerplate_pdf_create_surface (const char		 *name,
     return surface;
 }
 
-static cairo_status_t
-_cairo_boilerplate_pdf_finish (pdf_target_closure_t	*ptc,
-			       cairo_surface_t		*surface)
+cairo_status_t
+_cairo_boilerplate_pdf_finish_surface (cairo_surface_t		*surface)
 {
+    pdf_target_closure_t *ptc = cairo_surface_get_user_data (surface,
+							     &pdf_closure_key);
     cairo_status_t status;
 
     /* Both surface and ptc->target were originally created at the
@@ -155,13 +157,8 @@ _cairo_boilerplate_pdf_surface_write_to_png (cairo_surface_t *surface, const cha
 {
     pdf_target_closure_t *ptc = cairo_surface_get_user_data (surface, &pdf_closure_key);
     char    command[4096];
-    cairo_status_t status;
     int exitstatus;
 
-    status = _cairo_boilerplate_pdf_finish (ptc, surface);
-    if (status)
-	return status;
-
     sprintf (command, "./pdf2png %s %s 1",
 	     ptc->filename, filename);
 
@@ -181,14 +178,9 @@ _cairo_boilerplate_pdf_convert_to_image (cairo_surface_t *surface)
 {
     pdf_target_closure_t *ptc = cairo_surface_get_user_data (surface,
 							     &pdf_closure_key);
-    cairo_status_t status;
     FILE *file;
     cairo_surface_t *image;
 
-    status = _cairo_boilerplate_pdf_finish (ptc, surface);
-    if (status)
-	return cairo_boilerplate_surface_create_in_error (status);
-
     file = cairo_boilerplate_open_any2ppm (ptc->filename, 1);
     if (file == NULL)
 	return cairo_boilerplate_surface_create_in_error (CAIRO_STATUS_READ_ERROR);