Fully rework the conformance test suite

The current conformance test suite is suboptimal in many ways.

All tests are built into the same binary, which makes adding new tests,
builting tests, and running groups of tests much more awkward than it
needs to be. The first issue, especially, raises the bar of contribution
in a significant way, while the other two take their toll on the
maintainer. All of these changes were introduced back when we had both
Clutter and Cogl tests in tree, and because we were building the test
suite for every single change; since then, Cogl moved out of tree with
all its tests, and we build the conformance test suite only when running
the `check` make target.

This admittedly large-ish commit changes the way the conformance test
suite works, taking advantage of the changes in the GTest API and test
harness.

First of all, all tests are now built separately, using their own test
suite as defined by each separate file. All tests run under the TAP
harness provided by GTest and Automake, to gather a proper report using
the Test Anything Protocol without using the `gtester` harness and the
`gtester-report` script. We also use the Makefile rules provided by GLib
to vastly simplify the build environment for the conformance test suite.

On top of the changes for the build and harness, we also provide new API
for creating and running test suites for Clutter. The API is public,
because the test suite has to use it, but it's minimal and mostly
provides convenience wrappers around GTest that make writing test units
for Clutter easier.

This commit disables all tests in the conformance test suite, as well as
moving the data files outside of the tests/data directory; the next few
commits will re-establish the conformance test suite separately so we
can check that everything works in a reliable way.

32 files changed, 46 insertions, 829 deletions

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 8526b39fe..65b474a49 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -1,3 +1,3 @@
-SUBDIRS = accessibility data conform interactive micro-bench performance
+SUBDIRS = accessibility conform interactive micro-bench performance
 
-EXTRA_DIST = README
+EXTRA_DIST = README clutter-1.0.suppressions
diff --git a/tests/data/clutter-1.0.suppressions b/tests/clutter-1.0.suppressions
index f47498d16..f47498d16 100644
--- a/tests/data/clutter-1.0.suppressions
+++ b/tests/clutter-1.0.suppressions
diff --git a/tests/conform/ADDING_NEW_TESTS b/tests/conform/ADDING_NEW_TESTS
deleted file mode 100644
index 4a5a5dc5f..000000000
--- a/tests/conform/ADDING_NEW_TESTS
+++ /dev/null
@@ -1,65 +0,0 @@
-How to add new units to the Clutter Conformance test suite
--------------------------------------------------------------------------------
-
-If the new unit is not logically part of an existing file, you should create a
-new source file and add it to the build. The built files are listed in the
-Makefile.am template. Please, respect the sections already available there.
-
-When creating a new file, you should include "test-conform-common.h", as well
-as <clutter/clutter.h> and <glib.h>.
-
-Each new test unit should be contained in a function named following this
-pattern:
-
-  void
-  <section>_<unit> (void)
-  {
-  }
-
-For instance:
-
-  void
-  actor_allocation_changes (void)
-  {
-    /* test allocation changes */
-  }
-
-  void
-  rectangle_border (void)
-  {
-    /* test ClutterRectangle's border property */
-  }
-
-After adding the test unit, edit test-conform-main.c and add the unit to the
-list of tests, by using one of these three macros:
-
-  • TEST_CONFORM_SIMPLE (path, function_name);
-
-    @path is the unit path in the suite, and it's used to generate the
-          executable wrapper for running the unit as a stand-alone binary
-          and for the HTML report.
-    @function_name is the name of the function containing the newly added
-          test unit.
-
-    This is the simple form of test unit, and it will always be run.
-
-  • TEST_CONFORM_SKIP (condition, path, function_name);
-
-    @condition is used to decide whether to run the unit or not.
-
-    This macro will check @condition on start, and if it evaluates to TRUE
-    then the @function_name will be called and the unit executed; otherwise
-    the test will automatically succeed.
-
-  • TEST_CONFORM_TODO (path, function_name);
-
-    This macro will execute @function_name and will succeed if the unit
-    fails. This macro should be used for tests that are known to fail.
-
-Notes:
-
-  • Do not call clutter_init() in your units; initialization is handled
-    by the conformance test suite itself.
-
-  • All units are invoked in a new process, to prevent clobbering the
-    state.
diff --git a/tests/conform/Makefile.am b/tests/conform/Makefile.am
index 90f1f35b1..d0497d196 100644
--- a/tests/conform/Makefile.am
+++ b/tests/conform/Makefile.am
@@ -1,314 +1,43 @@
-include $(top_srcdir)/build/autotools/Makefile.am.silent
-
-NULL =
-
-BUILT_SOURCES =
-
-TESTS =
-check_PROGRAMS =
-check_SCRIPTS =
-
-EXTRA_DIST =
-DISTCLEANFILES =
-
-TEST_PROGS =
-
-# the common sources
-common_sources = \
-	test-conform-common.h 	\
-	test-conform-common.c 	\
-	test-conform-main.c	\
-	$(NULL)
-
-# the unit-specific sources; please: keep all sections in alphabetical order!
-units_sources =
-
-# animation tests
-units_sources += \
-	animator.c			\
-	behaviours.c			\
-	score.c				\
-	state.c				\
-	timeline.c			\
-	timeline-interpolate.c 		\
-	timeline-progress.c		\
-	timeline-rewind.c 		\
-	$(NULL)
-
-# actors tests
-units_sources += \
-	actor-anchors.c                	\
-	actor-graph.c			\
-	actor-destroy.c			\
-	actor-invariants.c 		\
-	actor-iter.c			\
-	actor-layout.c			\
-	actor-meta.c			\
-	actor-offscreen-redirect.c	\
-	actor-offscreen-limit-max-size.c\
-	actor-paint-opacity.c 		\
-	actor-pick.c 			\
-	actor-shader-effect.c		\
-	actor-size.c			\
-	binding-pool.c			\
-	cairo-texture.c    		\
-	group.c				\
-	interval.c			\
-	path.c 				\
-	rectangle.c 			\
-	texture-fbo.c			\
-	texture.c			\
-        text-cache.c               	\
-        text.c             		\
+include $(top_srcdir)/build/autotools/glib-tap.mk
+
+AM_CFLAGS = -g $(CLUTTER_CFLAGS) $(MAINTAINER_CFLAGS)
+LDADD = $(top_builddir)/clutter/libclutter-1.0.la $(CLUTTER_LIBS) -lm
+AM_LDFLAGS = -export-dynamic
+AM_CPPFLAGS = \
+	-DG_LOG_DOMAIN=\"Clutter-Conform\"	\
+	-I$(top_srcdir) 			\
+	-I$(top_builddir)			\
+	-DCOGL_DISABLE_DEPRECATION_WARNINGS	\
+	$(CLUTTER_DEPRECATED_CFLAGS)		\
+	$(CLUTTER_DEBUG_CFLAGS)			\
+	$(CLUTTER_PROFILE_CFLAGS)
+
+actor_tests = \
 	$(NULL)
 
-# objects tests
-units_sources += \
-	color.c				\
-	model.c				\
-	script-parser.c			\
-	units.c				\
-        $(NULL)
-
-# cally tests
-units_sources += \
-	cally-text.c			\
+general_tests = \
 	$(NULL)
 
-# events tests
-units_sources += \
-	events-touch.c			\
+deprecated_tests = \
 	$(NULL)
 
-if OS_WIN32
-SHEXT =
-else
-SHEXT = $(EXEEXT)
-endif
-
-EXTRA_DIST += ADDING_NEW_TESTS test-launcher.sh.in run-tests.sh
-DISTCLEANFILES += test-launcher.sh .gitignore
-
-# For convenience, this provides a way to easily run individual unit tests:
-.PHONY: wrappers clean-wrappers
-
-#UNIT_TESTS = `./test-conformance -l -m thorough | $(GREP) '^/'`
-
-wrappers: stamp-test-conformance
-	@true
-stamp-test-conformance: Makefile $(srcdir)/test-conform-main.c
-	@mkdir -p wrappers
-	@sed -n \
-		-e 's/^ \{1,\}TEST_CONFORM_SIMPLE *(.*"\([^",]\{1,\}\)", *\([a-zA-Z0-9_]\{1,\}\).*/\/conform\1\/\2/p' \
-		-e 's/^ \{1,\}TEST_CONFORM_SKIP *(.*"\([^",]\{1,\}\)", *\([a-zA-Z0-9_]\{1,\}\).*/\/conform\1\/\2/p' \
-		-e 's/^ \{1,\}TEST_CONFORM_TODO *(.*"\([^",]\{1,\}\)", *\([a-zA-Z0-9_]\{1,\}\).*/\/conform\1\/\2/p' \
-	$(srcdir)/test-conform-main.c > unit-tests
-	@chmod +x test-launcher.sh
-	@( echo "/stamp-test-conformance" ; \
-	   echo "/test-conformance" ; \
-	   echo "*.o" ; \
-	   echo "*.xml" ; \
-	   echo "*.html" ; \
-	   echo "*.test" ; \
-	   echo ".gitignore" ; \
-	   echo "test-suite.log" ; \
-	   echo "unit-tests" ; \
-	   echo "/wrappers/" ) > .gitignore
-	@for i in `cat unit-tests`; \
-	do \
-		unit=`basename $$i | sed -e s/_/-/g`; \
-		echo "  GEN      $$unit"; \
-		( echo "#!/bin/sh" ; echo "$(abs_builddir)/test-launcher.sh '$$i' \"\$$@\"" ) > $$unit$(SHEXT) ; \
-		( echo "#!/bin/sh" ; echo "exec $(abs_builddir)/test-conformance$(EXEEXT) -p $$i \"\$$@\"" ) > wrappers/$$unit$(SHEXT) ; \
-		( echo "test-conformance-clutter$(EXEEXT) -p $$i" ) > $(top_builddir)/build/win32/$$unit-clutter.bat ; \
-		( echo "test-conformance-clutter$(EXEEXT) -p $$i" ) >> $(top_builddir)/build/win32/test-conformance-clutter.bat ; \
-		chmod +x $$unit$(SHEXT); \
-		chmod +x wrappers/$$unit$(SHEXT); \
-		echo "/$$unit$(SHEXT)" >> .gitignore; \
-	done \
-	&& echo timestamp > $(@F)
-
-clean-wrappers:
-	@if test -f "unit-tests"; then \
-	  for i in `cat unit-tests`; \
-	  do \
-		unit=`basename $$i | sed -e s/_/-/g`; \
-		echo "  RM       $$unit"; \
-		rm -f $$unit$(SHEXT) ; \
-		rm -f wrappers/$$unit$(SHEXT) ; \
-	  done \
-	fi \
-	&& rm -rf wrappers \
-	&& rm -f unit-tests \
-	&& rm -f $(top_builddir)/build/win32/*.bat \
-	&& rm -f stamp-test-conformance
-
-test_conformance_CPPFLAGS = \
-	-DG_DISABLE_SINGLE_INCLUDES \
-	-DCOGL_ENABLE_EXPERIMENTAL_API \
-	-DG_DISABLE_DEPRECATION_WARNINGS \
-	-DCOGL_DISABLE_DEPRECATION_WARNINGS \
-	-DCLUTTER_DISABLE_DEPRECATION_WARNINGS \
-	-DTESTS_DATADIR=\""$(top_srcdir)/tests/data"\" \
-	-I$(top_srcdir) \
-	-I$(top_builddir) \
-	-I$(top_srcdir)/clutter \
-	-I$(top_builddir)/clutter
-
-test_conformance_CFLAGS = -g $(CLUTTER_CFLAGS)
-test_conformance_LDADD = $(top_builddir)/clutter/libclutter-@CLUTTER_API_VERSION@.la $(CLUTTER_LIBS) -lm
-test_conformance_LDFLAGS = -export-dynamic
-test_conformance_SOURCES = $(common_sources) $(units_sources)
-
-if OS_WIN32
-TESTS += test-conformance
-endif
-
-TEST_PROGS += test-conformance
-check_PROGRAMS += test-conformance
-check_SCRIPTS += wrappers
-
-test: wrappers
-	@export G_TEST_SRCDIR="$(abs_srcdir)" ; \
-	 export G_TEST_BUILDDIR="$(abs_builddir)" ; \
-	 $(top_srcdir)/tests/conform/run-tests.sh \
-	  ./test-conformance$(EXEEXT) -o test-report.xml
-
-test-verbose: wrappers
-	@export G_TEST_SRCDIR="$(abs_srcdir)" ; \
-	 export G_TEST_BUILDDIR="$(abs_builddir)" ; \
-	 $(top_srcdir)/tests/conform/run-tests.sh \
-	  ./test-conformance$(EXEEXT) -o test-report.xml --verbose
-
-.PHONY: test
-.PHONY: test-report perf-report full-report
-
-check-local: test
-
-GTESTER = gtester
-GTESTER_REPORT = gtester-report
-
-# test-report: run tests and generate report
-# perf-report: run tests with -m perf and generate report
-# full-report: like test-report: with -m perf and -m slow
-test-report perf-report full-report: ${TEST_PROGS}
-	@test -z "${TEST_PROGS}" || { \
-	  export GTESTER_LOGDIR=`mktemp -d "$(srcdir)/.testlogs-XXXXXX"` ; \
-	  if test -d "$(top_srcdir)/.git"; then \
-	    export REVISION="`git describe`" ;  \
-	  else \
-	    export REVISION="$(VERSION) $(CLUTTER_RELEASE_STATUS)" ; \
-	  fi ; \
-	  export TIMESTAMP=`date +%Y-%m-%dT%H:%M:%S%z` ; \
-	  case $@ in \
-	  test-report) test_options="-k";; \
-	  perf-report) test_options="-k -m=perf";; \
-	  full-report) test_options="-k -m=perf -m=slow";; \
-	  esac ; \
-	  export G_TEST_SRCDIR="$(abs_srcdir)" ; \
-	  export G_TEST_BUILDDIR="$(abs_builddir)" ; \
-	  $(top_srcdir)/tests/conform/run-tests.sh \
-	    ./test-conformance$(EXEEXT) \
-	    --verbose \
-	    $$test_options \
-	    -o `mktemp "$$GTESTER_LOGDIR/log-XXXXXX"` ; \
-	  echo '<?xml version="1.0"?>'              > $@.xml ; \
-	  echo '<report-collection>'               >> $@.xml ; \
-	  echo '<info>'                            >> $@.xml ; \
-	  echo '  <package>$(PACKAGE)</package>'   >> $@.xml ; \
-	  echo '  <version>$(VERSION)</version>'   >> $@.xml ; \
-	  echo "  <revision>$$REVISION</revision>" >> $@.xml ; \
-	  echo "  <date>$$TIMESTAMP</date>"        >> $@.xml ; \
-	  echo '</info>'                           >> $@.xml ; \
-	  for lf in `ls -L "$$GTESTER_LOGDIR"/.` ; do \
-	    sed '1,1s/^<?xml\b[^>?]*?>//' <"$$GTESTER_LOGDIR"/"$$lf" >> $@.xml ; \
-	  done ; \
-	  echo >> $@.xml ; \
-	  echo '</report-collection>' >> $@.xml ; \
-	  ${GTESTER_REPORT} --version 2>/dev/null 1>&2 ; test "$$?" != 0 || ${GTESTER_REPORT} $@.xml >$@.html ; \
-	  rm -rf "$$GTESTER_LOGDIR" ; \
-	}
-
-XML_REPORTS = \
-	test-report.xml 	\
-	perf-report.xml 	\
-	full-report.xml 	\
-	test-report-npot.xml 	\
-	perf-report-npot.xml 	\
-	full-report-npot.xml
-
-HTML_REPORTS = \
-	test-report.html 	\
-	perf-report.html 	\
-	full-report.html 	\
-	test-report-npot.html 	\
-	perf-report-npot.html 	\
-	full-report-npot.html
-
-dist-hook: $(top_builddir)/build/win32/vs9/test-conformance-clutter.vcproj $(top_builddir)/build/win32/vs10/test-conformance-clutter.vcxproj $(top_builddir)/build/win32/vs10/test-conformance-clutter.vcxproj.filters
-
-$(top_builddir)/build/win32/vs9/test-conformance-clutter.vcproj: $(top_srcdir)/build/win32/vs9/test-conformance-clutter.vcprojin
-	@for F in $(test_conformance_SOURCES); do \
-		case $$F in \
-		*.c)	echo '   <File RelativePath="..\..\..\tests\conform\'$$F'" />' \
-			;; \
-		esac; \
-	done > testconformance.sourcefiles
-	$(CPP) -P - <$(top_srcdir)/build/win32/vs9/test-conformance-clutter.vcprojin >$@
-	rm -f testconformance.sourcefiles
-
-$(top_builddir)/build/win32/vs10/test-conformance-clutter.vcxproj: $(top_srcdir)/build/win32/vs10/test-conformance-clutter.vcxprojin
-	@for F in $(test_conformance_SOURCES); do \
-		case $$F in \
-		*.c)	echo '    <ClCompile Include="..\..\..\tests\conform\'$$F'" />' \
-			;; \
-		esac; \
-	done >testconformance.vs10.sourcefiles
-	$(CPP) -P - <$(top_srcdir)/build/win32/vs10/test-conformance-clutter.vcxprojin >$@
-	rm -f testconformance.vs10.sourcefiles
-	
-$(top_builddir)/build/win32/vs10/test-conformance-clutter.vcxproj.filters: $(top_srcdir)/build/win32/vs10/test-conformance-clutter.vcxproj.filtersin
-	@for F in $(test_conformance_SOURCES); do \
-		case $$F in \
-		*.c)	echo '    <ClCompile Include="..\..\..\tests\conform\'$$F'"><Filter>Sources</Filter></ClCompile>' \
-			;; \
-		esac; \
-	done > testconformance.vs10.sourcefiles.filters
-	$(CPP) -P - < $(top_srcdir)/build/win32/vs10/test-conformance-clutter.vcxproj.filtersin > $@
-	rm -f testconformance.vs10.sourcefiles.filters
-
-# Let the VS9/VS10 Project files be cleared out before they are re-expanded...
-DISTCLEANFILES += \
-	$(top_builddir)/build/win32/vs9/test-conformance-clutter.vcproj	\
-	$(top_builddir)/build/win32/vs10/test-conformance-clutter.vcxproj \
-	$(top_builddir)/build/win32/vs10/test-conformance-clutter.vcxproj.filters
-
-# we override the clean-generic target to clean up the wrappers so
-# we cannot use CLEANFILES
-clean-generic: clean-wrappers clean-tests
-	$(QUIET_RM)rm -f $(XML_REPORTS) $(HTML_REPORTS)
-
-if ENABLE_INSTALLED_TESTS
-# installed tests
-insttestdir = $(libexecdir)/installed-tests/$(PACKAGE)/conform
-insttest_PROGRAMS = test-conformance
-
-testmetadir = $(datadir)/installed-tests/$(PACKAGE)
-testmeta_DATA = $(wildcard *.test)
-
-BUILT_SOURCES += tests
-endif
-
-.PHONY: tests clean-tests
-
-tests: stamp-test-conformance
-	@for i in `cat unit-tests`; do \
-		unit=`basename $$i | sed -e s/_/-/g`; \
-		echo "  GEN      $$unit"; \
-		echo "[Test]" > $$unit.test; \
-		echo "Type=session" >> $$unit.test; \
-		echo "Exec=$(libexecdir)/installed-tests/$(PACKAGE)/conform/test-conformance -p $$i" >> $$unit.test; \
-	done
-
-clean-tests:
-	$(QUIET_RM) rm -f *.test
+test_programs = $(actor_tests) $(general_tests) $(deprecated_tests)
+
+dist_test_data = $(script_ui_files)
+script_ui_files = $(addprefix scripts/,$(script_tests))
+script_tests = \
+	test-animator-1.json \
+	test-animator-2.json \
+	test-animator-3.json \
+	test-script-animation.json \
+	test-script-child.json \
+	test-script-implicit-alpha.json \
+	test-script-interval.json \
+	test-script-layout-property.json \
+	test-script-margin.json \
+	test-script-model.json \
+	test-script-named-object.json \
+	test-script-object-property.json \
+	test-script-single.json \
+	test-script-timeline-markers.json \
+	test-state-1.json
diff --git a/tests/conform/run-tests.sh b/tests/conform/run-tests.sh
deleted file mode 100755
index 03f5a7af3..000000000
--- a/tests/conform/run-tests.sh
+++ /dev/null
@@ -1,12 +0,0 @@
-#!/bin/sh
-
-BINARY=$1
-shift
-
-TMP=`./$BINARY -l -m thorough | grep '^/' | sed -e s/_/-/g`
-for i in $TMP
-do
-  TESTS="$TESTS wrappers/`basename $i`"
-done
-
-G_DEBUG=gc-friendly MALLOC_CHECK_=2 MALLOC_PERTURB_=$((${RANDOM:-256} % 256)) gtester "$@" $TESTS
diff --git a/tests/data/test-animator-1.json b/tests/conform/scripts/test-animator-1.json
index 2d6aab908..2d6aab908 100644
--- a/tests/data/test-animator-1.json
+++ b/tests/conform/scripts/test-animator-1.json
diff --git a/tests/data/test-animator-2.json b/tests/conform/scripts/test-animator-2.json
index 9059f57ed..9059f57ed 100644
--- a/tests/data/test-animator-2.json
+++ b/tests/conform/scripts/test-animator-2.json
diff --git a/tests/data/test-animator-3.json b/tests/conform/scripts/test-animator-3.json
index 26d779bee..26d779bee 100644
--- a/tests/data/test-animator-3.json
+++ b/tests/conform/scripts/test-animator-3.json
diff --git a/tests/data/test-script-animation.json b/tests/conform/scripts/test-script-animation.json
index d6ef90343..d6ef90343 100644
--- a/tests/data/test-script-animation.json
+++ b/tests/conform/scripts/test-script-animation.json
diff --git a/tests/data/test-script-child.json b/tests/conform/scripts/test-script-child.json
index 5e8ac5242..5e8ac5242 100644
--- a/tests/data/test-script-child.json
+++ b/tests/conform/scripts/test-script-child.json
diff --git a/tests/data/test-script-implicit-alpha.json b/tests/conform/scripts/test-script-implicit-alpha.json
index 5aab70d3d..5aab70d3d 100644
--- a/tests/data/test-script-implicit-alpha.json
+++ b/tests/conform/scripts/test-script-implicit-alpha.json
diff --git a/tests/data/test-script-interval.json b/tests/conform/scripts/test-script-interval.json
index 35fe5c22c..35fe5c22c 100644
--- a/tests/data/test-script-interval.json
+++ b/tests/conform/scripts/test-script-interval.json
diff --git a/tests/data/test-script-layout-property.json b/tests/conform/scripts/test-script-layout-property.json
index 17e9f4882..17e9f4882 100644
--- a/tests/data/test-script-layout-property.json
+++ b/tests/conform/scripts/test-script-layout-property.json
diff --git a/tests/data/test-script-margin.json b/tests/conform/scripts/test-script-margin.json
index 1f5289fa6..1f5289fa6 100644
--- a/tests/data/test-script-margin.json
+++ b/tests/conform/scripts/test-script-margin.json
diff --git a/tests/data/test-script-model.json b/tests/conform/scripts/test-script-model.json
index 9bd6aefc5..9bd6aefc5 100644
--- a/tests/data/test-script-model.json
+++ b/tests/conform/scripts/test-script-model.json
diff --git a/tests/data/test-script-named-object.json b/tests/conform/scripts/test-script-named-object.json
index 7210321c9..7210321c9 100644
--- a/tests/data/test-script-named-object.json
+++ b/tests/conform/scripts/test-script-named-object.json
diff --git a/tests/data/test-script-object-property.json b/tests/conform/scripts/test-script-object-property.json
index 18a02fb8b..18a02fb8b 100644
--- a/tests/data/test-script-object-property.json
+++ b/tests/conform/scripts/test-script-object-property.json
diff --git a/tests/data/test-script-single.json b/tests/conform/scripts/test-script-single.json
index e20a318df..e20a318df 100644
--- a/tests/data/test-script-single.json
+++ b/tests/conform/scripts/test-script-single.json
diff --git a/tests/data/test-script-timeline-markers.json b/tests/conform/scripts/test-script-timeline-markers.json
index f5fff5c42..f5fff5c42 100644
--- a/tests/data/test-script-timeline-markers.json
+++ b/tests/conform/scripts/test-script-timeline-markers.json
diff --git a/tests/data/test-state-1.json b/tests/conform/scripts/test-state-1.json
index 78ab5de89..78ab5de89 100644
--- a/tests/data/test-state-1.json
+++ b/tests/conform/scripts/test-state-1.json
diff --git a/tests/conform/test-conform-common.c b/tests/conform/test-conform-common.c
deleted file mode 100644
index 881a6b1cd..000000000
--- a/tests/conform/test-conform-common.c
+++ /dev/null
@@ -1,105 +0,0 @@
-#include "config.h"
-
-#include <clutter/clutter.h>
-#include <stdlib.h>
-
-#ifdef CLUTTER_WINDOWING_X11
-#include <X11/Xlib.h>
-#include <clutter/x11/clutter-x11.h>
-#endif
-
-#include "test-conform-common.h"
-
-/**
- * test_conform_simple_fixture_setup:
- *
- * Initialise stuff before each test is run
- */
-void
-test_conform_simple_fixture_setup (TestConformSimpleFixture *fixture,
-				   gconstpointer data)
-{
-  const TestConformSharedState *shared_state = data;
-  static int counter = 0;
-
-  if (counter != 0)
-    g_critical ("We don't support running more than one test at a time\n"
-                "in a single test run due to the state leakage that often\n"
-                "causes subsequent tests to fail.\n"
-                "\n"
-                "If you want to run all the tests you should run\n"
-                "$ make test-report");
-  counter++;
-
-#ifdef CLUTTER_WINDOWING_X11
-  if (clutter_check_windowing_backend (CLUTTER_WINDOWING_X11))
-    {
-      /* on X11 we need a display connection to run the test suite */
-      const gchar *display = g_getenv ("DISPLAY");
-      if (!display || *display == '\0')
-        {
-          g_print ("No DISPLAY found. Unable to run the conformance "
-                   "test suite without a display.\n");
-
-          exit (EXIT_SUCCESS);
-        }
-
-      /* enable XInput support */
-      clutter_x11_enable_xinput ();
-    }
-#endif
-
-  g_assert (clutter_init (shared_state->argc_addr, shared_state->argv_addr)
-            == CLUTTER_INIT_SUCCESS);
-
-#ifdef CLUTTER_WINDOWING_X11
-  /* A lot of the tests depend on a specific stage / framebuffer size
-   * when they read pixels back to verify the results of the test.
-   *
-   * Normally the asynchronous nature of X means that setting the
-   * clutter stage size may really happen an indefinite amount of time
-   * later but since the tests are so short lived and may only render
-   * a single frame this is not an acceptable semantic.
-   */
-  if (clutter_check_windowing_backend (CLUTTER_WINDOWING_X11))
-    XSynchronize (clutter_x11_get_default_display(), TRUE);
-#endif
-}
-
-
-/**
- * test_conform_simple_fixture_teardown:
- *
- * Cleanup stuff after each test has finished
- */
-void
-test_conform_simple_fixture_teardown (TestConformSimpleFixture *fixture,
-				      gconstpointer data)
-{
-  /* const TestConformSharedState *shared_state = data; */
-}
-
-void
-test_conform_get_gl_functions (TestConformGLFunctions *functions)
-{
-  functions->glGetString = (void *) cogl_get_proc_address ("glGetString");
-  g_assert (functions->glGetString != NULL);
-  functions->glGetIntegerv = (void *) cogl_get_proc_address ("glGetIntegerv");
-  g_assert (functions->glGetIntegerv != NULL);
-  functions->glPixelStorei = (void *) cogl_get_proc_address ("glPixelStorei");
-  g_assert (functions->glPixelStorei != NULL);
-  functions->glBindTexture = (void *) cogl_get_proc_address ("glBindTexture");
-  g_assert (functions->glBindTexture != NULL);
-  functions->glGenTextures = (void *) cogl_get_proc_address ("glGenTextures");
-  g_assert (functions->glGenTextures != NULL);
-  functions->glGetError = (void *) cogl_get_proc_address ("glGetError");
-  g_assert (functions->glGetError != NULL);
-  functions->glDeleteTextures =
-    (void *) cogl_get_proc_address ("glDeleteTextures");
-  g_assert (functions->glDeleteTextures != NULL);
-  functions->glTexImage2D = (void *) cogl_get_proc_address ("glTexImage2D");
-  g_assert (functions->glTexImage2D != NULL);
-  functions->glTexParameteri =
-    (void *) cogl_get_proc_address ("glTexParameteri");
-  g_assert (functions->glTexParameteri != NULL);
-}
diff --git a/tests/conform/test-conform-common.h b/tests/conform/test-conform-common.h
deleted file mode 100644
index f2d50dedd..000000000
--- a/tests/conform/test-conform-common.h
+++ /dev/null
@@ -1,52 +0,0 @@
-
-/* Stuff you put in here is setup once in main() and gets passed around to
- * all test functions and fixture setup/teardown functions in the data
- * argument */
-typedef struct _TestConformSharedState
-{
-  int	 *argc_addr;
-  char ***argv_addr;
-} TestConformSharedState;
-
-
-/* This fixture structure is allocated by glib, and before running each test
- * the test_conform_simple_fixture_setup func (see below) is called to
- * initialise it, and test_conform_simple_fixture_teardown is called when
- * the test is finished. */
-typedef struct _TestConformSimpleFixture
-{
-  /**/
-  int dummy;
-} TestConformSimpleFixture;
-
-typedef struct _TestConformTodo
-{
-  gchar *name;
-  void (* func) (TestConformSimpleFixture *, gconstpointer);
-} TestConformTodo;
-
-typedef struct _TestConformGLFunctions
-{
-  const guint8 * (* glGetString) (guint name);
-  void (* glGetIntegerv) (guint pname, int *params);
-  void (* glPixelStorei) (guint pname, int param);
-  void (* glBindTexture) (guint target, guint texture);
-  void (* glGenTextures) (int n, guint *textures);
-  guint (* glGetError) (void);
-  void (* glDeleteTextures) (int n, const guint *textures);
-  void (* glTexImage2D) (guint target, int level,
-                         int internalFormat,
-                         int width, int height,
-                         int border, guint format, guint type,
-                         const void *pixels);
-  void (* glTexParameteri) (guint target, guint pname, int param);
-} TestConformGLFunctions;
-
-void test_conform_get_gl_functions (TestConformGLFunctions *functions);
-
-void test_conform_simple_fixture_setup (TestConformSimpleFixture *fixture,
-					gconstpointer data);
-void test_conform_simple_fixture_teardown (TestConformSimpleFixture *fixture,
-					   gconstpointer data);
-
-gchar *clutter_test_get_data_file (const gchar *filename);
diff --git a/tests/conform/test-conform-main.c b/tests/conform/test-conform-main.c
deleted file mode 100644
index 00cc1c628..000000000
--- a/tests/conform/test-conform-main.c
+++ /dev/null
@@ -1,217 +0,0 @@
-#include "config.h"
-
-#include <clutter/clutter.h>
-
-#include <glib.h>
-#include <locale.h>
-#include <stdlib.h>
-
-#include "test-conform-common.h"
-
-static void
-test_conform_skip_test (TestConformSimpleFixture *fixture,
-                        gconstpointer             data)
-{
-  /* void */
-}
-
-void
-verify_failure (TestConformSimpleFixture *fixture,
-                gconstpointer             data)
-{
-  g_assert (FALSE);
-}
-
-static TestConformSharedState *shared_state = NULL;
-
-/* This is a bit of sugar for adding new conformance tests:
- *
- * - It adds an extern function definition just to save maintaining a header
- *   that lists test entry points.
- * - It sets up callbacks for a fixture, which lets us share initialization
- *   *code* between tests. (see test-conform-common.c)
- * - It passes in a shared *data* pointer that is initialised once in main(),
- *   that gets passed to the fixture setup and test functions. (See the
- *   definition in test-conform-common.h)
- */
-#define TEST_CONFORM_SIMPLE(NAMESPACE, FUNC)            G_STMT_START {  \
-  extern void FUNC (TestConformSimpleFixture *, gconstpointer);         \
-  g_test_add ("/conform" NAMESPACE "/" #FUNC,                           \
-	      TestConformSimpleFixture,                                 \
-	      shared_state, /* data argument for test */                \
-	      test_conform_simple_fixture_setup,                        \
-	      FUNC,                                                     \
-	      test_conform_simple_fixture_teardown);    } G_STMT_END
-
-/* this is a macro that conditionally executes a test if CONDITION
- * evaluates to TRUE; otherwise, it will put the test under the
- * "/skipped" namespace and execute a dummy function that will always
- * pass.
- */
-#define TEST_CONFORM_SKIP(CONDITION, NAMESPACE, FUNC)   G_STMT_START {  \
-  if (CONDITION) { TEST_CONFORM_SIMPLE (NAMESPACE, FUNC); }             \
-  else {                                                                \
-    g_test_add ("/skipped" NAMESPACE "/" #FUNC,                         \
-                TestConformSimpleFixture,                               \
-                shared_state, /* data argument for test */              \
-                test_conform_simple_fixture_setup,                      \
-                test_conform_skip_test,                                 \
-                test_conform_simple_fixture_teardown);                  \
-  }                                                     } G_STMT_END
-
-gchar *
-clutter_test_get_data_file (const gchar *filename)
-{
-  return g_test_build_filename (G_TEST_DIST, "..", "data", filename, NULL);
-}
-
-static void
-clutter_test_init (gint    *argc,
-                   gchar ***argv)
-{
-  /* Turning of sync-to-vblank removes a dependency on the specifics of the
-   * test environment. It also means that the timeline-only tests are
-   * throttled to a reasonable frame rate rather than running in tight
-   * infinite loop.
-   */
-  g_setenv ("CLUTTER_VBLANK", "none", FALSE);
-
-  g_test_init (argc, argv, NULL);
-
-  g_test_bug_base ("http://bugzilla.gnome.org/show_bug.cgi?id=%s");
-
-  /* Initialise the state you need to share with everything.
-   */
-  shared_state = g_new0 (TestConformSharedState, 1);
-  shared_state->argc_addr = argc;
-  shared_state->argv_addr = argv;
-}
-
-int
-main (int argc, char **argv)
-{
-  clutter_test_init (&argc, &argv);
-
-  /* This file is run through a sed script during the make step so the
-     lines containing the tests need to be formatted on a single line
-     each. To comment out a test use the SKIP macro. Using
-     #if 0 would break the script. */
-
-  TEST_CONFORM_SIMPLE ("/actor", actor_add_child);
-  TEST_CONFORM_SIMPLE ("/actor", actor_insert_child);
-  TEST_CONFORM_SIMPLE ("/actor", actor_raise_child);
-  TEST_CONFORM_SIMPLE ("/actor", actor_lower_child);
-  TEST_CONFORM_SIMPLE ("/actor", actor_replace_child);
-  TEST_CONFORM_SIMPLE ("/actor", actor_remove_child);
-  TEST_CONFORM_SIMPLE ("/actor", actor_remove_all);
-  TEST_CONFORM_SIMPLE ("/actor", actor_container_signals);
-  TEST_CONFORM_SIMPLE ("/actor", actor_destruction);
-  TEST_CONFORM_SIMPLE ("/actor", actor_anchors);
-  TEST_CONFORM_SIMPLE ("/actor", actor_pick);
-  TEST_CONFORM_SIMPLE ("/actor", actor_fixed_size);
-  TEST_CONFORM_SIMPLE ("/actor", actor_preferred_size);
-  TEST_CONFORM_SIMPLE ("/actor", actor_basic_layout);
-  TEST_CONFORM_SIMPLE ("/actor", actor_margin_layout);
-  TEST_CONFORM_SIMPLE ("/actor", actor_offscreen_redirect);
-  TEST_CONFORM_SIMPLE ("/actor", actor_offscreen_limit_max_size);
-  TEST_CONFORM_SIMPLE ("/actor", actor_shader_effect);
-
-  TEST_CONFORM_SIMPLE ("/actor/iter", actor_iter_traverse_children);
-  TEST_CONFORM_SIMPLE ("/actor/iter", actor_iter_traverse_remove);
-  TEST_CONFORM_SIMPLE ("/actor/iter", actor_iter_assignment);
-
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_initial_state);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_shown_not_parented);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_visibility_not_recursive);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_realized);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_realize_not_recursive);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_map_recursive);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_mapped);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_show_on_set_parent);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", clone_no_map);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_contains);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", default_stage);
-  TEST_CONFORM_SIMPLE ("/actor/invariants", actor_pivot_transformation);
-
-  TEST_CONFORM_SIMPLE ("/actor/meta", actor_meta_clear);
-
-  TEST_CONFORM_SIMPLE ("/actor/opacity", opacity_label);
-  TEST_CONFORM_SIMPLE ("/actor/opacity", opacity_rectangle);
-  TEST_CONFORM_SIMPLE ("/actor/opacity", opacity_paint);
-
-  TEST_CONFORM_SIMPLE ("/text", text_utf8_validation);
-  TEST_CONFORM_SIMPLE ("/text", text_set_empty);
-  TEST_CONFORM_SIMPLE ("/text", text_set_text);
-  TEST_CONFORM_SIMPLE ("/text", text_append_some);
-  TEST_CONFORM_SIMPLE ("/text", text_prepend_some);
-  TEST_CONFORM_SIMPLE ("/text", text_insert);
-  TEST_CONFORM_SIMPLE ("/text", text_delete_chars);
-  TEST_CONFORM_SIMPLE ("/text", text_delete_text);
-  TEST_CONFORM_SIMPLE ("/text", text_cursor);
-  TEST_CONFORM_SIMPLE ("/text", text_event);
-  TEST_CONFORM_SIMPLE ("/text", text_get_chars);
-  TEST_CONFORM_SIMPLE ("/text", text_cache);
-  TEST_CONFORM_SIMPLE ("/text", text_password_char);
-  TEST_CONFORM_SIMPLE ("/text", text_idempotent_use_markup);
-
-  TEST_CONFORM_SIMPLE ("/rectangle", rectangle_set_size);
-  TEST_CONFORM_SIMPLE ("/rectangle", rectangle_set_color);
-
-  TEST_CONFORM_SKIP (g_test_slow (), "/texture", texture_pick_with_alpha);
-  TEST_CONFORM_SKIP (g_test_slow (), "/texture", texture_fbo);
-  TEST_CONFORM_SKIP (g_test_slow (), "/texture/cairo", texture_cairo);
-
-  TEST_CONFORM_SIMPLE ("/interval", interval_initial_state);
-  TEST_CONFORM_SIMPLE ("/interval", interval_transform);
-
-  TEST_CONFORM_SIMPLE ("/path", path_base);
-
-  TEST_CONFORM_SIMPLE ("/binding-pool", binding_pool);
-
-  TEST_CONFORM_SIMPLE ("/model", list_model_populate);
-  TEST_CONFORM_SIMPLE ("/model", list_model_iterate);
-  TEST_CONFORM_SIMPLE ("/model", list_model_filter);
-  TEST_CONFORM_SIMPLE ("/model", list_model_from_script);
-  TEST_CONFORM_SIMPLE ("/model", list_model_row_changed);
-
-  TEST_CONFORM_SIMPLE ("/color", color_from_string_valid);
-  TEST_CONFORM_SIMPLE ("/color", color_from_string_invalid);
-  TEST_CONFORM_SIMPLE ("/color", color_to_string);
-  TEST_CONFORM_SIMPLE ("/color", color_hls_roundtrip);
-  TEST_CONFORM_SIMPLE ("/color", color_operators);
-
-  TEST_CONFORM_SIMPLE ("/units", units_constructors);
-  TEST_CONFORM_SIMPLE ("/units", units_string);
-  TEST_CONFORM_SIMPLE ("/units", units_cache);
-
-  TEST_CONFORM_SIMPLE ("/group", group_depth_sorting);
-
-  TEST_CONFORM_SIMPLE ("/script", script_single);
-  TEST_CONFORM_SIMPLE ("/script", script_child);
-  TEST_CONFORM_SIMPLE ("/script", script_implicit_alpha);
-  TEST_CONFORM_SIMPLE ("/script", script_object_property);
-  TEST_CONFORM_SIMPLE ("/script", script_animation);
-  TEST_CONFORM_SIMPLE ("/script", script_named_object);
-  TEST_CONFORM_SIMPLE ("/script", script_layout_property);
-  TEST_CONFORM_SIMPLE ("/script", animator_base);
-  TEST_CONFORM_SIMPLE ("/script", animator_properties);
-  TEST_CONFORM_SIMPLE ("/script", animator_multi_properties);
-  TEST_CONFORM_SIMPLE ("/script", state_base);
-  TEST_CONFORM_SIMPLE ("/script", script_margin);
-  TEST_CONFORM_SIMPLE ("/script", script_interval);
-
-  TEST_CONFORM_SKIP (g_test_slow (), "/timeline", timeline_base);
-  TEST_CONFORM_SIMPLE ("/timeline", timeline_markers_from_script);
-  TEST_CONFORM_SKIP (g_test_slow (), "/timeline", timeline_interpolation);
-  TEST_CONFORM_SKIP (g_test_slow (), "/timeline", timeline_rewind);
-  TEST_CONFORM_SIMPLE ("/timeline", timeline_progress_mode);
-  TEST_CONFORM_SIMPLE ("/timeline", timeline_progress_step);
-
-  TEST_CONFORM_SIMPLE ("/score", score_base);
-
-  TEST_CONFORM_SIMPLE ("/behaviours", behaviours_base);
-
-  TEST_CONFORM_SIMPLE ("/events", events_touch);
-
-  return g_test_run ();
-}
diff --git a/tests/conform/test-launcher.sh.in b/tests/conform/test-launcher.sh.in
deleted file mode 100755
index 0be13bda5..000000000
--- a/tests/conform/test-launcher.sh.in
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-UNIT_TEST_PATH=$1
-shift
-
-test -z ${UNIT_TEST_PATH} && {
-        echo "Usage: $0 /path/to/unit_test"
-        exit 1
-}
-
-UNIT_TEST=`basename ${UNIT_TEST_PATH}`
-
-echo "Running: ./test-conformance -p ${UNIT_TEST_PATH} $@"
-echo ""
-@abs_builddir@/test-conformance -p ${UNIT_TEST_PATH} "$@"
-
-echo ""
-echo "NOTE: For debugging purposes, you can run this single test as follows:"
-echo "$ libtool --mode=execute \\"
-echo "          gdb --eval-command=\"b ${UNIT_TEST}\" \\"
-echo "          --args ./test-conformance -p ${UNIT_TEST_PATH}"
-echo "or:"
-echo "$ env G_SLICE=always-malloc \\"
-echo "  libtool --mode=execute \\"
-echo "          valgrind ./test-conformance -p ${UNIT_TEST_PATH}"
diff --git a/tests/data/Makefile.am b/tests/data/Makefile.am
deleted file mode 100644
index 4e6878e88..000000000
--- a/tests/data/Makefile.am
+++ /dev/null
@@ -1,39 +0,0 @@
-NULL =
-EXTRA_DIST =
-
-json_files = \
-	test-script-animation.json		\
-	test-script-child.json			\
-	test-script-layout-property.json	\
-	test-script-implicit-alpha.json		\
-	test-script.json			\
-	test-script-named-object.json		\
-	test-script-object-property.json	\
-	test-script-signals.json		\
-	test-script-single.json			\
-	test-script-model.json			\
-	test-animator-1.json			\
-	test-animator-2.json			\
-	test-animator-3.json			\
-	test-state-1.json			\
-	test-script-timeline-markers.json	\
-	test-script-margin.json 		\
-	test-script-interval.json		\
-	$(NULL)
-
-EXTRA_DIST += $(json_files)
-
-png_files = \
-	redhand.png 		\
-	redhand_alpha.png	\
-	light0.png 		\
-	$(NULL)
-
-EXTRA_DIST += $(png_files)
-
-if ENABLE_INSTALLED_TESTS
-insttestdir = $(libexecdir)/installed-tests/$(PACKAGE)/data
-insttest_DATA = $(json_files) $(png_files)
-endif
-
-EXTRA_DIST += clutter-1.0.suppressions
diff --git a/tests/data/light0.png b/tests/data/light0.png
deleted file mode 100644
index 66871ddb3..000000000
--- a/tests/data/light0.png
+++ /dev/null
diff --git a/tests/data/redhand_alpha.png b/tests/data/redhand_alpha.png
deleted file mode 100644
index 4270765ed..000000000
--- a/tests/data/redhand_alpha.png
+++ /dev/null
diff --git a/tests/interactive/Makefile.am b/tests/interactive/Makefile.am
index 8e87edf44..8bea2bcad 100644
--- a/tests/interactive/Makefile.am
+++ b/tests/interactive/Makefile.am
@@ -148,7 +148,7 @@ test_interactive_SOURCES = test-main.c $(UNIT_TESTS)
 nodist_test_interactive_SOURCES = test-unit-names.h
 test_interactive_CFLAGS = $(CLUTTER_CFLAGS) $(GDK_PIXBUF_CFLAGS)
 test_interactive_CPPFLAGS = \
-	-DTESTS_DATADIR=\""$(abs_top_srcdir)/tests/data"\" \
+	-DTESTS_DATADIR=\""$(abs_srcdir)"\" \
 	-DG_DISABLE_SINGLE_INCLUDES \
 	-DGLIB_DISABLE_DEPRECATION_WARNINGS \
 	-DCOGL_DISABLE_DEPRECATION_WARNINGS \
@@ -161,8 +161,11 @@ test_interactive_LDFLAGS = -export-dynamic
 test_interactive_LDADD = $(CLUTTER_LIBS) $(GDK_PIXBUF_LIBS) $(common_ldadd) -lm
 
 EXTRA_DIST = \
-	wrapper.sh.in			\
-	$(top_builddir)/build/win32/test-unit-names.h
+	wrapper.sh.in \
+	$(top_builddir)/build/win32/test-unit-names.h \
+	test-script.json \
+	test-script-signals.json \
+	redhand.png
 
 DISTCLEANFILES = wrapper.sh .gitignore test-unit-names.h
 
diff --git a/tests/data/redhand.png b/tests/interactive/redhand.png
index c07d8acd3..c07d8acd3 100644
--- a/tests/data/redhand.png
+++ b/tests/interactive/redhand.png
diff --git a/tests/data/test-script-signals.json b/tests/interactive/test-script-signals.json
index 0d3f6bd25..0d3f6bd25 100644
--- a/tests/data/test-script-signals.json
+++ b/tests/interactive/test-script-signals.json
diff --git a/tests/data/test-script.json b/tests/interactive/test-script.json
index 54d4fb457..54d4fb457 100644
--- a/tests/data/test-script.json
+++ b/tests/interactive/test-script.json
diff --git a/tests/interactive/test-state-script.c b/tests/interactive/test-state-script.c
index e777d89bc..34b1b8789 100644
--- a/tests/interactive/test-state-script.c
+++ b/tests/interactive/test-state-script.c
@@ -4,7 +4,7 @@
 
 #include <clutter/clutter.h>
 
-#define TEST_STATE_SCRIPT_FILE  TESTS_DATADIR G_DIR_SEPARATOR_S "test-script-signals.json"
+#define TEST_STATE_SCRIPT_FILE  "test-script-signals.json"
 
 gboolean
 on_button_press (ClutterActor *actor,