tests: deterministically generate test suite definitions

The script "generate.py" is used to parse all test source files for unit
tests. These are then written into a "clar.suite" file, which can be
included by the main test executable to make available all test suites
and unit tests.

Our current algorithm simply collects all test suites inside of a dict,
iterates through its items and dumps them in a special format into the
file. As the order is not guaranteed to be deterministic for Python
dictionaries, this may result in arbitrarily ordered C structs. This
obviously defeats the purpose of reproducible builds, where the same
input should always result in the exact same output.

Fix this issue by sorting the test suites by name previous to dumping
them as structs. This enables reproducible builds for the libgit2_clar
file.

1 files changed, 5 insertions, 3 deletions

diff --git a/tests/generate.py b/tests/generate.py
index 6a6228f25..0a94d4952 100644
--- a/tests/generate.py
+++ b/tests/generate.py
@@ -207,16 +207,18 @@ class TestSuite(object):
             return False
 
         with open(output, 'w') as data:
-            for module in self.modules.values():
+            modules = sorted(self.modules.values(), key=lambda module: module.name)
+
+            for module in modules:
                 t = Module.DeclarationTemplate(module)
                 data.write(t.render())
 
-            for module in self.modules.values():
+            for module in modules:
                 t = Module.CallbacksTemplate(module)
                 data.write(t.render())
 
             suites = "static struct clar_suite _clar_suites[] = {" + ','.join(
-                Module.InfoTemplate(module).render() for module in sorted(self.modules.values(), key=lambda module: module.name)
+                Module.InfoTemplate(module).render() for module in modules
             ) + "\n};\n"
 
             data.write(suites)