Driver Rework Patch

This patch comprises of four different but closely related ideas. The
net result is fixing a large number of open issues with the driver
whilst making it simpler to understand.

1. Use the hash of the source file to determine whether the source file
has changed or not. This makes the recompilation checking more robust to
modern build systems which are liable to copy files around changing
their modification times.

2. Remove the concept of a "stable module", a stable module was one
where the object file was older than the source file, and all transitive
dependencies were also stable. Now we don't rely on the modification
time of the source file, the notion of stability is moot.

3. Fix TH/plugin recompilation after the removal of stable modules. The
TH recompilation check used to rely on stable modules. Now there is a
uniform and simple way, we directly track the linkables which were
loaded into the interpreter whilst compiling a module. This is an
over-approximation but more robust wrt package dependencies changing.

4. Fix recompilation checking for dynamic object files. Now we actually
check if the dynamic object file exists when compiling with -dynamic-too

Fixes #19774 #19771 #19758 #17434 #11556 #9121 #8211 #16495 #7277 #16093

1 files changed, 8 insertions, 5 deletions

diff --git a/testsuite/driver/testutil.py b/testsuite/driver/testutil.py
index bcea98bd17..48f33b1540 100644
--- a/testsuite/driver/testutil.py
+++ b/testsuite/driver/testutil.py
@@ -62,7 +62,7 @@ def getStdout(cmd_and_args: List[str]):
         raise Exception("stderr from command: %s\nOutput:\n%s\n" % (cmd_and_args, stderr.decode('utf-8')))
     return stdout.decode('utf-8')
 
-def lndir(srcdir: Path, dstdir: Path):
+def lndir(srcdir: Path, dstdir: Path, force_copy=False):
     # Create symlinks for all files in src directory.
     # Not all developers might have lndir installed.
     # os.system('lndir -silent {0} {1}'.format(srcdir, dstdir))
@@ -71,7 +71,7 @@ def lndir(srcdir: Path, dstdir: Path):
         src = srcdir / base
         dst = dstdir / base
         if src.is_file():
-            link_or_copy_file(src, dst)
+            link_or_copy_file(src, dst, force_copy)
         else:
             dst.mkdir()
             lndir(src, dst)
@@ -116,11 +116,14 @@ def symlinks_work() -> bool:
         return True
 
 if not symlinks_work():
-    def link_or_copy_file(src: Path, dst: Path):
+    def link_or_copy_file(src: Path, dst: Path, force_copy=False):
         shutil.copyfile(str(src), str(dst))
 else:
-    def link_or_copy_file(src: Path, dst: Path):
-        os.symlink(str(src), str(dst))
+    def link_or_copy_file(src: Path, dst: Path, force_copy=False):
+        if force_copy:
+            shutil.copyfile(str(src), str(dst))
+        else:
+            os.symlink(str(src), str(dst))
 
 class Watcher(object):
     def __init__(self, count: int) -> None: