Fix various issues with testsuite code on Windows

Summary:
Previously we would make direct calls to `diff` using `os.system`.
On Windows `os.system` is implemented using the standard
idiom `CreateProcess .. WaitForSingleObject ..`.

This again runs afoul with the `_exec` behaviour on Windows. So we ran
into some trouble where sometimes `diff` would return before it's done.

On tests which run multiple ways, such as `8086` what happens is that
we think the diff is done and continue. The next way tries to set things
up again by removing any previous directory. This would then fail with
and error saying the directory can't be removed. Which is true, because
the previous diff code/child is still running.

We shouldn't make any external calls to anything using `os.system`.
Instead just use `runCmd` which uses `timeout`. This also ensures that if
we hit the cygwin bug where diff or any other utility hangs, we kill it and
continue and not hang the entire test and leave hanging processes.

Further more we also:
Ignore error lines from `removeFile` from tools in the testsuite. This is a rather large
hammer to work around the fact that `hsc2hs` often tries to remove it's own file too early.
When this is patched the workaround can be removed. See Trac #9775

We mark `prog003` as skip. Since this test randomly fails and passes. For stability it's disabled
but it is a genuine bug which we should find. It's something with interface files being
overwritten. See Trac #11317

when `rmtree` hits a readonly file, the `onerror` handler is raised afterwards but not
during the tree walk. It doesn't allow you to recover and continue as we thought.
Instead you have to explicitly start again. This is why sometimes even though we
call `cleanup` before `os.mkdirs`, it would sometimes fail with an error that the
folder already exists. So we now do a second walk.

A new verbosity level (4) will strip the silent flags from `MAKE` invocations so you can actually
see what's going on.

Test Plan: ./validate on build bots.

Reviewers: bgamari, austin

Reviewed By: bgamari

Subscribers: mpickering, thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2894

GHC Trac Issues: #12661, #11317, #9775

1 files changed, 2 insertions, 2 deletions

diff --git a/testsuite/timeout/timeout.hs b/testsuite/timeout/timeout.hs
index d466495fe2..f72efe30ae 100644
--- a/testsuite/timeout/timeout.hs
+++ b/testsuite/timeout/timeout.hs
@@ -112,8 +112,8 @@ run secs cmd =
        -- We're explicitly turning off handle inheritance to prevent misc handles
        -- from being inherited by the child. Notable we don't want the I/O Completion
        -- Ports and Job handles to be inherited. So we mark them as non-inheritable.
-       setHandleInformation job cHANDLE_FLAG_INHERIT 0
-       setHandleInformation job cHANDLE_FLAG_INHERIT 0
+       setHandleInformation job    cHANDLE_FLAG_INHERIT 0
+       setHandleInformation ioPort cHANDLE_FLAG_INHERIT 0
 
        -- Now create the process suspended so we can add it to the job and then resume.
        -- This is so we don't miss any events on the receiving end of the I/O port.