tests: New FENCE script command to ensure event order

By default, emitted status text is allowed to match a status matcher
line from anywhere in the script.  Thus, a script like the
following:

    #?*FOO
    #?BAR

will happily accept a sequence of events like this:

    BAR
    *FOO

This loose ordering avoids test flakiness in the presence of
concurrent events, but can be problematic if the test wants to assert
that the BAR event is an effect of the FOO command and not
coincidental.  (A concrete example: assert that a greeter was
launched *because* the user session terminated, not in parallel with
the briefly-lived user session.)

Script authors may be tempted to assert causality by introducing a
WAIT like this:

    #?*WAIT
    #?*FOO
    #?BAR

but that would not have the desired effect because it will still
accept a BAR event before the FOO command (or even the WAIT command)
is executed.

The new FENCE command helps script authors assert causality by
ensuring that an emitted status does not match a line on the other
side of the fence.  For example, the following script works as
expected and asserts a causal relationship between FOO and BAR:

    #?*WAIT
    #?*FENCE
    #?*FOO
    #?BAR

The above script only works if the WAIT is long enough to sufficiently
rule out coincidence.  To see why, note that the following sequence of
events would be accepted by the above script:

    *WAIT
    *FENCE
    BAR
    *FOO

The order of the FENCE and FOO should not be switched, otherwise it
introduces a theoretical race condition: If the BAR event is caused by
the FOO command, and the BAR event happens to arrive before the FENCE
is executed, then the test will fail when it should not.  (Technically
it is not possible to lose the race right now because of how
run_commands is implemented, but scripts should not rely on an
implementation detail.  Besides, it doesn't eliminate the need for the
WAIT.)

This new command will be used in a future commit to add a regression
test for a Wayland multiseat bug.

1 files changed, 30 insertions, 7 deletions

diff --git a/tests/src/test-runner.c b/tests/src/test-runner.c
index c2b11a23..118c786f 100644
--- a/tests/src/test-runner.c
+++ b/tests/src/test-runner.c
@@ -52,8 +52,9 @@ typedef struct
  * concurrency, status messages emitted during testing do not have to appear in
  * the same order as their corresponding status matcher lines in the script.  In
  * effect, the test runner moves event matcher lines up or down to accommodate
- * the actual observed events.  It can move a matcher line up an unlimited
- * amount, but it will not move it down below the next command.  Details:
+ * the actual observed events.  It will not move a matcher line above a previous
+ * FENCE command (if one exists), nor will it move it below the next command
+ * (FENCE or otherwise).  Details:
  *
  *   - When the test code emits a status message, the test runner only considers
  *     the first status matcher line that meets ALL of the following
@@ -66,12 +67,15 @@ typedef struct
  *         prefix.  (Prefix is defined as the characters before the first space,
  *         if any.)
  *
+ *       * The status matcher line is before the next FENCE command, if one
+ *         exists.
+ *
  *     If no such line exists, or its regular expression does not match, the
  *     test fails.
  *
- *     The test runner does not care where the matching status matcher line is
- *     in the script.  The line could even be after a command line that has not
- *     yet executed.
+ *     Other than the above constraints, the test runner does not care where the
+ *     matching status matcher line is in the script.  The line could even be
+ *     after a command line that has not yet executed (except for FENCE).
  *
  *   - A command line will not be executed until every line above it is resolved
  *     (observed or executed, depending on the line type).
@@ -319,10 +323,17 @@ get_prefix (const gchar *text)
 static ScriptLine *
 get_script_line (const gchar *prefix)
 {
+    gboolean stop_at_fence = prefix != NULL;
     for (GList *link = script; link; link = link->next)
     {
         ScriptLine *line = link->data;
 
+        if (line->done)
+            continue;
+
+        if (stop_at_fence && strcmp (line->text, "*FENCE") == 0)
+            break;
+
         /* Ignore lines with other prefixes */
         if (prefix)
         {
@@ -331,8 +342,7 @@ get_script_line (const gchar *prefix)
                 continue;
         }
 
-        if (!line->done)
-            return line;
+        return line;
     }
 
     return NULL;
@@ -510,6 +520,18 @@ handle_command (const gchar *command)
         /* Restart status timeout */
         status_timeout = g_timeout_add (status_timeout_ms, status_timeout_cb, NULL);
     }
+    else if (strcmp (name, "FENCE") == 0)
+    {
+        /*
+         * Nothing special needs to be done here because FENCE's behavior is
+         * implemented elsewhere:
+         *   - run_commands ensures that every status matcher line above a
+         *     command (any command, not just FENCE) has been matched before
+         *     executing the command.
+         *   - When called from check_status, get_script_line stops at the next
+         *     FENCE.
+         */
+    }
     else if (strcmp (name, "ADD-SEAT") == 0)
     {
         const gchar *id = g_hash_table_lookup (params, "ID");
@@ -987,6 +1009,7 @@ run_commands (void)
              * avoid races, don't execute a command until all lines in the
              * script above the command's line are marked as done.  (This
              * function will be called again after the next status arrives.)
+             * The FENCE command in particular relies on this behavior.
              */
             return;