mtest: warn on invalid TAP output

In commit a7e458effadbc884eacf34528df3a57b60e43fe3 we stopped erroring
out on invalid TAP stream contents, with the rationale that "prove" has
become more lenient.

A close reading of the TAP spec indicates why, though:

> A TAP parser is required to not consider an unknown line as an error but
> may optionally choose to capture said line and hand it to the test
> harness, which may have custom behavior attached. This is to allow for
> forward compatability. Test::Harness silently ignores incorrect lines,
> but will become more stringent in the future. TAP::Harness reports TAP
> syntax errors at the end of a test run.

The goal of treating unknown lines as an error in the TAP parser is not
because unknown lines are fine and dandy. The goal is to allow
implementing future versions of TAP, and handling it via existing
parsers. Since Meson has both a parser and a harness, let's do exactly
that -- pass these lines as a distinctive status to the test harness,
then have the test harness complain.

1 files changed, 4 insertions, 0 deletions

diff --git a/unittests/taptests.py b/unittests/taptests.py
index 477d79794..6c2ccb0ec 100644
--- a/unittests/taptests.py
+++ b/unittests/taptests.py
@@ -37,6 +37,9 @@ class TAPParserTests(unittest.TestCase):
     def assert_error(self, events):
         self.assertEqual(type(next(events)), TAPParser.Error)
 
+    def assert_unexpected(self, events, **kwargs):
+        self.assertEqual(next(events), TAPParser.UnknownLine(**kwargs))
+
     def assert_bailout(self, events, **kwargs):
         self.assertEqual(next(events), TAPParser.Bailout(**kwargs))
 
@@ -255,6 +258,7 @@ class TAPParserTests(unittest.TestCase):
     def test_unexpected(self):
         events = self.parse_tap('1..1\ninvalid\nok 1')
         self.assert_plan(events, num_tests=1, late=False)
+        self.assert_unexpected(events, message='invalid', lineno=2)
         self.assert_test(events, number=1, name='', result=TestResult.OK)
         self.assert_last(events)