node.pyx: Support merging of conditional sequences

During composition of project.conf, it can happen that we
are including and composing node hierarchies before options
are resolved and before we are ready to evaluate conditional
statements.

Without this patch, conditional statements are treated like
regular sequences, causing existing conditional statements to
be overwritten by subsequently composited conditional statements
without ever having been resolved.

This patch introduces a special case for composing sequences,
when composing a sequence that is a conditional statement, the
sequence is appended instead of overwriting the underlying
list.

Note that precedence of declarations in including vs included
yaml fragments did raise some concern, and a test in the following
commit is added to cover this case. Composition of conditional lists
on top of other conditional lists are unconditionally appended,
however the semantics of includes ensures that the including fragment's
conditionals is always composited *on top* over the included fragment,
ensuring that we have the correct pecedence even when compositing
yet to be resolved conditional directives.

1 files changed, 13 insertions, 5 deletions

diff --git a/src/buildstream/node.pyx b/src/buildstream/node.pyx
index 394138311..9a0bd2eb6 100644
--- a/src/buildstream/node.pyx
+++ b/src/buildstream/node.pyx
@@ -214,7 +214,7 @@ cdef class Node:
 
     # _is_composite_list
     #
-    # Checks if the node is a Mapping with array composition
+    # Checks if the node is a Mapping with list composition
     # directives.
     #
     # Returns:
@@ -1070,14 +1070,14 @@ cdef class MappingNode(Node):
         cdef str key
 
         for key in self.value.keys():
-            if key in ['(>)', '(<)', '(=)']:
+            if key in ["(>)", "(<)", "(=)"]:
                 has_directives = True
             else:
                 has_keys = True
 
         if has_keys and has_directives:
             provenance = self.get_provenance()
-            raise LoadError("{}: Dictionary contains array composition directives and arbitrary keys"
+            raise LoadError("{}: Dictionary contains list composition directives and arbitrary keys"
                             .format(provenance), LoadErrorReason.INVALID_DATA)
 
         return has_directives
@@ -1412,8 +1412,16 @@ cdef class SequenceNode(Node):
                                   .format(self.get_provenance(),
                                           key,
                                           target_value.get_provenance()))
-        # Looks good, clobber it
-        target.value[key] = self
+
+        # If the target is a list of conditional statements, then we are
+        # also conditional statements, and we need to append ourselves
+        # to that list instead of overwriting it in order to preserve the
+        # conditional for later evaluation.
+        if type(target_value) is SequenceNode and key == "(?)":
+            (<SequenceNode> target.value[key]).value.extend(self.value)
+        else:
+            # Looks good, clobber it
+            target.value[key] = self
 
     cdef bint _is_composite_list(self) except *:
         return False