Merge branch 'test-gitlab-ci-ymls' into 'master'

Test templates and GitLabCI parser againts each other

## What does this MR do?

Test the available templates against the preprocessor and vice versa

## Are there points in the code the reviewer needs to double check?

The dynamic creation of tests seems a little hacked. Is there a cleaner way?

## Does this MR meet the acceptance criteria?

- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added -- Seems unneeded
- [x] All builds are passing
- [ ] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [ ] Branch has no merge conflicts with `master` (if you do - rebase it please)

See merge request !4898

1 files changed, 12 insertions, 0 deletions

diff --git a/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb b/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb
index 200ca6aeeea..2ca9f554b07 100644
--- a/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb
+++ b/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb
@@ -1206,5 +1206,17 @@ EOT
         end.to raise_error(GitlabCiYamlProcessor::ValidationError, "rspec job: dependencies parameter should be an array of strings")
       end
     end
+
+    describe "Validate configuration templates" do
+      templates = Dir.glob("#{Rails.root.join('vendor/gitlab-ci-yml')}/**/*.gitlab-ci.yml")
+
+      templates.each do |file|
+        it "does not return errors for #{file}" do
+          file = File.read(file)
+
+          expect { GitlabCiYamlProcessor.new(file) }.not_to raise_error
+        end
+      end
+    end
   end
 end