diff options
| author | Tyler Ball <tyleraball@gmail.com> | 2014-12-29 15:56:52 -0800 |
|---|---|---|
| committer | Tyler Ball <tyleraball@gmail.com> | 2014-12-29 15:56:52 -0800 |
| commit | 005f75e158ce702bb28fd6199e9654b3d1115d3b (patch) | |
| tree | d049236793b5ce8ade14f594218eac917ed31291 | |
| parent | b7b7dad4e476b3fde67f0d9881e15efe7e5b60ac (diff) | |
| parent | c1676b32aa08b618f4c2317676f5590388b3bc53 (diff) | |
| download | chef-005f75e158ce702bb28fd6199e9654b3d1115d3b.tar.gz | |
Merge pull request #2674 from opscode/audit-mode
Audit mode
55 files changed, 3465 insertions, 96 deletions
diff --git a/.gitignore b/.gitignore index a9e4338e2a..ecba9f4030 100644 --- a/.gitignore +++ b/.gitignore @@ -38,3 +38,6 @@ Berksfile.lock # Vagrant Vagrantfile .vagrant/ + +# Kitchen Tests Local Mode Data +kitchen-tests/nodes/* diff --git a/.kitchen.yml b/.kitchen.yml new file mode 100644 index 0000000000..ed49eb3e57 --- /dev/null +++ b/.kitchen.yml @@ -0,0 +1,82 @@ +driver: + name: vagrant + forward_agent: yes + customize: + cpus: 4 + memory: 4096 + synced_folders: + - ['.', '/home/vagrant/chef'] + +provisioner: + name: chef_zero + require_chef_omnibus: 12.0.0.rc.1 + +platforms: + - name: centos-5.10 + run_list: + - name: centos-6.5 + run_list: + - name: debian-7.2.0 + run_list: + - name: debian-7.4 + run_list: + - name: debian-6.0.8 + run_list: + - name: freebsd-9.2 + run_list: + - name: freebsd-10.0 + run_list: + - name: ubuntu-10.04 + run_list: + - name: ubuntu-12.04 + run_list: + - name: ubuntu-12.10 + run_list: + - name: ubuntu-13.04 + run_list: + - name: ubuntu-13.10 + run_list: + - name: ubuntu-14.04 + run_list: + # The following boxes are shared via VagrantCloud. Until kitchen-vagrant + # is updated you'll need to add the box manually: + # + # vagrant box add chef/windows-8.1-professional + # + # Please note this may require a `vagrant login` if the box is private. + # + # The following boxes are VMware only also. You can enable VMware Fusion + # as the default provider by copying `.kitchen.local.yml.vmware.example` + # over to `.kitchen.local.yml`. + # + - name: macosx-10.8 + driver: + box: chef/macosx-10.8 # private + - name: macosx-10.9 + driver: + box: chef/macosx-10.9 # private + - name: macosx-10.10 + driver: + box: chef/macosx-10.10 # private + # - name: windows-7-professional + # provisioner: + # name: windows_chef_zero + # require_chef_omnibus: 11.12.4 + # driver: + # box: chef/windows-7-professional # private + # - name: windows-8.1-professional + # provisioner: + # name: windows_chef_zero + # require_chef_omnibus: 11.12.4 + # driver: + # box: chef/windows-8.1-professional # private + # - name: windows-2008r2-standard + # provisioner: + # name: windows_chef_zero + # require_chef_omnibus: 11.12.4 + # driver: + # box: chef/windows-server-2008r2-standard # private + +suites: + - name: chef + run_list: diff --git a/CHANGELOG.md b/CHANGELOG.md index a8d9628347..0ea4b68264 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ * Update Chef to use RSpec 3. * Cleaned up script and execute provider + specs * Added deprecation warnings around the use of command attribute in script resources +* Audit mode feature added - see the RELEASE_NOTES for details ## 12.0.3 * [**Phil Dibowitz**](https://github.com/jaymzh): diff --git a/DOC_CHANGES.md b/DOC_CHANGES.md index bcf3a27286..7429baca2a 100644 --- a/DOC_CHANGES.md +++ b/DOC_CHANGES.md @@ -1,3 +1,37 @@ +<!--- +This file is reset every time a new release is done. This file describes changes that have not yet been released. + +Example Doc Change: +### Headline for the required change +Description of the required change. +--> + ### Chef now handles URI Schemes in a case insensitive manner -Previously, when a URI scheme contained all uppercase letters, Chef would reject the URI as invalid. In compliance with RFC3986, Chef now treats URI schemes in a case insensitive manner. This applies to all resources which accept URIs such as remote_file etc.
\ No newline at end of file +Previously, when a URI scheme contained all uppercase letters, Chef would reject the URI as invalid. In compliance with RFC3986, Chef now treats URI schemes in a case insensitive manner. This applies to all resources which accept URIs such as remote_file etc. + +### Experimental Audit Mode Feature + +There is a new command_line flag provided for `chef-client`: `--audit-mode`. This accepts 1 of 3 arguments: + +* `disabled` (default) - Audits are disabled and the phase is skipped. This is the default while Audit mode is an +experimental feature. +* `enabled` - Audits are enabled and will be performed after the converge phase. +* `audit-only` - Audits are enabled and convergence is disabled. Only audits will be performed. + +This can also be configured in your node's client.rb with the key `audit_mode` and a value of `:disabled`, `:enabled` or `:audit_only`. + +### Chef Why Run Mode Ignores Audit Phase + +Because most users enable `why_run` mode to determine what resources convergence will update on their system, the audit +phase is not executed. There is no way to get both `why_run` output and audit output in 1 single command. To get +audit output without performing convergence use the `--audit-mode` flag. + +#### Editors note 1 + +The `--audit-mode` flag should be a link to the documentation for that flag + +#### Editors node 2 + +This probably only needs to be a bullet point added to http://docs.getchef.com/nodes.html#about-why-run-mode under the +`certain assumptions` section diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index a99d44bc56..43c8f06d93 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,7 +1,59 @@ -# Chef Client Release Notes 12.2.0: +# Chef Client Release Notes 12.1.0: # Internal API Changes in this Release +## Experimental Audit Mode Feature + +This is a new feature intended to provide _infrastructure audits_. Chef already allows you to configure your infrastructure +with code, but there are some use cases that are not covered by resource convergence. What if you want to check that +the application Chef just installed is functioning correctly? If it provides a status page an audit can check this +and validate that the application has database connectivity. + +Audits are performed by leveraging [Serverspec](http://serverspec.org/) and [RSpec](https://relishapp.com/rspec) on the +node. As such the syntax is very similar to a normal RSpec spec. + +### Syntax + +```ruby +controls "Database Audit" do + + control "postgres package" do + it "should not be installed" do + expect(package("postgresql")).to_not be_installed + end + end + + let(:p) { port(111) } + control p do + it "has nothing listening" do + expect(p).to_not be_listening + end + end + +end +``` + +Using the example above I will break down the components of an Audit: + +* `controls` - This named block contains all the audits to be performed during the audit phase. During Chef convergence + the audits will be collected and ran in a separate phase at the end of the Chef run. Any `controls` block defined in + a recipe that is ran on the node will be performed. +* `control` - This keyword describes a section of audits to perform. The name here should either be a string describing +the system under test, or a [Serverspec resource](http://serverspec.org/resource_types.html). +* `it` - Inside this block you can use [RSpec expectations](https://relishapp.com/rspec/rspec-expectations/docs) to +write the audits. You can use the Serverspec resources here or regular ruby code. Any raised errors will fail the +audit. + +### Output and error handling + +Output from the audit run will appear in your `Chef::Config[:log_location]`. If an audit fails then Chef will raise +an error and exit with a non-zero status. + +### Further reading + +More information about the audit mode can be found in its +[RFC](https://github.com/opscode/chef-rfc/blob/master/rfc035-audit-mode.md) + # End-User Changes ## OpenBSD Package provider was added diff --git a/chef.gemspec b/chef.gemspec index f623f8bb82..52babdc5a1 100644 --- a/chef.gemspec +++ b/chef.gemspec @@ -35,16 +35,17 @@ Gem::Specification.new do |s| s.add_dependency 'plist', '~> 3.1.0' + # Audit mode requires these, so they are non-developmental dependencies now + %w(rspec-core rspec-expectations rspec-mocks).each { |gem| s.add_dependency gem, "~> 3.1" } + s.add_dependency "rspec_junit_formatter", "~> 0.2.0" + s.add_dependency "serverspec", "~> 2.7" + s.add_dependency "specinfra", "~> 2.10" + s.add_development_dependency "rack" # Rake 10.2 drops Ruby 1.8 support s.add_development_dependency "rake", "~> 10.1.0" - # rspec_junit_formatter 0.2.0 drops ruby 1.8.7 support - s.add_development_dependency "rspec_junit_formatter", "~> 0.2.0" - - %w(rspec-core rspec-expectations rspec-mocks).each { |gem| s.add_development_dependency gem, "~> 3.0" } - s.bindir = "bin" s.executables = %w( chef-client chef-solo knife chef-shell chef-apply ) diff --git a/kitchen-tests/.chef/client.rb b/kitchen-tests/.chef/client.rb index 5eb200a939..98f773d691 100644 --- a/kitchen-tests/.chef/client.rb +++ b/kitchen-tests/.chef/client.rb @@ -1,7 +1,8 @@ -chef_dir = File.expand_path(File.dirame(__FILE__)) -repo_dir = File.expand_path(Fild.join(chef_dir, '..')) +chef_dir = File.expand_path(File.dirname(__FILE__)) +repo_dir = File.expand_path(File.join(chef_dir, '..')) -log_level :info +log_level :info chef_repo_path repo_dir -local_mode true +local_mode true +cache_path "#{ENV['HOME']}/.cache/chef" diff --git a/kitchen-tests/cookbooks/audit_test/.gitignore b/kitchen-tests/cookbooks/audit_test/.gitignore new file mode 100644 index 0000000000..ec2a890bd3 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/.gitignore @@ -0,0 +1,16 @@ +.vagrant +Berksfile.lock +*~ +*# +.#* +\#*# +.*.sw[a-z] +*.un~ + +# Bundler +Gemfile.lock +bin/* +.bundle/* + +.kitchen/ +.kitchen.local.yml diff --git a/kitchen-tests/cookbooks/audit_test/.kitchen.yml b/kitchen-tests/cookbooks/audit_test/.kitchen.yml new file mode 100644 index 0000000000..be11e33081 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/.kitchen.yml @@ -0,0 +1,16 @@ +--- +driver: + name: vagrant + +provisioner: + name: chef_zero + +platforms: + - name: ubuntu-12.04 + - name: centos-6.5 + +suites: + - name: default + run_list: + - recipe[audit_test::default] + attributes: diff --git a/kitchen-tests/cookbooks/audit_test/Berksfile b/kitchen-tests/cookbooks/audit_test/Berksfile new file mode 100644 index 0000000000..0ac9b78cf7 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/Berksfile @@ -0,0 +1,3 @@ +source "https://supermarket.getchef.com" + +metadata diff --git a/kitchen-tests/cookbooks/audit_test/README.md b/kitchen-tests/cookbooks/audit_test/README.md new file mode 100644 index 0000000000..75e2f44808 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/README.md @@ -0,0 +1,12 @@ +# audit_test + +This cookbook has some basic recipes to test audit mode. + +In order to run these tests on your dev box: + +``` +$ bundle install +$ bundle exec chef-client -c kitchen-tests/.chef/client.rb -z -o audit_test::default -l debug +``` + +Expected JSON output for the tests will be printed to `debug` log. diff --git a/kitchen-tests/cookbooks/audit_test/chefignore b/kitchen-tests/cookbooks/audit_test/chefignore new file mode 100644 index 0000000000..80dc2d20ef --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/chefignore @@ -0,0 +1,95 @@ +# Put files/directories that should be ignored in this file when uploading +# or sharing to the community site. +# Lines that start with '# ' are comments. + +# OS generated files # +###################### +.DS_Store +Icon? +nohup.out +ehthumbs.db +Thumbs.db + +# SASS # +######## +.sass-cache + +# EDITORS # +########### +\#* +.#* +*~ +*.sw[a-z] +*.bak +REVISION +TAGS* +tmtags +*_flymake.* +*_flymake +*.tmproj +.project +.settings +mkmf.log + +## COMPILED ## +############## +a.out +*.o +*.pyc +*.so +*.com +*.class +*.dll +*.exe +*/rdoc/ + +# Testing # +########### +.watchr +.rspec +spec/* +spec/fixtures/* +test/* +features/* +Guardfile +Procfile + +# SCM # +####### +.git +*/.git +.gitignore +.gitmodules +.gitconfig +.gitattributes +.svn +*/.bzr/* +*/.hg/* +*/.svn/* + +# Berkshelf # +############# +Berksfile +Berksfile.lock +cookbooks/* +tmp + +# Cookbooks # +############# +CONTRIBUTING + +# Strainer # +############ +Colanderfile +Strainerfile +.colander +.strainer + +# Vagrant # +########### +.vagrant +Vagrantfile + +# Travis # +########## +.travis.yml diff --git a/kitchen-tests/cookbooks/audit_test/metadata.rb b/kitchen-tests/cookbooks/audit_test/metadata.rb new file mode 100644 index 0000000000..4a60104e92 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/metadata.rb @@ -0,0 +1,8 @@ +name 'audit_test' +maintainer 'The Authors' +maintainer_email 'you@example.com' +license 'all_rights' +description 'Installs/Configures audit_test' +long_description 'Installs/Configures audit_test' +version '0.1.0' + diff --git a/kitchen-tests/cookbooks/audit_test/recipes/default.rb b/kitchen-tests/cookbooks/audit_test/recipes/default.rb new file mode 100644 index 0000000000..4f634d73c1 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/recipes/default.rb @@ -0,0 +1,26 @@ +# +# Cookbook Name:: audit_test +# Recipe:: default +# +# Copyright (c) 2014 The Authors, All Rights Reserved. + +controls "basic control group" do + control "basic math" do + it "should pass" do + expect(2 - 2).to eq(0) + end + end +end + +controls "control group without top level control" do + it "should pass" do + expect(2 - 2).to eq(0) + end +end + +controls "control group with empty control" do + control "empty" +end + +controls "empty control group with block" do +end diff --git a/kitchen-tests/cookbooks/audit_test/recipes/error_duplicate_control_groups.rb b/kitchen-tests/cookbooks/audit_test/recipes/error_duplicate_control_groups.rb new file mode 100644 index 0000000000..77a4592e9d --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/recipes/error_duplicate_control_groups.rb @@ -0,0 +1,17 @@ +# +# Cookbook Name:: audit_test +# Recipe:: error_duplicate_control_groups +# +# Copyright (c) 2014 The Authors, All Rights Reserved. + +controls "basic control group" do + it "should pass" do + expect(2 - 2).to eq(0) + end +end + +controls "basic control group" do + it "should pass" do + expect(2 - 2).to eq(0) + end +end diff --git a/kitchen-tests/cookbooks/audit_test/recipes/error_no_block.rb b/kitchen-tests/cookbooks/audit_test/recipes/error_no_block.rb new file mode 100644 index 0000000000..76a8817b5d --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/recipes/error_no_block.rb @@ -0,0 +1,7 @@ +# +# Cookbook Name:: audit_test +# Recipe:: error_no_block +# +# Copyright (c) 2014 The Authors, All Rights Reserved. + +controls "empty control group without block" diff --git a/kitchen-tests/cookbooks/audit_test/recipes/error_orphan_control.rb b/kitchen-tests/cookbooks/audit_test/recipes/error_orphan_control.rb new file mode 100644 index 0000000000..d74acd6c6b --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/recipes/error_orphan_control.rb @@ -0,0 +1,13 @@ +# +# Cookbook Name:: audit_test +# Recipe:: error_orphan_control +# +# Copyright (c) 2014 The Authors, All Rights Reserved. + +controls "basic control group" do + it "should pass" do + expect(2 - 2).to eq(0) + end +end + +control "orphan control" diff --git a/kitchen-tests/cookbooks/audit_test/recipes/failed_specs.rb b/kitchen-tests/cookbooks/audit_test/recipes/failed_specs.rb new file mode 100644 index 0000000000..3225d3983e --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/recipes/failed_specs.rb @@ -0,0 +1,14 @@ +# +# Cookbook Name:: audit_test +# Recipe:: failed_specs +# +# Copyright (c) 2014 The Authors, All Rights Reserved. + +controls "basic control group" do + control "basic math" do + # Can not write a good control :( + it "should pass" do + expect(2 - 0).to eq(0) + end + end +end diff --git a/kitchen-tests/cookbooks/audit_test/recipes/serverspec_collision.rb b/kitchen-tests/cookbooks/audit_test/recipes/serverspec_collision.rb new file mode 100644 index 0000000000..70109d84b8 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/recipes/serverspec_collision.rb @@ -0,0 +1,31 @@ +# +# Cookbook Name:: audit_test +# Recipe:: serverspec_collision +# +# Copyright (c) 2014 The Authors, All Rights Reserved. + +file "/tmp/audit_test_file" do + action :create + content "Welcome to audit mode." +end + +controls "file auditing" do + describe "test file" do + it "says welcome" do + expect(file("/tmp/audit_test_file")).to contain("Welcome") + end + end +end + +file "/tmp/audit_test_file_2" do + action :create + content "Bye to audit mode." +end + +controls "end file auditing" do + describe "end file" do + it "says bye" do + expect(file("/tmp/audit_test_file_2")).to contain("Bye") + end + end +end diff --git a/kitchen-tests/cookbooks/audit_test/recipes/serverspec_support.rb b/kitchen-tests/cookbooks/audit_test/recipes/serverspec_support.rb new file mode 100644 index 0000000000..0396cc0de7 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/recipes/serverspec_support.rb @@ -0,0 +1,37 @@ +# +# Cookbook Name:: audit_test +# Recipe:: serverspec_support +# +# Copyright (c) 2014 The Authors, All Rights Reserved. + +file "/tmp/audit_test_file" do + action :create + content "Welcome to audit mode." +end + +# package "curl" do +# action :install +# end + +controls "serverspec helpers with types" do + control "file helper" do + it "says welcome" do + expect(file("/tmp/audit_test_file")).to contain("Welcome") + end + end + + control service("com.apple.CoreRAID") do + it { is_expected.to be_enabled } + it { is_expected.not_to be_running } + end + + # describe "package helper" do + # it "works" do + # expect(package("curl")).to be_installed + # end + # end + + control package("postgresql") do + it { is_expected.to_not be_installed } + end +end diff --git a/kitchen-tests/cookbooks/audit_test/recipes/with_include_recipe.rb b/kitchen-tests/cookbooks/audit_test/recipes/with_include_recipe.rb new file mode 100644 index 0000000000..ff39cde117 --- /dev/null +++ b/kitchen-tests/cookbooks/audit_test/recipes/with_include_recipe.rb @@ -0,0 +1,16 @@ +# +# Cookbook Name:: audit_test +# Recipe:: with_include_recipe +# +# Copyright (c) 2014 The Authors, All Rights Reserved. + +include_recipe "audit_test::serverspec_collision" + +controls "basic example" do + it "should pass" do + expect(2 - 2).to eq(0) + end +end + +include_recipe "audit_test::serverspec_collision" +include_recipe "audit_test::default" diff --git a/kitchen-tests/cookbooks/webapp/README.md b/kitchen-tests/cookbooks/webapp/README.md index e8de6ee467..f19ab46735 100644 --- a/kitchen-tests/cookbooks/webapp/README.md +++ b/kitchen-tests/cookbooks/webapp/README.md @@ -1,4 +1,3 @@ # webapp TODO: Enter the cookbook description here. - diff --git a/lib/chef/application/client.rb b/lib/chef/application/client.rb index 295dc2470e..40772c0f8f 100644 --- a/lib/chef/application/client.rb +++ b/lib/chef/application/client.rb @@ -239,9 +239,9 @@ class Chef::Application::Client < Chef::Application end option :audit_mode, - :long => "--[no-]audit-mode", - :description => "If not specified, run converge and audit phase. If true, run only audit phase. If false, run only converge phase.", - :boolean => true + :long => "--audit-mode MODE", + :description => "Enable audit-mode with `enabled`. Disable audit-mode with `disabled`. Skip converge and only perform audits with `audit-only`", + :proc => lambda { |mo| mo.gsub("-", "_").to_sym } IMMEDIATE_RUN_SIGNAL = "1".freeze @@ -280,6 +280,19 @@ class Chef::Application::Client < Chef::Application config_fetcher = Chef::ConfigFetcher.new(Chef::Config[:json_attribs]) @chef_client_json = config_fetcher.fetch_json end + + if mode = config[:audit_mode] || Chef::Config[:audit_mode] + expected_modes = [:enabled, :disabled, :audit_only] + unless expected_modes.include?(mode) + Chef::Application.fatal!(unrecognized_audit_mode(mode)) + end + + unless mode == :disabled + # This should be removed when audit-mode is enabled by default/no longer + # an experimental feature. + Chef::Log.warn(audit_mode_experimental_message) + end + end end def load_config_file @@ -400,4 +413,26 @@ class Chef::Application::Client < Chef::Application "#{"\n interval = #{Chef::Config[:interval]} seconds" if Chef::Config[:interval]}" + "\nEnable chef-client interval runs by setting `:client_fork = true` in your config file or adding `--fork` to your command line options." end + + def audit_mode_settings_explaination + "\n* To enable audit mode after converge, use command line option `--audit-mode enabled` or set `:audit_mode = :enabled` in your config file." + + "\n* To disable audit mode, use command line option `--audit-mode disabled` or set `:audit_mode = :disabled` in your config file." + + "\n* To only run audit mode, use command line option `--audit-mode audit-only` or set `:audit_mode = :audit_only` in your config file." + + "\nAudit mode is disabled by default." + end + + def unrecognized_audit_mode(mode) + "Unrecognized setting #{mode} for audit mode." + audit_mode_settings_explaination + end + + def audit_mode_experimental_message + msg = if Chef::Config[:audit_mode] == :audit_only + "Chef-client has been configured to skip converge and run only audits." + else + "Chef-client has been configured to run audits after it converges." + end + msg += " Audit mode is an experimental feature currently under development. API changes may occur. Use at your own risk." + msg += audit_mode_settings_explaination + return msg + end end diff --git a/lib/chef/application/solo.rb b/lib/chef/application/solo.rb index 6e568ddbb1..c3f5444ef7 100644 --- a/lib/chef/application/solo.rb +++ b/lib/chef/application/solo.rb @@ -211,6 +211,9 @@ class Chef::Application::Solo < Chef::Application config_fetcher = Chef::ConfigFetcher.new(Chef::Config[:json_attribs]) @chef_client_json = config_fetcher.fetch_json end + + # Disable auditing for solo + Chef::Config[:audit_mode] = :disabled end def setup_application diff --git a/lib/chef/audit/audit_event_proxy.rb b/lib/chef/audit/audit_event_proxy.rb new file mode 100644 index 0000000000..2512b8bfe2 --- /dev/null +++ b/lib/chef/audit/audit_event_proxy.rb @@ -0,0 +1,93 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +RSpec::Support.require_rspec_core "formatters/base_text_formatter" + +class Chef + class Audit + class AuditEventProxy < ::RSpec::Core::Formatters::BaseFormatter + ::RSpec::Core::Formatters.register self, :stop, :example_group_started + + # TODO I don't like this, but I don't see another way to pass this in + # see rspec files configuration.rb#L671 and formatters.rb#L129 + def self.events=(events) + @@events = events + end + + def events + @@events + end + + def example_group_started(notification) + if notification.group.parent_groups.size == 1 + # top level `controls` block + desc = notification.group.description + Chef::Log.debug("Entered `controls` block named #{desc}") + events.control_group_started(desc) + end + end + + def stop(notification) + Chef::Log.info("Successfully executed all `controls` blocks and contained examples") + notification.examples.each do |example| + control_group_name, control_data = build_control_from(example) + e = example.exception + if e + events.control_example_failure(control_group_name, control_data, e) + else + events.control_example_success(control_group_name, control_data) + end + end + end + + private + + def build_control_from(example) + described_class = example.metadata[:described_class] + if described_class + resource_type = described_class.class.name.split(':')[-1] + resource_name = described_class.name + end + + # The following code builds up the context - the list of wrapping `describe` or `control` blocks + describe_groups = [] + group = example.metadata[:example_group] + # If the innermost block has a resource instead of a string, don't include it in context + describe_groups.unshift(group[:description]) if described_class.nil? + group = group[:parent_example_group] + while !group.nil? + describe_groups.unshift(group[:description]) + group = group[:parent_example_group] + end + + # We know all of our examples each live in a top-level `controls` block - get this name now + outermost_group_desc = describe_groups.shift + + return outermost_group_desc, { + :name => example.description, + :desc => example.full_description, + :resource_type => resource_type, + :resource_name => resource_name, + :context => describe_groups, + :line_number => example.metadata[:line_number] + } + end + + end + end +end diff --git a/lib/chef/audit/audit_reporter.rb b/lib/chef/audit/audit_reporter.rb new file mode 100644 index 0000000000..a5dd9a6c48 --- /dev/null +++ b/lib/chef/audit/audit_reporter.rb @@ -0,0 +1,169 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'chef/event_dispatch/base' +require 'chef/audit/control_group_data' +require 'time' + +class Chef + class Audit + class AuditReporter < EventDispatch::Base + + attr_reader :rest_client, :audit_data, :ordered_control_groups, :run_status + private :rest_client, :audit_data, :ordered_control_groups, :run_status + + PROTOCOL_VERSION = '0.1.1' + + def initialize(rest_client) + @rest_client = rest_client + # Ruby 1.9.3 and above "enumerate their values in the order that the corresponding keys were inserted." + @ordered_control_groups = Hash.new + end + + def run_context + run_status.run_context + end + + def audit_phase_start(run_status) + Chef::Log.debug("Audit Reporter starting") + @audit_data = AuditData.new(run_status.node.name, run_status.run_id) + @run_status = run_status + end + + def audit_phase_complete + Chef::Log.debug("Audit Reporter completed successfully without errors.") + ordered_control_groups.each do |name, control_group| + audit_data.add_control_group(control_group) + end + end + + # If the audit phase failed, its because there was some kind of error in the framework + # that runs tests - normal errors are interpreted as EXAMPLE failures and captured. + # We still want to send available audit information to the server so we process the + # known control groups. + def audit_phase_failed(error) + # The stacktrace information has already been logged elsewhere + Chef::Log.debug("Audit Reporter failed.") + ordered_control_groups.each do |name, control_group| + audit_data.add_control_group(control_group) + end + end + + def run_completed(node) + post_auditing_data + end + + def run_failed(error) + post_auditing_data(error) + end + + def control_group_started(name) + if ordered_control_groups.has_key?(name) + raise Chef::Exceptions::AuditControlGroupDuplicate.new(name) + end + metadata = run_context.audits[name].metadata + ordered_control_groups.store(name, ControlGroupData.new(name, metadata)) + end + + def control_example_success(control_group_name, example_data) + control_group = ordered_control_groups[control_group_name] + control_group.example_success(example_data) + end + + def control_example_failure(control_group_name, example_data, error) + control_group = ordered_control_groups[control_group_name] + control_group.example_failure(example_data, error.message) + end + + # If @audit_enabled is nil or true, we want to run audits + def auditing_enabled? + Chef::Config[:audit_mode] != :disabled + end + + private + + def post_auditing_data(error = nil) + unless auditing_enabled? + Chef::Log.debug("Audit Reports are disabled. Skipping sending reports.") + return + end + + unless run_status + Chef::Log.debug("Run failed before audits were initialized, not sending audit report to server") + return + end + + audit_data.start_time = iso8601ify(run_status.start_time) + audit_data.end_time = iso8601ify(run_status.end_time) + + audit_history_url = "controls" + Chef::Log.debug("Sending audit report (run-id: #{audit_data.run_id})") + run_data = audit_data.to_hash + + if error + run_data[:error] = "#{error.class.to_s}: #{error.message}\n#{error.backtrace.join("\n")}" + end + + Chef::Log.debug "Audit Report:\n#{Chef::JSONCompat.to_json_pretty(run_data)}" + # Since we're posting compressed data we can not directly call post_rest which expects JSON + begin + audit_url = rest_client.create_url(audit_history_url) + rest_client.post(audit_url, run_data, headers) + rescue StandardError => e + if e.respond_to? :response + # 404 error code is OK. This means the version of server we're running against doesn't support + # audit reporting. Don't alarm failure in this case. + if e.response.code == "404" + Chef::Log.debug("Server doesn't support audit reporting. Skipping report.") + return + else + # Save the audit report to local disk + error_file = "failed-audit-data.json" + Chef::FileCache.store(error_file, Chef::JSONCompat.to_json_pretty(run_data), 0640) + Chef::Log.error("Failed to post audit report to server. Saving report to #{Chef::FileCache.load(error_file, false)}") + end + else + Chef::Log.error("Failed to post audit report to server (#{e})") + end + + if Chef::Config[:enable_reporting_url_fatals] + Chef::Log.error("Reporting fatals enabled. Aborting run.") + raise + end + end + end + + def headers(additional_headers = {}) + options = {'X-Ops-Audit-Report-Protocol-Version' => PROTOCOL_VERSION} + options.merge(additional_headers) + end + + def encode_gzip(data) + "".tap do |out| + Zlib::GzipWriter.wrap(StringIO.new(out)){|gz| gz << data } + end + end + + def iso8601ify(time) + time.utc.iso8601.to_s + end + + end + end +end diff --git a/lib/chef/audit/control_group_data.rb b/lib/chef/audit/control_group_data.rb new file mode 100644 index 0000000000..204d7f8070 --- /dev/null +++ b/lib/chef/audit/control_group_data.rb @@ -0,0 +1,140 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'securerandom' + +class Chef + class Audit + class AuditData + attr_reader :node_name, :run_id, :control_groups + attr_accessor :start_time, :end_time + + def initialize(node_name, run_id) + @node_name = node_name + @run_id = run_id + @control_groups = [] + end + + def add_control_group(control_group) + control_groups << control_group + end + + def to_hash + { + :node_name => node_name, + :run_id => run_id, + :start_time => start_time, + :end_time => end_time, + :control_groups => control_groups.collect { |c| c.to_hash } + } + end + end + + class ControlGroupData + attr_reader :name, :status, :number_succeeded, :number_failed, :controls, :metadata + + def initialize(name, metadata={}) + @status = "success" + @controls = [] + @number_succeeded = 0 + @number_failed = 0 + @name = name + @metadata = metadata + end + + + def example_success(control_data) + @number_succeeded += 1 + control = create_control(control_data) + control.status = "success" + controls << control + control + end + + def example_failure(control_data, details) + @number_failed += 1 + @status = "failure" + control = create_control(control_data) + control.details = details if details + control.status = "failure" + controls << control + control + end + + def to_hash + # We sort it so the examples appear in the output in the same order + # they appeared in the recipe + controls.sort! {|x,y| x.line_number <=> y.line_number} + h = { + :name => name, + :status => status, + :number_succeeded => number_succeeded, + :number_failed => number_failed, + :controls => controls.collect { |c| c.to_hash } + } + # If there is a duplicate key, metadata will overwrite it + add_display_only_data(h).merge(metadata) + end + + private + + def create_control(control_data) + ControlData.new(control_data) + end + + # The id and control sequence number are ephemeral data - they are not needed + # to be persisted and can be regenerated at will. They are only needed + # for display purposes. + def add_display_only_data(group) + group[:id] = SecureRandom.uuid + group[:controls].collect!.with_index do |c, i| + # i is zero-indexed, and we want the display one-indexed + c[:sequence_number] = i+1 + c + end + group + end + + end + + class ControlData + attr_reader :name, :resource_type, :resource_name, :context, :line_number + attr_accessor :status, :details + + def initialize(control_data={}) + control_data.each do |k, v| + self.instance_variable_set("@#{k}", v) + end + end + + def to_hash + h = { + :name => name, + :status => status, + :details => details, + :resource_type => resource_type, + :resource_name => resource_name + } + h[:context] = context || [] + h + end + end + + end +end diff --git a/lib/chef/audit/rspec_formatter.rb b/lib/chef/audit/rspec_formatter.rb new file mode 100644 index 0000000000..074a11bed3 --- /dev/null +++ b/lib/chef/audit/rspec_formatter.rb @@ -0,0 +1,37 @@ +# +# Author:: Serdar Sutay (<serdar@chef.io>) +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'rspec/core' + +class Chef + class Audit + class RspecFormatter < RSpec::Core::Formatters::DocumentationFormatter + RSpec::Core::Formatters.register self, :close + + # @api public + # + # Invoked at the very end, `close` allows the formatter to clean + # up resources, e.g. open streams, etc. + # + # @param _notification [NullNotification] (Ignored) + def close(_notification) + # Normally Rspec closes the streams it's given. We don't want it for Chef. + end + end + end +end diff --git a/lib/chef/audit/runner.rb b/lib/chef/audit/runner.rb new file mode 100644 index 0000000000..7ef17a4301 --- /dev/null +++ b/lib/chef/audit/runner.rb @@ -0,0 +1,178 @@ +# +# Author:: Claire McQuin (<claire@getchef.com>) +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +class Chef + class Audit + class Runner + + attr_reader :run_context + private :run_context + + def initialize(run_context) + @run_context = run_context + end + + def run + setup + register_controls + do_run + end + + def failed? + RSpec.world.reporter.failed_examples.size > 0 + end + + def num_failed + RSpec.world.reporter.failed_examples.size + end + + def num_total + RSpec.world.reporter.examples.size + end + + private + # Prepare to run audits: + # - Require files + # - Configure RSpec + # - Configure Specinfra/Serverspec + def setup + require_deps + configure_rspec + configure_specinfra + end + + # RSpec uses a global configuration object, RSpec.configuration. We found + # there was interference between the configuration for audit-mode and + # the configuration for our own spec tests in these cases: + # 1. Specinfra and Serverspec modify RSpec.configuration when loading. + # 2. Setting output/error streams. + # 3. Adding formatters. + # 4. Defining example group aliases. + # + # Moreover, Serverspec loads its DSL methods into the global namespace, + # which causes conflicts with the Chef namespace for resources and packages. + # + # We wait until we're in the audit-phase of the chef-client run to load + # these files. This helps with the namespacing problems we saw, and + # prevents Specinfra and Serverspec from modifying the RSpec configuration + # used by our spec tests. + def require_deps + require 'rspec' + require 'rspec/its' + require 'specinfra' + require 'serverspec/helper' + require 'serverspec/matcher' + require 'serverspec/subject' + require 'chef/audit/audit_event_proxy' + require 'chef/audit/rspec_formatter' + end + + # Configure RSpec just the way we like it: + # - Set location of error and output streams + # - Add custom audit-mode formatters + # - Explicitly disable :should syntax + # - Set :color option according to chef config + # - Disable exposure of global DSL + def configure_rspec + set_streams + add_formatters + disable_should_syntax + + RSpec.configure do |c| + c.color = Chef::Config[:color] + c.expose_dsl_globally = false + end + end + + # Set the error and output streams which audit-mode will use to report + # human-readable audit information. + # + # This should always be called before #add_formatters. RSpec won't allow + # the output stream to be changed for a formatter once the formatter has + # been added. + def set_streams + RSpec.configuration.output_stream = Chef::Config[:log_location] + RSpec.configuration.error_stream = Chef::Config[:log_location] + end + + # Add formatters which we use to + # 1. Output human-readable data to the output stream, + # 2. Collect JSON data to send back to the analytics server. + def add_formatters + RSpec.configuration.add_formatter(Chef::Audit::AuditEventProxy) + RSpec.configuration.add_formatter(Chef::Audit::RspecFormatter) + Chef::Audit::AuditEventProxy.events = run_context.events + end + + # Audit-mode uses RSpec 3. :should syntax is deprecated by default in + # RSpec 3, so we explicitly disable it here. + # + # This can be removed once :should is removed from RSpec. + def disable_should_syntax + RSpec.configure do |config| + config.expect_with :rspec do |c| + c.syntax = :expect + end + end + end + + # Set up the backend for Specinfra/Serverspec. :exec is the local system. + def configure_specinfra + Specinfra.configuration.backend = :exec + end + + # Iterates through the controls registered to this run_context, builds an + # example group (RSpec::Core::ExampleGroup) object per controls, and + # registers the group with the RSpec.world. + # + # We could just store an array of example groups and not use RSpec.world, + # but it may be useful later if we decide to apply our own ordering scheme + # or use example group filters. + def register_controls + add_example_group_methods + run_context.audits.each do |name, group| + ctl_grp = RSpec::Core::ExampleGroup.__controls__(*group.args, &group.block) + RSpec.world.register(ctl_grp) + end + end + + # Add example group method aliases to RSpec. + # + # __controls__: Used internally to create example groups from the controls + # saved in the run_context. + # control: Used within the context of a controls block, like RSpec's + # describe or context. + def add_example_group_methods + RSpec::Core::ExampleGroup.define_example_group_method :__controls__ + RSpec::Core::ExampleGroup.define_example_group_method :control + end + + # Run the audits! + def do_run + # RSpec::Core::Runner wants to be initialized with an + # RSpec::Core::ConfigurationOptions object, which is used to process + # command line configuration arguments. We directly fiddle with the + # internal RSpec configuration object, so we give nil here and let + # RSpec pick up its own configuration and world. + runner = RSpec::Core::Runner.new(nil) + runner.run_specs(RSpec.world.ordered_example_groups) + end + + end + end +end diff --git a/lib/chef/client.rb b/lib/chef/client.rb index 4f37bd0ee3..77f63671d7 100644 --- a/lib/chef/client.rb +++ b/lib/chef/client.rb @@ -25,6 +25,7 @@ require 'chef/log' require 'chef/rest' require 'chef/api_client' require 'chef/api_client/registration' +require 'chef/audit/runner' require 'chef/node' require 'chef/role' require 'chef/file_cache' @@ -43,6 +44,7 @@ require 'chef/formatters/doc' require 'chef/formatters/minimal' require 'chef/version' require 'chef/resource_reporter' +require 'chef/audit/audit_reporter' require 'chef/run_lock' require 'chef/policy_builder' require 'chef/request_id' @@ -209,6 +211,17 @@ class Chef end end + # Resource repoters send event information back to the chef server for processing. + # Can only be called after we have a @rest object + def register_reporters + [ + Chef::ResourceReporter.new(rest), + Chef::Audit::AuditReporter.new(rest) + ].each do |r| + events.register(r) + end + end + # Instantiates a Chef::Node object, possibly loading the node's prior state # when using chef-client. Delegates to policy_builder # @@ -246,7 +259,6 @@ class Chef @policy_builder ||= Chef::PolicyBuilder.strategy.new(node_name, ohai.data, json_attribs, @override_runlist, events) end - def save_updated_node if Chef::Config[:solo] # nothing to do @@ -260,6 +272,7 @@ class Chef def run_ohai ohai.all_plugins + @events.ohai_completed(node) end def node_name @@ -295,8 +308,7 @@ class Chef end # We now have the client key, and should use it from now on. @rest = Chef::REST.new(config[:chef_server_url], client_name, config[:client_key]) - @resource_reporter = Chef::ResourceReporter.new(@rest) - @events.register(@resource_reporter) + register_reporters rescue Exception => e # TODO: munge exception so a semantic failure message can be given to the # user @@ -307,18 +319,56 @@ class Chef # Converges the node. # # === Returns - # true:: Always returns true + # The thrown exception, if there was one. If this returns nil the converge was successful. def converge(run_context) - @events.converge_start(run_context) - Chef::Log.debug("Converging node #{node_name}") - @runner = Chef::Runner.new(run_context) - runner.converge - @events.converge_complete - true - rescue Exception - # TODO: should this be a separate #converge_failed(exception) method? - @events.converge_complete - raise + converge_exception = nil + catch(:end_client_run_early) do + begin + @events.converge_start(run_context) + Chef::Log.debug("Converging node #{node_name}") + @runner = Chef::Runner.new(run_context) + runner.converge + @events.converge_complete + rescue Exception => e + Chef::Log.error("Converge failed with error message #{e.message}") + @events.converge_failed(e) + converge_exception = e + end + end + converge_exception + end + + # We don't want to change the old API on the `converge` method to have it perform + # saving. So we wrap it in this method. + def converge_and_save(run_context) + converge_exception = converge(run_context) + unless converge_exception + begin + save_updated_node + rescue Exception => e + converge_exception = e + end + end + converge_exception + end + + def run_audits(run_context) + audit_exception = nil + begin + @events.audit_phase_start(run_status) + Chef::Log.info("Starting audit phase") + auditor = Chef::Audit::Runner.new(run_context) + auditor.run + if auditor.failed? + raise Chef::Exceptions::AuditsFailed.new(auditor.num_failed, auditor.num_total) + end + @events.audit_phase_complete + rescue Exception => e + Chef::Log.error("Audit phase failed with error message: #{e.message}") + @events.audit_phase_failed(e) + audit_exception = e + end + audit_exception end # Expands the run list. Delegates to the policy_builder. @@ -333,7 +383,6 @@ class Chef policy_builder.expand_run_list end - def do_windows_admin_check if Chef::Platform.windows? Chef::Log.debug("Checking for administrator privileges....") @@ -380,7 +429,7 @@ class Chef Chef::Log.debug("Chef-client request_id: #{request_id}") enforce_path_sanity run_ohai - @events.ohai_completed(node) + register unless Chef::Config[:solo] load_node @@ -396,11 +445,22 @@ class Chef run_context = setup_run_context - catch(:end_client_run_early) do - converge(run_context) + if Chef::Config[:audit_mode] != :audit_only + converge_error = converge_and_save(run_context) end - save_updated_node + if Chef::Config[:why_run] == true + # why_run should probably be renamed to why_converge + Chef::Log.debug("Not running audits in 'why_run' mode - this mode is used to see potential converge changes") + elsif Chef::Config[:audit_mode] != :disabled + audit_error = run_audits(run_context) + end + + if converge_error || audit_error + e = Chef::Exceptions::RunFailedWrappingError.new(converge_error, audit_error) + e.fill_backtrace + raise e + end run_status.stop_clock Chef::Log.info("Chef Run complete in #{run_status.elapsed_time} seconds") @@ -411,6 +471,7 @@ class Chef Chef::Platform::Rebooter.reboot_if_needed!(node) true + rescue Exception => e # CHEF-3336: Send the error first in case something goes wrong below and we don't know why Chef::Log.debug("Re-raising exception: #{e.class} - #{e.message}\n#{e.backtrace.join("\n ")}") diff --git a/lib/chef/config.rb b/lib/chef/config.rb index 4b83a0eca3..453a8f83da 100644 --- a/lib/chef/config.rb +++ b/lib/chef/config.rb @@ -320,6 +320,14 @@ class Chef default :ez, false default :enable_reporting, true default :enable_reporting_url_fatals, false + # Possible values for :audit_mode + # :enabled, :disabled, :audit_only, + # + # TODO: 11 Dec 2014: Currently audit-mode is an experimental feature + # and is disabled by default. When users choose to enable audit-mode, + # a warning is issued in application/client#reconfigure. + # This can be removed when audit-mode is enabled by default. + default :audit_mode, :disabled # Policyfile is an experimental feature where a node gets its run list and # cookbook version set from a single document on the server instead of diff --git a/lib/chef/dsl/audit.rb b/lib/chef/dsl/audit.rb new file mode 100644 index 0000000000..022bbcce01 --- /dev/null +++ b/lib/chef/dsl/audit.rb @@ -0,0 +1,51 @@ +# +# Author:: Tyler Ball (<tball@getchef.com>) +# Copyright:: Copyright (c) 2014 Opscode, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'chef/exceptions' + +class Chef + module DSL + module Audit + + # Can encompass tests in a `control` block or `describe` block + # Adds the controls group and block (containing controls to execute) to the runner's list of pending examples + def controls(*args, &block) + raise Chef::Exceptions::NoAuditsProvided unless block + + name = args[0] + if name.nil? || name.empty? + raise Chef::Exceptions::AuditNameMissing + elsif run_context.audits.has_key?(name) + raise Chef::Exceptions::AuditControlGroupDuplicate.new(name) + end + + # This DSL will only work in the Recipe class because that exposes the cookbook_name + cookbook_name = self.cookbook_name + metadata = { + cookbook_name: cookbook_name, + cookbook_version: self.run_context.cookbook_collection[cookbook_name].version, + recipe_name: self.recipe_name, + line_number: block.source_location[1] + } + + run_context.audits[name] = Struct.new(:args, :block, :metadata).new(args, block, metadata) + end + + end + end +end diff --git a/lib/chef/event_dispatch/base.rb b/lib/chef/event_dispatch/base.rb index 50d261cecd..25dd9fd1b2 100644 --- a/lib/chef/event_dispatch/base.rb +++ b/lib/chef/event_dispatch/base.rb @@ -225,6 +225,41 @@ class Chef def converge_complete end + # Called if the converge phase fails + def converge_failed(exception) + end + + ################################## + # Audit Mode Events + # This phase is currently experimental and these event APIs are subject to change + ################################## + + # Called before audit phase starts + def audit_phase_start(run_status) + end + + # Called when audit phase successfully finishes + def audit_phase_complete + end + + # Called if there is an uncaught exception during the audit phase. The audit runner should + # be catching and handling errors from the examples, so this is only uncaught errors (like + # bugs in our handling code) + def audit_phase_failed(exception) + end + + # Signifies the start of a `controls` block with a defined name + def control_group_started(name) + end + + # An example in a `controls` block completed successfully + def control_example_success(control_group_name, example_data) + end + + # An example in a `controls` block failed with the provided error + def control_example_failure(control_group_name, example_data, error) + end + # TODO: need events for notification resolve? # def notifications_resolved # end diff --git a/lib/chef/exceptions.rb b/lib/chef/exceptions.rb index 0868341849..b949e7b975 100644 --- a/lib/chef/exceptions.rb +++ b/lib/chef/exceptions.rb @@ -385,5 +385,45 @@ class Chef super "Found more than one provider for #{resource.resource_name} resource: #{classes}" end end + + class AuditControlGroupDuplicate < RuntimeError + def initialize(name) + super "Audit control group with name '#{name}' has already been defined" + end + end + class AuditNameMissing < RuntimeError; end + class NoAuditsProvided < RuntimeError + def initialize + super "You must provide a block with audits" + end + end + class AuditsFailed < RuntimeError + def initialize(num_failed, num_total) + super "Audit phase found failures - #{num_failed}/#{num_total} audits failed" + end + end + + # If a converge or audit fails, we want to wrap the output from those errors into 1 error so we can + # see both issues in the output. It is possible that nil will be provided. You must call `fill_backtrace` + # to correctly populate the backtrace with the wrapped backtraces. + class RunFailedWrappingError < RuntimeError + attr_reader :wrapped_errors + def initialize(*errors) + errors = errors.select {|e| !e.nil?} + output = "Found #{errors.size} errors, they are stored in the backtrace" + @wrapped_errors = errors + super output + end + + def fill_backtrace + backtrace = [] + wrapped_errors.each_with_index do |e,i| + backtrace << "#{i+1}) #{e.class} - #{e.message}" + backtrace += e.backtrace if e.backtrace + backtrace << "" + end + set_backtrace(backtrace) + end + end end end diff --git a/lib/chef/formatters/doc.rb b/lib/chef/formatters/doc.rb index 4a08b9d095..489888db8f 100644 --- a/lib/chef/formatters/doc.rb +++ b/lib/chef/formatters/doc.rb @@ -8,7 +8,9 @@ class Chef # "specdoc" class Doc < Formatters::Base - attr_reader :start_time, :end_time + attr_reader :start_time, :end_time, :successful_audits, :failed_audits + private :successful_audits, :failed_audits + cli_name(:doc) def initialize(out, err) @@ -16,6 +18,8 @@ class Chef @updated_resources = 0 @up_to_date_resources = 0 + @successful_audits = 0 + @failed_audits = 0 @start_time = Time.now @end_time = @start_time end @@ -32,12 +36,19 @@ class Chef @up_to_date_resources + @updated_resources end + def total_audits + successful_audits + failed_audits + end + def run_completed(node) @end_time = Time.now if Chef::Config[:why_run] puts_line "Chef Client finished, #{@updated_resources}/#{total_resources} resources would have been updated" else puts_line "Chef Client finished, #{@updated_resources}/#{total_resources} resources updated in #{elapsed_time} seconds" + if total_audits > 0 + puts_line " #{successful_audits}/#{total_audits} Audits succeeded" + end end end @@ -47,6 +58,9 @@ class Chef puts_line "Chef Client failed. #{@updated_resources} resources would have been updated" else puts_line "Chef Client failed. #{@updated_resources} resources updated in #{elapsed_time} seconds" + if total_audits > 0 + puts_line " #{successful_audits} Audits succeeded" + end end end @@ -151,6 +165,38 @@ class Chef unindent if @current_recipe end + def converge_failed(e) + # Currently a failed converge is handled the same way as a successful converge + converge_complete + end + + # Called before audit phase starts + def audit_phase_start(run_status) + puts_line "Starting audit phase" + end + + def audit_phase_complete + puts_line "Auditing complete" + end + + def audit_phase_failed(error) + puts_line "" + puts_line "Audit phase exception:" + indent + puts_line "#{error.message}" + error.backtrace.each do |l| + puts_line l + end + end + + def control_example_success(control_group_name, example_data) + @successful_audits += 1 + end + + def control_example_failure(control_group_name, example_data, error) + @failed_audits += 1 + end + # Called before action is executed on a resource. def resource_action_start(resource, action, notification_type=nil, notifier=nil) if resource.cookbook_name && resource.recipe_name diff --git a/lib/chef/monologger.rb b/lib/chef/monologger.rb index 464b21bdd3..f7d226f82e 100644 --- a/lib/chef/monologger.rb +++ b/lib/chef/monologger.rb @@ -1,5 +1,4 @@ require 'logger' - require 'pp' #== MonoLogger @@ -89,4 +88,3 @@ class MonoLogger < Logger end - diff --git a/lib/chef/recipe.rb b/lib/chef/recipe.rb index e54a1d98e3..621d93099b 100644 --- a/lib/chef/recipe.rb +++ b/lib/chef/recipe.rb @@ -24,6 +24,7 @@ require 'chef/dsl/platform_introspection' require 'chef/dsl/include_recipe' require 'chef/dsl/registry_helper' require 'chef/dsl/reboot_pending' +require 'chef/dsl/audit' require 'chef/mixin/from_file' @@ -40,6 +41,7 @@ class Chef include Chef::DSL::Recipe include Chef::DSL::RegistryHelper include Chef::DSL::RebootPending + include Chef::DSL::Audit include Chef::Mixin::FromFile include Chef::Mixin::Deprecation diff --git a/lib/chef/run_context.rb b/lib/chef/run_context.rb index 22679822a4..6803dc5796 100644 --- a/lib/chef/run_context.rb +++ b/lib/chef/run_context.rb @@ -50,6 +50,9 @@ class Chef # recipes, which is triggered by #load. (See also: CookbookCompiler) attr_accessor :resource_collection + # The list of audits (control groups) to execute during the audit phase + attr_accessor :audits + # A Hash containing the immediate notifications triggered by resources # during the converge phase of the chef run. attr_accessor :immediate_notification_collection @@ -73,6 +76,7 @@ class Chef @node = node @cookbook_collection = cookbook_collection @resource_collection = Chef::ResourceCollection.new + @audits = {} @immediate_notification_collection = Hash.new {|h,k| h[k] = []} @delayed_notification_collection = Hash.new {|h,k| h[k] = []} @definitions = Hash.new diff --git a/spec/functional/audit/rspec_formatter_spec.rb b/spec/functional/audit/rspec_formatter_spec.rb new file mode 100644 index 0000000000..43d3c2f6dd --- /dev/null +++ b/spec/functional/audit/rspec_formatter_spec.rb @@ -0,0 +1,53 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# Author:: Claire McQuin (<claire@getchef.com>) +# +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'spec_helper' +require 'spec/support/audit_helper' +require 'chef/audit/runner' +require 'rspec/support/spec/in_sub_process' +require 'chef/audit/rspec_formatter' + +describe Chef::Audit::RspecFormatter do + include RSpec::Support::InSubProcess + + let(:events) { double("events").as_null_object } + let(:audits) { {} } + let(:run_context) { instance_double(Chef::RunContext, :events => events, :audits => audits) } + let(:runner) { Chef::Audit::Runner.new(run_context) } + + let(:output) { double("output") } + # aggressively define this so we can mock out the new call later + let!(:formatter) { Chef::Audit::RspecFormatter.new(output) } + + around(:each) do |ex| + Sandboxing.sandboxed { ex.run } + end + + it "should not close the output using our formatter" do + in_sub_process do + expect_any_instance_of(Chef::Audit::RspecFormatter).to receive(:new).and_return(formatter) + expect(formatter).to receive(:close).and_call_original + expect(output).to_not receive(:close) + + runner.run + end + end + +end diff --git a/spec/functional/audit/runner_spec.rb b/spec/functional/audit/runner_spec.rb new file mode 100644 index 0000000000..aa35548f2f --- /dev/null +++ b/spec/functional/audit/runner_spec.rb @@ -0,0 +1,136 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'spec_helper' +require 'spec/support/audit_helper' +require 'chef/audit/runner' +require 'rspec/support/spec/in_sub_process' +require 'tempfile' + +## +# This functional test ensures that our runner can be setup to not interfere with existing RSpec +# configuration and world objects. When normally running Chef, there is only 1 RSpec instance +# so this isn't needed. In unit testing the Runner should be mocked appropriately. + +describe Chef::Audit::Runner do + + # The functional tests must be run in a sub_process. Including Serverspec includes the Serverspec DSL - this + # conflicts with our `package` DSL (among others) when we try to test `package` inside an RSpec example. + # Our DSL leverages `method_missing` while the Serverspec DSL defines a method on the RSpec::Core::ExampleGroup. + # The defined method wins our and returns before our `method_missing` DSL can be called. + # + # Running in a sub_process means the serverspec libraries will only be included in a forked process, not the main one. + include RSpec::Support::InSubProcess + + let(:events) { double("events").as_null_object } + let(:runner) { Chef::Audit::Runner.new(run_context) } + let(:stdout) { StringIO.new } + + around(:each) do |ex| + Sandboxing.sandboxed { ex.run } + end + + before do + Chef::Config[:log_location] = stdout + end + + describe "#run" do + + let(:audits) { {} } + let(:run_context) { instance_double(Chef::RunContext, :events => events, :audits => audits) } + let(:controls_name) { "controls_name" } + + it "Correctly runs an empty controls block" do + in_sub_process do + runner.run + end + end + + shared_context "passing audit" do + let(:audits) do + should_pass = lambda do + it "should pass" do + expect(2 - 2).to eq(0) + end + end + { controls_name => Struct.new(:args, :block).new([controls_name], should_pass)} + end + end + + shared_context "failing audit" do + let(:audits) do + should_fail = lambda do + it "should fail" do + expect(2 - 1).to eq(0) + end + end + { controls_name => Struct.new(:args, :block).new([controls_name], should_fail)} + end + end + + context "there is a single successful control" do + include_context "passing audit" + it "correctly runs" do + in_sub_process do + runner.run + + expect(stdout.string).to match(/1 example, 0 failures/) + end + end + end + + context "there is a single failing control" do + include_context "failing audit" + it "correctly runs" do + in_sub_process do + runner.run + + expect(stdout.string).to match(/Failure\/Error: expect\(2 - 1\)\.to eq\(0\)/) + expect(stdout.string).to match(/1 example, 1 failure/) + expect(stdout.string).to match(/# controls_name should fail/) + end + end + end + + describe "log location is a file" do + let(:tmpfile) { Tempfile.new("audit") } + before do + Chef::Config[:log_location] = tmpfile.path + end + + after do + tmpfile.close + tmpfile.unlink + end + + include_context "failing audit" + it "correctly runs" do + in_sub_process do + runner.run + + contents = tmpfile.read + expect(contents).to match(/Failure\/Error: expect\(2 - 1\)\.to eq\(0\)/) + expect(contents).to match(/1 example, 1 failure/) + expect(contents).to match(/# controls_name should fail/) + end + end + end + + end + +end diff --git a/spec/functional/resource/deploy_revision_spec.rb b/spec/functional/resource/deploy_revision_spec.rb index 7bc3da9a05..e5f5341fcd 100644 --- a/spec/functional/resource/deploy_revision_spec.rb +++ b/spec/functional/resource/deploy_revision_spec.rb @@ -45,11 +45,10 @@ describe Chef::Resource::DeployRevision, :unix_only => true do before(:all) do @ohai = Ohai::System.new - @ohai.all_plugins("os") + @ohai.all_plugins(["platform", "os"]) end let(:node) do - Chef::Node.new.tap do |n| n.name "rspec-test" n.consume_external_attrs(@ohai.data, {}) diff --git a/spec/functional/resource/git_spec.rb b/spec/functional/resource/git_spec.rb index 4f462b7cb6..9d3b82f19e 100644 --- a/spec/functional/resource/git_spec.rb +++ b/spec/functional/resource/git_spec.rb @@ -92,7 +92,7 @@ E before(:all) do @ohai = Ohai::System.new - @ohai.all_plugins("os") + @ohai.all_plugins(["platform", "os"]) end context "working with pathes with special characters" do diff --git a/spec/integration/client/client_spec.rb b/spec/integration/client/client_spec.rb index f4bb124781..62660bb852 100644 --- a/spec/integration/client/client_spec.rb +++ b/spec/integration/client/client_spec.rb @@ -239,4 +239,44 @@ EOM end end + + when_the_repository "has a cookbook with only an audit recipe" do + + before do + file 'config/client.rb', <<EOM +local_mode true +cookbook_path "#{path_to('cookbooks')}" +audit_mode :enabled +EOM + end + + it "should exit with a zero code when there is not an audit failure" do + file 'cookbooks/audit_test/recipes/succeed.rb', <<-RECIPE +controls "control group without top level control" do + it "should succeed" do + expect(2 - 2).to eq(0) + end +end + RECIPE + + result = shell_out("#{chef_client} -c \"#{path_to('config/client.rb')}\" -o 'audit_test::succeed'", :cwd => chef_dir) + expect(result.error?).to be_falsey + expect(result.stdout).to include("Successfully executed all `controls` blocks and contained examples") + end + + it "should exit with a non-zero code when there is an audit failure" do + file 'cookbooks/audit_test/recipes/fail.rb', <<-RECIPE +controls "control group without top level control" do + it "should fail" do + expect(2 - 2).to eq(1) + end +end + RECIPE + + result = shell_out("#{chef_client} -c \"#{path_to('config/client.rb')}\" -o 'audit_test::fail'", :cwd => chef_dir) + expect(result.error?).to be_truthy + expect(result.stdout).to include("Failure/Error: expect(2 - 2).to eq(1)") + end + end + end diff --git a/spec/support/audit_helper.rb b/spec/support/audit_helper.rb new file mode 100644 index 0000000000..8fd3f4d719 --- /dev/null +++ b/spec/support/audit_helper.rb @@ -0,0 +1,65 @@ +# This code comes from https://github.com/rspec/rspec-core/blob/master/spec/spec_helper.rb and +# https://github.com/rspec/rspec-core/blob/master/spec/support/sandboxing.rb + +# To leverage the sandboxing use an `around` block: +# around(:each) do |ex| +# Sandboxing.sandboxed { ex.run } +# end + +# rspec-core did not include a license on Github +# TODO when this API is exposed publicly from rspec-core, get rid of this copy pasta + +# Adding these as writers is necessary, otherwise we cannot set the new configuration. +# Only want to do this in the specs. +class << RSpec + attr_writer :configuration, :world +end + +class NullObject + private + def method_missing(method, *args, &block) + # ignore + end +end + +# TODO remove this when RSPec exposes this functionality publically +# https://github.com/rspec/rspec-core/pull/1808 +module Sandboxing + def self.sandboxed(&block) + orig_load_path = $LOAD_PATH.dup + orig_config = RSpec.configuration + orig_world = RSpec.world + orig_example = RSpec.current_example + new_config = RSpec::Core::Configuration.new + new_config.expose_dsl_globally = false + new_config.expecting_with_rspec = true + new_world = RSpec::Core::World.new(new_config) + RSpec.configuration = new_config + RSpec.world = new_world + object = Object.new + object.extend(RSpec::Core::SharedExampleGroup) + + (class << RSpec::Core::ExampleGroup; self; end).class_exec do + alias_method :orig_run, :run + def run(reporter=nil) + RSpec.current_example = nil + orig_run(reporter || NullObject.new) + end + end + + RSpec::Mocks.with_temporary_scope do + object.instance_exec(&block) + end + ensure + (class << RSpec::Core::ExampleGroup; self; end).class_exec do + remove_method :run + alias_method :run, :orig_run + remove_method :orig_run + end + + RSpec.configuration = orig_config + RSpec.world = orig_world + RSpec.current_example = orig_example + $LOAD_PATH.replace(orig_load_path) + end +end diff --git a/spec/unit/application/client_spec.rb b/spec/unit/application/client_spec.rb index c2d3ec0507..33af9bc5c1 100644 --- a/spec/unit/application/client_spec.rb +++ b/spec/unit/application/client_spec.rb @@ -18,18 +18,20 @@ require 'spec_helper' describe Chef::Application::Client, "reconfigure" do + let(:app) do + a = described_class.new + a.cli_arguments = [] + a + end + before do allow(Kernel).to receive(:trap).and_return(:ok) @original_argv = ARGV.dup ARGV.clear - @app = Chef::Application::Client.new - allow(@app).to receive(:trap) - allow(@app).to receive(:configure_opt_parser).and_return(true) - allow(@app).to receive(:configure_chef).and_return(true) - allow(@app).to receive(:configure_logging).and_return(true) - @app.cli_arguments = [] + allow(app).to receive(:trap) + allow(app).to receive(:configure_logging).and_return(true) Chef::Config[:interval] = 10 Chef::Config[:once] = false @@ -60,7 +62,7 @@ Configuration settings: interval = 600 seconds Enable chef-client interval runs by setting `:client_fork = true` in your config file or adding `--fork` to your command line options." ) - @app.reconfigure + app.reconfigure end end @@ -72,7 +74,7 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config it "should not terminate" do expect(Chef::Application).not_to receive(:fatal!) - @app.reconfigure + app.reconfigure end end @@ -83,7 +85,7 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config end it "should reconfigure chef-client" do - @app.reconfigure + app.reconfigure expect(Chef::Config[:interval]).to be_nil end end @@ -96,7 +98,7 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config end it "should set the interval to 1800" do - @app.reconfigure + app.reconfigure expect(Chef::Config.interval).to eq(1800) end end @@ -110,12 +112,12 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config end it "ignores the splay" do - @app.reconfigure + app.reconfigure expect(Chef::Config.splay).to be_nil end it "forces the interval to nil" do - @app.reconfigure + app.reconfigure expect(Chef::Config.interval).to be_nil end @@ -128,14 +130,89 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config let(:json_source) { "https://foo.com/foo.json" } before do + allow(app).to receive(:configure_chef).and_return(true) Chef::Config[:json_attribs] = json_source expect(Chef::ConfigFetcher).to receive(:new).with(json_source). and_return(config_fetcher) end it "reads the JSON attributes from the specified source" do - @app.reconfigure - expect(@app.chef_client_json).to eq(json_attribs) + app.reconfigure + expect(app.chef_client_json).to eq(json_attribs) + end + end + + describe "audit mode" do + shared_examples "experimental feature" do + before do + allow(Chef::Log).to receive(:warn) + end + + it "emits a warning that audit mode is an experimental feature" do + expect(Chef::Log).to receive(:warn).with(/Audit mode is an experimental feature/) + app.reconfigure + end + end + + shared_examples "unrecognized setting" do + it "fatals with a message including the incorrect setting" do + expect(Chef::Application).to receive(:fatal!).with(/Unrecognized setting #{mode} for audit mode/) + app.reconfigure + end + end + + shared_context "set via config file" do + before do + Chef::Config[:audit_mode] = mode + end + end + + shared_context "set via command line" do + before do + ARGV.replace(["--audit-mode", mode]) + end + end + + describe "enabled via config file" do + include_context "set via config file" do + let(:mode) { :enabled } + include_examples "experimental feature" + end + end + + describe "enabled via command line" do + include_context "set via command line" do + let(:mode) { "enabled" } + include_examples "experimental feature" + end + end + + describe "audit_only via config file" do + include_context "set via config file" do + let(:mode) { :audit_only } + include_examples "experimental feature" + end + end + + describe "audit-only via command line" do + include_context "set via command line" do + let(:mode) { "audit-only" } + include_examples "experimental feature" + end + end + + describe "unrecognized setting via config file" do + include_context "set via config file" do + let(:mode) { :derp } + include_examples "unrecognized setting" + end + end + + describe "unrecognized setting via command line" do + include_context "set via command line" do + let(:mode) { "derp" } + include_examples "unrecognized setting" + end end end end diff --git a/spec/unit/application/solo_spec.rb b/spec/unit/application/solo_spec.rb index 80f0bead8b..2a07ff38ad 100644 --- a/spec/unit/application/solo_spec.rb +++ b/spec/unit/application/solo_spec.rb @@ -18,13 +18,16 @@ require 'spec_helper' describe Chef::Application::Solo do + + let(:app) { Chef::Application::Solo.new } + before do allow(Kernel).to receive(:trap).and_return(:ok) - @app = Chef::Application::Solo.new - allow(@app).to receive(:configure_opt_parser).and_return(true) - allow(@app).to receive(:configure_chef).and_return(true) - allow(@app).to receive(:configure_logging).and_return(true) - allow(@app).to receive(:trap) + allow(app).to receive(:configure_opt_parser).and_return(true) + allow(app).to receive(:configure_chef).and_return(true) + allow(app).to receive(:configure_logging).and_return(true) + allow(app).to receive(:trap) + Chef::Config[:recipe_url] = false Chef::Config[:json_attribs] = false Chef::Config[:solo] = true @@ -32,10 +35,15 @@ describe Chef::Application::Solo do describe "configuring the application" do it "should set solo mode to true" do - @app.reconfigure + app.reconfigure expect(Chef::Config[:solo]).to be_truthy end + it "should set audit-mode to :disabled" do + app.reconfigure + expect(Chef::Config[:audit_mode]).to be :disabled + end + describe "when configured to not fork the client process" do before do Chef::Config[:client_fork] = false @@ -56,7 +64,7 @@ Configuration settings: interval = 600 seconds Enable chef-client interval runs by setting `:client_fork = true` in your config file or adding `--fork` to your command line options." ) - @app.reconfigure + app.reconfigure end end end @@ -68,7 +76,7 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config it "should set the interval to 1800" do Chef::Config[:interval] = nil - @app.reconfigure + app.reconfigure expect(Chef::Config[:interval]).to eq(1800) end end @@ -85,44 +93,46 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config end it "reads the JSON attributes from the specified source" do - @app.reconfigure - expect(@app.chef_client_json).to eq(json_attribs) + app.reconfigure + expect(app.chef_client_json).to eq(json_attribs) end end describe "when the recipe_url configuration option is specified" do + let(:tarfile) { StringIO.new("remote_tarball_content") } + let(:target_file) { StringIO.new } + before do Chef::Config[:cookbook_path] = "#{Dir.tmpdir}/chef-solo/cookbooks" Chef::Config[:recipe_url] = "http://junglist.gen.nz/recipes.tgz" + allow(FileUtils).to receive(:rm_rf).and_return(true) allow(FileUtils).to receive(:mkdir_p).and_return(true) - @tarfile = StringIO.new("remote_tarball_content") - allow(@app).to receive(:open).with("http://junglist.gen.nz/recipes.tgz").and_yield(@tarfile) - @target_file = StringIO.new - allow(File).to receive(:open).with("#{Dir.tmpdir}/chef-solo/recipes.tgz", "wb").and_yield(@target_file) + allow(app).to receive(:open).with("http://junglist.gen.nz/recipes.tgz").and_yield(tarfile) + allow(File).to receive(:open).with("#{Dir.tmpdir}/chef-solo/recipes.tgz", "wb").and_yield(target_file) allow(Chef::Mixin::Command).to receive(:run_command).and_return(true) end it "should create the recipes path based on the parent of the cookbook path" do expect(FileUtils).to receive(:mkdir_p).with("#{Dir.tmpdir}/chef-solo").and_return(true) - @app.reconfigure + app.reconfigure end it "should download the recipes" do - expect(@app).to receive(:open).with("http://junglist.gen.nz/recipes.tgz").and_yield(@tarfile) - @app.reconfigure + expect(app).to receive(:open).with("http://junglist.gen.nz/recipes.tgz").and_yield(tarfile) + app.reconfigure end it "should write the recipes to the target path" do - @app.reconfigure - expect(@target_file.string).to eq("remote_tarball_content") + app.reconfigure + expect(target_file.string).to eq("remote_tarball_content") end it "should untar the target file to the parent of the cookbook path" do expect(Chef::Mixin::Command).to receive(:run_command).with({:command => "tar zxvf #{Dir.tmpdir}/chef-solo/recipes.tgz -C #{Dir.tmpdir}/chef-solo"}).and_return(true) - @app.reconfigure + app.reconfigure end end end @@ -142,9 +152,9 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config end it "should fetch the recipe_url first" do - expect(@app).to receive(:fetch_recipe_tarball).ordered + expect(app).to receive(:fetch_recipe_tarball).ordered expect(Chef::ConfigFetcher).to receive(:new).ordered.and_return(config_fetcher) - @app.reconfigure + app.reconfigure end end @@ -153,18 +163,17 @@ Enable chef-client interval runs by setting `:client_fork = true` in your config Chef::Config[:solo] = true allow(Chef::Daemon).to receive(:change_privilege) - @chef_client = double("Chef::Client") - allow(Chef::Client).to receive(:new).and_return(@chef_client) - @app = Chef::Application::Solo.new + chef_client = double("Chef::Client") + allow(Chef::Client).to receive(:new).and_return(chef_client) # this is all stuff the reconfigure method needs - allow(@app).to receive(:configure_opt_parser).and_return(true) - allow(@app).to receive(:configure_chef).and_return(true) - allow(@app).to receive(:configure_logging).and_return(true) + allow(app).to receive(:configure_opt_parser).and_return(true) + allow(app).to receive(:configure_chef).and_return(true) + allow(app).to receive(:configure_logging).and_return(true) end it "should change privileges" do expect(Chef::Daemon).to receive(:change_privilege).and_return(true) - @app.setup_application + app.setup_application end end diff --git a/spec/unit/audit/audit_event_proxy_spec.rb b/spec/unit/audit/audit_event_proxy_spec.rb new file mode 100644 index 0000000000..899ba468b1 --- /dev/null +++ b/spec/unit/audit/audit_event_proxy_spec.rb @@ -0,0 +1,311 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# Author:: Claire McQuin (<claire@getchef.com>) +# +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'spec_helper' +require 'chef/audit/audit_event_proxy' + +describe Chef::Audit::AuditEventProxy do + + let(:stdout) { StringIO.new } + let(:events) { double("Chef::Events") } + let(:audit_event_proxy) { Chef::Audit::AuditEventProxy.new(stdout) } + + before do + Chef::Audit::AuditEventProxy.events = events + end + + describe "#example_group_started" do + + let(:description) { "poots" } + let(:group) { double("ExampleGroup", :parent_groups => parents, + :description => description) } + let(:notification) { double("Notification", :group => group) } + + context "when notified from a top-level example group" do + + let(:parents) { [double("ExampleGroup")] } + + it "notifies control_group_started event" do + expect(Chef::Log).to receive(:debug). + with("Entered \`controls\` block named poots") + expect(events).to receive(:control_group_started). + with(description) + audit_event_proxy.example_group_started(notification) + end + end + + context "when notified from an inner-level example group" do + + let(:parents) { [double("ExampleGroup"), double("OuterExampleGroup")] } + + it "does nothing" do + expect(events).to_not receive(:control_group_started) + audit_event_proxy.example_group_started(notification) + end + end + end + + describe "#stop" do + + let(:examples) { [] } + let(:notification) { double("Notification", :examples => examples) } + let(:exception) { nil } + let(:example) { double("Example", :exception => exception) } + let(:control_group_name) { "audit test" } + let(:control_data) { double("ControlData") } + + before do + allow(Chef::Log).to receive(:info) # silence messages to output stream + end + + it "sends a message that audits completed" do + expect(Chef::Log).to receive(:info).with("Successfully executed all \`controls\` blocks and contained examples") + audit_event_proxy.stop(notification) + end + + context "when an example succeeded" do + + let(:examples) { [example] } + let(:excpetion) { nil } + + before do + allow(audit_event_proxy).to receive(:build_control_from). + with(example). + and_return([control_group_name, control_data]) + end + + it "notifies events" do + expect(events).to receive(:control_example_success). + with(control_group_name, control_data) + audit_event_proxy.stop(notification) + end + end + + context "when an example failed" do + + let(:examples) { [example] } + let(:exception) { double("ExpectationNotMet") } + + before do + allow(audit_event_proxy).to receive(:build_control_from). + with(example). + and_return([control_group_name, control_data]) + end + + it "notifies events" do + expect(events).to receive(:control_example_failure). + with(control_group_name, control_data, exception) + audit_event_proxy.stop(notification) + end + end + + describe "#build_control_from" do + + let(:examples) { [example] } + + let(:example) { double("Example", :metadata => metadata, + :description => example_description, + :full_description => full_description, :exception => nil) } + + let(:metadata) { + { + :described_class => described_class, + :example_group => example_group, + :line_number => line + } + } + + let(:example_group) { + { + :description => group_description, + :parent_example_group => parent_group + } + } + + let(:parent_group) { + { + :description => control_group_name, + :parent_example_group => nil + } + } + + let(:line) { 27 } + + let(:control_data) { + { + :name => example_description, + :desc => full_description, + :resource_type => resource_type, + :resource_name => resource_name, + :context => context, + :line_number => line + } + } + + shared_examples "built control" do + + before do + if described_class + allow(described_class).to receive(:instance_variable_get). + with(:@name). + and_return(resource_name) + allow(described_class.class).to receive(:name). + and_return(described_class.class) + end + end + + it "returns the controls block name and example metadata for reporting" do + expect(events).to receive(:control_example_success). + with(control_group_name, control_data) + audit_event_proxy.stop(notification) + end + end + + describe "a top-level example" do + # controls "port 111" do + # it "has nobody listening" do + # expect(port("111")).to_not be_listening + # end + # end + + # Description parts + let(:group_description) { "port 111" } + let(:example_description) { "has nobody listening" } + let(:full_description) { group_description + " " + example_description } + + # Metadata fields + let(:described_class) { nil } + + # Example group (metadata[:example_group]) fields + let(:parent_group) { nil } + + # Expected returns + let(:control_group_name) { group_description } + + # Control data fields + let(:resource_type) { nil } + let(:resource_name) { nil } + let(:context) { [] } + + include_examples "built control" + end + + describe "an example with an implicit subject" do + # controls "application ports" do + # control port(111) do + # it { is_expected.to_not be_listening } + # end + # end + + # Description parts + let(:control_group_name) { "application ports" } + let(:group_description) { "#{resource_type} #{resource_name}" } + let(:example_description) { "should not be listening" } + let(:full_description) { [control_group_name, group_description, + example_description].join(" ") } + + # Metadata fields + let(:described_class) { double("Serverspec::Type::Port", + :class => "Serverspec::Type::Port", :name => resource_name) } + + # Control data fields + let(:resource_type) { "Port" } + let(:resource_name) { "111" } + let(:context) { [] } + + include_examples "built control" + end + + describe "an example in a nested context" do + # controls "application ports" do + # control "port 111" do + # it "is not listening" do + # expect(port(111)).to_not be_listening + # end + # end + # end + + # Description parts + let(:control_group_name) { "application ports" } + let(:group_description) { "port 111" } + let(:example_description) { "is not listening" } + let(:full_description) { [control_group_name, group_description, + example_description].join(" ") } + + # Metadata fields + let(:described_class) { nil } + + # Control data fields + let(:resource_type) { nil } + let(:resource_name) { nil } + let(:context) { [group_description] } + + include_examples "built control" + end + + describe "an example in a nested context including Serverspec" do + # controls "application directory" do + # control file("/tmp/audit") do + # describe file("/tmp/audit/test_file") do + # it "is a file" do + # expect(subject).to be_file + # end + # end + # end + # end + + # Description parts + let(:control_group_name) { "application directory" } + let(:outer_group_description) { "File \"tmp/audit\"" } + let(:group_description) { "#{resource_type} #{resource_name}" } + let(:example_description) { "is a file" } + let(:full_description) { [control_group_name, outer_group_description, + group_description, example_description].join(" ") } + + # Metadata parts + let(:described_class) { double("Serverspec::Type::File", + :class => "Serverspec::Type::File", :name => resource_name) } + + # Example group parts + let(:parent_group) { + { + :description => outer_group_description, + :parent_example_group => control_group + } + } + + let(:control_group) { + { + :description => control_group_name, + :parent_example_group => nil + } + } + + # Control data parts + let(:resource_type) { "File" } + let(:resource_name) { "/tmp/audit/test_file" } + let(:context) { [outer_group_description] } + + include_examples "built control" + end + end + end + +end diff --git a/spec/unit/audit/audit_reporter_spec.rb b/spec/unit/audit/audit_reporter_spec.rb new file mode 100644 index 0000000000..84d7ea82f0 --- /dev/null +++ b/spec/unit/audit/audit_reporter_spec.rb @@ -0,0 +1,393 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# Author:: Claire McQuin (<claire@getchef.com>) +# +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'spec_helper' + +describe Chef::Audit::AuditReporter do + + let(:rest) { double("rest") } + let(:reporter) { described_class.new(rest) } + let(:node) { double("node", :name => "sofreshsoclean") } + let(:run_id) { 0 } + let(:start_time) { Time.new(2014, 12, 3, 9, 31, 05, "-08:00") } + let(:end_time) { Time.new(2014, 12, 3, 9, 36, 14, "-08:00") } + let(:run_status) { instance_double(Chef::RunStatus, :node => node, :run_id => run_id, + :start_time => start_time, :end_time => end_time) } + + describe "#audit_phase_start" do + + it "notifies audit phase start to debug log" do + expect(Chef::Log).to receive(:debug).with(/Audit Reporter starting/) + reporter.audit_phase_start(run_status) + end + + it "initializes an AuditData object" do + expect(Chef::Audit::AuditData).to receive(:new).with(run_status.node.name, run_status.run_id) + reporter.audit_phase_start(run_status) + end + + it "saves the run status" do + reporter.audit_phase_start(run_status) + expect(reporter.instance_variable_get(:@run_status)).to eq run_status + end + end + + describe "#run_completed" do + + let(:audit_data) { Chef::Audit::AuditData.new(node.name, run_id) } + let(:run_data) { audit_data.to_hash } + + before do + allow(reporter).to receive(:auditing_enabled?).and_return(true) + allow(reporter).to receive(:run_status).and_return(run_status) + allow(rest).to receive(:create_url).and_return(true) + allow(rest).to receive(:post).and_return(true) + allow(reporter).to receive(:audit_data).and_return(audit_data) + allow(reporter).to receive(:run_status).and_return(run_status) + allow(audit_data).to receive(:to_hash).and_return(run_data) + end + + describe "a successful run with auditing enabled" do + it "sets run start and end times" do + iso_start_time = "2014-12-03T17:31:05Z" + iso_end_time = "2014-12-03T17:36:14Z" + + reporter.run_completed(node) + expect(audit_data.start_time).to eq iso_start_time + expect(audit_data.end_time).to eq iso_end_time + end + + it "posts audit data to server endpoint" do + endpoint = "api.opscode.us/orgname/controls" + headers = { + 'X-Ops-Audit-Report-Protocol-Version' => Chef::Audit::AuditReporter::PROTOCOL_VERSION + } + + expect(rest).to receive(:create_url). + with("controls"). + and_return(endpoint) + expect(rest).to receive(:post). + with(endpoint, run_data, headers) + reporter.run_completed(node) + end + + context "when unable to post to server" do + + let(:error) do + e = StandardError.new + e.set_backtrace(caller) + e + end + + before do + expect(rest).to receive(:post).and_raise(error) + allow(error).to receive(:respond_to?).and_call_original + end + + context "the error is an http error" do + + let(:response) { double("response", :code => code) } + + before do + expect(Chef::Log).to receive(:debug).with(/Sending audit report/) + expect(Chef::Log).to receive(:debug).with(/Audit Report/) + allow(error).to receive(:response).and_return(response) + expect(error).to receive(:respond_to?).with(:response).and_return(true) + end + + context "when the code is 404" do + + let(:code) { "404" } + + it "logs that the server doesn't support audit reporting" do + expect(Chef::Log).to receive(:debug).with(/Server doesn't support audit reporting/) + reporter.run_completed(node) + end + end + + shared_examples "non-404 error code" do + + it "saves the error report" do + expect(Chef::FileCache).to receive(:store). + with("failed-audit-data.json", an_instance_of(String), 0640). + and_return(true) + expect(Chef::FileCache).to receive(:load). + with("failed-audit-data.json", false). + and_return(true) + expect(Chef::Log).to receive(:error).with(/Failed to post audit report to server/) + reporter.run_completed(node) + end + + end + + context "when the code is not 404" do + include_examples "non-404 error code" do + let(:code) { "505" } + end + end + + context "when there is no code" do + include_examples "non-404 error code" do + let(:code) { nil } + end + end + + end + + context "the error is not an http error" do + + it "logs the error" do + expect(error).to receive(:respond_to?).with(:response).and_return(false) + expect(Chef::Log).to receive(:error).with(/Failed to post audit report to server/) + reporter.run_completed(node) + end + + end + + context "when reporting url fatals are enabled" do + + before do + allow(Chef::Config).to receive(:[]). + with(:enable_reporting_url_fatals). + and_return(true) + end + + it "raises the error" do + expect(error).to receive(:respond_to?).with(:response).and_return(false) + allow(Chef::Log).to receive(:error).and_return(true) + expect(Chef::Log).to receive(:error).with(/Reporting fatals enabled. Aborting run./) + expect{ reporter.run_completed(node) }.to raise_error(error) + end + + end + end + end + + context "when auditing is not enabled" do + + before do + allow(Chef::Log).to receive(:debug) + end + + it "doesn't send reports" do + expect(reporter).to receive(:auditing_enabled?).and_return(false) + expect(Chef::Log).to receive(:debug).with("Audit Reports are disabled. Skipping sending reports.") + reporter.run_completed(node) + end + + end + + context "when the run fails before audits" do + + before do + allow(Chef::Log).to receive(:debug) + end + + it "doesn't send reports" do + expect(reporter).to receive(:auditing_enabled?).and_return(true) + expect(reporter).to receive(:run_status).and_return(nil) + expect(Chef::Log).to receive(:debug).with("Run failed before audits were initialized, not sending audit report to server") + reporter.run_completed(node) + end + + end + end + + describe "#run_failed" do + + let(:audit_data) { Chef::Audit::AuditData.new(node.name, run_id) } + let(:run_data) { audit_data.to_hash } + + let(:error) { double("AuditError", :class => "Chef::Exception::AuditError", + :message => "Well that certainly didn't work", + :backtrace => ["line 0", "line 1", "line 2"]) } + + before do + allow(reporter).to receive(:auditing_enabled?).and_return(true) + allow(reporter).to receive(:run_status).and_return(run_status) + allow(reporter).to receive(:audit_data).and_return(audit_data) + allow(audit_data).to receive(:to_hash).and_return(run_data) + end + + it "adds the error information to the reported data" do + expect(rest).to receive(:create_url) + expect(rest).to receive(:post) + reporter.run_failed(error) + expect(run_data).to have_key(:error) + expect(run_data[:error]).to eq "Chef::Exception::AuditError: Well that certainly didn't work\n" + + "line 0\nline 1\nline 2" + end + + end + + shared_context "audit data" do + + let(:control_group_foo) { instance_double(Chef::Audit::ControlGroupData, + :metadata => double("foo metadata")) } + let(:control_group_bar) { instance_double(Chef::Audit::ControlGroupData, + :metadata => double("bar metadata")) } + + let(:ordered_control_groups) { + { + "foo" => control_group_foo, + "bar" => control_group_bar + } + } + + let(:audit_data) { instance_double(Chef::Audit::AuditData, + :add_control_group => true) } + + let(:run_context) { instance_double(Chef::RunContext, + :audits => ordered_control_groups) } + + before do + allow(reporter).to receive(:ordered_control_groups).and_return(ordered_control_groups) + allow(reporter).to receive(:audit_data).and_return(audit_data) + allow(reporter).to receive(:run_status).and_return(run_status) + allow(run_status).to receive(:run_context).and_return(run_context) + end + end + + describe "#audit_phase_complete" do + include_context "audit data" + + it "notifies audit phase finished to debug log" do + expect(Chef::Log).to receive(:debug).with(/Audit Reporter completed/) + reporter.audit_phase_complete + end + + it "collects audit data" do + ordered_control_groups.each do |_name, group| + expect(audit_data).to receive(:add_control_group).with(group) + end + reporter.audit_phase_complete + end + end + + describe "#audit_phase_failed" do + include_context "audit data" + + let(:error) { double("Exception") } + + it "notifies audit phase failed to debug log" do + expect(Chef::Log).to receive(:debug).with(/Audit Reporter failed/) + reporter.audit_phase_failed(error) + end + + it "collects audit data" do + ordered_control_groups.each do |_name, group| + expect(audit_data).to receive(:add_control_group).with(group) + end + reporter.audit_phase_failed(error) + end + end + + describe "#control_group_started" do + include_context "audit data" + + let(:name) { "bat" } + let(:control_group) { instance_double(Chef::Audit::ControlGroupData, + :metadata => double("metadata")) } + + before do + allow(Chef::Audit::ControlGroupData).to receive(:new). + with(name, control_group.metadata). + and_return(control_group) + end + + it "stores the control group" do + expect(ordered_control_groups).to receive(:has_key?).with(name).and_return(false) + allow(run_context.audits).to receive(:[]).with(name).and_return(control_group) + expect(ordered_control_groups).to receive(:store). + with(name, control_group). + and_call_original + reporter.control_group_started(name) + # stubbed :has_key? above, which is used by the have_key matcher, + # so instead we check the response to Hash's #key? because luckily + # #key? does not call #has_key? + expect(ordered_control_groups.key?(name)).to be true + expect(ordered_control_groups[name]).to eq control_group + end + + context "when a control group with the same name has been seen" do + it "raises an exception" do + expect(ordered_control_groups).to receive(:has_key?).with(name).and_return(true) + expect{ reporter.control_group_started(name) }.to raise_error(Chef::Exceptions::AuditControlGroupDuplicate) + end + end + end + + describe "#control_example_success" do + include_context "audit data" + + let(:name) { "foo" } + let(:example_data) { double("example data") } + + it "notifies the control group the example succeeded" do + expect(control_group_foo).to receive(:example_success).with(example_data) + reporter.control_example_success(name, example_data) + end + end + + describe "#control_example_failure" do + include_context "audit data" + + let(:name) { "bar" } + let(:example_data) { double("example data") } + let(:error) { double("Exception", :message => "oopsie") } + + it "notifies the control group the example failed" do + expect(control_group_bar).to receive(:example_failure). + with(example_data, error.message) + reporter.control_example_failure(name, example_data, error) + end + end + + describe "#auditing_enabled?" do + shared_examples "enabled?" do |true_or_false| + + it "returns #{true_or_false}" do + expect(Chef::Config).to receive(:[]). + with(:audit_mode). + and_return(audit_setting) + expect(reporter.auditing_enabled?).to be true_or_false + end + end + + context "when auditing is disabled" do + include_examples "enabled?", false do + let(:audit_setting) { :disabled } + end + end + + context "when auditing in audit-only mode" do + include_examples "enabled?", true do + let(:audit_setting) { :audit_only } + end + end + + context "when auditing is enabled" do + include_examples "enabled?", true do + let(:audit_setting) { :enabled } + end + end + end + +end diff --git a/spec/unit/audit/control_group_data_spec.rb b/spec/unit/audit/control_group_data_spec.rb new file mode 100644 index 0000000000..e21ab066fd --- /dev/null +++ b/spec/unit/audit/control_group_data_spec.rb @@ -0,0 +1,478 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# Author:: Claire McQuin (<claire@getchef.com>) +# +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'spec_helper' +require 'securerandom' + +describe Chef::Audit::AuditData do + + let(:node_name) { "noodles" } + let(:run_id) { SecureRandom.uuid } + let(:audit_data) { described_class.new(node_name, run_id) } + + let(:control_group_1) { double("control group 1") } + let(:control_group_2) { double("control group 2") } + + describe "#add_control_group" do + context "when no control groups have been added" do + it "stores the control group" do + audit_data.add_control_group(control_group_1) + expect(audit_data.control_groups).to include(control_group_1) + end + + end + + context "when adding additional control groups" do + + before do + audit_data.add_control_group(control_group_1) + end + + it "stores the control group" do + audit_data.add_control_group(control_group_2) + expect(audit_data.control_groups).to include(control_group_2) + end + + it "stores all control groups" do + audit_data.add_control_group(control_group_2) + expect(audit_data.control_groups).to include(control_group_1) + end + end + end + + describe "#to_hash" do + + let(:audit_data_hash) { audit_data.to_hash } + + it "returns a hash" do + expect(audit_data_hash).to be_a(Hash) + end + + it "describes a Chef::Audit::AuditData object" do + keys = [:node_name, :run_id, :start_time, :end_time, :control_groups] + expect(audit_data_hash.keys).to match_array(keys) + end + + describe ":control_groups" do + + let(:control_hash_1) { {:name => "control group 1"} } + let(:control_hash_2) { {:name => "control group 2"} } + + let(:control_groups) { audit_data_hash[:control_groups] } + + context "with no control groups added" do + it "is an empty list" do + expect(control_groups).to eq [] + end + end + + context "with one control group added" do + + before do + allow(audit_data).to receive(:control_groups).and_return([control_group_1]) + end + + it "is a one-element list containing the control group hash" do + expect(control_group_1).to receive(:to_hash).once.and_return(control_hash_1) + expect(control_groups.size).to eq 1 + expect(control_groups).to include(control_hash_1) + end + end + + context "with multiple control groups added" do + + before do + allow(audit_data).to receive(:control_groups).and_return([control_group_1, control_group_2]) + end + + it "is a list of control group hashes" do + expect(control_group_1).to receive(:to_hash).and_return(control_hash_1) + expect(control_group_2).to receive(:to_hash).and_return(control_hash_2) + expect(control_groups.size).to eq 2 + expect(control_groups).to include(control_hash_1) + expect(control_groups).to include(control_hash_2) + end + end + end + end +end + +describe Chef::Audit::ControlData do + + let(:name) { "ramen" } + let(:resource_type) { double("Service") } + let(:resource_name) { "mysql" } + let(:context) { nil } + let(:line_number) { 27 } + + let(:control_data) { described_class.new(name: name, + resource_type: resource_type, resource_name: resource_name, + context: context, line_number: line_number) } + + + describe "#to_hash" do + + let(:control_data_hash) { control_data.to_hash } + + it "returns a hash" do + expect(control_data_hash).to be_a(Hash) + end + + it "describes a Chef::Audit::ControlData object" do + keys = [:name, :resource_type, :resource_name, :context, :status, :details] + expect(control_data_hash.keys).to match_array(keys) + end + + context "when context is nil" do + + it "sets :context to an empty array" do + expect(control_data_hash[:context]).to eq [] + end + + end + + context "when context is non-nil" do + + let(:context) { ["outer"] } + + it "sets :context to its value" do + expect(control_data_hash[:context]).to eq context + end + end + end +end + +describe Chef::Audit::ControlGroupData do + + let(:name) { "balloon" } + let(:control_group_data) { described_class.new(name) } + + shared_context "control data" do + + let(:name) { "" } + let(:resource_type) { nil } + let(:resource_name) { nil } + let(:context) { nil } + let(:line_number) { 0 } + + let(:control_data) { + { + :name => name, + :resource_type => resource_type, + :resource_name => resource_name, + :context => context, + :line_number => line_number + } + } + + end + + shared_context "control" do + include_context "control data" + + let(:control) { Chef::Audit::ControlData.new(name: name, + resource_type: resource_type, resource_name: resource_name, + context: context, line_number: line_number) } + + before do + allow(Chef::Audit::ControlData).to receive(:new). + with(name: name, resource_type: resource_type, + resource_name: resource_name, context: context, + line_number: line_number). + and_return(control) + end + end + + describe "#new" do + it "has status \"success\"" do + expect(control_group_data.status).to eq "success" + end + end + + describe "#example_success" do + include_context "control" + + def notify_success + control_group_data.example_success(control_data) + end + + it "increments the number of successful audits" do + num_success = control_group_data.number_succeeded + notify_success + expect(control_group_data.number_succeeded).to eq (num_success + 1) + end + + it "does not increment the number of failed audits" do + num_failed = control_group_data.number_failed + notify_success + expect(control_group_data.number_failed).to eq (num_failed) + end + + it "marks the audit's status as success" do + notify_success + expect(control.status).to eq "success" + end + + it "does not modify its own status" do + expect(control_group_data).to_not receive(:status=) + status = control_group_data.status + notify_success + expect(control_group_data.status).to eq status + end + + it "saves the control" do + controls = control_group_data.controls + expect(controls).to_not include(control) + notify_success + expect(controls).to include(control) + end + end + + describe "#example_failure" do + include_context "control" + + let(:details) { "poop" } + + def notify_failure + control_group_data.example_failure(control_data, details) + end + + it "does not increment the number of successful audits" do + num_success = control_group_data.number_succeeded + notify_failure + expect(control_group_data.number_succeeded).to eq num_success + end + + it "increments the number of failed audits" do + num_failed = control_group_data.number_failed + notify_failure + expect(control_group_data.number_failed).to eq (num_failed + 1) + end + + it "marks the audit's status as failure" do + notify_failure + expect(control.status).to eq "failure" + end + + it "marks its own status as failure" do + notify_failure + expect(control_group_data.status).to eq "failure" + end + + it "saves the control" do + controls = control_group_data.controls + expect(controls).to_not include(control) + notify_failure + expect(controls).to include(control) + end + + context "when details are not provided" do + + let(:details) { nil } + + it "does not save details to the control" do + default_details = control.details + expect(control).to_not receive(:details=) + notify_failure + expect(control.details).to eq default_details + end + end + + context "when details are provided" do + + let(:details) { "yep that didn't work" } + + it "saves details to the control" do + notify_failure + expect(control.details).to eq details + end + end + end + + shared_examples "multiple audits" do |success_or_failure| + include_context "control" + + let(:num_success) { 0 } + let(:num_failure) { 0 } + + before do + if num_failure == 0 + num_success.times { control_group_data.example_success(control_data) } + elsif num_success == 0 + num_failure.times { control_group_data.example_failure(control_data, nil) } + end + end + + it "counts the number of successful audits" do + expect(control_group_data.number_succeeded).to eq num_success + end + + it "counts the number of failed audits" do + expect(control_group_data.number_failed).to eq num_failure + end + + it "marks its status as \"#{success_or_failure}\"" do + expect(control_group_data.status).to eq success_or_failure + end + end + + context "when all audits pass" do + include_examples "multiple audits", "success" do + let(:num_success) { 3 } + end + end + + context "when one audit fails" do + shared_examples "mixed audit results" do + include_examples "multiple audits", "failure" do + + let(:audit_results) { [] } + let(:num_success) { audit_results.count("success") } + let(:num_failure) { 1 } + + before do + audit_results.each do |result| + if result == "success" + control_group_data.example_success(control_data) + else + control_group_data.example_failure(control_data, nil) + end + end + end + end + end + + context "and it's the first audit" do + include_examples "mixed audit results" do + let(:audit_results) { ["failure", "success", "success"] } + end + end + + context "and it's an audit in the middle" do + include_examples "mixed audit results" do + let(:audit_results) { ["success", "failure", "success"] } + end + end + + context "and it's the last audit" do + include_examples "mixed audit results" do + let(:audit_results) { ["success", "success", "failure"] } + end + end + end + + context "when all audits fail" do + include_examples "multiple audits", "failure" do + let(:num_failure) { 3 } + end + end + + describe "#to_hash" do + + let(:control_group_data_hash) { control_group_data.to_hash } + + it "returns a hash" do + expect(control_group_data_hash).to be_a(Hash) + end + + it "describes a Chef::Audit::ControlGroupData object" do + keys = [:name, :status, :number_succeeded, :number_failed, + :controls, :id] + expect(control_group_data_hash.keys).to match_array(keys) + end + + describe ":controls" do + + let(:control_group_controls) { control_group_data_hash[:controls] } + + context "with no controls added" do + it "is an empty list" do + expect(control_group_controls).to eq [] + end + end + + context "with one control added" do + include_context "control" + + let(:control_list) { [control_data] } + let(:control_hash) { control.to_hash } + + before do + expect(control_group_data).to receive(:controls).twice.and_return(control_list) + expect(control_data).to receive(:to_hash).and_return(control_hash) + end + + it "is a one-element list containing the control hash" do + expect(control_group_controls.size).to eq 1 + expect(control_group_controls).to include(control_hash) + end + + it "adds a sequence number to the control" do + control_group_data.to_hash + expect(control_hash).to have_key(:sequence_number) + end + + end + + context "with multiple controls added" do + + let(:control_hash_1) { {:line_number => 27} } + let(:control_hash_2) { {:line_number => 13} } + let(:control_hash_3) { {:line_number => 35} } + + let(:control_1) { double("control 1", + :line_number => control_hash_1[:line_number], + :to_hash => control_hash_1) } + let(:control_2) { double("control 2", + :line_number => control_hash_2[:line_number], + :to_hash => control_hash_2) } + let(:control_3) { double("control 3", + :line_number => control_hash_3[:line_number], + :to_hash => control_hash_3) } + + let(:control_list) { [control_1, control_2, control_3] } + let(:ordered_control_hashes) { [control_hash_2, control_hash_1, control_hash_3] } + + before do + # Another way to do this would be to call #example_success + # or #example_failure per control hash, but we'd have to + # then stub #create_control and it's a lot of extra stubbing work. + # We can't stub the controls reader to return a list of + # controls because of the call to sort! and the following + # reading of controls. + control_group_data.instance_variable_set(:@controls, control_list) + end + + it "is a list of control group hashes ordered by line number" do + expect(control_group_controls.size).to eq 3 + expect(control_group_controls).to eq ordered_control_hashes + end + + it "assigns sequence numbers in order" do + control_group_data.to_hash + ordered_control_hashes.each_with_index do |control_hash, idx| + expect(control_hash[:sequence_number]).to eq idx + 1 + end + end + end + end + end + +end diff --git a/spec/unit/audit/rspec_formatter_spec.rb b/spec/unit/audit/rspec_formatter_spec.rb new file mode 100644 index 0000000000..471473e387 --- /dev/null +++ b/spec/unit/audit/rspec_formatter_spec.rb @@ -0,0 +1,29 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# Author:: Claire McQuin (<claire@getchef.com>) +# +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'spec_helper' +require 'chef/audit/rspec_formatter' + +describe Chef::Audit::RspecFormatter do + let(:formatter) { Chef::Audit::RspecFormatter.new(nil) } + it "should respond to close" do + expect(formatter).to respond_to(:close) + end +end diff --git a/spec/unit/audit/runner_spec.rb b/spec/unit/audit/runner_spec.rb new file mode 100644 index 0000000000..67590fecf9 --- /dev/null +++ b/spec/unit/audit/runner_spec.rb @@ -0,0 +1,133 @@ +# +# Author:: Tyler Ball (<tball@chef.io>) +# Copyright:: Copyright (c) 2014 Chef Software, Inc. +# License:: Apache License, Version 2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +require 'spec_helper' +require 'spec/support/audit_helper' +require 'chef/audit/runner' +require 'chef/audit/audit_event_proxy' +require 'chef/audit/rspec_formatter' +require 'rspec/support/spec/in_sub_process' + +describe Chef::Audit::Runner do + include RSpec::Support::InSubProcess + + let(:events) { double("events") } + let(:run_context) { instance_double(Chef::RunContext, :events => events) } + let(:runner) { Chef::Audit::Runner.new(run_context) } + + around(:each) do |ex| + Sandboxing.sandboxed { ex.run } + end + + describe "#initialize" do + it "correctly sets the run_context during initialization" do + expect(runner.instance_variable_get(:@run_context)).to eq(run_context) + end + end + + context "during #run" do + + describe "#setup" do + let(:log_location) { File.join(Dir.tmpdir, 'audit_log') } + let(:color) { false } + + before do + Chef::Config[:log_location] = log_location + Chef::Config[:color] = color + end + + it "sets all the config values" do + # This runs the Serverspec includes - we don't want these hanging around in all subsequent tests so + # we run this in a forked process. Keeps Serverspec files from getting loaded into main process. + in_sub_process do + runner.send(:setup) + + expect(RSpec.configuration.output_stream).to eq(log_location) + expect(RSpec.configuration.error_stream).to eq(log_location) + + expect(RSpec.configuration.formatters.size).to eq(2) + expect(RSpec.configuration.formatters).to include(instance_of(Chef::Audit::AuditEventProxy)) + expect(RSpec.configuration.formatters).to include(instance_of(Chef::Audit::RspecFormatter)) + expect(Chef::Audit::AuditEventProxy.class_variable_get(:@@events)).to eq(run_context.events) + + expect(RSpec.configuration.expectation_frameworks).to eq([RSpec::Matchers]) + expect(RSpec::Matchers.configuration.syntax).to eq([:expect]) + + expect(RSpec.configuration.color).to eq(color) + expect(RSpec.configuration.expose_dsl_globally?).to eq(false) + + expect(Specinfra.configuration.backend).to eq(:exec) + end + end + end + + describe "#register_controls" do + let(:audits) { [] } + let(:run_context) { instance_double(Chef::RunContext, :audits => audits) } + + it "adds the control group aliases" do + runner.send(:register_controls) + + expect(RSpec::Core::DSL.example_group_aliases).to include(:__controls__) + expect(RSpec::Core::DSL.example_group_aliases).to include(:control) + end + + context "audits exist" do + let(:audits) { {"audit_name" => group} } + let(:group) {Struct.new(:args, :block).new(["group_name"], nil)} + + it "sends the audits to the world" do + runner.send(:register_controls) + + expect(RSpec.world.example_groups.size).to eq(1) + # For whatever reason, `kind_of` is not working + # expect(RSpec.world.example_groups).to include(kind_of(RSpec::Core::ExampleGroup)) => FAIL + g = RSpec.world.example_groups[0] + expect(g.ancestors).to include(RSpec::Core::ExampleGroup) + expect(g.description).to eq("group_name") + end + end + end + + describe "#do_run" do + let(:rspec_runner) { instance_double(RSpec::Core::Runner) } + + it "executes the runner" do + expect(RSpec::Core::Runner).to receive(:new).with(nil).and_return(rspec_runner) + expect(rspec_runner).to receive(:run_specs).with([]) + + runner.send(:do_run) + end + end + end + + describe "counters" do + it "correctly calculates failed?" do + expect(runner.failed?).to eq(false) + end + + it "correctly calculates num_failed" do + expect(runner.num_failed).to eq(0) + end + + it "correctly calculates num_total" do + expect(runner.num_total).to eq(0) + end + end + +end diff --git a/spec/unit/client_spec.rb b/spec/unit/client_spec.rb index 10958d628c..2ec32b32ac 100644 --- a/spec/unit/client_spec.rb +++ b/spec/unit/client_spec.rb @@ -187,11 +187,12 @@ describe Chef::Client do end describe "a full client run" do - shared_examples_for "a successful client run" do + shared_context "a client run" do let(:http_node_load) { double("Chef::REST (node)") } let(:http_cookbook_sync) { double("Chef::REST (cookbook sync)") } let(:http_node_save) { double("Chef::REST (node save)") } let(:runner) { double("Chef::Runner") } + let(:audit_runner) { instance_double("Chef::Audit::Runner", :failed? => false) } let(:api_client_exists?) { false } @@ -204,7 +205,11 @@ describe Chef::Client do # --Client.register # Make sure Client#register thinks the client key doesn't # exist, so it tries to register and create one. - expect(File).to receive(:exists?).with(Chef::Config[:client_key]).exactly(1).times.and_return(api_client_exists?) + allow(File).to receive(:exists?).and_call_original + expect(File).to receive(:exists?). + with(Chef::Config[:client_key]). + exactly(:once). + and_return(api_client_exists?) unless api_client_exists? # Client.register will register with the validation client name. @@ -218,7 +223,7 @@ describe Chef::Client do # previous step. expect(Chef::REST).to receive(:new). with(Chef::Config[:chef_server_url], fqdn, Chef::Config[:client_key]). - exactly(1). + exactly(:once). and_return(http_node_load) # --Client#build_node @@ -246,11 +251,12 @@ describe Chef::Client do # --Client#converge expect(Chef::Runner).to receive(:new).and_return(runner) expect(runner).to receive(:converge).and_return(true) + end - # --ResourceReporter#run_completed - # updates the server with the resource history - # (has its own tests, so stubbing it here.) - expect_any_instance_of(Chef::ResourceReporter).to receive(:run_completed) + def stub_for_audit + # -- Client#run_audits + expect(Chef::Audit::Runner).to receive(:new).and_return(audit_runner) + expect(audit_runner).to receive(:run).and_return(true) end def stub_for_node_save @@ -269,11 +275,22 @@ describe Chef::Client do # Post conditions: check that node has been filled in correctly expect(client).to receive(:run_started) expect(client).to receive(:run_completed_successfully) + + # --ResourceReporter#run_completed + # updates the server with the resource history + # (has its own tests, so stubbing it here.) + expect_any_instance_of(Chef::ResourceReporter).to receive(:run_completed) + # --AuditReporter#run_completed + # posts the audit data to server. + # (has its own tests, so stubbing it here.) + expect_any_instance_of(Chef::Audit::AuditReporter).to receive(:run_completed) end before do Chef::Config[:client_fork] = enable_fork Chef::Config[:cache_path] = windows? ? 'C:\chef' : '/var/chef' + Chef::Config[:why_run] = false + Chef::Config[:audit_mode] = :enabled stub_const("Chef::Client::STDOUT_FD", stdout) stub_const("Chef::Client::STDERR_FD", stderr) @@ -282,11 +299,16 @@ describe Chef::Client do stub_for_node_load stub_for_sync_cookbooks stub_for_converge + stub_for_audit stub_for_node_save stub_for_run end + end - it "runs ohai, sets up authentication, loads node state, synchronizes policy, and converges" do + shared_examples_for "a successful client run" do + include_context "a client run" + + it "runs ohai, sets up authentication, loads node state, synchronizes policy, converges, and runs audits" do # This is what we're testing. client.run @@ -296,16 +318,12 @@ describe Chef::Client do end end - describe "when running chef-client without fork" do - include_examples "a successful client run" end describe "when the client key already exists" do - let(:api_client_exists?) { true } - include_examples "a successful client run" end @@ -344,7 +362,6 @@ describe Chef::Client do end describe "when a permanent run list is passed as an option" do - include_examples "a successful client run" do let(:new_runlist) { "recipe[new_run_list_recipe]" } @@ -374,6 +391,155 @@ describe Chef::Client do end end + describe "when converge fails" do + include_context "a client run" do + let(:e) { Exception.new } + def stub_for_converge + expect(Chef::Runner).to receive(:new).and_return(runner) + expect(runner).to receive(:converge).and_raise(e) + expect(Chef::Application).to receive(:debug_stacktrace).with an_instance_of(Chef::Exceptions::RunFailedWrappingError) + end + + def stub_for_node_save + expect(client).to_not receive(:save_updated_node) + end + + def stub_for_run + expect_any_instance_of(Chef::RunLock).to receive(:acquire) + expect_any_instance_of(Chef::RunLock).to receive(:save_pid) + expect_any_instance_of(Chef::RunLock).to receive(:release) + + # Post conditions: check that node has been filled in correctly + expect(client).to receive(:run_started) + expect(client).to receive(:run_failed) + + expect_any_instance_of(Chef::ResourceReporter).to receive(:run_failed) + expect_any_instance_of(Chef::Audit::AuditReporter).to receive(:run_failed) + end + end + + it "runs the audits and raises the error" do + expect{ client.run }.to raise_error(Chef::Exceptions::RunFailedWrappingError) do |error| + expect(error.wrapped_errors.size).to eq(1) + expect(error.wrapped_errors[0]).to eq(e) + end + end + end + + describe "when the audit phase fails" do + context "with an exception" do + include_context "a client run" do + let(:e) { Exception.new } + def stub_for_audit + expect(Chef::Audit::Runner).to receive(:new).and_return(audit_runner) + expect(audit_runner).to receive(:run).and_raise(e) + expect(Chef::Application).to receive(:debug_stacktrace).with an_instance_of(Chef::Exceptions::RunFailedWrappingError) + end + + def stub_for_run + expect_any_instance_of(Chef::RunLock).to receive(:acquire) + expect_any_instance_of(Chef::RunLock).to receive(:save_pid) + expect_any_instance_of(Chef::RunLock).to receive(:release) + + # Post conditions: check that node has been filled in correctly + expect(client).to receive(:run_started) + expect(client).to receive(:run_failed) + + expect_any_instance_of(Chef::ResourceReporter).to receive(:run_failed) + expect_any_instance_of(Chef::Audit::AuditReporter).to receive(:run_failed) + end + end + + it "should save the node after converge and raise exception" do + expect{ client.run }.to raise_error(Chef::Exceptions::RunFailedWrappingError) do |error| + expect(error.wrapped_errors.size).to eq(1) + expect(error.wrapped_errors[0]).to eq(e) + end + end + end + + context "with failed audits" do + include_context "a client run" do + let(:audit_runner) do + instance_double("Chef::Audit::Runner", :run => true, :failed? => true, :num_failed => 1, :num_total => 1) + end + + def stub_for_audit + expect(Chef::Audit::Runner).to receive(:new).and_return(audit_runner) + expect(Chef::Application).to receive(:debug_stacktrace).with an_instance_of(Chef::Exceptions::RunFailedWrappingError) + end + + def stub_for_run + expect_any_instance_of(Chef::RunLock).to receive(:acquire) + expect_any_instance_of(Chef::RunLock).to receive(:save_pid) + expect_any_instance_of(Chef::RunLock).to receive(:release) + + # Post conditions: check that node has been filled in correctly + expect(client).to receive(:run_started) + expect(client).to receive(:run_failed) + + expect_any_instance_of(Chef::ResourceReporter).to receive(:run_failed) + expect_any_instance_of(Chef::Audit::AuditReporter).to receive(:run_failed) + end + end + + it "should save the node after converge and raise exception" do + expect{ client.run }.to raise_error(Chef::Exceptions::RunFailedWrappingError) do |error| + expect(error.wrapped_errors.size).to eq(1) + expect(error.wrapped_errors[0]).to be_instance_of(Chef::Exceptions::AuditsFailed) + end + end + end + end + + describe "when why_run mode is enabled" do + include_context "a client run" do + + before do + Chef::Config[:why_run] = true + end + + def stub_for_audit + expect(Chef::Audit::Runner).to_not receive(:new) + end + + def stub_for_node_save + # This is how we should be mocking external calls - not letting it fall all the way through to the + # REST call + expect(node).to receive(:save) + end + + it "runs successfully without enabling the audit runner" do + client.run + + # fork is stubbed, so we can see the outcome of the run + expect(node.automatic_attrs[:platform]).to eq("example-platform") + expect(node.automatic_attrs[:platform_version]).to eq("example-platform-1.0") + end + end + end + + describe "when audits are disabled" do + include_context "a client run" do + + before do + Chef::Config[:audit_mode] = :disabled + end + + def stub_for_audit + expect(Chef::Audit::Runner).to_not receive(:new) + end + + it "runs successfully without enabling the audit runner" do + client.run + + # fork is stubbed, so we can see the outcome of the run + expect(node.automatic_attrs[:platform]).to eq("example-platform") + expect(node.automatic_attrs[:platform_version]).to eq("example-platform-1.0") + end + end + end + end diff --git a/spec/unit/dsl/audit_spec.rb b/spec/unit/dsl/audit_spec.rb new file mode 100644 index 0000000000..38707127f0 --- /dev/null +++ b/spec/unit/dsl/audit_spec.rb @@ -0,0 +1,43 @@ + +require 'spec_helper' +require 'chef/dsl/audit' + +class AuditDSLTester < Chef::Recipe + include Chef::DSL::Audit +end + +class BadAuditDSLTester + include Chef::DSL::Audit +end + +describe Chef::DSL::Audit do + let(:auditor) { AuditDSLTester.new("cookbook_name", "recipe_name", run_context) } + let(:run_context) { instance_double(Chef::RunContext, :audits => audits, :cookbook_collection => cookbook_collection) } + let(:audits) { {} } + let(:cookbook_collection) { {} } + + it "raises an error when a block of audits is not provided" do + expect{ auditor.controls "name" }.to raise_error(Chef::Exceptions::NoAuditsProvided) + end + + it "raises an error when no audit name is given" do + expect{ auditor.controls do end }.to raise_error(Chef::Exceptions::AuditNameMissing) + end + + context "audits already populated" do + let(:audits) { {"unique" => {} } } + + it "raises an error if the audit name is a duplicate" do + expect { auditor.controls "unique" do end }.to raise_error(Chef::Exceptions::AuditControlGroupDuplicate) + end + end + + context "included in a class without recipe DSL" do + let(:auditor) { BadAuditDSLTester.new } + + it "fails because it relies on the recipe DSL existing" do + expect { auditor.controls "unique" do end }.to raise_error(NoMethodError, /undefined method `cookbook_name'/) + end + end + +end diff --git a/spec/unit/exceptions_spec.rb b/spec/unit/exceptions_spec.rb index 6318ec9227..d35ecc8ec8 100644 --- a/spec/unit/exceptions_spec.rb +++ b/spec/unit/exceptions_spec.rb @@ -81,4 +81,50 @@ describe Chef::Exceptions do end end end + + describe Chef::Exceptions::RunFailedWrappingError do + shared_examples "RunFailedWrappingError expectations" do + it "should initialize with a default message" do + expect(e.message).to eq("Found #{num_errors} errors, they are stored in the backtrace") + end + + it "should provide a modified backtrace when requested" do + e.fill_backtrace + expect(e.backtrace).to eq(backtrace) + end + end + + context "initialized with nothing" do + let(:e) { Chef::Exceptions::RunFailedWrappingError.new } + let(:num_errors) { 0 } + let(:backtrace) { [] } + + include_examples "RunFailedWrappingError expectations" + end + + context "initialized with nil" do + let(:e) { Chef::Exceptions::RunFailedWrappingError.new(nil, nil) } + let(:num_errors) { 0 } + let(:backtrace) { [] } + + include_examples "RunFailedWrappingError expectations" + end + + context "initialized with 1 error and nil" do + let(:e) { Chef::Exceptions::RunFailedWrappingError.new(RuntimeError.new("foo"), nil) } + let(:num_errors) { 1 } + let(:backtrace) { ["1) RuntimeError - foo", ""] } + + include_examples "RunFailedWrappingError expectations" + end + + context "initialized with 2 errors" do + let(:e) { Chef::Exceptions::RunFailedWrappingError.new(RuntimeError.new("foo"), RuntimeError.new("bar")) } + let(:num_errors) { 2 } + let(:backtrace) { ["1) RuntimeError - foo", "", "2) RuntimeError - bar", ""] } + + include_examples "RunFailedWrappingError expectations" + end + + end end diff --git a/spec/unit/recipe_spec.rb b/spec/unit/recipe_spec.rb index e1a42362ef..e8c1358ba2 100644 --- a/spec/unit/recipe_spec.rb +++ b/spec/unit/recipe_spec.rb @@ -484,4 +484,11 @@ describe Chef::Recipe do expect(node[:tags]).to eql([]) end end + + describe "included DSL" do + it "should include features from Chef::DSL::Audit" do + expect(recipe.singleton_class.included_modules).to include(Chef::DSL::Audit) + expect(recipe.respond_to?(:controls)).to be true + end + end end |