Merge pull request #1555 from opscode/shain/search_filter

Result filtering on search (also known as Partial Search)

7 files changed, 439 insertions, 66 deletions

diff --git a/CHANGELOG.md b/CHANGELOG.md
index c160fae749..471a00500f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -117,6 +117,7 @@
 * Deprecate --distro / --template_file options in favor of --boostrap-template
 * Add `:node_ssl_verify_mode` & `:node_verify_api_cert` options to bootstrap
   to be able to configure these settings on the bootstrapped node.
+* Add partial_search dsl method to Chef::Search::Query, add result filtering to search.
 
 ## Last Release: 11.14.2
 
diff --git a/DOC_CHANGES.md b/DOC_CHANGES.md
index c08ab0097a..8b70f40c5a 100644
--- a/DOC_CHANGES.md
+++ b/DOC_CHANGES.md
@@ -89,3 +89,59 @@ Note that the service resource will also continue to set the startup type to aut
 DSL method `data_bag_item` now takes an optional String parameter `secret`, which is used to interact with encrypted data bag items.
 If the data bag item being fetched is encrypted and no `secret` is provided, Chef looks for a secret at `Chef::Config[:encrypted_data_bag_secret]`.
 If `secret` is provided, but the data bag item is not encrypted, then a regular data bag item is returned (no decryption is attempted).
+
+### Enhanced search functionality: result filtering
+#### Use in recipes
+`Chef::Search::Query#search` can take an optional `:filter_result` argument which returns search data in the form of the Hash specified. Suppose your data looks like
+```json
+{"languages": {
+  "c": {
+    "gcc": {
+      "version": "4.6.3",
+      "description": "gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) "
+    }
+  },
+  "ruby": {
+    "platform": "x86_64-linux",
+    "version": "1.9.3",
+    "release_date": "2013-11-22"
+  },
+  "perl": {
+    "version": "5.14.2",
+    "archname": "x86_64-linux-gnu-thread-multi"
+  },
+  "python": {
+    "version": "2.7.3",
+    "builddate": "Feb 27 2014, 19:58:35"
+  }
+}}
+```
+for a node running Ubuntu named `node01`, and you want to get back only information on which versions of c and ruby you have. In a recipe you would write
+```ruby
+search(:node, "platform:ubuntu", :filter_result => {"c_version" => ["languages", "c", "gcc", "version"],
+                                                    "ruby_version" => ["languages", "ruby", "version"]})
+```
+and receive
+```ruby
+[
+  {"url" => "https://api.opscode.com/organization/YOUR_ORG/nodes/node01",
+   "data" => {"c_version" => "4.6.3", "ruby_version" => "1.9.3"},
+  # snip other Ubuntu nodes
+]
+```
+If instead you wanted all the languages data (remember, `"languages"` is only one tiny piece of information the Chef Server stores about your node), you would have `:filter_result => {"languages" => ["laguages"]}` in your search query.
+
+For backwards compatibility, a `partial_search` method has been added to `Chef::Search::Query` which can be used in the same way as the `partial_search` method from the [partial_search cookbook](https://supermarket.getchef.com/cookbooks/partial_search). Note that this method has been deprecated and will be removed in future versions of Chef.
+
+#### Use in knife
+Search results can likewise be filtered by adding the `--filter-result` (or `-f`) option. Considering the node data above, you can use `knife search` with filtering to extract the c and ruby versions on your Ubuntu platforms:
+```bash
+$ knife search node "platform:ubuntu" --filter-result "c_version:languages.c.gcc.version, ruby_version:languages.ruby.version"
+1 items found
+
+:
+  c_version: 4.6.3
+  ruby_version: 1.9.3
+
+$
+```
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
index 30c5d0893e..f5a74d6a43 100644
--- a/RELEASE_NOTES.md
+++ b/RELEASE_NOTES.md
@@ -237,3 +237,9 @@ work properly if the remote server implemented only the Chef 10 API.
 ## CookbookSiteStreamingUploader now uses ssl_verify_mode config option
 The CookbookSiteStreamingUploader now obeys the setting of ssl_verify_mode in the client config. Was previously ignoring the
 config setting and always set to VERIFY_NONE.
+
+## Result filtering on `search` API.
+`search` can take an optional `:filter_result`, which returns search data in the form specified
+by the given Hash. This works analogously to the partial_search method from the [partial_search cookbook](https://supermarket.getchef.com/cookbooks/partial_search),
+with `:filter_result` replacing `:keys`. You can also filter `knife search` results by supplying the `--filter-result`
+or `-f` option and a comma-separated string representation of the filter hash.
diff --git a/lib/chef/exceptions.rb b/lib/chef/exceptions.rb
index f6db5dbe56..23e223f204 100644
--- a/lib/chef/exceptions.rb
+++ b/lib/chef/exceptions.rb
@@ -347,5 +347,7 @@ class Chef
       class EncodeError < RuntimeError; end
       class ParseError < RuntimeError; end
     end
+
+    class InvalidSearchQuery < ArgumentError; end
   end
 end
diff --git a/lib/chef/knife/search.rb b/lib/chef/knife/search.rb
index bc020c0445..34d12168b6 100644
--- a/lib/chef/knife/search.rb
+++ b/lib/chef/knife/search.rb
@@ -71,6 +71,11 @@ class Chef
         :long => "--query QUERY",
         :description => "The search query; useful to protect queries starting with -"
 
+      option :filter_result,
+        :short => "-f FILTER",
+        :long => "--filter-result FILTER",
+        :description => "Only bring back specific attributes of the matching objects; for example: \"ServerName=name, Kernel=kernel.version\""
+
       def run
         read_cli_args
         fuzzify_query
@@ -79,7 +84,6 @@ class Chef
           ui.use_presenter Knife::Core::NodePresenter
         end
 
-
         q = Chef::Search::Query.new
         escaped_query = URI.escape(@query,
                            Regexp.new("[^#{URI::PATTERN::UNRESERVED}]"))
@@ -87,14 +91,26 @@ class Chef
         result_items = []
         result_count = 0
 
-        rows = config[:rows]
-        start = config[:start]
+        search_args = Hash.new
+        search_args[:sort] = config[:sort]
+        search_args[:start] = config[:start]
+        search_args[:rows] = config[:rows]
+        if config[:filter_result]
+          search_args[:filter_result] = create_result_filter(config[:filter_result])
+        elsif (not ui.config[:attribute].nil?) && (not ui.config[:attribute].empty?)
+          search_args[:filter_result] = create_result_filter_from_attributes(ui.config[:attribute])
+        end
+
         begin
-          q.search(@type, escaped_query, config[:sort], start, rows) do |item|
-            formatted_item = format_for_display(item)
-            # if formatted_item.respond_to?(:has_key?) && !formatted_item.has_key?('id')
-            #   formatted_item['id'] = item.has_key?('id') ? item['id'] : item.name
-            # end
+          q.search(@type, escaped_query, search_args) do |item|
+            formatted_item = Hash.new
+            if item.is_a?(Hash)
+              # doing a little magic here to set the correct name
+              formatted_item[item["data"]["__display_name"]] = item["data"]
+              formatted_item[item["data"]["__display_name"]].delete("__display_name")
+            else
+              formatted_item = format_for_display(item)
+            end
             result_items << formatted_item
             result_count += 1
           end
@@ -149,10 +165,38 @@ class Chef
         end
       end
 
+      # This method turns a set of key value pairs in a string into the appropriate data structure that the
+      # chef-server search api is expecting.
+      # expected input is in the form of:
+      # -f "return_var1=path.to.attribute, return_var2=shorter.path"
+      #
+      # a more concrete example might be:
+      # -f "env=chef_environment, ruby_platform=languages.ruby.platform"
+      #
+      # The end result is a hash where the key is a symbol in the hash (the return variable)
+      # and the path is an array with the path elements as strings (in order)
+      # See lib/chef/search/query.rb for more examples of this.
+      def create_result_filter(filter_string)
+        final_filter = Hash.new
+        filter_string.gsub!(" ", "")
+        filters = filter_string.split(",")
+        filters.each do |f|
+          return_id, attr_path = f.split("=")
+          final_filter[return_id.to_sym] = attr_path.split(".")
+        end
+        return final_filter
+      end
+
+      def create_result_filter_from_attributes(filter_array)
+        final_filter = Hash.new
+        filter_array.each do |f|
+          final_filter[f] = f.split(".")
+        end
+        # adding magic filter so we can actually pull the name as before
+        final_filter["__display_name"] = [ "name" ]
+        return final_filter
+      end
+
     end
   end
 end
-
-
-
-
diff --git a/lib/chef/search/query.rb b/lib/chef/search/query.rb
index 4869ec1484..cc43efe1b1 100644
--- a/lib/chef/search/query.rb
+++ b/lib/chef/search/query.rb
@@ -23,6 +23,7 @@ require 'chef/node'
 require 'chef/role'
 require 'chef/data_bag'
 require 'chef/data_bag_item'
+require 'chef/exceptions'
 
 class Chef
   class Search
@@ -34,17 +35,112 @@ class Chef
         @rest = Chef::REST.new(url ||Chef::Config[:chef_server_url])
       end
 
-      # Search Solr for objects of a given type, for a given query. If you give
-      # it a block, it will handle the paging for you dynamically.
-      def search(type, query="*:*", sort='X_CHEF_id_CHEF_X asc', start=0, rows=1000, &block)
-        raise ArgumentError, "Type must be a string or a symbol!" unless (type.kind_of?(String) || type.kind_of?(Symbol))
 
-        response = @rest.get_rest("search/#{type}?q=#{escape(query)}&sort=#{escape(sort)}&start=#{escape(start)}&rows=#{escape(rows)}")
-        if block
-          response["rows"].each { |o| block.call(o) unless o.nil?}
+      # This search is only kept for backwards compatibility, since the results of the
+      # new filtered search method will be in a slightly different format
+      def partial_search(type, query='*:*', *args, &block)
+        Chef::Log.warn("DEPRECATED: The 'partial_search' api is deprecated, please use the search api with 'filter_result'")
+        # accept both types of args
+        if args.length == 1 && args[0].is_a?(Hash)
+          args_hash = args[0].dup
+          # partial_search implemented in the partial search cookbook uses the
+          # arg hash :keys instead of :filter_result to filter returned data
+          args_hash[:filter_result] = args_hash[:keys]
+        else
+          args_hash = {}
+          args_hash[:sort] = args[0] if args.length >= 1
+          args_hash[:start] = args[1] if args.length >= 2
+          args_hash[:rows] = args[2] if args.length >= 3
+        end
+
+        unless block.nil?
+          raw_results = search(type,query,args_hash)
+        else
+          raw_results = search(type,query,args_hash,&block)
+        end
+        results = Array.new
+        raw_results[0].each do |r|
+          results << r["data"]
+        end
+        return results
+      end
+
+      #
+      # New search input, designed to be backwards compatible with the old method signature
+      # 'type' and 'query' are the same as before, args now will accept either a Hash of
+      # search arguments with symbols as the keys (ie :sort, :start, :rows) and a :filter_result
+      # option.
+      #
+      # :filter_result should be in the format of another Hash with the structure of:
+      # {
+      #   :returned_name1 => ["path", "to", "variable"],
+      #   :returned_name2 => ["shorter", "path"]
+      # }
+      # a real world example might be something like:
+      # {
+      #   :ip_address => ["ipaddress"],
+      #   :ruby_version => ["languages", "ruby", "version"]
+      # }
+      #  this will bring back 2 variables 'ip_address' and 'ruby_version' with whatever value was found
+      # an example of the returned json may be:
+      # {"ip_address":"127.0.0.1", "ruby_version": "1.9.3"}
+      #
+      def search(type, query='*:*', *args, &block)
+        validate_type(type)
+        validate_args(args)
+
+        scrubbed_args = Hash.new
+
+        # argify everything
+        if args[0].kind_of?(Hash)
+          scrubbed_args = args[0]
+        else
+          # This api will be deprecated in a future release
+          scrubbed_args = { :sort => args[0], :start => args[1], :rows => args[2] }
+        end
+
+        # set defaults, if they haven't been set yet.
+        scrubbed_args[:sort] ||= 'X_CHEF_id_CHEF_X asc'
+        scrubbed_args[:start] ||= 0
+        scrubbed_args[:rows] ||= 1000
+
+        do_search(type, query, scrubbed_args, &block)
+      end
+
+      def list_indexes
+        @rest.get_rest("search")
+      end
+
+      private
+      def validate_type(t)
+        unless t.kind_of?(String) || t.kind_of?(Symbol)
+          msg = "Invalid search object type #{t.inspect} (#{t.class}), must be a String or Symbol." +
+            "Useage: search(:node, QUERY, [OPTIONAL_ARGS])" +
+            "        `knife search environment QUERY (options)`"
+          raise Chef::Exceptions::InvalidSearchQuery, msg
+        end
+      end
+
+      def validate_args(a)
+        max_args = 3
+        raise Chef::Exceptions::InvalidSearchQuery, "Too many arguments! (#{a.size} for <= #{max_args})" if a.size > max_args
+      end
+
+      def escape(s)
+        s && URI.escape(s.to_s)
+      end
+
+      # new search api that allows for a cleaner implementation of things like return filters
+      # (formerly known as 'partial search').
+      # Also args should never be nil, but that is required for Ruby 1.8 compatibility
+      def do_search(type, query="*:*", args=nil, &block)
+        query_string = create_query_string(type, query, args)
+        response = call_rest_service(query_string, args)
+        unless block.nil?
+          response["rows"].each { |rowset| block.call(rowset) unless rowset.nil?}
           unless (response["start"] + response["rows"].length) >= response["total"]
-            nstart = response["start"] + rows
-            search(type, query, sort, nstart, rows, &block)
+            args[:start] = response["start"] + args[:rows]
+            do_search(type, query, args, &block)
           end
           true
         else
@@ -52,14 +148,26 @@ class Chef
         end
       end
 
-      def list_indexes
-        @rest.get_rest("search")
+      # create the full rest url string
+      def create_query_string(type, query, args)
+        # create some default variables just so we don't break backwards compatibility
+        sort = args[:sort]
+        start = args[:start]
+        rows = args[:rows]
+
+        return "search/#{type}?q=#{escape(query)}&sort=#{escape(sort)}&start=#{escape(start)}&rows=#{escape(rows)}"
       end
 
-      private
-        def escape(s)
-          s && URI.escape(s.to_s)
+      def call_rest_service(query_string, args)
+        if args.key?(:filter_result)
+          response = @rest.post_rest(query_string, args[:filter_result])
+          response_rows = response['rows'].map { |row| row['data'] }
+        else
+          response = @rest.get_rest(query_string)
+          response_rows = response['rows']
         end
+        return response
+      end
     end
   end
 end
diff --git a/spec/unit/search/query_spec.rb b/spec/unit/search/query_spec.rb
index 7463e3bb3c..c7388a6234 100644
--- a/spec/unit/search/query_spec.rb
+++ b/spec/unit/search/query_spec.rb
@@ -20,80 +20,236 @@ require 'spec_helper'
 require 'chef/search/query'
 
 describe Chef::Search::Query do
-  before(:each) do
-    @rest = double("Chef::REST")
-    Chef::REST.stub(:new).and_return(@rest)
-    @query = Chef::Search::Query.new
-  end
+  let(:rest) { double("Chef::REST") }
+  let(:query) { Chef::Search::Query.new }
 
-  describe "search" do
-    before(:each) do
-      @response = {
+  shared_context "filtered search" do
+    let(:query_string) { "search/node?q=platform:rhel&sort=X_CHEF_id_CHEF_X%20asc&start=0&rows=1000" }
+    let(:server_url) { "https://api.opscode.com/organizations/opscode/nodes" }
+    let(:args) { { filter_key => filter_hash } }
+    let(:filter_hash) {
+      {
+        'env' => [ 'chef_environment' ],
+        'ruby_plat' => [ 'languages', 'ruby', 'platform' ]
+      }
+    }
+    let(:response) {
+      {
         "rows" => [
-          { "id" => "for you" },
-          { "id" => "hip hop" },
-          { "id" => "thought was down by law for you" },
-          { "id" => "kept it hard core for you" },
+          { "url" => "#{server_url}/my-name-is-node",
+            "data" => {
+              "env" => "elysium",
+              "ruby_plat" => "nudibranch"
+            }
+          },
+          { "url" => "#{server_url}/my-name-is-jonas",
+            "data" => {
+              "env" => "hades",
+              "ruby_plat" => "i386-mingw32"
+            }
+          },
+          { "url" => "#{server_url}/my-name-is-flipper",
+            "data" => {
+              "env" => "elysium",
+              "ruby_plat" => "centos"
+            }
+          },
+          { "url" => "#{server_url}/my-name-is-butters",
+            "data" => {
+              "env" => "moon",
+              "ruby_plat" => "solaris2",
+            }
+          }
         ],
         "start" => 0,
         "total" => 4
       }
-      @rest.stub(:get_rest).and_return(@response)
-    end
+    }
+  end
+
+  before(:each) do
+    Chef::REST.stub(:new).and_return(rest)
+    rest.stub(:get_rest).and_return(response)
+  end
+
+  describe "search" do
+    let(:response) { {
+      "rows" => [
+        { "name" => "my-name-is-node",
+          "chef_environment" => "elysium",
+          "platform" => "rhel",
+          "automatic" => {
+            "languages" => {
+              "ruby" => {
+                "platform" => "nudibranch",
+                "version" => "1.9.3",
+                "target" => "ming-the-merciless"
+              }
+            }
+          }
+        },
+        { "name" => "my-name-is-jonas",
+          "chef_environment" => "hades",
+          "platform" => "rhel",
+          "automatic" => {
+            "languages" => {
+              "ruby" => {
+                "platform" => "i386-mingw32",
+                "version" => "1.9.3",
+                "target" => "bilbo"
+              }
+            }
+          }
+        },
+        { "name" => "my-name-is-flipper",
+          "chef_environment" => "elysium",
+          "platform" => "rhel",
+          "automatic" => {
+            "languages" => {
+              "ruby" => {
+                "platform" => "centos",
+                "version" => "2.0.0",
+                "target" => "uno"
+              }
+            }
+          }
+        },
+        { "name" => "my-name-is-butters",
+          "chef_environment" => "moon",
+          "platform" => "rhel",
+          "automatic" => {
+            "languages" => {
+              "ruby" => {
+                "platform" => "solaris2",
+                "version" => "2.1.2",
+                "target" => "random"
+              }
+            }
+          }
+        },
+      ],
+      "start" => 0,
+      "total" => 4
+    } }
 
     it "should accept a type as the first argument" do
-      lambda { @query.search("foo") }.should_not raise_error
-      lambda { @query.search(:foo) }.should_not raise_error
-      lambda { @query.search(Hash.new) }.should raise_error(ArgumentError)
+      lambda { query.search("node") }.should_not raise_error
+      lambda { query.search(:node) }.should_not raise_error
+      lambda { query.search(Hash.new) }.should raise_error(Chef::Exceptions::InvalidSearchQuery, /(Hash)/)
     end
 
     it "should query for every object of a type by default" do
-      @rest.should_receive(:get_rest).with("search/foo?q=*:*&sort=X_CHEF_id_CHEF_X%20asc&start=0&rows=1000").and_return(@response)
-      @query = Chef::Search::Query.new
-      @query.search(:foo)
+      rest.should_receive(:get_rest).with("search/node?q=*:*&sort=X_CHEF_id_CHEF_X%20asc&start=0&rows=1000").and_return(response)
+      query.search(:node)
     end
 
     it "should allow a custom query" do
-      @rest.should_receive(:get_rest).with("search/foo?q=gorilla:dundee&sort=X_CHEF_id_CHEF_X%20asc&start=0&rows=1000").and_return(@response)
-      @query = Chef::Search::Query.new
-      @query.search(:foo, "gorilla:dundee")
+      rest.should_receive(:get_rest).with("search/node?q=platform:rhel&sort=X_CHEF_id_CHEF_X%20asc&start=0&rows=1000").and_return(response)
+      query.search(:node, "platform:rhel")
     end
 
     it "should let you set a sort order" do
-      @rest.should_receive(:get_rest).with("search/foo?q=gorilla:dundee&sort=id%20desc&start=0&rows=1000").and_return(@response)
-      @query = Chef::Search::Query.new
-      @query.search(:foo, "gorilla:dundee", "id desc")
+      rest.should_receive(:get_rest).with("search/node?q=platform:rhel&sort=id%20desc&start=0&rows=1000").and_return(response)
+      query.search(:node, "platform:rhel", "id desc")
     end
 
     it "should let you set a starting object" do
-      @rest.should_receive(:get_rest).with("search/foo?q=gorilla:dundee&sort=id%20desc&start=2&rows=1000").and_return(@response)
-      @query = Chef::Search::Query.new
-      @query.search(:foo, "gorilla:dundee", "id desc", 2)
+      rest.should_receive(:get_rest).with("search/node?q=platform:rhel&sort=id%20desc&start=2&rows=1000").and_return(response)
+      query.search(:node, "platform:rhel", "id desc", 2)
     end
 
     it "should let you set how many rows to return" do
-      @rest.should_receive(:get_rest).with("search/foo?q=gorilla:dundee&sort=id%20desc&start=2&rows=40").and_return(@response)
-      @query = Chef::Search::Query.new
-      @query.search(:foo, "gorilla:dundee", "id desc", 2, 40)
+      rest.should_receive(:get_rest).with("search/node?q=platform:rhel&sort=id%20desc&start=2&rows=40").and_return(response)
+      query.search(:node, "platform:rhel", "id desc", 2, 40)
+    end
+
+    it "should throw an exception if you pass to many options" do
+      lambda { query.search(:node, "platform:rhel", "id desc", 2, 40, "wrong") }
+        .should raise_error(Chef::Exceptions::InvalidSearchQuery, "Too many arguments! (4 for <= 3)")
     end
 
     it "should return the raw rows, start, and total if no block is passed" do
-      rows, start, total = @query.search(:foo)
-      rows.should equal(@response["rows"])
-      start.should equal(@response["start"])
-      total.should equal(@response["total"])
+      rows, start, total = query.search(:node)
+      rows.should equal(response["rows"])
+      start.should equal(response["start"])
+      total.should equal(response["total"])
     end
 
     it "should call a block for each object in the response" do
       @call_me = double("blocky")
-      @response["rows"].each { |r| @call_me.should_receive(:do).with(r) }
-      @query.search(:foo) { |r| @call_me.do(r) }
+      response["rows"].each { |r| @call_me.should_receive(:do).with(r) }
+      query.search(:node) { |r| @call_me.do(r) }
     end
 
     it "should page through the responses" do
       @call_me = double("blocky")
-      @response["rows"].each { |r| @call_me.should_receive(:do).with(r) }
-      @query.search(:foo, "*:*", nil, 0, 1) { |r| @call_me.do(r) }
+      response["rows"].each { |r| @call_me.should_receive(:do).with(r) }
+      query.search(:node, "*:*", nil, 0, 1) { |r| @call_me.do(r) }
+    end
+
+    context "when :filter_result is provided as a result" do
+      include_context "filtered search" do
+        let(:filter_key) { :filter_result }
+
+        before(:each) do
+          rest.should_receive(:post_rest).with(query_string, args[filter_key]).and_return(response)
+        end
+
+        it "should return start" do
+          start = query.search(:node, "platform:rhel", args)[1]
+          start.should == response['start']
+        end
+
+        it "should return total" do
+          total = query.search(:node, "platform:rhel", args)[2]
+          total.should == response['total']
+        end
+
+        it "should return rows with the filter applied" do
+          results = query.search(:node, "platform:rhel", args)[0]
+
+          results.each_with_index do |result, idx|
+            expected = response["rows"][idx]
+
+            result.should have_key("url")
+            result["url"].should == expected["url"]
+
+            result.should have_key("data")
+            filter_hash.keys.each do |filter_key|
+              result["data"].should have_key(filter_key)
+              result["data"][filter_key].should == expected["data"][filter_key]
+            end
+          end
+        end
+
+      end
+    end
+  end
+
+  describe "#partial_search" do
+    include_context "filtered search" do
+      let(:filter_key) { :keys }
+
+      it "should emit a deprecation warning" do
+        # partial_search calls search, so we'll stub search to return empty
+        query.stub(:search).and_return( [ [], 0, 0 ] )
+        Chef::Log.should_receive(:warn).with("DEPRECATED: The 'partial_search' api is deprecated, please use the search api with 'filter_result'")
+        query.partial_search(:node, "platform:rhel", args)
+      end
+
+      it "should return an array of filtered hashes" do
+        rest.should_receive(:post_rest).with(query_string, args[filter_key]).and_return(response)
+        results = query.partial_search(:node, "platform:rhel", args)
+
+        results.each_with_index do |result, idx|
+          expected = response["rows"][idx]
+
+          filter_hash.keys.each do |filter_key|
+            result.should have_key(filter_key)
+            result[filter_key].should == expected["data"][filter_key]
+          end
+        end
+      end
     end
   end
 end