diff options
author | Igor Shpakov <igorshp@gmail.com> | 2015-07-31 20:07:56 +0100 |
---|---|---|
committer | Lamont Granquist <lamont@scriptkiddie.org> | 2015-08-27 10:47:02 -0700 |
commit | bfa97d2ac03000b2335c9243f2e4fcc33610141c (patch) | |
tree | 0fa1534f86408ed3bc69c6f3136772f44cf1fb04 | |
parent | 096f4511f8418b6544ab02f30c0c23eba67bd964 (diff) | |
download | chef-bfa97d2ac03000b2335c9243f2e4fcc33610141c.tar.gz |
speed improvement for remote_directory() resource
problem:
doing ls(path) for a very large directory with > 1.8mil files takes
upwards of 10 minutes.
solution:
change filtering stage from files.reject! to files.reject
files.reject! does in place filtering and it looks like it copies the
whole array for every deletion. in my example it filtered out 300k rows,
so that's 300k array copies.
switching to files.reject decreases the runtime of that function from
650seconds down to 6-7 seconds.
-rw-r--r-- | lib/chef/provider/remote_directory.rb | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/lib/chef/provider/remote_directory.rb b/lib/chef/provider/remote_directory.rb index eaccce46cf..85ceb5cdae 100644 --- a/lib/chef/provider/remote_directory.rb +++ b/lib/chef/provider/remote_directory.rb @@ -67,7 +67,7 @@ class Chef ::File::FNM_DOTMATCH) # Remove current directory and previous directory - files.reject! do |name| + files = files.reject do |name| basename = Pathname.new(name).basename().to_s ['.', '..'].include?(basename) end |