Merge branch 'master' into mcquin/escape_globmcquin/escape_glob

Conflicts:
	CHANGELOG.md

4 files changed, 339 insertions, 7 deletions

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1c8a44b6a0..fe182840e0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -136,6 +136,7 @@
 * Set :ssl_verify_mode to :verify_peer by default.
 * Add homebrew provider for package resource, use it by default on OS X (Issue #1709)
 * Add escape_glob method to PathHelper, update glob operations.
+* Verify x509 properties of certificates in the :trusted_certs_dir during knife ssl check.
 
 ## Last Release: 11.14.2
 
diff --git a/DOC_CHANGES.md b/DOC_CHANGES.md
index 38a8a8c4f6..29fca72484 100644
--- a/DOC_CHANGES.md
+++ b/DOC_CHANGES.md
@@ -146,6 +146,228 @@ $ knife search node "platform:ubuntu" --filter-result "c_version:languages.c.gcc
 $
 ```
 
+# `knife ssl check` will verify X509 properties of your trusted certificates
+
+When you run `knife ssl check URL (options)` knife will verify if the certificate files, with extensions `*.crt` and `*.pem`
+in your `:trusted_certs_dir` have valid X509 certificate properties. Knife will generate warnings for certificates that
+do not meet X509 standards. OpenSSL **will not** use these certificates in verifying SSL connections.
+
+## Troubleshooting
+For each certificate that does not meet X509 specifications, a message will be displayed indicating why the certificate
+failed to meet these specifications. You may see output similar to
+
+```
+There are invalid certificates in your trusted_certs_dir.
+OpenSSL will not use the following certificates when verifying SSL connections:
+
+/path/to/your/invalid/certificate.crt: a message to help you debug
+```
+
+The documentation for resolving common issues with certificates is a work in progress. A few suggestions
+are outlined in the following sections. If you would like to help expand this documentation, please
+submit a pull request to [chef-docs](https://github.com/opscode/chef-docs) with your contribution.
+
+### Fetch the certificate again
+If the certificate was generated by your chef server, you may want to try downloading the certificate again.
+By default, the certificate is stored in the following location on the host where your chef-server runs:
+`/var/opt/chef-server/nginx/ca/SERVER_HOSTNAME.crt`. Copy that file into your `:trusted_certs_dir` using SSH,
+SCP, or some other secure method and run `knife ssl check URL (options)` again.
+
+### Generate a new certificate
+If you control the trusted certificate and you suspect it is bad (e.g., you've fetched the certificate again,
+but you're still getting warnings about it from `knife ssl check`), you might try generating a new certificate.
+
+#### Generate a certificate signing request
+If you used a certificate authority (CA) to authenticate your certificate, you'll need to generate
+a certificate signing request (CSR) to fetch a new certificate.
+
+If you don't have one already, you'll need to create an openssl configuration file. This example
+configuration file is saved in our current working directory as openssl.cnf
+
+```
+#
+# OpenSSL configuration file
+# ./openssl.cnf
+#
+
+[ req ]
+default_bits       = 1024       # Size of keys
+default_keyfile    = key.pem    # name of generated keys
+default_md         = md5        # message digest algorithm
+string_mask        = nombstr    # permitted characters
+distinguished_name = req_distinguished_name
+req_extensions     = v3_req
+
+[ req_distinguished_name ]
+# Variable name          Prompt string
+#---------------------   ----------------------------------
+0.organizationName     = Organization Name (company)
+organizationalUnitName = Organizational Unit Name (department, division)
+emailAddress           = Email Address
+emailAddress_max       = 40
+localityName           = Locality Name (city, district)
+stateOrProvinceName    = State or Province Name (full name)
+countryName            = Country Name (2 letter code)
+countryName_min        = 2
+countryName_max        = 2
+commonName             = Common Name (hostname, IP, or your name)
+commonName_max         = 64
+
+# Default values for the above, for consistency and less typing.
+# Variable name               Value
+#--------------------------   ------------------------------
+0.organizationName_default  = My Company
+localityName_default        = My Town
+stateOrProvinceName_default = State or Providence
+countryName_default         = US
+
+[ v3_req ]
+basicConstraints     = CA:FALSE   # This is NOT a CA certificate
+subjectKeyIdentifier = hash
+```
+
+You can use `openssl` to create a certificate from an existing private key
+```
+$ openssl req -new -extensions v3_req -key KEYNAME.pem -out REQNAME.pem -config ./openssl.cnf
+```
+or `openssl` can create a new private key simultaneously
+```
+$ openssl req -new -extensions v3_req -keyout KEYNAME.pem -out REQNAME.pem -config ./openssl.cnf
+```
+where `KEYNAME` is the path to your private key and `REQNAME` is the path to your CSR.
+
+You can verify your CSR was generated correctly
+```
+$ openssl req -noout -text -in REQNAME.pem
+```
+
+The final step is to submit your CSR to your certificate authority (CA) for signing.
+
+### Generate a self-signed (root) certificate
+You'll need to modify your openssl configuration file, or create a separate file, for
+generating root certificates.
+
+```
+#
+# OpenSSL configuration file
+# ./openssl.cnf
+#
+
+dir = .
+
+[ ca ]
+default_ca = CA_default
+
+[ CA_default ]
+serial        = $dir/serial
+database      = $dir/certindex.txt
+new_certs_dir = $dir/certs
+certificate   = $dir/cacert.pem
+private_key   = $dir/private/cakey.pem
+default_days  = 365
+default_md    = md5
+preserve      = no
+email_in_dn   = no
+nameopt       = default_ca
+certopt       = default_ca
+policy        = policy_match
+
+[ policy_match ]
+countryName            = match
+stateOrProvinceName    = match
+organizationName       = match
+organizationalUnitName = optional
+commonName             = supplied
+emailAddress           = optional
+
+[ v3_ca ]
+basicConstraints       = CA:TRUE   # This is a CA certificate
+subjectKeyIdentifier   = hash
+authorityKeyIdentifier = keyid:always,issuer:always
+```
+
+You can now create a root certificate. If you have a private key you would like
+to use
+```
+$ openssl req -new -x509 -extensions v3_ca -key KEYNAME.pem -out CERTNAME.pem -config ./openssl.cnf
+```
+or `openssl` can create a new private key simultaneously
+```
+$ openssl req -new -x509 -extensions v3_ca -keyout KEYNAME.pem -out CERTNAME.pem -config ./openssl.cnf
+```
+where `KEYNAME` is the path to your private key and `REQNAME` is the path to your CSR.
+
+At this point, you should add the generated certificate to your trusted certificates as well as
+replace the old server certificate. Furthermore, you should regenerate any certificates that
+were signed by the previous root certificate.
+
+For more information and an example on how to set up your server to generate certificates
+check out this post on [setting up OpenSSL to create certificates](http://www.flatmtn.com/article/setting-openssl-create-certificates).
+
+#### Signing certificates
+Use your root certificate to sign certificate requests sent to your server
+```
+$ openssl ca -out CERTNAME.pem -config ./openssl.cnf -infiles REQNAME.pem
+```
+This creates the certificate `CERTNAME.pem` generated from CSR `REQNAME.pem`. You
+should send `CERTNAME.pem` back to the client who generated the CSR.
+
+### Certificate attributes
+When creating certificates and certificate signing requests, you will be prompted for
+information via the command line. These are your certificate attributes.
+
+RDN | Name | Explanation | Examples
+:---: | :---: | --- | ---
+CN | Common Name | You server's FQDN, or YOUR_SERVER Certificate Authority if root certificate | mail.domain.com, *.domain.com, MyServer Certificate Authority
+OU | Organizational Unit | (Optional) Additional organization information. | mail server, R&D
+O | Organization | The exact name of your organization. Do not abbreviate. | DevOpsRUs Inc.
+L | Locality | The city where your organization is located | Seattle
+S | State or Province | The state or province where your organization is located. Do not abbreviate. | Washington
+C | Country Name | 2-letter ISO abbreviation for your country. | US
+ | Email Address | How you or another maintainer can be reached. | maintainers@devopsr.us
+
+If you examine the `policy_match` section in the openssl configuration file example from the section on generating
+self signed certificates, you'll see specifications that CSRs need to match the countryName, stateOrProvinceName,
+and the organizationName. CSRs whose CN, S, and O values do not match those of the root certificate will not be
+signed by that root certificate. You can modify these requirements as desired.
+
+### Key usage
+A keyUsage field can be added to your `v3_req` and `v3_ca` sections of your configuration file.
+Key usage extensions define the purpose of the public key contained in a certificate, limiting what
+it can and cannot be used for.
+
+Extension | Description
+--- | ---
+digitalSignature | Use when the public key is used with a digital signature mechanism to support security services other than non-repudiation, certificate signing, or CRL signing. A digital signature is often used for entity authentication and data origin authentication with integrity
+nonRepudiation | Use when the public key is used to verify digital signatures used to provide a non-repudiation service. Non-repudiation protects against the signing entity falsely denying some action (excluding certificate or CRL signing).
+keyEncipherment | Use when a certificate will be used with a protocol that encrypts keys.
+dataEncipherment | Use when the public key is used for encrypting user data, other than cryptographic keys.
+keyAgreement | Use when the sender and receiver of the public key need to derive the key without using encryption. This key can then can be used to encrypt messages between the sender and receiver. Key agreement is typically used with Diffie-Hellman ciphers.
+certificateSigning | Use when the subject public key is used to verify a signature on certificates. This extension can be used only in CA certificates.
+cRLSigning | Use when the subject public key is to verify a signature on revocation information, such as a CRL.
+encipherOnly | Use only when key agreement is also enabled. This enables the public key to be used only for enciphering data while performing key agreement.
+decipherOnly | Use only when key agreement is also enabled. This enables the public key to be used only for deciphering data while performing key agreement.
+[Source](http://www-01.ibm.com/support/knowledgecenter/SSKTMJ_8.0.1/com.ibm.help.domino.admin.doc/DOC/H_KEY_USAGE_EXTENSIONS_FOR_INTERNET_CERTIFICATES_1521_OVER.html)
+
+### Subject Alternative Names
+Subject alternative names (SANs) allow you to list host names to protect with a single certificate.
+To create a certificate using SANs, you'll need to add a `subjectAltName` field to your `v3_req` section
+in your openssl configuration file
+
+```
+[ v3_req ]
+basicConstraints     = CA:FALSE   # This is NOT a CA certificate
+subjectKeyIdentifier = hash
+subjectAltName       = @alt_names
+
+[alt_names]
+DNS.1 = kb.example.com
+DNS.2 = helpdesk.example.org
+DNS.3 = systems.example.net
+IP.1  = 192.168.1.1
+IP.2  = 192.168.69.14
+```
+
 ### Reboot resource in core
 The `reboot` resource will reboot the server, a necessary step in some installations, especially on Windows. If this resource is used with notifications, it must receive explicit `:immediate` notifications only: results of delayed notifications are undefined. Currently supported on Windows, Linux, and OS X; will work incidentally on some other Unixes.
 
diff --git a/lib/chef/knife/ssl_check.rb b/lib/chef/knife/ssl_check.rb
index e98469d5aa..f2d368ff39 100644
--- a/lib/chef/knife/ssl_check.rb
+++ b/lib/chef/knife/ssl_check.rb
@@ -106,6 +106,22 @@ class Chef
         end
       end
 
+      def verify_X509
+        cert_debug_msg = ""
+        trusted_certificates.each do |cert_name|
+          message = check_X509_certificate(cert_name)
+          unless message.nil?
+            cert_debug_msg << File.expand_path(cert_name) + ": " + message + "\n"
+          end
+        end
+
+        unless cert_debug_msg.empty?
+          debug_invalid_X509(cert_debug_msg)
+        end
+
+        true # Maybe the bad certs won't hurt...
+      end
+
       def verify_cert
         ui.msg("Connecting to host #{host}:#{port}")
         verify_peer_socket.connect
@@ -127,6 +143,35 @@ class Chef
         false
       end
 
+      def debug_invalid_X509(cert_debug_msg)
+        ui.msg("\n#{ui.color("Configuration Info:", :bold)}\n\n")
+        debug_ssl_settings
+        debug_chef_ssl_config
+
+        ui.warn(<<-BAD_CERTS)
+There are invalid certificates in your trusted_certs_dir.
+OpenSSL will not use the following certificates when verifying SSL connections:
+
+#{cert_debug_msg}
+
+#{ui.color("TO FIX THESE WARNINGS:", :bold)}
+
+We are working on documentation for resolving common issues uncovered here.
+
+* If the certificate is generated by the server, you may try redownloading the
+server's certificate. By default, the certificate is stored in the following
+location on the host where your chef-server runs:
+
+  /var/opt/chef-server/nginx/ca/SERVER_HOSTNAME.crt
+
+Copy that file to your trusted_certs_dir (currently: #{configuration.trusted_certs_dir})
+using SSH/SCP or some other secure method, then re-run this command to confirm
+that the server's certificate is now trusted.
+
+BAD_CERTS
+        # @TODO: ^ needs URL once documentation is posted.
+      end
+
       def debug_invalid_cert
         noverify_socket.connect
         issuer_info = noverify_socket.peer_cert.issuer
@@ -148,7 +193,7 @@ where your chef-server runs:
 
   /var/opt/chef-server/nginx/ca/SERVER_HOSTNAME.crt
 
-Copy that file to you trusted_certs_dir (currently: #{configuration.trusted_certs_dir})
+Copy that file to your trusted_certs_dir (currently: #{configuration.trusted_certs_dir})
 using SSH/SCP or some other secure method, then re-run this command to confirm
 that the server's certificate is now trusted.
 
@@ -197,17 +242,36 @@ ADVICE
 
       def run
         validate_uri
-        if verify_cert && verify_cert_host
+        if verify_X509 && verify_cert && verify_cert_host
           ui.msg "Successfully verified certificates from `#{host}'"
         else
           exit 1
         end
       end
 
+      private
+      def trusted_certificates
+        if configuration.trusted_certs_dir && Dir.exist?(configuration.trusted_certs_dir)
+          Dir.glob(File.join(configuration.trusted_certs_dir, "*.{crt,pem}"))
+        else
+          []
+        end
+      end
+
+      def check_X509_certificate(cert_file)
+        store = OpenSSL::X509::Store.new
+        cert = OpenSSL::X509::Certificate.new(IO.read(File.expand_path(cert_file)))
+        begin
+          store.add_cert(cert)
+          # test if the store can verify the cert we just added
+          unless store.verify(cert) # true if verified, false if not
+            return store.error_string
+          end
+        rescue OpenSSL::X509::StoreError => e
+          return e.message
+        end
+        return nil
+      end
     end
   end
 end
-
-
-
-
diff --git a/spec/unit/knife/ssl_check_spec.rb b/spec/unit/knife/ssl_check_spec.rb
index 32405a5977..bb803ce2ca 100644
--- a/spec/unit/knife/ssl_check_spec.rb
+++ b/spec/unit/knife/ssl_check_spec.rb
@@ -95,6 +95,49 @@ E
     end
   end
 
+  describe "verifying trusted certificate X509 properties" do
+    let(:name_args) { %w{https://foo.example.com:8443} }
+
+    let(:trusted_certs_dir) { File.join(CHEF_SPEC_DATA, "trusted_certs") }
+    let(:trusted_cert_file) { File.join(trusted_certs_dir, "example.crt") }
+
+    let(:store) { OpenSSL::X509::Store.new }
+    let(:certificate) { OpenSSL::X509::Certificate.new(IO.read(trusted_cert_file)) }
+
+    before do
+      Chef::Config[:trusted_certs_dir] = trusted_certs_dir
+      ssl_check.stub(:trusted_certificates).and_return([trusted_cert_file])
+      store.stub(:add_cert).with(certificate)
+      OpenSSL::X509::Store.stub(:new).and_return(store)
+      OpenSSL::X509::Certificate.stub(:new).with(IO.read(trusted_cert_file)).and_return(certificate)
+      ssl_check.stub(:verify_cert).and_return(true)
+      ssl_check.stub(:verify_cert_host).and_return(true)
+    end
+
+    context "when the trusted certificates have valid X509 properties" do
+      before do
+        store.stub(:verify).with(certificate).and_return(true)
+      end
+
+      it "does not generate any X509 warnings" do
+        expect(ssl_check.ui).not_to receive(:warn).with(/There are invalid certificates in your trusted_certs_dir/)
+        ssl_check.run
+      end
+    end
+
+    context "when the trusted certificates have invalid X509 properties" do
+      before do
+        store.stub(:verify).with(certificate).and_return(false)
+        store.stub(:error_string).and_return("unable to get local issuer certificate")
+      end
+
+      it "generates a warning message with invalid certificate file names" do
+        expect(ssl_check.ui).to receive(:warn).with(/#{trusted_cert_file}: unable to get local issuer certificate/)
+        ssl_check.run
+      end
+    end
+  end
+
   describe "verifying the remote certificate" do
     let(:name_args) { %w{https://foo.example.com:8443} }
 
@@ -117,6 +160,7 @@ E
     context "when the remote host's certificate is valid" do
 
       before do
+        ssl_check.should_receive(:verify_X509).and_return(true) # X509 valid certs (no warn)
         ssl_socket.should_receive(:connect) # no error
         ssl_socket.should_receive(:post_connection_check).with("foo.example.com") # no error
       end
@@ -148,6 +192,7 @@ E
 
       context "when the certificate's CN does not match the hostname" do
         before do
+          ssl_check.should_receive(:verify_X509).and_return(true) # X509 valid certs
           ssl_socket.should_receive(:connect) # no error
           ssl_socket.should_receive(:post_connection_check).
             with("foo.example.com").
@@ -167,6 +212,7 @@ E
 
       context "when the cert is not signed by any trusted authority" do
         before do
+          ssl_check.should_receive(:verify_X509).and_return(true) # X509 valid certs
           ssl_socket.should_receive(:connect).
             and_raise(OpenSSL::SSL::SSLError)
           ssl_socket_for_debug.should_receive(:connect)
@@ -184,4 +230,3 @@ E
   end
 
 end
-