Bug 1707130 NSS should use modern algorithms in PKCS#12 files by default r=mt

Also fixes:
Bug 452464 pk12util -o fails when -C option specifies AES or Camellia ciphers

Related:
Bug 1694689 Firefox should use modern algorithms in PKCS#12 files by default
Bug 452471 pk12util -o fails when -c option specifies pkcs12v2 PBE ciphers


The base of this fix is was a simple 3 line fix in pkcs12.c, changing the initial setting of cipher and cert cipher.

Overview for why this patch is larger than just 3 lines:
1. First issue was found in trying to change the mac hashing value.
   a. While the decrypt side knew how to handle SHA2 hashes, the equivalent code was not updated on the encrypt side. I refactored that code and placed the common function in p12local.c. Now p12e.c and p12d.c share common code to find the required function to produce the mac key.
   b. The prf hmac was hard coded to SHA1. I changed the code to pass the hmac matching the hashing algorithm for the mac. This required changes to p12e.c to calculate and pass the new hmac as well and adding new PK11_ExportEncryptedPrivateKey and PK11_ExportEncryptedPrivKey to take the PKCS #5 v2 parameters. I also corrected an error which prevented pkcs12 encoding of ciphers other than AES.
2. Once I've made my changes, I realized we didn't have a way of testing them. While we had code that verified that particular sets of parameters for pkcs12 worked together and could be listed and imported, we didn't have a way to verify what algorithms were actually generated by our tools.
  a. pk12util -l doesn't list the encryption used for the certs, so I updated pp to take a pkcs12 option. In doing so I had to update pp to handle indefinite encoding when decoding blocks. I also factored that decoding out in it's own function so the change only needed to be placed once. Finally I renabled a function which prints the output of an EncryptedPrivate key. This function was disabled long ago when the Encrypted Private key info was made private for NSS. It has since been exported, so these functions could easily be enabled (archeological note: I verified that this disabling was not a recent think I found I had done it back when I still have a netscape email address;).
 b. I updated tools.sh to us the new pp -t pkcs12 feature to verify that the key encryption, cert encryption, and hash functions matched what we expected when we exported a new key. I also updated tools.sh to handle the new hash variable option to pk12util.
 c. I discovered several tests commented out with comments that the don't work. I enabled those tests and discovered that they can now encrypt, but the can't decrypt because of pkcs12 policy. I updated the policy code, but I updated it to use the new NSS system wide policy mechanism. This enabled all the ciphers to work. There is still policy work to do. The pk12 policy currently only prevents ciphers from use in decrypting the certificates, not decrypting the keys and not encrypting. I left that for future work.
3. New options for pp and pk12util were added to the man pages for these tools.

---------------------------------------------------------------------------
With that in mind, here's a file by file description of the patch:

automation/abi-check/expected-report-libnss3.so.txt
-Add new exported functions. (see lib/nss/nss.def)

cmd/lib/basicutil.h:
-Removed the HAVE_EPV_TEMPLATE ifdefs (NSS has exported the Encrypted Private Key data structure for a while now.

cmd/lib/secutil.c:
global: Updated several functions to take a const char * m (message) rather than a char * m
global: Made the various PrintPKCS7 return an error code.
global: Added a state variable to be passed around the various PKCS7 Print functions. It gives the proper context to interpret PKCS7 Data Content. PKCS 12 used PKCS7 to package the various PKCS12 Safes and Bags.
-Updated SECU_StripTagAndLength to handle indefinite encoding, and to set the Error code.
-Added SECU_ExtractDERAndStep to grab the next DER Tag, Length, and Data.
-Updated secu_PrintRawStringQuotesOptional to remove the inline DER parsing and use SECU_ExtractDERAndStep().
-Updated SECU_PrintEncodedObjectID to return the SECOidTag just like SECU_PrintObjectID.
-Renable SECU_PrintPrivateKey
-Added secu_PrintPKCS12Attributes to print out the Attributes tied to a PKCS #12 Bag
-Added secu_PrintPKCS12Bag to print out a PKCS #12 Bag
-Added secu_PrintPKCS7Data, which uses the state to determine what it was printing out.
-Added secu_PrintDERPKCS7ContentInfo which is identical to the global function SECU_PrintPKCS7ContentInfo except it takes a state variable. The latter function now calls the former.
-Added secu_PrintPKCS12DigestInfo to print the Hash information of the Mac. DigestInfo is the name in the PKCS 12 spec.
-Added secu_PrintPKCS12MacData to print the Mac portion of the PKCS 12 file.
-Added SECU_PrintPKCS12 to print otu the pkcs12 file.

cmd/lib/secutil.h
-Added string for pkc12 for the command line of pp
reenabled SECU_PrintPrivateKey
-Added SECU_PrintPKCS12 for export.

cmd/pk12util/pk12util.c
-Added the -M option to specify a hash algorithm for the mac.
updated P12U_ExportPKCS12Object: pass the hash algorithm to the PasswordIntegrity handler.
-Added PKCS12U_FindTagFromString: generalized string to SECOidTag which only filters based on the oid having a matching PKCS #11 mechanism.
updated PKCS12U_MapCipherFromString to call use PKCS12U_FindTagFromString to get the candidate tag before doing it's post processing to decide if the tag is really an encryption algorithm.
-Added PKCS12U_MapHashFromString with is like MapCipherFromString except it verifies the resulting tag is a hash object.
-Updated main to 1) change the default cipher, change the default certCipher, and process the new hash argument. NOTE: in the old code we did not encrypt the certs in FIPS mode. That's because the certs were encrypted with RC4 in the default pkcs12 file, which wasn't a FIPS algorithm. Since AES is, we can use it independent on whether or not we are in FIPS mode.

cmd/pp/pp.c
-Added the pkcs12 option which calls SECU_PrintPKCS12 from secutil.c

lib/nss/nss.def
-Add exports to the new PK11_ExportEncryptedPrivKeyInfoV2 and PK11_ExportEncryptedPrivateKeyInfoV2 (V2 means PKCS 5 v2, not Version 2 of ExportEncrypted*Info).
-Add export for the old HASH_GetHMACOidTagByHashOidTag which should have been exported long ago to avoid the proliferation of copies of this function in places like ssl.

lib/pk11wrap/pk11akey.c
-Add PK11_ExportEncryptedPrivKeyInfoV2 (which the old function now calls), which takes the 3 PKCS 5 v2 parameters. The underlying pkcs5 code can fill in missing tags if necessary, but supplying all three gives the caller full control of the underlying pkcs5 PBE used.
-Add PK11_ExportEncryptedPrivateKeyInfoV2, same as the above function except it takes a cert which is used to look up the private key. It's the function that pkcs12 actually uses, but the former was exported for completeness.

lib/pk11wrap/pk11pub.h
-Added the new PK11_ExportEncryptedPriv*KeyInfoV2 functions.

lib/pkcs12/p12d.c
-Remove the switch statement and place it in p12local.c so that p12e.c can use the same function.

lib/pkc12/p12e.c
-Remove the unnecessary privAlg check so we can encode any mechanism we support. This only prevented encoding certificates in the pk12 file, not the keys.
-add code to get the hmac used in the pbe prf from the integrity hash, which is under application control.
-Do the same for key encryption, then use the new PK11_ExportEncryptedPrivateKeyInfo to pass that hash value.
-Use the new sec_pkcs12_algtag_to_keygen_mech so there is only one switch statement to update rather than 2.
-Update the hash data to old the length of the largest hash rather than the length of a SHA1 hash.

lib/pkcs12/p12local.c
- Add new function new sec_pkcs12_algtag_to_keygen_mech to factor out the common switch statement between p12e and p12d.

lib/pkcs12/p12local.h
-Export the new sec_pkcs12_algtag_to_keygen_mech

lib/pkcs12/p12plcy.c
-Map the old p12 policy functions to use the new NSS_GetAlgorithmPolicy. We keep the old table so that applications can change the policy with the old PKCS12 specific defines (so the old code keeps working). NOTE: policies now default to true rather than false.

lib/util/secoidt.h
-Add new NSS_USE_ALG_IN_PKCS12 used by pk11plcy.c
NOTE: I have not updated the policy table in pk11wrap/pk11pars.c, so we can't yet control pkcs12 policy with the nss system policy table. That's a patch for another time.

test/tools/tool.sh
-global: Remove trailing spaces
-global: DEFAULT is changed to 'default'
-Update the PBE mechanism to exactly match the string in secoid.c. PKCS #12 does case independent compares, so case doesn't matter there, but now I'm comparing to the output of pp, and I didn't want to spend the time to figure out case independent compares in bash.
-Add our defauts and shell variables at the top so there are easy to change in the future.
export_with_*** have all been colapsed into a single export_p12_file which handles taking 'default' and turning off that argument.
-Add for loops for the hash functions.
-Restore the camellia ciphers back now that they work.
-Restore the pkcs12V2pbe back now that they work.
-Collect various pbe types into single variables and use those variables in loops
-Reduce the number of tests ran in optimized mode (which takes 60x the time to do a pbe then than debug mode based on a larger iterator).
-Add verify_p12 which dumps out the p12 file and makes sure the expected CERT_ENCRYPTION, KEY_ENCRYPTION, and HASH are used.

doc/pp.xml
-Add pkcs12 option

doc/pk12util.xml
-Add -M option
-Update synopsis with options in the description but not in the synopsis

Differential Revision: https://phabricator.services.mozilla.com/D113699

1 files changed, 320 insertions, 186 deletions

diff --git a/tests/tools/tools.sh b/tests/tools/tools.sh
index 7cf1ef73f..19d8b1903 100644..100755
--- a/tests/tools/tools.sh
+++ b/tests/tools/tools.sh
@@ -1,4 +1,4 @@
-#! /bin/bash  
+#! /bin/bash
 #
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this
@@ -8,7 +8,7 @@
 #
 # mozilla/security/nss/tests/tools/tools.sh
 #
-# Script to test basic functionality of NSS tools 
+# Script to test basic functionality of NSS tools
 #
 # needs to work on all Unix and Windows platforms
 #
@@ -23,29 +23,20 @@
 ########################################################################
 
   export pkcs12v2pbeWithSha1And128BitRc4=\
-"PKCS #12 V2 PBE With SHA-1 and 128 Bit RC4"
+"PKCS #12 V2 PBE With SHA-1 And 128 Bit RC4"
 
   export pkcs12v2pbeWithSha1And40BitRc4=\
-"PKCS #12 V2 PBE With SHA-1 and 40 Bit RC4"
+"PKCS #12 V2 PBE With SHA-1 And 40 Bit RC4"
 
   export pkcs12v2pbeWithSha1AndTripleDESCBC=\
-"PKCS #12 V2 PBE With SHA-1 and 3KEY Triple DES-CBC"
+"PKCS #12 V2 PBE With SHA-1 And 3KEY Triple DES-CBC"
 
   export pkcs12v2pbeWithSha1And128BitRc2Cbc=\
-"PKCS #12 V2 PBE With SHA-1 and 128 Bit RC2 CBC"
+"PKCS #12 V2 PBE With SHA-1 And 128 Bit RC2 CBC"
 
   export pkcs12v2pbeWithSha1And40BitRc2Cbc=\
-"PKCS #12 V2 PBE With SHA-1 and 40 Bit RC2 CBC"
+"PKCS #12 V2 PBE With SHA-1 And 40 Bit RC2 CBC"
 
-  export pkcs12v2pbeWithMd2AndDESCBC=\
-"PKCS #5 Password Based Encryption with MD2 and DES-CBC"
-
-  export pkcs12v2pbeWithMd5AndDESCBC=\
-"PKCS #5 Password Based Encryption with MD5 and DES-CBC"
-
-  export pkcs12v2pbeWithSha1AndDESCBC=\
-"PKCS #5 Password Based Encryption with SHA-1 and DES-CBC"
-  
   export pkcs5pbeWithMD2AndDEScbc=\
 "PKCS #5 Password Based Encryption with MD2 and DES-CBC"
 
@@ -55,8 +46,28 @@
   export pkcs5pbeWithSha1AndDEScbc=\
 "PKCS #5 Password Based Encryption with SHA-1 and DES-CBC"
 
+  # if we change the defaults in pk12util, update these variables
+  export CERT_ENCRYPTION_DEFAULT="AES-128-CBC"
+  export KEY_ENCRYPTION_DEFAULT="AES-256-CBC"
+  export HASH_DEFAULT="SHA-256"
+
+  export PKCS5v1_PBE_CIPHERS="${pkcs5pbeWithMD2AndDEScbc},\
+${pkcs5pbeWithMD5AndDEScbc},\
+${pkcs5pbeWithSha1AndDEScbc}"
+  export PKCS12_PBE_CIPHERS="${pkcs12v2pbeWithSha1And128BitRc4},\
+${pkcs12v2pbeWithSha1And40BitRc4},\
+${pkcs12v2pbeWithSha1AndTripleDESCBC},\
+${pkcs12v2pbeWithSha1And128BitRc2Cbc},\
+${pkcs12v2pbeWithSha1And40BitRc2Cbc}"
+  export PKCS5v2_PBE_CIPHERS="RC2-CBC,DES-EDE3-CBC,AES-128-CBC,AES-192-CBC,\
+AES-256-CBC,CAMELLIA-128-CBC,CAMELLIA-192-CBC,CAMELLIA-256-CBC"
+  export PBE_CIPHERS="${PKCS5v1_PBE_CIPHERS},${PKCS12_PBE_CIPHERS},${PKCS5v2_PBE_CIPHERS}"
+  export PBE_CIPHERS_CLASSES="${pkcs5pbeWithSha1AndDEScbc},\
+${pkcs12v2pbeWithSha1AndTripleDESCBC},AES-256-CBC,default"
+  export PBE_HASH="SHA-1,SHA-224,SHA-256,SHA-384,SHA-512,default"
+
 ############################## tools_init ##############################
-# local shell function to initialize this script 
+# local shell function to initialize this script
 ########################################################################
 tools_init()
 {
@@ -117,7 +128,7 @@ list_p12_file()
 {
   echo "$SCRIPTNAME: Listing Alice's pk12 file"
   echo "pk12util -l ${1} -w ${R_PWFILE}"
-    
+
   ${BINDIR}/pk12util -l ${1} -w ${R_PWFILE} 2>&1
   ret=$?
   html_msg $ret 0 "Listing ${1} (pk12util -l)"
@@ -131,105 +142,70 @@ import_p12_file()
 {
   echo "$SCRIPTNAME: Importing Alice's pk12 ${1} file"
   echo "pk12util -i ${1} -d ${P_R_COPYDIR} -k ${R_PWFILE} -w ${R_PWFILE}"
-    
+
   ${BINDIR}/pk12util -i ${1} -d ${P_R_COPYDIR} -k ${R_PWFILE} -w ${R_PWFILE} 2>&1
   ret=$?
   html_msg $ret 0 "Importing ${1} (pk12util -i)"
   check_tmpfile
 }
 
-########################################################################
-# Export the key and cert to a p12 file using default ciphers
-########################################################################
-export_with_default_ciphers() 
-{
-  echo "$SCRIPTNAME: Exporting Alice's key & cert with [default:default] (pk12util -o)"
-  echo "pk12util -o Alice.p12 -n \"Alice\" -d ${P_R_ALICEDIR} \\"
-  echo "         -k ${R_PWFILE} -w ${R_PWFILE}"
-  ${BINDIR}/pk12util -o Alice.p12 -n "Alice" -d ${P_R_ALICEDIR} \
-                       -k ${R_PWFILE} -w ${R_PWFILE} 2>&1  
-  ret=$?  
-  html_msg $ret 0 "Exporting Alices's key & cert with [default:default] (pk12util -o)"
-  check_tmpfile
-  return $ret
-}
 
 ########################################################################
-# Exports key/cert to a p12 file, the key encryption cipher is specified
-# and the cert encryption cipher is blank for default.
+# Export the key and cert from the specified p12 file
 ########################################################################
-export_with_key_cipher() 
+export_p12_file()
 {
-  # $1 key encryption cipher   
-  echo "$SCRIPTNAME: Exporting with [${1}:default]"
-  echo "pk12util -o Alice.p12 -n \"Alice\" -d ${P_R_ALICEDIR} \\"
-  echo "         -k ${R_PWFILE} -w ${R_PWFILE} -c ${1}"
-  ${BINDIR}/pk12util -o Alice.p12 -n "Alice" -d ${P_R_ALICEDIR} \
-                     -k ${R_PWFILE} -w ${R_PWFILE} -c "${1}" 2>&1  
-  ret=$?  
-  html_msg $ret 0 "Exporting with [${1}:default] (pk12util -o)"
-  check_tmpfile
-  return $ret
-}
-
-########################################################################
-# Exports key/cert to a p12 file, the key encryption cipher is left
-# empty for default and the cert encryption cipher is specified.
-########################################################################
-export_with_cert_cipher() 
-{
-  # $1 certificate encryption cipher
-  echo "$SCRIPTNAME: Exporting with [default:${1}]"
-  echo "pk12util -o Alice.p12 -n \"Alice\" -d ${P_R_ALICEDIR} \\"
-  echo "         -k ${R_PWFILE} -w ${R_PWFILE} -C ${1}"
-  ${BINDIR}/pk12util -o Alice.p12 -n "Alice" -d ${P_R_ALICEDIR} \
-                     -k ${R_PWFILE} -w ${R_PWFILE} -C "${1}" 2>&1  
-  ret=$?  
-  html_msg $ret 0 "Exporting with [default:${1}] (pk12util -o)"
-  check_tmpfile
-  return $ret
-}
+  # $1 p12 file
+  # $2 cert to export
+  # $3 certdb
+  # $4 key encryption cipher or "default"
+  # $5 certificate encryption cipher or "default"
+  # $6 hash algorithm or "default"
+  KEY_CIPHER_OPT="-c"
+  KEY_CIPHER="${4}"
+  CERT_CIPHER_OPT="-C"
+  CERT_CIPHER="${5}"
+  HASH_ALG_OPT="-M"
+  HASH_ALG="${6}"
+
+  if [ "${KEY_CIPHER}" = "default" ]; then
+    KEY_CIPHER_OPT=""
+    KEY_CIPHER=""
+  fi
+  if [ "${CERT_CIPHER}" = "default" ]; then
+    CERT_CIPHER_OPT=""
+    CERT_CIPHER=""
+  fi
+  if [ "${HASH_ALG}" = "default" ]; then
+    HASH_ALG_OPT=""
+    HASH_ALG=""
+  fi
 
-########################################################################
-# Exports key/cert to a p12 file, both the key encryption cipher and
-# the cert encryption cipher are specified.
-########################################################################
-export_with_both_key_and_cert_cipher()
-{
-  # $1 key encryption cipher or ""
-  # $2 certificate encryption cipher or ""
-  
-  echo "pk12util -o Alice.p12 -n \"Alice\" -d ${P_R_ALICEDIR} \\"
-  echo "         -k ${R_PWFILE} -w ${R_PWFILE} -c ${1} -C ${2}"     
-  ${BINDIR}/pk12util -o Alice.p12 -n Alice -d ${P_R_ALICEDIR} \
+  echo "pk12util -o \"${1}\" -n \"${2}\" -d \"${3}\" \\"
+  echo "         -k ${R_PWFILE} -w ${R_PWFILE} \\"
+  echo "         ${KEY_CIPHER_OPT} \"${KEY_CIPHER}\" \\"
+  echo "         ${CERT_CIPHER_OPT} \"${CERT_CIPHER}\" \\"
+  echo "         ${HASH_ALG_OPT} \"${HASH_ALG}\""
+  ${BINDIR}/pk12util -o "${1}" -n "${2}" -d "${3}" \
                        -k ${R_PWFILE} -w ${R_PWFILE} \
-                       -c "${1}" -C "${2}" 2>&1  
-  ret=$?    
-  html_msg $ret 0 "Exporting with [${1}:${2}] (pk12util -o)"
+                       ${KEY_CIPHER_OPT} "${KEY_CIPHER}" \
+                       ${CERT_CIPHER_OPT} "${CERT_CIPHER}" \
+                       ${HASH_ALG_OPT} "${HASH_ALG}" 2>&1
+  ret=$?
+  html_msg $ret 0 "Exporting with [${4}:${5}:${6}] (pk12util -o)"
   check_tmpfile
+  verify_p12 "${1}" "${4}" "${5}" "${6}"
   return $ret
 }
 
 ########################################################################
-# Exports key and cert to a p12 file, both the key encryption cipher 
-# and the cert encryption cipher are specified. The key and cert are
-# imported and the p12 file is listed
+# Exports key and cert to a p12 file, the key encryption cipher,
+# the cert encryption cipher, and/or the hash algorithm are specified.
+# The key and cert are imported and the p12 file is listed
 ########################################################################
 export_list_import()
 {
-  # $1 key encryption cipher
-  # $2 certificate encryption cipher
-    
-  if [ "${1}" != "DEFAULT" -a "${2}" != "DEFAULT" ]; then
-      export_with_both_key_and_cert_cipher "${1}" "${2}"
-  elif [ "${1}" != "DEFAULT" -a "${2}" = "DEFAULT" ]; then
-      export_with_key_cipher "${1}"
-  elif [ "${1}" = "DEFAULT" -a "${2}" != "DEFAULT" ]; then
-      export_with_cert_cipher "${2}"
-  else
-      export_with_default_ciphers
-  fi
-    
+  export_p12_file Alice.p12 Alice "${P_R_ALICEDIR}" "${@}"
   list_p12_file Alice.p12
   import_p12_file Alice.p12
 }
@@ -239,20 +215,17 @@ export_list_import()
 # List the contents of and import from the p12 file.
 ########################################################################
 tools_p12_export_list_import_all_pkcs5pbe_ciphers()
-{  
-  # specify each on key and cert cipher
-  for key_cipher in "${pkcs5pbeWithMD2AndDEScbc}" \
-                    "${pkcs5pbeWithMD5AndDEScbc}" \
-                    "${pkcs5pbeWithSha1AndDEScbc}"\
-                    "DEFAULT"; do
-      for cert_cipher in "${pkcs5pbeWithMD2AndDEScbc}" \
-                         "${pkcs5pbeWithMD5AndDEScbc}" \
-                         "${pkcs5pbeWithSha1AndDEScbc}" \
-                         "DEFAULT"\
-                         "none"; do
-            export_list_import "${key_cipher}" "${cert_cipher}"
-      done       
+{
+  local saveIFS="${IFS}"
+  IFS=,
+  for key_cipher in ${PKCS5v1_PBE_CIPHERS} default; do
+      for cert_cipher in ${PKCS5v1_PBE_CIPHERS} default none; do
+          for hash in ${PBE_HASH}; do
+                  export_list_import "${key_cipher}" "${cert_cipher}" "${hash}"
+           done
+      done
   done
+  IFS="${saveIFS}"
 }
 
 ########################################################################
@@ -261,36 +234,16 @@ tools_p12_export_list_import_all_pkcs5pbe_ciphers()
 ########################################################################
 tools_p12_export_list_import_all_pkcs5v2_ciphers()
 {
-  # These should pass
-  for key_cipher in\
-    RC2-CBC \
-    DES-EDE3-CBC \
-    AES-128-CBC \
-    AES-192-CBC \
-    AES-256-CBC \
-    CAMELLIA-128-CBC \
-    CAMELLIA-192-CBC \
-    CAMELLIA-256-CBC; do
-
-#---------------------------------------------------------------
-# Bug 452464 - pk12util -o fails when -C option specifies
-# Camellia ciphers
-# FIXME Restore these to the list
-#    CAMELLIA-128-CBC, \
-#    CAMELLIA-192-CBC, \
-#    CAMELLIA-256-CBC, \
-#  when 452464 is fixed
-#---------------------------------------------------------------  
-    for cert_cipher in \
-      RC2-CBC \
-      DES-EDE3-CBC \
-      AES-128-CBC \
-      AES-192-CBC \
-      AES-256-CBC \
-      none; do
-	  export_list_import ${key_cipher} ${cert_cipher}
-	done
+  local saveIFS="${IFS}"
+  IFS=,
+  for key_cipher in ${PKCS5v2_PBE_CIPHERS} default; do
+      for cert_cipher in ${PKCS5v2_PBE_CIPHERS} default none; do
+          for hash in ${PBE_HASH}; do
+                  export_list_import "${key_cipher}" "${cert_cipher}" "${hash}"
+           done
+      done
   done
+  IFS="${saveIFS}"
 }
 
 ########################################################################
@@ -298,36 +251,50 @@ tools_p12_export_list_import_all_pkcs5v2_ciphers()
 # List the contents of and import from the p12 file.
 ########################################################################
 tools_p12_export_list_import_all_pkcs12v2pbe_ciphers()
-{ 
-#---------------------------------------------------------------
-# Bug 452471 - pk12util -o fails when -c option specifies pkcs12v2 PBE ciphers
-# FIXME - Restore these to the list 
-#                "${pkcs12v2pbeWithSha1And128BitRc4}" \
-#                "${pkcs12v2pbeWithSha1And40BitRc4}" \
-#	             "${pkcs12v2pbeWithSha1AndTripleDESCBC}" \
-#                "${pkcs12v2pbeWithSha1And128BitRc2Cbc}" \
-#                "${pkcs12v2pbeWithSha1And40BitRc2Cbc}" \
-#                "${pkcs12v2pbeWithMd2AndDESCBC}" \
-#                "${pkcs12v2pbeWithMd5AndDESCBC}" \
-#                "${pkcs12v2pbeWithSha1AndDESCBC}" \
-#                "DEFAULT"; do
-# when 452471 is fixed
-#---------------------------------------------------------------
-#  for key_cipher in \
-    key_cipher="DEFAULT"
-    for cert_cipher in "${pkcs12v2pbeWithSha1And128BitRc4}" \
-                  "${pkcs12v2pbeWithSha1And40BitRc4}" \
-                  "${pkcs12v2pbeWithSha1AndTripleDESCBC}" \
-                  "${pkcs12v2pbeWithSha1And128BitRc2Cbc}" \
-                  "${pkcs12v2pbeWithSha1And40BitRc2Cbc}" \
-                  "${pkcs12v2pbeWithMd2AndDESCBC}" \
-                  "${pkcs12v2pbeWithMd5AndDESCBC}" \
-                  "${pkcs12v2pbeWithSha1AndDESCBC}" \
-                  "DEFAULT"\
-                  "none"; do        
-	  export_list_import "${key_cipher}" "${cert_cipher}" 
-	done
-  #done
+{
+  local saveIFS="${IFS}"
+  IFS=,
+  for key_cipher in ${PKCS12_PBE_CIPHERS} ${PKCS5v1_PBE_CIPHERS} default; do
+      for cert_cipher in ${PKCS12_PBE_CIPHERS} ${PKCS5v1_PBE_CIPHERS} default none; do
+          for hash in ${PBE_HASH}; do
+                  export_list_import "${key_cipher}" "${cert_cipher}" "${hash}"
+           done
+      done
+  done
+  IFS="${saveIFS}"
+}
+
+########################################################################
+# Spot check all ciphers.
+# using the traditional tests, we wind up running almost 1300 tests.
+# This isn't too bad for debug builds in which the interator is set to 1000.
+# for optimized builds, the iterator is set to 60000, which means a 30
+# minute test will  now take more than 2 hours. This tests most combinations
+# and results in only about 300 tests. We are stil testing all ciphers
+# for both key and cert encryption, and we are testing them against
+# one of each class of cipher (pkcs5v1, pkcs5v2, pkcs12).
+########################################################################
+tools_p12_export_list_import_most_ciphers()
+{
+  local saveIFS="${IFS}"
+  IFS=,
+  for cipher in ${PBE_CIPHERS}; do
+    for class in ${PBE_CIPHERS_CLASSES}; do
+      # we'll test the case of cipher == class below the for loop
+      if [ "${cipher}" != "${class}" ]; then
+          export_list_import "${class}" "${cipher}" "SHA-1"
+          export_list_import "${cipher}" "${class}" "SHA-256"
+      fi
+    done
+    export_list_import "${cipher}" "none" "SHA-224"
+    export_list_import "${cipher}" "${cipher}" "SHA-384"
+  done
+  for class in ${PBE_CIPHERS_CLASSES}; do
+    for hash in ${PBE_HASH}; do
+      export_list_import "${class}" "${class}" "${hash}"
+    done
+  done
+  IFS="${saveIFS}"
 }
 
 #########################################################################
@@ -337,28 +304,28 @@ tools_p12_export_with_none_ciphers()
 {
   # use none as the key encryption algorithm default for the cert one
   # should fail
-  
+
   echo "pk12util -o Alice.p12 -n \"Alice\" -d ${P_R_ALICEDIR} \\"
-  echo "         -k ${R_PWFILE} -w ${R_PWFILE} -c none"     
+  echo "         -k ${R_PWFILE} -w ${R_PWFILE} -c none"
   ${BINDIR}/pk12util -o Alice.p12 -n Alice -d ${P_R_ALICEDIR} \
                        -k ${R_PWFILE} -w ${R_PWFILE} \
-                       -c none 2>&1  
+                       -c none 2>&1
   ret=$?
-  html_msg $ret 30 "Exporting with [none:default] (pk12util -o)"
+  html_msg $ret 30 "Exporting with [none:default:default] (pk12util -o)"
   check_tmpfile
 
   # use default as the key encryption algorithm none for the cert one
   # should pass
-  
+
   echo "pk12util -o Alice.p12 -n \"Alice\" -d ${P_R_ALICEDIR} \\"
-  echo "         -k ${R_PWFILE} -w ${R_PWFILE} -C none"     
+  echo "         -k ${R_PWFILE} -w ${R_PWFILE} -C none"
   ${BINDIR}/pk12util -o Alice.p12 -n Alice -d ${P_R_ALICEDIR} \
                        -k ${R_PWFILE} -w ${R_PWFILE} \
-                       -C none 2>&1  
+                       -C none 2>&1
   ret=$?
-  html_msg $ret 0 "Exporting with [default:none] (pk12util -o)"
+  html_msg $ret 0 "Exporting with [default:none:default] (pk12util -o)"
   check_tmpfile
- 
+  verify_p12 Alice.p12 "default" "none" "default"
 }
 
 #########################################################################
@@ -394,17 +361,18 @@ tools_p12_export_with_invalid_ciphers()
 tools_p12_export_list_import_with_default_ciphers()
 {
   echo "$SCRIPTNAME: Exporting Alice's email cert & key - default ciphers"
-  
-  export_list_import "DEFAULT" "DEFAULT"
+
+  export_list_import "default" "default" "default"
 
   echo "$SCRIPTNAME: Exporting Alice's email EC cert & key---------------"
   echo "pk12util -o Alice-ec.p12 -n \"Alice-ec\" -d ${P_R_ALICEDIR} -k ${R_PWFILE} \\"
   echo "         -w ${R_PWFILE}"
   ${BINDIR}/pk12util -o Alice-ec.p12 -n "Alice-ec" -d ${P_R_ALICEDIR} -k ${R_PWFILE} \
-       -w ${R_PWFILE} 2>&1 
+       -w ${R_PWFILE} 2>&1
   ret=$?
   html_msg $ret 0 "Exporting Alice's email EC cert & key (pk12util -o)"
   check_tmpfile
+  verify_p12 Alice-ec.p12 "default" "default" "default"
 
   echo "$SCRIPTNAME: Importing Alice's email EC cert & key --------------"
   echo "pk12util -i Alice-ec.p12 -d ${P_R_COPYDIR} -k ${R_PWFILE} -w ${R_PWFILE}"
@@ -460,9 +428,18 @@ tools_p12_import_rsa_pss_private_key()
 tools_p12()
 {
   tools_p12_export_list_import_with_default_ciphers
-  tools_p12_export_list_import_all_pkcs5v2_ciphers
-  tools_p12_export_list_import_all_pkcs5pbe_ciphers
-  tools_p12_export_list_import_all_pkcs12v2pbe_ciphers
+  # optimized builds have a larger iterator, so they can't run as many
+  # pkcs12 tests and complete in a reasonable time. Use the iterateration
+  # count from the previous tests to determine how many tests
+  # we can run.
+  iteration_count=$(pp -t p12 -i Alice-ec.p12 | grep "Iterations: " | sed -e 's;.*Iterations: ;;' -e 's;(.*).*;;')
+  echo "Iteration count=${iteration_count}"
+  if [ -n "${iteration_count}" -a  ${iteration_count} -le 10000 ]; then
+      tools_p12_export_list_import_all_pkcs5v2_ciphers
+      tools_p12_export_list_import_all_pkcs12v2pbe_ciphers
+  else
+      tools_p12_export_list_import_most_ciphers
+  fi
   tools_p12_export_with_none_ciphers
   tools_p12_export_with_invalid_ciphers
   tools_p12_import_old_files
@@ -473,7 +450,7 @@ tools_p12()
 
 ############################## tools_sign ##############################
 # local shell function pk12util uses a hardcoded tmp file, if this exists
-# and is owned by another user we don't get reasonable errormessages 
+# and is owned by another user we don't get reasonable errormessages
 ########################################################################
 check_tmpfile()
 {
@@ -484,6 +461,163 @@ check_tmpfile()
 }
 
 ############################## tools_sign ##############################
+# make sure the generated p12 file has the characteristics we expected
+########################################################################
+verify_p12()
+{
+  KEY_ENCRYPTION=$(map_cipher "${2}" "${KEY_ENCRYPTION_DEFAULT}")
+  CERT_ENCRYPTION=$(map_cipher "${3}" "${CERT_ENCRYPTION_DEFAULT}")
+  HASH=$(map_cipher "${4}" "${HASH_DEFAULT}")
+
+  STATE="NOBAGS"   # state records if we are in the key or cert bag
+  CERT_ENCRYPTION_NOT_FOUND=1
+  KEY_ENCRYPTION_NOT_FOUND=1
+  CERT_ENCRYPTION_FAIL=0
+  KEY_ENCRYPTION_FAIL=0
+  HASH_FAIL=0
+  TMP=$(mktemp /tmp/p12Verify.XXXXXX)
+  which pk12util
+  local saveIFS="${IFS}"
+  IFS=" 	\
+"
+  # use pp to dump the pkcs12 file, only the unencrypted portions are visible
+  # if there are multiple entries, we fail if any of those entries have the
+  # wrong encryption. We also fail if we can't find any encryption info.
+  # Use a file rather than a pipe so that while do can modify our variables.
+  # We're only interested in extracting the encryption algorithms are here,
+  # p12util -l will verify that decryption works properly.
+  pp -t pkcs12 -i ${1} -o ${TMP}
+  while read line ; do
+     # first up: if we see an unencrypted key bag, then we know that the key
+     # was unencrypted (NOTE: pk12util currently can't generate these kinds of
+     # files).
+     if [[ "${line}" =~ "Bag "[0-9]+" ID: PKCS #12 V1 Key Bag" ]]; then
+        KEY_ENCRYPTION_NOT_FOUND=0
+        if [ "${KEY_ENCRYPTION}" != "none" ]; then
+            KEY_ENCRYPTION_FAIL=1
+            echo "--Key encryption mismatch: expected \"${KEY_ENCRYPTION}\" found \"none\""
+        fi
+       continue
+     fi
+     # if we find the the Cert Bag, then we know that the certificate was not
+     # encrypted
+     if [[ "${line}" =~ "Bag "[0-9]+" ID: PKCS #12 V1 Cert Bag" ]]; then
+        CERT_ENCRYPTION_NOT_FOUND=0
+        if [ "${CERT_ENCRYPTION}" != "none" ]; then
+            CERT_ENCRYPTION_FAIL=1
+           echo "--Cert encryption mismatch: expected \"${CERT_ENCRYPTION}\" found \"none\""
+        fi
+        continue
+     fi
+     # we found the shrouded key bag, the next encryption informtion should be
+     # for the key.
+     if [[ "${line}" =~ "Bag "[0-9]+" ID: PKCS #12 V1 PKCS8 Shrouded Key Bag" ]]; then
+        STATE="KEY"
+        continue
+     fi
+     # If we found PKCS #7 Encrypted Data, it must be the encrypted certificate
+     # (well it could be any encrypted certificate, or a crl, but in p12util
+     # they will all have the same encryption value
+     if [[ "${line}" =  "PKCS #7 Encrypted Data:" ]]; then
+        STATE="CERT"
+        continue
+     fi
+     # check the Mac
+     if [[ "${line}" =~ "Mac Digest Algorithm ID: ".* ]]; then
+        MAC="${line##Mac Digest Algorithm ID: }"
+        if [ "${MAC}" != "${HASH}" ]; then
+            HASH_FAIL=1
+            echo "--Mac Hash mismatch: expected \"${HASH}\" found \"${MAC}\""
+        fi
+     fi
+     # check the KDF
+     if [[ "${line}" =~ "KDF algorithm: ".* ]]; then
+        KDF="${line##KDF algorithm: }"
+        if [ "${KDF}" != "HMAC ${HASH}" ]; then
+            HASH_FAIL=1
+            echo "--KDF Hash mismatch: expected \"HMAC ${HASH}\" found \"${KDF}\""
+        fi
+     fi
+     # Content Encryption Algorithm is the PKCS #5 algorithm ID.
+     if [[  "${line}" =~ .*"Encryption Algorithm: ".* ]]; then
+        # Strip the [Content ]EncryptionAlgorithm
+        ENCRYPTION="${line##Content }"
+        ENCRYPTION="${ENCRYPTION##Encryption Algorithm: }"
+        # If that algorithm id is PKCS #5 V2, then skip forward looking
+        # for the Cipher: field.
+        if [[ "${ENCRYPTION}" =~ "PKCS #5 Password Based Encryption v2"\ * ]]; then
+            continue;
+        fi
+        case ${STATE} in
+        "KEY")
+            KEY_ENCRYPTION_NOT_FOUND=0
+            if [ "${KEY_ENCRYPTION}" != "${ENCRYPTION}" ]; then
+                KEY_ENCRYPTION_FAIL=1
+                echo "--Key encryption mismatch: expected \"${KEY_ENCRYPTION}\" found \"${ENCRYPTION}\""
+            fi
+            ;;
+        "CERT")
+            CERT_ENCRYPTION_NOT_FOUND=0
+            if [ "${CERT_ENCRYPTION}" != "${ENCRYPTION}" ]; then
+                CERT_ENCRYPTION_FAIL=1
+                echo "--Cert encryption mismatch: expected \"${CERT_ENCRYPTION}\" found \"${ENCRYPTION}\""
+            fi
+            ;;
+        esac
+     fi
+     # handle the PKCS 5 case
+     if [[ "${line}" =~ "Cipher: ".* ]]; then
+        ENCRYPTION="${line#Cipher: }"
+        case ${STATE} in
+        "KEY")
+            KEY_ENCRYPTION_NOT_FOUND=0
+            if [ "${KEY_ENCRYPTION}" != "${ENCRYPTION}" ]; then
+                KEY_ENCRYPTION_FAIL=1
+                echo "--Key encryption mismatch: expected \"${KEY_ENCRYPTION}\" found \"${ENCRYPTION}\""
+            fi
+            ;;
+        "CERT")
+            CERT_ENCRYPTION_NOT_FOUND=0
+            if [ "${CERT_ENCRYPTION}" != "${ENCRYPTION}" ]; then
+                CERT_ENCRYPTION_FAIL=1
+                echo "--Cert encryption mismatch: expected \"${CERT_ENCRYPTION}\" found \"${ENCRYPTION}\""
+            fi
+            ;;
+        esac
+     fi
+  done < ${TMP}
+  IFS="${saveIFS}"
+  # we've scanned the file, set the return value to a combination of
+  # KEY and CERT state variables. If everything is as expected, they should
+  # add up to 0.
+  ret=$((${HASH_FAIL} * 10000 + ${KEY_ENCRYPTION_FAIL} * 1000 + ${KEY_ENCRYPTION_NOT_FOUND} * 100 + ${CERT_ENCRYPTION_FAIL} * 10 + ${CERT_ENCRYPTION_NOT_FOUND}))
+  rm -r ${TMP}
+  html_msg $ret 0 "Verifying p12 file generated with [${2}:${3}:${4}]"
+}
+
+#
+# this handles any mapping we need from requested cipher to
+# actual cipher. For instance ciphers which already have
+# PKCS 5 v1 PBE will be mapped to those pbes by pk12util.
+map_cipher()
+{
+   if [ "${1}" = "default" ]; then
+      echo "${2}"
+      return
+   fi
+   case "${1}" in
+   # these get mapped to the PKCS5 v1 or PKCS 12 attributes, not PKCS 5v2
+   RC2-CBC)
+      echo "${pkcs12v2pbeWithSha1And128BitRc2Cbc}"
+      return ;;
+   DES-EDE3-CBC)
+      echo "${pkcs12v2pbeWithSha1AndTripleDESCBC}"
+      return;;
+   esac
+   echo "${1}"
+}
+
+############################## tools_sign ##############################
 # local shell function to test basic functionality of signtool
 ########################################################################
 tools_sign()
@@ -554,7 +688,7 @@ tools_modutil()
 }
 
 ############################## tools_cleanup ###########################
-# local shell function to finish this script (no exit since it might be 
+# local shell function to finish this script (no exit since it might be
 # sourced)
 ########################################################################
 tools_cleanup()