Bug 1603628 Update NSS to handle PKCS #11 v3.0 r=ueno r=mt

Update to PKCS #11 v3.0 part 2.

Create the functions and switch to the C_Interface() function to fetch the PKCS #11 function table. Also PKCS #11 v3.0 uses a new fork safe interface. NSS can already handle the case if the PKCS #11 module happens to be fork safe (when asked by the application to refresh the tokens in the child process, NSS can detect that such a refresh is not necessary and continue. Softoken could also be put in fork_safe mode with an environment variable. With this patch it's the default, and NSS asks for the fork safe API by default. Technically softoken should implement the old non-fork safe interface when PKCS #11 v2.0 is called, but NSS no longer needs it, and doing so would double the number of PKCS #11 interfaces are needed. You can still compile with fork unsafe semantics, and the PKCS #11 V3.0 module will do the right thing and not include the fork safe flag. Firefox does not fork(), so for firefox this is simply code that is no longer compilied.

We now use C_GetInterface, which allows us to specify what kind of interface we want (PKCS #11 v3.0, PKCS #11 v2.0, fork safe, etc.). Vendor specific functions can now be accessed through the C_GetInterface. If the C_GetInterface function does not exists, we fall bak to the old C_GetFunctionList.

There are 24 new functions in PKCS #11 v3.0:
C_GetInterfaceList - return a table of all the supported interfaces
C_GetInterface - return a specific interface. You can specify interface name, version and flags separately. You can leave off any of these and you will get what the token thinks is the best match of the interfaces that meet the criteria. We do this in softoken by the order of the interface list.
C_SessionCancel - Cancel one or more multipart operation
C_LoginUser - Supply a user name to C_Login(). This function has no meaning for softoken, so it just returns CKR_OPERATION_NOT_INITIALIZED under the theory that if we in the future want to support usernames, the NSS db would need special initialization to make that happen.
C_Message* and C_*Message*  (20 functions in all) are the new AEAD interface (they are written generally so that it can be used for things other than AEAD). In this patch they are unimplemented (see the next patch).

This patch adds regular (NSC_) and FIPS (FC_) versions of these functions.
Also when creating the PKCS #11 v2.0 interface, we had to create a 2.0 specific version of C_GetInfo so that it can return a 2.40 in the CK_VERSION field rather than 3.00. We do this with #defines since all the function tables are generated automagically with pkcs11f.h.

Differential Revision: https://phabricator.services.mozilla.com/D67240

2 files changed, 21 insertions, 0 deletions

diff --git a/nss/automation/abi-check/new-report-libnss3.so.txt b/nss/automation/abi-check/new-report-libnss3.so.txt
new file mode 100644
index 000000000..edf5a1310
--- /dev/null
+++ b/nss/automation/abi-check/new-report-libnss3.so.txt
@@ -0,0 +1,13 @@
+
+1 function with some indirect sub-type change:
+
+  [C]'function SECStatus PK11_GetModInfo(SECMODModule*, CK_INFO*)' at pk11util.c:613:1 has some indirect sub-type changes:
+    parameter 1 of type 'SECMODModule*' has sub-type changes:
+      in pointed to type 'typedef SECMODModule' at secmodt.h:29:1:
+        underlying type 'struct SECMODModuleStr' at secmodt.h:44:1 changed:
+          type size changed from 1600 to 1664 (in bits)
+          1 data member insertion:
+            'CK_FLAGS SECMODModuleStr::flags', at offset 1600 (in bits) at secmodt.h:76:1
+          no data member change (1 filtered);
+
+
diff --git a/nss/automation/abi-check/new-report-libsoftokn3.so.txt b/nss/automation/abi-check/new-report-libsoftokn3.so.txt
new file mode 100644
index 000000000..1e825bede
--- /dev/null
+++ b/nss/automation/abi-check/new-report-libsoftokn3.so.txt
@@ -0,0 +1,8 @@
+
+4 Added functions:
+
+  'function CK_RV FC_GetInterface(CK_UTF8CHAR_PTR, CK_VERSION_PTR, CK_INTERFACE_PTR_PTR, CK_FLAGS)'    {FC_GetInterface@@NSS_3.52}
+  'function CK_RV FC_GetInterfaceList(CK_INTERFACE_PTR, CK_ULONG_PTR)'    {FC_GetInterfaceList@@NSS_3.52}
+  'function CK_RV NSC_GetInterface(CK_UTF8CHAR_PTR, CK_VERSION_PTR, CK_INTERFACE_PTR_PTR, CK_FLAGS)'    {NSC_GetInterface@@NSS_3.52}
+  'function CK_RV NSC_GetInterfaceList(CK_INTERFACE_PTR, CK_ULONG_PTR)'    {NSC_GetInterfaceList@@NSS_3.52}
+