GhostPDF - set the default for the key length when /V is 1

Bug #706668 "regression: new PDF engine can't handle a broken PDF file, falsely reports password protected"

Although the file is damaged (in fact it appears to be 2 PDF files,
concatenated) that's not the problem. The second file is the one which
we process, and that file is encrypted. We simply were not setting the
encryption key length for Revision 3 and we were only reading the
actual key /Length if /V is present and has the value 2 or 3.

This is basically because PDF encryption is a mess, and simply an
oversight. Fix it here.

1 files changed, 9 insertions, 5 deletions

diff --git a/pdf/pdf_sec.c b/pdf/pdf_sec.c
index 8e4dbb9e3..e968b89c5 100644
--- a/pdf/pdf_sec.c
+++ b/pdf/pdf_sec.c
@@ -1072,7 +1072,8 @@ static int pdfi_read_Encrypt_dict(pdf_context *ctx, int *KeyLen)
 
             if (code > 0)
                 *KeyLen = (int)f;
-        }
+        } else
+            *KeyLen = 40;
     }
 
     code = pdfi_dict_get_int(ctx, d, "P", &i64);
@@ -1407,10 +1408,13 @@ int pdfi_initialise_Decryption(pdf_context *ctx)
                 }
             }
             /* Revision 3 *may* be more than 40 bits of RC4 */
-            if (KeyLen != 0 && (KeyLen < 40 || KeyLen > 128 || KeyLen % 8 != 0)) {
-                pdfi_set_warning(ctx, 0, NULL, W_PDF_INVALID_DECRYPT_LEN, "pdfi_initialise_Decryption", NULL);
-                KeyLen = 128;
-            }
+            if (KeyLen != 0) {
+                if (KeyLen < 40 || KeyLen > 128 || KeyLen % 8 != 0) {
+                    pdfi_set_warning(ctx, 0, NULL, W_PDF_INVALID_DECRYPT_LEN, "pdfi_initialise_Decryption", NULL);
+                    KeyLen = 128;
+                }
+            } else
+                KeyLen = 40;
             if (ctx->encryption.StmF == CRYPT_NONE)
                 ctx->encryption.StmF = CRYPT_V2;
             if (ctx->encryption.StrF == CRYPT_NONE)