RT #127742: Hash keys are limited to 2 GB - throw an exception if hash keys are too long

We currently require hash keys to be less than 2**31 bytes long. But (a)
nothing actually tries to enforce that, and (b) if a Perl program tries to
create a hash with such a key (using a 64-bit system), we miscalculate the
size of a memory block, yielding a panic:

$ ./perl -e '+{ "x" x 2**31, undef }'
panic: malloc, size=18446744071562068026 at -e line 1.

Instead, check for this situation, and croak with an appropriate (new)
diagnostic in the unlikely event that it occurs.

This also involves changing the type of an argument to a public API function:
Perl_share_hek() previously took the key's length as an I32, but that makes
it impossible to detect over-long keys, so it must be SSize_t instead.

From Yves:
We also inject the length test into the PERL_HASH() macro, so that where
the macro is used *before* calling into any of the hv functions we can
avoid hashing a very long string only to throw an exception that it is
too long. Might as well fail fast.

1 files changed, 6 insertions, 4 deletions

diff --git a/hv_func.h b/hv_func.h
index e091c86b0f..8a92c362e9 100644
--- a/hv_func.h
+++ b/hv_func.h
@@ -93,10 +93,12 @@
     sbox32_seed_state96(seed + __PERL_HASH_SEED_BYTES , state + __PERL_HASH_STATE_BYTES);   \
 } STMT_END
 
-#define _PERL_HASH_WITH_STATE(state,str,len)                                                \
-    ((len <= SBOX32_MAX_LEN)                                                                \
-        ? sbox32_hash_with_state((state + __PERL_HASH_STATE_BYTES),(U8*)(str),(len))        \
-        : __PERL_HASH_WITH_STATE((state),(str),(len)))
+#define _PERL_HASH_WITH_STATE(state,str,len)                                            \
+    (LIKELY(len <= SBOX32_MAX_LEN)                                                      \
+        ? sbox32_hash_with_state((state + __PERL_HASH_STATE_BYTES),(U8*)(str),(len))    \
+        : UNLIKELY(len > (STRLEN) I32_MAX)                                              \
+          ? Perl_croak_nocontext("Sorry, hash keys must be smaller than 2**31 bytes")   \
+          : __PERL_HASH_WITH_STATE((state),(str),(len)))
 
 #endif