make EXTEND() and stack_grow() safe(r)

This commit fixes various issues around stack_grow() and its
two main wrappers, EXTEND() and MEXTEND(). In particular it behaves
very badly on systems with 32-bit pointers but 64-bit ints.

One noticeable effect of this is commit is that various usages of EXTEND()
etc will now start to give compiler warnings - usually because they're
passing an unsigned N arg when it should be signed. This may indicate
a logic error in the caller's code which needs fixing. This commit causes
several such warnings to appear in core code, which will be fixed in the
next commit.

Essentially there are several potential false negatives in this basic
code:

     if (PL_stack_max - p < (SSize_t)(n))
        stack_grow(sp,p,(SSize_t)(n));

where it incorrectly skips the call to stack_grow() and then the caller
tramples over the end of the stack because it assumes that it has in fact
been extended. The value of N passed to stack_grow() can also potentially
get truncated or wrapped.

Note that the N arg of stack_grow() is SSize_t and EXTEND()'s N arg is
documented as SSize_t.  In earlier times, they were both ints.
Significantly, this means that they are both signed, and always have been.

In detail, the problems and their solutions are:

1) N is a signed value: if negative, it could be an indication of a
    caller's invalid logic or wrapping in the caller's code. This should
    trigger a panic. Make it so by adding an extra test to EXTEND() to
    always call stack_grow if negative, then add a check and panic in
    stack_grow() (and other places too). This extra test will be constant
    folded when EXTEND() is called with a literal N.

2) If the caller passes an unsigned value of N, then the comparison is
    between a signed and an unsigned value, leading to potential
    wrap-around. Casting N to SSize_t merely hides any compiler warnings,
    thus failing to alert the caller to a problem with their code. In
    addition, where sizeof(N) > sizeof(SSize_t), the cast may truncate N,
    again leading to false negatives. The solution is to remove the cast,
    and let the caller deal with any compiler warnings that result.

3) Similarly, casting stack_grow()'s N arg can hide any warnings issued by
    e.g. -Wconversion. So remove it.  It still does the wrong thing if the
    caller uses a non-signed type (usually a panic in stack_grow()), but
    coders have slightly more chance of spotting issues at compile time
    now.

4) If sizeof(N) > sizeof(SSize_t), then the N arg to stack_grow() may get
   truncated or sign-swapped. Add a test for this (basically that N is too
   big to fit in a SSize_t); for simplicity, in this case just set N to
   -1 so that stack_grow() panics shortly afterwards. In platforms where
   this can't happen, the test is constant folded away.

With all these changes, the macro now looks in essence like:

     if ( n < 0 || PL_stack_max - p < n)
        stack_grow(sp,p,
            (sizeof(n) > sizeof(SSize_t) && ((SSize_t)(n) != n) ? -1 : n));

1 files changed, 4 insertions, 0 deletions

diff --git a/av.c b/av.c
index cb99ceb29b..2c4740b621 100644
--- a/av.c
+++ b/av.c
@@ -87,6 +87,10 @@ Perl_av_extend_guts(pTHX_ AV *av, SSize_t key, SSize_t *maxp, SV ***allocp,
 {
     PERL_ARGS_ASSERT_AV_EXTEND_GUTS;
 
+    if (key < -1) /* -1 is legal */
+        Perl_croak(aTHX_
+            "panic: av_extend_guts() negative count (%"IVdf")", (IV)key);
+
     if (key > *maxp) {
 	SV** ary;
 	SSize_t tmp;