make EXTEND() and stack_grow() safe(r)

This commit fixes various issues around stack_grow() and its
two main wrappers, EXTEND() and MEXTEND(). In particular it behaves
very badly on systems with 32-bit pointers but 64-bit ints.

One noticeable effect of this is commit is that various usages of EXTEND()
etc will now start to give compiler warnings - usually because they're
passing an unsigned N arg when it should be signed. This may indicate
a logic error in the caller's code which needs fixing. This commit causes
several such warnings to appear in core code, which will be fixed in the
next commit.

Essentially there are several potential false negatives in this basic
code:

     if (PL_stack_max - p < (SSize_t)(n))
        stack_grow(sp,p,(SSize_t)(n));

where it incorrectly skips the call to stack_grow() and then the caller
tramples over the end of the stack because it assumes that it has in fact
been extended. The value of N passed to stack_grow() can also potentially
get truncated or wrapped.

Note that the N arg of stack_grow() is SSize_t and EXTEND()'s N arg is
documented as SSize_t.  In earlier times, they were both ints.
Significantly, this means that they are both signed, and always have been.

In detail, the problems and their solutions are:

1) N is a signed value: if negative, it could be an indication of a
    caller's invalid logic or wrapping in the caller's code. This should
    trigger a panic. Make it so by adding an extra test to EXTEND() to
    always call stack_grow if negative, then add a check and panic in
    stack_grow() (and other places too). This extra test will be constant
    folded when EXTEND() is called with a literal N.

2) If the caller passes an unsigned value of N, then the comparison is
    between a signed and an unsigned value, leading to potential
    wrap-around. Casting N to SSize_t merely hides any compiler warnings,
    thus failing to alert the caller to a problem with their code. In
    addition, where sizeof(N) > sizeof(SSize_t), the cast may truncate N,
    again leading to false negatives. The solution is to remove the cast,
    and let the caller deal with any compiler warnings that result.

3) Similarly, casting stack_grow()'s N arg can hide any warnings issued by
    e.g. -Wconversion. So remove it.  It still does the wrong thing if the
    caller uses a non-signed type (usually a panic in stack_grow()), but
    coders have slightly more chance of spotting issues at compile time
    now.

4) If sizeof(N) > sizeof(SSize_t), then the N arg to stack_grow() may get
   truncated or sign-swapped. Add a test for this (basically that N is too
   big to fit in a SSize_t); for simplicity, in this case just set N to
   -1 so that stack_grow() panics shortly afterwards. In platforms where
   this can't happen, the test is constant folded away.

With all these changes, the macro now looks in essence like:

     if ( n < 0 || PL_stack_max - p < n)
        stack_grow(sp,p,
            (sizeof(n) > sizeof(SSize_t) && ((SSize_t)(n) != n) ? -1 : n));

1 files changed, 38 insertions, 9 deletions

diff --git a/pp.h b/pp.h
index 2d99a7242d..b497085a35 100644
--- a/pp.h
+++ b/pp.h
@@ -283,29 +283,58 @@ Does not use C<TARG>.  See also C<L</XPUSHu>>, C<L</mPUSHu>> and C<L</PUSHu>>.
 =cut
 */
 
+/* _EXTEND_SAFE_N(n): private helper macro for EXTEND().
+ * Tests whether the value of n would be truncated when implicitly cast to
+ * SSize_t as an arg to stack_grow(). If so, sets it to -1 instead to
+ * trigger a panic. It will be constant folded on platforms where this
+ * can't happen.
+ */
+
+#define _EXTEND_SAFE_N(n) \
+        (sizeof(n) > sizeof(SSize_t) && ((SSize_t)(n) != (n)) ? -1 : (n))
+
 #ifdef STRESS_REALLOC
 # define EXTEND(p,n)   STMT_START {                                     \
-                           sp = stack_grow(sp,p,(SSize_t) (n));         \
+                           sp = stack_grow(sp,p,_EXTEND_SAFE_N(n));     \
                            PERL_UNUSED_VAR(sp);                         \
                        } STMT_END
 /* Same thing, but update mark register too. */
 # define MEXTEND(p,n)   STMT_START {                                    \
                             const SSize_t markoff = mark - PL_stack_base; \
-                            sp = stack_grow(sp,p,(SSize_t) (n));        \
+                            sp = stack_grow(sp,p,_EXTEND_SAFE_N(n));    \
                             mark = PL_stack_base + markoff;             \
                             PERL_UNUSED_VAR(sp);                        \
                         } STMT_END
 #else
-# define EXTEND(p,n)   STMT_START {                                     \
-                         if (UNLIKELY(PL_stack_max - p < (SSize_t)(n))) { \
-                           sp = stack_grow(sp,p,(SSize_t) (n));         \
+
+/* _EXTEND_NEEDS_GROW(p,n): private helper macro for EXTEND().
+ * Tests to see whether n is too big and we need to grow the stack. Be
+ * very careful if modifying this. There are many ways to get things wrong
+ * (wrapping, truncating etc) that could cause a false negative and cause
+ * the call to stack_grow() to be skipped. On the other hand, false
+ * positives are safe.
+ * Bear in mind that sizeof(p) may be less than, equal to, or greater
+ * than sizeof(n), and while n is documented to be signed, someone might
+ * pass an unsigned value or expression. In general don't use casts to
+ * avoid warnings; instead expect the caller to fix their code.
+ * It is legal for p to be greater than PL_stack_max.
+ * If the allocated stack is already very large but current usage is
+ * small, then PL_stack_max - p might wrap round to a negative value, but
+ * this just gives a safe false positive
+ */
+
+#  define _EXTEND_NEEDS_GROW(p,n) ( (n) < 0 || PL_stack_max - p < (n))
+
+#  define EXTEND(p,n)   STMT_START {                                    \
+                         if (UNLIKELY(_EXTEND_NEEDS_GROW(p,n))) {       \
+                           sp = stack_grow(sp,p,_EXTEND_SAFE_N(n));     \
                            PERL_UNUSED_VAR(sp);                         \
                          } } STMT_END
 /* Same thing, but update mark register too. */
-# define MEXTEND(p,n)  STMT_START {                                     \
-                         if (UNLIKELY(PL_stack_max - p < (SSize_t)(n))) { \
-                           const SSize_t markoff = mark - PL_stack_base;  \
-                           sp = stack_grow(sp,p,(SSize_t) (n));         \
+#  define MEXTEND(p,n)  STMT_START {                                    \
+                         if (UNLIKELY(_EXTEND_NEEDS_GROW(p,n))) {       \
+                           const SSize_t markoff = mark - PL_stack_base;\
+                           sp = stack_grow(sp,p,_EXTEND_SAFE_N(n));     \
                            mark = PL_stack_base + markoff;              \
                            PERL_UNUSED_VAR(sp);                         \
                          } } STMT_END