Fix Bug 702181: SEGV when BufferSpace is at or above -K alloc limit

The issue with this file is that one of the pages is done in page mode because
it was so small, but the 'resize' call when doing page 2 was not checking the
allocation request against the limit, so the malloc would succeed, but the very
next (and all subsequent) allocations would fail. The gdev_prn_allocate would
capture the ecode, but would still call the clist_enable_multi_thread_render
resulting in the SEGV.

Add the check in the gs_heap_resize_object function, and make sure and leave
the 'monitor' after failure, then don't call clist_enable_multi_thread if the
resize failed.

1 files changed, 8 insertions, 6 deletions

diff --git a/base/gdevprn.c b/base/gdevprn.c
index c36550948..ed89cd7d0 100644
--- a/base/gdevprn.c
+++ b/base/gdevprn.c
@@ -520,13 +520,15 @@ gdev_prn_allocate(gx_device *pdev, gdev_prn_space_params *new_space_params,
             if (ecode == 0)
                 ecode = code;
 
-            if ( code >= 0 || (reallocate && pass > 1) )
+            if (code >= 0 || (reallocate && pass > 1))
                 ppdev->procs = gs_clist_device_procs;
-            /*
-             * Now the device is a clist device, we enable multi-threaded rendering.
-             * It will remain enabled, but that doesn't really cause any problems.
-             */
-            clist_enable_multi_thread_render(pdev);
+            if (code > 0) {
+                /*
+                 * Now the device is a clist device, we enable multi-threaded rendering.
+                 * It will remain enabled, but that doesn't really cause any problems.
+                 */
+                clist_enable_multi_thread_render(pdev);
+            }
         } else {
             /* Render entirely in memory. */
             gx_device *bdev = (gx_device *)pmemdev;