Fix SEGVs seen in overnight alphabits tests.

tests/pdf/Bug6901014_CityMap-evince.simplified.pdf.bitrgbtags.300.1.gs
tests/pdf/Bug6901014_CityMap-evince.simplified.pdf.pgmraw.300.0.gs
tests/pdf/Bug6901014_CityMap-evince.simplified.pdf.pgmraw.300.1.gs
tests/pdf/Bug6901014_CityMap-evince.simplified.pdf.ppmraw.300.1.gs
tests/pdf/Bug6901014_CityMap-evince.simplified.pdf.psdcmyk.300.1.gs
tests/pdf/Bug6901014_CityMap-evince.simplified.pdf.psdcmykog.600.1.gs
tests/pdf/Bug6901014_CityMap-evince.simplified.pdf.psdrgb.300.1	gs
tests/pdf/Bug6901014_CityMap-evince.simplified.pdf.tiffscaled.600.1.gs
tests_private/pdf/uploads/333.pdf.bitrgbtags.300.1.gs
tests_private/pdf/uploads/333.pdf.ppmraw.300.0.gs
tests_private/pdf/uploads/333.pdf.ppmraw.300.1.gs
tests_private/pdf/uploads/333.pdf.psdcmyk.300.1.gs
tests_private/pdf/uploads/333.pdf.psdcmyk16.300.1.gs
tests_private/pdf/uploads/333.pdf.psdcmykog.600.1.gs
tests_private/pdf/uploads/333.pdf.psdrgb.300.1.gs

These seem to be caused by us creating a memory device, then
detecting an error, and exiting with the device unused, but
half setup. That was enough to trip a problem when the device
is finalized by the gc system later.

Now, we move the operation that might cause an error to be before
the allocation of the memory device.

This leaves these files returning an error where previously they
didn't. This is less than ideal, so I've opened bug 706695 to track
that.

1 files changed, 7 insertions, 6 deletions

diff --git a/base/gspaint.c b/base/gspaint.c
index 30d78e0c6..58f88e9c4 100644
--- a/base/gspaint.c
+++ b/base/gspaint.c
@@ -232,18 +232,19 @@ alpha_buffer_init(gs_gstate * pgs, fixed extra_x, fixed extra_y, int alpha_bits,
     if (height > height2)
         height = height2;
     height <<= log2_scale.y;
-    mem = pgs->memory;
-    mdev = gs_alloc_struct(mem, gx_device_memory, &st_device_memory,
-                           "alpha_buffer_init");
-    if (mdev == 0)
-        return 0;		/* if no room, don't buffer */
     /* We may have to update the marking parameters if we have a pdf14 device
-       as our target.  Need to do while dev is still active in pgs */
+       as our target.  Need to do while dev is still active in pgs. Do this
+       before allocating the device to simplify cleanup. */
     if (dev_proc(dev, dev_spec_op)(dev, gxdso_is_pdf14_device, NULL, 0) > 0) {
         int code = gs_update_trans_marking_params(pgs);
         if (code < 0)
             return code;
     }
+    mem = pgs->memory;
+    mdev = gs_alloc_struct(mem, gx_device_memory, &st_device_memory,
+                           "alpha_buffer_init");
+    if (mdev == 0)
+        return 0;		/* if no room, don't buffer */
     gs_make_mem_abuf_device(mdev, mem, dev, &log2_scale,
                             alpha_bits, ibox.p.x << log2_scale.x, devn);
     mdev->width = width;