From 3c24479e370cd7745cf5a9c98b876fe61e85b4d5 Mon Sep 17 00:00:00 2001 From: Ken Sharp Date: Tue, 15 Nov 2022 20:22:58 +0000 Subject: xpswrite - limit the memory used by images to avoid memory exhaustion OSS-fuzz 53398 (and others) Because xpswrite uses libtiff to write images to XPS files, and libtiff won't offer us any way to use our memory manager, it is possible to get a corrupted image which tries to use silly amounts of memory, which leads to system memory being exhausted. In this commit; change the gs_memory_status_s structure to include the 'limit' from the memory allocator. Alter the various allocators to fill in the value (or set it to -1 if there is no limit). In the xpswrite device calculate the amount of memory needed for an uncompressed image, if it becomes negative we know we broke a 64-bit signed integer, so abort. If it didn't go negative, and the memory manager has a specified limit, check to see if the image will fit in that. If not return VMerror. This prevents at least some of the memory exhaustion errors with xpswrite and OSS-fuzz, because OSS-fuzz sets -K to limit memory use. --- devices/vector/gdevxps.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) (limited to 'devices') diff --git a/devices/vector/gdevxps.c b/devices/vector/gdevxps.c index d0f02f5d0..af0b32710 100644 --- a/devices/vector/gdevxps.c +++ b/devices/vector/gdevxps.c @@ -2102,6 +2102,35 @@ xps_begin_typed_image(gx_device *dev, num_components = gs_color_space_num_components(pcs); bits_per_pixel = pim->BitsPerComponent * num_components; + + { + /* This is a really hacky attempt to avoid running out of memory. This is + * inspired by various OSS-fuzz bugs but in particular 53398. This device + * uses the libtiff library to write images into the XPS file, libtiff won't + * let us use our own memory manager and have rejected patches to allow it + * to do so, so it uses system memory. If we have a badly broken file we + * might end up asking it to write a ridiculously large image which will + * cause it to eat all system memory. So here we try to limit it to any pre-defined + * limit. Also, if the width * height * 24 bits (RGB) goes negative then we know + * we've exceeded 2^(64 - 1) bytes, which is unreasonable too. + */ + int64_t memory_needed = (int64_t)pim->Width * (int64_t)pim->Height * 3; + gs_memory_status_t status; + + if (memory_needed < 0) { + gs_free_object(mem, pie, "xps_begin_image"); + return_error(gs_error_VMerror); + } + + gs_memory_status(dev->memory->gs_lib_ctx->memory, &status); + if (status.limit < (size_t)~1) { + if ( memory_needed > status.limit) { + gs_free_object(mem, pie, "xps_begin_image"); + return_error(gs_error_VMerror); + } + } + } + pie->decode_st.bps = bits_per_pixel / num_components; pie->bytes_comp = (pie->decode_st.bps > 8 ? 2 : 1); pie->decode_st.spp = num_components; -- cgit v1.2.1