Rework RunLengthEncoder.

The existing RunLengthEncoder relies on being able to read ahead
a few bytes, and then decide that it wants to ask the caller for
more data.

Unfortunately, this means that the clist cmd_compress_bitmap routine
can only call it in the "here is all the data in a solid block" case,
not in the "here is the data a line at a time" case.

This will become even more of a limitation when I rework
cmd_compress_bitmap to avoid overrun reads in a future commit.

The primary difference here is that we never backtrack on our reads
within the compression routine. We keep the last 1, 2 or 3 bytes
read in the state as n0, n1 and n2, and we insert literal bytes into a
array within the state for copying out later.

Because we now allow cmd_compress_bitmap to run in cases when it
didn't before, we trip over a bug in there. If height > 0 and
raster < width_bytes (for instance when raster = 0, so we can repeat
the same line several times), on 64bit builds the pointer
arithmetic goes wrong, and we end up accessing illegal memory.
Fix with a simple cast to int.

1 files changed, 9 insertions, 2 deletions

diff --git a/base/srlx.h b/base/srlx.h
index 41a474784..ebf172064 100644
--- a/base/srlx.h
+++ b/base/srlx.h
@@ -34,7 +34,12 @@ typedef struct stream_RLE_state_s {
     ulong record_size;
     /* The following change dynamically. */
     ulong record_left;		/* bytes left in current record */
-    int copy_left;		/* # of bytes waiting to be copied */
+    byte n0;
+    byte n1;
+    byte n2;
+    byte state;
+    int run_len;
+    byte literals[128];
 } stream_RLE_state;
 
 #define private_st_RLE_state()	/* in srle.c */\
@@ -47,7 +52,9 @@ typedef struct stream_RLE_state_s {
   ((ss)->record_left =\
    ((ss)->record_size == 0 ? ((ss)->record_size = max_uint) :\
     (ss)->record_size),\
-   (ss)->copy_left = 0)
+   (ss)->n0=(ss)->n1=(ss)->n2=0,\
+   (ss)->run_len=0,\
+   (ss)->state=0)
 extern const stream_template s_RLE_template;
 
 /* RunLengthDecode */