[smooth] Sub-banding protocol revision.

Rasterization sub-banding is utilized at large sizes while using
rather small fixed memory pool. Indeed it is possible to make an
educated guess how much memory is necessary at a given size for a
given glyph. It turns out that, for large majority of European glyphs,
you should store about 8 times more boundary pixels than their height.
Or, vice versa, if your memory pool can hold 800 pixels the band
height should be 100 and you should sub-band anything larger than
that. Should you still run out of memory, FreeType bisects the band
but you have wasted some time. This is what has been implemented in
FreeType since the beginning.

It was overlooked, however, that the top band could grow to twice the
default band size leading to unnecessary memory overflows there. This
commit fixes that. Now the bands are distributed more evenly and
cannot exceed the default size.

Now the magic number 8 is really suitable for rather simple European
scripts. For complex Chinese logograms the magic number should be 13
but that is subject for another day.

* src/smooth/ftgrays.c (gray_convert_glyph): Revise sub-banding
protocol.

2 files changed, 39 insertions, 11 deletions

diff --git a/ChangeLog b/ChangeLog
index 8cf5015be..e5bbe83fc 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,30 @@
+2016-07-08  Alexei Podtelezhnikov  <apodtele@gmail.com>
+
+	[smooth] Sub-banding protocol revision.
+
+	Rasterization sub-banding is utilized at large sizes while using
+	rather small fixed memory pool. Indeed it is possible to make an
+	educated guess how much memory is necessary at a given size for a
+	given glyph. It turns out that, for large majority of European glyphs,
+	you should store about 8 times more boundary pixels than their height.
+	Or, vice versa, if your memory pool can hold 800 pixels the band
+	height should be 100 and you should sub-band anything larger than
+	that. Should you still run out of memory, FreeType bisects the band
+	but you have wasted some time. This is what has been implemented in
+	FreeType since the beginning.
+
+	It was overlooked, however, that the top band could grow to twice the
+	default band size leading to unnecessary memory overflows there. This
+	commit fixes that. Now the bands are distributed more evenly and
+	cannot exceed the default size.
+
+	Now the magic number 8 is really suitable for rather simple European
+	scripts. For complex Chinese logograms the magic number should be 13
+	but that is subject for another day.
+
+	* src/smooth/ftgrays.c (gray_convert_glyph): Revise sub-banding
+	protocol.
+
 2016-07-07  suzuki toshiya  <mpsuzuki@hiroshima-u.ac.jp>
 
 	[mac] Fix Savannah bug #48417.
diff --git a/src/smooth/ftgrays.c b/src/smooth/ftgrays.c
index f0e380e40..12df14dc4 100644
--- a/src/smooth/ftgrays.c
+++ b/src/smooth/ftgrays.c
@@ -1893,27 +1893,28 @@ typedef ptrdiff_t  FT_PtrDist;
   gray_convert_glyph( RAS_ARG )
   {
     TCell        buffer[FT_MAX_GRAY_POOL];
-    const int    band_size = FT_MAX_GRAY_POOL / 8;
-    gray_TBand   bands[32];
-    gray_TBand*  band;
-    int          n, num_bands;
+    TCoord       band_size = FT_MAX_GRAY_POOL / 8;
+    int          num_bands;
     TCoord       min, max, max_y;
+    gray_TBand   bands[32];  /* enough to accommodate bisections */
+    gray_TBand*  band;
 
 
     /* set up vertical bands */
-    num_bands = (int)( ( ras.max_ey - ras.min_ey ) / band_size );
-    if ( num_bands == 0 )
-      num_bands = 1;
-    if ( num_bands >= 39 )
-      num_bands = 39;
+    if ( ras.count_ey > band_size )
+    {
+      /* two divisions rounded up */
+      num_bands = (int)( ( ras.count_ey + band_size - 1) / band_size );
+      band_size = ( ras.count_ey + num_bands - 1 ) / num_bands;
+    }
 
     min   = ras.min_ey;
     max_y = ras.max_ey;
 
-    for ( n = 0; n < num_bands; n++, min = max )
+    for (; min < max_y; min = max )
     {
       max = min + band_size;
-      if ( n == num_bands - 1 || max > max_y )
+      if ( max > max_y )
         max = max_y;
 
       bands[0].min = min;