lavf/qtpalette: Ignore greyscale bit in certain cases

The QuickTime File Format Specification states the following:

"Depth: A 16-bit integer that indicates the pixel depth of the
compressed image. Values of 1, 2, 4, 8 ,16, 24, and 32 indicate the
depth of color images. The value 32 should be used only if the image
contains an alpha channel. Values of 34, 36, and 40 indicate 2-, 4-, and
8-bit grayscale, respectively, for grayscale images."

There is no mention of value 33, i.e. 1-bit video (0x01) with the
greyscale bit (0x20) set. I therefore suggest that we ignore the
greyscale bit when processing 1-bit video. Another reason to do this is
that the first 1-bit sample file below will be displayed properly with
blue colors in QuickTime in Windows or Mac *in spite of* the greyscale
bit being set.

Also, QuickTime in Windows or Mac ignores the greyscale bit if the
video sample description contains a palette, regardless of bit depth.
This is undocumented behaviour, but I think we should do the same, and
it seems pretty logical after all, since one wouldn't really bother
putting a customized palette into a grayscale file anyway. See the
second 8-bit sample file below, which has the greyscale bit set, and
which contains a palette in the video sample description. In Windows or
Mac, it will be displayed with the palette in the sample description, in
spite of the greyscale bit being set.

Sample file 1 (1-bit QuickTime Animation):
https://drive.google.com/open?id=0B3_pEBoLs0faTThSek1EeXQ0ZHM
Earth Spin 1-bit qtrle orig.mov

Sample file 2 (8-bit QuickTime Animation):
https://drive.google.com/open?id=0B3_pEBoLs0fad2s0V1YzUWo5aDA
quiz-palette+gs.mov

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

1 files changed, 3 insertions, 1 deletions

diff --git a/libavformat/qtpalette.c b/libavformat/qtpalette.c
index 6544a551eb..a78b6af97a 100644
--- a/libavformat/qtpalette.c
+++ b/libavformat/qtpalette.c
@@ -51,7 +51,9 @@ int ff_get_qtpalette(int codec_id, AVIOContext *pb, uint32_t *palette)
         int color_count, color_start, color_end;
         uint32_t a, r, g, b;
 
-        if (greyscale) {
+        /* Ignore the greyscale bit for 1-bit video and sample
+         * descriptions containing a color table. */
+        if (greyscale && bit_depth > 1 && color_table_id) {
             int color_index, color_dec;
             /* compute the greyscale palette */
             color_count = 1 << bit_depth;