decoder: h264: improve AVC_REF_IDX_STATE for MVC.

Each Reference List Entry has Bit 6 set to one if the reference
picture is to be used as a long-term reference picture. However,
the H.264 standard, and subsequently the VA-API specs, makes it
possible to mark the picture as "used for short-term reference",
as "used for long-term reference", or even none of those flags.

This means we have to handle a minimum of 3 states. This doesn't
fit the range of a single bit. Let's examine how this could be
fixed from known practices.

There are cases where the picture is added to RefPicListX[] even
if it is not marked as "used for short-term reference" or "used
for long-term reference": MVC with inter-view reference components
or inter-view only reference components [H.8.4]. Ultimately, this
has an incidence on the value of colZeroFlag (8.4.1.2.2). Since
there is no way to program that, and that it depends on the picture
to be marked as "used for short-term reference" or not, then it
looks reasonable to imply Bit 6 (LongTermPicFlag) as a picture
that is *not* "used for short-term reference", i.e. thus including
genuine long-term reference pictures, and those that are neither
long-term reference nor short-term reference pictures.

In practice, this fixes MVCNV-2.264.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
(cherry picked from commit edbdc0e87919d8b7261d882a32b2d3c271660931)

1 files changed, 17 insertions, 1 deletions

diff --git a/src/i965_decoder_utils.c b/src/i965_decoder_utils.c
index 7e3d33ab..7833919f 100644
--- a/src/i965_decoder_utils.c
+++ b/src/i965_decoder_utils.c
@@ -349,8 +349,24 @@ avc_get_first_mb_bit_offset_with_epb(
 static inline uint8_t
 get_ref_idx_state_1(const VAPictureH264 *va_pic, unsigned int frame_store_id)
 {
+    /* The H.264 standard, and the VA-API specification, allows for at
+       least 3 states for a picture: "used for short-term reference",
+       "used for long-term reference", or considered as not used for
+       reference.
+
+       The latter is used in the MVC inter prediction and inter-view
+       prediction process (H.8.4). This has an incidence on the
+       colZeroFlag variable, as defined in 8.4.1.2.
+
+       Since it is not possible to directly program that flag, let's
+       make the hardware derive this value by assimilating "considered
+       as not used for reference" to a "not used for short-term
+       reference", and subsequently making it "used for long-term
+       reference" to fit the definition of Bit6 here */
+    const unsigned int ref_flags = VA_PICTURE_H264_SHORT_TERM_REFERENCE |
+        VA_PICTURE_H264_LONG_TERM_REFERENCE;
     const unsigned int is_long_term =
-        !!(va_pic->flags & VA_PICTURE_H264_LONG_TERM_REFERENCE);
+        ((va_pic->flags & ref_flags) != VA_PICTURE_H264_SHORT_TERM_REFERENCE);
     const unsigned int is_top_field =
         !!(va_pic->flags & VA_PICTURE_H264_TOP_FIELD);
     const unsigned int is_bottom_field =