codecs: h264dec: Improve the policy to infer max_num_reorder_frames.

The max_num_reorder_frames number can change the way we bumping the
pictures in the DPB. The smaller it is, the lower latency we will
get. So it is important for live mode streams, but it is not given
in VUI parameters sometimes. We now improve the policy to infer it:
1. Never guess it in the "strict" compliance.
2. For baseline and constrained baseline profiles, which do not have
   B frames, set it to 0.
3. For -intra only profiles, set it to 0.
4. Otherwise, not guess it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2501>

1 files changed, 13 insertions, 3 deletions

diff --git a/gst-libs/gst/codecs/gsth264decoder.c b/gst-libs/gst/codecs/gsth264decoder.c
index bded2c363..e3f7cab31 100644
--- a/gst-libs/gst/codecs/gsth264decoder.c
+++ b/gst-libs/gst/codecs/gsth264decoder.c
@@ -2181,9 +2181,19 @@ gst_h264_decoder_update_max_num_reorder_frames (GstH264Decoder * self,
     return TRUE;
   }
 
-  /* max_num_reorder_frames not present, infer from profile/constraints
-   * (see VUI semantics in spec) */
-  if (sps->constraint_set3_flag) {
+  if (priv->compliance == GST_H264_DECODER_COMPLIANCE_STRICT) {
+    gst_h264_dpb_set_max_num_reorder_frames (priv->dpb,
+        gst_h264_dpb_get_max_num_frames (priv->dpb));
+    return TRUE;
+  }
+
+  /* max_num_reorder_frames not present, infer it from profile/constraints. */
+  if (sps->profile_idc == 66 || sps->profile_idc == 83) {
+    /* baseline, constrained baseline and scalable-baseline profiles
+       only contain I/P frames. */
+    max_num_reorder_frames = 0;
+  } else if (sps->constraint_set3_flag) {
+    /* constraint_set3_flag may mean the -intra only profile. */
     switch (sps->profile_idc) {
       case 44:
       case 86: