avcodecmap: Don't try converting channel layouts with more than 64 channels1.16

We only support up to 64 channels in GStreamer with a specific layout so
it's safe to assume a NONE layout in this case.

Also the arrays of channel positions are allocated everywhere with 64
elements so don't try setting more than 64 to NONE as that will cause
stack corruptions and similar memory safety issues.

Thanks to Natalie Silvanovich for reporting this issue.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-libav/-/issues/92

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-libav/-/merge_requests/122>

1 files changed, 2 insertions, 2 deletions

diff --git a/ext/libav/gstavcodecmap.c b/ext/libav/gstavcodecmap.c
index 0c5a42f..6e0ab4c 100644
--- a/ext/libav/gstavcodecmap.c
+++ b/ext/libav/gstavcodecmap.c
@@ -124,7 +124,7 @@ gst_ffmpeg_channel_layout_to_gst (guint64 channel_layout, gint channels,
   guint nchannels = 0;
   gboolean none_layout = FALSE;
 
-  if (channel_layout == 0) {
+  if (channel_layout == 0 || channels > 64) {
     nchannels = channels;
     none_layout = TRUE;
   } else {
@@ -185,7 +185,7 @@ gst_ffmpeg_channel_layout_to_gst (guint64 channel_layout, gint channels,
     } else {
       guint i;
 
-      for (i = 0; i < nchannels; i++)
+      for (i = 0; i < nchannels && i < 64; i++)
         pos[i] = GST_AUDIO_CHANNEL_POSITION_NONE;
     }
   }