avcodec/decode: reset codec on receiving packet after EOF in compat_decode

In commit 061a0c14bb57 ("decode: restructure the core decoding code"), the
deprecated avcodec_decode_* APIs were reworked so that they called into the
new avcodec_send_packet / avcodec_receive_frame API. This had the side effect
of prohibiting sending new packets containing data after a drain
packet, but in previous versions of FFmpeg this "worked" and some
applications relied on it.

To restore some compatibility, reset the codec if we receive a new non-drain
packet using the old API after draining has completed. While this does
not give the same behaviour as the old API did, in the majority of cases
it works and it does not require changes to any other part of the decoding
code.

Fixes ticket #6775
Signed-off-by: James Cowgill <jcowgill@debian.org>
Signed-off-by: Marton Balint <cus@passwd.hu>

1 files changed, 5 insertions, 0 deletions

diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index 3f5b086f7e..f67b214759 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -749,6 +749,11 @@ static int compat_decode(AVCodecContext *avctx, AVFrame *frame,
 
     av_assert0(avci->compat_decode_consumed == 0);
 
+    if (avci->draining_done && pkt && pkt->size != 0) {
+        av_log(avctx, AV_LOG_WARNING, "Got unexpected packet after EOF\n");
+        avcodec_flush_buffers(avctx);
+    }
+
     *got_frame = 0;
     avci->compat_decode = 1;