avformat/mux: Don't use stack packet when writing interleaved packets

Currently the interleave_packet functions use a packet for
a new packet to be interleaved (may be NULL if there is none) and
a packet for output; said packet is always a stack packet in
interleaved_write_packet(). But all the interleave_packet functions
in use first move the packet to the packet list and then check whether
a packet can be returned, i.e. the effective lifetime of the new packet
ends before the packet for output is touched.

So one can use one packet both for input and output by adding a new
parameter that indicates whether there is a packet to add to the packet
list; there is just one complication: In case the muxer is flushed,
there is no packet available. This can be solved by reusing one of
the packets from AVFormatInternal. They are currently unused when
flushing in av_interleaved_write_frame().

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>

1 files changed, 15 insertions, 3 deletions

diff --git a/libavformat/avformat.h b/libavformat/avformat.h
index f054f7a405..a2af7e9f89 100644
--- a/libavformat/avformat.h
+++ b/libavformat/avformat.h
@@ -553,9 +553,21 @@ typedef struct AVOutputFormat {
     /**
      * A format-specific function for interleavement.
      * If unset, packets will be interleaved by dts.
-     */
-    int (*interleave_packet)(struct AVFormatContext *, AVPacket *out,
-                             AVPacket *in, int flush);
+     *
+     * @param s           An AVFormatContext for output. pkt will be added to
+     *                    resp. taken from its packet buffer.
+     * @param[in,out] pkt A packet to be interleaved if has_packet is set;
+     *                    also used to return packets. If no packet is returned
+     *                    (e.g. on error), pkt is blank on return.
+     * @param flush       1 if no further packets are available as input and
+     *                    all remaining packets should be output.
+     * @param has_packet  If set, pkt contains a packet to be interleaved
+     *                    on input; otherwise pkt is blank on input.
+     * @return 1 if a packet was output, 0 if no packet could be output,
+     *         < 0 if an error occurred
+     */
+    int (*interleave_packet)(struct AVFormatContext *s, AVPacket *pkt,
+                             int flush, int has_packet);
     /**
      * Test if the given codec can be stored in this container.
      *