介绍

FFmpeg的 libavcodec 模块完成音视频多媒体的编解码模块。
老版本的 FFmpeg 将avcodec_decode_video2()作为视频的解码函数 API，将avcodec_decode_audio4()作为音频的解码函数 API；从 3.4版本开始已经将二者标记为废弃过时 API（attribute_deprecated）。
新版本 FFmpeg 将 avcodec_send_packet() 与 avcodec_receive_frame() 作为音视频的解码函数 API，但同时仍然保留了对老接口的兼容，通过avcodec_decode_video2()、avcodec_decode_audio4()调用 compat_decode()完成对新 API 的封装。

//具体可以参考 FFmpeg 中 doc/APIchanges 中的记录.

2016-04-21 - 7fc329e - lavc 57.37.100 - avcodec.h
  Add a new audio/video encoding and decoding API with decoupled input
  and output -- avcodec_send_packet(), avcodec_receive_frame(),
  avcodec_send_frame() and avcodec_receive_packet().
  
2017-09-26 - b1cf151c4d - lavc 57.106.102 - avcodec.h
  Deprecate AVCodecContext.refcounted_frames. This was useful for deprecated
  API only (avcodec_decode_video2/avcodec_decode_audio4). The new decode APIs
  (avcodec_send_packet/avcodec_receive_frame) always work with reference
  counted frames.

视频解码详细流程

以解码 H264标准为例，从 main 函数到最后的 MB 宏块解码。

解码的核心 API 介绍

avcodec_send_packet()

API 申明介绍

 * @param avctx codec context
 * @param[in] avpkt The input AVPacket. Usually, this will be a single video
 *                  frame, or several complete audio frames.
 *                  Ownership of the packet remains with the caller, and the
 *                  decoder will not write to the packet. The decoder may create
 *                  a reference to the packet data (or copy it if the packet is
 *                  not reference-counted).
 *                  Unlike with older APIs, the packet is always fully consumed,
 *                  and if it contains multiple frames (e.g. some audio codecs),
 *                  will require you to call avcodec_receive_frame() multiple
 *                  times afterwards before you can send a new packet.
 *                  It can be NULL (or an AVPacket with data set to NULL and
 *                  size set to 0); in this case, it is considered a flush
 *                  packet, which signals the end of the stream. Sending the
 *                  first flush packet will return success. Subsequent ones are
 *                  unnecessary and will return AVERROR_EOF. If the decoder
 *                  still has frames buffered, it will return them after sending
 *                  a flush packet.
 *
 * @return 0 on success, otherwise negative error code:
 *      AVERROR(EAGAIN):   input is not accepted in the current state - user
 *                         must read output with avcodec_receive_frame() (once
 *                         all output is read, the packet should be resent, and
 *                         the call will not fail with EAGAIN).
 *      AVERROR_EOF:       the decoder has been flushed, and no new packets can
 *                         be sent to it (also returned if more than 1 flush
 *                         packet is sent)
 *      AVERROR(EINVAL):   codec not opened, it is an encoder, or requires flush
 *      AVERROR(ENOMEM):   failed to add packet to internal queue, or similar
 *      other errors: legitimate decoding errors
 */
int avcodec_send_packet(AVCodecContext *avctx, const AVPacket *avpkt);

avcodec_send_packet()定义解析
该 API 可以理解为 FFmpeg 完成音视频解码的核心函数，从函数的实现中可以看到，首先会调用av_bsf_send_packet()函数将输入的 AVPacket 拷贝到 buffer 中。最后当 buffer_frame为空的时候，就调用decode_receive_frame_internal()函数完成真正的解码。

decode_receive_frame_internal()内部会调用decode_simple_receive_frame()完成解码。

decode_simple_receive_frame()内部会调用decode_simple_internal()完成解码。

而在decode_simple_internal()通过函数指针(*decode())指向不同的解码器完成解码不同标准的解码过程；如果涉及到多线程解码，还会涉及到ff_thread_decode_frame()，多线程解码不详细介绍。
(*decode())函数指针
在 decode_simple_internal()通过ret = avctx->codec->decode(avctx, frame, &got_frame, pkt)实现对不同解码器的调用，完成具体的解码过程。

以h264解码为例，h264_decode_frame()完成了具体的解码过程，其中核心函数是 decode_nal_units()进行 NAUL 解码；其中ff_H2645_packet_split()进行码流解析， ff_h264_execute_decode_slices()进行 Slice 级解码。

在ff_h264_execute_decode_slices()核心函数是 decode_slice()，对每个 Slice 进行解码。

decode_slice()中是视频数据的核心解码过程，主要分成四个模块：熵解码、宏块解码、环路滤波、错误隐藏。其中 ff_h264_decode_mb_cabac()完成熵解码（如果熵解码是 cavlc，则会ff_h264_decode_mb_cavlac 完成熵解码）；ff_h264_decode_mb()完成宏块 MB 解码；loop_filter()完成环路滤波；er_add_slice()完成错误隐藏处理。

avcodec_receive_frame()

API 申明介绍

/**
 * Return decoded output data from a decoder.//从解码器返回解码后的输出数据
 *
 * @param avctx codec context
 * @param frame This will be set to a reference-counted video or audio
 *              frame (depending on the decoder type) allocated by the
 *              decoder. Note that the function will always call
 *              av_frame_unref(frame) before doing anything else.
 *
 * @return
 *      0:                 success, a frame was returned
 *      AVERROR(EAGAIN):   output is not available in this state - user must try
 *                         to send new input
 *      AVERROR_EOF:       the decoder has been fully flushed, and there will be
 *                         no more output frames
 *      AVERROR(EINVAL):   codec not opened, or it is an encoder
 *      AVERROR_INPUT_CHANGED:   current decoded frame has changed parameters
 *                               with respect to first decoded frame. Applicable
 *                               when flag AV_CODEC_FLAG_DROPCHANGED is set.
 *      other negative values: legitimate decoding errors
 */
int avcodec_receive_frame(AVCodecContext *avctx, AVFrame *frame);

avcodec_receive_frame()定义解析
从avcodec_receive_frame()函数内部实现可以看到，核心逻辑相对简单，先判断buffer_frame里是否有数据，如果有直接调用av_frame_move_ref()完成frame 拷贝过程，解码后数据从buffer_frame拷贝到 frame 中（即把数据从 AVCodecContext 中拷贝到 AVFrame 中）；如果 buffer_frame中没有数据，则需要调用decode_receive_frame_internal()完成具体的解码，该步骤和 send packet 模块相同。

官方解码实例

/*
 * Copyright (c) 2001 Fabrice Bellard
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 */

/**
 * @file
 * video decoding with libavcodec API example
 *
 * @example decode_video.c
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include <libavcodec/avcodec.h>

#define INBUF_SIZE 4096

static void pgm_save(unsigned char *buf, int wrap, int xsize, int ysize,
                     char *filename)
{
    FILE *f;
    int i;

    f = fopen(filename,"w");
    fprintf(f, "P5\n%d %d\n%d\n", xsize, ysize, 255);
    for (i = 0; i < ysize; i++)
        fwrite(buf + i * wrap, 1, xsize, f);
    fclose(f);
}

static void decode(AVCodecContext *dec_ctx, AVFrame *frame, AVPacket *pkt,
                   const char *filename)
{
    char buf[1024];
    int ret;

    ret = avcodec_send_packet(dec_ctx, pkt);
    if (ret < 0) {
        fprintf(stderr, "Error sending a packet for decoding\n");
        exit(1);
    }

    while (ret >= 0) {
        ret = avcodec_receive_frame(dec_ctx, frame);
        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
            return;
        else if (ret < 0) {
            fprintf(stderr, "Error during decoding\n");
            exit(1);
        }

        printf("saving frame %3d\n", dec_ctx->frame_number);
        fflush(stdout);

        /* the picture is allocated by the decoder. no need to
           free it */
        snprintf(buf, sizeof(buf), "%s-%d", filename, dec_ctx->frame_number);
        pgm_save(frame->data[0], frame->linesize[0],
                 frame->width, frame->height, buf);
    }
}

int main(int argc, char **argv)
{
    const char *filename, *outfilename;
    const AVCodec *codec;
    AVCodecParserContext *parser;
    AVCodecContext *c= NULL;
    FILE *f;
    AVFrame *frame;
    uint8_t inbuf[INBUF_SIZE + AV_INPUT_BUFFER_PADDING_SIZE];
    uint8_t *data;
    size_t   data_size;
    int ret;
    AVPacket *pkt;

    if (argc <= 2) {
        fprintf(stderr, "Usage: %s <input file> <output file>\n"
                "And check your input file is encoded by mpeg1video please.\n", argv[0]);
        exit(0);
    }
    filename    = argv[1];
    outfilename = argv[2];

    pkt = av_packet_alloc();
    if (!pkt)
        exit(1);

    /* set end of buffer to 0 (this ensures that no overreading happens for damaged MPEG streams) */
    memset(inbuf + INBUF_SIZE, 0, AV_INPUT_BUFFER_PADDING_SIZE);

    /* find the MPEG-1 video decoder */
    codec = avcodec_find_decoder(AV_CODEC_ID_MPEG1VIDEO);
    if (!codec) {
        fprintf(stderr, "Codec not found\n");
        exit(1);
    }

    parser = av_parser_init(codec->id);
    if (!parser) {
        fprintf(stderr, "parser not found\n");
        exit(1);
    }

    c = avcodec_alloc_context3(codec);
    if (!c) {
        fprintf(stderr, "Could not allocate video codec context\n");
        exit(1);
    }

    /* For some codecs, such as msmpeg4 and mpeg4, width and height
       MUST be initialized there because this information is not
       available in the bitstream. */

    /* open it */
    if (avcodec_open2(c, codec, NULL) < 0) {
        fprintf(stderr, "Could not open codec\n");
        exit(1);
    }

    f = fopen(filename, "rb");
    if (!f) {
        fprintf(stderr, "Could not open %s\n", filename);
        exit(1);
    }

    frame = av_frame_alloc();
    if (!frame) {
        fprintf(stderr, "Could not allocate video frame\n");
        exit(1);
    }

    while (!feof(f)) {
        /* read raw data from the input file */
        data_size = fread(inbuf, 1, INBUF_SIZE, f);
        if (!data_size)
            break;

        /* use the parser to split the data into frames */
        data = inbuf;
        while (data_size > 0) {
            ret = av_parser_parse2(parser, c, &pkt->data, &pkt->size,
                                   data, data_size, AV_NOPTS_VALUE, AV_NOPTS_VALUE, 0);
            if (ret < 0) {
                fprintf(stderr, "Error while parsing\n");
                exit(1);
            }
            data      += ret;
            data_size -= ret;

            if (pkt->size)
                decode(c, frame, pkt, outfilename);
        }
    }

    /* flush the decoder */
    decode(c, frame, NULL, outfilename);

    fclose(f);

    av_parser_close(parser);
    avcodec_free_context(&c);
    av_frame_free(&frame);
    av_packet_free(&pkt);

    return 0;
}