前言

  最近使用SDL2在鸿蒙系统(Harmoney OS)上截取视频播放过程中的数据，发现捕获的数据为空，然在windows上却可以正常捕获，欲定位其中问题并解决之。

一、SDL_RenderReadPixels简介

  SDL_RenderReadPixels用于从当前渲染目标（render target）读取像素数据，以便进行后续处理、保存图像或者其他用途。其参数定义及实现如下:
  renderer：指向用于渲染的 SDL_Renderer。
  rect：指向定义读取区域的 SDL_Rect。如果为 NULL，则读取整个渲染目标。
  format：希望得到的像素数据格式。使用 SDL 的像素格式，如 SDL_PIXELFORMAT_ARGB8888。
  pixels：指向用于存储读取到的像素数据的缓冲区。
  pitch：一行像素数据的字节数，即每行像素数据的步幅（宽度）。

SDL_RenderReadPixels(SDL_Renderer * renderer, const SDL_Rect * rect,
                     Uint32 format, void * pixels, int pitch)
{
    SDL_Rect real_rect;

    CHECK_RENDERER_MAGIC(renderer, -1);

    if (!renderer->RenderReadPixels) {
        return SDL_Unsupported();
    }

    FlushRenderCommands(renderer);  /* we need to render before we read the results. */

    if (!format) {
        format = SDL_GetWindowPixelFormat(renderer->window);
    }

    real_rect.x = renderer->viewport.x;
    real_rect.y = renderer->viewport.y;
    real_rect.w = renderer->viewport.w;
    real_rect.h = renderer->viewport.h;
    if (rect) {
        if (!SDL_IntersectRect(rect, &real_rect, &real_rect)) {
            return 0;
        }
        if (real_rect.y > rect->y) {
            pixels = (Uint8 *)pixels + pitch * (real_rect.y - rect->y);
        }
        if (real_rect.x > rect->x) {
            int bpp = SDL_BYTESPERPIXEL(format);
            pixels = (Uint8 *)pixels + bpp * (real_rect.x - rect->x);
        }
    }

    return renderer->RenderReadPixels(renderer, &real_rect,
                                      format, pixels, pitch);
}

1. 上面先确定当前系统有无适配渲染器的RenderReadPixels函数，在每个平台下都有自己的渲染器实现，如windows的D3D11，苹果的metal，索尼的psp、嵌入式系统如安卓和鸿蒙的opengles，及其他平台的opengl。

2. 使用FlushRenderCommands将当前渲染命令队列中的命令发送给GPU执行渲染，这里可以看到官方的注释说一定要在渲染之后才能读取到像素数据。【这里同时我也在github上问了下SDL的维护人员进行了确认：】
   

3. 如果传入的rect比实际渲染视口real_rect的大小要小，，则将更新pixels指向的起始捕获地址，这里如果是rgba8888格式，那么pitch就是1920(视频宽) * 4 = 7680

二、问题现象

  从上面的renderer->RenderReadPixels调到具体各平台的实现：
  在Open GLES上直接调用渲染器的driverdata的glReadPixels获取数据，但是此处得到的数据是个空值，具体也无法再往下跟了。

static int
GLES2_RenderReadPixels(SDL_Renderer * renderer, const SDL_Rect * rect,
                    Uint32 pixel_format, void * pixels, int pitch)
{
    GLES2_RenderData *data = (GLES2_RenderData *)renderer->driverdata;
    Uint32 temp_format = renderer->target ? renderer->target->format : SDL_PIXELFORMAT_ABGR8888;
    size_t buflen;
    void *temp_pixels;
    int temp_pitch;
    Uint8 *src, *dst, *tmp;
    int w, h, length, rows;
    int status;

    temp_pitch = rect->w * SDL_BYTESPERPIXEL(temp_format);
    buflen = rect->h * temp_pitch;
    if (buflen == 0) {
        return 0;  /* nothing to do. */
    }

    temp_pixels = SDL_malloc(buflen);
    if (!temp_pixels) {
        return SDL_OutOfMemory();
    }

    SDL_GetRendererOutputSize(renderer, &w, &h);

    data->glReadPixels(rect->x, renderer->target ? rect->y : (h-rect->y)-rect->h,
                       rect->w, rect->h, GL_RGBA, GL_UNSIGNED_BYTE, temp_pixels);
    if (GL_CheckError("glReadPixels()", renderer) < 0) {
        return -1;
    }

    /* Flip the rows to be top-down if necessary */
    if (!renderer->target) {
        SDL_bool isstack;
        length = rect->w * SDL_BYTESPERPIXEL(temp_format);
        src = (Uint8*)temp_pixels + (rect->h-1)*temp_pitch;
        dst = (Uint8*)temp_pixels;
        tmp = SDL_small_alloc(Uint8, length, &isstack);
        rows = rect->h / 2;
        while (rows--) {
            SDL_memcpy(tmp, dst, length);
            SDL_memcpy(dst, src, length);
            SDL_memcpy(src, tmp, length);
            dst += temp_pitch;
            src -= temp_pitch;
        }
        SDL_small_free(tmp, isstack);
    }

    status = SDL_ConvertPixels(rect->w, rect->h,
                               temp_format, temp_pixels, temp_pitch,
                               pixel_format, pixels, pitch);
    SDL_free(temp_pixels);

    return status;
}

  在windows下，同样的流程使用D3D11，确能正常获取到数据：

static int
D3D11_RenderReadPixels(SDL_Renderer * renderer, const SDL_Rect * rect,
                       Uint32 format, void * pixels, int pitch)
{
    D3D11_RenderData * data = (D3D11_RenderData *) renderer->driverdata;
    ID3D11Texture2D *backBuffer = NULL;
    ID3D11Texture2D *stagingTexture = NULL;
    HRESULT result;
    int status = -1;
    D3D11_TEXTURE2D_DESC stagingTextureDesc;
    D3D11_RECT srcRect = {0, 0, 0, 0};
    D3D11_BOX srcBox;
    D3D11_MAPPED_SUBRESOURCE textureMemory;

    /* Retrieve a pointer to the back buffer: */
    result = IDXGISwapChain_GetBuffer(data->swapChain,
        0,
        &SDL_IID_ID3D11Texture2D,
        (void **)&backBuffer
        );
    if (FAILED(result)) {
        WIN_SetErrorFromHRESULT(SDL_COMPOSE_ERROR("IDXGISwapChain1::GetBuffer [get back buffer]"), result);
        goto done;
    }

    /* Create a staging texture to copy the screen's data to: */
    ID3D11Texture2D_GetDesc(backBuffer, &stagingTextureDesc);
    stagingTextureDesc.Width = rect->w;
    stagingTextureDesc.Height = rect->h;
    stagingTextureDesc.BindFlags = 0;
    stagingTextureDesc.MiscFlags = 0;
    stagingTextureDesc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
    stagingTextureDesc.Usage = D3D11_USAGE_STAGING;
    result = ID3D11Device_CreateTexture2D(data->d3dDevice,
        &stagingTextureDesc,
        NULL,
        &stagingTexture);
    if (FAILED(result)) {
        WIN_SetErrorFromHRESULT(SDL_COMPOSE_ERROR("ID3D11Device1::CreateTexture2D [create staging texture]"), result);
        goto done;
    }

    /* Copy the desired portion of the back buffer to the staging texture: */
    if (D3D11_GetViewportAlignedD3DRect(renderer, rect, &srcRect, FALSE) != 0) {
        /* D3D11_GetViewportAlignedD3DRect will have set the SDL error */
        goto done;
    }

    srcBox.left = srcRect.left;
    srcBox.right = srcRect.right;
    srcBox.top = srcRect.top;
    srcBox.bottom = srcRect.bottom;
    srcBox.front = 0;
    srcBox.back = 1;
    ID3D11DeviceContext_CopySubresourceRegion(data->d3dContext,
        (ID3D11Resource *)stagingTexture,
        0,
        0, 0, 0,
        (ID3D11Resource *)backBuffer,
        0,
        &srcBox);

    /* Map the staging texture's data to CPU-accessible memory: */
    result = ID3D11DeviceContext_Map(data->d3dContext,
        (ID3D11Resource *)stagingTexture,
        0,
        D3D11_MAP_READ,
        0,
        &textureMemory);
    if (FAILED(result)) {
        WIN_SetErrorFromHRESULT(SDL_COMPOSE_ERROR("ID3D11DeviceContext1::Map [map staging texture]"), result);
        goto done;
    }

    /* Copy the data into the desired buffer, converting pixels to the
     * desired format at the same time:
     */
    if (SDL_ConvertPixels(
        rect->w, rect->h,
        D3D11_DXGIFormatToSDLPixelFormat(stagingTextureDesc.Format),
        textureMemory.pData,
        textureMemory.RowPitch,
        format,
        pixels,
        pitch) != 0) {
        /* When SDL_ConvertPixels fails, it'll have already set the format.
         * Get the error message, and attach some extra data to it.
         */
        char errorMessage[1024];
        SDL_snprintf(errorMessage, sizeof(errorMessage), "%s, Convert Pixels failed: %s", __FUNCTION__, SDL_GetError());
        SDL_SetError("%s", errorMessage);
        goto done;
    }

    /* Unmap the texture: */
    ID3D11DeviceContext_Unmap(data->d3dContext,
        (ID3D11Resource *)stagingTexture,
        0);

    status = 0;

done:
    SAFE_RELEASE(backBuffer);
    SAFE_RELEASE(stagingTexture);
    return status;
}

  可以看出ID3D11Device_CreateTexture2D首先创建了一个纹理，然后复制后缓冲区的一部分到纹理中，然后通过ID3D11DeviceContext_Map将暂存纹理的数据映射到 CPU 可访问的内存，最后将数据复制到所需的缓冲区，同时将像素转换为所需格式。
  从上面不难看出，两者的主要不同在于D3D11中创建了一个离屏的纹理，通过将这个纹理映射到内存中来读取数据，这样即使视频渲染和截屏操作不在同一个线程中，也可以捕获到正常的数据。

三、规避方案

1. 离屏纹理

  于是乎，想参照D3D11一样也搞个离屏纹理，然后使用离屏纹理的数据来生成图片，代码如下：

	SDL_Rect renderRect = { 0, 0, viewSize.width(), viewSize.height() };
	// 1.先创建纹理
	SDL_Texture* targetTexture = _createSdlTexture(m_sdlRender, SDL_PIXELFORMAT_ARGB8888, SDL_TEXTUREACCESS_TARGET, renderRect.w, renderRect.h, m_videoControl);
	if (!targetTexture)
		return false;
	// 2.设置渲染目标
	SDL_SetRenderTarget(m_sdlRender, targetTexture);
	// 3.拷贝纹理
	SDL_RenderCopyEx(m_sdlRender, m_videoYUVTexture, &m_videoOriginRect, &renderRect, 0, nullptr, SDL_FLIP_NONE);

	*ppImg = new QImage(renderRect.w, renderRect.h, QImage::Format_ARGB32);
	(*ppImg)->fill(0);

	uchar* data = (*ppImg)->bits();
	// 4.将拷贝的纹理中像素数据读到QImage中去
	SDL_RenderReadPixels(m_sdlRender, &renderRect, SDL_PIXELFORMAT_ARGB8888, (void*)data, (*ppImg)->bytesPerLine());
	// 5.销毁渲染目标
	SDL_SetRenderTarget(m_sdlRender, nullptr);

	_safeDestroy(targetTexture, &SDL_DestroyTexture, m_videoControl);

  这样可是可以，只不过只能适用于一张纹理的情况（欲哭无泪.jpg）,如果同时还要捕获视频的前景和背景，实现是可以实现，就太繁琐了。

2. ding! 灵光一现

  既然github上SDL的维护人员说SDL_RenderReadPixels一定要在SDL_RenderPresent之前调用才有数据，那SDL_RenderPresent里面是不是有方法让它不去交换前后缓冲区呢？如果不交换缓冲区，就可以捕获到当前前景+背景+视频帧的所有纹理数据了！
  带着这个疑问，去看了下SDL_RenderPresent的实现：

void
SDL_RenderPresent(SDL_Renderer * renderer)
{
    CHECK_RENDERER_MAGIC(renderer, );

    FlushRenderCommands(renderer);  /* time to send everything to the GPU! */

    /* Don't present while we're hidden */
    if (renderer->hidden) {
        return;
    }
    renderer->RenderPresent(renderer);
}

  嘿嘿，在交换缓冲区之前有个判断渲染器是否隐藏，如果隐藏就直接return了，好家伙，可以用这个来控制。又由于渲染器SDL_Render和SDL_Window是绑定的，因此就直接在SDL_RenderReadPixels前隐藏窗口就OK了：

	SDL_HideWindow(m_sdlWindow);
	SDL_RenderReadPixels(m_sdlRenderer, &destRect, SDL_PIXELFORMAT_ARGB8888, (void*)data, image.bytesPerLine());
	SDL_RaiseWindow(m_sdlWindow);

真是踏破铁鞋无觅处，得来全不费工夫
XD