Android平台播放透明视频
思路
设计一种特殊的视频,它的一半内容存储alpha信息,另一半内容存储rgb信息,接着通过OpenGL获取每个像素点的alpha值和rgb值进行混合,最后出来的画面就是带有透明效果的视频了。
可以上下的分,也可以左右的分,区别在于glsl的写法。
演示demo中的视频素材采用左右分的方式。
demo-video
效果
开发
主要用到MediaPlayer、SurfaceTexture、GLSurfaceview。
利用MediaPlayer配合SurfaceTexture,把视频画面转成纹理,作为输入,通过OpenGL渲染管线处理,最终画到GLSurfaceView上。
核心代码如下:
利用SurfaceTexture接收视频帧,方便二次处理。
mSurfaceTexture = new SurfaceTexture(mTexture);
mSurfaceTexture.setOnFrameAvailableListener(this);
Surface surface = new Surface(mSurfaceTexture);
...
mMediaPlayer.setSurface(surface);
...
shader代码(视频左边是alpha信息,右边的rgb信息)
public static final String VERTEX_SHADER = "uniform mat4 surfaceTransformMatrix;\n" +
"attribute vec4 position;\n" +
"attribute vec4 inputTextureCoordinate;\n" +
" \n" +
"varying vec2 rgbTextureCoordinate;\n" +
"varying vec2 alphaTextureCoordinate;\n" +
" \n" +
"void main()\n" +
"{\n" +
" gl_Position = position;\n" +
" float rgbXOrigin = 0.5;\n" +
" float alphaXOrigin = 0.0;\n" +
" float channelScale = 2.0;\n" +
"\n" +
" float rgbX = inputTextureCoordinate.x / channelScale + rgbXOrigin;\n" +
" float alphaX = inputTextureCoordinate.x / channelScale + alphaXOrigin;\n" +
"\n" +
" vec4 positionInRgbTexture = vec4(rgbX, inputTextureCoordinate.y, inputTextureCoordinate.zw);\n" +
" vec4 positionInAlphaTexture = vec4(alphaX, inputTextureCoordinate.y , inputTextureCoordinate.zw);\n" +
"\n" +
" rgbTextureCoordinate = (surfaceTransformMatrix * positionInRgbTexture).xy;\n" +
" alphaTextureCoordinate = (surfaceTransformMatrix * positionInAlphaTexture).xy;\n" +
"}";
public static final String ALPHA_BLEND_FRAGMENT_SHADER = "#extension GL_OES_EGL_image_external : require\n" +
"precision mediump float;\n" +
"varying vec2 rgbTextureCoordinate;\n" +
"varying vec2 alphaTextureCoordinate;\n" +
"\n" +
" uniform samplerExternalOES inputImageTexture;\n" +
" \n" +
" void main()\n" +
" {\n" +
" vec4 rgbColor = texture2D(inputImageTexture, rgbTextureCoordinate);\n" +
" float alphaColor = texture2D(inputImageTexture, alphaTextureCoordinate).g;\n" +
"\n" +
" gl_FragColor = vec4(rgbColor.rgb * alphaColor, alphaColor);\n" +
" }";
https://github.com/MRYangY/AlphaVideo
结语
- 标题虽然说是Android平台播放透明视频,但是因为是用OpenGL来做的,其他平台也可以用这个思路和shader来实现。
- 虽然透明视频效果出来了,但是应该发现,会有变形。这是因为存在比例不一致导致的,需要修改顶点坐标或者viewport,后面会再写篇文章细说。
参考
https://medium.com/go-electra/unlock-transparency-in-videos-on-android-5dc43776cc72#id_token=eyJhbGciOiJSUzI1NiIsImtpZCI6Ijc3NzBiMDg1YmY2NDliNzI2YjM1NzQ3NjQwMzBlMWJkZTlhMTBhZTYiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJuYmYiOjE2ODM0NDEyNDYsImF1ZCI6IjIxNjI5NjAzNTgzNC1rMWs2cWUwNjBzMnRwMmEyamFtNGxqZGNtczAwc3R0Zy5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsInN1YiI6IjExODIxNjI0MzUwNjQwOTM1MDQ1OSIsImVtYWlsIjoibXJ5YW5neTQyM0BnbWFpbC5jb20iLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwiYXpwIjoiMjE2Mjk2MDM1ODM0LWsxazZxZTA2MHMydHAyYTJqYW00bGpkY21zMDBzdHRnLmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwibmFtZSI6InJhaW4gcmFpbiIsInBpY3R1cmUiOiJodHRwczovL2xoMy5nb29nbGV1c2VyY29udGVudC5jb20vYS9BR05teXhhRU9SZVFUT0VMYzl1Yk9vYkt5ZTNNbk9iQUhZaVc2TWJIWFpPNj1zOTYtYyIsImdpdmVuX25hbWUiOiJyYWluIiwiZmFtaWx5X25hbWUiOiJyYWluIiwiaWF0IjoxNjgzNDQxNTQ2LCJleHAiOjE2ODM0NDUxNDYsImp0aSI6ImFjNTY0ZmMyM2YwMDA0ZDk1YWY2M2Y2MDcwNzFhODdiNWY0NGJiMjcifQ.qz1p119V5neaOVDn-VppVuay_5xPcaePXJwYgqtayUHNm4rrHuFMQqBJNQqGz-BF_Ik-z99JuKE12XDJwmaIiRa5ozW-x10f8Tr1yQ3ku8SEQnwwUi5LZFIck3UvYdvYHkesivNrlsE08YyybiR1zodeKkGwt9yNtzzEWzYUFxHrJOswbcJFpKG60B7Cri7GT_ttJml78dJHhl32o1mYj0wp_B0TyGciy2Pn88VrLS5sUIxNYu7sKKadtn2md7P3Rh-pbt9gLH7TxYkXMBrYD4AHxOM_bfhXfQmFOAvZgcfEznSWsh_2DvBmWELPsrc6c3o4420rMdAChw29E762WA
https://github.com/bytedance/AlphaPlayer