播放rtsp camera 内容

端口554在网络通信中用于Real Time Streaming Protocol(RTSP)。

gst-launch-1.0 playbin uri=rtsp://admin:WANGfengtu12@10.0.20.190:554/client0x
gst-launch-1.0 playbin uri=rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x

gst-launch-1.0 rtspsrc location=rtsp://admin:WANGfengtu12@10.0.20.70:554/client0x ! rtph264depay ! h264parse ! decodebin ! autovideosink
gst-launch-1.0 rtspsrc location=rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x ! rtph264depay ! h264parse ! decodebin ! autovideosink

尝试使用v4l2 去解码

gst-launch-1.0 rtspsrc location=rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x latency=10 ! rtph264depay ! h264parse ! v4l2h264dec capture-io-mode=4 ! v4l2convert output-io-mode=5 capture-io-mode=4 ! autovideosink sync=false

在这里插入图片描述

Ref

https://gstreamer.freedesktop.org/documentation/tutorials/playback/hardware-accelerated-video-decoding.html?gi-language=c
https://github.com/uutzinger/camera/blob/master/RTSP_RTP_gstreamer.md

NNStreamer

Env

Ubuntu18.04
x86-64

# 安装
$ sudo add-apt-repository ppa:nnstreamer/ppa
$ sudo apt-get update
$ sudo apt-get install nnstreamer

$ sudo add-apt-repository ppa:nnstreamer-example/ppa
$ sudo apt-get update
$ sudo apt-get install nnstreamer-example

sudo apt install nnstreamer-tensorflow2-lite
$ cd /usr/lib/nnstreamer/bin

./nnstreamer_example_pose_estimation_tflite 
python3 nnstreamer_example_object_detection_tflite.py

使用 rtsp 流去做推理 nnstreamer_example_object_detection_tflite.py

        self.pipeline = Gst.parse_launch(
            'rtspsrc location=rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x ! rtph264depay ! h264parse ! decodebin ! videoconvert ! videoscale ! '
            'video/x-raw,width=640,height=480,format=RGB ! tee name=t_raw '
            't_raw. ! queue leaky=2 max-size-buffers=2 ! videoscale ! video/x-raw,width=300,height=300 ! tensor_converter ! '
            'tensor_transform mode=arithmetic option=typecast:float32,add:-127.5,div:127.5 ! '
            'tensor_filter framework=tensorflow-lite model=' + self.tflite_model + ' ! '
            'tensor_decoder mode=bounding_boxes option1=mobilenet-ssd option2='
            + self.tflite_label + ' option3=' + self.tflite_box_prior + ' option4=640:480 option5=300:300 !'
            'compositor name=mix sink_0::zorder=2 sink_1::zorder=1 ! videoconvert ! ximagesink '
            't_raw. ! queue leaky=2 max-size-buffers=10 ! mix. '
        )

yolov5

gst-launch-1.0 v4l2src ! videoscale ! videoconvert ! video/x-raw,width=320,height=320,format=RGB,framerate=30/1,pixel-aspect-ratio=1/1 ! tee name=t   t. ! queue ! tensor_converter ! other/tensors,num_tensors=1,types=uint8,format=static,dimensions=3:320:320:1 !     tensor_transform mode=arithmetic option=typecast:float32,add:0.0,div:255.0 ! queue leaky=2 max-size-buffers=2 !     tensor_filter framework=tensorflow2-lite model=./yolov5s-fp16-320.tflite !     other/tensors,num_tensors=1,types=float32,format=static,dimensions=85:6300:1:1 !     tensor_decoder mode=bounding_boxes option1=yolov5 option2=./coco.txt option4=320:320 option5=320:320 !     video/x-raw,width=320,height=320,format=RGBA ! mix.sink_0   t. ! queue ! mix.sink_1 compositor name=mix sink_0::zorder=2 sink_1::zorder=1 ! videoconvert ! ximagesink

gst-launch-1.0 rtspsrc location=rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x ! rtph264depay ! h264parse ! decodebin ! videoconvert ! video/x-raw,width=320,height=320,format=RGB,framerate=30/1,pixel-aspect-ratio=1/1 ! tee name=t   t. ! queue ! tensor_converter ! other/tensors,num_tensors=1,types=uint8,format=static,dimensions=3:320:320:1 !     tensor_transform mode=arithmetic option=typecast:float32,add:0.0,div:255.0 ! queue leaky=2 max-size-buffers=2 !     tensor_filter framework=tensorflow2-lite model=./yolov5s-fp16-320.tflite !     other/tensors,num_tensors=1,types=float32,format=static,dimensions=85:6300:1:1 !     tensor_decoder mode=bounding_boxes option1=yolov5 option2=./coco.txt option4=320:320 option5=320:320 !     video/x-raw,width=320,height=320,format=RGBA ! mix.sink_0   t. ! queue ! mix.sink_1 compositor name=mix sink_0::zorder=2 sink_1::zorder=1 ! videoconvert ! ximagesink

使用opencv播放rtsp流

py

import cv2 as cv


cap = cv.VideoCapture('rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x',cv.CAP_GSTREAMER)
while cv.waitKey(1) < 0:
    hasFrame, frame = cap.read()

    if not hasFrame:
        break

    cv.imshow("rtsp", frame)

cpp

cv::VideoCapture cap;
cap.open("rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x",cv::CAP_GSTREAMER);


cv::Mat frame;

while(cv::waitKey(1) < 0) // Press any key to exit
{
    if (!cap.read(frame))
    {
        cerr << "No frames grabbed!\n";
        break;
    }
}
}

Ref

https://bbs.huaweicloud.com/blogs/315157

Q&A

python 使用gstreamer遇到的问题

No matching distribution found for gi
https://stackoverflow.com/questions/71369726/no-module-named-gi

sudo apt install libcairo2-dev libxt-dev libgirepository1.0-dev

pip install pycairo PyGObject

ffmpeg拉取rtsp流播放

ffmpeg -f v4l2 -i /dev/video0  -s 1280X720 -r 24 -vcodec libx264 -f rtsp rtsp://10.0.20.193:554/stream

ffplay -rtsp_transport tcp -fflags nobuffer rtsp://10.0.20.193/test

rk3588 使用gstreamer编解码，OpenEuler上

将硬编码相关的mpp 拷贝到设备上，https://gitee.com/wangyx0704/roc-rk3588s-pc-mppcodec
修改设备/dev/mpp_service 权限

# 下载相关的git 仓库
git clone https://gitee.com/wangyx0704/gst-d-softbus-sink.git
git clone https://gitee.com/wangyx0704/roc-rk3588s-pc-mppcodec.git

#安装相关的依赖
yum install opencv*  gcc-c++.aarch64  gcc.aarch64 -y
dnf install eigen3
dnf install -y gstreamer1 gstreamer1-devel gstreamer1-plugins-base gstreamer1-plugins-good gstreamer1-plugins-bad-free

dnf install -y gstreamer1* git make  cmake unzip pkg-config
rpm -e gstreamer1-plugins-base-help.noarch

#使用仓库中下载完的软件包，替换已经安装好的软件包
cd gst-d-softbus-sink/rpms
yum install -y gstreamer1-plugins-base-1.19.2/*.rpm
yum install -y gstreamer1-rtsp-server-1.19.2/*.rpm
将`roc-rk3588s-pc-mppcodec` 编解码库移动到系统目录`/usr/lib64`

常使用的pipeline的用法


# 解码
# 硬解
gst-launch-1.0 rtspsrc location="rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x" latency=0 drop-on-latency=true protocols=tcp ! rtph264depay ! h264parse ! mppvideodec ! video/x-raw,format=BGR ! appsink

# 软解
gst-launch-1.0 rtspsrc location="rtsp://admin:WANGfengtu12@10.0.20.61:554/client1x" latency=0 drop-on-latency=true protocols=tcp ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw,format=BGR ! appsink


# 编码
# 硬编
gst-launch-1.0 appsrc ! videoconvert ! mppvp8enc ! video/x-vp8, mapping=/stream1 ! rtspclientsink latency=0 location=rtsp://10.0.20.193:554/test

gst-launch-1.0 videotestsrc ! video/x-raw,width=1280,height=720 ! videoconvert ! mpph264enc ! video/x-h264, mapping=/stream1 ! rtspclientsink latency=0 location=rtsp://10.0.20.193:554/test

gst-launch-1.0 videotestsrc ! videoconvert ! mpph264enc ! rtspclientsink latency=0 location=rtsp://10.0.20.193:554/test

# 软编
gst-launch-1.0 videotestsrc ! videoconvert ! vp8enc ! rtspclientsink latency=0 location=rtsp://10.0.20.193:554/test

# 不能同时支持 软硬解码
gst-launch-1.0 rtspsrc location=rtsp://10.0.20.193/st1 latency=0 protocols=udp ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw, format=BGR ! videoconvert ! video/x-raw,format=YUY2,width=1280,height=720 ! mpph264enc ! rtspclientsink latency=100 location=rtsp://10.0.20.193:554/test

RK3588 Ubuntu使用

sudo apt install gstreamer1.0-rtsp
sudo apt install gstreamer1.0-rockchip1

OpenEuler 上面安装GStreamer

OE 22.03

sudo dnf install gstreamer*

# 目录 gstreamer1-rtsp-server-1.19.2
sudo rmp -Uvh *.rpm
# 直接远程安装
sudo dnf install gstreamer1-rtsp-server*

# rtspsrc
sudo dnf install gstreamer1-plugins-good*

# h264parse
sudo dnf install gstreamer1-plugins-bad*

# avdec_h264
sudo dnf install libgstlibav*

OE 20.03

# 卸载旧包
rpm -e gstreamer1-plugins-base-help.noarch

# 在下面目录依次执行
yum install *.rpm
# gstreamer1-plugins-base-1.19.2
#  gstreamer1-rtsp-server-1.19.2

rk3588 gstreamer pipeline 的使用

# 硬件编码
gst-launch-1.0 videotestsrc ! videoconvert ! mpph264enc ! rtspclientsink latency=0 location=rtsp://10.0.20.193:554/test

在这里插入图片描述

# 软件编码
gst-launch-1.0 videotestsrc ! videoconvert ! vp8enc ! rtspclientsink latency=0 location=rtsp://10.0.20.193:554/test

# 软解软编
gst-launch-1.0 rtspsrc location=rtsp://10.0.20.193/st1 latency=0 protocols=udp ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw, format=BGR ! videoconvert ! vp8enc ! rtspclientsink latency=100 location=rtsp://10.0.20.193:554/test

# 硬解硬编 （这种存在问题）
gst-launch-1.0 -v rtspsrc location=rtsp://10.0.20.193/st1 latency=0 ! rtph264depay ! h264parse ! mppvideodec ! videoconvert ! mpph264enc ! queue ! h264parse ! rtspclientsink location=rtsp://10.0.20.193:554/st2 latency=100

# 通过将mpp patch打上，重新编译内核 
export GST_VIDEO_CONVERT_USE_RGA=1
export GST_VIDEO_FLIP_USE_RGA=1
gst-launch-1.0 rtspsrc location=rtsp://10.0.20.193/input latency=0 protocols=udp ! queue ! rtph264depay ! h264parse ! mppvideodec ! videoconvert ! video/x-raw, format=BGR ! videoconvert ! video/x-raw,format=NV12 ! mpph264enc ! rtspclientsink latency=100 location=rtsp://10.0.20.193:554/test

Gstreamer server （完全不推荐）

1

#include <gst/gst.h>
#include <gst/rtsp-server/rtsp-server.h>

int main(int argc, char ** argv)
{
    GMainLoop *loop;
    GstRTSPMediaFactory *factory;

    char * pipe = "rtspsrc location=rtsp://10.0.20.193/st1 latency=0 protocols=udp ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw, format=BGR ! videoconvert ! video/x-raw,format=YUY2,width=1280,height=720 ! mpph264enc ! rtspclientsink latency=100 location=rtsp://10.0.20.193:554/test";
    loop = g_main_loop_new(NULL, FALSE);

    factory = gst_rtsp_media_factory_new();
    gst_rtsp_media_factory_set_launch(factory, pipe);
    gst_rtsp_media_factory_set_shared(factory, TRUE);
    gst_rtsp_media_factory_set_enable_rtcp(factory, FALSE);

    g_main_loop_run(loop);
    return 0;
}

编译选项

 # 错误
 gcc test_pipeline.c -o test_pipeline -lgstrtsp-1.0 `pkg-config --cflags --libs gstreamer-1.0` `pkg-config --cflags --libs gstreamer-plugins-base-1.0`
 # 正确
 gcc test_pipeline.c -o test_pipeline -lgstrtsp-1.0 `pkg-config --cflags --libs gstreamer-1.0` `pkg-config --cflags --libs gstreamer-rtsp-server-1.0`

2

#include <gst/gst.h>


int main(int argc, char **argv)
{
    GstElement *pipeline;
    GstBus * bus;
    GstMessage *msg;


    gst_init(&argc, &argv);

    pipeline = gst_parse_launch("rtspsrc location=rtsp://10.0.20.193/st1 latency=0 protocols=udp ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw, format=BGR ! videoconvert ! video/x-raw,format=YUY2,width=1280,height=720 ! mpph264enc ! rtspclientsink latency=100 location=rtsp://10.0.20.193:554/test", NULL);

    gst_element_set_state(pipeline, GST_STATE_PLAYING);



    bus = gst_element_get_bus(pipeline);

    msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);

    if(msg != NULL)
    {
        gst_message_unref(msg);
    }

    gst_object_unref(msg);

    gst_element_set_state(pipeline, GST_STATE_NULL);

    gst_object_unref(pipeline);
    return 0;
}

编译选项

gcc test_pipeline_2.c -o test_pipeline_2 `pkg-config --cflags --libs gstreamer-1.0` `pkg-config --cflags --libs gstreamer-rtsp-server-1.0`

使用ffmpeg 拉流保存到本地

ffmpeg -i rtsp://admin:WANGfengtu12@10.0.20.70:554/client0x  -vcodec copy out.h264

ffmpeg 推送本地流

ffmpeg -re  -stream_loop -1 -i D:\workspace\test.mp4  -vcodec copy -rtsp_transport tcp -f rtsp rtsp://192.168.3.176:554/s1

ffmpeg -re  -i rtsp://192.168.1.103/car/EdgeCar2 -vcodec copy -rtsp_transport tcp -f rtsp rtsp://192.168.1.109:554/s1

ffmpeg 拉流转发

ffmpeg -i rtsp://10.0.20.193/st1 -s 1080*720 -vcodec copy -f rtsp -rtsp_transport tcp rtsp://10.0.20.193:554/test
ffmpeg -i rtsp://10.0.20.193/st1 -acodec copy -vcodec copy -f rtsp -rtsp_transport tcp rtsp://10.0.20.193:554/test

推送到指定服务器

ffmpeg -re -stream_loop -1 -i test.mp4  -s 1280*720 -vcodec h264 -f rtsp -rtsp_transport tcp rtsp://10.0.20.221:554/test
ffmpeg -re -stream_loop -1 -i .\10daojishi.mp4 -vcodec copy -acodec copy -f rtsp rtsp://10.0.20.109:554/test

使用nvidia 硬编解码推流

ffmpeg -re -stream_loop -1 -i merge.mp4   -vcodec h264_nvenc -f rtsp -rtsp_transport udp rtsp://10.0.20.193:554/test

设置Gstreamer debug level

export GST_DEBUG=4

rtsp 流服务器

easy-darwin

ZLMediaKit

https://github.com/ZLMediaKit/ZLMediaKit/wiki/%E6%9C%8D%E5%8A%A1%E5%99%A8%E7%9A%84%E5%90%AF%E5%8A%A8%E4%B8%8E%E5%85%B3%E9%97%AD
https://github.com/ZLMediaKit/ZLMediaKit/wiki/ZLMediaKit%E6%8E%A8%E6%B5%81%E6%B5%8B%E8%AF%95

启动

cd /home/test/ZLMediaKit/release/linux/Debug
sudo ./MediaServer -d &

关闭

killall -2 MediaServer

推流

ffmpeg -re -i test.mp4 -vcodec h264 -acodec aac -f rtsp -rtsp_transport tcp rtsp://10.0.20.221/live/test

在这里插入图片描述

RTSP视频流相关的一些操作

播放rtsp camera 内容

Ref

NNStreamer

Env

使用opencv播放rtsp流

py

cpp

Ref

Q&A

python 使用gstreamer遇到的问题

ffmpeg拉取rtsp流播放

rk3588 使用gstreamer编解码，OpenEuler上

RK3588 Ubuntu使用

OpenEuler 上面安装GStreamer

OE 22.03

OE 20.03

rk3588 gstreamer pipeline 的使用

Gstreamer server （完全不推荐）

1

2

使用ffmpeg 拉流保存到本地

ffmpeg 推送本地流

ffmpeg 拉流转发

推送到指定服务器

使用nvidia 硬编解码 推流

设置Gstreamer debug level

rtsp 流服务器

easy-darwin

ZLMediaKit

相关文章

使用nvidia 硬编解码推流