技术背景
大牛直播SDK自2015年发布RTSP、RTMP直播播放模块，迭代从未停止，SmartPlayer功能强大、性能强劲、高稳定、超低延迟、超低资源占用。无需赘述，全自研内核，行业内一致认可的跨平台RTSP、RTMP直播播放器。本文以iOS平台为例，介绍下如何集成RTSP、RTMP播放模块。
技术对接
系统要求
- SDK支持iOS 9.0及以上版本；
- 支持的CPU架构：arm64（真机调试）。
准备工作
- 相关库：libSmartPlayerSDK.a
- 相关头文件：
- nt_common_media_define.h(如需转发或第三方数据对接)
- nt_event_define.h
- SmartPlayerSDK.h
- 如集需要引入的framework
- libbz.tbd
- Libbz2.tbd
- libiconv.tbd
- libstdc++.tbd
- Libc++.tbd
- Accelerate.framework
- AssetsLibrary.framework
- AudioToolBox.framework
- AVFoundation.framework
- CoreMedia.framework
- Foundation.framework
- GLKit.framework
- OpenGLES.framework
- UIKit.framework
- VideoToolBox.framework
- 如需集成到自己系统测试，请用大牛直播SDK的app name:
Info.plist-->右键Open As-->Source Code
添加或者编辑
<key>CFBundleName</key>
<string>SmartiOSPlayer</string>- 快照添加到“照片”权限:
Info.plist-->右键Open As-->Source Code 添加：
<key>NSPhotoLibraryUsageDescription</key>
<string>1</string>- 如需后台播放音频(添加后台播放权限)：
功能支持
iOS端，RTMP|RTSP直播播放，我们设计实现的功能如下：
- 音频：AAC/PCMA/PCMU/SPEEX(RTMP)；
- 视频：H.264；
- 播放协议：RTMP或RTSP；
- 支持纯音频、纯视频、音视频播放；
- 支持多实例播放；
- 支持网络状态、buffer状态等回调；
- [RTSP协议]支持RTSP TCP/UDP模式设置；
- [RTSP协议]支持RTSP TCP、UDP模式自动切换；
- [RTSP协议]支持RTSP超时时间设置，单位：秒；
- [RTSP协议]支持上报RTSP 401事件，如URL携带鉴权信息，会自动处理；
- 支持buffer time设置；
- 支持实时静音、取消静音；
- 支持首屏秒开功能(需服务器缓存GOP)；
- 支持超低延迟模式； 断网自动重连，支持视频追赶；
- 支持视频view实时旋转（0° 90° 180° 270°）；
- 支持视频view水平反转、垂直反转；
- 支持图像等比例缩放绘制；
- 支持实时快照；
- 支持实时音量调节；
- 支持YUV数据回调；
- 支持H.264|H.265数据回调；
- 支持AAC/SPEEX/PCMA/PCMU数据回调；
- 支持RTMP扩展H.265播放（Enhanced RTMP）；
- 支持扩展录像功能；
- 支持Unity3D接口;
- 支持H.264扩展SEI接收模块；
- 支持iOS 9.0及以上版本。
播放模块接口详解
| iOS播放端SDK接口详解 | |||
| 调用描述 | 接口 | 接口描述 | |
| 最先调用，创建播放实例，如成功返回player实例 | SmartPlayerInitPlayer | 初始化，创建player实例，此接口请第一个调用 | |
| Event回调 | SmartPlayerDelegate | 设置event callback，上层由handleSmartPlayerEvent处理 | |
| 软、硬解码设置 | SmartPlayerSetVideoDecoderMode | 设置是否用硬解码播放，如硬解码不支持，自动适配到软解码 0: 软解码; 1: 硬解码. | |
| 创建播放view | SmartPlayerCreatePlayView | x y width height 指定播放位置 | |
| 设置播放view | SmartPlayerSetPlayView | 设置播放view到底层SDK | |
| 释放播放view | SmartPlayeReleasePlayView | 释放播放view | |
| 视频回调 | 设置YUV回调 | SmartPlayerSetYuvBlock | 设置拉流时，视频YUV数据回调 |
| YUV回调 | PlayerYuvDataBlock | 提供解码后YUV/RGB数据接口，供用户自己render或进一步处理(如视频分析) | |
| 播放模式 | 缓冲时间设置 | SmartPlayerSetBuffer | 设置播放端缓存数据buffer，单位：毫秒，如不需buffer，设置为0 |
| 首屏秒开 | SmartPlayerSetFastStartup | 设置快速启动后，如果CDN缓存GOP，实现首屏秒开 | |
| 低延迟模式 | SmartPlayerSetLowLatencyMode | 针对类似于直播娃娃机等期待超低延迟的使用场景，超低延迟播放模式下，延迟可达到200~400ms | |
| 快速切换URL | SmartPlayerSwitchPlaybackUrl | 快速切换播放url，快速切换时，只换播放source部分，适用于不同数据流之间，快速切换 | |
| RTSP TCP/UDP模式设置 | SmartPlayerSetRTSPTcpMode | 设置RTSP TCP/UDP模式，如不设置，默认UDP模式 | |
| RTSP超时时间设置 | SmartPlayerSetRTSPTimeout | 设置RTSP超时时间，timeout单位为秒，必须大于0 | |
| 设置RTSP TCP/UDP自动切换 | SmartPlayerSetRTSPAutoSwitchTcpUdp | 对于RTSP来说，有些可能支持rtp over udp方式，有些可能支持使用rtp over tcp方式 为了方便使用，有些场景下可以开启自动尝试切换开关, 打开后如果udp无法播放，sdk会自动尝试tcp, 如果tcp方式播放不了,sdk会自动尝试udp. | |
| 实时静音 | SmartPlayerSetMute | 实时静音 | |
| 设置播放音量 | SmartPlayerSetAudioVolume | 播放端音量实时调节，范围[0,100]，0时为静音，100为原始流数据最大音量 | |
| 视频镜像旋转 | 旋转 | SmartPlayerSetRotation | 设置顺时针旋转, 注意除了0度之外， 其他角度都会额外消耗性能，当前支持 0度，90度, 180度, 270度 旋转 |
| 水平反转 | SmartPlayerSetFlipHorizontal | 设置视频水平反转 | |
| 垂直反转 | SmartPlayerSetFlipVertical | 设置视频垂直反转 | |
| 设置URL | SmartPlayerSetPlayURL | 设置播放或录像的url | |
| 开始播放 | SmartPlayerStart | 开始播放RTSP/RTMP流 | |
| 停止播放 | SmartPlayerStop | 停止播放RTSP/RTMP流 | |
| 销毁播放实例 | SmartPlayerUnInitPlayer | 结束时必须调用close接口释放资源 | |
录像模块接口详解
如需录像，录像相关的接口如下：
| iOS播放端录像SDK接口详解 | |||
| 调用描述 | 接口 | 接口描述 | |
| 录像设置 | 设置录像目录 | SmartPlayerSetRecorderDirectory | 设置录像文件目录 |
| 设置录像文件大小 | SmartPlayerSetRecorderFileMaxSize | 设置每个录像文件的大小，比如100M，超过这个大小后，会自动生成下一个录像文件 | |
| 音频转码 | SmartPlayerSetRecorderAudioTranscodeAAC | 设置录像时音频转AAC编码的开关 aac比较通用，sdk增加其他音频编码(比如speex, pcmu, pcma等)转aac的功能. | |
| 录制视频 | SmartPlayerSetRecorderVideo | 设置是否录视频，默认的话，如果视频源有视频就录，没有就不录, 但有些场景下可能不想录制视频，只想录音频，所以增加个开关 | |
| 录制音频 | SmartPlayerSetRecorderAudio | 设置是否录音频，默认的话，如果视频源有音频就录，没有就不录, 但有些场景下可能不想录制音频，只想录视频，所以增加个开关 | |
| 开始录像 | SmartPlayerStartRecorder | 开始录像 | |
| 停止录像 | SmartPlayerStopRecorder | 停止录像 | |
Event回调详解
由于iOS播放录像SDK和播放端SDK可组合使用，相关Event同步更新在iOS播放端SDK(如下图)：
| iOS播放端SDK Event回调说明 | |
| 事件ID | 事件描述 |
| EVENT_DANIULIVE_ERC_PLAYER_STARTED | 开始播放 |
| EVENT_DANIULIVE_ERC_PLAYER_CONNECTING | 播放端连接中 |
| EVENT_DANIULIVE_ERC_PLAYER_CONNECTION_FAILED | 播放端连接失败 |
| EVENT_DANIULIVE_ERC_PLAYER_CONNECTED | 播放端连接成功 |
| EVENT_DANIULIVE_ERC_PLAYER_DISCONNECTED | 播放端连接断开 |
| EVENT_DANIULIVE_ERC_PLAYER_STOP | 停止播放 |
| EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO | 返回视频宽、高信息 |
| EVENT_DANIULIVE_ERC_PLAYER_NO_MEDIADATA_RECEIVED | 收不到媒体数据(可能是URL错误) |
| EVENT_DANIULIVE_ERC_PLAYER_SWITCH_URL | 快速切换URL |
| EVENT_DANIULIVE_ERC_PLAYER_RECORDER_START_NEW_FILE | 开始一个新的录像文件(param3返回包含录像路径在内的录像文件名) |
| EVENT_DANIULIVE_ERC_PLAYER_ONE_RECORDER_FILE_FINISHED | 已生成一个录像文件(param3返回包含录像路径在内的录像文件名) |
| EVENT_DANIULIVE_ERC_PLAYER_CAPTURE_IMAGE | 播放端实时快照 |
| EVENT_DANIULIVE_ERC_PLAYER_START_BUFFERING | 开始缓冲数据 |
| EVENT_DANIULIVE_ERC_PLAYER_BUFFERING | 缓冲中(param1参数 会返回缓冲百分比) |
| EVENT_DANIULIVE_ERC_PLAYER_STOP_BUFFERING | 停止缓冲数据 |
| EVENT_DANIULIVE_ERC_PLAYER_DOWNLOAD_SPEED | 返回当前RTSP/RTMP流实时下载速度 |
| EVENT_DANIULIVE_ERC_PLAYER_RTSP_STATUS_CODE | RTSP收到错误码，可能是用户名、密码不对 |
逻辑调用
先说开始播放：
//
// ViewController.m
// SmartiOSPlayerV2
//
// Author: daniusdk.com
// WeChat: xinsheng120
// Created by daniulive on 2016/01/03.
//
- (void)playBtn:(UIButton *)button {
NSLog(@"playBtn only++");
button.selected = !button.selected;
if (button.selected)
{
if(is_playing_)
return;
[self InitPlayer];
//如需处理回调的用户数据+++++++++
__weak __typeof(self) weakSelf = self;
_smart_player_sdk.spUserDataCallBack = ^(int data_type, unsigned char *data, unsigned int size, unsigned long long timestamp, unsigned long long reserve1, long long reserve2, unsigned char *reserve3)
{
[weakSelf OnUserDataCallBack:data_type data:data size:size timestamp:timestamp reserve1:reserve1 reserve2:reserve2 reserve3:reserve3];
};
Boolean enableUserDataCallback = YES;
[_smart_player_sdk SmartPlayerSetUserDataCallback:enableUserDataCallback];
//如需处理回调的用户数据---------
if(![self StartPlayer])
{
NSLog(@"Call StartPlayer failed..");
}
[playbackButton setTitle:@"停止播放" forState:UIControlStateNormal];
is_playing_ = YES;
}
else
{
if ( !is_playing_ )
return;
[self StopPlayer];
if(!is_recording_)
{
[self UnInitPlayer];
}
[playbackButton setTitle:@"开始播放" forState:UIControlStateNormal];
is_mute_ = NO;
[muteButton setTitle:@"实时静音" forState:UIControlStateNormal];
is_playing_ = NO;
}
}其中，InitPlayer实现如下：
-(bool)InitPlayer
{
NSLog(@"InitPlayer++");
if(is_inited_player_)
{
NSLog(@"InitPlayer: has inited before..");
return true;
}
//NSString* in_cid = @"";
//NSString* in_key = @"";
//[SmartPlayerSDK SmartPlayerSetSDKClientKey:in_cid in_key:in_key reserve1:0 reserve2:nil];
_smart_player_sdk = [[SmartPlayerSDK alloc] init];
if (_smart_player_sdk ==nil ) {
NSLog(@"SmartPlayerSDK init failed..");
return false;
}
if (playback_url_.length == 0) {
NSLog(@"playback url is nil..");
return false;
}
if (_smart_player_sdk.delegate == nil)
{
_smart_player_sdk.delegate = self;
NSLog(@"SmartPlayerSDK _player.delegate:%@", _smart_player_sdk);
}
NSInteger initRet = [_smart_player_sdk SmartPlayerInitPlayer];
if ( initRet != DANIULIVE_RETURN_OK )
{
NSLog(@"SmartPlayerSDK call SmartPlayerInitPlayer failed, ret=%ld", (long)initRet);
return false;
}
[_smart_player_sdk SmartPlayerSetPlayURL:playback_url_];
//[self try_set_rtsp_url:playback_url_];
//超低延迟模式设置
[_smart_player_sdk SmartPlayerSetLowLatencyMode:(NSInteger)is_low_latency_mode_];
//buffer time设置
if(buffer_time_ >= 0)
{
[_smart_player_sdk SmartPlayerSetBuffer:buffer_time_];
}
//快速启动模式设置
[_smart_player_sdk SmartPlayerSetFastStartup:(NSInteger)is_fast_startup_];
NSLog(@"[SmartPlayerV2]is_fast_startup_:%d, buffer_time_:%ld", is_fast_startup_, (long)buffer_time_);
//RTSP TCP还是UDP模式
[_smart_player_sdk SmartPlayerSetRTSPTcpMode:is_rtsp_tcp_mode_];
//设置RTSP超时时间
NSInteger rtsp_timeout = 10;
[_smart_player_sdk SmartPlayerSetRTSPTimeout:rtsp_timeout];
//设置RTSP TCP/UDP自动切换
NSInteger is_tcp_udp_auto_switch = 1;
[_smart_player_sdk SmartPlayerSetRTSPAutoSwitchTcpUdp:is_tcp_udp_auto_switch];
//快照设置 如需快照 参数传1
[_smart_player_sdk SmartPlayerSaveImageFlag:save_image_flag_];
//如需查看实时流量信息，可打开以下接口
NSInteger is_report = 1;
NSInteger report_interval = 3;
[_smart_player_sdk SmartPlayerSetReportDownloadSpeed:is_report report_interval:report_interval];
//录像端音频，是否转AAC后保存
NSInteger is_transcode = 1;
[_smart_player_sdk SmartPlayerSetRecorderAudioTranscodeAAC:is_transcode];
//录制MP4文件 是否录制视频
NSInteger is_record_video = 1;
[_smart_player_sdk SmartPlayerSetRecorderVideo:is_record_video];
//录制MP4文件 是否录制音频
NSInteger is_record_audio = 1;
[_smart_player_sdk SmartPlayerSetRecorderAudio:is_record_audio];
is_inited_player_ = YES;
NSLog(@"InitPlayer--");
return true;
}
停止播放StopPlayer实现如下：
-(bool)StopPlayer
{
NSLog(@"StopPlayer++");
if (_smart_player_sdk != nil)
{
[_smart_player_sdk SmartPlayerStop];
}
if (!is_audio_only_) {
if (_glView != nil) {
[_glView removeFromSuperview];
[SmartPlayerSDK SmartPlayeReleasePlayView:(__bridge void *)(_glView)];
_glView = nil;
}
}
NSLog(@"StopPlayer--");
return true;
}UnInitPlayer实现如下：
-(bool)UnInitPlayer
{
NSLog(@"UnInitPlayer++");
if (_smart_player_sdk != nil)
{
[_smart_player_sdk SmartPlayerUnInitPlayer];
if (_smart_player_sdk.delegate != nil)
{
_smart_player_sdk.delegate = nil;
}
_smart_player_sdk = nil;
}
is_inited_player_ = NO;
NSLog(@"UnInitPlayer--");
return true;
}实时录像：
- (void)RecorderBtn:(UIButton *)button {
NSLog(@"record Stream only++");
button.selected = !button.selected;
if (button.selected)
{
if(is_recording_)
return;
[self InitPlayer];
//设置录像目录
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *recorderDir = [paths objectAtIndex:0];
if([_smart_player_sdk SmartPlayerSetRecorderDirectory:recorderDir] != DANIULIVE_RETURN_OK)
{
NSLog(@"Call SmartPlayerSetRecorderDirectory failed..");
}
//每个录像文件大小
NSInteger size = 200;
if([_smart_player_sdk SmartPlayerSetRecorderFileMaxSize:size] != DANIULIVE_RETURN_OK)
{
NSLog(@"Call SmartPlayerSetRecorderFileMaxSize failed..");
}
[_smart_player_sdk SmartPlayerStartRecorder];
[recButton setTitle:@"停止录像" forState:UIControlStateNormal];
is_recording_ = YES;
}
else
{
[_smart_player_sdk SmartPlayerStopRecorder];
[recButton setTitle:@"开始录像" forState:UIControlStateNormal];
if(!is_playing_)
{
[self UnInitPlayer];
}
is_recording_ = NO;
}
}实时快照：
- (void)SaveImageBtn:(UIButton *)button {
if ( _smart_player_sdk != nil )
{
//设置快照目录
NSLog(@"[SaveImageBtn] path++");
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *saveImageDir = [paths objectAtIndex:0];
NSLog(@"[SaveImageBtn] path: %@", saveImageDir);
NSString* symbol = @"/";
NSString* png = @".png";
// 1.创建时间
NSDate *datenow = [NSDate date];
// 2.创建时间格式化
NSDateFormatter *formatter = [[NSDateFormatter alloc] init];
// 3.指定格式
formatter.dateFormat = @"yyyyMMdd_HHmmss";
// 4.格式化时间
NSString *timeSp = [formatter stringFromDate:datenow];
NSString* image_name = [saveImageDir stringByAppendingString:symbol];
image_name = [image_name stringByAppendingString:timeSp];
image_name = [image_name stringByAppendingString:png];
NSLog(@"[SaveImageBtn] image_name: %@", image_name);
[_smart_player_sdk SmartPlayerSaveCurImage:image_name];
}
}Event回调处理如下：
- (NSInteger) handleSmartPlayerEvent:(NSInteger)nID param1:(unsigned long long)param1 param2:(unsigned long long)param2 param3:(NSString*)param3 param4:(NSString*)param4 pObj:(void *)pObj;
{
NSString* player_event = @"";
NSString* lable = @"";
if (nID == EVENT_DANIULIVE_ERC_PLAYER_STARTED) {
player_event = @"[event]开始播放..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTING)
{
player_event = @"[event]连接中..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTION_FAILED)
{
player_event = @"[event]连接失败..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CONNECTED)
{
player_event = @"[event]已连接..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_DISCONNECTED)
{
player_event = @"[event]断开连接..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_STOP)
{
player_event = @"[event]停止播放..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_RESOLUTION_INFO)
{
NSString *str_w = [NSString stringWithFormat:@"%ld", (long)param1];
NSString *str_h = [NSString stringWithFormat:@"%ld", (long)param2];
lable = @"[event]视频解码分辨率信息: ";
player_event = [lable stringByAppendingFormat:@"%@*%@", str_w, str_h];
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_NO_MEDIADATA_RECEIVED)
{
player_event = @"[event]收不到RTMP数据..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_SWITCH_URL)
{
player_event = @"[event]快速切换url..";
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_CAPTURE_IMAGE)
{
if ((int)param1 == 0)
{
NSLog(@"[event]快照成功: %@", param3);
lable = @"[event]快照成功:";
player_event = [lable stringByAppendingFormat:@"%@", param3];
tmp_path_ = param3;
image_path_ = [ UIImage imageNamed:param3];
UIImageWriteToSavedPhotosAlbum(image_path_, self, @selector(image:didFinishSavingWithError:contextInfo:), NULL);
}
else
{
lable = @"[event]快照失败";
player_event = [lable stringByAppendingFormat:@"%@", param3];
}
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_RECORDER_START_NEW_FILE)
{
lable = @"[event]录像写入新文件..文件名:";
player_event = [lable stringByAppendingFormat:@"%@", param3];
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_ONE_RECORDER_FILE_FINISHED)
{
lable = @"一个录像文件完成..文件名:";
player_event = [lable stringByAppendingFormat:@"%@", param3];
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_START_BUFFERING)
{
//NSLog(@"[event]开始buffer..");
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_BUFFERING)
{
NSLog(@"[event]buffer百分比: %lld", param1);
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_STOP_BUFFERING)
{
//NSLog(@"[event]停止buffer..");
}
else if (nID == EVENT_DANIULIVE_ERC_PLAYER_DOWNLOAD_SPEED)
{
NSInteger speed_kbps = (NSInteger)param1*8/1000;
NSInteger speed_KBs = (NSInteger)param1/1024;
lable = @"[event]download speed :";
player_event = [lable stringByAppendingFormat:@"%ld kbps - %ld KB/s", (long)speed_kbps, (long)speed_KBs];
}
else if(nID == EVENT_DANIULIVE_ERC_PLAYER_RTSP_STATUS_CODE)
{
lable = @"[event]RTSP status code received:";
player_event = [lable stringByAppendingFormat:@"%ld", (long)param1];
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
dispatch_async(dispatch_get_main_queue(), ^{
UIAlertController *aleView=[UIAlertController alertControllerWithTitle:@"RTSP错误状态" message:player_event preferredStyle:UIAlertControllerStyleAlert];
UIAlertAction *action_ok=[UIAlertAction actionWithTitle:@"确定" style:UIAlertActionStyleCancel handler:nil];
[aleView addAction:action_ok];
[self presentViewController:aleView animated:YES completion:nil];
});
});
}
else if(nID == EVENT_DANIULIVE_ERC_PLAYER_NEED_KEY)
{
player_event = @"[event]RTMP加密流，请设置播放需要的Key..";
}
else if(nID == EVENT_DANIULIVE_ERC_PLAYER_KEY_ERROR)
{
player_event = @"[event]RTMP加密流，Key错误，请重新设置..";
}
else
NSLog(@"[event]nID:%lx", (long)nID);
NSString* player_event_tag = @"当前状态:";
NSString* event = [player_event_tag stringByAppendingFormat:@"%@", player_event];
if ( player_event.length != 0)
{
NSLog(@"%@", event);
}
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
dispatch_async(dispatch_get_main_queue(), ^{
self.textPlayerEventLabel.text = event;
});
});
return 0;
}总结
iOS平台RTSP、RTMP直播播放模块，延迟低、资源占有少，性能优异。由于设备和系统比较单一，优先考虑硬解码，除了基础播放外，我们还实现了实时快照、实时录像、实时回调YUV数据、实时音量调节等，实际体验下来，iOS平台RTMP和RTSP，可以轻松毫秒级。
