WebRTC 系列（三、点对点通话，H5、Android、iOS）
上一篇博客中，我们已经实现了点对点通话，即一对一通话，这一次就接着实现多人通话。多人通话的实现方式呢也有好几种方案，这里我简单介绍两种方案。
一、多人通话方案
1.Mesh
多个客户端之间建立多个 PeerConnection，即如果有三个客户端 A、B、C，A 有两个 PeerConnection 分别与 B、C 通信，B 也是有两个 PeerConnection，分别与 A、C 通信，C 也是有两个 PeerConnection，分别与 A、B 通信，如图：
优点：服务端压力小，不需要对音视频数据做处理。
缺点：客户端编解码压力较大，传输的数据与通话人数成正比，兼容性较差。
2.Mixer
客户端只与服务器有一个 PeerConnection，有多个客户端时，服务端增加多个媒体流，由服务端来做媒体数据转发，如图：
优点：客户端只有一个连接，传输数据减少，服务端可对音视频数据预处理，兼容性好。
缺点：服务器压力大，通话人数过多时，服务器如果对音视频数据有预处理，可能导致通话延迟。
3.demo 方案选择
两种方案各有利弊，感觉在实际业务中，第二种方案更合适，毕竟把更多逻辑放在服务端更可控一点，我为了演示简单，就选用了第一种方案，下面就说说第一种方案的话，第一个人、第二个人、第三个人加入房间的流程是什么样的。
第一个人 A 加入房间：
- A 发送 join；
- 服务器向房间内其他所有人发送 otherJoin；
- 房间内没有其他人，结束。
第二个人 B 加入房间：
- B 发送 join；
- 服务器向房间内其他所有人发送 otherJoin；
- A 收到 otherJoin（带有 B 的 userId）；
- A 检查远端视频渲染控件集合中是否存在 B 的 userId 对应的远端视频渲染控件，没有则创建并初始化，然后保存；
- A 检查 PeerConnection 集合中是否存在 B 的 userId 对应的 PeerConnection，没有则创建并添加音轨、视轨，然后保存；
- A 通过 PeerConnection 创建 offer，获取 sdp；
- A 将 offer sdp 作为参数 setLocalDescription；
- A 发送 offer sdp（带有 A 的 userId）；
- B 收到 offer（带有 A 的 userId）；
- B 检查远端视频渲染控件集合中是否存在 A 的 userId 对应远端视频渲染控件，没有则创建并初始化，然后保存；
- B 检查 PeerConnection 集合中是否存在 A 的 userId 对应的 PeerConnection，没有则创建并添加音轨、视轨，然后保存；
- B 将 offer sdp 作为参数 setRemoteDescription；
- B 通过 PeerConnection 创建 answer，获取 sdp；
- B 将 answer sdp 作为参数 setLocalDescription；
- B 发送 answer sdp（带有 B 的 userId）；
- A 收到 answer sdp（带有 B 的 userId）；
- A 通过 userId 找到对应 PeerConnection，将 answer sdp 作为参数 setRemoteDescription。
第三个人 C 加入房间：
- C 发送 join；
- 服务器向房间内其他所有人发送 otherJoin；
- A 收到 otherJoin（带有 C 的 userId）；
- A 检查远端视频渲染控件集合中是否存在 C 的 userId 对应的远端视频渲染控件，没有则创建并初始化，然后保存；
- A 检查 PeerConnection 集合中是否存在 C 的 userId 对应的 PeerConnection，没有则创建并添加音轨、视轨，然后保存；
- A 通过 PeerConnection 创建 offer，获取 sdp；
- A 将 offer sdp 作为参数 setLocalDescription；
- A 发送 offer sdp（带有 A 的 userId）；
- C 收到 offer（带有 A 的 userId）；
- C 检查远端视频渲染控件集合中是否存在 A 的 userId 对应远端视频渲染控件，没有则创建并初始化，然后保存；
- C 检查 PeerConnection 集合中是否存在 A 的 userId 对应的 PeerConnection，没有则创建并添加音轨、视轨，然后保存；
- C 将 offer sdp 作为参数 setRemoteDescription；
- C 通过 PeerConnection 创建 answer，获取 sdp；
- C 将 answer sdp 作为参数 setLocalDescription；
- C 发送 answer sdp（带有 C 的 userId）；
- A 收到 answer sdp（带有 C 的 userId）；
- A 通过 userId 找到对应 PeerConnection，将 answer sdp 作为参数 setRemoteDescription。
- B 收到 otherJoin（带有 C 的 userId）；
- B 检查远端视频渲染控件集合中是否存在 C 的 userId 对应的远端视频渲染控件，没有则创建并初始化，然后保存；
- B 检查 PeerConnection 集合中是否存在 C 的 userId 对应的 PeerConnection，没有则创建并添加音轨、视轨，然后保存；
- B 通过 PeerConnection 创建 offer，获取 sdp；
- B 将 offer sdp 作为参数 setLocalDescription；
- B 发送 offer sdp（带有 B 的 userId）；
- C 收到 offer（带有 B 的 userId）；
- C 检查远端视频渲染控件集合中是否存在 B 的 userId 对应远端视频渲染控件，没有则创建并初始化，然后保存；
- C 检查 PeerConnection 集合中是否存在 B 的 userId 对应的 PeerConnection，没有则创建并添加音轨、视轨，然后保存；
- C 将 offer sdp 作为参数 setRemoteDescription；
- C 通过 PeerConnection 创建 answer，获取 sdp；
- C 将 answer sdp 作为参数 setLocalDescription；
- C 发送 answer sdp（带有 C 的 userId）；
- B 收到 answer sdp（带有 C 的 userId）；
- B 通过 userId 找到对应 PeerConnection，将 answer sdp 作为参数 setRemoteDescription。
依此类推，如果还有第四个用户 D 再加入房间的话，D 也会发送 join，然后 A、B、C 也会类似上述 3～17 步处理。
这期间的 onIceCandidate 回调的处理和之前类似，只是将生成的 IceCandidate 对象传递给对方时需要带上发送方（自己）的 userId，便于对方找到对应的 PeerConnection，以及接收方的 userId，便于服务器找到接收方的长连接。
这期间的 onAddStream 回调的处理也和之前类似，只是需要通过对方的 userId 找到对应的远端控件渲染控件。
二、信令服务器
信令服务器的依赖就不重复了，根据上述流程，我们需要引入用户的概念，但暂时我没有引入房间的概念，所以在测试的时候我认为只有一个房间，所有人都加入的同一个房间。
多人通话 WebSocket 服务端代码：
package com.qinshou.webrtcdemo_server;
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import com.google.gson.reflect.TypeToken;
import org.java_websocket.WebSocket;
import org.java_websocket.handshake.ClientHandshake;
import org.java_websocket.server.WebSocketServer;
import java.net.InetSocketAddress;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
/**
* Author: MrQinshou
* Email: cqflqinhao@126.com
* Date: 2023/2/8 9:33
* Description: 多人通话 WebSocketServer
*/
public class MultipleWebSocketServerHelper {
public static class WebSocketBean {
private String mUserId;
private WebSocket mWebSocket;
public WebSocketBean() {
}
public WebSocketBean(WebSocket webSocket) {
mWebSocket = webSocket;
}
public String getUserId() {
return mUserId;
}
public void setUserId(String userId) {
mUserId = userId;
}
public WebSocket getWebSocket() {
return mWebSocket;
}
public void setWebSocket(WebSocket webSocket) {
mWebSocket = webSocket;
}
}
private WebSocketServer mWebSocketServer;
private final List<WebSocketBean> mWebSocketBeans = new LinkedList<>();
// private static final String HOST_NAME = "192.168.1.104";
private static final String HOST_NAME = "172.16.2.172";
private static final int PORT = 8888;
private WebSocketBean getWebSocketBeanByWebSocket(WebSocket webSocket) {
for (WebSocketBean webSocketBean : mWebSocketBeans) {
if (webSocket == webSocketBean.getWebSocket()) {
return webSocketBean;
}
}
return null;
}
private WebSocketBean getWebSocketBeanByUserId(String userId) {
for (WebSocketBean webSocketBean : mWebSocketBeans) {
if (userId.equals(webSocketBean.getUserId())) {
return webSocketBean;
}
}
return null;
}
private WebSocketBean removeWebSocketBeanByWebSocket(WebSocket webSocket) {
for (WebSocketBean webSocketBean : mWebSocketBeans) {
if (webSocket == webSocketBean.getWebSocket()) {
mWebSocketBeans.remove(webSocketBean);
return webSocketBean;
}
}
return null;
}
public void start() {
InetSocketAddress inetSocketAddress = new InetSocketAddress(HOST_NAME, PORT);
mWebSocketServer = new WebSocketServer(inetSocketAddress) {
@Override
public void onOpen(WebSocket conn, ClientHandshake handshake) {
System.out.println("onOpen--->" + conn);
// 有客户端连接，创建 WebSocketBean，此时仅保存了 WebSocket 连接，但还没有和 userId 绑定
mWebSocketBeans.add(new WebSocketBean(conn));
}
@Override
public void onClose(WebSocket conn, int code, String reason, boolean remote) {
System.out.println("onClose--->" + conn);
WebSocketBean webSocketBean = removeWebSocketBeanByWebSocket(conn);
if (webSocketBean == null) {
return;
}
// 通知其他用户有人退出房间
JsonObject jsonObject = new JsonObject();
jsonObject.addProperty("msgType", "otherQuit");
jsonObject.addProperty("userId", webSocketBean.mUserId);
for (WebSocketBean w : mWebSocketBeans) {
if (w != webSocketBean) {
w.mWebSocket.send(jsonObject.toString());
}
}
}
@Override
public void onMessage(WebSocket conn, String message) {
System.out.println("onMessage--->" + message);
Map<String, String> map = new Gson().fromJson(message, new TypeToken<Map<String, String>>() {
}.getType());
String msgType = map.get("msgType");
if ("join".equals(msgType)) {
// 收到加入房间指令
String userId = map.get("userId");
WebSocketBean webSocketBean = getWebSocketBeanByWebSocket(conn);
// WebSocket 连接绑定 userId
if (webSocketBean != null) {
webSocketBean.setUserId(userId);
}
// 通知其他用户有其他人加入房间
JsonObject jsonObject = new JsonObject();
jsonObject.addProperty("msgType", "otherJoin");
jsonObject.addProperty("userId", userId);
for (WebSocketBean w : mWebSocketBeans) {
if (w != webSocketBean && w.getUserId() != null) {
w.mWebSocket.send(jsonObject.toString());
}
}
return;
}
if ("quit".equals(msgType)) {
// 收到退出房间指令
String userId = map.get("userId");
WebSocketBean webSocketBean = getWebSocketBeanByWebSocket(conn);
// WebSocket 连接解绑 userId
if (webSocketBean != null) {
webSocketBean.setUserId(null);
}
// 通知其他用户有其他人退出房间
JsonObject jsonObject = new JsonObject();
jsonObject.addProperty("msgType", "otherQuit");
jsonObject.addProperty("userId", userId);
for (WebSocketBean w : mWebSocketBeans) {
if (w != webSocketBean && w.getUserId() != null) {
w.mWebSocket.send(jsonObject.toString());
}
}
return;
}
// 其他消息透传
// 接收方
String toUserId = map.get("toUserId");
// 找到接收方对应 WebSocket 连接
WebSocketBean webSocketBean = getWebSocketBeanByUserId(toUserId);
if (webSocketBean != null) {
webSocketBean.getWebSocket().send(message);
}
}
@Override
public void onError(WebSocket conn, Exception ex) {
ex.printStackTrace();
System.out.println("onError");
}
@Override
public void onStart() {
System.out.println("onStart");
}
};
mWebSocketServer.start();
}
public void stop() {
if (mWebSocketServer == null) {
return;
}
for (WebSocket webSocket : mWebSocketServer.getConnections()) {
webSocket.close();
}
try {
mWebSocketServer.stop();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
mWebSocketServer = null;
}
public static void main(String[] args) {
new MultipleWebSocketServerHelper().start();
}
}
三、消息格式
传递的消息的话，相较于点对点通话，sdp 和 iceCandidate 中需要添加 fromUserId 和 toUserId 字段，另外还需要增加 join、otherJoin、quit、ohterQuit 消息：
// sdp
{
"msgType": "sdp",
"fromUserId": userId,
"toUserId": toUserId,
"type": sessionDescription.type,
"sdp": sessionDescription.sdp
}
// iceCandidate
{
"msgType": "iceCandidate",
"fromUserId": userId,
"toUserId": toUserId,
"id": iceCandidate.sdpMid,
"label": iceCandidate.sdpMLineIndex,
"candidate": iceCandidate.candidate
}
// join
{
"msgType": "join"
"userId": userId
}
// otherJoin
{
"msgType": "otherJoin"
"userId": userId
}
// quit
{
"msgType": "quit"
"userId": userId
}
// otherQuit
{
"msgType": "otherQuit"
"userId": userId
}
四、H5
代码与 p2p_demo 其实差不了太多，但是我们创建 PeerConnection 的时机需要根据上面梳理流程进行修改，发送的信令也需要根据上面定义的格式进行修改，布局中将远端视频渲染控件去掉，改成一个远端视频渲染控件的容器，每当有新的连接时创建新的远端视频渲染控件放到容器中，另外，WebSocket 需要额外处理 otherJoin 和 otherQuit 信令。
1.添加依赖
这个跟前两篇的一样，不需要额外引入。
2.multiple_demo.html
<html>
<head>
<title>Multiple Demo</title>
<style>
body {
overflow: hidden;
margin: 0px;
padding: 0px;
}
#local_view {
width: 100%;
height: 100%;
}
#remote_views {
width: 9%;
height: 80%;
position: absolute;
top: 10%;
right: 10%;
bottom: 10%;
overflow-y: auto;
}
.remote_view {
width: 100%;
aspect-ratio: 9/16;
}
#left {
width: 10%;
height: 5%;
position: absolute;
left: 10%;
top: 10%;
}
#p_websocket_state,
#input_server_url,
.my_button {
width: 100%;
height: 100%;
display: block;
margin-bottom: 10%;
}
</style>
</head>
<body>
<video id="local_view" width="480" height="270" autoplay controls muted></video>
<div id="remote_views">
</div>
<div id="left">
<p id="p_websocket_state">WebSocket 已断开</p>
<input id="input_server_url" type="text" placeholder="请输入服务器地址" value="ws://192.168.1.104:8888"></input>
<button id="btn_connect" class="my_button" onclick="connect()">连接 WebSocket</button>
<button id="btn_disconnect" class="my_button" onclick="disconnect()">断开 WebSocket</button>
<button id="btn_join" class="my_button" onclick="join()">加入房间</button>
<button id="btn_quit" class="my_button" onclick="quit()">退出房间</button>
</div>
</body>
<script type="text/javascript">
/**
* Author: MrQinshou
* Email: cqflqinhao@126.com
* Date: 2023/4/15 11:24
* Description: 生成 uuid
*/
function uuid() {
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function (c) {
var r = Math.random() * 16 | 0;
var v = c == 'x' ? r : (r & 0x3 | 0x8);
return v.toString(16);
});
}
</script>
<script type="text/javascript">
var localView = document.getElementById("local_view");
var remoteViews = document.getElementById("remote_views");
var localStream;
// let userId = uuid();
let userId = "h5";
let peerConnectionDict = {};
let remoteViewDict = {};
function createPeerConnection(fromUserId) {
let peerConnection = new RTCPeerConnection();
peerConnection.oniceconnectionstatechange = function (event) {
if ("disconnected" == event.target.iceConnectionState) {
let peerConnection = peerConnectionDict[fromUserId];
if (peerConnection != null) {
peerConnection.close();
delete peerConnectionDict[fromUserId];
}
let remoteView = remoteViewDict[fromUserId];
if (remoteView != null) {
remoteView.removeAttribute('src');
remoteView.load();
remoteView.remove();
delete remoteViewDict[fromUserId];
}
}
}
peerConnection.onicecandidate = function (event) {
console.log("onicecandidate--->" + event.candidate);
sendIceCandidate(event.candidate, fromUserId);
}
peerConnection.ontrack = function (event) {
console.log("remote ontrack--->" + event.streams);
let remoteView = remoteViewDict[fromUserId];
if (remoteView == null) {
return;
}
let streams = event.streams;
if (streams && streams.length > 0) {
remoteView.srcObject = streams[0];
}
}
return peerConnection;
}
function createOffer(peerConnection, fromUserId) {
peerConnection.createOffer().then(function (sessionDescription) {
console.log(fromUserId + " create offer success.");
peerConnection.setLocalDescription(sessionDescription).then(function () {
console.log(fromUserId + " set local sdp success.");
var jsonObject = {
"msgType": "sdp",
"fromUserId": userId,
"toUserId": fromUserId,
"type": "offer",
"sdp": sessionDescription.sdp
};
send(JSON.stringify(jsonObject));
}).catch(function (error) {
console.log("error--->" + error);
})
}).catch(function (error) {
console.log("error--->" + error);
})
}
function createAnswer(peerConnection, fromUserId) {
peerConnection.createAnswer().then(function (sessionDescription) {
console.log(fromUserId + " create answer success.");
peerConnection.setLocalDescription(sessionDescription).then(function () {
console.log(fromUserId + " set local sdp success.");
var jsonObject = {
"msgType": "sdp",
"fromUserId": userId,
"toUserId": fromUserId,
"type": "answer",
"sdp": sessionDescription.sdp
};
send(JSON.stringify(jsonObject));
}).catch(function (error) {
console.log("error--->" + error);
})
}).catch(function (error) {
console.log("error--->" + error);
})
}
function join() {
var jsonObject = {
"msgType": "join",
"userId": userId,
};
send(JSON.stringify(jsonObject));
}
function quit() {
var jsonObject = {
"msgType": "quit",
"userId": userId,
};
send(JSON.stringify(jsonObject));
for (var key in peerConnectionDict) {
let peerConnection = peerConnectionDict[key];
peerConnection.close();
delete peerConnectionDict[key];
}
for (var key in remoteViewDict) {
let remoteView = remoteViewDict[key];
remoteView.removeAttribute('src');
remoteView.load();
remoteView.remove();
delete remoteViewDict[key];
}
}
function sendOffer(offer, toUserId) {
var jsonObject = {
"msgType": "sdp",
"fromUserId": userId,
"toUserId": toUserId,
"type": "offer",
"sdp": offer.sdp
};
send(JSON.stringify(jsonObject));
}
function receivedOffer(jsonObject) {
let fromUserId = jsonObject["fromUserId"];
var peerConnection = peerConnectionDict[fromUserId];
if (peerConnection == null) {
// 创建 PeerConnection
peerConnection = createPeerConnection(fromUserId);
// 为 PeerConnection 添加音轨、视轨
for (let i = 0; localStream != null && i < localStream.getTracks().length; i++) {
const track = localStream.getTracks()[i];
peerConnection.addTrack(track, localStream);
}
peerConnectionDict[fromUserId] = peerConnection;
}
var remoteView = remoteViewDict[fromUserId];
if (remoteView == null) {
remoteView = document.createElement("video");
remoteView.className = "remote_view";
remoteView.autoplay = true;
remoteView.control = true;
remoteView.muted = true;
remoteViews.appendChild(remoteView);
remoteViewDict[fromUserId] = remoteView;
}
let options = {
"type": jsonObject["type"],
"sdp": jsonObject["sdp"]
}
// 将 offer sdp 作为参数 setRemoteDescription
let sessionDescription = new RTCSessionDescription(options);
peerConnection.setRemoteDescription(sessionDescription).then(function () {
console.log(fromUserId + " set remote sdp success.");
// 通过 PeerConnection 创建 answer，获取 sdp
peerConnection.createAnswer().then(function (sessionDescription) {
console.log(fromUserId + " create answer success.");
// 将 answer sdp 作为参数 setLocalDescription
peerConnection.setLocalDescription(sessionDescription).then(function () {
console.log(fromUserId + " set local sdp success.");
// 发送 answer sdp
sendAnswer(sessionDescription, fromUserId);
})
})
}).catch(function (error) {
console.log("error--->" + error);
});
}
function sendAnswer(answer, toUserId) {
var jsonObject = {
"msgType": "sdp",
"fromUserId": userId,
"toUserId": toUserId,
"type": "answer",
"sdp": answer.sdp
};
send(JSON.stringify(jsonObject));
}
function receivedAnswer(jsonObject) {
let fromUserId = jsonObject["fromUserId"];
var peerConnection = peerConnectionDict[fromUserId];
if (peerConnection == null) {
// 创建 PeerConnection
peerConnection = createPeerConnection(fromUserId);
// 为 PeerConnection 添加音轨、视轨
for (let i = 0; localStream != null && i < localStream.getTracks().length; i++) {
const track = localStream.getTracks()[i];
peerConnection.addTrack(track, localStream);
}
peerConnectionDict[fromUserId] = peerConnection;
}
var remoteView = remoteViewDict[fromUserId];
if (remoteView == null) {
remoteView = document.createElement("video");
remoteView.className = "remote_view";
remoteView.autoplay = true;
remoteView.control = true;
remoteView.muted = true;
remoteViews.appendChild(remoteView);
remoteViewDict[fromUserId] = remoteView;
}
let options = {
"type": jsonObject["type"],
"sdp": jsonObject["sdp"]
}
let sessionDescription = new RTCSessionDescription(options);
let type = jsonObject["type"];
peerConnection.setRemoteDescription(sessionDescription).then(function () {
console.log(fromUserId + " set remote sdp success.");
}).catch(function (error) {
console.log("error--->" + error);
});
}
function sendIceCandidate(iceCandidate, toUserId) {
if (iceCandidate == null) {
return;
}
var jsonObject = {
"msgType": "iceCandidate",
"fromUserId": userId,
"toUserId": toUserId,
"id": iceCandidate.sdpMid,
"label": iceCandidate.sdpMLineIndex,
"candidate": iceCandidate.candidate
};
send(JSON.stringify(jsonObject));
}
function receivedCandidate(jsonObject) {
let fromUserId = jsonObject["fromUserId"];
let peerConnection = peerConnectionDict[fromUserId];
if (peerConnection == null) {
return
}
let options = {
"sdpMLineIndex": jsonObject["label"],
"sdpMid": jsonObject["id"],
"candidate": jsonObject["candidate"]
}
let iceCandidate = new RTCIceCandidate(options);
peerConnection.addIceCandidate(iceCandidate);
}
function receivedOtherJoin(jsonObject) {
// 创建 PeerConnection
let userId = jsonObject["userId"];
var peerConnection = peerConnectionDict[userId];
if (peerConnection == null) {
peerConnection = createPeerConnection(userId);
for (let i = 0; localStream != null && i < localStream.getTracks().length; i++) {
const track = localStream.getTracks()[i];
peerConnection.addTrack(track, localStream);
}
peerConnectionDict[userId] = peerConnection;
}
var remoteView = remoteViewDict[userId];
if (remoteView == null) {
remoteView = document.createElement("video");
remoteView.className = "remote_view";
remoteView.autoplay = true;
remoteView.control = true;
remoteView.muted = true;
remoteViews.appendChild(remoteView);
remoteViewDict[userId] = remoteView;
}
// 通过 PeerConnection 创建 offer，获取 sdp
peerConnection.createOffer().then(function (sessionDescription) {
console.log(userId + " create offer success.");
// 将 offer sdp 作为参数 setLocalDescription
peerConnection.setLocalDescription(sessionDescription).then(function () {
console.log(userId + " set local sdp success.");
// 发送 offer sdp
sendOffer(sessionDescription, userId);
}).catch(function (error) {
console.log("error--->" + error);
})
}).catch(function (error) {
console.log("error--->" + error);
});
}
function receivedOtherQuit(jsonObject) {
let userId = jsonObject["userId"];
let peerConnection = peerConnectionDict[userId];
if (peerConnection != null) {
peerConnection.close();
delete peerConnectionDict[userId];
}
let remoteView = remoteViewDict[userId];
if (remoteView != null) {
remoteView.removeAttribute('src');
remoteView.load();
remoteView.remove();
delete remoteViewDict[userId];
}
}
navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(function (mediaStream) {
// 初始化 PeerConnectionFactory；
// 创建 EglBase；
// 创建 PeerConnectionFactory；
// 创建音轨；
// 创建视轨；
localStream = mediaStream;
// 初始化本地视频渲染控件；
// 初始化远端视频渲染控件；
// 开始本地渲染。
localView.srcObject = mediaStream;
}).catch(function (error) {
console.log("error--->" + error);
})
</script>
<script type="text/javascript">
var websocket;
function connect() {
let inputServerUrl = document.getElementById("input_server_url");
let pWebsocketState = document.getElementById("p_websocket_state");
let url = inputServerUrl.value;
websocket = new WebSocket(url);
websocket.onopen = function () {
console.log("onOpen");
pWebsocketState.innerText = "WebSocket 已连接";
}
websocket.onmessage = function (message) {
console.log("onmessage--->" + message.data);
let jsonObject = JSON.parse(message.data);
let msgType = jsonObject["msgType"];
if ("sdp" == msgType) {
let type = jsonObject["type"];
if ("offer" == type) {
receivedOffer(jsonObject);
} else if ("answer" == type) {
receivedAnswer(jsonObject);
}
} else if ("iceCandidate" == msgType) {
receivedCandidate(jsonObject);
} else if ("otherJoin" == msgType) {
receivedOtherJoin(jsonObject);
} else if ("otherQuit" == msgType) {
receivedOtherQuit(jsonObject);
}
}
websocket.onclose = function (error) {
console.log("onclose--->" + error);
pWebsocketState.innerText = "WebSocket 已断开";
}
websocket.onerror = function (error) {
console.log("onerror--->" + error);
}
}
function disconnect() {
websocket.close();
}
function send(message) {
if (!websocket) {
return;
}
websocket.send(message);
}
</script>
</html>多人通话至少需要三个端，我们就等所有端都实现了再最后来看效果。
五、Android
1.添加依赖
这个跟前两篇的一样，不需要额外引入。
2.布局
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="#FF000000"
android:keepScreenOn="true"
tools:context=".P2PDemoActivity">
<org.webrtc.SurfaceViewRenderer
android:id="@+id/svr_local"
android:layout_width="match_parent"
android:layout_height="0dp"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintDimensionRatio="9:16"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
<androidx.core.widget.NestedScrollView
android:layout_width="90dp"
android:layout_height="wrap_content"
android:layout_marginTop="30dp"
android:layout_marginEnd="30dp"
android:layout_marginBottom="30dp"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintTop_toTopOf="parent">
<androidx.appcompat.widget.LinearLayoutCompat
android:id="@+id/ll_remotes"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:orientation="vertical">
</androidx.appcompat.widget.LinearLayoutCompat>
</androidx.core.widget.NestedScrollView>
<androidx.appcompat.widget.LinearLayoutCompat
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:layout_marginStart="30dp"
android:layout_marginTop="30dp"
android:layout_marginEnd="30dp"
android:orientation="vertical"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent">
<androidx.appcompat.widget.AppCompatTextView
android:id="@+id/tv_websocket_state"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:text="WebSocket 已断开"
android:textColor="#FFFFFFFF" />
<androidx.appcompat.widget.AppCompatEditText
android:id="@+id/et_server_url"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:hint="请输入服务器地址"
android:textColor="#FFFFFFFF"
android:textColorHint="#FFFFFFFF" />
<androidx.appcompat.widget.AppCompatButton
android:id="@+id/btn_connect"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="连接 WebSocket"
android:textAllCaps="false" />
<androidx.appcompat.widget.AppCompatButton
android:id="@+id/btn_disconnect"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="断开 WebSocket"
android:textAllCaps="false" />
<androidx.appcompat.widget.AppCompatButton
android:id="@+id/btn_join"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="加入房间"
android:textSize="12sp" />
<androidx.appcompat.widget.AppCompatButton
android:id="@+id/btn_quit"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="退出房间"
android:textSize="12sp" />
</androidx.appcompat.widget.LinearLayoutCompat>
</androidx.constraintlayout.widget.ConstraintLayout>布局中将远端视频渲染控件去掉，改成一个远端视频渲染控件的容器，每当有新的连接时创建新的远端视频渲染控件放到容器中。
3.MultipleDemoActivity.java
package com.qinshou.webrtcdemo_android;
import android.content.Context;
import android.os.Bundle;
import android.text.TextUtils;
import android.view.View;
import android.view.ViewGroup;
import android.widget.EditText;
import android.widget.LinearLayout;
import android.widget.TextView;
import androidx.appcompat.app.AppCompatActivity;
import androidx.appcompat.widget.LinearLayoutCompat;
import org.json.JSONException;
import org.json.JSONObject;
import org.webrtc.AudioSource;
import org.webrtc.AudioTrack;
import org.webrtc.Camera2Capturer;
import org.webrtc.Camera2Enumerator;
import org.webrtc.CameraEnumerator;
import org.webrtc.DataChannel;
import org.webrtc.DefaultVideoDecoderFactory;
import org.webrtc.DefaultVideoEncoderFactory;
import org.webrtc.EglBase;
import org.webrtc.IceCandidate;
import org.webrtc.MediaConstraints;
import org.webrtc.MediaStream;
import org.webrtc.PeerConnection;
import org.webrtc.PeerConnectionFactory;
import org.webrtc.RtpReceiver;
import org.webrtc.SessionDescription;
import org.webrtc.SurfaceTextureHelper;
import org.webrtc.SurfaceViewRenderer;
import org.webrtc.VideoCapturer;
import org.webrtc.VideoDecoderFactory;
import org.webrtc.VideoEncoderFactory;
import org.webrtc.VideoSource;
import org.webrtc.VideoTrack;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
/**
* Author: MrQinshou
* Email: cqflqinhao@126.com
* Date: 2023/3/21 17:22
* Description: P2P demo
*/
public class MultipleDemoActivity extends AppCompatActivity {
private static final String TAG = MultipleDemoActivity.class.getSimpleName();
private static final String AUDIO_TRACK_ID = "ARDAMSa0";
private static final String VIDEO_TRACK_ID = "ARDAMSv0";
private static final List<String> STREAM_IDS = new ArrayList<String>() {{
add("ARDAMS");
}};
private static final String SURFACE_TEXTURE_HELPER_THREAD_NAME = "SurfaceTextureHelperThread";
private static final int WIDTH = 1280;
private static final int HEIGHT = 720;
private static final int FPS = 30;
private EglBase mEglBase;
private PeerConnectionFactory mPeerConnectionFactory;
private VideoCapturer mVideoCapturer;
private AudioTrack mAudioTrack;
private VideoTrack mVideoTrack;
private WebSocketClientHelper mWebSocketClientHelper = new WebSocketClientHelper();
// private String mUserId = UUID.randomUUID().toString();
private String mUserId = "Android";
private final Map<String, PeerConnection> mPeerConnectionMap = new ConcurrentHashMap<>();
private final Map<String, SurfaceViewRenderer> mRemoteViewMap = new ConcurrentHashMap<>();
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_multiple_demo);
((EditText) findViewById(R.id.et_server_url)).setText("ws://192.168.1.104:8888");
findViewById(R.id.btn_connect).setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
String url = ((EditText) findViewById(R.id.et_server_url)).getText().toString().trim();
mWebSocketClientHelper.connect(url);
}
});
findViewById(R.id.btn_disconnect).setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
mWebSocketClientHelper.disconnect();
}
});
findViewById(R.id.btn_join).setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
join();
}
});
findViewById(R.id.btn_quit).setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
quit();
}
});
mWebSocketClientHelper.setOnWebSocketListener(new WebSocketClientHelper.OnWebSocketClientListener() {
@Override
public void onOpen() {
runOnUiThread(new Runnable() {
@Override
public void run() {
((TextView) findViewById(R.id.tv_websocket_state)).setText("WebSocket 已连接");
}
});
}
@Override
public void onClose() {
runOnUiThread(new Runnable() {
@Override
public void run() {
((TextView) findViewById(R.id.tv_websocket_state)).setText("WebSocket 已断开");
}
});
}
@Override
public void onMessage(String message) {
ShowLogUtil.debug("message--->" + message);
try {
JSONObject jsonObject = new JSONObject(message);
String msgType = jsonObject.optString("msgType");
if (TextUtils.equals("sdp", msgType)) {
String type = jsonObject.optString("type");
if (TextUtils.equals("offer", type)) {
receivedOffer(jsonObject);
} else if (TextUtils.equals("answer", type)) {
receivedAnswer(jsonObject);
}
} else if (TextUtils.equals("iceCandidate", msgType)) {
receivedCandidate(jsonObject);
} else if (TextUtils.equals("otherJoin", msgType)) {
receivedOtherJoin(jsonObject);
} else if (TextUtils.equals("otherQuit", msgType)) {
receivedOtherQuit(jsonObject);
}
} catch (JSONException e) {
e.printStackTrace();
}
}
});
// 初始化 PeerConnectionFactory
initPeerConnectionFactory(MultipleDemoActivity.this);
// 创建 EglBase
mEglBase = EglBase.create();
// 创建 PeerConnectionFactory
mPeerConnectionFactory = createPeerConnectionFactory(mEglBase);
// 创建音轨
mAudioTrack = createAudioTrack(mPeerConnectionFactory);
// 创建视轨
mVideoCapturer = createVideoCapturer();
VideoSource videoSource = createVideoSource(mPeerConnectionFactory, mVideoCapturer);
mVideoTrack = createVideoTrack(mPeerConnectionFactory, videoSource);
// 初始化本地视频渲染控件，这个方法非常重要，不初始化会黑屏
SurfaceViewRenderer svrLocal = findViewById(R.id.svr_local);
svrLocal.init(mEglBase.getEglBaseContext(), null);
mVideoTrack.addSink(svrLocal);
// 开始本地渲染
// 创建 SurfaceTextureHelper，用来表示 camera 初始化的线程
SurfaceTextureHelper surfaceTextureHelper = SurfaceTextureHelper.create(SURFACE_TEXTURE_HELPER_THREAD_NAME, mEglBase.getEglBaseContext());
// 初始化视频采集器
mVideoCapturer.initialize(surfaceTextureHelper, MultipleDemoActivity.this, videoSource.getCapturerObserver());
mVideoCapturer.startCapture(WIDTH, HEIGHT, FPS);
}
@Override
protected void onDestroy() {
super.onDestroy();
if (mEglBase != null) {
mEglBase.release();
mEglBase = null;
}
if (mVideoCapturer != null) {
try {
mVideoCapturer.stopCapture();
} catch (InterruptedException e) {
e.printStackTrace();
}
mVideoCapturer.dispose();
mVideoCapturer = null;
}
if (mAudioTrack != null) {
mAudioTrack.dispose();
mAudioTrack = null;
}
if (mVideoTrack != null) {
mVideoTrack.dispose();
mVideoTrack = null;
}
for (PeerConnection peerConnection : mPeerConnectionMap.values()) {
peerConnection.close();
peerConnection.dispose();
}
mPeerConnectionMap.clear();
SurfaceViewRenderer svrLocal = findViewById(R.id.svr_local);
svrLocal.release();
for (SurfaceViewRenderer surfaceViewRenderer : mRemoteViewMap.values()) {
surfaceViewRenderer.release();
}
mRemoteViewMap.clear();
mWebSocketClientHelper.disconnect();
}
private void initPeerConnectionFactory(Context context) {
PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions.builder(context).createInitializationOptions());
}
private PeerConnectionFactory createPeerConnectionFactory(EglBase eglBase) {
VideoEncoderFactory videoEncoderFactory = new DefaultVideoEncoderFactory(eglBase.getEglBaseContext(), true, true);
VideoDecoderFactory videoDecoderFactory = new DefaultVideoDecoderFactory(eglBase.getEglBaseContext());
return PeerConnectionFactory.builder().setVideoEncoderFactory(videoEncoderFactory).setVideoDecoderFactory(videoDecoderFactory).createPeerConnectionFactory();
}
private AudioTrack createAudioTrack(PeerConnectionFactory peerConnectionFactory) {
AudioSource audioSource = peerConnectionFactory.createAudioSource(new MediaConstraints());
AudioTrack audioTrack = peerConnectionFactory.createAudioTrack(AUDIO_TRACK_ID, audioSource);
audioTrack.setEnabled(true);
return audioTrack;
}
private VideoCapturer createVideoCapturer() {
VideoCapturer videoCapturer = null;
CameraEnumerator cameraEnumerator = new Camera2Enumerator(MultipleDemoActivity.this);
for (String deviceName : cameraEnumerator.getDeviceNames()) {
// 前摄像头
if (cameraEnumerator.isFrontFacing(deviceName)) {
videoCapturer = new Camera2Capturer(MultipleDemoActivity.this, deviceName, null);
}
}
return videoCapturer;
}
private VideoSource createVideoSource(PeerConnectionFactory peerConnectionFactory, VideoCapturer videoCapturer) {
// 创建视频源
VideoSource videoSource = peerConnectionFactory.createVideoSource(videoCapturer.isScreencast());
return videoSource;
}
private VideoTrack createVideoTrack(PeerConnectionFactory peerConnectionFactory, VideoSource videoSource) {
// 创建视轨
VideoTrack videoTrack = peerConnectionFactory.createVideoTrack(VIDEO_TRACK_ID, videoSource);
videoTrack.setEnabled(true);
return videoTrack;
}
private PeerConnection createPeerConnection(PeerConnectionFactory peerConnectionFactory, String fromUserId) {
// 内部会转成 RTCConfiguration
List<PeerConnection.IceServer> iceServers = new ArrayList<>();
PeerConnection peerConnection = peerConnectionFactory.createPeerConnection(iceServers, new PeerConnection.Observer() {
@Override
public void onSignalingChange(PeerConnection.SignalingState signalingState) {
}
@Override
public void onIceConnectionChange(PeerConnection.IceConnectionState iceConnectionState) {
ShowLogUtil.debug("onIceConnectionChange--->" + iceConnectionState);
if (iceConnectionState == PeerConnection.IceConnectionState.DISCONNECTED) {
PeerConnection peerConnection = mPeerConnectionMap.get(fromUserId);
ShowLogUtil.debug("peerConnection--->" + peerConnection);
if (peerConnection != null) {
peerConnection.close();
mPeerConnectionMap.remove(fromUserId);
}
runOnUiThread(new Runnable() {
@Override
public void run() {
SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(fromUserId);
if (surfaceViewRenderer != null) {
((ViewGroup) surfaceViewRenderer.getParent()).removeView(surfaceViewRenderer);
mRemoteViewMap.remove(fromUserId);
}
}
});
}
}
@Override
public void onIceConnectionReceivingChange(boolean b) {
}
@Override
public void onIceGatheringChange(PeerConnection.IceGatheringState iceGatheringState) {
}
@Override
public void onIceCandidate(IceCandidate iceCandidate) {
ShowLogUtil.verbose("onIceCandidate--->" + iceCandidate);
sendIceCandidate(iceCandidate, fromUserId);
}
@Override
public void onIceCandidatesRemoved(IceCandidate[] iceCandidates) {
}
@Override
public void onAddStream(MediaStream mediaStream) {
ShowLogUtil.verbose("onAddStream--->" + mediaStream);
if (mediaStream == null || mediaStream.videoTracks == null || mediaStream.videoTracks.isEmpty()) {
return;
}
runOnUiThread(new Runnable() {
@Override
public void run() {
SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(fromUserId);
if (surfaceViewRenderer != null) {
mediaStream.videoTracks.get(0).addSink(surfaceViewRenderer);
}
}
});
}
@Override
public void onRemoveStream(MediaStream mediaStream) {
}
@Override
public void onDataChannel(DataChannel dataChannel) {
}
@Override
public void onRenegotiationNeeded() {
}
@Override
public void onAddTrack(RtpReceiver rtpReceiver, MediaStream[] mediaStreams) {
}
});
return peerConnection;
}
private void join() {
try {
JSONObject jsonObject = new JSONObject();
jsonObject.put("msgType", "join");
jsonObject.put("userId", mUserId);
mWebSocketClientHelper.send(jsonObject.toString());
} catch (JSONException e) {
e.printStackTrace();
}
}
private void quit() {
try {
JSONObject jsonObject = new JSONObject();
jsonObject.put("msgType", "quit");
jsonObject.put("userId", mUserId);
mWebSocketClientHelper.send(jsonObject.toString());
} catch (JSONException e) {
e.printStackTrace();
}
new Thread(new Runnable() {
@Override
public void run() {
for (PeerConnection peerConnection : mPeerConnectionMap.values()) {
peerConnection.close();
}
mPeerConnectionMap.clear();
}
}).start();
for (SurfaceViewRenderer surfaceViewRenderer : mRemoteViewMap.values()) {
((ViewGroup) surfaceViewRenderer.getParent()).removeView(surfaceViewRenderer);
}
mRemoteViewMap.clear();
}
private void sendOffer(SessionDescription offer, String toUserId) {
try {
JSONObject jsonObject = new JSONObject();
jsonObject.put("msgType", "sdp");
jsonObject.put("fromUserId", mUserId);
jsonObject.put("toUserId", toUserId);
jsonObject.put("type", "offer");
jsonObject.put("sdp", offer.description);
mWebSocketClientHelper.send(jsonObject.toString());
} catch (JSONException e) {
e.printStackTrace();
}
}
private void receivedOffer(JSONObject jsonObject) {
String fromUserId = jsonObject.optString("fromUserId");
PeerConnection peerConnection = mPeerConnectionMap.get(fromUserId);
if (peerConnection == null) {
// 创建 PeerConnection
peerConnection = createPeerConnection(mPeerConnectionFactory, fromUserId);
// 为 PeerConnection 添加音轨、视轨
peerConnection.addTrack(mAudioTrack, STREAM_IDS);
peerConnection.addTrack(mVideoTrack, STREAM_IDS);
mPeerConnectionMap.put(fromUserId, peerConnection);
}
runOnUiThread(new Runnable() {
@Override
public void run() {
SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(fromUserId);
if (surfaceViewRenderer == null) {
// 初始化 SurfaceViewRender ，这个方法非常重要，不初始化黑屏
surfaceViewRenderer = new SurfaceViewRenderer(MultipleDemoActivity.this);
surfaceViewRenderer.init(mEglBase.getEglBaseContext(), null);
surfaceViewRenderer.setLayoutParams(new LinearLayout.LayoutParams(dp2px(MultipleDemoActivity.this, 90), dp2px(MultipleDemoActivity.this, 160)));
LinearLayoutCompat llRemotes = findViewById(R.id.ll_remotes);
llRemotes.addView(surfaceViewRenderer);
mRemoteViewMap.put(fromUserId, surfaceViewRenderer);
}
}
});
String type = jsonObject.optString("type");
String sdp = jsonObject.optString("sdp");
PeerConnection finalPeerConnection = peerConnection;
// 将 offer sdp 作为参数 setRemoteDescription
SessionDescription sessionDescription = new SessionDescription(SessionDescription.Type.fromCanonicalForm(type), sdp);
peerConnection.setRemoteDescription(new MySdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
}
@Override
public void onSetSuccess() {
ShowLogUtil.debug(fromUserId + " set remote sdp success.");
// 通过 PeerConnection 创建 answer，获取 sdp
MediaConstraints mediaConstraints = new MediaConstraints();
finalPeerConnection.createAnswer(new MySdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
ShowLogUtil.verbose(fromUserId + "create answer success.");
// 将 answer sdp 作为参数 setLocalDescription
finalPeerConnection.setLocalDescription(new MySdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
}
@Override
public void onSetSuccess() {
ShowLogUtil.verbose(fromUserId + " set local sdp success.");
// 发送 answer sdp
sendAnswer(sessionDescription, fromUserId);
}
}, sessionDescription);
}
@Override
public void onSetSuccess() {
}
}, mediaConstraints);
}
}, sessionDescription);
}
private void sendAnswer(SessionDescription answer, String toUserId) {
try {
JSONObject jsonObject = new JSONObject();
jsonObject.put("msgType", "sdp");
jsonObject.put("fromUserId", mUserId);
jsonObject.put("toUserId", toUserId);
jsonObject.put("type", "answer");
jsonObject.put("sdp", answer.description);
mWebSocketClientHelper.send(jsonObject.toString());
} catch (JSONException e) {
e.printStackTrace();
}
}
private void receivedAnswer(JSONObject jsonObject) {
String fromUserId = jsonObject.optString("fromUserId");
PeerConnection peerConnection = mPeerConnectionMap.get(fromUserId);
if (peerConnection == null) {
peerConnection = createPeerConnection(mPeerConnectionFactory, fromUserId);
peerConnection.addTrack(mAudioTrack, STREAM_IDS);
peerConnection.addTrack(mVideoTrack, STREAM_IDS);
mPeerConnectionMap.put(fromUserId, peerConnection);
}
runOnUiThread(new Runnable() {
@Override
public void run() {
SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(fromUserId);
if (surfaceViewRenderer == null) {
// 初始化 SurfaceViewRender ，这个方法非常重要，不初始化黑屏
surfaceViewRenderer = new SurfaceViewRenderer(MultipleDemoActivity.this);
surfaceViewRenderer.init(mEglBase.getEglBaseContext(), null);
surfaceViewRenderer.setLayoutParams(new LinearLayout.LayoutParams(dp2px(MultipleDemoActivity.this, 90), dp2px(MultipleDemoActivity.this, 160)));
LinearLayoutCompat llRemotes = findViewById(R.id.ll_remotes);
llRemotes.addView(surfaceViewRenderer);
mRemoteViewMap.put(fromUserId, surfaceViewRenderer);
}
}
});
String type = jsonObject.optString("type");
String sdp = jsonObject.optString("sdp");
// 收到 answer sdp，将 answer sdp 作为参数 setRemoteDescription
SessionDescription sessionDescription = new SessionDescription(SessionDescription.Type.fromCanonicalForm(type), sdp);
peerConnection.setRemoteDescription(new MySdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
}
@Override
public void onSetSuccess() {
ShowLogUtil.debug(fromUserId + " set remote sdp success.");
}
}, sessionDescription);
}
private void sendIceCandidate(IceCandidate iceCandidate, String toUserId) {
try {
JSONObject jsonObject = new JSONObject();
jsonObject.put("msgType", "iceCandidate");
jsonObject.put("fromUserId", mUserId);
jsonObject.put("toUserId", toUserId);
jsonObject.put("id", iceCandidate.sdpMid);
jsonObject.put("label", iceCandidate.sdpMLineIndex);
jsonObject.put("candidate", iceCandidate.sdp);
mWebSocketClientHelper.send(jsonObject.toString());
} catch (JSONException e) {
e.printStackTrace();
}
}
private void receivedCandidate(JSONObject jsonObject) {
String fromUserId = jsonObject.optString("fromUserId");
PeerConnection peerConnection = mPeerConnectionMap.get(fromUserId);
if (peerConnection == null) {
return;
}
String id = jsonObject.optString("id");
int label = jsonObject.optInt("label");
String candidate = jsonObject.optString("candidate");
IceCandidate iceCandidate = new IceCandidate(id, label, candidate);
peerConnection.addIceCandidate(iceCandidate);
}
private void receivedOtherJoin(JSONObject jsonObject) throws JSONException {
String userId = jsonObject.optString("userId");
PeerConnection peerConnection = mPeerConnectionMap.get(userId);
if (peerConnection == null) {
// 创建 PeerConnection
peerConnection = createPeerConnection(mPeerConnectionFactory, userId);
// 为 PeerConnection 添加音轨、视轨
peerConnection.addTrack(mAudioTrack, STREAM_IDS);
peerConnection.addTrack(mVideoTrack, STREAM_IDS);
mPeerConnectionMap.put(userId, peerConnection);
}
runOnUiThread(new Runnable() {
@Override
public void run() {
SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(userId);
if (surfaceViewRenderer == null) {
// 初始化 SurfaceViewRender ，这个方法非常重要，不初始化黑屏
surfaceViewRenderer = new SurfaceViewRenderer(MultipleDemoActivity.this);
surfaceViewRenderer.init(mEglBase.getEglBaseContext(), null);
surfaceViewRenderer.setLayoutParams(new LinearLayout.LayoutParams(dp2px(MultipleDemoActivity.this, 90), dp2px(MultipleDemoActivity.this, 160)));
LinearLayoutCompat llRemotes = findViewById(R.id.ll_remotes);
llRemotes.addView(surfaceViewRenderer);
mRemoteViewMap.put(userId, surfaceViewRenderer);
}
}
});
PeerConnection finalPeerConnection = peerConnection;
// 通过 PeerConnection 创建 offer，获取 sdp
MediaConstraints mediaConstraints = new MediaConstraints();
peerConnection.createOffer(new MySdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
ShowLogUtil.verbose(userId + " create offer success.");
// 将 offer sdp 作为参数 setLocalDescription
finalPeerConnection.setLocalDescription(new MySdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
}
@Override
public void onSetSuccess() {
ShowLogUtil.verbose(userId + " set local sdp success.");
// 发送 offer sdp
sendOffer(sessionDescription, userId);
}
}, sessionDescription);
}
@Override
public void onSetSuccess() {
}
}, mediaConstraints);
}
private void receivedOtherQuit(JSONObject jsonObject) throws JSONException {
String userId = jsonObject.optString("userId");
PeerConnection peerConnection = mPeerConnectionMap.get(userId);
if (peerConnection != null) {
peerConnection.close();
mPeerConnectionMap.remove(userId);
}
runOnUiThread(new Runnable() {
@Override
public void run() {
SurfaceViewRenderer surfaceViewRenderer = mRemoteViewMap.get(userId);
if (surfaceViewRenderer != null) {
((ViewGroup) surfaceViewRenderer.getParent()).removeView(surfaceViewRenderer);
mRemoteViewMap.remove(userId);
}
}
});
}
public static int dp2px(Context context, float dp) {
float density = context.getResources().getDisplayMetrics().density;
return (int) (dp * density + 0.5f);
}
}其中 WebSocketClientHelper 跟之前一样的，其余逻辑跟 H5 是一样的。多人通话至少需要三个端，我们就等所有端都实现了再最后来看效果。
六、iOS
1.添加依赖
这个跟前两篇的一样，不需要额外引入。
2.MultipleDemoViewController.swift
//
// LocalDemoViewController.swift
// WebRTCDemo-iOS
//
// Created by 覃浩 on 2023/3/21.
//
import UIKit
import WebRTC
import SnapKit
class MultipleDemoViewController: UIViewController {
private static let AUDIO_TRACK_ID = "ARDAMSa0"
private static let VIDEO_TRACK_ID = "ARDAMSv0"
private static let STREAM_IDS = ["ARDAMS"]
private static let WIDTH = 1280
private static let HEIGHT = 720
private static let FPS = 30
private var localView: RTCEAGLVideoView!
private var remoteViews: UIScrollView!
private var peerConnectionFactory: RTCPeerConnectionFactory!
private var audioTrack: RTCAudioTrack?
private var videoTrack: RTCVideoTrack?
/**
iOS 需要将 Capturer 保存为全局变量，否则无法渲染本地画面
*/
private var videoCapturer: RTCVideoCapturer?
/**
iOS 需要将远端流保存为全局变量，否则无法渲染远端画面
*/
private var remoteStreamDict: [String : RTCMediaStream] = [:]
// private let userId = UUID().uuidString
private let userId = "iOS"
private var peerConnectionDict: [String : RTCPeerConnection] = [:]
private var remoteViewDict: [String : RTCEAGLVideoView] = [:]
private var lbWebSocketState: UILabel? = nil
private var tfServerUrl: UITextField? = nil
private let webSocketHelper = WebSocketClientHelper()
override func viewDidLoad() {
super.viewDidLoad()
// 表明 View 不要扩展到整个屏幕，而是在 NavigationBar 下的区域
edgesForExtendedLayout = UIRectEdge()
self.view.backgroundColor = UIColor.black
// WebSocket 状态文本框
lbWebSocketState = UILabel()
lbWebSocketState!.textColor = UIColor.white
lbWebSocketState!.text = "WebSocket 已断开"
self.view.addSubview(lbWebSocketState!)
lbWebSocketState!.snp.makeConstraints({ make in
make.left.equalToSuperview().offset(30)
make.right.equalToSuperview().offset(-30)
make.height.equalTo(40)
})
// 服务器地址输入框
tfServerUrl = UITextField()
tfServerUrl!.textColor = UIColor.white
tfServerUrl!.text = "ws://192.168.1.104:8888"
tfServerUrl!.placeholder = "请输入服务器地址"
tfServerUrl!.delegate = self
self.view.addSubview(tfServerUrl!)
tfServerUrl!.snp.makeConstraints({ make in
make.left.equalToSuperview().offset(30)
make.right.equalToSuperview().offset(-30)
make.height.equalTo(20)
make.top.equalTo(lbWebSocketState!.snp.bottom).offset(10)
})
// 连接 WebSocket 按钮
let btnConnect = UIButton()
btnConnect.backgroundColor = UIColor.lightGray
btnConnect.setTitle("连接 WebSocket", for: .normal)
btnConnect.setTitleColor(UIColor.black, for: .normal)
btnConnect.addTarget(self, action: #selector(connect), for: .touchUpInside)
self.view.addSubview(btnConnect)
btnConnect.snp.makeConstraints({ make in
make.left.equalToSuperview().offset(30)
make.width.equalTo(140)
make.height.equalTo(40)
make.top.equalTo(tfServerUrl!.snp.bottom).offset(10)
})
// 断开 WebSocket 按钮
let btnDisconnect = UIButton()
btnDisconnect.backgroundColor = UIColor.lightGray
btnDisconnect.setTitle("断开 WebSocket", for: .normal)
btnDisconnect.setTitleColor(UIColor.black, for: .normal)
btnDisconnect.addTarget(self, action: #selector(disconnect), for: .touchUpInside)
self.view.addSubview(btnDisconnect)
btnDisconnect.snp.makeConstraints({ make in
make.left.equalToSuperview().offset(30)
make.width.equalTo(140)
make.height.equalTo(40)
make.top.equalTo(btnConnect.snp.bottom).offset(10)
})
// 呼叫按钮
let btnCall = UIButton()
btnCall.backgroundColor = UIColor.lightGray
btnCall.setTitle("加入房间", for: .normal)
btnCall.setTitleColor(UIColor.black, for: .normal)
btnCall.addTarget(self, action: #selector(join), for: .touchUpInside)
self.view.addSubview(btnCall)
btnCall.snp.makeConstraints({ make in
make.left.equalToSuperview().offset(30)
make.width.equalTo(160)
make.height.equalTo(40)
make.top.equalTo(btnDisconnect.snp.bottom).offset(10)
})
// 挂断按钮
let btnHangUp = UIButton()
btnHangUp.backgroundColor = UIColor.lightGray
btnHangUp.setTitle("退出房间", for: .normal)
btnHangUp.setTitleColor(UIColor.black, for: .normal)
btnHangUp.addTarget(self, action: #selector(quit), for: .touchUpInside)
self.view.addSubview(btnHangUp)
btnHangUp.snp.makeConstraints({ make in
make.left.equalToSuperview().offset(30)
make.width.equalTo(160)
make.height.equalTo(40)
make.top.equalTo(btnCall.snp.bottom).offset(10)
})
webSocketHelper.setDelegate(delegate: self)
// 初始化 PeerConnectionFactory
initPeerConnectionFactory()
// 创建 EglBase
// 创建 PeerConnectionFactory
peerConnectionFactory = createPeerConnectionFactory()
// 创建音轨
audioTrack = createAudioTrack(peerConnectionFactory: peerConnectionFactory)
// 创建视轨
videoTrack = createVideoTrack(peerConnectionFactory: peerConnectionFactory)
let tuple = createVideoCapturer(videoSource: videoTrack!.source)
let captureDevice = tuple.captureDevice
videoCapturer = tuple.videoCapture
// 初始化本地视频渲染控件
localView = RTCEAGLVideoView()
localView.delegate = self
self.view.insertSubview(localView,at: 0)
localView.snp.makeConstraints({ make in
make.width.equalToSuperview()
make.height.equalTo(localView.snp.width).multipliedBy(16.0/9.0)
make.centerY.equalToSuperview()
})
videoTrack?.add(localView!)
// 开始本地渲染
(videoCapturer as? RTCCameraVideoCapturer)?.startCapture(with: captureDevice!, format: captureDevice!.activeFormat, fps: MultipleDemoViewController.FPS)
// 初始化远端视频渲染控件容器
remoteViews = UIScrollView()
self.view.insertSubview(remoteViews, aboveSubview: localView)
remoteViews.snp.makeConstraints { maker in
maker.width.equalTo(90)
maker.top.equalToSuperview().offset(30)
maker.right.equalToSuperview().offset(-30)
maker.bottom.equalToSuperview().offset(-30)
}
}
override func viewDidDisappear(_ animated: Bool) {
(videoCapturer as? RTCCameraVideoCapturer)?.stopCapture()
videoCapturer = nil
for peerConnection in peerConnectionDict.values {
peerConnection.close()
}
peerConnectionDict.removeAll(keepingCapacity: false)
remoteViewDict.removeAll(keepingCapacity: false)
remoteStreamDict.removeAll(keepingCapacity: false)
webSocketHelper.disconnect()
}
private func initPeerConnectionFactory() {
RTCPeerConnectionFactory.initialize()
}
private func createPeerConnectionFactory() -> RTCPeerConnectionFactory {
var videoEncoderFactory = RTCDefaultVideoEncoderFactory()
var videoDecoderFactory = RTCDefaultVideoDecoderFactory()
if TARGET_OS_SIMULATOR != 0 {
videoEncoderFactory = RTCSimluatorVideoEncoderFactory()
videoDecoderFactory = RTCSimulatorVideoDecoderFactory()
}
return RTCPeerConnectionFactory(encoderFactory: videoEncoderFactory, decoderFactory: videoDecoderFactory)
}
private func createAudioTrack(peerConnectionFactory: RTCPeerConnectionFactory) -> RTCAudioTrack {
let mandatoryConstraints : [String : String] = [:]
let optionalConstraints : [String : String] = [:]
let audioSource = peerConnectionFactory.audioSource(with: RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints))
let audioTrack = peerConnectionFactory.audioTrack(with: audioSource, trackId: MultipleDemoViewController.AUDIO_TRACK_ID)
audioTrack.isEnabled = true
return audioTrack
}
private func createVideoTrack(peerConnectionFactory: RTCPeerConnectionFactory) -> RTCVideoTrack? {
let videoSource = peerConnectionFactory.videoSource()
let videoTrack = peerConnectionFactory.videoTrack(with: videoSource, trackId: MultipleDemoViewController.VIDEO_TRACK_ID)
videoTrack.isEnabled = true
return videoTrack
}
private func createVideoCapturer(videoSource: RTCVideoSource) -> (captureDevice: AVCaptureDevice?, videoCapture: RTCVideoCapturer?) {
let videoCapturer = RTCCameraVideoCapturer(delegate: videoSource)
let captureDevices = RTCCameraVideoCapturer.captureDevices()
if (captureDevices.count == 0) {
return (nil, nil)
}
var captureDevice: AVCaptureDevice?
for c in captureDevices {
// 前摄像头
if (c.position == .front) {
captureDevice = c
break
}
}
if (captureDevice == nil) {
return (nil, nil)
}
return (captureDevice, videoCapturer)
}
private func createPeerConnection(peerConnectionFactory: RTCPeerConnectionFactory, fromUserId: String) -> RTCPeerConnection {
let configuration = RTCConfiguration()
// configuration.sdpSemantics = .unifiedPlan
// configuration.continualGatheringPolicy = .gatherContinually
// configuration.iceServers = [RTCIceServer(urlStrings: ["stun:stun.l.google.com:19302"])]
let mandatoryConstraints : [String : String] = [:]
// let mandatoryConstraints = [kRTCMediaConstraintsOfferToReceiveAudio: kRTCMediaConstraintsValueTrue,
// kRTCMediaConstraintsOfferToReceiveVideo: kRTCMediaConstraintsValueTrue]
let optionalConstraints : [String : String] = [:]
// let optionalConstraints = ["DtlsSrtpKeyAgreement" : kRTCMediaConstraintsValueTrue]
let mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)
return peerConnectionFactory.peerConnection(with: configuration, constraints: mediaConstraints, delegate: self)
}
@objc private func connect() {
webSocketHelper.connect(url: tfServerUrl!.text!.trimmingCharacters(in: .whitespacesAndNewlines))
}
@objc private func disconnect() {
webSocketHelper.disconnect()
}
@objc private func join() {
var jsonObject = [String : String]()
jsonObject["msgType"] = "join"
jsonObject["userId"] = userId
do {
let data = try JSONSerialization.data(withJSONObject: jsonObject)
webSocketHelper.send(message: String(data: data, encoding: .utf8)!)
} catch {
ShowLogUtil.verbose("error--->\(error)")
}
}
@objc private func quit() {
var jsonObject = [String : String]()
jsonObject["msgType"] = "quit"
jsonObject["userId"] = userId
do {
let data = try JSONSerialization.data(withJSONObject: jsonObject)
webSocketHelper.send(message: String(data: data, encoding: .utf8)!)
} catch {
ShowLogUtil.verbose("error--->\(error)")
}
for peerConnection in peerConnectionDict.values {
peerConnection.close()
}
peerConnectionDict.removeAll(keepingCapacity: false)
for (key, value) in remoteViewDict {
remoteViews.removeSubview(view: value)
}
remoteViewDict.removeAll(keepingCapacity: false)
}
private func sendOffer(offer: RTCSessionDescription, toUserId: String) {
var jsonObject = [String : String]()
jsonObject["msgType"] = "sdp"
jsonObject["fromUserId"] = userId
jsonObject["toUserId"] = toUserId
jsonObject["type"] = "offer"
jsonObject["sdp"] = offer.sdp
do {
let data = try JSONSerialization.data(withJSONObject: jsonObject)
webSocketHelper.send(message: String(data: data, encoding: .utf8)!)
} catch {
ShowLogUtil.verbose("error--->\(error)")
}
}
private func receivedOffer(jsonObject: [String : Any]) {
let fromUserId = jsonObject["fromUserId"] as? String ?? ""
var peerConnection = peerConnectionDict[fromUserId]
if (peerConnection == nil) {
// 创建 PeerConnection
peerConnection = createPeerConnection(peerConnectionFactory: peerConnectionFactory, fromUserId: fromUserId)
// 为 PeerConnection 添加音轨、视轨
peerConnection!.add(audioTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)
peerConnection!.add(videoTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)
peerConnectionDict[fromUserId] = peerConnection
}
var remoteView = remoteViewDict[fromUserId]
if (remoteView == nil) {
let x = 0
var y = 0
if (remoteViews.subviews.count == 0) {
y = 0
} else {
for i in 0..<remoteViews.subviews.count {
y += Int(remoteViews.subviews[i].frame.height)
}
}
let width = 90
let height = width / 9 * 16
remoteView = RTCEAGLVideoView(frame: CGRect(x: x, y: y, width: width, height: height))
remoteViews.appendSubView(view: remoteView!)
remoteViewDict[fromUserId] = remoteView
}
// 将 offer sdp 作为参数 setRemoteDescription
let type = jsonObject["type"] as? String
let sdp = jsonObject["sdp"] as? String
let sessionDescription = RTCSessionDescription(type: .offer, sdp: sdp!)
peerConnection?.setRemoteDescription(sessionDescription, completionHandler: { _ in
ShowLogUtil.verbose("\(fromUserId) set remote sdp success.")
// 通过 PeerConnection 创建 answer，获取 sdp
let mandatoryConstraints : [String : String] = [:]
let optionalConstraints : [String : String] = [:]
let mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)
peerConnection?.answer(for: mediaConstraints, completionHandler: { sessionDescription, error in
ShowLogUtil.verbose("\(fromUserId) create answer success.")
// 将 answer sdp 作为参数 setLocalDescription
peerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ in
ShowLogUtil.verbose("\(fromUserId) set local sdp success.")
// 发送 answer sdp
self.sendAnswer(answer: sessionDescription!, toUserId: fromUserId)
})
})
})
}
private func sendAnswer(answer: RTCSessionDescription, toUserId: String) {
var jsonObject = [String : String]()
jsonObject["msgType"] = "sdp"
jsonObject["fromUserId"] = userId
jsonObject["toUserId"] = toUserId
jsonObject["type"] = "answer"
jsonObject["sdp"] = answer.sdp
do {
let data = try JSONSerialization.data(withJSONObject: jsonObject)
webSocketHelper.send(message: String(data: data, encoding: .utf8)!)
} catch {
ShowLogUtil.verbose("error--->\(error)")
}
}
private func receivedAnswer(jsonObject: [String : Any]) {
let fromUserId = jsonObject["fromUserId"] as? String ?? ""
var peerConnection = peerConnectionDict[fromUserId]
if (peerConnection == nil) {
peerConnection = createPeerConnection(peerConnectionFactory: peerConnectionFactory, fromUserId: fromUserId)
peerConnection!.add(audioTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)
peerConnection!.add(videoTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)
peerConnectionDict[fromUserId] = peerConnection
}
DispatchQueue.main.async {
var remoteView = self.remoteViewDict[fromUserId]
if (remoteView == nil) {
let x = 0
var y = 0
if (self.remoteViews.subviews.count == 0) {
y = 0
} else {
for i in 0..<self.remoteViews.subviews.count {
y += Int(self.remoteViews.subviews[i].frame.height)
}
}
let width = 90
let height = width / 9 * 16
remoteView = RTCEAGLVideoView(frame: CGRect(x: x, y: y, width: width, height: height))
self.remoteViews.appendSubView(view: remoteView!)
self.remoteViewDict[fromUserId] = remoteView
}
}
// 收到 answer sdp，将 answer sdp 作为参数 setRemoteDescription
let type = jsonObject["type"] as? String
let sdp = jsonObject["sdp"] as? String
let sessionDescription = RTCSessionDescription(type: .answer, sdp: sdp!)
peerConnection!.setRemoteDescription(sessionDescription, completionHandler: { _ in
ShowLogUtil.verbose(fromUserId + " set remote sdp success.");
})
}
private func sendIceCandidate(iceCandidate: RTCIceCandidate, toUserId: String) {
var jsonObject = [String : Any]()
jsonObject["msgType"] = "iceCandidate"
jsonObject["fromUserId"] = userId
jsonObject["toUserId"] = toUserId
jsonObject["id"] = iceCandidate.sdpMid
jsonObject["label"] = iceCandidate.sdpMLineIndex
jsonObject["candidate"] = iceCandidate.sdp
do {
let data = try JSONSerialization.data(withJSONObject: jsonObject)
webSocketHelper.send(message: String(data: data, encoding: .utf8)!)
} catch {
ShowLogUtil.verbose("error--->\(error)")
}
}
private func receivedCandidate(jsonObject: [String : Any]) {
let fromUserId = jsonObject["fromUserId"] as? String ?? ""
let peerConnection = peerConnectionDict[fromUserId]
if (peerConnection == nil) {
return
}
let id = jsonObject["id"] as? String
let label = jsonObject["label"] as? Int32
let candidate = jsonObject["candidate"] as? String
let iceCandidate = RTCIceCandidate(sdp: candidate!, sdpMLineIndex: label!, sdpMid: id)
peerConnection!.add(iceCandidate)
}
private func receiveOtherJoin(jsonObject: [String : Any]) {
let userId = jsonObject["userId"] as? String ?? ""
var peerConnection = peerConnectionDict[userId]
if (peerConnection == nil) {
// 创建 PeerConnection
peerConnection = createPeerConnection(peerConnectionFactory: peerConnectionFactory, fromUserId: userId)
// 为 PeerConnection 添加音轨、视轨
peerConnection!.add(audioTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)
peerConnection!.add(videoTrack!, streamIds: MultipleDemoViewController.STREAM_IDS)
peerConnectionDict[userId] = peerConnection
}
DispatchQueue.main.async {
var remoteView = self.remoteViewDict[userId]
if (remoteView == nil) {
let x = 0
var y = 0
if (self.remoteViews.subviews.count == 0) {
y = 0
} else {
for i in 0..<self.remoteViews.subviews.count {
y += Int(self.remoteViews.subviews[i].frame.height)
}
}
let width = 90
let height = width / 9 * 16
remoteView = RTCEAGLVideoView(frame: CGRect(x: x, y: y, width: width, height: height))
self.remoteViews.appendSubView(view: remoteView!)
self.remoteViewDict[userId] = remoteView
}
}
// 通过 PeerConnection 创建 offer，获取 sdp
let mandatoryConstraints : [String : String] = [:]
let optionalConstraints : [String : String] = [:]
let mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)
peerConnection?.offer(for: mediaConstraints, completionHandler: { sessionDescription, error in
ShowLogUtil.verbose("\(userId) create offer success.")
if (error != nil) {
return
}
// 将 offer sdp 作为参数 setLocalDescription
peerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ in
ShowLogUtil.verbose("\(userId) set local sdp success.")
// 发送 offer sdp
self.sendOffer(offer: sessionDescription!, toUserId: userId)
})
})
}
private func receiveOtherQuit(jsonObject: [String : Any]) {
let userId = jsonObject["userId"] as? String ?? ""
Thread(block: {
let peerConnection = self.peerConnectionDict[userId]
if (peerConnection != nil) {
peerConnection?.close()
self.peerConnectionDict.removeValue(forKey: userId)
}
}).start()
let remoteView = remoteViewDict[userId]
if (remoteView != nil) {
remoteViews.removeSubview(view: remoteView!)
remoteViewDict.removeValue(forKey: userId)
}
remoteStreamDict.removeValue(forKey: userId)
}
}
// MARK: - RTCVideoViewDelegate
extension MultipleDemoViewController: RTCVideoViewDelegate {
func videoView(_ videoView: RTCVideoRenderer, didChangeVideoSize size: CGSize) {
}
}
// MARK: - RTCPeerConnectionDelegate
extension MultipleDemoViewController: RTCPeerConnectionDelegate {
func peerConnection(_ peerConnection: RTCPeerConnection, didChange stateChanged: RTCSignalingState) {
}
func peerConnection(_ peerConnection: RTCPeerConnection, didAdd stream: RTCMediaStream) {
ShowLogUtil.verbose("peerConnection didAdd stream--->\(stream)")
var userId: String?
for (key, value) in peerConnectionDict {
if (value == peerConnection) {
userId = key
}
}
if (userId == nil) {
return
}
remoteStreamDict[userId!] = stream
let remoteView = remoteViewDict[userId!]
if (remoteView == nil) {
return
}
if let videoTrack = stream.videoTracks.first {
ShowLogUtil.verbose("video track found.")
videoTrack.add(remoteView!)
}
if let audioTrack = stream.audioTracks.first{
ShowLogUtil.verbose("audio track found.")
audioTrack.source.volume = 8
}
}
func peerConnection(_ peerConnection: RTCPeerConnection, didRemove stream: RTCMediaStream) {
}
func peerConnectionShouldNegotiate(_ peerConnection: RTCPeerConnection) {
}
func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceConnectionState) {
if (newState == .disconnected) {
DispatchQueue.main.async {
var userId: String?
for (key, value) in self.peerConnectionDict {
if (value == peerConnection) {
userId = key
}
}
if (userId == nil) {
return
}
Thread(block: {
let peerConnection = self.peerConnectionDict[userId!]
if (peerConnection != nil) {
peerConnection?.close()
self.peerConnectionDict.removeValue(forKey: userId!)
}
}).start()
let remoteView = self.remoteViewDict[userId!]
if (remoteView != nil) {
self.remoteViews.removeSubview(view: remoteView!)
self.remoteViewDict.removeValue(forKey: userId!)
}
self.remoteStreamDict.removeValue(forKey: userId!)
}
}
}
func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceGatheringState) {
}
func peerConnection(_ peerConnection: RTCPeerConnection, didGenerate candidate: RTCIceCandidate) {
// ShowLogUtil.verbose("didGenerate candidate--->\(candidate)")
var userId: String?
for (key, value) in self.peerConnectionDict {
if (value == peerConnection) {
userId = key
}
}
if (userId == nil) {
return
}
self.sendIceCandidate(iceCandidate: candidate, toUserId: userId!)
}
func peerConnection(_ peerConnection: RTCPeerConnection, didRemove candidates: [RTCIceCandidate]) {
}
func peerConnection(_ peerConnection: RTCPeerConnection, didOpen dataChannel: RTCDataChannel) {
}
}
// MARK: - UITextFieldDelegate
extension MultipleDemoViewController: UITextFieldDelegate {
func textFieldShouldReturn(_ textField: UITextField) -> Bool {
textField.resignFirstResponder()
return true
}
}
// MARK: - WebSocketDelegate
extension MultipleDemoViewController: WebSocketDelegate {
func onOpen() {
lbWebSocketState?.text = "WebSocket 已连接"
}
func onClose() {
lbWebSocketState?.text = "WebSocket 已断开"
}
func onMessage(message: String) {
do {
let data = message.data(using: .utf8)
let jsonObject: [String : Any] = try JSONSerialization.jsonObject(with: data!) as! [String : Any]
let msgType = jsonObject["msgType"] as? String
if ("sdp" == msgType) {
let type = jsonObject["type"] as? String;
if ("offer" == type) {
receivedOffer(jsonObject: jsonObject);
} else if ("answer" == type) {
receivedAnswer(jsonObject: jsonObject);
}
} else if ("iceCandidate" == msgType) {
receivedCandidate(jsonObject: jsonObject);
} else if ("otherJoin" == msgType) {
receiveOtherJoin(jsonObject: jsonObject)
} else if ("otherQuit" == msgType) {
receiveOtherQuit(jsonObject: jsonObject)
}
} catch {
}
}
}
其中 UIScrollView 的 appendSubView 和 removeSubView 是我为 UIScrollView 添加的两个扩展方法，方便纵向添加和删除控件：
import UIKit
extension UIScrollView {
func appendSubView(view: UIView) {
let oldShowsHorizontalScrollIndicator = showsHorizontalScrollIndicator
let oldShowsVerticalScrollIndicator = showsVerticalScrollIndicator
showsHorizontalScrollIndicator = false
showsVerticalScrollIndicator = false
var y = 0.0
if (subviews.count == 0) {
y = 0
} else {
for i in 0..<subviews.count {
if ("_UIScrollViewScrollIndicator" == String(reflecting: type(of: subviews[i]))){
continue
}
y += subviews[i].frame.height
}
}
view.frame.origin.y = y
addSubview(view)
let contentSizeWidth = contentSize.width
// 重新计算 UIScrollView 内容高度
var contentSizeHeight = 0.0
for i in 0..<subviews.count {
if ("_UIScrollViewScrollIndicator" == String(reflecting: type(of: subviews[i]))){
continue
}
contentSizeHeight += subviews[i].frame.height
}
contentSize = CGSize(width: contentSizeWidth, height: contentSizeHeight)
showsHorizontalScrollIndicator = oldShowsHorizontalScrollIndicator
showsVerticalScrollIndicator = oldShowsVerticalScrollIndicator
}
func removeSubview(view: UIView) {
let oldShowsHorizontalScrollIndicator = showsHorizontalScrollIndicator
let oldShowsVerticalScrollIndicator = showsVerticalScrollIndicator
showsHorizontalScrollIndicator = false
showsVerticalScrollIndicator = false
var index = -1
for i in 0..<subviews.count {
if (subviews[i] == view) {
index = i
break
}
}
if (index == -1) {
return
}
for i in index+1..<subviews.count {
subviews[i].frame.origin.y = subviews[i].frame.origin.y-view.frame.height
}
view.removeFromSuperview()
let contentSizeWidth = contentSize.width
// 重新计算 UIScrollView 内容高度
var contentSizeHeight = 0.0
for i in 0..<subviews.count {
if ("_UIScrollViewScrollIndicator" == String(reflecting: type(of: subviews[i]))){
continue
}
contentSizeHeight += subviews[i].frame.height
}
contentSize = CGSize(width: contentSizeWidth, height: contentSizeHeight)
showsHorizontalScrollIndicator = oldShowsHorizontalScrollIndicator
showsVerticalScrollIndicator = oldShowsVerticalScrollIndicator
}
}好了，现在三端都实现了，我们可以来看看效果了。
七、效果展示
运行 MultipleWebSocketServerHelper 的 main() 方法，我们可以看到服务端已经开启，然后我们依次将 H5、Android、iOS 连接 WebSocket，再依次加入房间：
其中 iOS 在录屏的时候可能是系统限制，画面静止了，但其实跟另外两端是一样的，从另外两端的远端画面可以看到 iOS 是有在采集摄像头画面的。
八、总结
实现完成后可以感觉到多人呼叫其实也没有多难，跟点对点 Demo 的流程大致一样，只是我们需要重新定义创建 PeerConnection 的时机，但是流程仍然是不变的。以及信令有些许不同，信令这就是业务层面的，自己按需来设计，上面我定义的消息格式只是一个最简单的实现。
至此，WebRTC 单人和多人通话的 Demo 全部完成，这就说明它能满足我们基本的视频通话、视频会议等需求，至于丢包处理、美颜滤镜就是网络优化、图像处理相关的了，后续还会记录网络穿透如何去做，以及使用 WebRTC 时的一些小功能，比如屏幕录制、图片投屏、白板等这些视频会议常用的功能。
九、Demo
Demo 传送门
