Hi, I am using react-native-webrtc 124.0.3 and mediasoup-client 3.7.12,
I am unable to render the remote stream received over a recv transport using RTCView. I just get a black area or empty area. I saw this issue but I’m not sure if i can implement the workaround mentioned there because I don’t get mediastream generated by react-native-webrtc. If anybody has experienced this, please help. thanks
code to create recv transport and remote stream
const routerRtpCapabilities: any = await socketRequest(mediaSocket as Socket, "getRouterRTPCapabilities", {thread: route.params.thread})
const device = new Device();
setDevice(device);
await device.load({routerRtpCapabilities: routerRtpCapabilities.capabilities});
// Create receive transport on server side
const recvResp: any = await socketRequest(mediaSocket as Socket, "createTransport", {threadId: route.params.thread.id, sctpCapabilities: device.sctpCapabilities, direction: 'recv'})
// now create it locally on client side
const recvTransport = device.createRecvTransport(recvResp);
recvTransport.on("connectionstatechange", (state) => console.log("connectionstatechange", state));
// Store it in state
setRecvTransport(recvTransport)
// request all producers available in server and consume them.
const producers = await socketRequest(mediaSocket as Socket, "getProducers", {threadId: route.params.thread.id, userId: user?.id});
console.log("GOT PRODUCERS");
// console.log("PRODUCERS", producers);
(producers as any).forEach(async (producerId: string) => {
// create consumer on server side
const result = await socketRequest(mediaSocket as Socket, "consume", {threadId: route.params.thread.id, rtpCapabilities: device.rtpCapabilities, transportId: recvTransport.id, producerId, userId: user?.id})
// consume it on client side
const { id, kind, rtpParameters } = result as any;
const localConsumer = await recvTransport.consume({
id,
producerId,
kind,
rtpParameters,
});
// resume the consumer on server side
await socketRequest(mediaSocket as Socket, "resumeConsumer", {consumerId: id, threadId: route.params.thread.id });
localConsumer.resume()
const stream = new MediaStream;
stream.addTrack(localConsumer.track);
setRemoteStream(stream);
setRemoteURL(stream.toURL())
});
Client side producer stats after some time, where bytessent and all other such stats are always zero
Map {"CFD8:7F:37:4E:F2:92:00:40:71:9F:95:49:42:0E:A2:78:37:96:ED:82:85:ED:5A:96:E9:BB:BB:DA:33:18:0E:17" => {"base64Certificate": "MIIBFjCBvKADAgECAghEC0JXtbWmRzAKBggqhkjOPQQDAjARMQ8wDQYDVQQDDAZXZWJSVEMwHhcNMjQwNzI2MDc0NzUyWhcNMjQwODI2MDc0NzUyWjARMQ8wDQYDVQQDDAZXZWJSVEMwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARvSe8w2u3vDBKzbFqV9LMUDwY52qkXcKQGDLzt4SEzlKMgzpL/GOkX+O7Z3XKB8BWV73WEDO81NM+TdNYiK+1fMAoGCCqGSM49BAMCA0kAMEYCIQD4JhErVmRGxrfwi1x5vwDyvqfTVyVU4ndAP8PHYsPYEgIhAJRz09X4WXtjM/aLrNtjgf2HCDQEVx51eDGJS8UOq3Zv", "fingerprint": "D8:7F:37:4E:F2:92:00:40:71:9F:95:49:42:0E:A2:78:37:96:ED:82:85:ED:5A:96:E9:BB:BB:DA:33:18:0E:17", "fingerprintAlgorithm": "sha-256", "id": "CFD8:7F:37:4E:F2:92:00:40:71:9F:95:49:42:0E:A2:78:37:96:ED:82:85:ED:5A:96:E9:BB:BB:DA:33:18:0E:17", "timestamp": 1722066592798.311, "type": "certificate"}, "COT01_98_x-google-start-bitrate=1000" => {"clockRate": 90000, "id": "COT01_98_x-google-start-bitrate=1000", "mimeType": "video/VP8", "payloadType": 98, "sdpFmtpLine": "x-google-start-bitrate=1000", "timestamp": 1722066592798.311, "transportId": "T01", "type": "codec"}, "OT01V1935154532" => {"active": true, "bytesSent": 0, "codecId": "COT01_98_x-google-start-bitrate=1000", "firCount": 0, "framesEncoded": 0, "framesSent": 0, "headerBytesSent": 0, "hugeFramesSent": 0, "id": "OT01V1935154532", "keyFramesEncoded": 0, "kind": "video", "mediaSourceId": "SV8", "mid": "0", "nackCount": 0, "packetsSent": 0, "pliCount": 0, "qualityLimitationDurations": {"bandwidth": 0, "cpu": 0, "none": 120.026, "other": 0}, "qualityLimitationReason": "none", "qualityLimitationResolutionChanges": 0, "retransmittedBytesSent": 0, "retransmittedPacketsSent": 0, "rid": "r1", "rtxSsrc": 1158969486, "ssrc": 1935154532, "timestamp": 1722066592798.311, "totalEncodeTime": 0, "totalEncodedBytesTarget": 0, "totalPacketSendDelay": 0, "transportId": "T01", "type": "outbound-rtp"}, "OT01V1954824195" => {"active": true, "bytesSent": 0, "codecId": "COT01_98_x-google-start-bitrate=1000", "firCount": 0, "framesEncoded": 0, "framesSent": 0, "headerBytesSent": 0, "hugeFramesSent": 0, "id": "OT01V1954824195", "keyFramesEncoded": 0, "kind": "video", "mediaSourceId": "SV8", "mid": "0", "nackCount": 0, "packetsSent": 0, "pliCount": 0, "qualityLimitationDurations": {"bandwidth": 0, "cpu": 0, "none": 120.026, "other": 0}, "qualityLimitationReason": "none", "qualityLimitationResolutionChanges": 0, "retransmittedBytesSent": 0, "retransmittedPacketsSent": 0, "rid": "r2", "rtxSsrc": 1631519724, "ssrc": 1954824195, "timestamp": 1722066592798.311, "totalEncodeTime": 0, "totalEncodedBytesTarget": 0, "totalPacketSendDelay": 0, "transportId": "T01", "type": "outbound-rtp"}, "OT01V3353070453" => {"active": true, "bytesSent": 0, "codecId": "COT01_98_x-google-start-bitrate=1000", "firCount": 0, "framesEncoded": 0, "framesSent": 0, "headerBytesSent": 0, "hugeFramesSent": 0, "id": "OT01V3353070453", "keyFramesEncoded": 0, "kind": "video", "mediaSourceId": "SV8", "mid": "0", "nackCount": 0, "packetsSent": 0, "pliCount": 0, "qualityLimitationDurations": {"bandwidth": 0, "cpu": 0, "none": 120.026, "other": 0}, "qualityLimitationReason": "none", "qualityLimitationResolutionChanges": 0, "retransmittedBytesSent": 0, "retransmittedPacketsSent": 0, "rid": "r0", "rtxSsrc": 3051517795, "ssrc": 3353070453, "timestamp": 1722066592798.311, "totalEncodeTime": 0, "totalEncodedBytesTarget": 0, "totalPacketSendDelay": 0, "transportId": "T01", "type": "outbound-rtp"}, "SV8" => {"frames": 1359, "framesPerSecond": 11, "height": 1280, "id": "SV8", "kind": "video", "timestamp": 1722066592798.311, "trackIdentifier": "cf357ccc-9bb7-463a-8685-1febd92a9562", "type": "media-source", "width": 720}, "T01" => {"bytesReceived": 0, "bytesSent": 0, "dtlsRole": "client", "dtlsState": "new", "iceLocalUsernameFragment": "ViJV", "iceRole": "controlling", "iceState": "new", "id": "T01", "localCertificateId": "CFD8:7F:37:4E:F2:92:00:40:71:9F:95:49:42:0E:A2:78:37:96:ED:82:85:ED:5A:96:E9:BB:BB:DA:33:18:0E:17", "packetsReceived": 0, "packetsSent": 0, "selectedCandidatePairChanges": 0, "timestamp": 1722066592798.311, "type": "transport"}}
the track i receive from server looks like this
{"_constraints": {}, "_enabled": true, "_muted": false, "_peerConnectionId": 4, "_readyState": "live", "_settings": {}, "id": "85a4e6d7-044b-4c40-8977-90dac7f2d20a", "kind": "video", "label": "", "remote": true}
Here are the server logs
mediasoup createWorker() +10ms
mediasoup:Worker constructor() +1ms
mediasoup:Worker spawning worker process: /..../mediaserver/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=debug --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=10000 --rtcMaxPort=10100 +0ms
mediasoup:Channel constructor() +0ms
mediasoup:Channel [pid:84928] mediasoup-worker::mediasoup_worker_run() | starting mediasoup-worker process [version:3.14.8] +5ms
mediasoup:Channel [pid:84928] mediasoup-worker::mediasoup_worker_run() | little-endian CPU detected +0ms
mediasoup:Channel [pid:84928] mediasoup-worker::mediasoup_worker_run() | 64 bits architecture detected +0ms
mediasoup:Channel [pid:84928] Settings::PrintConfiguration() | <configuration> +0ms
mediasoup:Channel [pid:84928] Settings::PrintConfiguration() | logLevel: debug +0ms
mediasoup:Channel [pid:84928] Settings::PrintConfiguration() | logTags: info,ice,dtls,rtp,srtp,rtcp,rtx,bwe,score,simulcast,svc,sctp +0ms
mediasoup:Channel [pid:84928] Settings::PrintConfiguration() | rtcMinPort: 10000 +0ms
mediasoup:Channel [pid:84928] Settings::PrintConfiguration() | rtcMaxPort: 10100 +1ms
mediasoup:Channel [pid:84928] Settings::PrintConfiguration() | libwebrtcFieldTrials: WebRTC-Bwe-AlrLimitedBackoff/Enabled/ +0ms
mediasoup:Channel [pid:84928] Settings::PrintConfiguration() | </configuration> +0ms
mediasoup:Channel [pid:84928] DepLibUV::PrintVersion() | libuv version: "1.48.0" +0ms
mediasoup:Channel [pid:84928] DepOpenSSL::operator()() | openssl version: "OpenSSL 3.0.8 7 Feb 2023" +0ms
mediasoup:Channel [pid:84928] DepOpenSSL::operator()() | openssl CPU info: "CPUINFO: OPENSSL_armcap=0x7d" +0ms
mediasoup:Channel [pid:84928] DepLibSRTP::ClassInit() | libsrtp version: "libsrtp2 3.0.0" +0ms
mediasoup:Channel [pid:84928] DepUsrSCTP::ClassInit() | usrsctp +1ms
mediasoup:Channel [pid:84928] DepLibWebRTC::ClassInit() | libwebrtc field trials: "WebRTC-Bwe-AlrLimitedBackoff/Enabled/" +1ms
mediasoup:Channel [pid:84928] webrtc::FieldTrial::InitFieldTrialsFromString() | Setting field trial string: WebRTC-Bwe-AlrLimitedBackoff/Enabled/ +0ms
mediasoup:Channel [pid:84928] RTC::DtlsTransport::CreateSslCtx() | setting SRTP cryptoSuites for DTLS: SRTP_AEAD_AES_256_GCM:SRTP_AEAD_AES_128_GCM:SRTP_AES128_CM_SHA1_80:SRTP_AES128_CM_SHA1_32 +2ms
mediasoup:Channel [pid:84928] RTC::DtlsTransport::GenerateFingerprints() | sha-224 fingerprint: 3B:AD:03:CD:99:F9:DB:9A:0F:3F:49:3A:53:D0:BA:2D:34:0C:A5:36:63:7B:48:E7:99:C2:BC:36 +0ms
mediasoup:Channel [pid:84928] RTC::DtlsTransport::GenerateFingerprints() | sha-384 fingerprint: 89:67:C1:E2:D2:3C:15:86:14:FC:6C:65:23:30:33:58:67:9C:26:0F:21:68:36:C3:E4:A9:96:BC:53:A3:93:2D:91:1D:FC:D0:E7:7B:29:D7:3F:FE:65:5F:F1:6B:03:D0 +0ms
mediasoup:Channel [pid:84928] RTC::DtlsTransport::GenerateFingerprints() | sha-256 fingerprint: 33:FF:12:4A:87:21:B1:B6:20:5C:08:2E:35:00:AA:CA:FA:9E:7E:E4:EA:DB:B0:4B:7A:06:7F:4E:FC:F4:F4:CB +0ms
mediasoup:Channel [pid:84928] RTC::DtlsTransport::GenerateFingerprints() | sha-1 fingerprint: 5F:E0:56:F0:8B:D1:4F:8E:51:F3:7B:AD:F0:1F:2E:CD:89:F2:AD:32 +0ms
mediasoup:Channel [pid:84928] RTC::DtlsTransport::GenerateFingerprints() | sha-512 fingerprint: 42:D1:AA:EE:CE:CB:E1:82:F5:59:ED:EA:8D:20:D5:56:DE:FE:B2:8B:93:90:01:04:28:D1:87:56:80:D7:81:A7:FC:68:33:CB:75:62:DA:EB:3B:7F:7D:7B:18:0F:23:0C:38:38:EB:84:AE:5C:4D:08:0D:AD:AE:4A:92:6F:BE:98 +0ms
mediasoup:Worker worker process running [pid:84928] +10ms
mediasoup:Worker createWebRtcServer() +1ms
mediasoup:Channel request() [method:WORKER_CREATE_WEBRTCSERVER] +1ms
mediasoup:Channel [pid:84928] RTC::WebRtcServer::WebRtcServer() | UDP socket send buffer size: 9216, recv buffer size: 786896 +0ms
mediasoup:Channel [pid:84928] RTC::WebRtcServer::WebRtcServer() | TCP server send buffer size: 131072, recv buffer size: 131072 +0ms
mediasoup:Channel request succeeded [method:WORKER_CREATE_WEBRTCSERVER, id:1] +0ms
mediasoup:WebRtcServer constructor() +11ms
mediasoup:Worker createRouter() +35s
mediasoup:Channel request() [method:WORKER_CREATE_ROUTER] +35s
mediasoup:Channel request succeeded [method:WORKER_CREATE_ROUTER, id:2] +0ms
mediasoup:Router constructor() +0ms
mediasoup:Router createWebRtcTransport() +54ms
mediasoup:Channel request() [method:ROUTER_CREATE_WEBRTCTRANSPORT_WITH_SERVER] +56ms
mediasoup:Channel [pid:84915] DepUsrSCTP::Start() | usrsctp periodic check started +0ms
mediasoup:Channel request succeeded [method:ROUTER_CREATE_WEBRTCTRANSPORT_WITH_SERVER, id:3] +0ms
mediasoup:Transport constructor() +0ms
mediasoup:WebRtcTransport constructor() +0ms
setting listener 945bfa3a-69ed-4f40-a0a0-ce1bced9d814
mediasoup:Router createWebRtcTransport() +15ms
mediasoup:Channel request() [method:ROUTER_CREATE_WEBRTCTRANSPORT_WITH_SERVER] +15ms
mediasoup:Channel request succeeded [method:ROUTER_CREATE_WEBRTCTRANSPORT_WITH_SERVER, id:4] +0ms
mediasoup:Transport constructor() +14ms
mediasoup:WebRtcTransport constructor() +15ms
setting listener b8847651-9d37-4832-b00c-64e36c76ec83
connecting: b8847651-9d37-4832-b00c-64e36c76ec83
mediasoup:WebRtcTransport connect() +45ms
mediasoup:Channel request() [method:WEBRTCTRANSPORT_CONNECT] +46ms
mediasoup:Channel request succeeded [method:WEBRTCTRANSPORT_CONNECT, id:5] +0ms
mediasoup:Transport produce() +130ms
mediasoup:Channel request() [method:TRANSPORT_PRODUCE] +87ms
mediasoup:Channel request succeeded [method:TRANSPORT_PRODUCE, id:6] +1ms
mediasoup:Channel [pid:84915] RTC::Transport::HandleRequest() | enabling TransportCongestionControlServer with transport-cc +0ms
mediasoup:Producer constructor() +0ms
mediasoup:Router createWebRtcTransport() +4s
mediasoup:Channel request() [method:ROUTER_CREATE_WEBRTCTRANSPORT_WITH_SERVER] +3s
mediasoup:Channel request succeeded [method:ROUTER_CREATE_WEBRTCTRANSPORT_WITH_SERVER, id:7] +0ms
mediasoup:Transport constructor() +3s
mediasoup:WebRtcTransport constructor() +4s
setting listener 028ec2ac-7ef9-411e-b3e8-03d49494e5e6
mediasoup:Transport consume() +15ms
mediasoup:Channel request() [method:TRANSPORT_CONSUME] +18ms
mediasoup:Router createWebRtcTransport() +19ms
mediasoup:Channel request() [method:ROUTER_CREATE_WEBRTCTRANSPORT_WITH_SERVER] +0ms
mediasoup:Channel [pid:84915] RTC::SimulcastConsumer::CreateRtpStream() | [ssrc:548572422, payloadType:101] +0ms
mediasoup:Channel [pid:84915] RTC::SimulcastConsumer::CreateRtpStream() | FIR supported +0ms
mediasoup:Channel [pid:84915] RTC::SimulcastConsumer::CreateRtpStream() | NACK supported +0ms
mediasoup:Channel [pid:84915] RTC::SimulcastConsumer::CreateRtpStream() | PLI supported +0ms
mediasoup:Channel request succeeded [method:TRANSPORT_CONSUME, id:8] +0ms
mediasoup:Channel [pid:84915] RTC::Transport::HandleRequest() | enabling TransportCongestionControlClient with transport-cc +0ms
mediasoup:Consumer constructor() +0ms
mediasoup:Consumer setPreferredLayers() +0ms
mediasoup:Channel request() [method:CONSUMER_SET_PREFERRED_LAYERS] +1ms
mediasoup:Channel request succeeded [method:ROUTER_CREATE_WEBRTCTRANSPORT_WITH_SERVER, id:9] +0ms
mediasoup:Transport constructor() +2ms
mediasoup:WebRtcTransport constructor() +16ms
setting listener 6c1ed1d8-5fe7-469b-862a-4e6c4efd9200
mediasoup:Channel request succeeded [method:CONSUMER_SET_PREFERRED_LAYERS, id:10] +1ms
connecting: 6c1ed1d8-5fe7-469b-862a-4e6c4efd9200
mediasoup:WebRtcTransport connect() +59ms
mediasoup:Channel request() [method:WEBRTCTRANSPORT_CONNECT] +58ms
connecting: 028ec2ac-7ef9-411e-b3e8-03d49494e5e6
mediasoup:WebRtcTransport connect() +0ms
mediasoup:Channel request() [method:WEBRTCTRANSPORT_CONNECT] +0ms
mediasoup:Channel request succeeded [method:WEBRTCTRANSPORT_CONNECT, id:11] +0ms
mediasoup:Channel request succeeded [method:WEBRTCTRANSPORT_CONNECT, id:12] +0ms
mediasoup:Consumer resume() +125ms
mediasoup:Channel request() [method:CONSUMER_RESUME] +67ms
mediasoup:Channel request succeeded [method:CONSUMER_RESUME, id:13] +0ms
Consumer b2682f72-2541-4952-903b-068e18d7ab36 resumed
mediasoup:Transport produce() +141ms
mediasoup:Channel request() [method:TRANSPORT_PRODUCE] +16ms
mediasoup:Channel request succeeded [method:TRANSPORT_PRODUCE, id:14] +0ms
mediasoup:Channel [pid:84915] RTC::Transport::HandleRequest() | enabling TransportCongestionControlServer with transport-cc +0ms
mediasoup:Producer constructor() +4s
mediasoup:Transport consume() +4ms
mediasoup:Channel request() [method:TRANSPORT_CONSUME] +4ms
mediasoup:Channel [pid:84915] RTC::SimulcastConsumer::CreateRtpStream() | [ssrc:552480042, payloadType:101] +0ms
mediasoup:Channel [pid:84915] RTC::SimulcastConsumer::CreateRtpStream() | FIR supported +0ms
mediasoup:Channel [pid:84915] RTC::SimulcastConsumer::CreateRtpStream() | NACK supported +0ms
mediasoup:Channel [pid:84915] RTC::SimulcastConsumer::CreateRtpStream() | PLI supported +0ms
mediasoup:Channel request succeeded [method:TRANSPORT_CONSUME, id:15] +1ms
mediasoup:Channel [pid:84915] RTC::Transport::HandleRequest() | enabling TransportCongestionControlClient with transport-cc +0ms
mediasoup:Consumer constructor() +22ms
mediasoup:Consumer setPreferredLayers() +0ms
mediasoup:Channel request() [method:CONSUMER_SET_PREFERRED_LAYERS] +0ms
mediasoup:Channel request succeeded [method:CONSUMER_SET_PREFERRED_LAYERS, id:16] +0ms
connecting: 945bfa3a-69ed-4f40-a0a0-ce1bced9d814
mediasoup:WebRtcTransport connect() +115ms
mediasoup:Channel request() [method:WEBRTCTRANSPORT_CONNECT] +27ms
mediasoup:Channel request succeeded [method:WEBRTCTRANSPORT_CONNECT, id:17] +1ms
mediasoup:Consumer resume() +37ms
mediasoup:Channel request() [method:CONSUMER_RESUME] +9ms
mediasoup:Channel request succeeded [method:CONSUMER_RESUME, id:18] +0ms
Consumer caa48f96-add8-47cd-bb21-3001052ddf63 resumed