I am using mediasoup for an app built with React Native. Everything works perfectly fine while phones are on wifi. However, the phone do not transfer sound/audio if it is not on wifi. I tried to debug, and it looks like both devices successfuly produce and consume according to methods provided by mediasoup.
any ideas?
No idea without having logs. You need to check the logs in the server or even the client app and see what is happening.
Exactly. mediasoup (server) provides lot of logs and methods (getStats()) and events to check what is happening.
Those methods, events and logs are for the user to inspect. We cannot help if you don’t you it.
@mkh @ibc you guys were right. I tried producer.getStats() and it does not give me anything if I am using data. If I connect to wifi, it starts producing, but if I get disconnected from wifi, the bitrate becomes zero. Any ideas?
You should diagnose UDP/network issues when not using WiFi. That’s the only I can say.
You may also disable UDP in the server app and see if it works with TCP.
this is a log that I get while using LTE to connect via mediasoup.
I also used wireshark to diagnose network, but it looks like there is no problem associated with the nextwork itself. the empty array is the producerStat from a device connected using LTE instead of wifi
and below is a producer created by the device on LTE
the array that describes score is empty.
producer: Producer {
_events: [Object: null prototype] { '@close': [Function] },
_eventsCount: 1,
_maxListeners: Infinity,
_logger: Logger {
_debug: [Function: debug] {
namespace: 'mediasoup:Producer',
enabled: false,
useColors: true,
color: 2,
destroy: [Function: destroy],
extend: [Function: extend],
inspectOpts: {},
log: [Function: bound bound consoleCall]
},
_warn: [Function: debug] {
namespace: 'mediasoup:WARN:Producer',
enabled: false,
useColors: true,
color: 2,
destroy: [Function: destroy],
extend: [Function: extend],
inspectOpts: {},
log: [Function: bound bound consoleCall]
},
_error: [Function: debug] {
namespace: 'mediasoup:ERROR:Producer',
enabled: false,
useColors: true,
color: 4,
destroy: [Function: destroy],
extend: [Function: extend],
inspectOpts: {},
log: [Function: bound bound consoleCall]
}
},
_internal: {
routerId: '7008345f-370d-4626-92f4-c80e4311e1b8',
transportId: 'b4170b51-7cd9-47d4-85c1-26dde8cf9dad',
producerId: 'd6a272f8-1a63-4f31-86fe-b6d56995c0b7'
},
_data: {
kind: 'audio',
rtpParameters: {
mid: null,
codecs: [Array],
headerExtensions: [Array],
encodings: [Array],
rtcp: [Object]
},
type: 'simple',
consumableRtpParameters: {
codecs: [Array],
headerExtensions: [Array],
encodings: [Array],
rtcp: [Object]
}
},
_channel: Channel {
_events: [Object: null prototype] {
'b4170b51-7cd9-47d4-85c1-26dde8cf9dad': [Function],
'68f6f4c3-065f-4b75-bee3-6ac8b7511780': [Function],
'd6a272f8-1a63-4f31-86fe-b6d56995c0b7': [Function],
'2f01875e-4d08-4f35-a1ae-9fab0cf6b898': [Function],
'ef5190c4-dcf5-41ad-8deb-17ea59cfe8a4': [Function],
'83695f2d-2f6f-4c86-85f2-36bec17cc74c': [Function],
'c650486a-1e93-4998-8813-847bf1fa8cf2': [Function],
'1b08adf8-6aec-4cd7-bfaa-03389628be98': [Function]
},
_eventsCount: 8,
_maxListeners: Infinity,
_logger: Logger {
_debug: [Function],
_warn: [Function],
_error: [Function]
},
_workerLogger: Logger {
_debug: [Function],
_warn: [Function],
_error: [Function]
},
_closed: false,
_socket: Socket {
connecting: false,
_hadError: false,
_parent: null,
_host: null,
_readableState: [ReadableState],
readable: true,
_events: [Object: null prototype],
_eventsCount: 4,
_maxListeners: undefined,
_writableState: [WritableState],
writable: false,
allowHalfOpen: false,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: null,
_server: null,
[Symbol(asyncId)]: 9,
[Symbol(kHandle)]: [Pipe],
[Symbol(lastWriteQueueSize)]: 0,
[Symbol(timeout)]: null,
[Symbol(kBytesRead)]: 0,
[Symbol(kBytesWritten)]: 0
},
_nextId: 16,
_sents: Map {},
_recvBuffer: null
},
_closed: false,
_appData: {},
_paused: false,
_score: [],
_observer: EnhancedEventEmitter {
_events: [Object: null prototype] {
close: [Function],
pause: [Function],
resume: [Function]
},
_eventsCount: 3,
_maxListeners: Infinity,
_logger: Logger {
_debug: [Function],
_warn: [Function],
_error: [Function]
}
}
}
Instead of logging JS objects check your connectivity status. If ICE and DTLS does not connect, you’ll get absolutely nothing.
you were right. by checking the connectivity status like you have said, I found out ICE and DTLS do not connect if I try to connect with a device using LTE. do you have any recommendation to solve the problem? or would it be more of a react-native issue than a mediasoup issue?
Please, don’t assume that ICE connectivity issues are bugs in mediasoup-client or react-native. Check your network and check your TURN logs.
Unfortunately, I am not using my own TURN server, so I cannot check my TURN logs.
But I am using a list of STUN and TURN servers that I have been using for a year in order to implement WebRTC connection via mediasoup. so I feel like neither STUN nor TURN servers would be a problem.
I am passing the same iceServers that I have been using as a parameter for device.createSendTransport and device.createRecvTransport. Is there anything else that I should do in the server side in regards to iceServers?
You must do absolutely nothing in mediasoup server side in order to allow TURN. If you are setting TURN servers correctly in mediasoup-client then you should see relay ICE candidates. Just open chrome://webrtc-internals, then reload your web app in Chrome, and check the webrtc-internals. You should see those allocated TURN relay candidates. If you don’t see them I’m afraid I don’t know why, but this is not anything exotic and mediasoup-client does nothing but just bypass those iceServers to the peerconnection constructor.
I am setting TURN servers when I device.createSendTransport() and device.createRecvTransport(). I am building a mobile project with React Native, so I would have to look for another way to check the webrtc-internals to check TURN relay candidates.
But am I setting TURN servers correctly if I pass it as a parameter inside those two methods above?
this.sendTransport = this.device.createSendTransport({
id,
iceParameters,
iceCandidates,
dtlsParameters,
sctpParameters,
iceServers: this.iceServers
});
and where in mediasoup do the peerConnections do addIceCandidate? I couldn’t find it myself
I insist: you MUST NOT call addIceCandidate at all in order for TURN to work.
I don’t know how those this.iceServers look like. Can you show them? (you can replace the user/passwd).
this is what this.iceServers look like.
this.iceServers = [
{
username: 'testUserName',
url: 'turn:sh-turn1.xirsys.com:80?transport=udp',
credential: 'testCredentials'
},
{
username: 'testUserName',
url: 'turn:sh-turn1.xirsys.com:3478?transport=udp',
credential: 'testCredentials'
},
{
username: 'testUserName',
url: 'turn:sh-turn1.xirsys.com:80?transport=tcp',
credential: 'testCredentials'
},
{
username: 'testUserName',
url: 'turn:sh-turn1.xirsys.com:3478?transport=tcp',
credential: 'testCredentials'
},
{
username: 'testUserName',
url: 'turns:sh-turn1.xirsys.com:443?transport=tcp',
credential: 'testCredentials'
},
{
username: 'testUserName',
url: 'turns:sh-turn1.xirsys.com:5349?transport=tcp',
credential: 'testCredentials'
}
]
Below is a debug log from a mediasoup server for the connection made using a device on LTE
mediasoup createWorker() +0ms
mediasoup:Worker constructor() +0ms
mediasoup:Worker spawning worker process: /Users/byeongjinkang/connecting/connecting-media-server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
mediasoup:Channel[pid:88278] constructor() +0ms
mediasoup:Worker worker process running [pid:88278] +19ms
mediasoup:Worker createRouter() +0ms
mediasoup:Channel[pid:88278] request() [method:worker.createRouter, id:1] +16ms
mediasoup:Channel[pid:88278] request succeeded [method:worker.createRouter, id:1] +7ms
mediasoup:Router constructor() +0ms
mediasoup:Router createWebRtcTransport() +6s
mediasoup:Channel[pid:88278] request() [method:router.createWebRtcTransport, id:2] +6s
mediasoup:Channel[pid:88278] request succeeded [method:router.createWebRtcTransport, id:2] +1ms
mediasoup:Transport constructor() +0ms
mediasoup:WebRtcTransport constructor() +0ms
mediasoup:Router createWebRtcTransport() +40ms
mediasoup:Channel[pid:88278] request() [method:router.createWebRtcTransport, id:3] +39ms
mediasoup:Channel[pid:88278] request succeeded [method:router.createWebRtcTransport, id:3] +1ms
mediasoup:Transport constructor() +40ms
mediasoup:WebRtcTransport constructor() +40ms
mediasoup:WebRtcTransport connect() +32ms
mediasoup:Channel[pid:88278] request() [method:transport.connect, id:4] +32ms
mediasoup:Channel[pid:88278] request succeeded [method:transport.connect, id:4] +0ms
mediasoup:Transport produce() +160ms
mediasoup:Channel[pid:88278] request() [method:transport.produce, id:5] +129ms
mediasoup:Channel[pid:88278] request succeeded [method:transport.produce, id:5] +1ms
mediasoup:Producer constructor() +0ms
mediasoup:Router createWebRtcTransport() +1s
mediasoup:Channel[pid:88278] request() [method:router.createWebRtcTransport, id:6] +860ms
mediasoup:Channel[pid:88278] request succeeded [method:router.createWebRtcTransport, id:6] +1ms
mediasoup:Transport constructor() +863ms
mediasoup:WebRtcTransport constructor() +991ms
mediasoup:Router createWebRtcTransport() +12ms
mediasoup:Channel[pid:88278] request() [method:router.createWebRtcTransport, id:7] +11ms
mediasoup:Channel[pid:88278] request succeeded [method:router.createWebRtcTransport, id:7] +1ms
mediasoup:Transport constructor() +12ms
mediasoup:WebRtcTransport constructor() +12ms
mediasoup:WebRtcTransport connect() +6ms
mediasoup:Channel[pid:88278] request() [method:transport.connect, id:8] +6ms
mediasoup:Channel[pid:88278] request succeeded [method:transport.connect, id:8] +0ms
mediasoup:Transport consume() +22ms
mediasoup:Channel[pid:88278] request() [method:transport.consume, id:9] +16ms
mediasoup:Channel[pid:88278] request succeeded [method:transport.consume, id:9] +1ms
mediasoup:Consumer constructor() +0ms
mediasoup:Transport produce() +12ms
mediasoup:Channel[pid:88278] request() [method:transport.produce, id:10] +11ms
mediasoup:Channel[pid:88278] request succeeded [method:transport.produce, id:10] +0ms
mediasoup:Producer constructor() +908ms
mediasoup:Transport consume() +1ms
mediasoup:Channel[pid:88278] request() [method:transport.consume, id:11] +1ms
mediasoup:Channel[pid:88278] request succeeded [method:transport.consume, id:11] +1ms
mediasoup:Consumer constructor() +13ms
mediasoup:WebRtcTransport connect() +38ms
mediasoup:Channel[pid:88278] request() [method:transport.connect, id:12] +8ms
mediasoup:Channel[pid:88278] request succeeded [method:transport.connect, id:12] +0ms
mediasoup:WebRtcTransport connect() +125ms
mediasoup:Channel[pid:88278] request() [method:transport.connect, id:13] +125ms
mediasoup:Channel[pid:88278] request succeeded [method:transport.connect, id:13] +0ms
mediasoup:Consumer getStats() +1s
mediasoup:Channel[pid:88278] request() [method:consumer.getStats, id:14] +1s
mediasoup:Channel[pid:88278] request succeeded [method:consumer.getStats, id:14] +1ms
mediasoup:Consumer getStats() +9ms
mediasoup:Channel[pid:88278] request() [method:consumer.getStats, id:15] +8ms
mediasoup:Channel[pid:88278] request succeeded [method:consumer.getStats, id:15] +1ms
mediasoup:Producer getStats() +4s
mediasoup:Channel[pid:88278] request() [method:producer.getStats, id:16] +3s
mediasoup:Channel[pid:88278] request succeeded [method:producer.getStats, id:16] +1ms
mediasoup:Producer getStats() +907ms
mediasoup:Channel[pid:88278] request() [method:producer.getStats, id:17] +906ms
mediasoup:Channel[pid:88278] request succeeded [method:producer.getStats, id:17] +1ms
mediasoup:WARN:worker[pid:88278] RTC::DtlsTransport::OnSslInfo() | received DTLS warning alert: close notify +0ms
mediasoup:WARN:worker[pid:88278] RTC::DtlsTransport::Reset() | resetting DTLS transport +0ms
mediasoup:WARN:worker[pid:88278] RTC::WebRtcTransport::OnDtlsTransportClosed() | DTLS remotely closed +0ms
webRTC Transport dtlsState: closed 6UkOL3o4cBTKYQyJAAAB
ICE state changed to disconnected 6UkOL3o4cBTKYQyJAAAB
mediasoup:WARN:worker[pid:88278] RTC::DtlsTransport::OnSslInfo() | received DTLS warning alert: close notify +9ms
mediasoup:WARN:worker[pid:88278] RTC::DtlsTransport::Reset() | resetting DTLS transport +0ms
mediasoup:WARN:worker[pid:88278] RTC::WebRtcTransport::OnDtlsTransportClosed() | DTLS remotely closed +0ms
Those logs provide zero useful information. You may insist, but I said many times that mediasoup server is NOT aware of TURN settings and TURN usage in client side.
The spec clearly says that urls array must be used:
Anyway, this is about how react-native-webrtc handles those iceServers. It seems that it accepts the deprecated url string:
However, this is becoming much more a question about react-native-webrtc and TURN. I’m afraid I cannot help much more.