Multiple consumer video streams in a single receiving transport

mediasoup libraries
1

Hello,

I am in the process of porting Mediasoup client library to Dart to be used with Flutter and have encountered a problem when using multiple consumer video streams in a single receiving transport.
The first stream works fine and I can see its contents on the client side without issues.
However, when trying to create a second consumer over an existing receiving transport (it already has one consumer running), I don’t get any onAddTrack/onAddStream events on the client side.
It could be an issue with SDPs so here are SDPs:

Offer SDP for the first stream:

v=0
o=mediasoup-client 10000 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-lite
a=fingerprint:sha-512 37:3E:AD:8C:C6:E5:9F:CC:AF:12:BF:04:BE:95:2D:42:E3:52:A3:5D:42:B0:03:A9:D1:63:2C:E4:EC:29:30:C8:76:61:AA:FD:DF:EF:D8:BA:7F:B8:8B:19:2E:D1:D7:B1:35:4E:D8:6B:A6:93:7F:4B:16:74:8D:D8:3D:4C:0D:CD
a=msid-semantic: WMS *
a=group:BUNDLE video
m=video 7 UDP/TLS/RTP/SAVPF 101 102
c=IN IP4 127.0.0.1
a=rtpmap:101 VP8/90000
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 docs/native-code/rtp-hdrext/abs-send-time - src - Git at Google
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 draft-ietf-avtext-framemarking-07
a=extmap:11 urn:3gpp:video-orientation
a=extmap:12 urn:ietf:params:rtp-hdrext:toffset
a=setup:actpass
a=mid:video
a=msid:J74TmQMqZwjuRgTv 5fa96216-2ee6-451a-96fb-37a0376081e6
a=sendonly
a=ice-ufrag:v5ec6gnwlyh5p8q1
a=ice-pwd:ouvhzto5enmaexnuln0iwqx63c7ffe47
a=candidate:udpcandidate 1 udp 1076558079 10.0.1.3 10069 typ host
a=candidate:tcpcandidate 1 tcp 1076302079 10.0.1.3 10034 typ host tcptype passive
a=end-of-candidates
a=ice-options:renomination
a=ssrc:584877372 cname:J74TmQMqZwjuRgTv
a=ssrc:775741059 cname:J74TmQMqZwjuRgTv
a=ssrc:556569882 cname:372tS55XZEEKrdqQ
a=ssrc:556569882 msid:372tS55XZEEKrdqQ 00b6b0ae-72f2-4505-bcc6-a0e228210597
a=ssrc:723251905 cname:372tS55XZEEKrdqQ
a=ssrc:723251905 msid:372tS55XZEEKrdqQ 00b6b0ae-72f2-4505-bcc6-a0e228210597
a=ssrc-group:FID 584877372 775741059
a=ssrc-group:FID 556569882 723251905
a=rtcp-mux
a=rtcp-rsize

Offer SDP for the second stream:

v=0
o=mediasoup-client 10000 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-lite
a=fingerprint:sha-512 37:3E:AD:8C:C6:E5:9F:CC:AF:12:BF:04:BE:95:2D:42:E3:52:A3:5D:42:B0:03:A9:D1:63:2C:E4:EC:29:30:C8:76:61:AA:FD:DF:EF:D8:BA:7F:B8:8B:19:2E:D1:D7:B1:35:4E:D8:6B:A6:93:7F:4B:16:74:8D:D8:3D:4C:0D:CD
a=msid-semantic: WMS *
a=group:BUNDLE video
m=video 7 UDP/TLS/RTP/SAVPF 101 102
c=IN IP4 127.0.0.1
a=rtpmap:101 VP8/90000
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 docs/native-code/rtp-hdrext/abs-send-time - src - Git at Google
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 draft-ietf-avtext-framemarking-07
a=extmap:11 urn:3gpp:video-orientation
a=extmap:12 urn:ietf:params:rtp-hdrext:toffset
a=setup:actpass
a=mid:video
a=msid:J74TmQMqZwjuRgTv 5fa96216-2ee6-451a-96fb-37a0376081e6
a=sendonly
a=ice-ufrag:v5ec6gnwlyh5p8q1
a=ice-pwd:ouvhzto5enmaexnuln0iwqx63c7ffe47
a=candidate:udpcandidate 1 udp 1076558079 10.0.1.3 10069 typ host
a=candidate:tcpcandidate 1 tcp 1076302079 10.0.1.3 10034 typ host tcptype passive
a=end-of-candidates
a=ice-options:renomination
a=ssrc:584877372 cname:J74TmQMqZwjuRgTv
a=ssrc:775741059 cname:J74TmQMqZwjuRgTv
a=ssrc:556569882 cname:372tS55XZEEKrdqQ
a=ssrc:556569882 msid:372tS55XZEEKrdqQ 00b6b0ae-72f2-4505-bcc6-a0e228210597
a=ssrc:723251905 cname:372tS55XZEEKrdqQ
a=ssrc:723251905 msid:372tS55XZEEKrdqQ 00b6b0ae-72f2-4505-bcc6-a0e228210597
a=ssrc-group:FID 584877372 775741059
a=ssrc-group:FID 556569882 723251905
a=rtcp-mux
a=rtcp-rsize

Answer SDP for the first stream:

v=0
o=- 3666918098559640824 2 IN IP4 127.0.0.1
s=-
t=0 0
a=msid-semantic: WMS
a=group:BUNDLE video
a=null
m=video 9 UDP/TLS/RTP/SAVPF 101 102
c=IN IP4 0.0.0.0
a=rtpmap:101 VP8/90000
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=101
a=rtcp:9 IN IP4 0.0.0.0
a=extmap:12 urn:ietf:params:rtp-hdrext:toffset
a=extmap:4 docs/native-code/rtp-hdrext/abs-send-time - src - Git at Google
a=extmap:11 urn:3gpp:video-orientation
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 draft-ietf-avtext-framemarking-07
a=setup:active
a=mid:video
a=recvonly
a=ice-ufrag:MGvr
a=ice-pwd:wAaTa41gl7XCj8ALK20OyA36
a=fingerprint:sha-256 E7:C0:A1:7A:47:5D:DE:B9:05:AE:49:12:F4:83:5E:6D:EB:2E:B0:8C:0A:2B:FC:9A:64:17:CB:AC:26:75:9B:F1
a=ice-options:trickle
a=rtcp-mux
a=rtcp-rsize

Answer SDP for the second stream:

v=0
o=- 3666918098559640824 3 IN IP4 127.0.0.1
s=-
t=0 0
a=msid-semantic: WMS
a=group:BUNDLE video
a=null
m=video 51062 UDP/TLS/RTP/SAVPF 101 102
c=IN IP4 10.0.2.16
a=rtpmap:101 VP8/90000
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=101
a=rtcp:9 IN IP4 0.0.0.0
a=extmap:12 urn:ietf:params:rtp-hdrext:toffset
a=extmap:4 docs/native-code/rtp-hdrext/abs-send-time - src - Git at Google
a=extmap:11 urn:3gpp:video-orientation
a=extmap:5 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:6 draft-ietf-avtext-framemarking-07
a=setup:active
a=mid:video
a=recvonly
a=ice-ufrag:MGvr
a=ice-pwd:wAaTa41gl7XCj8ALK20OyA36
a=fingerprint:sha-256 E7:C0:A1:7A:47:5D:DE:B9:05:AE:49:12:F4:83:5E:6D:EB:2E:B0:8C:0A:2B:FC:9A:64:17:CB:AC:26:75:9B:F1
a=candidate:841689039 1 udp 2122260223 10.0.2.16 51062 typ host generation 0 network-id 5 network-cost 10
a=candidate:361502126 1 udp 2122187263 fec0::15:b2ff:fe00:0 38221 typ host generation 0 network-id 6 network-cost 10
a=candidate:841689039 1 udp 2122129151 10.0.2.16 57793 typ host generation 0 network-id 3 network-cost 900
a=candidate:361502126 1 udp 2122056191 fec0::15:b2ff:fe00:0 33665 typ host generation 0 network-id 4 network-cost 900
a=candidate:559267639 1 udp 2122005759 ::1 35839 typ host generation 0 network-id 2
a=candidate:1510613869 1 udp 2121932543 127.0.0.1 55056 typ host generation 0 network-id 1
a=ice-options:trickle
a=rtcp-mux
a=rtcp-rsize

One more possible cause: I can see the following WebRTC log line after trying to add remote SDP for the second stream:

I/peer_connection.cc(12331): (line 5567): Non-rejected SCTP m= section is needed to get the SSL Role of the SCTP transport.

Also, when enabling verbose WebRTC logging I can see another suspicious after trying to set Offer SDP for the second stream:

I/stats_collector.cc(13248): (line 396): Missing track ID for recv SSRC=556569882

Any help about why onAddTrack/onAddStream events are not generated will be much appreciated.

Thank you,
Leo

2

Some more log data:
During the setRemoteDescription() call for the first stream I see AddRecvStream being called:

I/webrtc_video_engine.cc(14844): (line 1190): AddRecvStream: {id:05792a6c-5f37-4e63-a6cf-95184d566fca;ssrcs:[849267223,190852275];ssrc_groups:{semantics:FID;ssrcs:[849267223,190852275]};cname:M7J/qIfucYSYjBfB;stream_ids:M7J/qIfucYSYjBfB;}
…
I/webrtc_video_engine.cc(14844): (line 1320): SetSink: ssrc:849267223 (ptr)

However, during setRemoteDescription() call for the second stream I see that there is no AddRecvStream call, but I see it a bit later in the log, and it appears there is no event handler supplied:

I/webrtc_video_engine.cc(14844): (line 1190): AddRecvStream (default stream): {ssrcs:[521246255];ssrc_groups:;stream_ids:;}
…
I/webrtc_video_engine.cc(14844): (line 1320): SetSink: ssrc:521246255 nullptr

Any ideas please?

Thank you,
Leo

3

If you are really porting mediasoup-client to Dart, why are you listening to those events? We don’t do that in mediasoup-client for a reason.

BTW you are using Plan-B instead of Unified-Plan. You should use Unified-Plan.

4

I know - that’s in my plans, but current Flutter’s WebRTC wrapper does not support unified-plan - its in the works. So for now I was going to get by using Plan B. Can I expect Mediasoup to support multiple streams (of the same kind) on a single transport when using Plan B?

5

It does, yes. Check the Chrome65 handler in mediasoup-client. And avoid onstream or ontrack.