Hi, i would like to use the directtransport functionality for some processing and i need to synchronize all the producer streams in order to know their wallclock relation. I was hoping to do this using RTCP sender report and use the rtp receive time until then. However the ‘rtcp’ event does not seem to be firing periodically even when multiple producers/consumers exist.
Is it possible that the RTCP event only fires upon injecting RTCP and not on incoming producer RTCP?
- You create a DirectTransport.
- You create a Producer in that DirectTransport via directTransport.produce().
- You call producer.send(rtpPacket).
- You now can listen for “rtcp” event in the DirectTransport.
Not sure if that’s what you are doing.
Thank you, however i am not trying to inject RTCP, but capture it in node the same way i can capture producer RTP with DirectTransport consumers. So i was looking for the equivalent of consmer.on(‘rtp’) but for RTCP. Does anything like this exist? If not, does mediasoup provide any api to get a producers sender report for stream synchronization in node?
I didn’t say that you need to inject RTCP.
No. mediasoup forwards RTP but it consumes and produces RTCP. It’s a RTCP endpoint, not an intermediary node. It doesn’t forward RTCP. In other words:
- Alice has a producer and sends RTP.
- mediasoup receives it.
- Bob consumers it so its consumer gets RTP.
- Alice sends RTCP.
- mediasoup receives it.
- Bob will NOT receive those RTCP packets.
- mediasoup will generate RTCP packets on its own for Bob.
So the ‘rtcp’ event in a DirectTransport emits with RTCP packets that the mediasoup worker needs to deliver to the producers and consumers you created over that DirectTransport.
There is no consumer.on('rtcp') because that’s not how RTCP works. A RTCP packet may contain info about many different sending/receiving streams so mediasoup cannot associate RTCP packets to a single producer or consumer.
1 Like
Correction, i am not trying to inject anything. Only capture. The producers are remote webrtc peers, not directtransport producers.
- Have router with a lot of Webrtc peers (basically video conference).
- Create DirectTransport in node.
- Create a direct transport consumer for each producer.
- I can capture their RTP with consumer.on(‘rtp’).
- I would like to capture their RTCP.
- RTCP event does not ever fire on transport.
The end goal is to be able to determine, on the node application, the time relation between producers, so that i can synchronize their videos in an encoding pipeline.
Yes, I know. As said above that’s not the purpose of “RTCP” event. It’s not intended to spy every RTCP packet sent by a producer to mediasoup.
Understood, thank you. As far as i know, external webrtc consumers (such as browser or gstreamer) use these sender reports (that mediasoup generates) to sync multiple streams between them, associating SR NTP with the RTP timestamp. Would you have a suggestion on how to do that in node where we do not have access to those reports? E.g i can not trust that 2 direct consumers receiving ‘RTP’ event at the same time should play the frame at the same time in a theoretical playback.
It’s impossible to do that in Node since, as you said, there is no access to those reports. I don’t have any suggestion, sorry. Your use case is the “receive many in mediasoup and make it behave as if it sending a continuous stream to another endpoint that only can consume a single stream”. That’s not covered.
Exactly, i’m trying to fake a dynamic ‘active speaker’ video stream that mangles video rtp from the participant that is the active speaker into a single continuous stream. And i was hoping this would let me get the timing right when changing the input stream. Ok, thanks again for your help.
Just to make sure, would you please confirm that there is no current way to get a wallclock/rtp timestamp association for a stream in mediasoup Node api?
I confirm.