I’ve been researching on ways to smooth out video and audio in a mediasoup router, potentially those that make mediasoup reselient against network congestions and other disruptions considering the fact that now people are streaming and having meetings more than ever.
From what I understand this requires changes to the sdp and appending a line that configures it on the sender’s side. How would one go about adding this to mediasoup and testing it out, because I believe it can have a major impact on perceived performance, especially in very low bandwidth scenarios.
And since this is related to WebRTC and not necessarily related to mediasoup, I’ve chosen to post this in the ‘Off Topic’ group.
Any and all the help / questions / critique will be appreciated.
Awesome. I would love to work on this and add it, this in my opinion will be a big win. But I think there is going to be some waiting before this gets any further, even if someone implemented this in mediasoup there still needs to be support on the clients.
Chrome is by far the only one that has added support for this. I did P2P tests where I streamed a video from my phone to a Chrome client and using the ‘playoutDelayHint’ I was able to smooth the jittery video out using a delay of 5 seconds.
I think there’s a way to get the underlying peer connection object for a consumer. I hope to do a Chromium test for this on mediasoup, I should add a ‘playoutDelayHint’ to the receiving peer connection objects and see if that makes any difference.
The only problem I see with that is currently in the docs I’ve not come across a way to set this parameter from the RTCRtpSender. the ‘playoutDelayHint’ is only available on chrome and can be set on the receiver.
I’m reading RFCs and docs and trying to figure out a way to attach the RTP header that will set this from mediasoup as it sends data out to the consumers. But that too is an assumption, I don’t have the knowledge yet to check weather the RTP header is supported on all clients specially ‘Safari’ - iOS is hectic af when it comes to support.
The craziest way that I can think of is packaging the encoder / decoder as a web assembly module and using data channels to send this data off and using canvas to render the video… That’s the only sure way of surely knowing that we have support for all that shit…
It would be great if you can point me to an implementation for this in the mediasoup - server side - library. I have configured the build environment locally I’m now going through code to understand where the changes will be required.
That’s not what I meant. You’ve said that I can look into the implementations for other header extensions and how MS handles that, I thought you might have a link to some resource that might point me in the right direction.
No worries though. Its all about the will, I’ll try my best. Thanks for all the advice. Much appreciated