How to forward stream from one mediasoup v3 server to another?

Idea is to use it for scalability, to enable more viewers than a single server can bear. If we re-encode stream with ffmpeg, it works just fine, but results in a high CPU load for encoding. Without encoding though, result looks like this:

here is the command and SDP we use:

[[11:18:18.865]] [LOG] ffmpeg -protocol_whitelist file,pipe,udp,rtp -i - -map 0:a:0 -c:a copy -map 0:v:0 -c:v copy -f tee “[select=a:f=rtp:ssrc=11111111:payload_type=101]rtp://127.0.0.1:10051?rtcpport=10070|[select=v:f=rtp:ssrc=22222222:payload_type=102]rtp://127.0.0.1:10024?rtcpport=10039”
[[11:18:18.870]] [LOG] v=0
o=- 0 0 IN IP4 127.0.0.1
s=-
c=IN IP4 127.0.0.1
t=0 0
m=audio 10200 RTP/AVP 101
a=rtcp-mux
a=rtpmap:101 opus/48000/2
a=fmtp:101 minptime=10;useinbandfec=1
m=video 10202 RTP/AVP 102
a=rtcp-mux
a=rtpmap:102 VP8/90000
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1

audio appears to be correct. if i save video+audio into a file instead of sending it, it is all correct.

any ideas? i seem to be sending out the stream in some incorrect way, but how to fix?

PS: just realized that

level-asymmetry-allowed=1;packetization-mode=1

have no meaning for VP8. Removing them does not change things though.

In a perfect world, i’d love to be able to do it without ffmpeg at all, if possible…

You don’t need ffmpeg to stream a RTP stream from one mediasoup v3 to another:

Of course i read this. It only says “application should implement something similar to what the router.pipeToRouter() method already does”. Is there any practical, real world example of this? We have spent about 90 hours of work time trying to put together an example, and still nowhere.

You may prefer to check the TypeScript version instead of the JS transpiled one:

I don’t think there are examples of that. Examples of what exactly? What you need to do is exactly the same as pipeToRouter() does but taking into account that, in your case, your mediasoup Worker instances live into different hosts, so you need to do signalling between both hosts to replicate what pipeToRouter() does. Just that. You can interconnect your servers via HTTP (in both directions), WebSocket, any PUB-SUB mechanism, etc. That’s up to you.

https://mediasoup.org/documentation/v3/scalability/ how serious is the concern in red outline here? is there any practical example of “re-encoder” endpoint? Thanks.

It’s super serious. No magic here. You can use libmediasoupclient (or other tools such as gstreamer/ffmpeg) to behave as a re-encoder. No public example of it AFAIK.

My client really really wants to NOT do any transcoding. I heard that Mediasoup throttles FIRs to no more than one per second. Is there a way to maybe increase this value to say once in 3 seconds or once in 5 seconds? If it takes changing C++ code, fine - can you point us at the line with constant?

My idea is that if it’s once per 5 seconds it means that some clients will have to wait for a few seconds for a keyframe, so fine, let it be, but the stream won’t swell up 3x in bitrate just ~1.4x which is acceptable?

*“i heard” is precisely, from here: https://github.com/versatica/mediasoup/issues/227
Currently the Producer.cpp class in mediasoup server already limits the number of PLI/FIR sent to the client. Basically, no more than 1 PLI/FIR per second is sent. If another PLI/FIR is requested in the meanwhile, it's delayed.

And yes, i just discussed with my client. Just simply switching that limit from one FIR in 1 second to one in 5 seconds, even if it has to be hardcoded, will do. How can i do that? Thanks!

My time is limited. Tell your client to make a donation to mediasoup and I’ll add a setting to configure that before tomorrow ends.

Sure! Give me a link where to donate, and the amount.
And yes, my client is a nonprofit, if that matters.

Whichever donation is ok: https://mediasoup.org/sponsors/#become-a-sponsor

Please at least hint on an amount. I don’t want to look rude or anything, just want to see what you expect. Thank you.

Donations are free and I don’t have any policy, bu tif you insist let’s say something between EUR 0 and 200 :slight_smile:

ah ok absolutely fine, will do right now thank you!

Done.

Thanks! Will post here when done.

Done in mediasoup 3.4.9:

Thanks! Will try Monday

But does it sound like a good solution to you? Apart from resulting in slightly increased bitrate for same quality of video, and having people wait an extra couple seconds for the first frame to arrive, does it allow for limitless scalability of the same stream without taking any additional technical debt, and without any transcodings?

In theory yes. Said that, if a viewer looses many packets and needs to ask a keyframe, given the 5 seconds “protection”, it may happen that such a viewer gets frozen video during up to 5 seconds (if other viewers are also requesting a keyframe).