Hi Community!
I recently came across mediasoup while going through WebRTC SFUs. Thank you for this amazing project! I went through the extensive documentation that you’ve done, over the weekend (notably the topics on scalability and architecture) and patched together a tiny proof of concept in Node.js, which works fine on my 8 core system.
But when I deploy it to AWS (a 4 cpu machine with 1 Tbps bandwidth), audio seems good, but the video lags and is out of sync at times and play as much as 20 seconds late. The situation worsens when I add more consumers. The physical location of server is nearest available. My video bitrate is less than 100kBps so that shouldn’t be a big deal (VP8 and Opus)
My application architecture is that I spawn a worker for each CPU, with a single router in each of them. A worker is designated as a producer worker to which all producers connect. After connection media is streamed to all consumer workers (routers). Consumers connect to consumer workers in round robin fashion to distribute load and consume specific producerIds (which gives a logical illusion of a room).
My CPU usage barely crosses 3% on all CPUs. Am I overoptimising this and doing something wrong? What server configuration do you suggest? The documentation suggests that a single worker can handle upto 500 peers.
That’s not expected. May be you are having problems in client side instead? Not sure how many real devices/browsers/computers you are using in the above scenario.
Also remember that real-time one-to-many with video is not the panacea. You need to do tricks:
Said that:
I assume you are using router.pipeToTransport().
If you experiment the video lag with a single producer in router1 and a single consumer in router2, and there is no network big issues between clients and the server, then let’s keep talking.
Hi! Thanks for the reply!
You mean router.pipeToRouter()? Yes, I did use that. Except that the router is on another worker.
I’m streaming about 80kBps (audio+video), with 10 tabs open on my device and another 10 on another device. I have a connection that supports this bandwidth (averages around 8mBps). I’m aware that UDP packets may get dropped on low bandwidth.
Got the clue here!
So instead of a worker that holds only producers, I’ll need to write a re-encoder right? Because there seems to be visible lag when piping to another worker…
Such a video lag should not happen. Let’s not assume that’s due to the usage of pipeToRouter. It may be CPU issues in client side or whatever. Please try your scenario with a single worker and a single router. If it also happens then it’s not due to the piped routers architecture.
Is this 10 tabs, each sending out 1 video stream (to the sfu) and each displaying 19 incoming videos?
If so, my guess is that this is cpu (or general chrome resource limits) on the client side. When you test locally on your dev machine are you testing with this same number of tabs/participants?
It’s quite hard to test “large” calls. We have scripts that run headless chrome on instances in the cloud, and local networks of several machines all routing through middle boxes that can individually degrade the network connections. Before we made this effort to build test infrastructure, we couldn’t really tell with any reliability what was happening in larger calls.
I switched back to connecting consumers and producers to the same worker. Works perfectly now. I’ll check again to see if I implemented it wrong.
I don’t think anything’s wrong with the client side because these internals are abstracted away.
I’m working on a one to many broadcasting (1 producer and 20 consumers) scenario so no issues with Chrome being a mammoth…
Haven’t really started working with the test infra yet, although the situation screams for it.
