Experience with Mediasoup

Hi Team,

Just wanted to share some numbers using mediasoup. We are currently working on a usecase of 1:many broadcasting, and yesterday we had a Performance Testing done. There were two producers and 1600 consumers(1 publisher : 800 subscribers). The system config we are using currently is as following,

CPU cores : 4
RAM : 30GB
OS : RHEL 7.4

We had this testing for 30 minutes and we were also capturing the server side parameters and the results were surprising.

Avg RAM utilization was around 6 GB,
Avg Cpu utilization was 44.6 %,
Processor load was less than 3% for these 30 minutes.

5 Likes

Awesome, some questions:

  • Does it use pipeToRouter()? how many workers used that way?

  • Is there video with simulcast/SVC?

  • Are receivers in “real” network with possible packet loss and so on?

2 Likes

Hi,

Yes it uses pipeToRouter(). We created 4 workers and on each worker only one router. Producers were present on router1 and from there we piped them to the other 3 routers created on other 3 workers. All consumers were able to consume the piped producers from router1. We had set the limit of each router to have only 400 consumers. Next 400 goes on next router.

Yes, audio producer was a simple one. Video producer was a simulcast producer with 3 encodings of maxbitrate(100000, 300000, 900000).

All receivers were in real network using different browsers and devices(android, macos, ios, windows). Browsers used were Safari, Chrome, Opera and Mozilla firefox. Transports were enabled with only UDP so there might be packet losses as well…

2 Likes

Nice, so it looks like good numbers, right? your thoughts?

2 Likes

Yes, ofcourse the numbers are really good. The limit of 400 was kept after reading the cosmo software publication and also after reading the documentation present on mediasoup.org

On the site it was mentioned that for a single worker we can have anywhere between 400-600 consumers. Therefore we restricted a maximum of 400 consumers on each router. Since we only have one router per worker.

We have many performance testings lined up for this month where we are planning to have more than 5000 subscribers which means 10000 consumers. There, we will also have scaling of producers across servers using pipe transports.

1 Like

Amazing. I’d love to have that document into Markdown format to add it to the website if you are ok with this.

Hi IBC,

Pardon me, which document are u referring to here?

I mean if you could write such a document/text so we add it to the website (with your company name and so on).

Hi,

I could definitely write a raw text document about these stats and mail you. I am sorry i cannot reveal my organization’s name. I hope you understand.

Sure, as you wish. Thanks a lot.

1 Like

Anytime, many many thanks to you guys for making such an awesome and powerful library.

2 Likes

This is exciting… Amit, love to discus this for our application of Media Soup as well.

Amit,
Great info. This info helps us estimate/size our deployments. Thank you for sharing.
-Reddy

1 Like

Hey Amit,

Congrats on the implementation! We’re just entering the same phase with our own implementation and I had a couple of questions about the hardware you’re running? Have you gone the virtualised route, cloud hosting or local?

These really are some cool to see just before we begin tackling it ourselves!

1 Like

Hi Dave,

We are using cloud hosting as of now.

1 Like

Which video resolutions are you using? The sent video is real (e.g. webcam) or generated from a test sequence?
Thanks

Hi Vittorio,

I did not set the resolution when i called the getUserMedia() api, i believe it took the default one depending on the device/browser.

Yes the video sent is a feed of real webcam comnected to device.

Ok, in this case test results are affected by the video content captured and encoded by the producer side.

I’ve performed some test with a similar configuration (using pipeToRouter) but using a recorded video as source (https://cloud.quavlive.com/s/gcw4ZLrfijJQbwP). You can find the first test results here: Video conference test: 1 producer / N consumers. Next week I will add some more scenarios with N>1 producers, that is more challenging because the number of output consumers without optimizations increases as N*(N-1).

4 Likes

Hi,

Would like to share our implementation and some questions :wink:

We are working on a solution where we have one producer (video/audio) and thousands of viewers.

We made our deployment using aws ec2 instances (m5n.8xlarge). We are using 32 workers (= # of vCPUs) and 1 router in each worker. We use pipeToRoute for all the workers (and pipeToExternalRoute which we implemented for other instances in the cluster).

In production we have several instances/machines.

We do not use simulcast or SVC, as we got very low-quality video when using simulcast (not sure why yet).

Video is maxed at ~ 100KB/s (VP8) and video is at ~4KB/s (opus).

We tested (synthetic tests…) and found that each aws instance, can handle 3000 users (3000 video consumers + 3000 audio consumer)

With 3000 users and on one ec2 instance (m5n.8xlarge), we are at:

CPU: 55%
Network Out: 315MB/s

Network In: 5.3MB/s

We do tests by opening mass number of servers on multiple aws regions with multiple instances of puppeteer on each which opens the web page with our session and save an image on a central machine - for us to review the image quality, and additional data we put on screen (bandwidth etc).

To lower the bandwidth required by the producer (standard mobile device), we set keyFrameRequestDelay to 4 seconds (didn’t implement ‘re-encoder’ yet) and we are setting ideal video in getUserMedia to 640x480

Still, we have a lot of unsolved areas and questions -

  • Did you implement a “re-encoder” as described in https://mediasoup.org/documentation/v3/scalability/
  • What is the bandwidth required by the producer? Does it change when there are multiple viewers?
  • Can you share how where you managed to test with multiple OSs?
  • Do you know what is the video quality your viewers receive?
  • Can you share your simulcast configuration?
3 Likes

So, m5n.8xlarge has 32 logical cores with AVX-512 (https://aws.amazon.com/ec2/instance-types/m5/), and you consumed 55% of it to send out 315MB/s or around 2.5 gbit/s of traffic. That closely matches my experience: about 0.019 of AVX2 cores per our typical stream which is 1.4mbit/s.

3 Likes