$$$ for a snippet of code for horizontal scaling using pipeTransports

Hi All,
First of all, thanks to the mediasoup team for such an amazing project, it’s truly a pleasure to work with.
I am putting together a prototype for scaling mediasoup across physical servers.
I know how to connect the 2 servers using pipeTransports, and I am implementing a version of what “pipeToRouter” does but across the physical hosts.
Since I am fairly new to medisoup, I would like to compare notes with somebody who has done this in a production setting, to make sure I build this the right way.
Willing to pay $$$. I need this ASAP, preferably today/this weekend.

the goal is to have the presenterRouter on host1 “replicated” on host2 so that viewerRouters on host2 can serve the producers on presenterRouter1 on host2. (few to many broadcast)

In host 1 I have worker 1, with presenter Router1, and this presenterRouter is being “pipedToRouter” to all other viewersRouters on the same host.
Additionally in this host, I am doing presenterRouter.createPipeTransport(…)
on host2 I have all workers and viewerRouters ready to accept connections

Now from the other physical server, I am getting the ip/port of the host1/presenterRouter1 and I am connecting to it.

Looking to get real production ready code for what needs to happen “next” on host2 to achieve the goal above

thanks!

I’ve been thinking on doing something similar, but didn’t have time yet to work on it. Subscribing to the issue to get notified in case there’s some interesting discussion here :slight_smile:

It sounds like you already have 95% of the work done. It would be helpful if you described specifically what your question is - what is the blank you want filled in? It sounds like this is more of an infrastructure question rather than how to use the code, since you’ve already figured that out. If so, would you describe your network topology/capability, number of servers, number of clients, and scaling needs?

A few ways way to consider doing this at mass scale:

  • Chaining servers. Server A streams to B which passes the stream off to C and so on. Least strain on individual servers, but adds latency at each hop.

  • Variation on the first idea, except A streams to B and C, which each stream to two sub servers, and so on.

  • A streams to all servers B-Z, and they stream to the clients. High load on A but minimal latency.

  • Multicast for devices on the same subnet. I have no idea if this is possible with WebRTC/mediasoup, but worthwhile investigating since it could help reduce strain on that subnet’s networking equipment for massive deployments.

HI!, dimochka, sorry If I was not clear, the part that I am missing is the “pipetorouter” implementation but for sending a specific producer in router1 in host 1 through the pipeTransport to host2, viewerRouter2

I’m currently running scaled in production right now at stumblechat.com;

This is alpha (3-4 days running) however for proof of concept go lurk… I run a main HTTPS front that talks to a broker server via web-socket. I can run as many front-ends on this broker (or if more needed on multiple brokers).

The broker in my case is a simple websocket connection that operates behind private LAN otherwise a safe proxy is made to connect regions. Its job is to essentially talk to all the chat-servers and route data to and from them to users or self taking as many short-cuts as it takes to make the least connections.

This keeps the chat-server up and running nicely keeping 100% accuracy on who’s around when they leave and handles 8K concurrency per core before choke/throttle needs applying.

For media-servers which was fun for me actually, I ask my broker server which one is available and this is based off whichever one doesn’t have too many users on it. I have two separate files, one for consumer side and one for producer.

How I connect these is simple, I connect my producers to every consumer server or if consumer becomes available tell all producers to subscribe with the pipe-transport.

This allows me to do some wicked scaling as you can load 40-60 producers per server and have room to cast them to x amount of consumer servers.

I generally run my ratios 1 producer to 2-4 consumers depending on operation but that’s easily sorted.
Some good tips, I’ve had real bad experiences running multi-core setups. I like to actually go single cores on the media-servers and spread it far and wide in case of attack and/or time-outs. I see too many large-core scenarios fail a large party to a simple jitter/packet loss.

Not looking to take on work cause I’m currently stuffed but hope this helps a bit, I can explain it more if needed.

1 Like

Hi @CosmosisT ,
the only part that I am missing is explained above

You’re trying to connect two different IPs? Utilizing the PipeTransport method?
For a producer connecter to consumers only scenario.

  1. When consumer comes online send a createpipetransport message (via your signalling method), this will share the listenIP and producer will create their pipe and connect that IP on received. If producer comes online, it tells any consumer to create the pipetransport first then send the listen IP to connect.

  2. Once connected, you would produce a stream and take that ID when someone wants to consume it and re-broadcast it from producer over to consumer to re-consume it and send it out to anyone who reaches that server and see’s it’s already available.

  3. Not really a third, it’s two giant steps. :slight_smile:

Thanks for clarifying. So it sounds like you have pipeTransports already set up. You are trying to implement a method that takes a producer on mediasoup server A, consumes it on A to send it to mediasoup server B, and then have it appear as a producer on B. The same way that pipeToRouter transforms a consumer on router A into a producer on router B.

I’m not exactly sure how best to do this, but the good news is the code already exists in pipeToRouter and is super readable. Check out the method in Router.ts in the mediasoup server code - it should be easy to adapt to your use case:

				pipeConsumer = await localPipeTransport!.consume(
					{
						producerId : producerId!
					});

				pipeProducer = await remotePipeTransport!.produce(
					{
						id            : producer.id,
						kind          : pipeConsumer!.kind,
						rtpParameters : pipeConsumer!.rtpParameters,
						paused        : pipeConsumer!.producerPaused,
						appData       : producer.appData
					});

So you’d want to use the consume() call on transport A, transmit the rtpParameters over the network separately (ie websocket), then pass those to the produce() call on transport B.

This is my best guess. I’ve never actually done this :joy:

Thanks you all. I’ll play with it this weekend ! thanks

1 Like