Handling server-wide congestion

Hey there,

I am currently facing an issue where due to too many users being connected to the same server (separate rooms, separate workers) the server’s uplink is becoming congested producing a choppy experience with both audio and video freezing intermittently.

It is my understanding that in these situations congestion control should kick in and tell producers to reduce their bitrate, fps, or quality, with the goal of maintaining a smooth experience even if degraded. However, upon further research it seems like this is only the case for individual producer/consumer streams, and there’s no default “SFU-wide” congestion control.

What are the usual strategies for dealing with this problem?

My current naive stopgap approach is to watch system resource usage and then based on room sizes, stream types, and consumer count dynamically adjust setMaxIncomingBitrate for the producers, however, I’m afraid in doing this I am messing up WebRTC’s own congestion control and possibly causing other issues.

That’s a problem you should avoid another way such as increasing uplink capacity of your server and, if not possible, by limiting number of users.

In an ideal world, I agree that’s how it should work. In this case, I hit a sudden influx of users in an uncommon location that overran my existing capacity, and procuring additional capacity took a couple of days, since this is a rather small service. I’m looking into how to smooth this out going forward.

From experience with platforms like Discord, it’s common for large SFU providers to degrade quality rather than let the stream go visibly choppy and freeze up, even without simulcast or special signaling involved. That’s the kind of end-to-end congestion control that seems to be almost entirely undocumented, and what I’m trying to track down.

Clients use Chrome or Firefox or Safari and all those browsers support sender side bandwidth estimation so if they server uplink is congested then browsers will detect it and adapt their sending bitrates. If this is not enough then yes you can use transport.setIncomingMaxBitrate() and call it again with a higher value when the congestion is gone or when some users left.