Ok so the media servers itself dont know about rooms, users etc, their job is to produce and consume whatever asked. They will not bother to check whether you are allowed to consume or not because that is the job of broker server or may be chat server?
Chat server should always be communicating with broker server its intensions. Unless you got a different means of communicating, broker is key.
A broker server should be the knowing of all that goes on, if things are out of sync it is to be corrected. Much of handling is done by a broker server that over-see’s much of what is going on. They keep sync and setups connected right.
Ok right now the rooms the participants in the room the workers, routers, transports of room are all managed by the media server I wrote.
So I will keep this rooms, participants related stuff on broker server and the mediaserver doesn’t need to know any of this its job is to produce and consume that is it rest.
It’s all up to you, so you can dedicate servers to rooms, or design a pool design where rooms may not necessarily use the full room max or may stay small in other words. Dynamically transistioning with a pool setup can save money when site or servers don’t use millions of users but even then you can still make it work. A single core can perform 20K completed requests a second, if you need the whole earth online then you adjust how your single core (broker) works. 
So to summarize most users here just need a single web-socket server where all media-servers connect/route from and this server alone will handle millions users no problem. If you need larger you need to put big-brain in charge to determine how.
Ok thanks brother for this knowledge.
We have some nice thorough conversation regarding this here as well:
This picture especially clears much of the stuff.
The design is similar still but progressed. I run more broker servers and I make them do diff tasks. This allows me to scale usage, but this design can in fact run thousands of users given media servers route right.
Thanks