Transport not connecting when nginx loadbalancer used

Deployment & Scalability
1

Hello everyone, I apologize for asking this question but I’ve been stuck over this for many days.

Basically, I’m trying to deploy my application on AWS EC2 instance. I am using docker-compose and nginx for creating multiple replicas of the same server to balance the load between.

And everything is working fine (signalling and stuff) but my webrtc transport are not conneting when I’m using the load balancer. It works when I deploy the same project with only one replica but as soon as I create multiple versions of it, the transport doesn’t connect anymore.

And I am pretty sure it has something to do with rtcMin and rtcMax ports of mediasoup worker not being exposed properly but I can’t figure out a way to do it

Following are my relevant files

docker-compose.yml

version: '3'
services:
  nginx:
    build: ./nginx
    ports:
      - "8000:8000"
    depends_on:
      - backend
    networks:
      - my_network

  backend:
    build:
      context: ~/Live-streaming/live-streaming-v2
    hostname: server1
    ports:
      - "8000"
      - "10000-10020/udp"
    networks:
      - my_network
    deploy:
      replicas: 3
      update_config:
        parallelism: 1
        delay: 10s
    volumes:
      - /usr/bin:/usr/bin
      - /usr/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu
      - /usr/share:/usr/share
    environment:
      - RTC_MIN_PORT=10000
      - RTC_MAX_PORT=10020


networks:
  my_network:
    driver: bridge

nginx configurations

upstream backend {
    ip_hash;
    server backend:8000;
}

server {
    listen 8000;

    # Handle general HTTP traffic with load balancing
    location / {
        proxy_pass http://backend;
        proxy_connect_timeout 5s;
        proxy_read_timeout 10s;
        proxy_send_timeout 10s;
        error_log /var/log/nginx/error.log;
        access_log /var/log/nginx/access.log;

        # Forward client IP address to the backend
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
    }

    # Handle WebSocket connections for Socket.IO
    location /socket.io/ {
        proxy_pass http://backend; # Same upstream as regular traffic
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host $host;

        # Optional timeout adjustments for WebSocket
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;

        # Log for debugging WebSocket issues
        error_log /var/log/nginx/socketio_error.log;
        access_log /var/log/nginx/socketio_access.log;
    }
   }

Dockerfile

FROM node:18
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
EXPOSE 8000
CMD ["npm", "start"]

Thanks a lot for taking your time to read this

2

Maybe you got Port Range Conflicts as docker compose doesn’t natively handle port ranges for replicas. When you define 10000-10020/udp, each replica might try to bind to the same host ports, resulting in conflicts.

Not sure if solve but you can try using Docker’s host networking mode for test.

backend:
  build:
    context: ~/Live-streaming/live-streaming-v2
  hostname: server1
  network_mode: host
  environment:
    - RTC_MIN_PORT=10000
    - RTC_MAX_PORT=10020

This ensures that the container uses the host’s network stack directly, avoiding port conflicts.

3

No the ports are being binded properly without conflict

image
image1009×531 15 KB

4 
- "10000-10020:10000-10020/udp"
5

If I bind the ports of my replicas to specific ports of the host then I can’t deploy multiple replicas of the same application which then defeats the purpose of loadbalancing. Plus it doesn’t work even after I deploy a single container with its ports binded to specific ports of the host like you suggested

6

Try testing it in a single container as follows:

Mapping TCP and UDP Ports for WebRTCTransport.

Ensure both TCP and UDP traffic for WebRTC media streams can pass through by mapping the ports as follows:

- "10000-10020:10000-10020/udp"
- "10000-10020:10000-10020/tcp"

Configuring Amazon EC2 for WebRTC with Elastic IP

  1. Set Inbound Rules for Security Groups
    Configure the following inbound rules in your AWS Security Group:

    Port Range Protocol Source
    8000 TCP 0.0.0.0/0
    10000-10020 TCP 0.0.0.0/0
    10000-10020 UDP 0.0.0.0/0

    These rules allow necessary TCP and UDP traffic for WebRTCTransport.

  2. Assign a Static Public IP (Elastic IP) to Your EC2 Instance
    To prevent the IP address from changing upon instance reboot, assign an Elastic IP:

    • Navigate to the Network & Security section in the AWS Management Console.
    • Click on Elastic IPs.
    • Select Allocate Elastic IP address to create a new Elastic IP.
    • Associate the newly created Elastic IP with your EC2 instance.

  3. Update config.js with Elastic IP and Restart the Instance

    • Edit the config.js file to set the Elastic IP as the announcedAddress.
      Example:
      announcedAddress: '<Your-Elastic-IP>',
    • Restart your Mediasoup instance to apply the changes.