I ran mediasoup on my production server. After hundreds of uses, the mediasoup worker took up about 6 G of memory. After everyone quit, the memory was not released. Because this is a production server, i can’t debug on it, so should I deploy a test environment for testing and then use a memory tool to try to find this memory leak?
Yes there is currently a memory leak with one of the dependencies that mediasoup uses. (debug)
The issue seems to have been fixed on the mediasoup side but has not yet been published.
For production I would recommend restarting the application once a while, or if not possible running multiple instances.
but what you mean is a memory leak in the js layer right? what i encountered is a memory leak from mediasoup worker.
Can you provide more info? It’s super hard to figure out whether there is a leak in C++ code without no information. Is it mediasoup v3? which exact version?
it’s 3.4.3, and i use pipeToRouter to expand the capacity that a single room can carry.
I also want to provide some more information, but the problem occurs on the production server, so I have limited information. I am setting up a test environment to simulate the production environment and see if I can get more detailed information. Some observed information: My production environment has three machines, and each room is allocated to these three machines in round-robin scheduling. The memory leak has nothing to do with the number of users. For example, Server A served 10000 users, and Server B only served 100 users, but Server B may have more memory leaks.
Next I am going to do two comparison tests:
- Compare the situation with and without packet loss to see if there is a difference.
- Compare the situation with and without pipeToRouter and see if there is a difference.
Am I going the right way? Any suggestions?
Please, run mediasoup with valgrind and provide us with the output file once you detect that the leak has happened.
Just set the following environment variables before running the server:
export MEDIASOUP_USE_VALGRIND="true" export MEDIASOUP_VALGRIND_OPTIONS="\ --leak-check=full|\ --track-fds=yes|\ --log-file=/tmp/mediasoup_valgrind_`date +%s`.log"
NOTE: You must have ,of course, valgrind installed.