FPS of a video stream drops dramatically after publisher sends a large file to server

mediasoup libraries
1

It happens more often when publishing a video from an iPhone. After about 2 minutes, the FPS comes back to normal. Also, if I stop and start video streaming after sharing a file, the FPS is great.

Small files don’t cause this issue.

I wonder if there is a potential that it could be something that Mediasoup can handle more gracefully?

Seems that bitrate drops. Producer stats before file upload:

bitrate: 978840
byteCount: 2574165
firCount: 0
fractionLost: 0
id: "tvsi0md2jsnrbm4n"
jitter: 25
kind: "video"
mediaType: "video"
mimeType: "video/VP8"
nackCount: 0
packetCount: 2447
packetsDiscarded: 0
packetsLost: 0
packetsRepaired: 0
pliCount: 3
sliCount: 0
ssrc: 3043613344
timestamp: 12973417209
transportId: 60411019
type: "inbound-rtp"

After:

bitrate: 30936
byteCount: 2720981
firCount: 0
fractionLost: 0
id: "tvsi0md2jsnrbm4n"
jitter: 88
kind: "video"
mediaType: "video"
mimeType: "video/VP8"
nackCount: 2
packetCount: 2596
packetsDiscarded: 0
packetsLost: 1
packetsRepaired: 1
pliCount: 4
sliCount: 0
ssrc: 3043613344
timestamp: 12973435536
transportId: 60411019
type: "inbound-rtp"

Mediasoup v2

2

I’ve searched this forum and found a similar issue:

3

Wonder if there is a way to reset bandwidth estimator without closing and starting a new video producer? I can see there is this->remoteBitrateEstimator.reset( call in mediasoup package but it’s called only on timeout and after new producer is created.

4

Use mediasoup v3 which implements Transport-CC for BWE and works much better than REMB.

5

I’ll definitely switch to v3 soon but since the switch is time-consuming, I hope there is a way to fix the issue in v2. From what you are saying though, I assume it’s not that easy to fix without hacks.

6

There is not fix possible. REMB is slow and there is no way to “reset the bandwidth estimator” because the user may consider that it has better uplink than the one reported by REMB. The problem is REMB and that’s why we added Transport-CC in v3.

mediasoup v2 is no longer supported BTW.

7

Thanks Iñaki

I know mediasoup v2 isn’t supported but I’ve found something interesting you might want to know.

The problem doesn’t seem to be related to uploading files at all! When I comment these following code out + make sure I pause producer and resume producer + replaceTrack, the FPS isn’t affected at all. One problem with this though is it will probably only work with iPadOS. I assume iOS will not let me successfully call getUserMedia when previous tracks aren’t stopped but I’ll double check that.

  // videoStream!.getVideoTracks()[0].stop()
  // videoProducer.track.stop()

Update: Tested with Pixel phone and an iPhone. All good! Will test on more phones.

8

You can just pause the producer in client side and server side (to notify consumers) so no RTP will be sent while uploading the file.

Anyway I insist that the problem would not happen using v3.

9

Anyway I insist that the problem would not happen using v3.

Thanks!

Just an update for those who is going to read this thread. Pausing producer doesn’t help and doesn’t cause this issue. It’s the fact that video stream isn’t coming to Mediasoup while producer/transport are still alive. And probably the reason why it’s happening is because getUserMedia() takes 0.5 seconds in Safari Mobile (haven’t tried reproducing in Android Chrome). In overall, the stream isn’t happening for 1.5 seconds in my case and it causes the issue with FPS sometimes. When I increase it to 5 seconds, it’s much easier to reproduce.

10

Update to the previous comment:
It wasn’t enough. Had to 1) deactivate the encoder + 2) activate it back only after uploading of the file finished

const parameters = this.videoProducer.transport.handler._pc
      .getTransceivers()[0]
      .sender.getParameters()

    parameters.encodings[0].active = false

    this.videoProducer.transport.handler._pc
      .getTransceivers()[0]
      .sender.setParameters(parameters)
1 Like