simulcast inconstant fps and some random freeze

mediasoup libraries
1

Hi,

I’m experiencing some issues with simulcast, I setup the video producer like that :

encodings: [
        { maxBitrate:  96000, scaleResolutionDownBy: 4 },
        { maxBitrate: 1500000, scaleResolutionDownBy: 1 }
]

I send a stream with 30 fps and the framesDecoded that return consumer.getStats() is inconstant : sometimes : 30 fps, sometimes 15 with an average of 23. (it is not very disturbing but it is noticeable a little).
I’ve tried vp8 and h264 and it’s the same. I did some other test with other values of encoding but it still the same

When I setup my stream with this encodings parameter (without simulcast) :

encodings: [
       { maxBitrate: 1500000, scaleResolutionDownBy: 1 }
]

I don’t have any problem, my fps is 30 all the time.

I also experiment some video freezing with simulcast very randomly, but it happens to all consumer in the same time, I did not notice any particularity in the logs when it’s happening.
I see this issue in github : https://github.com/versatica/mediasoup/issues/394 so I try with the new version, and I have some freeze maybe less often, it’s hard to understand when it is happening.

Thanks.

2

Because without simulcast, Chrome does not use per stream temporal layers.

3

Thank you for your reply.
I’m going to test the settings on these temporal layers and see if I can influence the fps of the stream. I’ll keep you posted if I find improvements

4

You cannot control the temporal layers that Chrome generates. That’s up to Chrome/libwebrtc.

5

Thanks for your answer.
Ok, I read some documentation about temporal layers, and I’m starting to understand the mechanisms of fps generation and if i understood correctly i could not receive my stream with 30 fps. (if I’m right we can close topic :slight_smile:)
So maybe the solution will be to produce 2 streams with one with low quality and to do the switch on client side with the analyze of consumer.getStats()

6

No. You don need two streams. Simulcast is already about sending two or three streams. But each streams has 2-3 temporal layers and mediasoup will relay to each consumer the max layers it can send to them based on downlink bandwidth estimation and user preferred layers (set via API in server side Consumer). There is API for that and lot of documentation about that. I cannot explain everything again here.

7

Ok, thanks, I will do more search and test with server side. I will share the result if it is conclusive

8

Sorry, I don’t want to waste your time.

So I investigated and played with the consumer.setPreferredLayers and the layerschange listener. With my client configuration I applied this config:

consumer.setPreferredLayers ({
        spatialLayer: 1,
        temporalLayer: 2
})

With this one my fps is between 20 and 30, with an average of 22 (the layerchange is not notified so I stay well all the time on the same layers)
My consumer has a score of 10 all the time.

when I change with these values:

consumer.setPreferredLayers ({
        spatialLayer: 1,
        temporalLayer: 1
})
consumer.setPreferredLayers ({
        spatialLayer: 1,
        temporalLayer: 0
})

I am with a regular FPS (15 for temporalLayer 1 and 7 for temporalLayer 0)

consumer.setPreferredLayers ({
        spatialLayer: 0,
        temporalLayer: 2
})

With this setting, I arrive at 30 FPS on a regular basis with low bitrate.

For the same bitrate when I apply the simulcast with the highest temporalLayer, I cannot recover 30 FPS on a regular basis.

Possibly the problem is more on the side of chrome encoding.

9

You can play with received spatial and temporal layers in the demo app:

CleanShot-2020-05-17-at-11.25.26@2x
CleanShot-2020-05-17-at-11.25.26@2x992×846 444 KB

10

NOTE: Since M80, Chrome just sends 2 temporal layers (instead of 3 as before) per simulcast stream. In the consumer side, a spatial layer represents a simulcast stream in the sender.

1 Like
11

Hi,
Is there any official webrtc chrome link that points to that Since M80 Chrome does not support 3 temporal layers?

12

The most official way to get it would be here I guess, as this was not put into a PSA in advance.

The flag in question can be seen here.

It was made default behavior for 100% at 4. December 2020.

13

Thanks for the pointer and documentation.