# Testing simulcast level switching **URL:** \ **Category:** mediasoup-demo\ **Created:** [May 18, 2020, 2:36pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036 "2020-05-18T14:36:41Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 18, 2020, 2:36pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/1 "2020-05-18T14:36:41Z")
Hi, I’m experimenting various settings for Simulcast. I found that in some cases, the spatial layer setting sent with `setPreferredLayers` is ignored. This always happens when the video source is a videoclip (not a real webcam source) sent as a fake system webcam, or using plain RTP transports. I prepared a modified GStreamer broadcaster script ([https://gist.github.com/vpalmisano/5716e8b52b258386fdd5c4fb93c97f11](https://gist.github.com/vpalmisano/5716e8b52b258386fdd5c4fb93c97f11)). You can test it using the well known Big Buck Bunny sequence ([https://peach.blender.org/download/](https://peach.blender.org/download/)). In my local setup I found that level=0 is never sent, even if manually selected using `setPreferredLayers`. ---
**Author:** ![ibc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/ibc/32/1540_2.png) [@ibc](https://mediasoup.discourse.group/u/ibc)\ **Post date:** [May 18, 2020, 2:45pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/2 "2020-05-18T14:45:53Z")
> [@vpalmisano](#): > > In my local setup I found that level=0 is never sent, even if manually selected using `setPreferredLayers` . Does `level=0` mean spatial 0 or temporal 0? BTW the proper way to diagnose this is by checking server side `Producer`’s stats to see what the producer is currently sending. You’ll see an array of stats, one for each ongoing stream (if simulcast and if the sender is really sending N streams). ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 18, 2020, 2:47pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/3 "2020-05-18T14:47:33Z")
> [@ibc](#): > > Does `level=0` mean spatial 0 or temporal 0? Spatial. I’m using only one temporal layer in the GStreamer script. ```auto producer.getStats(): [ { "bitrate": 53859, "byteCount": 1448769, "firCount": 0, "fractionLost": 0, "jitter": 0, "kind": "video", "mimeType": "video/VP8", "nackCount": 0, "nackPacketCount": 0, "packetCount": 6497, "packetsDiscarded": 0, "packetsLost": 0, "packetsRepaired": 0, "packetsRetransmitted": 0, "pliCount": 0, "score": 10, "ssrc": 2222, "timestamp": 1627638221, "type": "inbound-rtp" }, { "bitrate": 139811, "byteCount": 3926095, "firCount": 0, "fractionLost": 0, "jitter": 0, "kind": "video", "mimeType": "video/VP8", "nackCount": 0, "nackPacketCount": 0, "packetCount": 6897, "packetsDiscarded": 0, "packetsLost": 0, "packetsRepaired": 0, "packetsRetransmitted": 0, "pliCount": 0, "score": 10, "ssrc": 2223, "timestamp": 1627638221, "type": "inbound-rtp" }, { "bitrate": 467568, "byteCount": 12412844, "firCount": 0, "fractionLost": 0, "jitter": 0, "kind": "video", "mimeType": "video/VP8", "nackCount": 0, "nackPacketCount": 0, "packetCount": 12025, "packetsDiscarded": 0, "packetsLost": 0, "packetsRepaired": 0, "packetsRetransmitted": 0, "pliCount": 1, "score": 10, "ssrc": 2224, "timestamp": 1627638221, "type": "inbound-rtp" } ] ``` ```auto producer.dump(): { "id": "f4b3e65a-45f5-4093-8c1e-4aa39629263c", "kind": "video", "paused": false, "rtpMapping": { "codecs": [ { "mappedPayloadType": 101, "payloadType": 101 } ], "encodings": [ { "mappedSsrc": 225588197, "rid": null, "ssrc": 2222 }, { "mappedSsrc": 225588198, "rid": null, "ssrc": 2223 }, { "mappedSsrc": 225588199, "rid": null, "ssrc": 2224 } ] }, "rtpParameters": { "codecs": [ { "clockRate": 90000, "mimeType": "video/VP8", "parameters": {}, "payloadType": 101, "rtcpFeedback": [ { "parameter": "fir", "type": "ccm" }, { "type": "nack" }, { "parameter": "pli", "type": "nack" } ] } ], "encodings": [ { "codecPayloadType": 101, "ksvc": false, "maxBitrate": 512000, "scalabilityMode": "S1T1", "spatialLayers": 1, "ssrc": 2222, "temporalLayers": 1 }, { "codecPayloadType": 101, "ksvc": false, "maxBitrate": 1024000, "scalabilityMode": "S1T1", "spatialLayers": 1, "ssrc": 2223, "temporalLayers": 1 }, { "codecPayloadType": 101, "ksvc": false, "maxBitrate": 2048000, "scalabilityMode": "S1T1", "spatialLayers": 1, "ssrc": 2224, "temporalLayers": 1 } ], "headerExtensions": [], "rtcp": { "cname": "5ab6b651", "reducedSize": true } }, "rtpStreams": [ { "params": { "clockRate": 90000, "cname": "5ab6b651", "mimeType": "video/VP8", "payloadType": 101, "spatialLayers": 1, "ssrc": 2223, "temporalLayers": 1, "useDtx": false, "useFir": true, "useInBandFec": false, "useNack": true, "usePli": true }, "score": 10 }, { "params": { "clockRate": 90000, "cname": "5ab6b651", "mimeType": "video/VP8", "payloadType": 101, "spatialLayers": 1, "ssrc": 2224, "temporalLayers": 1, "useDtx": false, "useFir": true, "useInBandFec": false, "useNack": true, "usePli": true }, "score": 10 } ], "traceEventTypes": "", "type": "simulcast" } ``` ---
**Author:** ![ibc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/ibc/32/1540_2.png) [@ibc](https://mediasoup.discourse.group/u/ibc)\ **Post date:** [May 18, 2020, 2:55pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/4 "2020-05-18T14:55:25Z")
consumer.setPreferredLayers() will request a keyframe for the selected spatial layer (simulcast stream in this case) and become effective once such a keyframe is received in mediasoup, so I think that’s not happening. You can use produce.enableTrace() to enable keyframes in the producer and check it. Documented in the Debugging section of mediasoup. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 18, 2020, 2:59pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/5 "2020-05-18T14:59:20Z")
I modified the GStreamer script to produce 1 keyframe per second, nothing changed. > [@ibc](#): > > You can use produce.enableTrace() to enable keyframes in the producer and check it. Documented in the Debugging section of mediasoup. I will try now. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 18, 2020, 3:12pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/6 "2020-05-18T15:12:55Z")
> [@ibc](#): > > You can use produce.enableTrace() to enable keyframes in the producer and check it. Documented in the Debugging section of mediasoup. Using the browser as client and a fake webcam video source, I found that only 2 (over a total of 3) pli trace events are captured when I try to change levels. In the case of regular webcam source, I got 3 pli events, and the layer is always changed. This happens only when the video is generated, maybe a browser problem? This is the script I’m using to start a fake video source (on Linux): [v4l2loopback\_script · GitHub](https://gist.github.com/vpalmisano/17c3647e497629da5ec4aa9c15410a42) Edit: Using VP9, I get always 1 pli event (using SVC), but spatial layer is never changed as requested. ---
**Author:** ![ibc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/ibc/32/1540_2.png) [@ibc](https://mediasoup.discourse.group/u/ibc)\ **Post date:** [May 18, 2020, 3:47pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/7 "2020-05-18T15:47:53Z")
mediasoup sends a PLI for the ssrc of the stream for which it needs a keyframe. This’s been working fine for years. You can also use the `trace` event to log PLI requests sent by mediasoup to the sender: > **[mediasoup :: Debugging](https://mediasoup.org/documentation/v3/mediasoup/debugging/#producer-enabletraceevent-pli)** > > Cutting Edge WebRTC Video Conferencing > [@vpalmisano](#): > > Using VP9, I get always 1 pli event (using SVC), but spatial layer is never changed as requested. In VP9 there is a single SSRC so a single stream. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 18, 2020, 5:25pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/8 "2020-05-18T17:25:38Z")
Found the problem: the fake webcam source resolution should be at least 1280x720 in order to make simulcast work in a 3 levels configuration, with [4, 2, 1] scaling. It may be the same if you have a webcam with resolution \< 720p. ---
**Author:** ![ibc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/ibc/32/1540_2.png) [@ibc](https://mediasoup.discourse.group/u/ibc)\ **Post date:** [May 18, 2020, 5:42pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/9 "2020-05-18T17:42:29Z")
Nice. But the the missing spatial layer (the one you could not receive) was 2 and not 0. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 18, 2020, 5:45pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/10 "2020-05-18T17:45:53Z")
Level = 0 for me was the one with lowest resolution. ---
**Author:** ![ibc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/ibc/32/1540_2.png) [@ibc](https://mediasoup.discourse.group/u/ibc)\ **Post date:** [May 18, 2020, 5:51pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/11 "2020-05-18T17:51:20Z")
Exactly, so that’s always sent, but you said it wasn’t. Lowest one must be the first in the encodings array given to the server side Producer BTW ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 18, 2020, 5:55pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/12 "2020-05-18T17:55:30Z")
I think that when the fake webcam source sends a resolution that is less than the resolution requested by getUserMedia, the lowest level is not sent at all… ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 18, 2020, 5:58pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/13 "2020-05-18T17:58:05Z")
A question: do you know if there are some max/min limitation to the configurable parameters for simulcast? For instance, can I set `scaleResolutionDownBy=16`, or `maxBitrate=10` ? ---
**Author:** ![ibc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/ibc/32/1540_2.png) [@ibc](https://mediasoup.discourse.group/u/ibc)\ **Post date:** [May 18, 2020, 6:22pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/14 "2020-05-18T18:22:08Z")
> [@vpalmisano](#): > > A question: do you know if there are some max/min limitation to the configurable parameters for simulcast? For instance, can I set `scaleResolutionDownBy=16` , or `maxBitrate=10` ? There are ugly issues in libwebrtc if you specify encodings with too low `maxBitrate`, such as having zero RTP as result. I reported one of them to libwebrtc. So caution. No I don’t know these value limits. ---
**Author:** ![piranna](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/piranna/32/934_2.png) [@piranna](https://mediasoup.discourse.group/u/piranna)\ **Post date:** [May 12, 2021, 10:03am UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/15 "2021-05-12T10:03:59Z")
According to [media/engine/simulcast.cc - external/webrtc - Git at Google](https://chromium.googlesource.com/external/webrtc/+/master/media/engine/simulcast.cc#174), what it takes in account (at least in Chrome) is that the number of pixels (not resolution itself) is equal or higher to some standard resolution levels. Interest thing, according to [media/engine/simulcast.cc - external/webrtc - Git at Google](https://chromium.googlesource.com/external/webrtc/+/master/media/engine/simulcast.cc#87) the minimal limit to send 3 spatial layers is at least 960x540 (wide NTSC) pixels, not 1280x720 pixels (HD). @vpalmisano, what browser did you were using, Chrome or Firefox? Maybe Chrome changed the formats limits, or Firefox use other ones? ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 12, 2021, 10:08am UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/16 "2021-05-12T10:08:35Z")
I don’t know if Firefox applies the same configuration. If you comply the limits specified in the chromium source code, you always get exactly the number of spatial levels you set. ---
**Author:** ![piranna](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/piranna/32/934_2.png) [@piranna](https://mediasoup.discourse.group/u/piranna)\ **Post date:** [May 12, 2021, 10:19am UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/17 "2021-05-12T10:19:20Z")
So, we can be sure now that the 3 layers limit is 960x540, not 1280x720, isn’t it? I’m interested on this because I’m using the actual on-screen sizes to adjust the layer being used, so if I have an on-screen rendered small video tag, I can ask for a lower resolution layer and reduce client-side bandwidth and CPU usage 🙂 ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [May 12, 2021, 10:39am UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/18 "2021-05-12T10:39:46Z")
Yes, but this is strictly related to the chromium implementation. ---
**Author:** ![piranna](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/piranna/32/934_2.png) [@piranna](https://mediasoup.discourse.group/u/piranna)\ **Post date:** [May 12, 2021, 3:12pm UTC](https://mediasoup.discourse.group/t/testing-simulcast-level-switching/1036/19 "2021-05-12T15:12:37Z")
Ok, I’ve searched for the Firefox implementation, and I think it’s located at [VideoStreamFactory.cpp - mozsearch](https://searchfox.org/mozilla-central/source/dom/media/webrtc/libwebrtcglue/VideoStreamFactory.cpp#29), or at least it’s a similar table. It doesn’t have defined the number of spatial or temporal layers to use by default, but instead number can be variable based mostly on the bandwith needed for the desired resolution (not the resolution itself), provided config, and current bandwidth constraints, so if I understood it correctly, it seems Firefox could potentially generate up to 7 layers with a UHD video, when Chrome limit it to max 3. Can somebody confirm this? Firefox also tries to fulfill the requested number of layers, but layers will be ignored if scaled resolution is too much low that it dissapears, or scaled resolution doesn’t respect aspect ratio. Current bandwidth / resolution limits are: ```cpp // XXX Populate this based on a pref (which we should consider sorting because // people won't assume they need to). static VideoStreamFactory::ResolutionAndBitrateLimits kResolutionAndBitrateLimits[] = { // clang-format off {MB_OF(1920, 1200), KBPS(1500), KBPS(2000), KBPS(10000)}, // >HD (3K, 4K, etc) {MB_OF(1280, 720), KBPS(1200), KBPS(1500), KBPS(5000)}, // HD ~1080-1200 {MB_OF(800, 480), KBPS(200), KBPS(800), KBPS(2500)}, // HD ~720 {MB_OF(480, 270), KBPS(150), KBPS(500), KBPS(2000)}, // WVGA {tl::Max::value, KBPS(125), KBPS(300), KBPS(1300)}, // VGA {MB_OF(176, 144), KBPS(100), KBPS(150), KBPS(500)}, // WQVGA, CIF {0 , KBPS(40), KBPS(80), KBPS(250)} // QCIF and below // clang-format on }; ``` First columns are minimum resolution / bandwidth for each “level”. Values are different from the ones used in Chromium except 1280x720 (HD) and 480x270 (WVGA), but in both cases there are 7 entries, and in the case of Chrome, 480x270 is also the limit between one and two layers. I would need to check at least another implementation to see if it makes sense to have a common layers split, or do it per-browser and have a default one, of it default split layer would be senseless at all. What do you think?