We have been experimenting with VP9 SVC and can reproduce issues we are having in the mediasoup demo app. For screen sharing, what we would like to do is to provide multiple temporal layers with scalability mode L1T3 or higher temporal layers, such that frame rate is reduced rather than the user experiencing quality or spacial reductions in bandwidth limited situations. We have been experimenting with Chrome 80 and 84 canary and find that we are only ever seeing temporal layer 0 being forwarded by the SFU. We are seeing that by looking at the consumer remote stats on the client side, or looking at the consumer stats in the server side CLI. Its not clear that Chrome is producing more than just one layer, but on the client side we have verified the encodings going to the peerconnection are correct. We are using --force-fieldtrials=WebRTC-SupportVP9SVC/EnabledByFlag_1SL3TL/ on the command line. If we add spacial layers such as with L3T3, we see movement between spacial layers, but never see bandwidth on any temporal layers beyond 0.
Digging into the worker code a bit I find this bit of code in SvcConsumer::IncreaseLayer, around line 401 of SvcConsumer.cpp
temporalLayer = 0;
// Check bitrate of every temporal layer.
for (; temporalLayer < this->producerRtpStream->GetTemporalLayers(); ++temporalLayer)
{
// Ignore temporal layers lower than the one we already have (taking into account
// the spatial layer too).
// clang-format off
if (
spatialLayer == this->provisionalTargetSpatialLayer &&
temporalLayer <= this->provisionalTargetTemporalLayer
)
// clang-format on
{
continue;
}
requiredBitrate =
this->producerRtpStream->GetLayerBitrate(nowMs, spatialLayer, temporalLayer);
MS_DEBUG_DEV(
"testing layers %" PRIi16 ":%" PRIi16 " [virtual bitrate:%" PRIu32
", required bitrate:%" PRIu32 "]",
spatialLayer,
temporalLayer,
virtualBitrate,
requiredBitrate);
// If active layer, end iterations here. Otherwise move to next spatial layer.
if (requiredBitrate)
goto done;
else
break;
}
Two observations. The first is that the only layer that returns a non-zero requiredBitrate from the producerRtpStream is layer 0. Maybe the bug is in that code, or maybe Chrome isn’t sending data on those layers? The second is that it seems this loop will either jump to done or break out of this loop, so there isn’t any way to ever choose anything but temporalLayer 0 in the case that spatialLayer != this->provisionalTargetSpatialLayer. From my testing, provisional layers are -1:-1 in the loop. If I remove the bottom if condition, I see it cycle through all the layers, but I don’t get a working stream. I smell a bug in here… Any thoughts on a fix? Thanks for your help!