H264_SVC Temporal Layer Upgrade question

mediasoup libraries
1

I’ve been studying the H264_SVC::PayloadDescriptorHandler::Process method and have a question about the intended behavior of the temporal layer upgrade logic. I’ve encountered a scenario that I’d like to understand better.

The code block responsible for upgrading the temporal layer is as follows:

// ...
// Upgrade current temporal layer if needed.
if (context->GetTargetTemporalLayer() > context->GetCurrentTemporalLayer())
{
    // clang-format off
    if (
        packetTemporalLayer >= context->GetCurrentTemporalLayer() + 1 &&
        this->payloadDescriptor->s
    )
    // clang-format on
    {
        // ...
        tmpTemporalLayer = packetTemporalLayer;
    }
}
// ...

This logic checks if an upgrade is desired and if the incoming packet is a safe point to perform the switch (start of a frame). However, it does not constrain the upgrade to the targetTemporalLayer.

I would like to understand the original intent behind this design. Is this behavior intentional?

Thank you for your time and clarification.

2

I would say the correct behaviour would be moving closer to the target temporal layer, so replacing this check:

packetTemporalLayer >= context->GetCurrentTemporalLayer() + 1

by

packetTemporalLayer > context->GetCurrentTemporalLayer() &&
packetTemporalLayer <= context->GetTargetTemporalLayer()

The reason being, if we are inside this block we know that current temporal layer is to be upgraded, so if current packet temporal layer is greater than current and lowest or equal to target let’s increase the temporal layer closer to the target, and repeat with upcoming packets until we reach the target.

Do you recall any reason for the current code otherwise, @ibc?

3

In current code it may happen that we are in temporal layer 1, wish to move to layer 2 and end moving to layer 3, and that’s the issue right?

4

In current code it may happen that we are in temporal layer 1, wish to move to layer 2 and end moving to layer 3, and that’s the issue right?

No, in current code we may go beyond target temporal layer, since we only check that packet temporal layer is > current temporal layer.

5

Ok. Note that we have a similar bug in simulcast: consumer.setPreferredLayer(spatial: 1) makes mediasoup select layer 2 if layer 1 doesn't exist · Issue #1460 · versatica/mediasoup · GitHub

6

Issue created in GitHub: `SvcConsumer` may consume temporal layer higher than requested · Issue #1552 · versatica/mediasoup · GitHub

7

This potential issue has been fixed here by removing support for H264-SVC codec: