Problem using SimulcastConsumer on piped transports

Yes, but I can’t pipe a producer to the same router, because it will create e producer with the same id.

Sure, I did not mean “using the same router” but using the same worker :slight_smile:

In fact, there is no limitation, I was wrong, you can perfectly pipe router1 to router2 even if both routers have been created on the same worker.

Investigating the server logs I found that CanSwitchToSpatialLayer fails on spatial layers 0 and 2, and so they are excluded from the test loop.

Interesting. Let me explain what that method does:

  • Let’s imagine the remote simulcast producers sends 3 SSRC streams.
  • The first key frame is received for, let’s say, stream 2 (so “spatial layer 1” in the SimulcastConsumer).
  • The SimulcastConsumer selects it.
  • The SimulcastConsumer cannot switch to a different producer stream (for example, stream 3) until it has received RTCP SenderReports for both the already handled stream 2 and the desired stream 3.
    • This is because we need the NTP timestamp in both SR to correlate them and properly rewrite the RTP timestamp in packets forwarded by the SimulcastConsumer (otherwise we get lipsync, etc).

So, if you see that it means that, somehow, the PipeProducer in router_2 is NOT receiving RTCP Sender Reports from the PipeConsumer in router_1, so the SimulcastConsumer2 in router_2 can just select the first RTP stream that it received from the PipeConsumer (so the first RTP stream that the original sender sent).

And here the problem. This is super helpful !!!