How to forward simulcast streams from consumer to other producers?

mediasoup libraries
1

I’m trying to write an application where I forward packets from a WebRTCTransport through a DirectTransport. To do this, I make a WebRTCTransport producer (producer1), consume it with the DirectTransport (consumer1), and then use on('rtp') to send those packets through a second producer (producer2). Eventually, these packets are supposed to go to another WebRTCTransport to get to a client; the DirectTransport is there in order to do real-time packet analysis in JavaScript.

The issue I’m running into is that I can’t create producer2 using consumer1's rtpParameters if producer is simulcast. This is because simulcast consumers output a single stream with a scalability mode, rather than multiple streams; so then producer2 has to be SVC instead of simulcast. But mediasoup doesn’t (I think?) support any codecs with both simulcast (for producer1) and SVC (for producer2), so I can’t get any packets to go through the whole pipeline.

I assume, though, that what I’m trying to do must be supported somehow, since mediasoup is designed for architectures with forwarding in them, e.g. forwarding packets through PipeTransports to other mediasoup servers, etc. How is this done? Can I tell the simulcast producer to keep its streams separate so that I can forward them through other transports?

2

Consumers created in a DirectTransport are normal consumers and receive a single stream with a single spatial and temporal layer (no matter the Producer uses simulcast or SVC).

Consumers created in a PipeTransport are special “pipe consumers” and receive as many streams and layers as the original Producer sends.

So unfortunately this is not possible right now and we should include an option in directTransport.consume() to enable reception of all layers. If you wish, create a proper feature request in GitHub, but note I cannot give any ETA for now.

3

Thanks so much for your support! This is a critical core feature for our app, so we might invest some work into contributing this feature to mediasoup ourselves, if you think that it’s a doable task for non-core contributors and would be open to that PR. Do you think that we could handle it? Alternatively, if you think this is a complex feature that a core contributor needs to handle, we might have money to support someone to do it (I’d have to check, of course, but my managers have been open to it in the past).

4

Please contact us in private and let’s talk.