Using diffrent codecs for video and screen share

mediasoup libraries
1

I want to send (on the same transport) webcam video using h264 (to take advantage of HW acceleration) and screen share using VP8 (since it looks better).
I managed to “trick” chrome to use h264 as the first option (for the webcam) but what I really want is the ability to control the payloadType which will be used to encode each stream.

Is that even possible?

2

Why a trick? You don’t need to do any trick for that, just set H264 (with proper configuration) in the Router mediaCodecs as first codec and Chrome will use it without any trick.

This is a good feature to have, but it’s not done yet (it requires lot of work in client side).

However, you can do this:

  • Set both H264 and VP8 codecs in the mediasoup Router mediaCodecs.
  • Create 2 mediasoup-client Device instances in your client app (one for webcam and audio, another one just for sending screen sharing).
  • In the first one, before calling device1.load({ routerRtpCapabilities }), mangle the Router rtpCapabilities and put VP8 as first video codec.
  • In the second one, before calling device2.load({ routerRtpCapabilities }), mangle the Router rtpCapabilities and put H264 as first video codec.
  • Then create transports, etc.
3

The problem with two devices is that currently we have (mediasoup) device=user model, extending that a user can have multiple devices will required a lot of refactoring.

If you point me in the right direction I don’t mind publishing a PR to support that.

4

Adding the ability to select a codec for each Producer in the same sendTransport would also require a huge refactor in mediasoup-client. IMHO more than a refactor in your app for each user to hold two devices.

This is complex and I cannot give any guidance due to its complexity. This is a good to have feature but we need time to do it.

5

Donations are welcome :wink:

1 Like
6

Even if I spend every second of the day I wouldn’t have made the change you suggested to my application :slight_smile:

Good feature! passing the donation link to finance (It’s a big corp so it will take some time)

7

Great :slight_smile:

Please remember using the feature with mediasoup-client 3.6.2 (a bug was fixed yesteday). I’m using it right now and it works fine.