Should producer.replaceTrack() use producer's encodings?

mediasoup libraries
1

I create my video producer with:

const WEBCAM_SIMULCAST_ENCODINGS = [
  { maxBitrate:  144000, scaleResolutionDownBy: 4 },
  { maxBitrate: 400000, scaleResolutionDownBy: 2 },
];

sfu_camVideoProducer = await sfu_sendTransport.produce({
    track: localStream.getVideoTracks()[0],
    encodings: WEBCAM_SIMULCAST_ENCODINGS,
    codecOptions: {videoGoogleStartBitrate: 1000},
    appData: { mediaTag: 'cam-video' }
});

…and this works great; two separate tracks are available.

When a user (a) changes their video input or (b) turns the camera off & on, I call sfu_camVideoProducer.replaceTrack({track}), but this seems to only produce 1 track (at the lower encoding/maxBitrate).

Is this the expected behavior? If so, what is the recommended action when changing the video input device? Do you need to destroy the producer and re-create it from scratch?

2

Simulcast is about 2 different encodings of the same exact track.
When you replace it everything should work just fine, there should be still 2 encodings, but with a new track’s contents.

Also unless you have specific setup, usually one of the encodings has scaleResolutionDownBy: 1, so you’d be better off requesting lower resolution video in the first place.

3

This is not what’s happening for me though. I’m trying to think of where there could be bugs in my code.

But there’s not much room for error - you call sfu_camVideoProducer.replaceTrack({track}) client side, and that should do it, right?

If it had 2 encodings before, it should have 2 encodings now?

My setup doesn’t have a scaleResolutionDownBy: 1…didn’t seem necessary and adds to bandwidth requirements.

const WEBCAM_SIMULCAST_ENCODINGS = [
  { maxBitrate:  144000, scaleResolutionDownBy: 4 },
  { maxBitrate: 400000, scaleResolutionDownBy: 2 },
];
4

Yes, you can check that in dev tools. The only thing I can think of is if new video track is significantly smaller and browser decided to disable larger layer.

I mean if you request smaller video resolution from the browser you can use 2 and 1 coefficient instead of 4 and 2.

5

Exactly. Anyway mediasoup-client does not change encodings unless requested, so this is up to browser behavior.

6

Thank you guys, the problem was indeed with my code.

I forgot that when I startCamera for mediasoup, I don’t pass any constraints (other than deviceId) to getUserMedia, otherwise it tends to not work properly (for P2P, I pass constraints such as ideal resolution, echo, aspectRatio, etc.).

When changing cameras, I was still passing these constraints to getUserMedia. I updated my code and now all is well :+1: