I want to replace the track of consumer with some other track on server side just like we do producer.replaceTrack on client side. I couldn’t find anything in the docs related to this.
I am working on ‘speaker view’, same as zoom, so there will be a stream in the centre of screen which will switch dynamically based upon the person who is speaking.
The best way to achieve it is by doing replaceTrack on consumer from server. But replaceTrack is not available on server side.
The alternate way is to close consumer and create new consumer whenever speaker gets changed but that can give 2-5 seconds delay in video because it will have to create new consumer.
Is it possible to do replaceTrack on server side? If not then what can be the other option apart from the one I mentioned? Thanks.
Replacing a track in server side is a very complex task given that the source tracks being exchanged may have different properties (simulcast, codec, codec parameters, etc). We may try to implement it in future mediasoup v4 but no ETA.
You can create all consumers and dynamically pause/resume then at your convenience, no need to close and create new ones.
Ok pause/resume will be way better than close/create, thanks. But i can’t create all consumers because there may be many people in the call I think I will have to only consume streams of those who are speaking and then pause/resume between them. It will still give a delay incase a new person started speaking then for him I will have to create new consumer.
But this is the best way at the moment will go with it and wait for v4.
Well, I said “create consumers for all”, not just for those who are already speaking.
Creating consumers for all terifies me alot. I think if a consumer is in paused state then it has no impact on server load.
Then why does it terrify you?
It is because of the fact that we are creating thousands of consumers just to accomplish one thing that is replaceTrack. But I understand that it is necessity of the moment.
But before consuming all streams I will go with consuming streams of only the ones who are speaking and see if the delay is acceptable.
Sorry for bringing topic up if you solved the problem.
In my app, consumers are paused/resumed often. According to my experience with that, zeroRtpOnPause work well about cost and performance. And it Thats internally replaces track with null on pause.
Strange as it seems,
zeroRtpOnPause does not cause internal replace of the track with null. It did at first, but then this was changed in favor of inactivating the transceiver. And the latter only works with browsers that implement the transceiver API. So if you want to support older browsers like Chrome < 74 or Safari 11, you have to call
replaceTrack explicitly, depending on chosen handler or always (no harm, but excessive). I think there is actually a discrepancy in the documentation regarding this option that should be corrected. Or/and maybe
producer.pause behavior should be changed for the handlers that currently do nothing in their
Docs are fixed now: mediasoup :: API