I encountered an issue with using Firefox for my own mediasoup implementation where switching from screenshare to a webcam causes the webcam stream to be out of sync with the audio and the video quality being worse than starting the livestream with the webcam in the first place. I had a good look at my code to see if it was a bug on my end but couldn’t find anything obvious yet, so I wanted to see if the same issue could be reproduced on the demo site and it did on my Mac. Just wondering if anyone else noticed this issue as I am pretty sure it’s due to Firefox but I don’t know why. Chrome seems to be more reliable but the video quality isn’t as sharp as Firefox.
Are you switching using replaceTrack? If so I believe this is a known WebRTC bug. There was a discussion about a similar issue on this forum recently about the same - search this forum for zeroRtpOnPause.
Probably best solution would be to use separate producers for webcam and screen share, disableTrackOnPause, and make sure to pause producer on server side so the blank frames aren’t forwarded to consumers.
No I don’t use replaceTrack as the encodings used are different. I am creating separate producers for each stream the same way it is handled in the mediasoup demo. I am experiencing this issue on both my own server and the actual mediasoup demo site that is live right now. It may have to do with using the same webrtctransport to create the producers as rejoining as a subscriber and recreating the consumers for the webcam producer after closing the screenshare producer is still showing the stream to be out of sync with reduced video quality. One way to resolve is to recreate the webrtctransport and producer but that would require a host to rejoin the room. Were you able to reproduce this issue on the demo site with Firefox?
Oh I see. Demo isn’t my forte. Maybe one of the devs could weigh in. Have you looked at chrome://webrtc-internals for some insights? Best guess it’s still some WebRTC or network based idiosyncracy.
The issue does not occur on Chrome. It occurs on Firefox.
I think this tool offers some comparable information:
Sorry for the late reply. I did some testing on the mediasoup demo site and the stats provide a better explanation of what I am experiencing. I was able to observe the same issue on my end with my own implementation.
-
When entering a room using a webcam as the first video producer you will see that there are 3 streams produced as expected. From the consumer’s point of view I was able to view the stream at the highest resolution (1280x720).
-
When changing the video input to screenshare I noticed only 1 stream was being produced whereas in Chrome 2 streams are produced instead which should be the expected result.
-
Finally when switching back to webcam (i.e. new producer created due to different encodings), there were two scenarios that happened. The first scenario, only 1 stream was produced but at a lower resolution (320 x 180). The next scenario, 3 streams were produced but the score of the last two streams with higher resolutions quickly dropped to 0. From the consumer’s point of view only the 320 x 180 stream could be viewed and the other streams never recovered.
-
Consumer’s point of view for step 3 for both scenarios. You can see the producer scores for the last 2 streams are 0 where the first spatial layer is currently selected and the preferred setting is never honoured.
I am running Firefox 88 on Mac OS Mojave. This issue doesn’t occur in Chrome. Any ideas of why this could be happening? It seems like this is an issue on Firefox’s end. The solution that I came up with to avoid this issue is to create separate transports for camera inputs and sharing the screen.
Perhaps this is related to issue #150 and issue #104
Oh I am experiencing this issue by not just switching to screenshare but frequently closing and creating a new producer for a camera input on Firefox. That means creating a separate transport can still potentially run into issue.
Maybe worth opening a bugzilla ticket about this?
Were you able to reproduce this issue on your end? I am not sure what is the exact reason for this behaviour to make a report on Bugzilla as I am using the mediasoup demo to produce the issue. I need to learn how to read the webrtc stats to understand the root cause.
Honestly, I didn’t test, but overall seems like Firefox issue and we continue reporting various bugs to bugzilla from our end, so I wouldn’t be surprised that this doesn’t work properly because of bug in Firefox.
Oh, because you are also using Firefox and not just mediasoup and because mediasoup does not do anything on purpose to delay the screenshare video stream.
Yeah I know it’s not done intentionally by design. I am just saying I am not sure how to report it to Bugzilla because I don’t know what’s really happening under the hood in regards to Webrtc and I don’t want to mention mediasoup demo as a way to reproduce the issue as I think they are unrelated. I just wanted to know if anyone else is observing the same issue on Firefox. I’ve checked around and landed on issue #104 and issue #150 and thought they could be related and perhaps that something could be done on mediasoup’s end to handle it. I am sorry that I can’t contribute much as I have a lot to learn but I just wanted to raise awareness around this issue and help make mediasoup better. Sorry again if I came off as being ignorant.