In our app we have been noticing occasional issues with “black” video for extended / variable lengths of time. I am pretty sure this is due to our app aggressively pausing non-visible videos and resuming them when they become visible. This is likely introducing a lot of keyframe requests to the producer when multiple viewers hit a UI state which needs to unpause the same videos and this generally makes sense.
If we are not using mediasoups keyframeRequestDelay and thus getting the default of 0 seconds, it would seem that all keyframe requests (PLI) are just being sent directly onto the client. Non-mediasoup question: does anyone know if the client (Chrome) simply discards these requests if they arrive within the 1s timeout period? It seems like this would be the case.
I dug into the mediasoup KeyframeRequestManager implementation a bit and it appears to use a timer based approach for delaying and then requesting keyframes. And if a producer would receive multiple of these requests during this period (e.g. from multiple consumers resuming) they would all be satisfied by a single request once the timer period is up. e.g. if I have a delay of 5s, and I get 4 consumers requesting a keyframe 500ms apart, all 4 will receive the keyframe after the delay period. We won’t discard 3 of the requests, or “stack” them up.
So my theory is that by setting keyframeRequestDelay in mediasoup to 1s (generally matching Chrome’s setting) we’ll actually get better behavior because the requests won’t be simply discarded during that delay, but will be tried after that delay… and importantly, any requests during the delay period will be “grouped” into that next request.
Does this sound generally correct? This is based on the assumption that the client discards these requests during the timeout period (I haven’t dug into libwebrtc here to verify this and I understand this is a non-mediasoup topic). But is my understanding of the mediasoup implementation correct in that if a timeout period is “in progress” any further keyframe requests in that period will be satisfied once the timeout is complete?
I am trying to reconcile my understanding with this comment from another thread about keyframeRequestDelay, which says it will behave “similarly” and the keyword of confusion being “ignores”.
Thanks