video freezing and frame-rate drops with high package loss

mediasoup-demo
1

I used h.264/vp8 codec in behind 3 cases, and found that bursts of video freezed and frame-rate dropped with 20% uplink package loss with FEC & nack.
mode:
case 1: simple
case 2: svc
case 2: simulcast

mediasoup 3.5.15
OS: debian 10

I find video become more and more blur :
simple mode, the package loss is about 1% used FEC & nack, but video sometimes freezed and frame-rate dropped.
svc and simulcast mode, video always freezed and frame-rate dropped to 0fps.
I want to know why and how to reduce video freezed and frame-rate dropped and make video streams more smooth

2

Please, define what you mean with “FEC”. mediasouop does not support FEC for video codecs.

Other than that, we cannot do magic. If you apply 20% of uplink packet loss, the problem is between the browser and mediasoup, and it’s the browser the one who decides what it can send (resolution and frame rate) based on network uplink capacity, and not mediasoup. I assume you are using Chrome browser BTW.

3

Thanks, FEC(Forward Error Correction) as packet loss transport handle
I simulate uplink 20% packet loss case by clumsy, you mean client should adjust to send (resolution and frame rate) based on network uplink capacity. But the resolution is from 1080p downscale 480x270.
I find useinbandfec in code, does useinbandfec : 1 means using FEC mechanism？
and why I use NACK rtcp feedback, video stream is blur?

4

As I said mediasoup does NOT support FEC for video codecs.

Can you point to the corresponding documentation in mediasoup API?

Because, as I said, Chrome will limit the bitrate/resolution/frame-rate of its sending video stream based on network uplink quality.

You should also describe whether you are testing this in localhost or in a real server, and whether you are consuming the video in a browser running in the same or different computer in which you are applying such a 20% uplink packet loss.

5

I test it in chrome browser, use mediasoup demo with 20% uplink packet loss by clumsy software

6

Oh, a single line response that ignores everything I said in my previous response. Nice.

BTW what do you mean by “clumsy software”? Please, specify this.

7

clumsy software is simulate network software in windows operation system, for packet loss, latency, disorder etc.

clumsy homepage:
https://jagt.github.io/clumsy/cn/index.html

8

I’ve asked some questions above. You have not replied them. Hard to help that way.

9

Yes,I test this in a real server, and consumed the video in chrome browser run in different computer apply in a 20% uplink packet loss, and use nack vp8/h264 video codec, rtcp feedback, it always freezed and frame rate drops

10

Sorry, I don’t know how that software behaves. So compare it with other videoconference applications in the same scenario.

11

thank you all the same.

12

Actually, have you tested the same setup with existing WebRTC applications that use simulcast (such as Jitsi or Google Meet)?

13

I tested the same with licode and Google Meet, and found that as follow:

  1. use simple streams(not use svc or simulcast stream) to test, mediasoup demo the result is the same as google meet and licode, video qualitity is blur, but not halt.
  2. use svc or simulcast streams to test, licode and mediasoup stream halt a moment later.