Media streams produced by FFmpeg or GStreamer are not received via some ISPs

Integration
21

It seems GStreamer is sending keyframes but FFmpeg does not. Here’s the GStreamer keyframe trace:

{ 
  direction: 'in',
  info: {
    isKeyFrame: true,
    marker: 'false',
    payloadSize: 26,
    payloadType: 102,
    sequenceNumber: 26592,
    size: 38,
    spatialLayer: 0,
    ssrc: 22222222,
    temporalLayer: 0,
    timestamp: 2958783210,
  },
  timestamp: 5879089198,
  type: 'keyframe',
}
{ 
  direction: 'in',
  info: {
    isKeyFrame: true,
    marker: 'false',
    payloadSize: 26,
    payloadType: 102,
    sequenceNumber: 28896,
    size: 38,
    spatialLayer: 0,
    ssrc: 22222222,
    temporalLayer: 0,
    timestamp: 2959683210,
  },
  timestamp: 5879097728,
  type: 'keyframe',
}
{ 
  direction: 'in',
  info: {
    isKeyFrame: true,
    marker: 'false',
    payloadSize: 26,
    payloadType: 102,
    sequenceNumber: 29952,
    size: 38,
    spatialLayer: 0,
    ssrc: 22222222,
    temporalLayer: 0,
    timestamp: 2960093610,
  },
  timestamp: 5879102289,
  type: 'keyframe',
}
22

You should also enable “pli” type in producer.setTraceEvent() to see if those keyframes are triggered by PLIs. Otherwise they are mostly useless.

23

Here’s the log output for keyframe+pli:

{
  direction: 'in',
  info: {
    isKeyFrame: true,
    marker: 'false',
    payloadSize: 26,
    payloadType: 102,
    sequenceNumber: 26266,
    size: 38,
    spatialLayer: 0,
    ssrc: 22222222,
    temporalLayer: 0,
    timestamp: 1437098549
  },
  timestamp: 5961255102,
  type: 'keyframe'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961255179,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961256177,
  type: 'pli'
}

{
  direction: 'in',
  info: {
    isKeyFrame: true,
    marker: 'false',
    payloadSize: 26,
    payloadType: 102,
    sequenceNumber: 28577,
    size: 38,
    spatialLayer: 0,
    ssrc: 22222222,
    temporalLayer: 0,
    timestamp: 1437998549
  },
  timestamp: 5961265018,
  type: 'keyframe'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961265404,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961266404,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961267404,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961268405,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961269404,
  type: 'pli'
}

{
  direction: 'in',
  info: {
    isKeyFrame: true,
    marker: 'false',
    payloadSize: 26,
    payloadType: 102,
    sequenceNumber: 29632,
    size: 38,
    spatialLayer: 0,
    ssrc: 22222222,
    temporalLayer: 0,
    timestamp: 1438408949
  },
  timestamp: 5961269578,
  type: 'keyframe'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961269836,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  }
  ,
  timestamp: 5961270836,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961271836,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961272836,
  type: 'pli'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961273836,
  type: 'pli'
}

{
  direction: 'in',
  info: {
    isKeyFrame: true,
    marker: 'false',
    payloadSize: 26,
    payloadType: 102,
    sequenceNumber: 30866,
    size: 38,
    spatialLayer: 0,
    ssrc: 22222222,
    temporalLayer: 0,
    timestamp: 1438873349
  },
  timestamp: 5961274738,
  type: 'keyframe'
}

{
  direction: 'out',
  info: {
    ssrc: 22222222
  },
  timestamp: 5961275076,
  type: 'pli'
}

What now?

24

As a reminder, I’m not asking for info (there is no bug in mediasoup here) but telling you how to get info.

25

Okay :grin:

Any idea to go further?

26

@ibc
One question. How to force TCP when creating PlainRtpTransport just like WebRtcTransport.enableTcp option?

27

Hi,

PlainRtpTransport uses UDP as the only underlaying transport protocol.

28

Oops!

29

Which spec defines plain RTP over TCP transport?

30

I don’t know :man_shrugging:. I’m just asking.

BTW @ibc I really really don’t want to bother you with my noob or irrelevant questions. So please feel comfortable to completely ignore them.

31

Actually it was a real question. AFAIK the only way to send RTP and RTCP over TCP is defined in RFC 4571, but just WebRTC endpoints implement it.

32

I think I has totally same issue at specific isp
Client keeps requesting Keyframe but it never comes. ( Client is Chromium, TURN relay is enabled) to restore video stream after a slow connection.