RTP bytes received, but never sent

I’m sending some h264 packets over a plain transport, and the transport is showing rtpBytesReceived growing as normal. However, rtpBytesSent is always 0. The consumer getStats() shows that the inbound-rtp is hooked up correctly and bytes are coming in, but outbound-rtp byteCount is always 0.

Enabled rtp log tag with debug level, but nothing shows up.

What would cause RTP packets to be received by a transport, but never sent to a consumer that is properly hooked up to the producer?

For instance, lack of keyframe.

A keyframe is definitely being sent, but it is in “slices” to stay under the MTU. Would this make a difference? Does mediasoup just look at the h264 headers before forwarding?

mediasoup won’t send any video RTP packet to a remote consumer until:

  • The transport is connected (just applied to WebRTC).
  • A video key frame has been received from the producer. mediasoup will request the producer such a keyframe. Without that keyframe, the remote consumer won’t be able to render video anyway.

Are you using a plain transport for consuming that video? And are you using comedia: true in that recv transport? If so, please read about comedia here:

  • I am using a plain transport, the transport is connected, I use comedia and both client endpoints are sending video (and both transports return a tuple, verifying comedia functioned correctly). So each client is producing and consuming on the transport they are connected to.

  • The very first packets sent by producer are SPS and PPS NALUs, followed by a keyframe NALU. How does mediasoup determine a packet is a keyframe, does it just read the nal_unit_type in the header?

In order to check if your client is sending keyframes (and mediasoup detects them), enable the trace event (with type keyframe) in the server side Producer and print the evens:

This must be the problem, keyframes aren’t being recognized. I see in the parsing code, if there is no frame marking then it assumes the first byte of the payload is the NALU header. But aren’t the first 4 bytes usually the start code for the NALU (0x00000001), then followed by the NALU header?

I guess RTP streams don’t use start codes? Just assume that a packet is at the NALU boundary?

Edit: That was the issue, needed the first byte of the RTP payload to be the NALU header so that Mediasoup could determine what type of frame it is. I don’t know how useful it is, but NVEnc outputs start codes in the frame data - maybe Mediasoup could parse those away if that’s the first byte in the payload.

Honestly no idea about what you mean. We don’t know what NVEnc is. Feel free to propose a PR explaining all this and in which scenario it may happen, etc.