RTP bytes received, but never sent

mediasoup libraries
1

I’m sending some h264 packets over a plain transport, and the transport is showing rtpBytesReceived growing as normal. However, rtpBytesSent is always 0. The consumer getStats() shows that the inbound-rtp is hooked up correctly and bytes are coming in, but outbound-rtp byteCount is always 0.

Enabled rtp log tag with debug level, but nothing shows up.

What would cause RTP packets to be received by a transport, but never sent to a consumer that is properly hooked up to the producer?

2

For instance, lack of keyframe.

3

A keyframe is definitely being sent, but it is in “slices” to stay under the MTU. Would this make a difference? Does mediasoup just look at the h264 headers before forwarding?

4

mediasoup won’t send any video RTP packet to a remote consumer until:

  • The transport is connected (just applied to WebRTC).
  • A video key frame has been received from the producer. mediasoup will request the producer such a keyframe. Without that keyframe, the remote consumer won’t be able to render video anyway.

Are you using a plain transport for consuming that video? And are you using comedia: true in that recv transport? If so, please read about comedia here:

5

  • I am using a plain transport, the transport is connected, I use comedia and both client endpoints are sending video (and both transports return a tuple, verifying comedia functioned correctly). So each client is producing and consuming on the transport they are connected to.

  • The very first packets sent by producer are SPS and PPS NALUs, followed by a keyframe NALU. How does mediasoup determine a packet is a keyframe, does it just read the nal_unit_type in the header?

6

In order to check if your client is sending keyframes (and mediasoup detects them), enable the trace event (with type keyframe) in the server side Producer and print the evens:

7

This must be the problem, keyframes aren’t being recognized. I see in the parsing code, if there is no frame marking then it assumes the first byte of the payload is the NALU header. But aren’t the first 4 bytes usually the start code for the NALU (0x00000001), then followed by the NALU header?

I guess RTP streams don’t use start codes? Just assume that a packet is at the NALU boundary?

Edit: That was the issue, needed the first byte of the RTP payload to be the NALU header so that Mediasoup could determine what type of frame it is. I don’t know how useful it is, but NVEnc outputs start codes in the frame data - maybe Mediasoup could parse those away if that’s the first byte in the payload.

8

Honestly no idea about what you mean. We don’t know what NVEnc is. Feel free to propose a PR explaining all this and in which scenario it may happen, etc.

9

how did you resolve this @oilkillsbirds - did you have to strip out start codes?