Can't decode frames on client encoded using NVENC

Integration
1

I have been using x264enc for a while with no issues encoding/decoding frames. However I have recently switched to nvenc and am experiencing issues get a stream.

When I send a stream using gstreamer with the following pipeline:

C:\gstreamer\1.0\mingw_x86_64\bin\gst-launch-1.0.exe rtpbin name=rtpbin rtp-profile=avpf dxgiscreencapsrc ! queue ! videorate ! video/x-raw,framerate=20/1 ! videoconvert ! video/x-raw,format=I420,framerate=20/1 ! nvh264enc zerolatency=true preset=1 ! "video/x-h264,profile=(string)high" ! h264parse ! rtph264pay ssrc=22222222 ! "application/x-rtp,payload=(int)123" ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink bind-address=172.31.3.209 host=172.31.3.209 bind-port=5004 port=39629 rtpbin.send_rtcp_src_0 ! udpsink bind-address=172.31.3.209 host=172.31.3.209 port=26936 sync=false async=false udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0

As well as the following media soup options and video producers:

videoProducer = await video_producer_transport.produce({
    kind: "video",
    rtpParameters: {
        codecs: [
            {
                mimeType: "video/H264",
                clockRate: 90000,
                payloadType: 123,
                parameters: {
                    "packetization-mode": 1,
                    "profile-level-id": "640032",
                    "level-asymmetry-allowed": 1,
                },
            },
        ],
        encodings: [{ ssrc: 22222222, scalabilityMode: "S1T1" }],
    },
});

mediasoup_options = { worker: {
    rtcMinPort: 10000,
    rtcMaxPort: 10100,
    logLevel: "warn",
    logTags: ["info", "ice", "dtls", "rtp", "srtp", "rtcp"],
},
router: {
    mediaCodecs: [
        {
            kind: "audio",
            mimeType: "audio/opus",
            clockRate: 48000,
            channels: 2,
        },
        {
            kind: "video",
            mimeType: "video/H264",
            clockRate: 90000,
            parameters: {
                "packetization-mode": 1,
                "profile-level-id": "640032",
                "level-asymmetry-allowed": 1,
            },
        },
    ],
},
webRtcTransport: {
    listenIps: [
        { ip: get_local_ip_address(), announcedIp: external_ip_address },
    ],
    enableUdp: true,
    preferUdp: true,
    maxIncomingBitrate: 1500000,
    initialAvailableOutgoingBitrate: 1000000,
},

I am able to receive packets, and frames however almost all the frames are being dropped without a single frame being decoded

It shows a few key frames decoded however even that is very low

Here is my webrtc get stats

image
image2624Ɨ1576 525 KB

Is there any issue with my pipeline or is it to do with my mediasoup options?

1 Like
2

did you solve it?
Iā€™m having exactly the same problem
I need to send 10 h264 real-time videos, nvh264enc is my only option