How to let libmediasoupclient client sdk accept encoded video stream as input?

Integration
1

I need to implement a webrtc RTP sender client with decoupled 2 parts:

One is the H264 encoded stream source, which produce series of I- & P-frames (logically);
The other accepts the encoded video stream as input, and do RTP send stream to mediasoup SFU server.

The mediasoup-broadcaster-demo code depends on libmediasoupclient C++ client sdk, which further depends on libwebrtc, doesn’t provide useful sample code pieces for doing this.

When i take a look at the libwebrtc source, there is 2 concepts:

  1. VideoFrame which is normally taken as libwebrtc’s input type, but it’s YUV420p raw format and not encoded by codecs. The video encoding occurs internally in libwebrtc, which i don’t need, since i have encoded video stream as input…

  2. EncodedFrame which seems exactly what i need, but there is no convenient way to construct by parsing from a H264 encode video stream…

Look for advice.

Maybe i should not use libwebrtc but switch to directly use ffmpeg’s source to do this?

2

Previously i have experimenting use ffmpeg or gstreamer command line to copy the video stream part to RTP connection, but mediasoup server side seems stuck on a PLI keyframe request…

I’ve known there is a single directional mode RTP push stream case without RTCP feedback, (e.g. use a java forward proxy to RTP-packetize the encode video stream to a SFU server), this method doesn’t need to depend on libwebrtc, but loses the advanced feature of RTCP. How can mediasoup config support this?

3

I guess the more accurate question here is: when the webrtc receiver side is browser page, how to make it not send any RTCP feedback packet? The mediasoup SFU server just forward the receiver’s RTCP feedback packet, so if can disbale this, then a simple H264-over-RTP custom client is possible…

4

You could send raw h264 packet to mediasoup by plaintransport or directtransport， then you can consume the media via webrtctransport or other transport from mediasoup.

If you want to inject encoded stream source to mediasoup, you don’t need most of rtcp feedbacks as your producer cannot change the encoded stream。Using other tools instead of libmediasoupclient.

There are lots of work to do to use libwebrtc peerconnection api sending encoded packet.

5

plaintransport can accept raw h264 packet?

I’m now writing C++ code to do RTP-packetize h264 stream to RTP packets, and will also add codes to do RTCP-feedback response(extract I-frames from h264 stream as keyframe)

6

libwebrtc is ok for consumer side SDK, i thought…

i want to further custom the consumer side SDK to remove SDP negotiation and DTLS handshake to archieve 0-RTT performance, but that’s another question~

7

plaintransport can accept raw h264 packet?

No, RTP packer for h264 is needed.

8

See: rtc::VideoSinkInterface<webrtc::VideoFrame> 嘿嘿

9

This should be YUV data.