broadcasting a vp8 rtp stream from gstreamer

Integration
1

Hi community guys,

i failed to find a way to stream a vp8 encoded raw rtp stream from gst-launch to media soup to have it display the video (with the demo application)

i am looking for something equivalent to the following ffmpeg command

ffmpeg
-re
-v info
-stream_loop -1
-rtsp_transport tcp
-i {MEDIA_FILE} \ -map 0:v:0 \ -pix_fmt yuv420p -c:v libvpx -b:v 1000k -deadline realtime -cpu-used 4 \ -f tee \ "select=v:f=rtp:ssrc={VIDEO_SSRC}:payload_type={VIDEO_PT}]rtp://{transportIp}:${transportPort"

the transport is defined as rtpParameters:="{ “codecs”: [{ “mimeType”:“video/vp8”, “payloadType”:{VIDEO_PT}, \"clockRate\":90000 }], \"encodings\": [{ \"ssrc\":{VIDEO_SSRC} }] }" \

the trivial pipeline below doesnt work

! decodebin
! vp8enc ! rtpvp8pay pt={VIDEO_PT} ssrc={VIDEO_SSRC}
! udpsink host={transportIp} port={transportPort}

2

Definitely we are not experts in FFmpeg or GStreamer, so we refer to the documentation and forums of those projects:

1 Like
3

When using vp8/vp9, you should set the rtpvp{8,9}pay picture-id-mode={1,2} parameter. Moreover, you should use the rtpbin element (https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good/html/gst-plugins-good-plugins-rtpbin.html) in order to handle retransmissions and keyframe requests.

1 Like
4

@ibc investigating the rtcp feedbacks received by gstreamer, a found that when mediasoup sends a PLI request, gstreamer gets a RTCP packet with “sender_ssrc” equals to “0”, and so it fails to reply to the request.

In particular, the received RTCP packet has:

  • type = GST_RTCP_TYPE_PSFB (Payload-specific feedback)
  • fbtype = GST_RTCP_PSFB_TYPE_PLI
  • source_ssrc = 0

Full received RTCP buffer: ‘\x81\xce\x00\x02\x00\x00\x00\x00\x00\x98\x96\x80’

1 Like
5

Can you put a packet.Dump() after the PLI packet is created here?

We pass the media SSRC to it.

6

@vpalmisano refers to the source ssrc, which is 0 because the sender of the RTCP is not RTP source by any means. It must be set to 0 since mediasoup is only receiving RTP from that stream, it is not the typical bidirectional sendrecv SDP formed stream.

7

This is what I see doing it:

mediasoup:worker[pid:82416] RTC::RtpStreamRecv::RequestKeyFrame() | sending PLI [ssrc:4067265976]
RTC::RTCP::FeedbackPsPli::Dump() | <FeedbackPsPliPacket>
RTC::RTCP::Feedback::Dump() |   sender ssrc : 0
RTC::RTCP::Feedback::Dump() |   media ssrc  : 4067265976
RTC::RTCP::Feedback::Dump() |   size        : 12
RTC::RTCP::FeedbackPsPli::Dump() | </FeedbackPsPliPacket>

Clearly the PLI has a media ssrc field with the SSRC of the RTP stream. If GStreamer is expecting something different from 0 in the sender ssrc then it must be a bug in GStreamer as obviously we cannot put any other value on it.

8

Just to confirm, @vpalmisano can you modify this line in mediasoup as follows?

- RTC::RTCP::FeedbackPsPliPacket packet(0, GetSsrc());
+ RTC::RTCP::FeedbackPsPliPacket packet(GetSsrc(), GetSsrc());

and see what happens? Even if it works, GStreamer is wrong about it.

9

I have modified the received RTCP buffer received on UDP by GStreamer setting the SSRC sent in the “encodings” dict, now GStreamer is able to handle the PLI request.
If this is a GStremer bug, I can use this approach without modifications of mediasoup.

10

This seems an important issue and we are pretty sure we are doing it good in mediasoup. This may be related code in GStreamer. May you please report it in the GStreamer project?

11

I’m looking exactly at the same code. Maybe because it can handle different ssrc RTCP reports on the same UDP connection?

12

I don’t know if GStreamer can send different RTP streams on the same UDP connection (BUNDLE), probably it should be able to do it. And if so it must check the media ssrc in received PLIs.

I insist that mediasoup is correctly setting the media ssrc field in PLI packets. sender ssrc is another story and it’s 0 because there is no bidirectional RTP sending.

I have modified the received RTCP buffer received on UDP by GStreamer setting the SSRC sent in the “encodings” dict

So you are basically mangling the received PLI packet before GStreamer processes it. What are you exactly overriding in the PLI packet? the sender ssrc?

13

Yes, settings the same ssrc sent at the producer creation.

14

Will you report an issue in Gstreamer? I’d like to have that as a workaround :frowning:

BTW Gstreamer seems to look first for “sender ssrc” and, if no session is found then it looks for “media ssrc”. May be there is an issue in Gstreamer when reading those ssrcs (network order, etc)? Why is it finding a session by looking to “sender ssrc = 0”?

15

In order to find the exact bug in Gstreamer, @vpalmisano may you please try this?

16

I will try tomorrow. If the effect is the same of mangling the packet content, I think the result will be the same.

17

Question: when using PipeTransports, the PLI requests are sent to the pipe origin producer?

18

GStreamer first looks at the sender ssrc. If it gets NULL then it tries with media ssrc. So, if my proposal makes it work it means that, somehow, GStreamer is getting “something” when looking for sender ssrc = 0, which is crazy.

BTW: when you did the UDP packet mangling to set sender ssrc, did you properly write the ssrc in network order?

19

Yes. The flow is:

  • Consumer2 (in a WebRtcTransport in router2) requests a key frame to its Producer in router2.
  • Producer in router2 generates a PLI or FIR and sends it over its PipeTransport which communicates with router1.
  • The PLI/FIR reaches router1’s PipeTransport which delivers the PLI/FIR to the PipeConsumer in router1.
  • The PipeConsumer in router1 requests a keyframe to its associated Producer in router1.
  • The Producer in router1 sends a PLI or FIR to the producing endpoint.
20

I need to update my GStreamer to the latest version. The stable version on Ubuntu is 1.14.1 and there is no media src reference in the rtp_session_process_pli function (https://github.com/GStreamer/gst-plugins-good/blob/1.14.1/gst/rtpmanager/rtpsession.c#L2606).

Yes, I write the ssrc in network order.