Using GStreamer webrtcbin as MediaSoup client

Hi all,
I’m trying to use the webrtcbin (https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad/html/gst-plugins-bad-plugins-webrtcbin.html) GStreamer element for communicating with MediaSoup.
Without entering into the GStreamer details, the element expects to exchange the SDP info with a webrtc peer (a web example here https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/js/webrtc.js).
I’m trying to reconstruct the SDP message from the JSON returned from mediasoup server. The communication starts, but I get these errors:

RTC::Transport::ReceiveRtpPacket() | no suitable Producer for received RTP packet [ssrc:824347444, payloadType:96]

The ssrc value is the same sent into the rtpParameters when creating the producer. I don’t understand if the mid and rid values can be used if the ssrc is not found into ssrcTable.

Anyway, what I need to know is how medasoup can interoperate with a client without using the mediasoup-client library.
Thanks

mediasoup follows ORTC rules for matching a RTP stream against the corresponding RTCRtpReceiver (Producer in mediasoup):

  • If the ssrc of the RTP packet was announced in the rtpParameters then it’s associated Producer will exist in the ssrcTable.
  • If the ssrc is not found in the ssrcTable then mediasoup will check its MID RTP extension (if any) and look for the associated Producer in the midTable. If found, it will enter the ssrc into the ssrcTable.
  • Otherwise it will do the same with the ridTable.

More in detail:

You can print const dump = await transport.dump() to print those tables (among other info). This is Chrome sending audio and video (with MID in both audio/video Producers and RID based simulcast in video Producer):

  "rtpListener": {
    "midTable": {
      "0": "2b39a73f-a4d3-40de-a3bf-c61383cff686",
      "2": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04"
    },
    "ridTable": {
      "r0": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04",
      "r1": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04",
      "r2": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04"
    },
    "ssrcTable": {
      "80064589": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04",
      "1190225763": "2b39a73f-a4d3-40de-a3bf-c61383cff686",
      "1884500992": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04",
      "2712523004": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04",
      "2747658646": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04"
    }
  },

You can also print const dump = await producer.dump() to see its effective rtpParameters and see if they match your expectations. Also, producer.getStats() will show stats only for the RtpStreams created in the Producer, this is, those for which at least a valid RTP packet has been received.

Ok, I fixed a code error. I have no errors from mediasoup, but the video flow from GStreamer seems stopped.
One thing I don’t understand is the candidates exchange: I see that mediasoup sends back to the client its candidates list after WebRtcTransport creation, but the client never sends its own candidates list to the server. Is it right?

Resolved (problem on GStreamer pipeline). Now seems to work perfectly.

mediasoup is ICE Lite server so it doesn’t need client candidates

1 Like

Hi Vittorio,

Could you please share your working GStreamer pipeline?

Thanks,

I’m integrating webrtcbin into https://github.com/vpalmisano/mediasoupbin
What you need to do is converting the GStreamer generated SDP message into the dtlsParameters and rtpParameters structures and send them to the produce call to MediaSoup.

1 Like