# Using GStreamer webrtcbin as MediaSoup client **URL:** \ **Category:** Integration\ **Created:** [February 21, 2020, 5:29pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590 "2020-02-21T17:29:21Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [February 21, 2020, 5:29pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/1 "2020-02-21T17:29:21Z")
Hi all, I’m trying to use the webrtcbin ([https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad/html/gst-plugins-bad-plugins-webrtcbin.html](https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad/html/gst-plugins-bad-plugins-webrtcbin.html)) GStreamer element for communicating with MediaSoup. Without entering into the GStreamer details, the element expects to exchange the SDP info with a webrtc peer (a web example here [https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/js/webrtc.js](https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/js/webrtc.js)). I’m trying to reconstruct the SDP message from the JSON returned from mediasoup server. The communication starts, but I get these errors: ```auto RTC::Transport::ReceiveRtpPacket() | no suitable Producer for received RTP packet [ssrc:824347444, payloadType:96] ``` The `ssrc` value is the same sent into the `rtpParameters` when creating the producer. I don’t understand if the `mid` and `rid` values can be used if the `ssrc` is not found into `ssrcTable`. Anyway, what I need to know is how `medasoup` can interoperate with a client without using the `mediasoup-client` library. Thanks ---
**Author:** ![ibc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/ibc/32/1540_2.png) [@ibc](https://mediasoup.discourse.group/u/ibc)\ **Post date:** [February 21, 2020, 5:44pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/2 "2020-02-21T17:44:58Z")
mediasoup follows ORTC rules for matching a RTP stream against the corresponding `RTCRtpReceiver` (`Producer` in mediasoup): - If the ssrc of the RTP packet was announced in the `rtpParameters` then it’s associated `Producer` will exist in the `ssrcTable`. - If the ssrc is not found in the `ssrcTable` then mediasoup will check its `MID` RTP extension (if any) and look for the associated `Producer` in the `midTable`. If found, it will enter the ssrc into the `ssrcTable`. - Otherwise it will do the same with the `ridTable`. More in detail: - When a `Producer` is created (via `transport.produce()`) it’s inserted into these tables (within a specific `Transport`) as follows: - [https://github.com/versatica/mediasoup/blob/v3/worker/src/RTC/RtpListener.cpp?ts=2#L53](https://github.com/versatica/mediasoup/blob/v3/worker/src/RTC/RtpListener.cpp?ts=2#L53) - When a RTP packets arrives into a `Transport`, its `Producer` is found as follows: - [https://github.com/versatica/mediasoup/blob/v3/worker/src/RTC/RtpListener.cpp?ts=2#L169](https://github.com/versatica/mediasoup/blob/v3/worker/src/RTC/RtpListener.cpp?ts=2#L169) You can print `const dump = await transport.dump()` to print those tables (among other info). This is Chrome sending audio and video (with MID in both audio/video Producers and RID based simulcast in video Producer): ```json "rtpListener": { "midTable": { "0": "2b39a73f-a4d3-40de-a3bf-c61383cff686", "2": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04" }, "ridTable": { "r0": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04", "r1": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04", "r2": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04" }, "ssrcTable": { "80064589": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04", "1190225763": "2b39a73f-a4d3-40de-a3bf-c61383cff686", "1884500992": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04", "2712523004": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04", "2747658646": "cfe2c7cc-84ef-426c-b36d-a1d77e7feb04" } }, ``` You can also print `const dump = await producer.dump()` to see its effective `rtpParameters` and see if they match your expectations. Also, `producer.getStats()` will show stats **only** for the `RtpStreams` created in the `Producer`, this is, those for which at least a valid RTP packet has been received. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [February 24, 2020, 11:04am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/3 "2020-02-24T11:04:59Z")
Ok, I fixed a code error. I have no errors from mediasoup, but the video flow from GStreamer seems stopped. One thing I don’t understand is the candidates exchange: I see that mediasoup sends back to the client its candidates list after WebRtcTransport creation, but the client never sends its own candidates list to the server. Is it right? ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [February 24, 2020, 11:21am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/4 "2020-02-24T11:21:31Z")
Resolved (problem on GStreamer pipeline). Now seems to work perfectly. ---
**Author:** ![ibc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/ibc/32/1540_2.png) [@ibc](https://mediasoup.discourse.group/u/ibc)\ **Post date:** [February 24, 2020, 11:33am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/5 "2020-02-24T11:33:32Z")
mediasoup is ICE Lite server so it doesn’t need client candidates ---
**Author:** ![copiltembel](https://avatars.discourse-cdn.com/v4/letter/c/e9a140/32.png) [@copiltembel](https://mediasoup.discourse.group/u/copiltembel)\ **Post date:** [February 25, 2020, 9:50am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/6 "2020-02-25T09:50:36Z")
Hi Vittorio, Could you please share your working GStreamer pipeline? Thanks, ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [February 25, 2020, 10:50am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/7 "2020-02-25T10:50:14Z")
I’m integrating webrtcbin into [https://github.com/vpalmisano/mediasoupbin](https://github.com/vpalmisano/mediasoupbin) What you need to do is converting the GStreamer generated SDP message into the `dtlsParameters` and `rtpParameters` structures and send them to the `produce` call to MediaSoup. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [June 17, 2020, 9:47am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/8 "2020-06-17T09:47:08Z")
Hi all, I’m trying to finish the integration between GStreamer webrtcbin element and Mediasoup and I’ve found a blocking problem when I try to consume a WebRTC flow from Mediasoup. When used as receiver, webrtcbin configures its ICE connection as `controlled` ([https://github.com/GStreamer/gst-plugins-bad/blob/1.16.2/ext/webrtc/gstwebrtcbin.c#L2721](https://github.com/GStreamer/gst-plugins-bad/blob/1.16.2/ext/webrtc/gstwebrtcbin.c#L2721)) and mediasoup emits this warning: “peer indicates ICE-CONTROLLED in STUN Binding Request”, maybe because using ICE-lite? EDIT: this seems to be fixed in the latest release: [https://github.com/GStreamer/gst-plugins-bad/blob/master/ext/webrtc/gstwebrtcbin.c#L4469](https://github.com/GStreamer/gst-plugins-bad/blob/master/ext/webrtc/gstwebrtcbin.c#L4469) ---
**Author:** ![nazar-pc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/nazar-pc/32/369_2.png) [@nazar-pc](https://mediasoup.discourse.group/u/nazar-pc)\ **Post date:** [June 17, 2020, 11:05am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/9 "2020-06-17T11:05:22Z")
Yes, this was the issue I faced and on my request it is already fixed in master branch. Please note though that RTX is not working with renegotiations in that case though, so make sure to have `do-nack=false`. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [June 17, 2020, 3:28pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/10 "2020-06-17T15:28:36Z")
Do you mean this: [https://github.com/centricular/gstwebrtc-demos/blob/4351ec1e2b27117051409c63ce1a049f5a54cb31/android/app/src/main/jni/webrtc.c#L321](https://github.com/centricular/gstwebrtc-demos/blob/4351ec1e2b27117051409c63ce1a049f5a54cb31/android/app/src/main/jni/webrtc.c#L321) ? ---
**Author:** ![nazar-pc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/nazar-pc/32/369_2.png) [@nazar-pc](https://mediasoup.discourse.group/u/nazar-pc)\ **Post date:** [June 17, 2020, 3:33pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/11 "2020-06-17T15:33:52Z")
Yes, you can set that on transceiver, but it is already `false` by default, so should be fine out of the box. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [June 19, 2020, 5:08pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/12 "2020-06-19T17:08:52Z")
I checked the `do-nack` property in the transceiver, it is `False` by default. Anyway, after I setup the pipeline, I receive only some initial rtp packets, then the `nicesrc` element inside `webrtcbin` doesn’t receive nothing. I checked on Mediasoup side, the `consumer` is emitting rtp traces, so all seems fine on server side. My receiver setup flow: ```auto signalling.createWebRtcTransport -> transport signalling.consume(transport.id, producerId) -> consumer [transform the transport json into a SDP offer representation] webrtcbin.emit('set-remote-description', sdp_offer) webrtcbin.emit('add-ice-candidate', ) webrtcbin.emit('create-answer') -> client answer sdp webrtcbin.emit('set-local-description', answer) [get the dtlsParameters from answer sdp] signalling.transportConnect(transport, dtlsParameters) ``` ---
**Author:** ![nazar-pc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/nazar-pc/32/369_2.png) [@nazar-pc](https://mediasoup.discourse.group/u/nazar-pc)\ **Post date:** [June 19, 2020, 6:50pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/13 "2020-06-19T18:50:21Z")
Similar flow works for me, but I think I didn’t use `add-ice-candidate`, but rather had them in SDP itself. Essentially I wrote serializer to SDP such that it produces identical result to Chrome 74 handler from mediasoup-client. Also did you create paused consumer that is resumed afterwards just like with browsers? ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [June 19, 2020, 7:06pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/14 "2020-06-19T19:06:37Z")
> [@nazar-pc](#): > > Also did you create paused consumer that is resumed afterwards just like with browsers? Yes, this is an offer example: ```auto v=0 o=mediasoup-client 10000 1 IN IP4 0.0.0.0 s=- t=0 0 a=ice-lite a=fingerprint:sha-512 62:E3:4B:82:7C:9E:9E:82:FC:B0:16:D:FC:86:F3.XX.XX.XX a=msid-semantic: WMS * a=group:BUNDLE 0 m=video 7 UDP/TLS/RTP/SAVPF 101 106 c=IN IP4 127.0.0.1 a=rtpmap:101 VP8/90000 a=rtpmap:106 rtx/90000 a=rtcp-fb:101 nack a=rtcp-fb:101 nack pli a=setup:active a=mid:0 a=sendonly a=ice-ufrag:XXXXXXXX a=ice-pwd:XXXXXXXXXXXXXXXXXXXX a=candidate:hostcandidate 1 udp 1076558079 127.0.0.1 16885 typ host a=candidate:hostcandidate 1 udp 1076532479 10.73.0.137 10908 typ host a=candidate:hostcandidate 1 udp 1076506879 192.168.0.2 18188 typ host a=candidate:hostcandidate 1 udp 1076481279 10.0.0.137 10442 typ host a=ice-options:renomination a=rtcp-mux a=rtcp-rsize a=ssrc:332426526 cname:346d56e9 a=ssrc-group:FID 332426526 a=msid:346d56e9 b0d78830-b2cc-4e87-9763-8210a2680d9c ``` and `webrtcbin` answer: ```auto v=0 o=- 10000 1 IN IP4 0.0.0.0 s=- t=0 0 a=group:BUNDLE 0 m=video 9 UDP/TLS/RTP/SAVPF 101 c=IN IP4 0.0.0.0 a=ice-ufrag:/RaaVE9/USqq2c0fuM9T36/sxXXXXXX a=ice-pwd:7AaL26nZTyghXXXXXXXXXXXXXX a=mid:0 a=rtcp-mux a=setup:passive a=rtpmap:101 VP8/90000 a=rtcp-fb:101 nack pli a=recvonly a=fingerprint:sha-256 F9:B9:8A:EA:XXXXX ``` ---
**Author:** ![nazar-pc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/nazar-pc/32/369_2.png) [@nazar-pc](https://mediasoup.discourse.group/u/nazar-pc)\ **Post date:** [June 19, 2020, 7:23pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/15 "2020-06-19T19:23:24Z")
I think `a=fmtp:102 apt=101` is missing and try to change `a=setup:active` to `a=setup:actpass`, `a=end-of-candidates` would be nice to have as well. There is nothing really secret in serializing, so here is exactly what I’m using and what works (it is also covered by tests, but I didn’t publish them), hopefully it helps: [https://gist.github.com/nazar-pc/58ffd40db680ecae07198b15b919057e](https://gist.github.com/nazar-pc/58ffd40db680ecae07198b15b919057e) ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [June 20, 2020, 7:02am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/16 "2020-06-20T07:02:19Z")
> [@nazar-pc](#): > > `a=fmtp:102 apt=101` is missing Nothing changed after adding this line. The strange thing is that the peerconnection starts and I receive the first video frames that are correctly decoded. After some 5-10 received frames, the `nicesrc` element seems stalled, maybe there be some threading problem in the pipeline. > [@nazar-pc](#): > > change `a=setup:active` to `a=setup:actpass` With this setting a get this error: ```auto dtlsconnection gstdtlsconnection.c:964:handle_error: Fatal SSL error ``` > [@nazar-pc](#): > > hopefully it helps: [mediasoup-sdp-serializer.ts · GitHub](https://gist.github.com/nazar-pc/58ffd40db680ecae07198b15b919057e) Just tried, same result. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [June 20, 2020, 8:03am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/17 "2020-06-20T08:03:19Z")
I found the problem: `webrtcbin` element requires always a source element, even if in receive mode only ([https://github.com/centricular/gstwebrtc-demos/blob/master/sendonly/webrtc-recvonly-h264.c#L289](https://github.com/centricular/gstwebrtc-demos/blob/master/sendonly/webrtc-recvonly-h264.c#L289)). I fixed for now using this pipeline: ```auto audiotestsrc is-live=true num-buffers=100 ! opusenc ! rtpopuspay ! webrtcbin name=webrtcbin bundle-policy=max-bundle ! rtpvp8depay ! vp8dec name=sink ``` ---
**Author:** ![nazar-pc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/nazar-pc/32/369_2.png) [@nazar-pc](https://mediasoup.discourse.group/u/nazar-pc)\ **Post date:** [June 20, 2020, 8:57pm UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/18 "2020-06-20T20:57:49Z")
This is not true. I’m not having any sink pads on `webrtcbin` and it works just fine. ---
**Author:** ![coreyleoc](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/coreyleoc/32/393_2.png) [@coreyleoc](https://mediasoup.discourse.group/u/coreyleoc)\ **Post date:** [January 5, 2021, 12:45am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/19 "2021-01-05T00:45:28Z")
Are you creating the webrtcbin with gst\_element\_factory\_make or gst\_parse\_launch? It may be that you need to pass something to the webrtcbin sink if it is created with a launch string. ---
**Author:** ![vpalmisano](https://yyz2.discourse-cdn.com/free1/user_avatar/mediasoup.discourse.group/vpalmisano/32/43_2.png) [@vpalmisano](https://mediasoup.discourse.group/u/vpalmisano)\ **Post date:** [January 12, 2021, 9:33am UTC](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590/20 "2021-01-12T09:33:51Z")
> [@coreyleoc](#): > > gst\_parse\_launch I’m using `gst_parse_launch`, called dynamically inside my script. [Next page](https://mediasoup.discourse.group/t/using-gstreamer-webrtcbin-as-mediasoup-client/590.md?page=2)