Having trouble running mediasoup rust mutiopus example

williamlsh · December 2, 2021, 10:21am

Hi! I’m trying to run multiopus example, everything is ok except no audio to listen to (with maximum volume).

Environment: Ubuntu server x86

Running with paired frontend app. Sending data with docker run --rm -it --net=host restreamio/gstreamer:latest-prod-dbg with instruction in server side log:

rtpbin name=rtpbin \
  souphttpsrc location=https://www2.iis.fraunhofer.de/AAC/ChID-BLITS-EBU-Narration.mp4 ! \
  queue ! \
  decodebin ! \
  audioresample ! \
  audioconvert ! \
  opusenc inband-fec=true ! \
  queue ! \
  clocksync ! \
  rtpopuspay pt=100 ssrc=1111 ! \
  rtpbin.send_rtp_sink_0 \
  rtpbin.send_rtp_src_0 ! udpsink host=127.0.0.1 port=54869 sync=false async=false \
  rtpbin.send_rtcp_src_0 ! udpsink host=127.0.0.1 port=19360 sync=false async=false

Here’s the docker container log:

(gst-plugin-scanner:11): GStreamer-WARNING **: 10:04:44.976: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so': libva-wayland.so.2: cannot open shared object file: No such file or directory
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Redistribute latency...
Redistribute latency...
Got context from element 'souphttpsrc0': gst.soup.session=context, session=(SoupSession)NULL, force=(boolean)false;
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...8.2 %)
Got EOS from element "pipeline0".
Execution ended after 0:00:46.453850277
Setting pipeline to NULL ...
Freeing pipeline ...

Besides, there’s no incoming data in chrome://webrtc-internals/.

nazar-pc · December 2, 2021, 10:31am

What logs does application print with RUST_LOG=debug?

williamlsh · December 2, 2021, 10:43am

That’s all the log:

[2021-12-02T10:40:16Z INFO  actix_server::builder] Starting 2 workers
[2021-12-02T10:40:16Z INFO  actix_server::server] Actix runtime found. Starting in Actix runtime
[2021-12-02T10:40:57Z DEBUG mediasoup::worker_manager] create_worker()
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] new()
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] spawning worker with arguments [id:12ea079f-b028-4dad-9e33-d473e29725e3]:  --logLevel=debug --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --logTag=message --rtcMinPort=10000 --rtcMaxPort=59999
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] mediasoup-worker::mediasoup_worker_run() | starting mediasoup-worker process [version:0.9.0]
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] mediasoup-worker::mediasoup_worker_run() | little-endian CPU detected
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] mediasoup-worker::mediasoup_worker_run() | 64 bits architecture detected
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] Settings::PrintConfiguration() | <configuration>
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] Settings::PrintConfiguration() |   logLevel            : debug
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] Settings::PrintConfiguration() |   logTags             : info,ice,dtls,rtp,srtp,rtcp,rtx,bwe,score,simulcast,svc,sctp,message
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] Settings::PrintConfiguration() |   rtcMinPort          : 10000
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] Settings::PrintConfiguration() |   rtcMaxPort          : 59999
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] Settings::PrintConfiguration() | </configuration>
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] DepLibUV::PrintVersion() | libuv version: "1.42.0"
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] DepOpenSSL::operator()() | openssl version: "OpenSSL 1.1.1l  24 Aug 2021"
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] DepLibSRTP::ClassInit() | libsrtp version: "libsrtp2 2.4.2"
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] DepUsrSCTP::ClassInit() | usrsctp
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] webrtc::FieldTrial::InitFieldTrialsFromString() | Setting field trial string: WebRTC-Bwe-AlrLimitedBackoff/Enabled/
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::DtlsTransport::CreateSslCtx() | setting SRTP cryptoSuites for DTLS: SRTP_AEAD_AES_256_GCM:SRTP_AEAD_AES_128_GCM:SRTP_AES128_CM_SHA1_80:SRTP_AES128_CM_SHA1_32
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::DtlsTransport::GenerateFingerprints() | sha-1   fingerprint: 36:00:B9:F1:A4:6E:2F:FB:57:B1:59:B6:9B:9F:2E:86:A8:94:E5:41
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::DtlsTransport::GenerateFingerprints() | sha-224 fingerprint: 3F:A3:85:D6:B5:CF:06:04:73:C2:7A:94:27:1F:C0:D6:F7:78:AC:66:C6:2E:02:4A:74:CD:27:CE
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::DtlsTransport::GenerateFingerprints() | sha-256 fingerprint: 83:54:79:E6:4D:75:F6:11:59:41:7E:D2:38:45:8C:3B:68:BC:BD:EC:2D:B5:3C:79:9B:7F:10:D8:D7:41:12:9A
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::DtlsTransport::GenerateFingerprints() | sha-384 fingerprint: FF:49:7A:10:C9:F1:C4:CE:72:EC:41:8F:C9:82:31:5D:35:9F:09:F8:0E:EE:2C:D8:5F:79:13:22:80:F4:28:EA:4E:60:42:01:70:6D:75:2D:F8:00:59:47:15:64:01:89
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::DtlsTransport::GenerateFingerprints() | sha-512 fingerprint: 93:70:12:0E:A9:8B:BF:93:E2:62:31:46:5E:2F:5D:92:CB:C7:45:45:7C:22:EA:A5:5A:A6:01:83:8A:A6:11:3C:E8:6D:05:C3:E8:80:EF:B3:26:40:6D:53:CB:2D:00:DB:FC:D9:23:A5:1D:D1:4E:69:91:50:AA:C9:2F:F0:79:D3
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] worker thread running [id:12ea079f-b028-4dad-9e33-d473e29725e3]
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] create_router()
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request() [method:worker.createRouter, id:0]: WorkerCreateRouterRequest { internal: RouterInternal { router_id: RouterId(d6514ee9-0f51-4eca-83e3-9279fb07c24b) } }
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request succeeded [method:worker.createRouter, id:0]
[2021-12-02T10:40:57Z DEBUG mediasoup::router] new()
[2021-12-02T10:40:57Z DEBUG mediasoup::router] create_plain_transport()
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request() [method:router.createPlainTransport, id:1]: RouterCreatePlainTransportRequest { internal: TransportInternal { router_id: RouterId(d6514ee9-0f51-4eca-83e3-9279fb07c24b), transport_id: TransportId(22750abe-b59d-4058-a38a-b2a149741733) }, data: RouterCreatePlainTransportData { listen_ip: TransportListenIp { ip: 127.0.0.1, announced_ip: None }, port: None, rtcp_mux: false, comedia: true, enable_sctp: false, num_sctp_streams: NumSctpStreams { os: 1024, mis: 1024 }, max_sctp_message_size: 262144, sctp_send_buffer_size: 262144, enable_srtp: false, srtp_crypto_suite: AesCm128HmacSha180, is_data_channel: false } }
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request succeeded [method:router.createPlainTransport, id:1]
[2021-12-02T10:40:57Z DEBUG mediasoup::router::plain_transport] new()
[2021-12-02T10:40:57Z DEBUG mediasoup::router::plain_transport] produce()
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request() [method:transport.produce, id:2]: TransportProduceRequest { internal: ProducerInternal { router_id: RouterId(d6514ee9-0f51-4eca-83e3-9279fb07c24b), transport_id: TransportId(22750abe-b59d-4058-a38a-b2a149741733), producer_id: ProducerId(9254d5c1-fea2-44ef-97f3-97ea0c450968) }, data: TransportProduceData { kind: Audio, rtp_parameters: RtpParameters { mid: Some("AUDIO"), codecs: [Audio { mime_type: MultiChannelOpus, payload_type: 100, clock_rate: 48000, channels: 6, parameters: RtpCodecParametersParameters({"channel_mapping": String("0,1,4,5,2,3"), "coupled_streams": Number(2), "num_streams": Number(4), "useinbandfec": Number(1)}), rtcp_feedback: [] }], header_extensions: [], encodings: [RtpEncodingParameters { ssrc: Some(1111), rid: None, codec_payload_type: None, rtx: None, dtx: None, scalability_mode: None, scale_resolution_down_by: None, max_bitrate: None }], rtcp: RtcpParameters { cname: Some("1bdd74fc-90c6-4333-817a-dcde758c1122"), reduced_size: true, mux: None } }, rtp_mapping: RtpMapping { codecs: [RtpMappingCodec { payload_type: 100, mapped_payload_type: 100 }], encodings: [RtpMappingEncoding { ssrc: Some(1111), rid: None, scalability_mode: None, mapped_ssrc: 340423985 }] }, key_frame_request_delay: 0, paused: false } }
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request succeeded [method:transport.produce, id:2]
[2021-12-02T10:40:57Z DEBUG mediasoup::router::producer] new()
Plain transport created:
  RTP listening on 127.0.0.1:31221
  RTCP listening on 127.0.0.1:34800
  PT=100
  SSRC=1111
Use following command with GStreamer (1.20+) to play a sample audio:
gst-launch-1.0 \
  rtpbin name=rtpbin \
  souphttpsrc location=https://www2.iis.fraunhofer.de/AAC/ChID-BLITS-EBU-Narration.mp4 ! \
  queue ! \
  decodebin ! \
  audioresample ! \
  audioconvert ! \
  opusenc inband-fec=true ! \
  queue ! \
  clocksync ! \
  rtpopuspay pt=100 ssrc=1111 ! \
  rtpbin.send_rtp_sink_0 \
  rtpbin.send_rtp_src_0 ! udpsink host=127.0.0.1 port=31221 sync=false async=false \
  rtpbin.send_rtcp_src_0 ! udpsink host=127.0.0.1 port=34800 sync=false async=false
[2021-12-02T10:40:57Z DEBUG mediasoup::router] create_webrtc_transport()
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request() [method:router.createWebRtcTransport, id:3]: RouterCreateWebrtcTransportRequest { internal: TransportInternal { router_id: RouterId(d6514ee9-0f51-4eca-83e3-9279fb07c24b), transport_id: TransportId(d3dde8b2-07ca-44f1-abb2-868d3600c913) }, data: RouterCreateWebrtcTransportData { listen_ips: TransportListenIps([TransportListenIp { ip: 127.0.0.1, announced_ip: None }]), port: None, enable_udp: true, enable_tcp: false, prefer_udp: false, prefer_tcp: false, initial_available_outgoing_bitrate: 600000, enable_sctp: false, num_sctp_streams: NumSctpStreams { os: 1024, mis: 1024 }, max_sctp_message_size: 262144, sctp_send_buffer_size: 262144, is_data_channel: true } }
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request succeeded [method:router.createWebRtcTransport, id:3]
[2021-12-02T10:40:57Z DEBUG mediasoup::router::webrtc_transport] new()
WebSocket connection created
[2021-12-02T10:40:57Z DEBUG mediasoup::router::webrtc_transport] consume()
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request() [method:transport.consume, id:4]: TransportConsumeRequest { internal: ConsumerInternal { router_id: RouterId(d6514ee9-0f51-4eca-83e3-9279fb07c24b), transport_id: TransportId(d3dde8b2-07ca-44f1-abb2-868d3600c913), consumer_id: ConsumerId(b60e1705-4cf2-4598-9a9e-b00faa5783ba), producer_id: ProducerId(9254d5c1-fea2-44ef-97f3-97ea0c450968) }, data: TransportConsumeData { kind: Audio, rtp_parameters: RtpParameters { mid: Some("0"), codecs: [Audio { mime_type: MultiChannelOpus, payload_type: 100, clock_rate: 48000, channels: 6, parameters: RtpCodecParametersParameters({"channel_mapping": String("0,1,4,5,2,3"), "coupled_streams": Number(2), "num_streams": Number(4), "useinbandfec": Number(1)}), rtcp_feedback: [] }], header_extensions: [RtpHeaderExtensionParameters { uri: Mid, id: 1, encrypt: false }, RtpHeaderExtensionParameters { uri: AbsSendTime, id: 4, encrypt: false }, RtpHeaderExtensionParameters { uri: AudioLevel, id: 10, encrypt: false }], encodings: [RtpEncodingParameters { ssrc: Some(370504970), rid: None, codec_payload_type: None, rtx: None, dtx: None, scalability_mode: None, scale_resolution_down_by: None, max_bitrate: None }], rtcp: RtcpParameters { cname: Some("1bdd74fc-90c6-4333-817a-dcde758c1122"), reduced_size: true, mux: Some(true) } }, type: Simple, consumable_rtp_encodings: [RtpEncodingParameters { ssrc: Some(340423985), rid: None, codec_payload_type: None, rtx: None, dtx: None, scalability_mode: None, scale_resolution_down_by: None, max_bitrate: None }], paused: false, preferred_layers: None } }
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::SimpleConsumer::CreateRtpStream() | [ssrc:370504970, payloadType:100]
[2021-12-02T10:40:57Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::SimpleConsumer::CreateRtpStream() | in band FEC enabled
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request succeeded [method:transport.consume, id:4]
[2021-12-02T10:40:57Z DEBUG mediasoup::router::consumer] new()
Audio consumer created: b60e1705-4cf2-4598-9a9e-b00faa5783ba
[2021-12-02T10:40:57Z DEBUG mediasoup::router::webrtc_transport] connect()
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request() [method:transport.connect, id:5]: TransportConnectWebRtcRequest { internal: TransportInternal { router_id: RouterId(d6514ee9-0f51-4eca-83e3-9279fb07c24b), transport_id: TransportId(d3dde8b2-07ca-44f1-abb2-868d3600c913) }, data: TransportConnectRequestWebRtcData { dtls_parameters: DtlsParameters { role: Client, fingerprints: [Sha256 { value: "A0:E5:3A:35:D0:E3:88:41:BA:89:5D:8C:F8:72:9E:94:C4:C6:F3:61:02:39:88:9B:82:20:3C:F0:5C:DE:09:9F" }] } } }
[2021-12-02T10:40:57Z DEBUG mediasoup::worker::channel] request succeeded [method:transport.connect, id:5]
Consumer transport connected
[2021-12-02T10:41:28Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::PlainTransport::OnRtpDataReceived() | setting RTP tuple (comedia mode enabled)
[2021-12-02T10:41:28Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::Producer::CreateRtpStream() | [encodingIdx:0, ssrc:1111, rid:, payloadType:100]
[2021-12-02T10:41:28Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::Producer::CreateRtpStream() | in band FEC enabled
[2021-12-02T10:41:31Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::PlainTransport::OnRtcpDataReceived() | setting RTCP tuple (comedia mode enabled)
[2021-12-02T10:42:11Z DEBUG mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::Transport::HandleRtcpPacket() | ignoring received RTCP BYE
[2021-12-02T10:42:12Z WARN  mediasoup::worker] [id:12ea079f-b028-4dad-9e33-d473e29725e3] RTC::RtpStreamRecv::OnTimer() | RTP inactivity detected, resetting score to 0 [ssrc:1111]

nazar-pc · December 2, 2021, 10:57am

Just in case, did you notice that ports and hence command for gst-launch are different each time?

williamlsh · December 2, 2021, 10:59am

Yes, I did notice. Use the same gst-launch command for twice woud result unkown ip port error.

nazar-pc · December 2, 2021, 7:55pm

Not sure what happens then, worked for me last time I tried it

williamlsh · December 3, 2021, 6:28am

I found what’s wrong with my last testing. It’s in this api (TransportListenIps in mediasoup::prelude - Rust), it should be set to host ip instead of localhost when running frontend app on another host.

nazar-pc · December 3, 2021, 6:42am

Yes, examples are designed to be used on localhost, if you want to use them on the server, you might need to modify IPs.

williamlsh · December 3, 2021, 6:50am

What if the ip of that running server is dynamic? How should I config then?

nazar-pc · December 3, 2021, 6:53am

You probably don’t want to have server with dynamic IP or more specifically it is unlikely that you’ll encounter a server whose IP changes all the time. In either case you need to know how your server can be reached. This can be done with static configuration and determined by the app in some automated way, but that is not mediasoup’s job and definitely outside of the scope of the example app.

williamlsh · December 3, 2021, 7:06am

Alright, thanks.

Topic		Replies	Views
Injecting audio through GStreamer error! Integration	1	1148	June 23, 2020
Broadcast with Gst mediasoup libraries	3	588	June 21, 2020
Gstreamer -> PlainTransport send opus Integration	4	621	February 3, 2021
Paid development support or consultancy needed Job Opportunities 💰	0	266	May 17, 2022
Cannot setup WebRTCTransport to mediasoup server inside of Docker instance, using ip 127.0.0.1(gstreamer pipeline cmd publisher in docker) Integration	18	1474	December 2, 2021

Having trouble running mediasoup rust mutiopus example

Related Topics