[Flutter] No audio when connecting to the mediasoup demo server

Hi,

I am trying to consume an audio stream from a mobile app (written in flutter).
I have connected to the mediasoup demo and got an incoming audio stream.

When I check the track statistics I see that no packets are sent.

My guess is that there is a problem with the SDP manipulations during the “consume” call.
I would appreciate any pointers of where to look for the problem.

Thanks,
Yonatan

No easy response. Some points to consider:

  • mediasoup (server) exposes tons of logs and events and stats to verify whether packets are being sent to a remote client or not. Check them.
  • Ensure ICE and DTLS is connected.
  • Compare the SDPs of mediasoup-client and your lib for the same scenario.

Thanks for these points, indeed there seem to be an issue with ICE.

I am currently using trickle ICE and I get ice candidates from the peer connection.

But as far as I can understand from the client code I should use the ice candidate list from the server response.

Is this the correct approach? Or should I wait for the ice gathering to complete and use the candidate that were gathered?

Cheers,
Yonatan

mediasoup (server) is an ICE Lite server so no gathering is required from client. No ICE candidates are signalled from mediasoup-client to mediasoup server.

Hi Inaki,

I followed your advice and verified the SDP and ICE which seems ok.

I see an issue in the DTLS connection:
mediasoup:ERROR:Channel [pid:51152 RTC::DtlsTransport::CheckStatus() | OpenSSL error [desc:‘SSL status: SSL_ERROR_SSL’, error:‘error:141A10F4:SSL routines:ossl_statem_client_read_transition:unexpected message’]

Any idea what I can do differently to work around this?

Thanks,
Yonatan

Probably you are not respecting the DTLS role and are acting as DTLS client while you tell mediasoup server (via signalling) that you are DTLS server. However hard to figure out.

BTW is yours a public project? is the code somewhere?

This sounds like a promising direction, I will investigate this and let you know.

The code is not yet public, I will package it and make it public so you can have a look.
Bear in mind its still work in progress :slight_smile:

I’ve packaged the code into a new flutter plugin.
https://bitbucket.org/yonatannaor/flutter_mediasoup/src/master/

I verified its working on ios and android with the proper permissions.
I tried to follow the code structure of the JS mediasoup client. The one major change is that I don’t use the concept of handlers since this code is flutter only so I merged the handler functionality into the Device and Transport objects.

It is not yet organised since I am in the initial stages.
I believe this can greatly increase the native usage of mediasoup since it provides a unified codebase.
Flutter supports ios/android and even web (still not widely spread)

I’ll be happy to hear your feedback

Thanks. Definitely I have no idea of Dart language so I cannot provide good feedback. Just a few comments:

  • I see a Request class, which AFAIU is about network signalling, and should not be part of the lib itself.
  • I see you subscribe to PeerConnection events such as pc.onAddTrack and things like that. In mediasoup-client we completely avoid that since we do know which remote tracks we are producing in advance so we do not listen for “hey you got a new track that you already knew because you gave it to me via a remote SDP re-offer”.
  • I see pc.addStream(). No idea about current status of libwebrtc for Dart (does it even exist?) but definitely pc.addStream() is API from 8 years ago.

Honestly I do not like this point. We may have done the same in libmediasoupclient which is a C++ version of mediasoup-client that uses libwebrtc internally so we also only have a “handler” and still keep the design of mediasoup-client into it by splitting Transport and Handler.

The only advice I can give right now is: follow mediasoup-client “v3” branch (which has been updated recently BTW) step by step. If things are different any future change will be super hard to sync and figuring out why things don’t work won’t be easy.

I completely agree with all your points, and I accept the change regarding the transport/handler split
The code is mainly a POC so it is a bit of a mess, I will clean it up to follow the mediasoup client structure.

Your advice regarding the dtls role was point on, I am able to receive audio now!!

Thanks!

Great :slight_smile: