How to add media stream to a producer

Integration
41

i used docker because i communicate throw 2 differente micro serrvice and because i am on windows and the installation wasn’t easy at the beginning, now i think it work if i run it in localhost.
But i needed another micro seervice who manage the websockets events.

So the p roblem can come from the back end if i understand what you say ?
If the back end doesn’t return any response, it will return undefined or if i use an await promise he will be stuck and the console.log will never be triggered.

am i right ?

42

No. Once again: if the code await new Promise((resolve) => { this.sendTransport.on(‘connect’, ... does not result in “Uncaught (in promise)” error, than this.sendTransport and this.sendTransport.on are defined, because this code is executed immediately. So you can just proceed with this.sendTransport.produce(). Nothing from the outside can stop mediasoup-client from emitting a transport ‘connect’ event, providing that the arguments of the produce() are correct. And if something is incorrect, an exception is thrown.

43

So i don’t have to listen event connect on sendTransport ?
I can directly listen produce event ? And if i have the same error with produce event ?

44

if i try to use produce on sendTransport without listening conenct event before i have this error:

core.mjs:6412 ERROR Error: Uncaught (in promise): TypeError: no "connect" listener set into this transport
TypeError: no "connect" listener set into this transport

for this code:

await new Promise<void>(async (resolve) => {
        console.log('sendTransport', this.sendTransport);
        this.producer = await this.sendTransport.produce({
          track: this.localStream.getTracks()[0],
          codecOptions: {
            opusStereo: true,
            opusDtx: true,
          },
        });
      });
45

You have to listen both events. These events are in the middle of the transport.produce() function, and emitted when the client needs to exchange data with the server, with the help of you signalling (like in classic p2p WebRTC). Only the ‘connect’ event is emitted just once, when the transport is new, and the ‘produce’ event is issued every time you call transport.produce() with a new track.

46

yes transport.produce is triggered each time i update the track ok.
but like i explained you how i architect my front end and back end, everything seems ok for you ?
No error can come from back end ?
If the connect event can’t connect or don’t find the mediasoup server, he should return me an error no ?

47

when i check the documentation, for the connect we have this :

transport.on("connect", async ({ dtlsParameters }, callback, errback) =>
{
  // Signal local DTLS parameters to the server side transport.
  try
  {
    await mySignaling.send(
      "transport-connect",
      {
        transportId    : transport.id, 
        dtlsParameters : dtlsParameters
      });

    // Tell the transport that parameters were transmitted.
    callback();
  }
  catch (error)
  {
    // Tell the transport that something was wrong.
    errback(error);
  }
});

Maybe i have to call the back end server who have to send a response to resolve this issue ?

48

And btw in the doc.
We create transport like this:

const transport = device.createSendTransport(
  {
    id             : "0b38d662-ea00-4c70-9ae3-b675d6a89e09",
    iceParameters  : { ... },
    iceCandidates  : [ ... ],
    dtlsParameters : { ... },
    sctpParameters : { ... }
  });

With sctpParameters as parameter, i don’t don’t have sctpParameter in my transport options.
Again here the transport options i have:

{id: 'e95809b2-84f9-42a9-a349-d888a564303e', iceParameters: {…}, iceCandidates: Array(1), dtlsParameters: {…}}
dtlsParameters: {fingerprints: Array(5), role: 'auto'}
iceCandidates:  [{…}]
iceParameters: {iceLite: true, password: 'm7gdc4gn90r528y0roxepd1oatw2zprl', 
usernameFragment: '0peny2u53exv8ys6xavg3nnysbughndn'}
id: "e95809b2-84f9-42a9-a349-d888a564303e"
[[Prototype]]: Object

I know i am not far away to resolve that.

49

errback function is for handling the errors that may occur in signalling. To track the state of the сonnection you should listen ‘connectionstatechange’ event of the transport.

50

This is ok, sctpParameters are only needed for data channels I guess

51

@indie transport.on(‘connect’), transport.on(‘produce’) need to be listened, show use the code from your side step by step from transport creation to transport.produce(), in txt file to have a better look.

If you have this thing live somewhere then share the link, it is difficult to diagnose otherwise.

52

ok guys i guess i have something.
Now i don’t have anymore error.

Here what i have, i read your toughts and explain what i did.
First Here the full front end code for the device part:

await new Promise<void>(async (resolve) => {
        try {
          console.log('sendTransport', this.sendTransport);
          this.sendTransport.on('connect', ({ dtlsParameters }, callback) => {
            console.log("dtlsParameters");
            console.log(dtlsParameters);
            // Send dtlsParameters to the backend
            callback();
            resolve();
          });
          
          this.sendTransport.on("produce", async (parameters, callback, errback) =>
          {
            console.log("parameters produce");
            console.log(parameters);
            // console.log(dtlsParameters);
            // Send dtlsParameters to the backend
            // callback();
            resolve();
          });

          this.producer = await this.sendTransport.produce({
            track: this.localStream.getTracks()[0],
            codecOptions: {
              opusStereo: true,
              opusDtx: true,
            },
          });
          console.log('test 1');
          resolve();
        } catch (Exception) {
          console.log('Exception 1');
          console.log(Exception);
        }
      });

      console.log(1);

It trigger all debug messages.
I put both event ‘connect’ and ‘produce’ in an Promise, resolve both event.
If you call produce without conenct it doesn’t work and if you call connect without produce it doesn’t work either i guess.
So here is the solution above.

So it’s a victory i guess ?
Do i miss something ?
Do i have to send event to the back end now ? Or is the media sent to my sfu back server and everything is done for my producer step ?

For my toughts yes, i have to send the media stream to the sfu server so he can deliver the stream throw each consumer who connect to the room.
But maybe just identifying the producer transport is enough ?

53

I have updated the code for handling all track and not just the first one of the tracks, but produce event only send the first track, weird behavior.
Should i connect again and produce again for the second track ?

Like just putting my Promise block of code in:

this.localStream.getTracks().forEach(async (track) => {
 // Promise here
});

But that mean i listen again ‘connect’ event

54

Someone know how to set the docker local ip address to the server when wreating the server ?

55

i have this error in my front end when i try to consume a producer stream:
WebRTC: ICE failed, add a STUN server and see about:webrtc for more details

Is it related of why i don’t receive any stream ?
I have my track who look like this:

MediaStream { id: "{c1bfcd68-9688-4419-a86f-455038ecf9b3}", active: true, onaddtrack: null, onremovetrack: null }

active: true
​
id: "{c1bfcd68-9688-4419-a86f-455038ecf9b3}"
​
onaddtrack: null
​
onremovetrack: null
​
<prototype>: MediaStreamPrototype { getAudioTracks: getAudioTracks(), getVideoTracks: getVideoTracks(), getTracks: getTracks(), … }

Whatever ip address i have in the icecandidate i have the same result.
Now that i have the real IP of the container i still have the same result, no stream showing…

Is it tnormal to have a result like this for rthe MediaStream ?

MediaStream { id: "{a56f4832-8a64-4346-ae82-aa0899684505}", active: true, onaddtrack: null, onremovetrack: null }
56

And i have no websocket related to the server with mediasoup showing in my dev tools of my browser

57

if i understand well, and about what i saw, mediasoup doesn not need any stun and turn server ?
But i have this error showing:
WebRTC: ICE failed, add a STUN server and see about:webrtc for more details

Can the fact my stream is not showing in my srcObject of my video element come from there ?

58

if i use tcp over udp for my back end server, the event onconnectionstatechange return me ‘connected’.
Here is my configuration of my server:

const serverOptions: WebRtcServerOptions = {
            listenInfos: [
                {
                    ip: '0.0.0.0',
                    announcedIp: '127.0.0.1',
                    port: port,
                    protocol: 'tcp'
                }
            ],
            appData: {
                customData: 'This is my custom data'
            }
        };

But this doesn’t mean it is working ?
UDP protocol doesn’t work for me, if it would be ok without any error it should work in both TCP and UDP am i right ?

59

Does someone have an idea why if i use tcp i have onconnectionstatechange as ‘connected’ but the media stream is not showing in video element, and if i use udp, the onconnectionstatechange failed ?

60

This is the reason why it only works for TCP as you are only asking to make it work on TCP:

Do this instead to make it work on both TCP, UDP:

image
image2084×868 67.9 KB