Need Help - Unable to Produce

mediasoup libraries
1

Hi All,

I’m having some issue getting the app to produce. It fails whilst attempting to publish and get connected to the producer. I’m listening to the connectionstatechange event on the producer like so:

      switch (state) {
        case 'connecting':
          console.log('publishing...');
          break;

        case 'connected':
          console.log('published');
          break;

        case 'failed':
          console.log('failed');
          liveRef.producerTransport.close();
          break;

        default:
          break;
      }
    };

// later, I'm calling the producerTransport saved into a React Ref.

liveRef.producerTransport = liveRef.device.createSendTransport(params);
producerTransport.on('connectionstatechange', (eventState) => onProducerTransportConnectionStateChange(eventState) );

Perhaps you guys can spot what I’m doing wrong.

Thanks for your help,

2

I think we’ll need more information regarding your code workflow and network structure to identify the problem. The code you shared doesn’t have really much to do with this process - you can actually make the whole thing work without using this event at all.

Ensure that you followed all the steps detailed here: mediasoup :: Communication Between Client and Server

3

Absolutely nothing will happen in the Transport and connection level until you create a Producer on it. This is explained in the docs.

4

Thanks @ibc @dimoochka for the quick response. I appreciate it.

Please see more details regarding how I lay down the flow

 // -- For sending media logic

   // getRouterRtpCapabilities

    const data = await sendRequest(SocketActionEvents.GET_ROUTER_RTP_CAPABILITIES, {});
    await loadDevice(data);

    // createWebrtctTranspont on the server/router

    const params = await sendRequest(SocketActionEvents.CREATE_WEBRTC_TRANSPORT, {
      consuming: false,
      producing: true
    });
    
    // Replicate on the client side by creating the createSendTransport

    liveRef.producerTransport = liveRef.device.createSendTransport(params);
    console.log('createSendTransport:', liveRef.producerTransport);

    // --- subscribe to the `connect` & `produce` event --
   // ProducerTransportEvents -> {PRODUCE, CONNECTION_STATE_CHANGE, CONNECT}

    Object.values(ProducerTransportEvents).map(eventType => {
      liveRef.producerTransport.on(eventType, (eventData, callback, errback) => {
        switch (eventType) {
          case ProducerTransportEvents.CONNECT:
            onProducerTransportConnect(eventData, callback, errback);
            break;
          case ProducerTransportEvents.PRODUCE:
            onProducerTransportProduce(eventData, callback, errback);
            break;
          case ProducerTransportEvents.CONNECTION_STATE_CHANGE:
            onProducerTransportConnectionStateChange(eventData);
            break;
          default:
            console.log('Unable to find action for ' + eventType);
        }
      });
    });

  // For Receiving media

   // --- get capabilities --
      const data = await sendRequest(SocketActionEvents.GET_ROUTER_RTP_CAPABILITIES, {});
      await loadDevice(data);
    }

    // --- prepare transport ---
    console.log('--- createConsumerTransport --');
    if (!liveRef.consumerTransport) {
      // const params = await sendRequest(SocketActionEvents.CREATE_CONSUMER_TRANSPORT, 
      // {});
      const params = await sendRequest(SocketActionEvents.CREATE_WEBRTC_TRANSPORT, {
        consuming: true,
        producing: false
      });

      liveRef.consumerTransport = liveRef.device.createRecvTransport(params);

      // --- join & start publish --
      Object.values(ConsumerTransportEvents).map(eventType => {
        liveRef.consumerTransport.on(eventType, (eventData, callback, errback) => {
          switch (eventType) {
            case ConsumerTransportEvents.CONNECT:
              onConsumerTransportConnect(eventData, callback, errback);
              break;
            case ConsumerTransportEvents.CONNECTION_STATE_CHANGE:
              onConsumerTransportConnectionStateChange(eventData);
              break;
            default:
              console.log('Unable to find action for ' + eventType);
          }
        });
      });
      consumeCurrentProducers(liveRef.clientId);
    }
  }

The above is my current setup for both sending & receiving medias. After this, I’m now consuming the transport and passing the producerId, capabilities and codec.

I’m equally going through the referred documentation to ensure I haven’t missed an important step.

It’s all learning and additional clarity for me. Thanks again guys!

5

Do you have a produce() call on the client side to start off the whole process? You need to initiate this to start off the sendTransport connection.

7

Yes, I’m doing that here.

        const useAudio = true;
        if (useVideo) {
          const videoTrack = liveRef.localStream.getVideoTracks()[0];
          if (videoTrack) {
            const trackParams = { track: videoTrack };
            liveRef.videoProducer = await liveRef.producerTransport.produce(trackParams);
          }
        }
        if (useAudio) {
          const audioTrack = liveRef.localStream.getAudioTracks()[0];
          if (audioTrack) {
            const trackParams = { track: audioTrack };
            liveRef.audioProducer = await liveRef.producerTransport.produce(trackParams);
          }
        }
8

Ok cool. So assuming you’ve implemented the produce and connect events in the sendTransport correctly (and call the callbacks) then perhaps it’s a network / firewall issue. Turn on debug logs and also step through your code to make sure it’s all being executed.

9

Just to circle back… I’ve setup some debugging logs to see what’s going into transport.produce.

Upon calling transport.produce({ kind, rtpParameters, appData }), the resulted object looks like so:

{      
       iceRole: 'controlled',
        iceParameters: [Object],
        iceCandidates: [Array],
        iceState: 'new', // compared to the demo, this should be connected?
        iceSelectedTuple: undefined,
        dtlsParameters: [Object],
        dtlsState: 'new',
        dtlsRemoteCert: undefined,
        sctpParameters: undefined,
        sctpState: undefined
}

In the documentation, iceState` :"new" means No ICE Binding Requests have been received yet. Not sure what the missing part is.
When I ran the mediasoup-demo, iceState is connected at this point.

10

You may specify whether you are doing that in client side or server side.

What does it mean “the resulted object”? The documentation clearly explains what “object” it returns. And BTW transport.produce() resolves (it’s async) with a Producer instance.

To be clear: please, don’t try random things. Follow the documentation: mediasoup :: Communication Between Client and Server

Are you completing the required flow in client side as well?

Yes, the demo works.

11

@ibc thanks for taking time out to respond.

On the server side.

Resulted object meaning, the resolved promise value which is a Producer instance like you said.

I’m using the referenced documentation and the API documentations.

Retracing my steps with breakpoint. With a few more hours of debugging, should be able to find which part is missing/failing.

I’m running this in a docker container and passing my webRtcTransportOptions like so, hopefully that’s not part of the issue.

      webRtcTransportOptions: {

            listenIps: [
             {
               ip: "172.18.0.5", // docker ip address
               // announcedIp: "192.168.1.104", // private ip
            },
         ],
        ....additionalOptions,
     }
12

I haven’t noticed anything overtly wrong with your code so far…

Try turning on verbose mediasoup debugging on both the server and the client side - it gives a ton of information and will give you an idea of where your process is stalling. For further detail, try turning on logging on the server worker and using chrome://webrtc-internals. You could also compare logs between the working demo and your project if you’re still at a loss.

13

@bwoodlt Were you able to resolve this?
I’m having a similar issue.

14

@iamgroot I’m still fletching it out.

However with help from @dimoochka and @ibc I’ve made some progress.

I’m now producing. What I changed was the config,

    listenIps: [
        {
          ip: process.env.MEDIASOUP_LISTEN_IP || "0.0.0.0", 
          announcedIp: process.env.MEDIASOUP_ANNOUNCE_IP, // this is my public ip. 
        },
      ],

Also, set logLevel to debug like so in your mediasoup config object so you can get some logs on the mediasoup side:

 mediasoup: {

   ...otherSettings,
   workerSettings: {
     logLevel: "debug"
   }
 }

Try those and let us know.

Having said that, I’m still having some trouble getting feed whilst consuming. Video and audio request are being transmitted. Logs below:

| [29/05/2021 20:11:14.109] [INFO]   2021-05-29T20:11:14.109Z fc-live-server:INFO:Room Received a producer::score. id=4900f227-94d1-43fe-8e78-b73788f13c63, score [{"encodingIdx":0,"score":10,"ssrc":3576693883}], kind=audio request
| [29/05/2021 20:11:33.379] [INFO]   2021-05-29T20:11:33.379Z fc-live-server:INFO:Room Received a producer::score. id=0b9d4223-c847-4e3f-8682-b0a7e7fba0c1, score=[{"encodingIdx":0,"score":9,"ssrc":3746539812}], kind=video request
| [29/05/2021 20:11:34.564] [INFO]   2021-05-29T20:11:34.564Z fc-live-server:INFO:Room Received a producer::score. id=0b9d4223-c847-4e3f-8682-b0a7e7fba0c1, score=[{"encodingIdx":0,"score":10,"ssrc":3746539812}], kind=video request
| [29/05/2021 20:12:11.588] [INFO]   2021-05-29T20:12:11.588Z fc-live-server:INFO:Room Received a producer::score. id=0b9d4223-c847-4e3f-8682-b0a7e7fba0c1, score=[{"encodingIdx":0,"score":0,"ssrc":3746539812}], kind=video request
| [29/05/2021 20:12:55.795] [INFO]   2021-05-29T20:12:55.795Z fc-live-server:INFO Disconnected SocketIO Ytt1uW9f9KNrbJD6AAAZ from Room=1 userId=425 ...totalNoOfClient=1
15

In my case…I had set paused: true in the server side consumer object. I removed that and everything seemed to work fine.