Datachannels without GetUserMedia?

I have a question I am sure that I know the answer to… But my results are pointing to me being wrong. I wanted to ask: Must getUserMedia be called before a dataChannel can be established? I have included an example below, but my actual code has the behavior of needing the user to accept a getUserMedia dialog before the dataChannel will establish between client and server. Otherwise, the dataChannel fails with the webrtcSendTransport failing to reach the connected state, with an ice/turn failure. When a user accepts the getUserMedia reques (audio or video doesn’t matter) the data channel connects happily, even if I don’t pass any of the media streams to mediasoup, and only call the produceData on the transport.

//Called on page load, before user clicks "start" button
async connectIngressTransport(transport) {

    this.producerTransport = await this.webRTCDevice.createSendTransport({
      id: transport.id,
      iceParameters: transport.iceParameters,
      iceCandidates: transport.iceCandidates,
      //iceServers: iceServersJSON,
      dtlsParameters: transport.dtlsParameters,
      sctpParameters: transport.sctpParameters
    });

    this.producerTransport.on(
      "connect",
      async ({ dtlsParameters }, callback, errback) => {
        this.send("connectIngressTransport", {
          peerId: this.avatar_id,
          direction: this.producerTransport.direction,
          dtlsParameters: dtlsParameters
        });
        callback();
      }
    );

    this.producerTransport.on("connectionstatechange", connectionState => {
      console.warn("Producer transport state:", connectionState);
    });

    this.producerTransport.on(
      "producedata",
      async ( { sctpStreamParameters, label, protocol, appData }, callback, errback) => {
        // Send dataProducer data to server
        this.send("produceData", {
          producingPeer: this.avatar_id,
          producerOptions: {
            sctpStreamParameters: sctpStreamParameters,
            label: label,
            protocol: protocol,
            appData: appData
          }
        });
          // Wait for incoming producedData signal to set this value
          await this.waitFor(_ => this.movementProducerId !== undefined);
          callback({ id: this.movementProducerId });
    });
  }

//Called by user action button click
async start(func) {
  const frameMediaOptions = { audio: true,
                                                 video: false };
  await navigator.mediaDevices.getUserMedia(frameMediaOptions)
                          .catch( error => { console.log("Error getting shit", error) })
                          .finally( async () => { await this.producerTransport.produceData();
                                                            console.log("PRODUCED DATA") })
  }

The example above works as expected, but if I were to comment out the getUserMedia call and move the producerTransport.produceData to be alone, it would fail. – This leads me into questioning whether the getUserMedia is needed, even if dataChannels are the only thing being used. I have checked the values, for sctp and dtls and they seem to be ok. The transport has sctp enabled, and otherwise works with the same active code, except when getUserMedia is not called. :-/

Any insights would be appreciated. The links below seemed to touch on the topic, but didn’t lead to any ‘aha!’ moments.

It’s probably because your producerTransport isn’t ready yet.

Create a separate function that just handles the data producer creation this.producerTransport.produceData().

Then in here:

this.producerTransport.on("connectionstatechange", connectionState => {
      if (connectionState === 'connected') {
         // call function with dataProducer setup
      }

});

It was working more than likely because the producerTransport was set up by the time start was called by the user clicking the button. When you moved producerTransport.produceData to be alone it wasn’t waiting for producerTransport to be connected.

You didn’t say which were happens if you just create a DataProducer. Some browsers don’t allow creating a Data channel if getUserMedia is not called first due to privacy etc. But you should focus first on diagnosing the problem: does ICE connect? There are events and logs in client and server to check that.

Try it with the setTimeout in place of awaiting getUserMedia. It will show at once whether the latter is somehow involved or this is a timing problem.

I tried this based on your advice, and I think it will not work as you expect. I believe putting the call to produceData inside the connection state change would never be called if there was no other media. My thinking is that the producer transport fires both onConnect and onProduceData when the call to produceData is made. So if produce data is called from something that depends on it connecting, it will never fire. Its been my experience that when the first producer is put on the transport, it also connects the transport. Am I off base here with that thinking?

If I just create a data producer, it will not connect to the transport with an ice/turn failure error. Specifically, the transport state changes to “failed” and I get " WebRTC: ICE failed, add a STUN server and see about:webrtc for more details"

WebRTC ice candidates look ok, at least with the correct IP addresses etc.

Indeed, the mediasoup logs tell me some information, but its the same info that i have logged and seen in about:webrtc.

I will continue to check logs on the client and server, and see if I can spot any weirdness in my code. My browser is Firefox 78.6.1esr. I have googled if firefox needs getusermedia for the datachannel, but have not seen any answers. Do you happen to know which browsers require getusermedia before allowing datachannels/webrtctransports from connecting?

I will try this in chrome and see what happens as well. I would hate to think that getusermedia restricts the datachannel. Thank you for your answers here.

I did try some promise powered timeouts to await for certain times. I believe I have ruled out a timing issue. I set delays of 10 seconds in both the onConnect and onProduceData handlers of the transport. It doesn’t seem to be that. – If I run the same code, and flip the audio/video params to getusermedia as false, the datachannel wont open. As soon as I set one of them to true, and click the accept, all the mediasoup stuff works as expected, even if I dont actually give the audio/video streams to a transport.

In a bit of progress here, I used firefox version 84.0.2 and it seems to connect fine.

Err, will explore this more. Maybe the ESR version has extra measures in place, or there has been a policy change with getusermedia/datachannels between those versions.

Ill post back with more details. My dreams to support the long-terms-support versions of browsers is slowly fading into “ok, can we just get the latest browsers working?” :smiley: Luckily, it seems I can.

I’m not sure you did read this.

BTW does the same problem happen in Chrome?

I did read it, thank you again. This insight lead me to question and post about my version being Firefox ESR, as well as test different versions of firefox.

I have been able to tell so far that the latest versions of firefox 84.0.2 and chrome 87 work fine with my code. It does not work on Firefox 78.6.1esr. This leads me to think that the getusermedia is required in order to establish a datachannel in the ESR version of firefox, but not in the latest version. So as you said, it seems that there is some form of privacy consideration for a datachannel in different versions.

Blame Firefox :slight_smile:

1 Like

Yes, this issue is for sure rooted in how firefox handles webrtc. I am glad they fixed this in recent versions.

Yeah.

(and thanks :wink: )

Looks rather suspicious to me. Tried it in Firefox 78.6.1esr/Windows and didn’t encounter any problem (without getUserMedia of course).

I wonder if there is a deeper split in the condition. I am running debian10 with openbox. I would hope that the layering would prevent something falling back into OS-land, but this is direct device access we are talking about so I bet there are loads of edges to prick our fingers on.

I have moved on from that codepoint in development, but I will try it on my windows machine one day and see if I’m being paranoid or not. :wink:

Thanks for the observation!

I just can’t imagine what the connection between the data channels and getUserMedia could be. Suppose there is any, then what about the media streams?