Connection is established but no packets sent, DTLS stuck on not connected

I’m trying to stream from a mediasoup client in the browser, to a mediasoup server SFU, to a libmediasoupclient C++ client. Streaming works just fine from the client in the browser to the SFU, and I already have a setup to successfully consume that stream from the SFU into another browser client. My C++ client succeeds in connecting a transport and consumer as expected, but no data is transmitted over the transport for the stream. The transport itself passes through the various connection states until it gets to “completed”, but no RTP packets are sent for the stream itself. The server-side consumer’s trace event shows no packets whatsoever sent (including no pli or keyframe packets for video tracks).

UPDATE: Enabling debug logging on the SFU shows the following message every 2 seconds:

RTC::WebRtcTransport::OnRtcpDataReceived() | ignoring RTCP packet while DTLS not connected

But the logs say indicate that the DTLS handshake completes

The SFU side consumer connected to the C++ client receives packets from the source producer, but doesn’t send any to the C++ client itself. Here’s the server-side consumer stats:

[
{
    "bitrate": 0,
    "byteCount": 0,
    "firCount": 0,
    "fractionLost": 0,
    "kind": "audio",
    "mimeType": "audio/opus",
    "nackCount": 0,
    "nackPacketCount": 0,
    "packetCount": 0,
    "packetsDiscarded": 0,
    "packetsLost": 0,
    "packetsRepaired": 0,
    "packetsRetransmitted": 0,
    "pliCount": 0,
    "score": 10,
    "ssrc": 247034020,
    "timestamp": 23577950,
    "type": "outbound-rtp"
  },
  {
    "bitrate": 5441,
    "byteCount": 14715,
    "firCount": 0,
    "fractionLost": 0,
    "jitter": 19,
    "kind": "audio",
    "mimeType": "audio/opus",
    "nackCount": 0,
    "nackPacketCount": 0,
    "packetCount": 182,
    "packetsDiscarded": 0,
    "packetsLost": 0,
    "packetsRepaired": 0,
    "packetsRetransmitted": 0,
    "pliCount": 0,
    "roundTripTime": 0.1068115234375,
    "score": 10,
    "ssrc": 2838189885,
    "timestamp": 23577950,
    "type": "inbound-rtp"
  }
]

Here’s what the output of the Transport’s GetStats on the client looks like:

[
  {
    "base64Certificate": "MIIBFjCBvaADAgECAgkA2mCL0xbQn2EwCgYIKoZIzj0EAwIwETEPMA0GA1UEAwwGV2ViUlRDMB4XDTIxMDYxMDE5MDgwNloXDTIxMDcxMTE5MDgwNlowETEPMA0GA1UEAwwGV2ViUlRDMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEVqCuO/hgxx2+IFbqQ9/rMvepRdSpA1tdjhwrigVNyF3dUwUZOVJRsHFuodxGe8QyOsir+Y0XKvG/9X0EzC6mDjAKBggqhkjOPQQDAgNIADBFAiB30JDdDJ7dU/3ewBxhU9LIgIqcItpePvRLj4vJDL9IswIhAJHGkMIofKFXTxkfGv4hpPobmOYqwIUGZrubu0ZiC2Yi",
    "fingerprint": "B7:FF:85:0C:B2:D6:54:A1:12:9A:A0:4E:A2:D3:59:8A:5F:E1:48:26:65:B4:3A:5E:96:6C:24:42:92:50:D8:D4",
    "fingerprintAlgorithm": "sha-256",
    "id": "RTCCertificate_B7:FF:85:0C:B2:D6:54:A1:12:9A:A0:4E:A2:D3:59:8A:5F:E1:48:26:65:B4:3A:5E:96:6C:24:42:92:50:D8:D4",
    "timestamp": 1623438540621212,
    "type": "certificate"
  },
  {
    "base64Certificate": "MIIBUzCB/KADAgECAgN8ChUwCQYHKoZIzj0EATA0MRgwFgYDVQQKDA9tZWRpYXNvdXAyNTM4MTAxGDAWBgNVBAMMD21lZGlhc291cDI1MzgxMDAeFw0xMTA2MTQxOTA3MzhaFw0zMTA2MDkxOTA3MzhaMDQxGDAWBgNVBAoMD21lZGlhc291cDI1MzgxMDEYMBYGA1UEAwwPbWVkaWFzb3VwMjUzODEwMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAExT0VrofVnMcRKw1k+d+VSjhYUW9rnkK5X8PvFSw/OhXuzeTb7TNfKGk6BLhZYyZ5G65UJVvfzLmQvoKOHGDMtTAJBgcqhkjOPQQBA0cAMEQCICKWZjP4Hey2xMbxFkL46fOpOvjriKgl3pTif85AID78AiAf3gtNNK5lbrMFWs/N/8pnPI0zLChokrLJvVaIyt9oFw==",
    "fingerprint": "C6:98:AF:32:AB:61:A9:A9:FD:79:B3:C1:BC:93:8D:B4:B9:31:B0:97",
    "fingerprintAlgorithm": "sha-1",
    "id": "RTCCertificate_C6:98:AF:32:AB:61:A9:A9:FD:79:B3:C1:BC:93:8D:B4:B9:31:B0:97",
    "timestamp": 1623438540621212,
    "type": "certificate"
  },
  {
    "channels": 2,
    "clockRate": 48000,
    "id": "RTCCodec_0_Inbound_100",
    "mimeType": "audio/opus",
    "payloadType": 100,
    "sdpFmtpLine": "minptime=10;useinbandfec=1",
    "timestamp": 1623438540621212,
    "type": "codec"
  },
  {
    "channels": 2,
    "clockRate": 48000,
    "id": "RTCCodec_0_Outbound_100",
    "mimeType": "audio/opus",
    "payloadType": 100,
    "sdpFmtpLine": "minptime=10;useinbandfec=1",
    "timestamp": 1623438540621212,
    "type": "codec"
  },
  {
    "availableOutgoingBitrate": 300000,
    "bytesReceived": 772,
    "bytesSent": 936,
    "consentRequestsSent": 24,
    "currentRoundTripTime": 0,
    "id": "RTCIceCandidatePair_eVaEgvRe_ZVXsSeIK",
    "localCandidateId": "RTCIceCandidate_eVaEgvRe",
    "nominated": true,
    "priority": 4.623781745793842e+18,
    "remoteCandidateId": "RTCIceCandidate_ZVXsSeIK",
    "requestsReceived": 0,
    "requestsSent": 1,
    "responsesReceived": 25,
    "responsesSent": 0,
    "state": "succeeded",
    "timestamp": 1623438540621212,
    "totalRoundTripTime": 0.003,
    "transportId": "RTCTransport_0_1",
    "type": "candidate-pair",
    "writable": true
  },
  {
    "candidateType": "host",
    "deleted": false,
    "id": "RTCIceCandidate_ZVXsSeIK",
    "ip": "127.0.0.1",
    "isRemote": true,
    "port": 47815,
    "priority": 1076558079,
    "protocol": "udp",
    "timestamp": 1623438540621212,
    "transportId": "RTCTransport_0_1",
    "type": "remote-candidate"
  },
  {
    "candidateType": "host",
    "deleted": false,
    "id": "RTCIceCandidate_eVaEgvRe",
    "ip": "10.0.0.223",
    "isRemote": false,
    "networkType": "unknown",
    "port": 38507,
    "priority": 2122129151,
    "protocol": "udp",
    "timestamp": 1623438540621212,
    "transportId": "RTCTransport_0_1",
    "type": "local-candidate"
  },
  {
    "bytesReceived": 0,
    "fecPacketsDiscarded": 0,
    "fecPacketsReceived": 0,
    "headerBytesReceived": 0,
    "id": "RTCInboundRTPAudioStream_157459697",
    "isRemote": false,
    "jitter": 0,
    "kind": "audio",
    "mediaType": "audio",
    "packetsLost": 0,
    "packetsReceived": 0,
    "ssrc": 157459697,
    "timestamp": 1623438540621212,
    "trackId": "RTCMediaStreamTrack_receiver_3",
    "transportId": "RTCTransport_0_1",
    "type": "inbound-rtp"
  },
  {
    "concealedSamples": 0,
    "concealmentEvents": 0,
    "delayedPacketOutageSamples": 0,
    "detached": false,
    "ended": false,
    "id": "RTCMediaStreamTrack_receiver_3",
    "insertedSamplesForDeceleration": 0,
    "interruptionCount": 0,
    "jitterBufferDelay": 0,
    "jitterBufferEmittedCount": 0,
    "jitterBufferFlushes": 0,
    "jitterBufferTargetDelay": 0,
    "kind": "audio",
    "relativePacketArrivalDelay": 0,
    "remoteSource": true,
    "removedSamplesForAcceleration": 0,
    "silentConcealedSamples": 0,
    "timestamp": 1623438540621212,
    "totalAudioEnergy": 0,
    "totalInterruptionDuration": 0,
    "totalSamplesDuration": 0,
    "totalSamplesReceived": 0,
    "trackIdentifier": "c8730246-c04d-424b-a46a-14ae6cd80b6a",
    "type": "track"
  },
  {
    "id": "RTCMediaStream_ubWVgHt8J4yNNy5x",
    "streamIdentifier": "ubWVgHt8J4yNNy5x",
    "timestamp": 1623438540621212,
    "trackIds": [
      "RTCMediaStreamTrack_receiver_3"
    ],
    "type": "stream"
  },
  {
    "dataChannelsClosed": 0,
    "dataChannelsOpened": 0,
    "id": "RTCPeerConnection",
    "timestamp": 1623438540621212,
    "type": "peer-connection"
  },
  {
    "bytesReceived": 772,
    "bytesSent": 936,
    "dtlsCipher": "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "dtlsState": "connected",
    "id": "RTCTransport_0_1",
    "localCertificateId": "RTCCertificate_B7:FF:85:0C:B2:D6:54:A1:12:9A:A0:4E:A2:D3:59:8A:5F:E1:48:26:65:B4:3A:5E:96:6C:24:42:92:50:D8:D4",
    "remoteCertificateId": "RTCCertificate_C6:98:AF:32:AB:61:A9:A9:FD:79:B3:C1:BC:93:8D:B4:B9:31:B0:97",
    "selectedCandidatePairChanges": 1,
    "selectedCandidatePairId": "RTCIceCandidatePair_eVaEgvRe_ZVXsSeIK",
    "srtpCipher": "AES_CM_128_HMAC_SHA1_80",
    "timestamp": 1623438540621212,
    "tlsVersion": "FEFD",
    "type": "transport"
  }
]

And here’s the client’s consumer stats:

[
  {
    "base64Certificate": "MIIBUzCB/KADAgECAgMXftUwCQYHKoZIzj0EATA0MRgwFgYDVQQKDA9tZWRpYXNvdXAzMzYwMTgxGDAWBgNVBAMMD21lZGlhc291cDMzNjAxODAeFw0xMTA2MTQxOTA5MzZaFw0zMTA2MDkxOTA5MzZaMDQxGDAWBgNVBAoMD21lZGlhc291cDMzNjAxODEYMBYGA1UEAwwPbWVkaWFzb3VwMzM2MDE4MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEakMdoV3PCpk+rYztMuBge+g1XWqVsErWsbwXvvmLilhcYpXyxZGC1rmpY0dw3rMcZjdG4hcAEuXSRf4YZkUuxTAJBgcqhkjOPQQBA0cAMEQCIGowfrv4Hw9lM/YZ4zwS0HvYKewsOeaLa/6/qPPv9ihgAiAsRc9NEyRA4szDDrBwQWCRcfuppLxGV4uD+EY2e4qCTw==",
    "fingerprint": "8D:29:2D:AB:8B:F1:DE:64:18:AA:12:05:35:F0:06:3D:4B:5D:C0:A9",
    "fingerprintAlgorithm": "sha-1",
    "id": "RTCCertificate_8D:29:2D:AB:8B:F1:DE:64:18:AA:12:05:35:F0:06:3D:4B:5D:C0:A9",
    "timestamp": 1623440156820702,
    "type": "certificate"
  },
  {
    "base64Certificate": "MIIBFjCBvaADAgECAgkAslG62GQa4eAwCgYIKoZIzj0EAwIwETEPMA0GA1UEAwwGV2ViUlRDMB4XDTIxMDYxMDE5MzU1MFoXDTIxMDcxMTE5MzU1MFowETEPMA0GA1UEAwwGV2ViUlRDMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAENUNP1wZCZkNHAiQdW/3C1vy+jhezvRR4hAnBa2BK9f4H+rjthtxRc+HwQkHyupN4soVnlfKaYgHcW3lsg9DjGzAKBggqhkjOPQQDAgNIADBFAiEAxCLH6pz+jqGiVT9OA+0d9xEHi/OqDRc2dCiyf4uNK+4CIDuqnojIlwwJeN/taWpHMM1UcWhqwFGdUpzgmILGuBZR",
    "fingerprint": "E0:17:ED:12:5A:B7:32:F9:3D:DA:28:3C:F6:56:2C:49:28:EC:82:37:42:04:81:58:C3:05:4F:9D:B4:27:63:51",
    "fingerprintAlgorithm": "sha-256",
    "id": "RTCCertificate_E0:17:ED:12:5A:B7:32:F9:3D:DA:28:3C:F6:56:2C:49:28:EC:82:37:42:04:81:58:C3:05:4F:9D:B4:27:63:51",
    "timestamp": 1623440156820702,
    "type": "certificate"
  },
  {
    "availableOutgoingBitrate": 300000,
    "bytesReceived": 772,
    "bytesSent": 716,
    "consentRequestsSent": 5,
    "currentRoundTripTime": 0.001,
    "id": "RTCIceCandidatePair_mY2kyFbo_hJlP4l7R",
    "localCandidateId": "RTCIceCandidate_mY2kyFbo",
    "nominated": true,
    "priority": 4.623781745793842e+18,
    "remoteCandidateId": "RTCIceCandidate_hJlP4l7R",
    "requestsReceived": 0,
    "requestsSent": 1,
    "responsesReceived": 6,
    "responsesSent": 0,
    "state": "succeeded",
    "timestamp": 1623440156820702,
    "totalRoundTripTime": 0.003,
    "transportId": "RTCTransport_0_1",
    "type": "candidate-pair",
    "writable": true
  },
  {
    "candidateType": "host",
    "deleted": false,
    "id": "RTCIceCandidate_hJlP4l7R",
    "ip": "127.0.0.1",
    "isRemote": true,
    "port": 26071,
    "priority": 1076558079,
    "protocol": "udp",
    "timestamp": 1623440156820702,
    "transportId": "RTCTransport_0_1",
    "type": "remote-candidate"
  },
  {
    "candidateType": "host",
    "deleted": false,
    "id": "RTCIceCandidate_mY2kyFbo",
    "ip": "10.0.0.223",
    "isRemote": false,
    "networkType": "unknown",
    "port": 59375,
    "priority": 2122129151,
    "protocol": "udp",
    "timestamp": 1623440156820702,
    "transportId": "RTCTransport_0_1",
    "type": "local-candidate"
  },
  {
    "bytesReceived": 0,
    "fecPacketsDiscarded": 0,
    "fecPacketsReceived": 0,
    "headerBytesReceived": 0,
    "id": "RTCInboundRTPAudioStream_937718332",
    "isRemote": false,
    "jitter": 0,
    "kind": "audio",
    "mediaType": "audio",
    "packetsLost": 0,
    "packetsReceived": 0,
    "ssrc": 937718332,
    "timestamp": 1623440156820702,
    "trackId": "RTCMediaStreamTrack_receiver_3",
    "transportId": "RTCTransport_0_1",
    "type": "inbound-rtp"
  },
  {
    "concealedSamples": 0,
    "concealmentEvents": 0,
    "delayedPacketOutageSamples": 0,
    "detached": false,
    "ended": false,
    "id": "RTCMediaStreamTrack_receiver_3",
    "insertedSamplesForDeceleration": 0,
    "interruptionCount": 0,
    "jitterBufferDelay": 0,
    "jitterBufferEmittedCount": 0,
    "jitterBufferFlushes": 0,
    "jitterBufferTargetDelay": 0,
    "kind": "audio",
    "relativePacketArrivalDelay": 0,
    "remoteSource": true,
    "removedSamplesForAcceleration": 0,
    "silentConcealedSamples": 0,
    "timestamp": 1623440156820702,
    "totalAudioEnergy": 0,
    "totalInterruptionDuration": 0,
    "totalSamplesDuration": 0,
    "totalSamplesReceived": 0,
    "trackIdentifier": "3e44bce8-dc06-4dd0-b68a-e6df9fb7c2a8",
    "type": "track"
  },
  {
    "bytesReceived": 772,
    "bytesSent": 716,
    "dtlsCipher": "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "dtlsState": "connected",
    "id": "RTCTransport_0_1",
    "localCertificateId": "RTCCertificate_E0:17:ED:12:5A:B7:32:F9:3D:DA:28:3C:F6:56:2C:49:28:EC:82:37:42:04:81:58:C3:05:4F:9D:B4:27:63:51",
    "remoteCertificateId": "RTCCertificate_8D:29:2D:AB:8B:F1:DE:64:18:AA:12:05:35:F0:06:3D:4B:5D:C0:A9",
    "selectedCandidatePairChanges": 1,
    "selectedCandidatePairId": "RTCIceCandidatePair_mY2kyFbo_hJlP4l7R",
    "srtpCipher": "AES_CM_128_HMAC_SHA1_80",
    "timestamp": 1623440156820702,
    "tlsVersion": "FEFD",
    "type": "transport"
  }
]

Things I’ve tried:

  • Checking if the producer and client & SFU consumers are paused or ended (none of them are)
  • Adding a sink on the C++ client to see if any data is being received
  • Requesting GenerateKeyFrame() on the mediastreamtrack attached to the C++ client’s consumer

I suspect I might be making a simple mistake in my usage of libmediasoupclient. Any ideas on what debugging I could do?

First step is to compare the server signaling between your c++ client and the web client - differences may account for different behavior. You’re clearly doing things right since you have the web client working already.

Additionally you could enable debug logs for the actual worker threads on the SFU to get granular data on what the workers are doing and at what step they are hanging (ie. is any RTP actually being sent? if so, why is it not being received on the client side… etc). Worker threads generate a TON of messages in addition to the SFU messages you’re already seeing. Compare server logs between your working and dysfunctional scenarios to help you pinpoint the issue.

Enable the dtls log tag in the mediasoup worker and set the log level to debug to see any further logs related to DTLS.

NOTE: Are you providing mediasop with a DTLS certificate or are you letting it create a self signed one? If you are doing the former, try just not providing any certificate (mediasoup will create one). We have seen issues in the past related to the big size of provided certificates.

Thank you for the reply! Turns out I was not passing the dtls parameters correctly from client to server - it’s resolved now. Sorry for taking up your time.