First off, thank you for your amazing work on mediasoup. We have been using it heavily in our app and have found it to be a solid foundation to build on. We have a mediasoup v2 server side implementation that we have been using with a web based client side. That has been working well. At this point we are experimenting with using your c++ library and hoping to adapt it to connect to our mediasoup v2 server. It hasn’t been too hard to wire up, with some minimal changes to the library. At this point both sides seem to be happy with the arrangement but I haven’t been able to get a producer to send an audio track. Here is my setup code:
auto result = sendRequest("{\"method\":\"queryRoom\",\"target\":\"room\"}");
m_device->Load(result["rtpCapabilities"]);
json message = {
{ "method", "join" },
{ "target", "room" },
{ "peerName", m_peerName.toStdString() },
{ "rtpCapabilities", m_device->GetRtpCapabilities() },
{ "spy", false }
};
auto peers = sendRequest(message.dump());
auto sendTransportId = rtc::CreateRandomId();
auto rtp = m_device->getNativeRtpCapabilities();
auto fingerprint = rtp["fingerprint"];
json message2 = {
{ "method", "createTransport" },
{ "target", "peer" },
{ "id", sendTransportId },
{ "direction", "send" },
{ "options", {} },
{ "dtlsParameters", {
{ "role", "auto" },
{ "fingerprints", {
{
{ "algorithm", fingerprint.at("type").get<std::string>() },
{ "value", fingerprint.at("hash").get<std::string>() }
}
}}
}}
};
auto sendTransport = sendRequest(message2.dump());
json dtlsParams = sendTransport["dtlsParameters"];
json iceCandidates = sendTransport["iceCandidates"];
json iceParameters = sendTransport["iceParameters"];
m_sendTransport = m_device->CreateSendTransport(&m_broadcaster,
std::to_string(sendTransportId), iceParameters, iceCandidates, dtlsParams);
I had to add codec names to the rtpCapabilities so mediasoup v2 was happy. Also, createTransport requires dtlsParameters, so I had to extract that and add it to nativeRtpCapabilities as you can see above. I’m happy to share the minimal patch I’m using to get to this point.
To setup the producer I do this:
cricket::AudioOptions options;
m_audioSource = peerConnectionFactory->CreateAudioSource(options);
m_audioTrack = peerConnectionFactory->CreateAudioTrack("234324321", m_audioSource);
json codecOptions = {
{ "opusStereo", true },
{ "opusDtx", true }
};
m_audioProducer = m_sendTransport->Produce(&m_broadcaster, m_audioTrack, nullptr, &codecOptions);
My listener looks like this:
std::future<void> MediaBroadcaster::OnConnect(mediasoupclient::Transport *transport, const nlohmann::json &transportLocalParameters) {
std::promise<void> promise;
promise.set_value();
return promise.get_future();
}
void MediaBroadcaster::OnConnectionStateChange(mediasoupclient::Transport *transport, const std::string &connectionState) {
std::cout << "Connection state: " << connectionState << std::endl;
}
std::future<std::string> MediaBroadcaster::OnProduce(
mediasoupclient::SendTransport *transport,
const std::string &kind,
nlohmann::json rtpParameters,
const nlohmann::json &appData
) {
std::promise<std::string> promise;
unsigned int id = rtc::CreateRandomId();
nlohmann::json message = {
{ "method", "createProducer" },
{ "target", "peer" },
{ "id", id },
{ "kind", kind },
{ "transportId", std::stoul(transport->GetId()) },
{ "rtpParameters", rtpParameters },
{ "paused", false },
{ "appData", appData },
};
m_mediaService->sendRequest(message.dump());
std::cout << "Created producer with id: " << id << std::endl;
promise.set_value(std::to_string(id));
return promise.get_future();
}
I have to send the dtlsParameters to create the local transport, so I can’t utilize the onConnect mechanism, unless I misunderstand something. Everything looks good and the listener connection state goes from connected to completed but as soon as the producer is created the connection state goes to disconnected. Here is a raw dump of the final moments:
Connection state: checking
[TRACE] PeerConnection::OnIceGatheringChange()
[DEBUG] PeerConnection::OnIceGatheringChange() | new IceGatheringState:[complete]
Connection state: connected
Connection state: completed
Received: "{\"type\":\"data\",\"id\":\"9\",\"payload\":{\"data\":{\"sendMediaRequest\":\"{}\"}}}"
Created producer with id: 4024606819
[TRACE] Producer::Producer()
Producer created
Connection state: disconnected
I appreciate any suggestions anyone might have. I’m sure I’m missing something simple. Let me know if you need anything additional. Thank you!
