onProduce will block the application thread and freeze the application if callback not invoked immediately

mediasoup libraries
1

Although this is an iOS-specific question related to mediasoup-ios-client I thought to repost here in case someone has input to help resolve this or if we can make it more “async-friendly”;
And since the iOS framework actually is a wrapper around libMediasoupclient, maybe the problem is down to the C++ component in terms of the callback handler.

Having the below SendTransportListener.

class MediasoupSendTransportHandler : NSObject, SendTransportListener {
  
  fileprivate weak var delegate: SendTransportListener?
  private var parent: MediasoupRoomManager
  
  init(parent: MediasoupRoomManager) {
    self.parent = parent
  }
  
  func onProduce(_ transport: Transport!, kind: String!, rtpParameters: String!, appData: String!, callback: ((String?) -> Void)!) {
    print("SendTransport::onProduce " + kind + " rtpParameters = " + rtpParameters)
    parent.onLocalTransportProduce(transport: transport, kind: kind, rtpParameters: rtpParameters, appData: appData, callback: callback) // Passing the callback so that when async response from server received to invoke with the producer ID
//    callback("**") // Uncomment this line with a random string and it will unblock everything fine
  }
  
  func onConnect(_ transport: Transport!, dtlsParameters: String!) {
    print("SendTransport::onConnect dtlsParameters = " + dtlsParameters)
    parent.onLocalTransportConnect(transport: transport, dtlsParameters: dtlsParameters) // Async HTTP REST request, irrelevant for this problem since it is always working fine
  }
  
  func onConnectionStateChange(_ transport: Transport!, connectionState: String!) {
    print("SendTransport::onConnectionStateChange connectionState = " + connectionState)
  }
  
}

onLocalTransportProduce method to send async HTTP REST request to the server and retrieve the created producer ID.

func onLocalTransportProduce(transport: Transport, kind: String, rtpParameters: String, appData: String, callback: ((String?) -> Void)!) {
    let transportId = transport.getId()
    let jsonBody: JSON = ["kind": kind, "rtpParameters": JSON(parseJSON: rtpParameters)]
    // This is a Combine publisher using AlamoFire, for more details check below
    mediaApi.createProducer(roomId: roomId, broadcasterId: broadcasterId, transportId: transportId!, bodyJson: jsonBody)
      .sink(receiveCompletion: { completion in
        self.onReceiveCompletion(label: "mediaApi.createProducer \(transportId!)", completion: completion)
      }, receiveValue: { result in
        let producerId = result["id"].stringValue
       callback(producerId) // if you call this twice it will lead to an error regarding the state of the promise etc...see below. comment to avoid the error but ID will never passed to libMediasoupClient
      })
      .store(in: &disposables)
  }

libc++abi.dylib: terminating with uncaught exception of type std::__1::future_error: The state of the promise has already been set.

AlamoFire HTTP request to create a producer in server.

func createProducer(roomId: String, broadcasterId: String, transportId: String, bodyJson: JSON) -> AnyPublisher<JSON, MediasoupServerError> {
  let parameters : Parameters = bodyJson.dictionaryObject ?? [:]
  return session
    .request(
      makeCreateProducerUrlPath(roomId: roomId, broadcasterId: broadcasterId, transportId: transportId),
      method: .post,
      parameters: parameters,
      encoding: JSONEncoding.default)
    .publishData()
    .tryMap { result -> JSON in
      try JSON(data: result.data!)
  }
  .mapError { (error) -> MediasoupServerError in
    .network(description: error.localizedDescription)
  }
  .eraseToAnyPublisher()
}
}

Any idea of how to invoke the callback in the subscriber without getting the promise state changed error and of course not block the main thread?

2

I was taking a look at the mediasoup-broadcaster-demo and it looks like in the handler they are using an async HTTP request returning a future so I don’t believe this could be a libMediasoupClient issue and I am open to any help for a workaround :sweat:

3

The callback should only be called once, any more will lead to the above promise error.
The problem seems to be a race issue? You may need to do the request/callback on the same thread via GCD?
Although I’m not an expert in iOS concurrency :sneezing_face:

4

Yes, that is clear. Problem indeed seems to be a race issue; I am not sure how to accomplish that using GCD but keep researching.

On another note it is typical for applications to start background threads and pass callbacks around or provide a completion handler to do something when finished.

It feels we can improve the promise callback to support async so that when in onProduce and another background thread is created it doesn’t block the main thread.

I updated the GitHub issue.

1 Like