Mediasoup and Electron

Integration
1

Hi all,

Does anyone have any insight into implementing Mediasoup with Electron?

The issue I’m currently having is I cannot use ‘new mediasoup.Device();’ within the main process, as it is not exposed to the chromium frontend and thus detects no browser. However, if I attempt to use it within the frontend, I cannot pass it back to the backend main process as device has properties that cannot be serialised and therefore cannot be sent through IPC (Interprocess communcation) back to the main process.

I have attempted to search for any examples, here and elsewhere but have not found anything, so any help with be greatly appreciated.

Thanks in advance

2

It should not cause any issue, just convert it to json and send it to main process, just like socket io does.

why do you need it in main process by the way?

3

Hi, thanks for the reply and sorry if the following is a bit wack, I’m still learning so I’ll do my best at explaining.

I need to run const device = new Device(); to create the device object for my mediasoup client. I have to run this in my front-end as it has to be exposed to the chromium front-end to detect the browser (As I assume you cannot construct the device object without exposure to the browser.) This works.

However, I then have to load the rtp capabilities (ie await device.load({ routerRtpCapabilities });).

If I try to load the capabilities inside the ‘frontend process’ I get an error of DOMException: Failed to execute 'structuredClone' on 'Window': function Array() { [native code] } could not be cloned.

node_modules\mediasoup-client\lib\utils.js
function clone(value) {
    if (value === undefined) {
        return undefined;
    }
    else if (Number.isNaN(value)) {
        return NaN;
    }
    else if (typeof structuredClone === 'function') {
        // Available in Node >= 18.
---->   return structuredClone(value); <-- ERRORS HERE
    }
    else {
        return JSON.parse(JSON.stringify(value));
    }
}

If I attempt to return the Device object to the ‘back-end’ process to load the rtpcapabilities there, all functions have to be stripped (for inter-process communication via JSON). And therefore cannot be used to load the rtp capabilities once there.

(See below for comparison)

Correct Device Object

► _canProduceByKind: {audio: false, video: false}
  _extendedRtpCapabilities: undefined
► _handlerFactory: () => new Chrome111()
  _handlerName: "Chrome111"
  _loaded: false
► _observer: EnhancedEventEmitter {_events: {…}, _eventsCount: 0, _maxListeners: Infinity, Symbol(kCapture): false}
  _recvRtpCapabilities: undefined
  _sctpCapabilities: undefined
  handlerName: (...)
  loaded: (...)
  observer: (...)
  rtpCapabilities: (...)
  sctpCapabilities: (...)
▼ [[Prototype]]: Object
  ► canProduce: ƒ canProduce(kind)
  ► constructor: class Device
  ► createRecvTransport: ƒ createRecvTransport({ id, iceParameters, iceCandidates, dtlsParameters, sctpParameters, iceServers, iceTransportPolicy, additionalSettings, proprietaryConstraints, appData, })
  ► createSendTransport: ƒ createSendTransport({ id, iceParameters, iceCandidates, dtlsParameters, sctpParameters, iceServers, iceTransportPolicy, additionalSettings, proprietaryConstraints, appData, })
  ► createTransport: ƒ createTransport({ direction, id, iceParameters, iceCandidates, dtlsParameters, sctpParameters, iceServers, iceTransportPolicy, additionalSettings, proprietaryConstraints, appData, })
    handlerName: (...)
  ► load: ƒ async load({ routerRtpCapabilities, })
    loaded: (...)
    observer: (...)
    rtpCapabilities: (...)
    sctpCapabilities: (...)
  ► get handlerName: ƒ handlerName()
  ► get loaded: ƒ loaded()
  ► get observer: ƒ observer()
  ► get rtpCapabilities: ƒ rtpCapabilities()
  ► get sctpCapabilities: ƒ sctpCapabilities()
  ► [[Prototype]]: Object

Copied Device Object

► _canProduceByKind: {audio: false, video: false}
  _extendedRtpCapabilities: undefined
► _handlerFactory: ƒ ()
  _handlerName: "Chrome111"
  _loaded: false
► _observer: {_events: {…}, _eventsCount: 0, _maxListeners: Infinity, Symbol(kCapture): false}
  _recvRtpCapabilities: undefined
  _sctpCapabilities: undefined
▼ [[Prototype]]: Object
  ► constructor: ƒ Object()
  ► hasOwnProperty: ƒ hasOwnProperty()
  ► isPrototypeOf: ƒ isPrototypeOf()
  ► propertyIsEnumerable: ƒ propertyIsEnumerable()
  ► toLocaleString: ƒ toLocaleString()
  ► toString: ƒ toString()
  ► valueOf: ƒ valueOf()
  ► __defineGetter__: ƒ __defineGetter__()
  ► __defineSetter__: ƒ __defineSetter__()
  ► __lookupGetter__: ƒ __lookupGetter__()
  ► __lookupSetter__: ƒ __lookupSetter__()
  ► __proto__: (...)
  ► get __proto__: ƒ __proto__()
  ► set __proto__: ƒ __proto__()

Ideally it’s all in the main process, to expose as little as possible to the front end (for security in electron) and also the socket io server is all there so it reduces the amount of IPC needed. It seems from forum posts that Electron has been used with Mediasoup but I cant figure out how.

Any wisdom is appreciated, and thank you in advance.

4

There shouldn’t be any problem with that, What you are doing I think is that you get rtp capabilies from server over socket in main process and then you send it over to frontend process which causes the issue. Even in that case it shouldn’t cause any issues, I think the way you are sending it from main process or front end process there is something wrong there, try to print the rtp cababilites in console when you receive them in front end process.

By the way keep your socket as well in frontend process keeping it in main process will cause lots of complexities for you.

5

So here we are again with frontend issues due to usage of JS Proxy objects (Vue, MobX, etc). Please check closed issues in mediasoup-client in GitHub. Recently a similar issue was discussed in there.

6

You’re right, it was discussed, I should’ve checked closed issues.

(See issue here future viewer)
Mediasoup-Client Github ‘Failed to execute ‘structuredClone’ on ‘Window’’

It seems like there is no simple solution for Electron (or other similar approaches) due to the way IPC serialises its messages.

It is possible by removing context isolation and enabling node integration with electron, but that is unsecure. (However as my app will be used on an isolated LAN the security issue is fine.)

Thank you both for the assistance and clarification.

7

You could clone the object itself into a new one (so no longer team only) before passing it to mediasoup-client.