Server-side processing of WebRTC streams

Hello everyone

I’ve been using mediasoup for some time now, and I have to say, it’s really a life saver. I just want to know if it is possible, while using mediasoup to perform some AI processing on the video & audio streams using SDKs such as NVIDIA Maxine, on the server. My idea of going about it was to use a PlainRTPTransport to obtain the raw stream data and pass to the SDK, and then get the output from the SDK, and re-inject it into mediasoup using another PlainRTPTransport, but I can’t seem to be able to get the raw stream data. Any alternative solutions would be greatly appreciated.

this post will probably help you. they forward the stream to ffmpeg/gstreamer instead.

1 Like