zaidiqbal
February 16, 2023, 5:37pm
4
There is only one way which is to get the raw packets from mediasoup through direct or plain transport (not sure which one) and then consume them in ffmpeg, gstreamer, headless browser and process them and inject them back to mediasoup in the same way and then mediasoup will router it to the participants.
One simple way of doing this in your case can be that you use headless browser on server and then consume the video which you want to process and then process it using html canvas etc and then reproduce that stream from here to mediasoup again and then consume this processed stream in the rest of the participants just like normal stream.
Some things are described here:
Hello everyone
I’ve been using mediasoup for some time now, and I have to say, it’s really a life saver. I just want to know if it is possible, while using mediasoup to perform some AI processing on the video & audio streams using SDKs such as NVIDIA Maxine, on the server. My idea of going about it was to use a PlainRTPTransport to obtain the raw stream data and pass to the SDK, and then get the output from the SDK, and re-inject it into mediasoup using another PlainRTPTransport, but I can’t se…
There is a problem when I use ethand91/mediasoup3-record-demo(GitHub - ethand91/mediasoup3-record-demo: Simple Record Demo using Mediasoup 3 and GStreamer ) to record.But the problem is not his demo,I really helpless that I ask for help here.
I changed his demo,separate the ffmpeg part,becuse I want to record in another computer instead of the computer which the mediasoup server is running.The code ran well and I got video in the computer which the server is running,but I can’t got video in anot…
Hi!
does anyone have experience manipulating the RTP packets with DirectTransport with rtp.js library? I would like to extract the raw audio of the specific producer to manipulate in Nodejs
const room = Room.getRoom(roomId);
const router = room.router;
const directTransport = await router.createDirectTransport();
const audioProducer = getProducer(roomId, id, 'audio');
const consumer = await directTransport.consume({ producerId: audioProducer.id, rtpCapab…
I am testing recording of video and audio by creating a plain transport which is then consumed by FFMPEG process spawned by my app. When I play the recorded video, I see that the audio seems to jump ahead and gets out of sync with Video over time. The longer the video, the larger the displacement/ desync.
To debug this further, I removed the video producer/ consumer and just recorded the audio stream. I tested the audio by speaking every 5 seconds over a 30 second period and when I play the rec…