I’ve been playing with mediasoup / ffmpeg for a while and kicking the tires on the recording features.
I’m trying to figure out the best way to create a single recording file which can follow the active speaker. Effectively, when we have a new active speaker detected, I want to somehow “switch” the producer to which the consumer that belongs to the PlainTransport that connects FFmpeg.
This is proving to be a challenge. I went as far as trying to to create a proxy DirectTransport to manually try and do the binding via the consumer’s on(“rtc”) event and push that to the PlainTransports send(packet) method, but this didn’t work either, as the PlainTransport to which FFmpeg connects does not work with the send() API - and I’m not even sure this is a viable approach.
Trying my best to avoid doing something like recording every track separately then post-processing them all and trying to clip the right segments together from the other recordings, as this is very expensive on the CPU and achieving good audio-sync is also difficult.
Looking for any guidance on the best approach…