How to fill muted segments when recording rtp to ffmpeg?

I am recording audio streams to mp3 by creating plain transport to local ports and then running ffmpeg on those local port into mp3.

Basically I follow this excellent demo: GitHub - ethand91/mediasoup3-record-demo: Simple Record Demo using Mediasoup 3 and GStreamer and it works great!

The problem is that when I mute such audio stream, I pause the producer and the ffmpeg stops getting data which results in no output to mp3. I.e. all those mute segments are simply missing from the output mp3.
Instead I want to keep writing to mp3 while audio is muted, but writing zeros essentially, emulating mute periods.

Can’t figure out how to approach it :frowning:
Any idea how to do that? Any help is appreciated, thanks!

I guess I can record all those mute periods and then manipulate mp3 inserting empty segments there or some other similar way gluing final mp3 from various segments.

I understand how to do that ^^^, I just want to avoid it if it’s possible :slight_smile:

I guess I hope there is some magic exists when I can set some truly awesome property on a consumer/producer/etc and it will keep feeding ffmpeg with zeroes while the producer is paused.

No such thing?

If the producer is created with the default option of disableTrackOnPause (and without zeroRtpOnPause), then when you pause it on the client side, underlying transceiver continue to send normal frames with ‘silence’. If in addition the server-side producer is paused, this implicitly pauses its consumers, and they stop sending packets to the clients. You may consider explicitly pausing all server-side consumers of the producer except the one that is serving ffmpeg, without pausing the server-side producer.

You can use the aresample=async=1 filter in ffmpeg to replace missing timestamps with silence. This happens when the track is muted, but also happens when there are dropped packets.

This requires that you decode and re-encode the audio, but it sounds like you’re already doing this because you’re writing to mp3 and not opus.

3 Likes

It is now a bit more clear in my head, thank you!

Thank you Jonathan, the filter worked beautifully!