Opus audio crackling only appears after mediasoup relay (FFmpeg → PlainTransport → WebRTC Consumer) — ruled out FFmpeg, network, and browser rendering

mediasoup libraries
1

I’m feeding an FFmpeg-sourced RTSP camera’s audio into mediasoup via a PlainTransport producer, then consuming it via a standard WebRtcTransport consumer in the browser. The audio has a persistent crackling/static artifact — but only after passing through mediasoup. I’ve done extensive isolation testing and would appreciate any insight into what might be happening in the relay itself.

Setup:

  • Source: IP camera, RTSP, native audio is G.722 16kHz mono
  • FFmpeg transcodes to Opus for the mediasoup leg
  • mediasoup version: [3.19.17]
  • Node.js version: [22.17.0]

FFmpeg audio args:
-af aresample=48000 -c:a libopus -application voip -ar 48000 -ac 2 -b:a 40k

Producer Creation
const audioProducer = await entry.audioTransport.produce({
kind: "audio",
rtpParameters: {
codecs: [
{
mimeType: "audio/opus",
payloadType: 97,
clockRate: 48000,
channels: 2,
parameters: {
"sprop-stereo": 1,
"usedtx": 0,
},
},
],
encodings: [{ ssrc }],
},
});

What I’ve isolated so far (each step tested independently):

  1. :white_check_mark: FFmpeg resample alone → local WAV file → clean, no crackling
  2. :white_check_mark: FFmpeg full Opus encode (audio-only) → local .opus file → clean
  3. :white_check_mark: FFmpeg combined video+audio encode (matching production args) → local .mkv file → clean
  4. :white_check_mark: FFmpeg’s actual RTP output → received directly via ffplay (bypassing mediasoup entirely, using a manual SDP file) → clean
  5. :cross_mark: Same FFmpeg RTP stream → mediasoup PlainTransport producer → WebRtcTransport consumer → browser → crackling
  6. Confirmed the corruption is present in the raw MediaStreamTrack itself (captured directly via MediaRecorder from consumer.track in the browser, bypassing any <video> element / UI code) → still crackles

Producer and consumer stats during a crackling session (packetsLost, nackCount, jitter all clean):

// producer (inbound-rtp, what mediasoup received from ffmpeg)
{"packetsLost":0,"jitter":669,"nackCount":0,"score":10,"packetCount":190}

// consumer (outbound-rtp, what mediasoup sent to browser)
{"packetsLost":0,"jitter":658,"nackCount":0,"score":10,"packetCount":241}

Question: Since mediasoup terminates and re-originates the RTP stream (new SSRC/sequence/timestamp base) rather than doing a byte-level passthrough, is there a known issue or requirement around:

  • RTP timestamp granularity/consistency expected from non-browser (FFmpeg) Opus producers specifically?
  • DTX/FEC parameter mismatches between producer and consumer causing decode-side artifacts?
  • Any known interaction between PlainTransport comedia mode and Opus timestamp remapping for the WebRTC consumer leg?
2

This all looks good to me. If the audio is crackling, it’s probably something happening on the browser playout side rather than the mediasoup side. Here’s a couple things to try:

  1. Stream to ffprobe instead of ffplay and inspect the timestamps and durations. Normally, they should be 960 samples (20ms) apart, although I don’t think this is a hard requirement. If there’s anything odd about the timestamps or durations, that would be a good clue.
ffprobe -protocol_whitelist file,udp,rtp \
  -show_packets \
  -select_streams a \
  input.sdp
  1. Try the -re flag. You shouldn’t need this flag, because you’re streaming from a realtime device already. But, if something is buffering the audio and then sending it all too quickly, then the browser playout will sound degraded. It would be good to rule this out.
3

Probably ffmpeg is creating a local SDP that represents what it is being told to generate. Maybe there is a command line option to generate such a local SDK somewhere and taking a look to it may expose the problem. Usually these issues come from what ffmpeg is generating at RTP level despite everything looks good in mediasoup’s transport.produce() (which is just a description of what the endpoint is supposed to send, but it’s totally useless if the endpoint ends sending something different).

4

Thanks for the suggestions. I ran the ffprobe timestamp check you recommended, capturing the RTP stream directly from ffmpeg (before it reaches mediasoup):

pts=315840, 316800, 317760, 318720, 319680, 320640, 321600, 322560...

Every consecutive delta is exactly 960 (20ms at 48kHz) — completely regular, no drift, pts always equals dts, duration=960 throughout. This appears to rule out timestamp irregularity from ffmpeg as the cause.

I also tried the -re flag suggestion and confirmed no meaningful difference in the resulting audio (still real-time source, as expected).

Additionally, I ran a controlled side-by-side test: the exact same ffmpeg tee output that produces the crackling mediasoup/WebRTC audio is also sent, byte-for-byte identical RTP, directly to two other ffmpeg receivers (one writing an MP4 file, one writing HLS segments) — both of those play back with completely clean audio in Chrome. Only the mediasoup PlainTransport → WebRtcTransport consumer path produces the crackling, even though the source RTP is provably identical across all three destinations.

Given clean timestamps, clean transport stats (0 loss, score:10 on both producer and consumer), and clean playback via non-mediasoup receivers of the identical stream — is there anything else worth checking in mediasoup’s internal RTP relay for Opus specifically? Would enabling debug-level worker logs (logLevel: "debug" with specific logTags) surface anything useful here that I should share?

5

As I said above, seeing the SDP that ffmpeg internally uses/generates would help.

6

Thanks for the suggestion — this was a great diagnostic approach and did surface something real.

I generated ffmpeg’s actual internal SDP via -sdp_file:
m=audio 6000 RTP/AVP 97 b=AS:40 a=rtpmap:97 opus/48000/2 a=fmtp:97 sprop-stereo=1

Comparing this against my mediasoup producer’s rtpParameters, I found a genuine mismatch: I had usedtx: 0 set in the producer’s codec parameters, but ffmpeg’s actual SDP never declares usedtx at all — it only sets sprop-stereo=1. I removed the extraneous usedtx parameter so mediasoup’s configuration exactly matches ffmpeg’s real output.

Result: the crackling is unchanged. So this specific mismatch, while real, doesn’t appear to be the cause.

Given the SDP file only describes ffmpeg’s static, session-level intent rather than the actual real-time byte content of each RTP packet, I’m now considering a raw packet capture (Wireshark/tcpdump) to inspect the actual Opus TOC (Table of Contents) byte within each packet — to check for frame-size/mode irregularities at the byte level that wouldn’t show up in either the SDP or the transport-level stats we’ve already confirmed are clean (0 loss, score:10 throughout, from ffmpeg through mediasoup to the browser).

Does that seem like the right next diagnostic step, or is there something more specific in mediasoup’s own handling of Opus RTP packets you’d want me to check first (e.g., debug-level worker logs, or a specific trace event)?

7

I ran chrome://webrtc-internals during a crackling session and found something very concrete:

  • concealedSamples / totalSamplesReceived ≈ 89.5% — nearly all audio is decoder-concealed/synthesized, not real received data
  • totalInterruptionDuration ≈ 224s out of 263s total (~85% of playback time interrupted)
  • jitterBufferDelay averages ~2,466 ms per emitted sample — an enormously inflated jitter buffer
  • Meanwhile, the network-level jitter stat is only 42ms, and packetsLost: 0 throughout (confirmed via producer/consumer RTP stats on the mediasoup side too)

So the network path is clean and low-jitter, but the browser’s audio jitter buffer is behaving as though it’s under massive timing chaos — inflating to ~2.4 seconds and then concealing the vast majority of the stream.

Since mediasoup terminates the incoming PlainTransport RTP stream and re-originates a new one for the WebRtcTransport consumer (new SSRC, new sequence base, new timestamp base), I suspect the issue is in how that re-timestamping happens for this specific producer — possibly the initial timestamp offset/mapping, or some inconsistency in how the new RTP timestamp is derived from the original ffmpeg-sourced timestamps.

Is there a known consideration for how mediasoup maps original producer RTP timestamps to the new consumer stream, especially for PlainTransport-sourced (non-WebRTC-native) producers? Any specific worker-level debug logs or trace events you’d recommend to inspect this re-timestamping behavior directly?

8

If you want to compare the incoming and outgoing RTP you can enable `RtpDump` in Chrome and then retrieve the RTP in a .pcap out of it for analysis. Instructions here.

For audio we use the `SimpleProducerStreamManager` which does forward the RTP Timestamp as it is, so we don’t generate any Timestamp shift.

9

Thanks! I’m planning to switch from mediasoup to MediaMTX. Do you have any experience working with MediaMTX before?

10

Probably you don’t want to ask that question to mediasoup authors in the mediasoup support forum.