Outliter RTP timestamps causing synchronization issues

mediasoup libraries
1

I observe a rare issue when RTP timestamps of one of the tracks are way off comparing to other tracks, see this example with timestamps of 11 tracks over WebRTC connection:

680895002378
680828438931
27235570107
680832383284
680882493144
680744178737
680726552189
680723106192
680433869400
680770577490
680703423068

Those are in nanoseconds, but you can clearly see where the outlier is.

Those cause all kinds of issues in GStreamer application when trying to synchronize those tracks so I’m wondering if this is an expected behavior (doesn’t look like it) or there might be some bug in mediasoup (like integer overflow somewhere) that might cause this? There is always a chance that this is not a mediasoup bug, but it looks very suspicious to me considering that other tracks are fine.

I have no reproducible demo for this, but the issue does definitely happen in production and I can add some instrumentation if needed.

2

If you mean those, the latter could be an RTX packet or a RTP probation packet (SSRC 1234 always).

3

It was SSRC 948297492, not 1234, and it was an audio track without RTX with following timestamps (first 10):

27235570107
27248363451
27261055902
27273708144
27286382092
27299139527
27312040694
27325142916
29534676130
29554663002

It happens both to audio and video tracks and is always normal-looking when ignoring the rest of the tracks.

4

Would be nice seeing some dev logs pointing the original and new RTP Timestamp being set.

5

It is only happening in production with a very small number of cases.

My observation is that this only happens after renegotiation with newly added tracks, I haven’t see this happening to tracks added earlier in the middle of the way.

Maybe I can put some instrumentation in the worker code to log useful info? I’m still not as familiar with code base to figure out where to place it exactly. Something like logging some timestamps of all consumers on a transport once new consumer is added and resumed.

6

Maybe I can put some instrumentation in the worker code to log useful info

Not needed in this case after all. We use SimpleConsumer for audio, and there we don’t touch the RTP timestamp :slight_smile:. So that timestamp already comes that way from the endpoint.

7

Correct me if I’m wrong, but every endpoint should choose random initial RTP timestamp, but when someone consumes multiple tracks over the same WebRTC connection, those should be in the same relative time domain, am I right?

8

Nope. Each RTP stream has its own super random timestamp origin. When sending (or receiving) N tracks from same media source (same peer basically) the RTP absolute timestamp in the RTCP Sender Reports of each RTP stream are also random values but they must be incremented following the same source clock. We deal with that in SimulcastConsumer.cpp (for creating a single and continuos stream for the consumer when it switched from one stream to another).

1 Like
9

OK, that makes sense, thank you!
I’ll dig more into GStreamer side of things then.