delay in consuming rtpplaindata

mediasoup libraries
1

Hello,
I managed to consume RTP using ffmpeg but having a small issue at the start of the ffmpeg command.
I get lot of warnings for like 1minute, then ffmpeg starts writing correctly to the file. I am using the simplest basic SDP file:
c=IN IP4 192.168.56.1
m=video 5555 RTP/AVP 101
a=rtpmap:101 VP8/90000

I suspect that media soup is not sending keyframes in the first 1 minute?

Here is the log:

ffmpeg -protocol_whitelist file,pipe,udp,rtp -f sdp -i test.sdp -y -c copy mediasoup.webm
ffmpeg version 4.2.2 Copyright © 2000-2019 the FFmpeg developers
built with Apple clang version 11.0.0 (clang-1100.0.33.17)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
[sdp @ 0x7fec2c012600] Keyframe missing
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25908 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25909 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25910 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25911 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25912 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25913 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25914 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25915 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25913 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25914 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25913 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25914 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25914 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25913 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25914 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25915 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25571 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25570 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25571 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25572 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25571 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25572 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25573 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25574 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25575 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25576 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25577 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25575 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25576 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25577 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25576 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25577 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25228 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25228 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25227 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25228 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25229 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25228 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25229 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25229 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25230 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25231 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25232 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25233 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25234 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25235 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25236 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 25237 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24902 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24903 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24904 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24905 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24906 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24907 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24908 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24909 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24910 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24911 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24912 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24913 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24911 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24912 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24911 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24912 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24570 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24571 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24572 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24573 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24574 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24572 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24573 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24574 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24573 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24574 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24575 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24576 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24577 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24578 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24579 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24580 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24210 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24211 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24212 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24213 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24214 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24215 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24216 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24217 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24218 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24219 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24220 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24221 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24219 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24220 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24221 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 24220 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 23875 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 23875 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 23875 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
[sdp @ 0x7fec2c012600] RTP: missed 23875 packets
[sdp @ 0x7fec2c012600] max delay reached. need to consume packet
…
…
frame= 451 fps= 30 q=-1.0 Lsize= 2851kB time=00:00:15.49 bitrate=1506.9kbits/s speed=1.05x
video:2847kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.128698%

2

Have you tried the below?

When creating a consumer it's recommended to set paused to true, then transmit the consumer parameters to the consuming endpoint and, once the consuming endpoint has created its local side consumer, unpause the server side consumer using the resume() method.

Info can be found below @transport.consume

3

mediasoup does NOT send keyframes but just relays them when appropriate. mediasoup is a SFU and not a video encoder.

4

Thank you all for your reply, indeed it works now after setting it to pause true/flase afterwards but I still see some warning on ffmpeg side even though the video seems good enough, can I ignore those warnings?

sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15241 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15242 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15243 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15244 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15245 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15246 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15247 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15248 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15249 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15250 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15248 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15249 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15248 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15249 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15250 packets
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15251 packets
frame= 863 fps= 30 q=-1.0 size= 1861kB time=00:00:29.69 bitrate= 513.4kbits/s speed=frame= 878 fps= 30 q=-1.0 size= 2814kB time=00:00:30.19 bitrate= 763.5kbits/s speed=frame= 893 fps= 30 q=-1.0 size= 2814kB time=00:00:30.72 bitrate= 750.5kbits/s speed=frame= 902 fps= 30 q=-1.0 Lsize= 2973kB time=00:00:31.03 bitrate= 784.8kbits/s speed=1.02x
video:2966kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead

5

Hello again,
So what I did is I started the ffmpeg command FIRST and then executed the producer/consumer on media soup side. I am still getting those above errors
[sdp @ 0x7f8ede808200] max delay reached. need to consume packet
[sdp @ 0x7f8ede808200] RTP: missed 15250 packets
The pause is set to FALSE as I am running ffmpeg first to avoid missing any packets

Any hints on what I am doing wrong?
Thanks

6

Do not underestimate the Leeryver Symptom (in Spanish “Leer-y-ver”, in English “Read-and-see”):

“I never look at the logs when things work fine so, when they don’t, I look at the logs for the first time and whichever first warning/error line I see I blame it.”

7

I am reading, seeing but not finding any answer. I understand that media soup sends some corrections by ffmpeg does not how to use? read this

Do I have to with from ffmpeg to streamer to solve this?

8

I don’t know much about ffmeg or gstreamer, so this is not something that I want to answer. But yes, ffmpeg lacks lot of RTCP basic features (RTX, NACK, PLI) while gstreamer supports them. Said that, this does not mean that I can help much more with gstreamer.

1 Like