Forward Mediasoup RTP Transport to AWS Elemental Media Live

Integration
#1

Hello,

Thank you for the awesome mediasoup library. I have been playing around with it the last couple of weeks and really appreciate the amount of documentation and examples.

I was wondering if anyone has created an RTP transport using the mediasoup library and forwarded that stream to a 3rd party service such as AWS Elemental Media live for one-to-many streaming.

Found this example: https://d1.awsstatic.com/awselemental/workflowexamples/Workflow3_Example_FFMPEG_RTP_to_MediaLive_and_MediaPackage.pdf and wanted to see if this were possible using mediasoup.

AWS Elemental MediaLive supports RTP (MPEG TS) ingest with H.264 and AAC codecs and that example shows this code for how to use FFMPEG to transcode a video/audio file for publishing in AWS Elemental Media Live:

ffmpeg -re -i <source_file> -c copy -map 0 -f rtp_mpegts -fec prompeg=l=5:d=20
rtp://<IP>:5000

Would it be possible to forward mediasoupā€™s RTP transport to this service and transcode via FFMPEG on the fly?

I tried using the H.264 video codec and transcoding to AAC (because mediasoupā€™s audio codec is OPUS), and the RTP stream I created had choppy video.

These were my FFMPEG args:

    const commandArgs = [
      "-protocol_whitelist",
      "pipe,udp,rtp",
      "-fflags",
      "+genpts",
      "-f",
      "sdp",
      "-re",
      "-i",
      "pipe:0",
      "-vcodec",
      "copy",
      "acodec",
      "aac",
      "-f",
      "rtp_mpegts",
      "-fec",
      "prompeg=l=5:d=20",
      "rtp://127.0.0.1:5000"
    ];

I realize my post is not tied to support for the mediasoup library, my apologies if it does not belong here, but I wasnā€™t sure where else to post it and was curious if anyone else has tried this workflow. Thanks.

#2

mediasoup audio codec is not ā€œjustā€ OPUS. It supports tons of them:

Anyway, since you want to use WebRTC clients, you need mediasoup to use OPUS or any other audio codec supported by browsers (AAC is not).

Regarding your question: I donā€™t know what to say. You can forward plain RTP from mediasoup to other service. Such a service may transcode and then send it to a different service, etc. Cannot help much more since FFMpeg or Gstreamer is not my confort zone.

#3

Thank you for your response. Sorry for the noobish nature to my question. This is new technical territory for me. Iā€™ll look elsewhere for advice on transcoding the RTP stream. Thanks again for making this library available.

#4

No problem at all. In order to transcode, the flow coming to my mind is:

  1. browser opus+h264 to mediasoup.
  2. mediasoup via plain RTP transport(s) to a FFMpeg or GStreamer process(es).
  3. Make FFMpeg or GStreamer send transcoded media to the external service via RTP.
#5

Hi,

Did you manage to make it work correctly ?

I have something working using h264 & opus => aac. But there is an issue : when chrome goes to background then goes back to foreground again, ffmpeg stops with the following dts error :

Application provided invalid, non monotonically increasing dts to muxer in stream 0: 3465090 >= 3080070

Do you have any idea how to solve this ?

Here are copies of ffmpeg command line and sdp :

ffmpeg
-loglevel verbose
-protocol_whitelist pipe,udp,rtp
-debug_ts
-f sdp
-i pipe:0
-map 0:v:0
-c:v copy
-map 0:a:0
-c:a aac
-flags +global_header
-f rtp_mpegts
-fec prompeg=l=5:d=20
rtp://x.x.x.x:5000

v=0
o=- 0 0 IN IP4 127.0.0.1
s=-
c=IN IP4 127.0.0.1
t=0 0
m=video 28236 RTP/AVPF 125
a=rtcp:28237
a=rtpmap:125 H264/90000
a=sendonly
m=audio 28716 RTP/AVPF 111
a=rtcp:28717
a=rtpmap:111 opus/48000/2
a=sendonly
#6

I donā€™t have experience with AWS Elemental, but definitely have dealt with similar timestamp issues pulling audio and video out of the RTP streams and trying to maintain sync.

What works (mostly) for our use cases seems to be a combination of these flags:

// part of the command args array from our javascript code that forks ffmpeg ...

'-fflags', '+genpts',
'-output_ts_offset', (Date.now()/1000.0),
'-copyts'

One drawback to the above is that timestamps donā€™t start at zero ā€“ theyā€™re large numbers relative to what some media players expect to see in timestamps, because they are wall-clock times. ffmpeg and other lower-level toolchains are fine with them, though.

Let me know if any of the above doesnā€™t make sense and Iā€™ll look more at what weā€™re doing.

1 Like
#7

Hi all,
I am trying something similar trying to get VP8/OPUS from mediasoup into elemental.

Here is my config:

/opt/ffmpeg-git/ffmpeg -loglevel verbose -protocol_whitelist file,udp,rtp -debug_ts -i sdp1 -map 0:v:0 -c:v copy -map 0:a:0 -c:a aac -flags +global_header -f rtp_mpegts -fec prompeg=l=5:d=20 rtp://192.168.56.101:5000

v=0
o=- 0 0 IN IP4 127.0.0.1
s=-
c=IN IP4 127.0.0.1
t=0 0
m=video 5557 RTP/AVPF 101
a=rtcp: 5558
a=rtpmap:101 VP8/90000
a=sendonly
m=audio 5555 RTP/AVPF 102
a=rtcp:5556
a=rtpmap:102 opus/48000/2
a=sendonly

When I pay ffmepg I get the following:
[sdp @ 0x70d1300] setting jitter buffer size to 500
Last message repeated 1 times
[sdp @ 0x70d1300] Keyframe missing
[sdp @ 0x70d1300] RTP: dropping old packet received too late
Last message repeated 5 times
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6173 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6174 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6175 packets
[sdp @ 0x70d1300] RTP: dropping old packet received too late
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6169 packets
[sdp @ 0x70d1300] RTP: dropping old packet received too late
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6160 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6153 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6143 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6144 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6145 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6146 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6141 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6132 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6125 packets
[sdp @ 0x70d1300] max delay reached. need to consume packet
[sdp @ 0x70d1300] RTP: missed 6119 packets
[sdp @ 0x70d1300] Could not find codec parameters for stream 0 (Video: vp8, 1 reference frame, yuv420p): unspecified size
Consider increasing the value for the ā€˜analyzedurationā€™ and ā€˜probesizeā€™ options

replay rtp://192.168.56.101:5000 does nothing and only hangs.

Did you guys got it working? Appreciate a working example to base my testing on

Thank you