Sending audio stream from mediasoup to sound card in node app

Integration
1

Hi all,

I’m working on an app to receive multiple audio streams using webrtc. I’ve got it working in basic form using PeerJS with one peer as the main receiver, and multiple other peers sending audio to it. I then send the audio out to different local audio devices via mediaelements and setsinkid etc.

But I wanted to move it to a node app which removes that ability.

After going around the houses, I’m now trying mediasoup. I was hoping I’d be able to easily modify one of the examples (like the recording example) so that instead of sending to ffmpeg or gstreamer, it was played to a sound card (this is on a mac).

There must be some way of getting ffmpeg or gstreamer to rout the output to an audio output device but I can’t work it out and I’ve already spent a long time learning new things to get this far… ffmpeg seems like a whole lifetime of experience.

I could use chromium in the node app to replicate my browser app, but that seems overkill.

Is there a way to take the audio stream and send it to an audio device that anyone could help me with please?

I appreciate this isn’t core mediasoup, and I’ve noticed a few complaints in replies to other people re similar, so please, if you can help in anyway, that’s great. But if not, please don’t start a row. I’d rather have no replies than a row. We’ve all got more (and too many) things to do.

Thanks.

2

You can make ffmpeg or gstreamer play that stream to the sound car.

3

Thanks for the reply. I’ve been trying but without success. ffmpeg don’t have a real forum, the IRC group has totally ignored my question. I’ll see if I can find somewhere to get help on that or gstreamer unless anyone here knows how. Thanks again.

4

I think it would be the easiest for you to get audio from mediasoup as plain RTP stream, after which you should be able to consume it both in ffmpeg and gstreamer without major issues. Finding examples on how to use RTP with both should yield multiple matching results.

5

Thanks all.

Ok. I’ve managed to get the audio stream picked up by vlc, which is my preferred choice at the moment because it’s the only one out of ffmpeg, sox and vlc that I can get to output to my virtual sound cards at the moment…

I do have one problem though. This is all on the same PC, and I’m getting a 2 second delay from the sound going in to mediasoup and the sound coming out of vlc. If I play a file to the same device, it’s instant. My mic-in to the mac is instant (it’s through a digital mixer which acts as a usb device to the mac so there’s a tiny delay but I don’t have any additional delay set). Even if I set the output to my monitor (as in display monitor), rather than the mixer, I get the same delay, so it’s not the mixer.

I’m still using the same recording demo, sending audio only to external.

I’m using this sdp with vlc

v=0
o=- 0 0 IN IP4 127.0.0.1
s=-
c=IN IP4 127.0.0.1
t=0 0
m=audio 5004 RTP/AVP 111
a=rtcp:5005
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10;useinbandfec=1

Is there something I could be doing wrong, or something I can change to reduce the delay please?

Thanks

ps vlc doesn’t handle AVPF so have to set the sdp to AVP which I believe isn’t a huge issue as long as I don’t get packet loss. Would still prefer to use ffmpeg or sox though but just can’t find a way to get it to go to my output devices :frowning:

6

Most probably VLC is buffering it for smooth playback, ffmpeg has -fflags nobuffer, maybe you can tune VLC similarly.

1 Like
7

Yes, thanks Nazar. I thought that myself when I woke up today, will check it after the school run!

Would prefer to get ffmpeg working but I just can’t get it to see my output devices. Installed it via brew on a mac. Maybe I need to look at compiling it, although I have added an additional tap that adds some features but there still seems to be an issue with using coreaudio due to licensing issues which, for my use as a private in-house app, isn’t a worry as I won’t be distributing any binaries.

This is my task for this morning, and then I have most of what I need for my needs!

Thanks

8

Yep, that was it. Using this line

vlc vlc.sdp --auhal-audio-device=211 --network-caching=20

and it now works fine!

Guess most of what I need now is going to be non-mediasoup related? For instance, if I stop the stream and restart it, vlc doesn’t pick it up again but I presume that’s something I’ll need to monitor on the vlc side.

The other part of this is the video. I may just do the audio for now and use another way to get the video in, but ideally I’d be able to stream the video in to Ecamm Live using - in order of preference I think - NDI, Virtual Web Cameras, Syphon (but that means displaying it on the screen first), HDMI out on a raspberry PI into a capture card on the mac (cost of 4xPIs). But I’ll see how it goes syncing with using another method for the video first before I go down that road.

Thanks again for the help, appreciate it wasn’t core mediasoup but you’ve been very friendly!