Get raw audio from RTP packet with DirectTransport

Hi!

does anyone have experience manipulating the RTP packets with DirectTransport with rtp.js library? I would like to extract the raw audio of the specific producer to manipulate in Nodejs

        const room = Room.getRoom(roomId);
        const router = room.router;
        const directTransport = await router.createDirectTransport();
        const audioProducer = getProducer(roomId, id, 'audio');
        const consumer = await directTransport.consume({ producerId: audioProducer.id, rtpCapabilities: router.rtpCapabilities });
        consumer.on('rtp', (rtpPacket) => {
             const packet = new rtp.RtpPacket(rtpPacket);
             const payload = packet.getPayload();
        }

Thanks in advance

The answer is yes.

Perfect!

Can you share some examples to understand how to get the raw audio from RTP packet with rtp.js library?

Thank you :slight_smile:

I cannot, sorry. We write API documentation for something.

1 Like

Hi, thanks!

The documentation of the RTP.js API only says that getPayload return a Buffer but I don’t know how to process the audio data from here (RtpPacket | rtp.js - v0.0.1). I only need a little advice about how to do that

Thanks again <3

Me neither. And that’s not something that the library is supposed to help with.

What you want in general is to have RTP jitterbuffer, then depayer that will extract encoded audio from RTP and only then you can decode Opus or whatever codec you decided to use to get raw audio out of it.
This is what tools like libav/ffmpeg and GStreamer are often used for.

1 Like

Hi!,
I am getting directTransport does not have consume method, does the above mentioned code is correct?

Of course it has. Please rely on the API docs and then review your code. Something is wrong in it.

1 Like

Hi,
getting this error

ERROR:Room "request" failed [error:"UnsupportedError: consume() not implemented in DirectTransport at DirectTransport.consume (/usr/src/app/node_modules/mediasoup/lib/DirectTransport.js:92:15) at Record.publishProducerRtpStream (/usr/src/app/lib/Record.js:165:62)     at processTicksAndRejections (internal/process/task_queues.js:95:5) at async Record.startRecord (/usr/src/app/lib/Record.js:197:41) at async Room._handleRecording (/usr/src/app/lib/Room.js:569:4) at async Room._handleSocketRequest (/usr/src/app/lib/Room.js:1812:5)"]

Probably you are using some old mediasoup version but you didn’t even mention it.

Hi,
I am using v3.5.14
I did some console logs for directTansport.consume and got this [AsyncFunction: consume], but while doing const rtpDirectTransportConsumer = await directTransport.consume({ producerId: producer.id, }); I am getting that error

Then update mediasoup to the latest version.

1 Like

I have updated the mediasoup and event is working fine now (Thank you), I have successfully saved rtp dump using rtp.js library
Now I just want a direction on how to convert these packets to audio/video file later on server, I have read about rtp replay using rtptools but I don’t know how to create that rtpdump file format which rtp replay require in its payload
Just want to know what are the steps now it will require to convert these packets to audio video
jitter buffer, gstreamer, rtptools etc and in which order?

Hi. Why not using plainTransport to GStreamer? GStreamers handle all the things, and jitter buffer that is not an easy thing to deal with, specially video, at your own.

Hi, I don’t want to pass directly to gstreamer because if my platform starting to produce multiple record request than it can also hinder the group video calls (I don’t want to spend that much on my servers), I want to delegate the recording process to another server which will one by one process recordings RTP dump and will produce final video file for users, I don’t want it realtime.