How to seperate the audio&video data of the diferent channel ?

mediasoup libraries
1

I want to get some channels the YUV(for video ) and PCM (for audio) data, But
the peerconnection was created one only by libmediasoupclient , so I get the data by the function
in webrtc::AudioTrackSinkInterface OnData
class AudioTrackSinkInterface {
public:
virtual void OnData(const void* audio_data,
int bits_per_sample,
int sample_rate,
size_t number_of_channels,
size_t number_of_frames) = 0;

Now, I hold the one peerconnection , but I hold some consumers , I can’t indentify the different consumer from the peerconnection.
How to seperate the audio&video data of the diferent channel ?

2

You are the one that creates the Consumers in your libmediasoupclient based application, so you can access the consumer.track of each Consumer independently.

I don’t know why are you pasting the webrtc::AudioTrackSinkInterface class above, but you are not supposed to need it. Said that, you may need to disable the default audio routing of libwebrtc in OSX or Linux so it does not go directly to the audio output device.

3

And no, you do not need separate PeerConnections for that.

4

Yes, I want to record all the consumers’s data, so I plan to get the video and audio from the each PeerConnection ,then mix and write to file .

5

You may need to override the default webrtc::AudioDeviceModule class which in Linux and OSX directly sends the received audio to the speakers:

#include "modules/audio_device/include/audio_device.h"

class AudioDeviceDummy : public webrtc::AudioDeviceModule {
  // override things
}