How to add media stream to a producer

Integration
1

Hi,

I am using NodeJS with mediasoup v3 for creating a sfu server using mediasoup v3.
But i don’t understand if what i do is well and how to add the media stream that i get from the front end to the producer.

I would like to know, how to add a media stream to a pipe producer, so every consumer who join the stream can get the media stream and see a livestream or video conference.

Here is my code for creating the pipeProducer on a nodejs back end:

async createPipeProducer(userStreamerId: string, mediaStream: MediaStream) {
const router = await this.createRouterFromWorker(this.worker);

    const transport = await router.createWebRtcTransport(
        {
            webRtcServer : this.server, // Or ListenIps
            // listenIps    : [ { ip: "192.168.0.111", announcedIp: "88.12.10.41" } ],
            enableUdp    : true,
            enableTcp    : true,
            preferUdp    : true
        }
    );

    transport.on("icestatechange", (iceState) => {
        console.log(`ICE state changed to ${iceState}`);
    });

    // Get the supported RTP capabilities
    const rtpCapabilities = router.rtpCapabilities;

    // Create the producer
    const pipeProducer = await transport.produce({
        kind: 'video',
        rtpParameters: rtpCapabilities["video"].codecs.find(codec => codec.mimeType.toLowerCase() === "video/vp8"),
        appData: { type: 'pipe' }
    });

    // Set the media stream as the source of the producer
    const track = mediaStream.getVideoTracks()[0];
    track.onended = () => {
        pipeProducer.pause();
    };

    // It's in this part that i am stuck
    const mediaSender = await pipeProducer.getStats();
    const rtpSenders = mediaSender[0].rtpSenders;
    mediaSender.send(track);

    return pipeProducer;
}

Am i making everything great ? Expect for add a media stream to the producer, so every consumer can join a specific producer’s stream.
I can provide more informations if it can help you to help me.
Thanks for your help.

2

You are mixing server side code with client side code, we do this on server side:

And this on client side:

I think you have missed many things in the documentation, perhaps a look at the demo code will give you good understanding.

Further more you have mentioned about ‘piping’, this is not used normally but only used when we need to share producer between routers within a server on between separate servers.

Apart from this the basic step to produce are these:

  • Create router, workers etc on server
  • Create transport on server, and based upon the result create transport on client as well
  • Use transport.produce() method to produce track on the transport you just created. It will give you producer
  • After that ‘producer’ event will fire on transport you created, and on that event you will send producer details on your server.
  • After that ‘connect’ event will fire on transport you created, and on that event you will send transport details so finally peer connection will be established and track will start flowing to the server.
3

and how to add the media stream to a producer then ? In the front end or in the back end ?

4

i have asked chatGPT and createWebRtcTransport can be used in both back end and front end.
Here is his answer:

createWebRtcTransport is part of the mediasoup library and can be used in both the front-end and the back-end.

In a server-side application, createWebRtcTransport would typically be called on the server to create a WebRTC transport object that can be used to send and receive media streams from clients. In a client-side application, createWebRtcTransport would typically be called on the browser to create a WebRTC transport object that can be used to send and receive media streams from the server.

So, createWebRtcTransport is not specific to the front-end or the back-end, but rather it is used in both contexts.

5

I understand the flow now.
But i have checked the mediasoup-client library and it seems that the connect() method doesn’t exist.
why that ?
Do i still ahve to connect before using produce() method ?
If yes and if it doesn’t exist anymore, by what should i change that ?

Thanks for your help.

6

The first call to produce will initiate the connection process internally. You should catch the “connect” event on the transport and manage the signalling.

7

createWebRtcTransport is used in backend while createSendTransport/ createRecvTransport is used on client side.

Follow the docs brother, I think you are rushing through the docs, take your time and it will all reveal to you.

8

There is no connect() method in mediasoup-client, on client side we have ‘connect’ event while on server side we have connect() method:

When you call produce() on client side then it will trigger ‘connect’ event on transport automatically on that event you will signal to your server some parameters and on server side you will call transport.connect() method and then these transports are connected.

9

I did that, can you confirm me it is correct in this way ?
I first create a new instance of Device:

const device = new Device();

Then i use createSendTransport method from device :slight_smile:

this.sendTransport = device.createSendTransport(responseStreamCreated.transport.transportOptions);

Then listen the connect event:

this.sendTransport.on(‘connect’, ({ dtlsParameters }, callback) => {
// Send dtlsParameters to the backend
callback();
});

From now it seems to work, but once i try to isten produce event to add the media stream audio and media stream video to make a stream like that:

this.sendTransport.on(‘produce’, async (parameters, callback, errback) => {
try {
this.localStream.getTracks().forEach(async (track) => {
const producer = await this.sendTransport.produce({
track: this.localStream.getTracks()[0],
codecOptions: {
opusStereo: true,
opusDtx: true,
},
});
callback({ id: producer.id });
});
} catch (err) {
errback(err);
}
});

Is everything alright ?

10

This is wrong. You shouldn’t call transport.produce() method inside ‘produce’ event, it should be reverse i.e you should call transport.produce() outside somewhere after creating transport and registering for event, produce() method will automatically call ‘produce’, ‘connect’ events. And on these events you will signal data to your server

11

Read the documentation, it describes what and when has to be done pretty well.

12

but the problem is i have an error on this.sendTransport.on, on method doesn’t exist

13

Then the problem is you are not creating transport correctly. To create transport on client side, which you are creating, you must create transport on server before that. And from the server’s response you should create the transport on client side and then listen to events.

Try logging this.sendTransport and see what it contains.

14

yes i created a transport in back end, then get the transport information using websocket, from the response i built the transport like that:

this.sendTransport = device.createSendTransport(responseStreamCreated.transport.transportOptions);

Here is the console.log response:

 1. Transport {_events: {…}, _eventsCount: 0, _maxListeners: Infinity, _closed: false, _connectionState: 'new', …}

  1. _appData: {}
  2. _awaitQueue: AwaitQueue {pendingTasks: Map(0), nextTaskId: 0, stopping: false}
  3. _canProduceByKind: {audio: true, video: true}
  4. _closed: false
  5. _connectionState: "new"
  6. _consumerCloseInProgress: false
  7. _consumerCreationInProgress: false
  8. _consumerPauseInProgress: false
  9. _consumerResumeInProgress: false
  10. _consumers: Map(0) {size: 0}
  11. _dataConsumers: Map(0) {size: 0}
  12. _dataProducers: Map(0) {size: 0}
  13. _direction: "send"
  14. _events: {connect: ƒ, produce: ƒ}
  15. _eventsCount: 2
  16. _extendedRtpCapabilities: {codecs: Array(2), headerExtensions: Array(11)}
  17. _handler: Chrome74 {_events: {…}, _eventsCount: 2, _maxListeners: Infinity, _mapMidTransceiver: Map(0), _sendStream: MediaStream, …}
  18. _id: "049f3994-14ea-411b-8f5b-3833839d44c0"
  19. _maxListeners: Infinity
  20. _maxSctpMessageSize: null
  21. _observer: EnhancedEventEmitter {_events: {…}, _eventsCount: 0, _maxListeners: Infinity}
  22. _pendingCloseConsumers: Map(0) {size: 0}
  23. _pendingConsumerTasks: []
  24. _pendingPauseConsumers: Map(0) {size: 0}
  25. _pendingResumeConsumers: Map(0) {size: 0}
  26. _probatorConsumerCreated: false
  27. _producers: Map(0) {size: 0}
  28. appData: (...)
  29. closed: (...)
  30. connectionState: (...)
  31. direction: (...)
  32. handler: (...)
  33. id: (...)
  34. observer: (...)
  35. [[Prototype]]: EnhancedEventEmitter
15

I have connect and produce event on my object, but it’s maybe not by using .on that it will work ?

16

Why do you think that on method does not exist? It is in the EnhancedEventEmitter prototype.

17

i just don’t understand why i have an error that sendTransport is undefined then if it exist.
I have this exact error:

core.mjs:6412 ERROR Error: Uncaught (in promise): TypeError: Cannot read properties of undefined (reading 'on')
TypeError: Cannot read properties of undefined (reading 'on')
18

this is the transportOptions i am passing to this.sendTransport = device.createSendTransport(responseStreamCreated.transport.transportOptions);:

{id: '48aded87-b36f-42f3-b050-b6cf74aa3264', iceParameters: {…}, iceCandidates: Array(1), dtlsParameters: {…}}
dtlsParameters
: 
{fingerprints: Array(5), role: 'auto'}
iceCandidates
: 
[{…}]
iceParameters
: 
{iceLite: true, password: 'qrohkckjxofhawa0fwlm15h8edwenpbt', usernameFragment: 'oxi7u6mpzmtfoqcexe70znbqwvyclfdf'}
id
: 
"48aded87-b36f-42f3-b050-b6cf74aa3264"
[[Prototype]]
: 
Object
19

You definitely call the on method not on what createSendTransport returned (and which is logged in the console.log). It is some uninitialized variable: “TypeError: Cannot read properties of undefined”.

20

This error may be coming from somewhere else, make sure where it is coming from, show the exact line from developer tools where this error is occurring.

Also console.log(this.sendTransport); console.log(this.sendTransport.on); together after transport is created and show the result.