How to add media stream to a producer

I my front end:

transport.on return:

ƒ on(eventName, listener) {
    super.on(eventName, listener);
    return this;
  }

And just transport return:

Transport {_events: {…}, _eventsCount: 0, _maxListeners: Infinity, _closed: false, _connectionState: 'new', …}
_appData
: 
{}
_awaitQueue
: 
AwaitQueue {pendingTasks: Map(0), nextTaskId: 0, stopping: false}
_canProduceByKind
: 
{audio: true, video: true}
_closed
: 
false
_connectionState
: 
"new"
_consumerCloseInProgress
: 
false
_consumerCreationInProgress
: 
false
_consumerPauseInProgress
: 
false
_consumerResumeInProgress
: 
false
_consumers
: 
Map(0) {size: 0}
_dataConsumers
: 
Map(0) {size: 0}
_dataProducers
: 
Map(0) {size: 0}
_direction
: 
"send"
_events
: 
{connect: ƒ, produce: ƒ}
_eventsCount
: 
2
_extendedRtpCapabilities
: 
{codecs: Array(2), headerExtensions: Array(11)}
_handler
: 
Chrome74 {_events: {…}, _eventsCount: 2, _maxListeners: Infinity, _mapMidTransceiver: Map(0), _sendStream: MediaStream, …}
_id
: 
"815de9c8-22a2-47ac-88e7-765a273ac426"
_maxListeners
: 
Infinity
_maxSctpMessageSize
: 
null
_observer
: 
EnhancedEventEmitter {_events: {…}, _eventsCount: 0, _maxListeners: Infinity}
_pendingCloseConsumers
: 
Map(0) {size: 0}
_pendingConsumerTasks
: 
[]
_pendingPauseConsumers
: 
Map(0) {size: 0}
_pendingResumeConsumers
: 
Map(0) {size: 0}
_probatorConsumerCreated
: 
false
_producers
: 
Map(0) {size: 0}
appData
: 
(...)
closed
: 
(...)
connectionState
: 
(...)
direction
: 
(...)
handler
: 
(...)
id
: 
(...)
observer
: 
(...)
[[Prototype]]
: 
EnhancedEventEmitter

connect and product is in _event from transport object:

_events
: 
connect
: 
({ dtlsParameters }, callback) => {…}
produce
: 
ƒ (_x, _x2, _x3)
_eventsCount
: 
2

i think i have more information:

This is what i get when i check what i hae from connect and produce:

_events
: 
connect
: 
({ dtlsParameters }, callback) => {…}
length
: 
2
name
: 
""
arguments
: 
[Exception: TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them at Function.invokeGetter (<anonymous>:3:28)]
caller
: 
[Exception: TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them at Function.invokeGetter (<anonymous>:3:28)]
[[FunctionLocation]]
: 
streaming-livestream.component.ts:159
[[Prototype]]
: 
ƒ ()
[[Scopes]]
: 
Scopes[3]
produce
: 
ƒ (_x, _x2, _x3)
length
: 
3
name
: 
""
prototype
: 
constructor
: 
ƒ (_x, _x2, _x3)
[[Prototype]]
: 
Object
arguments
: 
[Exception: TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them at Function.invokeGetter (<anonymous>:3:28)]
caller
: 
[Exception: TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them at Function.invokeGetter (<anonymous>:3:28)]
[[FunctionLocation]]
: 
streaming-livestream.component.ts:168
[[Prototype]]
: 
ƒ ()
[[Scopes]]
: 
Scopes[4]

Btw i am using Angular, it’s in a streaming component.

This indicates that on function exists, the error you are getting is from somewhere else. You need to get deeper into it. What is the mediasoup-client version you are using?

This is normal.

I am using version 3.6 of mediasoup-client

yes that’s weird because i ahve debugged and remove every other part of my code and it crash here.
If i comment the sendTransport.on i have no more error, if i uncomment it tthe error come back

Remove everything from the callback and test it. Also make sure you are loading rtp capabilities on device via device.load() after creating device.

I had already load with device.load() before like that:

const device = new Device();
const routerRtpCapabilities: RtpCapabilities = responseStreamCreated.transport.rtpCapabilities;
await device.load({routerRtpCapabilities});

The routerRtpCapabilities object look like this:

{id: 'd1362acb-970f-426f-80cd-6c8551035adc', iceParameters: {…}, iceCandidates: Array(1), dtlsParameters: {…}}
dtlsParameters
: 
fingerprints
: 
Array(5)
0
: 
{algorithm: 'sha-512', value: '86:A3:17:16:43:3A:FB:BF:C5:8C:2C:26:B8:5C:A7:47:E4…2:60:C4:7B:05:4F:B7:A9:70:93:4E:01:F4:01:89:46:11'}
1
: 
{algorithm: 'sha-256', value: 'F7:0B:E9:B0:FE:4D:CD:7D:AE:58:02:AE:E9:DA:F0:4D:AD:85:E3:27:94:71:66:3E:77:4B:68:AF:F3:ED:9D:3A'}
2
: 
{algorithm: 'sha-384', value: 'A6:C4:B2:FE:5F:CF:76:BD:06:F1:36:AA:E8:DF:E6:7F:61…6:C6:8E:D6:EE:A9:B2:28:32:5A:D8:5C:E6:A9:97:55:32'}
3
: 
{algorithm: 'sha-1', value: '47:8B:36:F4:34:62:C4:54:03:96:07:13:BB:9A:7C:AD:E0:00:7E:67'}
4
: 
{algorithm: 'sha-224', value: '7D:E7:E9:FF:1E:B0:79:6F:51:61:31:FE:90:7E:BF:74:C1:EC:77:C1:63:93:34:3A:E1:E0:D5:8F'}
length
: 
5
[[Prototype]]
: 
Array(0)
role
: 
"auto"
[[Prototype]]
: 
Object
iceCandidates
: 
Array(1)
0
: 
{foundation: 'udpcandidate', ip: '127.0.0.1', port: 4000, priority: 1076558079, protocol: 'udp', …}
length
: 
1
[[Prototype]]
: 
Array(0)
iceParameters
: 
iceLite
: 
true
password
: 
"yz0hkkg3tc31e8igvg96txl8ihcuj1n9"
usernameFragment
: 
"0h2jqyuhzyhf46wqhujcvj7uoniotvhj"
[[Prototype]]
: 
Object
id
: 
"d1362acb-970f-426f-80cd-6c8551035adc"
[[Prototype]]
: 
Object

This is device object after load:

Device {_loaded: true, _observer: EnhancedEventEmitter, _handlerName: 'Chrome74', _extendedRtpCapabilities: {…}, _handlerFactory: ƒ, …}
_canProduceByKind
: 
{audio: true, video: true}
_extendedRtpCapabilities
: 
{codecs: Array(2), headerExtensions: Array(11)}
_handlerFactory
: 
() => new Chrome74()
_handlerName
: 
"Chrome74"
_loaded
: 
true
_observer
: 
EnhancedEventEmitter {_events: {…}, _eventsCount: 0, _maxListeners: Infinity}
_recvRtpCapabilities
: 
{codecs: Array(3), headerExtensions: Array(8)}
_sctpCapabilities
: 
{numStreams: {…}}
handlerName
: 
(...)
loaded
: 
(...)
observer
: 
(...)
rtpCapabilities
: 
(...)
sctpCapabilities
: 
(...)
[[Prototype]]
: 
Object

And i have commented the the callback:

this.sendTransport.on('connect', ({ dtlsParameters }, callback) => {
        console.log("dtlsParameters");
        console.log(dtlsParameters);
        // Send dtlsParameters to the backend
        // callback();
      });

It looks like you somehow call this.sendTransport.on before the result of the device.createSendTransport is assigned to this.sendTransport. Just some general programming mistake, not connected with mediasoup.

1 Like

@indie do verify this, you are doing something wrong in the code as described by @snnz , I suggest you run the demo version on your system first and follow the steps from there.

Yes but how it can’t be assigned since i called it before even with an usefull await

And when i console.log it seems to work well, am i right ?
Maybe i didn’t saw something important

I avoid the error by doing this:

await new Promise((resolve) => {
this.sendTransport.on(‘connect’, ({ dtlsParameters }, callback) => {
console.log(“dtlsParameters”);
console.log(dtlsParameters);
// Send dtlsParameters to the backend
callback();
resolve();
});
});

But it never trigger the console.log(“dtlsParameters”); so i suppose it’s not the behavior expected.

For the context, i click on a button, this button call a custom event socket who create a object in my databse for me to know that a stream is created.
Than i have a redis publisher event who call a event who is listened in another micro service who create the router, then the producer who create a transport.
This transport options is returned to my other back end who return this object to the front end.
From there i did the code i previously wrote where i create a new Device, load it, and create sendTransport.
There i try to listen both event connect and produce who bring me the undefined error.

Can i architect my front end and back end like that ?
Or do i have to use specific event from mediasoup-client library to communicate and create server and worker ?
Maybe the error can come from there, but i don’t know it would be weird, because we can create a new transport from device, from a transport options object without even calling the back end.
I followed the doc from mediasoup-client from npm here: mediasoup - npm

I provide my function in back end who to the main job for each stream created:

async createPipeProducer(userStreamerId: string, stream: any) {
        try {
            const router = await this.createRouterFromWorker(this.worker);
            const transport = await this.createTransportForRouter(router);

            // Get the supported RTP capabilities
            const rtpCapabilities: mediasoup.types.RtpCapabilities = router.rtpCapabilities;

            const codec: mediasoup.types.RtpCodecCapability = rtpCapabilities.codecs.find(codecs => codecs.mimeType.toLowerCase() === "video/vp8");

            if (!codec) {
                throw new Error(`Router does not support this mime type`);
            }

            const rtpCodecParameters: mediasoup.types.RtpCodecParameters = {
                payloadType: codec.preferredPayloadType,
                clockRate: codec.clockRate,
                mimeType: codec.mimeType,
                parameters: {
                  'x-google-start-bitrate': 1000,
                },
            };

            const rtpParameters: mediasoup.types.RtpParameters = {
                codecs: [rtpCodecParameters],
                rtcp: {
                  cname: 'mediasoup-client',
                  reducedSize: true,
                  mux: true,
                },
                encodings: [{ ssrc: 1001 }],
            };

            // Create the producer
            const pipeProducer = await transport.produce({
                kind: 'video',
                rtpParameters: rtpParameters,
                appData: { type: 'pipe' }
            });

            const transportOptions = {
                id: transport.id,
                iceParameters: transport.iceParameters,
                iceCandidates: transport.iceCandidates,
                dtlsParameters: transport.dtlsParameters,
            };

            const room: Room = {
                producer: pipeProducer,
                transports: new Map().set(transport.id, transport),
                router: router,
                streamId: stream.streaming_id,
            };
    
            this.rooms.set(stream.streaming_id, room);

            /**
             * We return transport to the front end, transport will be used to send the media stream from
             * the front end to the back end using mediasoup-client library 
             */
            return {
                producerId: pipeProducer.id,
                transportOptions: transportOptions,
                rtpCapabilities: router.rtpCapabilities
            }
        } catch (Exception) {
            console.log("createPipeProducer Exception");
            console.log(Exception);
        }
    }

Hope this can be help with more context.
Does i miss something ?

ps: maybe my console.log in my promise is never trigger because he try to connect to the sfu server and he can’t ?
Btw i have all my back end containerized with docker.
The front end is not, so not in the same network as the back end server, if this information can help

This means the ‘connect’ event is never triggered (but this.sendTransport.on is ok, otherwise an exception would have been thrown). However it should be triggered when transport.produce is called for the first time, unless something is wrong with the arguments of that call, in which case there again would have been an exception.

if i don’t add produce event, connect event is never triggered ?
If i only do this in an await Promise it’s never triggered, otherwise i got the undefined error.

Can the problem come from the fact i am not in the same network since i have used docker for my back end and not my front end where i try to trigger the connect event ?

Or can it come from my browser ?
But problem shouldn’t come from there…

This case is checked. If transport.produce() is called without a ‘produce’ event listener set, it throws an exception (‘no “produce” listener set into this transport’).

await new Promise((resolve) => { this.sendTransport.on(‘connect’, ... calls the function passed to the promise constructor immediately, so it the same as direct this.sendTransport.on(‘connect’, ..., except for the inner resolve(). I.e. there should not be undefined error if you simply call this.sendTransport.on(‘connect’... at this point.

yes so what i can i do at this point ?
Can the error come from the back end ?

I can only suggest to test one thing at a time. If ‘undefined’ errors occur, and the expected events are not received, all these dockers, different networks, etc. just add more confusion. Why not just make sure that the principal procedure works in the simplest environment, i.e. with the server and the client both on the localhost?