How to add media stream to a producer

I can show you consumer code too, i wanted to go step by step to be sure if i didn’t forgot anything.
For producer i have inbound-rtp and outbount-rtp traffic.

I just shared with you the producer side.
In server side i have for the producer:

ICE state changed to connected
{
id: '11990f5a-c8ef-4b77-90e4-1ecb99f27d38',
clientId: '-gkGLcsPRPycWptzAAAZ',
streamId: '654fbc0e-d8ec-4992-b27c-efdaec83f8fc'
}
ICE state changed to completed

If it seems ok for you, i can share you more code or consumer side.

Ok go ahead.

async getStreamSFUServer() {
    this.streamingService.returnConsumer().subscribe(async (consumerResponse: any) => {
      const device = new Device();
      const consumer = JSON.parse(consumerResponse);

      const routerRtpCapabilities = consumer.rtpCapabilities;
      await device.load({routerRtpCapabilities});

      // consumer.transportOptions.iceServers = iceServers.iceServers;
      const receiveTransport = await device.createRecvTransport(consumer.transportOptions);
      console.log("consumer.producer_id 1");
      console.log(consumer.producer_id);
      console.log("receiveTransport 1");
      console.log(receiveTransport);

This is the console.log(receiveTransport):


      receiveTransport.on('connectionstatechange', (state) => {
        console.log("connectionstatechange 1");
        console.log(state);
        if (state === 'failed' || state === 'closed') {
          // Handle transport failure
          console.log("connectionstatechange failure 1");
        }
      });

      // receiveTransport.on('transportclose', () => {
      //   // handle the "connect" event
      // });

      // receiveTransport.on('trackended', () => {
      //   // handle the "connect" event
      // });

      // receiveTransport.on('connect', () => {
      //   // handle the "connect" event
      // });

      // receiveTransport.on('connect', () => {
      //   // handle the "connect" event
      // });

      // receiveTransport.on('connect', () => {
      //   // handle the "connect" event
      // });

      await new Promise<void>(async (resolve) => {
        try {
          receiveTransport.on('connect', ({ dtlsParameters }, callback) => {
            console.log('dtlsParameters receiver');
            console.log(dtlsParameters);

            callback();
          });
          
          console.log('before resolve Consume');
          resolve();
        } catch (Exception) {
          console.log('Exception Mediasoup Consume');
          console.log(Exception);
        }
      });

      console.log('consumer receiver');
      console.log(consumer);

      this.streamingService.consumerConsume({transportId: consumer.producer_id, });

      this.streamingService.returnConsumerConsume().subscribe(async (consumer: any) => {
        console.log('consumer receiver subscribe');
        console.log(consumer);
        const consumerTransport = await receiveTransport.consume(({
          id: consumer.id,
          producerId: consumer.producerId,
          rtpParameters: consumer.rtpParameters,
          kind: consumer.kind
        }));

        console.log('consumerTransport receiver');
        console.log(consumerTransport);


        const mediaStream = new MediaStream();
        console.log("consumer.track 1 22 43333");
        console.log(consumerTransport.track);

        consumerTransport.track.enabled = true;
        
 
        consumerTransport.on('@getstats', (getstats) => {
          console.log("getstats on event");
          console.log(getstats);
        });

        const rtpReceiver = consumerTransport.rtpReceiver;
        console.log("rtpReceiver");
        console.log(rtpReceiver);
        const statsReport = await rtpReceiver.getStats();

        statsReport.forEach((report) => {

Never print any console.log().

          if (report.type === 'inbound-rtp') {
            console.log(`Bytes received: ${report.bytesReceived}`);
            console.log(`Packets received: ${report.packetsReceived}`);
            console.log(`Packets lost: ${report.packetsLost}`);
          }
        });
      
        console.log("this.streamVideo");
        console.log(this.streamVideo);

        const supportedCodecs = [
          'video/webm; codecs="vp8, opus"',
          'video/mp4; codecs="avc1.4d401f, mp4a.40.2"',
          'video/ogg; codecs="theora, vorbis"'
        ];

        if (MediaSource.isTypeSupported(supportedCodecs[0])) {
          console.log('vp8 codec is supported');
        } else {
          console.log('vp8 codec is not supported');
        }

        mediaStream.getTracks().forEach((track) => {
          if (track.kind === "video") {
            console.log("MediaStream contains video track.");
            mediaStream.addTrack(track);
          }
          if (track.kind === "audio") {
            console.log("MediaStream contains audio track.");
            mediaStream.addTrack(track);
          }
        });

It return well: vp8 codec is supported


        this.streamVideo.nativeElement.srcObject = mediaStream;
        this.streamVideo.nativeElement.autoplay = true;
        this.streamVideo.nativeElement.muted = false; // This doesn't work

muted is true, this.streamVideo.nativeElement.muted = false; doesn’t work

        this.streamVideo.nativeElement.play();
      });
    });
  }

Back end for consumer:

How i create a consumer:

async createConsumer(stream: any) {
        try {

            const transport = await this.createTransportForRouter(this.stream.get(stream.indexWorker).router, this.stream.get(stream.indexWorker).server);

            // Get the supported RTP capabilities
            const rtpCapabilities: mediasoup.types.RtpCapabilities = this.stream.get(stream.indexWorker).router.rtpCapabilities;

            const producterId = stream.producer_id;
            console.log("producterId hhhhhhhhhhhhhhhhhhhhhhhh");
            console.log(producterId);
            const canConsume = await this.stream.get(stream.indexWorker).router.canConsume({producerId: stream.producer_id, rtpCapabilities: rtpCapabilities});
            console.log("canConsume");
            console.log(canConsume);

can consumer return true

            console.log("transport.id consumer");
            console.log(transport.id);
            this.transports.set(transport.id, transport);

            const transportOptions = {
                id: transport.id,
                iceParameters: transport.iceParameters,
                iceCandidates: transport.iceCandidates,
                dtlsParameters: transport.dtlsParameters,
            };

            return {
                producerId: stream.producer_id,
                type: "consume",
                rtpCapabilities: rtpCapabilities,
                transport: transport,
                transportOptions: transportOptions,
            };
        } catch (Exception) {
            console.log("createConsumer Exception");
            console.log(Exception);
        }
    }

When transport connect is triggered:

async consumerTransportConnect(transportConnect) {
        try {
            await this.transports.get(transportConnect.transportId).connect({dtlsParameters: transportConnect.dtlsParameters});
        } catch (Exception) {
            console.log("Exception");
            console.log(Exception);
        }
    }
async consumerConsume(transportConnect) {
        try {
            console.log('consumerConsume');
            console.log('transportConnect.transportOptions.id');
            console.log(transportConnect.transportOptions.id);
            const consumerTransport = await this.transports.get(transportConnect.transportOptions.id);

            // transport can now consume and return a consumer
            const consumer = await consumerTransport.consume({
                producerId: this.rooms.get(transportConnect.streaming_id).producer.id,
                rtpCapabilities: transportConnect.rtpCapabilities,
                paused: true,
            });

            consumer.on('transportclose', () => {
                console.log('transport close from consumer');
            });

            consumer.on('producerclose', () => {
                console.log('producer of consumer closed');
            });

            this.consumers.set(consumer.id, consumer);

            // to send back to the Client
            const params = {
                id: consumer.id,
                producerId: this.rooms.get(transportConnect.streaming_id).producer.id,
                kind: consumer.kind,
                rtpParameters: consumer.rtpParameters,

                userId: transportConnect.user_vieawer_id,
                clientId: transportConnect.clientId
            }

            this.publisher.publish('returnConsumerConsume', JSON.stringify(params));
        } catch (Exception) {
            console.log("Exception");
            console.log(Exception);
        }
    }

Sorry for answering late, i try to provide the most clean explanation with most informations

You have created the consumer in paused state, and you are not resuming it, this leads to track being muted. So resume the consumer properly or do not created it in paused state.

I tried paused to false and true in back end, same result.
And to change it to false i have to do consumer.resume ? I already did it.

If you are creating consumer with paused true then you need to do consumer.resume() otherwise no need of consumer.resume(). Rest looks all good to me

Yes consumer.resume should be in back end, i can add it again in consumerConsume function.
But that can’t be the reasonn why i have no rtp traffic ?

This is the reason if consumer is created with paused true. Not otherwise.
All looks good to me, the only way to check it further is that you share some working thing so that I can run on my system and see what is causing the issue.

No wonder. Why are you calling canConsume with router’s rtpCapabilities? There should be client device’s receive capabilities instead.

ok i will change that, that’s right.

yes i test and tell you.
Sharing code would complicated but i could try

I have run many demo, but they are not well made, so event with the docker version, when there is one, i have issue.
I find one porject who seems to work but not clear.

I already did consumer resume.
But i am thinking about something, in my consumer side i have to click twice for the consumer to be connected and once it is connected, connect event appears after consume.

canConsume doesn’t work on Consumer object

Everything is connected with the server, Producer side and Consumer side.
Producer side have inbound-rtp and outbound-rtp traffic.
But Consumer side doesn’t have inbound-rtp and outbound-rtp traffic…

canConsume is a router method, it will not work on consumer object.

that’s exactly waht i said yes, so i had right when i said that and when i used canConsume on router.
I tried canConsume two times, both times returned me true, so that’s ok.

I think i have an idea i will exploit is:
The connection is consumer side is in status “connected” after the consumer consume.
I have the idea since a moment that the status need to be connected before the consumer consume.
What do you think ?

Producer’s consumeable rtpParameters is a subset of the router’s rtpCapabilities. So if you call canConsume with the router’s rtpCapabilities, the result is predefined. It just makes no sense.