Ok so it this is the code and the video tag where is the play() called:
const { track } = consumer
const remoteMediaStream = new MediaStream([track])
console.log(remoteMediaStream.getTracks().length > 0)
remoteVideoRef.current.srcObject = remoteMediaStream
setRemoteVideoObject(remoteMediaStream)
webrtc.emit('resume', ({ roomId }))
//Video element
<video id='fan-video' ref={remoteVideoRef} /*srcobject={remoteVideoObject}*/ autoPlay playsInline className='video-player'></video>
you need to do remoteVideoRef.current.play() after above line.
Does this affect the fact that in the line below the consumer will be resumed?
The play() is unfortunately not working i tried it an refreshed it. still not displaying anything but I logged the ref and it shows the mediastream and active is true…It wont let me reply for the rest of the day as I have reached my limit.
no, it doesn’t matter
Just in case: make sure that autoplay policy requirements are satisfied.
this is especially important if you are on mac or iphone.
Sorry for the late response the site has limited my ability to send messages due to my new account, but I’ve tried everything and its still not working. I added another producer and consumer for audio and it’s successfully sending and receiving both tracks but it’s not displaying it properly on the other side for the viewer.
There is problem the way you display the video, make sure you are not facing autoplay related issues. Give id ‘videoid’ to the video element and go to console and see the output of the following code:
$(‘#videoid’)[0].srcObject;
$(‘#videoid’)[0].srcObject.getActiveTracks();
Or their equivalent in vanila js
This is the last sequence of code, all of the logs are returning correctly including the jquery log:
await webrtc.emit('consume', {
rtpCapabilities: device.rtpCapabilities,
roomId
}, async ({ params }) => {
if(params.error){
console.log("cannot consume")
return
}
console.log(params)
videoConsumer = await consumerTransport.consume({
id: params.videoId,
producerId: params.videoProducerId,
kind: 'video',
rtpParameters: params.videoRtpParameters
})
audioConsumer = await consumerTransport.consume({
id: params.audioId,
producerId: params.audioProducerId,
kind: 'audio',
rtpParameters: params.audioRtpParameters
})
const videoTrack = videoConsumer.track
const audioTrack = audioConsumer.track
console.log("Received consumers: ", videoConsumer, audioConsumer)
console.log("Tracks: ", videoTrack, audioTrack)
const remoteMediaStream = new MediaStream()
remoteMediaStream.addTrack(videoTrack)
remoteMediaStream.addTrack(audioTrack)
remoteVideoRef.current.srcObject = remoteMediaStream
// remoteVideoRef.current.play()
console.log("Remote reference: ", remoteVideoRef.current)
console.log("Remote video: ", remoteVideoRef.current.srcObject)
// setRemoteVideoObject(remoteMediaStream)
remoteVideoRef.current.onloadedmetadata = () => {
console.log('Video metadata loaded')
remoteVideoRef.current.play()
console.log("Jquery Data: ", $('#model-video')[0].srcObject, $('#model-video')[0].srcObject.getTracks())
}
remoteVideoRef.current.onerror = (error) => {
console.log('Video error:', error)
}
webrtc.emit('resume', ({ roomId }))
This is what is returned in the console but nothing is showing in the ui, it did appear once with no audio but when i refresh it goes away and didn’t return:
Jquery Data:
MediaStream {id: 'a270db7a-e544-449f-8cef-b737339d7bff', active: true, onaddtrack: null, onremovetrack: null, onactive: null, …}
(2) [MediaStreamTrack, MediaStreamTrack]
0
:
MediaStreamTrack {kind: 'audio', id: '0b6a7784-00b8-4fc7-b0cc-cc9ba92c41cd', label: '0b6a7784-00b8-4fc7-b0cc-cc9ba92c41cd', enabled: true, muted: false, …}
1
:
MediaStreamTrack {kind: 'video', id: 'c43972c0-b3a9-4721-9e63-1235229e943c', label: 'c43972c0-b3a9-4721-9e63-1235229e943c', enabled: true, muted: false, …}
length
:
2
This play() have not effect either? Do you see any errors on console? Enable controls of video element and play from there manually. Show us the screenshot of your page.
I got it to work by setting the video element to muted, you were correct it was an autoplay issue. Thank you for all of your help I really appreciate it.
Update:
It works when i add muted to the video element but it stays muted which is expected, however when I try to unmute it freezes the stream:
remoteVideoRef.current.onloadedmetadata = () => {
console.log('Video metadata loaded')
remoteVideoRef.current.play()
// remoteVideoRef.current.muted = false
console.log("Jquery Data: ", $('#model-video')[0].srcObject, $('#model-video')[0].srcObject.getTracks())
}
This is expected, browser’s autoplay policy will not allow us to trick this way as mentioned here in stackoverflow:
I have a full topic here, regarding audio autoplay:
As per that you have 2 options:
Always do getUserMedia() i.e GUM request before playing the streams, by doing this browser allows playing the stream without any autoplay issues, this will always work
If you don’t want to do GUM then before playing video you must ask user for interaction, show a modal and show some text and a button when user clicks that button play the stream it will work. The only problem is that it will only work for the streams being played right after that button click, and if you have stream like few seconds after that and you want to play then you will again face autoplay issue for this specific stream. So if you have 1 stream and want to avoid autoplay, do this otherwise do the first one GUM that will surely work.
Thanks again, you were a major help.
Glad to help
I have another question, how would we go about the announcedIp section in production, if the frontend is on a different server/domain than the backend and the backend server uses nginx as a reverse proxy? Your help would be greatly appreciated again. I added the server IP as the announcedIp and it doesn’t work in production.
You should use the public ip of server not the private one. Not sure about reverse proxy thing.
Must be the reverse proxy because I’m using the public Ip and it’s not working.
I’ve disabled the firewall for the backend server and it still isnt working, could the problem be something with the fact that my frontend is hosted using netlify?
No issue with the frontend, please make sure the tcp/udp ports which you are providing to mediasoup are open on your server.