ok, what does mediasoup NodeJS server say?
Let me repeat my question. You said:
Everything perfect, mediasoup NodeJS server also saying that
So I ask you: what does your mediasoup NodeJS server say? āI got answer same as offerā is not a valid response, in fact it means absolutely nothing since mediasoup does not produce/consume SDPs.
Looks like flutter canāt resolve offer and answer from server for video
Ok, you donāt answer direct and specific questions. Donāt expect more answers from me.
Here is coming video offer from webrtc demos
v=0
o=- 5506019393115635759 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS 5ecd04c7-17a4-4441-9255-5cf5eefaeb78
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:gWxl
a=ice-pwd:Mf7xxeZXwKI1VJKedLSmCfdg
a=ice-options:trickle renomination
a=fingerprint:sha-256 52:E2:F0:E3:63:FA:68:8B:78:8E:F1:E0:9F:13:70:37:F9:3C:41:6D:5A:68:8B:57:2E:A1:91:32:E2:22:13:43
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:102 ILBC/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:2583589951 cname:ncGWwvbSa7zYrtJZ
a=ssrc:2583589951 msid:5ecd04c7-17a4-4441-9255-5cf5eefaeb78 975df2f5-341b-4802-9835-24ade367de46
a=ssrc:2583589951 mslabel:5ecd04c7-17a4-4441-9255-5cf5eefaeb78
a=ssrc:2583589951 label:975df2f5-341b-4802-9835-24ade367de46
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 124 125
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:gWxl
a=ice-pwd:Mf7xxeZXwKI1VJKedLSmCfdg
a=ice-options:trickle renomination
a=fingerprint:sha-256 52:E2:F0:E3:63:FA:68:8B:78:8E:F1:E0:9F:13:70:37:F9:3C:41:6D:5A:68:8B:57:2E:A1:91:32:E2:22:13:43
a=setup:actpass
a=mid:video
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 H264/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:127 red/90000
a=rtpmap:124 rtx/90000
a=fmtp:124 apt=127
a=rtpmap:125 ulpfec/90000
a=ssrc-group:FID 1560641331 3490930342
a=ssrc:1560641331 cname:ncGWwvbSa7zYrtJZ
a=ssrc:1560641331 msid:5ecd04c7-17a4-4441-9255-5cf5eefaeb78 6ee272f4-8882-409d-b758-450b3d041056
a=ssrc:1560641331 mslabel:5ecd04c7-17a4-4441-9255-5cf5eefaeb78
a=ssrc:1560641331 label:6ee272f4-8882-409d-b758-450b3d041056
a=ssrc:3490930342 cname:ncGWwvbSa7zYrtJZ
a=ssrc:3490930342 msid:5ecd04c7-17a4-4441-9255-5cf5eefaeb78 6ee272f4-8882-409d-b758-450b3d041056
a=ssrc:3490930342 mslabel:5ecd04c7-17a4-4441-9255-5cf5eefaeb78
a=ssrc:3490930342 label:6ee272f4-8882-409d-b758-450b3d041056
here is answer
v=0
o=- 3904721005637389161 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS 54752986-b410-472f-999e-5da5f287d621
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 9 102 0 8 105 13 110 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:s3Bu
a=ice-pwd:j81V9taPlwf5wNJQ7xAWeOvE
a=ice-options:trickle renomination
a=fingerprint:sha-256 BB:7C:E2:B4:6B:E4:03:7C:66:5C:A8:8A:1F:97:7F:DA:C2:42:78:4E:EB:89:DF:06:3F:F6:CC:43:2E:32:8F:5C
a=setup:active
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:9 G722/8000
a=rtpmap:102 ILBC/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:2417119888 cname:3VtSTGpu9smHV320
a=ssrc:2417119888 msid:54752986-b410-472f-999e-5da5f287d621 4faa3928-09ee-480b-bbc9-b2769e355093
a=ssrc:2417119888 mslabel:54752986-b410-472f-999e-5da5f287d621
a=ssrc:2417119888 label:4faa3928-09ee-480b-bbc9-b2769e355093
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 127 124 125
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:s3Bu
a=ice-pwd:j81V9taPlwf5wNJQ7xAWeOvE
a=ice-options:trickle renomination
a=fingerprint:sha-256 BB:7C:E2:B4:6B:E4:03:7C:66:5C:A8:8A:1F:97:7F:DA:C2:42:78:4E:EB:89:DF:06:3F:F6:CC:43:2E:32:8F:5C
a=setup:active
a=mid:video
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:127 red/90000
a=rtpmap:124 rtx/90000
a=fmtp:124 apt=127
a=rtpmap:125 ulpfec/90000
a=ssrc-group:FID 3498875224 2800042824
a=ssrc:3498875224 cname:3VtSTGpu9smHV320
a=ssrc:3498875224 msid:54752986-b410-472f-999e-5da5f287d621 cb7aff57-7ed7-47e9-8a8e-2079f6f10fce
a=ssrc:3498875224 mslabel:54752986-b410-472f-999e-5da5f287d621
a=ssrc:3498875224 label:cb7aff57-7ed7-47e9-8a8e-2079f6f10fce
a=ssrc:2800042824 cname:3VtSTGpu9smHV320
a=ssrc:2800042824 msid:54752986-b410-472f-999e-5da5f287d621 cb7aff57-7ed7-47e9-8a8e-2079f6f10fce
a=ssrc:2800042824 mslabel:54752986-b410-472f-999e-5da5f287d621
a=ssrc:2800042824 label:cb7aff57-7ed7-47e9-8a8e-2079f6f10fce
Lovely
Hi! Can you contact me via skype iluha90? Iāve got a commercial task related to current subject for you.
Hi, first of all, thank you so much for this project! You rock!
Second, is there any advantage in using the flutter-webrtc package that you specify in the pubspec.yaml?
The latest version of flutter_webrtc 3.X has added a host of improvements it seems.
Hi,
The reason I made this fork is that I had to quickly fix some bugs and add some additional functionality.
I did post pull requests on all the changes that I made.
You are welcome to check if the latest version works as expected, if it does let me know and Iāll update the codebase to point to the latest flutter_webrtc.
Cheers
Hi,
So I used the latest flutter-webrtc code, and it seems they do not have rtc_rtp_sender.dart and receiver files.
Also I get this error from RTCPeerConnection:
Error: The setter āonAddTrack2ā isnāt defined for the class āRTCPeerConnectionā.
Rest everything seems fine after 2 changes:
webrtc.dart is now flutter_webrtc.dart
&
navigator.getUserMedia is now MediaDevices.getUserMedia
Do you have any thoughts as to what I can do here? Iām new to dart and am afraid to tamper with the flutter installation directoryā¦
Hi Yonatan,
If itās not too much to ask, can you please send your socket connection backend? Iām trying to use NodeJS with socket.io and for a web app, I used mediasoup in a certain way. However, your request response structure is a little confusing primarily because Iām new to dart. Can you please atleast show me how your server side socket requests are handled so that the link becomes clear to me?
Hi Servo97,
I am using the Protoo package (https://www.npmjs.com/package/protoo) it was written by IƱaki Baz Castillo who authored mediasoup.
I find it very well written and very convenient to use Iāve used it in several projects.
You can checkout the mediasoup demo server implementation - Iāve based a lot of my backend on these concepts:
Hi Yonatan, I have almost solved the entire communication puzzle. 1 last thing I canāt figure out is how are you setting the mediastream which you receive from other producers on the current consumer?
connectToSocket() {
if(null==_socket){
print("Socket is null");
return;
}
_socket.on("newProducers", (data){
print("NEW PRODUCERS!");
print(data);
for(var producerId in data){
print(producerId['producer_id']);
getConsumeStream(producerId).then((data){
});
}
});
_socket.connect();
}
getConsumeStream(producerId) async{
var rtpCapabilities = this.device.rtpCapabilities;
var data = await _sendMessageWithACK('consume', {
"rtpCapabilities":rtpCapabilities,
"consumerTransportId": _recvTransport.id, // might be
"producerId":producerId['producer_id']
});
var id, kind, rtpParameters = data;
_recvTransport.consume(id:id, kind: kind, rtpParameters: rtpParameters);
// var stream = new MediaStream();
// stream.addTrack(consumer.track);
// return {"consumer":consumer,"stream":stream,"kind":kind,};
}
In the javascript implementation, the last 3 lines give me the stream from the consumer object. However, I canāt understand in where in your code do you get the remote stream from? This is the last thing that I need to figure out.
I can see that when you add a stream to your peer connection, you call onAddRemoteStream. But when I call _recvTransport.consume(), it is not getting called. FYI, Iām not implementing your _answer method (is that causing some issue?).
Have a look in the media soup demo project, the relevant code is in Room.js line 615
You have to pass in the produce ID that is generated by the server, I am not sure what you are doing in the client with āconsumeā message.
I suggest that you start from the working demo code and make your modifications on top of that.
Hi, Iām sorry I didnāt update. The problem is solved. The error was that the data object actually does not separate like this in dart:
var id, kind, rtpParameters = data;
So I had to call data[āidā], blah, blah to send it to the _recvTransport.consume function. Sorry for my noobness!
Also, Now Iām dealing with some weird error regarding cameras:
android.hardware.camera2.CameraAccessException: CAMERA_ERROR (3): cancelRequest:456: Camera 0: Error clearing streaming request: Function not implemented (-38)
Many people faced similar issues in pure android development environments. I donāt know if this problem is carried over to flutter from somewhere.
Anyhoo, Iām really grateful to you for making this amazing port! You seriously rock!
Hi Inaki, can you help me decide the webrtc transport options?
My mediasoup server is running on NodeJS with Socket.io on my localhost. I use Nginx to reverse proxy my domain name to this localhost. In the config.js file, my configuration is as follows:
// WebRtcTransport settings
webRtcTransport: {
listenIps: [
{
ip: '0.0.0.0',
announcedIp:'127.0.0.1' // replace by public IP address
}
],
maxIncomingBitrate: 1500000,
initialAvailableOutgoingBitrate: 1000000
},
I am a little confused here as to what do these fields need to be. Shall I put my public IP here or keep it localhost?
TLDR: My NodeJS server is running on Localhost. So the outside communication is coming through Nginx. What are the appropriate values that I should put in these fields.
No direct mentions, please. Iām not the only one in this forum.
Regarding your questions check the mediasoup API documentation and FAQ. I cannot explain these things again and again, we wrote docs for a reason.
I have obviously read the documentation, and I did not find my answer there, hence asking here.
This is the part of the documentation:https://mediasoup.org/documentation/v3/mediasoup/api/#Router-methods
Nowhere does it mention how to setup config.js for reverse proxy kind of scenarios. Iām sorry for bothering you, but I wouldnāt have asked if I found my answer somewhere on the internet / in the documentations.
There is no āconfig.jsā in mediasoup and your question is not related to the Router class at all. Check the mediasoup FAQ in the website.