That’s just part of the demo, and I don’t know how you are trying to replicate that in your app.
Just learning 
and it’s more clear every day!
I get it, I will have to check the device.rtpCapabilities.codecs to see what type of encodings to send to the server, force VP9, etc is something to configure only if available on the device.
Just look at how this is done in the demo. The codec parameter in produce() may be a specific codec in device.rtpParameters.codecs (if matching codec is found) or undefined otherwise, which means the first available codec in the client (respecting the codec order in the Router) will be chosen.
Awesome! I will change the code to match that logic 
@ibc I am a bit confused about the rtpCapabilities in mediasoup Device and the rtpCapabilities sent to the server passed in transport?.on('produce', async ({ kind, rtpParameters, appData }, callback, errback) => {...} callback, they don’t match and so while I find in device.rtpCapabilities.codecs desired VP9 or H264 codecs server producer doesn’t have any of these when created.
The only time I send the device RTP capabilities is when a peer joins the room and set the capabilities to the peer.data.rtpCapabilities.
Here are the details and data.
Server Side:
device RTP capabilities sent from the client on join.
mediaserver:INFO:./Logger join - peer.data.rtpCapabilities {
codecs: [
{
mimeType: 'audio/opus',
kind: 'audio',
preferredPayloadType: 100,
clockRate: 48000,
channels: 2,
parameters: [Object],
rtcpFeedback: [Array]
},
{
mimeType: 'video/VP8',
kind: 'video',
preferredPayloadType: 101,
clockRate: 90000,
parameters: {},
rtcpFeedback: [Array]
},
{
mimeType: 'video/rtx',
kind: 'video',
preferredPayloadType: 102,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: []
},
{
mimeType: 'video/VP9',
kind: 'video',
preferredPayloadType: 103,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: [Array]
},
{
mimeType: 'video/rtx',
kind: 'video',
preferredPayloadType: 104,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: []
},
{
mimeType: 'video/H264',
kind: 'video',
preferredPayloadType: 105,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: [Array]
},
{
mimeType: 'video/rtx',
kind: 'video',
preferredPayloadType: 106,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: []
},
{
mimeType: 'video/H264',
kind: 'video',
preferredPayloadType: 107,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: [Array]
},
{
mimeType: 'video/rtx',
kind: 'video',
preferredPayloadType: 108,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: []
}
],
headerExtensions: [
{
kind: 'audio',
uri: 'urn:ietf:params:rtp-hdrext:sdes:mid',
preferredId: 1,
preferredEncrypt: false,
direction: 'sendrecv'
},
{
kind: 'video',
uri: 'urn:ietf:params:rtp-hdrext:sdes:mid',
preferredId: 1,
preferredEncrypt: false,
direction: 'sendrecv'
},
{
kind: 'audio',
uri: 'http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time',
preferredId: 4,
preferredEncrypt: false,
direction: 'sendrecv'
},
{
kind: 'video',
uri: 'http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time',
preferredId: 4,
preferredEncrypt: false,
direction: 'sendrecv'
},
{
kind: 'video',
uri: 'http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01',
preferredId: 5,
preferredEncrypt: false,
direction: 'sendrecv'
},
{
kind: 'video',
uri: 'http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07',
preferredId: 6,
preferredEncrypt: false,
direction: 'sendrecv'
},
{
kind: 'audio',
uri: 'urn:ietf:params:rtp-hdrext:ssrc-audio-level',
preferredId: 10,
preferredEncrypt: false,
direction: 'sendrecv'
},
{
kind: 'video',
uri: 'urn:3gpp:video-orientation',
preferredId: 11,
preferredEncrypt: false,
direction: 'sendrecv'
},
{
kind: 'video',
uri: 'urn:ietf:params:rtp-hdrext:toffset',
preferredId: 12,
preferredEncrypt: false,
direction: 'sendrecv'
}
]
}
Request ‘produce’ sent from the client when local client audio transport.produce invoked to create a server producer.
mediaserver:INFO:./Logger on produce request rtpParameters {
codecs: [
{
mimeType: 'audio/opus',
payloadType: 111,
clockRate: 48000,
channels: 2,
parameters: [Object],
rtcpFeedback: [Array]
}
],
headerExtensions: [
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:mid',
id: 4,
encrypt: false,
parameters: {}
},
{
uri: 'http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time',
id: 2,
encrypt: false,
parameters: {}
},
{
uri: 'http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01',
id: 3,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:ssrc-audio-level',
id: 1,
encrypt: false,
parameters: {}
}
],
encodings: [ { ssrc: 4185228809, dtx: false } ],
rtcp: { cname: 'RRIzxbR+TuPeuktn', reducedSize: true },
mid: '0'
}
Request 'produce' sent from the client when local client video transport.produce invoked to create a server producer.
This is where I don’t get why transport.on('produce'...) RTP parameters are not the same as client RTP capabilities.
I guess I can send the device RTP capabilities (that will error, I tested it) but I’d like to learn why the delegate has different RTP parameters; Is it a server router initialization;
mediaserver:INFO:./Logger on produce request rtpParameters {
codecs: [
{
mimeType: 'video/VP8',
payloadType: 96,
clockRate: 90000,
parameters: {},
rtcpFeedback: [Array]
},
{
mimeType: 'video/rtx',
payloadType: 97,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: []
}
],
headerExtensions: [
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:mid',
id: 4,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id',
id: 5,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id',
id: 6,
encrypt: false,
parameters: {}
},
{
uri: 'http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time',
id: 2,
encrypt: false,
parameters: {}
},
{
uri: 'http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01',
id: 3,
encrypt: false,
parameters: {}
},
{
uri: 'http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07',
id: 8,
encrypt: false,
parameters: {}
},
{
uri: 'urn:3gpp:video-orientation',
id: 13,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:toffset',
id: 14,
encrypt: false,
parameters: {}
}
],
encodings: [
{
ssrc: 1297492880,
rtx: [Object],
active: true,
scaleResolutionDownBy: 4,
maxBitrate: 500000,
scalabilityMode: 'S1T3',
dtx: false
},
{
ssrc: 1297492881,
rtx: [Object],
active: true,
scaleResolutionDownBy: 2,
maxBitrate: 1000000,
scalabilityMode: 'S1T3',
dtx: false
},
{
ssrc: 1297492882,
rtx: [Object],
active: true,
scaleResolutionDownBy: 1,
maxBitrate: 5000000,
scalabilityMode: 'S1T3',
dtx: false
}
],
rtcp: { cname: 'RRIzxbR+TuPeuktn', reducedSize: true },
mid: '1'
}
Please, let’s avoid direct naming. I’m not the only one supposed to reply in this forum.
You mean rtpParameters rather than rtpCapabilities. Those are different things.
So VP8 is before VP9 (due the order of codecs you passed to your mediasoup Router) so VP8 will be chosen by your client unless you force VP9 or H264 via API. And I am not aware that you are forcing VP9 or H264 at all so it’s perfectly fine that it chooses VP8 (and expected).
Use the codec parameter in ProducerOptions in mediasoup-client: mediasoup :: API
Check the mediasoup-demo to see a usage example.
Sounds good.
Yes! I edited the response
What API is that to force a codec?
I tried that but throws an error since RTP parameters do not include VP9 or H264 which I believe you mention forcing via API in previous statement.
Well, that’s the API to force a codec instead of using the first available one (VP8). If you don’t use it you cannot force a codec.
I tried that but throws an error since RTP parameters do not include VP9 or H264
I don’t understand. You are calling transport.produce(), you don’t have any rtpParameters yet. rtpParameters will be generated after you call transport.produce() and you’ll get them in the transport.on('produce') event.
Please, print the full object you are passing into the codec option in transport.produce().
OK, I get it how you dictate it via the codec options.
Passing H264 codec and below encodings server will not error but I don’t get the stream on the other side, this is the initial issue I had (sounds like transport H264 doesn’t work nicely with my client;)
mediaserver:INFO:./Logger produce rtpParameters
{
codecs: [
{
mimeType: 'video/H264',
payloadType: 124,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: [Array]
},
{
mimeType: 'video/rtx',
payloadType: 120,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: []
}
],
headerExtensions: [
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:mid',
id: 4,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id',
id: 5,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id',
id: 6,
encrypt: false,
parameters: {}
},
{
uri: 'http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time',
id: 2,
encrypt: false,
parameters: {}
},
{
uri: 'http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01',
id: 3,
encrypt: false,
parameters: {}
},
{
uri: 'http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07',
id: 8,
encrypt: false,
parameters: {}
},
{
uri: 'urn:3gpp:video-orientation',
id: 13,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:toffset',
id: 14,
encrypt: false,
parameters: {}
}
],
encodings: [
{
ssrc: 2239202200,
rtx: [Object],
active: true,
scaleResolutionDownBy: 4,
maxBitrate: 500000,
scalabilityMode: 'S1T3',
dtx: false
},
{
ssrc: 2239202201,
rtx: [Object],
active: true,
scaleResolutionDownBy: 2,
maxBitrate: 1000000,
scalabilityMode: 'S1T3',
dtx: false
},
{
ssrc: 2239202202,
rtx: [Object],
active: true,
scaleResolutionDownBy: 1,
maxBitrate: 5000000,
scalabilityMode: 'S1T3',
dtx: false
}
],
rtcp: { cname: 'pqdo0WPNhfBRb02t', reducedSize: true },
mid: '1'
}
When forcing VP9 now worked! (not sure where the bug was before I couldn’t make it but it looks fine now) with the below RTP parameters passed to the server.
And stream comes across to the consumer side as opposed with the H264 case.
mediaserver:INFO:./Logger produce rtpParameters
{
codecs: [
{
mimeType: 'video/VP9',
payloadType: 100,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: [Array]
},
{
mimeType: 'video/rtx',
payloadType: 101,
clockRate: 90000,
parameters: [Object],
rtcpFeedback: []
}
],
headerExtensions: [
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:mid',
id: 4,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id',
id: 5,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id',
id: 6,
encrypt: false,
parameters: {}
},
{
uri: 'http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time',
id: 2,
encrypt: false,
parameters: {}
},
{
uri: 'http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01',
id: 3,
encrypt: false,
parameters: {}
},
{
uri: 'http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07',
id: 8,
encrypt: false,
parameters: {}
},
{
uri: 'urn:3gpp:video-orientation',
id: 13,
encrypt: false,
parameters: {}
},
{
uri: 'urn:ietf:params:rtp-hdrext:toffset',
id: 14,
encrypt: false,
parameters: {}
}
],
encodings: [
{
ssrc: 1656228626,
rtx: [Object],
active: true,
scalabilityMode: 'S3T3_KEY',
dtx: false
}
],
rtcp: { cname: 'zoE2/Oj3wG7Kw673', reducedSize: true },
mid: '1'
}
I will open a separate thread for the H264 since this thread is getting bigger and for reference for others;
I have the following questions if you’re kind to answer, very helpful so far, and making better decisions on how to build the actual app!
I am using a flag as found in your demo, useSimulcast to decide encodings with the code below.
Sending undefined encodings still works producing and consuming video/audio, how this affects the results; quality;
if (this.useSimulcast) {
if (vp9 && this.forceVP9) {
encodings = WEBCAM_KSVC_ENCODINGS
} else {
encodings = WEBCAM_SIMULCAST_ENCODINGS
}
}
This might be more related DevOps question, when/why should I enable TCP for webRtcTransportOptions in the server?
if (forceTcp) {
webRtcTransportOptions.enableUdp = false
webRtcTransportOptions.enableTcp = true
}
Amazing work! Thanks for all your help.