H264 OpenH264 / ExternalEncoder in Chrome

Following this thread in the deprecated mailing list, and after some more testing and checks, here some conclusion about H264 encoder in Chrome:

OpenH264 encoder/decoder

mediasoup (v3) Router codec:

{
	kind       : 'video',
	mimeType   : 'video/h264',
	clockRate  : 90000,
	parameters :
	{
		'packetization-mode'      : 1,
		'profile-level-id'        : '42e01f',
		'level-asymmetry-allowed' : 1
	}
}

SDP answer m= section generated by mediasoup-client (v3):

m=video 44759 UDP/TLS/RTP/SAVPF 125 107
c=IN IP4 5.5.5.5
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:udpcandidate 1 udp 1076302079 5.5.5.5 44759 typ host generation 0
a=ice-ufrag:dwm3pvlc52vmyixb
a=ice-pwd:y1zsaw216rozrr2c3mwhjdnivc9ogwjp
a=ice-options:renomination
a=fingerprint:sha-512 15:5A:74:F5:F3:CE:BA:33:87:E9:BC:D1:9D:4F:8A:63:FD:4D:B3:57:C8:F7:0B:9E:5A:74:C3:E3:30:A4:D6:EB:D0:96:3A:C5:FB:5A:CF:68:D8:AD:E4:F0:B6:24:62:3A:96:91:A1:3F:6A:3C:65:3D:63:7F:AB:36:03:80:65:BA
a=setup:active
a=mid:2
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:12 urn:3gpp:video-orientation
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f;x-google-start-bitrate=1000
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=simulcast:recv r0;r1;r2

chrome://webrtc-stats (with “Legazy Non-Standard) getStats() API” enabled):

Statistics ssrc_2812663329_send
* codecImplementationName	OpenH264

There is proper simulcast. This is, Chrome sends 3 H264 streams.

H264 ExternalEncoder (hardware encoder)

mediasoup (v3) Router codec:

{
	kind       : 'video',
	mimeType   : 'video/h264',
	clockRate  : 90000,
	parameters :
	{
		'packetization-mode'      : 1,
		'profile-level-id'        : '42001f',
		'level-asymmetry-allowed' : 1
	}
}

Create a Producer with simulcast in mediasoup-client (v3).

SDP answer generated by mediasoup-client:

m=audio 7 UDP/TLS/RTP/SAVPF 111
c=IN IP4 127.0.0.1
a=rtpmap:111 opus/48000/2
a=fmtp:111 stereo=1;usedtx=1
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=setup:active
a=mid:1
a=recvonly
a=ice-ufrag:5wy7xrdgt27ii6db
a=ice-pwd:qtvse4e5qjofzze4zm0x39rqfcxgo07l
a=candidate:udpcandidate 1 udp 1076302079 5.5.5.5 43975 typ host
a=end-of-candidates
a=ice-options:renomination
a=rtcp-mux
a=rtcp-rsize
m=video 7 UDP/TLS/RTP/SAVPF 102 122
c=IN IP4 127.0.0.1
a=rtpmap:102 H264/90000
a=rtpmap:122 rtx/90000
a=fmtp:102 packetization-mode=1;level-asymmetry-allowed=1;profile-level-id=42001f;x-google-start-bitrate=1000
a=fmtp:122 apt=102
a=rtcp-fb:102 goog-remb 
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack 
a=rtcp-fb:102 nack pli
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:12 urn:3gpp:video-orientation
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=setup:active
a=mid:2
a=recvonly
a=ice-ufrag:5wy7xrdgt27ii6db
a=ice-pwd:qtvse4e5qjofzze4zm0x39rqfcxgo07l
a=candidate:udpcandidate 1 udp 1076302079 5.5.5.5 43975 typ host
a=end-of-candidates
a=ice-options:renomination
a=rtcp-mux
a=rtcp-rsize
a=rid:r0 recv
a=rid:r1 recv
a=rid:r2 recv
a=simulcast:recv r0;r1;r2

chrome://webrtc-stats (with “Legazy Non-Standard) getStats() API” enabled):

Statistics ssrc_1264496030_send
* codecImplementationName	ExternalEncoder

There is NOT simulcast. This is, Chrome just sends a single H264 stream (THIS IS NOT GOOD). Also, the remote Consumer cannot render it, probably due to a well known Chrome H264 issue. However it does not even work if the mediasoup-client Producer does not use simulcast, so here we have a separate issue (already reported in mediasoup issue tracker here).

Conclusion

Chrome enables H264 ExternalEncoder depending on the H264 parameters of the negotiated codec. For example, it uses ExternalEncoder if the remote SDP just enables an H264 codec with 'packetization-mode': 1, 'profile-level-id': '42001f'. However, it does not produce simulcast streams.

Thank you for looking into this in detail. This is amazing. We probably have a special use-case for mediasoup as simulcast is really not important for us at this point. Interesting observation of different behavior is, that using profile 42001f still uses OpenH264 instead of external encoder for us. And that is on machines where HW acceleration is available and works fine with Munge SDP or AppRTC. We do appreciate the time and effort you’re putting into this issue and we’ll be grateful for any other ideas on why this could be the case.

Interesting observation of different behavior is, that using profile 42001f still uses OpenH264 instead of external encoder for us

Could you please share the SDPs (local description and remote description) from chrome://webrtc-internals for such session, as well as the room media codecs you are providing to mediasoup server?

@pavelserbajlo,

Setting this exact codec description to the media codec provided to server.Room() I’m seeing that the H264 ExternalEncoder is being used.

			{
				kind: 'video',
				name: 'H264',
				clockRate: 90000,
				parameters:
				{
					'packetization-mode': 1,
					'profile-level-id': '42001f'
				}
             }

This has been tested in mediasoup-demo (v2) with chrome version 74.0.3729.169.

localDescription sdp:

v=0
o=- 7378254510298434705 3 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=msid-semantic: WMS
m=audio 57871 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 10.0.0.47
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:3588808435 1 udp 2113937151 10.0.0.47 57871 typ host generation 0 network-cost 999
a=candidate:2180346033 1 udp 2113939711 2601:646:c300:fb0f:4cfe:8197:78b4:c03f 57872 typ host generation 0 network-cost 999
a=ice-ufrag:RsEO
a=ice-pwd:rTJ4wakyVlPpkVoq5LzyIpiu
a=ice-options:trickle
a=fingerprint:sha-256 E2:7F:03:4F:37:2C:33:89:BE:BA:83:6F:1C:E1:3C:BE:00:12:E1:B0:0B:93:E9:70:E2:EE:22:7F:DE:84:0B:F0
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:- 6feaeb29-bdc0-4d7b-bb9c-a29f3fc8aeab
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:3605588459 cname:hT/eeeoYFKaN3mMq
a=ssrc:3605588459 msid:- 6feaeb29-bdc0-4d7b-bb9c-a29f3fc8aeab
a=ssrc:3605588459 mslabel:-
a=ssrc:3605588459 label:6feaeb29-bdc0-4d7b-bb9c-a29f3fc8aeab
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 122 127 121 125 107 108 109 124 120 123 119 114 115 116
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:RsEO
a=ice-pwd:rTJ4wakyVlPpkVoq5LzyIpiu
a=ice-options:trickle
a=fingerprint:sha-256 E2:7F:03:4F:37:2C:33:89:BE:BA:83:6F:1C:E1:3C:BE:00:12:E1:B0:0B:93:E9:70:E2:EE:22:7F:DE:84:0B:F0
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:12 urn:3gpp:video-orientation
a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:- ce0eb71b-848b-4c60-ad5b-86d243f6a3fc
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:122 rtx/90000
a=fmtp:122 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:121 rtx/90000
a=fmtp:121 apt=127
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:124 H264/90000
a=rtcp-fb:124 goog-remb
a=rtcp-fb:124 transport-cc
a=rtcp-fb:124 ccm fir
a=rtcp-fb:124 nack
a=rtcp-fb:124 nack pli
a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=124
a=rtpmap:123 H264/90000
a=rtcp-fb:123 goog-remb
a=rtcp-fb:123 transport-cc
a=rtcp-fb:123 ccm fir
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032
a=rtpmap:119 rtx/90000
a=fmtp:119 apt=123
a=rtpmap:114 red/90000
a=rtpmap:115 rtx/90000
a=fmtp:115 apt=114
a=rtpmap:116 ulpfec/90000
a=ssrc-group:FID 1696337601 1237085717
a=ssrc:1696337601 cname:hT/eeeoYFKaN3mMq
a=ssrc:1696337601 msid:- ce0eb71b-848b-4c60-ad5b-86d243f6a3fc
a=ssrc:1696337601 mslabel:-
a=ssrc:1696337601 label:ce0eb71b-848b-4c60-ad5b-86d243f6a3fc
a=ssrc:1237085717 cname:hT/eeeoYFKaN3mMq
a=ssrc:1237085717 msid:- ce0eb71b-848b-4c60-ad5b-86d243f6a3fc
a=ssrc:1237085717 mslabel:-
a=ssrc:1237085717 label:ce0eb71b-848b-4c60-ad5b-86d243f6a3fc

remoteDescription sdp:

v=0
o=- 48917412 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=msid-semantic: WMS
a=ice-lite
m=audio 10006 RTP/SAVPF 111
c=IN IP4 10.0.0.47
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:udpcandidate 1 udp 1080142079 10.0.0.47 10006 typ host generation 0
a=candidate:tcpcandidate 1 tcp 1078862079 10.0.0.47 10087 typ host tcptype passive generation 0
a=ice-ufrag:6f8fnf2cbe41zdry
a=ice-pwd:zwl88or5crhi4igj8tyll1km1oi3djdw
a=ice-options:renomination
a=fingerprint:sha-512 6E:6D:1F:52:27:E1:90:D1:5E:E5:AA:16:4C:F1:AC:82:81:AC:D3:B3:8E:5F:E2:18:C0:42:ED:88:E5:15:F5:49:B8:8A:CA:43:94:40:DB:10:EF:AD:7E:E5:97:AA:A3:7F:B1:6F:B1:D7:92:7E:E9:1D:FF:D7:09:A4:9A:AB:16:40
a=setup:active
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:111 opus/48000/2
a=fmtp:111 cbr=1;useinbandfec=1
m=video 10006 RTP/SAVPF 102 122
c=IN IP4 10.0.0.47
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:udpcandidate 1 udp 1080142079 10.0.0.47 10006 typ host generation 0
a=candidate:tcpcandidate 1 tcp 1078862079 10.0.0.47 10087 typ host tcptype passive generation 0
a=ice-ufrag:6f8fnf2cbe41zdry
a=ice-pwd:zwl88or5crhi4igj8tyll1km1oi3djdw
a=ice-options:renomination
a=fingerprint:sha-512 6E:6D:1F:52:27:E1:90:D1:5E:E5:AA:16:4C:F1:AC:82:81:AC:D3:B3:8E:5F:E2:18:C0:42:ED:88:E5:15:F5:49:B8:8A:CA:43:94:40:DB:10:EF:AD:7E:E5:97:AA:A3:7F:B1:6F:B1:D7:92:7E:E9:1D:FF:D7:09:A4:9A:AB:16:40
a=setup:active
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:12 urn:3gpp:video-orientation
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 packetization-mode=1;profile-level-id=42001f
a=rtpmap:122 rtx/90000
a=fmtp:122 apt=102

server room config:

export const MediaRoomConfig = [
  {
    kind        : 'audio',
    name        : 'opus',
    clockRate   : 48000,
    payloadType : 100,
    channels    : 2,
    parameters : {
      useinbandfec : 1,
      cbr: 1,
    }
  },
  {
    kind: 'video',
    name: 'H264',
    clockRate: 90000,
    parameters:
    {
      'packetization-mode': 1,
      'profile-level-id': '42001f'
    }
  }
];

checking ssrc:

codecImplementationName	OpenH264

Chrome Version 75.0.3770.90 (Official Build) (64-bit)
mediasoup (v2)

Any pointers appreciated.

At least in MacOS:

Make sure Hardware-accelerated video decode flag (chrome://flags) is enabled.

Otherwise the HW encoder is not used.

It is enabled and external encoder works properly in Munge SDP and AppRTC in the same browser. There must some other variable considered we’re missing at this point. I’m having hard time finding any reasonable description of what exactly the ExternalEncoder needs on the input side to behave properly. It seems the problem lies outside of SDP somewhere.

In order to have a start point., have you tried mediasoup-demo (v2)? Does it work for you there?

Try adding level-asymmetry-allowed=1 param in the room codec.

Nope, trying this: https://v3demo.mediasoup.org/?roomId=xxx&forceH264=true I can see OpenH264 is used.

mediasoup-demo (v2) from webrtc-internals:

localDescription sdp:

type: offer, sdp: v=0
o=- 5903509325747470903 3 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0 1
a=msid-semantic: WMS 1ad9c42e-40dc-42d2-8765-521f4984dba3
m=audio 62460 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126
c=IN IP4 10.0.0.47
a=rtcp:9 IN IP4 0.0.0.0
a=candidate:3588808435 1 udp 2113937151 10.0.0.47 62460 typ host generation 0 network-cost 999
a=candidate:2180346033 1 udp 2113939711 2601:646:c300:fb0f:4cfe:8197:78b4:c03f 62461 typ host generation 0 network-cost 999
a=ice-ufrag:GOoy
a=ice-pwd:b48267w/Gnvo3EdTvqP9rDmK
a=ice-options:trickle
a=fingerprint:sha-256 ED:F2:1B:24:BF:C0:9C:0F:70:6B:D9:9E:18:F2:1F:C9:A4:0D:91:64:25:77:7B:B6:A3:83:7E:71:29:85:82:1F
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:1ad9c42e-40dc-42d2-8765-521f4984dba3 35c71fe6-ef92-4134-b4c2-3c7ad7846182
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:2351042407 cname:5GjYnVChjGUSn7fT
a=ssrc:2351042407 msid:1ad9c42e-40dc-42d2-8765-521f4984dba3 35c71fe6-ef92-4134-b4c2-3c7ad7846182
a=ssrc:2351042407 mslabel:1ad9c42e-40dc-42d2-8765-521f4984dba3
a=ssrc:2351042407 label:35c71fe6-ef92-4134-b4c2-3c7ad7846182
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 122 127 121 125 107 108 109 124 120 123 119 114 115 116
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:GOoy
a=ice-pwd:b48267w/Gnvo3EdTvqP9rDmK
a=ice-options:trickle
a=fingerprint:sha-256 ED:F2:1B:24:BF:C0:9C:0F:70:6B:D9:9E:18:F2:1F:C9:A4:0D:91:64:25:77:7B:B6:A3:83:7E:71:29:85:82:1F
a=setup:actpass
a=mid:1
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:12 urn:3gpp:video-orientation
a=extmap:2 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=msid:1ad9c42e-40dc-42d2-8765-521f4984dba3 2dc84b83-68a9-46b0-b2a4-c3875cd0fd13
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:122 rtx/90000
a=fmtp:122 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:121 rtx/90000
a=fmtp:121 apt=127
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:124 H264/90000
a=rtcp-fb:124 goog-remb
a=rtcp-fb:124 transport-cc
a=rtcp-fb:124 ccm fir
a=rtcp-fb:124 nack
a=rtcp-fb:124 nack pli
a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=124
a=rtpmap:123 H264/90000
a=rtcp-fb:123 goog-remb
a=rtcp-fb:123 transport-cc
a=rtcp-fb:123 ccm fir
a=rtcp-fb:123 nack
a=rtcp-fb:123 nack pli
a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032
a=rtpmap:119 rtx/90000
a=fmtp:119 apt=123
a=rtpmap:114 red/90000
a=rtpmap:115 rtx/90000
a=fmtp:115 apt=114
a=rtpmap:116 ulpfec/90000
a=rid:r0 send
a=rid:r1 send
a=rid:r2 send
a=simulcast:send r0;r1;r2

remoteDescription sdp:

type: answer, sdp: v=0
o=mediasoup-client 10000 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-lite
a=fingerprint:sha-512 02:36:D9:0A:38:D5:AD:CB:96:44:39:0C:C5:CC:C6:43:BA:59:96:FC:04:36:8D:5E:F1:F4:2A:87:83:A8:89:4C:D7:A6:9A:0C:D6:9B:D4:77:42:23:A0:5C:86:B9:12:7C:BC:10:AD:AB:54:A3:8E:7B:97:B3:4D:FD:27:E5:BA:A6
a=msid-semantic: WMS *
a=group:BUNDLE 0 1
m=audio 7 UDP/TLS/RTP/SAVPF 111
c=IN IP4 127.0.0.1
a=rtpmap:111 opus/48000/2
a=fmtp:111 useinbandfec=1;stereo=1;usedtx=1
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=setup:active
a=mid:0
a=recvonly
a=ice-ufrag:l65xt4etc2zprx4t
a=ice-pwd:qf6ai1j61v8mse9y3s3vvuw3ewhb5b67
a=candidate:udpcandidate 1 udp 1076302079 62.210.59.78 56069 typ host
a=end-of-candidates
a=ice-options:renomination
a=rtcp-mux
a=rtcp-rsize
m=video 7 UDP/TLS/RTP/SAVPF 125 107
c=IN IP4 127.0.0.1
a=rtpmap:125 H264/90000
a=rtpmap:107 rtx/90000
a=fmtp:125 packetization-mode=1;level-asymmetry-allowed=1;profile-level-id=42e01f;x-google-start-bitrate=1000
a=fmtp:107 apt=125
a=rtcp-fb:125 goog-remb 
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack 
a=rtcp-fb:125 nack pli
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:5 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=extmap:13 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:8 http://tools.ietf.org/html/draft-ietf-avtext-framemarking-07
a=extmap:12 urn:3gpp:video-orientation
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=setup:active
a=mid:1
a=recvonly
a=ice-ufrag:l65xt4etc2zprx4t
a=ice-pwd:qf6ai1j61v8mse9y3s3vvuw3ewhb5b67
a=candidate:udpcandidate 1 udp 1076302079 62.210.59.78 56069 typ host
a=end-of-candidates
a=ice-options:renomination
a=rtcp-mux
a=rtcp-rsize
a=rid:r0 recv
a=rid:r1 recv
a=rid:r2 recv
a=simulcast:recv r0;r1;r2

In this case I can see your demo is probably using just profile 42e01f. Any way to force 42001f ?

Inaki: Same behavior with level-asymmetry-allowed=1 in our app

Aha, got it!

It depends on the constraints provided to getUserMedia().

For instance, the following constraints plus correct H264 profile-level-id, make chrome use external encoder:

{ width: { ideal: 640 }, height: { ideal: 480 } }
or
{ width: { ideal: 1280 }, height: { ideal: 720 } }

Whereas for a lower width and height values the external encoder is NOT activated. Ie:

{ width: { ideal: 320 }, height: { ideal: 240 } }

Yes, the v3demo enables a single H264 codec configuration with these parameters:

'packetization-mode'      : 1,
'profile-level-id'        : '42e01f',
'level-asymmetry-allowed' : 1

As conclusion from previous experiments, and as @jmillan said, whether Chrome (in OSX at least) uses external encoder or OpenH264 depends on the remote H264 codec configuration (profile-level-id must be “42001f”) plus the height/width constraints given to getUserMedia() to get the local video track.

Yeah, the size constraint really is the actual issue here. Thank you for unraveling yet another blackbox weirdness around WebRTC and Chrome :slight_smile:

another observation I noticed – it doesn’t matter what the height is for as long as width is at least 640. Maybe somebody finds this information interesting one day.

I assume Chrome can use the external encoder as far as the requested video size matches one of the values natively supported by the webcam and the selected H264 configuration also matches the encoding produced by the webcam. Yes, all this is very “dark”…