AV1 support is closed, but now with AV1 support that is expected to ship in Chromium 90 or even 89, it is relevant again.
And looks like @vpalmisano started working on it in his fork 
, so exciting!
2 Likes
I have started to add the required values to support AV1X codec, but at the moment the parser class is only a stub (a copy of VP9).
I’m stuck with this bug that prevents chromium to encode AV1 packets inside the webrtc stack:
1170699 - chromium - An open-source project to help move the web forward. - Monorail, so I’m unable to receive any packet.
Moreover, it’s not clear if both simulcast and svc will be supported, or only svc.
I suspect it can already be fixed in recent Chromium Nightly and Chrome Canary builds, did you try those?
Tested right now the latest chromium linux build (90.0.4405.0), same errors.
So the changes mentioned in the issue point to this: 11215febb9500c597c940a40f1b9fe86c939413f - src.git - Git at Google
Which says that scalability mode is now required.
I think if you force SVC one way or another it should start working.
OK, next nightly should have a fix for this regression
Yes, we should wait this revision to be approved: 3faea70d1a1eef602eaff02fa67ece20734619c7 - src - Git at Google
Update: fixed in 90.0.4407.0
Simulcast: multiple encoded bitstreams, each with a different SSRC, separately decodable, signalled.
SVC: multiple resolutions, single bitstream, single SSRC, only base layer separately decodable, no need for signalling.
K-SVC: multiple resolutions, single bitstream, single SSRC, several layers can be decoded independently, no need for signalling.
traditional simulcast is not recommended with AV1, K-SVC is preferred. See discussion during the RTP payload specification here:
SVC and simulcast achieve the same thing, but SVC will allow for almost immediate resolution change, while simulcast need to wait for next I-Frame, among other things.
3 Likes
Now if I enable the same SVC mechanism implemented for VP9 (addLegacySimulcast), I get this error in chrome:
[111581:111591:0202/180713.063283:INFO:video_send_stream_impl.cc(247)] VideoSendStreamInternal: {encoder_settings: { experiment_cpu_load_estimator: on}}, rtp: {ssrcs: [1632335588, 1632335589, 1632335590], rids: [], mid: '0', rtcp_mode: RtcpMode::kReducedSize, max_packet_size: 1200, extmap-allow-mixed: false, extensions: [{uri: http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01, id: 3}, {uri: http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time, id: 2}, {uri: urn:3gpp:video-orientation, id: 13}, {uri: urn:ietf:params:rtp-hdrext:sdes:mid, id: 4}, {uri: urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id, id: 6}, {uri: urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id, id: 5}], lntf: {enabled: false}, nack: {rtp_history_ms: 1000}, ulpfec: {ulpfec_payload_type: -1, red_payload_type: -1, red_rtx_payload_type: -1}, payload_name: AV1X, payload_type: 35, raw_payload: false, flexfec: {payload_type: -1, ssrc: 0, protected_media_ssrcs: []}, rtx: {ssrcs: [4055469294, 4055469295, 4055469296], payload_type: 36}, c_name: DRkJ1htAg67kzYkd}, rtcp_report_interval_ms: 1000, send_transport: (Transport), render_delay_ms: 0, target_delay_ms: 0, suspend_below_min_bitrate: off}
...
[111581:111586:0202/180713.078326:INFO:video_stream_encoder.cc(978)] ReconfigureEncoder:
[111581:111586:0202/180713.078379:WARNING:libaom_av1_encoder.cc(167)] Simulcast is not implemented by LibaomAv1Encoder.
AFAIK AV1 uses the standard scalabilityMode property in the single encodings entry to define number of spatial/temporal layers, but I don’t think it’s implemented in Canary yet.
When I use the single encodings entry { scalabilityMode: 'S3T3_KEY' } I get this log from chrome:
Scalability mode is not set, using 'NONE'.
I see. I’m afraid I don’t know current status of that in Chromium master.
In the meantime I have updated my git forks (obviously the AV1X class parsing is incomplete for now):
1 Like
We should wait for this:
https://chromium.googlesource.com/external/webrtc/+/master/video/video_stream_encoder.cc#268
IMHO, all is there already, and the bug 12000 is not related.
However, supporting AV1 SVC is harder than it looks on the SFU side.
I would suggest that you start with single layer AV1 RTP payload parsing, and then only look at the dependency descriptor, chains, IDD, … . Those (single layer AV1 RTP on one hand, SVC support on the other) are two chunks of work of equivalent complexity. Divide and conquer?
Why should this be needed at all (within a SFU)? AFAIR frame-marking RTP extension is not available for AV1 but, in theory, there is yet another “similar” RTP extension for it. Am I totally wrong?
In the case of single layer AV1 I used the same approach of chromium (modules/rtp_rtcp/source/video_rtp_depacketizer_av1.cc - external/webrtc - Git at Google), that is checking if N=1 => keyframe found (RTP Payload Format For AV1). With SVC I need to go more deeply inside the packet structure.
Maybe (Un)related question: when insertable streams are used, the packet content could be received encrypted/scrambled?
You are correct, Framemarkign as been “downgraded” to experimental, and frame marking paragraph has been removed for the VP9 spec, last week.
For AV1, it is replaced by the Dependency Descriptor (annex of the Av1 RTP payload specification), which defines the notion of chains, IDD, and so on and so forth. That s what I was referring to.
Thanks. Just to clarify (I am far from the spec now): is such a Dependency Description a new RTP extension header?