Disabling rtp orientation header extension for mobile Safari

mediasoup libraries
1

Hi,

We implemented a workaround for orientation issues when sending video from iOS, and thought we’d post it here in case it’s useful to others.

We support Firefox, and we record video tracks using PlainRtpTransport/ffmpeg. For both of these use cases, video received from iOS clients is not properly rotated. Firefox does not support the video-orientation rtp header extension. And ffmpeg doesn’t either (or, perhaps, we have failed to figure out how to pass the orientation signaling through to ffmpeg in a usable fashion.)

So, removing the urn:3gpp:video-orientation from the sdp generated by mediasoup is a good workaround for these orientation issues. (Removing the extension from the sdp forces the client to encode the video rotated, rather than signaling rotation out of band.)

Here’s how we disable the extension, prior to calling the mediasoup-client device.load() function:

      log('loading soup device', routerRtpCapabilities);
      // remove rtp header orientation extension for mobile Safari,
      // because neither Firefox nor ffmpeg support it. so we need
      // Safari to encode the video as rotated, rather than just
      // signal rotation changes
      if (getBrowserName() === 'Safari' && browserMobile_p()) {
        routerRtpCapabilities.headerExtensions =
          routerRtpCapabilities.headerExtensions.filter(
            (ext) => ext.uri !== 'urn:3gpp:video-orientation'
          );
      }
      await this.soupDevice.load({ routerRtpCapabilities });

In our tests, Android Chrome clients work fine, which suggests that Android Chrome is not actually using this header extension. (We plan to revisit this to test more heavily on Android, however, and will update this post if we learn anything new.)

–
Kwin

3 Likes
2

@kwindla may you please an issue in the mediasoup-website project about this? You can just copy the exact same text as above into it. We’ll definitely document this in the website. Thanks.

1 Like