Get video resolution server side or from client producer

mediasoup libraries
1

Hi! Is it somehow possible to get the video resolution on the server side?

The usecase is to show it in stats for each participant. The only alternative in our case would be to somehow get it on the client side from the producer. But also there my digging has come up empty as there seems to be inconsistencies between browsers and for example in Firefox I was not able to get it when digging into producer.rtpSender.

I am interested in the actual resolution being sent over the transport. Not the MediaStreamTrack resolution that is obtained from the camera. The two can differ when for example scaleResolutionDownBy is used in encodings.

2

mediasoup does not decode so it’s not possible to do this in server side unless you also consume the track in server side using any other RTP tool and inspect the resolution of the frames.

3

Not possible without decoding today, but maybe one day: https://webrtc.googlesource.com/src/+/refs/heads/master/docs/native-code/rtp-hdrext/video-layers-allocation00

As you receive video on the other side and likely decode it for showing in the browser, you can observe video resolution for the layer you currently receiving, this may fit your use case. Firefox doesn’t expose as much stats as we all would like unfortunately.

1 Like
4

Such a header is not supported by mediasoup yet.

5

Thanks for the replies! I knew mediasoup being a SFU doesn’t decode the video but my hope there would be some metadata where the resolution is exposed. Seems like in the future at some point this might be the case.

Nazar: yes we grab the resolution on the consumer side but have two challenges with this:

  1. We do not consume the stream that the participant sends, only from all other participants. So we have consumer stats for everyone but not yourself.
  2. We can’t easily log this server side. We could grab it from the consumers and send it to the server I guess but it’s far from ideal.

Now that we talked about it, maybe we can just create a consumer (well actually just rtpreceiver) that ingests the data from the rtpsender locally without even going through the mediasoup server. Just to get the stats :slight_smile:

Thanks for the production discussion!

6

It wouldn’t be the same stats I’m afraid. Actual resolution will also depend on networking conditions, so if you do everything locally it will likely be different from what other consumers get over real network.

7

Just a note, in Chome the RTCPeerConnection.getStats() exposed by mediasoup-client Transport.getStats() returns the real resolution. The problem is that you would need continuous polling and process this nasty structure :slight_smile:
Also, getStats() doesn’t return identical information in Chrome, Safari and FF.