Firefox and redundant Sender Extended Report packets

mediasoup libraries
1

I know from the ealier topics that these packets are expected for the disabled SDP sections. However, combined with the current solution of this issue, it creates quite a flow of packets in Firefox if many producers are created and closed: SDP is growing indefinitely since no sections are reused, and the browser sends a packet for each section. What’s more, it keeps them sending even when there is no track in any transceiver. Somehow, this does not look right…

2

The issue you mentioned was already fixed over a year ago according to comments.

Moreover, sections are reused since https://github.com/versatica/mediasoup-client/commit/c0da3089167440e53c57c87f8f235bcdd5f2bd8f was merged a few months ago.

There must be something about the way you use the library I think.

3

In case of Firefox, sections are intentionally not reused. Take a look at the send method of the Firefox60 handler.

4

True, sorry. I think if there it is the best to find or report upstream Firefox issue on Bugzilla then.

5

It’s kind of complicated. Exactly what is the bug here? That issue #104, as far as I understand, was caused by the fact that mediasoup client is trying to ‘guess’ what is in the local offer received from the browser without even looking into it. The browser developers can always ask: “Why would you do this in the first plaсe? Just check what we offer.”

6

Well, at least m= reuse support would be nice to have, not sure if there is already ticket for it.

7

In this case m= reuse conflicts with the method of making an answer that matches the offer. I am not familiar in detail with the technology: the comment in the send method of the Firefox handler says that Firefox should reuse the closed media section, but fails. Is that really so? The client adds new transceiver to the peer connection. Is it supposed to be associated with a previously closed section (if there is one) according to some specification?

8

Well, yes. The comment says that because yes, Firefox doing that. Now, it seems that we (mediasoup team) are responsible fo a bug in Firefox. Actually we are not. So you see a comment written by us telling that there is an issue in Firefox, you wonder whether it’s true or not and you blame us.

If you want to test this wrong behavior, change Firefox handler in mediasoup-client and make the SDP section reusing mechanism so it behaves as in latest Chrome. Then do some tests (create mic/webcam producers, close them, create them again, etc) and you’ll see the problem.

Just blaming us because we wrote a comment about a problem in Firefox does not mean that the issue is in mediasoup-client.

Yes, as i Chrome. But Firefox fails doing this.

And then what? how to know which one was the new generated SDP m section? There are rules in the WebRTC spec. If a m= section is closed (this is, if the local or remote port of the m= section is 0) and you create a new transceiver, it will reuse such a m= section. But Firefox randomly fails on this.

9

Well, I investigated a little what really happens there. It seems that Firefox does reuse the m= sections, but in its own way: it chooses not just first available closed section, but the section where the track kind matches the track kind in the new transceiver. And the RemoteSdp.getNextMediaSectionIdx() method just does not work. So I tried this (directly in JS code):

  1. Added the following method to the RemoteSdp class:
    send2({ offerMediaObject, localMedia, offerRtpParameters, answerRtpParameters, codecOptions, extmapAllowMixed = false }) {
        const mediaSection = new MediaSection_1.AnswerMediaSection({
            iceParameters: this._iceParameters,
            iceCandidates: this._iceCandidates,
            dtlsParameters: this._dtlsParameters,
            plainRtpParameters: this._plainRtpParameters,
            planB: this._planB,
            offerMediaObject,
            offerRtpParameters,
            answerRtpParameters,
            codecOptions,
            extmapAllowMixed
        });

	let idx = this._midToIndex.get(mediaSection.mid);
	if (idx)
	{
            // The same as in _replaceMediaSection
            this._mediaSections[idx] = mediaSection;
            this._sdpObject.media[idx] = mediaSection.getObject();
	}
	else
	{
            // Add new section and synchronize RemoteSdp with the local offer
            if (!this._firstMid)
                this._firstMid = mediaSection.mid;

            idx = this._mediaSections.length;
            this._mediaSections.push(mediaSection);
            this._sdpObject.media.push(mediaSection.getObject());
            this._midToIndex.set(mediaSection.mid, idx);

            let mediaSections = [], media = [], midToIndex = new Map();
            idx = 0;
            for (let s of localMedia)
            {
                let mid = String(s.mid), j = this._midToIndex.get(mid);
                mediaSections.push(this._mediaSections[j]);
                media.push(this._sdpObject.media[j]);
                midToIndex.set(mid, idx++);
            }
            this._mediaSections = mediaSections;
            this._sdpObject.media = media;
            this._midToIndex = midToIndex;

            this._regenerateBundleMids();
	}
 }

  1. Restored this._remoteSdp.closeMediaSection in the stopSending of Firefox handler

  2. In the send method of the handler, instead of

	const offerMediaObject = localSdpObject.media[localSdpObject.media.length - 1];
        ...
	this._remoteSdp.send(...);

I do:

    const offerMediaObject = localSdpObject.media[localSdpObject.media.findIndex(s => s.mid == localId)];
        ...
    this._remoteSdp.send2({
            offerMediaObject,
            localMedia: localSdpObject.media,
            offerRtpParameters: sendingRtpParameters,
            answerRtpParameters: sendingRemoteRtpParameters,
            codecOptions,
            extmapAllowMixed: true
        });

And it appears to work ok. I am not sure however whether this is compatible with everything else.

2 Likes
10

Nice. This is something I was afraid of in the past. This is not what the standard says so this deserves a proper bug report in Firefox tracker.

Other than that, please report this in the mediasoup-client GitHub repository with all the info you can provide and I’ll take care of it eventually. Yes, we assume that “closed m= section reusing” does not require same kind, but Firefox may behave that way so we need to change a few things.

1 Like
11

Yes, we assume that “closed m= section reusing” does not require same kind

Doesn’t this mean skating on thin ice? To me, finding where the new transceiver has landed in the local offer appears to be a straightforward approach, I would have rather used it than relied on some guess about what browsers “should” do, even without Firefox issue.

12

Ok, just tell me how to do that.

NOTE: The WebRTC API is as it is. And yes, it’s a pain. And SDP is a pain. But that’s not my fault, and that’s not mediasoup-client’s fault. We just have to deal with that.

NOTE 2: If you don’t report this issue (with your findings above) in GH we’ll forget about it.

13

I arranged it as a PR, please take a look.

1 Like
14

Added an issue referencing this PR (#150)

15

I posted this issue on the mediasoup-demo forum and it seems to be related to issue #104. When closing and creating video producers too frequently on Firefox when using simulcast (For me this happens on the 3rd attempt), stream layers aren’t produced or their scores drop to 0 where streams are consumed at the lowest resolution.