Modify gstreamer.sh to try H264 RTP push stream, but failed

mediasoup-demo
1

The default gstreamer.sh uses VP8, but i want H264, and modify it as below:

${HTTPIE_COMMAND} -v \
	POST ${SERVER_URL}/rooms/${ROOM_ID}/broadcasters/${BROADCASTER_ID}/transports/${videoTransportId}/producers \
	kind="video" \
	rtpParameters:="{ \"codecs\": [{ \"mimeType\":\"video/h264\", \"payloadType\":${VIDEO_PT}, \"clockRate\":90000 }], \"parameters\":{ \"packetization-mode\": 1, \"profile-level-id\": \"42e01f\", \"level-asymmetry-allowed\": 1, \"x-google-start-bitrate\": 1000}, \"encodings\": [{ \"ssrc\":${VIDEO_SSRC} }] }" \
	> /dev/null

&

gst-launch-1.0 \
	rtpbin name=rtpbin \
	filesrc location=${MEDIA_FILE} \
	! qtdemux name=demux \
	demux.video_0 \
	! queue \
	! decodebin \
	! videoconvert \
	! x264enc qp-min=18 \
	! rtph264pay pt=${VIDEO_PT} ssrc=${VIDEO_SSRC} \
	! rtpbin.send_rtp_sink_0 \
	rtpbin.send_rtp_src_0 ! udpsink host=${videoTransportIp} port=${videoTransportPort} \
	rtpbin.send_rtcp_src_0 ! udpsink host=${videoTransportIp} port=${videoTransportRtcpPort} sync=false async=false \
...

However, client side reports error:

http: warning: HTTP 500 unsupported codec [mimeType:video/h264, payloadType:101]

& i got the server side async error stack trace:

UnsupportedError: unsupported codec [mimeType:video/h264, payloadType:101]
    at Object.getProducerRtpParametersMapping (/Users/chenzx/repo/mediasoup-demo/server/node_modules/mediasoup/lib/ortc.js:500:19)
    at PlainTransport.produce (/Users/chenzx/repo/mediasoup-demo/server/node_modules/mediasoup/lib/Transport.js:251:33)
    at Room.createBroadcasterProducer (/Users/chenzx/repo/mediasoup-demo/server/lib/Room.js:544:20)
    at /Users/chenzx/repo/mediasoup-demo/server/server.js:291:33
    at Layer.handle [as handle_request] (/Users/chenzx/repo/mediasoup-demo/server/node_modules/express/lib/router/layer.js:95:5)
    at next (/Users/chenzx/repo/mediasoup-demo/server/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/Users/chenzx/repo/mediasoup-demo/server/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/Users/chenzx/repo/mediasoup-demo/server/node_modules/express/lib/router/layer.js:95:5)
    at /Users/chenzx/repo/mediasoup-demo/server/node_modules/express/lib/router/index.js:281:22
    at param (/Users/chenzx/repo/mediasoup-demo/server/node_modules/express/lib/router/index.js:354:14)

The rtpParameters’s parameters part is copied from config.js, why the line const rtpMapping = ortc.getProducerRtpParametersMapping(rtpParameters, routerRtpCapabilities); in Transport.js failed?

2

Please read the documentation of mediasoup about how to set available codecs in the Router instance of mediasoup.

3

i didn’t get it: the config.js has 4 codecs, i just use the 4th item in client rtpParametrs, why not match?

4

I said something about reading the documentation. Please, don’t ignore it when you reply.

5

The documentation says “see the RFC 6184 for more details”, but i don’t think it’s quick solution~

I’m planning to write a unit test case to investigate the weird match case…

6

ok, i find the problem: need to put parameters nested into the codecs item, previously i put it outside of codecs part.

7

rtpParameters is a typo, sorry for my miss.
but gstreamer.sh + H264 RTP push stream still failed
i guess because i use the x264enc in gstreamer pipeline string, while x264enc is in gst-plugins-ugly, so there MUST be something wrong with this plugin…?

8

since the input mp4 file is already H264 constrained basic profile encoded, i re-tried the gstreamer pipeline string below:

gst-launch-1.0 \
	rtpbin name=rtpbin \
	filesrc location=${MEDIA_FILE} \
	! qtdemux name=demux \
	demux.video_0 \
	! queue \
	! rtph264pay pt=${VIDEO_PT} ssrc=${VIDEO_SSRC} \
	! rtpbin.send_rtp_sink_0 \
	rtpbin.send_rtp_src_0 ! udpsink host=${videoTransportIp} port=${videoTransportPort} \
	rtpbin.send_rtcp_src_0 ! udpsink host=${videoTransportIp} port=${videoTransportRtcpPort} sync=false async=false \
...

The client console successfully displays the progress timer:

>>> creating Broadcaster...
>>> creating mediasoup PlainTransport for producing audio...
>>> creating mediasoup PlainTransport for producing video...
>>> creating mediasoup audio Producer...
>>> creating mediasoup video Producer...
>>> running gstreamer...
设置暂停管道 ...
管道正在 PREROLLING ...
重新分配延迟时间...
管道被 PREROLLED ...
设置播放管道 ...
New clock: GstSystemClock
0:00:10.1 / 0:01:31.6 (11.1 %)

But there is nothing response in receiver browser page…

9

H264 codec negotiation is hard. You are not mentioning anything about profile-level-id, which determines the configuration of H264 and is critical that both sides support. I cannot help with that but you should investigate it and tell your H264 encoder to encode using a specific H264 profile that is supported by the receivers, and include such a H264 configuration codec in the mediasoup Router.

10

I’m trying really hard to get a non HTTP 500 from

${HTTPIE_COMMAND} -v \
        POST ${SERVER_URL}/rooms/${ROOM_ID}/broadcasters/${BROADCASTER_ID}/transports/${videoTransportId}/producers \
        kind="video" \
        rtpParameters:="{ \"codecs\": [{ \"mimeType\":\"video/h264\", \"payloadType\":${VIDEO_PT}, \"clockRate\":90000, \"preferredPayloadType\": ${VIDEO_PT} }], \"encodings\": [{ \"ssrc\":${VIDEO_SSRC} }], \"parameters\":{ \"packetization-mode\": 1, \"profile-level-id\": \"${PROFILE_LEVEL_ID}\", \"level-asymmetry-allowed\": 1, \"x-google-start-bitrate\": 1000 } }" \
        > /dev/null

I modified ffmpeg.sh from mediasoup-demo/ffmpeg.sh at v3 · versatica/mediasoup-demo · GitHub to publish a baseline constrained H.264 video with ffmpeg but I failed.

http: warning: HTTP 500 unsupported codec [mimeType:video/h264, payloadType:101]

I tried everything, it simply doesn’t work for me. How did you get it working? Could you share your config.js and full commandline?