I’ve confirmed that chrome can play H264 video via WebRTC (using Janus server), no special config needed(audio still using Opus encoding).
Does mediasoup support H264 streaming? There is no detailed documentation about that.
Please help clarify this. thanks!
Yes there is
Hi, ibc
i tried with the below config:
{
mimeType : 'video/H264',
clockRate : 90000,
payloadType : 96,
parameters : {
"packetization-mode" : 1,
"profile-level-id": "640032",
"x-google-start-bitrate" : 1000,
"level-asymmetry-allowed" : 1
},
rtcpFeedback : [
{
"type" : "nack",
"parameter" : "pli"
},
{
"type" : "ccm",
"parameter" : "fir"
},
{
"type" : "goog-remb",
"parameter" : ""
},
{
"type" : "transport-cc",
"parameter" : ""
}
], // FFmpeg does not support NACK nor PLI/FIR. ?
}
This can support chrome’s page2page H264 streaming with 640*480 size.
But when i use the ffmpeg RTP push command(video only):
const callRTPStreamPushCmd = `ffmpeg \
-re \
-v info \
-stream_loop -1 \
-i ${input_mp4_filepath} \
-an \
-c:v copy \
-f rtp \
-sdp_file video.sdp \
"[select=v:f=rtp:ssrc=22222222:payload_type=96]rtp://127.0.0.1:${ports.videoRtp}?rtcpport=${ports.videoRtcp}"`
Failed.
And also tried using gstreamer: (modified according to mediasoup-demo/gstreamer.sh at v3 · versatica/mediasoup-demo · GitHub)
const callRTPStreamPushCmd = `gst-launch-1.0 \
rtpbin name=rtpbin \
filesrc location=${input_mp4_filepath} \
! qtdemux name=demux \
demux.video_0 \
! queue \
! decodebin \
! rtph264pay pt=96 ssrc=22222222 picture-id-mode=2 \
! rtpbin.send_rtp_sink_0 \
rtpbin.send_rtp_src_0 ! udpsink host=127.0.0.1 port=${ports.videoRtp} \
rtpbin.send_rtcp_src_0 ! udpsink host=127.0.0.1 port=${ports.videoRtcp} sync=false async=false`
Also failed. How can i do a H264 RTP stream push with ffmpeg or gstreamer?
You need to configure profile-level-id to be the same as the video you’re using AND browser in question should support that particular profile. It is only really safe to use constrained baseline profile if you want H264 to work everywhere.
Hi nazar-pc,
How can i know which is the correct profile-level-id with my test input mp4 file? (i don’t want to use any transcoding at client side) and how can i know if chrome browser’s WebRTC+H264 stack supports this profile?
My test input mp4 file is 1080p with aac audio encoding(audio is not used here), is inout resolution the reason?
Try ffprobe and see what it says about the file.
If you take a look to test folder in h246-profile-level-id library (created by us and used in mediasoup) you may figure it out. Otherwise you have to read the H264 RFC spec.
I tried the value ffmpeg geneated in sdp file & the value from chrome://webrtc-internals page: 640028、640032、640015, none works
% ffprobe ./test.mp4
…
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './test.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.76.100
Duration: 00:01:31.61, start: 0.000000, bitrate: 1295 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 640x360 [SAR 1:1 DAR 16:9], 1158 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
Metadata:
handler_name : Core Media Video
vendor_id : [0][0][0][0]
Stream #0:1(zho): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
H264 High profile will not work in many browsers when sent over WebRTC (it will using <video> tag though).
I have converted the test.mp4 to constrained basic profile, but still not works:
% ffmpeg -i ./test_640p.mp4 -vcodec libx264 -acodec aac -b:a 256k -profile:v baseline test_640p_basic.mp4
The symptom: chrome page cannot receive RTP video stream data.
mediasoup server side log:
mediasoup:Channel request() [method:transport.consume, id:6] +5ms
mediasoup:Channel [pid:87109] RTC::SimpleConsumer::CreateRtpStream() | [ssrc:365044946, payloadType:96] +1ms
mediasoup:Channel [pid:87109] RTC::SimpleConsumer::CreateRtpStream() | FIR supported +0ms
mediasoup:Channel [pid:87109] RTC::SimpleConsumer::CreateRtpStream() | NACK supported +0ms
mediasoup:Channel [pid:87109] RTC::SimpleConsumer::CreateRtpStream() | PLI supported +0ms
mediasoup:Channel request succeeded [method:transport.consume, id:6] +1ms
mediasoup:Consumer constructor() +0ms
mediasoup:WebRtcTransport connect() +18ms
mediasoup:Channel request() [method:transport.connect, id:7] +10ms
mediasoup:Channel request succeeded [method:transport.connect, id:7] +1ms
mediasoup:Channel [pid:87109] RTC::IceServer::HandleTuple() | transition from state 'new' to 'connected' +8ms
mediasoup:Channel [pid:87109] RTC::WebRtcTransport::OnIceServerSelectedTuple() | ICE selected tuple +0ms
mediasoup:Channel [pid:87109] RTC::WebRtcTransport::OnIceServerConnected() | ICE connected +0ms
mediasoup:Channel [pid:87109] RTC::WebRtcTransport::MayRunDtlsTransport() | running DTLS transport in local role 'server' +1ms
mediasoup:Channel [pid:87109] RTC::WebRtcTransport::OnDtlsTransportConnecting() | DTLS connecting +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::Run() | running [role:server] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | DTLS handshake start +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'before SSL initialization'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | role: server, waiting:'before SSL initialization'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'before SSL initialization'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS read client hello'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS write server hello'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS write certificate'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS write key exchange'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS write certificate request'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS write server done'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | role: server, waiting:'SSLv3/TLS write server done'] +1ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS write server done'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS read client certificate'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS read client key exchange'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS read certificate verify'] +1ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS read change cipher spec'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS read finished'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS write change cipher spec'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | [role:server, action:'SSLv3/TLS write finished'] +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::OnSslInfo() | DTLS handshake done +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::CheckRemoteFingerprint() | valid remote fingerprint +0ms
mediasoup:Channel [pid:87109] RTC::DtlsTransport::GetNegotiatedSrtpCryptoSuite() | chosen SRTP crypto suite: SRTP_AEAD_AES_256_GCM +0ms
mediasoup:Channel [pid:87109] RTC::WebRtcTransport::OnDtlsTransportConnected() | DTLS connected +0ms
mediasoup:Consumer resume() +33ms
mediasoup:Channel request() [method:consumer.resume, id:8] +11ms
mediasoup:Channel request succeeded [method:consumer.resume, id:8] +1ms
mediasoup:Channel [pid:87109] RTC::Transport::HandleRtcpPacket() | PLI received, requesting key frame for Consumer [sender ssrc:1, media ssrc:365044946] +1ms
mediasoup:Channel [pid:87109] RTC::IceServer::HandleTuple() | transition from state 'connected' to 'completed' +33ms
mediasoup:Channel [pid:87109] RTC::WebRtcTransport::OnIceServerCompleted() | ICE completed +0ms
What do you mean by “not works”? I don’t see any errors server-side, check browser logs and dev tools, see if it found decoder for this video.
from chrome://webrtc-internals page, i can see 2 rtc inbound rtp video stream, no data received displayed here, normally there is H264 encoding info here
normally if i use chrome page client to publish, mediasoup server log:
mediasoup:Producer constructor() +0ms
mediasoup:Channel [pid:87330] RTC::IceServer::HandleTuple() | transition from state 'connected' to 'completed' +41ms
mediasoup:Channel [pid:87330] RTC::WebRtcTransport::OnIceServerCompleted() | ICE completed +0ms
mediasoup:Channel [pid:87330] RTC::Producer::GetRtpStream() | ignoring RTX packet for not yet created RtpStream (ssrc lookup) +137ms
mediasoup:WARN:Channel [pid:87330] RTC::Producer::ReceiveRtpPacket() | no stream found for received packet [ssrc:3741614746] +0ms
mediasoup:Channel [pid:87330] RTC::Producer::CreateRtpStream() | [encodingIdx:0, ssrc:2379022241, rid:, payloadType:123] +0ms
mediasoup:Channel [pid:87330] RTC::Producer::CreateRtpStream() | FIR supported +0ms
mediasoup:Channel [pid:87330] RTC::Producer::CreateRtpStream() | NACK supported +0ms
mediasoup:Channel [pid:87330] RTC::Producer::CreateRtpStream() | PLI supported +0ms
mediasoup:Channel [pid:87330] RTC::Producer::ReceiveRtpPacket() | key frame received [ssrc:2379022241, seq:4653] +0ms
and then open subscribe page:
mediasoup:Channel [pid:87330] RTC::Transport::HandleRtcpPacket() | PLI received, requesting key frame for Consumer [sender ssrc:1, media ssrc:399602566] +0ms
mediasoup:Channel [pid:87330] RTC::IceServer::HandleTuple() | transition from state 'connected' to 'completed' +52ms
mediasoup:Channel [pid:87330] RTC::WebRtcTransport::OnIceServerCompleted() | ICE completed +0ms
mediasoup:Channel [pid:87330] RTC::Producer::ReceiveRtpPacket() | key frame received [ssrc:2379022241, seq:22431] +1ms
mediasoup:Channel [pid:87330] RTC::SimpleConsumer::SendRtpPacket() | sync key frame received +0ms
mediasoup:Channel [pid:87330] RTC::SimpleConsumer::SendRtpPacket() | sending sync packet [ssrc:399602566, seq:1, ts:503037965] from original [seq:22431] +0ms
The difference is, when using ffmpeg/gstream to do RTP stream push, seems no “key frame received” log?
What is GOP/key frame interval in that video? It might be expected behavior if you have key frame interval of 8 seconds to have no video for around 8 seconds.
i wait about ~20s but there is no response in subscribe page.
i guess it’s not GOP problem.
something error happened, causing the RTP streaming stuck
according to 使用 WebRTC 的 GPU 加速流式传输 | 云架构中心 | Google Cloud
i changed rtp config to profile-level-id=42001f，payloadType=102
but still not working
There must be something else. Maybe port or ssrc is incorrect in ffmpeg command.
i checked the browser page ==> browser page WebRTC streaming case, in chrome://webrtc-internals , i could see that the outbound video stream and the inbound one have different SSRC, also, why there is 2 inbound video streams?
initially, when i used the mediasoup’s official ffmpeg rtp push case(H264 file converted to VP8 encoding, everything works fine.) but why when i tried to directly use H264 video data, the streaming proc is blocked by something unknown?
When i modify the ffmpeg RTP push stream command string by changing “libvpx” to “libx264”, H264 streaming is OK, except for “profile-level-id” is re-negotiated from “4d0032” in setting to “4d0015”.
I can see the server log says:
mediasoup:Channel [pid:12857] RTC::Producer::ReceiveRtpPacket() | key frame received [ssrc:22222222, seq:8122]
But the browser page consumer/player side is not playing video fluent, looks like “jumping from keyframe to next keyframe”? wierd.
But my mp4 file is already H264 encoded, why need re-encoding to let everything OK?
I’ll retry gstreamer case.