mediasoup-client direct access to h264 stream on device that does not have h264 hardware decoder

mediasoup libraries
1

How possible to access into h264 stream on device that unable decode this kinds of stream? i would like to add software decoder.

2

You indicate mediasoup-client in the subject, which implies client side (browser mainly).

In server side mediasoup you can make use of DirectTransport to consume into Node.js the RTP from a producer.

Beware that this just provides you with the RTP packets which contain the media in the payload. That is not the h264 stream itself.

1 Like
3

@jmillan could you please point to any related sample?

4

Look for recording examples in the Examples section of the website.

1 Like
5

@ibc, could you please point to datachannel sample unable to implement mediasoup datachannel into mediasoup-client?

6

Please, avoid direct naming in the forum. You are not asking things to just me but to any member of the forum.

mediasoup-demo app has DataChannel.

1 Like
7

On server we created data consumer on webrtc transport, and created data producer on direct transport, and connected data consumer with data producer.

On web client we created data consumer on webrtc, but in API require sctpStreamParameters, that we unable to specify because data producer created on direct transport does not have this parameter.

Please advice.

8

Sorry, what you are trying to achieve is not clear for me. Not enough info.

You must create the client side DataConsumer using the parameters of its associated DataConsumer in server side. Forget the DataProducer.

1 Like
9

thank you for prompt reply.

here more background about what we try accomplish.
we have rtp source stream, we create producer on plain transport for rtp source stream.
we need to transfer this stream to web client via DataChannel.

Could you please advice how to do implement it?

10

You cannot consume RTP via DataChannel.

1 Like
11

How we can transfer rtp stream to web client as a data? We would like to custom handle it decoding.

12

Then you must create a DataProducer in a DirectTransport, create a consuming DataConsumer in a WebRtcTransport, and call dataProducer.send(YOUR_CUSTOM_MEDIA_PACKETS). How you get those YOUR_CUSTOM_MEDIA_PACKETS is up to you. Obviously you cannot use a Producer in a PlainTransport for that. You need to get the media packets produced by your RTP source stream into JavaScript within your mediasoup Node.js app and call dataProducer.send() with them.

1 Like
13

Thank you,

Could you please elaborate about getting packets produced by RTP source stream into js within mediasoup? You means direct from UDP port? :astonished:

14
1 Like
15

Sorry, for not clear question.

We have camera that stream h264 over udp (RTP packages) we would like to send this stream to web client via webrtc DataChannel.

In case I understand you correctly : we need source data direct from UDP stream and call DirectTransport dataProducer.send() and connected WebRTC dataConsumer will receive this data?

Or elsewhere please explain how to get packets (from RTP stream using mediasoup) that we need to send DirectTransport dataProducer.send() ?

16

I think I already explained it:

  • If you want to send video over DataChannel to WebRTC browsers/clients then you need a DataProducer in mediasoup server created on top of a DirectTransport.
  • And you need a DataConsumer for each WebRTC client that want to receive it.
  • And you need to get RTP packets in Node.js from your video source in server side so, once you have them in Node Buffer binary format, you call dataProducer.send(buffer).

Now, if the question is: “how do I get RTP packets from my source in Node.js?” then that depends entirely on how your video source is. If it’s a ffmpeg or gstreamer, you may try to get those RTP packets in raw (instead of sending them via regular UDP). Sorry, no idea about how to do that.

Another option is:

  • Your source sends regular plain RTP over UDP to mediasoup Router using a plain transport by creating a normal Producer on that plain transport.
  • You create a DirectTransport in that Router and call consume() to consume the previous producer.
  • In the new Consumer you have, subscribe to rtp event so you’ll get all RTP packets from the Producer in Node in Buffer binary format.
  • Then you call dataProducer.send(buffer).
1 Like
18

Thank you Iñaki,

Following your advice we created DataProducer on DirectTransport and DataConsumer on WebRTCTransport, that connected to this DataProducer, when try to connect on client To DataConsumer we get error:

Uncaught (in promise) TypeError: params is not an object
at Object.validateSctpStreamParameters (app-bundle.js:11625)
validateSctpStreamParameters @ app-bundle.js:11625

mediasoup-client v3.6.12

What can be the issue with the client?

19

@ibc, sorry by mention you by name.
Could you please help, us with the issue in above post?

20

Assuming that error is in client side (mediasoup-client) I don’t know which params you are passing in transport.consumeData(), but they are probably wrong (“not an object”).

21

By your recommendation, we created data producer based on DirectTransport.
Connected WebRTC data consumer to this data producer.
as defined in API documentation DataProducerOptions there no SctpStreamParameters for DirectTransport.

From exception we undestand that client API require this parameters.

Could you please advise how to resolve it ?