Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to handle 2-way video and identify incoming tracks #1253

Open
fusionfoxy opened this issue Sep 12, 2024 · 0 comments
Open

how to handle 2-way video and identify incoming tracks #1253

fusionfoxy opened this issue Sep 12, 2024 · 0 comments

Comments

@fusionfoxy
Copy link
Contributor

I am streaming video between a browser and libdatachannel in a scenario where multiple video feeds may exist on both sides and I am having trouble identifying incoming video tracks.

Scenario:

  • Libdatachannel side calls addTrack() with a video track with SSRC: 42, MSID: "foo"
  • Negotiation happens and browser gets onTrack event
  • Browser sees MSID: "foo" and can know which of the feeds available on the libdatachannel side this is.
  • Some time later, the browser calls addTrack() with a video track with SSRC: 24, MSID: "bar"
  • Negotiation happens, the browser reuses the m= section from the existing track
  • onTrack is not triggered on libdatachannel side, existing rtc::Track simply starts receiving RTP data (in addition to the RTCP from its own track). The existing rtc::Track does not seem to provide me any insights into the SSRC added by the browser or its msid. This means the libdatachannel side cannot identify that it is the "bar" feed the browser is sending.

Questions*

rtc::Track seems to act like Transceivers do in the browser in that it seems to represent an entire m= section in the SDP. The onMessage callback receives data from both video tracks. It also kind of acts like a browser MediaStreamTrack in that you can use it with an existing mid with addTrack() to add an SSRC to an existing m= section.

  1. Is there no clear distinction between tracks and transceivers in libdatachannel?
  2. It seems like onTrack should be triggered when the browser adds its track or am I missing something?
  3. If onTrack should be triggered, would it be invoked with another rtc::Track instance representing the received feed(meaning the contained SSRCs would be those of the browser)?
  4. When adding the browser track, how can i get the msid of the browser track on the libdatachannel side?
  5. Calling addSSRC(42, "foo", "bar", "baz") will produce an offer with ssrc: 42, cname: "foo", msid: "bar baz" as I would expect. This makes me believe cname and msid is handled properly by libdatachannel. Is this only true when sending tracks, not when receiving?

Side notes
When browser feeds are created, the browser picks the msid randomly. I am conveying its meaning separately allowing me to assign meaning to the MSID "bar" value from the scenario.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant