Safari local video track issue

[rickary]

• I have verified that the issue occurs with the latest twilio-video.js release and is not marked as a known issue in the CHANGELOG.md.
• I reviewed the Common Issues and open GitHub issues and verified that this report represents a potentially new issue.
• I verified that the Quickstart application works in my environment. - SEE BELOW
• I am not sharing any Personally Identifiable Information (PII)
  or sensitive account information (API keys, credentials, etc.) when reporting this issue.

Issue noticed in our React app where Safari 15 users (any device) were not able to see their local video feed (just seeing a black screen instead). This is happening both when we use createLocalTracks and publications taken from the rooms. The other user(s) see the published feed ok and the recordings for the room are fine. It’s just affecting the local participant. Audio is not affected.

Found this issue https://developer.apple.com/forums/thread/690523 which appears related however both of the suggested 'fixes' don’t resolve the issue. Instead of seeing a black screen the first frame of the video is rendered but then freezes.

Note that this doesn’t happen 100% of the time. We are seeing the issue approx 25-30% of the time (more frequently on iOS devices than on mac os).

Flipping the camera or turning the video on/off seems to resolve the issue. Not seeing any console errors/warnings.

We have been able to replicate the issue using video-quickstart-js where the remote video plays but we just see a black screen for the local video however on the second device you can see both videos.

Unsure whether this is another issue with Safari 15. Any investigation appreciated.

Software versions:

• Browser(s): Safari 15.3
• Operating System: ios/mac os
• twilio-video.js: 2.20.1 (have just upgraded from 2.18.3 which had the same issue)
• Third-party libraries (e.g., Angular, React, etc.): React

[makarandp0]

Hello @rickary, Thank you for reporting this - We are not aware of any open issues that would prevent local video track from rendering. Does this issue repro even when the participant does not join the room?

Given that the track is seen by remote participants, looks like nothing is wrong with the video track itself.

Can you post any client side debug logs when the issue repros which may also give us some clues.

Thanks,
Makarand

Thanks,
Makarand

[rickary]

Hi thanks for reply.

Yes we are seeing the issue with the local video preview as well as the track. As above, it’s not happening every time. Sometimes we can see an issue with the local preview but then when we enter the room the video is fine. Sometimes vice-versa. And sometimes no issues at all.

Our app is one-way video so we cloned https://github.com/twilio-labs/function-templates/tree/main/video and ran that to see how two-way video is affected and was able to reproduce (safari: remote video ok, local video black. Other browser: both video feeds ok).

It doesn’t seem anything is wrong with the track itself or the Twilio connection but with the <video> element maybe?

What logs specifically may be useful? I’m not seeing any errors/warnings from Twilio.

[gavinlefebvre]

I think we ran into something similar- I attached a callback to the <video> element loadstart event to check the LocalVideoTrack's MediaStreamTrack muted state, since starting with Safari 15 the WebKit engine has really leaned into that property, even though they're somewhat inconsistent firing the mute and unmute events.

[benmesander]

I have reproduced this issue with the Twilio demo video application with room id RM0679f3946646354b3d8e9f6304863336 - I also made a mp4 recording of the issue. I will try opening a zendesk ticket as well. My device is an Apple iPhone 7, running iOS 15.3.1, using Safari browser.

twilio-black-screen.mp4

[YenTingWu]

Same issue here. In some cases, I can't even reopen the local video feed by toggling the video button. Other users are still seeing my video feed on their devices.

room id: RM19d8ec3a5d44754764844199bb00b437
device: iphone8
os: ios 15.3.1
browser: chrome 100

[shaibt]

Also experiencing something similar to what @rickary is describing:
Safari iOS 15.X, user creates local tracks w/o issue and later on when connecting to the room (previously acquired tracks provided in tracks param of connect()) , in ~25% of cases, the local video preview is grey. video track is streaming and is published - remote participants in the room are seeing the video. Just the local video preview is not playing.
When local user restarts their video (stop->unpublish->reacquire->publish) no problem.
Seems like a timing issue w/ the <video> element and the sequence of connecting to the room.

[douglasOutboxup]

I'm also experiencing something similar.
I have a room for just 2 participants and the local track is just visible when the remote track is disabled or not connected.
The issue is occurring both on iOS and MacOS Safari.

[timmydoza]

Hey Everybody!

Just wanted to make sure that everybody here saw the possible workaround for this issue that we have found. You can see my comment in this other ticket here: #1775 (comment)

What we found, is that Safari can occasionally fail to display a local video track if this CSS is used to mirror the video: transform: rotateY(180deg);. I'm not sure why Safari has issues with this CSS, but it seems to be a problem that lies with the Safari browser and not the Twilio Video SDK.

The workaround is to use a simpler 2d transform (rotateY() is a 3d transform) instead: transform: scaleX(-1);.

If you are using our Twilio Video React App, please note that this workaround has been included in version 0.6.5 which was released today.

In our internal testing, we found that this completely solves the issue in the most recent version of desktop and iOS Safari (version 15.5), although we have heard from some users that the issue can still be seen periodically.

If you are using CSS to mirror your local video track, please try the workaround above and let us know if it helps! We're curious to hear about what results you see. If you find that the issue still persists, please try to give us as much detail as you can about the steps that you are taking to reproduce the issue, in addition to the specific browser version(s) that you are using.

Thanks!!

[vbabenko]

We applied the workaround yesterday and at the current moment we already see a complaint that the issue is reproduced for iPad Chrome. Potentially, it's also reproduced for the Safari browser...

Room SID: RM3fe29f6671b476d5e022b28b0bfdf466
Device: iPad

In our case, it reproduced when we change the video layout from one type to another. If the user repeats manipulation again it helps to restore the video

2022-08-04T18:33:02.342Z warn [LocalAudioTrack #1: 72c5320c-d720-4299-a024-05454a26b027] Failed to detect silence: AbortError: The operation was aborted.
2022-08-04T18:33:02.343Z warn [LocalAudioTrack #1: 72c5320c-d720-4299-a024-05454a26b027] Unintentionally paused: [object HTMLAudioElement]
2022-08-04T18:33:02.344Z warn [LocalAudioTrack #1: 72c5320c-d720-4299-a024-05454a26b027] Unintentionally paused: [object HTMLAudioElement]

[cindyloo]

I'm also seeing this on Mac iOS 15.5 Safari

[JosueRodriguezDev]

We've seen this issue on iOS safari as well

We applied the workaround yesterday and at the current moment we already see a complaint that the issue is reproduced for iPad Chrome. Potentially, it's also reproduced for the Safari browser...

Room SID: RM3fe29f6671b476d5e022b28b0bfdf466 Device: iPad

In our case, it reproduced when we change the video layout from one type to another. If the user repeats manipulation again it helps to restore the video

2022-08-04T18:33:02.342Z warn [LocalAudioTrack #1: 72c5320c-d720-4299-a024-05454a26b027] Failed to detect silence: AbortError: The operation was aborted.
2022-08-04T18:33:02.343Z warn [LocalAudioTrack #1: 72c5320c-d720-4299-a024-05454a26b027] Unintentionally paused: [object HTMLAudioElement]
2022-08-04T18:33:02.344Z warn [LocalAudioTrack #1: 72c5320c-d720-4299-a024-05454a26b027] Unintentionally paused: [object HTMLAudioElement]