Duplicate Captions for MP4 Content

[wildg-ensemble]

What version of Hls.js are you using?

1.5.8

What browser (including version) are you using?

Chrome 124.0.6367.93 (Official Build) (arm64)

What OS (including version) are you using?

MacOS Ventura (13.5)

Test stream

No response

Configuration

{
  debug: true,
  autoStartLoad: false
}

Additional player setup steps

N/A

Checklist

• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
• The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. Play MP4 VOD content with at least one subtitle option
2. Enable subtitles

Expected behaviour

There should only be one set of subtitles

What actually happened?

Subtitles are duplicated. The issue seems to stem from changes between hls.js v1.4.14 to v1.5 (Change logs). Playing content on v1.4.14 does not have duplicate captions

Console output

HlsPlayerInstance.ts:16 [log] > Debug logs enabled for "Hls instance" in hls.js version 1.5.8
HlsPlayerInstance.ts:133 [log] > attachMedia
HlsPlayerInstance.ts:133 [log] > [buffer-controller] created media source: MediaSource
hls.mjs:174 [log] > [buffer-controller] Media source opened
HlsPlayerInstance.ts:63 [log] > stopLoad
HlsPlayerInstance.ts:63 [log] > loadSource:[REDACTED]
HlsPlayerInstance.ts:63 [log] > [stream-controller]: Trigger BUFFER_RESET
hls.mjs:174 [log] > [level-controller]: manifest loaded, 3 level(s) found, first bitrate: 1614036
hls.mjs:174 [log] > setting initial bwe to 1614036
hls.mjs:174 [log] > [buffer-controller] 2 bufferCodec event(s) expected
HlsPlayerInstance.ts:44 [log] > startLoad(41.286797)
HlsPlayerInstance.ts:44 [log] > [abr] picked start tier {"codecSet":"avc1,mp4a","videoRanges":["SDR"],"preferHDR":false,"minFramerate":0,"minBitrate":1614036}
HlsPlayerInstance.ts:44 [info] > [abr] switch candidate:0->0 adjustedbw(1614036)-bitrate=0 ttfb:0.1 avgDuration:0.0 maxFetchDuration:4.0 fetchDuration:0.1 firstSelection:true codecSet:avc1,mp4a videoRange:SDR hls.loadLevel:-1
HlsPlayerInstance.ts:44 [log] > [level-controller]: Switching to level 0 (540p SDR avc1,mp4a @1614036) from level -1
HlsPlayerInstance.ts:44 [log] > [audio-track-controller]: Updating audio tracks, 2 track(s) found in group(s): program_audio
HlsPlayerInstance.ts:128 [warn] > [audio-track-controller]: Invalid audio track id: -1
HlsPlayerInstance.ts:44 [log] > [audio-track-controller]: Switching to audio-track 0 "Default Audio" lang:eng group:program_audio channels:undefined
HlsPlayerInstance.ts:44 [log] > [audio-stream-controller]: Reset loading state
HlsPlayerInstance.ts:44 [log] > [audio-stream-controller]: STOPPED->IDLE
HlsPlayerInstance.ts:44 [log] > [audio-stream-controller]: IDLE->STOPPED
HlsPlayerInstance.ts:44 [log] > [subtitle-track-controller]: Updating subtitle tracks, 1 track(s) found in "subs" group-id
HlsPlayerInstance.ts:182 [log] > [subtitle-track-controller]: Switching to subtitle-track 0 "English" lang:en group:subs
HlsPlayerInstance.ts:44 [log] > [level-controller]: Loading level index 0 with [REDACTED]
HlsPlayerInstance.ts:44 [log] > [stream-controller]: STOPPED->IDLE
HlsPlayerInstance.ts:44 [log] > [audio-stream-controller]: STOPPED->WAITING_TRACK
HlsPlayerInstance.ts:44 [log] > [subtitle-track-controller]: Loading subtitle playlist for id 0
HlsPlayerInstance.ts:44 [log] > [subtitle-stream-controller]: STOPPED->IDLE
hls.mjs:174 [log] > [subtitle-track-controller]: Subtitle track 0 "English" lang:en group:subs loaded [1-17]
hls.mjs:174 [log] > [subtitle-stream-controller]: Subtitle track 0 loaded [1,17][part-17--1],duration:2697.6519999999996
hls.mjs:174 [log] > [subtitle-stream-controller]: Loading fragment 2 cc: 1 of [1-17] track: 0, target: 41.287
hls.mjs:174 [log] > [subtitle-stream-controller]: IDLE->FRAG_LOADING
hls.mjs:174 [log] > [stream-controller]: Level 0 loaded [1,441][part-441--1], cc [0, 9] duration:2697.571999999994
hls.mjs:174 [log] > [buffer-controller] Updating Media Source duration to 2697.572
hls.mjs:174 [log] > [stream-controller]: Loading fragment 5 cc: 1 of [1-441] level: 0, target: 41.287
hls.mjs:174 [log] > [stream-controller]: IDLE->FRAG_LOADING
hls.mjs:174 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 5 p: -1 level: 0 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 36.036
        initSegmentChange: true
hls.mjs:174 [log] > [mp4-remuxer]: ISGenerated flag reset
hls.mjs:174 [log] > [mp4-remuxer]: initPTS & initDTS reset
hls.mjs:174 [log] > [mp4-remuxer]: reset next timestamp
hls.mjs:174 [log] > manifest codec:mp4a.40.2,mp4a.40.5, ADTS type:2, samplingIndex:4
hls.mjs:174 [log] > parsed codec:mp4a.40.2, rate:44100, channels:2
hls.mjs:174 [log] > [stream-controller]: FRAG_LOADING->PARSING
hls.mjs:174 [log] > [stream-controller]: Android: force audio codec to mp4a.40.2
hls.mjs:174 [log] > [stream-controller]: Swapping manifest audio codec "mp4a.40.2,mp4a.40.5" for "mp4a.40.2"
hls.mjs:174 [log] > [stream-controller]: Init audio buffer, container:audio/mp4, codecs[selected/level/parsed]=[mp4a.40.2/mp4a.40.2,mp4a.40.5/mp4a.40.2]
hls.mjs:174 [log] > [stream-controller]: Init video buffer, container:video/mp4, codecs[level/parsed]=[avc1.4d401f/avc1.4d401f]
hls.mjs:174 [log] > [buffer-controller] 1 bufferCodec event(s) expected audio,video
hls.mjs:174 [log] > [buffer-controller] creating sourceBuffer(audio/mp4;codecs=mp4a.40.2)
hls.mjs:174 [log] > [buffer-controller] creating sourceBuffer(video/mp4;codecs=avc1.4d401f)
hls.mjs:174 [log] > [audio-stream-controller]: InitPTS for cc: 1 found from main: -2514072
hls.mjs:174 [log] > [stream-controller]: Loaded fragment 5 of level 0
hls.mjs:174 [log] > [transmuxer.ts]: Flushed fragment 5 of level 0
hls.mjs:174 [log] > [stream-controller]: PARSING->PARSED
hls.mjs:174 [log] > [stream-controller]: Buffered main sn: 5 of level 0 (frag:[36.036-42.045] > buffer:[36.054-42.042])
hls.mjs:174 [log] > [stream-controller]: PARSED->IDLE
hls.mjs:174 [log] > [stream-controller]: seek to target start position 41.286797 from current time 0
hls.mjs:174 [info] > [abr] switch candidate:0->2 adjustedbw(114460416)-bitrate=111373181 ttfb:0.0 avgDuration:6.0 maxFetchDuration:0.8 fetchDuration:0.2 firstSelection:false codecSet:avc1,mp4a videoRange:SDR hls.loadLevel:0
hls.mjs:174 [log] > [stream-controller]: Adapting to level 2 from level 0
hls.mjs:174 [log] > [level-controller]: Switching to level 2 (720p SDR avc1,mp4a @3087235) from level 0
hls.mjs:174 [log] > [level-controller]: Loading level index 2 with [REDACTED]
hls.mjs:174 [log] > [stream-controller]: IDLE->WAITING_LEVEL
hls.mjs:174 [log] > [stream-controller]: media seeking to 41.287, state: WAITING_LEVEL
hls.mjs:174 [log] > [audio-stream-controller]: media seeking to 41.287, state: WAITING_TRACK
hls.mjs:174 [log] > [subtitle-stream-controller]: media seeking to 41.287, state: FRAG_LOADING
hls.mjs:174 [log] > [subtitle-stream-controller]: Loaded fragment 2 of level 0
hls.mjs:174 [log] > [subtitle-stream-controller]: FRAG_LOADING->IDLE
hls.mjs:174 [log] > [subtitle-stream-controller]: Buffered subtitle sn: 2 of track 0 (frag:[NaN-NaN] > buffer:[30.030-330.030])
hls.mjs:174 [log] > [stream-controller]: Media seeked to 41.287
hls.mjs:174 [log] > [stream-controller]: Level 2 loaded [1,441][part-441--1], cc [0, 9] duration:2697.571999999994
hls.mjs:174 [log] > [stream-controller]: WAITING_LEVEL->IDLE
hls.mjs:174 [log] > [stream-controller]: Loading fragment 6 cc: 1 of [1-441] level: 2, target: 42.042
hls.mjs:174 [log] > [stream-controller]: IDLE->FRAG_LOADING
hls.mjs:174 [log] > [transmuxer-interface, main]: Starting new transmux session for sn: 6 p: -1 level: 2 id: 1
        discontinuity: false
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 42.042
        initSegmentChange: false
hls.mjs:174 [log] > [mp4-remuxer]: ISGenerated flag reset
hls.mjs:174 [log] > [mp4-remuxer]: initPTS & initDTS reset
hls.mjs:174 [log] > [mp4-remuxer]: reset next timestamp
hls.mjs:174 [log] > manifest codec:mp4a.40.2,mp4a.40.5, ADTS type:2, samplingIndex:4
hls.mjs:174 [log] > parsed codec:mp4a.40.2, rate:44100, channels:2
hls.mjs:174 [log] > [stream-controller]: FRAG_LOADING->PARSING
hls.mjs:174 [log] > [stream-controller]: Android: force audio codec to mp4a.40.2
hls.mjs:174 [log] > [stream-controller]: Swapping manifest audio codec "mp4a.40.2,mp4a.40.5" for "mp4a.40.2"
hls.mjs:174 [log] > [stream-controller]: Init audio buffer, container:audio/mp4, codecs[selected/level/parsed]=[mp4a.40.2/mp4a.40.2,mp4a.40.5/mp4a.40.2]
hls.mjs:174 [log] > [stream-controller]: Init video buffer, container:video/mp4, codecs[level/parsed]=[avc1.4d401f/avc1.4d401f]
hls.mjs:174 [log] > [stream-controller]: Loaded fragment 6 of level 2
hls.mjs:174 [log] > [transmuxer.ts]: Flushed fragment 6 of level 2
hls.mjs:174 [log] > [stream-controller]: PARSING->PARSED
hls.mjs:174 [log] > [stream-controller]: Buffered main sn: 6 of level 2 (frag:[42.042-48.059] > buffer:[36.054-48.048])
hls.mjs:174 [log] > [stream-controller]: PARSED->IDLE
hls.mjs:174 [log] > [stream-controller]: Loading fragment 7 cc: 1 of [1-441] level: 2, target: 48.048
hls.mjs:174 [log] > [stream-controller]: IDLE->FRAG_LOADING
hls.mjs:174 [log] > [stream-controller]: FRAG_LOADING->PARSING
hls.mjs:174 [log] > [stream-controller]: Loaded fragment 7 of level 2
hls.mjs:174 [log] > [transmuxer.ts]: Flushed fragment 7 of level 2
hls.mjs:174 [log] > [stream-controller]: PARSING->PARSED
hls.mjs:174 [log] > [stream-controller]: Buffered main sn: 7 of level 2 (frag:[48.048-54.073] > buffer:[36.054-54.054])
hls.mjs:174 [log] > [stream-controller]: PARSED->IDLE

Chrome media internals output

No response

[robwalch]

Please provide a link to the HLS content (subtitle playlist and segments) that reproduce this issue.

[robwalch]

According to the logs only one subtitle playlist and segment was loaded, so this could very well be an issue with the WebVTT content. We cannot confirm or troubleshoot without sample content.

[jcekstrom]

@robwalch I can reproduce it. I'll try and get a video together that demonstrates it. Might be the presence of 608 and WebVTT captions both being present.

[robwalch]

We have test assets with both 608 and WebVTT subtitles. They each get their own respective TextTracks which are not enabled at the same time.

[robwalch]

@jcekstrom,

I can reproduce it.

The label cannot reproduce means that the issue does not include the information or assets needed to reproduce the issue (or root cause and fix it).

I'll try and get a video together that demonstrates it

Please do. That is exactly what we need ( Need sample stream ). Thanks!

[robwalch]

@wildg-ensemble,

The issue seems to stem from changes between hls.js v1.4.14 to v1.5 (Change logs). Playing content on v1.4.14 does not have duplicate captions

I did look through all the changes between releases and could not find anything that jumped out as a root cause. A repro asset is critical to bisecting build changes and finding the root cause.

[jcekstrom]

I played around with the manifests... It looks like the issue is tied to having

#EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS,GROUP-ID="CC",LANGUAGE="eng",NAME="english",INSTREAM-ID="CC1"
#EXT-X-MEDIA:TYPE=SUBTITLES,NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="eng",GROUP-ID="subs",URI="master_caption.m3u8"

Where the caption names are the same but not. Notice that the CC NAME="english", while the VTT NAME="English". If I change the name of the VTT to "eng_vtt", I have 2 choices for captions show up in the drop down, and I no longer get dual captions.

So to reproduce, we just need to create an asset with both VTT and 608 captions, and label them with only a case difference.

[robwalch]

Can you share the output of video.textTracks?

[robwalch]

With the same playlist, when I open the HTMLVideoElement captions menu in Chrome or Safari, I am presented with the options "Off", "English", "Unknown CC", and "Spanish". The last two options are the 608 captions. Picking "English" only enables/shows the subtitle TextTrack, but I don't have to pick it because it is the default track (DEFAULT=YES,AUTOSELECT=YES).

So to reproduce, we just need to create an asset with both VTT and 608 captions, and label them with only a case difference.

That doesn't work for me. I'm not sure how you are getting 608 captions enabled this way with 1.5.8. Please provide a page that reproduces the issue.

Only one track is showing and the NAME of the CC item is not applied and does not impact selection using the media element controls or hls.subtitleTrack = index:

> video.textTracks
[
  {
    activeCues: TextTrackCueList {length: 0, item: function, getCueById: function}
    cues: TextTrackCueList {0: VTTCue, 1: VTTCue, 2: VTTCue, 3: VTTCue, 4: VTTCue, 5: VTTCue, 6: VTTCue, 7: VTTCue, 8: VTTCue, 9: VTTCue, …}
    id: ""
    inBandMetadataTrackDispatchType: ""
    kind: "subtitles"
    label: "English"
    language: "eng"
    mode: "showing"
  },
  {
    activeCues: TextTrackCueList {length: 0, item: function, getCueById: function}
    cues: TextTrackCueList {0: VTTCue, 1: VTTCue, 2: VTTCue, 3: VTTCue, 4: VTTCue, 5: VTTCue, length: 6, item: function, getCueById: function}
    id: ""
    inBandMetadataTrackDispatchType: ""
    kind: "captions"
    label: "Unknown CC"
    language: ""
    mode: "hidden"
  },
  {
    activeCues: TextTrackCueList {length: 0, item: function, getCueById: function}
    cues: TextTrackCueList {0: VTTCue, 1: VTTCue, 2: VTTCue, 3: VTTCue, 4: VTTCue, 5: VTTCue, 6: VTTCue, 7: VTTCue, 8: VTTCue, 9: VTTCue, …}
    id: ""
    inBandMetadataTrackDispatchType: ""
    kind: "captions"
    label: "Spanish"
    language: "es"
    mode: "hidden"
}

[robwalch]

Are you using another player API or custom controls on top of HLS.js? HLS.js does not use TYPE=CLOSED-CAPTIONS to label TextTracks it creates or filter them (#4920). It only uses CLOSED-CAPTIONS=NONE on variants to disable 608 parsing for an item.

[robwalch]

Hi @jcekstrom,

You should use a different NAME attribute value for captions to differentiate them from subtitles. As another workaround you could also try adding the characteristics attribute to the #EXT-X-MEDIA:TYPE=CLOSED-CAPTIONS
tag (even though it is redundant): CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog,public.accessibility.describes-music-and-sound".

I've posted a fix with #6440. Please let me know if it resolves the issue for you.

[wildg-ensemble]

Hey @robwalch, apologies for the late response. I was unable to find another asset that would re-produce the issue. However, it seems #6440 fixed the issue on my end, thank you for your help!