Malformed sound on Pale Moon #6110

pnaf · 2024-01-13T15:57:19Z

What version of Hls.js are you using?

v1.5.1

What browser (including version) are you using?

Pale Moon 32.5.2 (Linux 64-bit GTK3)

What OS (including version) are you using?

Ubuntu 22.04.3 LTS

Test stream

https://test-streams.mux.dev/x36xhzz/url_6/193039199_mp4_h264_aac_hq_7.m3u8

Configuration

{
  "debug": true,
  "enableWorker": true,
  "lowLatencyMode": true,
  "backBufferLength": 90
}

Additional player setup steps

No response

Checklist

The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
The issue occurs in the stable client (latest release) on https://hlsjs.video-dev.org/demo and not just on my page
The issue occurs in the latest client (main branch) on https://hlsjs-dev.video-dev.org/demo and not just on my page
The stream has correct Access-Control-Allow-Origin headers (CORS)
There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

Download and run Pale Moon for Linux (I'm running the GTK3 version).
On Pale Moon, navigate to the demo page, and notice that video and audio play correctly when playing the "Big Buck Bunny - adaptive qualities" or "Big Buck Bunny - 480p only" videos.
Click "Tools", "Preferences", "Advanced". Then, under the "Compatibility" section, set the user agent mode to "Native" or "Gecko Compatibilty" (selecting either of these options merely causes the string Firefox to not appear on the browser's user agent).
Refresh the demo page, and play either video.

Expected behaviour

Audio continues to play correctly.

What actually happened?

Audio plays at an extremely low pitch with crackling. Meanwhile, audio continues to play perfectly fine on all other websites like YouTube and Twitch.

Console output

Using Hls.js config: Object { debug: true, enableWorker: true, lowLatencyMode: true, backBufferLength: 90 }  
hls-demo.js:24433:12
[log] > Debug logs enabled for "Hls instance" in hls.js version 1.5.1  
hls.js:555:24
[log] > stopLoad  
hls.js:28585:14
[log] > loadSource:https://test-streams.mux.dev/x36xhzz/url_6/193039199_mp4_h264_aac_hq_7.m3u8  
hls.js:28552:14
[log] > [stream-controller]: Trigger BUFFER_RESET  
hls.js:27598:12
[log] > attachMedia  
hls.js:28524:14
[log] > [buffer-controller] created media source: MediaSource  
hls.js:18338:14
[log] > [buffer-controller] Media source opened  
hls.js:18223:15
[log] > [level-controller]: manifest loaded, 1 level(s) found, first bitrate: 0  
hls.js:26275:16
[log] > [buffer-controller] 1 bufferCodec event(s) expected  
hls.js:18330:12
[log] > startLoad(-1)  
hls.js:28574:14
[log] > [level-controller]: Switching to level 0 (SDR @0) from level -1  
hls.js:26489:14
[log] > [stream-controller]: STOPPED->IDLE  
hls.js:10668:16
[log] > [subtitle-stream-controller]: STOPPED->IDLE  
hls.js:10668:16
[log] > [stream-controller]: Level 0 loaded [0,63][part-63--1], cc [0, 0] duration:634.6  
hls.js:27644:12
[log] > [buffer-controller] Updating Media Source duration to 634.600  
hls.js:18873:14
[log] > [stream-controller]: Loading fragment 0 cc: 0 of [0-63] level: 0, target: 0  
hls.js:9923:12
[log] > [stream-controller]: IDLE->FRAG_LOADING  
hls.js:10668:16
[log] > injecting Web Worker for "main"  
hls.js:15916:22
[log] > [transmuxer-interface, main]: Starting new transmux session for sn: 0 p: -1 level: 0 id: 1
        discontinuity: true
        trackSwitch: true
        contiguous: false
        accurateTimeOffset: true
        timeOffset: 0
        initSegmentChange: true  
hls.js:16017:16
[log] > [stream-controller]: Loaded fragment 0 of level 0  
hls.js:9657:18
[log] > Debug logs enabled for "main" in hls.js version 1.5.1  
[...]:555:24
[log] > [mp4-remuxer]: ISGenerated flag reset  
hls.js:16125:38
[log] > [mp4-remuxer]: initPTS & initDTS reset  
hls.js:16125:38
[log] > [mp4-remuxer]: reset next timestamp  
hls.js:16125:38
[log] > manifest codec:undefined, ADTS type:2, samplingIndex:4  
hls.js:16125:38
[log] > parsed codec:mp4a.40.5, rate:44100, channels:2  
hls.js:16125:38
[log] > [stream-controller]: FRAG_LOADING->PARSING  
hls.js:10668:16
[log] > [stream-controller]: Init audio buffer, container:audio/mp4, codecs[selected/level/parsed]=[//mp4a.40.5]  
hls.js:28159:14
[log] > [stream-controller]: Init video buffer, container:video/mp4, codecs[level/parsed]=[/avc1.64001f]  
hls.js:28164:14
[log] > [buffer-controller] 0 bufferCodec event(s) expected audio,video  
hls.js:18494:14
[log] > [buffer-controller] creating sourceBuffer(audio/mp4;codecs=mp4a.40.5)  
hls.js:18944:19
[log] > [buffer-controller] creating sourceBuffer(video/mp4;codecs=avc1.64001f)  
hls.js:18944:19
[log] > [audio-stream-controller]: InitPTS for cc: 0 found from main: 900909  
hls.js:16260:14
[log] > [transmuxer.ts]: Flushed fragment 0 of level 0  
hls.js:16125:38
[log] > [stream-controller]: PARSING->PARSED  
hls.js:10668:16
[log] > [stream-controller]: Buffered main sn: 0 of level 0 (frag:[0.000-10.023] > buffer:[0.023-10.008])  
hls.js:9798:12
[log] > [stream-controller]: PARSED->IDLE  
hls.js:10668:16
[log] > [stream-controller]: Loading fragment 1 cc: 0 of [0-63] level: 0, target: 10.008  
hls.js:9923:12
[log] > [stream-controller]: IDLE->FRAG_LOADING  
hls.js:10668:16
[warn] > skipping hole, adjusting currentTime from 0 to 0.1  
hls.js:27120:18
[log] > [stream-controller]: Loaded fragment 1 of level 0  
hls.js:9657:18
[log] > [stream-controller]: media seeking to 0.100, state: FRAG_LOADING  
hls.js:9564:12
[log] > [audio-stream-controller]: media seeking to 0.100, state: STOPPED  
hls.js:9564:12
[log] > [subtitle-stream-controller]: media seeking to 0.100, state: IDLE  
hls.js:9564:12
[log] > [stream-controller]: Media seeked to 0.100  
hls.js:27583:14
[log] > [stream-controller]: FRAG_LOADING->PARSING  
hls.js:10668:16
[log] > [transmuxer.ts]: Flushed fragment 1 of level 0  
hls.js:16125:38
[log] > [stream-controller]: PARSING->PARSED  
hls.js:10668:16
[log] > [stream-controller]: Buffered main sn: 1 of level 0 (frag:[10.008-20.023] > buffer:[0.023-19.992])  
hls.js:9798:12
[log] > [stream-controller]: PARSED->IDLE  
hls.js:10668:16
[log] > [stream-controller]: Loading fragment 2 cc: 0 of [0-63] level: 0, target: 19.992  
hls.js:9923:12
[log] > [stream-controller]: IDLE->FRAG_LOADING  
hls.js:10668:16
[log] > [stream-controller]: Loaded fragment 2 of level 0  
hls.js:9657:18
[log] > [stream-controller]: FRAG_LOADING->PARSING  
hls.js:10668:16
[log] > [transmuxer.ts]: Flushed fragment 2 of level 0  
hls.js:16125:38
[log] > [stream-controller]: PARSING->PARSED  
hls.js:10668:16
[log] > [stream-controller]: Buffered main sn: 2 of level 0 (frag:[19.992-30.023] > buffer:[0.023-30.000])  
hls.js:9798:12
[log] > [stream-controller]: PARSED->IDLE  
hls.js:10668:16
[log] > [stream-controller]: Loading fragment 3 cc: 0 of [0-63] level: 0, target: 30  
hls.js:9923:12
[log] > [stream-controller]: IDLE->FRAG_LOADING  
hls.js:10668:16
[log] > [stream-controller]: Loaded fragment 3 of level 0  
hls.js:9657:18
[log] > [stream-controller]: FRAG_LOADING->PARSING  
hls.js:10668:16
[log] > [transmuxer.ts]: Flushed fragment 3 of level 0  
hls.js:16125:38
[log] > [stream-controller]: PARSING->PARSED  
hls.js:10668:16
[log] > [stream-controller]: Buffered main sn: 3 of level 0 (frag:[30.000-40.023] > buffer:[0.023-40.008])  
hls.js:9798:12
[log] > [stream-controller]: PARSED->IDLE  
hls.js:10668:16
[log] > [stream-controller]: Loading fragment 4 cc: 0 of [0-63] level: 0, target: 40.008  
hls.js:9923:12
[log] > [stream-controller]: IDLE->FRAG_LOADING  
hls.js:10668:16
[log] > [stream-controller]: Loaded fragment 4 of level 0  
hls.js:9657:18
[log] > [stream-controller]: FRAG_LOADING->PARSING  
hls.js:10668:16
[log] > [transmuxer.ts]: Flushed fragment 4 of level 0  
hls.js:16125:38
[log] > [stream-controller]: PARSING->PARSED  
hls.js:10668:16
[log] > [stream-controller]: Buffered main sn: 4 of level 0 (frag:[40.008-50.023] > buffer:[0.023-49.993])  
hls.js:9798:12
[log] > [stream-controller]: PARSED->IDLE  
hls.js:10668:16
[log] > [stream-controller]: Loading fragment 5 cc: 0 of [0-63] level: 0, target: 49.993  
hls.js:9923:12
[log] > [stream-controller]: IDLE->FRAG_LOADING  
hls.js:10668:16
[log] > [stream-controller]: Loaded fragment 5 of level 0  
hls.js:9657:18
[log] > [stream-controller]: FRAG_LOADING->PARSING  
hls.js:10668:16
[log] > [transmuxer.ts]: Flushed fragment 5 of level 0  
hls.js:16125:38
[log] > [stream-controller]: PARSING->PARSED  
hls.js:10668:16
[log] > [stream-controller]: Buffered main sn: 5 of level 0 (frag:[49.993-60.023] > buffer:[0.023-60.000])  
hls.js:9798:12
[log] > [stream-controller]: PARSED->IDLE

Chrome media internals output

No response

The text was updated successfully, but these errors were encountered:

robwalch · 2024-01-13T16:32:47Z

Hi @pnaf,

Click "Tools", "Preferences", "Advanced". Then, under the "Compatibility" section, set the user agent mode to "Native" or "Gecko Compatibilty" (selecting either of these options merely causes the string Firefox to not appear on the browser's user agent).

To confirm:

If the UA does not include "Firefox" when running HLS.js in Pale Moon 32.5.2 (Linux 64-bit GTK3) on Ubuntu 22.04.3 LT, then HE-AAC v1 audio is not played correctly?

After a quick search, I am inclined to think that this issue is related to this workaround:

hls.js/src/demux/audio/adts.ts

Lines 65 to 76 in cd12094

    
           if (/firefox/i.test(userAgent)) { 
        
             if (adtsSamplingIndex >= 6) { 
        
               adtsObjectType = 5; 
        
               config = new Array(4); 
        
               // HE-AAC uses SBR (Spectral Band Replication) , high frequencies are constructed from low frequencies 
        
               // there is a factor 2 between frame sample rate and output sample rate 
        
               // multiply frequency by 2 (see table below, equivalent to substract 3) 
        
               adtsExtensionSamplingIndex = adtsSamplingIndex - 3; 
        
             } else { 
        
               adtsObjectType = 2; 
        
               config = new Array(2); 
        
               adtsExtensionSamplingIndex = adtsSamplingIndex;

~~The same code exists in v1.4 and earlier (in demux/adts.ts) so I assume that this is not a regression.~~

I'm assuming that this workaround is required for the version of Pale Moon you are running or any gecko based user-agent requiring this workaround.

robwalch · 2024-01-13T16:53:08Z

For historical context, this workaround goes way back to the initial M2TS AAC demuxer contribution (#121).

https://github.com/video-dev/hls.js/blame/fa8cbdb73c713434db62ddc747795437299fac03/src/demux/aacdemuxer.js#L111-L129

A feature-detection based alternative to user-agent detection would be nice. Maybe we could test for moz prefixed JavaScript APIs rather than depending on the UA (const isFirefox = 'mozInnerScreenX' in self)?

Meanwhile, audio continues to play perfectly fine on all other websites like YouTube and Twitch.

YouTube and Twitch are not necessarily using AAC SBR (HE-AAC) with less than 24kHz. A better comparison would be other against other MSE based player using exactly the same HLS asset and setting. Does the same thing happen using video.js v8 or shaka-player?

pnaf · 2024-01-14T15:59:28Z

Hi @robwalsh,

If the UA does not include "Firefox" when running HLS.js in Pale Moon 32.5.2 (Linux 64-bit GTK3) on Ubuntu 22.04.3 LT, then HE-AAC v1 audio is not played correctly?

Yes. As far as I can tell, the example streams I mentioned seem to have HE-AAC v1 audio.

YouTube and Twitch are not necessarily using AAC SBR (HE-AAC) with less than 24kHz. A better comparison would be other against other MSE based player using exactly the same HLS asset and setting. Does the same thing happen using video.js v8 or shaka-player?

I'm not hearing this sound issue with any of the Video.js demos or any of the MPEG-TS demos for Shaka Player.

In fact, I played both the "adaptive qualities" and "480p only" versions of Big Buck Bunny from the HLS.js demo page on Shaka Player, and the audio from both videos played perfectly fine. Just to double-check, I played the same Big Buck Bunny videos using HLS.js and Shaka Player on different tabs, and HLS.js consistently continued to play malformed sound.

#6111 fixes the issue locally on my end, although I absolutely agree that we should switch to a feature-detection method at some time in the future.

robwalch · 2024-01-14T16:55:07Z

I'm not hearing this sound issue with any of the Video.js demos or any of the MPEG-TS demos for Shaka Player.

In fact, I played both the "adaptive qualities" and "480p only" versions of Big Buck Bunny from the HLS.js demo page on Shaka Player, and the audio from both videos played perfectly fine. Just to double-check, I played the same Big Buck Bunny videos using HLS.js and Shaka Player on different tabs, and HLS.js consistently continued to play malformed sound.

If that's the case we should investigate what the difference in remuxed output is - there may be a fix that removes the workaround.

robwalch · 2024-08-09T20:51:53Z

Hi @pnaf. I have a change that removes the user-agent specific workarounds in #6627. I would like to know if this resolves the malformed sound issues you've been experiencing. You can try it here: https://bugfix-adts-to-esds.hls-js-4zn.pages.dev/demo/

pnaf · 2024-08-28T13:45:46Z

Hi @robwalch. Sorry, I just saw your message now.

Yes, on Pale Moon 33.3.0, I can see that the issue no longer occurs on the v1.5.14-pr.bugfix-adts-to-esds.4e9516d2 demo page.

Thank you!

pnaf added Bug Needs Triage If there is a suspected stream issue, apply this label to triage if it is something we should fix. labels Jan 13, 2024

robwalch added browser: Firefox Browser issue If there is an underlying issue with the browser that hls.js is running on, this tag should be used. and removed Needs Triage If there is a suspected stream issue, apply this label to triage if it is something we should fix. labels Jan 13, 2024

pnaf mentioned this issue Jan 14, 2024

Use AAC SBR (HE-AAC) workaround on Pale Moon #6111

Merged

3 tasks

robwalch added this to the 1.6.0 milestone Jan 22, 2024

robwalch mentioned this issue May 16, 2024

Media not buffered (no AV overlap) #6252

Closed

5 tasks

robwalch added the Verify Fixed An unreleased bug fix has been merged and should be verified before closing. label Aug 8, 2024

robwalch mentioned this issue Aug 9, 2024

ADTS to esds remuxing enhancements #6627

Merged

3 tasks

robwalch closed this as completed in #6627 Aug 27, 2024

robwalch mentioned this issue Sep 5, 2024

Hls Audio Stream sounds different on Firefox and Chrome #2686

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Malformed sound on Pale Moon #6110

Malformed sound on Pale Moon #6110

pnaf commented Jan 13, 2024

robwalch commented Jan 13, 2024 •

edited

Loading

robwalch commented Jan 13, 2024 •

edited

Loading

pnaf commented Jan 14, 2024

robwalch commented Jan 14, 2024

robwalch commented Aug 9, 2024

pnaf commented Aug 28, 2024

Malformed sound on Pale Moon #6110

Malformed sound on Pale Moon #6110

Comments

pnaf commented Jan 13, 2024

What version of Hls.js are you using?

What browser (including version) are you using?

What OS (including version) are you using?

Test stream

Configuration

Additional player setup steps

Checklist

Steps to reproduce

Expected behaviour

What actually happened?

Console output

Chrome media internals output

robwalch commented Jan 13, 2024 • edited Loading

robwalch commented Jan 13, 2024 • edited Loading

pnaf commented Jan 14, 2024

robwalch commented Jan 14, 2024

robwalch commented Aug 9, 2024

pnaf commented Aug 28, 2024

robwalch commented Jan 13, 2024 •

edited

Loading

robwalch commented Jan 13, 2024 •

edited

Loading