macOS decoding error: PIPELINE_ERROR_DECODE: VDA Error 4 #5632
Comments
iameli
added
Bug
Needs Triage
|
Here's the transmuxed fMP4 dumped from the player. Attempting to decode that with VideoToolbox-accelerated ffmpeg causes problems. And then that last part repeats a bunch. Full ffmpeg debug log if that's helpful. Remuxing the source m3u8 doesn't cause any issues at all: |
|
Duplicated MOOV and SDTP atoms? What's in that TS? |
|
@robwalch It's just these three files: The only unusual thing I can think of there is that there are three tracks — H264, AAC, and Vorbis audio. |
|
The first and last segment play OK. Early in the playback (and decoding) of the second segment, is when the error occurs. All three have the opus track. This should be ignored completely. There's a DVB stream type of 6 that HLS.js is ignoring. I dumped the fmp4 output for the second segment and was able to play the audio and video files with QuickTime. No errors when run through ffprobe (5.1.1). I can reproduce the pipeline error in Chrome: |
|
Here's the AVC mp4 which should be playable from 5-10s (0-5 and 10-15 empty): video-2023-07-12T01_46_32.359Z.mp4Let me know if there's an issue with the muxing of any of the samples or timestamps. I don't see the problem, but happy to work on a fix if someone can identify it in this example. |
|
Huh... interesting! In that video, I'm noticing that the first second of data claims to contain 59 frames, and that the 60th frame is twice as long in duration as the others. EDIT: And I cross-referenced with the original TS data (to be clear, above is about the video in the message preceding this message) for the same time period, and that data does contain 60 distinct timestamps of equal length in the first GOP. So... yeah, something in the muxer is somehow combining that last frame of the GOP with the first frame of the next and considering it a single frame. Hopefully that's enough to figure out why it's happening? |
I can't image... Using config: load the sample stream in the description and break/step through hls.js/src/remux/mp4-remuxer.ts Line 413 in 70e2c34 It's counting 299 input samples with 836 NAL units. Sample duration handled here and each sample has a duration of (1440 or 1530 / 90000): hls.js/src/remux/mp4-remuxer.ts Lines 585 to 597 in 70e2c34 I don't see how frames would get combined by the remuxer. Maybe the issue is in tsdemuxer > avc-video-parser when parsing AVC NAL units: |
|
Yeah, this is almost certainly indeed a problem with the avc-video-parser... Note that the second GOP's first frame does not start with a AUD NAL unit, while every other frame in the TS data does. It seems the NVENC encoder occasionally neglects to insert an AUD before some keyframes, but otherwise always does it. The logic seems to assume AUDs are either always or never there, and that assumption does not hold for this stream. 🤔 When an AUD is seen, this logic happens before creating the new frame: hls.js/src/demux/video/avc-video-parser.ts Lines 164 to 166 in b84baee But, when a AUD is missing, instead this logic happens, and the existing frame data is never pushed: Copying those three lines over to here should resolve the issue, as in the presence of a keyframe you can safely assume any remaining frame data waiting to be pushed belongs to a separate frame. On that note, there's another case that should probably also use this logic, when a type-1 NAL unit is a keyframe, here: (does not apply to this stream specifically, but the same thing could happen here for a theoretical other stream) |
|
Went ahead and made a PR for my suggested solution. 👍 |
|
Confirmed that @Thulinma's branch fixes the problem for me! Here's a built version of the player that plays back smoothly. EDIT: Published to npm as |
…s them to be merged with their preceding frame
… be merged with their preceding frame (#5660)
robwalch
added
Confirmed
and removed
Needs Triage
|
Thanks for the bug report and the fix. Changes are available in dev and in v1.4.9: |
What version of Hls.js are you using?
v1.4.7
What browser (including version) are you using?
Latest Brave & Firefox on macOS
What OS (including version) are you using?
macOS Ventura 13.3.1 (a)
Test stream
https://docs-livepeer-live-cors.livepeer.workers.dev/eli/966061CB-D79D-46A6-996C-699D87FD4D2D/hls-playback-problem/problem.m3u8
Configuration
Additional player setup steps
Seems to break in all cases where macOS hardware decoding is involved. Using software decoding in an Ubuntu VM, I get a hiccup in the video but nothing fatal. On Windows using NVDEC I get completely clean playback; this is consistent with the original encoder being NVENC.
Checklist
Steps to reproduce
Just playback the stream at https://docs-livepeer-live-cors.livepeer.workers.dev/eli/966061CB-D79D-46A6-996C-699D87FD4D2D/hls-playback-problem/problem.m3u8
Expected behaviour
Clean playback
What actually happened?
Console output
Chrome media internals output
{ "properties": { "render_id": 1655, "player_id": 11, "created": "2023-07-06 17:55:57.476 UTC", "origin_url": "https://hlsjs.video-dev.org/", "kFrameUrl": "https://hlsjs.video-dev.org/demo/?src=https%3A%2F%2Ftest-streams.mux.dev%2Fx36xhzz%2Fx36xhzz.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsInN0b3BPblN0YWxsIjpmYWxzZSwiZHVtcGZNUDQiOmZhbHNlLCJsZXZlbENhcHBpbmciOi0xLCJsaW1pdE1ldHJpY3MiOi0xfQ==", "kFrameTitle": "hls.js demo", "url": "blob:https://hlsjs.video-dev.org/07aac4ef-1f4e-47bd-9e17-c1076ad0679b", "kTextTracks": [], "info": "Effective playback rate changed from 0 to 1", "kRendererName": "RendererImpl", "pipeline_state": "kStopped", "kAudioTracks": [ { "bytes per channel": 2, "bytes per frame": 4, "channel layout": "STEREO", "channels": 2, "codec": "aac", "codec delay": 0, "discard decoder delay": false, "encryption scheme": "Unencrypted", "has extra data": false, "profile": "unknown", "sample format": "Signed 16-bit", "samples per second": 48000, "seek preroll": "0us" } ], "kVideoTracks": [ { "alpha mode": "is_opaque", "codec": "h264", "coded size": "3840x2160", "color space": { "matrix": "BT709", "primaries": "BT709", "range": "LIMITED", "transfer": "BT709" }, "encryption scheme": "Unencrypted", "has extra data": false, "hdr metadata": "unset", "natural size": "3840x2160", "orientation": "0°", "profile": "h264 high", "visible rect": "0,0 3840x2160" } ], "kIsAudioDecryptingDemuxerStream": false, "kAudioDecoderName": "FFmpegAudioDecoder", "kIsPlatformAudioDecoder": false, "kIsVideoDecryptingDemuxerStream": false, "kVideoDecoderName": "VDAVideoDecoder", "kIsPlatformVideoDecoder": true, "debug": "ISO-BMFF container metadata for video frame indicates that the frame is a keyframe, but the video frame contents indicate the opposite.", "seek_target": 5.810637, "dimensions": "3840x2160", "kResolution": "3840x2160", "event": "kPause", "error": { "cause": { "code": 1, "data": {}, "group": "DecoderStatus", "message": "", "stack": [ { "file": "media/gpu/ipc/service/vda_video_decoder.cc", "line": 919 } ] }, "code": 3, "data": {}, "group": "PipelineStatus", "message": "", "stack": [ { "file": "media/renderers/video_renderer_impl.cc", "line": 591 } ] }, "duration": 15.007999, "pipeline_buffering_state": { "for_suspended_start": false, "state": "BUFFERING_HAVE_ENOUGH" } }, "events": [ { "time": 0, "key": "created", "value": "2023-07-06 17:55:57.476 UTC" }, { "time": 0.1940000057220459, "key": "origin_url", "value": "https://hlsjs.video-dev.org/" }, { "time": 0.19699999690055847, "key": "kFrameUrl", "value": "https://hlsjs.video-dev.org/demo/?src=https%3A%2F%2Ftest-streams.mux.dev%2Fx36xhzz%2Fx36xhzz.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsInN0b3BPblN0YWxsIjpmYWxzZSwiZHVtcGZNUDQiOmZhbHNlLCJsZXZlbENhcHBpbmciOi0xLCJsaW1pdE1ldHJpY3MiOi0xfQ==" }, { "time": 0.19900000095367432, "key": "kFrameTitle", "value": "hls.js demo" }, { "time": 0.3269999921321869, "key": "url", "value": "blob:https://hlsjs.video-dev.org/07aac4ef-1f4e-47bd-9e17-c1076ad0679b" }, { "time": 0.3310000002384186, "key": "kTextTracks", "value": [] }, { "time": 0.33799999952316284, "key": "info", "value": "ChunkDemuxer" }, { "time": 0.3439999967813492, "key": "kRendererName", "value": "RendererImpl" }, { "time": 0.5300000011920929, "key": "pipeline_state", "value": "kStarting" }, { "time": 101.41899999976158, "key": "kAudioTracks", "value": [ { "bytes per channel": 2, "bytes per frame": 4, "channel layout": "STEREO", "channels": 2, "codec": "aac", "codec delay": 0, "discard decoder delay": false, "encryption scheme": "Unencrypted", "has extra data": false, "profile": "unknown", "sample format": "Signed 16-bit", "samples per second": 48000, "seek preroll": "0us" } ] }, { "time": 101.87800000607967, "key": "kVideoTracks", "value": [ { "alpha mode": "is_opaque", "codec": "h264", "coded size": "3840x2160", "color space": { "matrix": "BT709", "primaries": "BT709", "range": "LIMITED", "transfer": "BT709" }, "encryption scheme": "Unencrypted", "has extra data": false, "hdr metadata": "unset", "natural size": "3840x2160", "orientation": "0°", "profile": "h264 high", "visible rect": "0,0 3840x2160" } ] }, { "time": 107.00499999523163, "key": "kIsAudioDecryptingDemuxerStream", "value": false }, { "time": 107.0080000013113, "key": "kAudioDecoderName", "value": "FFmpegAudioDecoder" }, { "time": 107.00900000333786, "key": "kIsPlatformAudioDecoder", "value": false }, { "time": 107.0160000026226, "key": "info", "value": "Selected FFmpegAudioDecoder for audio decoding, config: codec: aac, profile: unknown, bytes_per_channel: 2, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Signed 16-bit, bytes_per_frame: 4, seek_preroll: 0us, codec_delay: 0, has extra data: false, encryption scheme: Unencrypted, discard decoder delay: false, target_output_channel_layout: STEREO, target_output_sample_format: Unknown sample format, has aac extra data: true" }, { "time": 107.04700000584126, "key": "info", "value": "Cannot select DecryptingVideoDecoder for video decoding" }, { "time": 135.83300000429153, "key": "kIsVideoDecryptingDemuxerStream", "value": false }, { "time": 135.89200000464916, "key": "kVideoDecoderName", "value": "VDAVideoDecoder" }, { "time": 135.89299999177456, "key": "kIsPlatformVideoDecoder", "value": true }, { "time": 135.90299999713898, "key": "info", "value": "Selected VDAVideoDecoder for video decoding, config: codec: h264, profile: h264 high, level: not available, alpha_mode: is_opaque, coded size: [3840,2160], visible rect: [0,0,3840,2160], natural size: [3840,2160], has extra data: false, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}" }, { "time": 135.91499999165535, "key": "pipeline_state", "value": "kPlaying" }, { "time": 153.00900000333786, "key": "debug", "value": "Prepared video sample is not conformant" }, { "time": 153.01299999654293, "key": "debug", "value": "ISO-BMFF container metadata for video frame indicates that the frame is a keyframe, but the video frame contents indicate the opposite." }, { "time": 164.74899999797344, "key": "seek_target", "value": 5.810637 }, { "time": 164.8180000036955, "key": "pipeline_state", "value": "kSeeking" }, { "time": 164.9599999934435, "key": "pipeline_state", "value": "kPlaying" }, { "time": 251.76099999248981, "key": "dimensions", "value": "3840x2160" }, { "time": 251.76500000059605, "key": "kResolution", "value": "3840x2160" }, { "time": 255.27300000190735, "key": "info", "value": "Effective playback rate changed from 0 to 1" }, { "time": 255.37700000405312, "key": "event", "value": "kPlay" }, { "time": 522.7230000048876, "key": "error", "value": "VDA Error 4" }, { "time": 522.7939999997616, "key": "error", "value": { "code": 1, "data": {}, "group": "DecoderStatus", "message": "", "stack": [ { "file": "media/gpu/ipc/service/vda_video_decoder.cc", "line": 919 } ] } }, { "time": 522.8069999963045, "key": "error", "value": "video decode error!" }, { "time": 522.8859999924898, "key": "error", "value": { "cause": { "code": 1, "data": {}, "group": "DecoderStatus", "message": "", "stack": [ { "file": "media/gpu/ipc/service/vda_video_decoder.cc", "line": 919 } ] }, "code": 3, "data": {}, "group": "PipelineStatus", "message": "", "stack": [ { "file": "media/renderers/video_renderer_impl.cc", "line": 591 } ] } }, { "time": 523.027999997139, "key": "pipeline_state", "value": "kStopping" }, { "time": 523.2479999959469, "key": "event", "value": "kPause" }, { "time": 523.3770000040531, "key": "pipeline_state", "value": "kStopped" }, { "time": 221.16499999165535, "key": "duration", "value": 15.007999 }, { "time": 255.2099999934435, "key": "pipeline_buffering_state", "value": { "for_suspended_start": false, "state": "BUFFERING_HAVE_ENOUGH" } } ] }EDIT: My encoder settings for this stream, just in case that matters. 12 Mbps 4k60 with B-Frames disabled.
EDIT 2: Shaka Player and Bitmovin's HLS Player seem to run into this issue as well. But THEOplayer doesn't! Fascinating.
The text was updated successfully, but these errors were encountered: