Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WebVTT is rejected when using header text #1017

Closed
dvoracek-slub opened this issue Jan 7, 2022 · 2 comments · Fixed by #1349
Closed

WebVTT is rejected when using header text #1017

dvoracek-slub opened this issue Jan 7, 2022 · 2 comments · Fixed by #1349
Labels
status: archived Archived and locked; will not be updated

Comments

@dvoracek-slub
Copy link

System info

Operating System: any
Shaka Packager Version: e1b0c7c (current master)

Issue and steps to reproduce the problem

The spec allows to place a header text after the WEBVTT magic (https://www.w3.org/TR/webvtt1/#file-structure, 3.), which Packager does not recognize and rejects.

Packager Command:

packager in=text.vtt,stream=text,output=text_processed.vtt

What is the expected result?
Packager generates a WebVTT file named "text_processed.vtt" without error. (Ideally, I suppose, the header text should be carried over to the generated file.)

What happens instead?
Packager aborts with the following message:

[0107/200000:INFO:demuxer.cc(89)] Demuxer::Run() on file 'text.vtt'.
[0107/200000:INFO:demuxer.cc(155)] Initialize Demuxer for file 'text.vtt'.
[0107/200000:ERROR:webvtt_parser.cc(231)] Failed to read WEBVTT header - should be WEBVTT but was WEBVTT Header Text
[0107/200000:ERROR:packager_main.cc(554)] Packaging Error: 8 (PARSER_FAILURE): Cannot parse media file text.vtt

text.vtt

WEBVTT Header Text

00:00.000 --> 00:01.000
Cue Text

00:01.000 --> 00:02.000
Cue Text
@Canta Canta mentioned this issue Feb 3, 2022
Closed
@Canta
Copy link
Contributor

Canta commented Feb 4, 2022

Hi there @dvoracek-slub .

I've faced the same issue on a different situation (live streaming), and already sent a pull request with changes that make shaka packager not rejecting the webvtts.
However, I see the whole shaka group of software is dealing with infrastructure changes, and my pull request fails because of building steps configurations (something unrelated to my code). I can't tell when is that going to be fixed. So, if you happen to need a fix for your issue, you may take a look at the PR and apply the changes yourself locally on your side.

Here are the changes: https://github.com/google/shaka-packager/pull/1027/files
And here's the explanation for all of it: #685 (comment)

@dvoracek-slub
Copy link
Author

Thanks @Canta! Currently my workaround is to preprocess the WEBVTT file, but I'll consider using a custom build.

@Canta Canta mentioned this issue Feb 22, 2024
Merged
@github-actions github-actions bot added the status: archived Archived and locked; will not be updated label Apr 24, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 24, 2024
SteveR-PMP added a commit to SteveR-PMP/shaka-packager that referenced this issue Aug 28, 2024
@SteveR-PMP @petzeb
@sr1990 @cosmin @wjywbs @marcusspangenberg @XingzhaoYun @joeyparrish @Canta @koln67 @shaka-bot
fix: VR-4186, fix HLS ordering in shaka packager (shaka-project#29)
1c73e48 
* fix: http_file: Close upload cache on task exit (shaka-project#1348)

In some cases it can happen that the http server responds with a
non-successful status code without reading the response body. In this
case curl may decide not to read from the cache since there is really no
point in sending data to the server. In case some other thread of shaka
has already called HttpFile::Flush it may end up deadlocked there
waiting for the cache to either close or become empty. Thus, we close
the cache when leaving the main thread as no data will be read by curl
after it has finished anyways.

Closes shaka-project#1347

* feat: Add support for single file TS for HLS (shaka-project#934)

This is based on comments at
shaka-project#891. The muxer is deciding
whether to write to a single file or a segment file based on the
configuration.

Example:
```
../packager 'in=TOS.ts,stream=video,output=tos_video.ts,playlist_name=tos_video.m3u8' \
            'in=TOS.ts,stream=audio,output=tos_audio.ts,playlist_name=tos_audio.m3u8' \
           --hls_master_playlist_output tos.m3u8
```
Tested the content using Exoplayer.

---------

Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>

* feat: Add PlayReady support in HLS. (shaka-project#1011)

Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>

* feat: Add input support for EBU Teletext in MPEG-TS (shaka-project#1344)

Replaces shaka-project#1181

* Add support for EBU Teletext input following Level 1.5 of the core
specification ETSI EN 300 706 V1.2.1 (2003-04).
* Add support for webvtt in MP4 segments output.

Closes shaka-project#272

---------

Co-authored-by: Marcus Spangenberg <marcus.spangenberg@eyevinn.se>

* feat: add startwithSAP/subsegmentstartswithSAP for audio tracks (shaka-project#1346)

Add startwithSAP/subsegmentstartswithSAP for aac, ac3, ec3 and ac4 audio tracks according to LIVE or VOD profile.

Replaces shaka-project#1055
Partial solution for shaka-project#364

---------

Co-authored-by: Xingzhao Yun <xyun@dolby.com>
Co-authored-by: Joey Parrish <joeyparrish@google.com>

* feat: Allow LIVE UDP WebVTT input (shaka-project#1349)

An updated version of PR shaka-project#1027

That previous PR was done using 2021 code, and there were many changes
in the codebase from there, so a rebase was needed and also some minor
tweak here and there. But it's the same code, just reimplemented on a
newer codebase.

If you want to take a look at this in action, after building shaka
packager with this PR's code included, try this commands in 3 different
simultaneous bash sessions:

1. Video UDP input: `ffmpeg -f lavfi -re -i
"testsrc=s=320x240:r=30,format=yuv420p" -c:v h264 -sc_threshold 0 -g 30
-keyint_min 30 -r 30 -a53cc 1 -b:v 150k -preset ultrafast -r 30 -f
mpegts "udp://127.0.0.1:10000?pkt_size=1316"`
2. WebVTT UDP input: `for sec in $(seq 0 9999) ; do printf
"%02d:%02d.000 --> %02d:%02d.000\ntest second ${sec}\n\n" "$(( ${sec} /
60 ))" "$(( ${sec} % 60 ))" "$(( (${sec} + 1) / 60 ))" "$(( (${sec} + 1)
% 60 ))" ; sleep 1 ; done > /dev/udp/127.0.0.1/12345`
3. shaka packager command line: `timeout 60
path/to/build/packager/packager
'in=udp://127.0.0.1:10000?timeout=8000000,stream_selector=0,init_segment=240_init.m4s,segment_template=240_$Number%09d$.m4s,bandwidth=150000'
'in=udp://127.0.0.1:12345?timeout=8000000,stream_selector=0,input_format=webvtt,format=webvtt+mp4,init_segment=text_init.m4s,segment_template=text_$Number%09d$.m4s,language=eng,dash_roles=subtitle'
--mpd_output ./manifest.mpd --segment_duration 3.2
--suggested_presentation_delay 3.2 --min_buffer_time 3.2
--minimum_update_period 3.2 --time_shift_buffer_depth 60
--preserved_segments_outside_live_window 1 --default_language=eng
--dump_stream_info 2>&1`

Note the added `input_format=webvtt` to the shaka packager command's
second selector. That's new from this PR. If you don't use that, shaka's
format autodetection will not detect the webvtt format from the input,
as explained in
shaka-project#685 (comment).
Try the command without it if you want to.

Fixes shaka-project#685
Fixes shaka-project#1017

---------

Co-authored-by: Daniel Cantarín <canta@canta.com.ar>

* fix: Always log to stderr by default (shaka-project#1350)

This tweaks the default config for stderrthreshold from absl/log so that
we always get logs to stderr by default, as we did in v2. The --quiet
and --v flags that existed in v2 can still be used to modify the log
level, as well as the new --minloglevel from absl/log.

Issue shaka-project#1325

* feat: Add support for ALAC codec (shaka-project#1299)

Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>

* fix: Fix missing newline at the end of usage (shaka-project#1352)

Without this, the command prompt will not appear on its own line after
running `packager` with no arguments.

* build: Disable warnings in third-party code (shaka-project#1353)

We already disabled warnings-as-errors in third-party code, but this
goes futher to completely disable third-party warnings. They are mostly
noise, and distracting. If they aren't errors, we shouldn't have our
build logs flooded with them.

* fix: AudioSampleEntry size caluations due to bad merge (shaka-project#1354)

from ALAC pull request

* feat: Portable, fully-static release executables on Linux (shaka-project#1351)

This adds the option FULLY_STATIC to create fully-static executables.

To create portable, fully-static release executables on Linux, we need
to use musl instead of glibc. Static executables from glibc are not
portable.

The popular musl-gcc wrapper does not support C++, so instead we use a
full musl cross-compiler toolchain in the build workflow.

To build FULLY_STATIC, the user must point to the appropriate
cross-compiler, as we do in the workflow. On systems where musl is the
native libc (such as Alpine Linux), this is not necessary.

I have also read that musl's allocator is not very fast in
multi-threaded applications. So when FULLY_STATIC is enabled, we will
also enable mimalloc, a replacement allocator that is very fast.

I tested a very basic packaging command to compare speeds of dynamic
glibc, static musl, and static musl+mimalloc:

dynamic glibc:
runs: 2.527, 2.798, 2.703, 2.756, 2.959
avg = 2.749, std dev = 0.156s

static musl:
runs: 2.813, 2.920, 3.129, 3.003, 2.738
avg = 2.921s, std dev = 0.154s

static musl+mimalloc:
runs: 2.291, 2.034, 2.415, 2.303, 2.265
avg = 2.262s, std dev = 0.140s

The mimalloc build is 82% faster than musl default allocator, 77% faster
than glibc, and has more consistent runtime characteristics (lower
standard deviation).

* fix: Restore support for legacy FairPlay system ID (shaka-project#1357)

Fixes shaka-project#1356 which was caused by the fix in shaka-project#1281 which updated this to
use the correct FairPlay system ID. However since old versions
recognized the previous system ID this restores support for it to avoid
breaking clients.

* fix: use a better estimate of frame rate for cases with very short first sample durations (shaka-project#838)

Use the second sample in mp4 and webm formats. shaka-project#835 had issues with
merging due to golden file conflicts. Because we cannot make dependent
pull requests, this is a replica of shaka-project#835.

---------

Signed-off-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>

* chore(main): release 3.0.0 (shaka-project#1312)

## [3.0.0](shaka-project/shaka-packager@v2.6.1...v3.0.0) (2024-02-28)

### ⚠ BREAKING CHANGES

* Update all dependencies
* Drop Python 2 support in all scripts
* Replace glog with absl::log, tweak log output and flags
* Replace gyp build system with CMake

### Features & Bug Fixes

Too many to list... See CHANGELOG.md for a full accounting!

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

* ci: Add missing arm64 (self-hosted) builds on release (shaka-project#1358)

The build invocation from the release workflow needs settings to trigger
self-hosted builds, including Linux arm64.

* fix(CI): Add Mac-arm64 to build matrix (shaka-project#1359)

Support for this came out in January, 2024. Explicit macos versions seem
to be necessary for now, until GitHub offers "latest" labels targeting
specific architectures.

* ci: Update actions (shaka-project#1360)

This silences warnings about outdated node runtimes in CI jobs.

 - Trivial upgrades:
   - actions/checkout@v3 => v4
   - actions/deploy-pages@v2 => v4
   - actions/upload-pages-artifact@v2 => v3
   - mxschmitt/action-tmate@v3.6 => v3.17
   - mikepenz/action-junit-report@v3.8.0 => v4
 - Upgrades that required small changes or deeper testing:
   - actions/download-artifact@v3 => v4
   - actions/upload-artifact@v3 => v4
   - google-github-actions/release-please-action@v3 => v4
 - Replaced:
   - svenstaro/upload-release-action@v2 => gh CLI

See also:
- https://github.com/actions/upload-artifact/blob/main/docs/MIGRATION.md

* chore(main): release 3.0.1 (shaka-project#1361)

* fix: duplicate representation id for TTML when forced ordering is on (shaka-project#1364)

Fixes shaka-project#1362

* chore(main): release 3.0.2 (shaka-project#1365)

:robot: I have created a release *beep* *boop*
---


##
[3.0.2](shaka-project/shaka-packager@v3.0.1...v3.0.2)
(2024-03-07)


### Bug Fixes

* duplicate representation id for TTML when forced ordering is on
([shaka-project#1364](shaka-project#1364))
([0fd815a](shaka-project@0fd815a)),
closes
[shaka-project#1362](shaka-project#1362)

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

* force exact command-line ordering for HLS by breaking grouping
---------

Signed-off-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: Peter Zebühr <33069501+petzeb@users.noreply.github.com>
Co-authored-by: sr90 <sr1990003@gmail.com>
Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>
Co-authored-by: wjywbs <wjywbs@users.noreply.github.com>
Co-authored-by: Marcus Spangenberg <marcus.spangenberg@eyevinn.se>
Co-authored-by: Xingzhao Yun <xyun@dolby.com>
Co-authored-by: Joey Parrish <joeyparrish@google.com>
Co-authored-by: Daniel Cantarín <omega_canta@yahoo.com>
Co-authored-by: Daniel Cantarín <canta@canta.com.ar>
Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com>
Co-authored-by: Anthony Lu <67125539+koln67@users.noreply.github.com>
Co-authored-by: Shaka Bot <13107131+shaka-bot@users.noreply.github.com>
Co-authored-by: Cosmin Stejerean <cstejerean@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
status: archived Archived and locked; will not be updated
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Allow LIVE UDP WebVTT input Canta/shaka-packager
2 participants
@Canta @dvoracek-slub