Missing HLS outputs on macOS

[Romantic-LiXuefeng]

Use case:
I want to use Sintel.2010.720p.mkv , Sintel.2010.English.vtt and Sintel.2010.Chinese.vtt as the input. And just want to output HLS VOD stream. After transcode and package, the output not includes HLS manifest and the subtitle format is error.

input_vod_config.yaml

# List of inputs.
inputs:
    # Name of the input file.
    # This example can be downloaded from https://storage.googleapis.com/shaka-streamer-assets/sample-inputs/Sintel.2010.4k.mkv
  - name: Sintel.2010.720p.mkv

    # The track number.
    track_num: 0
    # The media type of the input. Can be audio or video.
    media_type: video
    # Frame rate in seconds.
    frame_rate: 24.0
    # Resolution of the input.
    resolution: 720p
    # Start time of video to encode.
    start_time: 00:00:00
    # End time of video to encode.
    end_time: 00:01:00

    # A second track (audio) from the same input file.
  - name: Sintel.2010.720p.mkv
    track_num: 1
    media_type: audio
    start_time: 00:00:00
    end_time: 00:01:00

    # https://storage.googleapis.com/shaka-streamer-assets/sample-inputs/Sintel.2010.English.vtt
  - name: Sintel.2010.English.vtt
    media_type: text
    language: en

    # https://storage.googleapis.com/shaka-streamer-assets/sample-inputs/Sintel.2010.Chinese.vtt
  - name: Sintel.2010.Chinese.vtt
    media_type: text
    language: zh

pipeline_vod_config.yaml

streaming_mode: vod

# A list of resolutions to encode.
# For VOD, you can specify many more resolutions than you would with live,
# since the encoding does not need to be done in real time.
# - 4k
# - 1080p
# - 720p

resolutions:
  - 480p
  - 360p

# The number of audio channels to output.
channels: 2

# The codecs to encode with.
# - aac
# - opus
# - h264
# - vp9
audio_codecs:
  - aac
video_codecs:
  - h264

# Manifest format (dash, hls or both)
# - hls
# - dash
manifest_format:
  - hls


# Length of each segment in seconds.
segment_size: 10

# Forces the use of SegmentTemplate in DASH.
# segment_per_file: True

Running on Mac OS:

$ python3.7 shaka-streamer -i config_files/input_vod_config.yaml -p config_files/pipeline_vod_config.yaml -o ../../shaka_packager/streamer/vod 

[Romantic-LiXuefeng]

I'm not sure whether the subtitle output format (m4s) is correct. I hope the text format is webvtt.

[joeyparrish]

I'm not sure if I understand your report. Please tell me if I got some of the details wrong.

After transcode and package, the output not includes HLS manifest

Are you saying there are no m3u8 files in the output folder? I can't reproduce this on Linux, but I'll try again on macOS.

and the subtitle format is error.

What exactly do you mean by this? If there is no HLS manifest, how do you know that there's a problem with the subtitles?

[Romantic-LiXuefeng]

[Romantic-LiXuefeng]

@joeyparrish The output result is as the picture described. There are 2 problems:

1. There are no m3u8 files in the output folder.
2. The text output isn't cut between start time(00:00:00) and end time(00:01:00) and the text output format is also m4s, hope to be webvtt.

[joeyparrish]

I'm not sure why you don't have m3u8 files. I can't reproduce that on Linux, but I've yet to try to reproduce the issue on macOS. I'll be able to try that tomorrow.

As for the text, we can't cut or process text in any way. I'll make sure to update the code to reject this combination and the docs to match.

[joeyparrish]

I can now confirm that there are no HLS outputs on macOS. I don't know why. It might be a bug in Shaka Packager, which is responsible for creating the HLS playlists and DASH manifest. I'll keep digging.

[Angels-group]

Manifest files (m3u8 and mpd) appear only after full transcoding (segmentation).
If you use live type (streaming_mode: live) - the manifest appears immediately.

I thought it was a streamer feature :)
But I also did not like this behavior :)
Checked on mac os 10.14.6

[joeyparrish]

The behavior of waiting to write the manifest for VOD is actually controlled by Shaka Packager.

In my test on macOS, even after transcoding was complete, I found no HLS playlists. This was not the case on Linux. So there may be a Packager bug on Mac.

@kqyang, I'll prepare a bug report on this soon.

[kqyang]

@joeyparrish The problem could be related to the specific content or command options as there is no problem packaging Shaka Packager test content: https://travis-ci.org/google/shaka-packager.

I'll try to re-produce the problem after seeing your bug report.

[joeyparrish]

I've narrowed it down to subtitles. Without them, everything is working correctly on macOS. With them, I get no HLS playlists. I'll send you a minimal repro with a specific command-line soon.

Here's the minimal configs in Shaka Streamer to repro the issue:

# input.yaml
inputs:
  - name: Sintel.2010.720p.mkv
    media_type: audio
    end_time: 00:01

  - name: Sintel.2010.English.vtt
    media_type: text
    language: en

# pipeline.yaml
streaming_mode: vod
resolutions:
  - 360p
manifest_format:
  - hls

This works as expected on Linux, but not macOS.

[joeyparrish]

I've determined that this is not a Packager bug after all. It has to do with the timing of the shutdown sequence in Shaka Streamer. FFmpeg shuts down first, and then we're killing Packager before it's done.