Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix multi-period DASH with descriptive audio #4629

Merged
merged 27 commits into from
Oct 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
90b4527
Merge remote-tracking branch 'upstream/master'
dsparacio Oct 29, 2020
49d611f
Merge branch 'master' of https://github.com/google/shaka-player
dsparacio Jan 25, 2021
93fdcd3
Merge remote-tracking branch 'upstream/master'
littlespex May 4, 2021
0b7ac5b
Merge branch 'google:master' into master
littlespex Sep 8, 2021
6db0e3c
Merge branch 'google:master' into master
littlespex Sep 15, 2021
efaebee
Merge branch 'google:master' into master
littlespex Sep 20, 2021
7fed0d0
Merge branch 'google:master' into master
littlespex Sep 22, 2021
7a30028
Merge branch 'google:master' into master
littlespex Sep 30, 2021
0168a7f
Merge branch 'google:master' into master
littlespex Oct 18, 2021
6b64092
Merge branch 'google:master' into master
littlespex Nov 2, 2021
a0cdf79
Merge branch 'google:master' into master
littlespex Dec 16, 2021
28564dd
Merge branch 'google:master' into master
littlespex Jan 7, 2022
5fb344e
Merge branch 'google:master' into master
littlespex Jan 10, 2022
1766930
Merge branch 'google:master' into master
littlespex Jan 12, 2022
b0cad7c
Merge branch 'google:master' into master
littlespex Jan 18, 2022
868b75b
Merge branch 'google:master' into master
littlespex Feb 4, 2022
95589c9
Merge branch 'shaka-project:main' into master
littlespex Mar 15, 2022
f331382
Merge branch 'shaka-project:main' into master
littlespex Jun 13, 2022
d14c111
Merge branch 'shaka-project:main' into master
littlespex Jun 29, 2022
b700d3b
Merge branch 'shaka-project:main' into master
littlespex Sep 21, 2022
0d9ea95
Merge branch 'shaka-project:main' into master
littlespex Oct 21, 2022
ef5177b
Merge branch 'shaka-project:main' into master
littlespex Oct 25, 2022
937638e
Merge branch 'shaka-project:main' into master
littlespex Oct 26, 2022
6f0d4fe
Merge branch 'shaka-project:main' into master
littlespex Oct 27, 2022
956679e
Perform primary check after role check to facilitate better period co…
littlespex Oct 27, 2022
6338d7e
Rename tests
littlespex Oct 27, 2022
8d1c69e
no message
littlespex Oct 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions lib/util/periods.js
Original file line number Diff line number Diff line change
Expand Up @@ -1180,15 +1180,6 @@ shaka.util.PeriodCombiner = class {
return false;
}

// If the language doesn't match, but the candidate is the "primary"
// language, then that should be preferred as a fallback.
if (!best.primary && candidate.primary) {
return true;
}
if (best.primary && !candidate.primary) {
return false;
}

Comment on lines -1183 to -1191
Copy link
Contributor Author

@littlespex littlespex Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joeyparrish Removing this completely caused a number of issues, but moving it below the role check solves the issue.

// If language-based differences haven't decided this, look at roles. If
// the candidate has more roles in common with the output, upgrade to the
// candidate.
Expand Down Expand Up @@ -1222,6 +1213,15 @@ shaka.util.PeriodCombiner = class {
return false;
}

// If the language doesn't match, but the candidate is the "primary"
// language, then that should be preferred as a fallback.
if (!best.primary && candidate.primary) {
return true;
}
if (best.primary && !candidate.primary) {
return false;
}

// If language-based and role-based features are equivalent, take the audio
// with the closes channel count to the output.
const channelsBetterOrWorse =
Expand Down
160 changes: 149 additions & 11 deletions test/util/periods_unit.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ describe('PeriodCombiner', () => {
/** @type {shaka.util.PeriodCombiner} */
let combiner;

const makeAudioStreamWithRoles = (id, roles, primary = true) => {
const stream = makeAudioStream('en');
stream.originalId = id;
stream.roles = roles;
stream.primary = primary;
return stream;
};

beforeEach(() => {
combiner = new shaka.util.PeriodCombiner();
});
Expand Down Expand Up @@ -1127,28 +1135,157 @@ describe('PeriodCombiner', () => {
const variants = combiner.getVariants();

expect(variants).toEqual(jasmine.arrayWithExactContents([
makeAVVariant(1080, 'en'),
makeAVVariant(1080, 'en'),
makeAVVariant(1080, 'en', 2, ['role1', 'role2']),
makeAVVariant(1080, 'en', 2, ['role1']),
]));

// We can use the originalId field to see what each track is composed of.
const audio1 = variants[0].audio;
expect(audio1.roles).toEqual(['role1', 'role2']);
expect(audio1.originalId).toBe('stream1,stream3');
expect(variants[0].audio.originalId).toBe('stream1,stream3');
expect(variants[1].audio.originalId).toBe('stream2,stream4');
});

const audio2 = variants[1].audio;
expect(audio2.roles).toEqual(['role1']);
expect(audio2.originalId).toBe('stream2,stream4');
it('Matches streams with roles in common', async () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test simulates the stream that originally manifested the issue. It is a multi-period steam where all periods have the same number of video and audio adaptation sets, and the audio adaptations both have roles, one main and one description.

/** @type {!Array.<shaka.util.PeriodCombiner.Period>} */
const periods = [
{
id: '1',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
makeAudioStreamWithRoles('stream2', ['description'], false),
],
textStreams: [],
imageStreams: [],
},
{
id: '2',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
makeAudioStreamWithRoles('stream2', ['description'], false),
],
textStreams: [],
imageStreams: [],
},
];

await combiner.combinePeriods(periods, /* isDynamic= */ false);
const variants = combiner.getVariants();

console.log(variants);

expect(variants.length).toBe(4);

expect(variants).toEqual(jasmine.arrayWithExactContents([
makeAVVariant(720, 'en', 2, ['main']),
makeAVVariant(1080, 'en', 2, ['main']),
makeAVVariant(720, 'en', 2, ['description']),
makeAVVariant(1080, 'en', 2, ['description']),
]));

// We can use the originalId field to see what each track is composed of.
expect(variants[0].audio.originalId).toBe('stream1,stream1');
expect(variants[1].audio.originalId).toBe('stream1,stream1');
expect(variants[2].audio.originalId).toBe('stream2,stream2');
expect(variants[3].audio.originalId).toBe('stream2,stream2');
});

it('Matches streams with mismatched roles', async () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test simulates a similar stream to the one above, but this time with periods representing pre-roll and mid-roll ads that only have main audio tracks.

/** @type {!Array.<shaka.util.PeriodCombiner.Period>} */
const periods = [
{
id: '0',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
],
textStreams: [],
imageStreams: [],
},
{
id: '1',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
makeAudioStreamWithRoles('stream2', ['description'], false),
],
textStreams: [],
imageStreams: [],
},
{
id: '2',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
],
textStreams: [],
imageStreams: [],
},
{
id: '3',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
makeAudioStreamWithRoles('stream2', ['description'], false),
],
textStreams: [],
imageStreams: [],
},
];

await combiner.combinePeriods(periods, /* isDynamic= */ false);
const variants = combiner.getVariants();

console.log(variants);

expect(variants.length).toBe(4);

expect(variants).toEqual(jasmine.arrayWithExactContents([
makeAVVariant(720, 'en', 2, ['main']),
makeAVVariant(1080, 'en', 2, ['main']),
makeAVVariant(720, 'en', 2, ['description', 'main']),
makeAVVariant(1080, 'en', 2, ['description', 'main']),
]));

// We can use the originalId field to see what each track is composed of.
expect(variants[0].audio.originalId)
.toBe('stream1,stream1,stream1,stream1');

expect(variants[1].audio.originalId)
.toBe('stream1,stream1,stream1,stream1');

expect(variants[2].audio.originalId)
.toBe('stream1,stream2,stream1,stream2');

expect(variants[3].audio.originalId)
.toBe('stream1,stream2,stream1,stream2');
});

it('The number of variants stays stable after many periods ' +
'when going between similar content and varying ads', async () => {
// This test is based on the content from
// https://github.com/shaka-project/shaka-player/issues/2716
// that used to cause our period flattening logic to keep
// creating new variants for every new period added.
// It's ok to create a few additional varinats/streams,
// It's ok to create a few additional variants/streams,
// but we should stabilize eventually and keep the number
// of variants from growing indefinitely.

Expand Down Expand Up @@ -1334,7 +1471,7 @@ describe('PeriodCombiner', () => {
expect(isCandidateBetter).toBe(WORSE);

// Make sure it works correctly whether it's the candidate or the best
// value that is equel to the output.
// value that is equal to the output.
isCandidateBetter = PeriodCombiner.compareClosestPreferLower(
/* output= */ 5, /* bestValue= */ 3, /* candidateValue= */ 5);
expect(isCandidateBetter).toBe(BETTER);
Expand Down Expand Up @@ -1461,11 +1598,12 @@ describe('PeriodCombiner', () => {
* @param {number=} channels
* @return {shaka.extern.Variant}
*/
function makeAVVariant(height, language, channels = 2) {
function makeAVVariant(height, language, channels = 2, roles = []) {
const variant = jasmine.objectContaining({
language,
audio: jasmine.objectContaining({
language,
roles,
channelsCount: channels,
}),
video: jasmine.objectContaining({
Expand Down