Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fix multi-period DASH with descriptive audio #4629

Merged
merged 27 commits into from
Oct 28, 2022
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
90b4527
Merge remote-tracking branch 'upstream/master'
dsparacio Oct 29, 2020
49d611f
Merge branch 'master' of https://github.com/google/shaka-player
dsparacio Jan 25, 2021
93fdcd3
Merge remote-tracking branch 'upstream/master'
littlespex May 4, 2021
0b7ac5b
Merge branch 'google:master' into master
littlespex Sep 8, 2021
6db0e3c
Merge branch 'google:master' into master
littlespex Sep 15, 2021
efaebee
Merge branch 'google:master' into master
littlespex Sep 20, 2021
7fed0d0
Merge branch 'google:master' into master
littlespex Sep 22, 2021
7a30028
Merge branch 'google:master' into master
littlespex Sep 30, 2021
0168a7f
Merge branch 'google:master' into master
littlespex Oct 18, 2021
6b64092
Merge branch 'google:master' into master
littlespex Nov 2, 2021
a0cdf79
Merge branch 'google:master' into master
littlespex Dec 16, 2021
28564dd
Merge branch 'google:master' into master
littlespex Jan 7, 2022
5fb344e
Merge branch 'google:master' into master
littlespex Jan 10, 2022
1766930
Merge branch 'google:master' into master
littlespex Jan 12, 2022
b0cad7c
Merge branch 'google:master' into master
littlespex Jan 18, 2022
868b75b
Merge branch 'google:master' into master
littlespex Feb 4, 2022
95589c9
Merge branch 'shaka-project:main' into master
littlespex Mar 15, 2022
f331382
Merge branch 'shaka-project:main' into master
littlespex Jun 13, 2022
d14c111
Merge branch 'shaka-project:main' into master
littlespex Jun 29, 2022
b700d3b
Merge branch 'shaka-project:main' into master
littlespex Sep 21, 2022
0d9ea95
Merge branch 'shaka-project:main' into master
littlespex Oct 21, 2022
ef5177b
Merge branch 'shaka-project:main' into master
littlespex Oct 25, 2022
937638e
Merge branch 'shaka-project:main' into master
littlespex Oct 26, 2022
6f0d4fe
Merge branch 'shaka-project:main' into master
littlespex Oct 27, 2022
956679e
Perform primary check after role check to facilitate better period co…
littlespex Oct 27, 2022
6338d7e
Rename tests
littlespex Oct 27, 2022
8d1c69e
no message
littlespex Oct 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions lib/util/periods.js
Original file line number Diff line number Diff line change
Expand Up @@ -1180,15 +1180,6 @@ shaka.util.PeriodCombiner = class {
return false;
}

// If the language doesn't match, but the candidate is the "primary"
// language, then that should be preferred as a fallback.
if (!best.primary && candidate.primary) {
return true;
}
if (best.primary && !candidate.primary) {
return false;
}

Comment on lines -1183 to -1191
Copy link
Contributor Author

@littlespex littlespex Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joeyparrish Removing this completely caused a number of issues, but moving it below the role check solves the issue.

// If language-based differences haven't decided this, look at roles. If
// the candidate has more roles in common with the output, upgrade to the
// candidate.
Expand Down Expand Up @@ -1222,6 +1213,15 @@ shaka.util.PeriodCombiner = class {
return false;
}

// If the language doesn't match, but the candidate is the "primary"
// language, then that should be preferred as a fallback.
if (!best.primary && candidate.primary) {
return true;
}
if (best.primary && !candidate.primary) {
return false;
}

// If language-based and role-based features are equivalent, take the audio
// with the closes channel count to the output.
const channelsBetterOrWorse =
Expand Down
161 changes: 159 additions & 2 deletions test/util/periods_unit.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ describe('PeriodCombiner', () => {
/** @type {shaka.util.PeriodCombiner} */
let combiner;

const makeAudioStreamWithRoles = (id, roles, primary = true) => {
const stream = makeAudioStream('en');
stream.originalId = id;
stream.roles = roles;
stream.primary = primary;
return stream;
};

beforeEach(() => {
combiner = new shaka.util.PeriodCombiner();
});
Expand Down Expand Up @@ -1141,14 +1149,163 @@ describe('PeriodCombiner', () => {
expect(audio2.originalId).toBe('stream2,stream4');
});

it('Matches streams with primary in common', async () => {
littlespex marked this conversation as resolved.
Show resolved Hide resolved
littlespex marked this conversation as resolved.
Show resolved Hide resolved
/** @type {!Array.<shaka.util.PeriodCombiner.Period>} */
const periods = [
{
id: '1',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
makeAudioStreamWithRoles('stream2', ['description'], false),
],
textStreams: [],
imageStreams: [],
},
{
id: '2',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
makeAudioStreamWithRoles('stream2', ['description'], false),
],
textStreams: [],
imageStreams: [],
},
];

await combiner.combinePeriods(periods, /* isDynamic= */ false);
const variants = combiner.getVariants();

console.log(variants);

expect(variants.length).toBe(4);

expect(variants).toEqual(jasmine.arrayWithExactContents([
makeAVVariant(720, 'en'),
makeAVVariant(1080, 'en'),
makeAVVariant(720, 'en'),
makeAVVariant(1080, 'en'),
]));

// We can use the originalId field to see what each track is composed of.
const audio1 = variants[0].audio;
expect(audio1.roles).toEqual(['main']);
expect(audio1.originalId).toBe('stream1,stream1');

const audio2 = variants[1].audio;
expect(audio2.roles).toEqual(['main']);
expect(audio2.originalId).toBe('stream1,stream1');

const audio3 = variants[2].audio;
expect(audio3.roles).toEqual(['description']);
expect(audio3.originalId).toBe('stream2,stream2');

const audio4 = variants[3].audio;
expect(audio4.roles).toEqual(['description']);
expect(audio4.originalId).toBe('stream2,stream2');
});

it('Matches streams with mismatched primary', async () => {
littlespex marked this conversation as resolved.
Show resolved Hide resolved
littlespex marked this conversation as resolved.
Show resolved Hide resolved
/** @type {!Array.<shaka.util.PeriodCombiner.Period>} */
const periods = [
{
id: '0',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
],
textStreams: [],
imageStreams: [],
},
{
id: '1',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
makeAudioStreamWithRoles('stream2', ['description'], false),
],
textStreams: [],
imageStreams: [],
},
{
id: '2',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
],
textStreams: [],
imageStreams: [],
},
{
id: '3',
videoStreams: [
makeVideoStream(720),
makeVideoStream(1080),
],
audioStreams: [
makeAudioStreamWithRoles('stream1', ['main']),
makeAudioStreamWithRoles('stream2', ['description'], false),
],
textStreams: [],
imageStreams: [],
},
];

await combiner.combinePeriods(periods, /* isDynamic= */ false);
const variants = combiner.getVariants();

console.log(variants);

expect(variants.length).toBe(4);

expect(variants).toEqual(jasmine.arrayWithExactContents([
makeAVVariant(720, 'en'),
makeAVVariant(1080, 'en'),
makeAVVariant(720, 'en'),
makeAVVariant(1080, 'en'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add an optional roles argument to makeAVVariant? I think this part of the test would be clearer if we could see the expected roles here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

]));

// We can use the originalId field to see what each track is composed of.
const audio1 = variants[0].audio;
expect(audio1.roles).toEqual(['main']);
expect(audio1.originalId).toBe('stream1,stream1,stream1,stream1');

const audio2 = variants[1].audio;
expect(audio2.roles).toEqual(['main']);
expect(audio2.originalId).toBe('stream1,stream1,stream1,stream1');

const audio3 = variants[2].audio;
expect(audio3.roles).toEqual(['description', 'main']);
expect(audio3.originalId).toBe('stream1,stream2,stream1,stream2');

const audio4 = variants[3].audio;
expect(audio4.roles).toEqual(['description', 'main']);
expect(audio4.originalId).toBe('stream1,stream2,stream1,stream2');
});

it('The number of variants stays stable after many periods ' +
'when going between similar content and varying ads', async () => {
// This test is based on the content from
// https://github.com/shaka-project/shaka-player/issues/2716
// that used to cause our period flattening logic to keep
// creating new variants for every new period added.
// It's ok to create a few additional varinats/streams,
// It's ok to create a few additional variants/streams,
// but we should stabilize eventually and keep the number
// of variants from growing indefinitely.

Expand Down Expand Up @@ -1334,7 +1491,7 @@ describe('PeriodCombiner', () => {
expect(isCandidateBetter).toBe(WORSE);

// Make sure it works correctly whether it's the candidate or the best
// value that is equel to the output.
// value that is equal to the output.
isCandidateBetter = PeriodCombiner.compareClosestPreferLower(
/* output= */ 5, /* bestValue= */ 3, /* candidateValue= */ 5);
expect(isCandidateBetter).toBe(BETTER);
Expand Down