Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core(metrics): support FCP for all frames with devtools throttling #11874

Merged
merged 17 commits into from
Jan 19, 2021
20 changes: 20 additions & 0 deletions lighthouse-cli/test/fixtures/perf/frame-metrics-inner.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<html>
<head>
</head>
<body>
<div id="space"></div>
<h1>THIS IS THE INNER FRAME LCP AND FCP</h1>
<script>
const space = document.getElementById('space');
setTimeout(() => {
space.style.height = 200;
}, 200);
setTimeout(() => {
space.style.height = 400;
}, 400);
setTimeout(() => {
space.style.height = 600;
}, 600);
</script>
</body>
</html>
38 changes: 38 additions & 0 deletions lighthouse-cli/test/fixtures/perf/frame-metrics.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
<html>
<head>
</head>
<body>
<iframe id="frame" src="frame-metrics-inner.html" width="100%" height="700"></iframe>
<div id="space"></div>
<div>
<span id="content"></span>
</div>
<script>
const space = document.getElementById('space');
const content = document.getElementById('content');

const keepAliveInterval = setInterval(() => {
const start = new Date();
let now = start;
while (now - start < 51) {
now = new Date();
}
}, 1000);
setTimeout(() => clearInterval(keepAliveInterval), 8000);

// The innerHTML will be the LCP and FCP of the main frame.
// It is painted after the inner frame FCP, and it is smaller than the inner frame LCP.
// The metrics for all frames will therefore be different than the metrics for the main frame.
setTimeout(() => {
content.innerHTML = 'This is the main frame LCP and FCP.';

// Add a small layout shift in the main frame.
// More layout shifts occur in the inner frame.
// CLS for all frames will therefore, be different than CLS for main frame.
setTimeout(() => {
space.style.height = '100';
}, 50);
}, 5000);
</script>
</body>
</html>
Original file line number Diff line number Diff line change
Expand Up @@ -323,4 +323,31 @@ module.exports = [
},
},
},
{
lhr: {
requestedUrl: 'http://localhost:10200/perf/frame-metrics.html',
finalUrl: 'http://localhost:10200/perf/frame-metrics.html',
audits: {
'metrics': {
score: null,
details: {
type: 'debugdata',
items: [
{
firstContentfulPaint: '>5000',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

beautiful 👍

firstContentfulPaintAllFrames: '<5000',
largestContentfulPaint: '>5000',
largestContentfulPaintAllFrames: '<5000',
cumulativeLayoutShift: '0.001 +/- 0.0005',
cumulativeLayoutShiftAllFrames: '0.068 +/- 0.0005',
},
{
lcpInvalidated: false,
},
],
},
},
},
},
},
];
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/**
* @license Copyright 2020 The Lighthouse Authors. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
*/
'use strict';

const makeComputedArtifact = require('../computed-artifact.js');
const ComputedMetric = require('./metric.js');

class FirstContentfulPaintAllFrames extends ComputedMetric {
/**
* @return {Promise<LH.Artifacts.LanternMetric>}
*/
static computeSimulatedMetric() {
// TODO: Add support for all frames in lantern.
throw new Error('FCP All Frames not implemented in lantern');
}

/**
* @param {LH.Artifacts.MetricComputationData} data
* @return {Promise<LH.Artifacts.Metric>}
*/
static async computeObservedMetric(data) {
const {traceOfTab} = data;

return {
timing: traceOfTab.timings.firstContentfulPaintAllFrames,
timestamp: traceOfTab.timestamps.firstContentfulPaintAllFrames,
};
}
}

module.exports = makeComputedArtifact(FirstContentfulPaintAllFrames);
6 changes: 6 additions & 0 deletions lighthouse-core/computed/metrics/timing-summary.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
const TraceOfTab = require('../trace-of-tab.js');
const Speedline = require('../speedline.js');
const FirstContentfulPaint = require('./first-contentful-paint.js');
const FirstContentfulPaintAllFrames = require('./first-contentful-paint-all-frames.js');
const FirstMeaningfulPaint = require('./first-meaningful-paint.js');
const LargestContentfulPaint = require('./largest-contentful-paint.js');
const LargestContentfulPaintAllFrames = require('./largest-contentful-paint-all-frames.js');
Expand Down Expand Up @@ -44,6 +45,7 @@ class TimingSummary {
const traceOfTab = await TraceOfTab.request(trace, context);
const speedline = await Speedline.request(trace, context);
const firstContentfulPaint = await FirstContentfulPaint.request(metricComputationData, context);
const firstContentfulPaintAllFrames = await requestOrUndefined(FirstContentfulPaintAllFrames, metricComputationData); // eslint-disable-line max-len
const firstMeaningfulPaint = await FirstMeaningfulPaint.request(metricComputationData, context);
const largestContentfulPaint = await requestOrUndefined(LargestContentfulPaint, metricComputationData); // eslint-disable-line max-len
const largestContentfulPaintAllFrames = await requestOrUndefined(LargestContentfulPaintAllFrames, metricComputationData); // eslint-disable-line max-len
Expand All @@ -67,6 +69,8 @@ class TimingSummary {
// Include the simulated/observed performance metrics
firstContentfulPaint: firstContentfulPaint.timing,
firstContentfulPaintTs: firstContentfulPaint.timestamp,
firstContentfulPaintAllFrames: firstContentfulPaintAllFrames && firstContentfulPaintAllFrames.timing, // eslint-disable-line max-len
firstContentfulPaintAllFramesTs: firstContentfulPaintAllFrames && firstContentfulPaintAllFrames.timestamp, // eslint-disable-line max-len
firstMeaningfulPaint: firstMeaningfulPaint.timing,
firstMeaningfulPaintTs: firstMeaningfulPaint.timestamp,
largestContentfulPaint: largestContentfulPaint && largestContentfulPaint.timing,
Expand Down Expand Up @@ -97,6 +101,8 @@ class TimingSummary {
observedFirstPaintTs: traceOfTab.timestamps.firstPaint,
observedFirstContentfulPaint: traceOfTab.timings.firstContentfulPaint,
observedFirstContentfulPaintTs: traceOfTab.timestamps.firstContentfulPaint,
observedFirstContentfulPaintAllFrames: traceOfTab.timings.firstContentfulPaintAllFrames,
observedFirstContentfulPaintAllFramesTs: traceOfTab.timestamps.firstContentfulPaintAllFrames,
observedFirstMeaningfulPaint: traceOfTab.timings.firstMeaningfulPaint,
observedFirstMeaningfulPaintTs: traceOfTab.timestamps.firstMeaningfulPaint,
observedLargestContentfulPaint: traceOfTab.timings.largestContentfulPaint,
Expand Down
38 changes: 32 additions & 6 deletions lighthouse-core/computed/trace-of-tab.js
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,30 @@ class TraceOfTab {
// We'll check that we got an FCP here and re-type accordingly so all of our consumers don't
// have to repeat this check.
const traceOfTab = await LHTraceProcessor.computeTraceOfTab(trace);
const {timings, timestamps, firstContentfulPaintEvt} = traceOfTab;
const {firstContentfulPaint: firstContentfulPaintTiming} = timings;
const {firstContentfulPaint: firstContentfulPaintTs} = timestamps;
const {
timings,
timestamps,
firstContentfulPaintEvt,
firstContentfulPaintAllFramesEvt,
} = traceOfTab;
const {
firstContentfulPaint: firstContentfulPaintTiming,
firstContentfulPaintAllFrames: firstContentfulPaintAllFramesTiming,
} = timings;
const {
firstContentfulPaint: firstContentfulPaintTs,
firstContentfulPaintAllFrames: firstContentfulPaintAllFramesTs,
} = timestamps;

if (
!firstContentfulPaintEvt ||
firstContentfulPaintTiming === undefined ||
firstContentfulPaintTs === undefined
firstContentfulPaintTs === undefined ||
// FCP-AF will only be undefined if FCP is also undefined.
// These conditions are for enforcing types and should never actually trigger.
!firstContentfulPaintAllFramesEvt ||
firstContentfulPaintAllFramesTiming === undefined ||
firstContentfulPaintAllFramesTs === undefined
) {
throw new LHError(LHError.errors.NO_FCP);
}
Expand All @@ -66,8 +83,17 @@ class TraceOfTab {
return {
...traceOfTab,
firstContentfulPaintEvt,
timings: {...timings, firstContentfulPaint: firstContentfulPaintTiming},
timestamps: {...timestamps, firstContentfulPaint: firstContentfulPaintTs},
firstContentfulPaintAllFramesEvt,
timings: {
...timings,
firstContentfulPaint: firstContentfulPaintTiming,
firstContentfulPaintAllFrames: firstContentfulPaintAllFramesTiming,
},
timestamps: {
...timestamps,
firstContentfulPaint: firstContentfulPaintTs,
firstContentfulPaintAllFrames: firstContentfulPaintAllFramesTs,
},
};
}
}
Expand Down
35 changes: 27 additions & 8 deletions lighthouse-core/lib/tracehouse/trace-processor.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@
* 4. Return all those items in one handy bundle.
*/

/** @typedef {Omit<LH.Artifacts.TraceTimes, 'firstContentfulPaint'> & {firstContentfulPaint?: number}} TraceTimesWithoutFCP */
/** @typedef {Omit<LH.Artifacts.TraceTimes, 'firstContentfulPaint'|'firstContentfulPaintAllFrames'> & {firstContentfulPaint?: number, firstContentfulPaintAllFrames?: number}} TraceTimesWithoutFCP */
/** @typedef {Omit<TraceTimesWithoutFCP, 'traceEnd'>} TraceTimesWithoutFCPAndTraceEnd */
/** @typedef {Omit<LH.Artifacts.TraceOfTab, 'firstContentfulPaintEvt'|'timings'|'timestamps'> & {timings: TraceTimesWithoutFCP, timestamps: TraceTimesWithoutFCP, firstContentfulPaintEvt?: LH.Artifacts.TraceOfTab['firstContentfulPaintEvt']}} TraceOfTabWithoutFCP */
/** @typedef {Omit<LH.Artifacts.TraceOfTab, 'firstContentfulPaintEvt'|'firstContentfulPaintAllFramesEvt'|'timings'|'timestamps'> & {timings: TraceTimesWithoutFCP, timestamps: TraceTimesWithoutFCP, firstContentfulPaintEvt?: LH.Artifacts.TraceOfTab['firstContentfulPaintEvt'], firstContentfulPaintAllFramesEvt?: LH.Artifacts.TraceOfTab['largestContentfulPaintAllFramesEvt']}} TraceOfTabWithoutFCP */
/** @typedef {'lastNavigationStart'|'firstResourceSendRequest'} TimeOriginDeterminationMethod */
/** @typedef {Omit<LH.TraceEvent, 'name'|'args'> & {name: 'FrameCommittedInBrowser', args: {data: {frame: string, url: string, parent?: string}}}} FrameCommittedEvent */
/** @typedef {Omit<LH.TraceEvent, 'name'|'args'> & {name: 'largestContentfulPaint::Invalidate'|'largestContentfulPaint::Candidate', args: {data?: {size?: number}, frame: string}}} LCPEvent */
Expand Down Expand Up @@ -611,15 +611,26 @@ class TraceProcessor {
});
const frameIdToRootFrameId = this.resolveRootFrames(frames);

// Filter to just events matching the frame ID, just to make sure.
// Filter to just events matching the main frame ID, just to make sure.
const frameEvents = keyEvents.filter(e => e.args.frame === mainFrameIds.frameId);

// Filter to just events matching the main frame ID or any child frame IDs.
const frameTreeEvents = keyEvents.filter(e => {
return e.args &&
e.args.frame &&
frameIdToRootFrameId.get(e.args.frame) === mainFrameIds.frameId;
});
// In practice, there should always be FrameCommittedInBrowser events to define the frame tree.
// Unfortunately, many test traces do not include FrameCommittedInBrowser events due to minification.
// This ensures there is always a minimal frame tree and events so those tests don't fail.
let frameTreeEvents = [];
if (frameIdToRootFrameId.has(mainFrameIds.frameId)) {
frameTreeEvents = keyEvents.filter(e => {
return e.args.frame && frameIdToRootFrameId.get(e.args.frame) === mainFrameIds.frameId;
});
} else {
log.warn(
'trace-of-tab',
'frameTreeEvents may be incomplete, make sure the trace has FrameCommittedInBrowser events'
);
frameIdToRootFrameId.set(mainFrameIds.frameId, mainFrameIds.frameId);
frameTreeEvents = frameEvents;
}

// Compute our time origin to use for all relative timings.
const timeOriginEvt = this.computeTimeOrigin(
Expand All @@ -630,6 +641,11 @@ class TraceProcessor {
// Compute the key frame timings for the main frame.
const frameTimings = this.computeKeyTimingsForFrame(frameEvents, {timeOriginEvt});

// Compute FCP for all frames.
const fcpAllFramesEvt = frameTreeEvents.find(
e => e.name === 'firstContentfulPaint' && e.ts > timeOriginEvt.ts
);

// Compute LCP for all frames.
const lcpAllFramesEvt = this.computeValidLCPAllFrames(frameTreeEvents, timeOriginEvt).lcp;

Expand Down Expand Up @@ -658,6 +674,7 @@ class TraceProcessor {
timeOrigin: frameTimings.timings.timeOrigin,
firstPaint: frameTimings.timings.firstPaint,
firstContentfulPaint: frameTimings.timings.firstContentfulPaint,
firstContentfulPaintAllFrames: maybeGetTiming(fcpAllFramesEvt && fcpAllFramesEvt.ts),
firstMeaningfulPaint: frameTimings.timings.firstMeaningfulPaint,
largestContentfulPaint: frameTimings.timings.largestContentfulPaint,
largestContentfulPaintAllFrames: maybeGetTiming(lcpAllFramesEvt && lcpAllFramesEvt.ts),
Expand All @@ -669,6 +686,7 @@ class TraceProcessor {
timeOrigin: frameTimings.timestamps.timeOrigin,
firstPaint: frameTimings.timestamps.firstPaint,
firstContentfulPaint: frameTimings.timestamps.firstContentfulPaint,
firstContentfulPaintAllFrames: fcpAllFramesEvt && fcpAllFramesEvt.ts,
firstMeaningfulPaint: frameTimings.timestamps.firstMeaningfulPaint,
largestContentfulPaint: frameTimings.timestamps.largestContentfulPaint,
largestContentfulPaintAllFrames: lcpAllFramesEvt && lcpAllFramesEvt.ts,
Expand All @@ -679,6 +697,7 @@ class TraceProcessor {
timeOriginEvt: frameTimings.timeOriginEvt,
firstPaintEvt: frameTimings.firstPaintEvt,
firstContentfulPaintEvt: frameTimings.firstContentfulPaintEvt,
firstContentfulPaintAllFramesEvt: fcpAllFramesEvt,
firstMeaningfulPaintEvt: frameTimings.firstMeaningfulPaintEvt,
largestContentfulPaintEvt: frameTimings.largestContentfulPaintEvt,
largestContentfulPaintAllFramesEvt: lcpAllFramesEvt,
Expand Down
20 changes: 18 additions & 2 deletions lighthouse-core/test/audits/__snapshots__/metrics-test.js.snap
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ Object {
"firstCPUIdle": 863,
"firstCPUIdleTs": 23466886143,
"firstContentfulPaint": 863,
"firstContentfulPaintAllFrames": 683,
"firstContentfulPaintAllFramesTs": 23466705983,
"firstContentfulPaintTs": 23466886143,
"firstMeaningfulPaint": 863,
"firstMeaningfulPaintTs": 23466886143,
Expand All @@ -24,6 +26,8 @@ Object {
"observedDomContentLoaded": 596,
"observedDomContentLoadedTs": 23466619325,
"observedFirstContentfulPaint": 863,
"observedFirstContentfulPaintAllFrames": 683,
"observedFirstContentfulPaintAllFramesTs": 23466705983,
"observedFirstContentfulPaintTs": 23466886143,
"observedFirstMeaningfulPaint": 863,
"observedFirstMeaningfulPaintTs": 23466886143,
Expand Down Expand Up @@ -62,6 +66,8 @@ Object {
"firstCPUIdle": 3790,
"firstCPUIdleTs": undefined,
"firstContentfulPaint": 2289,
"firstContentfulPaintAllFrames": undefined,
"firstContentfulPaintAllFramesTs": undefined,
"firstContentfulPaintTs": undefined,
"firstMeaningfulPaint": 2758,
"firstMeaningfulPaintTs": undefined,
Expand All @@ -77,6 +83,8 @@ Object {
"observedDomContentLoaded": 1513,
"observedDomContentLoadedTs": 713038536140,
"observedFirstContentfulPaint": 1122,
"observedFirstContentfulPaintAllFrames": 1122,
"observedFirstContentfulPaintAllFramesTs": 713038144775,
"observedFirstContentfulPaintTs": 713038144775,
"observedFirstMeaningfulPaint": 1122,
"observedFirstMeaningfulPaintTs": 713038144775,
Expand All @@ -85,8 +93,8 @@ Object {
"observedFirstVisualChange": 1105,
"observedFirstVisualChangeTs": 713038128064,
"observedLargestContentfulPaint": 1122,
"observedLargestContentfulPaintAllFrames": undefined,
"observedLargestContentfulPaintAllFramesTs": undefined,
"observedLargestContentfulPaintAllFrames": 1122,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, why did this one change? I wouldn't have expected movement here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, if there were no FrameTreeCommittedInBrowser events, only FCP-AF would fall back to FCP because we needed to avoid a NO_FCP error. LCP-AF didn't need to fall back because a NO_LCP_ALL_FRAMES error didn't brick a bunch of existing tests.

Now that frameTreeEvents falls back to frameEvents, LCP-AF falls back to LCP because it looks at frameTreeEvents.

"observedLargestContentfulPaintAllFramesTs": 713038144775,
"observedLargestContentfulPaintTs": 713038144775,
"observedLastVisualChange": 1722,
"observedLastVisualChangeTs": 713038745064,
Expand Down Expand Up @@ -115,6 +123,8 @@ Object {
"firstCPUIdle": 1582,
"firstCPUIdleTs": 225415754204,
"firstContentfulPaint": 499,
"firstContentfulPaintAllFrames": 499,
"firstContentfulPaintAllFramesTs": 225414670885,
"firstContentfulPaintTs": 225414670885,
"firstMeaningfulPaint": 783,
"firstMeaningfulPaintTs": 225414955343,
Expand All @@ -130,6 +140,8 @@ Object {
"observedDomContentLoaded": 560,
"observedDomContentLoadedTs": 225414732309,
"observedFirstContentfulPaint": 499,
"observedFirstContentfulPaintAllFrames": 499,
"observedFirstContentfulPaintAllFramesTs": 225414670885,
"observedFirstContentfulPaintTs": 225414670885,
"observedFirstMeaningfulPaint": 783,
"observedFirstMeaningfulPaintTs": 225414955343,
Expand Down Expand Up @@ -168,6 +180,8 @@ Object {
"firstCPUIdle": 4313,
"firstCPUIdleTs": undefined,
"firstContentfulPaint": 1337,
"firstContentfulPaintAllFrames": undefined,
"firstContentfulPaintAllFramesTs": undefined,
"firstContentfulPaintTs": undefined,
"firstMeaningfulPaint": 1553,
"firstMeaningfulPaintTs": undefined,
Expand All @@ -183,6 +197,8 @@ Object {
"observedDomContentLoaded": 560,
"observedDomContentLoadedTs": 225414732309,
"observedFirstContentfulPaint": 499,
"observedFirstContentfulPaintAllFrames": 499,
"observedFirstContentfulPaintAllFramesTs": 225414670885,
"observedFirstContentfulPaintTs": 225414670885,
"observedFirstMeaningfulPaint": 783,
"observedFirstMeaningfulPaintTs": 225414955343,
Expand Down
Loading