Add a standard track/category field to the spec

[bgirard]

I touched on this need in this discussion many years ago: #68 (comment) .

Frankly I've ran into this many times. You can see some screenshots in that discussion for how heavily user timings are used at Meta on web. More recently I'm looking at a tooling integration where this would be useful again. React-native already support user-timings but I'd like to forward it to the system's Perfetto on certain platforms and builds. Tracks are an important feature of Perfetto because it organizes the events in the Perfetto frontend.

Ideally I'd like to see the following happen here:

• We spec and document an optional track/category field to mark and measure. We document the expectation that it helps categories events into tracks in the UI to organize them together. This would all be non-normative expectations of downstream tools of course.
• Perhaps worth discussing if we want to introduce the concept of heavy/slow categories that can be enabled/disabled from tracing. This is concept that common to almost all tracing system (Perfetto, about:tracing, Firefox's Platform Profiler and many more tools). But we can defer this to another discussion.
• Consumers of the performance APIs (Browsers, react-native, perf libraries) can then optionally follow the standard recommendation and better display the marks in their respective front-ends.

[mmocny]

Regarding track, here's a simple strawman:

• We have options.detail dictionary for User Timing
• As you suggest, I think this would be a good fit for a non-normative suggestion.

In Chromium:

• We already extract values from PerformanceUserTiming and emit TRACE events which are picked up by tooling such as DevTools perf panel and Perfetto.
• TRACE events already support custom track by name.
• Chromium explicitly uses a hash of name, startTime, and endTime to define the track name of marks/measures, but we could use this options.detail.track value if provided.

(We may also want to change the default hash strategy to merge more things...)

---

Regarding the concept of optional slow/heavy categories:

• For cases where these are unconditionally measured and included in the performance timeline, you may want to hide / collapse the UI overload by default in developer tooling. I think track helps here.
• For cases where you want to optionally-enable marks/measures only if tracing is running (because theu are expensive to compute at runtime), well, I wonder if this should be solved with DevTools extensions and/or custom injected JS or cookie settings? As far as I know for React you already need to install DevTools extension to enable advanced instrumentation?

[bgirard]

TRACE events already support custom track by name.

If you mean Perfetto (trace-viewer too?) yes but not tools like DevTools, the primary frontend for performance.measure for most web product developers, there's no way to separate events into track in the frontend and you end up with something unmanageable like in my linked discussion example from a few years ago.

here's a simple strawman

That would be great. Suggesting a options.detail.track string as the standard for product and tools to communicate this information. Or perhaps options.detail.track.name to be more future proof if we ever needed to specify more track information in the future.

By glancing over at the Perfetto SDK/API, the one feature we would want would be a way to name the track. But that's not essential.

[mmocny]

If you mean Perfetto (trace-viewer too?) yes but not tools like DevTools, the primary frontend for performance.measure for most web product developers, there's no way to separate events into track in the frontend and you end up with something unmanageable like in my linked discussion example from a few years ago.

Fair enough! I think that becomes a distinct feature request for the DevTools product.

That would be great. Suggesting a options.detail.track string as the standard for product and tools to communicate this information. Or perhaps options.detail.track.name to be more future proof if we ever needed to specify more track information in the future.

+1

By glancing over at the Perfetto SDK/API, the one feature we would want would be a way to name the track. But that's not essential.

I know that capability is supported by Perfetto UI, but I guess it isn't "exposed"?

[bgirard]

I know that capability is supported by Perfetto UI, but I guess it isn't "exposed"?

Yes, it's supported. Here's an example of what I have so far with a private implementation of this proposal in React Native (showing the tracks coming from UserTimings, not shown are the existing native tracks). There's a good chance that these improvement will be merged within open source React Native eventually and fixing this spec issue would help:

There's shared JavaScript code running both on Web and React Native. I'm moving all the performance logging code away from custom logger to use UserTimings with this custom track feature. React Native then creates custom tracks using the Perfetto SDK and ideally someday Chrome/Firefox Devtools will implement this feature as well.

[and-oli]

Jumping in a little late to the conversation but folks might find this interesting.

This is an experimental API we recently made available in Chrome DevTools for extending the Performance panel. It basically fulfills the request for customizing the track User Timings appear within in the timeline: https://bit.ly/rpp-e11y.

It relies on the detail field of User Timings as the payload container and specifies a format for the data that allows to customize how the timing is displayed, including track, color and details info.

Extension data is expected to be contained under detail.devtools. For example, if you wanted to add data under a track called "React Profiler", you could do so by making a call similar to:

    const measureOptions = {
     // ...
      detail: {
        devtools: {
          metadata: {
            dataType:"track-entry",
            extensionName: 'React extension',
          },
          color: 'primary',
          // Track the entry will be displayed in
          track: 'React Profiler',
         }
      },
    };
    performance.measure( "A measurement", measureOptions);

The API is experimental because we are still looking for feedback from the ecosystem to launch something that's valuable, any thoughts would be appreciated!

[bgirard]

That sounds promising. I'll give it a shot in the next 2 weeks and will share some feedback.

[bgirard]

Found some time to test it. At first it didn't work after enabling the flag but I tried it in Canary and it worked there. I was able to easily retrofit Facebook Web's performance measures to this suggested API and get the following view:

There's more useful logging, like I showed in Perfetto, that we would expose now that we have the concept of tracks. Having this API standardized would really help and React Native would likely follow suit as well.

My spec relevant feedback is:

• Minimal overhead should be the primary concern. This API carries a lot of data per measure. Ideally we want to support high volumes of logging that. For instance a viable use case for this API would be measure for React component render. So a reasonable target might be 1,000 events/sec. Redefining the track and its metadata per each marker could be a bit redundant and expensive. But the web/client code could retain the 'devtools' sub-object and pass the same instance of the object for each mark. Although this wont work if we add keys that change every mark like detailsPair. We might want to split the object into a clear 'track' object that can be re-used, or have a separate API to define track metadata once. That's what the Perfetto SDK does BTW.
• I think 'trackName' is a better term here but I don't feel too strongly.
• Color should be optional.
• hintText is a nice addition, but it's not mandatory if it holds up this proposal.
• detailsPairs is a nice API as well. It would be nice getting an API to inject a custom component there (security nightmare I'm sure). Or perhaps just richer data like object reference (similar to console where you can expand objects), or images for features like screenshot. But again I wouldn't hold up the proposal on any of these issues.
• We could consider adding an API for the page to list what advanced category it supports and letting the devtool toggle them. Then the page could make a quick check if something like say 'expensive React perf logging' is enabled. I'm talking about categories that are expensive to even compute the measure for. I think that's similar to Chrome's "Disabled by default" category.

Non spec relevant feedback:

• I was expecting to see this as a sub-view of Timings. Even just ordering it higher in the view near Timings would be nice. Otherwise it can get missed easily.
• Happy to provide more feedback 1:1

[and-oli]

Thank you so much for taking the time to test the API and come back with such detailed feedback!

Minimal overhead should be the primary concern.

One way we thought about avoiding overhead in hot paths was by calling the performance.measure API with the measurements after the fact (since it can be tricky to ensure the call to the API is quick) . Say for example measurements are taken efficiently inside React component rendering, but the calls to the API are added at the end of the recording with their original timestamps, which UserTimings L3 allows to do. We are still figuring out what's the most ergonomic and maintainable way to achieve this, especially because the calls to the API might happen after trace events stop being buffered when the recording stops, but it's technically possible. How does this sound to you?

Color should be optional.

That's a good point and I agree. Will make sure we consider this.

detailsPairs is a nice API as well. It would be nice getting an API to inject a custom component there

This is something we have been thinking of but aren't sure what shape will be the best for it. We are waiting to get more feedback of possible use cases to have a better idea of this, but I think it isn't a blocker since it could be added progressively .

We could consider adding an API for the page to list what advanced category it supports and letting the devtool toggle them.

To make sure I'm following, would this work as a "filter" to allow the developer to select what will be recorded from an extension? If so, that could be pretty interesting. Will investigate if this is also beneficial for other potential early adopters.

I was expecting to see this as a sub-view of Timings. Even just ordering it higher in the view near Timings would be nice. Otherwise it can get missed easily.

That's also a fair point. Will also make sure we consider moving extension tracks closer to timings and likely above the main thread. Also note that you can customize the track order in the Performance panel (right click on the track name -> configure tracks)

Happy to provide more feedback 1:1

Sounds good! I appreciate that. I'll definitely set up something if the conversation extends and needs more space!

[bgirard]

calling the performance.measure API with the measurements after the fact

That moves a lot of complexity on to user code, regresses app size to implement a buffering mechanism. It would be much better for UserTimings to buffer itself if it needs to defer something costly. Web binding is cheap enough on all platform. So it's really just about not having too much redundant data in the object, or being able to easily reuse references between calls.

but the calls to the API are added at the end of the recording

That's the other problem. There's no way to observe that from web APIs currently. There's no 'flush your performance entries now' API. And as long as we can solve the problem above, I think we should continue to punt on adding one.

To make sure I'm following, would this work as a "filter" to allow the developer to select what will be recorded from an extension?

Yes exactly. It's common to want to select which 'categories' you'll be logging and paying for the cost for.

[and-oli]

After looking into the options to natively work around the performance overhead of the performance API, we are also exploring using the console API instead of performance. In particular, because performance (including measure/mark) is meant to buffer user observable data, which contributes to the API's slowness and is difficult to bypass natively because of the semantics of the API itself. console on the other hand is meant specifically to interact with devtools. For example, the sole purpose of console.timeStamp is to add data to the DevTools Performance panel, so it can be more easily optimized natively for performance, especially in the no-op cases (when not profiling / tracing with the perf panel).

We've added an exploration for this in the whatwgh repo: whatwg/console#140

The main disadvantage of this approach is the disregard for the existing adoption of user timings in the ecosystem, since people are using it for adding stuff to the performance tool but also to power telemetry / logs / etc. An alternative would be to enable performance.measure/mark instrumentation only on profiling/dev builds of apps but still the performance of the profiled application itself would be affected, potentially leading to misleading recordings.

[bgirard]

meant to buffer user observable data, which contributes to the API's slowness and is difficult to bypass natively because of the semantics of the API itself.

I agree. In React Native's implementation this is adding significant overhead. This is being optimized in facebook/react-native#45122. But I agree that fundamentally the semantics of the API are going to get in the way here.

Personally I don't feel strongly if it should go under either console or performance. The slight problem with putting it under console is that the data should be going under the performance tab of DevTools, not the console tab. So that also feels smelly.

Ultimately I think either approach has pros and cons. I think having cross vendor alignment here is really the most important. I would personally support either variant. If we go with measure we need to either have a fast implementation or add an option to avoid creating `performanceEntry's.