Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse browser compatibility metadata #144

Merged
merged 8 commits into from
Feb 2, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions tool/generator/bcd.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
// Copyright (c) 2024, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import 'dart:convert' hide json;
import 'dart:js_interop';

import 'package:path/path.dart' as p;

import 'filesystem_api.dart';

/// A class to read from the browser-compat-data files and parse interface and
/// property status (standards track, experimental, deprecated) and supported
/// browser (chrome, safari, firefox) info.
class BrowserCompatData {
static BrowserCompatData read() {
final path =
p.join('node_modules', '@mdn', 'browser-compat-data', 'data.json');
final content = (fs.readFileSync(
path.toJS,
JSReadFileOptions(encoding: 'utf8'.toJS),
) as JSString)
.toDart;

final api = (jsonDecode(content) as Map)['api'] as Map<String, dynamic>;
final interfaces = api.keys
.where((key) => !key.startsWith('_'))
devoncarew marked this conversation as resolved.
Show resolved Hide resolved
.map((key) => BCDInterfaceStatus(key, api[key] as Map<String, dynamic>))
.toList();
return BrowserCompatData(Map.fromIterable(
interfaces,
key: (i) => (i as BCDInterfaceStatus).name,
));
}

final Map<String, BCDInterfaceStatus> interfaces;

BrowserCompatData(this.interfaces);

BCDInterfaceStatus? interfaceFor(String name) => interfaces[name];
devoncarew marked this conversation as resolved.
Show resolved Hide resolved
}

class BCDInterfaceStatus extends BCDItem {
late final Map<String, BCDPropertyStatus> properties;

BCDInterfaceStatus(super.name, super.json) {
final names = json.keys.where((key) => !key.startsWith('_'));
properties = Map.fromIterable(
names,
value: (key) => BCDPropertyStatus(
key as String, json[key] as Map<String, dynamic>, this),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MDN exposes compatibility info for dictionary interfaces with an _option suffix e.g. https://github.com/mdn/browser-compat-data/blob/cfa15e085ceb88cdee391ae09bd52ee3674d5ecc/api/MediaDevices.json#L264C10-L264C29. We could handle this here by having a separate "dictionaryOptions" field in BCDInterfaceStatus or we can just do the lookup in the generation script.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, interesting. This would be metadata about parameters for an interface's method?

It looks like these options exist off properties? So we'd want BCDInterfaceStatus to contain BCDPropertyStatus items, and for those items to have optional options (anything w/ an _option suffix).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to help me think about the organization in the bcd.json file, there are ~16 foo_option instances in there (all fields off of interface properties?).

https://gist.github.com/devoncarew/81c987aa55637b6014c5b400a7edf022

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funnily enough, that info is actually a separate thing altogether and I think they're suffixed with _parameter: https://github.com/mdn/browser-compat-data/blob/c2761cb84874ff5b9b0a230e7a557a5a43b9b939/api/EventTarget.json#L160C10-L160C27. :D Also see https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener#browser_compatibility.

This is for what options can go in an options object that is passed to an API. https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia#browser_compatibility

There's also "_parameter_optional" apparently for optional parameters: https://github.com/mdn/browser-compat-data/blob/c2761cb84874ff5b9b0a230e7a557a5a43b9b939/api/HTMLTableRowElement.json#L378.

I think we may want to validate all the strings and check for _ to make sure we're handling the various types of compatibility info. There's also some guidance here: https://github.com/mdn/browser-compat-data/blob/main/docs/data-guidelines/index.md#parameters-and-parameter-object-features, but it doesn't seem to catch every variant unfortunately. Realistically, there are probably not enough APIs that use all the variants for this to matter from our perspective, but something to watch out for.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, taking another pass through this, it looks like neither _option nor _parameter show up in the first two levels of the data (the interfaces and their properties).

Separately, I do see some types we're generating interop code for that aren't represented in the BCD data. I expect we'll need to adjust how we parse and use the data as we start to filter how much of the IDL we generate code for.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, taking another pass through this, it looks like neither _option nor _parameter show up in the first two levels of the data (the interfaces and their properties).

Right, we can do this in another CL to grab more information about a specific member or constructor.

Separately, I do see some types we're generating interop code for that aren't represented in the BCD data. I expect we'll need to adjust how we parse and use the data as we start to filter how much of the IDL we generate code for.

I suspect the majority of these cases to be because the types are new and are still being drafted, and therefore MDN doesn't have information on them. It'll be useful to validate that, but I suspect if the type is not present, we can safely omit the type.

);
}

BCDPropertyStatus? propertyFor(String name) => properties[name];
}

class BCDPropertyStatus extends BCDItem {
final BCDInterfaceStatus parent;

BCDPropertyStatus(super.name, super.json, this.parent);

bool get statusDifferentFromParent =>
parent.statusDescription != statusDescription;
}

abstract class BCDItem {
final String name;
final Map<String, dynamic> json;

BCDItem(this.name, this.json);

Map<String, dynamic> get _compat => json['__compat'] as Map<String, dynamic>;
Map<String, dynamic> get _status => _compat['status'] as Map<String, dynamic>;
Map<String, dynamic> get _support =>
_compat['support'] as Map<String, dynamic>;

bool get deprecated => _status['deprecated'] as bool? ?? false;
bool get experimental => _status['experimental'] as bool? ?? false;
bool get standardTrack => _status['standard_track'] as bool? ?? false;

List<String> get status => [
if (standardTrack) 'standards-track',
if (deprecated) 'deprecated',
if (experimental) 'experimental',
];

String get statusDescription => status.join(', ');
devoncarew marked this conversation as resolved.
Show resolved Hide resolved

bool get chromeSupported => _versionAdded('chrome');
bool get firefoxSupported => _versionAdded('firefox');
bool get safariSupported => _versionAdded('safari');

List<String> get browsers => [
if (chromeSupported) 'chrome',
if (firefoxSupported) 'firefox',
if (safariSupported) 'safari',
];

String get browsersDescription => browsers.join(', ');

int get browserCount => browsers.length;

bool _versionAdded(String browser) {
devoncarew marked this conversation as resolved.
Show resolved Hide resolved
final map = (_support[browser] is List
? (_support[browser] as List).first
: _support[browser]) as Map<String, dynamic>;

if (map.containsKey('version_removed')) {
return false;
}

final value = map['version_added'];
if (value is String) return true;
if (value is bool) return value;
return false;
}

@override
String toString() => '$name ($browsersDescription) [$statusDescription]';
}
13 changes: 6 additions & 7 deletions tool/generator/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tool/generator/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@
"author": "Dart project authors",
"license": "BSD 3",
"dependencies": {
"@mdn/browser-compat-data": "^5.5.2",
"@webref/css": "^6.10.0",
"@webref/idl": "^3.23.0"
},
"devDependencies": {
"web-specs": "^2.74.1",
"webidl2": "^24.2.2"
}
}
85 changes: 7 additions & 78 deletions tool/generator/translator.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,13 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import 'dart:convert';
import 'dart:js_interop';

import 'package:code_builder/code_builder.dart' as code;
import 'package:path/path.dart' as p;

import 'banned_names.dart';
import 'filesystem_api.dart';
import 'bcd.dart';
import 'singletons.dart';
import 'type_aliases.dart';
import 'type_union.dart';
Expand Down Expand Up @@ -443,15 +442,15 @@ class Translator {
final _includes = <idl.Includes>[];

late String _currentlyTranslatingUrl;
late WebSpecs webSpecs;
late BrowserCompatData browserCompatData;

/// Singleton so that various helper methods can access info about the AST.
static Translator? instance;

Translator(
this.packageRoot, this._librarySubDir, this._cssStyleDeclarations) {
instance = this;
webSpecs = WebSpecs.read();
browserCompatData = BrowserCompatData.read();
}

/// Set or update partial interfaces so we can have a unified interface
Expand Down Expand Up @@ -496,10 +495,6 @@ class Translator {
final libraryPath = '$_librarySubDir/${shortName.kebabToSnake}.dart';
assert(!_libraries.containsKey(libraryPath));

// TODO: Use the info from the spec to skip generation of some libraries.
// ignore: unused_local_variable
final spec = webSpecs.specFor(shortName)!;

final library = _Library(this, '$packageRoot/$libraryPath');
_libraries[libraryPath] = library;

Expand Down Expand Up @@ -762,6 +757,7 @@ class Translator {
code.ExtensionType _extensionType({
required String jsName,
required String dartClassName,
required BCDInterfaceStatus? interfaceStatus,
required List<String> implements,
required _OverridableConstructor? constructor,
required List<_OverridableOperation> operations,
Expand Down Expand Up @@ -808,6 +804,8 @@ class Translator {
// private classes, and make their first character uppercase in the process.
final dartClassName = isNamespace ? '\$${capitalize(jsName)}' : jsName;

final status = browserCompatData.interfaceFor(name);

// We create a getter for namespaces with the expected name. We also create
// getters for a few pre-defined singleton classes.
final getterName = isNamespace ? jsName : singletons[jsName];
Expand All @@ -826,6 +824,7 @@ class Translator {
_extensionType(
jsName: jsName,
dartClassName: dartClassName,
interfaceStatus: status,
implements: implements,
constructor: interfacelike.constructor,
operations: operations,
Expand Down Expand Up @@ -885,73 +884,3 @@ class Translator {
return dartLibraries;
}
}

class WebSpecs {
static WebSpecs read() {
final path = p.join('node_modules', 'web-specs', 'index.json');
final content = (fs.readFileSync(
path.toJS,
JSReadFileOptions(encoding: 'utf8'.toJS),
) as JSString)
.toDart;
return WebSpecs(
(jsonDecode(content) as List)
.map((json) => WebSpec(json as Map<String, dynamic>))
.toList(),
);
}

final List<WebSpec> specs;

WebSpecs(this.specs);

WebSpec? specFor(String shortName) {
for (final spec in specs) {
if (spec.shortname == shortName) {
return spec;
}
}

for (final spec in specs) {
if (spec.seriesShortname == shortName) {
return spec;
}
}

return null;
}
}

class WebSpec {
final Map<String, dynamic> json;

WebSpec(this.json);

String get url => json['url'] as String;

String get shortname => json['shortname'] as String;

String? get seriesShortname {
if (!json.containsKey('series')) return null;
return (json['series'] as Map)['shortname'] as String?;
}

String get standing => json['standing'] as String;

List<String> get categories {
if (json.containsKey('categories')) {
return (json['categories'] as List).cast<String>();
} else {
return const [];
}
}

String? get releaseStatus {
if (!json.containsKey('release')) return null;
return (json['release'] as Map)['status'] as String?;
}

@override
String toString() =>
'$shortname $url $standing [${categories.join(',')}] $releaseStatus';
}
Loading