Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse browser compatibility metadata #144

Merged
merged 8 commits into from
Feb 2, 2024
Merged

Parse browser compatibility metadata #144

merged 8 commits into from
Feb 2, 2024

Conversation

devoncarew
Copy link
Member

@devoncarew devoncarew commented Jan 15, 2024

  • parse the browser-compat-data for the APIs we generate

See also https://github.com/mdn/browser-compat-data.

This PR does not use the info currently; follow-ups to this PR should likely use this info to reduce the amount of API that we generate (i.e., don't generate an API that does not support standards track; don't generate an API w/ no supported browsers; consider not generating an API w/ only one supported browser).


  • I’ve reviewed the contributor guide and applied the relevant portions to this PR.
Contribution guidelines:

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

@kevmoo
Copy link
Member

kevmoo commented Jan 15, 2024

How does the annotation help end users of this package? I love the idea of using this data to prune the output.

@devoncarew
Copy link
Member Author

How does the annotation help end users of this package? I love the idea of using this data to prune the output.

It'll probably depend on what we choose for our criteria to emit or exclude generation of APIs. If they're all the same (supports standards track and all three browsers) than the metadata won't provide any real value.

But the APIs currently support the full gamut of maturity and browser support:

  • a few aren't standards track
  • a few don't support any (desktop) browsers
  • many only support one browser (i.e., only chrome, or, only firefox)
  • a few support two but not all three browsers

@devoncarew
Copy link
Member Author

devoncarew commented Jan 15, 2024

Not standards track:

  • BeforeInstallPromptEvent
  • HTMLPortalElement

no supported browsers:

  • ContactAddress
  • ContactsManager
  • ContentIndex
  • ContentIndexEvent
  • DevicePosture
  • NDEFMessage
  • NDEFRecord
  • NDEFReader
  • NDEFReadingEvent
  • XRHand
  • XRJointSpace
  • XRJointPose
  • XRCompositionLayer
  • XRProjectionLayer
  • XRQuadLayer
  • XRCylinderLayer
  • XREquirectLayer
  • XRCubeLayer
  • XRSubImage
  • XRWebGLSubImage
  • XRMediaBinding
  • XRLayerEvent
only one supported browser:
  • WebGLTimerQueryEXT (chrome)
  • EXT_disjoint_timer_query (chrome)
  • EXT_disjoint_timer_query_webgl2 (chrome)
  • IdentityCredential (chrome)
  • IdentityProvider (chrome)
  • WEBGL_compressed_texture_pvrtc (safari)
  • Accelerometer (chrome)
  • LinearAccelerationSensor (chrome)
  • GravitySensor (chrome)
  • AmbientLightSensor (chrome)
  • XRAnchor (chrome)
  • XRAnchorSet (chrome)
  • AudioSession (safari)
  • BackgroundFetchManager (chrome)
  • BackgroundFetchRegistration (chrome)
  • BackgroundFetchRecord (chrome)
  • BackgroundFetchEvent (chrome)
  • BackgroundFetchUpdateUIEvent (chrome)
  • SyncManager (chrome)
  • SyncEvent (chrome)
  • BatteryManager (chrome)
  • CookieStore (chrome)
  • CookieStoreManager (chrome)
  • CookieChangeEvent (chrome)
  • ExtendableCookieChangeEvent (chrome)
  • PasswordCredential (chrome)
  • FederatedCredential (chrome)
  • CSSScopeRule (chrome)
  • PaintWorkletGlobalScope (chrome)
  • PaintRenderingContext2D (chrome)
  • PaintSize (chrome)
  • CSSPseudoElement (firefox)
  • CSSStartingStyleRule (chrome)
  • ViewTransition (chrome)
  • CaretPosition (firefox)
  • DataCue (safari)
  • DeprecationReportBody (chrome)
  • DocumentPictureInPicture (chrome)
  • DocumentPictureInPictureEvent (chrome)
  • EditContext (chrome)
  • TextUpdateEvent (chrome)
  • TextFormat (chrome)
  • TextFormatUpdateEvent (chrome)
  • CharacterBoundsUpdateEvent (chrome)
  • PerformanceElementTiming (chrome)
  • EyeDropper (chrome)
  • HTMLFencedFrameElement (chrome)
  • FencedFrameConfig (chrome)
  • Fence (chrome)
  • GamepadPose (firefox)
  • Sensor (chrome)
  • SensorErrorEvent (chrome)
  • Gyroscope (chrome)
  • VisibilityStateEntry (chrome)
  • Navigation (chrome)
  • NavigationHistoryEntry (chrome)
  • NavigationTransition (chrome)
  • NavigateEvent (chrome)
  • NavigationDestination (chrome)
  • NavigationCurrentEntryChangeEvent (chrome)
  • IdleDetector (chrome)
  • Ink (chrome)
  • InkPresenter (chrome)
  • InputDeviceCapabilities (chrome)
  • InterventionReportBody (chrome)
  • Scheduling (chrome)
  • Profiler (chrome)
  • Keyboard (chrome)
  • KeyboardLayoutMap (chrome)
  • LayoutShift (chrome)
  • LayoutShiftAttribution (chrome)
  • FontData (chrome)
  • PerformanceLongTaskTiming (chrome)
  • TaskAttributionTiming (chrome)
  • Magnetometer (chrome)
  • BeforeInstallPromptEvent (chrome)
  • MediaSourceHandle (chrome)
  • CropTarget (chrome)
  • BrowserCaptureMediaStreamTrack (chrome)
  • MediaStreamTrackProcessor (chrome)
  • NetworkInformation (chrome)
  • OrientationSensor (chrome)
  • AbsoluteOrientationSensor (chrome)
  • RelativeOrientationSensor (chrome)
  • PaymentManager (chrome)
  • CanMakePaymentEvent (chrome)
  • PaymentRequestEvent (chrome)
  • PeriodicSyncManager (chrome)
  • PeriodicSyncEvent (chrome)
  • HTMLPortalElement (chrome)
  • Presentation (chrome)
  • PresentationRequest (chrome)
  • PresentationAvailability (chrome)
  • PresentationConnectionAvailableEvent (chrome)
  • PresentationConnection (chrome)
  • PresentationConnectionCloseEvent (chrome)
  • PresentationReceiver (chrome)
  • PresentationConnectionList (chrome)
  • PushSubscriptionChangeEvent (safari)
  • XRCamera (chrome)
  • Sanitizer (firefox)
  • CaptureController (chrome)
  • ScrollTimeline (chrome)
  • ViewTimeline (chrome)
  • FragmentDirective (chrome)
  • Serial (chrome)
  • SerialPort (chrome)
  • BarcodeDetector (chrome)
  • SharedStorageWorklet (chrome)
  • SharedStorageWorkletGlobalScope (chrome)
  • SharedStorageOperation (chrome)
  • SharedStorageRunOperation (chrome)
  • SharedStorageSelectURLOperation (chrome)
  • SharedStorage (chrome)
  • WindowSharedStorage (chrome)
  • WorkletSharedStorage (chrome)
  • SpeechGrammarList (chrome)
  • TimeEvent (firefox)
  • TrustedHTML (chrome)
  • TrustedScript (chrome)
  • TrustedScriptURL (chrome)
  • TrustedTypePolicyFactory (chrome)
  • TrustedTypePolicy (chrome)
  • NavigatorUAData (chrome)
  • URLPattern (chrome)
  • VirtualKeyboard (chrome)
  • LaunchParams (chrome)
  • LaunchQueue (chrome)
  • Bluetooth (chrome)
  • BluetoothDevice (chrome)
  • BluetoothRemoteGATTServer (chrome)
  • BluetoothRemoteGATTService (chrome)
  • BluetoothRemoteGATTCharacteristic (chrome)
  • BluetoothCharacteristicProperties (chrome)
  • BluetoothRemoteGATTDescriptor (chrome)
  • BluetoothUUID (chrome)
  • OTPCredential (chrome)
  • AudioSinkInfo (chrome)
  • MediaStreamTrackAudioSourceNode (firefox)
  • AudioDecoder (chrome)
  • AudioEncoder (chrome)
  • EncodedAudioChunk (chrome)
  • AudioData (chrome)
  • ImageDecoder (chrome)
  • ImageTrackList (chrome)
  • ImageTrack (chrome)
  • WGSLLanguageFeatures (chrome)
  • GPUExternalTexture (chrome)
  • GPUPipelineError (chrome)
  • GPURenderBundle (chrome)
  • GPUError (chrome)
  • GPUInternalError (chrome)
  • HID (chrome)
  • HIDDevice (chrome)
  • HIDConnectionEvent (chrome)
  • HIDInputReportEvent (chrome)
  • RTCIdentityAssertion (firefox)
  • WebTransportSendStream (firefox)
  • WebTransportReceiveStream (firefox)
  • USB (chrome)
  • USBConnectionEvent (chrome)
  • USBInTransferResult (chrome)
  • USBOutTransferResult (chrome)
  • USBIsochronousInTransferPacket (chrome)
  • USBIsochronousInTransferResult (chrome)
  • USBIsochronousOutTransferPacket (chrome)
  • USBIsochronousOutTransferResult (chrome)
  • USBDevice (chrome)
  • USBConfiguration (chrome)
  • USBInterface (chrome)
  • USBAlternateInterface (chrome)
  • USBEndpoint (chrome)
  • XRDepthInformation (chrome)
  • XRCPUDepthInformation (chrome)
  • XRWebGLDepthInformation (chrome)
  • XRHitTestSource (chrome)
  • XRTransientInputHitTestSource (chrome)
  • XRHitTestResult (chrome)
  • XRTransientInputHitTestResult (chrome)
  • XRRay (chrome)
  • XRLightProbe (chrome)
  • XRLightEstimate (chrome)
  • XRSystem (chrome)
  • XRSession (chrome)
  • XRRenderState (chrome)
  • XRFrame (chrome)
  • XRSpace (chrome)
  • XRReferenceSpace (chrome)
  • XRBoundedReferenceSpace (chrome)
  • XRView (chrome)
  • XRViewport (chrome)
  • XRRigidTransform (chrome)
  • XRPose (chrome)
  • XRViewerPose (chrome)
  • XRInputSource (chrome)
  • XRInputSourceArray (chrome)
  • XRLayer (chrome)
  • XRWebGLLayer (chrome)
  • XRSessionEvent (chrome)
  • XRInputSourceEvent (chrome)
  • XRInputSourcesChangeEvent (chrome)
  • XRReferenceSpaceEvent (chrome)
  • XRWebGLBinding (chrome)
  • WindowControlsOverlay (chrome)
  • WindowControlsOverlayGeometryChangeEvent (chrome)
  • ScreenDetails (chrome)
  • ScreenDetailed (chrome)
missing one browser (2 of 3 supported):
  • CSPViolationReportBody (chrome, safari)
  • EXT_texture_norm16 (chrome, safari)
  • KHR_parallel_shader_compile (chrome, safari)
  • OVR_multiview2 (chrome, firefox)
  • WEBGL_compressed_texture_etc (chrome, safari)
  • WEBGL_compressed_texture_etc1 (chrome, safari)
  • WEBGL_multi_draw (chrome, safari)
  • ContentVisibilityAutoStateChangeEvent (chrome, firefox)
  • FontFaceSetLoadEvent (chrome, firefox)
  • CSSStyleValue (chrome, safari)
  • StylePropertyMapReadOnly (chrome, safari)
  • StylePropertyMap (chrome, safari)
  • CSSUnparsedValue (chrome, safari)
  • CSSVariableReferenceValue (chrome, safari)
  • CSSKeywordValue (chrome, safari)
  • CSSNumericValue (chrome, safari)
  • CSSUnitValue (chrome, safari)
  • CSSMathValue (chrome, safari)
  • CSSMathSum (chrome, safari)
  • CSSMathProduct (chrome, safari)
  • CSSMathNegate (chrome, safari)
  • CSSMathInvert (chrome, safari)
  • CSSMathMin (chrome, safari)
  • CSSMathMax (chrome, safari)
  • CSSMathClamp (chrome, safari)
  • CSSNumericArray (chrome, safari)
  • CSSTransformValue (chrome, safari)
  • CSSTransformComponent (chrome, safari)
  • CSSTranslate (chrome, safari)
  • CSSRotate (chrome, safari)
  • CSSScale (chrome, safari)
  • CSSSkew (chrome, safari)
  • CSSSkewX (chrome, safari)
  • CSSSkewY (chrome, safari)
  • CSSPerspective (chrome, safari)
  • CSSMatrixComponent (chrome, safari)
  • CSSImageValue (chrome, safari)
  • CustomStateSet (chrome, firefox)
  • PerformanceEventTiming (chrome, firefox)
  • EventCounts (chrome, firefox)
  • FileSystemWritableFileStream (chrome, firefox)
  • AudioTrack (chrome, firefox)
  • External (chrome, firefox)
  • ImageCapture (chrome, firefox)
  • LargestContentfulPaint (chrome, firefox)
  • CanvasCaptureMediaStreamTrack (chrome, safari)
  • OverconstrainedError (chrome, safari)
  • DeviceMotionEventAcceleration (chrome, firefox)
  • DeviceMotionEventRotationRate (chrome, firefox)
  • PictureInPictureWindow (chrome, safari)
  • PictureInPictureEvent (chrome, safari)
  • RemotePlayback (chrome, safari)
  • ReportBody (chrome, safari)
  • Report (chrome, safari)
  • ReportingObserver (chrome, safari)
  • IdleDeadline (chrome, firefox)
  • Scheduler (chrome, firefox)
  • TaskPriorityChangeEvent (chrome, firefox)
  • TaskController (chrome, firefox)
  • TaskSignal (chrome, firefox)
  • SpeechRecognition (chrome, safari)
  • SpeechRecognitionErrorEvent (chrome, safari)
  • SpeechRecognitionAlternative (chrome, safari)
  • SpeechRecognitionResult (chrome, safari)
  • SpeechRecognitionResultList (chrome, safari)
  • SpeechRecognitionEvent (chrome, safari)
  • SpeechGrammar (chrome, firefox)
  • ReadableStreamBYOBReader (chrome, firefox)
  • ReadableByteStreamController (chrome, firefox)
  • ReadableStreamBYOBRequest (chrome, firefox)
  • Touch (chrome, firefox)
  • TouchList (chrome, firefox)
  • TouchEvent (chrome, firefox)
  • VideoDecoder (chrome, safari)
  • VideoEncoder (chrome, safari)
  • EncodedVideoChunk (chrome, safari)
  • VideoFrame (chrome, safari)
  • VideoColorSpace (chrome, safari)
  • GPUSupportedLimits (chrome, firefox)
  • GPUSupportedFeatures (chrome, firefox)
  • GPUAdapterInfo (chrome, firefox)
  • GPU (chrome, firefox)
  • GPUAdapter (chrome, firefox)
  • GPUDevice (chrome, firefox)
  • GPUBuffer (chrome, firefox)
  • GPUTexture (chrome, firefox)
  • GPUTextureView (chrome, firefox)
  • GPUSampler (chrome, firefox)
  • GPUBindGroupLayout (chrome, firefox)
  • GPUBindGroup (chrome, firefox)
  • GPUPipelineLayout (chrome, firefox)
  • GPUShaderModule (chrome, firefox)
  • GPUCompilationMessage (chrome, firefox)
  • GPUCompilationInfo (chrome, firefox)
  • GPUComputePipeline (chrome, firefox)
  • GPURenderPipeline (chrome, firefox)
  • GPUCommandBuffer (chrome, firefox)
  • GPUCommandEncoder (chrome, firefox)
  • GPUComputePassEncoder (chrome, firefox)
  • GPURenderPassEncoder (chrome, firefox)
  • GPURenderBundleEncoder (chrome, firefox)
  • GPUQueue (chrome, firefox)
  • GPUQuerySet (chrome, firefox)
  • GPUCanvasContext (chrome, firefox)
  • GPUDeviceLostInfo (chrome, firefox)
  • GPUValidationError (chrome, firefox)
  • GPUOutOfMemoryError (chrome, firefox)
  • GPUUncapturedErrorEvent (chrome, firefox)
  • MIDIInputMap (chrome, firefox)
  • MIDIOutputMap (chrome, firefox)
  • MIDIAccess (chrome, firefox)
  • MIDIPort (chrome, firefox)
  • MIDIInput (chrome, firefox)
  • MIDIOutput (chrome, firefox)
  • MIDIMessageEvent (chrome, firefox)
  • MIDIConnectionEvent (chrome, firefox)
  • RTCTransformEvent (firefox, safari)
  • RTCRtpScriptTransformer (firefox, safari)
  • RTCRtpScriptTransform (firefox, safari)
  • RTCPeerConnectionIceErrorEvent (chrome, safari)
  • RTCIceTransport (chrome, safari)
  • RTCError (chrome, safari)
  • RTCErrorEvent (chrome, safari)
  • WebTransportDatagramDuplexStream (chrome, firefox)
  • WebTransport (chrome, firefox)
  • WebTransportBidirectionalStream (chrome, firefox)
  • WebTransportError (chrome, firefox)
  • VTTRegion (firefox, safari)

@devoncarew
Copy link
Member Author

Upleveling this a bit, I think the important things are:

  • giving us the ability to not generate code for APIs which are generally not supported / available / generally used
  • giving the users additional info about the APIs we do generate, when that info is useful to them

That 2nd point could be via annotations, a table in the generated docs, ...

@bivens-dev
Copy link

There is a push amongst from W3C and all the major browser teams to standardise some of that comparability data via “Baseline” which as an end user I absolutely would find super helpful to know about in the code.

@devoncarew devoncarew marked this pull request as draft January 23, 2024 19:00
@devoncarew
Copy link
Member Author

There is a push amongst from W3C and all the major browser teams to standardise some of that comparability data via “Baseline” which as an end user I absolutely would find super helpful to know about in the code.

Thanks for the pointer! I took a look at baseline; the information in there looks less comprehensive than what we have in browser-compat-data today. But perhaps baseline will evolve to be the best system-of-record for the info?

In any case, we're likely going to look at what the best system to record for this info is (baseline? browser-compat-data? the w3c specs themselves?) and use that to evolve how many of these APIs that we generate, and whether we generate any w/ some experimental verbiage.

Moving this PR to a draft as we're less likely to land this specific implementation.

@devoncarew devoncarew marked this pull request as ready for review January 30, 2024 17:28
@devoncarew devoncarew changed the title Added browser compatibility metadata to the generated API. Parse browser compatibility metadata Jan 30, 2024
@devoncarew
Copy link
Member Author

@srujzs - this PR has been updated to parse the browser compat metadata but not to use it during generation (it no longer generates annotations w/ the data).

@srujzs
Copy link
Contributor

srujzs commented Jan 30, 2024

@srujzs - this PR has been updated to parse the browser compat metadata but not to use it during generation (it no longer generates annotations w/ the data).

Cool, thanks! Is this ready for review or are you iterating on it still?

@devoncarew
Copy link
Member Author

It's ready for review now -

Copy link
Contributor

@srujzs srujzs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Mostly just some nits.

tool/generator/bcd.dart Outdated Show resolved Hide resolved
tool/generator/bcd.dart Outdated Show resolved Hide resolved
tool/generator/bcd.dart Outdated Show resolved Hide resolved
tool/generator/bcd.dart Outdated Show resolved Hide resolved
properties = Map.fromIterable(
names,
value: (key) => BCDPropertyStatus(
key as String, json[key] as Map<String, dynamic>, this),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MDN exposes compatibility info for dictionary interfaces with an _option suffix e.g. https://github.com/mdn/browser-compat-data/blob/cfa15e085ceb88cdee391ae09bd52ee3674d5ecc/api/MediaDevices.json#L264C10-L264C29. We could handle this here by having a separate "dictionaryOptions" field in BCDInterfaceStatus or we can just do the lookup in the generation script.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, interesting. This would be metadata about parameters for an interface's method?

It looks like these options exist off properties? So we'd want BCDInterfaceStatus to contain BCDPropertyStatus items, and for those items to have optional options (anything w/ an _option suffix).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to help me think about the organization in the bcd.json file, there are ~16 foo_option instances in there (all fields off of interface properties?).

https://gist.github.com/devoncarew/81c987aa55637b6014c5b400a7edf022

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funnily enough, that info is actually a separate thing altogether and I think they're suffixed with _parameter: https://github.com/mdn/browser-compat-data/blob/c2761cb84874ff5b9b0a230e7a557a5a43b9b939/api/EventTarget.json#L160C10-L160C27. :D Also see https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener#browser_compatibility.

This is for what options can go in an options object that is passed to an API. https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia#browser_compatibility

There's also "_parameter_optional" apparently for optional parameters: https://github.com/mdn/browser-compat-data/blob/c2761cb84874ff5b9b0a230e7a557a5a43b9b939/api/HTMLTableRowElement.json#L378.

I think we may want to validate all the strings and check for _ to make sure we're handling the various types of compatibility info. There's also some guidance here: https://github.com/mdn/browser-compat-data/blob/main/docs/data-guidelines/index.md#parameters-and-parameter-object-features, but it doesn't seem to catch every variant unfortunately. Realistically, there are probably not enough APIs that use all the variants for this to matter from our perspective, but something to watch out for.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, taking another pass through this, it looks like neither _option nor _parameter show up in the first two levels of the data (the interfaces and their properties).

Separately, I do see some types we're generating interop code for that aren't represented in the BCD data. I expect we'll need to adjust how we parse and use the data as we start to filter how much of the IDL we generate code for.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, taking another pass through this, it looks like neither _option nor _parameter show up in the first two levels of the data (the interfaces and their properties).

Right, we can do this in another CL to grab more information about a specific member or constructor.

Separately, I do see some types we're generating interop code for that aren't represented in the BCD data. I expect we'll need to adjust how we parse and use the data as we start to filter how much of the IDL we generate code for.

I suspect the majority of these cases to be because the types are new and are still being drafted, and therefore MDN doesn't have information on them. It'll be useful to validate that, but I suspect if the type is not present, we can safely omit the type.

@devoncarew devoncarew merged commit 393ed83 into main Feb 2, 2024
11 checks passed
@devoncarew devoncarew deleted the browser_compat branch February 2, 2024 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants