feat(compiler): add type declaration with relay config file types #4569

noghartt · 2024-01-07T00:54:46Z

resolve #4037

captbaritone · 2024-01-09T17:22:03Z

I love this idea, and I've wanted it for some time. I was hopeful that there would be some way to use serde to derive this, but so far I haven't found such a took that works with our somewhat complicated setup.

I'm a little hesitant to merge this given that I don't think it's fully complete. For example I don't see feature flags or mutli-project config. Would you be interested in taking it the rest of the way and capturing the full config? If so, I think this would be a great addition.

If we do go this route, I'd love to add a few more things:

Comments on the relevent config structs reminding people that any changes to the struct should also be added to these types.
Documentation updates adding the types as comments, similar to what next-js does (/** @type {import('next').NextConfig} */

Thanks for taking the time to propose this!

noghartt · 2024-01-09T18:05:11Z

I'm a little hesitant to merge this given that I don't think it's fully complete. For example I don't see feature flags or mutli-project config. Would you be interested in taking it the rest of the way and capturing the full config? If so, I think this would be a great addition.

I'd love to take the rest of the config and documentate it.

But yes, it isn't fully complete because I didn't find some good documentations related to the rest of the config. I saw that the relay-config crate has other fields, but I couldn't find anything that documentate these configurations, any idea where I can find it?

What I follow was the README from the react-compiler package, but I think that is a little outdated, right?

tobias-tengler · 2024-01-09T18:15:56Z

If it's just about adding the TypeScript declarations, I think you can work of the declaration file I derived from the config last year: https://github.com/facebook/relay/blob/d873d5b8927b4a5bb6ca2c4b9a4c504ea0723b6b/packages/relay-compiler/index.d.ts
I would also like to continue my effort regarding the VSCode integration for config completion in JSON files: #4162 Now that I have some more experience with the compiler, maybe I have more luck than back then :D

sibelius

cool

captbaritone · 2024-01-09T18:48:09Z

@noghartt

I couldn't find anything that documentate these configurations, any idea where I can find it?

Sadly the best source of truth is the Rust crate. Luckily most of the fields are commented with public-facing comments. You can find it here: https://github.com/facebook/relay/blob/main/compiler/crates/relay-compiler/src/config.rs#L931

You'll need to learn a bit about how Serde (which is a Rust crate that can derived a parser for a Rust structure based on its definition) works to understand what the JS/JSON shape will look like. The main thinks to look out for:

Each child value/object will have its own deserialization logic, usually derived from a Serde derived trait at the structs declaration (simple values like strings/paths/etc generally have intuitive inputs)
rename_all = "camelCase" converts snake_case names to camelCase
#[serde(flatten)] will push all the keys of the child object up into the parent object.

Thank you so much for your interest in this. The lack of documentation for the Relay config is a huge gap, and TypeScript types will offer both in-editor support and human readable documentation that we can point people to.

noghartt · 2024-01-10T17:03:02Z

cc @captbaritone @tobias-tengler

I improved the types and wrote more JSDocs based on the config.rs as mentioned before. Improve the doc too, what do you think?

captbaritone

So exciting! This is going to be such a good improvement. A few requests:

Could you do another pass to ensure no properties which are actually optional are marked as non-optional?
In the Rust code, can you add a comment at the top of each struct that is part of this config with a note that any changes made to the struct should also be made to these types?

packages/relay-compiler/index.d.ts

captbaritone · 2024-01-10T18:30:34Z

packages/relay-compiler/index.d.ts

+  variableNamesComment?: boolean;
+  featureFlags?: RelayConfigFeatureFlags;
+  /** A placeholder for allowing extra information in the config file */
+  extra: unknown;


Presumably this should be optional?

This should probably just be [configKey: string]: any?

I pushed a fix into this tht turns it into an optional field.

This should probably just be [configKey: string]: any?

In case, I think that if we should type it, we should type accordingly to the serde_json::Value struct, that can be any JSON-friendly value. What do you think?

captbaritone · 2024-01-10T18:40:29Z

Another point: We don't have authoritative docs on this config. As a baseline we should link out to this definition from our docs as a "Here's where you can go to find out what options are available in the config". As a stretch, I wonder if it would be possible to use something like https://www.npmjs.com/package/docusaurus-plugin-typedoc to generate an actual docs page (or pages) from this?

Maybe we can start with the former and then (optionally) explore the later in a separate PR.

…te index.d.ts accordingly to struct

noghartt · 2024-01-10T21:07:33Z

Another point: We don't have authoritative docs on this config. As a baseline we should link out to this definition from our docs as a "Here's where you can go to find out what options are available in the config". As a stretch, I wonder if it would be possible to use something like https://www.npmjs.com/package/docusaurus-plugin-typedoc to generate an actual docs page (or pages) from this?

Maybe we can start with the former and then (optionally) explore the later in a separate PR.

Would be really cool if we could have a docs specifically for these configs, explaining which things each field are doing. I think that we can derive some docs from this config, like one about feature flags and what are the features related.

noghartt · 2024-01-15T13:48:10Z

Anything else I need to fix, what do you think?

cc @captbaritone @tobias-tengler

captbaritone · 2024-01-16T17:26:35Z

packages/relay-compiler/index.d.ts

+   */
+  jsModuleFormat?: 'commonjs' | 'haste';
+  /** Options for configuring the output of compiler diagnostics. */
+  diagnosticReportConfig: RelayConfigDiagnosticReport;


I think we need to understand defaults a little better, since I know a config without this value (for example, but others as well) is accepted by the compiler.

Ah, looking a bit more closely, it looks like the struct itself is annotated with default. Can you check each of the structs and see how that impacts these types?

relay/compiler/crates/relay-compiler/src/config.rs

Line 666 in ece1fd9

#[serde(deny_unknown_fields, rename_all = "camelCase", default)]

I checked all the structs and update some JSDocs with new @default values. I think that everything is OK now, what do you think?

captbaritone · 2024-01-16T17:27:59Z

website/docs/getting-started/installation-and-setup.md

@@ -74,7 +75,7 @@ module.exports = {
 ```

 This configuration also can be specified in `"relay"` section of the `package.json` file.
-For more details, and configuration options see: [Relay Compiler Configuration](https://github.com/facebook/relay/tree/main/packages/relay-compiler)
+For more details, and configuration options see: [Relay Compiler Configuration](https://github.com/facebook/relay/tree/main/packages/relay-compiler/index.d.ts)


Love this! Could you also add a similar note to https://github.com/facebook/relay/blob/main/packages/relay-compiler/README.md#configuration?

Added! I updated the README with a note admonition containing the same info.

packages/relay-compiler/index.d.ts

facebook-github-bot · 2024-01-18T00:08:42Z

@captbaritone has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

captbaritone · 2024-01-18T00:10:46Z

I've imported this, but it will race with #4585 which changes the config types. I'll try to fix it up internally before I land.

captbaritone · 2024-01-18T00:14:10Z

Hey. After importing this, I'm seeing a bunch more errors. How have you been validating this addition? Could you find a way to ensure it's working end to end on a given config file? Maybe include a screenshot of what errors look like in VSCode?

captbaritone · 2024-01-18T00:15:08Z

packages/relay-compiler/index.d.ts

+  | { 
+      /** The path of your schema file. */
+      schema: string 
+    };


I don't think a semicolon is valid here.

I think that this isn't a problem, but I removed it to follow the pattern

captbaritone · 2024-01-18T00:15:11Z

packages/relay-compiler/index.d.ts

+   * ```
+   */
+  diagnosticReportConfig?: RelayConfigDiagnosticReportConfig;
+  featureFlags?: RelayFeatureFlags;


This type is not defined.

captbaritone · 2024-01-18T00:15:32Z

packages/relay-compiler/index.d.ts

+   * (TypeScript only) Whether to use the `import type` syntax introduced in Typescript version 3.8. This will prevent warnings from `importsNotUsedAsValues`.
+   */
+  useImportTypeSyntax?: boolean;
+  customScalarTypes?: CustomScalarTypes;


This type is not defined

captbaritone · 2024-01-18T00:15:43Z

packages/relay-compiler/index.d.ts

+     * 
+     * @default false
+     */
+    noFutureProofEnums?: NoFutureProofEnums;


This type is not defined

I defined this type, fixed

captbaritone · 2024-01-18T00:16:05Z

packages/relay-compiler/index.d.ts

+   * inlined or not depends on whether it actually uses that directive.
+   */
+  no_inline?: RelayConfigFeatureFlag;
+  enable_3d_branch_arg_generation?: bool;


bool is not a valid TypeScript type.

Oh, nice catch, fixed

captbaritone · 2024-01-18T00:16:14Z

packages/relay-compiler/index.d.ts

+  /** Create normalization nodes for client edges to client objects. */
+  emit_normalization_nodes_for_client_edges?: boolean;
+  /** Fully build the normalization AST for Resolvers. */
+  enable_resolver_normalization_ast?: bool;


bool is not a valid TypeScript type.

captbaritone · 2024-01-18T00:37:08Z

@noghartt Can you please add a test plan that demonstrates this working end to end? For example, a screenshot of it reporting errors on an invalid config file in VSCode? (and one of it not erroring on a valid config file)?

noghartt · 2024-01-18T01:04:49Z

@noghartt Can you please add a test plan that demonstrates this working end to end? For example, a screenshot of it reporting errors on an invalid config file in VSCode? (and one of it not erroring on a valid config file)?

Of course. But in case, VSCode and LSP does not infer type checking over type infered objects that comes from JSDocs annotation, so it'll won't trigger a visual error for the final user, only if they have a relay.config.ts and let the tsserver does it for us.

But in case, I took some screenshots that shows this type working, I'll let also a test plan containing how I test it:

Test plan

build the packages using gulp with the command yarn gulp mainrelease
create a new relay.config.js file
add type annotations targeting the exported RelayConfig type from ./dist build package

the boilerplate

// relay.config.js

/** @type {import('./dist/relay-compiler').RelayConfig} */
module.exports = {
  // ...
}

Screenshots

starting single project

multi project config

feature flags

As an example, even the next.config.js aren't being typechecked by the LSP.

facebook-github-bot · 2024-01-18T17:21:59Z

@captbaritone has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

alunyov · 2024-01-19T14:27:27Z

compiler/crates/common/src/feature_flags.rs

@@ -17,6 +17,7 @@ use serde::Serialize;

 use crate::Rollout;

+/// **NOTE**: Every change on this struct should be reflected into the `relay-compiler/index.d.ts` file.


I wonder if we can make use of something like this: https://github.com/Aleph-Alpha/ts-rs here?

We would generate these typescript definitions based on the actual rust types?

I did some POCs with specta yesterday, it seems to works well. I think that we can bring it to these structs too.

They have just a limitation around some JSDocs comments, like the @default and others, but it's working.

Nice, @noghartt! I think this would be a much better solution to always keep these definitions in sync.

@noghartt Just checking in here. Were you able to make any progress with Spectra? Do you think it can offer a viable automated solution?

@noghartt Just checking in here. Were you able to make any progress with Spectra? Do you think it can offer a viable automated solution?

I think that we can bring an automated solution yes. But as I commented, specta doesn't seem to fit all the types for us, as an example, it doesn't fill the JSDocs related fields, like the @default. I couldn't find a workaround for that scenario.

Except that, it seems to works well. Do you think that we can bring this improvement in this PR or maybe moving to another?

Summary: One huge usability issue with Relay today is that the compiler config is wholely undocumented. To understand what options are available you either need to _try_ an option and try to decipher the sometime cryptic validation errors, or read the Rust code. This PR attempts to start to close that gap. Our Rust structs and comments _are_ a good source of truth for this info, so ideally we still use that as our documentation/validation. So, here we derive [Json Schema](https://json-schema.org/) from our Rust structs/enums/types/serde tags. In VSCode this allows for nice autocomplete, hover tool tips, and yellow squiggles if you make an error (much nicer than a cryptic validation error when you run the compiler). Some next steps this could enable: 1. Enable this editor experience directly if you have the Relay VSCode extension installed via the [JSON Schema contribution point](https://code.visualstudio.com/api/references/contribution-points#contributes.jsonValidation) 2. Derive a docs page about the [Docusaurus JSON Schema plugin](https://jy95.github.io/docusaurus-json-schema-plugin/) https://github.com/facebook/relay/assets/162735/7f5e807b-069c-4344-9e9c-506f3369b042 ## Next Steps There are a few unanswered things here that we'll need to consider: 1. How do we publish this schema such that users can find it and ensure that they use the correct schema for the version of the compiler they are using? 1. Especially tricky if we want to enable it automatically in VSCode 2. Maybe we could have the compiler validate that you've added the right value for `$schema`? 3. Can we get the docs plugin to work? (I think it might require Docusaurus 3) 4. Are there bugs? There are some serde attributes and little deserialization tricks which we might not have modeled correctly here. Difficult to know if we've captured them all. 5. Add more descriptions to our structs. Some fields are undocumented. This would provide us more incentive to get those struct/field/variant `///` comments very helpful and up to date. ## Acknowledgment This PR aims to be a more maintainable and reliable alternative to #4569. Thanks for noghartt for pushing the discussion of how we might fill this gap. This approach was also previously explored in other PRs: * #4227 by tbezman * #4162 by tobias-tengler Pull Request resolved: #4723 Reviewed By: tyao1 Differential Revision: D58936067 Pulled By: captbaritone fbshipit-source-id: c58488da4981eed285ee705ac65321146cfb576b

noghartt · 2024-06-25T23:41:36Z

Well done by @captbaritone on this PR #4723. We can follow this improvement and bring some type emission following the ideas of JsonSchema!

feat(compiler): add type declaration with relay config file types

2b9cf4a

facebook-github-bot added the CLA Signed label Jan 7, 2024

sibelius reviewed Jan 9, 2024

View reviewed changes

noghartt added 2 commits January 10, 2024 11:52

feat(compiler): add more types and jsdoc

b1f18d3

docs(relay): improve docs with @type imported on relay.config.js setup

1f7eb8f

captbaritone requested changes Jan 10, 2024

View reviewed changes

noghartt added 3 commits January 10, 2024 17:46

feat(compiler): improve types on index.d.ts

3bfdc10

feat(compiler): add NOTE comment on config structs to be able to upda…

bea4df7

…te index.d.ts accordingly to struct

docs(compiler): replace docs for index.d.ts

37ca368

noghartt requested a review from captbaritone January 10, 2024 20:50

Merge branch 'main' into feat/config-file-declaration

74cab82

Merge branch 'main' into feat/config-file-declaration

8ed8f09

captbaritone requested changes Jan 16, 2024

View reviewed changes

noghartt added 2 commits January 16, 2024 19:18

feat(compiler: update README

533f4c1

feat(compiler): update @default type JSDocs

13a0a96

noghartt requested a review from captbaritone January 17, 2024 00:19

captbaritone requested changes Jan 17, 2024

View reviewed changes

packages/relay-compiler/index.d.ts Outdated Show resolved Hide resolved

packages/relay-compiler/index.d.ts Outdated Show resolved Hide resolved

fix(compiler): bring optional to types

5deb0b8

noghartt requested a review from captbaritone January 17, 2024 02:29

captbaritone requested changes Jan 18, 2024

View reviewed changes

fix(compiler): define missing types

47b75c4

noghartt requested a review from captbaritone January 18, 2024 00:33

alunyov reviewed Jan 19, 2024

View reviewed changes

noghartt mentioned this pull request Jan 31, 2024

chore(doc): improve relay compiler doc for persist #4608

Closed

captbaritone mentioned this pull request Jun 23, 2024

Generate JSON schema for compiler config #4723

Closed

noghartt closed this Jun 25, 2024

noghartt deleted the feat/config-file-declaration branch June 25, 2024 23:41

		@@ -17,6 +17,7 @@ use serde::Serialize;

		use crate::Rollout;

		/// NOTE: Every change on this struct should be reflected into the `relay-compiler/index.d.ts` file.

feat(compiler): add type declaration with relay config file types #4569

feat(compiler): add type declaration with relay config file types #4569

Conversation

noghartt commented Jan 7, 2024

captbaritone commented Jan 9, 2024

noghartt commented Jan 9, 2024

tobias-tengler commented Jan 9, 2024

sibelius left a comment

Choose a reason for hiding this comment

captbaritone commented Jan 9, 2024

noghartt commented Jan 10, 2024

captbaritone left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

noghartt Jan 10, 2024 • edited Loading

Choose a reason for hiding this comment

captbaritone commented Jan 10, 2024

noghartt commented Jan 10, 2024

noghartt commented Jan 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

facebook-github-bot commented Jan 18, 2024

captbaritone commented Jan 18, 2024

captbaritone commented Jan 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

captbaritone commented Jan 18, 2024

noghartt commented Jan 18, 2024

Test plan

Screenshots

facebook-github-bot commented Jan 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

noghartt commented Jun 25, 2024

noghartt Jan 10, 2024 •

edited

Loading