Proposal for JSDoc comment support to inform a better JavaScript editing experience.

[CyrusNajmabadi]

Pull Request here: #2646

Problem

Providing a good JavaScript editing experience in a cross-platform/cross-editor manner is challenging. JavaScript is a highly dynamic language, and users may write code in a manner that our default TypeScript language rules don't provide a good experience for.

For example, a user may write the following:

var val = "some_value";
if (someExpr()) {
     val = somethingThatReturnsNumber();
}
val     // <-- completion here.

TypeScript's view of this variable is that it is a string (due to its initializer). Leading to an IDE experience like so:

(notice the lack of number properties)

The solution to this in TypeScript code is easy, just write your code as:

var val: number|string = "some_value";
if (someExpr()) {
     val = somethingThatReturnsNumber();
}
val.     // <-- completion here.

Of course, then you're not typing JavaScript anymore, defeating the purpose of providing a good JavaScript editing experience. :-)

Proposed Solution

There is, fortunately, an approach already taken by the JavaScript community to help out here. JSDoc comments support a number of mechanisms to allow declaring valuable information about their declarations. As part of our work to better support editing JavaScript using the TypeScript compiler, we can inform the typing experience based on these JSDoc comments. These comments already appear to be widely used in the JavaScript community, and there is ample tooling support for them already in many other tools.

Here's an example of how we can improve our editing scenario if we take advantage of these annotations:

/** @type {number|string} */
var val = "some_value";
if (someExpr()) {
     val = somethingThatReturnsNumber();
}
val.     // <-- completion here.

If we allow these JSDoc type annotations to inform the compiler's type-system in JavaScript files, then we can see a very nice set of functionality light up immediately like so:


(notice both the string and number properties listed).

I've gone ahead and added support for a subset of JSDoc annotations that make sense in our compiler's existing system. This is an initial starting point for JSDoc support, and I am performing continued investigations to see what other sort of JSDoc concepts are used commonly, and what we may be able to cheaply provide support for. Current support has been added for a wide variety of types that we see in JSDoc types including:

1. Normal primitive types. i.e.: number, string, etc.
2. Type references. i.e.: ArrayBuffer. When these are to types already in some .d.ts files, this is naturally understood by the compiler.
3. Generic types. i.e. Array<number>.
4. The * type. This is the all type for JSDoc, and maps to our any type.
5. The ? type. This maps to the unknown type.
6. The union type. i.e. number|string.
7. The nullable type. i.e. number? or ?number. There is no direct representation for this in our compiler, so we map this simply to the underlying element type (in this case number).
8. The non-nullable type. i.e. number! or !number. There is no direct representation for this in our compiler, so we map this simply to the underlying element type (in this case number).
9. The record type. i.e. { a: number, b: string }

Additionally, based on how we've seen JSDoc used in practice on GitHub, we also support some common types like

1. The array type. i.e. number[]
2. The tuple type. i.e. [number,string]

On top of the types that we understand and can map to our existing TypeScript compiler types, we also understand function, param and return annotations. For example, a variable with an annotation of the form:

/** @type {function(new:ArrayBuffer,number)} */
var createBuffer;

Is understood to have a normal TypeScript construct signature of:

Within function or param annotations we support

1. The new-type annotation. i.e. function(new:ArrayBuffer,number). This represents a construct signature.
2. The this-type annotation. i.e. function(this:ArrayBuffer,string). The this annotation indicates the type that this has within the body of the function. Interestingly enough, there is no TypeScript way to do the same thing (something we might want to reconsider).
3. The reset annotation. i.e. function(...number). This naturally maps to our own ... - rest parameter concept.
4. The optional annotation. i.e. function(number=). This naturally maps to our ? - optional parameter concept.
5. The Template annotations. i.e. @template T. These naturally map to our own Type-Parameters.

Here are examples of a few of these interesting cases:

Implementation

Adding support for this was fairly simple and fit nicely within our existing compiler with very little effort. The work was broken down into the following pieces:

1. Parsing. This was fairly simple. I added parsing functionality to support handling JSDoc comments. And, when parsing a JavaScript (.js) file, we simply look for and parse out these constructs on declarations they make sense for (currently, functions and variables).
2. Binding. When the binder is binding a JavaScript file, we look for these JSDoc comments and handle certain parts of them appropriately. This includes handling @template annotations and appropriately adding their Type-Parameters to the parent function. Handling record types ({ ... }) in a similar manner to how we handle TypeScript Object-Type-Literals. And handling function types function(type1, etc.) in a similar manner to how we handle TypeScript call/construct signatures.
3. Checking. In certain targeted locations, we provide specialized code-paths if we are in a JavaScript file. For example, when asking 'what is the return type of this function?' (i.e. getReturnTypeFromBody), we check if this is a JavaScript node, and if it has an @return annotation. If so, we defer to that as the canonical source of truth about this function's return type. After all, if this user declared it as such, then we should respect that declaration. If no such annotation can be found, we fall back to our normal inference model. The places touched in the checker are fairly minimal and reuse existing compiler functionality as much as possible.
4. Emitting. No work was done here. There is no concept of 'emit' for a JavaScript file. It's already in its final form.

The work was done in the core compiler itself as it is the most natural place for this understanding. By doing the understanding at the compiler layer, all IDE features naturally light-up, with no additional effort. If this work was done at the Services layer it would require an enormous amount of effort. To understand this, let's work with a simple example:

If the Services layer wanted to handle this properly, it would not be sufficient to merely understand the types of x and y. It would also have to understand how those types flowed and changed through arbitrary expression. This is a somewhat simple case here but would already require the Services layer to implement understanding of binary expressions. This immediately spirals into too much complexity the moment we would need to understand the gamut of other cases supported in the compiler (like overload resolution, contextual types, generics, etc.)

Feedback is welcome on the current design and implementation. I will be spending the immediate present doing more investigations into how JSDoc is used in the wild to better flesh things out from here.

Thanks!

[JsonFreeman]

looks good!

[danquirk]

The current plan is that this should only effect the editing experience in JavaScript files. We should consider whether we want this to work in TypeScript files as well. A couple reasons:

• As noted, the JSDoc experience actually adds features that TypeScript doesn't currently support, namely 'this' typing which is a much requested feature.(Meta-issue: this disambiguator #513).
• For users migrating from JavaScript to TypeScript it would be unfortunate if after renaming their files to .ts suddenly their editing experience degrades as the significant effort they put into JSDoc comments is no longer leveraged by the editor. In an ideal world we could simply promote their JSDoc comments to real type annotations via an IDE command.

This does create additional complications where there are now 2 type annotations for a given element that we would want to verify and keep in sync. It's probably not an option to delete the JSDoc comments since there're plenty of good reasons to have some later portion in your pipeline still use JSDoc comments even after the TypeScript compiler is done with your file(s).

[aozgaa]

@danquirk: Regarding the second point, if the user wants to have JSDoc annotations in their code, then perhaps a solution would be to

1. support converting JSDoc comments to TypeScript type annotations, and
2. Add facilities for emitting type information in the form of JSDoc comments.

Then the programmer needs maintain only one copy of the annotations in the form of TS files.

There remains the outstanding question of whether we might be emitting too much type information, rendering the output file bloated for the remaining tools in the pipeline.

[danquirk]

@aozgaa great minds ;) #2916