Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Modularize Library #6974

Closed
3 of 4 tasks
yuit opened this issue Feb 9, 2016 · 13 comments
Closed
3 of 4 tasks

Proposal: Modularize Library #6974

yuit opened this issue Feb 9, 2016 · 13 comments
Assignees
Labels
Committed The team has roadmapped this issue Fixed A PR has been merged for this issue Suggestion An idea for TypeScript @types Relates to working with .d.ts files (declaration/definition files) from DefinitelyTyped

Comments

@yuit
Copy link
Contributor

yuit commented Feb 9, 2016

Problem

Currently TypeScript compiler accepts a single compiler option, target, which specify both version of library to be included during compilation and version of JavaScript for emitting code together.
The behaviour presents limitations in two parts of the pipeline: consuming the default library and granularly controlling how JavaScript will be emitted (e.g. what features to be down-level etc.).
The proposal will mainly focus on the first issue regarding using the default library.

The current behavior of target option doesn't allow users to have a fine-grain control on what library or feature of the library to be included, and user cannot independently control library's version from emit JavaScript version. As summarized in #4692 Proposal: Granular Targeting and #4168 Normalize our lib files by compiler setting

  • Emit ES5 JavaScript while using ES6 library during design-time (and vice-versa).
  • Include only sub-part of the ES6/ESnext library
  • Modifying default libraries can conflict with users-defined ones.
  • Using host-specific library.

Workaround

As @yortus points out in the #4692 Proposal: Granular Targeting , there are possible workarounds for above issues by maintaing customized version of the default library while target another version of emit JavaScript.
Such approach is cumbersome to maintain and consume any necessary fixes of the default library.

Related Issues:

  • #4692 Proposal: Granular Targeting
  • #4168 Normalize our lib files by compiler settings
  • #3215 New APIs added to lib.d.ts may break client codes. Allow duplicated members in interfaces? Make lib.d.ts overridable?
  • #3005 Using ES6 type default library when targetting ES5 output

Proposed Solution

There are two parts to the proposed solution:

  1. Introducing a new compiler flag --lib. The flag will be used to allow users to granularly control the default library.
    Note: target flag will remain. Its behaviors with --lib flag will be discussed below.
  2. Breaking down the default library into smaller files, especially ES6 library and any future JavaScript library.

Compiler Flag --lib

The flag's options (see below for the full list) allows users to specify what library to be included into the compilation.
The flag's options can be separated into following categories:

  • JavaScript only:

    es3

    es5

    es6

  • Host only:

    node

    dom

    dom.iterable

    webworker

    scripthost

  • ES6 or ESNext by-features options:

    es6.array

    es6.collection

    es6.function

    es6.generator

    es6.iterable

    es6.math

    es6.number

    es6.object

    es6.promise

    es6.proxy

    es6.reflect

    es6.regexp

    es6.string

    es6.symbol

    es6.symbol.wellknown

    es7.array.include

The --lib flag is an optional flag and multiple options can be used in combination (e.g --lib es6,dom.iterable) and each option will be mapped to associated library.
So in this example, files: lib.es6.d.ts and lib.dom.iterable.d.ts will be loaded into the compilation..
When --lib is specified, --target will be used solely to indicate version of emitting JavaScript.

In additional to above behavior, we still need to determine what to be included when --lib is not specified, should it remain the same, including version of library specifying by --target and all host libraries, or including every library. -> _Current implementation is to include version of the library using --target

The proposal will only affect when the compiler attempts to create compilation in Program.ts. Other parts of the pipeline will not be affected by the proposal.

Usage:

The --lib flag will take an options in the form of string separated by comma. Space will not be allowed as a separator. This rule apply to both command line and tsconfig

Valid

tsc --target es5 --lib es6
tsc --target es5 --lib es5,es6.array
"compilerOptions": {
    ...
    "lib": "es5,es6.array"
}

Invalid command-line

tsc --target es5--lib es5, es6.array  // es6.array will be considered a file-name
tsc --target es5 --lib es5 es6.array  // es6.array will be considered a file-name
"compilerOptions": {
    ...
    "lib": "es5, es6.array"
}

Working Items:

@yuit yuit added Suggestion An idea for TypeScript Committed The team has roadmapped this issue labels Feb 9, 2016
@yuit yuit self-assigned this Feb 9, 2016
@yuit yuit added this to the TypeScript 2.0 milestone Feb 9, 2016
@yortus
Copy link
Contributor

yortus commented Feb 9, 2016

Thanks @yuit, great to see progress on this.

I'm wondering about how this handles cases where there is crossover between features. For example, consider the following project setup:

Example Project

  • Browser-based, so it must either avoid unsupported features or ensure they are adequately polyfilled.
  • Promise is polyfilled by the project as necessary so we can rely on that.
  • Generic iterables are not widely supported (eg not supported in any IE or Safari version) so should not be used in source code. This means we want only one of the following two definitions:
interface PromiseConstructor {
    ...
    all<T>(values: ArrayLike<T | PromiseLike<T>>): Promise<T[]>;  // Do include this
    all<T>(values: Iterable<T | PromiseLike<T>>): Promise<T[]>;   // DO NOT include this
}
  • Well-known symbols like Symbol.species and Symbol.toStringTag are not widely supported so should not be used in source code.

How to get just the right type definitions?

For the above project, I'd like to tell tsc: "I want to include definitions for Promise in the build, but NOT definitions for well-known symbols or methods that take generic iterables". That way, I get type safety for Promises and compile-time failure (rather than runtime failure) if I try to use unsupported constructs like Promise.all(someIterable).

I believe situations like this really get to the point of granular targeting. I've used Promise as an example but there are similar crossover issues with RegExp, Date, typed arrays, Map, Set, and other builtins. We want early failure for referencing things we already know aren't going to work reliably at run time. The compiler can really help here by letting us catch these errors at transpile time, rather than runtime.

How to handle feature crossover?

I may have missed something, but I don't see any handling of feature crossover in this proposal as it is currently stated. It looks to me like the following would be the best that can be done:

  • Promise in es6.d.ts is currently all-or-nothing, including definitions that use well-known symbols and generic iterables, even if we explicitly want to avoid them.
  • Alternatively, there could be a separate symbols.d.ts with well-known symbol declarations placed there, and taken out of places like promise.d.ts It could use declaration merging to add to existing declarations, like interface PromiseConstructor { [Symbol.species]: Function; }.
  • But the declaration-merging approach will sometimes introduce declarations that we've told the compiler not to include. E.g. if we said we want to include symbols but not promises, then we'll still have a Promise and PromiseConstructor defined by symbols.d.ts.

This is where a tsc-internal mechanism that does the equivalent of conditional compilation shines - it starts with all known definitions and filters out anything not supported on a line-by-line basis, which covers cross-over between features perfectly at arbitrary levels of granuarity. See for example this es6.d.ts. I'm NOT suggesting conditional compilation, just a purely internal mechanism that does effectively the same thing to get the correct definitions, with no more and no less than what the user asked for via --lib.

@yuit
Copy link
Contributor Author

yuit commented Feb 9, 2016

@yortus Thanks for the feedback and pointing out some holes in the proposal.

How to get just the right type definitions?

For the above project, I'd like to tell tsc: "I want to include definitions for Promise in the build, but NOT definitions for well-known symbols or methods that take generic iterables".

Yes you are right that the current proposal when you pick feature such as promise.d.ts, it is all or nothing. You cannot just pick only sub-part of the promise.d.t.

So building upon your proposed solution, we can 1) separate symbol into its own file (probably same as iterator) 2) separate symbol in Promise or other es6 features in to their own files like es6.promiseWithSymbol.d.ts which will get included when you specified that you want es6.symbol. I am trying to not to overly granularly separate the files so we don't end up creating large amounts of tiny lib. though for this as you suggest here there may not be too bad. Also, to clarify, I think these smaller files like es6.promiseWithSymbol.d.ts should not be exposed as options to users.

@yuit yuit mentioned this issue Feb 9, 2016
4 tasks
@yortus
Copy link
Contributor

yortus commented Feb 10, 2016

@yuit sounds reasonable :)

I am trying to not to overly granularly separate the files so we don't end up creating large amounts of tiny lib

I agree there are definitely diminishing returns to supporting finer granularity. However, I do think the choice of "grains" should be informed by how the browsers/JS runtimes actually add feature support over time, which is what this issue is meant to address after all.

For example, the Symbol builtin is widely supported now, but well-known symbols still have very little support. I can definitely make use of Symbol in my node projects today, but I must avoid pretty much all well-known symbols. And they have pretty different use-cases anyway.

So, while I would not suggest having a separate flag for each of the 2 dozen or so well-known symbols, I would suggest having a separate flag for ES6.Symbol and for ES6.WellKnownSymbols, based on their actual availability.

I think these smaller files like es6.promiseWithSymbol.d.ts should not be exposed as options to users

Agreed, the number and names of all these .d.ts files can just be an implementation detail known to tsc. It can pick the right set of files according to a naming convention based on the supplied --lib flags.

@alexeagle
Copy link
Contributor

@mhegazy note that it would be nice for one of the new lib.es6* files to be the minimal one for
tsc --target es6 --noLib node_modules/typescript/lib/lib.d.ts app.ts
(from #5504)

@yuit
Copy link
Contributor Author

yuit commented Mar 9, 2016

@alexeagle what you mean by lib.es6.* to be minimal? Do you mean you just want es5 lib? ( like what you describe in the issue #5504)?

@alexeagle
Copy link
Contributor

First preference would be to support --target es6 with only the es5 lib, with a fix for #5504

If that's not possible for some reason, then I'd like to have the lib.d.ts that expresses just the types needed by the compiler to make it work, to make the workaround less painful. (I have Duplicate identifier in our internal build because of the types I had to specify in our custom std lib)

@yuit
Copy link
Contributor Author

yuit commented Mar 9, 2016

@alexeagle Thanks for the clarification. The new compiler flag will allow you to get your preference fix for #5504 😄
You will now be able to do --target es6 --lib es5 (and include any es6 per-feature library)

@mhegazy
Copy link
Contributor

mhegazy commented Apr 4, 2016

This should be fixed by #7715. --lib support should be available now in typescript@next and in the next TypeScript release.

@basarat
Copy link
Contributor

basarat commented Apr 24, 2016

Going through and adding to alm. I know it says "lib": "es5, es6.array". But wouldn't "lib": ["es5","es6.array"] be better in tsconfig.json? Is this how future array options (if there aren't any yet) be represented in tsconfig.json's compilerOptions?

Thanks! 🌹

@basarat
Copy link
Contributor

basarat commented Apr 24, 2016

More questions:

  • What would be the result of getDefaultLibFileName in language service host (perhaps we should start going with FileNames (notice s) for these apis.
  • Similarly getDefaultLibFileName should have an s suffix.
  • Also does the user type es6.array or es2015.array. Note: the file is called lib.es2015.array.ts

Here is a sample of the function I needed to delegate to eventually:

export const getDefaultLibFilePaths = (options: ts.CompilerOptions): string[] => {
    if (options.noLib) {
        return [];
    }
    if (options.lib) {
        /** Note: this might need to be more fancy at some point. E.g. user types `es6.array` but we need to get `es2015.array` */
        return options.lib.map((fileName) => fileFromLibFolder(`lib.${fileName}.d.ts`));
    }
    return [fileFromLibFolder(ts.getDefaultLibFileName(options))];
}

With this I have it working in alm. Feedback appreciated 🌹

lib support

@basarat
Copy link
Contributor

basarat commented Apr 24, 2016

Is this how future array options (if there aren't any yet) be represented in tsconfig.json's compilerOptions

In fact rootDirs is an array from the discussion : #8245 (comment) Maybe lib should be too in tsconfig.json

@basarat
Copy link
Contributor

basarat commented Apr 25, 2016

Verified by @2426021684 (TypeStrong/atom-typescript#918 (comment)) it is an array, as it should be 🌹

@yuit
Copy link
Contributor Author

yuit commented Jun 23, 2016

Update our decision on to ship node.d.ts or not to ship, the outcome is that we will not ship the node.d.ts as our default lib for the following reasons:

  • what version of node.d.ts we will support?
  • nodejs constantly ship new version and cause change in node.d.ts, how we will ship the update in node.d.ts that we already ship (especially problematic in VS case)?
  • additional effort to maintain yet a separate node.t.s instead of just in "@types"
  • shipping node.d.ts doesn't seem to provide any much easier workflow, in comparison to just install one from "@types"

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Committed The team has roadmapped this issue Fixed A PR has been merged for this issue Suggestion An idea for TypeScript @types Relates to working with .d.ts files (declaration/definition files) from DefinitelyTyped
Projects
None yet
Development

No branches or pull requests

5 participants