Migrate the TypeScript Project to Use Modules

[DanielRosenwasser]

Follow-up from #34899

Today, TypeScript is authored in a relatively outdated style that uses what are called "namespaces" (formerly "internal modules"). This used to have some advantages in being able to globally access the compiler in the browser, but nowadays bundlers make this possible without exposing internal authoring details.

Benefits of Modules

Authoring with namespaces has several disadvantages today, and moving to using ES module syntax per file would likely solve those issues. Here are some of the advantages to moving to modules.

Dog-fooding

Using namespaces separates our team from the way that the community authors code. We have to learn about bugs related to module resolution from external bug reports rather than hitting them ourselves on the team. That means we can't dog-food our product the same as most modern users.

We would also hit some of the challenges module users hit with external tooling (more on this later) but this is a good thing! It will help us build the appropriate empathy to improve the situation for users.

Leaner package size with share-able bundles

TypeScript ships at least 5 different JavaScript files that are meant for different purposes - but because of the nature of namespaces, the contents of every core file is duplicated in each output file - basically, think of this as us shipping 5 virtually identical .exes because of their dependencies statically linked rather than being split into .dlls. So as a result, you end up with checker.ts's contents included 5 times in every install of typescript.

Ideally, tsc.js and typescript.js could share a majority of the code generated from src/compiler.

As an end result, we would fix #27891

Faster builds and editing

Every keystroke involves re-checking the entire program, which involves merging every global namespace's symbol tables. Not clear how long this takes in practice, but you wouldn't need to wait for that per-keystroke when editing checker.ts.

Slightly more tree-shakeable

If you're only using TypeScript's parser, you shouldn't have to pay for the checker too. Would be nice to enable API consumers like this.

---

Challenges

Not stopping the world

We cannot stop the team from authoring code - ideally, nobody is touching the tops of most files, so this shouldn't be a huge problem.

Have break off part of utilities.ts into privateUtilities.ts

Because stripInternal does't work on an entire file level. On the other hand, that's really handy because you don't need to use stripInternal, you just never export from that module

.d.ts bundling

We'd presumably leverage something like API Extractor. This would be new infrastructure work, and it would have to integrate into our test suite (since we already do .d.ts baselining for our API.

Death by 1000 imports

Not clear whether this is really a problem or just annoying, but people keep bringing it up.

[sandersn]

Reminder: 1000 is 1/36 of the total lines of checker.ts. That's fewer imports per source line than most Javascript I've written.

[fatcerberus]

I don’t care so much about the tree-shaking, I just want to be able to consume TS using import instead of require(). You can’t use require() at all in Node in ESM files to my knowledge; I allow this in miniSphere but you then have to use an absolute path:
https://github.com/fatcerberus/spectacles-i/blob/master/buildtime/ts-tool.js#L35

I want the above to be an relative path but I can’t because in my engine, using import would try to load it as ESM—and fail.

[kitsonk]

This would resolve #32949 as well so 👍 🎉

[kitsonk]

Also, .d.ts generation would be a good dog-fooding exercise of some of what we have to deal with in creating libraries based on TypeScript, so what you likely solve there would benefit all of us.

[fatcerberus]

Reminder: 1000 is 1/36 of the total lines of checker.ts.

To be fair, though, checker.ts is absolutely massive. It's the only time I've ever encountered a source file in a project so large that GitHub totally refuses to display it. 😆

(To be clear: I don't really buy the death-by-1000-imports argument. As long as a program is well-structured, this doesn't become an issue in my experience.)

[kitsonk]

(To be clear: I don't really buy the death-by-1000-imports argument. As long as a program is well-structured, this doesn't become an issue in my experience.)

And if you use a type self development language and environment, management of the imports is pretty easy... 😆

[NoelAbrahams]

@DanielRosenwasser You've missed out the biggest con: Moving from organising code in virtual hierarchies (ie namespaces) to a system based on the physical file system.

[kitsonk]

How is that a con?

[pauldraper]

How is that a con?

I assume because the organization of the API has to conform the organization of development (source files).

But this is the reason barrel modules exist.

[jakebailey]

For those following this issue, the module conversion PR has been posted: #51387