Publish @‍types Packages to the GitHub Registry

[DanielRosenwasser]

Today, @types packages are only published to the npm registry. This accounts for a large amount of traffic, in large part because of automatic type acquisition (ATA), a feature that leverages @types packages to provide code completion (a.k.a. IntelliSense) for JavaScript users. Because this functionality is crucial for editors, it's important to provide a source of redundancy to ensure reliability. Additionally, this is a good way to give users an option to select their registry.

[sandersn]

Here's the current plan:

Summary

Automatic Type Acquisition will slowly move over to the github package registry completely. All @types/* packages will remain on npm for human consumption, but will be mirrored on @definitelytyped/* for editor consumption.

3.7-beta (October)

• types-publisher starts publishing to @definitelytyped/types-publisher in addition to types-registry on npm (Publish types-registry to npm AND github package registry types-publisher#661)
• Prototype version of ATA reads from @definitelytyped/types-publisher, but requires you to define an environment variable and have a global .npmrc with //npm.pkg.github.com with package reading rights (Use @definitelytyped/types-registry for ATA #33791)

3.8 (November with release in January)

• Types-publisher test-publishes DT packages to @testtypepublishing/* on Github.
• Backfill script mirrors existing @types/* packages from NPM to Github.
• ATA adds telemetry for success rate, install times and which registry a package came from.
• Automated test checks that DT tests pass for each @types/* package after installation.
• Types-registry remains on npm, is updated and is treated as the main copy for the purposes of consistency checks when publishing.
• Types-publisher publishes DT packages to @types/* on both NPM and Github. (This doesn't need to happen until 3.9 but the sooner the better.)

3.9

• ATA reads DT packages from @types/* on Github, with a fallback to npm.
• ATA starts reading @definitelytyped/types-registry with fallback to npm.
• ATA adds a --registry flag to control which registry to use. If not provided, the default is NPM. NPM remains as a fallback. VS Code and VS need to start passing this flag.

4.0

• Types-registry is deprecated on npm. Publishing treats @definitelytyped/types-registry as the only package.
• ATA reads its registry only from @definitelytyped/types-registry

Addendum: Transitive skeletal dependents

If we publish to @types on github, skeletal dependents are not needed to resolve DT packages correctly. They would solely improve performance when ATA installs non-@types dependencies.

• Evaluate "transitive skeletal dependents" -- see below.
• "transitive skeletal dependents" -- possible enemy type in next Final Fantasy?

On @definitelytyped/*, publish versions of npm packages stripped of everything but d.ts files.
Publish packages that:

1. ship their own types.
2. @types/* packages depend on.
3. Are licence-compatible with the process?

Benefits

• Still less load on npm from ATA.
• Less disk space for ATA clients
• Less check time for ATA clients (maybe?)
• Less bandwidth for ATA clients

Notes

I believe github's proxy will already cache dependents of @definitelytyped/* packages, so skeletal depedents would only reduce the number of times that ATA installs an npm-based package from 1 (with caching) to 0 (with a skeletal dependent on GHPR), whereas today it's one for each time an @types/* package depends on it. So less load on npm is already part of @definitelytyped/* packages.

For the other 3 benefits, we need to know:

• Is the current resource consumption on the client a problem?
• Would skeletal dependents reduce the resource consumption enough?

Open design questions

• What triggers the publication of a skeletal package, and what version is published?
• What packages are candidates for skeletal packages?
• Will @types/* start depending on @DefinitelyTyped skeletal packages?
• What happens in duplicate typing scenarios?

E.g, a JS user installs pikaday and moment. @definitelytyped/pikaday depends on the skeletal dependency @definitelytyped/moment, but the user also has moment with its own shipped typings in their node_modules

Evaluation ideas

• Popular packages + simulation
• Stress testing + simulation
• Telemetry + simulation
• Surveys (new or existing)

[weswigham]

Prototype version of ATA reads from @definitelytyped/types-publisher, but requires you to define an environment variable and have a global .npmrc with //npm.pkg.github.com with package reading rights

We had a couple companies a long time ago ask for ATA to be able to be configured to point at an internal registry. This'll effectively enable that.

[sandersn]

@weswigham Only if the registry or the package name is configurable, right? I was literally thinking of

installTypesRegistry('github') || installTypesRegistry('npm')

for 3.8.

[DanielRosenwasser]

4.0

Very presumptuous ;)

[weswigham]

Only if the registry or the package name is configurable, right?

Well, I mean, I think enabling all custom registries follows the "zero, one, or infinity possibilities" rule.

[sandersn]

Update: As of this morning, types-publisher is publishing updates to packages at @testtypepublishing/*. I'll watch it for a while to shake the bugs out. When I have time I'll write a script to fill in old versions of packages as well.

[weswigham]

@sandersn you wanna update the summary here with the results of today's discussion?

[sandersn]

Yes, and the schedule needs to be updated as well.

[sandersn]

Moving to 3.9; the 3.8 parts of this task are done.

[sandersn]

This is actually more likely to be 4.1, but that's not a milestone yet.

[sandersn]

After discussing this, we decided that this feature doesn't make sense with Microsoft owning both github and npm. There's no gain in redundancy anymore.