Publishable NPM packages for mainnet

[mhofman]

Fixes #8366

Description

The NPM packages generated by the release-mainnet1B branch, including the ones related to agoric-upgrade-11 are broken in various ways such that they are unusable to develop a dapp using agoric-sdk packages installed from NPM. Most of these issues have been fixed in trunk since then. This backports build changes to make published NPM packages work for the release-mainnet1B branch.

In particular:

• Picks ci(local-npm): resume integration testing of NPM canary #7883 for the xsnap build fix and integration test changes. Some commits in the PR were unrelated but benign enough to bring along
• Pick some commits from PRs with related integration test changes: introduce vm-config package to fix local-npm test #8141, ci: lerna publish sequentially #8092, ci(getting-started): run (but ignore) checks with npm and npx #8250 and Sync Endo: 2023-08-08 #8314
• Pins the @endo/* package versions to avoid picking up newer published endo versions when installing from NPM (the agoric-sdk yarn.lock does not apply). This is needed because some endo changes are more breaking than advertised.
• Tweaks some CI jobs for compatibility with the release branch

This branch was constructed as follow:

# Branch Agoric-mfig-local-npm
label base-Agoric-mfig-local-npm
pick ca530f884 ci(local-npm): resume integration testing of NPM canary
pick b4a598469 build(deps): tweak `package.json`s
pick f33f4e20e chore(telemetry): use newer `@opentelemetry` with matching versions
pick 9364d7d8e build(deps): refresh `yarn.lock`
pick ff85a7af2 fix(deploy-scripts-support): correct bare module resolution
label Agoric-mfig-local-npm
reset base-Agoric-mfig-local-npm
merge -C 33535095a Agoric-mfig-local-npm # Merge pull request #7883 from Agoric/mfig-local-npm

# Pull request #8141
pick c3b3b5fbc ci: timeout agoric-cli integration-test

# Pull request #8092
pick 014a2ed8f ci: lerna publish sequentially

# Pull request #8250
pick 82ca74656 ci(getting-started): run (but ignore) checks with `npm` and `npx`
pick a59b9355e chore(deployment): properly set `packageJson.name`
pick 793426002 ci(getting-started): break out `registry.sh`
pick dd2873ddc chore(inter-protocol): do not publish bundles to NPM

# Pull request #8314
pick eb6c9e494 ci(integration): update for `restore-node`
pick 7b684a626 fix(agoric): relax override mistake using `@endo/init/legacy.js`
pick 06aa08a10 ci(registry): publish Endo in CI registry, update versions

# Pull request #8368
pick 098bb791c build(cosmos): prevent macOS `filesize exceeds vmsize`

# Pull request #8369
pick 57802f970 ci: jobs compatibility with release branch

# Pull request #8372
pick 2b7c650ea deps(agoric): cleanup deps (#8372)

# s/("@endo/([^"]+)": ")\^/$1"/
pick 875341d0b build(deps): pin endo package versions

# s/"@endo/bundle-source": "2.5.1"/"@endo/bundle-source": "2.5.2-upstream-rollup"/
pick f01767409 build(deps): bump @endo/bundle-source

# Some manual changes
pick 8f8782bc5 ci: tweak integration checks

Security Considerations

None

Scaling Considerations

None

Documentation Considerations

None

Testing Considerations

Locally tested using scripts/registry.sh ci. Will check CI integration tests as well

Upgrade Considerations

This should have no impact on a future chain software upgrade that includes these changes.

We should follow up this PR by going through the release process and publish new NPM packages with the appropriate dist tag. agoric-upgrade-11 may be appropriate.

[mhofman]

@michaelfig I'm not sure what is failing in the getting-started tests.

[michaelfig]

@michaelfig I'm not sure what is failing in the getting-started tests.

It's an old @endo/bundle-source which is incompatible with NPM installs, since it is dragging in a rollup from Github (automatically running the prepare script which invokes husky, which fails):

$ npm info @endo/bundle-source@2.5.2
[...]
dependencies:
[...]
rollup: github:endojs/endo#rollup-2.7.1-patch-1

Fixed by @turadg in https://github.com/endojs/endo/blob/5b8e97b445e0da0c92b2496cda574321f6758c34/packages/bundle-source/package.json#L42

[mhofman]

It's an old @endo/bundle-source which is incompatible with NPM installs

That's what I just figured out. Are we ok with just yarn support then ? Still weird that yarn doesn't puke, I guess because of a different node_modules layout.

[michaelfig]

@michaelfig I'm not sure what is failing in the getting-started tests.

Now I see the getting-started test is timing out. I'm not sure why.

[mhofman]

@warner @michaelfig I have confirmed in local tests, CI and manual npx agoric@other-dev install other-dev in the KREAd repo that this seems to be working as expected. I'd like @turadg to do a little more manual testing in a dapp that using the other-dev tag (published from this PR), that no other untested issues remain.

[warner]

@mhofman walked me through the need for this, and the solution they picked, and it sounds good to me