Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce vm-config package to fix local-npm test #8141

Merged
merged 7 commits into from
Aug 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ jobs:
with:
node-version: '18.x'

- name: yarn bench (vats)
- name: yarn bench (boot)
shell: bash
run: |
cd packages/vats && yarn bench
cd packages/boot && yarn bench
echo "METRIC_VALUE=`cat benchmark-stress-vaults.json | jq .avgPerVaultMs`" >> $GITHUB_ENV

- uses: ./.github/actions/publish-metric
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:
strategy:
matrix:
cli: [link-cli, local-npm]
timeout-minutes: 40
steps:
- uses: actions/checkout@v3
with:
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/test-all-packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,9 @@ jobs:
- name: yarn test (vats)
if: (success() || failure())
run: cd packages/vats && yarn ${{ steps.vars.outputs.test }} | $TEST_COLLECT
- name: yarn test (vm-config)
if: (success() || failure())
run: cd packages/vm-config && yarn ${{ steps.vars.outputs.test }} | $TEST_COLLECT
- name: yarn test (zone)
if: (success() || failure())
run: cd packages/zone && yarn ${{ steps.vars.outputs.test }} | $TEST_COLLECT
Expand Down
2 changes: 1 addition & 1 deletion docs/env.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Affects: `ag-chain-cosmos`, `ag-solo`

Purpose: to set the specifier for the chain/sim-chain's `vatconfig.json`

Description: defaults to `@agoric/boot/decentral-core-config.json`
Description: defaults to `@agoric/vm-config/decentral-core-config.json`

Lifetime: until we don't want to allow user control of the chain's vat config

Expand Down
2 changes: 1 addition & 1 deletion golang/cosmos/x/swingset/types/default-params.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ var (
DefaultBeansPerMinFeeDebit = DefaultBeansPerFeeUnit.Quo(sdk.NewUint(5)) // $0.2
DefaultBeansPerStorageByte = DefaultBeansPerFeeUnit.Quo(sdk.NewUint(500)) // $0.002

DefaultBootstrapVatConfig = "@agoric/boot/decentral-core-config.json"
DefaultBootstrapVatConfig = "@agoric/vm-config/decentral-core-config.json"

DefaultPowerFlagFees = []PowerFlagFee{
NewPowerFlagFee("SMART_WALLET", sdk.NewCoins(sdk.NewInt64Coin("ubld", 10_000_000))),
Expand Down
1 change: 1 addition & 0 deletions packages/agoric-cli/src/sdk-package-names.js
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ export default [
"@agoric/time",
"@agoric/vat-data",
"@agoric/vats",
"@agoric/vm-config",
"@agoric/wallet",
"@agoric/wallet-backend",
"@agoric/xsnap",
Expand Down
2 changes: 1 addition & 1 deletion packages/agoric-cli/src/start.js
Original file line number Diff line number Diff line change
Expand Up @@ -528,7 +528,7 @@ export default async function startMain(progname, rawArgs, powers, opts) {
const agServerResolve = spec =>
require.resolve(spec, { paths: [serverDir] });
const coreConfigPath = agServerResolve(
'@agoric/boot/decentral-core-config.json',
'@agoric/vm-config/decentral-core-config.json',
);
const economyTemplPath = agServerResolve(
'@agoric/cosmic-swingset/economy-template.json',
Expand Down
File renamed without changes.
3 changes: 0 additions & 3 deletions packages/boot/index.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
// @ts-check
// Ambient types
import '@agoric/vats/src/core/types.js';
import '@agoric/zoe/src/contractFacet/types.js';
import '@agoric/zoe/src/types.js';
import '@agoric/zoe/tools/types-ambient.js';
6 changes: 3 additions & 3 deletions packages/boot/package.json
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
{
"name": "@agoric/boot",
"version": "0.1.0",
"private": true,
"description": "Config and utilities to bootstrap an Agoric chain",
"type": "module",
"main": "./index.js",
"repository": "https://github.com/Agoric/agoric-sdk",
"scripts": {
"build": "exit 0",
"prepack": "tsc --build jsconfig.build.json",
"postpack": "git clean -f '*.d.ts*'",
"test": "ava",
"test:xs": "SWINGSET_WORKER_TYPE=xs-worker ava 'test/bootstrapTests/**/test-*.js' 'test/upgrading/**/test-*.js'",
"bench": "ava --config ./ava.bench.config.js",
"lint-fix": "yarn lint:eslint --fix",
"lint": "run-s --continue-on-error lint:*",
"lint:types": "tsc -p jsconfig.json",
Expand All @@ -25,6 +24,7 @@
"@agoric/swingset-vat": "^0.32.2",
"@agoric/vat-data": "^0.5.2",
"@agoric/vats": "^0.15.1",
"@agoric/vm-config": "^0.1.0",
"@agoric/zoe": "^0.26.2",
"@endo/marshal": "^0.8.6",
"@endo/bundle-source": "^2.5.2",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -106,15 +106,15 @@ test.after.always(t => t.context.shutdown());

const rows = [];
const perfObserver = new PerformanceObserver(items => {
items.getEntries().forEach(entry => {
for (const entry of items.getEntries()) {
// @ts-expect-error cast
const { vaultsOpened, round } = entry.detail;
rows.push({
name: `${round}:${vaultsOpened}`,
durationMs: entry.duration,
avgPerVaultMs: entry.duration / vaultsOpened,
});
});
}
});
perfObserver.observe({ entryTypes: ['measure'] });

Expand Down
2 changes: 1 addition & 1 deletion packages/boot/test/bootstrapTests/liquidation.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ export const likePayouts = ({ Bid, Collateral }) => ({
export const makeLiquidationTestContext = async t => {
console.time('DefaultTestContext');
const swingsetTestKit = await makeSwingsetTestKit(t, 'bundles/vaults', {
configSpecifier: '@agoric/boot/decentral-main-vaults-config.json',
configSpecifier: '@agoric/vm-config/decentral-main-vaults-config.json',
});

const { runUtils, storage } = swingsetTestKit;
Expand Down
2 changes: 1 addition & 1 deletion packages/boot/test/bootstrapTests/supports.js
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ export const makeRunUtils = (controller, log = (..._) => {}) => {

export const getNodeTestVaultsConfig = async (
bundleDir = 'bundles',
specifier = '@agoric/boot/decentral-itest-vaults-config.json',
specifier = '@agoric/vm-config/decentral-itest-vaults-config.json',
) => {
const fullPath = await importMetaResolve(specifier, import.meta.url).then(
u => new URL(u).pathname,
Expand Down
2 changes: 1 addition & 1 deletion packages/boot/test/bootstrapTests/test-demo-config.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ const test = anyTest;

const makeDefaultTestContext = async t => {
const swingsetTestKit = await makeSwingsetTestKit(t, 'bundles/demo-config', {
configSpecifier: '@agoric/boot/decentral-demo-config.json',
configSpecifier: '@agoric/vm-config/decentral-demo-config.json',
});
return swingsetTestKit;
};
Expand Down
2 changes: 1 addition & 1 deletion packages/boot/test/bootstrapTests/test-vats-restart.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ const { Fail } = assert;
/** @file Bootstrap test of restarting (almost) all vats */

// main/production config doesn't have initialPrice, upon which 'open vaults' depends
const PLATFORM_CONFIG = '@agoric/boot/decentral-itest-vaults-config.json';
const PLATFORM_CONFIG = '@agoric/vm-config/decentral-itest-vaults-config.json';
/** @typedef {Awaited<ReturnType<typeof makeSwingsetTestKit>>} SwingsetTestKit */

export const makeTestContext = async t => {
Expand Down
8 changes: 4 additions & 4 deletions packages/boot/test/bootstrapTests/test-vaults-upgrade.js
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ test.serial('audit bootstrap exports', async t => {
return anObj;
};
const m = makeMarshal(undefined, saveBootstrapIface);
oids.forEach(oid => {
for (const oid of oids) {
for (const [_ix, ev] of transcript) {
for (const sc of ev.sc) {
if (sc.s[0] === 'send') {
Expand All @@ -240,7 +240,7 @@ test.serial('audit bootstrap exports', async t => {
}
}
}
});
}

const exportedInterfaces = Object.fromEntries(
[...toIface.values()].map(iface => [iface.replace(/^Alleged: /, ''), true]),
Expand Down Expand Up @@ -485,9 +485,9 @@ test.serial(
'vattp',
'timer',
].map(vatNamed);
swingsetVats.forEach(vatID => {
for (const vatID of swingsetVats) {
delete todo[vatID];
});
}

// TODO? test that powerStore is in baggage?
// by inspection, we see that it is:
Expand Down
2 changes: 1 addition & 1 deletion packages/boot/test/bootstrapTests/test-zcf-upgrade.js
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ export const makeZoeTestContext = async t => {
console.time('ZoeTestContext');
/** @type {SwingsetTestKit} */
const swingsetTestKit = await makeSwingsetTestKit(t, 'bundles/zoe', {
configSpecifier: '@agoric/boot/decentral-demo-config.json',
configSpecifier: '@agoric/vm-config/decentral-demo-config.json',
});

const { controller, runUtils } = swingsetTestKit;
Expand Down
36 changes: 17 additions & 19 deletions packages/boot/test/test-boot-config.js
Original file line number Diff line number Diff line change
@@ -1,21 +1,26 @@
// @ts-check
import { test as anyTest } from '@agoric/swingset-vat/tools/prepare-test-env-ava.js';
import '@agoric/swingset-liveslots/tools/prepare-test-env.js';

import anyTest from 'ava';
import { spawn as ambientSpawn } from 'child_process';
import { promises as fsPromises } from 'fs';
import { resolve as importMetaResolve } from 'import-meta-resolve';
import path from 'path';

import { extractCoreProposalBundles } from '@agoric/deploy-script-support/src/extract-proposal.js';
import { mustMatch } from '@agoric/store';
import { loadSwingsetConfigFile, shape as ssShape } from '@agoric/swingset-vat';
import { provideBundleCache } from '@agoric/swingset-vat/tools/bundleTool.js';
import { extractCoreProposalBundles } from '@agoric/deploy-script-support/src/extract-proposal.js';

/**
* @type {import('ava').TestFn<
* Awaited<ReturnType<typeof makeTestContext>>
* >}
*/
const test = anyTest;
const importConfig = configName =>
importMetaResolve(`@agoric/vm-config/${configName}`, import.meta.url).then(
u => new URL(u).pathname,
);

const test =
/** @type {import('ava').TestFn<Awaited<ReturnType<typeof makeTestContext>>>}} */ (
anyTest
);

const PROD_CONFIG_FILES = [
'decentral-main-vaults-config.json',
Expand Down Expand Up @@ -59,7 +64,6 @@ const makeTestContext = async () => {
const pathname = new URL(import.meta.url).pathname;
const dirname = path.dirname(pathname);
const pathResolve = (...ps) => path.join(dirname, ...ps);
const asset = (...ps) => fsPromises.readFile(pathResolve(...ps), 'utf-8');

const cacheDir = pathResolve('..', 'bundles');
const bundleCache = await provideBundleCache(cacheDir, {}, s => import(s));
Expand All @@ -68,7 +72,6 @@ const makeTestContext = async () => {
const runViz = pspawn(vizTool, { spawn: ambientSpawn });

return {
asset,
bundleCache,
cacheDir,
pathResolve,
Expand All @@ -83,11 +86,9 @@ test.before(async t => {
//#endregion

test('Bootstrap SwingSet config file syntax', async t => {
const { asset } = t.context;

await Promise.all(
CONFIG_FILES.map(async f => {
const txt = await asset('..', f);
const txt = await fsPromises.readFile(await importConfig(f), 'utf-8');
const config = harden(JSON.parse(txt));
t.notThrows(() => mustMatch(config, ssShape.SwingSetConfig), f);
}),
Expand Down Expand Up @@ -140,14 +141,13 @@ const checkBundle = async (t, sourceSpec, seen, name, configSpec) => {
};

test('no test-only code is in production configs', async t => {
const { pathResolve } = t.context;
const { entries } = Object;

const seen = new Set();

for await (const configSpec of PROD_CONFIG_FILES) {
t.log('checking config', configSpec);
const fullPath = pathResolve('..', configSpec);
const fullPath = await importConfig(configSpec);
const config = await loadSwingsetConfigFile(fullPath);
if (!config) throw t.truthy(config, configSpec); // if/throw refines type
const { bundles } = config;
Expand All @@ -162,14 +162,12 @@ test('no test-only code is in production configs', async t => {
});

test('no test-only code is in production proposals', async t => {
const { pathResolve } = t.context;

const seen = new Set();

for await (const configSpec of PROD_CONFIG_FILES) {
t.log('checking config', configSpec);
const getProposals = async () => {
const fullPath = pathResolve('..', configSpec);
const fullPath = await importConfig(configSpec);
const config = await loadSwingsetConfigFile(fullPath);
if (!config) throw t.truthy(config, configSpec); // if/throw refines type
const { coreProposals } = /** @type {any} */ (config);
Expand All @@ -189,7 +187,7 @@ test('no test-only code is in production proposals', async t => {
test('bootstrap permit visualization snapshot', async t => {
const { runViz } = t.context;

const cmd = runViz(['@agoric/boot/decentral-itest-vaults-config.json']);
const cmd = runViz(['@agoric/vm-config/decentral-itest-vaults-config.json']);
const output = async () => {
const parts = [];
cmd.child.stdout?.on('data', chunk => parts.push(chunk));
Expand Down
2 changes: 1 addition & 1 deletion packages/boot/tools/authorityViz.js
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ const loadConfig = async (specifier, { resolve, readFile }) => {
*/
const main = async (args, { stdout, fsp, meta }) => {
const [specifier, ...opts] = args;
specifier || Fail`Usage: $0 @agoric/boot/decentral-...json`;
specifier || Fail`Usage: $0 @agoric/vm-config/decentral-...json`;

const config = await loadConfig(specifier, {
resolve: meta.resolve,
Expand Down
8 changes: 4 additions & 4 deletions packages/cosmic-swingset/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -141,18 +141,18 @@ scenario2-setup:

t1/decentral-economy-config.json: $(ECONOMY_PROPOSALS)
jq -s '.[0] * { coreProposals: .[1] }' \
../boot/decentral-core-config.json \
../vm-config/decentral-core-config.json \
'$(ECONOMY_PROPOSALS)' > t1/decentral-economy-config.json

scenario2-run-chain-economy: t1/decentral-economy-config.json
echo "DEPRECATED use: CHAIN_BOOTSTRAP_VAT_CONFIG=@agoric/boot/decentral-itest-vaults-config.json scenario2-run-chain"
echo "DEPRECATED use: CHAIN_BOOTSTRAP_VAT_CONFIG=@agoric/vm-config/decentral-itest-vaults-config.json scenario2-run-chain"
CHAIN_BOOTSTRAP_VAT_CONFIG="$$PWD/t1/decentral-economy-config.json" \
$(MAKE) scenario2-run-chain

# We want to use the same configuration that will be deployed to the next
# devnet.agoric.net so that developers can test their code against it locally.
scenario2-run-chain: ../boot/decentral-devnet-config.json
CHAIN_BOOTSTRAP_VAT_CONFIG="$${CHAIN_BOOTSTRAP_VAT_CONFIG-$$PWD/../boot/decentral-devnet-config.json}" \
scenario2-run-chain: ../vm-config/decentral-devnet-config.json
CHAIN_BOOTSTRAP_VAT_CONFIG="$${CHAIN_BOOTSTRAP_VAT_CONFIG-$$PWD/../vm-config/decentral-devnet-config.json}" \
OTEL_EXPORTER_PROMETHEUS_PORT=$(OTEL_EXPORTER_PROMETHEUS_PORT) \
$(AGC) --home=t1/n0 start --log_level=warn $(AGC_START_ARGS)

Expand Down
1 change: 1 addition & 0 deletions packages/cosmic-swingset/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
"@agoric/swing-store": "^0.9.1",
"@agoric/swingset-vat": "^0.32.2",
"@agoric/telemetry": "^0.6.2",
"@agoric/vm-config": "^0.1.0",
"@endo/far": "^0.2.19",
"@endo/import-bundle": "^0.3.5",
"@endo/init": "^0.5.57",
Expand Down
3 changes: 2 additions & 1 deletion packages/cosmic-swingset/src/sim-params.js
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,8 @@ export const defaultBeansPerUnit = [
makeStringBeans(BeansPerXsnapComputron, defaultBeansPerXsnapComputron),
];

const defaultBootstrapVatConfig = '@agoric/boot/decentral-demo-config.json';
const defaultBootstrapVatConfig =
'@agoric/vm-config/decentral-demo-config.json';

export const defaultPowerFlagFees = [
makePowerFlagFee('SMART_WALLET', [makeCoin('ubld', 10_000_000n)]),
Expand Down
2 changes: 1 addition & 1 deletion packages/deployment/scripts/integration-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ then
solodir="$LOADGEN"/_agstate/agoric-servers/testnet-8000
"$thisdir/../../solo/bin/ag-solo" init "$solodir" --webport=8000
SOLO_ADDR=$(cat "$solodir/ag-cosmos-helper-address")
VAT_CONFIG="@agoric/boot/decentral-demo-config.json"
VAT_CONFIG="@agoric/vm-config/decentral-demo-config.json"
fi

"$thisdir/docker-deployment.cjs" > deployment.json
Expand Down
2 changes: 1 addition & 1 deletion packages/inter-protocol/scripts/start-local-chain.sh
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ make scenario2-setup-nobuild >>"$CHAIN_LOG" 2>&1
echo "Starting the chain..."
# use -economy target to get the kitchen sink
# disable pruning to keep all history https://docs.desmos.network/fullnode/overview/
make AGC_START_ARGS="--pruning=nothing" CHAIN_BOOTSTRAP_VAT_CONFIG=@agoric/boot/decentral-itest-vaults-config.json scenario2-run-chain >>"$CHAIN_LOG" 2>&1 &
make AGC_START_ARGS="--pruning=nothing" CHAIN_BOOTSTRAP_VAT_CONFIG=@agoric/vm-config/decentral-itest-vaults-config.json scenario2-run-chain >>"$CHAIN_LOG" 2>&1 &
Copy link
Member

@michaelfig michaelfig Aug 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this (and the reference in test-proposal-stuff.js) mean that inter-protocol/package.json should take a dependency on @agoric/vm-config?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, because this is instructing cosmic-swingset what to do. That's the package that really has the dependency. Same as how the JSONs in vm-config that specify a slate of packages doesn't itself depend on them (the thing that reads their source does).

Also this script is not long for this world #7914

make wait-for-cosmos

# xxx sleep to let it settle
Expand Down
2 changes: 1 addition & 1 deletion packages/inter-protocol/test/test-proposal-stuff.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { test as anyTest } from '@agoric/zoe/tools/prepare-test-env-ava.js';
import * as ambientFs from 'fs';
import { resolve as importMetaResolve } from 'import-meta-resolve';

const configSpecifier = '@agoric/boot/decentral-itest-vaults-config.json';
const configSpecifier = '@agoric/vm-config/decentral-itest-vaults-config.json';
const noop = harden(() => {});

/** @type {import('ava').TestFn<ReturnType<typeof makeTestContext>>} */
Expand Down
1 change: 0 additions & 1 deletion packages/vats/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
"test": "ava",
"test:c8": "c8 $C8_OPTIONS ava",
"test:xs": "exit 0",
"bench": "ava --config ./ava.bench.config.js",
"lint-fix": "yarn lint:eslint --fix",
"lint": "run-s --continue-on-error lint:*",
"lint:types": "tsc -p jsconfig.json",
Expand Down
13 changes: 13 additions & 0 deletions packages/vm-config/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# VM Config

Factored out of @agoric/vats, by way of @agoric/boot.

This is similar to `@agoric/boot` but because that has the integration testing of bootstrap, it depends on almost everything.

The configs themselves have no dependencies. Keeping them separate allows packages to depend on them without depending on the world.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like useful justification.

But I wonder to what extent it's true. The configs do reference other packages:

  "coreProposals": [
    "@agoric/vats/scripts/init-core.js",
    "@agoric/vats/scripts/init-network.js",
    {
      "module": "@agoric/inter-protocol/scripts/init-core.js",

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah… in a way it's not true. It's only true in the way that you can get the package without any other dependencies.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thinking more, this is fully true. the configs have no dependencies. the runtime that wants to build or execute those configs does.


# Future ideas

- [ ] move some things from agoric/vats to agoric/boot (needed only to bootstrap)
- [ ] move authorityViz to vm-config
- [ ] consider moving defaultBootstrapVatConfig out of cosmic-swingset (by making sim-chain configurable)
Empty file added packages/vm-config/index.js
Empty file.
Loading