Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(synthetic-chain)!: bundle info with dir #104

Merged
merged 2 commits into from
Feb 16, 2024
Merged

fix(synthetic-chain)!: bundle info with dir #104

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cdc4eff
fix(synthetic-chain)!: bundle info with dir
mhofman Feb 14, 2024
4201216
chore: release 0.0.7 alpha 1
turadg Feb 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/synthetic-chain/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@agoric/synthetic-chain",
"version": "0.0.6",
"version": "0.0.7-1",
"description": "Utilities to build a chain and test proposals atop it",
"bin": "dist/cli/cli.js",
"main": "./dist/lib/index.js",
Expand Down
11 changes: 7 additions & 4 deletions packages/synthetic-chain/src/lib/core-eval-support.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { agoric } from './cliHelper.js';
import { getISTBalance, mintIST } from './econHelpers.js';
import { ExecutionContext } from 'ava';
import { StaticConfig } from './core-eval.js';
import path from 'node:path';

// move to unmarshal.js?
const makeBoardUnmarshal = () => {
Expand Down Expand Up @@ -95,6 +96,8 @@ const importBundleCost = (bytes: number, price = 0.002) => {
};

export type BundleInfo = {
name: string;
dir: string;
bundles: string[];
evals: { permit: string; script: string }[];
};
Expand Down Expand Up @@ -150,13 +153,13 @@ export const ensureISTForInstall = async (
export const readBundles = async (dir: string) => {
const files = await fsp.readdir(dir);
const names = files.filter(f => f.endsWith('.js')).map(f => f.slice(0, -3));
const buildAssets = {} as Record<string, BundleInfo>;
const bundleInfos: BundleInfo[] = [];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change this type? This is the only breaking change I see. Is it necessary?

I factored it that way to enforce that there's on bundle per name, an assumption I think the user will have. Admittedly, given that goal the loop should have enforced it.

Copy link
Member Author

@mhofman mhofman Feb 14, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change this type? This is the only breaking change I see. Is it necessary?

Correct, the return value of readBundles and the param of passCoreEvalProposal are the only breaking changes. I didn't see them used anywhere so figured it was safe to break. It's necessary to break passCoreEvalProposal because it needs the directory information to come from somewhere. Before it was assuming a submission subdir

I factored it that way to enforce that there's on bundle per name, an assumption I think the user will have. Admittedly, given that goal the loop should have enforced it.

As you noticed, the change from bundleAssets -> bundleInfos doesn't change the logic at all. While it does technically allow multiple bundle infos to share the same name, I don't think it's a huge problem if that's the case?

Edit: Happy to go back to a map if preferred.

Copy link
Member

@turadg turadg Feb 14, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it does technically allow multiple bundle infos to share the same name, I don't think it's a huge problem if that's the case?

Not huge, but worse. As I think about this more, there's no good reason to export this functionality from the package so I'm not concerned with the breaking change. I'll probably follow up with another breaking change (affecting nobody) to drop these exports. So refactor at will.

I might also add then a validation that names aren't reused, unless you want to add that now. It's probably easiest to do with a map so I might end up changing this back at that point.

for (const name of names) {
const evals = [{ permit: `${name}-permit.json`, script: `${name}.js` }];
const content = await fsp.readFile(`${dir}/${name}.js`, 'utf8');
const content = await fsp.readFile(path.join(dir, `${name}.js`), 'utf8');
const bundleIds = content.matchAll(/b1-[a-z0-9]+/g);
const bundles = Array.from(bundleIds).map(id => `${id}.json`);
buildAssets[name] = { evals, bundles };
bundleInfos.push({ evals, bundles, name, dir });
}
return buildAssets;
return bundleInfos;
};
87 changes: 28 additions & 59 deletions packages/synthetic-chain/src/lib/core-eval.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,43 +32,15 @@ export const staticConfig = {
swingstorePath: '~/.agoric/data/agoric/swingstore.sqlite',
};

export type StaticConfig = typeof staticConfig & {
bundleInfos?: Record<string, BundleInfo>;
};

const makeFakeWebCache = (base: string): WebCache => {
return {
getText(segment: string) {
return fsp.readFile(path.join(base, segment), 'utf8');
},
async storedPath(segment: string) {
return path.join(base, segment);
},
async size(segment: string) {
const info = await fsp.stat(path.join(base, segment));
return info.size;
},
toString() {
return 'fake web cache';
},
async remove() {
console.warn('noop remove');
},
};
};
export type StaticConfig = typeof staticConfig;

/**
* Provide access to the outside world via context.
*
* TODO: refactor overlap with mn2-start.test.js
*/
const makeTestContext = async (staticConfig: StaticConfig) => {
// assume filenames don't overlap
const bundleAssets = makeFakeWebCache('submission');
console.log(`bundleAssets: ${bundleAssets}`);

const config = {
bundleAssets,
chainId: 'agoriclocal',
...staticConfig,
};
Expand All @@ -84,20 +56,16 @@ const makeTestContext = async (staticConfig: StaticConfig) => {
return { agd, agoric, swingstore, config, before, fetch };
};

export const passCoreEvalProposal = async (
bundleMap: Record<string, BundleInfo>,
) => {
export const passCoreEvalProposal = async (bundleInfos: BundleInfo[]) => {
// XXX vestige of Ava
const config = {
...staticConfig,
bundleInfos: bundleMap,
};
const context = await makeTestContext(config);
const bundleInfos = Object.values(bundleMap);

await step('bundles not yet installed', async () => {
const loaded = loadedBundleIds(context.swingstore);
for (const [name, { bundles, evals }] of Object.entries(bundleMap)) {
for (const { name, bundles, evals } of bundleInfos) {
console.log(
name,
evals[0].script,
Expand Down Expand Up @@ -138,11 +106,12 @@ export const passCoreEvalProposal = async (
};

await step('bundle names: compartmentMap.entry', async () => {
const { bundleAssets } = context.config;
for (const { bundles, evals } of bundleInfos) {
for (const { bundles, dir } of bundleInfos) {
for (const bundleRef of bundles) {
const { fileName } = bundleDetail(bundleRef);
const bundle = JSON.parse(await bundleAssets.getText(fileName));
const bundle = JSON.parse(
await fsp.readFile(path.join(dir, fileName), 'utf8'),
);
const entry = await bundleEntry(bundle);
console.log(entry, fileName.slice(0, 'b1-12345'.length));
assert(entry.compartment);
Expand All @@ -153,21 +122,23 @@ export const passCoreEvalProposal = async (

const sum = (xs: number[]) => xs.reduce((a, b) => a + b, 0);

const readBundleSizes = async (assets: WebCache) => {
const readBundleSizes = async () => {
const bundleSizes = await Promise.all(
bundleInfos
.map(({ bundles }) =>
bundles.map(b => assets.size(bundleDetail(b).fileName)),
)
.flat(),
bundleInfos.flatMap(({ bundles, dir }) =>
bundles.map(async b => {
const { fileName } = bundleDetail(b);
const stat = await fsp.stat(path.join(dir, fileName));
return stat.size;
}),
),
);
const totalSize = sum(bundleSizes);
return { bundleSizes, totalSize };
};

await step('ensure enough IST to install bundles', async () => {
const { agd, config } = context;
const { totalSize } = await readBundleSizes(config.bundleAssets);
const { totalSize } = await readBundleSizes();

await ensureISTForInstall(agd, config, totalSize, {
log: console.log,
Expand All @@ -176,13 +147,13 @@ export const passCoreEvalProposal = async (

await step('ensure bundles installed', async () => {
const { agd, swingstore, agoric, config } = context;
const { chainId, bundleAssets } = config;
const { chainId } = config;
const loaded = loadedBundleIds(swingstore);
const from = agd.lookup(config.installer);

let todo = 0;
let done = 0;
for (const { bundles } of bundleInfos) {
for (const { bundles, dir } of bundleInfos) {
todo += bundles.length;
for (const bundle of bundles) {
const { id, fileName, endoZipBase64Sha512 } = bundleDetail(bundle);
Expand All @@ -192,7 +163,7 @@ export const passCoreEvalProposal = async (
continue;
}

const bundleRd = await bundleAssets.storedPath(fileName);
const bundleRd = path.join(dir, fileName);
const result = await agd.tx(
['swingset', 'install-bundle', `@${bundleRd}`, '--gas', 'auto'],
{ from, chainId, yes: true },
Expand All @@ -216,7 +187,7 @@ export const passCoreEvalProposal = async (
await step('core eval proposal passes', async () => {
const { agd, swingstore, config } = context;
const from = agd.lookup(config.proposer);
const { chainId, deposit, bundleAssets } = config;
const { chainId, deposit } = config;
// @ts-expect-error FIXME
const info = { title: config.title, description: config.description };
// @ts-expect-error FIXME
Expand All @@ -233,14 +204,12 @@ export const passCoreEvalProposal = async (
}
}

const evalNames = bundleInfos
.map(({ evals }) => evals)
.flat()
.map(e => [e.permit, e.script])
.flat();
const evalPaths = await Promise.all(
evalNames.map(e => bundleAssets.storedPath(e)),
);
const evalPaths = bundleInfos.flatMap(({ evals, dir }) => {
return evals
.flatMap(e => [e.permit, e.script])
.map(file => path.join(dir, file));
});

const result = await agd.tx(
[
'gov',
Expand All @@ -262,7 +231,7 @@ export const passCoreEvalProposal = async (
};

export const evalBundles = async (dir: string) => {
const bundleMap = await readBundles(dir);
const bundleInfos = await readBundles(dir);

await passCoreEvalProposal(bundleMap);
await passCoreEvalProposal(bundleInfos);
};