Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: create llm basic dispute bot #4630

Merged
merged 5 commits into from
Sep 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions packages/llm-bot/src/dispute-bot/DisputeDisputableRequests.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import {
DisputerStrategy,
OptimisticOracleClientV2,
OptimisticOracleClientV2FilterDisputeable,
} from "../core/OptimisticOracleV2";
import { Logger, BotParams } from "./common";

export async function disputeDisputableRequests(logger: typeof Logger, params: BotParams): Promise<void> {
const oov2 = new OptimisticOracleClientV2(params.provider);

// Update the client with the latest block range.
const oov2ClientUpdated = await oov2.updateWithBlockRange();

const requests = Array.from(oov2ClientUpdated.requests.values());
const oov2FilterDisputable = new OptimisticOracleClientV2FilterDisputeable();

const filteredRequests = await oov2FilterDisputable.filter(requests);

const disputable = await Promise.all(filteredRequests.map(DisputerStrategy.process));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is process() a static method on DisputerStrategy? Are all strategies intended to be built that way? Seems like stated unless might ultimately be necessary for future (non trivial) strategies. Either way, that's something we can worry about in a follow up PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can any of these calls error? i usually try to avoid processing things in a list like this since errors can prevent some from executing. if no errors are expected though this is fine


for (const request of disputable) {
logger.info({
at: "LLMDisputeBot",
message: "Disputing request",
request,
});
// TODO: Dispute the request.
}

console.log("Done speeding up prices.");
}
27 changes: 27 additions & 0 deletions packages/llm-bot/src/dispute-bot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# LLM Dispute Bot
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As usual, 👌 documentation!


The LLM Dispute Bot can check for disputable requests with LLM logic and dispute them.

The main entry point to LLM Dispute Bot is running:

```
node ./packages/llm-bot/dist/dispute-bot/index.js
```

All the configuration should be provided with following environment variables:

- `CHAIN_ID` is network number.
- `NODE_URLS_X` is an array of RPC node URLs replacing `X` in variable name with network number from `CHAIN_ID`.
- `NODE_URL_X` is a single RPC node URL replacing `X` in variable name with network number from `CHAIN_ID`. This is considered only if matching `NODE_URLS_X` is not provided.
- `MNEMONIC` is a mnemonic for a wallet that has enough funds to pay for transactions.
- `GCKMS_WALLET` is a GCKMS wallet that has enough funds to pay for transactions. If this is provided, `MNEMONIC` is ignored.
- `NODE_RETRIES` is the number of retries to make when a node request fails (defaults to `2`).
- `NODE_RETRY_DELAY` is the delay in seconds between retries (defaults to `1`).
- `NODE_TIMEOUT` is the timeout in seconds for node requests (defaults to `60`).
- `POLLING_DELAY` is value in seconds for delay between consecutive runs, defaults to 1 minute. If set to 0 then running in serverless mode will exit after the loop.
- `DISPUTE_DISPUTABLE_REQUESTS` is boolean enabling/disabling disputes with LLM bot (`false` by default).
- `SLACK_CONFIG` is a JSON object containing `defaultWebHookUrl` for the default Slack webhook URL.
- `BLOCK_LOOKBACK` is the number of blocks to look back from the current block to look for past resolution events.
See default values in blockDefaults in index.ts
- `MAX_BLOCK_LOOKBACK` is the maximum number of blocks to look back per query.
See default values in blockDefaults in index.ts
71 changes: 71 additions & 0 deletions packages/llm-bot/src/dispute-bot/common.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
import { getGckmsSigner, getMnemonicSigner, getRetryProvider } from "@uma/common";
import { Signer, Wallet } from "ethers";

import type { Provider } from "@ethersproject/abstract-provider";

export { OptimisticOracleV3Ethers } from "@uma/contracts-node";
export { Logger } from "@uma/financial-templates-lib";
export { getContractInstanceWithProvider } from "../utils/contracts";

export const ARBITRUM_CHAIN_ID = 42161;
export const OPTIMISM_CHAIN_ID = 10;
export const POLYGON_CHAIN_ID = 137;

export interface BotModes {
disputeDisputableRequests: boolean;
}

export interface BlockRange {
start: number;
end: number;
}

export interface BotParams {
chainId: number;
provider: Provider;
botModes: BotModes;
signer: Signer;
blockLookback: number;
maxBlockLookBack: number;
pollingDelay: number;
}

export const initBotParams = async (env: NodeJS.ProcessEnv): Promise<BotParams> => {
if (!env.CHAIN_ID) throw new Error("CHAIN_ID must be defined in env");
const chainId = Number(env.CHAIN_ID);

// Creating provider will check for other chainId specific env variables.
const provider = getRetryProvider(chainId) as Provider;

// Default to 1 minute polling delay.
const pollingDelay = env.POLLING_DELAY ? Number(env.POLLING_DELAY) : 60;

let signer;
if (process.env.GCKMS_WALLET) {
signer = ((await getGckmsSigner()) as Wallet).connect(provider);
} else {
// Throws if MNEMONIC env var is not defined.
signer = (getMnemonicSigner() as Signer).connect(provider);
}

const botModes = {
disputeDisputableRequests: env.DISPUTE_DISPUTABLE_REQUESTS === "true",
};

const blockLookback = Number(env.BLOCK_LOOKBACK);
const maxBlockLookBack = Number(env.MAX_BLOCK_LOOKBACK);

return {
chainId,
provider,
botModes,
signer,
blockLookback,
maxBlockLookBack,
pollingDelay,
};
};

export const startupLogLevel = (params: BotParams): "debug" | "info" => {
return params.pollingDelay === 0 ? "debug" : "info";
};
54 changes: 54 additions & 0 deletions packages/llm-bot/src/dispute-bot/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
import { delay, waitForLogger } from "@uma/financial-templates-lib";
import { BotModes, initBotParams, Logger, startupLogLevel } from "./common";
import { disputeDisputableRequests } from "./DisputeDisputableRequests";

const logger = Logger;

async function main() {
const params = await initBotParams(process.env);

logger[startupLogLevel(params)]({
at: "LLMDisputeBot",
message: "LLMDisputeBot started 🤖",
botModes: params.botModes,
});

const cmds = {
disputeRequestsEnabled: disputeDisputableRequests,
};

for (;;) {
const runCmds = Object.entries(cmds)
.filter(([mode]) => params.botModes[mode as keyof BotModes])
.map(([, cmd]) => cmd(logger, { ...params }));

for (const cmd of runCmds) {
await cmd;
}

if (params.pollingDelay !== 0) {
await delay(params.pollingDelay);
} else {
await delay(5); // Set a delay to let the transports flush fully.
await waitForLogger(logger);
break;
}
}
}

main().then(
() => {
process.exit(0);
},
async (error) => {
logger.error({
at: "LLMDisputeBot",
message: "LLMDisputeBot error🚨",
error,
});
// Wait 5 seconds to allow logger to flush.
await delay(5);
await waitForLogger(logger);
process.exit(1);
}
);
87 changes: 87 additions & 0 deletions packages/llm-bot/test/DisputeRequests.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
import { ExpandedERC20Ethers, OptimisticOracleV2Ethers } from "@uma/contracts-node";
import { SpyTransport, createNewLogger } from "@uma/financial-templates-lib";
import { assert } from "chai";
import sinon from "sinon";
import { disputeDisputableRequests } from "../src/dispute-bot/DisputeDisputableRequests";
import { BotModes, MonitoringParams } from "../src/dispute-bot/common";
import { defaultOptimisticOracleV2Identifier } from "./constants";
import { optimisticOracleV2Fixture } from "./fixtures/OptimisticOracleV2.Fixture";
import { Provider, Signer, hre, toUtf8Bytes } from "./utils";

const ethers = hre.ethers;

const createMonitoringParams = async (): Promise<MonitoringParams> => {
// get chain id
const chainId = await hre.ethers.provider.getNetwork().then((network) => network.chainId);
// get hardhat signer
const [signer] = await ethers.getSigners();
// Bot modes are not used as we are calling monitor modules directly.
const botModes: BotModes = {
disputeDisputableRequests: true,
};
return {
chainId: chainId,
provider: ethers.provider as Provider,
pollingDelay: 0,
botModes,
signer,
maxBlockLookBack: 1000,
blockLookback: 1000,
};
};

describe("LLMDisputeDisputableRequests", function () {
let bondToken: ExpandedERC20Ethers;
let optimisticOracleV2: OptimisticOracleV2Ethers;
let requester: Signer;
let proposer: Signer;
let disputer: Signer;

const bond = ethers.utils.parseEther("1000");

const question = "This is just a test question";
const ancillaryData = toUtf8Bytes(question);

beforeEach(async function () {
// Signer from ethers and hardhat-ethers are not version compatible, thus, we cannot use the SignerWithAddress.
[requester, proposer, disputer] = (await ethers.getSigners()) as Signer[];

// Get contract instances.
const optimisticOracleV2Contracts = await optimisticOracleV2Fixture();

bondToken = optimisticOracleV2Contracts.bondToken;
optimisticOracleV2 = optimisticOracleV2Contracts.optimisticOracleV2;

// Fund proposer and disputer with bond amount and approve Optimistic Oracle V2 to spend bond tokens.
await bondToken.addMinter(await requester.getAddress());
await bondToken.mint(await proposer.getAddress(), bond);
await bondToken.mint(await disputer.getAddress(), bond);
await bondToken.connect(proposer).approve(optimisticOracleV2.address, bond);
await bondToken.connect(disputer).approve(optimisticOracleV2.address, bond);
});

it("Disputes disputable requests", async function () {
await (
await optimisticOracleV2.requestPrice(defaultOptimisticOracleV2Identifier, 0, ancillaryData, bondToken.address, 0)
).wait();

await (await bondToken.connect(proposer).approve(optimisticOracleV2.address, bond)).wait();
await (
await optimisticOracleV2
.connect(proposer)
.proposePrice(
await requester.getAddress(),
defaultOptimisticOracleV2Identifier,
0,
ancillaryData,
ethers.utils.parseEther("1")
)
).wait();

const spy = sinon.spy();
const spyLogger = createNewLogger([new SpyTransport({}, { spy: spy })]);

await disputeDisputableRequests(spyLogger, await createMonitoringParams());
assert.equal(spy.getCall(0).lastArg.at, "LLMDisputeBot");
});
});