-
Notifications
You must be signed in to change notification settings - Fork 178
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: create llm basic dispute bot #4630
Merged
md0x
merged 5 commits into
master
from
pablo/uma-1760-create-llm-dispute-bot-wrapper-base
Sep 3, 2023
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
31 changes: 31 additions & 0 deletions
31
packages/llm-bot/src/dispute-bot/DisputeDisputableRequests.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
import { | ||
DisputerStrategy, | ||
OptimisticOracleClientV2, | ||
OptimisticOracleClientV2FilterDisputeable, | ||
} from "../core/OptimisticOracleV2"; | ||
import { Logger, BotParams } from "./common"; | ||
|
||
export async function disputeDisputableRequests(logger: typeof Logger, params: BotParams): Promise<void> { | ||
const oov2 = new OptimisticOracleClientV2(params.provider); | ||
|
||
// Update the client with the latest block range. | ||
const oov2ClientUpdated = await oov2.updateWithBlockRange(); | ||
|
||
const requests = Array.from(oov2ClientUpdated.requests.values()); | ||
const oov2FilterDisputable = new OptimisticOracleClientV2FilterDisputeable(); | ||
|
||
const filteredRequests = await oov2FilterDisputable.filter(requests); | ||
|
||
const disputable = await Promise.all(filteredRequests.map(DisputerStrategy.process)); | ||
|
||
for (const request of disputable) { | ||
logger.info({ | ||
at: "LLMDisputeBot", | ||
message: "Disputing request", | ||
request, | ||
}); | ||
// TODO: Dispute the request. | ||
} | ||
|
||
console.log("Done speeding up prices."); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# LLM Dispute Bot | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As usual, 👌 documentation! |
||
|
||
The LLM Dispute Bot can check for disputable requests with LLM logic and dispute them. | ||
|
||
The main entry point to LLM Dispute Bot is running: | ||
|
||
``` | ||
node ./packages/llm-bot/dist/dispute-bot/index.js | ||
``` | ||
|
||
All the configuration should be provided with following environment variables: | ||
|
||
- `CHAIN_ID` is network number. | ||
- `NODE_URLS_X` is an array of RPC node URLs replacing `X` in variable name with network number from `CHAIN_ID`. | ||
- `NODE_URL_X` is a single RPC node URL replacing `X` in variable name with network number from `CHAIN_ID`. This is considered only if matching `NODE_URLS_X` is not provided. | ||
- `MNEMONIC` is a mnemonic for a wallet that has enough funds to pay for transactions. | ||
- `GCKMS_WALLET` is a GCKMS wallet that has enough funds to pay for transactions. If this is provided, `MNEMONIC` is ignored. | ||
- `NODE_RETRIES` is the number of retries to make when a node request fails (defaults to `2`). | ||
- `NODE_RETRY_DELAY` is the delay in seconds between retries (defaults to `1`). | ||
- `NODE_TIMEOUT` is the timeout in seconds for node requests (defaults to `60`). | ||
- `POLLING_DELAY` is value in seconds for delay between consecutive runs, defaults to 1 minute. If set to 0 then running in serverless mode will exit after the loop. | ||
- `DISPUTE_DISPUTABLE_REQUESTS` is boolean enabling/disabling disputes with LLM bot (`false` by default). | ||
- `SLACK_CONFIG` is a JSON object containing `defaultWebHookUrl` for the default Slack webhook URL. | ||
- `BLOCK_LOOKBACK` is the number of blocks to look back from the current block to look for past resolution events. | ||
See default values in blockDefaults in index.ts | ||
- `MAX_BLOCK_LOOKBACK` is the maximum number of blocks to look back per query. | ||
See default values in blockDefaults in index.ts |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
import { getGckmsSigner, getMnemonicSigner, getRetryProvider } from "@uma/common"; | ||
import { Signer, Wallet } from "ethers"; | ||
|
||
import type { Provider } from "@ethersproject/abstract-provider"; | ||
|
||
export { OptimisticOracleV3Ethers } from "@uma/contracts-node"; | ||
export { Logger } from "@uma/financial-templates-lib"; | ||
export { getContractInstanceWithProvider } from "../utils/contracts"; | ||
|
||
export const ARBITRUM_CHAIN_ID = 42161; | ||
export const OPTIMISM_CHAIN_ID = 10; | ||
export const POLYGON_CHAIN_ID = 137; | ||
|
||
export interface BotModes { | ||
disputeDisputableRequests: boolean; | ||
} | ||
|
||
export interface BlockRange { | ||
start: number; | ||
end: number; | ||
} | ||
|
||
export interface BotParams { | ||
chainId: number; | ||
provider: Provider; | ||
botModes: BotModes; | ||
signer: Signer; | ||
blockLookback: number; | ||
maxBlockLookBack: number; | ||
pollingDelay: number; | ||
} | ||
|
||
export const initBotParams = async (env: NodeJS.ProcessEnv): Promise<BotParams> => { | ||
if (!env.CHAIN_ID) throw new Error("CHAIN_ID must be defined in env"); | ||
const chainId = Number(env.CHAIN_ID); | ||
|
||
// Creating provider will check for other chainId specific env variables. | ||
const provider = getRetryProvider(chainId) as Provider; | ||
|
||
// Default to 1 minute polling delay. | ||
const pollingDelay = env.POLLING_DELAY ? Number(env.POLLING_DELAY) : 60; | ||
|
||
let signer; | ||
if (process.env.GCKMS_WALLET) { | ||
signer = ((await getGckmsSigner()) as Wallet).connect(provider); | ||
} else { | ||
// Throws if MNEMONIC env var is not defined. | ||
signer = (getMnemonicSigner() as Signer).connect(provider); | ||
} | ||
|
||
const botModes = { | ||
disputeDisputableRequests: env.DISPUTE_DISPUTABLE_REQUESTS === "true", | ||
}; | ||
|
||
const blockLookback = Number(env.BLOCK_LOOKBACK); | ||
const maxBlockLookBack = Number(env.MAX_BLOCK_LOOKBACK); | ||
|
||
return { | ||
chainId, | ||
provider, | ||
botModes, | ||
signer, | ||
blockLookback, | ||
maxBlockLookBack, | ||
pollingDelay, | ||
}; | ||
}; | ||
|
||
export const startupLogLevel = (params: BotParams): "debug" | "info" => { | ||
return params.pollingDelay === 0 ? "debug" : "info"; | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
import { delay, waitForLogger } from "@uma/financial-templates-lib"; | ||
import { BotModes, initBotParams, Logger, startupLogLevel } from "./common"; | ||
import { disputeDisputableRequests } from "./DisputeDisputableRequests"; | ||
|
||
const logger = Logger; | ||
|
||
async function main() { | ||
const params = await initBotParams(process.env); | ||
|
||
logger[startupLogLevel(params)]({ | ||
at: "LLMDisputeBot", | ||
message: "LLMDisputeBot started 🤖", | ||
botModes: params.botModes, | ||
}); | ||
|
||
const cmds = { | ||
disputeRequestsEnabled: disputeDisputableRequests, | ||
}; | ||
|
||
for (;;) { | ||
const runCmds = Object.entries(cmds) | ||
.filter(([mode]) => params.botModes[mode as keyof BotModes]) | ||
.map(([, cmd]) => cmd(logger, { ...params })); | ||
|
||
for (const cmd of runCmds) { | ||
await cmd; | ||
} | ||
|
||
if (params.pollingDelay !== 0) { | ||
await delay(params.pollingDelay); | ||
} else { | ||
await delay(5); // Set a delay to let the transports flush fully. | ||
await waitForLogger(logger); | ||
break; | ||
} | ||
} | ||
} | ||
|
||
main().then( | ||
() => { | ||
process.exit(0); | ||
}, | ||
async (error) => { | ||
logger.error({ | ||
at: "LLMDisputeBot", | ||
message: "LLMDisputeBot error🚨", | ||
error, | ||
}); | ||
// Wait 5 seconds to allow logger to flush. | ||
await delay(5); | ||
await waitForLogger(logger); | ||
process.exit(1); | ||
} | ||
); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
import { ExpandedERC20Ethers, OptimisticOracleV2Ethers } from "@uma/contracts-node"; | ||
import { SpyTransport, createNewLogger } from "@uma/financial-templates-lib"; | ||
import { assert } from "chai"; | ||
import sinon from "sinon"; | ||
import { disputeDisputableRequests } from "../src/dispute-bot/DisputeDisputableRequests"; | ||
import { BotModes, MonitoringParams } from "../src/dispute-bot/common"; | ||
import { defaultOptimisticOracleV2Identifier } from "./constants"; | ||
import { optimisticOracleV2Fixture } from "./fixtures/OptimisticOracleV2.Fixture"; | ||
import { Provider, Signer, hre, toUtf8Bytes } from "./utils"; | ||
|
||
const ethers = hre.ethers; | ||
|
||
const createMonitoringParams = async (): Promise<MonitoringParams> => { | ||
// get chain id | ||
const chainId = await hre.ethers.provider.getNetwork().then((network) => network.chainId); | ||
// get hardhat signer | ||
const [signer] = await ethers.getSigners(); | ||
// Bot modes are not used as we are calling monitor modules directly. | ||
const botModes: BotModes = { | ||
disputeDisputableRequests: true, | ||
}; | ||
return { | ||
chainId: chainId, | ||
provider: ethers.provider as Provider, | ||
pollingDelay: 0, | ||
botModes, | ||
signer, | ||
maxBlockLookBack: 1000, | ||
blockLookback: 1000, | ||
}; | ||
}; | ||
|
||
describe("LLMDisputeDisputableRequests", function () { | ||
let bondToken: ExpandedERC20Ethers; | ||
let optimisticOracleV2: OptimisticOracleV2Ethers; | ||
let requester: Signer; | ||
let proposer: Signer; | ||
let disputer: Signer; | ||
|
||
const bond = ethers.utils.parseEther("1000"); | ||
|
||
const question = "This is just a test question"; | ||
const ancillaryData = toUtf8Bytes(question); | ||
|
||
beforeEach(async function () { | ||
// Signer from ethers and hardhat-ethers are not version compatible, thus, we cannot use the SignerWithAddress. | ||
[requester, proposer, disputer] = (await ethers.getSigners()) as Signer[]; | ||
|
||
// Get contract instances. | ||
const optimisticOracleV2Contracts = await optimisticOracleV2Fixture(); | ||
|
||
bondToken = optimisticOracleV2Contracts.bondToken; | ||
optimisticOracleV2 = optimisticOracleV2Contracts.optimisticOracleV2; | ||
|
||
// Fund proposer and disputer with bond amount and approve Optimistic Oracle V2 to spend bond tokens. | ||
await bondToken.addMinter(await requester.getAddress()); | ||
await bondToken.mint(await proposer.getAddress(), bond); | ||
await bondToken.mint(await disputer.getAddress(), bond); | ||
await bondToken.connect(proposer).approve(optimisticOracleV2.address, bond); | ||
await bondToken.connect(disputer).approve(optimisticOracleV2.address, bond); | ||
}); | ||
|
||
it("Disputes disputable requests", async function () { | ||
await ( | ||
await optimisticOracleV2.requestPrice(defaultOptimisticOracleV2Identifier, 0, ancillaryData, bondToken.address, 0) | ||
).wait(); | ||
|
||
await (await bondToken.connect(proposer).approve(optimisticOracleV2.address, bond)).wait(); | ||
await ( | ||
await optimisticOracleV2 | ||
.connect(proposer) | ||
.proposePrice( | ||
await requester.getAddress(), | ||
defaultOptimisticOracleV2Identifier, | ||
0, | ||
ancillaryData, | ||
ethers.utils.parseEther("1") | ||
) | ||
).wait(); | ||
|
||
const spy = sinon.spy(); | ||
const spyLogger = createNewLogger([new SpyTransport({}, { spy: spy })]); | ||
|
||
await disputeDisputableRequests(spyLogger, await createMonitoringParams()); | ||
assert.equal(spy.getCall(0).lastArg.at, "LLMDisputeBot"); | ||
}); | ||
}); |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is process() a static method on DisputerStrategy? Are all strategies intended to be built that way? Seems like stated unless might ultimately be necessary for future (non trivial) strategies. Either way, that's something we can worry about in a follow up PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can any of these calls error? i usually try to avoid processing things in a list like this since errors can prevent some from executing. if no errors are expected though this is fine