Actions queue

[fordN]

Background

The action queue aims to decouple action execution from decision making, provide oversight of decisions to indexers, and provide a more clear delineation of concerns in the software. The indexer management server will handle external interactions (data fetching and executing actions) while the indexer agent will be focused on managing the reconciliation loop decision making process and ongoing management of allocations. By moving transaction execution and data fetching to the indexer management server, providing the option to turn off the agents allocation management, and providing an allocation management interface hosted by the the indexer management server we open up the design space for 3rd party decision making software to replace or supplement the indexer-agent.

Changes Summary

• Add action queue models and resolvers to indexer-common
• Add actions module to indexer-cli
• Add actions worker that polls and executes approved action queue
  items
• Add new oversight allocation management mode to the indexer-agent (description below)
• Update indexer-agent to add items to queue instead of executing them immediately
• Add action-queue usage guide to /docs
• Update README with up to date usage --help instructions for each component

Usage

The action execution worker will only grab items from the action queue to execute if they have ActionStatus = approved. In the recommended path actions are added to the queue with ActionStatus = queued, so they must then be approved in order to be executed on-chain. The indexer-agent now has 3 management modes (set using --allocation-management or INDEXER_AGENT_ALLOCATION_MANAGEMENT): auto, manual, and oversight.

Allocation management modes:

• auto: The indexer-agent will act similarly to the current paradigm. When it identifies allocation actions it will add them to the queue with ActionStatus = approved and the execution worker process will pick it up within 30 seconds and execute it.
• manual: The indexer-agent will not add any items to the action queue in this mode. It will host the indexer-management server which can be interacted with manually or integrated with 3rd party tools to add actions to the action queue and execute them.
• oversight: The indexer-agent will add run its reconciliation loop to make allocation decisions and will add proposed actions to the action queue with ActionStatus = queued. These actions will then require approval before they will be executed.

Actions CLI

An actions module has been added to the indexer-cli for interacting with the actions queue.

# Fetch all
./bin/graph-indexer indexer actions get all

# Fetch by status
./bin/graph-indexer indexer actions get --status queued

# Queue allocate action (allocateFrom())
./bin/graph-indexer indexer actions queue allocate QmeqJ6hsdyk9dVbo1tvRgAxWrVS3rkERiEMsxzPShKLco6 5000

# Queue reallocate action (closeAndAllocate())
./bin/graph-indexer indexer actions queue reallocate QmeqJ6hsdyk9dVbo1tvRgAxWrVS3rkERiEMsxzPShKLco6 0x4a58d33e27d3acbaecc92c15101fbc82f47c2ae5 55000

# Queue unallocate action (closeAllocation())
./bin/graph-indexer indexer actions queue unallocate QmeqJ6hsdyk9dVbo1tvRgAxWrVS3rkERiEMsxzPShKLco6 0x4a58d33e27d3acbaecc92c15101fbc82f47c2ae

# Approve multiple actions for execution
./bin/graph-indexer indexer actions approve 1 3 5 

[hopeyen]

Hey Ford! This PR is amazing! 😻 A ton of refactoring and design. Thank you so much! I have a few comments for you, other than that, the manual mode of action queue works amazingly with the optimizer

[hopeyen]

LGTM:)
Just a small question about ActionType.collect that was previously considered, I guess we don't need it since the worker will handle it along with unallocate and reallocate?

[fordN]

LGTM:) Just a small question about ActionType.collect that was previously considered, I guess we don't need it since the worker will handle it along with unallocate and reallocate?

Yeah, collect is just always performed after a successful unnallocate or reallocate action.