feat: high priority processing of economic activity

[mhofman]

closes: #5966
closes: #6576
closes: #6964
refs: #5334

Best reviewed commit-by-commit

Description

This change introduces a high priority queue, with cosmic swingset processing actions from that queue before timer polls (#6964) and non-priority actions (#5966).

It also introduces a new chain storage prefix highPrioritySenders where it expects to find entries for each address to be considered for priority processing. This path is top level and will require a new a new "root" chain storage created for it in bootstrap. JS should populate that collection by creating a child node for the address, and setting any non-empty value. To remove the address, it should set the node to the empty string.

When considering messages, the inbound ante handler will check whether the message is from a high priority sender, and allow the message regardless of the inbound queue size.

Security Considerations

This introduces a way to bypass inbound queue size checks, but gated on the verified signer.

Scaling Considerations

The lookup of the high priority status of the sender must be done for every message going into the mempool, and is thus done in O(1) by a direct path lookup.

Documentation Considerations

TBD

Testing Considerations

Added a unit test for the inbound checks in golang
We do not have tests covering the routing of messages, either on the golang or the JS side, and I didn't add any.
In a perfect world we should have an integration test for #5334 verifying the end-to-end behavior.

[mhofman]

cc @turadg @dckc

[JimLarson]

Please don't mix reformatting with substantial changes in a single commit - separate reformatting and other whitespace-only changes, and maybe mechanical renaming, into a separate commit so that the reviewer is free to skim it.

[mhofman]

Yeah I took a shortcut and described the about to be semantics. I'll try to update the comment to first reflect the naming change before updating again when changing the semantics.

[mhofman]

Let me know if the new commit-by-commit change addresses your concern.

[JimLarson]

Note: "Priority Queue" is a data structure which lets you remove the highest-priority item efficiently, so calling the high-priority queue the "priority queue" is potentially confusing. Consider calling it the "high priority queue", which leaves room for additional priority levels in the future. Avoid calling it the "economy queue" as a) it mixes mechanism with policy, and b) suggests a lower priority by analogy with airline boarding.

[JimLarson]

LGTM with one minor issue to consider.

[gibson042]

What block is "the block" here (e.g., is it presumptive next block still under construction)? And what are the "allotments"?

[gibson042]

Suggestion: Move the string concatenation into the caller (where it already exists anyway in the case of pushAction) and refactor into something like

Suggested change

	func (k Keeper) queueIndex(ctx sdk.Context, queuePath string, position string) (sdk.Int, error) {
	func (k Keeper) getInt(ctx sdk.Context, path string, default *sdk.Int) (sdk.Int, error) {

 	// Get the current queue tail, defaulting to zero if its vstorage doesn't exist.
 	// The `tail` is the value of the next index to be inserted
-	tail, err := k.queueIndex(ctx, inboundQueuePath, "tail")
+	tail, err := k.getInt(ctx, inboundQueuePath + ".tail", zero)
 	if err != nil {
 		return err
 	}

 func (k Keeper) queueLength(ctx sdk.Context, queuePath string) (sdk.Int, error) {
-	head, err := k.queueIndex(ctx, queuePath, "head")
+	head, err := k.getInt(ctx, queuePath + ".head", zero)
 	if err != nil {
 		return sdk.NewInt(0), err
 	}
-	tail, err := k.queueIndex(ctx, queuePath, "tail")
+	tail, err := k.getInt(ctx, queuePath + ".tail", zero)
 	if err != nil {
 		return sdk.NewInt(0), err
 	}
 }

It could even be moved down into vstorage/keeper, and/or the queue concept could be abstracted here into e.g. queuePush(ctx sdk.Context, queuePath string) error and queueLength(ctx sdk.Context, queuePath string) (sdk.Int, error) rather than the mismatched pushAction and queueLength.

[mhofman]

Refactored and moved to vstorage mostly as suggested (minus the default value).

[gibson042]

I think there's a slightly more conventional way to express this.

Suggested change

	const queueRawStorage = makeBufferedStorage(
	makePrefixedBridgeStorage(
	sendToChain,
	`${queuePath}.`,
	'setWithoutNotify',
	x => x,
	x => x,
	),
	);
	return harden({
	...queueRawStorage.kvStore,
	commit: queueRawStorage.commit,
	abort: queueRawStorage.abort,
	});
	const { kvStore, commit, abort } = makeBufferedStorage(
	makePrefixedBridgeStorage(
	sendToChain,
	`${queuePath}.`,
	'setWithoutNotify',
	x => x,
	x => x,
	),
	);
	return harden({ ...kvStore, commit, abort });

[mhofman]

Duh

[gibson042]

Suggested change

	* golang side will push into the queue, updating `<prefix>tail` and setting
	* `<prefix><index>`.
	* The JS side will shift the queue, updating `<prefix>head` and reading and
	* deleting `<prefix><index>`.
	* golang side will push into the queue, updating the index stored at key
	* `${queuePath}.tail` and setting data for key `${queuePath}.${index}`.
	* The JS side will shift the queue, updating the index at key
	* `${queuePath}.head` and reading and deleting `${queuePath}.${index}`.

[mhofman]

@Mergifyio refresh

[mergify]

refresh

✅ Pull request refreshed