Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cosmos): Add a "CapData" vstorage RPC endpoint #8056

Merged
merged 11 commits into from
Jul 28, 2023
Merged
54 changes: 53 additions & 1 deletion golang/cosmos/proto/agoric/vstorage/query.proto
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,18 @@ option go_package = "github.com/Agoric/agoric-sdk/golang/cosmos/x/vstorage/types

// Query defines the gRPC querier service
service Query {
// Return an arbitrary vstorage datum.
// Return the raw string value of an arbitrary vstorage datum.
rpc Data(QueryDataRequest) returns (QueryDataResponse) {
option (google.api.http).get = "/agoric/vstorage/data/{path}";
}

// Return a formatted representation of a vstorage datum that must be
// a valid StreamCell with CapData values, or standalone CapData.
rpc CapData(QueryCapDataRequest)
returns (QueryCapDataResponse) {
option (google.api.http).get = "/agoric/vstorage/capdata/{path}";
}

// Return the children of a given vstorage path.
rpc Children(QueryChildrenRequest)
returns (QueryChildrenResponse) {
Expand All @@ -37,6 +44,51 @@ message QueryDataResponse {
];
}

// QueryCapDataRequest contains a path and formatting configuration.
message QueryCapDataRequest {
string path = 1 [
(gogoproto.jsontag) = "path",
(gogoproto.moretags) = "yaml:\"path\""
];
// mediaType must be an actual media type in the registry at
// https://www.iana.org/assignments/media-types/media-types.xhtml
// or a special value that does not conflict with the media type syntax.
// The only valid value is "JSON Lines", which is also the default.
string media_type = 2 [
(gogoproto.jsontag) = "mediaType",
(gogoproto.moretags) = "yaml:\"mediaType\""
];
// itemFormat, if present, must be the special value "flat" to indicate that
// the deep structure of each item should be flattened into a single level
// with kebab-case keys (e.g., `{ "metrics": { "min": 0, "max": 88 } }` as
// `{ "metrics-min": 0, "metrics-max": 88 }`).
string item_format = 3 [
(gogoproto.jsontag) = "itemFormat",
(gogoproto.moretags) = "yaml:\"itemFormat\""
];
// remotableValueFormat indicates how to transform references to opaque but
// distinguishable Remotables into readable embedded representations.
// * "object" represents each Remotable as an `{ id, allegedName }` object.
// * "string" represents each Remotable as a bracketed string such as `[Alleged: IST brand {}]`.
Comment on lines +71 to +72
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is important documentation. is it published somewhere using godoc ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that I know of (as mentioned in the PR summary, we don't seem to have any documentation of RPC endpoints). Does anyone have a place where they would expect to find it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't seem to have any documentation of RPC endpoints

It's far from perfect, but we have >0 for inter protocol RPC stuff: https://github.com/Agoric/agoric-sdk/tree/master/packages/inter-protocol#reading-data-off-chain

As to the rest, yes, it's been a hole in our docs for some time; I put what notes I learn in...

In some sense, this is a different issue, since that issue was about documentation of the form "Agoric stuff is based on cosmos-sdk; go read XYZ to find out how the lower level works." It turns out that there's no obvious candidate for XYZ, btw.

Does anyone have a place where they would expect to find it?

The inter-protocol precedent suggests a readme under golang/cosmos/x/vstorage

Copy link
Member Author

@gibson042 gibson042 Jul 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added golang/cosmos/x/vstorage/README.md.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, it turns out that the documentation is available via godoc of the generated pb.go.

string remotable_value_format = 10 [
(gogoproto.jsontag) = "remotableValueFormat",
(gogoproto.moretags) = "yaml:\"remotableValueFormat\""
];
}

// QueryCapDataResponse represents the result with the requested formatting,
// reserving space for future metadata such as media type.
message QueryCapDataResponse {
string block_height = 1 [
(gogoproto.jsontag) = "blockHeight",
(gogoproto.moretags) = "yaml:\"blockHeight\""
];
string value = 10 [
(gogoproto.jsontag) = "value",
(gogoproto.moretags) = "yaml:\"value\""
];
}

// QueryChildrenRequest is the vstorage path children query.
message QueryChildrenRequest {
string path = 1 [
Expand Down
32 changes: 16 additions & 16 deletions golang/cosmos/x/swingset/types/msgs.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

52 changes: 52 additions & 0 deletions golang/cosmos/x/vstorage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Virtual Storage

This module manages "[IAVL](https://github.com/cosmos/iavl)" chain storage data with a hierarchical keyspace in which each key is a "[path](./types/path_keys.go)" composed of zero or more dot-separated nonempty segments in a restricted alphabet. It exposes gRPC endpoints to arbitrary external clients for reading data, and internal read/write interfaces for use by SwingSet (which itself manages further subtree-scoped attenuation).

## Internal Go interface

[Keeper](./keeper/keeper.go)
* generic
* GetChildren
* GetEntry
* HasEntry
* HasStorage
* SetStorage[AndNotify]
* StreamCell-oriented (a StreamCell captures a block height and an array of values)
* AppendStorageValue[AndNotify]
* queue-oriented (a queue stores items at paths like "$prefix.$n", documenting
the n for the next item to be consumed at "$prefix.head" and the n for the next
next item to be pushed at "$prefix.tail" such that the queue is empty when both
head and tail store the same n)
* GetQueueLength
* PushQueueItem

## Internal JSON interface

This is used by the SwingSet "bridge".

[Receive](./vstorage.go) with input `{ "method": "...", "args": [...] }`
* generic
* method "entries", args path
* method "get"/"has", args path
* method "set"/"setWithoutNotify", args [[path, value?], ...]
* method "children", args path
* method "values", args path (returns values for children in the same order as method "children")
* method "size", args path (returns the count of children)
* StreamCell-oriented
* method "append", args [[path, value?], ...]

## External protobuf interface

gRPC via [Querier](./keeper/keeper.go)
and CometBFT method "abci_query" with params `{ "path": "/agoric.vstorage.Query/...", "data": "<hexadecimal representation of serialized protobuf>" }` via the same
* /agoric.vstorage.Query/CapData
* /agoric.vstorage.Query/Children
* /agoric.vstorage.Query/Data

## Arbitrary-response HTTP interface

This depends upon appModule `LegacyQuerierHandler` functionality that is [removed from cosmos-sdk as of v0.47](https://github.com/cosmos/cosmos-sdk/blob/fa4d87ef7e6d87aaccc94c337ffd2fe90fcb7a9d/CHANGELOG.md#api-breaking-changes-3)

[legacy querier](./keeper/querier.go)
* /custom/vstorage/children/$path
* /custom/vstorage/data/$path
Loading