Skip to content
This repository has been archived by the owner on Nov 19, 2024. It is now read-only.

feat: Add metrics about subnet usage on the state tree #191

Merged
merged 31 commits into from
Oct 4, 2023
Merged
Changes from 20 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
623521a
feat: Add metrics about subnet usage on the state tree
dsarlis Jun 28, 2023
4738ccc
Use a subnet endpoint
dsarlis Jul 17, 2023
fff992a
Fix capitalization
dsarlis Jul 17, 2023
81ec858
One more
dsarlis Jul 17, 2023
cd51408
Reorganize certified state reads section
dsarlis Jul 17, 2023
de4c34a
One more capital
dsarlis Jul 17, 2023
b3aa932
Apply suggestions from code review
dsarlis Jul 18, 2023
207bbee
Some more fixes after review
dsarlis Jul 18, 2023
ec99c44
define the meaning of total
mraszyk Jul 25, 2023
60fcd41
rephrase
mraszyk Jul 25, 2023
c9c5666
Merge branch 'master' into dimitris/subnet-id
mraszyk Jul 25, 2023
ff7caeb
Merge branch 'master' into dimitris/subnet-id
mraszyk Jul 25, 2023
41ae5bf
update may_read_path_for_canister
mraszyk Jul 25, 2023
6de6e26
Merge branch 'master' into dimitris/subnet-id
mraszyk Aug 4, 2023
68d301a
Merge branch 'master' into dimitris/subnet-id
mraszyk Aug 15, 2023
41a5905
Merge branch 'master' into dimitris/subnet-id
mraszyk Aug 23, 2023
39f7a81
More specific names for the metrics
dsarlis Aug 29, 2023
7847f9f
Updates to type of metrics
dsarlis Sep 1, 2023
b738b9c
Update
dsarlis Sep 1, 2023
05d7cc4
Use nat
dsarlis Sep 1, 2023
e56404f
add metrics to the formal semantics
mraszyk Sep 4, 2023
b8f1f98
Review comments
dsarlis Sep 4, 2023
e0efbc7
update read_state access restrictions
mraszyk Sep 4, 2023
7053bca
Merge branch 'master' into dimitris/subnet-id
mraszyk Sep 5, 2023
ff4ee7f
do not allow reading metrics with canister read state
mraszyk Sep 10, 2023
cc22cd5
Merge branch 'master' into dimitris/subnet-id
mraszyk Sep 12, 2023
e38777d
add a deprecation announcement
mraszyk Sep 12, 2023
c35966b
add note to formal semantics
mraszyk Sep 12, 2023
0ead6f0
Merge branch 'master' into dimitris/subnet-id
mraszyk Sep 13, 2023
b8afd4b
update conditions of read state certificate validity
mraszyk Sep 24, 2023
777e584
update changelog
mraszyk Oct 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 65 additions & 24 deletions spec/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -439,6 +439,16 @@ The state tree contains information about the topology of the Internet Computer.
principal = bytes .size (0..29)
tagged<t> = #6.55799(t) ; the CBOR tag

- `/subnet/<subnet_id>/metrics` (blob)
dsarlis marked this conversation as resolved.
Show resolved Hide resolved
mraszyk marked this conversation as resolved.
Show resolved Hide resolved

A collection of subnet-wide metrics related to this subnet's current resource usage and/or performance. The metrics are a CBOR map with the following fields:

- `num_canisters` (`nat`): The number of canisters on this subnet.
- `total_canister_state` (`nat`): The total size of the state in bytes taken by canisters on this subnet since this subnet was created.
- `total_consumed_cycles` (`map`): The total number of cycles consumed by canisters on this subnet since this subnet was created. It's a map of two values, a low part of type `nat` and a high part of type `opt nat`.
- `num_update_transactions` (`nat`): The total number of transactions processed on this subnet since this subnet was created.
dsarlis marked this conversation as resolved.
Show resolved Hide resolved


:::note

Because this uses the lexicographic ordering of princpials, and the byte distinguishing the various classes of ids is at the *end*, this range by construction conceptually includes principals of various classes. This specification needs to take care that the fact that principals that are not canisters may appear in these ranges does not cause confusion.
Expand Down Expand Up @@ -511,15 +521,15 @@ The concrete mechanism that users use to send requests to the Internet Computer

- At `/api/v2/canister/<effective_canister_id>/call` the user can submit (asynchronous, potentially state-changing) calls.

- At `/api/v2/canister/<effective_canister_id>/read_state` the user can read various information about the state of the Internet Computer. In particular, they can poll for the status of a call here.
- At `/api/v2/canister/<effective_canister_id>/read_state` or `/api/v2/subnet/<subnet_id>/read_state` the user can read various information about the state of the Internet Computer. In particular, they can poll for the status of a call here.

- At `/api/v2/canister/<effective_canister_id>/query` the user can perform (synchronous, non-state-changing) query calls.

- At `/api/v2/status` the user can retrieve status information about the Internet Computer.

In these paths, the `<effective_canister_id>` is the [textual representation](#textual-ids) of the [*effective* canister id](#http-effective-canister-id).

Requests to `/api/v2/canister/<effective_canister_id>/call`, `/api/v2/canister/<effective_canister_id>/read_state`, and `/api/v2/canister/<effective_canister_id>/query` are POST requests with a CBOR-encoded request body, which consists of a authentication envelope (as per [Authentication](#authentication)) and request-specific content as described below.
Requests to `/api/v2/canister/<effective_canister_id>/call`, `/api/v2/canister/<effective_canister_id>/read_state`, `/api/v2/subnet/<subnet_id>/read_state`, and `/api/v2/canister/<effective_canister_id>/query` are POST requests with a CBOR-encoded request body, which consists of a authentication envelope (as per [Authentication](#authentication)) and request-specific content as described below.

:::note

Expand Down Expand Up @@ -628,7 +638,7 @@ The functionality exposed via the [The IC management canister](#ic-management-ca

### Request: Read state {#http-read-state}

In order to read parts of the [The system state tree](#state-tree), the user makes a POST request to `/api/v2/canister/<effective_canister_id>/read_state`. The request body consists of an authentication envelope with a `content` map with the following fields:
In order to read parts of the [The system state tree](#state-tree), the user makes a POST request to `/api/v2/canister/<effective_canister_id>/read_state` or `/api/v2/subnet/<subnet_id>/read_state`. The subnet form should be used when the information to be retrieved is subnet specific, i.e., when requesting paths with the prefix `/time` or `/subnet`. The request body consists of an authentication envelope with a `content` map with the following fields:

- `request_type` (`text`): Always `read_state`

Expand Down Expand Up @@ -756,7 +766,7 @@ It must be contained in the canister ranges of a subnet, otherwise the correspon

:::note

The expectation is that user-side agent code shields users and developers from the notion of effective canister ID, in analogy to how the System API interface shields canister developers from worrying about routing.
The expectation is that user-side agent code shields users and developers from the notion of effective canister id, in analogy to how the System API interface shields canister developers from worrying about routing.

The Internet Computer blockchain mainnet does not support `provisional_create_canister_with_cycles` and thus all calls to this method are rejected independently of the effective canister id.

Expand Down Expand Up @@ -4643,9 +4653,9 @@ Query response

#### Certified state reads

The user can read elements of the *state tree*, using a `read_state` request to `/api/v2/canister/<ECID>/read_state`.
The user can read elements of the *state tree*, using a `read_state` request to `/api/v2/canister/<ECID>/read_state` or `/api/v2/subnet/<subnet_id>/read_state`.

Submitted request
Submitted request to `/api/v2/canister/<ECID>/read_state`
`E`

Conditions
Expand All @@ -4655,7 +4665,7 @@ Conditions
E.content = ReadState RS
TS = verify_envelope(E, RS.sender, S.system_time)
S.system_time <= RS.ingress_expiry
∀ path ∈ RS.paths. may_read_path(S, R.sender, path)
∀ path ∈ RS.paths. may_read_path_for_canister(S, R.sender, path)
dsarlis marked this conversation as resolved.
Show resolved Hide resolved
∀ (["request_status", Rid] · _) ∈ RS.paths. ∀ R ∈ dom(S.requests). hash_of_map(R) = Rid => R.canister_id ∈ TS

```
Expand All @@ -4665,33 +4675,64 @@ A record with

- `{certificate: C}`

The predicate `may_read_path` is defined as follows, implementing the access control outlined in [Request: Read state](#http-read-state):

may_read_path(S, _, ["time"]) = True
may_read_path(S, _, ["subnet"]) = True
may_read_path(S, _, ["subnet", sid]) = True
may_read_path(S, _, ["subnet", sid, "public_key"]) = True
may_read_path(S, _, ["subnet", sid, "canister_ranges"]) = True
may_read_path(S, _, ["request_status", Rid]) =
may_read_path(S, _, ["request_status", Rid, "status"]) =
may_read_path(S, _, ["request_status", Rid, "reply"]) =
may_read_path(S, _, ["request_status", Rid, "reject_code"]) =
may_read_path(S, _, ["request_status", Rid, "reject_message"]) =
may_read_path(S, _, ["request_status", Rid, "error_code"]) =

The predicate `may_read_path_for_canister` is defined as follows, implementing the access control outlined in [Request: Read state](#http-read-state):

may_read_path_for_canister(S, _, ["time"]) = True
may_read_path_for_canister(S, _, ["subnet"]) = True
may_read_path_for_canister(S, _, ["subnet", sid]) = True
may_read_path_for_canister(S, _, ["subnet", sid, "public_key"]) = True
may_read_path_for_canister(S, _, ["subnet", sid, "canister_ranges"]) = True
may_read_path_for_canister(S, _, ["request_status", Rid]) =
may_read_path_for_canister(S, _, ["request_status", Rid, "status"]) =
may_read_path_for_canister(S, _, ["request_status", Rid, "reply"]) =
may_read_path_for_canister(S, _, ["request_status", Rid, "reject_code"]) =
may_read_path_for_canister(S, _, ["request_status", Rid, "reject_message"]) =
may_read_path_for_canister(S, _, ["request_status", Rid, "error_code"]) =
∀ (R ↦ (_, ECID')) ∈ dom(S.requests). hash_of_map(R) = Rid => RS.sender == R.sender ∧ ECID == ECID'
may_read_path(S, _, ["canister", cid, "module_hash"]) = cid == ECID
may_read_path(S, _, ["canister", cid, "controllers"]) = cid == ECID
may_read_path(S, _, ["canister", cid, "metadata", name]) = cid == ECID ∧ UTF8(name) ∧
may_read_path_for_canister(S, _, ["canister", cid, "module_hash"]) = cid == ECID
may_read_path_for_canister(S, _, ["canister", cid, "controllers"]) = cid == ECID
may_read_path_for_canister(S, _, ["canister", cid, "metadata", name]) = cid == ECID ∧ UTF8(name) ∧
(cid ∉ dom(S.canisters[cid]) ∨
S.canisters[cid] = EmptyCanister ∨
name ∉ (dom(S.canisters[cid].public_custom_sections) ∪ dom(S.canisters[cid].private_custom_sections)) ∨
name ∈ dom(S.canisters[cid].public_custom_sections) ∨
(name ∈ dom(S.canisters[cid].private_custom_sections) ∧ RS.sender ∈ S.controllers[cid])
)
may_read_path(S, _, _) = False
may_read_path_for_canister(S, _, _) = False

where `UTF8(name)` holds if `name` is encoded in UTF-8.


Submitted request to `/api/v2/subnet/<subnet_id>/read_state`
`E`

Conditions

```html

E.content = ReadState RS
TS = verify_envelope(E, RS.sender, S.system_time)
S.system_time <= RS.ingress_expiry
∀ path ∈ RS.paths. may_read_path_for_subnet(S, RS.sender, path)

```

Read response
A record with

- `{certificate: C}`


The predicate `may_read_path_for_subnet` is defined as follows, implementing the access control outlined in [Request: Read state](#http-read-state):
mraszyk marked this conversation as resolved.
Show resolved Hide resolved

may_read_path_for_subnet(S, _, ["time"]) = True
may_read_path_for_subnet(S, _, ["subnet"]) = True
may_read_path_for_subnet(S, _, ["subnet", sid]) = True
may_read_path_for_subnet(S, _, ["subnet", sid, "public_key"]) = True
may_read_path_for_subnet(S, _, ["subnet", sid, "canister_ranges"]) = True
may_read_path_for_subnet(S, _, _) = False

The response is a certificate `cert`, as specified in [Certification](#certification), which passes `verify_cert` (assuming `S.root_key` as the root of trust), and where for every `path` documented in [The system state tree](#state-tree) that is a suffix of a path in `RS.paths` or of `["time"]`, we have

lookup_in_tree(path, cert.tree) = lookup_in_tree(path, state_tree(S))
Expand Down
Loading