Skip to content

Commit

Permalink
#2124: update JSON example and include json validator in docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@cwschilly
cwschilly committed May 29, 2024
1 parent 6617858 commit 0447f98
Showing 1 changed file with 159 additions and 38 deletions.
197 changes: 159 additions & 38 deletions docs/md/node-lb-data.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,41 +52,51 @@ and optionally the `index`, `home`, and `collection_id` for that object.
"entity": {
"collection_id": 7,
"home": 0,
"id": 12884901888,
"id": 3407875,
"index": [
3
12
],
"migratable": true,
"type": "object"
},
"node": 0,
"resource": "cpu",
"subphases": [
{
"id": 0,
"time": 0.014743804931640625
"time": 1.1263000033068238e-05
},
{
"id": 1,
"time": 1.1333999964335817e-05
}
],
"time": 0.014743804931640625
"time": 3.379300005690311e-05
},
{
"entity": {
"collection_id": 7,
"home": 0,
"id": 4294967296,
"id": 3145731,
"index": [
1
11
],
"migratable": true,
"type": "object"
},
"node": 0,
"resource": "cpu",
"subphases": [
{
"id": 0,
"time": 0.013672113418579102
"time": 1.1653000001388136e-05
},
{
"id": 1,
"time": 1.1435000033088727e-05
}
],
"time": 0.013672113418579102
"time": 3.452300006756559e-05
}
]
},
Expand All @@ -97,21 +107,51 @@ and optionally the `index`, `home`, and `collection_id` for that object.
"entity": {
"collection_id": 7,
"home": 0,
"id": 12884901888,
"id": 3407875,
"index": [
12
],
"migratable": true,
"type": "object"
},
"node": 0,
"resource": "cpu",
"subphases": [
{
"id": 0,
"time": 3.207300005669822e-05
},
{
"id": 1,
"time": 1.1347999816280208e-05
}
],
"time": 5.658399982166884e-05
},
{
"entity": {
"collection_id": 7,
"home": 0,
"id": 3145731,
"index": [
3
11
],
"migratable": true,
"type": "object"
},
"node": 0,
"resource": "cpu",
"subphases": [
{
"id": 0,
"time": 0.014104127883911133
"time": 1.3647000059791026e-05
},
{
"id": 1,
"time": 1.1320000112391426e-05
}
],
"time": 0.014104127883911133
"time": 3.787500008911593e-05
}
]
}
Expand All @@ -132,50 +172,40 @@ types, like an `object` or `node` depending on the type of communication.
{
"communications": [
{
"bytes": 262.0,
"bytes": 1456.0,
"from": {
"home": 1,
"home": 0,
"id": 1,
"migratable": false,
"type": "object"
},
"messages": 1,
"messages": 26,
"to": {
"home": 0,
"id": 4294967296,
"home": 1,
"id": 5,
"migratable": false,
"type": "object"
},
"type": "SendRecv"
},
{
"bytes": 96.0,
"bytes": 1456.0,
"from": {
"home": 0,
"id": 4294967296,
"type": "object"
},
"messages": 1,
"to": {
"id": 1,
"type": "node"
},
"type": "CollectionToNode"
},
{
"bytes": 259.0,
"from": {
"id": 0,
"type": "node"
"migratable": false,
"type": "object"
},
"messages": 1,
"messages": 26,
"to": {
"home": 0,
"id": 0,
"home": 2,
"id": 9,
"migratable": false,
"type": "object"
},
"type": "NodeToCollection"
"type": "SendRecv"
}
],
"id": 0
]
}
]
}
Expand All @@ -198,6 +228,97 @@ The type of communication lines up with the enum
For all the broadcast-like edges, the communication logging will occur on the
receive of the broadcast side (one entry per broadcast recipient).

\section JSON_data_files_validator.py

All input JSON files will be validated using the `JSON_data_files_validator.py` found in the `scripts` directory, which ensures that a given JSON adheres to the following schema:

\code{.py}
Schema(
{
Optional('type'): And(str, lambda a: a in allowed_types_data,
error=f"{self.get_error_message(allowed_types_data)} must be chosen"),
Optional('metadata'): {
Optional('type'): And(str, lambda a: a in allowed_types_data,
error=f"{self.get_error_message(allowed_types_data)} must be chosen"),
Optional('rank'): int,
Optional('shared_node'): {
'id': int,
'size': int,
'rank': int,
'num_nodes': int,
},
Optional('phases'): {
Optional('count'): int,
'skipped': {
'list': [int],
'range': [[int]],
},
'identical_to_previous': {
'list': [int],
'range': [[int]],
},
},
Optional('attributes'): dict
},
'phases': [
{
'id': int,
'tasks': [
{
'entity': {
Optional('collection_id'): int,
'home': int,
'id': int,
Optional('index'): [int],
'type': str,
'migratable': bool,
Optional('objgroup_id'): int
},
'node': int,
'resource': str,
Optional('subphases'): [
{
'id': int,
'time': float,
}
],
'time': float,
Optional('user_defined'): dict,
Optional('attributes'): dict
},
],
Optional('communications'): [
{
'type': str,
'to': {
'type': str,
'id': int,
Optional('home'): int,
Optional('collection_id'): int,
Optional('migratable'): bool,
Optional('index'): [int],
Optional('objgroup_id'): int,
},
'messages': int,
'from': {
'type': str,
'id': int,
Optional('home'): int,
Optional('collection_id'): int,
Optional('migratable'): bool,
Optional('index'): [int],
Optional('objgroup_id'): int,
},
'bytes': float
}
],
Optional('user_defined'): dict
},
]
}
)
\endcode

\section lb-spec-file LB Specification File
In order to customize when LB output is enabled and disabled, a LB
specification file can be passed to \vt via a command-line flag:
Expand Down

0 comments on commit 0447f98

Please sign in to comment.