Skip to content

Commit

Permalink
Documentation update (atlarge-research#224)
Browse files Browse the repository at this point in the history
  • Loading branch information
Radu-Nicolae authored and DanteNiewenhuis committed Apr 29, 2024
1 parent f7e374a commit d8ac0fd
Show file tree
Hide file tree
Showing 5 changed files with 236 additions and 31 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ database/opendc_testing/*
# Old credential setup file
keys.json

# Demo
/demo/

# Traces
/traces/

Expand Down
125 changes: 125 additions & 0 deletions site/docs/documentation/Input/Scenario.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
The scenario of a simulation is defined using a JSON file. A scenario consists of one or more topologies, one or more
workloads, one or more allocation policies, a name and a number of times the simulation is being run.

## Schema

The schema for the scenario file is provided in [schema](ScenarioSchema)
In the following section, we describe the different components of the schema.

### General Structure

| Variable | Type | Required? | Default | Description |
|----------------------|----------------------------------------------|-----------|-------|--------------------------------------------------------------------------|
| name | string | no | "" | Name of the scenario, used for identification and referencing. |
| topologies | List[[Topology](#topology)] | yes | N/A | List of topologies used in the scenario. |
| workloads | List[[Workload](#workload)] | yes | N/A | List of workloads to be executed within the scenario. |
| allocationPolicies | List[[AllocationPolicy](#allocationpolicy)] | yes | N/A | Allocation policies used for resource management in the scenario. |
| failureModels | List[[FailureModel](#failuremodel)] | no | empty | List of failure models to simulate various types of failures. |
| exportModels | List[[ExportModel](#exportmodel)] | no | empty | Specifications for exporting data from the simulation. |
| carbonTracePaths | List[string] | no | null | Paths to carbon footprint trace files. |
| outputFolder | string | no | "output" | Directory where the simulation outputs will be stored. |
| initialSeed | integer | no | 0 | Seed used for random number generation to ensure reproducibility. |
| runs | integer | no | 1 | Number of times the scenario should be run. |

### Topology

| Variable | Type | Required? | Default | Description |
|-------------|--------|-----------|---------|---------------------------------------------------------------------|
| pathToFile | string | yes | N/A | Path to the JSON file defining the topology. |

### Workload

| Variable | Type | Required? | Default | Description |
|-------------|--------|-----------|---------|---------------------------------------------------------------------|
| pathToFile | string | yes | N/A | Path to the file containing the workload trace. |
| type | string | yes | N/A | Type of the workload (e.g., "ComputeWorkload"). |

### AllocationPolicy

| Variable | Type | Required? | Default | Description |
|-------------|--------|-----------|---------|---------------------------------------------------------------------|
| policyType | string | yes | N/A | Type of allocation policy (e.g., "BestFit", "FirstFit"). |

### FailureModel

| Variable | Type | Required? | Default | Description |
|-------------|--------|-----------|---------|---------------------------------------------------------------------|
| modelType | string | yes | N/A | Type of failure model to simulate specific operational failures. |

### ExportModel

| Variable | Type | Required? | Default | Description |
|-------------|--------|-----------|---------|---------------------------------------------------------------------|
| exportType | string | yes | N/A | Specifies the type of data export model for simulation results. |


## Examples
In the following section, we discuss several examples of Scenario files. Any scenario file can be verified using the
JSON schema defined in [schema](TopologySchema).

### Simple

The simplest scneario that can be provided to OpenDC is shown below:
```json
{
"topologies": [
{
"pathToFile": "topologies/topology1.json"
}
],
"workloads": [
{
"pathToFile": "traces/bitbrains-small",
"type": "ComputeWorkload"
}
],
"allocationPolicies": [
{
"policyType": "Mem"
}
]
}
```

This scenario creates a simulation from file topology1, located in the topologies folder, with a workload trace from the
bitbrains-small file, and an allocation policy of type Mem. The simulation is run once (by default), and the default
name is "".

### Complex
Following is an example of a more complex topology:
```json
{
"topologies": [
{
"pathToFile": "topologies/topology1.json"
},
{
"pathToFile": "topologies/topology2.json"
},
{
"pathToFile": "topologies/topology3.json"
}
],
"workloads": [
{
"pathToFile": "traces/bitbrains-small",
"type": "ComputeWorkload"
},
{
"pathToFile": "traces/bitbrains-large",
"type": "ComputeWorkload"
}
],
"allocationPolicies": [
{
"policyType": "Mem"
},
{
"policyType": "Mem-Inv"
}
]
}
```

This scenario runs a total of 12 experiments. We have 3 topologies (3 datacenter configurations), each simulated with
2 distinct workloads, each using a different allocation policy (either Mem or Mem-Inv).
81 changes: 81 additions & 0 deletions site/docs/documentation/Input/ScenarioSchema.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
Below is the schema for the Scenario JSON file. This schema can be used to validate a scenario file.
A scenario file can be validated using a JSON schema validator, such as https://www.jsonschemavalidator.net/.

```json
{
"$schema": "OpenDC/Scenario",
"$defs": {
"topology": {
"type": "object",
"properties": {
"pathToFile": {
"type": "string"
}
},
"required": [
"pathToFile"
]
},
"workload": {
"type": "object",
"properties": {
"pathToFile": {
"type": "string"
},
"type": {
"type": "string"
}
},
"required": [
"pathToFile",
"type"
]
},
"allocationPolicy": {
"type": "object",
"properties": {
"policyType": {
"type": "string"
}
},
"required": [
"policyType"
]
}
},
"properties": {
"name": {
"type": "string"
},
"topologies": {
"type": "array",
"items": {
"$ref": "#/$defs/topology"
},
"minItems": 1
},
"workloads": {
"type": "array",
"items": {
"$ref": "#/$defs/workload"
},
"minItems": 1
},
"allocationPolicies": {
"type": "array",
"items": {
"$ref": "#/$defs/allocationPolicy"
},
"minItems": 1
},
"runs": {
"type": "integer"
}
},
"required": [
"topologies",
"workloads",
"allocationPolicies",
]
}
```
56 changes: 26 additions & 30 deletions site/docs/documentation/Input/Topology.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
The topology of a datacenter is defined using a JSON file. A topology consist of one or more clusters.
Each cluster consist of at least one host on which jobs can be executed. Each host consist of one or more CPUs, a memory unit and a power model.
Each cluster consist of at least one host on which jobs can be executed. Each host consist of one or more CPUs,
a memory unit and a power model.

## Schema
The schema for the topology file is provided in [schema](TopologySchema).

The schema for the topology file is provided in [schema](TopologySchema).
In the following section, we describe the different components of the schema.

### Cluster
Expand Down Expand Up @@ -54,24 +56,22 @@ In the following section, we describe the different components of the schema.
| maxPower | string | Watt | yes | N/A | The power draw of a host when using max capacity in Watt |
| idlePower | integer | Watt | yes | N/A | The power draw of a host when idle in Watt |


## Examples
In the following section, we discuss several examples of topology files. Any topology file can be verified using the

In the following section, we discuss several examples of topology files. Any topology file can be verified using the
JSON schema defined in [schema](TopologySchema).

### Simple

The simplest data center that can be provided to OpenDC is shown below:

```json
{
"clusters":
[
"clusters": [
{
"hosts" :
[
"hosts": [
{
"cpus":
[
"cpus": [
{
"coreCount": 16,
"coreSpeed": 1000
Expand All @@ -91,19 +91,18 @@ This is creates a data center with a single cluster containing a single host. Th
with a speed of 1 Ghz, and 100 MiB RAM memory.

### Count

Duplicating clusters, hosts, or CPUs is easy using the "count" keyword:

```json
{
"clusters":
[
"clusters": [
{
"count": 2,
"hosts" :
[
"hosts": [
{
"count": 5,
"cpus":
[
"cpus": [
{
"coreCount": 16,
"coreSpeed": 1000,
Expand All @@ -119,26 +118,26 @@ Duplicating clusters, hosts, or CPUs is easy using the "count" keyword:
]
}
```
This topology creates a datacenter consisting of 2 clusters, both containing 5 hosts. Each host contains 10 16 core CPUs.

This topology creates a datacenter consisting of 2 clusters, both containing 5 hosts. Each host contains 10 16 core
CPUs.
Using "count" saves a lot of copying.

### Complex

Following is an example of a more complex topology:

```json
{
"clusters":
[
"clusters": [
{
"name": "C01",
"count": 2,
"hosts" :
[
"hosts": [
{
"name": "H01",
"count": 2,
"cpus":
[
"cpus": [
{
"coreCount": 16,
"coreSpeed": 1000
Expand All @@ -147,8 +146,7 @@ Following is an example of a more complex topology:
"memory": {
"memorySize": 1000000
},
"powerModel":
{
"powerModel": {
"modelType": "linear",
"idlePower": 200.0,
"maxPower": 400.0
Expand All @@ -157,8 +155,7 @@ Following is an example of a more complex topology:
{
"name": "H02",
"count": 2,
"cpus":
[
"cpus": [
{
"coreCount": 8,
"coreSpeed": 3000
Expand All @@ -167,8 +164,7 @@ Following is an example of a more complex topology:
"memory": {
"memorySize": 100000
},
"powerModel":
{
"powerModel": {
"modelType": "square",
"idlePower": 300.0,
"maxPower": 500.0
Expand All @@ -180,5 +176,5 @@ Following is an example of a more complex topology:
}
```

This topology defines two types of hosts with different coreCount, and coreSpeed.
This topology defines two types of hosts with different coreCount, and coreSpeed.
Both types of hosts are created twice.
2 changes: 1 addition & 1 deletion site/docs/documentation/Input/TopologySchema.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Below is the schema for the Topology JSON file. This schema can be used to validate a topology file.
A topology file can be validated using using a JSON schema validator, such as https://www.jsonschemavalidator.net/.
A topology file can be validated using a JSON schema validator, such as https://www.jsonschemavalidator.net/.

```json
{
Expand Down

0 comments on commit d8ac0fd

Please sign in to comment.