diff --git a/.gitignore b/.gitignore index 1d06ddc00..e88cb7b7a 100644 --- a/.gitignore +++ b/.gitignore @@ -11,6 +11,9 @@ database/opendc_testing/* # Old credential setup file keys.json +# Demo +/demo/ + # Traces /traces/ diff --git a/site/docs/documentation/Input/Scenario.md b/site/docs/documentation/Input/Scenario.md new file mode 100644 index 000000000..e5c4c58ca --- /dev/null +++ b/site/docs/documentation/Input/Scenario.md @@ -0,0 +1,125 @@ +The scenario of a simulation is defined using a JSON file. A scenario consists of one or more topologies, one or more +workloads, one or more allocation policies, a name and a number of times the simulation is being run. + +## Schema + +The schema for the scenario file is provided in [schema](ScenarioSchema) +In the following section, we describe the different components of the schema. + +### General Structure + +| Variable | Type | Required? | Default | Description | +|----------------------|----------------------------------------------|-----------|-------|--------------------------------------------------------------------------| +| name | string | no | "" | Name of the scenario, used for identification and referencing. | +| topologies | List[[Topology](#topology)] | yes | N/A | List of topologies used in the scenario. | +| workloads | List[[Workload](#workload)] | yes | N/A | List of workloads to be executed within the scenario. | +| allocationPolicies | List[[AllocationPolicy](#allocationpolicy)] | yes | N/A | Allocation policies used for resource management in the scenario. | +| failureModels | List[[FailureModel](#failuremodel)] | no | empty | List of failure models to simulate various types of failures. | +| exportModels | List[[ExportModel](#exportmodel)] | no | empty | Specifications for exporting data from the simulation. | +| carbonTracePaths | List[string] | no | null | Paths to carbon footprint trace files. | +| outputFolder | string | no | "output" | Directory where the simulation outputs will be stored. | +| initialSeed | integer | no | 0 | Seed used for random number generation to ensure reproducibility. | +| runs | integer | no | 1 | Number of times the scenario should be run. | + +### Topology + +| Variable | Type | Required? | Default | Description | +|-------------|--------|-----------|---------|---------------------------------------------------------------------| +| pathToFile | string | yes | N/A | Path to the JSON file defining the topology. | + +### Workload + +| Variable | Type | Required? | Default | Description | +|-------------|--------|-----------|---------|---------------------------------------------------------------------| +| pathToFile | string | yes | N/A | Path to the file containing the workload trace. | +| type | string | yes | N/A | Type of the workload (e.g., "ComputeWorkload"). | + +### AllocationPolicy + +| Variable | Type | Required? | Default | Description | +|-------------|--------|-----------|---------|---------------------------------------------------------------------| +| policyType | string | yes | N/A | Type of allocation policy (e.g., "BestFit", "FirstFit"). | + +### FailureModel + +| Variable | Type | Required? | Default | Description | +|-------------|--------|-----------|---------|---------------------------------------------------------------------| +| modelType | string | yes | N/A | Type of failure model to simulate specific operational failures. | + +### ExportModel + +| Variable | Type | Required? | Default | Description | +|-------------|--------|-----------|---------|---------------------------------------------------------------------| +| exportType | string | yes | N/A | Specifies the type of data export model for simulation results. | + + +## Examples +In the following section, we discuss several examples of Scenario files. Any scenario file can be verified using the +JSON schema defined in [schema](TopologySchema). + +### Simple + +The simplest scneario that can be provided to OpenDC is shown below: +```json +{ + "topologies": [ + { + "pathToFile": "topologies/topology1.json" + } + ], + "workloads": [ + { + "pathToFile": "traces/bitbrains-small", + "type": "ComputeWorkload" + } + ], + "allocationPolicies": [ + { + "policyType": "Mem" + } + ] +} +``` + +This scenario creates a simulation from file topology1, located in the topologies folder, with a workload trace from the +bitbrains-small file, and an allocation policy of type Mem. The simulation is run once (by default), and the default +name is "". + +### Complex +Following is an example of a more complex topology: +```json +{ + "topologies": [ + { + "pathToFile": "topologies/topology1.json" + }, + { + "pathToFile": "topologies/topology2.json" + }, + { + "pathToFile": "topologies/topology3.json" + } + ], + "workloads": [ + { + "pathToFile": "traces/bitbrains-small", + "type": "ComputeWorkload" + }, + { + "pathToFile": "traces/bitbrains-large", + "type": "ComputeWorkload" + } + ], + "allocationPolicies": [ + { + "policyType": "Mem" + }, + { + "policyType": "Mem-Inv" + } + ] +} +``` + +This scenario runs a total of 12 experiments. We have 3 topologies (3 datacenter configurations), each simulated with +2 distinct workloads, each using a different allocation policy (either Mem or Mem-Inv). diff --git a/site/docs/documentation/Input/ScenarioSchema.md b/site/docs/documentation/Input/ScenarioSchema.md new file mode 100644 index 000000000..bd800fd71 --- /dev/null +++ b/site/docs/documentation/Input/ScenarioSchema.md @@ -0,0 +1,81 @@ +Below is the schema for the Scenario JSON file. This schema can be used to validate a scenario file. +A scenario file can be validated using a JSON schema validator, such as https://www.jsonschemavalidator.net/. + +```json +{ + "$schema": "OpenDC/Scenario", + "$defs": { + "topology": { + "type": "object", + "properties": { + "pathToFile": { + "type": "string" + } + }, + "required": [ + "pathToFile" + ] + }, + "workload": { + "type": "object", + "properties": { + "pathToFile": { + "type": "string" + }, + "type": { + "type": "string" + } + }, + "required": [ + "pathToFile", + "type" + ] + }, + "allocationPolicy": { + "type": "object", + "properties": { + "policyType": { + "type": "string" + } + }, + "required": [ + "policyType" + ] + } + }, + "properties": { + "name": { + "type": "string" + }, + "topologies": { + "type": "array", + "items": { + "$ref": "#/$defs/topology" + }, + "minItems": 1 + }, + "workloads": { + "type": "array", + "items": { + "$ref": "#/$defs/workload" + }, + "minItems": 1 + }, + "allocationPolicies": { + "type": "array", + "items": { + "$ref": "#/$defs/allocationPolicy" + }, + "minItems": 1 + }, + "runs": { + "type": "integer" + } + }, + "required": [ + "topologies", + "workloads", + "allocationPolicies", + ] +} +``` diff --git a/site/docs/documentation/Input/Topology.md b/site/docs/documentation/Input/Topology.md index e54190788..8a57a1836 100644 --- a/site/docs/documentation/Input/Topology.md +++ b/site/docs/documentation/Input/Topology.md @@ -1,8 +1,10 @@ The topology of a datacenter is defined using a JSON file. A topology consist of one or more clusters. -Each cluster consist of at least one host on which jobs can be executed. Each host consist of one or more CPUs, a memory unit and a power model. +Each cluster consist of at least one host on which jobs can be executed. Each host consist of one or more CPUs, +a memory unit and a power model. ## Schema -The schema for the topology file is provided in [schema](TopologySchema). + +The schema for the topology file is provided in [schema](TopologySchema). In the following section, we describe the different components of the schema. ### Cluster @@ -54,24 +56,22 @@ In the following section, we describe the different components of the schema. | maxPower | string | Watt | yes | N/A | The power draw of a host when using max capacity in Watt | | idlePower | integer | Watt | yes | N/A | The power draw of a host when idle in Watt | - ## Examples -In the following section, we discuss several examples of topology files. Any topology file can be verified using the + +In the following section, we discuss several examples of topology files. Any topology file can be verified using the JSON schema defined in [schema](TopologySchema). ### Simple The simplest data center that can be provided to OpenDC is shown below: + ```json { - "clusters": - [ + "clusters": [ { - "hosts" : - [ + "hosts": [ { - "cpus": - [ + "cpus": [ { "coreCount": 16, "coreSpeed": 1000 @@ -91,19 +91,18 @@ This is creates a data center with a single cluster containing a single host. Th with a speed of 1 Ghz, and 100 MiB RAM memory. ### Count + Duplicating clusters, hosts, or CPUs is easy using the "count" keyword: + ```json { - "clusters": - [ + "clusters": [ { "count": 2, - "hosts" : - [ + "hosts": [ { "count": 5, - "cpus": - [ + "cpus": [ { "coreCount": 16, "coreSpeed": 1000, @@ -119,26 +118,26 @@ Duplicating clusters, hosts, or CPUs is easy using the "count" keyword: ] } ``` -This topology creates a datacenter consisting of 2 clusters, both containing 5 hosts. Each host contains 10 16 core CPUs. + +This topology creates a datacenter consisting of 2 clusters, both containing 5 hosts. Each host contains 10 16 core +CPUs. Using "count" saves a lot of copying. ### Complex + Following is an example of a more complex topology: ```json { - "clusters": - [ + "clusters": [ { "name": "C01", "count": 2, - "hosts" : - [ + "hosts": [ { "name": "H01", "count": 2, - "cpus": - [ + "cpus": [ { "coreCount": 16, "coreSpeed": 1000 @@ -147,8 +146,7 @@ Following is an example of a more complex topology: "memory": { "memorySize": 1000000 }, - "powerModel": - { + "powerModel": { "modelType": "linear", "idlePower": 200.0, "maxPower": 400.0 @@ -157,8 +155,7 @@ Following is an example of a more complex topology: { "name": "H02", "count": 2, - "cpus": - [ + "cpus": [ { "coreCount": 8, "coreSpeed": 3000 @@ -167,8 +164,7 @@ Following is an example of a more complex topology: "memory": { "memorySize": 100000 }, - "powerModel": - { + "powerModel": { "modelType": "square", "idlePower": 300.0, "maxPower": 500.0 @@ -180,5 +176,5 @@ Following is an example of a more complex topology: } ``` -This topology defines two types of hosts with different coreCount, and coreSpeed. +This topology defines two types of hosts with different coreCount, and coreSpeed. Both types of hosts are created twice. diff --git a/site/docs/documentation/Input/TopologySchema.md b/site/docs/documentation/Input/TopologySchema.md index 9f8f7575a..d8d3aa48d 100644 --- a/site/docs/documentation/Input/TopologySchema.md +++ b/site/docs/documentation/Input/TopologySchema.md @@ -1,5 +1,5 @@ Below is the schema for the Topology JSON file. This schema can be used to validate a topology file. -A topology file can be validated using using a JSON schema validator, such as https://www.jsonschemavalidator.net/. +A topology file can be validated using a JSON schema validator, such as https://www.jsonschemavalidator.net/. ```json {