Update documentation

Signed-off-by: David Schall <david.schall@ed.ac.uk>
vhive-serverless · Jun 18, 2022 · 9b5acbc · 9b5acbc
1 parent ddd1940
commit 9b5acbc
Show file tree

Hide file tree

Showing 22 changed files with 155 additions and 132 deletions.
diff --git a/.github/configs/functions-under-test.txt b/.github/configs/functions-under-test.txt
@@ -4,18 +4,4 @@ fibonacci-python
 aes-go
 aes-nodejs
 aes-python
-auth-go
-auth-nodejs
-auth-python
-geo
-profile
-rate
-recommendation
-reservation
-user
-productcatalogservice
-currencyservice
-paymentservice
-shippingservice
-emailservice
-recommendationservice
+
diff --git a/.github/configs/wordlist.txt b/.github/configs/wordlist.txt
@@ -706,4 +706,26 @@ qcow
 qcow2
 permalink
 fs
-fw
+fw
+
+gu
+wkdir
+ExperimentConfigs
+ExperimentStats
+subclasses
+ipc
+str
+AtomicSimpleCPU
+TimingSimpleCPU
+SimpleMemory
+OoO
+DDR
+Skylake
+
+subfolder
+PID
+JIT
+runtime
+Jupyter
+jupyter
+microarchitect's
diff --git a/README.md b/README.md
@@ -7,38 +7,46 @@
 [![Build base disk image](https://github.com/ease-lab/vSwarm-u/actions/workflows/create_base_disk.yml/badge.svg)](https://github.com/ease-lab/vSwarm-u/actions/workflows/create_base_disk.yml)
 [![Function CI for gem5 Simulator](https://github.com/ease-lab/vSwarm-u/actions/workflows/function_ci.yml/badge.svg)](https://github.com/ease-lab/vSwarm-u/actions/workflows/function_ci.yml)
 
-[![](https://img.shields.io/badge/doc-latest-green)](https://ease-lab.github.io/vSwarm-u/)
-<!-- ![](https://img.shields.io/static/v1?label=&message=documentation&color=green) -->
+[![doc](https://img.shields.io/badge/doc-latest-green)](https://ease-lab.github.io/vSwarm-u/)
 ![GitHub release (latest by date)](https://img.shields.io/github/v/release/ease-lab/vSwarm-u)
-
-
-
-<!-- ![GitHub Workflow Status](https://img.shields.io/github/workflow/status/ease-lab/vSwarm-u/function_ci.yml?logo=Build%20kernel) -->
-
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 <a href="https://twitter.com/intent/follow?screen_name=ease_lab" target="_blank">
 <img src="https://img.shields.io/twitter/follow/ease_lab?style=social&logo=twitter" alt="follow on Twitter"></a>
 
+<!-- ![GitHub Workflow Status](https://img.shields.io/github/workflow/status/ease-lab/vSwarm-u/function_ci.yml?logo=Build%20kernel) -->
+
+<!-- ![](https://img.shields.io/static/v1?label=&message=documentation&color=green) -->
 
 <!-- ![vSwarm-u Header](docs/figures/vhive_hdr.jpg) -->
 
-> **Warning**
-> *vSwarm-u is still in beta state but will be released soon for the ISCA'22 conference.*
+<span style="font-weight:700;font-size:17px">
+Serverless computing represent a new class of workloads, challenging the microarchitecture of modern CPU's. The mission of vSwarm-&mu; is to provide microarchitect's with the tools they need to address those challenges in next generation CPU's.
+</span>
 
-## Mission
+<!-- <span style="color:green;font-weight:700;font-size:20px">
+    markdown color font styles
+</span> -->
 
-Serverless computing has emerged as a widely used paradigm for deploying services in the cloud. In serverless, developers organize their application as a set of functions, which are invoked on-demand in response to a trigger, such as user request or an invocation by another function.
 
-Recent studies of production data  reveal drastic differences in the characteristics of serverless workloads compared to conventional cloud workloads: short execution time and infrequent invocation of function instances. Performance studies  further finds that serverless workloads are inefficient when running on modern CPUs designed for traditional long-running workloads. To make serverless workload execution efficient, there is a strong need to understand more about the detailed implications serverless workload characteristics have on modern hardware.
+## Challenges of Serverless Host Simulation
 
-However, existing platforms that support the required level of detail make significant simplifications in the test setup and the software stack to achieve feasible simulation times. Prior work often lacks the key layers of the serverless software stack, such as containerization and HTTP-level communication fabric, to simplify and increase the simulation speed. However, the short execution time of serverless functions leads to a significant fraction of execution time spent in system layers. Such simplifications may result in wrong experimental data and, consequently, mislead the systems researchers.
+Serverless computing has emerged as a widely used paradigm for deploying services in the cloud. However, with their unique characteristics of short execution times and infrequent invocation of function instances serverless workloads set new demands on both the microarchitecture of modern server CPU’s as well as the research tools built to design this microarchitecture.
 
-With vSwarm-&mu; we are addressing the challenges of serverless host server simulation and allow researchers to conduct experiments with systems representative of a modern serverless cloud. To achieve this, the vSwarm-u framework integrates various serverless workloads packages as containerized functions and featuring the full communication stack with gem5, the state-of-the-art research platform for system- and microarchitecture. This allows researcher to perform cycles accurate simulation of the representative serverless software stack in gem5’s full system mode.
+gem5 is the state-of-the-art research platform for system- and microarchitecture but does not support key layers of the serverless software stack in its reference setup. Moreover, the common way of ignoring large parts of the system stack and simplifying test scenarios in favour of simulation speed is not applicable to serverless. Due to its significant part faction of execution time spend in communication and system stack components such simplifications may result in wrong experimental data and, consequently, mislead research.
 
-Furthermore, vSwarm-u includes the infrastructure to drive function instances running on the simulated serverless host server without interfering or simplifying the complexity of the test system. The robust evaluation methodology allows benchmarking and microarchitecture analysis in a realistic scenario.
 
-## vSwarm-u Design
-<img src="docs/figures/vswarm-u-design.jpg" title="vSwarm-u design" height="200"/>
+## The vSwarm-&mu; Framework
+
+| Serverless software stack on gem5 | Serverless Test infrastructure |
+|---|---|
+| <img src="./docs/figures/serverless-stack.jpg" title="Serverless Stack"/> | <img src="./docs/figures/serverless-system.jpg" title="vSwarm-u design"/> |
+
+vSwarm-&mu; offers two main components to address the challenges of serverless host simulation. A full serverless software stack including kernel, OS and packages compatible with gem5. Furthermore, the infrastructure to drive function instances running on the simulated serverless host server without interfering or simplifying the complexity of the test system.
+
+That will equip microarchitect's and researchers with the tools they need to identify the real bottlenecks in modern hardware when running serverless workloads.
+
+Interested? Find more information in the [methodology](./docs/methodology.md) section and the remaining documentation.
+
 
 ## Referencing our work
 
@@ -47,6 +55,7 @@ advice for potential extensions of vSwarm-&mu; and always open for collaboration
 
 Please cite our [paper](https://ease-lab.github.io/ease_website/pubs/JUKEBOX_ISCA22.pdf) that has been recently accepted to ISCA 2022:
 
+
 ```
 @inproceedings{schall:lukewarm,
   author    = {Schall, David and
@@ -75,8 +84,6 @@ Please cite our [paper](https://ease-lab.github.io/ease_website/pubs/JUKEBOX_ISC
 vSwarm-&mu; can be deployed on premises or in the cloud, with support for nested virtualization. We provide [a quick-start guide](./docs/quick_start.md)
 that describes the initial setup, as well as how to set up benchmarking experiments.
 
-You can view the vSwarm-&mu; documentation [here](docs/).
-
 
 ### Getting help and contributing
 
@@ -100,15 +107,13 @@ The software is maintained at the [EASE lab](https://easelab.inf.ed.ac.uk/) as p
 ## Directory Structure
 
 - `simulation` contains everything related to run simulations.
-- `setup` contains all scripts and makefiles to setup vSwarm-u
+- `setup` contains all scripts and makefiles to setup vSwarm-&mu;
 - `tools` includes a client that can be instrumented for gem5
 - `runner` is for setting up self-hosted GitHub Actions runners.
 - `docs` contains the documentation on a number of relevant topics.
 
 
 
-
-
 ## Known problems
 
 Sometimes the simulator get stuck during while running with the detailed core model. We are working on it and try to make the setup more stable.

diff --git a/docs/figures/conventional-system.jpg b/docs/figures/conventional-system.jpg
diff --git a/docs/figures/conventional-workloads.jpg b/docs/figures/conventional-workloads.jpg
diff --git a/docs/figures/serverless-stack.jpg b/docs/figures/serverless-stack.jpg
diff --git a/docs/figures/vswarm-u-design.jpg → docs/figures/serverless-system.jpg b/docs/figures/vswarm-u-design.jpg → docs/figures/serverless-system.jpg
diff --git a/docs/figures/serverless-workloads.jpg b/docs/figures/serverless-workloads.jpg
diff --git a/docs/figures/skylake-system.jpg b/docs/figures/skylake-system.jpg
diff --git a/docs/figures/two-core.jpg b/docs/figures/two-core.jpg
diff --git a/docs/figures/two-machine.jpg b/docs/figures/two-machine.jpg
diff --git a/docs/index.md b/docs/index.md
@@ -9,30 +9,29 @@ has_children: false
 
 <!-- # vSwarm-&mu; Documentation -->
 
+Serverless computing represent a new class of workloads, challenging the microarchitecture of modern CPU's. The mission of vSwarm-&mu; is to provide microarchitect's with the tools they need to address those challenges in next generation CPU's.
+{: .fs-6 .fw-300 }
 
-> **Warning**
-> {: .label .label-red }
->*<font color="red">vSwarm-u is still in beta state but will be released soon for the ISCA'22 conference.</font>*
 
-[Serverless computing represent new challenges for modern CPUs!](https://ease-lab.github.io/ease_website/pubs/JUKEBOX_ISCA22.pdf)
-With vSwarm-&mu; we aim to enable microarchitectural research for serverless
-{: .fs-6 .fw-300 }
+## Challenges of Serverless Host Simulation
+
+Serverless computing has emerged as a widely used paradigm for deploying services in the cloud. However, with their unique characteristics of short execution times and infrequent invocation of function instances serverless workloads set new demands on both the microarchitecture of modern server CPU’s as well as the research tools built to design this microarchitecture.
 
+gem5 is the state-of-the-art research platform for system- and microarchitecture but does not support key layers of the serverless software stack in its reference setup. Moreover, the common way of ignoring large parts of the system stack and simplifying test scenarios in favour of simulation speed is not applicable to serverless. Due to its significant part faction of execution time spend in communication and system stack components such simplifications may result in wrong experimental data and, consequently, mislead research.
 
-## Mission
 
-Serverless computing has emerged as a widely used paradigm for deploying services in the cloud. In serverless, developers organize their application as a set of functions, which are invoked on-demand in response to a trigger, such as user request or an invocation by another function.
+## The vSwarm-&mu; Framework
 
-Recent studies of production data  reveal drastic differences in the characteristics of serverless workloads compared to conventional cloud workloads: short execution time and infrequent invocation of function instances. Performance studies  further finds that serverless workloads are inefficient when running on modern CPUs designed for traditional long-running workloads. To make serverless workload execution efficient, there is a strong need to understand more about the detailed implications serverless workload characteristics have on modern hardware.
+| Serverless software stack on gem5 | Serverless Test infrastructure |
+|---|---|
+| <img src="./figures/serverless-stack.jpg" title="Serverless Stack"/> | <img src="./figures/serverless-system.jpg" title="vSwarm-u design"/> |
 
-However, existing platforms that support the required level of detail make significant simplifications in the test setup and the software stack to achieve feasible simulation times. Prior work often lacks the key layers of the serverless software stack, such as containerization and HTTP-level communication fabric, to simplify and increase the simulation speed. However, the short execution time of serverless functions leads to a significant fraction of execution time spent in system layers. Such simplifications may result in wrong experimental data and, consequently, mislead the systems researchers.
+vSwarm-&mu; offers two main components to address the challenges of serverless host simulation. A full serverless software stack including kernel, OS and packages compatible with gem5. Furthermore, the infrastructure to drive function instances running on the simulated serverless host server without interfering or simplifying the complexity of the test system.
 
-With vSwarm-&mu; we are addressing the challenges of serverless host server simulation and allow researchers to conduct experiments with systems representative of a modern serverless cloud. To achieve this, the vSwarm-u framework integrates various serverless workloads packages as containerized functions and featuring the full communication stack with gem5, the state-of-the-art research platform for system- and microarchitecture. This allows researcher to perform cycles accurate simulation of the representative serverless software stack in gem5’s full system mode.
+That will equip microarchitect's and researchers with the tools they need to identify the real bottlenecks in modern hardware when running serverless workloads.
 
-Furthermore, vSwarm-u includes the infrastructure to drive function instances running on the simulated serverless host server without interfering or simplifying the complexity of the test system. The robust evaluation methodology allows benchmarking and microarchitecture analysis in a realistic scenario.
+Interested? Find more information in the [methodology](./methodology.md) section and the remaining documentation.
 
-## vSwarm-u Design
-<img src="figures/vswarm-u-design.jpg" title="vSwarm-u design" height="200"/>
 
 ## Referencing our work
 
@@ -93,7 +92,7 @@ The software is maintained at the [EASE lab](https://easelab.inf.ed.ac.uk/) as p
 ## Directory Structure
 
 - `simulation` contains everything related to run simulations.
-- `setup` contains all scripts and makefiles to setup vSwarm-u
+- `setup` contains all scripts and makefiles to setup vSwarm-&mu;
 - `tools` includes a client that can be instrumented for gem5
 - `runner` is for setting up self-hosted GitHub Actions runners.
 - `docs` contains the documentation on a number of relevant topics.

diff --git a/docs/methodology.md b/docs/methodology.md
@@ -0,0 +1,10 @@
+---
+layout: default
+title: Home
+nav_order: 1
+has_children: false
+---
+
+# Methodology: What is vSwarm-&mu;?
+
+> Please give me one more day to add this :)
diff --git a/docs/quick_start.md b/docs/quick_start.md
@@ -5,7 +5,7 @@ nav_order: 2
 has_children: false
 ---
 
-# vSwarm-u: Quickstart with Serverless and gem5
+# vSwarm-&mu;: Quickstart with Serverless and gem5
 
 This guide describes how to quickly setup your machine and run the first cycle accurate simulations of serverless functions using gem5.
 
@@ -23,7 +23,7 @@ Brew yourself a coffee ☕️😉 in the meantime. It can take its time to to se
 
 3. **Download latest release artifacts** *(~3 min)*
 
-   With vSwarm-u we distribute a compiled kernel, client and ready to use and tested disk image. You can find the latest release of those resources on the [release page](https://github.com/ease-lab/vSwarm-u/releases) of the repo. Refer to the [resources](setup/setup.md#get-resources) section in the documentation for more details how to download and use them.
+   With vSwarm-&mu; we distribute a compiled kernel, client and ready to use and tested disk image. You can find the latest release of those resources on the [release page](https://github.com/ease-lab/vSwarm-u/releases) of the repo. Refer to the [resources](setup/setup.md#get-resources) section in the documentation for more details how to download and use them.
 
 
 ### Test Downloads

diff --git a/docs/simulation/functions.md b/docs/simulation/functions.md
@@ -13,7 +13,7 @@ Currently we support and tested 21 functions from our benchmark suite vSwarm
 
 
 ## Standalone kernels
-The benchmark suite contains three functions that implement the same functionality in different runtime. With this the difference in the runtime's can be explored. The runtime's golang, nodejs, python are most common used for serverless and are representatives for a compiled, JIT-compiled and interpreted language. Details about the functions can be found in [vSwarm](https://github.com/ease-lab/vSwarm/tree/main/benchmarks)
+The benchmark suite contains three functions that implement the same functionality in different runtime. With this the difference in the runtimes can be explored. The runtimes golang, nodejs, python are most common used for serverless and are representatives for a compiled, JIT-compiled and interpreted language. Details about the functions can be found in [vSwarm](https://github.com/ease-lab/vSwarm/tree/main/benchmarks)
 
 
 | Benchmark      | Name | Languages implemented  | gem5 support |

diff --git a/docs/simulation/instrumentation.md b/docs/simulation/instrumentation.md
@@ -19,20 +19,32 @@ nav_order: 10
 
 ---
 
-In contrast to conventional workloads serverless function run for very short amount of time. The challenge for those kind of workloads however that due to the short nature the interval where do measurements need to be defined more precise.
+In contrast to conventional workloads serverless function run for short amount of time. Furthermore we might want to run several invocations to compare different inputs. The challenge is that now the intervals where to do measurements need to defined more precise.
 
 <figure>
   <img src="../figures/trigger-points.jpg" title="Instrumentation points" />
     <figcaption> <i><b>Figure 1:</b> Overall simulation flow and several intervals to be measured.</i></figcaption>
 </figure>
 
-Figure 1 show the the overall simulation and different interval that are interesting to do measurements. The following describe how different points in this flow are instrumented.
+Figure 1 show the the overall simulation and different intervals that are interesting to do measurements. The following describe how different points in this flow are instrumented.
 
-## M5 Fail codes
-For driving the simulator fail codes are executed at various points during the simulation.  via the m5 binary
+## M5 fail codes
+For driving the simulator `m5 fail <code>` codes are executed at various points during the simulation. Using `m5 fail` instead of `m5 exit` has the advantage to send also a code which can encode more information. Refer to the templates in your initial working directory.
 
 
-## Instrumented go client.
+## Client side instrumentation
 Using the `m5` binary is fine for rough trigger points. However, it is not very precise as some OS interaction is involved when executing a command. (Need to load the binary, allocate memory,...)
 
-In order to get around this and get more precise trigger points the repo includes a instrumented to client. Under to hood it takes advantage of the [`m5ops`]() magic instruction
+In order to get around this issue and get more precise trigger points the repo includes a instrumented to client in `tools/client/`. Under to hood it takes advantage of the [`m5ops`]() magic instruction. Refer to the gem5 documentation to for more information how magic instructions work.
+
+For convenience we wrap gem5's magic instructions in a go package to be found in the m5 subfolder of the client.
+
+To run the client with instrumentation set the flag `-m5ops` when running the client.
+Note that m5 magic instruction take advantage of `mmap` which is only available with root permissions. This is not an issue issue when running the client on the disk image as we have root permissions there. However, it will raise a warning when running the client on your local machine.
+
+
+## Server side instrumentation
+For the server side we instrumented the linux scheduler by hooking up a `FuncEvent` on the `__schedule` symbol. With compiling also the `thread_info` into the kernel we can get the previous and next PID for every scheduling event.
+This enables us to react on certain scheduling events.
+
+However, this is still work in progress and will be rolled out soon.