Capture data with Kepler using a simple process

[zanete]

Why: Sub of #1027
What: A locally runnable script that runs a simple process and captures data using Kepler

Scope of work:

• figure out a common infrastructure that will support Kepler
• Set up the process to run a simple process and capture data with Kepler
• Research if this setup is possible on MacOS
• Step by step commands that can be run locally to replicate

[zanete]

need to install linux VM and transfer the architecture setup to ubuntu today as kepler doesn't work on MacOS

[narekhovhannisyan]

@jawache @jmcook1186 I have tried the same Kubernetes setup on Windows machine (grabbed one from @MariamKhalatova), and was able to get Kepler output file. Grafana visualization is not working yet, however we got data! Let’s investigate and see if it’s something useful.
Interesting facts: Kepler and Scaphandre are not working on macOS, and they won’t work on VMs too if they are running on top of macOS.

metrics.txt

[zanete]

discuss with @jmcook1186 to make sure we have useful information

[jawache]

Thanks @narekhovhannisyan.

NOTE: @jmcook1186 if Kepler and Scaphandre doesn't work on MacOS then using them in the training course is quite problematic, just a note re: the course that we should probably just give people the observations in a CSV (including some Kepler observations) and focus on helping them take the observations and turn them into impacts rather than running a process and capturing observations.

[jmcook1186]

@jawache @narekhovhannisyan and I sync'd on this today - here's the notes I took:

What has been done so far:

• sourced a windows laptop becuase kep[ler and scaphandre dont work on macos
• started a minicube cluster in default namespace with a hello node simple server that serves the current time
• installed kepler in a separate namespace
• kepler observes the minicube cluster and returns a text dataset in an odd format that can be read manually but not parsed programmatically yet
• the data kepler returns ioncludes container energy in kJ for each timestep
• kepler results are all 0 at the moment, not sure why.
• The Kepler docs are here - they have not proven sufficient to get up and running, and some extra exploration has been required: https://sustainable-computing.io/installation/kepler-helm/
• Scaphandre has been installed too, but values are not available yet.
• kepler and scaphandre are conflicting - they require config values that conflict with one another - it is probably possible to work around but looks complicated to run both on the same machine.

[zanete]

@narekhovhannisyan please summarise what you did and what you found in a bulleted list and we can close

[narekhovhannisyan]

@zanete I first started installing the infrastructure on macOS and found out that Kepler and Scaphandre can't be installed.
Secondly, I've tried to run it on Ubuntu VM which was unsuccessful either.
By the next step, the infra was laid down on a Windows machine. Setting up everything was successful; however, Intel Rapl support was missing on a Windows machine. So, the only use case is the Linux machine.

[zanete]

@narekhovhannisyan please provide a little bit more detail in a step-by-step precisely so that it could be replicated 🙏

[narekhovhannisyan]

Prerequisites: Docker

Steps to install:

1. Minikube and Hello-Node

Download and install Minikube according to this doc: until the point Use the kubectl create command to create a Deployment that manages a Pod. The Pod runs a Container based on the provided Docker image. (included).

By the end of this step we will have minikube installed, and hello-node application deployed.

2. Helm

Download and install Helm
This will help to manage repositories and services in minikube.

3.1 Kepler

Deploy Kepler using Helm Charts. With this documentation I had problems with Prometheus and could not get it running and showing results, however, it's possible to expose Kepler with its specific port and get the outputs by requesting to localhost:<KEPLER-PORT>.

3.2 Scaphandre

Deploy Scaphandre using Helm Chart. Following steps from this documentation, you will be able to install Prometheus and Graphana too for monitoring purposes.

[zanete]

@jmcook1186 FYI ☝