This repository contains the necessary setup tools for working in CloudLab. The intention is to standardize and speed-up the process of setting up the cloudlab environment for various research projects. Though it is extensively used on cloudlab, it can be used to setup any linux machine. Use it as a standalone setup tool for your experiments or import it as a submodule in your repository.
This repository contains the setup scripts for the following tools:
- Generic - contains generic setup tools like Go, docker, etc.
- eBPF - contains setup tools for eBPF.
- DPDK - contains setup tools for DPDK.
- OpenNetVM - contains setup tools for OpenNetVM.
- Kubernetes - contains setup tools for Kubernetes.
- Create an account on CloudLab.
- Create an experiment using a profile of your choice. This repository is tested with Ubuntu 22.04 nodes, so make sure to select the appropriate profile. If you don't know which profile to select use the following:
- Profile: small-lan
- Physical Node Type: c220g2/c220g5
- OS Image: Ubuntu 22.04
- Wait for the experiment to be ready. Once the experiment is ready, cloudlab will show the status as "ready".
- Copy the public IP address of the node (ex:
c220g2-010606.wisc.cloudlab.us
,c220g2-010608.wisc.cloudlab.us
, etc) and paste it to the.cloudlab/cloudlab_config.mk
in the formatNODE_NAME=<public_ip>
. Example:
# Node 1
NODE_0=c220g2-010606.wisc.cloudlab.us
# Node 1
NODE_1=c220g2-010608.wisc.cloudlab.us
# Node 2
NODE_2=c220g2-010604.wisc.cloudlab.us
- Update the
CLOUDLAB_USERNAME
in the.cloudlab/cloudlab_config.mk
file with your CloudLab username. - Update the
SSH_KEY_PATH
in the.cloudlab/cloudlab_config.mk
file with the path to your SSH private key.
To rsync the repository code to the cloudlab node, run the following command:
make cl-sync-code NODE={NODE_ID}
This will sync the code to the ~/src directory by default. To specify the destination directory, run the following command:
make cl-sync-code NODE={NODE_ID} REMOTE_DIR={DEST_DIR}
To ssh into the cloudlab node, run the following command:
make cl-ssh-host NODE={NODE_ID}
To scp files from the cloudlab node, run the following command:
make cl-scp-from-host SCP_SRC=<path-to-remote-source> SCP_DEST=<path-to-local-dest> NODE=<node-id>
To rsync from the cloudlab node to the local machine, run the following command:
make cl-sync-from-host NODE=NODE_0
This copies the code from ~/src/<project>
to the local machine. To specify the source directory, run the following command:
make cl-sync-from-host NODE=NODE_0 REMOTE_DIR=<remote-dir> REMOTE_SUBDIR=<remote-subdir>
To run a command on the cloudlab node, run the following command:
make cl-run-cmd NODE=<node-id> COMMAND=<command>
For building your own projects with the help of CloudLab, it is recommended to add this repository as a submodule to your project. This will allow you to keep the setup tools in sync with the latest changes. Here are the steps to add this repository as a submodule:
- Create a directory in your project to store this repository as a submodule. For example, create a directory called
setup
in your project.
mkdir setup
- In the directory, add this repository as a submodule.
cd setup
git submodule add https://github.com/rutu-sh/cloudlab-tools.git
- Initialize the submodule.
git submodule update --init --recursive
- In the root Makefile of your project, add the following code to the start of the file.
CL_DIR=${CURDIR}/.cloudlab
TOOLS_SRC_DIR=${CURDIR}/setup/cloudlab-tools
include setup/cloudlab-tools/cloudlab_tools.mk
- In the root of your project, run the following command to setup cloudlab configurations.
make cl-setup
This will create a .cloudlab
directory in the root of your project with a cloudlab_config.mk
file inside the directory. Add this directory to your .gitignore
file.
In the cloudlab_config.mk
add the values for SSH_KEY_PATH
and CLOUDLAB_USERNAME
. Then add node-specific configurations as:
NODE_NAME=<public_ip>