CB-Tumblebug (CB-TB for short) is a system for managing multi-cloud infrastructure consisting of resources from multiple cloud service providers. (Cloud-Barista)
- Overview, Features, Architecture
- Supported Cloud Providers and Resource Types
- This is for reference only, and we do not guarantee its functionality. Regular updates are made.
- The support for Kubernetes is currently mostly a WIP, and even when available, it offers only a limited set of features.
Note: Ongoing Development of CB-Tumblebug
CB-TB has not reached version 1.0 yet. We welcome any new suggestions, issues, opinions, and contributors!
Please note that the functionalities of Cloud-Barista are not yet stable or secure.
Be cautious if you plan to use the current release in a production environment.
If you encounter any difficulties using Cloud-Barista,
please let us know by opening an issue or joining the Cloud-Barista Slack.
Note: Localization and Globalization of CB-Tumblebug
As an open-source project initiated by Korean members,
we aim to encourage participation from Korean contributors during the initial stages of this project.
Therefore, the CB-TB repository will accept the use of the Korean language in its early stages.
However, we hope this project will thrive regardless of contributors' countries in the long run.
To facilitate this, the maintainers recommend using English at least for
the titles of Issues, Pull Requests, and Commits, while accommodating local languages in the contents.
- Deploy a Multi-Cloud Infra with GPUs and Enjoy muiltple LLMs in parallel (YouTube)
- LLM-related scripts
- Linux (recommend:
Ubuntu 22.04) - Docker and Docker Compose
- Golang (recommend:
v1.23.0) to build the source
Open source packages used in this project
-
Clone the CB-Tumblebug repository:
git clone https://github.com/cloud-barista/cb-tumblebug.git $HOME/go/src/github.com/cloud-barista/cb-tumblebug cd ~/go/src/github.com/cloud-barista/cb-tumblebug
Optionally, you can register aliases for the CB-Tumblebug directory to simplify navigation:
echo "alias cdtb='cd $HOME/go/src/github.com/cloud-barista/cb-tumblebug'" >> ~/.bashrc echo "alias cdtbsrc='cd $HOME/go/src/github.com/cloud-barista/cb-tumblebug/src'" >> ~/.bashrc echo "alias cdtbtest='cd $HOME/go/src/github.com/cloud-barista/cb-tumblebug/src/testclient/scripts'" >> ~/.bashrc source ~/.bashrc
-
Check Docker Compose Installation:
Ensure that Docker Engine and Docker Compose are installed on your system. If not, you can use the following script to install them (note: this script is not intended for production environments):
# download and install docker with docker compose curl -sSL get.docker.com | sh # optional: add user to docker groupd sudo groupadd docker sudo usermod -aG docker ${USER} newgrp docker # test the docker works docker run hello-world
-
Start All Components Using Docker Compose:
To run all components, use the following command:
cd ~/go/src/github.com/cloud-barista/cb-tumblebug sudo docker compose up
This command will start all components as defined in the preconfigured docker-compose.yaml file. For configuration customization, please refer to the guide.
The following components will be started:
- ETCD: CB-Tumblebug KeyValue DB
- CB-Spider: a Cloud API controller
- CB-MapUI: a simple Map-based GUI web server
- CB-Tumblebug: the system with API server
After running the command, you should see output similar to the following:
Now, the CB-Tumblebug API server is accessible at: http://localhost:1323/tumblebug/api Additionally, CB-MapUI is accessible at: http://localhost:1324
Note: Before using CB-Tumblebug, you need to initialize it.
To provisioning multi-cloud infrastructures with CB-TB, it is necessary to register the connection information (credentials) for clouds, as well as commonly used images and specifications.
-
Create
credentials.yamlfile and input your cloud credentials-
Overview
credentials.yamlis a file that includes multiple credentials to use API of Clouds supported by CB-TB (AWS, GCP, AZURE, ALIBABA, etc.)- It should be located in the
~/.cloud-barista/directory and securely managed. - Refer to the
template.credentials.yamlfor the template
-
Create
credentials.yamlthe fileAutomatically generate the
credentials.yamlfile in the~/.cloud-barista/directory using the CB-TB scriptcd ~/go/src/github.com/cloud-barista/cb-tumblebug ./init/genCredential.sh
-
Input credential data
Put credential data to
~/.cloud-barista/credentials.yaml(Reference: How to obtain a credential for each CSP)### Cloud credentials for credential holders (default: admin) credentialholder: admin: alibaba: # ClientId(ClientId): client ID of the EIAM application # Example: app_mkv7rgt4d7i4u7zqtzev2mxxxx ClientId: # ClientSecret(ClientSecret): client secret of the EIAM application # Example: CSEHDcHcrUKHw1CuxkJEHPveWRXBGqVqRsxxxx ClientSecret: aws: # ClientId(aws_access_key_id) # ex: AKIASSSSSSSSSSS56DJH ClientId: # ClientSecret(aws_secret_access_key) # ex: jrcy9y0Psejjfeosifj3/yxYcgadklwihjdljMIQ0 ClientSecret: ...
-
-
Encrypt
credentials.yamlintocredentials.yaml.encTo protect sensitive information,
credentials.yamlis not used directly. Instead, it must be encrypted usingencCredential.sh. The encrypted filecredentials.yaml.encis then used byinit.py. This approach ensures that sensitive credentials are not stored in plain text.- Encrypting Credentials
init/encCredential.sh
If you need to update your credentials, decrypt the encrypted file using
decCredential.sh, make the necessary changes tocredentials.yaml, and then re-encrypt it.- Decrypting Credentials
init/decCredential.sh
- Encrypting Credentials
-
(INIT) Register all multi-cloud connection information and common resources
-
How to register
Refer to README.md for init.py, and execute the
init.pyscript. (enter 'y' for confirmation prompts)cd ~/go/src/github.com/cloud-barista/cb-tumblebug ./init/init.sh
-
The credentials in
~/.cloud-barista/credentials.yaml.enc(encrypted file from thecredentials.yaml) will be automatically registered (all CSP and region information recorded incloudinfo.yamlwill be automatically registered in the system)- Note: You can check the latest regions and zones of CSP using
update-cloudinfo.pyand review the file for updates. (contributions to updates are welcome)
- Note: You can check the latest regions and zones of CSP using
-
Common images and specifications recorded in the
cloudimage.csvandcloudspec.csvfiles in theassetsdirectory will be automatically registered. -
init.pywill apply the hybrid encryption for secure transmission of credentials- Retrieve RSA Public Key: Use the
/credential/publicKeyAPI to get the public key. - Encrypt Credentials: Encrypt credentials with a randomly generated
AESkey, then encrypt theAESkey with theRSA public key. - Transmit Encrypted Data: Send
the encrypted credentialsandAES keyto the server. The server decrypts the AES key and uses it to decrypt the credentials.
This method ensures your credentials are securely transmitted and protected during registration. See init.py for a Python implementation. In detail, check out Secure Credential Registration Guide (How to use the credential APIs)
- Retrieve RSA Public Key: Use the
-
-
-
Shutting down CB-TB and related components
-
Stop all containers by
ctrl+cor type the commandsudo docker compose stop/sudo docker compose down(When a shutdown event occurs to CB-TB, the system will be shutting down gracefully: API requests that can be processed within 10 seconds will be completed)
-
In case of cleanup is needed due to internal system errors
- Check and delete resources created through CB-TB
- Delete CB-TB & CB-Spider metadata using the provided script
cd ~/go/src/github.com/cloud-barista/cb-tumblebug ./init/cleanDB.sh
-
-
Upgrading the CB-TB & CB-Spider versions
The following cleanup steps are unnecessary if you clearly understand the impact of the upgrade
- Check and delete resources created through CB-TB
- Delete CB-TB & CB-Spider metadata
cd ~/go/src/github.com/cloud-barista/cb-tumblebug ./init/cleanDB.sh
- Restart with the upgraded version
- Using CB-TB MapUI (recommended)
- Using CB-TB REST API (recommended)
- Using CB-TB Test Scripts
- With CB-MapUI, you can create, view, and control Mutli-Cloud infra.
- CB-MapUI is a project to visualize the deployment of MCI in a map GUI.
- CB-MapUI also run with CB-Tumblebug by default (edit
dockercompose.yamlto disable)- If you run the CB-MapUI container using the CB-TB script, excute
cd ~/go/src/github.com/cloud-barista/cb-tumblebug ./scripts/runMapUI.sh
- If you run the CB-MapUI container using the CB-TB script, excute
- Access via web browser at http://{HostIP}:1324
-
Access to REST API dashboard
-
http://[IP]:1323/tumblebug/api
- Upsteam online API document:
- Upsteam online API document:
-
REST API AUTH
CB-TB API is encoded with
basic access authenticationby default. (not fully secured yet!)You need to encode the
UsernameandPasswordentered during server startup in Base64 and include it in the API header.
-
-
Using individual APIs
- Create resources required for VM provisioning by using Resource(multi-cloud infrastructure resources) management APIs
- Create, view, control, execute remote commands, shut down, and delete MCI using the MCI(multi-cloud infrastructure service) management APIs
- CB-TB optimal and dynamic provisioning
src/testclient/scripts/ provides Bash shell-based scripts that simplify and automate the MCI (MC-Infra) provisioning procedures, which require complex steps.
[Note] Details
- Step 1: Setup Test Environment
- Step 2: Integrated Tests
- Go to
src/testclient/scripts/ - Configure
conf.env- Provide basic test information such as CB-Spider and CB-TB server endpoints, cloud regions, test image names, test spec names, etc.
- Much information for various cloud types has already been investigated and input, so it can be used without modification. (However, check for charges based on the specified spec)
- How to modify test VM image:
IMAGE_NAME[$IX,$IY]=ami-061eb2b23f9f8839c - How to modify test VM spec:
SPEC_NAME[$IX,$IY]=m4.4xlarge
- How to modify test VM image:
- Configure
testSet.env- Set the cloud and region configurations to be used for MCI provisioning in a file (you can change the existing
testSet.envor copy and use it) - Specify the types of CSPs to combine
- Change the number in NumCSP= to specify the total number of CSPs to combine
- Specify the types of CSPs to combine by rearranging the lines in L15-L24 (use up to the number specified in NumCSP)
- Example: To combine aws and alibaba, change NumCSP=2 and rearrange
IndexAWS=$((++IX)),IndexAlibaba=$((++IX))
- Specify the regions of the CSPs to combine
- Go to each CSP setting item
# AWS (Total: 21 Regions) - Specify the number of regions to configure in
NumRegion[$IndexAWS]=2(in the example, it is set to 2) - Set the desired regions by rearranging the lines of the region list (if
NumRegion[$IndexAWS]=2, the top 2 listed regions will be selected)
- Go to each CSP setting item
- Be aware!
- Be aware that creating VMs on public CSPs such as AWS, GCP, Azure, etc. may incur charges.
- With the default setting of
testSet.env, TestClouds (TestCloud01,TestCloud02,TestCloud03) will be used to create mock VMs. TestCloud01,TestCloud02,TestCloud03are not real CSPs. They are used for testing purposes (do not support SSH into VM).- Anyway, please be aware of cloud usage costs when using public CSPs.
- Set the cloud and region configurations to be used for MCI provisioning in a file (you can change the existing
-
You can test the entire process at once by executing
create-all.shandclean-all.shincluded insrc/testclient/scripts/sequentialFullTest/βββ sequentialFullTest # Automatic testing from cloud information registration to NS creation, Resource creation, and MCI creation βββ check-test-config.sh # Check the multi-cloud infrastructure configuration specified in the current testSet βββ create-all.sh # Automatic testing from cloud information registration to NS creation, Resource creation, and MCI creation βββ gen-sshKey.sh # Generate SSH key files to access MCI βββ command-mci.sh # Execute remote commands on the created MCI (multiple VMs) βββ deploy-nginx-mci.sh # Automatically deploy Nginx on the created MCI (multiple VMs) βββ create-mci-for-df.sh # Create MCI for hosting CB-Dragonfly βββ deploy-dragonfly-docker.sh # Automatically deploy CB-Dragonfly on MCI and set up the environment βββ clean-all.sh # Delete all objects in reverse order of creation βββ create-k8scluster-only.sh # Create a K8s cluster for the multi-cloud infrastructure specified in the testSet βββ get-k8scluster.sh # Get K8s cluster information for the multi-cloud infrastructure specified in the testSet βββ clean-k8scluster-only.sh # Delete the K8s cluster for the multi-cloud infrastructure specified in the testSet βββ force-clean-k8scluster-only.sh # Force delete the K8s cluster for the multi-cloud infrastructure specified in the testSet if deletion fails βββ add-k8snodegroup.sh # Add a new K8s node group to the created K8s cluster βββ remove-k8snodegroup.sh # Delete the newly created K8s node group in the K8s cluster βββ set-k8snodegroup-autoscaling.sh # Change the autoscaling setting of the created K8s node group to off βββ change-k8snodegroup-autoscalesize.sh # Change the autoscale size of the created K8s node group βββ deploy-weavescope-to-k8scluster.sh # Deploy weavescope to the created K8s cluster βββ executionStatus # Logs of the tests performed (information is added when testAll is executed and removed when cleanAll is executed. You can check the ongoing tasks)
-
MCI Creation Test
-
./create-all.sh -n shson -f ../testSetCustom.env# Create MCI with the cloud combination configured in ../testSetCustom.env -
Automatically proceed with the process to check the MCI creation configuration specified in
../testSetCustom.env -
Example of execution result
Table: All VMs in the MCI : cb-shson ID Status PublicIP PrivateIP CloudType CloudRegion CreatedTime -- ------ -------- --------- --------- ----------- ----------- aws-ap-southeast-1-0 Running xx.250.xx.73 192.168.2.180 aws ap-southeast-1 2021-09-17 14:59:30 aws-ca-central-1-0 Running x.97.xx.230 192.168.4.98 aws ca-central-1 2021-09-17 14:59:58 gcp-asia-east1-0 Running xx.229.xxx.26 192.168.3.2 gcp asia-east1 2021-09-17 14:59:42 [DATE: 17/09/2021 15:00:00] [ElapsedTime: 49s (0m:49s)] [Command: ./create-mci-only.sh all 1 shson ../testSetCustom.env 1] [Executed Command List] [Resource:aws-ap-southeast-1(28s)] create-resource-ns-cloud.sh (Resource) aws 1 shson ../testSetCustom.env [Resource:aws-ca-central-1(34s)] create-resource-ns-cloud.sh (Resource) aws 2 shson ../testSetCustom.env [Resource:gcp-asia-east1(93s)] create-resource-ns-cloud.sh (Resource) gcp 1 shson ../testSetCustom.env [MCI:cb-shsonvm4(19s+More)] create-mci-only.sh (MCI) all 1 shson ../testSetCustom.env [DATE: 17/09/2021 15:00:00] [ElapsedTime: 149s (2m:29s)] [Command: ./create-all.sh -n shson -f ../testSetCustom.env -x 1]
-
-
MCI Removal Test (Use the input parameters used in creation for deletion)
./clean-all.sh -n shson -f ../testSetCustom.env# Perform removal of created resources according to../testSetCustom.env- Be aware!
- If you created MCI (VMs) for testing in public clouds, the VMs may incur charges.
- You need to terminate MCI by using
clean-allto avoid unexpected billing. - Anyway, please be aware of cloud usage costs when using public CSPs.
-
Generate MCI SSH access keys and access each VM
-
./gen-sshKey.sh -n shson -f ../testSetCustom.env# Return access keys for all VMs configured in MCI -
Example of execution result
... [GENERATED PRIVATE KEY (PEM, PPK)] [MCI INFO: mc-shson] [VMIP]: 13.212.254.59 [MCIID]: mc-shson [VMID]: aws-ap-southeast-1-0 ./sshkey-tmp/aws-ap-southeast-1-shson.pem ./sshkey-tmp/aws-ap-southeast-1-shson.ppk ... [SSH COMMAND EXAMPLE] [VMIP]: 13.212.254.59 [MCIID]: mc-shson [VMID]: aws-ap-southeast-1-0 ssh -i ./sshkey-tmp/aws-ap-southeast-1-shson.pem cb-user@13.212.254.59 -o StrictHostKeyChecking=no ... [VMIP]: 35.182.30.37 [MCIID]: mc-shson [VMID]: aws-ca-central-1-0 ssh -i ./sshkey-tmp/aws-ca-central-1-shson.pem cb-user@35.182.30.37 -o StrictHostKeyChecking=no
-
-
Verify MCI via SSH remote command execution
./command-mci.sh -n shson -f ../testSetCustom.env# Execute IP and hostname retrieval for all VMs in MCI
-
K8s Cluster Test (WIP: Stability work in progress for each CSP)
./create-resource-ns-cloud.sh -n tb -f ../testSet.env` # Create Resource required for K8s cluster creation ./create-k8scluster-only.sh -n tb -f ../testSet.env -x 1 -z 1` # Create K8s cluster (-x maximum number of nodes, -z additional name for K8s node group and K8s cluster) ./get-k8scluster.sh -n tb -f ../testSet.env -z 1` # Get K8s cluster information ./add-k8snodegroup.sh -n tb -f ../testSet.env -x 1 -z 1` # Add a new K8s node group to the K8s cluster ./change-k8snodegroup-autoscalesize.sh -n tb -f ../testSet.env -x 1 -z 1` # Change the autoscale size of the specified K8s node group ./deploy-weavescope-to-k8scluster.sh -n tb -f ../testSet.env -y n` # Deploy weavescope to the created K8s cluster ./set-k8snodegroup-autoscaling.sh -n tb -f ../testSet.env -z 1` # Change the autoscaling setting of the new K8s node group to off ./remove-k8snodegroup.sh -n tb -f ../testSet.env -z 1` # Delete the newly created K8s node group ./clean-k8scluster-only.sh -n tb -f ../testSet.env -z 1` # Delete the created K8s cluster ./force-clean-k8scluster-only.sh -n tb -f ../testSet.env -z 1` # Force delete the created K8s cluster if deletion fails ./clean-resource-ns-cloud.h -n tb -f ../testSet.env` # Delete the created Resource
-
Setup required tools
-
Install: git, gcc, make
sudo apt update sudo apt install make gcc git
-
Install: Golang
-
Check https://golang.org/dl/ and setup Go
-
Download
wget https://go.dev/dl/go1.23.0.linux-amd64.tar.gz; sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.23.0.linux-amd64.tar.gz
-
Setup environment
echo 'export PATH=$PATH:/usr/local/go/bin:$HOME/go/bin' >> ~/.bashrc echo 'export GOPATH=$HOME/go' >> ~/.bashrc
source ~/.bashrc echo $GOPATH go env go version
-
-
-
-
Run Docker Compose with the build option
To build the current CB-Tumblebug source code into a container image and run it along with the other containers, use the following command:
cd ~/go/src/github.com/cloud-barista/cb-tumblebug sudo DOCKER_BUILDKIT=1 docker compose up --build
This command will automatically build the CB-Tumblebug from the local source code and start it within a Docker container, along with any other necessary services as defined in the
docker-compose.ymlfile.DOCKER_BUILDKIT=1setting is used to speed up the build by using the go build cache technique.
-
Build the Golang source code using the Makefile
cd ~/go/src/github.com/cloud-barista/cb-tumblebug/src make
All dependencies will be downloaded automatically by Go.
The initial build will take some time, but subsequent builds will be faster by the Go build cache.
Note To update the Swagger API documentation, run
make swag- API documentation file will be generated at
cb-tumblebug/src/api/rest/docs/swagger.yaml - API documentation can be viewed in a web browser at http://localhost:1323/tumblebug/api (provided when CB-TB is running)
- Detailed information on how to update the API
- API documentation file will be generated at
-
Set environment variables required to run CB-TB (in another tab)
- Check and configure the contents of
cb-tumblebug/conf/setup.env(CB-TB environment variables, modify as needed)- Apply the environment variables to the system
cd ~/go/src/github.com/cloud-barista/cb-tumblebug source conf/setup.env
- (Optional) Automatically set the TB_SELF_ENDPOINT environment variable (an externally accessible address) using a script if needed
- This is necessary if you want to access and control the Swagger API Dashboard from outside when CB-TB is running
cd ~/go/src/github.com/cloud-barista/cb-tumblebug source ./scripts/setPublicIP.sh
- Apply the environment variables to the system
- Check and configure the contents of
-
Execute the built cb-tumblebug binary by using
make runcd ~/go/src/github.com/cloud-barista/cb-tumblebug/src make run
CB-TB welcomes improvements from both new and experienced contributors!
Check out CONTRIBUTING.
Thanks goes to these wonderful people (emoji key):
![@allcontributors[bot]](https://avatars.githubusercontent.com/in/23186?s=64&v=4){kind=small}
![@dependabot[bot]](https://avatars.githubusercontent.com/in/29110?s=64&v=4){kind=small}
![@github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=64&v=4){kind=small}
