Krateo Composable FinOps Database Handler

This service offers a set of endpoints to connect to the Krateo's CrateDB instance and use notebooks to compute data starting from SQL queries.

This service requires CrateDB to be installed in the Kubernetes cluster.

Summary

1. Overview
2. Architecture
3. API
4. Examples
5. Installation

Overview

This webservice acts as a proxy for all requests to the database, including the possibility of performing computations on the stored data. The structure is: the database requires username/password authentication for the HTTP endpoint. The password is stored in a Kubernetes secret. The RBAC to the secret should allow to filter who can access the webservice-database functionality. The handler calls the database HTTP endpoint to perform queries (both input and output). The result is returned by the endpoint.

Architecture

API

All endpoints must have the basic auth header field compiled with the username and password of the database.

• POST /upload: the webservice receives the data (divided into chunks) and directly uploads it into the specified table in the database
• POST /compute/<compute_name>: calls the specified compute notebook with the POST body data being the parameters required by the given algorithm, encoded in JSON as parameter_name=parameter_value
• POST /compute/<compute_name>/upload: uploads the specified notebook into the database with the name <compute_name>
• GET /compute/list: lists all available compute notebooks

Examples

Upload endpoint:

curl -X POST -u <db-user>:<db-password> http://finops-database-handler.finops:8088/upload?table=<table_name>&type=<cost|resource> -d "<metrics>"

Compute endpoint:

curl -X POST -u <db-user>:<db-password> http://finops-database-handler.finops:8088/compute/cyclic \
    --header "Content-Type: application/json" \
    --data '{"table_name":"testfocus_res"}'

Compute endpoint upload:

curl -X POST -u <db-user>:<db-password> http://finops-database-handler.finops:8088/compute/cyclic/upload --data-binary "@cyclic.py"

For a notebook example, see ./notebook_samples/cyclic.py

Compute endpoint list:

curl -u <db-user>:<db-password> http://finops-database-handler.finops:8088/compute/list

Installation

The webservice can be installed through the HELM chart:

helm repo add krateo https://charts.krateo.io
helm repo update krateo
helm install finops-database-handler krateo/finops-database-handler

You need to configure the environment variables in the HELM chart of the webservice to connect to the database on the HTTP endpoint of CrateDB: CRATE_HOST and CRATE_PORT.