A framework to optimize cloud computing costs, using mixed integer programming.
This library should be used together with optiframe
and pulp
.
optiframe
is the underlying optimization framework. This library provides modules for cloud computing that you can use with optiframe
.
pulp
is used to implement the mixed integer program. You only need to use it if you want to add additional modules or if you want to configure the solver.
pip install optiframe
This library provides multiple modules that you can use in optiframe
for modeling cloud cost optimization problems:
base_module
: This module represents the basic decision of which cloud resources should be deployed on which cloud services. It also adds instance demands and flat (upfront) base costs for cloud services. This module must always be added, all other modules depend on it.performance_module
: A module for performance requirements. It allows you to define performance criteria (such as vCPUs and RAM) and the corresponding demand & supply. Use-based pricing models can also be represented with this module.network_module
: This module provides the means to encode network connections, maximum latency requirements and network traffic costs.multi_cloud_module
: If multiple cloud service providers are considered for the decision, this module can be used. It allows you to assign the cloud services to the providers, specify migration cost and enforce a minimum and maximum number of providers to be used.service_limits_module
: If a cloud service is under very high demand and only a limited number of instances is available for purchase, this module can encode these requirements.
Here is a small example demonstrating how to use this library:
from pulp import LpMinimize
from optiframe import Optimizer, SolutionObjValue
from cloud_resource_matcher.modules.base import BaseData, BaseSolution, base_module
from cloud_resource_matcher.modules.performance import PerformanceData, performance_module
# Specify the data of the problem instance
base_data = BaseData(...)
performance_data = PerformanceData(...)
solution = (
Optimizer("cloud_cost_optimization", sense=LpMinimize)
# Configure which modules you want to use
.add_modules(base_module, performance_module)
# Add the data of the problem instance
.initialize(base_data, performance_data)
# Obtain the optimal solution to the problem
.solve()
)
# Extract the total cost of the solution
cost = solution[SolutionObjValue].objective_value
# Determine which cloud resource should be matched to which cloud service
matching = solution[BaseSolution]
You can also take a look at the examples
folder for more detailed examples.
The test/case_studies
folder also contains examples based on the pricing examples from cloud service providers.
This library uses pulp
under the hood and is therefore agnostic to the solver backend that you can use.
By default, it uses the CBC solver, which is pre-bundled with pulp
.
However, it's not very fast, so you probably want to change it.
You can pass any solver object from pulp
into the .solve(...)
method.
Take a look at this documentation for instructions on how to install and configure the solvers.
Here is a small glossary of terms that are used across this project:
- Cloud resource (CR): Anything you want to deploy on the cloud, such as virtual machines, serverless functions or databases.
- Cloud service (CS): An offer that you can buy in the cloud, for example Google Cloud C3, Amazon S3 or Azure Functions.
- Cloud service provider (CSP): A company offering cloud services. For example Google Cloud, AWS or Microsoft Azure.
- Mixed integer program (MIP): A mathematical description of optimization problems. Solvers can use this to return an optimal solution to the problem. See also mixed integer programming.
- Module: A component that implements one set of decision criteria for the optimization problem. For example, the network module implements functionality to represent network traffic and latencies. By choosing the modules you want to use, you can configure the functionality of the optimizer. If a decision criteria or pricing model you need to use is not implemented yet, you can define your own modules.
- Solver: A program implementing multiple algorithms to solve mixed integer programs to optimality. Examples include CBC (open source), SCIP (open source) and Gurobi (commercial).
We use Poetry as a package manager, so you have to install it to properly run the project locally.
Then you can fork and clone the repository and run poetry install
to install all dependencies.
We use several tools to ensure a consistent level of code quality:
- Run
poetry run pytest
to run the test suit for the whole project. - Run
poetry run mypy .
to check for typing errors. - Run
poetry run black .
to format all Python files. - Run
poetry run ruff .
to lint all Python files.
This project is available under the terms of the MIT license