YOTSE is a powerful tool designed to enable computational experiments using any software, with a particular emphasis on running optimization algorithm on HPC clusters with optimal resource allocation.
Originally it was developed for use with NetSquid in the context of the Quantum Internet Alliance. However, the tools in this library are aimed at being as generally applicable as possible and should fit simulation scripts written in any language with minimal requirements on output format. YOTSE offers a set of predefined functions for performing parameter explorations and optimizations in a scalable and efficient manner.
Documentation can be found here.
Note: If you use this package for research purposes, consider checking out the underlying optimization libraries and citing them.
To install this project, simply:
pip install yotse
If you want to contribute to the project, then clone the repository as follows: (Make sure you have the necessary permissions.)
git clone https://github.com/SURFQuantum/yotse.git
Once the repository is cloned, navigate to the project's root directory, with:
cd yotse
If you do not have poetry already installed you need will need to install it first. Then install yotse using:
poetry install
To verify your installation you can now do:
poetry run tests && poetry run examples
YOTSE is versatile, catering to a wide range of software. While it has a special focus on NetSquid, it is flexible enough to accommodate any software you wish to use for your computational experiments.
Our library offers predefined functions for parameter exploration and optimization. We base our tools on the QCG-PilotJob project, a job manager executable both locally and in HPC centers. You can find more information about QCG-PilotJob in their official docs or in the original paper.
The optimization component of YOTSE involves a base class that can be used with any external optimization library. This flexibility allows you to tailor your optimization process to your specific needs and preferences.
Detailed examples of usage are provided in the /examples
directory.
---
title: Workflow
---
flowchart LR
p("`user_pre
(Experiment)`")
a(analysis script)
s(user script)
c{{"`csv output
input params, cost
(Core)`"}}
S[[Best Solution]]
q["`QcgPilotJob
(Executor)`"]
G["`GenericOptimization
(Optimizer)`"]
p --> q -- schedules jobs of --> s --> c --> G -- generates new datapoints for --> q
G --> S
s -.-> a -.-> c
In order to make usage with HPC resources and SLURM as smooth and simple as possible, yotse can generate its own SLURM files ready for execution.
Simply specify your SLURM parameters (such as required nodes or time) and modules to be imported in the SystemSetup
and then execute
python <your_yotse_script.py> --slurm
This will generate a SLURM script for your (default name: slurm.job
) which you can then submit by:
sbatch slurm.job
In order to keep dependencies as clean as possible, we suggest using two virtual environments for SLURM execution:
- A virtual environment that has all dependencies of your own script installed and can execute your owns script without errors. Pass this as
SystemSetup.venv
. - A virtual environment that has
yotse
installed. Pass this asSystemSetup.slurm_venv
such that it is activated before SLURM executes the yotse commands.
---
title: Class diagram
---
flowchart TB
E["Executor"]
c["cost function"]
O["Optimizer"]
G["GenericOptimization"]
D[("database_from_csv")]
Ex["Experiment"]
E --> Ex --> O
E -- generates --> O -- contains --> G -- contains --> D
c -- replaced by --x D
D -- fitness fun in --> G
We appreciate contributions. To contribute:
- Fork the project
- Create your feature branch (
git checkout -b feature/MyNewFeature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin feature/MyNewFeature
) - Execute
make verify
to check if your changes pass the required checks - Open a Pull Request
To run tests on the project, navigate to the project's root directory and:
poetry run tests
If you want to open the coverage report do:
poetry run show-cov
and the coverage report should open in your default browser.
To run the examples, in the root directory execute
poetry run examples
This project is licensed under the terms of the Apache License 2.0. For more details, refer to the LICENSE file in the project root.