Skip to content

Toolkit for evaluating and monitoring AI models in clinical settings

License

Notifications You must be signed in to change notification settings

ethanhkim/cyclops

 
 

Repository files navigation

cyclops Logo


PyPI PyPI - Python Version code checks integration tests docs codecov docker

cyclops is a toolkit for facilitating research and deployment of ML models for healthcare. It provides a few high-level APIs namely:

  • data - Create datasets for training, inference and evaluation. We use the popular 🤗 datasets to efficiently load and slice different modalities of data
  • models - Use common model implementations using scikit-learn and PyTorch
  • tasks - Use common ML task formulations such as binary classification or multi-label classification on tabular, time-series and image data
  • evaluate - Evaluate models on clinical prediction tasks
  • monitor - Detect dataset shift relevant for clinical use cases
  • report - Create model report cards for clinical ML models

cyclops also provides example end-to-end use case implementations on clinical datasets such as

🐣 Getting Started

Installing cyclops using pip

python3 -m pip install pycyclops

cyclops has many optional dependencies that are used for specific functionality. For example, the monai library is used for loading DICOM images to create datasets. Hence, monai can be installed using python3 -m pip install pycyclops[monai]. Specific sets of dependencies are listed below.

Dependency pip extra Notes
xgboost xgboost Allows use of XGBoost model
torch torch Allows use of PyTorch models
torchvision torchvision Allows use of Torchvision library
torchxrayvision torchxrayvision Uses TorchXRayVision library
monai monai Uses MONAI to load and transform images
alibi alibi Uses Alibi for additional explainability functionality
alibi-detect alibi-detect Uses Alibi Detect for dataset shift detection

🧑🏿‍💻 Developing

Using poetry

The development environment can be set up using poetry. Hence, make sure it is installed and then run:

python3 -m poetry install
source $(poetry env info --path)/bin/activate

In order to install dependencies for testing (codestyle, unit tests, integration tests), run:

python3 -m poetry install --with test

API documentation is built using Sphinx and can be locally built by:

python3 -m poetry install --with docs
cd docs
make html SPHINXOPTS="-D nbsphinx_allow_errors=True"

Contributing

Contributing to cyclops is welcomed. See Contributing for guidelines.

If you need to build the documentations locally, make sure to install Pandoc in addition to docs poetry group.

📓 Notebooks

To use jupyter notebooks, the python virtual environment can be installed and used inside an IPython kernel. After activating the virtual environment, run:

python3 -m ipykernel install --user --name <name_of_kernel>

Now, you can navigate to the notebook's Kernel tab and set it as <name_of_kernel>.

🎓 Citation

Reference to cite when you use cyclops in a project or a research paper:

@article {Krishnan2022.12.02.22283021,
	author = {Krishnan, Amrit and Subasri, Vallijah and McKeen, Kaden and Kore, Ali and Ogidi, Franklin and Alinoori, Mahshid and Lalani, Nadim and Dhalla, Azra and Verma, Amol and Razak, Fahad and Pandya, Deval and Dolatabadi, Elham},
	title = {CyclOps: Cyclical development towards operationalizing ML models for health},
	elocation-id = {2022.12.02.22283021},
	year = {2022},
	doi = {10.1101/2022.12.02.22283021},
	publisher = {Cold Spring Harbor Laboratory Press},
	URL = {https://www.medrxiv.org/content/early/2022/12/08/2022.12.02.22283021},
	journal = {medRxiv}
}

About

Toolkit for evaluating and monitoring AI models in clinical settings

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 96.7%
  • JavaScript 2.0%
  • Jinja 1.3%