This repository contains the implementation of our full paper accepted for the IEEE EMBC 2024 Conference.
If you find our work useful, please consider citing it.
M. Y. Jabarulla, S. Oeltze-Jafra, P. Beerbaum and T. Uden, "MedDoc-Bot: A Chat Tool for Comparative Analysis of Large Language Models in the Context of the Pediatric Hypertension Guideline," 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA, 2024, pp. 1-4, doi: 10.1109/EMBC53108.2024.10781509.
You can access the full article on arXiv.
MedDoc-Bot: A Chat Tool for Comparative Analysis of Large Language Models in the Context of the Pediatric Hypertension Guideline
- The MedDoc-Bot interface CODE allows users to choose from four quantized Language Model Models (LLMs) to chat with multiple PDF documents.The models used for our evaluations are downloaded from huggingface (Link provided below).
- In our clinical use case, we assessed each model's performance by interpreting the hypertension in children and adolescents ESC guidelines PDF document. Source
- The original pediatric hypertension guidelines Link contain text, tables, and figures on twelve pages. We carefully transformed figures and tables into textual representations to enhance interpretation and extraction. This involves providing detailed captions, extracting numerical data, and describing visual features in text Transformed Document For Visual Element Analysis.
- Evaluation involved using a benchmark dataset crafted by a pediatric specialist with four years of experience in pediatric cardiology manually generated twelve questions and corresponding responses by meticulously reviewing the pediatric hypertension guidelines. Dataset.
- Evaluated models' accuracy, chrF, and METEOR score Detailed Results.
A Streamlit-Powered Chat Tool for interpreting Multi-PDF Document using Four Large Language Models.
 |
 |
|---|---|
 |
 |
https://docs.conda.io/en/latest/miniconda.html
conda create -n MedDoc-Bot python=3.11
conda activate MedDoc-Bot
| System | GPU | Command |
|---|---|---|
| Windows | NVIDIA | pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 |
| Windows | CPU only | pip3 install torch torchvision torchaudio |
The up-to-date commands can be found here: https://pytorch.org/get-started/locally/.
For NVIDIA, you also need to install the CUDA runtime libraries:
conda install -y -c "nvidia/label/cuda-12.1.1" cuda-runtime
git clone https://github.com/yaseen28/MedDoc-Bot
cd MedDoc-Bot
pip install -r requirements.txt
(i) Llama-2 {Version: llama-2-13b.Q5_K_S.gguf} Link
(ii) MedAlpaca {Version: medalpaca-13b.Q5_K_S.gguf} Link
(iii) Meditron {Version: meditron-7b.Q5_K_S.gguf} Link
(iv) Mistral {Version: mistral-7b-instruct-v0.2.Q5_K_M.gguf} Link
NOTE!! Please ensure that you rename the model file to match the name listed in the 'Select Model' dropdown in the browser.
conda activate MedDoc-Bot
cd MedDoc-Bot
streamlit run Main_MedDoc-Bot.py
You can now view your Streamlit app in your browser.
Local URL: http://localhost:8501
User
User@123
If you find our work useful, please consider citing it.
@INPROCEEDINGS{10781509, author={Jabarulla, Mohamed Yaseen and Oeltze-Jafra, Steffen and Beerbaum, Philipp and Uden, Theodor}, booktitle={2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)}, title={MedDoc-Bot: A Chat Tool for Comparative Analysis of Large Language Models in the Context of the Pediatric Hypertension Guideline}, year={2024}, volume={}, number={}, pages={1-4}, keywords={Hypertension;Measurement;Analytical models;Visualization;Large language models;Biological system modeling;Portable document format;Meteors;Reliability;Guidelines;Medical language models;Streamlit;Medical Guidelines PDF Chatbot;Clinical care}, doi={10.1109/EMBC53108.2024.10781509}} }