-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update docs and yaml for baselines #292
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,46 @@ | ||
# Reproduce Baselines | ||
:hourglass_flowing_sand: Coming soon! :hourglass_flowing_sand: | ||
|
||
Reproducing baselines used in this study is very similar to the adaptation of the FM as essentially we are just using the FM weights for adaptation. | ||
|
||
## Randomly initialized | ||
|
||
We provide the YAML configuration to train the random init baseline at `experiments/baselines/supervised_training/supervised_random_init.yaml` | ||
|
||
By default, we configure this for Task 1. You can adapt this for Task 2 and Task 3 by searching for `Note: ` comments in the YAML that outline what must be changed. | ||
|
||
You can start training by running this in the root code folder, | ||
```bash | ||
lighter fit --config_file ./experiments/baselines/supervised_training/supervised_random_init.yaml | ||
``` | ||
|
||
## Transfer learning | ||
We provide the YAML configuration to train the transfer learning baseline at `experiments/baselines/supervised_training/supervised_finetune.yaml` | ||
|
||
This baseline is only used for Task 2 and Task 3 as we use the random init baseline from Task 1 for the transfer. Follow the `Note: ` comments to switch between Task 2 and Task 3 configurations. | ||
|
||
You can start training by running this in the root code folder, | ||
```bash | ||
lighter fit --config_file ./experiments/baselines/supervised_training/supervised_finetune.yaml | ||
``` | ||
|
||
## Med3D / MedicalNet | ||
Original repo: https://github.com/Tencent/MedicalNet | ||
|
||
We have provided re-implementations of Med3D to fit into our YAML workflows at `experiments/baselines/med3d/finetune.yaml`. Again, the `Note: ` comments help adapt for different tasks. | ||
|
||
|
||
You can start training by running this in the root code folder, | ||
```bash | ||
lighter fit --config_file ./experiments/baselines/med3d/finetune.yaml | ||
``` | ||
|
||
## Models Genesis | ||
Original repo: https://github.com/MrGiovanni/ModelsGenesis | ||
|
||
We have provided re-implementations of Models Genesis to fit into our YAML workflows at `experiments/baselines/models_genesis/finetune.yaml`. Again, the `Note: ` comments help adapt for different tasks. | ||
|
||
|
||
You can start training by running this in the root code folder, | ||
```bash | ||
lighter fit --config_file ./experiments/baselines/models_genesis/finetune.yaml | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,7 +6,7 @@ The FM was adapted by either fine-tuning all its weights or by freezing its weig | |
|
||
We provide the YAML configuration for this at `experiments/adaptation/fmcib_finetune.yaml`. | ||
|
||
By default, we configure this for Task 1. You can adapt this for Task 2 and Task 3 by searching for 'Note: ' comments in the YAML that outline what must be changed. Make sure you download the weights for the pre-trained foundation model before attempting to reproduce this training. | ||
By default, we configure this for Task 1. You can adapt this for Task 2 and Task 3 by searching for `Note: ` comments in the YAML that outline what must be changed. Make sure you download the weights for the pre-trained foundation model before attempting to reproduce this training. | ||
|
||
|
||
You can start training by running this in the root code folder, | ||
Comment on lines
6
to
12
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The term "hyper-parameter" is typically spelled as one word: "hyperparameter." Consider updating this for consistency with common usage in machine learning documentation. - hyper-parameter optimization framework
+ hyperparameter optimization framework
The phrase "In order to" can be simplified to "To" for conciseness without losing meaning. - In order to perform the modelling, you can run
+ To perform the modelling, you can run
Consider reducing the number of consecutive spaces before "using - for instance, using `--scoring roc_auc`
+ for instance, using `--scoring roc_auc`
The phrase "to try our the modelling process" seems to contain a typo. It should likely be "to try out the modelling process." - to try our the modelling process
+ to try out the modelling process |
||
|
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -12,10 +12,10 @@ lighter predict --config_file ./experiments/inference/extract_features.yaml | |||||
!!! note | ||||||
While the above pipeline will allow you to extract features, we provide an easier and simpler, recommended API to do this. Please refer to [Quick Start](../getting-started/quick-start.md) or [Cloud Quick Start](../getting-started/cloud-quick-start.md) | ||||||
|
||||||
However, this method might be preferred when features need to be extracted from different models (used as baselines in our study). Follow the `Note:` in the corresponding config file to change model paths. | ||||||
|
||||||
## Running predictions from our supervised models (Finetuned FM/ Baselines) | ||||||
However, this method might be preferred when features need to be extracted from different models (used as baselines in our study). Follow the `Note:` in the corresponding config file to change model paths and use different baselines tested. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To enhance clarity and readability, consider rephrasing "However, this method might be preferred when features need to be extracted from different models" to start with an uppercase letter and simplify the sentence structure. - However, this method might be preferred when features need to be extracted from different models
+ However, this method is useful for extracting features from various models Committable suggestion
Suggested change
|
||||||
|
||||||
## Running predictions from our supervised models (Finetuned FM/ Baselines) | ||||||
|
||||||
To run predictions from our models (both supervised and self-supervised), we provide YAML files that can be run with the lighter interface. These are found in `experiments/inference`, namely `get_predictions.yaml` for getting the predictions. | ||||||
|
||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There's a spelling mistake in "Beofre" which should be corrected to "Before." - Beofre running the predictions config, if you haven't downloaded the `models` folder contraining all our baselines, you can do so using
+ Before running the predictions config, if you haven't downloaded the `models` folder containing all our baselines, you can do so using Committable suggestion
Suggested change
Comment on lines
12
to
21
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The word "contraining" seems to be a typo. Correct it to "containing." - contraining all our baselines
+ containing all our baselines |
||||||
|
@@ -37,3 +37,5 @@ lighter predict --config_file ./experiments/inference/get_predictions.yaml | |||||
``` | ||||||
As with the previous YAMLS, please follow the 'Note:' tags to place appropriate data paths and change relevant parameters. This YAML is to be used if you want to get target predictions from the models. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The term "YAMLS" is not standard and could be confusing. Consider using "YAML files" for clarity. - As with the previous YAMLS, please follow the 'Note:' tags
+ As with the previous YAML files, please follow the 'Note:' tags Committable suggestion
Suggested change
|
||||||
|
||||||
!!! note | ||||||
The predictions can be extracted for different tasks as well as different baselines by following the `Note:` comments. |
This file was deleted.
This file was deleted.
This file was deleted.
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider adding a comma after "FM" and "essentially" for better readability and to improve the flow of the sentence.
Committable suggestion