Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation for evaluate package #543

Merged
merged 31 commits into from
Feb 11, 2024

Conversation

rjavadi
Copy link
Collaborator

@rjavadi rjavadi commented Jan 9, 2024

PR Type (Documentation)

Short Description

Improving docs for evaluate module.

I changed the main theme to provide easier navigation, and add same-page navigation on the right side navigation panel.
I'm going to add new experimental metrics in another PR.

@rjavadi rjavadi changed the title [WIP] Improve documentation for evaluate package Improve documentation for evaluate package Jan 29, 2024
fcogidi and others added 7 commits January 29, 2024 18:25
* Fix capitalization of Python

* Change ValueError to TypeError and update docstrings
* Add ROC metrics to experimental module

* fix typo

* Fix docstrings
* Add AUROC metric to experimental module

* Refactor binary and multiclass ROC functions

* Refactor tests to use a common thresholds list

* Fix mypy error
* Add regression support

* update docstrings

* Ignore `no-any-return` errors
* integrate experimental metrics with other modules

* add average precision metric to experimental metrics package

* fix tutorials

* Add type hints and keyword arguments to metrics classes

* Update nbsphinx version to 0.9.3

* Update nbconvert version to 7.14.2

* Fix type annotations and formatting issues

* Update kernel display name in mortality_prediction.ipynb

* Add guard clause to prevent module execution on import

* Update `torch_distributed.py` with type hints

* Add multiclass and multilabel average precision metrics

* Change jupyter kernel

* Fix type annotations for metric values in ClassificationPlotter

---------

Co-authored-by: Amrit K <amritk@vectorinstitute.ai>
@rjavadi
Copy link
Collaborator Author

rjavadi commented Jan 30, 2024

@amrit110 I added new dependencies and generated a new poetry.lock, what's the best way to resolve its conflicts?

@amrit110 amrit110 added the documentation Improvements or additions to documentation label Jan 30, 2024
@amrit110
Copy link
Member

amrit110 commented Jan 30, 2024

@rjavadi i've resolved the conflict now, had to merge against main and then just fix the pyproject.toml file manually and then do poetry lock --no-update. I've pushed the sphinx-book-theme dependency to be under the docs section. This will make sure that its not part of the cyclops package install but rather a development dependency. I also fixed some of the formatting issues. You can get the errors by running pre-commit run --all-files before a commit.

@amrit110
Copy link
Member

amrit110 commented Jan 30, 2024

I feel the new theme improves navigation, but it breaks a few things because there were custom html files with footer and also the color scheme gets thrown off course. The rest of the PR is good, @fcogidi should review more on how the user guide for the evaluate package reads.

Copy link
Collaborator

@fcogidi fcogidi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's possible, I'd suggest not having any Evaluation under the User Guide section, but as a separate section. Also, when the Evaluation section is clicked, it shows all the available modules and functions. I'd suggest not doing that, but instead, only expanding the drop-down and forcing the user to click on a specific module to see the details about it.

Inside the User Guide section, I'd put things like how to get started - installing cyclops, for example. For the evaluate package, things like how to implement a metric and how to set up the evaluate method would be useful in the User Guide section.

@rjavadi
Copy link
Collaborator Author

rjavadi commented Feb 2, 2024

@fcogidi @amrit110 I'm going to make the changes you mentioned.
I still couldn't find a solution to get rid of that list of items that appear when clicking on Evaluation.

@rjavadi rjavadi force-pushed the improve-documentation-for-evaluate-package branch from 7727e76 to 5fa38ae Compare February 9, 2024 18:59
@amrit110 amrit110 merged commit 1bf5eb2 into main Feb 11, 2024
9 checks passed
@amrit110 amrit110 deleted the improve-documentation-for-evaluate-package branch February 11, 2024 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants