Add new struct called RMSEVariable and necessary functionalities for plotting #91
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #91 +/- ##
==========================================
+ Coverage 96.91% 97.61% +0.70%
==========================================
Files 10 11 +1
Lines 712 922 +210
==========================================
+ Hits 690 900 +210
Misses 22 22 ☔ View full report in Codecov by Sentry. |
Sbozzolo
reviewed
Sep 20, 2024
ph-kev
force-pushed
the
kp/leaderboard
branch
4 times, most recently
from
fad4a5d
to
5d0e4c3
Compare
ph-kev
force-pushed
the
kp/leaderboard
branch
from
5d0e4c3
to
4101f61
Compare
Sbozzolo
reviewed
Sep 20, 2024
This commit adds a struct called `RMSEVariable` which hold all the information necessary for comparing the root mean square errors between different models across different categories for a single variable.
This commit adds the functionality of reading RMSEs values from a CSV file.
This commit adds indexing to the `RMSEVariable`. In other words, the struct behaves like an array. We allow indexing by both integers and strings. We also made the decision to support indexing using only a single string. For instance, `rmse_var["model_name"]` will return the row of RMSEs across the different categories for that model. However, we do not support `rmse_var[1]` because it is unclear whether this should return a row (similar to the string case) or the first value of the entry (similar to if the struct behaves like an array).
This commit adds functionality to add a single or multiple categories, models, or units.
This commit adds functionality to find the best single model, worst single model, and median though `Leaderboard.find_best_single_model`, `Leaderboard.find_worst_single_model`, `Leaderboard.median`. For handling NaNs, all NaNs are converted to positive Inf or negative Inf as appopriate for finding the best single model or worst single model. For finding the median, the NaNs are filtered out.
This commit adds functionality to plot a boxplot for a single `RMSEVariable` and a leaderboard/heatmap for multiple `RMSEVariable`s. For handling NaNs, the box plot filter out NaNs. For the leaderboard, if a NaN is present, the cell corresponding to the NaN will not be filled out.
ph-kev
force-pushed
the
kp/leaderboard
branch
from
4101f61
to
cd24b3a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes Add struct and constructor for storing RMSEs across model names and seasons #88, closes Add
find_best_single_model,find_worst_single_model, andmedianfor struct that store RMSEs #89, closes Addplot_box_plotandplot_leaderboardusing structs that store RMSEs #90This PR adds a new struct called
RMSEVariablefor storing root mean squared errors (RMSEs) over categories (e.g. seasons and annual) and models (e.g. CliMA). The features include forRMSEVariableincludeConstructors
Reading RMSEs from CSV file
Indexing
Adding categories, models, and units
Methods for summary statistics
Plotting box plots and leaderboard
A combined plot of the box plots and leaderboard would like something below.
