Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a new column #168

Merged
merged 10 commits into from
Jan 27, 2025
Merged

Conversation

Expertium
Copy link
Contributor

I added a column with input features. Ngl, I feel like my idea of using abbreviations isn't that great. But typing all that out would make the table huge, and adding more columns isn't great either, so I guess using a ton of abbreviations is the lesser evil.
@1DWalker idk about the RMSE bins exploit thingy, do we need to specify some input features? I mean, it uses interval lengths and grades...I think?

I added a column with input features. Ngl, I feel like my idea of using abbreviations isn't that great. But typing all that out would make the table huge, and adding more columns isn't great either, so I guess using a ton of abbreviations is the lesser evil.
@1DWalker
Copy link
Contributor

I think RMSE bins exploit can be considered "IL+G"

@Expertium
Copy link
Contributor Author

Expertium commented Jan 26, 2025

Do you think this column should be moved to the left, between "Log Loss" and "Parameters"?
Or maybe do the opposite and move "Parameters" to the far right?

@1DWalker
Copy link
Contributor

I think it's fine as it is.

@L-M-Sherlock
Copy link
Member

Please add them to here as well:

srs-benchmark/evaluate.py

Lines 90 to 128 in 0cad528

models = (
[
(dev_mode_name, None),
("LSTM-short-secs-equalize_test_with_non_secs", 8869),
("GRU-P-short", 297),
("GRU-P", 297),
("FSRS-5-recency", 19),
("FSRS-5-preset", 19),
("FSRS-rs", 19),
("FSRS-5", 19),
("FSRS-5-disable_short_term", 17),
("FSRS-4.5", 17),
("FSRS-5-deck", 19),
("FSRS-5-binary", 15),
("FSRSv4", 17),
("DASH", 9),
("GRU", 39),
("DASH[MCM]", 9),
("DASH-short", 9),
("DASH[ACT-R]", 5),
("FSRSv2", 14),
("FSRS-5-pretrain", 4),
("FSRSv3", 13),
("NN-17", 39),
("FSRS-5-dry-run", 0),
("ACT-R", 5),
("FSRSv1", 7),
("AVG", 0),
("Anki", 7),
("HLR", 3),
("HLR-short", 3),
("SM2-trainable", 6),
("Anki-dry-run", 0),
("SM2-short", 0),
("SM2", 0),
("Ebisu-v2", 0),
("Transformer", 127),
("RMSE-BINS-EXPLOIT", 0),
]

@Expertium
Copy link
Contributor Author

@L-M-Sherlock this PR is ready

Copy link
Member

@L-M-Sherlock L-M-Sherlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@L-M-Sherlock L-M-Sherlock merged commit b0c4f38 into open-spaced-repetition:main Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants