Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to output model settings in metric output #39

Open
J535D165 opened this issue Aug 4, 2022 · 0 comments
Open

Add option to output model settings in metric output #39

J535D165 opened this issue Aug 4, 2022 · 0 comments
Labels
help wanted Extra attention is needed

Comments

@J535D165
Copy link
Member

J535D165 commented Aug 4, 2022

It would be nice to extend the asreview metrics command with model settings. Ideally with a flag that is disabled by default.

Example:

asreview metrics my_file.asreview --include-metrics
 "asreviewVersion": "1.0",
    "apiVersion": "1.0",
    "model": {
          ...
    }
    "data": {
        "items": [
            {
                "id": "recall",
                "title": "Recall",
                "value": [
                    [
                        0.1,
                        1.0
                    ],
                    [
                        0.25,
                        1.0
                    ],
                    [
                        0.5,
                        1.0
                    ],
                    [
                        0.75,
                        1.0
                    ],
                    [
                        0.9,
                        1.0
                    ]
                ]
            },
            {
                "id": "wss",
                "title": "Work Saved over Sampling",
                "value": [
                    [
                        0.95,
                        0.8913851624373686
                    ]
                ]
            },
            {
                "id": "erf",
                "title": "Extra Relevant record Found",
                "value": [
                    [
                        0.1,
                        0.9047619047619048
                    ]
                ]
            },
            {
                "id": "atd",
                "title": "Average time to discovery",
                "value": 101.71428571428571
            },
            {
                "id": "td",
                "title": "Time to discovery",
                "value": [
                    [
                        3898,
                        22
                    ],
                    [
                        284,
                        23
                    ],
                    [
                        592,
                        25
                    ],
                    ...
                    [
                        2382,
                        184
                    ],
                    [
                        5479,
                        224
                    ],
                    [
                        3316,
                        575
                    ]
                ]
            }
        ]
    }
}
@J535D165 J535D165 added the help wanted Extra attention is needed label Aug 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant