Issue 159 new evaluation #160
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
k15z
reviewed
Jun 26, 2020
EVALUATION.md
Outdated
| # SDV Evaluation | ||
|
|
||
| After using SDV to model your database and generate a synthetic version of it you | ||
| might want to evaluate how similar the syntehtic data is to your real data. |
EVALUATION.md
Outdated
| might want to evaluate how similar the syntehtic data is to your real data. | ||
|
|
||
| SDV has an evaluation module with a simple function that allows you to compare | ||
| the syntehtic data to your real data using [SDMetrics](https://github.com/sdv-dev/SDMetrics) and |
| return synth, real, metadata | ||
|
|
||
|
|
||
| def evaluate(synth, real=None, metadata=None, root_path=None, table_name=None, get_report=False): |
There was a problem hiding this comment.
I think this function signature needs to be updated, real/metadata should be required?
There was a problem hiding this comment.
No, the rationale here is that the only thing truly required is the synthetic data.
Then, you can either pass real data without metadata (and we build the metadata on the fly), or metadata without real data (then we load the real data from the metadata) or both.
This logic is embeded in the _validate_arguments method, which tries to build whatever is necessary to evaluate from as little input from the user as possible.
JonathanDZiegler
pushed a commit
to JonathanDZiegler/SDV
that referenced
this pull request
Feb 7, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolve #159
Replace evaluation subpackage with new evaluation module that uses SDMetrics.
Also add new
EVALUATION.mdmodule with usage instructionsAlso update the Makefile, add readme and tutorials tests and add github actions.