Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zero metrics for all algorithms, but no error reported #69

Closed
natke opened this issue May 31, 2019 · 2 comments
Closed

Zero metrics for all algorithms, but no error reported #69

natke opened this issue May 31, 2019 · 2 comments

Comments

@natke
Copy link

natke commented May 31, 2019

I ran Model Builder on the attached dataset targeting the multiclass classification task.

All metrics for all algorithms are 0.

I tried running for 100 seconds and 1000 seconds with the same result.

I tried .csv and .tsv format

scenarios.xlsx

The log file from Model Builder is also attached
debug_log.txt

(I created a pipeline manually in ML.NET:

var pipeline = mlContext.Transforms.Conversion.MapValueToKey("Label", "Task")
    .Append(mlContext.Transforms.Text.FeaturizeText("Scenario_Features", "Scenario"))
    .Append(mlContext.Transforms.Concatenate("Features", "Scenario_Features"))
    .AppendCacheCheckpoint(mlContext)
    .Append(mlContext.MulticlassClassification.Trainers.OneVersusAll\
        (mlContext.BinaryClassification.Trainers.AveragedPerceptron()) 
    .Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedTask", "PredictedLabel")));

and that gave micro/macro accuracy of 0.75/0.83 with a few seconds of training)

ModelBuilder version: 16.0.1905.2002
Visual Studio version: 16.0.4

@justinormont
Copy link

This is likely an instance of a cross validation bug which we've seen before.

That bug is being fixed in dotnet/machinelearning#3794.

Feel free to subscribe to notifications on that PR for updates.

@rustd
Copy link
Contributor

rustd commented Jun 3, 2019

Duplicate of dotnet/machinelearning#3794

@rustd rustd marked this as a duplicate of dotnet/machinelearning#3794 Jun 3, 2019
@rustd rustd closed this as completed Jun 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants