Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent learning pipeline from adding null transform model to the pipeline #154

Merged
merged 3 commits into from
May 15, 2018
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/Microsoft.ML/LearningPipeline.cs
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,9 @@ public PredictionModel<TInput, TOutput> Train<TInput, TOutput>()

if (transformModels.Count > 0)
{
transformModels.Insert(0,lastTransformModel);
if (lastTransformModel != null)
transformModels.Insert(0, lastTransformModel);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

transformModels [](start = 24, length = 15)

would be nice to add test coverage

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


var modelInput = new Transforms.ModelCombiner
{
Models = new ArrayVar<ITransformModel>(transformModels.ToArray())
Expand Down
37 changes: 37 additions & 0 deletions test/Microsoft.ML.Tests/LearningPipelineTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
// See the LICENSE file in the project root for more information.

using Microsoft.ML;
using Microsoft.ML.Runtime.Api;
using Microsoft.ML.TestFramework;
using Microsoft.ML.Transforms;
using System.Linq;
using Xunit;
using Xunit.Abstractions;
Expand Down Expand Up @@ -42,5 +44,40 @@ public void CanAddAndRemoveFromPipeline()
pipeline.Add(new Trainers.StochasticDualCoordinateAscentRegressor());
Assert.Equal(3, pipeline.Count);
}

class InputData
{
[Column(ordinal: "1")]
public string F1;
}

class TransformedData
Copy link
Contributor

@Ivanidzo4ka Ivanidzo4ka May 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

explicit access modifer, same for class above #Closed

{
[ColumnName("F1")]
public float[] TransformedF1;
}

[Fact]
public void TransformOnlyPipeline()
{
const string _dataPath = @"..\..\Data\breast-cancer.txt";
var pipeline = new LearningPipeline();
pipeline.Add(new TextLoader<InputData>(_dataPath, useHeader: false));
pipeline.Add(new CategoricalHashOneHotVectorizer("F1") { HashBits = 10, Seed = 314489979, OutputKind = CategoricalTransformOutputKind.Bag });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this issue only affecting transforms only pipeline?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah

var model = pipeline.Train<InputData, TransformedData>();
var predictionModel = model.Predict(new InputData() { F1 = "5" });

Assert.NotNull(predictionModel);
Assert.NotNull(predictionModel.TransformedF1);
Assert.Equal(1024, predictionModel.TransformedF1.Length);

for (int index = 0; index < 1024; index++)
if (index == 265)
Assert.Equal(1, predictionModel.TransformedF1[index]);
else
Assert.Equal(0, predictionModel.TransformedF1[index]);

predictionModel.TransformedF1 = null;
Copy link
Contributor

@Ivanidzo4ka Ivanidzo4ka May 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

predictionModel.TransformedF1 = null; [](start = 11, length = 38)

out of curiosity, why? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compiler warning that is treated as an error.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

}
}
}