-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adhere to scikit-learn estimator interface #94
Comments
One addendum to the above, which is related to the use of the models in pipelines: with tf.device('/device:GPU:2'):
# call fit here and will this even work correctly with our current code? |
I'll take a stab at this. I don't have much experience with sklearn though, so it'll probably take a while to understand the desired API. |
I have familiarized myself with the scikit-learn architecture. I think the best way forward is to create a test like this one for all our estimators and then gradually make it pass for each one. |
Continued in #116. |
While working on #116, I noticed a pretty fundamental incompatibility between our API and scikit-learn assumes X and Y are numpy arrays of dtype float64 / int64 and shape ( We could of course try to flatten our data. The inputs could pretty straightforwardly be concatenated together, the output could be considered as classification with each possible ranking being its own class. Thats not particularly elegant however and actually loses information in the case of the output. The other option is to just adhere to the sklearn API as much as possible, but ignore their data shape assumptions. I do not know how much functionality that will break. |
I noticed one problem with the data shape: I am inclined to ignore the shape requirement. If need be, we can write appropriate transformers, which can transform the data in a pipeline. |
Is there anything still missing for this issue, or can we close it? |
Our scikit-learn API conformance is not perfect. Support for I think the first four points of this issue are addressed. The In summary: The most important parts should be covered and I expect that we will get better conformance by using |
Then I would say we can close this particular issue, since the issues with get_params can be worked around and we are going to switch to PyTorch soon anyway. |
Rationale
Most of the learners implemented in cs-ranking already implement an interface similar to the one described in https://scikit-learn.org/stable/developers/develop.html#rolling-your-own-estimator,
i.e., we usually have a
fit
andpredict
method implemented.For users to be able to use all learners effortlessly in a scikit-learn pipeline.Pipeline or to apply model_selection.GridSearchCV, we should make sure that all additional requirements are also fulfilled.
To do
get_params
andset_params
to set parameters. This is important, sinceGridSearchCV
or BayesSearchCV callset_params
for hyperparameter optimization.sklearn.base.BaseEstimator
implements basic versions of these. The current way we handle hyperparameters should be deprecated.__init__
, but rather infit
itself.set_params
is supposed to do exactly the same thing as__init__
with respect to parameters._
.clone
methods for each learner.Most of these changes are independent of each other and could be done using separate branches.
The text was updated successfully, but these errors were encountered: