Expose raw predictions in java wrapper (#1118 #1233) #1244
Conversation
|
Sorry, can you resolve conflicts on this? We're dealing with a bit of a mess because I had to back out the concurrent java interface patch. |
|
@JohnLangford Resolved. |
|
@jmorra Can you take a look at this? It's short. |
| jfloatArray j_labels = env->NewFloatArray(num_values); | ||
| for (int i=0 ; i<num_values; i++) { | ||
| jfloat f[] = { vec->l.cs.costs[i].partial_prediction }; | ||
| env->SetFloatArrayRegion(j_labels, i, 1, (float*)f); |
There was a problem hiding this comment.
I may be a little lost here, but how are you sure that the length of f is 1?
There was a problem hiding this comment.
Expected a double here instead of an array because there should be one value. I ran tests to verify array sizes are all 1 in this case. My understanding is there could be cases where array sizes can be >1 (adf?). I can investigate further if we decide to go with the PR.
| * | ||
| * @param example a single vw example string | ||
| * @return Raw prediction | ||
| */ |
There was a problem hiding this comment.
I don't think this code should go here. From what I understand about VW, the action of getting a raw prediction is applicable to a wider variety of cases than just multi class prediction. @JohnLangford can you comment on this. If this is the case then we'd want to put this code higher up in the chain and allow more learners to take advantage of it. If not, then this seems fine. @deaktator what do you think?
|
I'm somewhat uncomfortable about the patch because it's using some prediction information squirreled away in the label rather than the prediction data. This was a hack Hal introduced some time ago which is certainly useful, but since then we've update our prediction types. https://github.com/JohnLangford/vowpal_wabbit/blob/master/vowpalwabbit/example.h#L44 |
|
I wonder if you can use |
|
With the right flags, yes. The action-score representation is also available. @tilayealemu are you using --csoaa or --csoaa_adf? |
|
I'm using --csoaa. My goal is to rank all actions from best to worst. I have tried both scores and probabilities. Neither of them worked (discussed under #1233). Raw predictions were understandable and usable. Others have raised raw predictions as well [1, 2]. But if there is a way to get raw without this PR then I'm all for that. [1] https://groups.yahoo.com/neo/groups/vowpal_wabbit/conversations/topics/5958 |
|
After looking at this for a while now I think the more appropriate change to make is to add the ability for the CSOAA learner to return action scores. On the LDF front there exists a flag called |
|
I think Jon Morra is right here---it's better (and more uniform) to have the system --csoaa have the same return types as --csoaa_ldf. This implies some c++ coding. Are you up for that? |
|
@JohnLangford this is out of my area of expertise. I've been struggling with it for a number of hours now and I'm not really sure how to do this. Is there any chance you could take a stab at it? |
|
Saw the c++ code too, and it was beyond me. Happy to help if we decide to update any of the java code though. I know it better now. |
|
I'll look into it, but there are some higher priorities at the moment
(build is breaking...).
…-John
On Wed, Jun 28, 2017 at 8:02 AM, Tilaye Yismaw Alemu < ***@***.***> wrote:
Saw the c++ code too, and it was beyond me. Happy to help if we decide to
update any of the java code though. I know it better now.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1244 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAE25vKSe7Zwi46UFk-tcSXugXFe2q32ks5sIkDFgaJpZM4NbPmc>
.
|
|
@JohnLangford I might need this feature pretty soon, I was wondering if you had any ETA on when you might be able to look at this. |
|
I'm working towards a release before ICML. Hopefully something addressing this can go in. |
|
I've decided to not deal with this right now, although a pull request remains welcome. |
No description provided.