Fix "average loss" bug in cb_explore.cc

[marco-rossi29]

action and observation->action were one 1-index and 0-index based

Fixes #1811

[marco-rossi29]

@jackgerrits This changes the default behavior, so tests are failing. How do you suggest to proceed?

[JohnLangford]

Update the tests.

[marco-rossi29]

To prove that this was indeed a bug, we can take a look at Test 121.

First example from data file (train-sets/rcv1_raw_cb_small.vw) is:
1:1:0.5 | tuesday year million short compan vehicl ...

Action 1 is played both in the buggy version and in the correct version with p = 0.975.
CMD output of first line in the old version was:

average  since         example        example  current  current  current
loss     last          counter         weight    label  predict features
0.050000 0.050000            1            1.0        1 1:0.975000      280

CMD output of first line in new version is:

average  since         example        example  current  current  current
loss     last          counter         weight    label  predict features
1.950000 1.950000            1            1.0        1 1:0.975000      280

Who is correct?

By definition:
ave. loss = Reward / P_log * P_policy = 1 / 0.5 * 0.975 = 1.95.
In the buggy version ave. loss = 0.05 = 1 / 0.5 * 0.025, where 0.025 was the probability of playing action 2. The missmatch caused by the bug.
QED

[JohnLangford]

Merged, thanks :-)