Implement the finding of true confidence intervals (aka MINOS)

[beniz]

This is the MINOS subroutine implemented by Minuit and used in ROOT.

See page 7 (349) in http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.158.9157&rep=rep1&type=pdf for details.

[beniz]

The original article points to http://www.amazon.fr/Statistical-Methods-Experimental-Physics-Edition/dp/9812705279/ref=sr_1_2?ie=UTF8&qid=1399901929&sr=8-2&keywords=Statistical+Methods+in+Experimental+Physics pages 234-236 as the underlying theory.

This is turns refer to the 'modern' term of the multidimensional extension as the 'profile likelihood'.

From there, there are several implementations, a simple one to begin with appears to be outlined here:
http://www.fdmold.uni-freiburg.de/~araue/pmwiki.php/Projects/ProfileLikelihoodApproach

[beniz]

Commit ec17c45 brings usable minos type confidence intervals computation to the library.

This is still experimental and though unit-tested, it has not been thoroughly tested on the regular function set. In order to try it out at this early stage, for a deviation of 0.1 from the found mininum, and using otherwise default parameters for computing the profile likelihood:

./test_functions -fname fsphere -dim 10 -le

yields:

confidence intervals:
dim 0 in [-0.285108,0.285108]
dim 1 in [-0.288124,0.288124]
dim 2 in [-0.289218,0.289218]
dim 3 in [-0.285644,0.285644]
dim 4 in [-0.287044,0.287044]
dim 5 in [-0.298543,0.298543]
dim 6 in [-0.281206,0.281206]
dim 7 in [-0.297881,0.297881]
dim 8 in [-0.275348,0.275348]
dim 9 in [-0.294037,0.294037]

and

./test_functions -fname elli -dim 10 -le

yields:

confidence intervals:
dim 0 in [-0.281297,0.281297]
dim 1 in [-0.131263,0.131263]
dim 2 in [-0.0621184,0.0621184]
dim 3 in [-0.0288797,0.0288797]
dim 4 in [-0.013169,0.013169]
dim 5 in [-0.00582101,0.00582101]
dim 6 in [-0.00281503,0.00281503]
dim 7 in [-0.00131911,0.00131911]
dim 8 in [-0.000600367,0.000600367]
dim 9 in [-0.000287616,0.000287616]

The library also embeds the ability to compute a 'full' profile likelihood 'curve', that is a set of points such that confidence intervals can later be obtained for various values of the deviation from the original minimum.

At this stage, the confidence intervals returned on rosenbrock and rastrigin cannot be validated and may well be faulty (results vary with values of -le_samplesize).

[beniz]

Last fix yields good results on even rosenbrock, sphere, elli and ellirot in up to 20-D (tested). Will proceed with tests up to 100-D.

[beniz]

Confidence intervals through profile likelihood now more stable across sphere, elli, ellirot, rosenbrock and rastrigin.

The iterative process uses several consecutive optimizations, each of which reuses the minimum from the previous call. However, passing the covariance matrix along does not yield good results as the search often fits an early stopping criteria. For this reason, and for now, the covariance matrix is resetted to the identity upon every new optimization call.

[beniz]

merged into dev branch, will be part of next release.