Recommended neos setup

[alexander-held]

Hi, I am interested to try out neos. What is the currently recommended way to get a working setup?

pypi (mentioned in the readme under the installation section) provides version 0.0.1, while this repository also has a tag for 0.0.2. There have been a number of changes after both these tags, so I tried to install from master. Going through the example in the readme, the model validation fails in pyhf:

Traceback (most recent call last):
  File "test.py", line 74, in <module>
    for i, (network, metrics, epoch_time) in enumerate(train_network(maxN)):
  File "test.py", line 66, in train_network
    state, value, network = update_and_value(i, state, 1.0)
  File "test.py", line 61, in update_and_value
    value, grad = jax.value_and_grad(loss)(net, mu)
  File "test.py", line 43, in loss
    return get_loss(params, test_mu)[0]
  File "/neos/neos/infer.py", line 50, in get_expected_CLs
    m, bonlypars = model_maker([params, hyperparams])
  File "/neos/neos/makers.py", line 346, in nn_model_maker
    m = from_spec(yields)
  File "/neos/neos/makers.py", line 342, in from_spec
    return pyhf.Model(spec)
  File "/usr/local/lib/python3.8/site-packages/pyhf/pdf.py", line 556, in __init__
    utils.validate(self.spec, self.schema, version=self.version)
  File "/usr/local/lib/python3.8/site-packages/pyhf/utils.py", line 51, in validate
    raise InvalidSpecification(err, schema_name)
jax._src.traceback_util.FilteredStackTrace: pyhf.exceptions.InvalidSpecification: Traced<ShapedArray(float64[3])>with<JVPTrace(level=2/1)>
  with primal = Traced<ShapedArray(float64[3])>with<DynamicJaxprTrace(level=0/1)>
       tangent = Traced<ShapedArray(float64[3]):JaxprTrace(level=1/1)> is not of type 'array'.
	Path: channels[0].samples[0].data
	Instance: Traced<ShapedArray(float64[3])>with<JVPTrace(level=2/1)>
  with primal = Traced<ShapedArray(float64[3])>with<DynamicJaxprTrace(level=0/1)>
       tangent = Traced<ShapedArray(float64[3]):JaxprTrace(level=1/1)> Schema: model.json

I also gave the binder linked in the readme a try, some notebooks run but not all (the link points to a notebook that does not exist anymore).

Two minor things I came across while looking through the readme: the http://gradhep.github.io/neos link points to the wrong location, and the demo_kde_pyhf.ipynb notebook linked at the bottom does not exist anymore.

[phinate]

Thanks for this @alexander-held! I think this is a problem with the wrong pyhf version -- were you using the differentiable_jsonpyhf branch?

I wouldn't worry either way for now, because I think there are other problems that need cleaning up on the pyhf side that popped out of nowhere, so I'll try and fix these things at once. Thanks for trying and pointing out the link problems :)

[alexander-held]

I was using the feat/change-default-backend branch specified in the setup.py

neos/setup.py

Line 24 in 68a7e32

requirements.append('pyhf @ git+https://github.com/phinate/pyhf.git@feat/change-default-backend')

when installing from master. Thanks, I'll give it another try when things are more stable again then.