Implement backend='auto' and make it the default

[jcmgray]

Description

This implements backend='auto' for contractions and makes it the default. This works currently by inferring the module from the class of the first array supplied. A slight detail is that sometimes the module of an array does not supply tensordot etc. (maybe it has subclassed just numpy.ndarray for example) and in this case 'numpy' is still used.

Hopefully this kind of thing will eventually be made redundant by __array_function__ (#46) but for the moment it makes it a bit more convenient to work with other array libraries.

Status

• Ready to go

[codecov-io]

Codecov Report

Merging #81 into master will decrease coverage by 0.34%.
The diff coverage is 80%.

[dgasmith]

I like the idea of this, but worried that this could potentially break code. We seem to be in quite a few projects at the moment and could change behavior.

@fritzo Thoughts here?

[dgasmith]

What if someone has subclassed a NumPy array or similar or uses an unknown backend? We should fall back to NumPy here as well.

[jcmgray]

So I think the behavior you describe is what happens here. For example, in my library quimb I have a subclass of ndarray so the initial inferred backend is quimb. However this check for quimb.tensordot (which is cached) means 'numpy' is used still since unlike dask, tensorflow etc. quimb does not provide tensordot.

[jcmgray]

I like the idea of this, but worried that this could potentially break code. We seem to be in quite a few projects at the moment and could change behavior.

Yeah I definitely appreciate this might be a significant change and would be good to get anybody's feedback! I think for almost all use cases there should be no change in behaviour however.

The only really 'breaking' edge case I can think of, is if someone is using non-numpy arrays from a library that does provide tensordot but also is also for some reason relying by default on numpy.tensordot being the backend function. In this case, the libraries tensordot will now be used instead.

[fritzo]

IIUC this would not break any Pyro code since we always specify backend, and actually we usually specify a custom backend like pyro.ops.einsum.torch_log.

[jcmgray]

IIUC this would not break any Pyro code since we always specify backend, and actually we usually specify a custom backend like pyro.ops.einsum.torch_log.

Yes no change for explicit backend - which I imagine is all uses in other projects since this has until now been essentially required.

---

The overall logic is really that if you supply a mylib array, and mylib implements tensor functions, you almost definitely (by default) want to use that backend and so supplying backend='mylib' feels redundant. Of course, if you really wanted to use numpy tensor functions you can explicitly set backend='numpy' but since this is the much more niche case it feels fine to be more verbose.

[dgasmith]

I think this represents a fundamental shift where we are no longer focused on a NumPy backend, but are agnostic. I would consider this a major change (3.x) as we change our philosophy statement (something that could be argued that we have done anyways). This would make me a bit more comfortable as we are potentially breaking code, thoughts?

[jcmgray]

Yes that seems like a reasonable approach to me!

[dgasmith]

LGTM