Add Duplicated methods
#192
Conversation
| shared .= 1 | ||
| Optimisers.update!(st2, model, grad) | ||
| model.x ≈ [0.8] # This is wrong, but don't make it a test. | ||
| # Ideally, perhaps the 3-arg update! could notice that grad.x===grad.y, and not accumulate the gradient in this case? |
There was a problem hiding this comment.
I get what you mean, but do we promise to handle shared gradient arrays? Might be easier to add a warning in the docs. Maybe a broken test outside of the Enzyme test suite if we really care about this.
There was a problem hiding this comment.
At present (after this PR) I think the usual path with Zygote and this new path with Enzyme should always agree.
Whether the normal path can auto-detect this (as in the comment) I'm not really sure. It's possible to make Zygote return two components which are ===, and it's conceivable that this could happen with shared parameters.
Nevertheless, detecting this would probably be better than not. In real use, e.g. you use the same layer twice, Zygote is certainly going to return two new arrays for shared parameters.
* add Duplicated methods * add test * test for shared params + minimal docs * remove 1.6 CI * indent by two spaces * fix doctest
This moves part of FluxML/Flux.jl#2471 to an extension here.
Bumps Julia support to 1.10.
PR Checklist