Add instancenorm and alpha_dropout implementations #180
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
avik-pal
force-pushed
the
ap/luxlib_parity
branch
from
82ba1e5 to
d366e3f
Compare
Codecov ReportBase: 88.88% // Head: 88.81% // Decreases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## main #180 +/- ##
==========================================
- Coverage 88.88% 88.81% -0.07%
==========================================
Files 19 19
Lines 1278 1279 +1
==========================================
Hits 1136 1136
- Misses 142 143 +1
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
avik-pal
force-pushed
the
ap/luxlib_parity
branch
from
d366e3f to
a759abd
Compare
avik-pal
added a commit
that referenced
this pull request
Nov 3, 2024
avik-pal
added a commit
that referenced
this pull request
Nov 3, 2024
* perf: cleanup the benchmarking script * perf: add benchmarks for Zygote * perf: try reclaiming memory * fix: incorrect system parameters * perf: temporarily disable non-dense benchmarks [skip tests] * ci(benchmark): allow proceed on failure [skip tests] * perf: update polyalg selection for matmul and matmuladd * test: ensure no additional allocations for matmul * fix: typo in AMDGPU batched matmul * perf: restore running all benchmarks * docs: add link to benchmarks * ci: fix benchmarks config * test: run allocs test only on CPU * fix: mixed-precision use Octavian if possible * feat: add traits to fuse activation functions [skip ci] * perf: selective vectorization of operations bias_add/activation * perf: fused bias activation for certain operations * perf: optimize batchnorm implementation * perf: don't fuse tanh * perf: run specific benchmarks * perf: be conservative while fusing activation functions * refactor: qualify CPU functions with `_cpu` * perf: restore running all benchmarks * fix(tracker): expand custom Tracker AD for wrapper types * fix: subtyping correction * test: ignore tests for batched_vec (not our code) * perf: faster version of groupnorm * ci: run downstream testing only on pull requests * refactor: remove unnecessary forced inlining * refactor: move PartialFunctions into a module * refactor: move utilities into Utils * refactor: move device agnostic functions to `DeviceAgnostic` * test: separate out the testing project file * refactor: move internal functions into separate modules * test: separate out the testing project file * fix: incorrect internal calls * refactor: remove unnecessary turbo loop * perf: don't rely on compile time branch removal for KA * perf: static ndrange kernel launches * perf: let it autotune * refactor: use multiple dispatch for cleaner kernels * refactor: disable cpu codegen for kernels * fix: nicer information for fallback mixed-precision matmul * fix: allow zero-sized arrays in bias_activation * fix: don't restrict bias_act to number * fix: don't restrict traits/ext/utils to number * fix: more aggressive type specialization * chore: update version * fix: broken qa tests * fix: use `fmap_with_path` to correctly identify all internal states * chore: apply formatting suggestion Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix: don't error on detecting arrays with undefined entries * refactor: move ChainRulesCore into an extension * fix: skip enzyme tests if it is a pre-release * chore: bump version for release * fix: decide internal operation based on unwrapped arrays * fix: avoid wrappers for SVector using `insert_batch_dim` * fix: enzyme forward mode with octavian * feat: swap Enzyme forward rules along with reverse * test: simple enzyme forward test to check no crash * chore: bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * feat: add `unsafe_free!` * feat: add DeviceIterator (and support parallel Device DataLoader) * test: basic tests for free-ing data * refactor: simplify parallel dataloader * test: DataLoader aggressive freeing * docs: add docstrings for `DeviceIterator` * refactor: deprecate "Explicit" in favor of "Lux" * chore: add deprecation for the single arg outputsize * fix: remove old uses of Explicit * fix!: remove deprecations * chore: add exports for abstract layers * refactor: move Functors and Setfield into ext * fix!: remove hacky version of outputsize * feat: add `AbstractLuxWrapperLayer` * refactor: cleanup extension usage * test: update test to new API * test: extension loading errors * feat: support functors for WrappedLayer * test: LuxWrappedLayer tested * test: don't qualify unnecessarily * refactor: cleanup internal functions * fix!: remove default slow handling of outputsize * fix: update removed API * test: update old tests * fix!: remove unused `inputsize` * fix: add fmap_with_path support * chore: fix formatting * feat: default call for wrapper layers * fix: remove hacky usage of module getproperty rrules * fix: accidental dual usage of `ofeltype_array` * feat: auto-training mode and strict checks * chore: bump compat for LuxCore to 1, (keep existing compat) (#147) Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> * feat: extend the layernorm API * test: more detailed layernorm testing * chore: bump version for release * fix!: remove deprecations for 1.0 release * chore!: remove Reexport of NNlib (will be done via Lux) * perf: add NNlib to benchmarks deps * fix: remove unused explicit imports * chore: update to using LuxCore@1.0 * fix!: remove dropout branching based on size * fix!: change the default layernorm dims * chore: bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * feat: add enzyme reverse rules for `fused_dense!` * test: add tests for the enzyme fused_dense rules * fix: typo in reverse rule * test: run tests with more activations * feat: instancenorm with running statistics * fix: fixes for testing * fix: modify the dropout testing * fix: windows testing for dropout * chore(deps): bump crate-ci/typos from 1.24.3 to 1.24.5 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.5) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump peter-evans/create-pull-request from 6 to 7 Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump peter-evans/create-pull-request from 6 to 7 Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.3 to 1.24.5 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.5) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump peter-evans/create-pull-request from 6 to 7 Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.3 to 1.24.5 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.5) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump peter-evans/create-pull-request from 6 to 7 (#19) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump peter-evans/create-pull-request from 6 to 7 Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.3 to 1.24.5 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.5) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * test: add tests comparing the fused op with unfused op * fix: improve load times by moving CRC to ext * fix: remove UnrolledUtilities dep * fix: remove UnrolledUtilities dep * chore: bump minimum MLDataDevices version * fix: dropout tests are no longer broken * chore: accidentally left deprecations file * fix: missing enzyme rules for matmuladd! (CUDA support) * test: incorrect condition * test: incorrect function name * fix: zero out shadows * fix: enzyme reverse bias needs a check on Const * chore: bump crate-ci/typos from 1.24.5 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.5 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.5...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * feat: better test integration in test_gradients * feat: add test_gradients macro * chore: apply formatting suggestion Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix: update to use test_gradients macro * fix: bias needs to add accum gradients * chore: bump `EnzymeCore` version * CompatHelper: bump compat for EnzymeCore in [weakdeps] to 0.8, (keep existing compat) * chore: bump version for release --------- Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> Co-authored-by: Avik Pal <avikpal@mit.edu> * chore: install latest enzyme version * chore: update Enzyme version * chore: bump minimum versions * ci: update buildkite settings * feat: wider support for batched_matmul * perf: benchmark fallback batched_matmul * feat: slow fallback conv impl * feat: parallel fallback batchedmm * ci(buildkite): add GPU testing for Metal and oneAPI * test: check for FP64 support * fix: convert element type before broadcasting * fix: dispatch for NNlib conv * ci(buildkite): disable testing for Metal and oneAPI * chore: bump version * feat: update minimum version of Enzyme to 0.13 * feat: support within_gradient for Enzyme * refactor: rename within_gradient to within_autodiff * fix: update forward rules to new API * fix: use known on the return type * fix: forward enzyme rules * fix: broken enzyme tests * feat: support runtime activity for enzyme * fix: check was accidentally broken * chore(deps): bump crate-ci/typos from 1.24.5 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.5 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.5...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.3 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.5 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.5 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.5...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.5 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.5 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.5...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * fix: rollback custom gelu implementation * feat: XLADevice via Reactant * chore: apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * chore: bump version * feat: more extensive testing of XLA backend * fix: incorrect function call * test: rename * test: incorrect env var * fix: copy to XLA in main thread * fix: don't support pre-moving the data * fix: urgent patch for reactant breakage * chore: bump crate-ci/typos from 1.24.6 to 1.25.0 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.25.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.25.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump crate-ci/typos from 1.24.6 to 1.25.0 (#41) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.25.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.25.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.24.6 to 1.25.0 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.25.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.25.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.6 to 1.25.0 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.25.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.25.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.6 to 1.26.0 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.26.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.26.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * ci: run on `1.10` and `1` (#57) * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` (#81) * ci: run on 1.10 and 1 * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` (#43) * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` * test: mark truncated normal on Metal as unbroken * ci: run buildkite on `1.10` and `1` * chore: bump peter-evans/create-pull-request from 6 to 7 (#40) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ci: run tests only on `1.10` for now (#172) * fix: relax cublaslt types (#173) * docs: add Flux.jl to the README (#83) After FluxML/Flux.jl#2492 also Flux relies on MLDataDevices. * chore: bump crate-ci/typos from 1.25.0 to 1.26.0 (#58) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.25.0 to 1.26.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.25.0...v1.26.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.25.0 to 1.26.0 (#44) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.25.0 to 1.26.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.25.0...v1.26.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.25.0 to 1.26.0 (#174) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.25.0 to 1.26.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.25.0...v1.26.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump compat for GPUArrays in [weakdeps] to 11, (keep existing compat) (#86) Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> * chore: bump version for release * chore: bump compat for GPUArrays in [weakdeps] to 11, (keep existing compat) (#46) Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> * chore: bump compat for GPUArraysCore to 0.2, (keep existing compat) (#47) Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> Co-authored-by: Avik Pal <avikpal@mit.edu> * chore: bump version for release * feat: add fallbacks for unknown objects (#87) * feat: add fallbacks for unknown objects * feat: handle RNGs and undef arrays gracefully * test: RNG movement * test: functions and closures * refactor: move `JuliaSIMD` deps to extensions (#175) * fix: remove LV.vmap! usage * fix: remove LV handling for bias_activation * fix: remove LV usage in dropout * refactor: move LV and octavian behind an extension * docs: add docs for loading packages * refactor: move SLEEFPirates to an ext * fix: enzyme rules for batched matmul * fix: patch more enzyme issues * feat: add a preference to disable loop vectorization * fix: incorrect dispatch called * fix: enzyme segfault bypass * feat: define isleaf (#84) * isleaf * exclude * add tests and docs * more tests * import functors * fix test * chore: reduce min compat * chore: run formatter * chore: bump version for release * fix: handle bitstypes and wrapped arrays in isleaf (#88) * bitstype and wrapped arrays * fixes * fix import * bound * cleanup * chore: fix min version of LinearAlgebra * chore: run formatter --------- Co-authored-by: Avik Pal <avik.pal.2017@gmail.com> Co-authored-by: Avik Pal <avikpal@mit.edu> * fix: task switching in AMDGPU complex batched_matmul (#178) * ci(buildkite): add downstream testing for NeuralOperators * perf: restore old batched_mul * fix: disable threading for certain devices * revert: "perf: restore old batched_mul" This reverts commit a8c0f3b4615f96a8773577e16fac61ba310d8123. * fix: correctly handle adjoints of wrapped arrays (#90) * fix: correctly handle adjoints of wrapped arrays * fix: use fast paths for adapt * fix: adapt ranges to JuliaGPU/Adapt.jl#86 * chore(deps): bump crate-ci/typos from 1.25.0 to 1.26.8 (#44) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.25.0 to 1.26.8. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.25.0...v1.26.8) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.26.0 to 1.26.8 (#49) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.26.0 to 1.26.8. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.26.0...v1.26.8) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.26.0 to 1.26.8 (#60) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.26.0 to 1.26.8. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.26.0...v1.26.8) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: missing import; fixes #179 (#180) * chore: bump crate-ci/typos from 1.26.0 to 1.26.8 (#93) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.26.0 to 1.26.8. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.26.0...v1.26.8) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ci: merge LuxCUDA testing scripts * ci: merge LuxCore testing scripts * ci: merge WeightInitializers testing scripts * ci: add WI to pipeline launch * ci: add MLDataDevices to pipeline launch * ci: change 1.10 to "lts" * test: LuxCore test fixes * ci: soft fail MLDataDevices * ci: add a central downstream testing * ci: partially migrate LuxLib CI * ci: remove name field * ci: minor fixes to build scripts * ci: move LuxTestUtils CI scripts * ci: update LuxLib workflow * ci: update LuxLib workflows * ci: split out downstream testing * ci: fix certain pipelines * ci: minor tweaks * fix: workflows * test: use local LuxCUDA for tests * fix: use develop * docs: update * fix: add dev packages * docs: dev required packages * perf: merge the benchmarks --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> Co-authored-by: Christopher Rackauckas <accounts@chrisrackauckas.com> Co-authored-by: Carlo Lucibello <carlo.lucibello@gmail.com>
avik-pal
added a commit
that referenced
this pull request
Nov 3, 2024
* perf: cleanup the benchmarking script * perf: add benchmarks for Zygote * perf: try reclaiming memory * fix: incorrect system parameters * perf: temporarily disable non-dense benchmarks [skip tests] * ci(benchmark): allow proceed on failure [skip tests] * perf: update polyalg selection for matmul and matmuladd * test: ensure no additional allocations for matmul * fix: typo in AMDGPU batched matmul * perf: restore running all benchmarks * docs: add link to benchmarks * ci: fix benchmarks config * test: run allocs test only on CPU * fix: mixed-precision use Octavian if possible * feat: add traits to fuse activation functions [skip ci] * perf: selective vectorization of operations bias_add/activation * perf: fused bias activation for certain operations * perf: optimize batchnorm implementation * perf: don't fuse tanh * perf: run specific benchmarks * perf: be conservative while fusing activation functions * refactor: qualify CPU functions with `_cpu` * perf: restore running all benchmarks * fix(tracker): expand custom Tracker AD for wrapper types * fix: subtyping correction * test: ignore tests for batched_vec (not our code) * perf: faster version of groupnorm * ci: run downstream testing only on pull requests * refactor: remove unnecessary forced inlining * refactor: move PartialFunctions into a module * refactor: move utilities into Utils * refactor: move device agnostic functions to `DeviceAgnostic` * test: separate out the testing project file * refactor: move internal functions into separate modules * test: separate out the testing project file * fix: incorrect internal calls * refactor: remove unnecessary turbo loop * perf: don't rely on compile time branch removal for KA * perf: static ndrange kernel launches * perf: let it autotune * refactor: use multiple dispatch for cleaner kernels * refactor: disable cpu codegen for kernels * fix: nicer information for fallback mixed-precision matmul * fix: allow zero-sized arrays in bias_activation * fix: don't restrict bias_act to number * fix: don't restrict traits/ext/utils to number * fix: more aggressive type specialization * chore: update version * fix: broken qa tests * fix: use `fmap_with_path` to correctly identify all internal states * chore: apply formatting suggestion Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix: don't error on detecting arrays with undefined entries * refactor: move ChainRulesCore into an extension * fix: skip enzyme tests if it is a pre-release * chore: bump version for release * fix: decide internal operation based on unwrapped arrays * fix: avoid wrappers for SVector using `insert_batch_dim` * fix: enzyme forward mode with octavian * feat: swap Enzyme forward rules along with reverse * test: simple enzyme forward test to check no crash * chore: bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.23.6 to 1.24.1 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.6 to 1.24.1. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.23.6...v1.24.1) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * feat: add `unsafe_free!` * feat: add DeviceIterator (and support parallel Device DataLoader) * test: basic tests for free-ing data * refactor: simplify parallel dataloader * test: DataLoader aggressive freeing * docs: add docstrings for `DeviceIterator` * refactor: deprecate "Explicit" in favor of "Lux" * chore: add deprecation for the single arg outputsize * fix: remove old uses of Explicit * fix!: remove deprecations * chore: add exports for abstract layers * refactor: move Functors and Setfield into ext * fix!: remove hacky version of outputsize * feat: add `AbstractLuxWrapperLayer` * refactor: cleanup extension usage * test: update test to new API * test: extension loading errors * feat: support functors for WrappedLayer * test: LuxWrappedLayer tested * test: don't qualify unnecessarily * refactor: cleanup internal functions * fix!: remove default slow handling of outputsize * fix: update removed API * test: update old tests * fix!: remove unused `inputsize` * fix: add fmap_with_path support * chore: fix formatting * feat: default call for wrapper layers * fix: remove hacky usage of module getproperty rrules * fix: accidental dual usage of `ofeltype_array` * feat: auto-training mode and strict checks * chore: bump compat for LuxCore to 1, (keep existing compat) (#147) Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> * feat: extend the layernorm API * test: more detailed layernorm testing * chore: bump version for release * fix!: remove deprecations for 1.0 release * chore!: remove Reexport of NNlib (will be done via Lux) * perf: add NNlib to benchmarks deps * fix: remove unused explicit imports * chore: update to using LuxCore@1.0 * fix!: remove dropout branching based on size * fix!: change the default layernorm dims * chore: bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.1 to 1.24.3 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.1 to 1.24.3. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.1...v1.24.3) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * feat: add enzyme reverse rules for `fused_dense!` * test: add tests for the enzyme fused_dense rules * fix: typo in reverse rule * test: run tests with more activations * feat: instancenorm with running statistics * fix: fixes for testing * fix: modify the dropout testing * fix: windows testing for dropout * chore(deps): bump crate-ci/typos from 1.24.3 to 1.24.5 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.5) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump peter-evans/create-pull-request from 6 to 7 Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump peter-evans/create-pull-request from 6 to 7 Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.3 to 1.24.5 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.5) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump peter-evans/create-pull-request from 6 to 7 Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.3 to 1.24.5 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.5) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump peter-evans/create-pull-request from 6 to 7 (#19) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump peter-evans/create-pull-request from 6 to 7 Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.3 to 1.24.5 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.5. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.5) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * test: add tests comparing the fused op with unfused op * fix: improve load times by moving CRC to ext * fix: remove UnrolledUtilities dep * fix: remove UnrolledUtilities dep * chore: bump minimum MLDataDevices version * fix: dropout tests are no longer broken * chore: accidentally left deprecations file * fix: missing enzyme rules for matmuladd! (CUDA support) * test: incorrect condition * test: incorrect function name * fix: zero out shadows * fix: enzyme reverse bias needs a check on Const * chore: bump crate-ci/typos from 1.24.5 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.5 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.5...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * feat: better test integration in test_gradients * feat: add test_gradients macro * chore: apply formatting suggestion Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix: update to use test_gradients macro * fix: bias needs to add accum gradients * chore: bump `EnzymeCore` version * CompatHelper: bump compat for EnzymeCore in [weakdeps] to 0.8, (keep existing compat) * chore: bump version for release --------- Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> Co-authored-by: Avik Pal <avikpal@mit.edu> * chore: install latest enzyme version * chore: update Enzyme version * chore: bump minimum versions * ci: update buildkite settings * feat: wider support for batched_matmul * perf: benchmark fallback batched_matmul * feat: slow fallback conv impl * feat: parallel fallback batchedmm * ci(buildkite): add GPU testing for Metal and oneAPI * test: check for FP64 support * fix: convert element type before broadcasting * fix: dispatch for NNlib conv * ci(buildkite): disable testing for Metal and oneAPI * chore: bump version * feat: update minimum version of Enzyme to 0.13 * feat: support within_gradient for Enzyme * refactor: rename within_gradient to within_autodiff * fix: update forward rules to new API * fix: use known on the return type * fix: forward enzyme rules * fix: broken enzyme tests * feat: support runtime activity for enzyme * fix: check was accidentally broken * chore(deps): bump crate-ci/typos from 1.24.5 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.5 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.5...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.3 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.3 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.3...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.5 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.5 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.5...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.5 to 1.24.6 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.5 to 1.24.6. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.5...v1.24.6) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * fix: rollback custom gelu implementation * feat: XLADevice via Reactant * chore: apply suggestions from code review Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * chore: bump version * feat: more extensive testing of XLA backend * fix: incorrect function call * test: rename * test: incorrect env var * fix: copy to XLA in main thread * fix: don't support pre-moving the data * fix: urgent patch for reactant breakage * chore: bump crate-ci/typos from 1.24.6 to 1.25.0 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.25.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.25.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump crate-ci/typos from 1.24.6 to 1.25.0 (#41) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.25.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.25.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.24.6 to 1.25.0 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.25.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.25.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.6 to 1.25.0 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.25.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.25.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: bump crate-ci/typos from 1.24.6 to 1.26.0 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.24.6 to 1.26.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.24.6...v1.26.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * ci: run on `1.10` and `1` (#57) * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` (#81) * ci: run on 1.10 and 1 * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` (#43) * ci: run on `1.10` and `1` * ci: run on `1.10` and `1` * test: mark truncated normal on Metal as unbroken * ci: run buildkite on `1.10` and `1` * chore: bump peter-evans/create-pull-request from 6 to 7 (#40) Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6 to 7. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v6...v7) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ci: run tests only on `1.10` for now (#172) * fix: relax cublaslt types (#173) * docs: add Flux.jl to the README (#83) After FluxML/Flux.jl#2492 also Flux relies on MLDataDevices. * chore: bump crate-ci/typos from 1.25.0 to 1.26.0 (#58) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.25.0 to 1.26.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.25.0...v1.26.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.25.0 to 1.26.0 (#44) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.25.0 to 1.26.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.25.0...v1.26.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.25.0 to 1.26.0 (#174) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.25.0 to 1.26.0. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.25.0...v1.26.0) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump compat for GPUArrays in [weakdeps] to 11, (keep existing compat) (#86) Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> * chore: bump version for release * chore: bump compat for GPUArrays in [weakdeps] to 11, (keep existing compat) (#46) Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> * chore: bump compat for GPUArraysCore to 0.2, (keep existing compat) (#47) Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> Co-authored-by: Avik Pal <avikpal@mit.edu> * chore: bump version for release * feat: add fallbacks for unknown objects (#87) * feat: add fallbacks for unknown objects * feat: handle RNGs and undef arrays gracefully * test: RNG movement * test: functions and closures * refactor: move `JuliaSIMD` deps to extensions (#175) * fix: remove LV.vmap! usage * fix: remove LV handling for bias_activation * fix: remove LV usage in dropout * refactor: move LV and octavian behind an extension * docs: add docs for loading packages * refactor: move SLEEFPirates to an ext * fix: enzyme rules for batched matmul * fix: patch more enzyme issues * feat: add a preference to disable loop vectorization * fix: incorrect dispatch called * fix: enzyme segfault bypass * feat: define isleaf (#84) * isleaf * exclude * add tests and docs * more tests * import functors * fix test * chore: reduce min compat * chore: run formatter * chore: bump version for release * fix: handle bitstypes and wrapped arrays in isleaf (#88) * bitstype and wrapped arrays * fixes * fix import * bound * cleanup * chore: fix min version of LinearAlgebra * chore: run formatter --------- Co-authored-by: Avik Pal <avik.pal.2017@gmail.com> Co-authored-by: Avik Pal <avikpal@mit.edu> * fix: task switching in AMDGPU complex batched_matmul (#178) * ci(buildkite): add downstream testing for NeuralOperators * perf: restore old batched_mul * fix: disable threading for certain devices * revert: "perf: restore old batched_mul" This reverts commit a8c0f3b4615f96a8773577e16fac61ba310d8123. * fix: correctly handle adjoints of wrapped arrays (#90) * fix: correctly handle adjoints of wrapped arrays * fix: use fast paths for adapt * fix: adapt ranges to JuliaGPU/Adapt.jl#86 * chore(deps): bump crate-ci/typos from 1.25.0 to 1.26.8 (#44) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.25.0 to 1.26.8. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.25.0...v1.26.8) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.26.0 to 1.26.8 (#49) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.26.0 to 1.26.8. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.26.0...v1.26.8) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump crate-ci/typos from 1.26.0 to 1.26.8 (#60) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.26.0 to 1.26.8. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.26.0...v1.26.8) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: missing import; fixes #179 (#180) * chore: bump crate-ci/typos from 1.26.0 to 1.26.8 (#93) Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.26.0 to 1.26.8. - [Release notes](https://github.com/crate-ci/typos/releases) - [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md) - [Commits](crate-ci/typos@v1.26.0...v1.26.8) --- updated-dependencies: - dependency-name: crate-ci/typos dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ci: merge LuxCUDA testing scripts * ci: merge LuxCore testing scripts * ci: merge WeightInitializers testing scripts * ci: add WI to pipeline launch * ci: add MLDataDevices to pipeline launch * ci: change 1.10 to "lts" * test: LuxCore test fixes * ci: soft fail MLDataDevices * ci: add a central downstream testing * ci: partially migrate LuxLib CI * ci: remove name field * ci: minor fixes to build scripts * ci: move LuxTestUtils CI scripts * ci: update LuxLib workflow * ci: update LuxLib workflows * ci: split out downstream testing * ci: fix certain pipelines * ci: minor tweaks * fix: workflows * test: use local LuxCUDA for tests * fix: use develop * docs: update * fix: add dev packages * docs: dev required packages * perf: merge the benchmarks --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: CompatHelper Julia <compathelper_noreply@julialang.org> Co-authored-by: Christopher Rackauckas <accounts@chrisrackauckas.com> Co-authored-by: Carlo Lucibello <carlo.lucibello@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.