Use confection for configurations #745
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…fg. Adjust cnfig.py.
shadeMe
reviewed
Aug 24, 2022
|
@rmitsch: can you also run the spaCy test suite with this branch, via our explosion bot? |
|
@explosion-bot please test_gpu --run-on spaCy --run-on-branch master --thinc-branch current_pr |
URL: https://buildkite.com/explosion-ai/spacy-gpu-test-suite/builds/110 |
|
@explosion-bot please test_gpu --run-on spaCy --run-on-branch master --thinc-branch current_pr |
URL: https://buildkite.com/explosion-ai/spacy-gpu-test-suite/builds/111 |
|
@svlandeg spaCy test suite run succeded after latest commit. |
adrianeboyd
added a commit
that referenced
this pull request
Oct 12, 2022
* Move compatiblity-related code into a separate `compat` module (#652) * Add `compat` module to encapsulate imports of optional 3rd party frameworks/libraries * Replace references to compat code in `.util` with references to `.compat` Remove `cupy_ops. has_cupy` , `backends.has_cupy`, and `api.has_cupy` * Update example notebook * `util.set_active_gpu`: Return `None` if GPU is unavailable * `util`: Import tensorflow and mxnet with shorthand names Fix markdown formatting * `api`: Re-export `has_cupy` from `compat` * `backends`: Preserve `has_cupy` export for bwd-compat, remove superfluous imports * Revert "Update example notebook" This reverts commit 9f068a4. * `util`: Revert changes to `set_active_gpu`, raise an error if no GPU is detected Clarify docs * NumpyOps: Add a method to get a table of C BLAS functions (#643) * NumpyOps: Add a method to get a table of C BLAS functions This table can be used for downstream `cdef nogil` functions that need to use a BLAS function from the BLAS implementation used by an Ops subclass. * Bump blis requiment to >=0.9.0,<0.10.0 * NumpyOps: do not construct CBlas on every NumpyOps.cblas() call * api-backends: Fix superfluous wording * Fix a unit test in the PyTorch wrapper (#663) * Fix a unit test in the PyTorch wrapper This test checked whether the allocator was set to the PyTorch allocator when the PyTorch shim is used. However, this is not the case when PyTorch is installed, but CuPy isn't, so the test would fail. Since this test relies on CuPy, disable it when CuPy is not available. * Fix merge fallout * `CupyOps`: Simplify `asarray` (#661) * `CupyOps`: Simplify `asarray` * Remove `cast_array` flag and use `astype` unconditionally * Revert unconditional call to `astype` * Remove no-op * NumpyOps: Better type-casting in `asarray` (#656) * `NumpyOps`: Better type-casting in `asarray` * Simplify `dtype` check * Update thinc/backends/numpy_ops.pyx Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com> * Simplify casting further, avoid copies if possible * Remove no-op Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com> * Fix out-of-bounds writes in NumpyOps/CupyOps (#664) * Fix out-of-bounds writes in NumpyOps/CupyOps - Using `{CupyOps,NumpyOps}.adam` with incompatible shapes for weights, gradients, or moments resulted in out-of-bound writes. - Using `NumpyOps.adam` with non-float32 arrays resulted filling arrays with incorrect data. * Remove print debugging remnants Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com> * More print debugging remnants Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com> Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com> * Set version to v8.1.0.dev0 (#666) * Fix model.copy() bug where layer used more than once (#659) * Fix model.copy() bug where layer used more than once * Expand functionality to include shims * Corrections after review * Added default for Model._copy() * `conftest.py`: Handle exception caused by `pytest` options being added twice in CI builds (#670) * Auto-format code with `black` + Pin `black` requirement (#673) * Add `autoblack` GitHub action * Fix command * Add `black` to `requirements.txt` * Add support for bot-invoked slow tests (#672) * `Shim`: Fix potential data race when allocated on different threads * Fix two warnings (#676) - torch.nn.functional.sigmoid is deprecated in favor of torch.sigmoid. - Clip cosh input in sechsq to avoid overflow. * Replace use of gpu_is_available with has_cupy_gpu (#675) * Replace use of gpu_is_available with has_cupy_gpu This PR is in preparation of better non-CUDA device support. Once we support non-CUDA GPUs, there may be GPUs available that are not 'CuPy GPUs'. In all places where we use `gpu_is_available` we actually mean: is 'CuPy available with a CUDA GPU'? So, this PR replaces uses of `gpu_is_available` to `has_cupy_gpu`. This allows us to use `gpu_is_available` in the future to check if any GPU is available. In addition to that, some code had expressions like ``` has_cupy and gpu_is_available() ``` This PR simplify such conditions to `has_cupy_gpu`, since `has_cupy_gpu` implies that `has_cupy`. * Remove unused import * Improve error message when no CUDA GPU is found * Fix another error message when no CUDA GPU is found * Fixes for slow tests (#671) * `test_uniqued`: Disable test timing for `test_uniqued_doesnt_change_result` (#678) * `test_to_categorical`: Ensure that `label_smoothing < 0.5` (#680) * `test_to_categorical`: Ensure that `label_smoothing < 0.5` * Use `exclude_max` instead of clamping to `0.49` * test_ops: do not lower precision in conversion to Torch tensor (#681) * test_ops: do not lower precision in conversion to Torch tensor float64 test values close to zero were rounded by conversion to a float32 Torch tensor, resuling in mismatches between Thinc and Torch gradients. This change prevents the loss in precision. * test_ops: compare arrays on same device in Torch comparison * test_maxout: compare arrays with same precision * Add `test_slow_gpu` explosion-bot command * Auto-format code with black (#682) Co-authored-by: explosion-bot <explosion-bot@users.noreply.github.com> * Azure: pin protobuf to fix Tensorflow * Extend typing_extensions to <4.2.0 (#689) * xp2{tensorflow,torch}: convert NumPy arrays using dlpack (#686) * xp2{tensorflow,torch}: convert NumPy arrays using dlpack Newer versions of NumPy can expose arrays as dlpack capsules. Use this functionality (when supported) to speed up NumPy -> Torch/Tensorflow array conversion. * Fix up copy paste error * `test_model_gpu`: Use TF memory pool if available, feature-gate test (#688) * `test_model_gpu`: Use TF memory pool if available, feature-gate test * Fix typo * `test_predict_extensive`: Disable test time monitoring * Fix imports, use `has_cupy_gpu` for forward-compat * `conftest`: Use `pytest_sessionstart` to enable TF GPU memory growth * Bump version to v8.1.0.dev1 (#694) * `NumpyOps`: Do not use global for `CBlas` (#697) * Merge pytorch-device branch into master (
Goal
Use https://github.com/explosion/confection/ for processing configurations in
thinc. Remove tests that have been integrated into theconfectiontest suite.Description
Integration of
confectionintothinc, removal of redundant tests.Types of change
Refactoring.
Checklist