✨ Extend Annotation to Support Init from WKB #639

John-P · 2023-07-14T10:40:25Z

I have created this as a proof of concept in regard to my comments on #632.

I have extended the Annotation class to enable the creation from either a set of coordinates and a geometry type (int) for direct decoding of the WKB OR from a shapely geometry object. The shapely geometry is created lazily, and vice versa; the coordinates can be accessed lazily if the Annotation was created from a Shapely object.

Summary

Enables init of Annotation from WKB bytes instead of from Shapely geometry object. This can shorten the path from database WKB -> Shapely -> Numpy Array for applications which just need the raw coordinates and not any Shapely/geos methods.
Move decode_wkb to an Annotation static method. This could potentially be optimized with numba or similar in future.
Lazily create a Shapely geometry object, WKB, or raw numpy coordinates as needed. This can avoid converting WKB to shapely and then to coordinates and go directly from WKB to coordinates.
Add tests.
Update code relying on as_wkb.
Bug fixes including fixing exceptions on appending unsupported geometry and generation of coordinates from WKB in decode_wkb.
Ensures the coordinates generates from Shapely and from decode_wkb via Annotation.coords are in the same format as each other. I.e. 1x2 array for points, nx2 array for lines, list of nx2 arrays for poylgons and lists for the multi forms.

…toolbox into storage-updates

…toolbox into wkb-decoding

- Minor docstring changes

for more information, see https://pre-commit.ci

It is now more consistent with to_geojson and to_feature.

for more information, see https://pre-commit.ci

shaneahmed · 2023-07-14T12:00:37Z

Please can you run ruff with current settings in #625 after adding from __future__ import annotations to the top. This will fix type annotations to be consistent with the new format.

…toolbox into wkb-decoding

tests/test_annotation_stores.py

shaneahmed

Added some comments on tests/test_annotation_stores.py. Will continue with the rest of the files.

tests/test_annotation_stores.py

…ization.py

…ation-ext # Conflicts: # tests/test_annotation_stores.py # tests/test_visualization.py

for more information, see https://pre-commit.ci

…ation-ext # Conflicts: # tiatoolbox/utils/visualization.py

shaneahmed · 2023-08-11T08:43:03Z

I am happy with this PR. I have raised an issue related to a test which fails on Windows #692

shaneahmed · 2023-08-11T10:52:55Z

@measty Please let me know if you are happy with this PR before I merge it with develop.

measty

Just left a few minor comments. Happy to approve it, looks good.

Thanks for adding this!

tests/test_annotation_stores.py

@TomastpPereira

## 1.5.0 (2023-12-15) ### Major Updates and Feature Improvements - Adds the bokeh visualization tool. #684 - The tool allows a user to launch a server on their machine to visualise whole slide images, overlay the results of deep learning algorithms or to select a patch from whole slide image and run TIAToolbox deep learning engines. - This tool powers the TIA demos server. For details please see https://tiademos.dcs.warwick.ac.uk/. - Extends Annotation to Support Init from WKB #639 - Adds `IOConfig` for NuClick in `pretrained_model.yaml` #709 - Adds functions to save the TIAToolbox Engine outputs to Zarr and AnnotationStore files. #724 - Adds Support for QuPath Annotation Imports #721 ### Changes to API - Adds `model.to(device)` and `model.load_model_from_file()` functionality to make it compatible with PyTorch API. #733 - Replaces `pretrained` with `weights` to make the engines compatible with the new PyTorch API. #621 - Adds support for high-level imports for various utility functions and classes such as `WSIReader`, `PatchPredictor` and `imread` #606, #607, - Adds `tiatoolbox.typing` for type hints. #619 - Fixes incorrect file size saved by `save_tiles`, issue with certain WSIs raised by @TomastpPereira - TissueMasker transform now returns mask instead of a list. #748 - Fixes #732 ### Bug Fixes and Other Changes - Fixes `pixman` incompability error on Colab #601 - Removes `shapely.speedups`. The module no longer has any affect in Shapely >=2.0. #622 - Fixes errors in the slidegraph example notebook #608 - Fixes bugs in WSI Registration #645, #670, #693 - Fixes the situation where PatchExtractor.get_coords() can return patch coords which lie fully outside the bounds of a slide. #712 - Fixes #710 - Fixes #738 raised by @xiachenrui ### Development related changes - Replaces `flake8` and `isort` with `ruff` #625, #666 - Adds `mypy` checks to `root` and `utils` package. This will be rolled out in phases to other modules. #723 - Adds a module to detect file types using magic number/signatures #616 - Uses `poetry` for version updates instead of `bump2version`. #638 - Removes `setup.cfg` and uses `pyproject.toml` for project configurations. - Reduces runtime for some unit tests e.g., #627, #630, #631, #629 - Reuses models and datasets in tests on GitHub actions by utilising cache #641, #644 - Set up parallel tests locally #671 **Full Changelog:** v1.4.0...v1.5.0 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: mostafajahanifar <74412979+mostafajahanifar@users.noreply.github.com> Co-authored-by: John Pocock <John-P@users.noreply.github.com> Co-authored-by: DavidBAEpstein <David.Epstein@warwick.ac.uk> Co-authored-by: David Epstein <22086916+DavidBAEpstein@users.noreply.github.com> Co-authored-by: Ruqayya Awan <18444369+ruqayya@users.noreply.github.com> Co-authored-by: Mark Eastwood <20169086+measty@users.noreply.github.com> Co-authored-by: adamshephard <39619155+adamshephard@users.noreply.github.com> Co-authored-by: adamshephard <adam.shephard@warwick.ac.uk> Co-authored-by: Abdol <a@fkrtech.com> Co-authored-by: Jiaqi-Lv <60471431+Jiaqi-Lv@users.noreply.github.com> Co-authored-by: Abishek <abishekraj6797@gmail.com> Co-authored-by: Dmitrii Blaginin <blaginin@mbp.lan>

@TomastpPereira

## 1.5.0 (2023-12-15) ### Major Updates and Feature Improvements - Adds the bokeh visualization tool. #684 - The tool allows a user to launch a server on their machine to visualise whole slide images, overlay the results of deep learning algorithms or to select a patch from whole slide image and run TIAToolbox deep learning engines. - This tool powers the TIA demos server. For details please see https://tiademos.dcs.warwick.ac.uk/. - Extends Annotation to Support Init from WKB #639 - Adds `IOConfig` for NuClick in `pretrained_model.yaml` #709 - Adds functions to save the TIAToolbox Engine outputs to Zarr and AnnotationStore files. #724 - Adds Support for QuPath Annotation Imports #721 ### Changes to API - Adds `model.to(device)` and `model.load_model_from_file()` functionality to make it compatible with PyTorch API. #733 - Replaces `pretrained` with `weights` to make the engines compatible with the new PyTorch API. #621 - Adds support for high-level imports for various utility functions and classes such as `WSIReader`, `PatchPredictor` and `imread` #606, #607, - Adds `tiatoolbox.typing` for type hints. #619 - Fixes incorrect file size saved by `save_tiles`, issue with certain WSIs raised by @TomastpPereira - TissueMasker transform now returns mask instead of a list. #748 - Fixes #732 ### Bug Fixes and Other Changes - Fixes `pixman` incompability error on Colab #601 - Removes `shapely.speedups`. The module no longer has any affect in Shapely >=2.0. #622 - Fixes errors in the slidegraph example notebook #608 - Fixes bugs in WSI Registration #645, #670, #693 - Fixes the situation where PatchExtractor.get_coords() can return patch coords which lie fully outside the bounds of a slide. #712 - Fixes #710 - Fixes #738 raised by @xiachenrui ### Development related changes - Replaces `flake8` and `isort` with `ruff` #625, #666 - Adds `mypy` checks to `root` and `utils` package. This will be rolled out in phases to other modules. #723 - Adds a module to detect file types using magic number/signatures #616 - Uses `poetry` for version updates instead of `bump2version`. #638 - Removes `setup.cfg` and uses `pyproject.toml` for project configurations. - Reduces runtime for some unit tests e.g., #627, #630, #631, #629 - Reuses models and datasets in tests on GitHub actions by utilising cache #641, #644 - Set up parallel tests locally #671 **Full Changelog:** v1.4.0...v1.5.0

measty and others added 16 commits June 23, 2023 14:11

storage updates

214e637

byte geom decoding

5952ca0

Merge branch 'develop' of https://github.com/TissueImageAnalytics/tia…

e634918

…toolbox into storage-updates

add wkb decoding via numpy

362bc0a

Merge branch 'develop' of https://github.com/TissueImageAnalytics/tia…

65aaffa

…toolbox into wkb-decoding

deepsource fixes

7414195

add tests

1e5b7ef

further tests

faf4450

min area test

502025c

warnings test

d2c1c87

📝 Minor docstring changes

7bbce45

- Minor docstring changes

[pre-commit.ci] auto fixes from pre-commit.com hooks

e39d46b

for more information, see https://pre-commit.ci

Merge branch 'develop' into wkb-decoding

6d145da

🎨 Change to_wkb_geometry to to_wkb For Consistency

5c62547

It is now more consistent with to_geojson and to_feature.

🐛 Correct Return Type for to_wkb

fe1c809

✨ Extend Annotation To Support Creation From Coords

4b8006c

John-P requested a review from measty July 14, 2023 10:40

pre-commit-ci bot and others added 3 commits July 14, 2023 10:41

[pre-commit.ci] auto fixes from pre-commit.com hooks

638a057

for more information, see https://pre-commit.ci

🐛 Require coords or geometry in init

35de8ac

🎨 Remove Redundant Conditional

2fefa27

shaneahmed added 2 commits July 14, 2023 13:04

Merge branch 'develop' into wkb-decoding

4e02aab

Merge branch 'wkb-decoding' into wkb-decoding-annotation-ext

948d650

shaneahmed added the enhancement New feature or request label Jul 14, 2023

shaneahmed added this to the Release v1.5.0 milestone Jul 14, 2023

shaneahmed assigned John-P Jul 14, 2023

shaneahmed and others added 4 commits July 14, 2023 13:09

Update tests/test_visualization.py

b3c8920

Merge branch 'wkb-decoding' into wkb-decoding-annotation-ext

7dc0e30

address comments

56ca0f6

Merge branch 'develop' of https://github.com/TissueImageAnalytics/tia…

2cb1921

…toolbox into wkb-decoding