Use of write_multiscale_labels for image #107
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 tasks
|
This feature has now been released as part of |
will-moore
changed the title
will-moore
force-pushed
the
use_write_multiscale_labels
branch
from
2c41b9f to
6ebc1f3
Compare
sbesson
requested changes
Jun 15, 2022
There was a problem hiding this comment.
Tested using the same set of samples as the ones generated in #116
To compare the output, I used the following minimal script which should compare arrays and flag difference in attributes
import zarr
import sys
z1 = zarr.open(sys.argv[1])
z2 = zarr.open(sys.argv[2])
if z1.attrs != z2.attrs:
print("Different top-level attributes")
assert len(z1) == len(z2)
for a in z1.arrays():
key = a[0]
assert (z1[key][:] == z2[key][:]).all()
if z1['labels'].attrs != z2['labels'].attrs:
print("Different labels attributes")
for g in z1['labels'].groups():
label_name = g[0]
if z1['labels'][label_name].attrs != z2['labels'][label_name].attrs:
print("Different image-label attributes")
assert len(z1['labels'][label_name]) == len(z2['labels'][label_name])
for a in z1['labels'][label_name].arrays():
key = a[0]
assert (z1['labels'][label_name][key][:] == z2['labels'][label_name][key][:]).all()The execution of this script reveals no difference in the binary data content when comparing the arrays but a few mismatches in the attributes
(conversion) [sbesson@pilot-zarr1-dev omero-cli-zarr_116]$ for i in *.zarr; do echo $i && python diff.py $i ../omero-cli-zarr_107/$i; done
10501752.zarr
Different top-level attributes
Different image-label attributes
4496763.zarr
Different top-level attributes
Different image-label attributes
5514375.zarr
Different top-level attributes
Different image-label attributes
Different image-label attributes
6001247.zarr
Different top-level attributes
Different image-label attributes
The top-level attribute difference comes from the version of the creation tool and is expected
(conversion) [sbesson@pilot-zarr1-dev omero-cli-zarr_116]$ for i in *.zarr; do diff $i/.zattrs ../omero-cli-zarr_107/$i/.zattrs; done
4c4
< "version": "0.3.1.dev33+gf305941"
---
> "version": "0.3.1.dev40+gcd84f02"
4c4
< "version": "0.3.1.dev33+gf305941"
---
> "version": "0.3.1.dev40+gcd84f02"
4c4
< "version": "0.3.1.dev33+gf305941"
---
> "version": "0.3.1.dev40+gcd84f02"
4c4
< "version": "0.3.1.dev33+gf305941"
---
> "version": "0.3.1.dev40+gcd84f02"For the attribute of the image-label layer, the new code adds the name of the multiscales but the version in the image-label specification seems to be lost and should be restored
(conversion) [sbesson@pilot-zarr1-dev omero-cli-zarr_116]$ find * -mindepth 2 -iname .zattrs -exec diff ../omero-cli-zarr_107/{} {} \;
23991c23991,23992
< }
---
> },
> "version": "0.4"
24076d24076
< "name": "0",
6029c6029,6030
< }
---
> },
> "version": "0.4"
6152d6152
< "name": "0",
609c609,610
< }
---
> },
> "version": "0.4"
700d700
< "name": "Cell",
609c609,610
< }
---
> },
> "version": "0.4"
700d700
< "name": "Chromosomes",
681c681,682
< }
---
> },
> "version": "0.4"
764d764
< "name": "0",I will also have a go at testing a plate labels export
|
Unfortunately, my test of plate labels export failes using sample plates from The export of a single image with non overlapping polygons works as expected Barring the plate label export workflow, the only remaining issue is the loss of the |
|
ome/ome-zarr-py#206 proposes to address the regression mentioned in #107 (review) by updating the underlying API to write the version metadata. |
sbesson
requested changes
Jun 20, 2022
Co-authored-by: Sébastien Besson <seb.besson@gmail.com>
|
@sbesson Thanks for releasing ome-zarr-py 0.5.0 and bumping the requirements here. |
|
I am retesting the PR with the data regeneration and validation process described above. Once this is successful, yes I'd propose to merge and release |
|
I would like to update as many of the sample OME-NGFF datasets as possible, since that allows viewing the labels on the web (zarr.js). I guess it might be useful to keep a sample as 64-bit, in case someone wants to test with it, but in general I'd like to replace existing labels, since having both is confusing / redundant. Happy to help with regeneration/update etc. |
sbesson
approved these changes
Jun 21, 2022
There was a problem hiding this comment.
Retested the conversion workflow and confirmed that the arrays are identical and the only metadata difference are 1- the creator version 2- the addition of image names
(conversion) [sbesson@pilot-zarr1-dev omero-cli-zarr_107]$ for i in *.zarr; do echo $i && python diff.py $i ../omero-cli-zarr_116/$i; done
10501752.zarr
Different top-level attributes
Different image-label attributes
4496763.zarr
Different top-level attributes
Different image-label attributes
5514375.zarr
Different top-level attributes
Different image-label attributes
Different image-label attributes
6001247.zarr
Different top-level attributes
Different image-label attributes
(conversion) [sbesson@pilot-zarr1-dev omero-cli-zarr_107]$ for i in *.zarr; do diff $i/.zattrs ../omero-cli-zarr_116/$i/.zattrs; done
4c4
< "version": "0.3.1.dev41+g5c73169"
---
> "version": "0.3.1.dev33+gf305941"
4c4
< "version": "0.3.1.dev41+g5c73169"
---
> "version": "0.3.1.dev33+gf305941"
4c4
< "version": "0.3.1.dev41+g5c73169"
---
> "version": "0.3.1.dev33+gf305941"
4c4
< "version": "0.3.1.dev41+g5c73169"
---
> "version": "0.3.1.dev33+gf305941"
(conversion) [sbesson@pilot-zarr1-dev omero-cli-zarr_107]$ find * -mindepth 2 -iname .zattrs -exec diff ../omero-cli-zarr_116/{} {} \;
24076a24077
> "name": "0",
6152a6153
> "name": "0",
700a701
> "name": "Cell",
700a701
> "name": "Chromosomes",
764a765
> "name": "0",
and confirmed that the dtype of the labels is still int8 or int16 as per #107
(conversion) [sbesson@pilot-zarr1-dev omero-cli-zarr_107]$ find * -mindepth 3 -iname .zarray -exec grep dtype {} \;
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "<i2",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
"dtype": "|i1",
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Testing
write_multiscale_labels()added in ome/ome-zarr-py#178