-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.bs
1851 lines (1489 loc) · 78.1 KB
/
index.bs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<pre class='metadata'>
Title: Next-generation file formats (NGFF)
Shortname: ome-ngff
Level: 1
Status: LS-COMMIT
Status: w3c/ED
Group: ome
URL: https://ngff.openmicroscopy.org/latest/
Repository: https://github.com/ome/ngff
Issue Tracking: Forums https://forum.image.sc/tag/ome-ngff
Logo: http://www.openmicroscopy.org/img/logos/ome-logomark.svg
Local Boilerplate: header yes
Local Boilerplate: copyright yes
Boilerplate: style-darkmode off
Markup Shorthands: markdown yes
Editor: Josh Moore, University of Dundee (UoD) https://www.dundee.ac.uk, https://orcid.org/0000-0003-4028-811X
Editor: Sébastien Besson, University of Dundee (UoD) https://www.dundee.ac.uk, https://orcid.org/0000-0001-8783-1429
Editor: Constantin Pape, European Molecular Biology Laboratory (EMBL) https://www.embl.org/sites/heidelberg/, https://orcid.org/0000-0001-6562-7187
Editor: John Bogovic, Hughes Medical Institute Janelia (HHMI) https://www.janelia.org/, https://orcid.org/0000-0002-4829-9457
Text Macro: NGFFVERSION 0.5-dev
Abstract: This document contains next-generation file format (NGFF)
Abstract: specifications for storing bioimaging data in the cloud.
Abstract: All specifications are submitted to the https://image.sc community for review.
Status Text: The current released version of this specification is
Status Text: <a href="../0.4/index.html">0.4</a>. Migration scripts
Status Text: will be provided between numbered versions. Data written with these latest changes
Status Text: (an "editor's draft") will not necessarily be supported.
</pre>
Introduction {#intro}
=====================
Bioimaging science is at a crossroads. Currently, the drive to acquire more,
larger, preciser spatial measurements is unfortunately at odds with our ability
to structure and share those measurements with others. During a global pandemic
more than ever, we believe fervently that global, collaborative discovery as
opposed to the post-publication, "data-on-request" mode of operation is the
path forward. Bioimaging data should be shareable via open and commercial cloud
resources without the need to download entire datasets.
At the moment, that is not the norm. The plethora of data formats produced by
imaging systems are ill-suited to remote sharing. Individual scientists
typically lack the infrastructure they need to host these data themselves. When
they acquire images from elsewhere, time-consuming translations and data
cleaning are needed to interpret findings. Those same costs are multiplied when
gathering data into online repositories where curator time can be the limiting
factor before publication is possible. Without a common effort, each lab or
resource is left building the tools they need and maintaining that
infrastructure often without dedicated funding.
This document defines a specification for bioimaging data to make it possible
to enable the conversion of proprietary formats into a common, cloud-ready one.
Such next-generation file formats layout data so that individual portions, or
"chunks", of large data are reference-able eliminating the need to download
entire datasets.
Why "<dfn export="true"><abbr title="Next-generation file-format">NGFF</abbr></dfn>"? {#why-ngff}
-------------------------------------------------------------------------------------------------
A short description of what is needed for an imaging format is "a hierarchy
of n-dimensional (dense) arrays with metadata". This combination of features
is certainly provided by <dfn export="true"><abbr title="Hierarchical Data Format 5">HDF5</abbr></dfn>
from the <a href="https://www.hdfgroup.org">HDF Group</a>, which a number of
bioimaging formats do use. HDF5 and other larger binary structures, however,
are ill-suited for storage in the cloud where accessing individual chunks
of data by name rather than seeking through a large file is at the heart of
parallelization.
As a result, a number of formats have been developed more recently which provide
the basic data structure of an HDF5 file, but do so in a more cloud-friendly way.
In the [PyData](https://pydata.org/) community, the Zarr [[zarr]] format was developed
for easily storing collections of [NumPy](https://numpy.org/) arrays. In the
[ImageJ](https://imagej.net/) community, N5 [[n5]] was developed to work around
the limitations of HDF5 ("N5" was originally short for "Not-HDF5").
Both of these formats permit storing individual chunks of data either locally in
separate files or in cloud-based object stores as separate keys.
A [current effort](https://zarr-specs.readthedocs.io/en/core-protocol-v3.0-dev/protocol/core/v3.0.html)
is underway to unify the two similar specifications to provide a single binary
specification. The editor's draft will soon be entering a [request for comments (RFC)](https://github.com/zarr-developers/zarr-specs/issues/101) phase with the goal of having a first version early in 2021. As that
process comes to an end, this document will be updated.
OME-NGFF {#ome-ngff}
--------------------
The conventions and specifications defined in this document are designed to
enable next-generation file formats to represent the same bioimaging data
that can be represented in \[OME-TIFF](http://www.openmicroscopy.org/ome-files/)
and beyond. However, the conventions will also be usable by HDF5 and other sufficiently advanced
binary containers. Eventually, we hope, the moniker "next-generation" will no longer be
applicable, and this will simply be the most efficient, common, and useful representation
of bioimaging data, whether during acquisition or sharing in the cloud.
Note: The following text makes use of OME-Zarr [[ome-zarr-py]], the current prototype implementation,
for all examples.
Document conventions
--------------------
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”,
“RECOMMENDED”, “MAY”, and “OPTIONAL” are to be interpreted as described in
[RFC 2119](https://tools.ietf.org/html/rfc2119).
<p>
<dfn>Transitional</dfn> metadata is added to the specification with the
intention of removing it in the future. Implementations may be expected (MUST) or
encouraged (SHOULD) to support the reading of the data, but writing will usually
be optional (MAY). Examples of transitional metadata include custom additions by
implementations that are later submitted as a formal specification. (See [[#bf2raw]])
</p>
Some of the JSON examples in this document include commments. However, these are only for
clarity purposes and comments MUST NOT be included in JSON objects.
On-disk (or in-cloud) layout {#on-disk}
=======================================
An overview of the layout of an OME-Zarr fileset should make
understanding the following metadata sections easier. The hierarchy
is represented here as it would appear locally but could equally
be stored on a web server to be accessed via HTTP or in object storage
like S3 or GCS.
OME-Zarr is an implementation of the OME-NGFF specification using the Zarr
format. Arrays MUST be defined and stored in a hierarchical organization as
defined by the
[version 2 of the Zarr specification ](https://zarr.readthedocs.io/en/stable/spec/v2.html).
OME-NGFF metadata MUST be stored as attributes in the corresponding Zarr
groups.
Images {#image-layout}
----------------------
The following layout describes the expected Zarr hierarchy for images with
multiple levels of resolutions and optionally associated labels.
See [[#multiscale-md]] for details.
For this example we assume an image with 5 dimensions and axes called `t,c,z,y,x`.
<pre>
. # Root folder, potentially in S3,
│ # with a flat list of images by image ID.
│
├── 123.zarr # One image (id=123) converted to Zarr.
│
└── 456.zarr # Another image (id=456) converted to Zarr.
│
├── .zgroup # Each image is a Zarr group, or a folder, of other groups and arrays.
├── .zattrs # Group level attributes are stored in the .zattrs file and include
│ # "multiscales" and "omero" (see below). In addition, the group level attributes
│ # must also contain "_ARRAY_DIMENSIONS" if this group directly contains multi-scale arrays.
│
├── 0 # Each multiscale level is stored as a separate Zarr array,
│ ... # which is a folder containing chunk files which compose the array.
├── n # The name of the array is arbitrary with the ordering defined by
│ │ # by the "multiscales" metadata, but is often a sequence starting at 0.
│ │
│ ├── .zarray
│ │
│ │
│ └─ t # Chunks are stored with the nested directory layout.
│ └─ c # All but the last chunk element are stored as directories.
│ └─ z # The terminal chunk is a file. Together the directory and file names
│ └─ y # provide the "chunk coordinate" (t, c, z, y, x), where the maximum coordinate
│ └─ x # will be `dimension_size / chunk_size`.
│
└── labels
│
├── .zgroup # The labels group is a container which holds a list of labels to make the objects easily discoverable
│
├── .zattrs # All labels will be listed in `.zattrs` e.g. `{ "labels": [ "original/0" ] }`
│ # Each dimension of the label should be either the same as the
│ # corresponding dimension of the image, or `1` if that dimension of the label
│ # is irrelevant.
│
└── original # Intermediate folders are permitted but not necessary and currently contain no extra metadata.
│
└── 0 # Multiscale, labeled image. The name is unimportant but is registered in the "labels" group above.
├── .zgroup # Zarr Group which is both a multiscaled image as well as a labeled image.
├── .zattrs # Metadata of the related image and as well as display information under the "image-label" key.
│
├── 0 # Each multiscale level is stored as a separate Zarr array, as above, but only integer values
│ ... # are supported.
└── n
</pre>
High-content screening {#hcs-layout}
------------------------------------
The following specification defines the hierarchy for a high-content screening
dataset. Three groups must be defined above the images:
- the group above the images defines the well and MUST implement the
[well specification](#well-md). All images contained in a well are fields
of view of the same well
- the group above the well defines a row of wells
- the group above the well row defines an entire plate i.e. a two-dimensional
collection of wells organized in rows and columns. It MUST implement the
[plate specification](#plate-md)
<pre>
. # Root folder, potentially in S3,
│
└── 5966.zarr # One plate (id=5966) converted to Zarr
├── .zgroup
├── .zattrs # Implements "plate" specification
├── A # First row of the plate
│ ├── .zgroup
│ │
│ ├── 1 # First column of row A
│ │ ├── .zgroup
│ │ ├── .zattrs # Implements "well" specification
│ │ │
│ │ ├── 0 # First field of view of well A1
│ │ │ │
│ │ │ ├── .zgroup
│ │ │ ├── .zattrs # Implements "multiscales", "omero"
│ │ │ ├── 0
│ │ │ │ ... # Resolution levels
│ │ │ ├── n
│ │ │ └── labels # Labels (optional)
│ │ ├── ... # Fields of view
│ │ └── m
│ ├── ... # Columns
│ └── 12
├── ... # Rows
└── H
</pre>
Metadata {#metadata}
====================
The various `.zattrs` files throughout the above array hierarchy may contain metadata
keys as specified below for discovering certain types of data, especially images.
"bioformats2raw.layout" (transitional) {#bf2raw}
------------------------------------------------
[=Transitional=] "bioformats2raw.layout" metadata identifies a group which implicitly describes a series of images.
The need for the collection stems from the common "multi-image file" scenario in microscopy. Parsers like Bio-Formats
define a strict, stable ordering of the images in a single container that can be used to refer to them by other tools.
In order to capture that information within an OME-NGFF dataset, `bioformats2raw` internally introduced a wrapping layer.
The bioformats2raw layout has been added to v0.4 as a transitional specification to specify filesets that already exist
in the wild. An upcoming NGFF specification will replace this layout with explicit metadata.
<h4 id="bf2raw-layout" class="no-toc">Layout</h4>
Typical Zarr layout produced by running `bioformats2raw` on a fileset that contains more than one image (series > 1):
<pre>
series.ome.zarr # One converted fileset from bioformats2raw
├── .zgroup
├── .zattrs # Contains "bioformats2raw.layout" metadata
├── OME # Special group for containing OME metadata
│ ├── .zgroup
│ ├── .zattrs # Contains "series" metadata
│ └── METADATA.ome.xml # OME-XML file stored within the Zarr fileset
├── 0 # First image in the collection
├── 1 # Second image in the collection
└── ...
</pre>
<h4 id="bf2raw-attributes" class="no-toc">Attributes</h4>
The top-level `.zattrs` file must contain the `bioformats2raw.layout` key:
<pre class=include-code>
path: examples/bf2raw/image.json
highlight: json
</pre>
If the top-level group represents a plate, the `bioformats2raw.layout` metadata will be present but
the "plate" key MUST also be present, takes precedence and parsing of such datasets should follow [[#plate-md]]. It is not
possible to mix collections of images with plates at present.
<pre class=include-code>
path: examples/bf2raw/plate.json
highlight: json
</pre>
The `.zattrs` file within the OME group may contain the "series" key:
<pre class=include-code>
path: examples/ome/series-2.json
highlight: json
</pre>
<h4 id="bf2raw-details" class="no-toc">Details</h4>
Conforming groups:
- MUST have the value "3" for the "bioformats2raw.layout" key in their `.zattrs` metadata at the top of the hierarchy;
- SHOULD have OME metadata representing the entire collection of images in a file named "OME/METADATA.ome.xml" which:
- MUST adhere to the OME-XML specification but
- MUST use `<MetadataOnly/>` elements as opposed to `<BinData/>`, `<BinaryOnly/>` or `<TiffData/>`;
- MAY make use of the [minimum specification](https://docs.openmicroscopy.org/ome-model/6.2.2/specifications/minimum.html).
Additionally, the logic for finding the Zarr group for each image follows the following logic:
- If "plate" metadata is present, images MUST be located at the defined location.
- Matching "series" metadata (as described next) SHOULD be provided for tools that are unaware of the "plate" specification.
- If the "OME" Zarr group exists, it:
- MAY contain a "series" attribute. If so:
- "series" MUST be a list of string objects, each of which is a path to an image group.
- The order of the paths MUST match the order of the "Image" elements in "OME/METADATA.ome.xml" if provided.
- If the "series" attribute does not exist and no "plate" is present:
- separate "multiscales" images MUST be stored in consecutively numbered groups starting from 0 (i.e. "0/", "1/", "2/", "3/", ...).
- Every "multiscales" group MUST represent exactly one OME-XML "Image" in the same order as either the series index or the group numbers.
Conforming readers:
- SHOULD make users aware of the presence of more than one image (i.e. SHOULD NOT default to only opening the first image);
- MAY use the "series" attribute in the "OME" group to determine a list of valid groups to display;
- MAY choose to show all images within the collection or offer the user a choice of images, as with <dfn export="true"><abbr title="High-content screening">HCS</abbr></dfn> plates;
- MAY ignore other groups or arrays under the root of the hierarchy.
"coordinateSystems" metadata {#coord-sys-md}
--------------------------
A "coordinate system" is a collection of "axes" / dimensions with a name. Every coordinate system:
- MUST contain the field "name". The value MUST be a non-empty string that is unique among `coordinateSystem`s.
- MUST contain the field "axes", whose value is an array of valid "axes" (see below).
<div class=example>
```json
{
"name" : "volume_micrometers",
"axes" : [
{"name": "z", "type": "space", "unit": "micrometer"},
{"name": "y", "type": "space", "unit": "micrometer"},
{"name": "x", "type": "space", "unit": "micrometer"}
]
}
```
</div>
The order of the `"axes"` list matters and defines the index of each array dimension and coordinates for points in that
coordinate system. For the above example, the `"x"` dimension is the first dimension. The "dimensionality" of a coordinate system
is indicated by the length of its "axes" array. The "volume_micrometers" example coordinate system above is three dimensional (3D).
The axes of a coordinate system (see below) give information about the types, units, and other properties of the coordinate
system's dimensions. Axis `name`s may contain semantically meaningful information, but can be arbitrary. As a result, two
coordinate systems that have identical axes in the same order may not be "the same" in the sense that measurements at the same
point refer to different physical entities and therefore should not be analyzed jointly. Tasks that require images, annotations,
regions of interest, etc., SHOULD ensure that they are in the same coordinate system (same name, with identical axes) or can be
transformed to the same coordinate system before doing analysis. See the example below.
<div class=example>
Two instruments simultaneously image the same sample from two different angles, and the 3D data from both instruments are
calibrated to "micrometer" units. Two samples are collected ("sampleA" and "sampleB"). An analysis of sample A requires
measurements from both instruments' images at certain points in space. Suppose a region of interest (ROI) is determined from the
image obtained from instrument 2, but quantification from that region is needed for instrument 1. Since measurements were
collected at different angles, a measurement by instrument 1 at the point with coordinates (x,y,z) may not correspond to the
measurement at the same point in instrument 2 (i.e., it may not be the same physical location in the sample). To analyze both
images together, they must be in the same coordinate system.
The set of coordinate transformations ([[#trafo-md]]) encodes relationships between coordinate systems, specifically, how to
convert points and images to different coordinate systems. Implementations can apply the coordinate transform to images or
points in coordinate system "sampleA_instrument2" to bring them into the "sampleA_instrument1" coordinate system. In this case,
the ROI should be transformed to the "sampleA_image1" coordinate system, then used for quantification with the instrument 1
image.
```json
"coordinateSystems" : [
{
"name" : "sampleA-instrument1",
"axes" : [
{"name": "z", "type": "space", "unit": "micrometer"},
{"name": "y", "type": "space", "unit": "micrometer"},
{"name": "x", "type": "space", "unit": "micrometer"}
]
},
{
"name" : "sampleA-instrument2",
"axes" : [
{"name": "z", "type": "space", "unit": "micrometer"},
{"name": "y", "type": "space", "unit": "micrometer"},
{"name": "x", "type": "space", "unit": "micrometer"}
]
}
],
"coordinateTransformations": [
{
"type": "affine":
"path": "../sampleA_instrument2-to-instrument1"
"input": "sampleA_instrument2",
"output": "sampleA_instrument1"
}
]
```
</div>
### "axes" metadata
"axes" describes the dimensions of a coordinate systems. It is a list of dictionaries, where each dictionary describes a dimension (axis) and:
- MUST contain the field "name" that gives the name for this dimension. The values MUST be unique across all "name" fields.
- SHOULD contain the field "type". It SHOULD be one of the strings "array", "space", "time", "channel", "coordinate", or
"displacement" but MAY take other string values for custom axis types that are not part of this specification yet.
- MAY contain the field "discrete". The value MUST be a boolean, and is `true` if the axis represents a discrete dimension.
- SHOULD contain the field "unit" to specify the physical unit of this dimension. The value SHOULD be one of the following strings, which are valid units according to UDUNITS-2.
- Units for "space" axes: 'angstrom', 'attometer', 'centimeter', 'decimeter', 'exameter', 'femtometer', 'foot', 'gigameter', 'hectometer', 'inch', 'kilometer', 'megameter', 'meter', 'micrometer', 'mile', 'millimeter', 'nanometer', 'parsec', 'petameter', 'picometer', 'terameter', 'yard', 'yoctometer', 'yottameter', 'zeptometer', 'zettameter'
- Units for "time" axes: 'attosecond', 'centisecond', 'day', 'decisecond', 'exasecond', 'femtosecond', 'gigasecond', 'hectosecond', 'hour', 'kilosecond', 'megasecond', 'microsecond', 'millisecond', 'minute', 'nanosecond', 'petasecond', 'picosecond', 'second', 'terasecond', 'yoctosecond', 'yottasecond', 'zeptosecond', 'zettasecond'
- MAY contain the field "longName". The value MUST be a string, and can provide a longer name or description of an axis and its properties.
If part of [[#multiscale-md]], the length of "axes" MUST be equal to the number of dimensions of the arrays that contain the image data.
<div class=example>
Examples of valid axes:
```json
[
{"name": "x", "type": "space", "unit": "micrometer"},
{"name": "t", "type": "time", "unit": "second", "longName": "Unix Epoch time"},
{"name": "c", "type": "channel", "discrete": true},
{"name": "i0", "type": "array"},
{"name": "c", "type": "coordinate", "discrete" : true },
{"name": "v", "type": "displacement", "discrete": true },
{"name": "freq", "type": "frequency", "unit": "megahertz"}
]
```
</div>
Arrays are inherently discrete (see Array coordinate systems, below) but are often used to store discrete samples of a
continuous variable. The continuous values "in between" discrete samples can be retrieved using an *interpolation* method. If an
axis is continuous (`"discrete" : false`), it indicates that interpolation is well-defined. Axes representing `space` and
`time` are usually continuous. Similarly, joint interpolation across axes is well-defined only for axes of the same `type`. In
contrast, discrete axes (`"discrete" : true`) may be indexed only by integers. Axes of representing a `channel`, `coordinate`,
or `displacement` are usually discrete.
Note: The most common methods for interpolation are "nearest neighbor", "linear", "cubic", and "windowed sinc". Here, we refer
to any method that obtains values at real valued coordinates using discrete samples as an "interpolator". As such, label images
may be interpolated using "nearest neighbor" to obtain labels at points along the continuum.
<div class=example>
For the coordinate system:
```json
{
"name" : "index and interpolation",
"axes" : [
{"name": "t", "type": "time"},
{"name": "c", "type": "channel", "discrete": true},
{"name": "y", "type": "space"},
{"name": "x", "type": "space"}
]
}
```
Indexing an image at the point `(0.1, 0.2, 0.3, 0.4)` is not valid, because the value of the first coordinate (`0.1`) refers
to the discrete axis `"c"`. Indexing an image at the point `(1, 0.2, 0.3, 0.4)` is valid.
</div>
### Array coordinate systems
Every array has a default coordinate system whose parameters need not be explicitly defined. Its name is the path to the array
in the container, its axes have `"type":"array"`, are unitless, and have default "name"s. The ith axis has `"name":"dim_i"`
(these are the same default names used by [xarray](https://docs.xarray.dev/en/stable/user-guide/terminology.html)).
<div class=example>
For example, a 3D array at path `my/data/array` defines the coordinate system:
```json
{
"name" : "my/data/array",
"axes" : [
{"name": "dim_0", "type": "array"},
{"name": "dim_1", "type": "array"},
{"name": "dim_2", "type": "array"}
]
}
```
though this object should not and need not explicitly appear in metadata.
</div>
The dimensionality of each array coordinate system equals the dimensionality of its corresponding zarr array. The axis with
name `"dim_i"` is the ith element of the `"axes"` list. The axes and their order align with the `shape`
attribute in the zarr array attributes (in `.zarray`), and whose data depends on the byte order used to store
chunks. As described in the [zarr array metadata](https://zarr.readthedocs.io/en/stable/spec/v2.html#arrays),
the last dimension of an array in "C" order are stored contiguously on disk or in-memory when directly loaded.
<div class=example>
For example, if `my/data/array/.zarray` contains:
```json
{
"chunks": [ 4, 3, 5 ],
"compressor": null,
"dtype": "|u1",
"fill_value": 0,
"filters": null,
"order": "C",
"shape": [ 4, 3, 5 ],
"zarr_format": 2
}
```
Then `dim_0` has length 4, `dim_1` has length 3, and `dim_2` has length 5.
</div>
The name and axes names MAY be customized by including a `arrayCoordinateSystem` field in
the user-defined attributes of the array whose value is a coordinate system object. The length of
`axes` MUST be equal to the dimensionality. The value of `"type"` for each object in the
axes array MUST equal `"array"`.
<div class=example>
<pre class=include-code>
path: examples/coordSystems/arrayCoordSys.json
highlight: json
</pre>
Note that dimension `i` is contiguous in memory.
</div>
### Coordinate convention
**The pixel/voxel center is the origin of the continuous coordinate system.**
It is vital to consistently define relationship between the discrete/array and continuous/interpolated
coordinate systems. A pixel/voxel is the continuous region (rectangle) that corresponds to a single sample
in the discrete array, i.e., the area corresponding to nearest-neighbor (NN) interpolation of that sample.
The center of a 2d pixel corresponding to the origin `(0,0)` in the discrete array is the origin of the continuous coordinate
system `(0.0, 0.0)` (when the transformation is the identity). The continuous rectangle of the pixel is given by the
half-open interval `[-0.5, 0.5) x [-0.5, 0.5)` (i.e., -0.5 is included, +0.5 is excluded). See chapter 4 and figure 4.1 of the ITK Software Guide [[itk]].
"coordinateTransformations" metadata {#trafo-md}
------------------------------------------------
"coordinateTransformations" describe the mapping between two coordinate systems (defined by "axes").
For example, to map an array's discrete coordinate system to its corresponding physical coordinates.
Coordinate transforms are in the "forward" direction. They represent functions from *points* in the
input space to *points* in the output space.
- MUST contain the field "type".
- MUST contain any other fields required by the given "type" (see table below).
- MUST contain the field "output", unless part of a `sequence` or `inverseOf` (see details).
- MUST contain the field "input", unless part of a `sequence` or `inverseOf` (see details).
- MAY contain the field "name". Its value MUST be unique across all "name" fields for coordinate transformations.
- Parameter values MUST be compatible with input and output space dimensionality (see details).
<table>
<tr><th>`identity`
<td>
<td>The identity transformation is the default transformation and is typically not explicitly defined.
<tr><th>`mapAxis`
<td>`"mapAxis":Dict[String:String]`
<td> A `maxAxis` transformation specifies an axis permutation as a map between axis names.
<tr><th>`translation`
<td> one of: <br>`"translation":List[number]`, <br>`"path":str`
<td>translation vector, stored either as a list of numbers (`"translation"`) or as binary data at a location
in this container (`path`).
<tr><th>`scale`
<td> one of: <br>`"scale":List[number]`, <br>`"path":str`
<td>scale vector, stored either as a list of numbers (`scale`) or as binary data at a location in this
container (`path`).
<tr><th>`affine`
<td> one of: <br>`"affine":List[List[number]]`, <br>`"path":str`
<td>affine transformation matrix stored as a flat array stored either with json uing the affine field
or as binary data at a location in this container (path). If both are present, the binary values at path should be used.
<tr><th>`rotation`
<td> one of: <br>`"rotation":List[number]`, <br>`"path":str`
<td>rotation transformation matrix stored as an array stored either
with json or as binary data at a location in this container (path).
If both are present, the binary parameters at path are used.
<tr><th>`sequence`
<td> `"transformations":List[Transformation]`
<td>A sequence of transformations, Applying the sequence applies the composition of all transforms in the list, in order.
<tr><th>`displacements`
<td>`"path":str`<br>`"interpolation":str`
<td>Displacement field transformation located at (path).
<tr><th>`coordinates`
<td>`"path":str`<br>`"interpolation":str`
<td>Coordinate field transformation located at (path).
<tr><th>`inverseOf`
<td>`"transform":Transform`
<td>The inverse of a transformation. Useful if a transform is not closed-form invertible. See Forward and inverse for details and examples.
<tr><th>`bijection`
<td>`"forward":Transform`<br>`"inverse":Transform`
<td>Explicitly define an invertible transformation by providing a forward transformation and its inverse.
<tr><th>`byDimension`
<td>`"transformations":List[Transformation]`
<td>Define a high dimensional transformation using lower dimensional transformations on subsets of
dimensions.
<thead>
<tr><th>type<th>fields<th>description
</table>
Conforming readers:
- MUST parse `identity`, `scale`, `translation` transformations;
- SHOULD parse `mapAxis`, `affine` transformations;
- SHOULD be able to apply transformations to points;
- SHOULD be able to apply transformations to images;
Coordinate transformations from array to physical coordinates MUST be stored in multiscales ([[#multiscale-md]]),
and MUST be duplicated in the attributes of the zarr array. Transformations between different images MUST be stored in the
attributes of a parent zarr group. For transformations that store data or parameters in a zarr array, those zarr arrays SHOULD
be stored in a zarr group `"coordinateTransformations"`.
<pre>
store.zarr # Root folder of the zarr store
│
├── .zattrs # coordinate transformations describing the relationship between two image coordinate systems
│ # are stored in the attributes of their parent group.
│ # transformations between 'volume' and 'crop' coordinate systems are stored here.
│
├── coordinateTransformations # transformations that use array storage go in a "coordinateTransformations" zarr group.
│ └── displacements # for example, a zarr array containing a displacement field
│ ├── .zattrs
│ └── .zarray
│
├── volume
│ ├── .zattrs # group level attributes (multiscales)
│ └── 0 # a group containing the 0th scale
│ └── image # a zarr array
│ ├── .zattrs # physical coordinate system and transformations here
│ └── .zarray # the array attributes
└── crop
├── .zattrs # group level attributes (multiscales)
└── 0 # a group containing the 0th scale
└── image # a zarr array
├── .zattrs # physical coordinate system and transformations here
└── .zarray # the array attributes
</pre>
### Additional details
Most coordinate transformations MUST specify their input and output coordinate systems using `input` and `output` with a string value
corresponding to the name of a coordinate system. The coordinate system's name may be the path to an array, and therefore may
not appear in the list of coordinate systems.
Exceptions are if the the coordinate transformation appears in the `transformations` list of a `sequence` or is the
`transformation` of an `inverseOf` transformation. In these two cases input and output SHOULD be omitted (see below for
details).
Transformations in the `transformations` list of a `byDimensions` transformation MUST provide `input` and `output` as arrays
of strings corresponding to axis names of the parent transformation's input and output coordinate systems (see below for
details).
<div class=example>
The sequence transformation's input corresponds to an array coordinate system at path "my/array".
```json
"coordinateSystems" : [
{ "name" : "in", "axes" : [{"name" : "j"}, {"name":"i"}] },
{ "name" : "outScale", "axes" : [{"name" : "y"}, {"name":"x"}] },
{ "name" : "outSeq", "axes" : [{"name" : "y"}, {"name":"x"}] },
{ "name" : "outInv", "axes" : [{"name" : "y"}, {"name":"x"}] },
{ "name" : "outByDim", "axes" : [{"name" : "y"}, {"name":"x"}] }
],
"coordinateTransformations" : [
{
"type": "scale",
"input" : "in",
"output" : "outScale",
"scale" : [ 0.5, 1.2 ]
},
{
"type" : "sequence",
"input" : "my/array",
"output" : "outSeq",
"transformations" : [
{ "type": "scale", "scale" : [ 0.5, 0.6 ] },
{ "type": "translation", "translation" : [ 2, 5 ] }
]
},
{
"type": "inverseOf",
"input" : "in",
"output" : "outInv",
"transformation" : {
"type": "displacements",
"path": "path/to/displacements"
}
},
{
"type": "byDimension",
"input" : "in",
"output" : "outDim",
"transformations" : [
{ "type" : "translation", "translation" : [1], "input" : ["i"], "output" : ["x"]},
{ "type" : "scale", "scale" : [2.0], "input" : ["j"], "output" : ["y"]}
]
}
]
```
</div>
Coordinate transformations are functions of *points* in the input space to *points* in the output space. We call this the "forward" direction.
Points are ordered lists of coordinates, where a coordinate is the location/value of that point along its corresponding axis.
The indexes of axis dimensions correspond to indexes into transformation parameter arrays. For example, the scale transformation above
defines the function:
```
x = 0.5 * i
y = 1.2 * j
```
i.e., the mapping from the first input axis to the first output axis is determined by the first scale parameter.
When rendering transformed images and interpolating, implementations may need the "inverse" transformation - from the output to
the input coordinate system. Inverse transformations will not be explicitly specified when they can be computed in closed form
from the forward transformation. Inverse transformations used for image rendering may be specified using the `inverseOf`
transformation type, for example:
```json
{
"type": "inverseOf",
"transformation" : {
"type": "displacements",
"path": "path/to/displacements",
}
}
```
Implementations SHOULD be able to compute and apply the inverse of some coordinate transformations when they
are computable in closed-form (as the [Transformation types](#transformation-types) section below indicates). If an
operation is requested that requires the inverse of a transformation that can not be inverted in closed-form,
implementations MAY estimate an inverse, or MAY output a warning that the requested operation is unsupported.
#### Matrix transformations
Two transformation types ([affine](#affine) and [rotation](#rotation)) are parametrized by matrices. Matrices are applied to
column vectors that represent points in the input coordinate system. The first (last) axis in a coordinate system is the top
(bottom) entry in the column vector. Matrices are stored as two-dimensional arrays, either as json or in a zarr array. When
stored as a 2D zarr array, the first dimension indexes rows and the second dimension indexes columns (e.g., an array of
`"shape":[3,4]` has 3 rows and 4 columns). When stored as a 2D json array, the inner array contains rows (e.g. `[[1,2,3],
[4,5,6]]` has 2 rows and 3 columns).
<div class=example>
For matrix transformations, points in the coordinate system:
```
{ "name" : "in", "axes" : [{"name" : "z"}, {"name" : "y"}, {"name":"x"}] },
```
are represented as column vectors:
```
[z]
[y]
[x]
```
As a result, transforming the point `[z,y,x]=[1,2,3]` with the matrix `[[0,1,0],[-1,0,0],[0,0,-1]]`
results in the point [2,-1,3] because it is computed with the matrix-vector multiplication:
```
[ 0 1 0] [1] [ 2]
[-1 0 0] [2] = [-1]
[ 0 0 -1] [3] [-3]
```
</div>
### Transformation types
Input and output dimensionality may be determined by the value of the "input" and "output" fields, respectively. If the value
of "input" is an array, it's length gives the input dimension, otherwise the length of "axes" for the coordinate
system with the name of the "input" value gives the input dimension. If the value of "input" is an array, it's
length gives the input dimension, otherwise it is given by the length of "axes" for the coordinate system with
the name of the "input". If the value of "output" is an array, its length gives the output dimension,
otherwise it is given by the length of "axes" for the coordinate system with the name of the "output".
#### <a name="identity">identity</a>
`identity` transformations map input coordinates to output coordinates without modification. The position of
the ith axis of the output coordinate system is set to the position of the ith axis of the input coordinate
system. `identity` transformations are invertible.
<div class=example>
<pre class=include-code>
path: examples/transformations/identity.json
highlight: json
</pre>
defines the function:
```
x = i
y = j
```
</div>
#### <a name="mapAxis">mapAxis</a>
`mapAxis` transformations describe axis permutations as a mapping of axis names. Transformations MUST include a `mapAxis` field
whose value is an object, all of whose values are strings. If the object contains `"x":"i"`, then the transform sets the value
of the output coordinate for axis "x" to the value of the coordinate of input axis "i" (think `x = i`). For every axis in its output coordinate
system, the `mapAxis` MUST have a corresponding field. For every value of the object there MUST be an axis of the input
coordinate system with that name. Note that the order of the keys could be reversed.
<div class=example>
<pre class=include-code>
path: examples/transformations/mapAxis1.json
highlight: json
</pre>
The "equivalent to identity" transformation defines the function:
```
x = i
y = j
```
and the "permutation" transformation defines the function
```
x = j
y = i
```
</div>
<div class=example>
<pre class=include-code>
path: examples/transformations/mapAxis2.json
highlight: json
</pre>
The "projection_down" transformation defines the function:
```
x = b
```
and the "projection_up" transformation defines the function:
```
x = a
y = b
z = b
```
</div>
#### <a name="translation">translation</a>
`translation` transformations are special cases of affine transformations. When possible, a
translation transformation should be preferred to its equivalent affine. Input and output dimensionality MUST be
identical and MUST equal the the length of the "translation" array (N). `translation` transformations are
invertible.
<dl>
<dt><strong>path</strong></dt>
<dd> The path to a zarr-array containing the translation parameters.
The array at this path MUST be 1D, and its length MUST be `N`.</dd>
<dt><strong>scale</strong></dt>
<dd> The scale parameters stored as a JSON list of numbers. The list MUST have length `N`.</dd>
</dl>
<div class=example>
<pre class=include-code>
path: examples/transformations/translation.json
highlight: json
</pre>
defines the function:
```
x = i + 9
y = j - 1.42
```
</div>
#### scale
`scale` transformations are special cases of affine transformations. When possible, a scale transformation
SHOULD be preferred to its equivalent affine. Input and output dimensionality MUST be identical and MUST equal
the the length of the "scale" array (N). Values in the `scale` array SHOULD be non-zero; in that case, `scale`
transformations are invertible.
<dl>
<dt><strong>path</strong></dt>
<dd> The path to a zarr-array containing the scale parameters.
The array at this path MUST be 1D, and its length MUST be `N`.</dd>
<dt><strong>scale</strong></dt>
<dd> The scale parameters stored as a JSON list of numbers. The list MUST have length `N`.</dd>
</dl>
<div class=example>
<pre class=include-code>
path: examples/transformations/scale.json
highlight: json
</pre>
defines the function:
```
x = 3.12 * i
y = 2 * j
```
</div>
#### <a name="affine">affine</a>
`affine`s are [matrix transformations](#matrix-transformations) from N-dimensional inputs to M-dimensional outputs are
represented as the upper `(M)x(N+1)` sub-matrix of a `(M+1)x(N+1)` matrix in [homogeneous
coordinates](https://en.wikipedia.org/wiki/Homogeneous_coordinates) (see examples). This transformation type may be (but is not
necessarily) invertible when `N` equals `M`. The matrix MUST be stored as a 2D array either as json or as a zarr array.
<dl>
<dt><strong>path</strong></dt>
<dd> The path to a zarr-array containing the affine parameters.
The array at this path MUST be 2D whose shape MUST be `M x (N+1)`.</dd>
<dt><strong>affine</strong></dt>
<dd> The affine parameters stored in JSON. The matrix MUST be stored as 2D nested array where the outer array MUST be length
`M` and the inner arrays MUST be length `N+1`.</dd> </dl>
<div class=example>
A 2D-2D example:
<pre class=include-code>
path: examples/transformations/affine2d2d.json
highlight: json
</pre>
defines the function:
```
x = 1*i + 2*j + 3
y = 4*i + 5*j + 6
```
it is equivalent to this matrix-vector multiplication in homogeneous coordinates:
```
[ 1 2 3 ][ i ] [ x ]
[ 4 5 6 ][ j ] = [ y ]
[ 0 0 1 ][ 1 ] [ 1 ]
```
where the last row `[0 0 1]` is omitted in the JSON representation.
</div>
<div class=example>
An example with two dimensional inputs and three dimensional outputs.
Note that the order of the axes can in general be determined by the application or user.
These axes relate to the memory or on-disk order insofar as the last dimension is contiguous
when the zarr array is c-order (the default for zarr version 2, and the only option for zarr version 3).
<pre class=include-code>
path: examples/transformations/affine2d3d.json
highlight: json
</pre>
defines the function:
```
x = 1*i + 2*j + 3
y = 4*i + 5*j + 6
z = 7*i + 8*j + 9
```
it is equivalent to this matrix-vector multiplication in homogeneous coordinates:
```
[ 1 2 3 ][ i ] [ x ]
[ 4 5 6 ][ j ] = [ y ]
[ 7 8 9 ][ 1 ] [ z ]
[ 0 0 1 ] [ 1 ]
```
where the last row `[0 0 1]` is omitted in the JSON representation.
</div>
#### <a name="rotation">rotation</a>
`rotation`s are [matrix transformations](#matrix-transformations) that are special cases of affine transformations. When possible, a rotation