[WIP] Multispectral support

[michaelni]

This is not to merge yet, but for discussion and of course if anyone wants to contribute/add to it

[JeromeMartinez]

color_transfer_characteristic is common to YUV (+colour_primaries, matrix_coefficients), shouldn't it be in a color info dedicated part e.g. at the end of the configuration box?

[michaelni]

hmm, the idea was that we need something to go from physical energy to binary values and that often is not linear. Maybe color_transfer_characteristic is a poor term for this.

[JeromeMartinez]

transfer_characteristic wording would be misleading as the name is already used elsewhere (prefix 'color' would not be enough for differentiation IMO), so in that case it should be another name.

[pjotrek-b]

• color_translation?
• color_conversion?
• color_interpretation?

possibly add _characteristic to any of those?

[retokromer]

On my side, I made good experiences definitying the wavelength (in nm) for each spectral band.

[michaelni]

On my side, I made good experiences definitying the wavelength (in nm) for each spectral band.

I used hz in the hope that the whole relevant EM spectrum could be represented and as its a fundamental unit. (quite possibly ive missed something of course). with nm we would need fractional values and not just integers if we wanted to store X-ray images and use this field to hold some acccurate X-ray frequency information

[retokromer]

I’m afraid, I’m probably missing something! It seems to me that the integers vs. fractions issue is the same when expressed in Hz or in m, which are actually both SI base units. (Context: I made that choice, because it’s the information characterising the filters we use… Anyway, the conversion between them is elementary.)

[michaelni]

If one wanted to use integers to handle the value, with Hz that works for most of the relevant spectrum with also very good precision for the range one is able to make pictures with. visual, UV, ir, xrays are all high frequency so lots of digits precission. With really long radio waves making images will be very hard as the antenna has to scale up size wise. OTOH with meters as the base unit most of the relevant spectrum falls below 1.0 so fractional numbers need to be handled. Also with wavelength we would need to clarify if that is wavelength in air, in vacuum or in the medium the picture was taken.
But ultimately it of course makes no real difference either way can be used to store the value.
In practice maybe we should use a mantisse * 2^exponent style storage as fractions as in a/b would result in values for either a or b not fitting in 64bit integers. Ill try to come up with some better suggestion in a future patch revission

[retokromer]

nit Hz

[pjotrek-b]

Sorry for jumping in (3 years+ later) 😄 :
Does any of you have any particular real-world examples in mind (or access to them)?
Like the xrays, long radio wave images, air/vacuum/etc recordings @michaelni mentioned?

I'm wondering if it'd make sense not to raw-parametrize this, but rather define/offer presets which can be tuned in the code, rather than as arguments? Then access these interpretation options by a given name shortcut?
(If I understood the subject alright)

I assume that any area of expertise that would create such recordings will very likely have some sets of "common parameters" - therefore domain-specific "insider terms" for those imaging types.

Like FFmpeg having "dvd_pal" containing a set of sane/valid parameters, we could have something like "xray_10nm" or "10nm-water-hot". Dunno. Something like that?

Wouldn't this also allow arbitrary choice of data-types (per imaging-type) for proper calculations?
Integers, fractions, etc - "different strokes for different (imaging type) folks" - so to say 😜

If I'm right, then this is tricky to handle in a Standards paper definition... But if x264 can be a standard and do presets, so could ffv1, I presume? 😉

[michaelni]

If you need samples there should be lots of public stuff from astronomy and satellites. For xrays and MRT the word to search for is probably dicom image. One source for public domain samples would be any images you own the rights to, like from a medical exam. I didnt really try to find free proper raw sample images for these things previously ...

[retokromer]

Just out of curiosity: is there any work going on here? (I will be in Vienna from Thursday to Sunday and could meet if useful. Then again the week 6–10 January.)

[kieranjol]

I was at a pronom workshop recently and there were some Dicom images. Perhaps pronom have these images available as part of a test suite?

…

On Tue 25 Oct 2022 at 23:31, Michael Niedermayer ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In ffv1.md <#138 (comment)>: > +| 4 | Retro-Reflective | +| 5 | Mask | +| 6 | Depth | +| 7 | Motion | +| 8 | Error | +| 9 | Force Field | +| 10 | Polarization | +| 11-999 | Reserved | + +### per_plane_metadata_count + +`per_plane_metadata_count` allows future extension of the information we store about planes + +### center_frequency + +`center_frequency` indicates the center frequency in hz. Or 0 if unknown or not applicable. If you need samples there should be lots of public stuff from astronomy and satellites. For xrays and MRT the word to search for is probably dicom image. One source for public domain samples would be any images you own the rights to, like from a medical exam. I didnt really try to find free proper raw sample images for these things previously ... — Reply to this email directly, view it on GitHub <#138 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAITFPSJIQUYEXON2SJ556TWFBGS5ANCNFSM4GPFEWBA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[retokromer]

FYI: I will present on this topic again in a larger context at NTTW6.