The SPECIAL_FEATURES, CUSTOMISATION and ANOMALY field are unconstrained

[matdon17]

In the Argo metadata fields SPECIAL_FEATURES, CUSTOMISATION and ANOMALY the population is entirely free-text and useful of entries varies considerably. Can we, and should we, better constrain these fields to make them more consistent and more useful?

Un-defined acronyms widely used, and the fields SPECIAL_FEATURES and CUSTOMISATION in particular are used interchangeably

SPECIAL_FEATURES examples:
‘Buoyancy diagnostics are being placed in the auxiliary directory, as the reporting of such data has not been standardized.’ (this came up a lot, should someone be working on standardisation of buoyancy diagnostic data? It doesn’t really sound like a ‘special feature’ to me though… More like a general note but they didn’t have anywhere better to put it)
‘This float has transmitted auxiliary CTD profile data as part of its ice-algorithm firmware. This data is of limited depth range and is considered technical data. As such it is being placed in the auxiliary directory. Other limited profile data may also b’ (exact quote, I assume there’s a character limit)
‘DPF, Time of Day profile control, Near Surface Temperature’
‘Ice Sensing Algorithm’
‘Park and profile, deep profile first, time of day profile control, ice detection and evasion’

CUSTOMISATION examples:
Apf9iSbdSbe41cp
Apf9iSbe41CPStsPalCF/Float equipped with Seabird Sea Surface Temp & Salinity and Acoustic Rain Gauge (Passive Acoustic Listener)
Small Piston
DPF, SST
DPF, SST, NST

ANOMALY examples:
‘reached profile 200 -> now a surface drifter’
‘n/a’

How do we approach improving this situation?

[vpaba]

This may be of interest to you @mbelbeoch, @anthoninlize, @roswri

[mbelbeoch]

I agree we need to survey these fields, and make proposal for a controlled vocabulary, if there is any useful content.
We will ask a global export to Coriolis and start to work on it.

[vturpin]

I have requested a review of the "SPECIAL_FEATURE" to the GDAC. We will see what will come out of this.

What is the objective of better constraining this ?
Can we use special feature to indicate if the float goes in the aux\ repository ? for exemple when a sensor is on test ?

I understand the un-clarity of SPECIAL_FEATURE, ANOMALY, CUSTOMIZATION. But I don't understand exactly what we want to do with it.

[vturpin]

The Extraction of the field "SPECIAL REQUEST" since 2020, (about 1900 float with non null value) gave the following results:

42 differents value of "SPECIAL REQUEST" that could be reduced in the following 15 themes:

• Hull caracteristics
• Ice invoidance
• Grouding
• Buoyancy
• Depth profil first
• Experimental sensor
• Polar Ocean profiling system
• Time of day profile control
• Air pump energy consumption limit
• Auxiliary directory
• Near Surface Temperature
• ID (no idea what it means)
• SST
• Park and profile
• Pressure activation

[mscanderbeg]

Victor, thanks for looking at the SPECIAL_FEATURES string - I've been wondering what is actually in there.

My understanding of the SPECIAL_FEATURES string is that it is used to alert users that an experimental sensor is on the float and that data for that sensor exists in the AUX directory. This is critical for notifying countries if a float with some special sensor wanders into their EEZ. This page has been updated with examples of strings to include in the SPECIAL_FEATURES string: https://argo.ucsd.edu/data/auxiliary-directory/
I think it would be helpful to be able to quickly find out the parameter measured by the experimental sensor.
I also think it would be useful if a controlled vocabulary could be developed around the 'time of day' issue. For example, it will become important to easily determine which floats are programmed to come up at a certain time of day in order to remove them from some types of analyses.
From the User Manual, examples for SPECIAL_FEATURES also include ice avoidance algorithm. Again, it would be nice to quickly determine which floats have ice avoidance algorithms on them, so a controlled vocab around this may be helpful as well.

I think ANOMALY is there to describe any anomalies or problems the float had. It also seems to be where the ENDING_CAUSE and ENDING_CAUSE_CATEGORY go which looks to be a constrained field.

CUSTOMISATION appears to be used to describe changes made to the float after purchase and before deployment. I could see overlap here with SPECIAL_FEATURES if an extra sensor is added.

[vturpin]

Here is what the user manual says exactly about SPECIAL_FEATURES:
"This vocabulary captures the possible SPECIAL_FEATURES in a float’s characteristics. Additional float features can be specified here such as algorithms used by the float. The SPECIAL_FEATURES variable in a float metadata file must be filled by the DAC, to indicate there are AUX data files (see https://argo.ucsd.edu/data/auxiliary-directory)"

So the current purpose of SPECIAL FEATURE is mutliple :

1. Notify the special variables (Experimental sensors)
2. Notify the special algorithms (Ice avoidance, Grounding, time of day)
3. Notify the special float caracteristics (6000m hull)
   On my view, this is too broad for one metadata if we want to constrain it.

Here is what the user manual says about CUSTOMIZATION:
"Free form field to record changes made to the float after manufacture and before deployment, i.e. this could be the customisation institution plus a list of modifications."
Indeed there is an overlap between SPECIAL_FEATURES and CUSTOMIZATION for sensors as it is not stated that experimental sensor should go in SPECIAL_FEATURES.

Here is what the user manual says about ANOMALY:
"This field describes any anomaly or problem the float may have had"
ANOMALY is fine as soon as ENDING_CAUSE and ENDING_CAUSE_CATEGORY is excluded from it.

What's our need exactly ? Here is a suggestion in order of importance:

1. Automatically find out the paramters measured by experimental senors (for a proper implmentation of the IOC resolution).
2. Automatically determine which floats have special algorithm on them (this cover Ice avoindance, time surfacing, grouding)
   2-bis) keep track of ANOMALY and monitor ENDING_CAUSE properly.
3. Keep track of any specifiv change of a float after purchasing that can not fall under controlled vocab of 1) and 2).

What is your opinion ?

[vturpin]

I found this discussion related to the same topic.
#32