TG2-AMENDMENT_GEOGRAPHY_STANDARDIZED

[godfoder]

TestField	Value
GUID	ba2d868c-afa3-409a-836b-fdcea9f75945
Label	AMENDMENT_GEOGRAPHY_STANDARDIZED
Description	Propose amendment to one or more of the values dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality using bdq:sourceAuthority.
TestType	Amendment
Darwin Core Class	Location
Information Elements ActedUpon	dwc:continent
	dwc:country
	dwc:countryCode
	dwc:stateProvince
	dwc:county
	dwc:municipality
Information Elements Consulted
Expected Response	EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available or if the combination of the values of dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality could not be unambiguously resolved in the bdq:sourceAuthority; AMENDED if the values of dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality could be unambiguously interpreted from values in the bdq:sourceAuthority; otherwise NOT_AMENDED.
Data Quality Dimension	Conformance
Term-Actions	GEOGRAPHY_STANDARDIZED
Parameter(s)	bdq:sourceAuthority
Source Authority	bdq:sourceAuthority default = "The Getty Thesaurus of Geographic Names (TGN)" [https://www.getty.edu/research/tools/vocabularies/tgn/index.html]
Specification Last Updated	2023-09-18
Examples	[dwc:continent="", dwc:country="Australia", dwc:countryCode="", dwc:stateProvince="WA", dwc:county="", dwc:municipality="": Response.status=AMENDED, Response.result=dwc:continent="Australia", dwc:country="Australia", dwc:countryCode="AU", dwc:stateProvince="Western Australia", dwc:county="", dwc:municipality="", Response.comment="The state of WA in Australia was amended to Western Australia and the country code added"]
	[dwc:continent="", dwc:country="Aus", dwc:countryCode="", dwc:stateProvince="WA", dwc:county="", dwc:municipality="": Response.status=NOT_AMENDED, Response.result="", Response.comment="The information provided was ambiguous and changes were made"]
Source	VertNet, Kurator
References	Chapman, AD and Wieczorek, JR (2020). Georeferencing Best Practices. Copenhagen: GBIF Secretariat (https://doi.org/10.15468/doc-gg7h-s853) Vertnet (2019). DwCVocabs (https://github.com/VertNet/DwCVocabs/blob/master/vocabs/Geography.csv) Getty Reseach Institute (2017). Getty Thesaurus of Geographic Names Online (https://www.getty.edu/research/tools/vocabularies/tgn/index.html)
Example Implementations (Mechanisms)	Kurator
Link to Specification Source Code	https://github.com/VertNet/toolkit, https://github.com/kurator-org/kurator-validation/blob/master/packages/kurator_dwca/workflows/dwca_geography_cleaner.yaml
Notes	The terms up the hierarchy may be able to be filled, but if a term used is deprecated in the hierarchy in the lookup, terms down the hierarchy as well can be filled. A Parameter setting the Geography Source Authority must be made otherwise the default is The Getty Thesaurus of Geographic Names (TGN). This test is not recommended be implemented because of one or more of the following criteria: Unavailable vocabularies; available vocabularies are ambiguous; too difficult to code; too complex to currently implement; implementation could lead to ambiguous or inaccurate results.

[godfoder]

This needs to be worked on with #95 further before becoming a core test.

[ArthurChapman]

This is the only test that we have an AMENDMENT_...STANDARDIZED without a corresponding VALIDATION..._NOTSTANDARD. In this case we decided it was too difficult to implement the VALIDATION. However, does this mean we need some sort of prerequisite to say that one or more of the administrative geographic terms (dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality) is not standard?

[Tasilee]

We do appear to need at least one equivalent validation. John?

[chicoreus]

I concur a matching validation/problem is needed. The Kurator workflow listed in link to source code is actually an implementation of such a problem finder. There is a Kurator workflow that matches this test as currently phrased as an amendment: https://github.com/kurator-org/kurator-validation/blob/master/packages/kurator_dwca/workflows/dwca_geography_cleaner.yaml
(I've updated the issue to reflect this, and referenced the assessor workflow in the Validation #139

[chicoreus]

I've added a matching validation/problem for discussion as #139. I'm not convinced by the discussion we had in Gainesville that it is too difficult to implement a matching validation, as both validation and amendment have implementations in the wild (in VertNet, and code from VertNet ported into Kurator).

[chicoreus]

Excluding waterBody and including Continent raises a host of issues. The current text references "administrative geographic terms", but dwc:continent isn't a administrative/political term in the same way that the other listed terms (dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality) are. Material from offshore but within some country's EEZ would be expected to have a value for dwc:country but none for dwc:continent, and would be distinguished from non-marine material from the same country by the values of waterBody and continent. As an amendment, this test could all to easily incorrectly propose filling in continent in such a case. Also, some places have multiple competing opinions about continental placement. VertNet places Hawaii in Oceania, but the Getty Thesaurus places it in North and Central America, with an alternative placement in Oceania. http://www.getty.edu/vow/TGNFullDisplay?find=Hawaii&place=&nation=&prev_page=1&english=Y&subjectid=7007249 We might be safer limiting this core test to just country to municipality terms, and having another test to look at water bodies, islands, etc. We skipped marine hierarchies in Gainesville, but that is something the OBIS team should probably weigh in on.

[Tasilee]

Thanks @chicoreus. I bow to those of us with greater experience. "Target authority" here is significant.

[ArthurChapman]

Looks like you have a way of validating @chicoreus. I think we could go ahead with #139 unless @tucotuco has some further objections. And agreed with waterbodies and marine

[tucotuco]

I suggest the Expected Response:

'EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available or if the combination of the values of dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality could not be unambiguously resolved from the bdq:sourceAuthority; AMENDED if the values of dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality could be unambiguously interpreted from values in bdq:sourceAuthority; otherwise NOT_AMENDED'

in place of:

'EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available or if the combination of the values of dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality could not be unambiguously resolved from the bdq:sourceAuthority; AMENDED the values of dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality if could be unambiguously interpreted from values in bdq:sourceAuthority; otherwise NOT_AMENDED'

[chicoreus]

78640f09-8353-411a-800e-9b6d498fb1c9 duplicates #95, replacing with ba2d868c-afa3-409a-836b-fdcea9f75945

[Tasilee]

@tucotuco: Seems to me that there is redundancy. Why not simply

'EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; AMENDED if the values of dwc:continent, dwc:country, dwc:countryCode, dwc:stateProvince, dwc:county, dwc:municipality could be unambiguously interpreted from values in bdq:sourceAuthority; otherwise NOT_AMENDED'

?

If the ER as you had it was not able to be ambiguously resolved against the source authority, you would have EXTERNAL_PREREQUISITES_NOT_MET, which doesn't seem appropriate given the context of other tests.

[tucotuco]

@Tasilee I like this simplification. If we can resolve #95 to give us the answer about ambiguity, the output of that test could simply be checked to know if this one would be able to AMEND.

[Tasilee]

The zoom discussion with @ArthurChapman, @tucotuco and @chicoreus today concluded that tests #95, #139 and #118 were going to be very difficult to implement properly given the lack of a consistent geographic terms hierarchy by comparison with the taxonomic terms. Note the issues arising from the table above for example. We will therefore remove these tests from CORE.

In their place, we will

1. Add a test for dwc:stateProvice found to complement TG2-VALIDATION_COUNTRY_FOUND #21 (which we will rename)
2. Add a test for dwc:country dwc:stateProvince combo exist at least once in the bdq:sourceAuthority (country-state/province consistent)
3. Add a test for dwc:country dwc:stateProvince combo exists exactly once in the bdq:sourceAuthority ((country-state/province unambiguous)

[chicoreus]

Changed Field to TestField, added ActedUpon/Consulted, added date specification last modified.

[ArthurChapman]

Changed "Output Type" to TestType

[ArthurChapman]

@Tasilee - I thought this was a DO NOT IMPLEMENT rather than Supplementary given our definitions.

[Tasilee]

Specifications updated to align with the current template