Make sure Event amendments and validation specifications are consistent

[chicoreus]

We need to make sure that the set of amendments that populate Event terms from each other and the validation that checks them for consistency reflect the current thinking of consistent and compliant.

There is a clause in #52 that needs to be carried into #67 (or removed from #52, but I don't think that's our intent):

(3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year

Contrast with the matching clause in #52:

(1) dwc:year is EMPTY or dwc:eventDate has a precision of one year or finer and the provided value of dwc:year matches the year expressed in dwc:eventDate, and

We aren't specifying here that the dwc:year should only be popluated if dwc:eventDate is within a single year, only that it has a precision of a year or finer. Thus given dwc:eventDate="1981/1982" or dwc:eventDate="1981-12-30/1982-01-03" #52 won't fill in dwc:year, but #67, as currently framed, will treat dwc:year=1982 as consistent.

I think, though I could be wrong, that #52 expresses our desire, as in the table above.

We also probably need to similarly be explicit about month and day. Start/End day of year are different, as we expect them to be poplulated if eventDate is a range, even if it spans more than one year, so long as there is a precision of one day or finer.

Amendments that fill eventDate from other terms:
#86 AMENDMENT_EVENTDATE_FROM_VERBATIM
#93 AMENDMENT_EVENTDATE_FROM_YEARMONTHDAY
#132 AMENDMENT_EVENTDATE_FROM_YEARSTARTDAYOFYEARENDDAYOFYEAR
Amendment that backfills the other terms from eventDate:
#52 AMENDMENT_EVENT_FROM_EVENTDATE
Test that the event terms are consistent:
#67 VALIDATION_EVENT_CONSISTENT

[chicoreus]

Current specifications:

#86 AMENDMENT_EVENTDATE_FROM_VERBATIM

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is not EMPTY or the value of dwc:verbatimEventDate is EMPTY or not unambiguously interpretable as an ISO 8601-1 date; FILLED_IN the value of dwc:eventDate if an unambiguous ISO 8601-1 date was interpreted from dwc:verbatimEventDate; otherwise NOT_AMENDED

#93 AMENDMENT_EVENTDATE_FROM_YEARMONTHDAY

INTERNAL _PREREQUISITES_NOT_MET if dwc:eventDate is not EMPTY or dwc:year is EMPTY or is not interpretable as a valid year; FILLED_IN the value of dwc:eventDate if an ISO 8601-1 date was interpreted from the values in dwc:year, dwc:month and dwc:day; otherwise NOT_AMENDED.

#132 AMENDMENT_EVENTDATE_FROM_YEARSTARTDAYOFYEARENDDAYOFYEAR

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate was not EMPTY or any of dwc:year, dwc:startDayOfYear, or dwc:endDayOfYear were EMPTY or any of the values in dwc:year, dwc:startDayOfYear, or dwc:endDayOfYear were not independently interpretable; FILLED_IN the value of dwc:eventDate from values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear if the value of dwc:startDayOfYear is less than the value of dwc:endDayOfYear; otherwise NOT_AMENDED

#52 AMENDMENT_EVENT_FROM_EVENTDATE

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

#67 VALIDATION_EVENT_CONSISTENT

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY, or all of dwc:year, dwc:month, dwc:day, dwc:startDayOfYear and dwc:endDayOfYear are EMPTY; COMPLIANT if all of the following conditions are met (1) dwc:year is EMPTY or dwc:eventDate has a precision of one year or finer and the provided value of dwc:year matches the year expressed in dwc:eventDate, and (2) dwc:month is EMPTY or dwc:eventDate has a precision of one month or finer and the provided value in dwc:month matches the month represented by dwc:eventDate, and (3) dwc:day is EMPTY or dwc:eventDate has a precision of a day or less and the provided value in dwc:day matches the day represented by dwc:eventDate, and (4) dwc:startDayOfYear is empty or dwc:eventDate has a precision of one day or finer and the provided value in dwc:startDayOfYear matches the start day of the year of the range represented by dwc:eventDate, and (5) dwc:endDayOfYear is empty or dwc:eventDate has a precision of one day or finer and the provided value in dwc:endDayOfYear matches the end day of the year of the range represented by dwc:eventDate; otherwise NOT_COMPLIANT.

[chicoreus]

#86 and #93 probably need a statement about retaining the original precision, e.g. year=1980, month-1 should become eventDate 1980-01, not 1980-01-01/1980-01-31.

#132 is asymmetrical with #52. #132 only populating event date if start day is less than end day, while #52 can fill start day of year with a value greater than end day of year, if the range spans a year boundary. This is by design, to be conservative, as a start day of year larger than end day of year would indicate a range spanning a year boundary, where dwc:year can only hold one of the years.

#52 and #67 aren't consistent in handling dwc:day dwc:month and dwc:year for both reduced precision eventDates and eventDates representing ranges.

[Tasilee]

I agree @chicoreus we need to be consistent. I can't see edits to the Expected responses are needed for #86, #93 or #132. I do agree that we should edit the Notes about retaining original/input precision and @ArthurChapman is working on that now.

Your edit to the Expected response of #67 seems rational.

[ArthurChapman]

My comments on each of these

Current specifications:

#86 AMENDMENT_EVENTDATE_FROM_VERBATIM

The Expected Response wording look OK to me

**Add to Notes: "When running the test, the original precision, e.g. year=1980, month=1 should be retained, e.g. dwc:eventDate should become 1980-01, not 1980-01-01/1980-01-31."

#93 AMENDMENT_EVENTDATE_FROM_YEARMONTHDAY

The Expected Response wording look OK to me

**Add to Notes: "When running the test, the original precision, e.g. dwc:year=1980, dwc:month=1 should be retained, e.g. dwc:eventDate should become 1980-01, not 1980-01-01/1980-01-31."

#132 AMENDMENT_EVENTDATE_FROM_YEARSTARTDAYOFYEARENDDAYOFYEAR

The Expected Response wording look OK to me

Add to Note: ... "... Note that this test is only for cases that fall within the one year (as given in dwc:year) and hence "dwc:startDayOfYear will always be less than dwc:endDayOfYear". [or do we just leave this as being obvious from the Expected Response.]

#52 AMENDMENT_EVENT_FROM_EVENTDATE

The Expected Response wording look OK to me.

#67 VALIDATION_EVENT_CONSISTENT

I don't see that this is inconsistent with #52 - one is a VALIDATION and one an AMENDMENT. The VALIDATIOn is requiring that all conditions are met to be COMPLIENT, where as the AMENDMENT is a series of independent actions (or). To me both are logical and I don't see inconsistencies

I have made one small suggestion in the INTERNAL_PREREQUISITES_NOTMET

Suggest:

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1, or all of dwc:year, dwc:month, dwc:day, dwc:startDayOfYear and dwc:endDayOfYear are EMPTY; COMPLIANT if all of the following conditions are met (1) dwc:year is EMPTY or dwc:eventDate has a precision of one year or finer and the provided value of dwc:year matches the year expressed in dwc:eventDate, and (2) dwc:month is EMPTY or dwc:eventDate has a precision of one month or finer and the provided value in dwc:month matches the month represented by dwc:eventDate, and (3) dwc:day is EMPTY or dwc:eventDate has a precision of a day or less and the provided value in dwc:day matches the day represented by dwc:eventDate, and (4) dwc:startDayOfYear is empty or dwc:eventDate has a precision of one day or finer and the provided value in dwc:startDayOfYear matches the start day of the year of the range represented by dwc:eventDate, and (5) dwc:endDayOfYear is empty or dwc:eventDate has a precision of one day or finer and the provided value in dwc:endDayOfYear matches the end day of the year of the range represented by dwc:eventDate; otherwise NOT_COMPLIANT.

[Tasilee]

Minor suggestions (for #67) to align phrasing with other tests

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1, or all of dwc:year, dwc:month, dwc:day, dwc:startDayOfYear and dwc:endDayOfYear are EMPTY; COMPLIANT if all of the following conditions are met (1) dwc:year is EMPTY or dwc:eventDate has a precision of one year or finer and the value of dwc:year matches the year in dwc:eventDate, and (2) dwc:month is EMPTY or dwc:eventDate has a precision of one month or finer and the value in dwc:month matches the month in dwc:eventDate, and (3) dwc:day is EMPTY or dwc:eventDate has a precision of a day or less and the value in dwc:day matches the day in dwc:eventDate, and (4) dwc:startDayOfYear is EMPTY or dwc:eventDate has a precision of one day or finer and the value in dwc:startDayOfYear matches the start day of the year of the range in dwc:eventDate, and (5) dwc:endDayOfYear is EMPTY or dwc:eventDate has a precision of one day or finer and the value in dwc:endDayOfYear matches the end day of the year of the range in dwc:eventDate; otherwise NOT_COMPLIANT.

[chicoreus]

@ArthurChapman "INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1," We definitely don't want to make that change, that is a responsibility of another validation (#66 VALIDATION_EVENTDATE_STANDARD) . By specifying that here we are entangling the two distinct tests.

@ArthurChapman The inconsistency isn't so much between the tests, but between the tests and our understanding of what is needed for the darwin core terms for year, month, and day to contain compliant values, and thus the tests treating both consistency and compliance. The inconsistency is that the amendment will produce values that the validation will assert are NOT_COMPLIANT, given that the latest proposal for the validation covers the case that dwc:day should not be popluated when dwc:eventDate contains a range of more than one day, per our reading of the darwin core definition, we almost certainly don't want that...

Amendment (current version):

(1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer,

Validation (latest proposal, current with our thinking):

(3) dwc:day is EMPTY or dwc:eventDate has a precision of a day or less and is within a single day and the provided value in dwc:day matches the day represented by dwc:eventDate, and

The amendment would take dwc:eventDate=1982-01-03 and set dwc:day to 3. The validation would assert that this is NOT_COMPLIANT.

In essence, we need:

(1) To be sure that "within a single" applies to year and month as well as day (? @tucotuco )

(2) Change the language of the validation and the amendment to be in sync with the expectations from (1).

The current language probably isn't inconsistent between the amendment and the validation, but the current language isn't up to date with our understanding of what consistent and compliant means for dwc:eventDate, dwc:year, dwc:month. dwc:day, dwc:StartDayOfYear, dwc:endDayOfYear.

[chicoreus]

@Tasilee see the current proposal for #67

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY, or all of dwc:year, dwc:month, dwc:day, dwc:startDayOfYear and dwc:endDayOfYear are EMPTY; COMPLIANT if all of the following conditions are met (1) dwc:year is EMPTY or dwc:eventDate has a precision of one year or finer and and is within a single year and the provided value of dwc:year matches the year expressed in dwc:eventDate, and (2) dwc:month is EMPTY or dwc:eventDate has a precision of one month or finer and is within a single month and the provided value in dwc:month matches the month represented by dwc:eventDate, and (3) dwc:day is EMPTY or dwc:eventDate has a precision of a day or less and is within a single day and the provided value in dwc:day matches the day represented by dwc:eventDate, and (4) dwc:startDayOfYear is empty or dwc:eventDate has a precision of one day or finer and the provided value in dwc:startDayOfYear matches the start day of the year of the range represented by dwc:eventDate, and (5) dwc:endDayOfYear is empty or dwc:eventDate has a precision of one day or finer and the provided value in dwc:endDayOfYear matches the end day of the year of the range represented by dwc:eventDate; otherwise NOT_COMPLIANT.

[ArthurChapman]

Sorry - I am getting lost @chicoreus - you say

The amendment would take dwc:eventDate=1982-01-03 and set dwc:day to 3. The validation would assert that this is NOT_COMPLIANT.

Why would that fail

dwc:eventDate has a precision of a day or less and is within a single day and the provided value in dwc:day matches the day represented by dwc:eventDate

as far as I can see, it has a precision of a day is within a single day (3) and the provided value in dwc:day (3) = 3 in the dwc:eventDate - ?

[chicoreus]

On Sun, 25 Jun 2023 18:31:25 -0700 Arthur Chapman ***@***.***> wrote: The amendment would take dwc:eventDate=1982-01-03 and set dwc:day to 3.

Too late at night... that should be dwc:eventDate=1982-01-03/1982-03-04, which has a precision of 1 day, but represents a range of more than one day (by the reading of precision in ISO dates as yyyy or yyyy/yyyy having a precision or year, yyyy-mm or yyyy-mm/yyyy-mm having a precision of month, and yyyy-mm-dd and yyyy-mm-dd/yyyy-mm-dd and yyyy-ddd and yyyy-ddd/yyyy-ddd having a precision of day. Coming out whether date ranges should poplulate year month and day or not, with John strongly asserting they should not, and the definition of dwc:day being clear that it should not.

[chicoreus]

Discussion in TG2 call. dwc:year, dwc:month, dwc:day should not be populated if the dwc:eventDate is a range with a span longer than the target, thus for an eventDate that is a range spanning more than a month boundary, dwc:month should not be popluated. So an amendment should not populate month if event date spans a month boundary, and the validation should fail if dwc:month contains a value and the eventDate spans a month boundary.

Thus:

#52 AMENDMENT_EVENT_FROM_EVENTDATE

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

And:

#67 VALIDATION_EVENT_CONSISTENT

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY, or all of dwc:year, dwc:month, dwc:day, dwc:startDayOfYear and dwc:endDayOfYear are EMPTY; COMPLIANT if all of the following conditions are met (1) dwc:year is EMPTY or dwc:eventDate has a precision of one year or finer and and is within a single year and the provided value of dwc:year matches the year expressed in dwc:eventDate, and (2) dwc:month is EMPTY or dwc:eventDate has a precision of one month or finer and is within a single month and the provided value in dwc:month matches the month represented by dwc:eventDate, and (3) dwc:day is EMPTY or dwc:eventDate has a precision of a day or less and is within a single day and the provided value in dwc:day matches the day represented by dwc:eventDate, and (4) dwc:startDayOfYear is empty or dwc:eventDate has a precision of one day or finer and the provided value in dwc:startDayOfYear matches the start day of the year of the range represented by dwc:eventDate, and (5) dwc:endDayOfYear is empty or dwc:eventDate has a precision of one day or finer and the provided value in dwc:endDayOfYear matches the end day of the year of the range represented by dwc:eventDate; otherwise NOT_COMPLIANT.

Further discussion, accept @ArthurChapman s notes on the issues.