Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simply datetime values (timestamps) by moving to RFC3339 #469

Open
bernhardreiter opened this issue Mar 2, 2022 · 14 comments
Open

Simply datetime values (timestamps) by moving to RFC3339 #469

bernhardreiter opened this issue Mar 2, 2022 · 14 comments
Assignees
Labels
csaf 2.1 csaf 2.1 work editor-revision already worked on in the editor revision test related to tests in this repo

Comments

@bernhardreiter
Copy link
Contributor

CSAF-2.0 has ISO 8601 as normative reference.
Allowing a subset of this, specially RFC3339 with the further restriction of only using UTC,
makes the standard easier to parse and write.

Because CSAF is an exchange format for automation in the security context,
there does not seem to be a case for timezones or all iso 8601 variants.

https://datatracker.ietf.org/doc/html/rfc3339 says

The following profile of ISO 8601 [ISO8601] dates SHOULD be used in
   new protocols on the Internet.  

and

some sites may wish to emit times in UTC only.

so the ABNF should be used with this line changed to:

 time-offset     = "Z"
@bernhardreiter
Copy link
Contributor Author

Another argument for using RFC3339 is ROLIE, as RFC3339 is the normative reference there.

@sthagen
Copy link
Contributor

sthagen commented Mar 2, 2022

I am (for now) not in favor of this proposal.

There are no easy choices here as only "different" but not "less severe" problems await.
I learned from discussions in the converter project that RFC3339 cannot accept 20220302T123456Z (needing colons and dashes.

That alone renders the that RFC as normative reference "replacing" ISO-8601 quite useless.

I remember we arrived at the "silent no zone means Z decision" within the TC for CVRF v1.2 as many documents in the field had no time zones at all and this only "shifted" (bend) times but did not "invalidate complete documents".

@sthagen sthagen self-assigned this Mar 2, 2022
@bernhardreiter
Copy link
Contributor Author

bernhardreiter commented Mar 2, 2022

that RFC3339 cannot accept 20220302T123456Z (needing colons and dashes.)

This is correct (from my reading).

But what is the use case for 20220302T123456Z CSAF-2.0? A converter can easily rewrite the values.

The same with a missing 'Z', if the converter is to assume a local time and does the conversion, it is fine, but I wouldn't leave the ambiguity open in a new standard.

It is much better to do this once in the conversion step than to make all reader implementations cope with the extra complexity unnecessarily.

@bernhardreiter
Copy link
Contributor Author

Did more reading: the JSON schemas already refer to date-time, which is RFC 3339 according to
https://json-schema.org/draft/2020-12/json-schema-validation.html
so most CSAF-2.0 will already be using RFC3339. So this issue is about

  • doing this more clearly in the document and remove ISO 8601 as normative reference and replace it with RFC 3339
  • doing a further simplification by only allowing UTC as indicated by an ending "Z" with out time-numoffset

@tschmidtb51
Copy link
Contributor

I'm happy to change ISO 8601 to RFC3339 as we implicit state that through "format": "date-time" with in the schema.

I'm not in favor of only allowing UTC. Some programming languages seem to have problems with the Z and use explicit time-numoffset. Moreover, every tool would need to have an extra check for that as it would not be covered by the JSON schema. IMHO - the burdens outweigh the benefits at this time - especially as we would deviate from RFC3339.

tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Mar 2, 2022
- resolves oasis-tcs#469
- remove reference to ISO 8601
- add reference to RFC3339
@sthagen
Copy link
Contributor

sthagen commented Mar 2, 2022

In case we later ask OASIS to submit CSAF v2.0 to ISO it is not convincing for me to remove ISO-8601. The converter arguments seem weak, as we could just use milliseconds since epoch and trust converters to do it right. The usecase of colon free timestamps is always the slugness when using as parts of filenames and URL parts.

In summary I think we are too late in the process for v2.0 to preform this change.

tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Mar 2, 2022
- addresses parts of oasis-tcs#469
- add informative reference to RFC3339
@tschmidtb51 tschmidtb51 mentioned this issue Mar 2, 2022
@bernhardreiter
Copy link
Contributor Author

The usecase of colon free timestamps is always the slugness when using as parts of filenames and URL parts.

As filename parts a special format maybe preferable, but then ISO 8601 is still having way to many possibilities.
If a format suitable for good filenames is needed, then it IMO probably should be one that has only one variant, very close to RFC3339 with UTC only.

@sthagen
Copy link
Contributor

sthagen commented Mar 2, 2022

My position is clear, I think. Time rules humanity not the other way around - for that we are good enough with ISO-8601.

We are already very good on the parsing of the temporal data. Any complications that remain are due to the difficulty of a multitude of producers and their preferences and capabilities.

Personally, I do not really perceive the simplification promise from that opinionated RFC.

Lastly, ROLIE is just one option from many and the JSON schema mostly a shape sketch - I would not use these details as pretext to have the tail wag the dog

In a later version - maybe v3.0 we may have more experience with the ecosystem to decide based on evidence what is best to describe our temporal aspects - it should in no case be that some specific implementations rule the standard for easing, as the standard should serve the whole system of participants.

@bernhardreiter
Copy link
Contributor Author

bernhardreiter commented Apr 7, 2022

Time rules humanity not the other way around - for that we are good enough with ISO-8601.

The CSAF standard is for automation, such as code that can run exact and with a minimum of potential variants. Using only one variant of datetimes with UTC, is adhering to the "minimal principle" of data format design (If you want a longer background read on this principle, here is an article about this I wrote many years ago: https://fsfe.org/freesoftware/standards/minimalisticstandards.en.html )

Some programming languages seem to have problems with the Z and use explicit time-numoffset.

(Which programming languages did you see with this problem? Is this a reading or a writing problem? Most general purpose language do have an RFC3339 parser in a internal or external library available and writing dates with a format string is also something I see almost eyerywhere.)

Moreover, every tool would need to have an extra check for that as it would not be covered by the JSON schema.

Thought about it, this is an argument with weight, because if using a simpler data format here will complicate things elsewhere, it would not be helpful.

Using the UTC variant does not seems to create major problems for implementations, though:

  • It could be solved on the reading side by just using an existing RFC3339 parser, which will accept the UTC variant, so there is no code difference or complication. And if the format has to be parsed manually, it is less complicated code.
  • On the writing side, the format is mostly directly assembled and it is quite easy to do so, so again this keeps the code simple.
  • The the specification side, exchanging the ABNF rule in one point is clear and simple enough.
  • The only issue maybe potential schemas for a specification, but once defined it should be fine (I haven't checked if JSON schemas already have a predefined RFC3339 type).

From my view the recommendation ist: use the UTC subset of RFC3339, unless the timeshift value is really needed by a compelling use case.

@sthagen
Copy link
Contributor

sthagen commented Apr 7, 2022

When the web was young, we all found it refreshingly easy to parse ISO-8601 - the real one, coming along in real data - and not some human only targeting local display of time we encountered before.

Automating successfully - in my experience - rarely means we bend reality, but instead find some minimal algorithmic "Bending" to deal with real data. Nice try, but ...

Real people, filling in temporal data may do so by hand using a simple text editor and then often prefer local times. Not every software is produced in a UTC culture constantly triggering mental excercises to remove the cognitive dissonance (esp. in regions not as privileged as Europe where it often is just subtraction/addition of 1 or 2 hours depending on the temperature 😉).

Converters can always act along the chain when automation is important and the community operates under a transitive trust model (allowing an intermediate node to "refactor" a signed document).

To me, the CSAF specification has to offer a good user experience also in producing and consuming - not to maximize use cases in the middle of some possible chain. We already have a two-level compliance framework:

  1. the approximate shape checking JSON Schema (already often very noisy because we can only use available ready made constraints and formulations thereof)
  2. the additional tests we describe in dedicated sections of the prose specification and provide counter examples for

I imagine and hope that many open source CSAF documents will be created by hand and stored as well as downloaded from github, gitlab, sourcehut, and other platforms once to their final destination.

No intermediate automation involved.

In summary: I see the value of a temporal format spec on our abstraction level in providing support for cooperative exchange to use only the most simple codification. We even offer categorial data fields as free text to maximize use and accessibility.

Propose to close with no action.

@bernhardreiter
Copy link
Contributor Author

bernhardreiter commented Apr 8, 2022

(Part of the main action has already been taken if I understand correctly. I've only put forward more arguments between using RFC3339 plain allowing timeshifts or the RFC3339 subset that only allows UTC.)

As for cognitive load:

  • When local time would be allowed in all its ISO-8601 variants, the point it time would not be clearly defined, that would mean losing information that is potentially useful.
  • Between the RFC3339 variants, when doing it as human, less variants are also better.

RFC3339 with only UTC allowed

  • Writing: Need to know the current timezone difference and do the calculation.
  • Reading: also, know the current offset and do one calculation to get the info to my local time

RFC3339 with timezone

  • For writing with timezone, you also need to know the current timezone difference, but no need for the calculation
  • For reading you need to know the current offset and know you must do two calculations (difference of offsets and then applying it to the time).

I think reading will happen more often than writing, the RFC 3339 with UTC variant is posing least mental work on humans.

@tschmidtb51
Copy link
Contributor

I suggest to clarify what we expect to see in a date-time.

tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Oct 25, 2024
- addresses parts of oasis-tcs#469
- add section on Date and Time to explain the rules
@tschmidtb51 tschmidtb51 mentioned this issue Oct 25, 2024
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Oct 25, 2024
- addresses parts of oasis-tcs#469
- specify . as frac-sec separator
- explicitly state no empty timezones
- add reference to ABNF of RFC 3339 section 5.6
@tschmidtb51
Copy link
Contributor

tschmidtb51 commented Oct 25, 2024

  • add text
  • make RFC 3339 and ISO 8601 normative
  • add tests (see below)

tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Oct 25, 2024
- addresses parts of oasis-tcs#469
- move RFC 3339 and ISO 8601 to normative references
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Oct 25, 2024
- addresses parts of oasis-tcs#469
- correct format
- add filename to etc/bind.txt
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Oct 25, 2024
- addresses parts of oasis-tcs#469
- link section in conformance targets
@tschmidtb51 tschmidtb51 added the csaf 2.1 csaf 2.1 work label Nov 19, 2024
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Nov 23, 2024
- addresses parts of oasis-tcs#469
- add links to standards
- add rules regarding separator
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Nov 23, 2024
- addresses parts of oasis-tcs#469
- add mandatory test to check date-time rules
- add invalid example
- add valid example
@tschmidtb51
Copy link
Contributor

  • check all values
  • check all rules

tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Nov 23, 2024
- addresses parts of oasis-tcs#469
- exclude schema test failing testfile
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Dec 2, 2024
- addresses parts of oasis-tcs#469
- add links to standards
- add rules regarding separator
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Dec 2, 2024
- addresses parts of oasis-tcs#469
- add mandatory test to check date-time rules
- add invalid example
- add valid example
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Dec 2, 2024
- addresses parts of oasis-tcs#469
- exclude schema test failing testfile
tschmidtb51 added a commit to tschmidtb51/csaf that referenced this issue Dec 2, 2024
- addresses parts of oasis-tcs#469
- add test into bind.txt
@tschmidtb51 tschmidtb51 added test related to tests in this repo editor-revision already worked on in the editor revision labels Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
csaf 2.1 csaf 2.1 work editor-revision already worked on in the editor revision test related to tests in this repo
Projects
None yet
Development

No branches or pull requests

3 participants