Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Be more clear about format support (3.1.1) #3726

Merged
merged 4 commits into from
Apr 25, 2024

Conversation

handrews
Copy link
Member

This came up in conversation with @karenetheridge .

Most 2020-12 validators do not validate it, which is a regular point of confusion worth highlighting.

If accepted, I will port this to 3.2.0. It is not directly relevant to 3.0.4, as format was handled differently in that version of JSON Schema - still confusing and often not supporting useful validation, but it would require a different explanation.

Most 2020-12 validators do not validate it, which is a regular point
of confusion worth highlighting.
@handrews handrews added this to the v3.1.1 milestone Apr 20, 2024
@handrews handrews requested a review from a team April 20, 2024 19:11
@handrews handrews added the clarification requests to clarify, but not change, part of the spec label Apr 20, 2024
versions/3.1.1.md Outdated Show resolved Hide resolved
Copy link
Contributor

@mikekistler mikekistler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should be adding observations about tooling to the specification, particularly when this may change in the future. What will we do in 5 years if all the common validators have corrected their behavior to handle format correctly? Can we at that time change old versions of the spec to remove these observations, or do we leave them in? And if we leave them, won't that be confusing to readers?

@@ -146,7 +146,8 @@ Data types in the OAS are based on the types supported by the [JSON Schema Speci
Note that `integer` as a type is also supported and is defined as a JSON number without a fraction or exponent part.
Models are defined using the [Schema Object](#schemaObject), which is a superset of JSON Schema Specification Draft 2020-12.

<a name="dataTypeFormat"></a>As defined by the [JSON Schema Validation vocabulary](https://tools.ietf.org/html/draft-bhutton-json-schema-validation-00#section-7.3), data types can have an optional modifier property: `format`.
<a name="dataTypeFormat"></a>As defined by the [JSON Schema Validation specification](https://tools.ietf.org/html/draft-bhutton-json-schema-validation-00#section-7.3), data types can have an optional modifier property: `format`. Note that by default, JSON Schema draft 2020-12 validators _do not_ validate `format`, but instead treat it as an annotation. Support for `format` validation, either in best-effort form with the [default `format-annotation` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.1), or with the less-commonly-implemented [`format-assertion` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.2), varies among JSON Schema implementations.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<a name="dataTypeFormat"></a>As defined by the [JSON Schema Validation specification](https://tools.ietf.org/html/draft-bhutton-json-schema-validation-00#section-7.3), data types can have an optional modifier property: `format`. Note that by default, JSON Schema draft 2020-12 validators _do not_ validate `format`, but instead treat it as an annotation. Support for `format` validation, either in best-effort form with the [default `format-annotation` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.1), or with the less-commonly-implemented [`format-assertion` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.2), varies among JSON Schema implementations.
<a name="dataTypeFormat"></a>As defined by the [JSON Schema Validation specification](https://tools.ietf.org/html/draft-bhutton-json-schema-validation-00#section-7.3), data types can have an optional modifier property: `format`. Note that by default, JSON Schema draft 2020-12 validators _do not_ validate `format`, but instead treat it as an annotation. Support for `format` validation, either in best-effort form with the [default `format-annotation` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.1), or with the less-commonly-implemented [`format-assertion` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.2), varies among JSON Schema validator implementations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand the JSON Schema specification correctly, the OpenAPI 3.1 schema declares that it is based on https://json-schema.org/draft/2020-12/schema, which declares that format is just an annotation and not an assertion and can be ignored by validators:

    "https://json-schema.org/draft/2020-12/vocab/format-annotation": true,

So ignoring format is correct behavior for validators and explicitly allowed by how OpenAPI 3.1 uses JSON Schema.

As long as we do not change that part of OpenAPI 3.1, calling out that format is more of a comment and less of an instruction will also remain valid.

@handrews is my interpretation correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ralfhandl Yes, that's correct. Perhaps it might be better to change the wording a bit as the wording I have here makes it sound like it's a tooling problem. It's actually that the spec changed between draft-04 and 2020-12 regarding how format is expected to be handled. Although it's slightly more complicated as there are two ways to get format validation: one is enabling a "best effort" validation mode that matches what has been historically implemented for the keyword, which can be done with the format-annotation vocabulary, but is done by the person running validation and can't be controlled by the schema or meta-schema. The other is the strict validation mode of the format-assertion vocabulary, which is not widely implemented in practice and requires making a meta-schema change that most people don't seem inclined to do (or support).

@mikekistler I think some wording changes here would make it more clear that this is not a transient thing, but the expected result of the specification.

Co-authored-by: Mike Kistler <mikekistler@microsoft.com>
@ralfhandl ralfhandl self-requested a review April 24, 2024 09:44
Copy link
Contributor

@ralfhandl ralfhandl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why we should worry about JSON Schema validators in the OpenAPI specification. The distinction between "annotation" and "assertion" is surely relevant for JSON Schema.

For API descriptions it seems irrelevant, especially as format is prominently mentionend in section Data Types as

OAS defines additional formats to provide fine detail for primitive data types.

If an API description declares that a property has type: 'string', format: 'date', this most likely means that the API implementation will refuse to accept any strings that do not contain a valid date. Ignoring the "fine detail" provided via the format keyword will result in 400 Bad Request.

@handrews
Copy link
Member Author

I wonder why we should worry about JSON Schema validators in the OpenAPI specification.

Because the whole point of establishing JSON Schema compatibility in 3.1 was to offload JSON Schema support to JSON Schema tools. JSON Schema validators are effectively part of the OAS 3.1 tooling ecosystem.

If an API description declares that a property has type: 'string', format: 'date', this most likely means that the API implementation will refuse to accept any strings that do not contain a valid date. Ignoring the "fine detail" provided via the format keyword will result in 400 Bad Request.

In my experience people expect format to actually validate in JSON Schema, get really confused when it doesn't, and consider it a bug in something. If it's in the context of OpenAPI, they tend to blame OpenAPI.

I'll try re-writing this, but while I really try to avoid bloating the specification, I do think that addressing persistent problems is worth a sentence or two.

@handrews
Copy link
Member Author

@ralfhandl @mikekistler I think I have now made this as minimal as possible while still setting user expectations correctly.

@ralfhandl
Copy link
Contributor

Not sure which expectations we are setting here: "Don't use format because it may not be validated"?

@ralfhandl ralfhandl self-requested a review April 25, 2024 09:10
@ralfhandl ralfhandl dismissed their stale review April 25, 2024 09:11

Neither objecting nor agreeing to the change

@handrews
Copy link
Member Author

Not sure which expectations we are setting here: "Don't use format because it may not be validated"?

"Don't expect format to be validated automatically."

It means what it says, no more, no less. Because I've spent nearly 10 years now answering this question, and @karenetheridge has dealt with it plenty as she's implemented both JSON Schema and OAS, and we both thought it would be helpful to head off some of that confusion.

Copy link
Contributor

@lornajane lornajane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good clarification, thanks

Copy link
Contributor

@mikekistler mikekistler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! 👍

@lornajane lornajane merged commit ddbd53f into OAI:v3.1.1-dev Apr 25, 2024
1 check passed
@handrews handrews deleted the format-vocab branch April 25, 2024 19:25
lornajane added a commit that referenced this pull request May 1, 2024
Be more clear about "format" validation behavior (3.2.0 port of #3726)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clarification requests to clarify, but not change, part of the spec schema-object
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants