Proposal to enable support for alternate schemas

[darrelmiller]

After reading the updated documentation for examples in PR 636 I realized we may be able to use the exact same approach for schemas.

As I understand it, in OpenAPI.vNext, examples come in one of three flavours:

• A JSON object, which infers the payload is some media type that is based on JSON.
• A string, which infers the contents of the example string comply with the declared media type of the response.
• A $ref which points to a file, that contains a sequence of bytes that correspond to the media type of the response.

The documentation for examples states that if an example is a string, there is no guarantee that tooling will understand the media type of the example and be able to validate it.

I'm proposing that we use the same mechanism to allow the Schema property to describe different schema types. We would introduce the ability to assign a string primitive to the schema object, or use a $ref to point to a file containing a schema. A new optional peer property called schematype is needed to identify the format of schema value. Schematype values should be standardized by the OpenAPI specification using some kind of registry of known values, however, there is no requirement for tooling to support all schema types to be compliant with the OpenAPI specification.

Users of OpenAPI who wish to continue using only JSON and JSON Schema to describe their APIs can do so with no changes. Those who wish to use Avro #466, or JSON Content Rules to describe their JSON can use a string or $ref schema value. Using schemas like XSD, Relax-NG, Schematron, Kwalify for YAML, or ABNF for parameter values becomes possible.

Those who do wish to support non-JSON content get the same mechanism for both Schemas and Examples.

Building on the "representations/content" proposal , a header object could be described like this:

{
   "headers": {
       "my-user-agent" : {
          "content"  : {
             "text/plain" : {
                     "schematype" : "HTTP-ABNF",
                     "schema" : "product *( RWS ( product / comment ) )",
                     "examples" : [
                                  "foo/v1.0 (Yo! like my app?) OS/2.0"
                          ]
                   }
           }
      }
  }
}

This also brings another interesting possibility. Consider the following path parameter.

{
  "name": "username",
  "in": "path",
  "description": "username to fetch",
  "required": true,
  "content": {
    "text/plain": {
       "schematype":"rfc6570",
       "schema" : ";username*" 
    }
  }
}

This approach, in effect, enables all of the URI template goodness. Matrix parameters, path segment parameters, unencoded parameters. I'm not suggesting this is the best way to bring RFC 6570 support to OpenAPI, but it is the least intrusive way to do it.

And one just one more thing... For those people who want to use "pure" JSON Schema, for whatever is the current definition of pure, so that external tools can validate the schemas, then they could choose to use this syntax for schemas instead of the Open API flavoured JSON Schema.

[ePaul]

I somehow fear that by opening the way to allow all kinds of schemas, and in the same time do not require implementations to actually support any of them, we get to the situation that someone can say "My API has an OpenAPI 3.0 compliant definition" but no one can then actually use this API definition with most tools which implement this.

Thus we'll get several dialects of OpenAPI, like "OpenAPI 3.0 with just standard OpenAPI schemas", "OpenAPI with RelaxNG", etc. I'm not sure this is what we want.

On the other hand, maybe this will lead to tooling to become more modularized, so you can e.g. plug in support for specific schema types in Swagger-Codegen.

[dilipkrish]

Seems weird for a parameter to have a content especially for path/query params. Sorry I haven't had much time to keep up with the changes but parameters today has one of (type|schema). Are we thinking of unifying/pluralizing that to just schemas perhaps?

for e.g.

{
  "name": "username",
  "in": "path",
  "description": "username to fetch",
  "required": true,
  "schemas": {
    "text/plain": {
       "schematype":"rfc6570",
       "schema" : ";username*" 
    }
  }
}

[darrelmiller]

@dilipkrish Currently OpenAPI 3.0 does not support type anymore for parameters, you must use "schema" for primitive types. See #762 for more info.

[darrelmiller]

@ePaul I completely understand and share your concern. However, it seems to me to be the lesser of the evils.

[fehguy]

I'm pretty sure using arbitrary references to schemas will mean, no tooling will support them. I think that is a much worse evil and render the specification pretty much a try-it-out-only tool.

[darrelmiller]

@fehguy Fair enough. I will withdraw my proposal :-)

[jharmn]

IMO we overlooked a simple option on this. If we force a reference to an external schema file, then the OpenAPI doc is still parseable, and references to external schema could be resolved when needed.
This would fulfill many requests to handle multiple schema types, and decouple the parsing of OpenAPI from JSON Schema, along with other schema types.

{
  "name": "username",
  "in": "path",
  "description": "username to fetch",
  "required": true,
  "content": {
    "application/xml": {
       "schematype":"XSD",
       "schemaRef" : "schema/username.xsd" 
    }
  }
}

Perhaps JSON Schema could still be supported in-line, but any other schema should be external file refs only. I've intentionally avoided $ref, as that has specific usage in JSON parsing that would break when referencing non-JSON formats.

[darrelmiller]

@jharmn If schematype is a limited fixed enum, then maybe that would address @fehguy 's concerns about lack of tooling support.

The only downside to requiring things to be defined externally is that now we sort of need a standard packaging format for exporting OpenAPI definitions because they could be a set of files.

[olensmar]

just my 5 cents; I've talked to several teams transitioning from SOAP to REST but wanting to keep the XML-Schemas in which they've modeled their domains, allowing them to reference an external existing schema would definitely help in this scenario as JSON Schema is not a viable alternative in most cases

[fehguy]

this will leave us with probably zero tools that support the full spec. Protobuf parsing? XSD? It'll be a mess

[darrelmiller]

@fehguy Assuming documentation tooling would take these "foreign" payload schemas and render them as text embedded into the documentation. What other tooling uses payload schemas?

I'm guessing client generation tooling wouldn't be able to generate strongly typed models for payloads, but they should be able to treat them as a stream of bytes payload.

I'm just trying to understand the scope of what would break.

[fehguy]

If we say that you can put a schema in for definition or validation, the expectation will be that the tooling will honor it. Take a look at the issues raised for XML in swagger-ui, for example--once you "support it" then your tool sucks if it doesn't work.

I see literally no value in putting the reference as a schema, if tooling doesn't do anything with it. If we just want it as a reference doc, then we can label it as an externalDoc or something other than 'this response adheres to this schema', which is setting the expectation that it is an enforced constraint.

[fehguy]

@webron to chime in on support for form data (still waiting...)

[DavidBiesack]

There is much more than "no value". There is descriptive value for both consumers and implementers of the API. Just because tools can't handle it yet does not mean users can't get value, even if they have to code it themselves (until the tools catch up). My API that consumes XML can use the schema to validate a request body at runtime (though I would not use the Open API doc; I'm more likely to use a cached copy of the reference schema), and clients know what schema to use in case they want to use it (with other tools) to generate client code like custom marshallers.

[ghost]

Since json-schema and the subset in openapi is not as robust as xmlscehema and others, I think this is a good idea. Despite some shortcomings I stick with xmlschema because nothing else is close enough to generate and describe more than simple structures.

[olensmar]

I general I don't think it's a good idea to let existing tooling dictate what we put in future versions of the spec - but on the other hand the spec shouldn't be too "academic" for its own good and push tooling vendors away if is too complex to support. In this case I'm stuck in the middle; I understand that creating tooling that supports all kinds of schema languages is unrealistic - but it's equally unrealistic to expect users to translate their existing schemas to JSON Schema to be able to use the spec (which seems to be a common scenario).

Middle ground could be to have an "official extension" - ("x-schema" or something) for this that tooling can optionally support - but that puts this at risk of going down the WS-* route, which is a mess to say the least.

Another possibility could be to have a (very) fixed set of schema types (JSON-Schema and XML-Schema) initially supported in OAS 3.0 - and others to be considered further down the road if a general need arises.

[dilipkrish]

Unrelated to this issue, but related to a class of issues like this one, I like the idea of having a tooling aware advancement of the spec. However, Id like to suggest an auxiliary program, to incubate, incorporate spec extensions people organically come up with, outside of this process, into the fold. Some good examples of this are the influence of googles guava library on idk 8 or the gradle plugin ecosystem.

I'm not sure what that looks like or how to orchestrate something like this, but I'm sure there are numerous extension that solve some of the problems described in issues like this, that vendors/companies are building tooling around their extensions. AWS Api Gateway integration comes to mind as an example. Rather than re-think a solved problem, it would be useful to showcase/vote on such extensions to move into the spec as a first class solution.

[fehguy]

@dilipkrish we did discuss how we handle "experimental" features in the spec, which are quite different from extensions.

@olensmar there are many reasons why we decided to not do this, I suggest reviewing the discussion in the recordings if you want to get caught up. It's not just a tooling issue, and unfortunately it's a little late in the process to chime in. The spec can't be in flux forever

[skjolber]

As a member 'of the industry', I can't believe this

https://github.com/OAI/OpenAPI-Specification/blob/3.0.0-rc0/versions/3.0.md#xml-object

is in your standard and not XML Schema.

I think it is fair to be pragmatic about the adopters of the standard being able to support the full spec, however I cannot see those conserns apply to XML Schema due to the sheer volume of open-source tools in virtually all programming languages.

So my 5 cents is, XML Schema is not 'legacy support', it is there to stay permanently, and in 2017 I have to recommend writing SOAP services over OpenAPI 3.0.

[webron]

If that's your analogy, then you don't understand what OpenAPI is.

[skjolber]

Please, xml-object makes WSDL look good. I think the suggestions within this issue limits the complexity while providing great value to existing XML-based ecosystems.

[DavidBiesack]

I would like to again suggest reconsidering some support for XML .xsd schema files, such as that proposed above, or even allowing a xsd elements for a Content Object (xsd would be mutually exclusive with the Content Object's schema adn SHOULD only be used for XML media types).

For example, we have a Geo/Maps API which is primarily a JSON API, but it supports application/gml+xml for a request body on a POST... I have zero interest in converting those schema to annotated JSON schema.We don't care that the existing tools don't process this.I would rather be able to correctly specify the operation and its XML schema.

[darrelmiller]

@DavidBiesack I believe it is high on everyone's priority list, but I believe it can be added as an additive feature (i.e. non-breaking) and I would really rather not try and rush a solution because we want to squeeze it into the 3.0 release timeframe.

[DavidBiesack]

good to hear @darrelmiller . When I presented OpenAPI 3.0.0-rc1 at TRI REST meetup, XML schema was one of the not-there-yet enhancements the audience most wanted.

[ioggstream]

@darrelmiller should we close this as duplicate of #1532 ?