Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XContent format Versioning #11184

Closed
colings86 opened this issue May 15, 2015 · 9 comments
Closed

XContent format Versioning #11184

colings86 opened this issue May 15, 2015 · 9 comments
Labels
:Core/Infra/REST API REST infrastructure and utilities >feature high hanging fruit Team:Core/Infra Meta label for core/infra team

Comments

@colings86
Copy link
Contributor

We have the concept of versioning on the transport wire format which allows us to make changes to the product whilst maintaining backwards compatibility with nodes on previous versions (but still on the same major version). We currently have no such versioning for the REST request/response format. For the request format we tend to add backward compatibility support in the Parsers to parse the old parameters as well as the new ones (these tend to be jumbled together with the new parameters with no clear indication they are deprecated formats still there for bwc), but for the REST Response format we don't have any way of knowing what version of the format the client expects so cannot maintain bwc with changes.

We could add a query parameter to every REST action called format whose value would specify which version the server should expect for request formats and which version the client expects for response formats. This would allow us to have the same kind of bwc logic that we have for the transport layer (if the format parameter is missing then it will default of the server's version or possibly to trying all available formats).

We could then also have a new API which would return the XContentTypes and format versions supported by the server. Clients could then use this API to determine whether they are compatible with the server on client initialisation and work out which version they need to specify in the format parameter.

This would be a big and long term change and certainly would not happen overnight as we would have to slowly introduce the bwc code in the XContent parsers and renderers.

@colings86 colings86 added discuss :Core/Infra/REST API REST infrastructure and utilities labels May 15, 2015
@jpountz
Copy link
Contributor

jpountz commented May 15, 2015

I agree we need such a thing if we want to be able to change the response formats across major versions.

@nik9000
Copy link
Member

nik9000 commented May 15, 2015

We could add a query parameter to every REST action called format whose value would specify which version the server should expect for request formats and which version the client expects for response formats.

MediaWiki uses formatversion for something similar to this. It feels to me like exactly the kind of this that accept headers were built for though. Either way I'm +1 on the concept .

@colings86
Copy link
Contributor Author

We discussed this in FixItFriday and we decided that for now, as breaking changes in the REST API response are rare we will support a legacy flag in the query parameters which indicates which feature should respond in the legacy format e.g. ?legacy=aggs,completion. The feature can then check in the toXContent() for the legacy flag if it has a legacy format and then respond appropriately

@rjernst
Copy link
Member

rjernst commented Jun 3, 2016

While I understand the need, I am concerned with the semantics. When do we ever remove the "legacy" format if we dont know what version(s) that format was associated with? What if there are two breaking changes that occur "close" together (say within a major release)? While that may be unlikely, not accounting for it could be a real problem if it ever did happen. Unless what we mean is "use the previous major version format" which assumes we do not break the rest api response format within a major version (which I think we already try to follow that rule)?

@rjernst
Copy link
Member

rjernst commented Jun 3, 2016

Also, why can it not be as simple as a boolean? I think if you want the "legacy" format (whatever that means) the entire response should be in that format, not just pieces?

@clintongormley
Copy link
Contributor

Unless what we mean is "use the previous major version format" which assumes we do not break the rest api response format within a major version (which I think we already try to follow that rule)?

Yes, exactly.

I think if you want the "legacy" format (whatever that means) the entire response should be in that format, not just pieces?

Maybe, not sure... We're not promising that the response syntax will be exactly the same as the previous major version, just that we'll revert a major breaking change. Also, it may be that somebody migrates their queries, but hasn't had time to do aggs yet. not sure

@nik9000
Copy link
Member

nik9000 commented Mar 12, 2018

@colings86 is this still a thing you think we should do? We've done fairly well holding breaking changes until the next major release but I expect there are major changes that we might want to do that'd be quite difficult without this.

@colings86
Copy link
Contributor Author

@nik9000 yes it's something I still think we should do. It's true that we have been (and still are) able to hold breaking changes to the request side of the REST API for major versions and have been able to deprecate old API parameters there etc. But the impetus for this change for me was actually more on the response side where we currently have no way of making any breaking changes in a user friendly way because there is no way for the user to transition to a new response format before upgrading to the version where the breaking change is made. For example see #17652. This change would need to move sub aggregations to their own part of the aggregation response, but there is no way for us to make this change at the moment because we have no way for the client to specify whether they need to read the old or the new format of thee response

@thecoop
Copy link
Member

thecoop commented Oct 25, 2024

This is possible using REST API versioning - on a major version upgrade, you can choose to continue to use the previous REST API that should be compatible for requests and responses

@thecoop thecoop closed this as completed Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/REST API REST infrastructure and utilities >feature high hanging fruit Team:Core/Infra Meta label for core/infra team
Projects
None yet
Development

No branches or pull requests

7 participants