Add ADR for decoupling from IETF #1277
Conversation
Co-authored-by: Ethan <133719+notEthan@users.noreply.github.com>
Co-authored-by: Ethan <133719+notEthan@users.noreply.github.com>
Co-authored-by: Ethan <133719+notEthan@users.noreply.github.com>
|
I pushed draft 2. This time I replaced the mention of "governance" with "specification development lifecycle (SDLC)". I don't think "governance" was the right word and its use confuses the purpose of the ADR. |
|
Hold up a bit... I feel like this misrepresents many things. This view seems to assume JSON Schema is only used to standardize validators, but we must also think about the larger standards community: One of the benefits of an RFC is other standards can normatively reference it, and use JSON Schema to define their JSON-based syntaxes. Independently publishing a specification does not permit this.
I'm not sure what this is referring to. We switched from serially numbering the meta-schemas to using dates, but this was unrelated to how the I-D system works. (We never published the meta-schemas through the I-D process, they were non-normative.)
This doesn't have to be true if you don't want it to be. An RFC is not the complete and closed description of everything the system is capable of doing. It's supposed to be able to evolve without updates to the specification text. All an RFC has to do is describe is the essential characteristics of evaluating a JSON Schema: the keywords that everybody is guaranteed to support, and any extension mechanisms.
Expecting use in production would be a strong reason to publish an RFC.
This also seems like a good argument to get the specification published as an RFC.
All of those interactions were with members of the general public. |
|
@awwright prominent members of the JSON working group told us (in email I can no longer find b/c their email software is such a mess) that they would likely throw away the current spec and start over. And that appears to be the sort of thing that is going on with JSON Path. You can't dismiss that as "members of the general public" because these were people who had just published the JSON RFC and have deep knowledge of the considerations involved. I don't think most of us disagree with the idea that it would be great if we published JSON Schema as a proper standards-track RFC. While I understand @jdesrosiers's concerns about evolution, I tend to agree with you that that could probably be sorted out somehow. But I do not see a path for standardizing what we consider to be JSON Schema. People who would have influence have directly told us that there is no such path. The JSON Path process seems to reinforce that. So why do you think we should pursue the IETF? You are asking us to believe that such a process would go in a radically different direction than we have been told by knowledgeable people, and can see in a similar project. I just can't take your recommendation seriously in the face of that. You have not offered any explanation for why you think an IETF working group would go well for us, in the face of significant evidence that it would not. |
The appropriate thing to do here would be to discuss this with downstream projects. Obviously OpenAPI and AsyncAPI, which self-publish, are not going to significantly care. We can talk with the WoT folks for a W3C project. We can talk to FHIR for a government regulatory project (if no one else has a contact with them, I might know someone who can help). We can probably find a few more. Let's get some real feedback from real projects on what they need from us, including what would make them comfortable referencing a non-standards-body-published specification. It may be that they need certain governance assurances, which is where the OpenJS foundation might actually help. It may be that they need to see a way for certain stakeholders to have a seat at the table, or some sort of formal review and input process. We should explore what people actually need, rather than react to assumptions. |
|
Hi @handrews. I'm subscribed to the watch list for the JSON schema spec. I represent the U.S. as part of ANSI (i.e. the American National Standards Institute) and am an expert member of the U.S. National Committee on global standards for the Smart Grid / Power Systems industry. In this capacity I serve on the IEC International Standards body which is an SDO that works closely with other groups such as ISO and IEEE. Work we've undertaken over the last couple years would categorize us as a 'downstream project'. Specifically, we have drafted new data exchange standards that were dependent upon work done as part of draft-2019-09 and later pivot to adjust for draft-2020-12 after later decisions were made with regards to 2019-09. @Relequestual and I have chatted off and on early on at a few points over this time period. As implied by your comments...many standards bodies (including the IEC) lean strongly that any dependent "standards" themselves be stable, normative publications from recognized SDO's (e.g. IETF, W3C, ISO, IEEE, etc.) Originally we were under the auspices that the direction of the JSON schema spec was as an eventual IETF RFC. This generally provides the requisite assurance needed to proceed. At present though we are in a holding pattern. In the interim period we've gone ahead and implemented two early reference implementations against the draft publication. One for potential use for data exchanges amongst E.U. power systems. So to summarize, my feedback is that we absolutely need some form of definitive processes in place to sanction what can be publicly recognized as a "normatively published standard" of the JSON schema specification. This must carry a bit more weight and have more than just the perception of it as a "de facto standard" if you know what I mean. Anyway, thanks for engaging and soliciting outside thoughts/perspectives. I commend the work you and the team have done and understand the at times rather painful and laborious process that drafting standards entails. |
|
@admin-cimug thanks so much for this comment, it's tremendously helpful. I'd like to understand how a couple of possible scenarios might impact you. Currently, @jdesrosiers (from our core team) and @ioggstream (as the spec editor - he is associate with the Italian government so I suspect he has some similar constraints as you) are working on registering the JSON Schema media types through an IETF informational-track RFC. This effort is already under the aegis of the IETF's HTTP API working group. (no one here is opposed to the IETF in general, or thinks that everyone involved with it is hostile to JSON Schema!) Informational-track RFCs are what is used to document de-facto standards in a more official way, but it's not clear to me how well that satisfies your requirements. Here's the most relevant thing RFC 1796 says about informational RFCs:
We are still working out exactly what the boundary between the media type specification and the main specification of JSON Schema is, but it at minimum needs to specify how to identify the version/dialect of JSON Schema, so that an implementation can figure out which document(s) govern further processing. So, questions that arise:
|
|
@admin-cimug a few more questions:
|
I want to make sure this document accurately represents your side of the discussion even if you don't agree with the conclusion. It would not have been merged without your feedback.
Good point. I will add this to the document. And, thank you @admin-cimug for providing an example of when this might be needed.
This is not a meta-schema issue. This is about how we represent releases of JSON Schema. People don't implement draft-handrews-json-schema-01 or draft-bhutton-json-schema-01, they implement draft-07 or 2020-12. We issue our releases as I-Ds, but the details are hidden behind our versioning system because the I-D versions don't fit our needs. For example, we have been releasing patch updates to clarify things or fix bugs. We don't want these non-functional updates to change the JSON Schema version because that would be confusing and unnecessary.
I'll add this perspective to the document. I agree that there is a possible path here, but it means we have to put up with the I-D process and it's incompatibility with our needs until we get there. I don't think we're particularly close to ready for an RFC release.
Not everything that is used in production needs to be recognized by a standards body. I'm not aware of single programming language that has an RFC with IETF or any other standard body. No one has any issue using those languages. Why should it be any different for us?
I wasn't involved, but first hand reports are telling me otherwise. I'm going the leave this in the document. I feel that it accurately represents the way many of the core contributors to the project feel about working with IETF even if there are potentially misunderstanding involved. However, I can add a reference to the relatively new HTTPAPI working group that is far more likely to be receptive to our project than the WG that we interacted with in the past. |
Incorporating feedback from Henry and Austin.
Yeah I think it would be good to show that we do not have a blanked negative reaction to or dislike of the IETF. Our concerns come from specific past statements that getting JSON Schema adopted by a working group would more or less involve starting over. If there turns out to be a working group that would take a different approach, that would be worth considering. It is worth noting (here, not necessarily in the ADR text) that the HTTPAPI working group is producing informational, rather than standards-track RFCs, at least for the ones we're involved with. The prior discussions were in the context of a standards-track working group/RFC. |
I don't think that's true. There was discussion about "tracks" at one point and it was decided that the group would default to standards track unless there was reason not to. The links, problem+json, deprecation, and rate-limiting efforts are all standards track if I'm not mistaken. |
Ah, this is good news! @admin-cimug we may need to add the "media type registration is a standards-track RFC, with the rest of JSON Schema behavior described some other way" as an option. @jdesrosiers I was going by this setting which produces "Intended Status: Informational" in the current published I-D. But perhaps that is outdated? |
I'm not 100% sure of the timeline, but I'm pretty sure that the document was created before this was discussed and no one noticed that it still says "informational". I certainly didn't notice. |
@handrews / @jdesrosiers : First off...Henry you'd commented:
I completely agree and don't think that the JSON schema spec team should find themselves cornered into a position to start over if that's where you find yourself at. There are many instances in my own involvement in standards where technologies have evolved/matured and have been introduced and accepted as the "baseline" for a standard from which future evolution can occur. To give you one practical example is the OData (Open Data Protocol) standard. It is an ISO/IEC approved, OASIS standard that defines an option for a set of best practices for building and consuming RESTful APIs. I don't know the nuances or the day-to-day of the IETF RFC process as well as I do our own at the IEC but hopefully there is some flexibility on that front.
I hope this helps. Please feel free to run past further questions or if I may have missed anthing. ~Todd |
|
@admin-cimug thank you so much for this information! It is very helpful, especially since you were able to confer with a wider audience. While none of this should be taken as a commitment (and I do not speak for the whole JSON Schema Org), I think this will all work out well. While we have not made a final decision, there is movement towards dropping the "draft" signifier with the next release. It just means that we have to decide what our compatibility process is going forwards (a tricky topic) since we wouldn't be able to just wave our hands about breakages between drafts, and we need to by clear as to why people should rely on our publication if it is no longer an IETF draft. Barring a truly unexpected development, there will be an IETF RFC on one of the acceptable tracks registering the We are already listed as a Linux Foundation project (under Web and Application Development) through our affiliation with the OpenJS Foundation, where we are listed as an incubation-status project. Having only recently formally joined, we're still working through our checklist to exit incubation status. See json-schema-org/community#129 for progress – I'm not sure if that's exactly the checklist for exiting incubation but it's stuff we are doing to finish onboarding, at least, so if anyone needs evidence that we're working on it, there it is in writing. We need to get more of this information onto our web site, which is undergoing a redesign, but it sounds like if we officially publish a road map in this direction, it would reassure the formal standards-based community that we are making real progress towards being a properly cite-able specification. Please let me know if you think of anything that would still be a concern. We understand that things look a bit shaky right now, but we want folks to know that this is a transition rather than heading off into the wild west 🤠 🤓 |
|
@admin-cimug Thank you for the feedback! I'll incorporate that into the document. If you are comfortable with OpenAPI, there's no reason you shouldn't be comfortable with what we end up with. |
Include feedback about referencing JSON Schema in other standards without the backing of a recognized standards body.
Mention OpenJS / Linux Foundation and add Henry as a decider
adr/2022-08-decouple-from-ietf.md
Outdated
| * Several members of the JSON Schema team have had poor interactions with IETF | ||
| and don't feel that working with them would be productive. This is a | ||
| relatively minor consideration. If we thought IETF was right for JSON Schema, |
There was a problem hiding this comment.
Since this is going to be a formal public document, I've been thinking about this wording. I'm also going to explain some things here because they keep getting asked (most recently by @devinbost), so I spent a few hours tracking down the history. Skip down to the last section for wording recs.
Contacting the old JSON working group
When I contacted the JSON working group in January 2018 (see #526), this quite helpful response from Paul Hoffman indicated some likely difficulties:
There appears to be interest in something around schema for JSON; it is
not clear if there is interest in starting from the specific spec at
https://github.com/json-schema-org/json-schema-spec. I bet there is some
interest in that, and some interest in something else. This is quite
typical of requests for the IETF to adopt work that originated outside
the IETF, with mixed results in any of of "adopt with the intention of
making only minor changes", "adopt and make major changes", and "start
from scratch while getting inspiration from the existing work".
While the above paragraph suggests that maybe we could have gotten it adopted, there were zero encouraging responses or expressions of interest aside from Paul's response. There were several non-productive criticisms and dismissals on such topics as not needing minimum other than 0 or 1, and not needing maximum at all. Of course, individual poor-quality feedback is not necessarily representative, but it was the lack of anything resembling constructive engagement was the more important concern. Sadly I can't link to the absence of something.
In response to that discussion, someone else brought up a project called JSON Content Rules, which got much more interest (although notably, no further drafts were ever produced and the project was abandoned). Somewhat later, the project that eventually became JSON Type Definition also got more attention (although note that it is an "experimental" RFC and was never adopted by a working group).
The important point here is not that some people were grumpy and hostile (I'm... really not in a position to throw stones there), it's that no one was actively interested, and instead those who participated actively favored other projects.
The JSON Path experience
While I don't want to put anyone on the spot, we do know that JSON Path's adoption by a working group has led to more changes in that specification than were expected or wanted by at least some participants. It essentially illustrates the fear discussed above: that the momentum would be away from what works for our existing user community.
Of course, that doesn't meant the same thing would happen, but it is very relevant (and involves some overlap of people).
A perusal of email archives and GitHub issues and PRs also reveals a number of incompatible expectations in working styles and process. Whether or not this qualifies as a "negative experience" depends on one's expectation, but it would probably be negative for us.
Suggested wording
I would suggest replacing the "poor interactions" wording with something like:
Several members of the JSON Schema team have interacted with JSON-related IETF working groups. Some of these interactions demonstrated an indifference or hostility to JSON Schema, and a preference for projects taking a different approach. Equally important was a lack of any active interest or constructive engagement. Finally, we were informed that any schema project for JSON would not necessarily start from JSON Schema as a base, indicating that a "JSON Schema" working group would quite likely not involve JSON Schema itself. This impression has been reinforced by observing the amount of change introduced to JSON Path as a consequence of its adoption by an IETF working group. While we have a good relationship with the relatively new HTTPAPIs working group, the combination of these other experiences with other mismatches between our project and the IETF process contributes to our reluctance to move forward through the iETF.
There was a problem hiding this comment.
While the above paragraph suggests that maybe we could have gotten it adopted, there were zero encouraging responses or expressions of interest aside from Paul's response. There were several non-productive criticisms and dismissals on such topics as not needing
minimumother than 0 or 1, and not needingmaximumat all. Of course, individual poor-quality feedback is not necessarily representative, but it was the lack of anything resembling constructive engagement was the more important concern. Sadly I can't link to the absence of something.
@handrews , in reviewing the mail archives you provided and acquainting myself with the references to criticisms posed it only reinforces the decision to go in a different direction. The criticisms of minimum or maximum, for example, I patently would not support. In the IEC standards we're working on we have a distinct need for exactly such keywords to properly define the contextual model mappings into JSON schema syntax. This is "real world stuff". I don't think valuable time/energy should be spent arguing fundamental constructs such as these in the spec.
There was a problem hiding this comment.
I'm using the suggestion. A first-hand account is much preferred to my second-hand account.
|
@admin-cimug Thanks for joining and sharing your thoughts. Really appreciate you taking the time to not only answer questions, but put them to your wider group. This matters and will inform our decisions. |
No problem Ben. I particularly appreciate the level of engagement you, Henry and the spec team have demonstrated. I much prefer to work in that type of collaborative dynamic. |
|
Also pinging @davidgtonge who commented on #526 a while back with a perspective from the ISO. If you don't have time to go through everything here, this comment summarizes the general direction of the most likely (but not committed) approach for us. |
Feedback from Henry. The main changes are updating the section on our prior experience with IETF and clarifying that our descision not to use IETF doesn't apply to media type registration or supporting components such as Relative JSON Pointer.
I'm going to wait until @Relequestual contributes his notes about W3C. Then we should be ready. |
|
@jdesrosiers this is looking great now! I left one or two threads "unresolved" but only because they have useful info from which I think any new visitors would benefit. specifically this thread with the account of the JSON working group history. But please consider everything from me prior to this comment to be resolved and feel free to press the "resolve" buttons if needed to merge or un-draft. |
There was a problem hiding this comment.
I've added my comments regarding W3C. I'm happy with the rest of the document.
Please feel free to remove the draft status on this PR @jdesrosiers =]
(Edit: Looks like I could make it ready for review myself, but as you said "should", I'll leave you to confirm)
|
Editor hat on: There’s certain uses of the I-D process we can reevaluate, but the text here is essentially a change to the charter that's out of scope. However all of the problems here are fixable by an editing process, and with robust interoperability requirements. Let's try fixing these underlying problems first. (Editor hat off) What do interoperability requirements have to do with this? The feeling that the I-D system is unsuitable for JSON Schema, or that an RFC would mark the end of our work, comes primarily from the lack of interoperability requirements. In the I-D process, new publications replace old ones in their entirety. This poses a problem when we’ve removed or changed features that validators have already implemented. Implementors (correctly) want to publish stable, production-ready packages that aren't going to break just because we published a new document. So in order to produce a stable package from an unstable (draft) source, they have to make some additional assumptions, out-of-spec, to provide stability. This process often conflates the specification documents, the meta-schemas, and the media type. These are all different things, and they are "versioned" differently:
Again, it should be apparent that JSON Schema (as a whole) is unversioned (each document describes its own expiration policy). But since this is not suitable for stable software packages, they adopt $schema as a version identifier to detect use of obsolete (since removed) features. This is tolerable, not because the specification says to do it this way, but rather we expect that software packages to avoid breakage. (It was my intention that a different mechanism would be used for reverse compatibility, but I never wrote that in, so that’s not what ended up happening.) We want to accommodate these implementations, so we’ve run with this. Each release is considered a new “dialect” among many. But when we treat old and new releases as equivalent, we can’t use the I-D system for its intended purpose, or publish simple revisions in response to feedback, and so the I-D process feels obsolete. By (effectively) publishing multiple versions in parallel, and a new concern arises: if just one is published as an RFC, wouldn’t we have to publish future changes as another RFC, subject to the whims of the working group? Wouldn’t development slow down because of that? To avoid this possibility, the idea is to independently publish “vocabularies” or “dialects”. But this only helps software packages. Suppose you’re an API and you publish a schema, where Validator A only implements the latest JSON Schema Core & validation, and Validator B reads $schema and will ignore certain requirements in the current JSON Schema Core as necessary. In this environment, two validators can be “compliant” (at least with the tests) yet disagree on if an instance is valid/invalid, and this is bad. Or, Validator C errors on anything that’s not draft-4, and Validator D errors on anything that’s not draft2020-12, so no one schema will work for everyone. It defeats the point of a standard, which is interoperability. Independent publishing does not solve these problems, it just covers them up. The RFC process is designed to promote interoperability, so it’s no wonder we’re having a difficult time with it. The solution to these problems is to define robust interoperability requirements that will be guaranteed well into the future. What are the solutions? (1) A strong reverse compatibility policy This means we cannot remove behavior from the core specification, unless we’ve determined it can be implemented as a module, extension, or This is mostly my fault. In the past we've just yanked out text that we don't like, but the specification isn't supposed to describe what's good, as such, it's supposed to describe how to be interoperable. By making backwards compatibility interoperable, implementations will no longer need to vary from the specification text in order to maintain stability. This relieves the pressure on us to maintain support for “dialects” of old and expired specifications, and allows us to reap the benefits of the I-D process. (2) Implement changes as extensions. We prototype changes to the specification as optional extensions. These extensions can then be tested within specialized ecosystems. This would allow us to publish I-Ds that solicit feedback without locking the JSON Schema universe into a particular expectation. Vocabularies are partial solution, but more fundamental changes to the core semantics still lack an extension mechanism. (3) Robust, ubiquitous extension mechanisms Solution parts (1) and (2) are procedural changes that assume there’s an extension mechanism. We still need to define this concrete extension mechanism. In Email or HTTP, you want reliable messages that are delivered even if you don’t understand a part, so unrecognized headers are ignored. In JSON Schema, the functionality of JSON Schema is more analogous to a programming language: If you encounter an undefined function, you don’t want it ignored; you want an error. Yet we copy the Email behavior even though this doesn’t make sense in our particular case. Another hint that ignoring unknown keywords is the wrong approach is that known keywords with a bad argument do error. (Email ignored malformed headers whenever possible.) Either bad arguments should be ignored, or unrecognized keywords should error. It doesn’t make sense to have it both ways. To fix this, we need a concrete way to tell when there’s unsupported validation keywords in use, so that validators won’t return a determinate (valid/invalid) result. The most straightforward way to do this is to error on unrecognized keywords. Keywords can become "recognized" by any standard mechanism, including:
Finally, sometimes authors do want keywords to be ignored, especially annotation keywords, so there should be some way to flag keywords that serve no validation-related function, even if the rest of the semantics remains unknown. This would allow new publications that are backwards compatible with older implementations, and it would allow functionality to be defined separately from the essential semantics. By defining essential semantics separately, this would relieve the need to publish “dialects” as an RFC. The PR notes
And
I think we can accomplish this in fairly short order, and in a way that expands our ability to make changes and experiment. I see two realistic paths (among others). (1) Over the past couple months I’ve tossed around a couple of ideas for the HTTP WG and the HTTP APIs WG that have been well received, and I’m making it a personal goal to be at IETF 116 Yokohama and propose a BOF around at least one of those ideas. (BOF=“Birds of a Feather”, the most common way to form a WG or persuade one to take up work.) If that goes well, then I hope to have the JSON Schema Core document ready for a BOF at IETF 117 San Francisco. This is about 10 months away. (2) We can form a Community Group at the W3C. This is simple to do and requires 5 people, including one Chair. These community groups can publish reports and submit them for adoption by the W3C. (3) There’s other standards organizations that could publish a standard, e.g. ISO (recall that ISO also publishes JSON as ISO/IEC 21778:2017), but they are less open typically require some sort of corporate sponsorship. It’s possible the OpenJS Foundation could help here, but I don’t know enough. The IETF process I think is better evolved and more open, though the W3C has more expertise in hypermedia formats. |
|
@awwright In agreeing and ratifying this ADR, the consensus of the JSON Schema Org rejects your poposals in your latest comment, explicitly 1: looking to form an IETF BOF, and 2: looking to form a W3C community group. The ADR should stand and we should operate accordingly unless there is new information. |
@Relequestual / @handrews / @awwright : I apologize that I've been checked out here for a few weeks. I hosted our first return to in-person IEC Joint WG global standards meetings this past week so I was completely out of pocket. As part of this I had an opportunity to meet in-person with some of our expert members from the National Committees of E.U. member states (we had representation from France, U.K., Hungary, Ireland, Norway, Sweden amongst others with some having to attending remotely due to corporate restrictions still in place for COVID). As I think I previously mentioned we have pending I.S. publications waiting on JSON schema due to it being the core dependency. Given that we've collectively made significant investment in a reference implementation and preliminary work for use of the draft publication for use in European energy related data exchanges. this is a very high priority area with lots of visibility (also we have other standards in process that will, in turn, depend on the publication of this one.) During this past week's meetings we had several sessions that were held where questions came up as to when we will finally be able to push this to publication. This has become definitely a critical topic. I provided a status report of the discussions we've had over the past month. @awrights points on interoperability is really a central point that remains fundamentally important. Even though there's a rejection of his IETF BOF and W3C related proposals. His point on APIs and interoperability is important:
When I originally presented my early standards work at the DOE's PNNL Research Lab 3 years ago what I presented got the green light as the result of what was perceived as clear intentions to undertake the IETF RFC process. Now, I understand the JSON spec teams' concerns so I support what you're trying to do as the team wrestle the process through. I just don't want to lose the key goals of interoperability along the way as @awrights is pointing out. Another concern I have is that we find a way forward that will prevent another three years to establish something from which other SDOs can build upon. Finally, a key concern I have is that we're losing external confidence/credibility. I have no doubt that the team is fully competent but its the optics that I'm trying to manage at this stage. I have the E.U. National Committees looking for something definitive and I'm now cornered into trying to find a path forward to publication. I'm wrestling with whether to back off and publish our core standard as a Technical Specification as opposed to an I.S. (this is not ideal at all) or do I back off further and publish it as a Technical Recommendation (?) I don't even consider that a descent option. Anyway, I trust the team here will get my point. Any possibility we can brainstorm together a short term path forward that minimally puts the "scaffolding" in place upon which to build the long term path you want to take? This scaffolding being enough to manage the optics and build confidence in the path forward? ~Todd |
|
@admin-cimug We've been discussing all of the issues @awwright brings up here for a while and they have been well received. We're still discussing what our process will be going forward, but if the proposal on the table is adopted, all of your concerns about interoperability and compatibility will be addressed. The plan is to release a "stable" spec as soon as possible. I think releasing by the end of first quarter 2023 is achievable. By "stable" I mean all the interoperability/compatibility concerns would be addressed. Once the stable version is released, all changes from then on would follow strict backwards and forwards compatibility rules. New features would go through a process similar to what EMCAScript uses where features have to be implemented and show real-world use and benefit before they can be considered part of the stable spec. Of course, in your situation, your spec would limit to only the stable JSON Schema spec because you lose interoperability guarantees when using unstable features. You can view the proposal and join the conversation at, https://github.com/orgs/json-schema-org/discussions/234. And, again, thank you for your feedback. It's incredibly helpful for us. |
|
@awwright I'm going to keep this short because the decision has already been made and I don't want to drag this out. But, you put a lot of effort into your comment and you deserve a response.
I've noticed you refer to yourself as "the Editor of JSON Schema" a few times recently. I don't know what you mean by that. We don't officially have such a role. As far as I know we don't have a charter for JSON Schema. Maybe we should have one, but I'm quite sure we don't.
As I've told you many times, your interoperability/compatibility concerns have been well received and have broad agreement. Addressing this issue is a top priority for the next release. Your explanation was good, but it wasn't necessary. You're preaching to the disciples at this point.
I think your plan is reasonable, but it's a too vague in a lot of areas for me to have confidence. The biggest problem in my opinion is that I don't think we can create a stable extension mechanism in any reasonable amount of time, especially since you still haven't articulated what you have in mind. Currently our extension mechanism is the vocabulary system. We put it out knowing that it was an experiment and would require some real-world use to illicit feedback on what problems there were and how it can be improved. It's very much a work in progress. Whether your idea of an extension mechanism is based on the vocabulary system or something new entirely, I'm convinced it will take real use to vet and get right. That's going to take years. Your plan for an RFC core with extensions is dependent on having this extension mechanism in place. That means being a "draft" for quite a while longer, which is not something we're willing to do. I'm still interested in what you have in mind regarding extension mechanisms. You've sold me on just about everything you've proposed lately other than IETF, so I expect I'd like what you have to say. But, let's create a new issue for that conversation and not in the context of IETF since this decision has already been made. |
|
@admin-cimug are the "optics" concerns not addressed by what we previously discussed regarding an IETF media type registration plus our own publication in the context of being an OpenJS Foundation / Linux Foundation project? Particularly if we move out of OpenJS "incubation" status and into the next one (which I think is "impact" but I'm not sure)? |
@jdesrosiers / @handrews : First, I wish I was more up to speed on all past discussions so I'm genuinely grateful for you and the team's patience as you graciously allow me to voice my questions and comments. Thank you for confirming/clarifying the interoperability related concerns and that the team has discussed and has plans to address along the way. It's a core tenet for the IEC standards we publish so was not totally clear from @awwright original post if this was going to be fully addressed (and thus my last response). Now, I just reviewed the discussions link you sent. This is great. Had I caught it earlier I'm sure it would have helped.
@handrews I think we may be close but there's a final level of assurance I think I'm fishing for but it may be just be the need for a bit more discussion here. First, after we talked and in lieu of my meetings last weeks the key question that came out of things is timing. I did check out OpenJS Foundation as we discussed and see JSON Schema listed as an Incubation Project. The question now is as to the timeline for completing the on-boarding checklist to join the foundation? Do you see any obstacles or concerns that may present new roadblocks that might derail us as the IETF RFC processes did? If I knew definitively that JSON schema was close to on-boarding I'd definitely have more peace of mind. This help address the "optics". A charter would be nice to have but I think official acceptance with a charter to come later like we discussed would really help. I apologize that I'm jittery on this. There's a two fold reason why. The first is the previous three year window. The second is addressing how we move forward and timing of OpenJS, etc. The the most efficient/ideal standardization window I now have is probably 12 months from the time I officially submit what we term an IEC NWIP (i.e. essentially a "work proposal" for a new standard.) I would have originally done this a couple years ago but we waited. Our plan (i.e. myself and my European counterparts) has been to submit a nearly complete late stage draft of an I.S. (thereby shortening the review cycles). Such a move is what we would term a "fast track" process. This approach bought me time the past couple of years as we waited for the originally suggested IETF RFC process to progress for JSON schema. That (for very understandable reasons that you've explained) stalled out. Now at last weeks meeting the urgency has escalated. Hopefully that helps explain what remains. So I see a couple of open items for us:
Now, I don't have the bandwidth to catch myself up on all related and relevant discussion so if any of my above questions should be documented and obvious elsewhere than thank in advance for bearing with me. |
Levelling up our status in the OpenJS Foundation is tracked here: json-schema-org/community#129 I can't foresee any roadblocks. It has just been a matter of time and priority.
In truth, I can't say. What I will say is, even though we're "only" an incubation project, we are still considered proper members of the OpenJS Foundation. It's not a "half measure", but a signal of commitment to see it over the line. I made a personal promise to do everything in my power to see JSON Schema go beyond the incubation status.
We try to discuss as much as possible in public. I don't think we've discussed much about this proposal which hasn't later been made public, either there or during one of our open weekly meetings. Sorry if I've not addressed every part of your comment, but I hope that's still useful. Thanks again for sharing and allowing us to hear your important feedback and comments... we want to see this more from yourself and others! 🙌 |
I'm probably the person with the most objections/divergence regarding the SDLC proposal. However, I am (and have been) actively working on getting to "yes" with it, which is why there's not a competing proposal. I also do not intend to drag it out: as I have told the rest of the team, if there is a strong majority sentiment to accept it that develops before I can articulate any sort of requested changes or alternatives, I will go with that majority sentiment. So while I cannot give you an exact timeline or assurance, as the person who is the least on board of those who have commented to date, I can guarantee that it is moving towards a resolution. My concerns at this point are mostly around the level at which we define stability and compatibility: whether it is keyword-centric or feature-centric. Which I realize is very vague but I am working on un-vague-ing it 😅
It will start for the next release. We would like to put one out in January 2023, although given the level of stability/compatibility guarantees needed to do this, it might be more like March 2023. We would not wait until 2024 or shift the stability promotion month, we'd just have an abbreviated cycle and do the next round of promotions in January 2024 and continue from there. Given that the team is partially volunteer (if anyone wants to sponsor my or other volunteer work with JSON Schema pls reach out to the org), and that the full time staff have a variety of responsibilities beyond just spec-writing (including OpenJS onboarding, community engagement, test suite development, etc.), we can't rush the next release but so much. But January 2023 would be our ideal, and we'll get as close to it as we can. |
Thanks Henry for your response. You got a grin out of me 😁 ... LOL. Your summary is just what I needed. That Jan-2023 to Mar-2023 timeframe should actually work well. If we accomplish what you've summarized it should clear away the concerns and I don't see any issues then having the JSON schema specification included in our normative references and expressing it's association with the Linux Foundation (I believe this removes the whole "optics" thing). Based on this our goal can be to update our draft I.S. during one of the review cycles and align it with that initial stable release for Jan or Mar. @Relequestual : Ben, 🙂 thanks for pointing me to: json-schema-org/community#129 that will be helpful for tracking puposes. |
|
@admin-cimug great! Just to be clear, I cannot (on behalf of the org) commit to the Jan-March 2023 timeframe. But we are targeting January so I think that range is realistic. If we cannot hit that range, I would (personally, not an org commitment although we could probably discuss such a thing) expect that we would publish some sort of guidance as to what is going on. A delay would presumably mean that we found an important compatibility problem with our process, and that would be the kind of thing we'd want to communicate. Presumably, it would be something that anyone waiting on a stable spec would want to see resolved anyway, |
|
@jdesrosiers I apologize for not having a more concrete proposal sooner, there's many moving parts and I'm hesitant to propose changes that involve the vocabulary system, because that's so highly utilized. I hope some of my regular updates show that there's interesting developments in the space, though. I'm trying to figure out how I can devote more time to have a workable solution ready by March, when I would like to take other HTTP-related work to IETF 116. Regarding "editor," that's how the document itself lists my title. This is ever since I asked the previous editors about publishing an update. As the scope has grown, I've turned over various aspects of editing and publishing to different people over time, particularly so that different people oversee different documents, which now list multiple editors and authors; but I still edit Core and oversee interoperability and Internet related issues. My intention was always that our organization would serve as a forum for implementors, and to promote the standard, but that nobody owns the specification, we just steward it until it can be adopted by a venue (or multiple venues) with experience in Internet standards. We don't have a formal "charter" as such but the I-D process is a written down process, and deciding to change the process or the venue is unusual. |
|
@awwright creating and publishing a formal charter is part of our OpenJS Foundation todo list: json-schema-org/community#129 Given that we are no longer simply continuing the approach that we inherited from the previous editorial team around 2016 (or whenever you first picked it up), I think that increases the importance and urgency of sorting out the charter. |
Here's a fist draft for an ADR for decoupling from JSON Schema. I'm sure it needs some cleanup and fleshing out in places. I'll revise tomorrow.
The scope of this ADR is decoupling from from IETF. That doesn't include the decision to stop using "draft", but enables us to make that decision later.