JSON Schema
Specification Development Life Cycle (a collaborative proposal) #671
Replies: 15 comments 36 replies
-
|
"Version" is an overloaded term. The above process defines a single version as "a collection/sequence of compatible releases." Do we want to use "version" for this, or something else? |
Beta Was this translation helpful? Give feedback.
-
|
The meta-schema URI needs to declare the version as well as the release. Open to other proposals. Proposal 1
Proposal 2
This has the added benefit of visually indicating that it's an identifier, not a location. We would need to register the |
Beta Was this translation helpful? Give feedback.
-
|
I'm strongly in favor of Proposal 2 here. One of the most common confusions among users comes from our use of URLs for This also results in tooling that defaults to automatically attempting to download resources even though the spec very specifically and explicitly recommends against it. Using a That said, I don't have a problem also aligning the URI and the URL it's served from. With an Obtaining the download URL is as simple as transforming the URI scheme from Keeping this URI/URL alignment allows for easy downloads, while having a dedicated URI scheme provides a strong indication that it's not intended as a locator. We need to lead by example by setting that precedent ourselves. |
Beta Was this translation helpful? Give feedback.
-
|
Here's some analysis of the various options.
|
Beta Was this translation helpful? Give feedback.
-
|
Governance: For people posting ideas (the "concept" stage), we could have a "proposal" issue template in the spec repo that would help people think about certain aspects of the proposal before posting it. For example:
|
Beta Was this translation helpful? Give feedback.
-
|
How can we know when an experimental feature is ready to be promoted? Usage?I'm hesitant on requiring the usage metrics since many devs will prefer not to use experimental features, especially if there's a chance the feature could change. I think the worst case is we add a feature that no one uses and it's just deprecated later, but that would also mean wasted time for us and implementers. Minimum time?Should there be a "minimum unchanged time" before a proposal is promoted to stable? What about for breaking changes? For example, should we require that a proposal remain unchanged for six months before releasing it? |
Beta Was this translation helpful? Give feedback.
-
I was thinking about this. The main reason not to require 100% of implementations support a feature is that a lot of implementations aren't actively maintained or do maintenance only and it's hard to identify which is which. Most implementations are someone's hobby that they might not have kept up over the years. So, what if we maintain a group of member implementations? Any implementation is welcome to join and must renew their membership annually. This would give us a list of implementations that are actively maintained and serious about keeping up with new features. Then we could go as far as to say 100% of member implementations must support the feature. |
Beta Was this translation helpful? Give feedback.
-
|
I really like that, Jason. It would require a level of governance (@benjagm), but it's a good piece to this puzzle. |
Beta Was this translation helpful? Give feedback.
-
In a way, it is not a bad idea but there needs to be quite a bit of administrative work. It is more than governance, you would need to be a legal entity if we discuss membership. That is going in the standardization development organization direction. W3C has a rule like that but a membership attachment of an implementation is not required for the Web of Things (WoT) work. There can be also issues where an implementation doesn't necessarily belong to a single person. For example, Siemens is a W3C member but we contribute to open-source WoT implementations under the Eclipse Foundation, which has no formal relationship to W3C. Still our implementations are counted for the standardization work. I am not objecting nor strongly in favour but it is a relatively important decision to take. |
Beta Was this translation helpful? Give feedback.
-
|
I had something much more informal in mind. I don't see a need for a legal entity. It's just more of an informal way to track active implementations. They aren't contributing anything. They're just indicating their desire to be considered when we make our decisions. However, we have a contact who can advise us about legal stuff, so we can bring it up with him. Thanks for flagging that for us. |
Beta Was this translation helpful? Give feedback.
-
|
Maybe "membership" carries a common definition/idea, and we just need a different word for this. |
Beta Was this translation helpful? Give feedback.
-
|
Should a stalled proposal expire? If an existing proposal loses traction for whatever reason, should the proposal be removed? Does removing an experimental feature from the meta-schema constitute a breaking change? (The keyword is allowed but will not be in future meta-schema releases.) |
Beta Was this translation helpful? Give feedback.
-
|
In W3C, if a new feature doesn't have enough implementations or adoption, it is removed before a final release (called a W3C Recommendation). |
Beta Was this translation helpful? Give feedback.
-
|
In that case, the feature is still defined, but not required. It seems to be in a perpetual "proposal" state. I expect these W3C Recommendations are still rather complete as well. I'm really asking about proposals that never reach any kind of completion. |
Beta Was this translation helpful? Give feedback.
-
|
Do proposals need their own versions so that implementers can indicate their support? A subtle behavior change (non-syntax) could be confusing to a user of an outdated implementation. |
Beta Was this translation helpful? Give feedback.
-
|
Can a deprecated feature return to "stable" status? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, perhaps by going through the Proposal and Experimental phases again first. |
Beta Was this translation helpful? Give feedback.
-
|
@karenetheridge would that mean that they are subject to change? If so, that would relate to re-introducing keywords in later versions. |
Beta Was this translation helpful? Give feedback.
-
|
I wouldn't have a problem with undeprecating a feature if we change our mind, but any changes to the feature still wouldn't be allowed. It would have to go back to Stable. It couldn't go back to Proposal/Experimental. |
Beta Was this translation helpful? Give feedback.
-
|
Can a removed keyword be re-introduced in a later version (a breaking change has occurred after the keyword was removed). If not, does the meta-schema need to enforce this? E.g. {
// ...
"properties": {
"foo": false,
"bar": false,
// ...
}
} |
Beta Was this translation helpful? Give feedback.
-
|
That shouldn't be necessary, because of the |
Beta Was this translation helpful? Give feedback.
-
|
Should we then define a "removed keyword registry"? (This is protection from future-us, primarily.) Should the meta-schema link to such a registry to enforce that they're not re-introduced? |
Beta Was this translation helpful? Give feedback.
-
|
Tracking these changes are good in any case I would say. Talking hypothetically but if there are different circumstances, those keywords should be possible to reintroduce. |
Beta Was this translation helpful? Give feedback.
-
|
Governance: Do we need some sort of incentive program to encourage implementor support for experimental features? |
Beta Was this translation helpful? Give feedback.
-
|
@gregsdennis - What would be the incentive? |
Beta Was this translation helpful? Give feedback.
-
|
I don't know. This thread is to discuss whether we should define one and whether we can even support such a thing. We'll probably have another discussion somewhere to determine what it could be and actually ratify it. |
Beta Was this translation helpful? Give feedback.
-
|
Personal opinion: If a feature does not get any interest, it should not be adopted into the release. It is in the proposer's interest to implement it somewhere and gather support from other implementers. This would slow down the process but would build up more consensus. |
Beta Was this translation helpful? Give feedback.
-
That's 100% the goal and IMO one of the most important changes from our previous way of working. We used to first release a feature and then wait for people to implement it. The goal now is to shift to get implementations first and the feature can only be released once implementations and usage show that it's something people actually want and doesn't have any major problems. |
Beta Was this translation helpful? Give feedback.
-
|
When you say "version", I think this means "major version"? As a version is an atomic thing, e.g. "1.2.3", but a "major version" can encompass a set of versions (where all such versions are internally compatible). I don't mean to bikeshed on the version format; that can be figured out later, whether semver or something else. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, what I mean by "version" is analogous to "major version" in semver. This is why I kinda want to get away from the word "version" (I have a thread for what it could be named). |
Beta Was this translation helpful? Give feedback.
-
|
On known/unknown/experimental/supported/unsupported keywords... "Implementations MUST refuse to process schemas which contain unknown keywords." -> clarify this to mean "..where a keyword is not defined by any vocabulary used in the evaluation" or perhaps "..any vocabulary defined by the dialect" [although dialect isn't defined yet.. so that suggests a glossary is needed?] "The keyword prefix x- defines a safe space for users to introduce custom annotations" - are we saying that x- keywords can only be annotations, and have no associated functionality? In the past we've acknowledged that some x- keywords might actually do something, just something recognized onlyby a specific implementation and not formally defined in a vocabulary. Is this change intentional? Was there an ADR produced for it? The next section, which says "To support custom keywords, implementations MUST create a custom meta-schema..." would be unnecessary in that case, as now it's implying that even to use an x- keyword one must define it in a new vocabulary. So I think some rephrasingis needed here, as "experimental" and "custom" seem to be used interchangeably and I'm not sure if they're both referencing the same thing. Which of these is intended to be true?
I would have assumed the first, and would add an explicit caveat that when not using a custom vocabulary (in conjunction with an x- keyword), interoperability is explicitly NOT assumed or expected. |
Beta Was this translation helpful? Give feedback.
-
Well, the problem is that vocabularies are being extracted into the Feature Life Cycle, so the spec can't use them yet, but it also still needs to allow for custom keywords. It needs some mechanism other than vocabs to do that. Thus, "unknown keywords" needs to merely encompass"what the implementation understands." It's not the best, and not interoperable at all, but it's the best we can do until vocabs/dialects are sorted out. |
Beta Was this translation helpful? Give feedback.
-
Yes, my understanding was that As with the above comment, this needs to be independent of vocabs since they're being extracted.
Yes. We also have a blog post, though, that describes the reasoning and purpose.
No, Implementations which provide/support custom keywords need to create a custom top-level meta-schema similar to this: {
"$schema": "https://json-schema.org/draft/next/schema",
"$id": "https://example.com/custom-schema",
"$dynamicAnchor": "meta",
"title": "My custom meta-schema",
"allOf": [
{"$ref": "meta/core"},
{"$ref": "meta/applicator"},
{"$ref": "meta/unevaluated"},
{"$ref": "meta/validation"},
{"$ref": "meta/meta-data"},
{"$ref": "meta/format-annotation"},
{"$ref": "meta/content"}
],
"type": ["object", "boolean"],
"$comment": "Defines some custom keywords",
"properties": {
"custom-keyword": {
// ...
}
}
}
Neither, primarily because vocabs are being extracted and can't be relied upon. Instead, one can introduce new experimental keywords (which can do anything), but not in the |
Beta Was this translation helpful? Give feedback.
-
|
Re the Lifecycle, I would make a clear distinction between features that can be wholly implemented with a new vocabulary using an existing major version (i.e. it does not require modifications to existing functionality of JSON Schema as a whole or to existing vocabularies and keywords), and features that require changing something essential to JSON Schema as a whole (say something that modifies how errors are produced, or the order of evaluation of keywords, or reference canonicalization etc). In particular, a vocabulary might have a different versioning lifecycle than a specification version (perhaps multiple major revisions of a vocabulary could be released that are compatible with a specific specification major version). |
Beta Was this translation helpful? Give feedback.
-
|
Vocabulary versioning is an interesting idea that should be discussed once we extract, fix, and re-introduce vocabs. For now, we need to consider them as not part of the spec. |
Beta Was this translation helpful? Give feedback.
-
This is implying that any implementation that supports release N must also support N-1, N-2 etc (presumably back to the first "stable" release). Is that realistic, if there are major changes between versions? There may be fundamental changes in the way evaluation is done between versions such that it is not practical to support both of them in the same implementation. |
Beta Was this translation helpful? Give feedback.
-
If an implementation supports the third release in a version, then it automatically supports all previous releases within that version becaues each release is backward compatible within its version. The real question is, "How many versions is an implementation expected to support?" which leads me to...
Jason and I had some conversation about this. I was thinking that we could recommend that implementations only support the latest, maybe the latest two, versions in order to provide pressure on users to update their schemas. Jason's point was that implementations are going to support what they want, including all the way back to draft 3. This was very much coming from the other way than what you're thinking, though. I wouldn't require an implementation to support previous versions, no. |
Beta Was this translation helpful? Give feedback.
-
|
Regarding the release cycle period, the proposal says:
Has there been any discussions on this? I am not sure what is the benefit of constraining to an annual release cycle. From my point of view, it can be even slower. |
Beta Was this translation helpful? Give feedback.
-
|
The inspiration for this comes from ECMAScript. Previously, their releases were defined by a fixed set of features. When there were delays in one or two of those features, the whole release was delayed. This means that people had to wait sometimes years for features that were ready because of a some other feature that wasn't ready. In their new process, whatever is ready to go on release day is released. Releases are often small, but users get access to more features sooner. We have he same problem in JSON Schema. We have several things that have been ready for release for years, but haven't been because of other things holding up release. By emulating ECMAScript's solution, we hope to get features out to users sooner. By sticking to compatibility guarantees this more frequent release cycle shouldn't be the burden it previously was every time we release a new version. I had occurred to me that it's possible that we could get to Jan 1 and nothing is ready for release. I don't think that's too much of a concern. We can just skip that release. |
Beta Was this translation helpful? Give feedback.
-
|
Ok I see the reason and find it positive. Just that ECMAScript's new features look not ready at all from the user pov (https://compat-table.github.io/compat-table/es6/ or newer ones) so I would doubt upgrading to 2022 version. I think tools like bowtie can generate such tables and at least we can track everything and keep things transparent. |
Beta Was this translation helpful? Give feedback.
-
|
Annually was what @jdesrosiers proposed with his first SDLC concept. I've just stuck with that. I think a regular cadence is good. If we don't have any new features to release, maybe it's just editorial stuff, or we can just skip a year. (My browser hasn't updated before I posted. I now see the response above.) |
Beta Was this translation helpful? Give feedback.
-
|
The topic of availability is not discussed above. Does the JSON Schema Organization guarantee the availability of the specifications in a stable URI? I am guessing yes but it would be very important to clearly express this. Something along the lines of "As long as the JSON Schema Organization continues to exist, all stable specifications will be available with a stable URI" |
Beta Was this translation helpful? Give feedback.
-
|
Oh, then yes. The spec will have a stable URI on https://json-schema.org. I didn't include that because I felt the publishing location was implied. @egekorkan I did include it in my PR. |
Beta Was this translation helpful? Give feedback.
-
|
Yes I have meant the specification but the meta-schema also belongs to it in a way, so that is also of concern.
While this is fine, it does not express a guarantee that the URL will never change or be removed or whether the document behind the URL will change or not. It can be still deprecated or superseded by another spec version (sort of implied by the feature life cycle). Now that I have written this, the discussion here does not take a specification document lifecycle into account but focuses on the feature lifecycle (which is also important). The W3C document lifecycle is available at https://www.w3.org/2023/Process-20230612/#rec-track in the first figure here. The final stage of a document is Recommendation, where it is mentioned: A W3C Recommendation normally retains its status indefinitely. There is a lot of text in that document that takes a bunch of conditions into account. |
Beta Was this translation helpful? Give feedback.
-
I'm happy to add that to the PR. (done) I do mention this in the opening text of this discussion, though:
|
Beta Was this translation helpful? Give feedback.
-
|
@egekorkan what are your thoughts of a URL that redirects to the latest release within a version. For example, if version 1 has then We'll be doing this with the meta-schema |
Beta Was this translation helpful? Give feedback.
-
Sorry for missing that. I think all the messages are being conveyed.
That is good. For a spec that references to it, they can use the EDIT: Moved it to this thread instead of creating a new one. |
Beta Was this translation helpful? Give feedback.
-
A note before we begin
@jdesrosiers and I have spent a couple months building this offline in the hopes that we can present a completed proposal to the team (or at least as close to complete as possible).
The process laid out below is the initial process and may need to be updated as new features are introduced.
For example, since vocabularies are being extracted from the spec into the feature life cycle, support for custom keywords needs to happen another way (which is described below). When vocabularies are eventually promoted to stable status, this process will need to be updated so that vocabularies are recommended way to extend JSON Schema.
The point is that this process is not set in stone, and we can continue to evolve it as we need to.
I've also started a number of threads with questions that Jason and I thought could use some input from the rest of the team. Please read through those and provide your thoughts in the appropriate thread. If you have other questions, please feel free to start new threads. I will update this main post as those questions are resolved.
UPDATE
A PR to create a process document has been opened: json-schema-org/json-schema-spec#1500
Specification Development Life Cycle
Stability guarantees, versioning, and working with breaking changes
Releases will occur annually (e.g. 1 Jan) and will contain features which have achieved "stable" status.
A version consists of a set of consecutive releases which contain no breaking changes (also see the "Meta-schema" section regarding identifying versions).
Stability is implicitly and indefinitely guaranteed within a version.
If a breaking change is necessary, the Core Team will publicly announce the pending breaking change. Typically, the change will be included in the next release, however The Core Team reserves the right to delay releasing breaking changes. The Core Team MAY decide to withhold individual breaking changes until later versions. (Not all changes need to be included in the next version.)
Specification development only occurs in the next release. Published releases will not be updated retroactively. Fixing errata, such as spelling mistakes, MAY be permitted, however changes to feature descriptions might be too much for a back-fix. The Core Team will decide the severity of a back-fix on a case-by-case basis.
Changes within a version
A new release within a version MAY include non-functional changes. Examples of non-functional changes are:
Functional changes within a version MUST be compatible with existing functionality and MUST follow the full feature life cycle. Examples of compatible functional changes are:
If any functionality is broken as a result of a particular change, no matter how little the expected impact, it is considered a breaking change.
Unknown keywords
In order to support future-compatibility, keywords which are not known by the implementation MUST be disallowed.
The keyword prefix
x-defines a safe space for users to introduce custom annotations without the need for an explicit custom keyword.Implementations MUST refuse to process schemas which contain unknown keywords.
Meta-schema
The published meta-schema will be closed, enforcing the restriction on unknown keywords by using an
unevaluatedPropertieskeyword with thefalseschema.The custom annotation space is enforced in the meta-schema by using a
patternPropertieskeyword with a^x-pattern with atrueschema. (This can be added to the Core vocabulary meta-schema.)Experimental keywords will be added to the appropriate vocabulary meta-schema with a schema which is empty (no validation) except for a
$commentkeyword to explain that the keyword is experimental and where to find more information. (See the Proposal stage.)To support custom keywords, implementations MUST create a custom meta-schema which defines their keywords and:
unevaluatedPropertiesit contains)"$dynamicAnchor": "meta"Feature life cycle
New features MUST progress through a sequence of stages before being added to the specification, and existing stable features must be formally deprecated before being removed. The stages of the life cycle, in order, are:
Concept
The feature life cycle begins with an idea expressed in a GitHub issue in the specification repository. Initial discussion may occur in Slack or another space, but the life cycle does not formally begin until a GitHub issue is created.
The issue and subsequent discussion should cover how the feature could work, use cases, alternatives, whether it’s a breaking change, etc., with a goal of deciding rough requirements.
During this stage, members of the Core Team will implement private prototypes of the ideas expressed in the issue to get a feel for how it integrates with the stable features. Questions to address:
Proposal
Once the idea has been formed and agreed upon, a proposal will be written, separate from the specification (format TBD), with the goal of precisely defining specification changes.
In addition to the proposal, the published meta-schema for the appropriate vocabulary will be updated at this stage to include any new keywords with a schema containing only a
$commentkeyword to explain that the keyword is experimental and where to find more information. This is done so that these keywords are allowed but are not validated as their definition may change during the proposal/prototype process, and to permit different implementations to support different variations of the proposal throughout this process. It will be up to the implementation to validate these keywords in accordance with their support.Finally, tests are added to the Test Suite (somewhere that indicates they’re still WIP).
Experimentation
With specification language defined, the meta-schema updated, and tests provided, implementations may begin to support the new feature.
It is expected that feedback from implementers and users will result in refinements to the proposal. Sweeping changes MAY occur, but are highly discouraged.
In order to proceed, a minimum number (TBD) of implementations MUST support the feature, and there MUST be evidence of use in the wild (metrics TBD).
Users MUST NOT expect experimental features to be interoperable across implementations.
Stable
The proposal is incorporated into the specification, and the feature will be required as of the next release.
The appropriate vocabulary meta-schema is updated to include a subschema that validates the feature's syntax requirements.
The appropriate tests are moved out of WIP status.
Deprecated
If a feature is no longer useful, e.g. it has been replaced, it may be deprecated.
Implementations MUST still support deprecated features.
Removed
Features MUST remain in the "deprecated" state for a minimum time (TBD). After this time, they are removed with the release of the next version (may not be the next release).
Features to be extracted
propertyDependenciesThis feature is new and should be added via the feature life cycle.
Vocabularies
This feature is incomplete (and needs to undergo some changes anyway) and should be added via the feature life cycle.
Because this is being extracted, the specification will need to permit implementations to support custom keywords. However, since this practice produces non-interoperable schemas, it should be discouraged. Text like the following is recommended:
Later, when the vocabularies feature is promoted to a stable, it will be touted as the recommended, interoperable way to support custom keywords.
Output formats
It has been determined that the output formats defined in 2020-12 are geared toward machine consumers, but there are other consumers that the output does not consider, like human readers.
The output formats will be extracted to a new specification, allowing other specifications to fulfill the needs of other target audiences.
The Core specification will only identify terminology that should be used across all output format specifications in order to foster consistency.
Implementation support
Implementations MUST indicate the year of the latest release for which they fully support all stable features. For instance, if an implementation fully supports Release 2024 but only 3 of 4 features from Release 2025, then that implementation is said to support up to Release 2024. Implementations SHOULD also indicate the version associated with the release.
Proposal support
Wider support across implementations increases the likelihood that a feature will be used.
However, it's recognized that implementations are typically open source and often hobby projects. As a result, it's somewhat unrealistic to expect many implementations to stay up-to-date with proposals since many maintainers likely won't have time.
An implementation which does not recognize a proposed feature or the particular form for a proposed feature (i.e. the proposal has iterated) MUST refuse to process the schema.
Beta Was this translation helpful? Give feedback.
All reactions