Clarifying spec requirements for Interop 2025

[foolip]

The Interop 2024 proposal template said this about the specification:

The feature must defined by a standards-track specification from one of the following organizations:

• IETF (Standard Track)
• Khronos Group
• TC39 (Stage 2+)
• W3C (Recommendation Track)
• WHATWG

This issue tracks changes or refinements we want to make. So far:

• Should we accept ISO specs?
• What's the bar for Wasm features?

[foolip]

For Wasm, I would suggest Phase 2+ as the bar.

[jensimmons]

Let's add:

• Alliance for Open Media
• ISO

In fact, I'd recommend adding them to the current process proposal. And adding an "etc" to the end. I don't believe we are intending to gate proposals by saying "oh that web standards org doesn't count, only these ones do". It seems our intention is to say "this must be standardized in an industry web standards org". And it's ok if we add some as we go, as we think of technology we want to include. The availability of tests will be more of a blocker than which standards org something is part of. If we someday want to exclude a particular standards org, then we can do so later.

How about changing the language to this:

The feature must defined by a standards-track specification from one of the following organizations:

• IETF (Standard Track)
• Khronos Group
• TC39 (Stage 2+)
• W3C (Recommendation Track)
• WHATWG
• WASM (Phase 2+)
• Alliance for Open Media
• ISO
• or other similar industry web standards organizations

[jensimmons]

I created a suggested amendment to the proposal PR at #591

[jgraham]

We got the feedback internally that Phase 2 is too early for WASM.

For something to be in Interop we'd expect it to be reaching Phase 4 by the end of the year, since we'd presumably have multiple implementations all passing the testsuite. In practice that seems extremely unlikely unless something is in Phase 3 already, and accepting Phase 2 proposals risks rushing the standards process, and requiring difficult, incompatible, changes to already shipping features.

[jensimmons]

So is your recommendation that we say "WASM (Phase 3+)" or "WASM (Phase 4+" @jgraham ?

[jgraham]

Working backwards, I'd expect that after Interop a proposal would be in at least Phase 4 i.e. multiple implementations would pass the testsuite. Therefore that doesn't make sense as an precondition for being in Interop in the first place.

Phase 3 requires a testsuite to exist, which is a necessary precondition. It doesn't require complete formal spec text to exist, which does also seem like a requirement, and one we have for other specs in other groups.

So in practice I think "Phase 3" is the minimum bar, but for a proposal to be accepted, we'd additionally look for a usable specification, and agreement that it is unlikely to undergo substantive change. However since those additional criteria don't map cleanly to the phases, I think "Phase 3" is the right formal requirement.

[jensimmons]

I agree that any specification should be judged independently to assess whether it's actually ready or not. The levels are applied inconsistently, plus 'it depends' factors in heavily.

While we are at it... I don't think W3C requiring (Recommendation Track) is accurate. We often include things from W3C that are in Working Draft — if we deem they are ready. Rec is an incredibly high bar. And just not what we are doing.

No one is suggesting that if a standard has a certain level, then that's all it needs to be considered 'done enough'. We are defining a minimum bar, where any standard that doesn't have at least this should be considered disqualified.

[zcorpan]

@jensimmons Recommendation Track and Recommendation are different things, see https://www.w3.org/2023/Process-20230612/#rec-track

i.e. any of FPWD, WD, CR, PR, REC are rec-track.

[jensimmons]

Ah, you are correct. Nevermind my comment then. Yes, on track to become recommendation.

[foolip]

Let's go with Phase 3+ for WebAssembly then. I've suggested an edit at #591 (comment)

[foolip]

This has been integrated into #591 now.

[jsnkuhn]

I'd like to recommend including links to explainers of what thing like "Recommendation Track" mean instead of links to the wider organizations involved. This explanation https://www.w3.org/2023/Process-20230612/#rec-track is considerably more useful than sending us generically to w3.org. Honestly, I've been in web dev for almost 20 years at this point and have only really started to grasp the process for specs since I started posting in the csswg github a few years ago. I'd imagine most devs would not be able to explain to you what "Recommendation Track" specifically means or even know where to go to get that explanation.