Retrospective: transparency in proposal selection and feedback

[foolip]

This is part of #274.

⚠️ First, I want to acknowledge that this is potentially a controversial topic, and I'm trying to represent things as fairly as I can. Still, how this played out will look different to different people and this is my perspective, not a neutral/objective summary.

As we were starting proposal evaluation – around the time of #240 – we discovered that there was a difference of opinion/preference within the interop team about what parts of our process should be public. What we ended up doing is documented in Interop 2023 Proposal Selection. In summary only the outcome of the process was made public, and feedback on proposals was given by the interop team jointly, after considerable review of that feedback.

A few things didn't work great, from my point of view:

• The issue was not discussed in RFC 116: Interop 2023 rfcs#116 and not documented in the RFC. My assumption and intention was that the process would be transparent, making per-proposal decisions by consensus, and documenting those decisions in meeting notes or equivalent. I made this assumption because almost everything in web standards and WPT is public by default, and the Interop 2022 positions were made public via GitHub issues.
• Because we didn't have agreement to make individual positions public, we had to write joint feedback on proposals. In some cases the result was crisp and clear, IMHO better than individual feedback would have been. But in other cases the feedback was watered down since we didn't always agree about importance or priority, and used a common response instead.
• It's unclear what expectations we have for participants talking about their own positions on features in public, since doing so would implicitly say something about others' positions.

I'll not suggest any changes to the process in this issue, but I'd like to revisit this for Interop 2024 and I think we'd all benefit from documenting this clearly up-front before the process starts.

[slightlyoff]

@dandclark and I have been chatting, and while Microsoft has a concern about the lack of transparency in votes and vetoes, it would seem to me that the larger issue is that we're no longer feeding Interop from a needs assessment survey.

That is to say, the legitimacy of selections has historically derived from direct evidence of developer interest, whereas in recent years that has not been so tightly paired with the categories that have made the cut.

This risks eroding the project's value and and creating the perception that selections are political or can be gamed. A lack of transparency in voting and vetoes potentially bolsters this perception. Until and unless the selection process is re-grounded on evidence, sunlight regarding proposal support feels like an essential minimum to ensure the effort doesn't fall into disrepute.

[foolip]

@slightlyoff I'm glad to hear that Microsoft has also been discussing the process, that's great!

On developer interest, I think that we actually did pretty well with gathering feedback this time around:

• Angular input on Interop 2023 #238
• Results of MDN CSS & HTML survey #245
• Results of MDN APIs & JavaScript survey #246
• Results of State of CSS 2022 Browser Incompatibilities #248

There hasn't been a big survey like the MDN Web Developer Needs Assessment since 2020, but I agree that would be valuable. Over in https://www.w3.org/community/webdx/ we're looking at various forms of shared research, and I hope that for next year we'll have more input than we could get from State of CSS + MDN short surveys + frameworks this year.

Anyway, can I suggest another issue or comment in #274 about developer needs?

[myakura]

(reducted my comment which was meant to be posted in #285 (comment). sorry for the noise).