Agenda for Nov 3 meeting

[foolip]

Here's the agenda for our meeting on Thursday, Nov 3:

• State of CSS 2022, results of the 3 freeform questions
• MDN short surveys design #196 is now live
• Expanding 2021 and 2022 focus areas for 2023 #119
  • Agreement to carry forward 15 focus areas as a small part of the score? (see @jgraham's comment in particular)

Previous meeting: #240

[nairnandu]

Attendees: @chrishtr @dlibby- @gsnedders @jensimmons @jgraham @meyerweb @nairnandu @nt1m @rachelandrew @robnyman @tantek

Brief meeting notes:

• State of CSS - Summary from @foolip

  • The survey is now closed! 14115 people took the survey and 10147 finished it.
    • https://2022.stateofcss.com/en-US/usage live but WIP
    • https://2022.stateofcss.com/en-US/usage is most relevant to us
    • This spreadsheet has freeform responses
    • Also worth looking at trends for specific features, like backdrop-filter
    • Significant work to link results for each proposal, if anyone wants to help we can divide up the work
    • [@jgraham] Browser incompatibility graph is maybe more interesting to this group. Appears that 90% of the respondents don't have an opinion
    • [@jensimmons] Interesting to look at the raw data
    • [@gsnedders] There is a lot of overlap, like css nesting was in the incompatibility section and in the “what you would like to see” section
    • [@chrishtr] tl;dr - please review the data

• MDN survey - Summary from @foolip

  • Running since Monday, results preview from Ruth already shared (and attached to this email)
  • Ranking look roughly plausible, but unclear how many responses are needed for it to stabilize, given question type
  • Plan to post a comment on each affected proposal when survey is done
  • [@robnyman] Some insights
    • “Web components” is quite well represented. Same with “file systems” and “push notifications”.
    • Some are specific, whereas some are very broad - like codecs
    • CSS Media queries and Math functions were well represented in the CSS part of the survey

• [@gsnedders] Positions process

  • [@chrishtr] Would like to have Philip in the discussion. Lets put it on the agenda for next week.

• Expanding 2021 and 2022 focus areas for 2023 #119 - Agreement to carry forward 15 focus areas as a small part of the score?

  • tl;dr - needs more discussion on some areas
  • [@jgraham] general question is what do we do with focus areas from last year. In 2022, we carried over ones from 2021, with slightly reduced weight. Are we doing that again? Should we do something different? One option would to be make a new proposal for bringing forward one from last year. There are a few options:
    • Pull proposals forward as-is
    • Add more tests to the proposal this year. This would retroactively affect scores from previous years
    • Ignore tests that are already passing. Carry forward only the failing tests. Specifically verify that tests are valuable and if needed, adjust.
  • [@chrishtr] The focus areas are important. Both to avoid regressing and also improving those areas. Efforts on regressions should be minimal. New tests - most of it resulting from the bug fixes we have done. We should fill in those gaps and improve interop over time. Fixing those tests across all browsers needs investment and we should do it anyway. Maybe give minimal scores as we bring them forward.
  • [@jgraham] We should not retroactively change scores from previous years. That said, agree that new tests are important. Agree that pulling everything forward with minimal scoring. In retrospect, we should handle this ahead of time.
  • [@jensimmons] isolating each year sounds good. It's a bad idea to include new tests in last year’s focus areas. It's a good idea to keep these tests around, even if all browsers are passing. However, we should give it a very low score. This gives a signal that we are maintaining a list of core tests and ensuring that nobody regresses on those. If we propose a new focus area for this year, we should not tie with something from previous years.
  • [@jgraham] Long term - are we building up a list of things? Its okay to say this was a thing for Interop and got it to 100%. I am more inclined to say that we should focus on things we are doing this year and not on things from previous years even if its a small %
  • [@jensimmons] Maybe we can have the list from previous years, as an appendix. Viewport as an example - do we pull that out? There is more that can be done.
  • [@jgraham] For viewport example - maybe in 2024 we have really good mobile testing infra and we can test that again, as a new focus area
  • [@gsnedders] We can break this down to 3 smaller questions
    1. Yes/No - on carrying forward proposals?
       • [@jgraham] there is consensus on moving all 2021 and 2022
    2. [@gsnedders] Do we want to expand on things from past years?
       • [@gsnedders] Do we want to carry forward all passing tests?
       • [@nt1m] We could do a survey for which focus areas to include?
       • [@chrishtr] It might be best to not ask for re-prioritize
       • [@chrishtr] If we carry over the failing tests there is a risk of mis-representation on the quality of Interop (starting from zero on a feature).
       • [@jgraham] Carry over everything with a minimal score
       • [@nt1m] strange to talk about this at a test level. Can we look at this from a focus-area level? Arbitrary to split tests for a feature between 2 years
       • [@jgraham] Do we think this is about a set of tests that represent a feature in a time period
       • [@nt1m] If a new feature is added in a category, it should be
       • [@gsnedders] worthwhile splitting couple of things. Do we allow the scope to increase by including new additions to the specification?
       • [@jensimmons] Grid focus area for 2021 was Grid Level 1. There is additional refinement being made, albeit small changes. Grid Level 2 is not included and it is separate. New features need to be a new focus area.
       • [@nt1m] It’s a bit early to talk about weighting since we don't have focus-areas for 2023.
    3. [@gsnedders] For things where we want to expand coverage - do we want to consider those as new focus area proposals in 2023
       • [@jensimmons] 2 parts to this question:
         • Do we add new features?
         • Where do we draw the line in terms of new features vs new tests in old features?
       • [@gsnedders] There is consensus that we do not want to increase the scope. New tests is the area that needs a decision
       • [@jgraham] We could have a separate focus area for the new tests, something that is generic.
  • [@gsnedders] Re-visit this next week