Workweek reftests

Reftests

• jack: Do we want to run 1:1 or ??
• mbrubeck: current status is reftests no longer depend on monitor density - if we want to run hidpi, we need to set that explicitly.
• jack: should we test it both ways?
• mbrubeck: sure, we could run all tests at 200% zoom, except for the ones known to break. lots of our tests test layout; zooming happens in the compositor after layout is done, so a few tests to ensure things are scaling as expected.
• jack: what is a good reftest for hidpi?
• mbrubeck: want to know that text is rendered crisply; images are scaled up and not blurry.
• jack: not sure how to test text.
• mbrubeck: could have a reference image like acid1
• pcwalton: that will break on different platforms. have one that is hidpi, one that is not, test that they're different, so it's a smoketest.
• mbrubeck: a bunch of people are on macbooks, so we should notice obvious problems too.
• jack: ok, file a bug about that.
• jack: csswg tests have a different format, so gw will write a new harness using kmc's parser.
• gw: came across it when working on float issues, so pulled in some css2.1 reftests and they all failed. new format supports chaining links (a should match b and c) and not match (a should not match b). once kmc's parser is ready, do we want to do something where we preprocess the new format into our .list one, or what?
• jack: writing something to write out gecko's format is about the same amount of code to just run the new format in the harness. right now reftest harness takes list of .list files as input, could change it to take html files as well.
• gw: how long for the parser?
• kmc: a few weeks? if you just need head tags, you can probably do that today since it's just tokenization.
• gw: chromium uses a python parser script to pull out the content; I could do that, but if ours will be done soon I can wait.
• jack: does anyone here not know what reftests are and how to write them?
• silence
• jack: we should be more selective about merging commits that don't have reftests. mrobinson has been making reftests that fail before he starts working on something, then makes a PR and merges it commented it out. if we had expected-fail support in the new format, that would be even better...