WCAG 2 and APCA Comparison #131
Replies: 28 comments 17 replies
-
Andy this is a real eye-opener! The 49%. Incorrect passes is really hard hitting. I wondered if you had published this study with thousands of colours in the public domain? If not, please can you explain how the thousands of colours pairs were generated, was it just three random numbers all between 0 and 255 for the foreground and similar again for the background? Also, do you have a figure for how many colours WCAG 2 fails when it ought to pass? Many thanks. |
Beta Was this translation helpful? Give feedback.
-
@sdw32 I have not published the complete model, it was summer of 2019, and an earlier model. Now, pretty sure 2.x would do worse, but not gonna beat a dead horse. They were pure random tuples, I'd need to dig it up to see what domain they were in, but most likely linear 0.0-1.0, for each channel, then apply gamma and multiply by 255, so: rnd()^0.45 * 255 It is important to point out that no designer would use pure random colors, it was only to point out the deep confirmation bias that surrounds that. |
Beta Was this translation helpful? Give feedback.
-
Thanks for the follow-up. I think this issue remains important to fully investigate, contrast ratio thresholds are far from dead! I think are presently used to determine the outcome of ADA lawsuits and Section 508 compliance for procurement contracts, and will continue to do so for the indefinite future! As a group in the LVTF, we still have a decision to make regarding whether the supplemental guidance uses ratios, or APCA, or both. If we do anything other than using ratios on their own, we will have to write some content justifying why we didn't just use contrast ratios. Understanding the full extent to which the ratios model is broken would helpfully inform this decision I think. Did you also have a value for the number of colours that failed the ratios model when they ought to have passed? I'm imagining something like an Excel spreadsheet that can calculate both WCAG2 and APCA contrasts from RGB input values for foreground and background, which could be randomly generated in Excel. I wondered if this is something that you already had, or if you would be able to make it? This page looks like it has the VBA custom functions for WCAG contrast, so would just need similar VBA custom functions for APCA [https://ramblings.mcpher.com/color-fiesta/playing-around-with-colors-in-vba/#Calculate_contrast_ratio] Happy to chat further on this topic during the regular meetings. |
Beta Was this translation helpful? Give feedback.
-
Hi @sdw32
Are you talking about the WCAG 2 ratio? or ratios in general? or the ratios of some specific national standard of some nation? There is no contrast ratio requirement under current ADA unless something changed very recently. As for lawsuits, not sure the context you mean. The WCAG 2.x contrast ratio is different from other standards, and not directly compatible with any of them, and only exists for use in web-context. It's also not accurate (perceptually or otherwise) as has been shown over and over. Some other standards (ISO, ANSI, etc) may use some variant of Weber or Michelson, or something completely different. Most of them are NOT ratios. Australia for instance uses something I can only call "bizarre" and resorts to brute force due to its incorrect shape. The UK uses LRV difference, specifying a 30 point difference. But once again, this is not a perceptual standard. Compared to APCA, a 30 point LRV difference would rate anywhere from a low Lc 19 (if one color was white) up to Lc 47 if one color is black. This is the exact reverse of the WCAG 2.x problem, but as the UK standard relates to physical signage it's possibly a less serious problem than WCAG 2.x, as physical signage is also dependent on ambient illumination, so forcing contrast higher for darker colors has some benefit. Still, it is a brute force solution. (And when I say opposite problem, WCAG 2.x causes darker colors to have lower contrast by over reporting).
I've written about 145,000 words on this, so I'm guessing you haven't read much of my work? I won't ask you to read all the in depth material, but here are a few very brief articles with many visual examples: The Lighter Side of Dark BackgroundsOrange You Wondering About Contrast?WCAG 2 vs APCA • A Contrast in Applied MathsThe issue of using contrast ratios to describe perceived contrast has a long history of wavy pseudoscience as described in this paper by Dr. Arditi. This kind of thing is partly why I am doing this research, to add to the knowledge in this area, and the papers I am preparing focus on these kinds of issues.
I've posted this and others several times. I'm surprised this is the first you heard this as I've written about it a lot. I did several runs with 1000 colors (500 pairs). There were some more recent test runs where the incorrect pass rate was as high as 75%(depending on criteria such as use case). This was from 2019. I'm not inclined to continue much of this, as the proactive choice is developing better solutions going forward. The few articles and examples are all in response to specific requests, but I'd rather focus on other positive actions.
Yes I already have of course. There is an APCA spreadsheet pre-built (LibreOffice) at the APCA GitHub repo. And the rest I have written about extensively in thread 695 and others. As a side note, when writing posts here, enclose URLs in parentheses not square brackets. Use the square brackets for the clickable text. Basically like this:
Note there is no space between the ] and the ( As for the content on the link you shared, there is enough wrong there that I suggest you disregard. Here is an oft cited site for maths for displays and color etc: http://brucelindbloom.comBruce uses the ASTM values, sometimes very slightly different than some other sources. I hear that easyrgb is also good, but have not investigated that site much. I use the CIE Colorimetry (2004 and 2007), ITU, IEC, and SMTPE standards as appropriate, writing my own code, as I don't use any code snips off internet sites anywhere because apparently very few people actually know the correct math, much less how vision works, and it's rather shocking. Even some commercial software like Matlab got it wrong. On github there is a python library called ColourScience that's solid, but can't vouch for many others. |
Beta Was this translation helpful? Give feedback.
-
Thanks for all the additional info. I have now completed my own exploration of APCA Perceived Lightness Contrast versus WCAG2 contrast ratios. I programmed both as custom functions in Excel, then used the spreadsheet to compare passes and fails for white text on coloured backgrounds, and black text on coloured backgrounds. Perhaps unsurprisingly, I agree with the conclusion that WCAG2 is not fit for purpose. When one of the colours is black, it incorrectly passes a great deal of colour combinations, which can be so dark that they are almost completely unreadable. When one of the colours is white, it also incorrectly fails a great deal of colour combinations that are actually fine. For my own analysis in Excel, I did completely random integers in the range 0-255 for both foreground and background colours. I ran circa 5000 combinations, obviously lots of these were completely nonsensical, but 1393 colour pairs were considered viable by one algorithm or the other. Of these,
Having completed my exploration, I am now of the opinion that the minimum contrast threshold is within the supplemental guidance for WCAG2 should be based on APCA Perceived Lightness Contrast. I don't currently believe that it's worth spending effort trying to develop patches to fix the WCAG2 algorithms, it would be difficult to develop a patch for the dark pairs problem, which didn't make the bright colour pairs problem even worse. (and vice versa). I will look forward to further discussions on this topic. |
Beta Was this translation helpful? Give feedback.
-
Thanks for the follow-up. Apologies, my previous post was not clear, I was testing 3:1 WCAG2 against perceived lightness contrast>45. I have just now repeated the analysis based on testing 4.5:1 WCAG2 against perceived lightness contrast>60. On this basis, the number of colours considered viable by either algorithm reduces to 596 colour pairs. Of these:
This is now pretty close to the toss of the coin, which Andy has previously reported. Regarding patches, increasing the contrast ratio would of course help to solve the dark colour pairs problem, but it would also make the bright colour pairs problem worse. In order to properly solve it, I think we would need to make the viable threshold depend on the brightness of the brightest colour, but I fear this would end up being too complicated to be practically useful. I will look forward to discussing this topic further in our regular calls. |
Beta Was this translation helpful? Give feedback.
-
Thank you Sam it's good to have independent verification. Yea the only "patches" that are practical as the few I mentioned, but they too are somewhat brute force. The very rapid and wide acceptance and integration into contrast tools caught me by surprise. But this also means if there is a WCAG 2.3, the better bet than a patch is a WCAG 2.3 version of APCA, though this is not without issues, in particular if the old method "has to pass too" that kills the flexibility of a lot of APCA... I'd need to evaluate how much impact that would be. The reason is that WCAG 2 states 3:1 for "everything" and then 4.5:1 for small text. Converting to WCAG 2 ratios (roughly in my head) APCA's spread is much wider, as large non-text items that are non-lexical like a button form can be Lc30 which is about 2:1, moving up the scale, APCA is actually stricter except when the text is brighter than about But on that point, one of the guidance areas not yet mentioned is an overall page background that is not brighter than about #ddd. Here's a demonstration of that reason: |
Beta Was this translation helpful? Give feedback.
-
Regarding page backgrounds - I actually prefer to the full white background it is much easier for me to read than the other one you provided. I'd note that Word, outlook, slack, onenote, etc. all have black on white as well as the default. I know I may be an exception and so understand that for most people including people with some types of visual impairments it is not best - but for some low vision users that black on full white is needed -- so we need to at least acknowledge that some people require that and may want to change settings for that - there are no absolutes in this world and some people need the extremes. The bright background keeps me focused. I would personally agree that full white text on a totally black background is not good for reading for me and is much more problematic probably for a larger set of users. |
Beta Was this translation helpful? Give feedback.
-
I have now refined my analysis that tests randomly generated colours, on the situation is even worse than previously reported. This time around, I have focused the analysis on two real-world scenarios that are particularly relevant:
On this basis, the false pass of dark colours only occurs in the first scenario, and the false fail of bright colours only occurs in the second scenario. Considering the first scenario, I generated 2000 random colours, 1000 colour text on black background and 1000 black text on colour background. Of these 2000 colours, WCAG2 suggested 1286 passed AA 4.5to1. APCA suggested 570 passed perceived lightness contrast>60. This gives a total of 716 colours that passed WCAG2 when they ought to have failed, a whopping 55.7%. So, if one of the colours is black, and the other is random, WCAG2 appears to be less accurate than tossing a coin. If we look at the second scenario, I generated 2000 random colours, 1000 colour text on white background and 1000 white text on colour background. Of these 2000 colours, WCAG2 suggested 733 passed AA 4.5to1. APCA suggested 1122 passed perceived lightness contrast>60. This gives a total of 389 colours that failed WCAG2 when they ought to have passed, which is 34.7% of the 1122 colours that ought to have passed. I made a webpage where the left hand column showcases the entire set of WCAG2 false passes (black & random colour). The right-hand column showcases the entire set of WCAG2 false fails (white & random colour). http://www.cedc.tools/pairs.html. For the sake of this webpage, I set the font-family to Arial and the font-size to 12 PX. I sorted the table by APCA perceived lightness contrast, so the most epic incorrect pass for black & random colour is at the top of the left-hand column. This is R56,G130,B88 text on pure black background, which just passes 4.5:1 but is perceived lightness contrast = 30. The most epic incorrect fail for white & random colour is at the bottom of the right-hand column. This is white text on R43, G133, B113 background, which just fails 4.5:1, but is perceived lightness contrast = 76. The bottom line here is that for display screens in typical lighting conditions, the eye perceives the brightness contrast of text as a difference, not as a ratio. If the model based on ratios was correct, then everything in the left-hand column of http://www.cedc.tools/pairs.html should be easier to read than everything in the right-hand column. Evidently, this is not the case. I will look forward to discussing the implications of this in due course. |
Beta Was this translation helpful? Give feedback.
-
@sdw32 I think it really helpful to have this comparison chart for people to compare and also get and idea of the colors we are talking about. I generally I agree with the most of the dark combos on the left being harder to read and causing issues - I agree this is a limitation in the current algorithm. 2 in the dark combos that I found no problems with that were failed were: who but new as now two what come very how up to she use want me on if out their 9.65:1 PLC=59.7 | In terms of the white text ones - while there are some that WCAG unfairly flags that are really pretty easy to read - there are some on the list of APCA passes that are very hard for me to read. I like that APCA is a sliding scale because size also does make a difference and for me which color is the foreground and which is the background make a big difference. For example, a white text on green may be ok for me but a green text on white background is not. Also some of these that are problems at 12pt for me likely would be ok at 17pt normal weight for me. |
Beta Was this translation helpful? Give feedback.
-
Hi Sam and Jonathan @sdw32 @mraccess77 Independent Validation ThanksThank you Sam for your deep analysis and validation here, especially as my paper on APCA is not ready to submit, and I know some have been eager to see some level of peer review. You are one of the few other maths-accomplished individuals here, and it's very good to hear this feedback. I've had to work doubly hard to ensure I am not falling into any confirmation bias which is a very real issue in perception research. Contrast ThresholdsFirst, Just as an FYI, 59.7 can count as 60. The spec indicates rounding to the nearest integer. But do keep in mind that size is a critical component. Also, when used in a pass/fail way, the Lc60 and Lc75 levels are part of the "simple" APCA, originally intended as a replacement for WCAG 2 — the "full matrix" APCA does not set those as a pass/fail, but a sliding scale for a given font. As we move forward, I expect two distinct options for designers, one a sliding scale for design and testing to accomodate more flexible design needs, or the alternate choice for a site is the simple test version, which is set up more like the WCAG 2 SCs. Size/Weight Thresholds etc.APCA contrast is tightly connected to font weight and size for fonts smaller than about 32px. I AM working on the lookup tables, and the current lookup tables are a little off by a px or two. On Sam's experiment page, a 12px font is used. But per the APCA guidelines, 12px is discouraged for any fluent text, and needs to be Lc 90 if used for fluent purposes. The "standard" is moving to 16px at Lc 75, and 24px at Lc 75 and might relax from these values to 14px and 21px for 400 weight. Assuming weight 400-500:
These values are still under study. For a dollars to donuts comparison, at Lc60 I suggest using no less than 21px, and at Lc75 no less than 14px for a 400 weight font. History and Early EvaluationsAgain Sam thank you for the independent verification... I am working on more than one paper regarding this, and I intend on gathering additional empirical data before I do, though there seems to be some impatience regarding me publishing. I also realize that the volume of my writing has been hard for some to digest — I've been stating these issues since early in thread 695, and your experiments here echo my early experiments. it's useful to hear it in a different voice. Some key experiments are still online. Here are a couple links if you are curious about some of the background and early development. The list of experiments that I had made public are here: https://www.myndex.com/WEB/Perception Some of them have the link "offline" to prevent search engine linking and the resultant confusion, but I can give you a link to any that are listed, and some links are obvious as I outline below. But please keep in mind that I am FAR past these as a source of data. These are more of historical interest. HistoricalThese links are a few of the historical investigations. They contain obsolete material. We are far past this initial material, and the most recent material has ventured into a proprietary area and will be released when published.
TL;DRThanked Sam for the independent verification review, listed a few specifics and work in progress RE: the lookup tables. |
Beta Was this translation helpful? Give feedback.
-
Thanks to Andy for the detailed follow-up. I had a quick look at issue 695 and it is unbelievably long! Congratulations for your years of persistence on this topic. Regarding the size of the text on my test page, I was primarily interested in comparing one contrast calculation algorithm against the other, so I had intentionally set the text to be 12 PX for everything, which is slightly on the small side, intending to give both algorithms a good workout at something close to the limit of readability. However I can see why this has caused confusion as the PLC threshold that I was testing at was not intended to be applied to that text size. I have now changed the text size to 16 PX, even though this still isn't quite at the actual intended size threshold for PLC>60, it makes all the text easier to read from both algorithms, while still being small enough that the differences between the algorithms is still apparent. Furthermore, I've been trying to think about the contextual factors that would influence the severity of the dark-pairs false pass and bright pairs false fail issues with WCAG2 (4.5to1). Lots of these factors have already been mentioned before in 695 and elsewhere, but please can you check if I have summarised these correctly?
I had a go at playing with the brightness and contrast settings on my monitor, and this didn't seem to favour either of the columns of my test patches In order to consider the significance of the colour of blank space on the screen, I made a version of test page where the entire background is black and made a separate page where the entire background is white. The dark pairs appear to be considerably more readable with the black page background. I also had a look at my test pages on some different laptops and monitors that I have easy access to. The dark pairs appeared to be more readable on some of these monitors than others. So, I would be glad to find out more detail regarding the contextual factors for the experiments that were performed in order to calibrate the APCA model, and the extent to which these generalise to real-world usage of laptops, monitors, phones and tablets. Also, I'd be glad to know if I have missed / misunderstood anything critical, or if any of the above effects could be better described. Many thanks. |
Beta Was this translation helpful? Give feedback.
-
Thanks to Andy for all the additional details. As I understand things, the current state of play is that the contrast calculation algorithm, and some experimental results have been published, but the full set of experimental results that support the latest version of the contrast calculation algorithm have not been published. Additionally, there is a proposed algorithm that gives a minimum font-size for a given perceived lightness contrast and font-weight, but this algorithm has not been published, and neither has the evidence that supports it. From my limited experience with guidelines and IP, I would imagine that any algorithm or look up table that needs to be used in order to follow the guideline will need to be royalty-free for anybody else to use, and program their own software based on this. I would have thought any evidence that is necessary to support this algorithm ought to be reviewed before the algorithm is incorporated as part of the guidelines. If the supporting evidence has already been published then this is straightforward. If the supporting evidence is not intended to be published, then I wonder if NDA arrangements might be needed to enable such a review process? Furthermore, I would imagine that if someone creates software that uses the algorithms that eventually become part of WCAG guidelines, this software does not need to be offered freely. I must confess I am a little confused as to where the boundary is in this regard between the WCAG3 (silver) guidelines, and the tool that is here: https://www.myndex.com/APCA/. Nevertheless, for the supplemental guidance that we are discussing for WCAG2, I believe Andy has proposed contributing an APCA-Lite version that just contains a few of the size/contrast cut-offs, so this may make things a bit simpler. For the more imminent issue of the supplemental guidance, I'd be glad if someone could confirm if I have understood the above correctly, and/or describe the expected timeline/roadmap/procedures regarding access to review the empirical evidence that supports any algorithm that is proposed to be included in supplemental guidance. And finally, I'll summarise my current position on the APCA contrast algorithm as follows: my preliminary review indicates that APCA contrast calculation is better than WCAG2 contrast ratios, so I'd be delighted to see the supplemental guidance use this method instead of contrast ratios. However, I'm nevertheless reluctant to support the APCA algorithm, without having had the chance to review the empirical evidence that confirms it is better, for the range of expected real-world use cases. Would be grateful for any guidance or further help can be offered on this topic, many thanks. |
Beta Was this translation helpful? Give feedback.
-
@sdw32 Any terms related to the use of tools would be worked out at the W3C and Accessibility Guidelines level and not with the co-facilitators of the Low Vision Task Force. Ability to create tools that use the formula and access to research that validates it is an important aspect that they are aware of. In terms of how it impacts supplemental guidance - this is an area where we will need to keep flexibility in how we write the guidance so it could stand with, without depending on how the timing of those discussions go. They are tracking that issue - but I think it's worthwhile for the group and others seeking to use the tools to be aware of this dependency. |
Beta Was this translation helpful? Give feedback.
-
Hi @sdw32 and @mraccess77 At the moment, only my patent attorney and I discuss my IP. Because studies are ongoing, and delayed due to COVID, some things are necessarily still under wraps. I was very open initially, but due to the general atmosphere here I've been a little more careful, which is not my normal open operating method. I have nevertheless provided everything needed to duplicate my efforts. I have indicated where this information is. For the most resent refinements, there is some empirical data that is not collated into charts or graphs yet. Theory of operation, conformance, uniformity, comparitive superiority, have all been demonstrated with that information publicly available. APCA has been integrated into dozens of tools, and people are very receptive. I have shown in multiple articles with visual examples the efficacy. PublicationsThat said, I am working toward publication. I am funding this project myself, so there will not be any open access as the costs to submit to these pay for play deals is in the thousands. As a writer, I am used to being paid to write... in the scientific community, scientists are required to PAY to be published in open access journals. While they are not written in an academic format, the following articles have substantial visual comparisons, and are written in a more casual manner for a general audience. GENERAL-PUBLIC-ORIENTED & PUBLIC FACING:Fonts for Accessibilityhttps://www.myndex.com/PUB/PDF/AccessibleFontsD.pdf ContrastPart I:Orange You Wondering About Contrast? Answering some contrast questions, and demonstrating a real solution to the infamous orange conundrum. Part II:The Lighter Side of Dark Backgrounds An article comparing some parts of APCA with the old WCAG 2 contrast methods. NEW! Part III: WCAG 2 vs APCA Contrast ShootoutA gist answering some recent questions regarding APCA, and comparisons and examples of the old (WCAG 2 1.4.3) and the new WCAG 3 with APCA. Determining Luminance Contrast A discussion and code examples for determining Luminance, and various methods of determining luminance contrast. (Written prior to invention of SAPC/ACPA) LinksThe Myndex Linktree A bunch of links relating to APCA contrast anc color. SAPC and APCA The WCAG3/Silver Contrast Method and Algorithm repository. This is the canonical source for the latest version of APCA. SAPC Contrast Research Tools The WCAG3/Silver Contrast Algorithm beta site. Includes interactive experiments that demonstrate the concepts. perception studies site A list of publicly available experiments. CE17 includes a white paper with theory of operation. ColorNEW! Let's Flip for Color! If you want your text to be either black or white if the user selects some random color, just where is that inflection point? Hint: It's NOT 18% Y.Part I:For The Luv of Color An article comparing CIE Lab and Luv colorspaces Part II:Will Work for Color A follow up article on working spaces and related considerations. Introduces the concept of "Web Working Spacelets". COLORSPACES - The Primal Frontier A brief Look at the math that helps model how we see. And let's not forget the VIsual Contrast WIki, and try the experiments at the SAPC site. AS I HAVE ALSO STATEDThe following existing research has played a large role:
And obviously I am not listing the several threads such as 695. Below are a number of infographics created in this project. Thank you, Andy Andrew Somers THE REVOLUTION WILL BE READABLE™ANNEX: INFOGRAPHICSThese are copyright and not for distribution, not to be used without permission, here for discussion only.Demonstration: it's not too much contrast, it's too much white.This is an example of one of the guidelines I am working on relating to excessive contrast or luminance. WCAG 2 vs APCA — ShootoutChromaticity diagramCritical font size, based largely on the Bailey/Lovie Kitchin researchIllustration of the problems with font sizing and font metrics in generalA lookup chart of equivelelnt pixel densityAPCA to WCAG2 comparisonsRGB Spectral outputsContrast Sensitivity and Font WeightKey factor in creating the APCA font lookup tables. User interface of SAPC research softwareThis is the software I developed for determining the constants (exponents) for APCA. Not seen are the stimulus panels viewed my test subjects. I am not particularly comfortable showing these more fully as they are part of on going studies. Various studies or experimentsPlot of perceptual error of WCAG 2Three Equal Contrasts — but are all three equal in readability?Examples of context sensitivity.CVD depicted |
Beta Was this translation helpful? Give feedback.
-
I have slightly refined the demo pages that I created. http://www.cedc.tools/pairs1.html shows the whole page with a completely black background, with a white rectangle behind the text in the right-hand column. http://www.cedc.tools/pairs2.html shows exactly the same colour combinations, but this time the whole page has a completely white background, with black rectangles behind the text in the left-hand column. This demonstration shows that dark pairs are considerably more readable when the background of the whole page is black, compared to when the background of the whole page is white. Similarly, I would imagine that the dark pairs would be more readable in a dark room, as compared to a brightly lit room. Nevertheless, I think it's fair to say model ought to be calibrated to screens with white backgrounds in typical indoor lighting, in which case the dark pairs false pass is still of considerable concern. |
Beta Was this translation helpful? Give feedback.
-
Thanks Bruce for the follow-up. Inspired by your suggestions, I have now made several more demo pages. I have separated things out so that random colour backgrounds are now on a different page from random colour text I have also edited my code so that can produce tables with a variable number of columns, and reduced the number of words per line and tightened up margins a bit. So, there are now the following demo pages: Thinking about it some more, presenting these pages completely separately is better than the previous approach as the effect of darkness is now much better counterbalanced. Additionally, if you're interested in spot reading rather than fluent body reading, I have produced demo pages with exactly the same colour combinations, but each cell only contains the numbers for Perceived Lightness Contrast and contrast ratios, so you get a lot more colours on the screen at once. I will look forward to further discussions on this topic |
Beta Was this translation helpful? Give feedback.
-
Thanks Bruce for the comments, I agree that 'and also' is correct, and in fact, thinking about it further, communicating the difference between WCAG2 and APCA is more straightforward for spot reading. I've also been thinking about this topic further, and have now produced my best ever demonstration page showing how WCAG2 incorrectly predicts readability of text and colour backgrounds. The latest demo page is here http://www.cedc.tools/rback_numbers_vs.html Essentially, in every case, if WCAG2 is correct then every instance of black text ought to be more readable than every instance of white text, and all the black text is a pass. if APCA is correct, then every instance of white text ought to be more readable than every instance of black text, and every instance of black text is a fail, and every instance of white text is a pass. This page is clearly counterbalanced for darkness adaption, so I believe it's theoretically sound and also extremely compelling! For those interested in the details, I generated 4000 random colour backgrounds sRGB(X,Y,Z) in the range 0-255. Then, for each background, I calculated WCAG2 ratio and APCA for black text on that colour background. Then I also calculated the same for white text on that colour background. Then I looked for specific instances where WCAG2 said the black text was better contrast than the white text, and WCAG2 said the black text was a pass (>4.5:1), and APCA said that in fact the white text was better contrast than the black text, and the black text was a fail (APCA<60), and that the white text is a pass APCA>60. Given that I was generating colours completely randomly, and looking for a quite specific outcome with all five of these conditionals being true, I was expecting to find a few edge case colours where WCAG2 performed particularly poorly. In fact, out of the 4000 random colours, 907 (23%) matched this particular set of conditionals. This is even more shocking if you consider that only 2656 of the random colours passed WCAG2 in the first place. So, if you restrict attention to the colours that WCAG2 passed, this particular epic fail condition occurs for 34% of these colours. To produce the demo page, I filtered the 907 epic fail conditions to only present the colours that were most different to each other (in Euclidean sRGB coordinates). There's some obvious improvement that can be made in the filtering, but the page is pretty compelling 'as is'!! |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
@alastc the recent experiment (and percentages) @sdw32 did was only for color pairs @Myndex — the bar graph @alastc describes, I agree, would be very valuable. I am expecting the APCA line to be a smooth curve, mostly above the 2.x line, but maybe a dip below. |
Beta Was this translation helpful? Give feedback.
-
Thanks Bruce, so it's a sub-set of colour combinations. I think it's important (when describing percentages of correct/incorrect) to indicate what that is as a proportion of the whole and whether it is for 'normal' visual perception or incorporating impairments. |
Beta Was this translation helpful? Give feedback.
-
Thanks to Alistair and Bruce for the follow-ups, and I also notice Andy sent a detailed reply on a new thread here Myndex/SAPC-APCA#55. So I'll just add a few additional comments here. For my earlier post on the random colour generator and percent false fails, I said this
When generating completely random colours for foreground or background, one has to bear in mind that plenty of these will be completely nonsensical, and no human would ever even attempt to try and make a website with those colours. As such, percentage of the whole isn't a particularly interesting statistic. I addressed this by instead reporting 'if a random colour passes WCAG2, what percentage of these ought to have failed'. I maintain that this scope is much more relevant, as it only examines them the subset of things that passed, which means the percentage reflects colour combinations that might actually appear on real-world websites. Nevertheless, it's also important to point out that my first analysis assumes that APCA gets it right and WCAG2.x gets it wrong. I believe Andy has evidence that this is the case within his controlled research experiments, and it looks to be the case on the screen in my office, but population-based evidence that APCA performs better in real world devices/environments is still pending (and Andy is planning a study for this). So, in my later article, I reframed things to instead pose a question regarding which algorithm is better, and shows some examples that might help people come to their own conclusions, and to act as a call to action for further research. This latest article is here http://www.cedc.tools/rback_numbers_vs1.html Regarding the algorithms and colour vision impairments, to the best of my knowledge, both algorithms are models of full colour vision, and neither attempt to model colour impairments. I've heard it said lots of times that WCAG 2.x helps to ensure sufficient contrast for people with colour impairments, but to my knowledge there is no theoretical justification for this statement, because the perceived lightness of different colours gets differently affected when one of the cones is impaired. That said, WCAG 2.x appears to have issues with false passing when one of the colours is black, and black vs red is a particular issue for protans. APCA tends to give a much lower score when one of the colours is black, so one could say it's less likely to pass colour combinations that would be problematic for protans. To do the job properly though, to the best of my knowledge, it ought to be possible to adapt either or both APCA and WCAG 2.x in order to model colour impairments. As a first starting point, I would try using a colour impairment simulation algorithm to transforming the RGB values, then run the contrast algorithm on the transformed RGB values. I'd be glad to discuss this further, but I think that belongs on a different thread. I will look forward to further discussions on this topic. |
Beta Was this translation helpful? Give feedback.
-
Um, we have tested sites which were using #ddd on #fff, personally I'd include that in the 'no human should attempt this' category! Another source of problems is where people have tried to colour-code their navigation/sections, and they keep adding colours as the site grows. That leads to some poor (and you might consider 'random') colour combinations because they stick with either white-text or black-text across all of them. It's less common these days, but I remember testing sites like this. Please don't underestimate how much good has been done by having a contrast measure. There are a lot of colour combinations which would fail both that have been prevented by the current guideline. It definitely needs improvement, but unless you're on the front-line working with designers and developers it's hard to see the effect. It's a tricky area, as I think Bruce mentioned that a lot of the combinations that should fail (but don't in WCAG2) are not particularly attractive and don't tend to get used by designers. I think if we want to focus on a sub-set of colour combinations it would be better to take a sample from what's in use. E.g. There are a couple of projects which have done mass-scale analysis of website code, maybe one of them could provide a list of popular colour combinations? If we can't create a sub-set based on real usage, then I think it would be worth including a 'ridiculous' set that is random across the colour space, perhaps as a control group compared to a set that focuses on the colours which are different between models. Another thought I had when looking at the dark & light example pages was that the background colour has quite a big impact. If I switched the background to black, the dark background combinations seemed much better (to me). I'm not sure a guideline can account for that, but I think it would be worth having that as a variable in any experiments.
Right, I thought that must be the case but wasn't sure.
I'm a little wary of that because most people looking will not have a visual impairment, and from observing usability testing with low-vis people, my expectations didn't always match the outcome. For example, pink/white (even glaring, saturated pink) caused issues for a participant a while back. Orange & white, even blue on white that seemed clear to me weren't for participants. That's anecdotal, but a few examples like that caused me to stop assuming visibility to me meant visibility to others.
If that has come from me, I think it is based a logical outcome from having luminance contrast, compared to not. One of the 'old school' tests is to grey-scale the interface and see whether you can still use it. If it works mono-chromatically, it is more likely (but not certain) to work for everyone. Seeing the examples does make me wonder if we are at a stage where we can start getting help with testing the models? Probably need to start a new thread, but how about: Several sets of colour combinations are gathered, each includes a full set of hues but gather from:
A web form is created that includes on page 1:
Page 2:
Page 3: Thanks for taking part. I'm starting off with a strawman method to see if this (simple) approach is reasonable? If something like this is reasonable, we could write it up and get help with running the testing. |
Beta Was this translation helpful? Give feedback.
-
I've been working on the font lookup tables, and part of the process in terms of font analysis goes along with some of what's being discussed in this thread, and here are some examples for further discussion: On this link are examples of several fonts, lined up to make clear the variations in weight. There are also a number of presentations where the lightness contrast is adjusted with the font size and weight to maintain perceptual uniformity. On this link are evaluations of the most recent font lookup table (0.1.5. G). These are the lookup tables the valuations reference: |
Beta Was this translation helpful? Give feedback.
-
Thanks to Andy @Myndex for providing this extra detail. Personally, I'm currently skeptical of the accuracy of providing average results for 'sans serif' typefaces, based on body height and font-weight. As has already been discussed, each typeface has its own interpretation of what font-weight means, and the ratio of x-height to body height is very different for different typefaces. For each of the typefaces shown on the demo page, I would suggest calculating the ratio of x- height to body height, and calculating the ratio of stroke width to x-height. For the font look up table, I would have thought the values in the table ought to reference x-height (instead of body height), and 'the ratio of stroke width / x-height' (instead of font-weight). Then, another look up table can show, for a variety of typefaces, how x-height relates to body height, and how stroke width / x-height relates to font-weight. This could all be programmed such that a user selects the typeface from a drop-down list, and then gets the values of body height and font-weight, calculated from the generic tables. It would be great to include Verdana on the demonstration page as it's such an outlier, and perfectly demonstrates why body height and font weight are not useful properties for the purposes of guidelines. I will look forward to further discussions on this topic, many thanks. |
Beta Was this translation helpful? Give feedback.
-
I would ask for inclusion of more pedestrian of web fonts. Am I correct to understand that lvtf has consensus that x-height is a key metric? If so, I agree that we should take pains to include it with any of these sorts of tables. (Even if it is not currently a CSS property.) |
Beta Was this translation helpful? Give feedback.
-
Hi @Myndex, thanks for all the follow-up detail. I would just like to add one further comment regarding your text ...
I had never imagined designers would use the font look up table directly. I think the lookup table allows software programmers to make applications that can present this data in a useful way. So, I maintain that the lookup table should be based on x-height, and this table will likely sit as a database back end for a website /software. The front end of the website / software might for example have drop-down list to choose the font-name, and then show the CSS font-size property (i.e. body height) for that particular font-name, which the software/website calculates from its source database of x-heights, and the ratio of x-height to body height, which will be known for the list of font-names. |
Beta Was this translation helpful? Give feedback.
-
Thanks @Myndex for your feedback. Inspired by your comments, I thought of an alternative middle ground, something a bit like the following: Produce the look up table for fonts where the x-height is 50% of the body height, state this explicitly in the intro to the table, and state a list of common fonts where this ratio is true (Arial etc). After the table, give an example for how the table can be adjusted to account for fonts that have a different ratio, so that the x-height would be the same as one of the reference fonts. So, for example with Verdana, the x-height is 55% of the body height, so all the font-size values in the table would need to be multiplied by 50/55 = 0.91. So Arial 22px is equivalent to Verdana 20px. Furthermore, I wonder if font-weight can be handled in the same way. Arial 400 regular has stroke width = 8.8% of body height. Verdana 400 has stroke width = 9.2% of body height, so is roughly equivalent to Arial 400 (or more exactly, 400*9.2/8.8=418, although this doesn't exist obviously, so 400 is still the closest match). Similarly, Arial 700 has stroke width = 13.7% of body height. Verdana 700 has stroke width = 17.6% of body height, so is roughly equivalent to Arial 900 (or more exactly, 700*17.6/13.7=899, so 900 is the closest match). I'm going to take a guess that the stroke width of Arial 900 to Arial 700 isn't exactly in the proportion of 900/700, but I'd also assume the approximation is probably close enough for practical purposes? Any further feedback on this topic would be gratefully received! Many thanks. |
Beta Was this translation helpful? Give feedback.
-
On the APCA Github repo, I posted my portion of a discussion with some of the early adopters of APCA. Here is the key infographic fromthat post:
WCAG 2 and APCA Comparison
Here is the old WCAG 2 1.4.3 contrast guideline against the new WCAG 3 APCA contrast guideline. As the links below also show, this is not an isolated case. Using thousands of random colors, WCAG 2 incorrectly passed 49% colors it tested — meaning passing colors that were too low in contrast to read.
(click image to view it full size/full screen)
Comparison Article: The Lighter Side of Dark Backgrounds
Comparison Article: Orange You Wondering About Contrast?
Regarding consistency concerns between the old and new:
It is not possible to create something that works correctly
and also be consistent to something that is incorrect.
Fortunately, there is enough overlap that a "transition" zone can be had, and some of the tools developed by other developers have a "pass all" setting, to pass both WCAG 2 and WCAG 3 at once, though the trade off is a loss of the flexibility that APCA normally adds. I am actually very surprised at the number of early adopters that have made APCA a part of their contrast tools, literally dozens of tools out there now!
-Andy
Myndex/SAPC-APCA#21
Beta Was this translation helpful? Give feedback.
All reactions