video-caption
accessibility audit always shows as passing even in error scenarios
#14439
Closed
1 task done
Labels
FAQ
[X] Yes, my issue is not about a specific accessibility audit (file with axe-core instead).URL
https://accessibility-experiments-90d5b5.glitch.me/lighthouse-video-caption-audit/index.html
What happened?
Same issue as #14438 and #14354, this time for
video-caption
. This is the last rule I was suspecting to have this problem!The
video-caption
audit based on Axe’s video-caption rule always shows as passing when it’s applicable, even when Axe itself reports an "incomplete" / "needs review" result.Sample page: https://accessibility-experiments-90d5b5.glitch.me/lighthouse-video-caption-audit/index.html
Lighthouse results for video-caption on this page (JSON export)
Axe CLI results for the same page
Command:
axe -r video-caption https://accessibility-experiments-90d5b5.glitch.me/lighthouse-video-caption-audit/index.html --save video-caption.json && cat video-caption.json
Output:
What did you expect?
Lighthouse’s accessibility audit results should be consistent with those of Axe. In this instance, Axe reports an "incomplete", so it shouldn’t be a pass with Lighthouse. I would have expected either a "Additional items to manually check", or an error.
What have you tried?
Aside from setting up the demo page to reproduce this definitively, I also identified the Axe change that likely caused this – switching the rule from reporting errors to always reporting incomplete: axe-core#285, dequelabs/axe-core@30be884.
How were you running Lighthouse?
Chrome DevTools
Lighthouse Version
9.6.6
Chrome Version
108
Node Version
No response
OS
Mac
Relevant log output
No response
The text was updated successfully, but these errors were encountered: