Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Empty <html> lang attributes report as fail of html-has-lang, pass for html-lang-valid #3624

Closed
1 task done
thibaudcolas opened this issue Aug 28, 2022 · 7 comments
Closed
1 task done

Empty <html> lang attributes report as fail of html-has-lang, pass for html-lang-valid #3624

thibaudcolas opened this issue Aug 28, 2022 · 7 comments
Labels
configuration rules / checks customisation pr A pr has been created for the issue rules Issue or false result from an axe-core rule

Comments

@thibaudcolas
Copy link
Contributor

Product

axe-core

Product Version

4.4.2

Latest Version

  • I have tested the issue with the latest version of the product

Issue Description

For <html lang="">, html-has-lang fails, and html-lang-valid passes. This is a similar issue to #3623, where I would expect the rules to be consistent with one-another.

Expectation

Both rules should be consistent. If lang="" is indeed a valid lang attribute value, then html-has-lang should pass (both rules should pass). If it’s not a valid value, then either both rules should fail, or only html-lang-valid should fail.

Actual

Currently, <html lang=""> passes html-lang-valid.

How to Reproduce

Here are two sites where this happens, and Axe CLI commands:

axe -r html-has-lang,html-lang-valid https://www.koldijkbv.nl/ --save empty-lang.json
axe -r html-has-lang,html-lang-valid https://loyalty.posnania.eu/ --save empty-lang.json

In each case, see the html-has-lang violation while html-lang-valid is marked as a "pass".

Additional context

Similarly to #3623, I was also tempted to report this to Lighthouse instead / in addition to Axe, as its overly simplified audit messages make this even more confusing. It would report:

  • An error "<html> element does not have a [lang] attribute"
  • And a pass "<html> element has a valid value for its [lang] attribute"
@thibaudcolas thibaudcolas added the ungroomed Ticket needs a maintainer to prioritize and label label Aug 28, 2022
@thibaudcolas thibaudcolas changed the title Empty html lang attributes report as fail of html-has-lang, pass for html-lang-valid Empty <html> lang attributes report as fail of html-has-lang, pass for html-lang-valid Aug 28, 2022
@WilcoFiers
Copy link
Contributor

We try to avoid having axe-core report the same problem more than once. There's not much more to it than that. It's important to know that even if a rule passes, it's always possible for there to be some other issue.

We could update the rule selector of html-lang-valid to skip empty lang attributes. That way the rule would be inapplicable, rather than passed. Would that have helped with your concern?

@WilcoFiers WilcoFiers added question and removed ungroomed Ticket needs a maintainer to prioritize and label labels Aug 29, 2022
@thibaudcolas
Copy link
Contributor Author

Hi @WilcoFiers, thank you for looking into this!

Researching this further, I learned that the HTML page has lang attribute ACT rule specifically expects a failure for <html lang="">, so I’m hesitant to suggest any changes to Axe’s html-has-lang’s . Looking at HTML page lang attribute has valid language tag, its applicability says:

[…] has a lang attribute that is neither empty ("") nor only ASCII whitespace […]

… so your suggestion to have the rule marked as inapplicable in this case seems spot on.

Unfortunately although this scenario is documented in the ACT rule description, there isn’t a test case for it there, so implementations seem to differ (Axe passes, Alfa seems to report as inapplicable).

@WilcoFiers
Copy link
Contributor

@thibaudcolas ACT implementations are not required to consistently report on passed / inapplicable. Those two outcomes are effectively interchangeable as far as ACT is concerned. There are multiple reasons for that. Many tools don't even report inapplicable, for example. Those should still be able to implement ACT rules. It also comes from that WCAG doesn't have the concept of inapplicable. A success criterion is either satisfied, or it isn't.

What ultimately matters is that tools/testers only fail things that should fail.

I'll leave this ticket open for us to update this selector. It should be a fairly straightforward tweak to the rules elector.

@WilcoFiers WilcoFiers added rules Issue or false result from an axe-core rule configuration rules / checks customisation and removed question labels Aug 30, 2022
@thibaudcolas
Copy link
Contributor Author

Cool!

I'll leave this ticket open for us to update this selector. It should be a fairly straightforward tweak to the rules elector.

Would this be something along the lines of html[lang]:not([lang=""]), html[xml\\:lang]:not([xml\\:lang=""]) in place of html-lang-valid’s existing html[lang], html[xml\\:lang]? If so I’ll happily give this a shot if it helps.

@WilcoFiers
Copy link
Contributor

@thibaudcolas Pretty much, yes. We'll need some integration tests to go with it in test/integration/full/html-lang-valid/. A PR would be much appreciated!

thibaudcolas added a commit to thibaudcolas/axe-core that referenced this issue Sep 17, 2022
@thibaudcolas
Change html-lang-valid selector so the rule only applies when attrs h…
bce5b20 
…ave a value. Fix dequelabs#3624
thibaudcolas added a commit to thibaudcolas/axe-core that referenced this issue Sep 17, 2022
@thibaudcolas
Change html-lang-valid selector so the rule only applies when attrs h…
ccfe834 
…ave a value. Fix dequelabs#3624
thibaudcolas added a commit to thibaudcolas/axe-core that referenced this issue Sep 17, 2022
@thibaudcolas
Change html-lang-valid selector so the rule only applies when attrs h…
8b9dbf8 
…ave a value. Fix dequelabs#3624
thibaudcolas added a commit to thibaudcolas/axe-core that referenced this issue Sep 20, 2022
@thibaudcolas
Change html-lang-valid selector so the rule only applies when attrs h…
24c0701 
…ave a value. Fix dequelabs#3624
@thibaudcolas thibaudcolas mentioned this issue Sep 20, 2022
Merged
@thibaudcolas
Copy link
Contributor Author

I gave this a go at #3663 but unfortunately got stuck with an issue with the CSS selector processing.

thibaudcolas added a commit to thibaudcolas/axe-core that referenced this issue Sep 24, 2022
@thibaudcolas
Change html-lang-valid selector so the rule only applies when attrs h…
8b4829b 
…ave a value. Fix dequelabs#3624
thibaudcolas added a commit to thibaudcolas/axe-core that referenced this issue Oct 6, 2022
@thibaudcolas
Change html-lang-valid selector so the rule only applies when attrs h…
eb04e60 
…ave a value. Fix dequelabs#3624
thibaudcolas added a commit to thibaudcolas/axe-core that referenced this issue Oct 6, 2022
@thibaudcolas
Change html-lang-valid selector so the rule only applies when attrs h…
95b2224 
…ave a value. Fix dequelabs#3624
@straker straker added the pr A pr has been created for the issue label Nov 22, 2022
@padmavemulapati
Copy link

Validated with the latest code base,
for the test snippet <html lang=""> , html-has-lang failing and as well valid-lang is in inapplicable

Image

Image

This was referenced Dec 5, 2022
Closed
Closed
@attest-team-ci attest-team-ci mentioned this issue Dec 12, 2022
Merged
@Shubhabrata08 Shubhabrata08 mentioned this issue Jan 13, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
configuration rules / checks customisation pr A pr has been created for the issue rules Issue or false result from an axe-core rule
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@WilcoFiers @thibaudcolas @straker @padmavemulapati