[WebDriver] Testability of web components

[AutomatedTester]

When web authors are creating their web applications they are going to need to be able to test and interact with elements that may be in a shadow DOM.

There have been requests and a proposal for how to interact with the Shadow DOM as well to this issue tracker.

The main concerns from WebDriver are about finding elements and then interacting with them.

Finding Elements.

People using testing frameworks need a meaningful way to find elements on the page. Currently if a Selenium WebDriver user wants to find an element within the DOM they would do

driver.find_element(By.CSS, "<insert selector>")

Which is the equivalent of doing

document.querySelector("<insert selector>");

This obviously only searches the document and never goes into the shadow DOM. If we wanted to search in the shadow DOM via JavaScript we would just switch out document and can mimic that in WebDriver client bindings. This will only work though for open shadow DOMs.

I know there have been proposals that suggest testing frameworks could do their own thing to circumvent the closed shadow DOM. Unfortunately this will lead to a situation where we are hoping that browsers do the right thing leading to interop issues if it is not defined properly. There are also issues about injecting JavaScript onto the page as this could lead to race conditions and also doesn't preserve the page as a user would see it if people wanted to test it "as is".

It would be good if we can have a mechanism to look into shadow DOMs

Interaction with elements

WebDriver allows users to interact, where possible, as the user would. If an element is (obscured)[https://w3c.github.io/webdriver/index.html#dfn-obscuring] we notify the user the element they are trying to click will have the click intercepted by another element. We do this by getting the paint order from document.elementsFromPoint(...)spec link. Currently there is w3c/csswg-drafts#556 that discusses this but, for encapsulation reasons, it doesn't want to return elements in the shadow DOM. We can work around this by recursively going down until we can't go any further and then error. If the Shadow DOM work give a mechanism that can be interoperable that make life easier.

[annevk]

I'm not sure I understand, but we cannot have web-exposed APIs for any of this. These would have to be privileged APIs that only WebDriver can use.

[rniwa]

For finding an element, we can add a special flag / mode to WebDriver where all shadow trees are treated as open in that script's context for example to let any script access any author-defined shadow trees. Alternatively, we can add a WebDriver command which finds the shadow tree of a given element.

For interacting with elements, it seems that we need some kind of WebDriver command to find an element across shadow boundaries.

[rniwa]

Like @annevk said, we definitely can't have a Web exposed API for these things. That would defeat the whole point of encapsulation.

[jgraham]

I think there is some confusion here :)

It's clear that testing APIs should be able to pierce the shadow trees in ways that content APIs are forbidden to. This is already how e.g. devtools behaves, so it's clearly also something that's possible to implement in browsers.

The WebDriver issue (w3c/webdriver#1320) contains a couple of plausible suggestions for how to handle this, basically variants of @rniwa's suggeseted "given an element get me the shadow root". One (from @gsnedders) is literally that; the other one is "Find shadow elements" where you use a (standard) selector to get a set of elements but return the shadow root attached to the elements (if any) rather than the elements themselves.

One other suggestion that came up there was to ressurrect the shadow piercing CSS selectors but only for a test context. My suspicion is that's a bigger implementation change and we should hold off until we have a better feeling for the pain points in testing this stuff.

The other question was around interactability testing; WebDriver uses elementsFromPoint to figure out if events like click should reach an element. Naively this doesn't work for shadow DOM since elementsFromPoint called on the document will always result in the top-level shadow host being returned. However it looks like there's some intent to add elementsFromPoint to the DocumentOrShadowRoot interface, which I think solves this problem (MDN lists it as a member, but afaict the CSSOM Spec lists it on Document only). Assuming this is on DocumentOrShadowRoot, I think you just need to perform the interactability check on all the ShadowRoot or Document nodes up to the root.

So my suspicion is that modulo the CSSOM spec not putting the necessary interfaces on ShadowRoot, we have enough information/features to put together a complete proposal and ask for review.

[caridy]

I'm in favor of "given an element get me the shadow root" as a WebDriver specific API to reach into any shadow, whether it is open or closed (this should not work on native elements though). Maybe WebDriver can implement this feature directly, just replacing the Element.attachShadow before executing any user-land code. This is probably enough to allow a controlled traversing.

I'm not ok with "Find shadow elements". I have seen what that can cause in big apps, and it goes directly against the web component encapsulation, which should also apply to tests and automation IMO.

[jgraham]

I don't think "Find shadow elements" as proposed goes against encapsulation? The proposal is specifically not shadow-piercing CSS; it's approximately:

fn findShadow(root, selector) {
  return root.querySelectorAll(selector).filter(x => x.shadowRoot).map(x => x.shadowRoot)
}

but with the ability to see the shadowRoot irrespective of the open vs closed status.

[rniwa]

And only expose that to WebDriver? It's definitely not okay to have such an API in the production web environment.

[jgraham]

Yes, of course. There's no intent here to change web-exposed APIs, especially not in a way that breaks encapsulation.

[rniwa]

Okay, another API people have suggested in the last was a way to get a list of all shadow roots in a given document.

I think it's best to enumerate a list of concrete scenarios where this kine of API is useful / needed, and evaluate which one of dozen options we've come up would work best.

[LarsDenBakker]

Not being able to do webdriver-like tests is a major blocker for adoption of shadow dom. Especially for enterprise.

[annevk]

@LarsDenBakker it's clear something is needed, what's desirable is a clearer description of what is needed. What kind of primitives are you looking for that would let you accomplish your goals?

[hayatoito]

FYI. A related issue in Chromium is https://bugs.chromium.org/p/chromium/issues/detail?id=829713, which was already closed.

Quoted from https://bugs.chromium.org/p/chromium/issues/detail?id=829713#c13

Filed
whatwg/dom#665
w3c/csswg-drafts#2908

For webdriver, I found an old existing thread and pinged there
w3c/webdriver#350

[emilio]

FWIW the only primitive we needed to expose for devtools is the ability to poke at closed shadow roots. But that's a no-go if it's web-exposed unfortunately.

[LarsDenBakker]

@annevk I think that the core of the issue is that shadow dom tries to be a 'one size fits all' solution.

Usually people speak of web components in the sense of UI component libraries: <my-button>, <my-selector>, <my-dropdown> etc. and in those cases shadow dom with all it's encapsulation works great. Even when writing webdriver tests you don't want to click on the <button> element that's inside of <my-button>. The <my-button> component itself is interactable, so clicking that should trigger the right action.

I think the main problem here is when you build larger apps using web component and shadow dom. A component that represents a page can contain multiple interactable elements, and it makes sense to use style encapsulation there. But there is no requirement for the hardcode encapsulation (dom nodes, querySelector, ids, aria attributes, activeElement, events, focus etc.) that you also get when you add shadow dom. For non-shared components, it can actually be very useful to be able to poke around the internals of these components, that's how the web has always worked.

My point here is that often when you want something to be exposed in a test, you might also wanted to have it exposed in some way in production code.

Being able to have more control over the level of encapsulation would go a long way in simplifying the testing of components. Besides that, I think a few extra tools would help:

• ::part from http://tabatkins.github.io/specs/css-shadow-parts/ can help with enabling components to expose individual elements which can be interacted with
• something like the removed ::shadow selector can help by querying the shadow dom of an individual element

In the end though, we've been testing large web component apps for a couple of years now and we simply have a deepQuerySelector script which we inject in our webdriver tests. It lets us write regular CSS selector and it recursively walks through shadow roots until it finds the first matching element. It's hacky and bad for performance, but very effective. Custom elements are a great tool here, as we can use the element names in the selectors. So I'm wondering if just adding /deep/ back for testing will not just solve the problem.

[diervo]

Let me share our experience at Salesforce since we just transition a couple of thousand Selenium tests to work with shadow DOM, and our pages are quite dynamic and complex (thousands of multi-author/multi-version web components running within a page) and I believe we check very well the "enterprise"box 😄.

TL;DR: Very simple utilities were sufficient to have an "enterprise grade test codebase" migrated to use ShadowDOM and Web Components.

As @rniwa said above, any API that breaks encapsulation is a non-starter since it will defeat the whole purpose of web components and shadow DOM, but also, would be a foot-gun that would be fundamentally miss-used and ultimately would put us in the same position of "reach-all-the-things" approach we are in today.

Precisely at our scale, we have suffered deeply from developers using complex global queries that rely on all sort of specific tree-structures and hierarchies. Terrible for scale and maintainability.

Using Shadow DOM is really helping us and our developers to "do the right thing" since they are now forced to follow a very explicit structure and a set of repeatable query patterns.

All we have done to enable this, is to provide a couple of helper methods to our WebDriver classes that facilitate navigation and traversal:

/**
* Returns a test.util.webdriver.WebElement class that represents the elements shadowRoot property if it
* exists. Otherwise, returns the element parameter passed into this function.
* 
* Only call this function if you need to support scenarios where a shadowRoot property on an element may or may
* not be present. This is helpful for utility methods that run in different environments.
* 
* An important note here is if the shadowRoot property exists on an element then a ShadowRootWebElement class will
* be returned. This class attempts to represent what a shadowRoot would be on the client. Thus, not all standard
* WebElement APIs will be available. Clicking, for example, does not make sense to do on the shadowRoot on the
* client and thus will throw an error. See test.util.webdriver.ShadowRootWebElement for more details.
* 
* @param element the element to expand the shadowRoot property of
* @return a ShadowRootWebElement class representing the shadowRoot if it exists, otherwise the WebElement passed in
*/
WebElement expandShadowIfPresent(WebElement element);

/**
* Returns a test.util.webdriver.ShadowRootWebElement class that represents the elements shadowRoot property.
* 
* @param element the element to expand the shadowRoot property of
* @return a ShadowRootWebElement class representing the shadowRoot
*/
ShadowRootWebElement expandShadow(WebElement element);

We have also a series of what we call "pageObjects" that are fundamentally an abstraction of what a page looks like and how can I interact with different pieces of it, which again fits perfectly with the WC and Shadow DOM model.

As @caridy points out, it will be great to have these utilities as part of the Selenium protocol, but all of those should be just sugar on top of the current available APIs. Adding these in form of atoms in all drivers is pretty straight forward (we have already done some POCs).

We want to cover more ground and find more use-cases before doing a formal proposal, but we will be happy to work wit anybody interested in designing this APIs. I believe just a couple of helpers like the ones I described previously should be more than enough to cover any complex use case in a scalable and modular way.

For the record, we have even fixed the IE11 driver to "fully" support ShadowDOM so we can actually build an enterprise level system working with WebComponents and ShadowDOM.

[gsnedders]

@diervo Thanks for that! I think that's exactly the sort of feedback we were wanting.

[jgraham]

@diervo So are all your shadow DOMs open, or is the point that you explictly don't want to test closed shadow DOMs when they're embedded? Otherwise I don't see how injecting content js (i.e. WebDriver atoms) can work here. If you explicity don't want to open up shadow DOMs, how do you handle the case where you have a <my-video> and want to click the play button in a test (for example)?

In any case it seems like the primitive that you're looking for is basically a "give me the shadow root for this element" one, rather than something more complex. That's promising, since it's also the simplest possible thing. As @gsnedders says, thanks for the helpful feedback :)

[caridy]

@diervo So are all your shadow DOMs open.

@jgraham, they could be open or closed, depending on some other boundaries (e.g.: is the child and parent from the same developer, or not). But that is irrelevant because we could bypass that by patching the attachShadow() mechanism to always store a reference of the shadowRoot somewhere, e.g..: elm.__nakedShadow__, so our expandShadow mechanism can use that instead of just elm.shadowRoot which is dictated by the mode configuration.

[AutomatedTester]

@diervo So are all your shadow DOMs open.

@jgraham, they could be open or closed, depending on some other boundaries (e.g.: is the child and parent from the same developer, or not). But that is irrelevant because we could bypass that by patching the attachShadow() mechanism to always store a reference of the shadowRoot somewhere, e.g..: elm.__nakedShadow__, so our expandShadow mechanism can use that instead of just elm.shadowRoot which is dictated by the mode configuration.

We need to have something where we are not trying to run JS before something else could be. This approach, which is fine for now, will lead to a point where the atoms lose the race. We need a solution that never hits that situation.

As for expandShadow, this is kind of along the same lines as my original PR for a "switch to shadow root`. We can change it to "getShadowRoot" or whatever really and then carry on from that point. It can easily be integrated into most peoples abstractions for their web application.

[LarsDenBakker]

@diervo what does a selector look like for you? Do you need to explicitly specify each element that has a shadow root along the way?

[caridy]

We need to have something where we are not trying to run JS before something else could be. This approach, which is fine for now, will lead to a point where the atoms lose the race. We need a solution that never hits that situation.

@AutomatedTester yes, that's why we are supportive of such API, so we don't have to play the race game.

@diervo what does a selector look like for you? Do you need to explicitly specify each element that has a shadow root along the way?

Yes, you have to get inside the shadow at every level, unless you rely on our high level API to access page segments as @diervo described above. We don't want arbitrary queries to return stuff from inside a shadow. We did entertained the idea of using something equivalent to ::shadow selector (IIRC), but that might confuse folks that they might be able to use the same selector in production code, which will not work.

[LarsDenBakker]

That will make the tests very brittle, right? As you need to spell most of the DOM hierarchy. When you release frequently this is very subject to change.

I really think the ::part spec can help out a lot here to help expose nodes expressively.

[rniwa]

@LarsDenBakker: Note querySelector("a::part(b)") WOULD NOT return an element inside the shadow tree of a exposed as a part b. querySelector and querySelectorAll, etc... always match against nodes in context object's tree.

[caridy]

That will make the tests very brittle, right?

@LarsDenBakker yes, and that's why we have an APP specific API (the high level API that @diervo explained above) for what we call pageObjects, which is basically saying: "find me the record being shown on the main area of the page, I want to test that area", and then you drill down from there instead of having to drill down from the very top, but again, that's APP's specific logic. Such API can be implemented based on "switch to shadow root" that we are discussing here.

[rniwa]

@diervo

All we have done to enable this, is to provide a couple of helper methods to our WebDriver classes that facilitate navigation and traversal

What I'd like to know is why you needed those methods. Here are some examples of concrete use cases:

• I want to write an integration test, and make sure when I click on a "Done" button inside a calendar widget's shadow tree, it would reflect the change back to the timeline widget's shadow tree.
• I want to write an unit test for a chart component, and I want to verify that when I update the data set associated with it via a method on the chart, the canvas element inside its shadow tree gets redrawn.

We need a bunch of these very concrete, down-to-earth examples so that what API would suit the needs of each use case best. Without them, we'd be talking in abstract which makes all the discussions more opinion-based rather than evidence-based, and it would make a much harder to reach a consensus.

[ghost]

Just in case this discussion is still alive.

We're developing mid-sized (~250k sloc of client code) web app built from ground up using web components. All of the shadow roots are open. Depth of shadow roots is about 12-15 levels at max. All third-party components have opened shadow roots too.

For unit tests, and inter-component integration tests there is no need for any additional API since all internals are known and easily accessible. It's true even for third-party components like vaadin-, paper-. For example to check that clicking on date input opens custom calendar popup, we can access shadowRoot directly in tests code, since they are being executed in browser using karma. I think it's safe to assume that most of unit-testing today is done in this or similar way.

E2E on the other hand is tricky. My guess is that we invented the same wheel as @diervo did. To select elements in shadow DOM we collect selectors that include explicit information about shadow bounds (just like the >>> ) and then for every query in WebDriver we inject JS script that traverses DOM and returns found element.

There is big chance I'm missing something since our E2E test base is extremely small due to considerable friction caused by the fact that explicit selectors are very fragile and require a lot of maintenance.
But testing looks possible w/o additional API from browsers. Perhaps wider adoption of WC will be possible when E2E-testing ecosystem gets mature in this regard. Solutions like Selenium IDE, Katalon, Cypress.io seem to work on shadow DOM support. I didn't tested it myself, but given this I assume TestCafe already has some basic support.

IMHO the benefits of having native way of selecting elements behind shadow DOMs are following:

• eliminating race-conditions caused by the need to inject helper content scripts
• more "honest" E2E testing that does not require modification of page under test, keeping it as close to production as possible
• not sure on this one, but I assume there will be performance improvement compared to the method I described above (injecting JS into page and traversing DOM).

I hope this was useful

[dwt]

I'd like to add, that we have super useful xpath libraries like capybara, that allow to express things like 'find the input that has a label that contains this text'. (Of course the actual XPATH expressions are way more complex). This allows to define web tests in such a more succinct way, it's not even funny.

However the current situation of WebComponents and especially the non piercing aspect of selectors (and especially xpath) makes the approach of developing reusable xpath expressions for ease of selection a complete non starter. :-(

As I haven't seen that discussed in the above thread (sorry if I missed it) I am really hoping to add that to the proposed solution.