feat: expand #shadow-root elements automatically in parseWithCheerio helper
#2396
Conversation
github-actions
bot
added
t-tooling
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request is neither linked to an issue or epic nor labeled as adhoc!
packages/playwright-crawler/src/internals/utils/playwright-utils.ts
Outdated
Show resolved
Hide resolved
|
I guess we can merge this one, or do you guys think we should make this configurable? I don't think it will add a perf cost, but it could produce some unwanted junk.. |
Configuration will make the feature hard to discover, but the potential for over-selecting is also large - I'd even call it breaking. We could enable it by default and allow disabling it... |
|
Yes definitely, I meant configurable opt-out, not making it opt-in. I guess I will add it to be safer. |
…heerio` helper Custom HTML elements have their content isolated under a `#shadow-root` property, this PR handles its expansion by traversing the DOM and looking for all custom components with a shadow root, inlining its contents. This way, we can traverse the inside of a custom component via cheerio, as well as get the text content later on.
You need to traverse the whole DOM to find them, so it feels fine to me, you would have to do it one way or the other.
Agreed, let's be safe, and let's make it hidden initially so we don't need to convince anyone that its a good idea to have it there :] |
B4nan
changed the title
parseWithCheerio helperparseWithCheerio helper
Custom HTML elements have their content isolated under a
#shadow-rootproperty, this PR handles its expansion by traversing the DOM and looking for all custom components with a shadow root, inlining its contents. This way, we can traverse the inside of a custom component via cheerio, as well as get the text content later on.The behavior is enabled by default and can be disabled via
ignoreShadowRoots: truein the crawler options.