refactor(code_block): remove all snapshot tests and redundant describe blocks #8185

weronikaolejniczak · 2024-11-25T16:38:08Z

Summary

As discussed on #8176, the EuiCodeBlock component's testing suite could use improvements, mainly:

removing the redundant wrapping describe block over only one test case (comment),
updating the toMatchSnapshot (aka snapshot testing) to relevant assertions using Jest matchers (comment).

Some additional comments

I added some stories (Annotations, Highlighted Lines and Start Value) to showcase the component's capabilities with linesNumber prop. You cannot easily control it with the playground controls. Additionally, we can test the annotation behavior and have a VRT test.

Also, I should mention I do like a single it('renders') first test emitting a snapshot

I removed all snapshots (except for initial render). A snapshot can still get overwritten without much notice, targeted assertions are better.

QA

Unit Tests

Checkout the branch (I recommend using GitHub Pull Requests if you're working in VSCode, makes it so much faster to checkout a branch PR).
Run yarn workspace @elastic/eui test-unit or yarn workspace @elastic/eui test-unit code_block (to target only the EuiCodeBlock component).

Visual Regression Tests

Checkout the branch.
Run yarn workspace @elastic/eui storybook to open the Storybook.
Run yarn workspace @elastic/eui visual-regression or yarn workspace @elastic/eui --storiesFilter EuiCodeBlock (to target only the EuiCodeBlock component).

closes elastic#8004

- remove the wrapping describe, - update the assertion to toHaveAttribute

…e blocks

cee-chen · 2024-11-25T17:15:42Z

packages/eui/changelogs/upcoming/8176.md

We should remove this changelog!

edit: oh, it looks like all the copy aria label commits came along for the ride in this PR - we probably want to rebase them out

I did this on purpose, based the branch off the other one so that once it's merged, I can easily rebase onto master without conflicts. BUT somehow there is a conflict anyway... 👀

cee-chen · 2024-11-25T17:16:11Z

packages/eui/src/components/code/code_block.styles.ts

- * 2.0 and the Server Side Public License, v 1; you may not use this file except
- * in compliance with, at your election, the Elastic License 2.0 or the Server
- * Side Public License, v 1.
- */


🤦 Thank you so much for catching this!

cee-chen · 2024-11-25T17:17:34Z

packages/eui/src/components/code/code_block.test.tsx

@@ -22,107 +23,59 @@ describe('EuiCodeBlock', () => {
      <EuiCodeBlock {...requiredProps}>{code}</EuiCodeBlock>
    );

-    expect(container.firstChild).toMatchSnapshot();
+    expect(container).toBeInTheDocument();


I'd like just one it('renders') HTML snapshot at the top of the test file, so we can quickly examine output classes and DOM and see any diffs if added. Remaining tests should generally be specific assertions however

quickly examine output classes and DOM and see any diffs if added

Is this commonly used within EUI? Do you often check for DOM differences? In what situations do you find it necessary? I can add it back, no problem 👌🏻, but I want to understand the context better.

From my perspective, snapshot tests are generally not very useful because you have to manually assert whether the markup differences are expected. While assertions ensure that the component renders and doesn't crash, which is the most basic regression test. I don't have much experience with how useful snapshot tests are at the complexity level of EUI though.

Nevermind, I added the snapshot test back while keeping the toBeInTheDocument 😄

cee-chen · 2024-11-25T17:18:34Z

packages/eui/.loki/reference/chrome_mobile_Editors_Syntax_EuiCodeBlock_Highlighted_Lines.png

Thank you so much for updating these VRT screenshots!! 🎉

cee-chen · 2024-11-25T17:37:22Z

packages/eui/src/components/code/code_block.stories.tsx

+export const NoPaddingSize: Story = {
+  tags: ['vrt-only'],
+  args: {
+    children: htmlCode,
+    language: 'html',
+    paddingSize: 'none',
+  },
+};
+
+export const SmallPaddingSize: Story = {
+  tags: ['vrt-only'],
+  args: {
+    children: htmlCode,
+    language: 'html',
+    paddingSize: 's',
+  },
+};
+
+export const MediumPaddingSize: Story = {
+  tags: ['vrt-only'],
+  args: {
+    children: htmlCode,
+    language: 'html',
+    paddingSize: 'm',
+  },
+};
+
+export const LargePaddingSize: Story = {


I think my only hesitation with manually writing all these tests one-by-one is 1. it's not DRY, and 2. it's incredibly easy to miss a size. Ideally I'd prefer it if Loki could go through our controls and take a screenshot of each option, but I don't know if that's available yet.

My 2c: until we can address this holistically across all components I'd prefer not to add VRT for each prop and its options, but I'll defer to @mgadewoll on that

I agree 💯. After writing these stories, I had the exact same thought: "Is it scalable, and is it maintainable?". My answer was "no and no". I decided to push this anyway to get your insights because maybe you'd have an idea on how we could improve it.

if Loki could go through our controls and take a screenshot of each option

There's no automated solution for testing all controls implicitly. The way Loki works, you have to create a story per each prop combination you want it to take a screenshot of. Besides, I don't think it's sensible for all of the controls to be visually tested for regression. Some do not have visual results or are not as crucial.

My 2c: until we can address this holistically across all components I'd prefer not to add VRT for each prop and its options

I can agree here. We can investigate this on a separate task and make a decision together. If not to somehow automate it, at least to establish the protocol of what type of prop combinations do we want to test. It's not without a cost after all.

We still have to test this interface somehow though. Snapshot tests - no, as we discussed before, unit tests - what, asserting a class? It's implementation details and very prone to breaking. VRT really seems to be the best suited for the job.

I'd agree to not add the granular stories for VRT just now and instead we should first look into these things:

a) is there a way to automate this better within Loki's limitations

b) we eventually want/need to migrate off Loki considering it's non-maintained by now and we can't use it in CI, so adding additional VRT stories now might be unnecessary work if the new solution can handle this - hopefully - more elegantly

So if the question is: if we remove the snapshots here now, what do we do? Then maybe we don't remove the snapshots for those just yet? 🤔

Edit: Or we accept class/attribute assertions in unit tests for now until we can migrate to more fitting VRT.

@cee-chen @mgadewoll how about I write those tests again but using assertions for classes? I'll remove the stories and we think about Loki separately?

Yep, of course. 👍 We don't need to solve the Loki issue in this PR. It's a full grown investigation by itself.

cee-chen · 2024-11-25T17:40:51Z

packages/eui/src/components/code/code_block.test.tsx

    });

+    // TODO: Update the assertion


Was this TODO addressed? I'd personally query for:

const { container } = render(...) const highlightedLine = container.querySelector('[class*="euiCodeBlock__lineText-isHighlighted"]'); expect(highlightedLine).toBeInTheDocument();

😬 thanks for catching this! I totally forgot. I don't exactly like that we have to query it like this but I guess there's no other way.

cee-chen · 2024-11-25T17:41:49Z

packages/eui/src/components/code/code_block.test.tsx

-      expect(container.firstChild).toMatchSnapshot();
+
+      expect(
+        container.querySelectorAll('.euiCodeBlock__lineNumberWrapper > span')[0]


Suggested change

container.querySelectorAll('.euiCodeBlock__lineNumberWrapper > span')[0]

container.querySelector('.euiCodeBlock__lineNumber')

Oh, if we're querying multiple lines, I'd store const lineNumbers = container.querySelectorAll('.euiCodeBlock__lineNumber') as a var after the render.

Very good suggestion! 👍🏻

cee-chen · 2024-11-25T17:42:05Z

packages/eui/src/components/code/code_block.test.tsx

+        container.querySelectorAll('.euiCodeBlock__lineNumberWrapper > span')[0]
+      ).toHaveAttribute('data-line-number', '10');
+      expect(
+        container.querySelectorAll('.euiCodeBlock__lineNumberWrapper > span')[1]


Suggested change

container.querySelectorAll('.euiCodeBlock__lineNumberWrapper > span')[1]

container.querySelectorAll('.euiCodeBlock__lineNumber')[1]

Oh yes, this makes sense but when querying by .euiCodeBlock__lineNumber it couldn't find it. That's why I went with the . euiCodeBlock__lineNumberWrapper. I'll try again though, maybe I had a typo or something.

cee-chen · 2024-11-25T17:43:43Z

packages/eui/src/components/code/code_block.test.tsx

-      expect(container.firstChild).toMatchSnapshot();
+
+      expect(
+        container.querySelectorAll('.euiCodeBlock__lineNumberWrapper').length


Let's target the specific number class instead of the wrapper

Suggested change

container.querySelectorAll('.euiCodeBlock__lineNumberWrapper').length

container.querySelectorAll('.euiCodeBlock__lineNumber').length

As above: #8185 (comment)

I'll try it again.

cee-chen · 2024-11-25T17:44:27Z

packages/eui/src/components/code/code_block.test.tsx

    });
  });

-  describe('line numbers', () => {
+  describe('Line numbers', () => {


[not a change request, just my opinion] - personally I don't care about sentence capitalization in test names. no strong preference either way though, so it's whatever y'all want to do

I don't have a strong preference either, I just believe it's a bit more readable from the console as opposed to everything being lowercase.

cee-chen · 2024-11-25T17:47:06Z

packages/eui/src/components/code/code_block.test.tsx

@@ -174,40 +130,38 @@ describe('EuiCodeBlock', () => {
          {code}
        </EuiCodeBlock>
      );
-      expect(container.firstChild).toMatchSnapshot();
+
+      expect(container).toBeInTheDocument();


Depending on the size of the snapshot, I wouldn't hate this being left as a snapshot - it's not bad to get an idea of what kind of DOM our virtualization library renders and what inline styles it's adding.

If it's a crapton of lines that would cause people's eyes to glaze over however, I'm good with leaving it as an assertion:

Suggested change

expect(container).toBeInTheDocument();

expect(container.querySelector('[class*="euiCodeBlock__code-isVirtualized"]')).toBeInTheDocument();

Depending on the size of the snapshot, I wouldn't hate this being left as a snapshot - it's not bad to get an idea of what kind of DOM our virtualization library renders and what inline styles it's adding.

That's what I thought initially but after revisiting our discussion about snapshot tests, reading different opinions and reflecting on it myself, I came to a conclusion that testing the component structure is too prone to misinterpretation and human error while deciding if the diff is expected or not. It's easy to skip by just updating the snapshot. The core aspects we want to test (like rendering, important classes, or roles) can be tested with proper assertions.

Still, the suggestion you made ☝🏻 I don't like that we have to query by a class like that but there's no other way I see in a unit test and I do like that the class points more to whether the virtualization is applied or not. BUT I believe the better way to test it is with Cypress, so making sure only the visible items are rendered with EuiCodeBlock that has a isVirtualized prop set to true.

Let me know what you think and we decide together how it's better to test virtualization.

Actually, I just added back the snapshot 😄 If this is useful nevertheless then it doesn't hurt to leave it while we remove all the other prop snapshots which are more tedious than useful 👍🏻

Motivation: to avoid scalability and maintenance issues

- add back snapshot tests for initial render and virtualization - simplify some assertions - add unit tests for the: transparentBackground, overflowHeight, paddingSize and fontSize

for paddingSize and fontSize

kibanamachine · 2024-11-29T13:51:15Z

Preview staging links for this PR:

Docs site: https://eui.elastic.co/pr_8185/
Storybook: https://eui.elastic.co/pr_8185/storybook
New docs: https://eui.elastic.co/pr_8185/new-docs

elasticmachine · 2024-11-29T13:51:48Z

💚 Build Succeeded

Buildkite Build
Commit: 969214e

History

💚 Build #3155 succeeded 3294750
💔 Build #3148 failed 32ac68d
💔 Build #3146 failed 6b5bc29
💚 Build #3136 succeeded 014385a

weronikaolejniczak added 7 commits November 22, 2024 19:16

feat(code_block): allow for a custom copy button aria-label

9cb1e61

closes elastic#8004

refactor(code_block): rename copyAriaLabel and remove docs

0ccfd50

chore: update the changelog

dbeb3be

refactor(code_block): improve the copyAriaLabel test case

acc44d3

- remove the wrapping describe, - update the assertion to toHaveAttribute

refactor(code_block): remove all snapshot tests and redundant describ…

950d663

…e blocks

chore(code_block): add VRT tests for the EuiCodeBlock

d6ae46c

chore(code_block): remove duplicated copyright comment

014385a

weronikaolejniczak added the skip-changelog label Nov 25, 2024

weronikaolejniczak requested a review from a team as a code owner November 25, 2024 16:38

weronikaolejniczak changed the title ~~Refactor/8180 refactor code block tests~~ @weronikaolejniczak refactor(code_block): remove all snapshot tests and redundant describe blocks Nov 25, 2024

weronikaolejniczak changed the title ~~@weronikaolejniczak refactor(code_block): remove all snapshot tests and redundant describe blocks~~ refactor(code_block): remove all snapshot tests and redundant describe blocks Nov 25, 2024

weronikaolejniczak mentioned this pull request Nov 25, 2024

feat(code_block): allow for a custom copy button aria-label #8176

Merged

12 tasks

cee-chen reviewed Nov 25, 2024

View reviewed changes

weronikaolejniczak added 2 commits November 27, 2024 13:08

chore(code_block): remove VRT tests

9299032

Motivation: to avoid scalability and maintenance issues

chore(code_block): improve the unit test suite

6b5bc29

- add back snapshot tests for initial render and virtualization - simplify some assertions - add unit tests for the: transparentBackground, overflowHeight, paddingSize and fontSize

weronikaolejniczak linked an issue Nov 27, 2024 that may be closed by this pull request

[EuiCodeBlock] Improve the Jest testing suite #8180

Open

weronikaolejniczak and others added 3 commits November 27, 2024 17:54

Merge branch 'main' into refactor/8180-refactor-code-block-tests

32ac68d

chore(code_block): use toHaveStyleRule and test.each

3294750

for paddingSize and fontSize

chore: remove old changelog file

969214e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(code_block): remove all snapshot tests and redundant describe blocks #8185

refactor(code_block): remove all snapshot tests and redundant describe blocks #8185

weronikaolejniczak commented Nov 25, 2024 •

edited

Loading

cee-chen Nov 25, 2024 •

edited

Loading

weronikaolejniczak Nov 26, 2024

cee-chen Nov 25, 2024

cee-chen Nov 25, 2024

weronikaolejniczak Nov 26, 2024

weronikaolejniczak Nov 28, 2024

cee-chen Nov 25, 2024

cee-chen Nov 25, 2024

weronikaolejniczak Nov 26, 2024

mgadewoll Nov 26, 2024 •

edited

Loading

weronikaolejniczak Nov 26, 2024

mgadewoll Nov 26, 2024

cee-chen Nov 25, 2024 •

edited

Loading

weronikaolejniczak Nov 26, 2024

cee-chen Nov 25, 2024

cee-chen Nov 25, 2024

weronikaolejniczak Nov 26, 2024

cee-chen Nov 25, 2024

weronikaolejniczak Nov 26, 2024

cee-chen Nov 25, 2024

weronikaolejniczak Nov 26, 2024

cee-chen Nov 25, 2024

weronikaolejniczak Nov 26, 2024

cee-chen Nov 25, 2024

weronikaolejniczak Nov 26, 2024

weronikaolejniczak Nov 28, 2024

kibanamachine commented Nov 29, 2024

elasticmachine commented Nov 29, 2024

	container.querySelectorAll('.euiCodeBlock__lineNumberWrapper > span')[0]
	container.querySelector('.euiCodeBlock__lineNumber')

	expect(container).toBeInTheDocument();
	expect(container.querySelector('[class*="euiCodeBlock__code-isVirtualized"]')).toBeInTheDocument();

refactor(code_block): remove all snapshot tests and redundant describe blocks #8185

Are you sure you want to change the base?

refactor(code_block): remove all snapshot tests and redundant describe blocks #8185

Conversation

weronikaolejniczak commented Nov 25, 2024 • edited Loading

Summary

Some additional comments

QA

Unit Tests

Visual Regression Tests

cee-chen Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mgadewoll Nov 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cee-chen Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibanamachine commented Nov 29, 2024

elasticmachine commented Nov 29, 2024

💚 Build Succeeded

History

weronikaolejniczak commented Nov 25, 2024 •

edited

Loading

cee-chen Nov 25, 2024 •

edited

Loading

mgadewoll Nov 26, 2024 •

edited

Loading

cee-chen Nov 25, 2024 •

edited

Loading