Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support the official vscode test API and Test Explorer (v1.59) #742

Merged
merged 10 commits into from
Aug 21, 2021

Conversation

connectdotz
Copy link
Collaborator

@connectdotz connectdotz commented Aug 4, 2021

The official test api and explorer are finally ready and should be released in v1.59. This PR aimed to support the test explorer via the new test API so we can offer the long-requested feature:

test-explorer

The explorer support is driven by the existing engine (mainly from TestResultProvider), therefore, completely backward compatible and all config and functionality behave exactly as before. The code change is mainly in the new test-provider folder, plus some minor code changes in the existing code to support more interactive test configurations from the explorer and notify the test-provider when jest test results changed.

rc.1:
vscode-jest-4.1.0-rc.1.vsix.zip

updated rc.2:
vscode-jest-4.1.0-rc.2.vsix.zip

update rc3:
vscode-jest-4.1.0-rc.3.vsix.zip

@coveralls
Copy link

coveralls commented Aug 4, 2021

Pull Request Test Coverage Report for Build 1141539487

  • 544 of 558 (97.49%) changed or added relevant lines in 14 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+0.2%) to 94.245%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/test-provider/test-provider.ts 86 88 97.73%
src/JestExt/process-listeners.ts 20 23 86.96%
src/JestExt/core.ts 57 61 93.44%
src/test-provider/test-item-data.ts 258 263 98.1%
Files with Coverage Reduction New Missed Lines %
src/TestResults/match-by-context.ts 1 94.44%
Totals Coverage Status
Change from base Build 980: 0.2%
Covered Lines: 2459
Relevant Lines: 2557

💛 - Coveralls

@connectdotz
Copy link
Collaborator Author

FYI, CI will fail until @types/vscode 1.59 releases as there is a breaking change for StatusBar.tooltip type. In order to work with the proposed API, vscode recommends removing @types/vscode and download the typing file directly (see typings folder). But I did not modify CI script to perform this since this is just a transient situation.

@connectdotz connectdotz changed the title support the official vscode test API and Test Explorer (v1.59 insider) support the official vscode test API and Test Explorer (v1.59) Aug 6, 2021
@connectdotz
Copy link
Collaborator Author

ok, 1.59 has been released today, the extension should work in the official vscode now. @orta @stephtr do you guys have some cycle to take a look and play with it? Please let me know if you have any comments, thanks.

@distante
Copy link

distante commented Aug 6, 2021

Works pretty nice on a newly updated vscode! I hope this gets released to the official channel soon. 👍

@distante
Copy link

distante commented Aug 6, 2021

@connectdotz I have notice something

When I save an spec file and it auto runs I see the ? below the red/green test status in the file gutter.

It also does not change the status in the test explorer until the test are done.
image

@connectdotz
Copy link
Collaborator Author

@distante your observation is accurate. The current sync point (between the jest runner and the explorer) is indeed when the test result is available, i.e. run is over. The editor and statusBar, on the other hand, could update status when the test "starts". Thus the discrepancy you saw...

I wasn't sure if the difference is noticeable, well, now I know 😅

Let me see if we can close the gap...

@stephtr
Copy link
Member

stephtr commented Aug 7, 2021

ok, 1.59 has been released today, the extension should work in the official vscode now. @orta @stephtr do you guys have some cycle to take a look and play with it? Please let me know if you have any comments, thanks.

Unfortunately, I don't have any resources available, but I installed the RC and gave it a quick try. Looks nice so far!

I also noticed something similar to distante: When the tests are finished, the green circled checkmarks appear. If one then removes a line within the file (without saving) and clicks onto the green checkmark, differently looking checkmarks appear: One at the test's old location and another one at the test's new location. If one saves the changes, everything is fine again.

image

1. implement runEvent system to synchronize existing and new provider
2. added new testExplorer setting to fine tune UI experience
3. deprecate our own inline error and in favor of explorer's
@connectdotz
Copy link
Collaborator Author

connectdotz commented Aug 16, 2021

hmmm... looks like we ran out of the free credit in travis-ci.com... ok I guess it's time to move on to another CI system... @orta do you have any opinion? Looks like jest itself uses circle-ci, but that was before github actions... thoughts?

[updated]
I take it back, looks like jest is using github actions...

@connectdotz
Copy link
Collaborator Author

meanwhile, I made some changes to address the comments:

  1. refactored the mechanism of how the existing system communicates with the new test-provider (supporting vscode TestExplorer). Now using an event mechanism for both run and session processes, so components like test-provider can register and process the same events JestExt received, without modifying JestExt. This would also enable us to further decouple components like statusBar, debug-provider, coverage in the future...
  2. change the jest run command to always report colors so we can show nice native jest output in the test explorer's terminal. (we could remove the existing output channel... anybody has a strong opinion about this?)
  3. replace our own inline-error messaging functionality with explorer's
  4. added a new setting jest.testExplorer to fine-tune some experience:
    • enable/disable testExplorer support
    • hiding/showing the original status decoration when test explorer is enabled.
    • showing/hiding test explorer's inline error messaging
  5. added an information-message popup on the start-up to remind people about test-explorer support since by default the test explorer will be enabled and they will not see the familiar "?" status decorations
  6. adding the new vscode Testing category in package.json

vscode-jest-4.1.0-rc.2.vsix.zip

@connectdotz connectdotz mentioned this pull request Aug 16, 2021
@stephtr
Copy link
Member

stephtr commented Aug 16, 2021

During my brief test it worked quite nice!
I only noticed one issue (I don't think it happened with the 4.0 release):

  • Have a (passing) test file, committed to Git.
  • Add a test failure (like expect(2).toBe(1);)
  • Save the file => test runs, test shows up as failure
  • Undo the change & save => test doesn't rerun, appears as being broken, despite being correct

@connectdotz
Copy link
Collaborator Author

@stephtr thanks for testing, are you on watch mode?

@stephtr
Copy link
Member

stephtr commented Aug 17, 2021

@stephtr thanks for testing, are you on watch mode?

Yes, exactly.

@connectdotz
Copy link
Collaborator Author

I suspect it could be a jest watch issue, I think I have seen similar behavior before, the reverted file has timestamp equal or less than the committed file thus not triggering the test run... if you turn on the test explorer's output, you would probably see message like 'No tests found related to files changed since last commit.'

If you get a chance, can you test executing explicit test from test explorer UI in a non-watch mode project in windows. These individual test run is new (in the past the we only supported file level test run), wanted to make sure it works for window users...

@stephtr
Copy link
Member

stephtr commented Aug 17, 2021

You're completely right, it's really due to no changes since last commit.

I just tested manual test execution in the test explorer, for me it is working fine!
As usual, great work!

@connectdotz
Copy link
Collaborator Author

connectdotz commented Aug 18, 2021

update for rc3:

address part of #749, particularly about status not update until the run was completed. Even though the status did get updated (on the file level) during the run, @agos had a point that it is easy to miss it, especially when there are lots of tests within. rc3 will do a deep status update for "scheduled" and "started" tests so it should be a lot obvious now...

FYI, also migrated our CI to github actions.

vscode-jest-4.1.0-rc.3.vsix.zip

@connectdotz connectdotz merged commit 12dca20 into jest-community:master Aug 21, 2021
@connectdotz connectdotz deleted the test-provider-1.59 branch August 21, 2021 15:26
legend1202 pushed a commit to legend1202/vscode-jest that referenced this pull request Jun 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants