api integration tests

[akosyakov]

What it does

New docs rendered:

• theia cli test command
• API integration testing

fix #6409

• ci integration
• browser integration (rewrite UI tests)
• expose loader source map handling
• electron integration - out of scope, needs Electron 5, tracked in [tests] Improve electron example tests #317 (comment)
• theia cli integration -> theia test to run tests
• separate generated webpack config from default and make expose-loader optional only when someone wants to add tests
• coverage report - https://github.com/istanbuljs/puppeteer-to-istanbul
  • only against bundled files, without remapping to original sources
  • lcov reporting does not work
  • see api tests coverage #6892 as a follow-up
• concurrent page testing -> one can write own script which spawn multiple servers and run different test suites on them
• different VS Code extensions testing -> all theia options are supported, so one can pass --plugins directly to theia test command
• extract reusable tests for theia apps -> published under @theia/api-tests
• doc how to develop, run and debug integration tests, what to test and what not
• CHANGELOG update
• clean up ci scripts from chrome installation and so on

How to test

• Run yarn test in examples/browser to run integration tests.
• Run yarn test:debug in examples/browser to debug integration tests. A new chrome instance will be opened. Refresh to rerun tests and grep query param to filter tests.

Review checklist

• as an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• as a reviewer, I agree to behave in accordance with the review guidelines

[akosyakov]

This PR already replaces old UI wdio test with API based tests which actually find timing issues with our APIs, memory leaks in react scm and git extensions and make shell API more complete.

[svenefftinge]

This is pretty cool! Is it possible to write tests in typescript as well?

[akosyakov]

Is it possible to write tests in typescript as well?

I suppose so, if it compiles to code which can be executed by chrome, keeps requires and does not need any 3rd party deps then it should work. I have not done it because ts-check became very good. The whole webpack repo for example checked with this without any compilation. But i can switch if you want.

[akosyakov]

I'm merging if the build is green, will add more monaco tests in gitlens PR for newly exposed api.

[akosyakov]

@eclipse-theia/ecd-theia-committers Travis does not seem to pick up my latest commits, does anyone know what can be done to trigger the build?

[marcdumais-work]

@eclipse-theia/ecd-theia-committers Travis does not seem to pick up my latest commits, does anyone know what can be done to trigger the build?

Probably something is wrong with Travis ATM. I restarted the job that failed for the latest commit on master, and after a few minutes it's still in Waiting to be queued state:

https://travis-ci.com/eclipse-theia/theia/jobs/278277336

update: it has started now.

Maybe you can amend your top PR commit so the SHA changes and force-push: hopefully that will wake travis up.

[paul-marechal]

The .travis file is broken :)