[GSoC] Multi-Project Test Suite

[PeterSenpai]

Summary

This is a draft PR for my GSoC application (More details can be added later.)

Project description

Automated testing for projects and deployments that we expect EUI to support. Currently, we rely on manual build steps or testing suites (functional and/or unit tests) provided by consuming applications to check environment compatibility and catch instances of breaking API changes. Conversion to and adoption of TypeScript has helped at a low level, but we need broader, less manual means of understanding support coverage. (related issues: issue#3058, issue#2760)

Target:

For projects that consume EUI.

• Some of the targeted projects:
  1. Gatsby EUI starter
  2. Create react app
  3. CodeSandbox
  4. NextJs

Outcome:

• Developers will have a command-line interface to execute the tests for different projects.
• The tool can test installation, build process and detect the failure.
• The tests are configurable and extensible.
• It should also be integrated with CI tools.

Example:

>>> integration-test gatsby-starter --install
>>> integration-test --config ./integration-test/config/config.json

Overall plan:

1. Design and create a prototype test script for one of the projects such as EUI Gatsby starter.
2. Implement the essential scripts that interact with multiple projects with different operations.
3. Make the tests configurable and extensible. Improve the CLI.
4. Containerize and optimize the suite and integrate it with CI tools.

(More details are shared in Google Doc.)

Checklist

• Check against all themes for compatibility in both light and dark modes
• Checked in mobile
• Checked in IE11 and Firefox
• Props have proper autodocs
• Added documentation examples
• Added or updated jest tests
• Checked for breaking changes and labeled appropriately
• Checked for accessibility including keyboard-only and screenreader modes
• A changelog entry exists and is marked appropriately

[kibanamachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[thompsongl]

Hey, @PeterSenpai, thanks for your interest!

We're not yet to the point of having open PRs against GSoC work. You can keep this work locally for now, but it's important to go through the full application process: https://github.com/elastic/gsoc#application-instructions

[PeterSenpai]

Hi @thompsongl, really sorry about this. 😳

The creation of this PR is following one of the steps in the instruction Link to a pull request you have submitted to the project you want to work on.

And apparently I misunderstand this instruction? I also have another PR(merged) against one of the good first issue.

Another question I have is that according to the instruction I have to add you and Chandler as commenters of my google docs. Right now I have added Chandler and Phillpp, is that good enough?

[thompsongl]

No problem at all!

Link to a pull request you have submitted to the project you want to work on.

Yes, your other merged PR is sufficient for that step.

Please add me as a commenter on your docs, also. Thanks!

[PeterSenpai]

Hi @thompsongl, you mind share your mail address? Or you can probably ask Philipp b/c when I shared with Philipp, he said he can share with all the mentors, so I didn't really ask yours. (I found Chandler's thru GitHub and Philipp's thru the picture in the instruction page) 😳

[tushar22-tg]

I too had send my proposal but didn't got any review from mentors , I think they are busy or might be avoiding applicants,

[thompsongl]

@PeterSenpai I have access now 👍

We'll be reviewing applications in the next couple of days.