Include and run upstream rntester-e2e tests in our repo

[jonthysell]

Summary

We already have our own version of the upstream rn-tester package in the form of @react-native/tester with our overrides and our own @react-native-windows/tester with additional Windows examples/pages, and finally our own e2e-test-app for running XAML snapshot tests with wdio/WinAppDriver.

However, upstream RN also has an rn-tester-e2e package which contains a suite of jest tests against the rn-tester app using wido targeting ios and android platforms. We should bring and adapt the upstream rn-tester-e2e package into our repo so we can also run those E2E jest tests but targeting windows instead.

Motivation

We already know how to run wido tests against our e2e-test-app but right now it's only for XAML snapshot testing. If there is a suite of wido tests upstream, already written, it seems like we could greatly expand our test coverage for pretty cheap, while also getting the benefits of checking our platform's behavior against the "expectation" from RN.

Basic Example

No response

Open Questions

No response

[chrisglein]

@chiaramooney Can you evaluate this alongside the general Fabric E2E testing plan? #11422

[chiaramooney]

Looked into this. Here's what I've gathered:

Initial Thoughts

• The work so far looks primarily to be about putting the infrastructure in place to do Jest testing with wdio in the react-native repo. Currently, there is only ONE test page and it tests the Button component.
• The infrastructure was build and added to the repo by Lorenzo. I believe the hope is that Meta and the community will add more tests now that the infrastructure is in place. The changes landed 2 days ago, so we'll have to watch the test coverage builds over time.
• This does raise a question that if this is the infrastructure Meta is moving towards, show we base our test app for Fabric off of this setup instead of our current infrastructure. One benefit we get is that we align with Meta and get to take in any testing infrastructure they build/fix. The downside is that there may be testing we have built on Paper that needs to be reworked to align with the new infrastructure.
• My initial impression is that this seems like much less infrastructure than what we have built in RNW. Maybe my first impression is wrong, but why is it that we have so much more code to get to a similar setup of Jest with WinDriverIO? Is our setup more robust in some way?
• Is there value in getting this to work on Paper before we try to get it to work on Fabric. This way we can work through all of the repo/Windows configuration issues separate from the "is this just not working on Fabric yet?" issues.

Implementation Details

• The tests involve clicking on a component and seeing that results/messages are displayed correctly. There are interactive tests in comparison to tree dump tests. We have interactive tests in our own e2e-test-app, but the coverage of those tests is fairly minimal.
• The tests are running of the RNTester app. If we decide to implement this we'll want to redirect the path to test our @react-native-windows/tester app package.
• There are two seperate yarn script commands for iOS and android. We'll need to add our own Windows script command.
• The README is very descriptive and has lots of information on how the tests work and the file structure for the package.

@chrisglein @jonthysell fyi

[jonthysell]

Wow, they were talking about this many months ago, I didn't realize it both literally just landed and has only 2 actual tests. I guess we have two choices:

1. Get this set up in our repo ASAP so if there's any issues where we need breaking changes upstream , we can get those fixed before the test suite is filled with tests that depend on the "broken" setup.
2. Put this on the back burner and wait and see if the test suite even does get filled with tests in the first place.

I think either is acceptable depending on current resourcing and prioritization. If someone can freely take this on right now, it might be worth it to get started, otherwise this can probably wait until someone has nothing else to do.

Thanks @chiaramooney for looking into this.

[chiaramooney]

@jonthysell Totally agree with those options! I think the tradeoff would be that I would probably be the person to tackle getting this set up. So do we want to prioritize working on this first (and getting it to work on paper and fabric) or continuing with the original fabric testing plan?