Hydrogen Testing framework

[cartogram]

Goal: Create an opinionated testing framework for testing hydrogen apps and react server components. Make testing hydrogen apps easy and save our users from the overhead of choosing and setting up a testing framework —which is tedious.

Some initial ideas and points for discussion:

• We'd need to support multiple environments (server, worker, dom) and replicate an Oxygen environment as accurately as possible
• Augment our vite setup and use vitest to share configuration between build, dev and test.
• Include utilities for e2e browser and unit tests —under the hood these would know how to spin up a server, open a headless browser, read text, etc... and suffice this in a commerce-centric way expect(page).toDirectToCheckout()
• Perhaps we build on top of vitest and provide methods that mimic and proxy to the runtime, it, describe, etc..., if we find we need more fine-grained control.
• Provide custom matchers to create a more natural sounding expectations, ie: expect(...).toHavePerformedStorefrontRequest() and tailored to specific commerce interactions toHaveAddedItemsToCart([...lineItems])
• Provide some common utils for mocking and other undifferentiated aspects of unit testing shopify storefronts/hydrogen apps.
• Could have a shared layer with app development.

Alternately we could provide an easy way for apps to extend an existing testing framework.

cc @pepicrft @benjaminsehl pulling our thread here to get more 👀 and comments.

Related issues/Prs

#670
#669
#698
#800

[benjaminsehl]

Leaning into an existing testing framework that best fits our principles (like Vitest, as we've previously mentioned) would be my preferred route. Let's minimize maintenance here to maximize quality.

[davidhousedev]

Provide custom matchers to create a more natural sounding expectations, ie: expect(...).toHavePerformedStorefrontRequest() and tailored to specific commerce interactions toHaveAddedItemsToCart([...lineItems])

The ability to inspect and mock both Storefront requests and requests made via useQuery will be very important for our use case because we will be leveraging Shopify APIs and our own custom APIs. Additionally, we use useQuery to fetch external CMS data, so mocking responses from those queries will be helpful to simulate different UI presentations.

If mocking/spying on cache keys is a feasible approach to support at a framework level, I could see that working very well for us!

I'll list out a few test cases we'd want to be able to cover:

• CMS returns different data via a promise wrapped within a useQuery, multiple tests would assert on how the page renders differently depending on the response from the CMS.
• CMS returns data which modifies downstream requests to Storefront API and internal APIs, tests validate that server components make appropriate Storefront API requests based on responses from the CMS
• Server components modify their behavior based on session contents. Ideally, we could configure a session as a precondition then assert on the different ways that pages render based on that test's session config.