Project: client-side page data requests (/_next/data/ URLs) broken

[eatyourgreens]

Package
app-project

Describe the bug
Client-side workflow routing is broken in mobile Safari.

EDIT: the underlying cause of this bug is that Next's data request URLs are broken, because they aren't being proxied through to a NextJS app. eg. https://www.zooniverse.org/_next/data/cy3jPk61vsMFAGo-Zi51-/projects/msalmon/hms-nhs-the-nautical-health-service/classify/workflow/16926/subject-set/82738.json

To Reproduce

1. Go to https://frontend.preview.zooniverse.org/projects/cseidenstuecker/every-name-counts
2. Click one of the workflow links.
3. Get the generic error message: 'An unexpected error has occurred'.

Loading a workflow page directly will work eg.
https://frontend.preview.zooniverse.org/projects/darkeshard/engaging-the-crowds-2020/classify/workflow/16099

Expected behavior
The workflow page should load with that workflow shown in the classifier.

• OS: iOS 13.7
• Browser: Safari

[eatyourgreens]

I think the problem might be related to the page data requests going to a 404 page in PFE.
https://frontend.preview.zooniverse.org/_next/data/_d8G2fIA7qwq8iBc9Fpnt/projects/cseidenstuecker/every-name-counts/classify/workflow/15904.json

[sentry-io]

Sentry issue: FEM-APP-PROJECT-CZ

[eatyourgreens]

The problem goes away if you browse the NextJS app directly on your phone:
https://fe-project.preview.zooniverse.org/projects/cseidenstuecker/every-name-counts

The problem seems to be with proxying requests to /_next/data/. Those don't use the asset prefix so will be requested from the window origin: https://frontend.preview.zooniverse.org in this case.

[camallen]

The problem goes away if you browse the NextJS app directly on your phone:
https://fe-project.preview.zooniverse.org/projects/cseidenstuecker/every-name-counts

The problem seems to be with proxying requests to /_next/data/. Those don't use the asset prefix so will be requested from the window origin: https://frontend.preview.zooniverse.org in this case.

OOI do we know why this isn't an issue on other browsers?

[eatyourgreens]

In other browsers, the page is refreshed from the server after the client-side fetch fails. See https://github.com/zooniverse/operations/issues/497#issuecomment-699907536

[eatyourgreens]

Fixed on staging by adding a routing rule to forward https://frontend.preview.zooniverse.org/_next/data/ to https://fe-project.preview.zooniverse.org/_next/data/.

Note that this prevents us from using client-side routing for the About pages, since they would also request data from https://frontend.preview.zooniverse.org/_next/data/. NextJS does not allow prefixes for data requests, only for static files in /_next/static.

[eatyourgreens]

Let's see how this works on staging before deciding how we'd like to fix it in production.

[eatyourgreens]

If Next 9.5 allows us to run multiple Next apps on the same domain, then #1886 may fix this by allowing us to run app-project on www.zooniverse.org/projects/ and app-content-pages on www.zooniverse.org/about/. At the moment, both run on www.zooniverse.org/, meaning we can only set up a proxy rule (www.zooniverse.org/_next/data) for one of them.

[eatyourgreens]

I'm looking at this in production now, for the HMS NHS beta. Requests are being made for the workflow and subject set JSON, as expected eg.
https://www.zooniverse.org/_next/data/cy3jPk61vsMFAGo-Zi51-/projects/msalmon/hms-nhs-the-nautical-health-service/classify/workflow/16926/subject-set/82738.json

Our nginx setup redirects those to static.zooniverse.org, which responds with a 403 error eg.
https://static.zooniverse.org/www.zooniverse.org/_next/data/cy3jPk61vsMFAGo-Zi51-/projects/msalmon/hms-nhs-the-nautical-health-service/classify/workflow/16926/subject-set/82738.json

[eatyourgreens]

This has been fixed in production by proxying data requests to fe-project.zooniverse.org/_next/data/.

It can be reopened if we find we need to proxy requests for both NextJS apps.