Skip to content

Commit

Permalink
Dynamic APIs (#60645)
Browse files Browse the repository at this point in the history
formalizes the concept of dynamic APIs inside Next to allow for varying
semantics beyond just staticGenerationBailout.

### Dynamic APIs

#### `markCurrentScopeAsDynamic`
useful to bail out of default caching semantics but does not imply a
Request specific data source was read. critically, this semantic is
ignored if you are inside a cache scope

#### `trackDynamicDataAccessed`
Must be called before reading any data source that is derived from
Request specific data. Currently this is `cookies()`, `headers()`, and
`searchParams`. This kind of data access inside a cache scope is
forbidden (it always should have been, but now it will error).

#### `trackDynamicFetch`
This one is unideal but the complexity of patch-fetch's current
implementation necessitates it for now. Essentially it will postpone if
we are prerendering. Long term this should be eliminated with a refactor
of patch fetch.


### Other Improvements
Also removes the `staticGenerationBailout` implementation as it has been
replaced with more specific logic in the places it was previously being
used.

One area that has also been enhanced is the proxy for app-route modules.
Previously we proxied the Request every time however when we are doing
non-static generation executions we generally don't want the overhead of
wrapping the request. In the refactor here I also improved the runtime
performance by using static proxy handlers and I believe I also fixed a
few bugs related to `clone` and `url`

In general there has been a bit of refactoring to clarify how we should
handle various render/execution states and a reduction in implicit side
effects for proper execution.

Another callout to notice is that app-route modules do not attempt a
static generation if they are force-dynamic regardless of the PPR
setting. Previously the PPR setting would opt them into this code path
which is not necessary because PPR itself does not work for routes, only
pages.

Closes NEXT-2099
  • Loading branch information
gnoff committed Jan 24, 2024
1 parent 78c9793 commit b5772b8
Show file tree
Hide file tree
Showing 52 changed files with 1,831 additions and 666 deletions.
1 change: 0 additions & 1 deletion docs/02-app/02-api-reference/04-functions/next-request.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,6 @@ The following options are available:
| -------------- | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `basePath` | `string` | The [base path](/docs/app/api-reference/next-config-js/basePath) of the URL. |
| `buildId` | `string` \| `undefined` | The build identifier of the Next.js application. Can be [customized](/docs/app/api-reference/next-config-js/generateBuildId). |
| `url` | `URL` | The URL object. |
| `pathname` | `string` | The pathname of the URL. |
| `searchParams` | `Object` | The search parameters of the URL. |

Expand Down
6 changes: 4 additions & 2 deletions packages/next/cache.d.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
export { unstable_cache } from 'next/dist/server/web/spec-extension/unstable-cache'
export { revalidatePath } from 'next/dist/server/web/spec-extension/revalidate-path'
export { revalidateTag } from 'next/dist/server/web/spec-extension/revalidate-tag'
export {
revalidateTag,
revalidatePath,
} from 'next/dist/server/web/spec-extension/revalidate'
export { unstable_noStore } from 'next/dist/server/web/spec-extension/unstable-no-store'
4 changes: 2 additions & 2 deletions packages/next/cache.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
const cacheExports = {
unstable_cache: require('next/dist/server/web/spec-extension/unstable-cache')
.unstable_cache,
revalidateTag: require('next/dist/server/web/spec-extension/revalidate-tag')
revalidateTag: require('next/dist/server/web/spec-extension/revalidate')
.revalidateTag,
revalidatePath: require('next/dist/server/web/spec-extension/revalidate-path')
revalidatePath: require('next/dist/server/web/spec-extension/revalidate')
.revalidatePath,
unstable_noStore:
require('next/dist/server/web/spec-extension/unstable-no-store')
Expand Down
11 changes: 2 additions & 9 deletions packages/next/src/build/templates/app-route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,8 @@ const routeModule = new AppRouteRouteModule({
// Pull out the exports that we need to expose from the module. This should
// be eliminated when we've moved the other routes to the new format. These
// are used to hook into the route.
const {
requestAsyncStorage,
staticGenerationAsyncStorage,
serverHooks,
headerHooks,
staticGenerationBailout,
} = routeModule
const { requestAsyncStorage, staticGenerationAsyncStorage, serverHooks } =
routeModule

const originalPathname = 'VAR_ORIGINAL_PATHNAME'

Expand All @@ -51,8 +46,6 @@ export {
requestAsyncStorage,
staticGenerationAsyncStorage,
serverHooks,
headerHooks,
staticGenerationBailout,
originalPathname,
patchFetch,
}
10 changes: 8 additions & 2 deletions packages/next/src/build/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1580,7 +1580,13 @@ export async function isPageStatic({
const routeModule: RouteModule =
componentsResult.ComponentMod?.routeModule

let supportsPPR = false

if (pageType === 'app') {
if (ppr && routeModule.definition.kind === RouteKind.APP_PAGE) {
supportsPPR = true
}

const ComponentMod: AppPageModule = componentsResult.ComponentMod

isClientComponent = isClientReference(componentsResult.ComponentMod)
Expand Down Expand Up @@ -1650,7 +1656,7 @@ export async function isPageStatic({
// If force dynamic was set and we don't have PPR enabled, then set the
// revalidate to 0.
// TODO: (PPR) remove this once PPR is enabled by default
if (appConfig.dynamic === 'force-dynamic' && !ppr) {
if (appConfig.dynamic === 'force-dynamic' && !supportsPPR) {
appConfig.revalidate = 0
}

Expand Down Expand Up @@ -1748,7 +1754,7 @@ export async function isPageStatic({
// When PPR is enabled, any route may be completely static, so
// mark this route as static.
let isPPR = false
if (ppr && routeModule.definition.kind === RouteKind.APP_PAGE) {
if (supportsPPR) {
isPPR = true
isStatic = true
}
Expand Down
21 changes: 21 additions & 0 deletions packages/next/src/client/components/client-page.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
'use client'
import { createDynamicallyTrackedSearchParams } from './search-params'

export function ClientPageRoot({
Component,
props,
}: {
Component: React.ComponentType<any>
props: { [props: string]: any }
}) {
// We expect to be passed searchParams but even if we aren't we can construct one from
// an empty object. We only do this if we are in a static generation as a performance
// optimization. Ideally we'd unconditionally construct the tracked params but since
// this creates a proxy which is slow and this would happen even for client navigations
// that are done entirely dynamically and we know there the dynamic tracking is a noop
// in this dynamic case we can safely elide it.
props.searchParams = createDynamicallyTrackedSearchParams(
props.searchParams || {}
)
return <Component {...props} />
}
17 changes: 12 additions & 5 deletions packages/next/src/client/components/draft-mode.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import type { DraftModeProvider } from '../../server/async-storage/draft-mode-provider'

import { staticGenerationBailout } from './static-generation-bailout'
import { staticGenerationAsyncStorage } from './static-generation-async-storage.external'
import { trackDynamicDataAccessed } from '../../server/app-render/dynamic-rendering'

export class DraftMode {
/**
Expand All @@ -15,14 +16,20 @@ export class DraftMode {
return this._provider.isEnabled
}
public enable() {
if (staticGenerationBailout('draftMode().enable()')) {
return
const store = staticGenerationAsyncStorage.getStore()
if (store) {
// We we have a store we want to track dynamic data access to ensure we
// don't statically generate routes that manipulate draft mode.
trackDynamicDataAccessed(store, 'draftMode().enable()')
}
return this._provider.enable()
}
public disable() {
if (staticGenerationBailout('draftMode().disable()')) {
return
const store = staticGenerationAsyncStorage.getStore()
if (store) {
// We we have a store we want to track dynamic data access to ensure we
// don't statically generate routes that manipulate draft mode.
trackDynamicDataAccessed(store, 'draftMode().disable()')
}
return this._provider.disable()
}
Expand Down
62 changes: 30 additions & 32 deletions packages/next/src/client/components/headers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,45 +4,46 @@ import {
} from '../../server/web/spec-extension/adapters/request-cookies'
import { HeadersAdapter } from '../../server/web/spec-extension/adapters/headers'
import { RequestCookies } from '../../server/web/spec-extension/cookies'
import { requestAsyncStorage } from './request-async-storage.external'
import { actionAsyncStorage } from './action-async-storage.external'
import { staticGenerationBailout } from './static-generation-bailout'
import { DraftMode } from './draft-mode'
import { trackDynamicDataAccessed } from '../../server/app-render/dynamic-rendering'
import { staticGenerationAsyncStorage } from './static-generation-async-storage.external'
import { getExpectedRequestStore } from './request-async-storage.external'

export function headers() {
if (
staticGenerationBailout('headers', {
link: 'https://nextjs.org/docs/app/building-your-application/rendering/static-and-dynamic#dynamic-rendering',
})
) {
return HeadersAdapter.seal(new Headers({}))
}
const requestStore = requestAsyncStorage.getStore()
if (!requestStore) {
throw new Error(
`Invariant: headers() expects to have requestAsyncStorage, none available.`
)
const callingExpression = 'headers'
const staticGenerationStore = staticGenerationAsyncStorage.getStore()

if (staticGenerationStore) {
if (staticGenerationStore.forceStatic) {
// When we are forcing static we don't mark this as a Dynamic read and we return an empty headers object
return HeadersAdapter.seal(new Headers({}))
} else {
// We will return a real headers object below so we mark this call as reading from a dynamic data source
trackDynamicDataAccessed(staticGenerationStore, callingExpression)
}
}

const requestStore = getExpectedRequestStore(callingExpression)
return requestStore.headers
}

export function cookies() {
if (
staticGenerationBailout('cookies', {
link: 'https://nextjs.org/docs/app/building-your-application/rendering/static-and-dynamic#dynamic-rendering',
})
) {
return RequestCookiesAdapter.seal(new RequestCookies(new Headers({})))
}
const callingExpression = 'cookies'
const staticGenerationStore = staticGenerationAsyncStorage.getStore()

const requestStore = requestAsyncStorage.getStore()
if (!requestStore) {
throw new Error(
`Invariant: cookies() expects to have requestAsyncStorage, none available.`
)
if (staticGenerationStore) {
if (staticGenerationStore.forceStatic) {
// When we are forcing static we don't mark this as a Dynamic read and we return an empty cookies object
return RequestCookiesAdapter.seal(new RequestCookies(new Headers({})))
} else {
// We will return a real headers object below so we mark this call as reading from a dynamic data source
trackDynamicDataAccessed(staticGenerationStore, callingExpression)
}
}

const requestStore = getExpectedRequestStore(callingExpression)

const asyncActionStore = actionAsyncStorage.getStore()
if (
asyncActionStore &&
Expand All @@ -57,11 +58,8 @@ export function cookies() {
}

export function draftMode() {
const requestStore = requestAsyncStorage.getStore()
if (!requestStore) {
throw new Error(
`Invariant: draftMode() expects to have requestAsyncStorage, none available.`
)
}
const callingExpression = 'draftMode'
const requestStore = getExpectedRequestStore(callingExpression)

return new DraftMode(requestStore.draftMode)
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,13 @@ export type RequestAsyncStorage = AsyncLocalStorage<RequestStore>

export const requestAsyncStorage: RequestAsyncStorage =
createAsyncLocalStorage()

export function getExpectedRequestStore(callingExpression: string) {
const store = requestAsyncStorage.getStore()
if (!store) {
throw new Error(
`Invariant: \`${callingExpression}\` expects to have requestAsyncStorage, none available.`
)
}
return store
}
73 changes: 73 additions & 0 deletions packages/next/src/client/components/search-params.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import type { ParsedUrlQuery } from 'querystring'

import { staticGenerationAsyncStorage } from './static-generation-async-storage.external'
import { trackDynamicDataAccessed } from '../../server/app-render/dynamic-rendering'
import { ReflectAdapter } from '../../server/web/spec-extension/adapters/reflect'

/**
* Takes a ParsedUrlQuery object and either returns it unmodified or returns an empty object
*
* Even though we do not track read access on the returned searchParams we need to
* return an empty object if we are doing a 'force-static' render. This is to ensure
* we don't encode the searchParams into the flight data.
*/
export function createUntrackedSearchParams(
searchParams: ParsedUrlQuery
): ParsedUrlQuery {
const store = staticGenerationAsyncStorage.getStore()
if (store && store.forceStatic) {
return {}
} else {
return searchParams
}
}

/**
* Takes a ParsedUrlQuery object and returns a Proxy that tracks read access to the object
*
* If running in the browser will always return the provided searchParams object.
* When running during SSR will return empty during a 'force-static' render and
* otherwise it returns a searchParams object which tracks reads to trigger dynamic rendering
* behavior if appropriate
*/
export function createDynamicallyTrackedSearchParams(
searchParams: ParsedUrlQuery
): ParsedUrlQuery {
const store = staticGenerationAsyncStorage.getStore()
if (!store) {
// we assume we are in a route handler or page render. just return the searchParams
return searchParams
} else if (store.forceStatic) {
// If we forced static we omit searchParams entirely. This is true both during SSR
// and browser render because we need there to be parity between these environments
return {}
} else if (!store.isStaticGeneration && !store.dynamicShouldError) {
// during dynamic renders we don't actually have to track anything so we just return
// the searchParams directly. However if dynamic data access should error then we
// still want to track access. This covers the case in Dev where all renders are dynamic
// but we still want to error if you use a dynamic data source because it will fail the build
// or revalidate if you do.
return searchParams
} else {
// We need to track dynamic access with a Proxy. We implement get, has, and ownKeys because
// these can all be used to exfiltrate information about searchParams.
return new Proxy({} as ParsedUrlQuery, {
get(target, prop, receiver) {
if (typeof prop === 'string') {
trackDynamicDataAccessed(store, `searchParams.${prop}`)
}
return ReflectAdapter.get(target, prop, receiver)
},
has(target, prop) {
if (typeof prop === 'string') {
trackDynamicDataAccessed(store, `searchParams.${prop}`)
}
return Reflect.has(target, prop)
},
ownKeys(target) {
trackDynamicDataAccessed(store, 'searchParams')
return Reflect.ownKeys(target)
},
})
}
}
15 changes: 0 additions & 15 deletions packages/next/src/client/components/searchparams-bailout-proxy.ts

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ import type { Revalidate } from '../../server/lib/revalidate'

import { createAsyncLocalStorage } from './async-local-storage'

type PrerenderState = {
hasDynamic: boolean
}

export interface StaticGenerationStore {
readonly isStaticGeneration: boolean
readonly pagePath?: string
Expand All @@ -16,12 +20,8 @@ export interface StaticGenerationStore {
readonly isRevalidate?: boolean
readonly isUnstableCacheCallback?: boolean

/**
* If defined, this function when called will throw an error postponing
* rendering during the React render phase. This should not be invoked outside
* of the React render phase as it'll throw an error.
*/
readonly postpone: ((reason: string) => never) | undefined
// When this exists (is not null) it means we are in a Prerender
prerenderState: null | PrerenderState

forceDynamic?: boolean
fetchCache?:
Expand All @@ -36,7 +36,6 @@ export interface StaticGenerationStore {
forceStatic?: boolean
dynamicShouldError?: boolean
pendingRevalidates?: Record<string, Promise<any>>
postponeWasTriggered?: boolean

dynamicUsageDescription?: string
dynamicUsageStack?: string
Expand All @@ -59,3 +58,13 @@ export type StaticGenerationAsyncStorage =

export const staticGenerationAsyncStorage: StaticGenerationAsyncStorage =
createAsyncLocalStorage()

export function getExpectedStaticGenerationStore(callingExpression: string) {
const store = staticGenerationAsyncStorage.getStore()
if (!store) {
throw new Error(
`Invariant: \`${callingExpression}\` expects to have staticGenerationAsyncStorage, none available.`
)
}
return store
}
Loading

0 comments on commit b5772b8

Please sign in to comment.