Skip to content

Commit

Permalink
Implement new client-side router (#37551)
Browse files Browse the repository at this point in the history 
## Client-side router for `app` directory

This PR implements the new router that leverages React 18 concurrent features like Suspense and startTransition.
It also integrates with React Server Components and builds on top of it to allow server-centric routing that only renders the part of the page that has to change.

It's one of the pieces of the implementation of https://nextjs.org/blog/layouts-rfc.

## Details

I'm going to document the differences with the current router here (will be reworked for the upgrade guide)

### Client-side cache

In the current router we have an in-memory cache for getStaticProps data so that if you prefetch and then navigate to a route that has been prefetched it'll be near-instant. For getServerSideProps the behavior is different, any navigation to a page with getServerSideProps fetches the data again.

In the new model the cache is a fundamental piece, it's more granular than at the page level and is set up to ensure consistency across concurrent renders. It can also be invalidated at any level.

#### Push/Replace (also applies to next/link)

The new router still has a `router.push` / `router.replace` method.

There are a few differences in how it works though:

- It only takes `href` as an argument, historically you had to provide `href` (the page path) and `as` (the actual url path) to do dynamic routing. In later versions of Next.js this is no longer required and in the majority of cases `as` was no longer needed. In the new router there's no way to reason about `href` vs `as` because there is no notion of "pages" in the browser.
- Both methods now use `startTransition`, you can wrap these in your own `startTransition` to get `isPending`
- The push/replace support concurrent rendering. When a render is bailed by clicking a different link to navigate to a completely different page that still works and doesn't cause race conditions.
- Support for optimistic loading states when navigating

##### Hard/Soft push/replace

Because of the client-side cache being reworked this now allows us to cover two cases: hard push and soft push.

The main difference between the two is if the cache is reused while navigating. The default for `next/link` is a `hard` push which means that the part of the cache affected by the navigation will be invalidated, e.g. if you already navigated to `/dashboard` and you `router.push('/dashboard')` again it'll get the latest version. This is similar to the existing `getServerSideProps` handling.

In case of a soft push (API to be defined but for testing added `router.softPush('/')`) it'll reuse the existing cache and not invalidate parts that are already filled in. In practice this means it's more like the `getStaticProps` client-side navigation because it does not fetch on navigation except if a part of the page is missing.

#### Back/Forward navigation

Back and Forward navigation ([popstate](https://developer.mozilla.org/en-US/docs/Web/API/Window/popstate_event)) are always handled as a soft navigation, meaning that the cache is reused, this ensures back/forward navigation is near-instant when it's in the client-side cache. This will also allow back/forward navigation to be a high priority update instead of a transition as it is based on user interaction. Note: in this PR it still uses `startTransition` as there's no way to handle the high priority update suspending which happens in case of missing data in the cache. We're working with the React team on a solution for this particular case.

### Layouts

Note: this section assumes you've read [The layouts RFC](https://nextjs.org/blog/layouts-rfc) and [React Server Components RFC](https://reactjs.org/blog/2020/12/21/data-fetching-with-react-server-components.html)

React Server Components rendering leverages the Flight streaming mechanism in React 18, this allows sending a serializable representation of the rendered React tree on the server to the browser, the client-side React can use this serialized representation to render components client-side without the JavaScript being sent to the browser. This is one of the building blocks of Server Components. This allows a bunch of interesting features but for now I'll keep it to how it affects layouts.

When you have a `app/dashboard/layout.js` and `app/dashboard/page.js` the page will render as children of the layout, when you add another page like `app/dashboard/integrations/page.js` that page falls under the dashboard layout as well. When client-side navigating the new router automatically figures out if the page you're navigating to can be a smaller render than the whole page, in this case `app/dashboard/page.js` and `app/dashboard/integrations/page.js` share the `app/dashboard/layout.js` so instead of rendering the whole page we render below the layout component, this means the layout itself does not get re-rendered, the layout's `getServerSideProps` would not be called, and the Flight response would only hold the result of `app/dashboard/integrations/page.js`, effectively giving you the smallest patch for the UI.

---

Note: the commits in this PR were mostly work in progress to ensure it wasn't lost along the way. The implementation was reworked a bunch of times to where it is now.

Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
Co-authored-by: JJ Kasper <22380829+ijjk@users.noreply.github.com>
  • Loading branch information
@timneutkens @huozhi @ijjk
3 people committed Jul 6, 2022
1 parent 6e2c382 commit f113141
Show file tree
Hide file tree
Showing 41 changed files with 1,931 additions and 437 deletions.
2 changes: 1 addition & 1 deletion package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@
"eslint-plugin-import": "2.22.1",
"eslint-plugin-jest": "24.3.5",
"eslint-plugin-react": "7.23.2",
"eslint-plugin-react-hooks": "4.2.0",
"eslint-plugin-react-hooks": "4.5.0",
"event-stream": "4.0.1",
"execa": "2.0.3",
"express": "4.17.0",
Expand Down
191 changes: 86 additions & 105 deletions packages/next/build/webpack/loaders/next-app-loader.ts
View file
Original file line number Diff line number Diff line change
@@ -1,63 +1,93 @@
import path from 'path'
import type webpack from 'webpack5'
import { NODE_RESOLVE_OPTIONS } from '../../webpack-config'
import { getModuleBuildInfo } from './get-module-build-info'

function pathToUrlPath(pathname: string) {
let urlPath = pathname.replace(/^private-next-app-dir/, '')

// For `app/layout.js`
if (urlPath === '') {
urlPath = '/'
}

return urlPath
}

async function resolvePathsByPage({
name,
async function createTreeCodeFromPath({
pagePath,
resolve,
removeExt,
}: {
name: 'layout' | 'loading'
pagePath: string
resolve: (pathname: string) => Promise<string | undefined>
removeExt: (pathToRemoveExtensions: string) => string
}) {
const paths = new Map<string, string | undefined>()
const parts = pagePath.split('/')
let tree: undefined | string
const splittedPath = pagePath.split('/')
const appDirPrefix = splittedPath[0]

const segments = ['', ...splittedPath.slice(1)]
const isNewRootLayout =
parts[1]?.length > 2 && parts[1]?.startsWith('(') && parts[1]?.endsWith(')')
segments[0]?.length > 2 &&
segments[0]?.startsWith('(') &&
segments[0]?.endsWith(')')

let isCustomRootLayout = false

for (let i = parts.length; i >= 0; i--) {
const pathWithoutSlashLayout = parts.slice(0, i).join('/')
// segment.length - 1 because arrays start at 0 and we're decrementing
for (let i = segments.length - 1; i >= 0; i--) {
const segment = removeExt(segments[i])
const segmentPath = segments.slice(0, i + 1).join('/')

if (!pathWithoutSlashLayout) {
// First item in the list is the page which can't have layouts by itself
if (i === segments.length - 1) {
// Use '' for segment as it's the page. There can't be a segment called '' so this is the safest way to add it.
tree = `['', {}, {page: () => require('${pagePath}')}]`
continue
}
const layoutPath = `${pathWithoutSlashLayout}/${name}`
let resolvedLayoutPath = await resolve(layoutPath)
let urlPath = pathToUrlPath(pathWithoutSlashLayout)

// For segmentPath === '' avoid double `/`
const layoutPath = `${appDirPrefix}${segmentPath}/layout`
// For segmentPath === '' avoid double `/`
const loadingPath = `${appDirPrefix}${segmentPath}/loading`

const resolvedLayoutPath = await resolve(layoutPath)
const resolvedLoadingPath = await resolve(loadingPath)

// if we are in a new root app/(root) and a custom root layout was
// not provided or a root layout app/layout is not present, we use
// a default root layout to provide the html/body tags
const isCustomRootLayout = name === 'layout' && isNewRootLayout && i === 2

if (
name === 'layout' &&
(isCustomRootLayout || i === 1) &&
!resolvedLayoutPath
) {
resolvedLayoutPath = await resolve('next/dist/lib/app-layout')
}
paths.set(urlPath, resolvedLayoutPath)
isCustomRootLayout = isNewRootLayout && i === 1

// Existing tree are the children of the current segment
const children = tree

tree = `['${segment}', {
${
// When there are no children the current index is the page component
children ? `children: ${children},` : ''
}
}, {
${
resolvedLayoutPath
? `layout: () => require('${resolvedLayoutPath}'),`
: ''
}
${
resolvedLoadingPath
? `loading: () => require('${resolvedLoadingPath}'),`
: ''
}
}]`

// if we're in a new root layout don't add the top-level app/layout
if (isCustomRootLayout) {
break
}
}
return paths

return `const tree = ${tree};`
}

function createAbsolutePath(appDir: string, pathToTurnAbsolute: string) {
return pathToTurnAbsolute.replace(/^private-next-app-dir/, appDir)
}

function removeExtensions(
extensions: string[],
pathToRemoveExtensions: string
) {
const regex = new RegExp(`(${extensions.join('|')})$`.replace(/\./g, '\\.'))
return pathToRemoveExtensions.replace(regex, '')
}

const nextAppLoader: webpack.LoaderDefinitionFunction<{
Expand All @@ -71,7 +101,7 @@ const nextAppLoader: webpack.LoaderDefinitionFunction<{
const buildInfo = getModuleBuildInfo((this as any)._module)
buildInfo.route = {
page: name.replace(/^app/, ''),
absolutePagePath: appDir + pagePath.replace(/^private-next-app-dir/, ''),
absolutePagePath: createAbsolutePath(appDir, pagePath),
}

const extensions = pageExtensions.map((extension) => `.${extension}`)
Expand All @@ -81,89 +111,40 @@ const nextAppLoader: webpack.LoaderDefinitionFunction<{
}
const resolve = this.getResolve(resolveOptions)

const loadingPaths = await resolvePathsByPage({
name: 'loading',
pagePath: pagePath,
resolve: async (pathname) => {
try {
return await resolve(this.rootContext, pathname)
} catch (err: any) {
if (err.message.includes("Can't resolve")) {
return undefined
}
throw err
}
},
})

const loadingComponentsCode = []
for (const [loadingPath, resolvedLoadingPath] of loadingPaths) {
if (resolvedLoadingPath) {
this.addDependency(resolvedLoadingPath)
// use require so that we can bust the require cache
const codeLine = `'${loadingPath}': () => require('${resolvedLoadingPath}')`
loadingComponentsCode.push(codeLine)
} else {
const resolver = async (pathname: string) => {
try {
const resolved = await resolve(this.rootContext, pathname)
this.addDependency(resolved)
return resolved
} catch (err: any) {
const absolutePath = createAbsolutePath(appDir, pathname)
for (const ext of extensions) {
this.addMissingDependency(
path.join(appDir, loadingPath, `layout${ext}`)
)
}
}
}

const layoutPaths = await resolvePathsByPage({
name: 'layout',
pagePath: pagePath,
resolve: async (pathname) => {
try {
return await resolve(this.rootContext, pathname)
} catch (err: any) {
if (err.message.includes("Can't resolve")) {
return undefined
}
throw err
const absolutePathWithExtension = `${absolutePath}${ext}`
this.addMissingDependency(absolutePathWithExtension)
}
},
})

const componentsCode = []
for (const [layoutPath, resolvedLayoutPath] of layoutPaths) {
if (resolvedLayoutPath) {
this.addDependency(resolvedLayoutPath)
// use require so that we can bust the require cache
const codeLine = `'${layoutPath}': () => require('${resolvedLayoutPath}')`
componentsCode.push(codeLine)
} else {
for (const ext of extensions) {
this.addMissingDependency(path.join(appDir, layoutPath, `layout${ext}`))
if (err.message.includes("Can't resolve")) {
return undefined
}
throw err
}
}

// Add page itself to the list of components
componentsCode.push(
`'${pathToUrlPath(pagePath).replace(
new RegExp(`(${extensions.join('|')})$`),
''
// use require so that we can bust the require cache
)}': () => require('${pagePath}')`
)
const treeCode = await createTreeCodeFromPath({
pagePath,
resolve: resolver,
removeExt: (p) => removeExtensions(extensions, p),
})

const result = `
export const components = {
${componentsCode.join(',\n')}
};
export const loadingComponents = {
${loadingComponentsCode.join(',\n')}
};
export ${treeCode}
export const AppRouter = require('next/dist/client/components/app-router.client.js').default
export const LayoutRouter = require('next/dist/client/components/layout-router.client.js').default
export const hooksClientContext = require('next/dist/client/components/hooks-client-context.js')
export const __next_app_webpack_require__ = __webpack_require__
`

return result
}

Expand Down
7 changes: 0 additions & 7 deletions packages/next/client/app-index.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,6 @@ import { createFromReadableStream } from 'next/dist/compiled/react-server-dom-we

export const version = process.env.__NEXT_VERSION

// History replace has to happen on bootup to ensure `state` is always populated in popstate event
window.history.replaceState(
{ url: window.location.toString() },
'',
window.location.toString()
)

const appElement: HTMLElement | Document | null = document

let reactRoot: any = null
Expand Down

0 comments on commit f113141

Please sign in to comment.