refactor: move fs based inc cache to next-server #36246

huozhi · 2022-04-18T16:30:21Z

In edge runtime, node.js polyfills should not be applied, move more nodejs specific code (fs related) to next server before removing them

huozhi · 2022-04-18T16:36:00Z

test/integration/react-18/test/index.test.js

@@ -67,11 +68,13 @@ function runTestsAgainstRuntime(runtime) {
          invalidPage.write(`export const value = 1`)
        }
        nextConfig.replace("// runtime: 'edge'", `runtime: '${runtime}'`)
+        indexPage.replace("// runtime: 'edge'", `runtime: '${runtime}'`)


Setting page runtime to edge, this wasn't coverred before cause it will be statically optimized.
Adding this to avoid the basic build failed when removing the nodejs polyfills

ijjk · 2022-04-18T16:53:22Z

Stats from current PR

Default Build (Decrease detected ✓)

General Overall increase ⚠️

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
buildDuration	18.6s	18.7s	⚠️ +40ms
buildDurationCached	7.2s	7.4s	⚠️ +161ms
nodeModulesSize	485 MB	485 MB	⚠️ +2.14 kB

Page Load Tests Overall decrease ⚠️

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
/ failed reqs	0	0	✓
/ total time (seconds)	3.655	3.772	⚠️ +0.12
/ avg req/sec	683.93	662.71	⚠️ -21.22
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.571	1.598	⚠️ +0.03
/error-in-render avg req/sec	1591.54	1564.07	⚠️ -27.47

Client Bundles (main, webpack)

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
925.HASH.js gzip	179 B	179 B	✓
framework-HASH.js gzip	42 kB	42 kB	✓
main-HASH.js gzip	28.5 kB	28.5 kB	✓
webpack-HASH.js gzip	1.44 kB	1.44 kB	✓
Overall change	72.1 kB	72.1 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
_app-HASH.js gzip	1.36 kB	1.36 kB	✓
_error-HASH.js gzip	192 B	192 B	✓
amp-HASH.js gzip	309 B	309 B	✓
css-HASH.js gzip	327 B	327 B	✓
dynamic-HASH.js gzip	3.04 kB	3.04 kB	✓
head-HASH.js gzip	351 B	351 B	✓
hooks-HASH.js gzip	920 B	920 B	✓
image-HASH.js gzip	5.74 kB	5.74 kB	✓
index-HASH.js gzip	263 B	263 B	✓
link-HASH.js gzip	2.36 kB	2.36 kB	✓
routerDirect..HASH.js gzip	320 B	320 B	✓
script-HASH.js gzip	392 B	392 B	✓
withRouter-HASH.js gzip	319 B	319 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	16 kB	16 kB	✓

Client Build Manifests

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
_buildManifest.js gzip	461 B	461 B	✓
Overall change	461 B	461 B	✓

Rendered Page Sizes

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
index.html gzip	532 B	532 B	✓
link.html gzip	545 B	545 B	✓
withRouter.html gzip	527 B	527 B	✓
Overall change	1.6 kB	1.6 kB	✓

Default Build with SWC (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
buildDuration	21.7s	21.5s	-218ms
buildDurationCached	7.2s	7.3s	⚠️ +91ms
nodeModulesSize	485 MB	485 MB	⚠️ +2.14 kB

Page Load Tests Overall increase ✓

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
/ failed reqs	0	0	✓
/ total time (seconds)	3.798	3.752	-0.05
/ avg req/sec	658.19	666.33	+8.14
/error-in-render failed reqs	0	0	✓
/error-in-render total time (seconds)	1.634	1.576	-0.06
/error-in-render avg req/sec	1529.85	1586.22	+56.37

Client Bundles (main, webpack)

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
925.HASH.js gzip	178 B	178 B	✓
framework-HASH.js gzip	42.3 kB	42.3 kB	✓
main-HASH.js gzip	28.9 kB	28.9 kB	✓
webpack-HASH.js gzip	1.45 kB	1.45 kB	✓
Overall change	72.8 kB	72.8 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
_app-HASH.js gzip	1.35 kB	1.35 kB	✓
_error-HASH.js gzip	179 B	179 B	✓
amp-HASH.js gzip	313 B	313 B	✓
css-HASH.js gzip	325 B	325 B	✓
dynamic-HASH.js gzip	3.02 kB	3.02 kB	✓
head-HASH.js gzip	351 B	351 B	✓
hooks-HASH.js gzip	921 B	921 B	✓
image-HASH.js gzip	5.78 kB	5.78 kB	✓
index-HASH.js gzip	261 B	261 B	✓
link-HASH.js gzip	2.44 kB	2.44 kB	✓
routerDirect..HASH.js gzip	322 B	322 B	✓
script-HASH.js gzip	393 B	393 B	✓
withRouter-HASH.js gzip	317 B	317 B	✓
85e02e95b279..7e3.css gzip	107 B	107 B	✓
Overall change	16.1 kB	16.1 kB	✓

Client Build Manifests

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
_buildManifest.js gzip	457 B	457 B	✓
Overall change	457 B	457 B	✓

Rendered Page Sizes

	vercel/next.js canary	huozhi/next.js refactor/inc-cache	Change
index.html gzip	530 B	530 B	✓
link.html gzip	544 B	544 B	✓
withRouter.html gzip	526 B	526 B	✓
Overall change	1.6 kB	1.6 kB	✓

Commit: 2e4bc59

ijjk · 2022-04-19T19:00:36Z

Failing test suites

Commit: 2e4bc59

yarn testheadless test/integration/react-streaming-and-server-components/test/switchable-runtime.test.js

Switchable runtime (prod) > should build /edge as a dynamic page with the edge runtime
Switchable runtime (prod) > should build /edge-rsc as a dynamic page with the edge runtime
Switchable runtime (prod) > should support etag header in the web server

Expand output

● Switchable runtime (prod) › should build /edge as a dynamic page with the edge runtime

thrown: "Exceeded timeout of 90000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

  147 |   })
  148 |
> 149 |   it('should build /edge as a dynamic page with the edge runtime', async () => {
      |   ^
  150 |     await testRoute(context.appPort, '/edge', {
  151 |       isStatic: false,
  152 |       isEdge: true,

  at integration/react-streaming-and-server-components/test/switchable-runtime.test.js:149:3
  at Object.<anonymous> (integration/react-streaming-and-server-components/test/switchable-runtime.test.js:49:1)

● Switchable runtime (prod) › should build /edge-rsc as a dynamic page with the edge runtime

thrown: "Exceeded timeout of 90000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

  155 |   })
  156 |
> 157 |   it('should build /edge-rsc as a dynamic page with the edge runtime', async () => {
      |   ^
  158 |     await testRoute(context.appPort, '/edge-rsc', {
  159 |       isStatic: false,
  160 |       isEdge: true,

  at integration/react-streaming-and-server-components/test/switchable-runtime.test.js:157:3
  at Object.<anonymous> (integration/react-streaming-and-server-components/test/switchable-runtime.test.js:49:1)

● Switchable runtime (prod) › should support etag header in the web server

thrown: "Exceeded timeout of 90000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

  256 |   })
  257 |
> 258 |   it('should support etag header in the web server', async () => {
      |   ^
  259 |     const res = await fetchViaHTTP(context.appPort, '/edge', '', {
  260 |       headers: {
  261 |         // Make sure the result is static so an etag can be generated.

  at integration/react-streaming-and-server-components/test/switchable-runtime.test.js:258:3
  at Object.<anonymous> (integration/react-streaming-and-server-components/test/switchable-runtime.test.js:49:1)

Read more about building and testing Next.js in contributing.md.

yarn testheadless test/integration/react-18/test/index.test.js

Concurrent mode in the edge runtime dev > throws if useFlushEffects is used more than once
Concurrent mode in the edge runtime dev > throws if useFlushEffects is called on the client
Concurrent mode in the edge runtime prod > throws if useFlushEffects is used more than once
Concurrent mode in the edge runtime prod > throws if useFlushEffects is called on the client

Expand output

● Concurrent mode in the edge runtime dev › throws if useFlushEffects is used more than once

thrown: "Exceeded timeout of 90000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

  17 |   }
  18 |
> 19 |   it('throws if useFlushEffects is used more than once', async () => {
     |   ^
  20 |     await renderViaHTTP(context.appPort, '/use-flush-effect/multiple-calls')
  21 |     expect(context.stderr).toContain(
  22 |       'Error: The `useFlushEffects` hook cannot be used more than once.'

  at Object._default [as default] (integration/react-18/test/concurrent.js:19:3)
  at Object.runTests (integration/react-18/test/index.test.js:52:17)
  at lib/next-test-utils.js:786:13
  at runSuite (lib/next-test-utils.js:754:3)
  at Object.runDevSuite (lib/next-test-utils.js:797:10)
  at runTests (integration/react-18/test/index.test.js:89:42)
  at runTestsAgainstRuntime (integration/react-18/test/index.test.js:49:3)
  at Object.<anonymous> (integration/react-18/test/index.test.js:84:1)

● Concurrent mode in the edge runtime dev › throws if useFlushEffects is called on the client

page.goto: Timeout 30000ms exceeded.
=========================== logs ===========================
navigating to "http://localhost:38669/use-flush-effect/client", waiting until "load"
============================================================

  149 |       this.activeTrace = encodeURIComponent(url)
  150 |     }
> 151 |     await page.goto(url, { waitUntil: 'load' })
      |                ^
  152 |   }
  153 |
  154 |   back(): BrowserInterface {

  at Playwright.loadPage (lib/browsers/playwright.ts:151:16)
  at Object.webdriver [as default] (lib/next-webdriver.ts:106:3)
  at withBrowser (integration/react-18/test/concurrent.js:10:17)
  at Object.<anonymous> (integration/react-18/test/concurrent.js:27:5)

● Concurrent mode in the edge runtime prod › throws if useFlushEffects is used more than once

thrown: "Exceeded timeout of 90000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

  17 |   }
  18 |
> 19 |   it('throws if useFlushEffects is used more than once', async () => {
     |   ^
  20 |     await renderViaHTTP(context.appPort, '/use-flush-effect/multiple-calls')
  21 |     expect(context.stderr).toContain(
  22 |       'Error: The `useFlushEffects` hook cannot be used more than once.'

  at Object._default [as default] (integration/react-18/test/concurrent.js:19:3)
  at Object.runTests (integration/react-18/test/index.test.js:52:17)
  at lib/next-test-utils.js:786:13
  at runSuite (lib/next-test-utils.js:754:3)
  at Object.runProdSuite (lib/next-test-utils.js:807:10)
  at runTests (integration/react-18/test/index.test.js:90:43)
  at runTestsAgainstRuntime (integration/react-18/test/index.test.js:49:3)
  at Object.<anonymous> (integration/react-18/test/index.test.js:84:1)

● Concurrent mode in the edge runtime prod › throws if useFlushEffects is called on the client

page.goto: Timeout 30000ms exceeded.
=========================== logs ===========================
navigating to "http://localhost:37813/use-flush-effect/client", waiting until "load"
============================================================

  149 |       this.activeTrace = encodeURIComponent(url)
  150 |     }
> 151 |     await page.goto(url, { waitUntil: 'load' })
      |                ^
  152 |   }
  153 |
  154 |   back(): BrowserInterface {

  at Playwright.loadPage (lib/browsers/playwright.ts:151:16)
  at Object.webdriver [as default] (lib/next-webdriver.ts:106:3)
  at withBrowser (integration/react-18/test/concurrent.js:10:17)
  at Object.<anonymous> (integration/react-18/test/concurrent.js:27:5)

Read more about building and testing Next.js in contributing.md.

huozhi · 2022-04-20T10:23:31Z

pages with edge and rsc are still relying on inc-cache, shouldn't fully move it to node server only

ijjk added created-by: Next.js team PRs by the Next.js team type: next labels Apr 18, 2022

huozhi mentioned this pull request Apr 18, 2022

Switchable server runtime #31506

Closed

15 tasks

huozhi commented Apr 18, 2022

View reviewed changes

huozhi changed the title ~~refactor: move fs based inc cache to next-serevr~~ refactor: move fs based inc cache to next-server Apr 18, 2022

huozhi added 3 commits April 19, 2022 20:37

refactor: move fs based inc cache to next-serevr

481e61a

move response cache

c66f5b1

update types

2e4bc59

huozhi force-pushed the refactor/inc-cache branch from 98b1da1 to 2e4bc59 Compare April 19, 2022 18:38

huozhi closed this Apr 20, 2022

huozhi deleted the refactor/inc-cache branch April 20, 2022 10:23

github-actions bot locked as resolved and limited conversation to collaborators May 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: move fs based inc cache to next-server #36246

refactor: move fs based inc cache to next-server #36246

huozhi commented Apr 18, 2022

huozhi Apr 18, 2022

ijjk commented Apr 18, 2022 •

edited

ijjk commented Apr 19, 2022 •

edited

huozhi commented Apr 20, 2022

refactor: move fs based inc cache to next-server #36246

refactor: move fs based inc cache to next-server #36246

Conversation

huozhi commented Apr 18, 2022

huozhi Apr 18, 2022

Choose a reason for hiding this comment

ijjk commented Apr 18, 2022 • edited

Stats from current PR

ijjk commented Apr 19, 2022 • edited

Failing test suites

huozhi commented Apr 20, 2022

ijjk commented Apr 18, 2022 •

edited

ijjk commented Apr 19, 2022 •

edited