@next/swc is not a development dependency (blowing Heroku slug size)

[carlosbaraza]

What version of Next.js are you using?

12.0.7

What version of Node.js are you using?

14.15.4

What browser are you using?

Chrome

What operating system are you using?

Linux

How are you deploying your application?

Heroku

Describe the Bug

Heroku limits the compressed size of the app folder to 500 MB. Somehow, the total compressed size of the app went from 200 to 600, after upgrading from Next 11 to 12.

After investigating the problem, the issue is that somehow, all the binaries for SWC are included in the build and not removed when running npm prune.

$ du -ha --max-depth 3 /app | sort -hr | grep @next
481M	/app/node_modules/@next
48M	/app/node_modules/@next/swc-linux-x64-gnu
48M	/app/node_modules/@next/swc-android-arm64
47M	/app/node_modules/@next/swc-linux-x64-musl
47M	/app/node_modules/@next/swc-linux-arm64-musl
47M	/app/node_modules/@next/swc-linux-arm64-gnu
47M	/app/node_modules/@next/swc-linux-arm-gnueabihf
44M	/app/node_modules/@next/swc-win32-x64-msvc
40M	/app/node_modules/@next/swc-win32-arm64-msvc
40M	/app/node_modules/@next/swc-darwin-x64
38M	/app/node_modules/@next/swc-win32-ia32-msvc
38M	/app/node_modules/@next/swc-darwin-arm64
1.8M	/app/node_modules/@next/react-dev-overlay
68K	/app/node_modules/@next/react-refresh-utils
32K	/app/node_modules/@next/env
28K	/app/node_modules/@next/polyfill-module
....

The actual problem is that because those are binaries, they barely compress, so they really break the slug size limit of heroku.

Expected Behavior

Given that the build was already completed, I imagine it is unnecessary to have this packages in production. We should allow these packages to be removed by npm prune.

To Reproduce

Deploy any Next 12 app to Heroku.

[balazsorban44]

Some possible docs changing PRs that can help to address this right now:

#32255
#32258

[carlosbaraza]

For Heroku and NextJS 12.0.7, I had to add the following custom Heroku build steps to package.json:

Disable SWC compiler because of #31855

"heroku-postbuild": "echo '{\"presets\": [\"next/babel\"]}' > .babelrc && npm run build",

Remove SWC and .next/cache to reduce the slug size:

"heroku-cleanup": "rm -rf .next/cache && find node_modules/@next -depth -maxdepth 1 -type d -name 'swc*' -exec rm -rf {} \\; -prune #Remove cache and SWC binaries to reduce heroku slug size",

Hope this workaround helps someone.

[carlosbaraza]

@balazsorban44 I guess the outputStandalone flag could work, but it would still need a custom server to handle the static files if you wanted to host it all in one container without having to integrate a CDN? I wonder how well it plays with another server.

[abriginets]

@carlosbaraza your solution looks really awesome, I'll try it later today. But still I think it's a bad idea from NextJS devs to auto include all the binaries in the bundle. For example, Prisma only includes the engine binary you need.

[wi-ski]

For anyone else - this is what got us below the heroku slug size limit:

rm -rf .next/cache;
# Remove these from node_modules because very large
find node_modules/@next -depth -maxdepth 1 -type d -name 'swc*' -exec rm -rf {} \\; -prune;
find node_modules/@swc -depth -maxdepth 1 -type d -name 'core-win*' -exec rm -rf {} \\; -prune;
find node_modules/@swc -depth -maxdepth 1 -type d -name 'core-android*' -exec rm -rf {} \\; -prune;

[ijjk]

Hi, this should not be the case unless you are force installing all optional dependencies. Further you can optimize deployment size by leveraging the output: 'standalone' Next.js config as documented here https://nextjs.org/docs/advanced-features/output-file-tracing#automatically-copying-traced-files

I'm gonna close this as this shouldn't be an issue in the latest version with up to date versions of your package manager.

[wi-ski]

@ijjk - Thank you for those pointers, seriously. For those tripping over this issue in the future: https://stackoverflow.com/questions/69159313/prevent-optional-dependencies-to-be-installed-on-heroku

To set 'optional dependencies' on heroku:

$ heroku config:set --app $APP_NAME NPM_CONFIG_OMIT=optional

@ijjk Do you have any insight into how you use the 'standalone'-ed bundles with a custom server???

For example:

// This handler in my custom server
const nextJsHandler = app.getRequestHandler(); // Does the presence of the `.next/standalone` make this behave differently? It doesnt seem using next@12.2.5

[wi-ski]

@ijjk - Rounding back on this, it does not appear that NPM_CONFIG_OMIT works when declared on heroku (we still see those binaries installed 😢). Is there a different config you're describing?

Also - forgot to remove core-darwin* deps in the previous script. Added them below:

find node_modules/@next -depth -maxdepth 1 -type d -name 'swc*' -exec rm -rf {} \; -prune;
find node_modules/@swc -depth -maxdepth 1 -type d -name 'core-win*' -exec rm -rf {} \; -prune;
find node_modules/@swc -depth -maxdepth 1 -type d -name 'core-android*' -exec rm -rf {} \; -prune;
find node_modules/@swc -depth -maxdepth 1 -type d -name 'core-darwin*' -exec rm -rf {} \; -prune;

[ijjk]

@wi-ski how are you installing your dependencies only the relevant version should be installed for your platform

yarn

pnpm

standalone mode creates a server.js file already the same setup can be copied to a custom server and used installed although tracing the necessary dependencies for the custom server will need to be done separately as custom servers are external and Next.js isn't controlling them.

[github-actions]

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.