Recommended way to build multiple lambdas with tree-shaking

[doteric]

Hello,
So I'm trying to figure out the most convenient way to build lambdas using the serverless framework.
The problem I am facing is tree-shaking of the lambdas. Serverless basically includes everything in the zip even if the setting individually is set to true.

What I need (requirements):

• Multiple lambdas built from the same serverless and package.json config
• Tree shaking of source files and node_modules
• Retain file/directory structure of output (not bundle into one file)
• Simple TypeScript compilation is fine (for example webpack compiles with additional stuff that I would like to avoid)

Initially I tried serverless-plugin-tree-shake, but it looks outdate, doesn't seem to work with serverless >=3.0 and had some edge case problems.
Later I went for serverless-webpack and managed to put a configuration together where I would retain the file structure using the splitChunks feature, but this causes some additional unneeded complexity and also webpack produces unwanted extra code and modifies it in a specific way.
I also considered serverless-esbuild, but it doesn't seem to support retaining the file/dir structure.

So my question is - what would be the recommended way for me considering the requirements I listed above?
Appreciate any suggestions and comments, cheers

[samuel-pipelaunch]

Have the same challenge. Any updates?

[doteric]

Hey @samuel-pipelaunch ,
So I've decided to drop that one point "Retain file/directory structure of output (not bundle into one file)", because I couldn't find anything that suited every single of my point and started to use esbuild (retaining the structure could maybe be possible in the future with esbuild though https://esbuild.github.io/api/#splitting). Personally I only deploy lambdas using terraform therefore I decided to drop serverless as a whole and created a "function packager" of my own.
https://www.npmjs.com/package/funpack (https://github.com/doteric/funpack)
Feel free to use it and report and issues or ideas you have.

[nponeccop]

Serverless excludeDevDependencies had a bug where it treated runtime dependencies of your development dependencies as runtime depencencies and didn't include them. So the first thing you try is to disable the exclusion and use yarn install --prod before serverless deploy. It will require some jumping over hoops (e.g. serverless plugins should be installed outside of your node_modules).

• Then you can use yarnclean. The defaul yarnclean does a decent job on cleaning some junk, and I found it unproblematic in the long run after you fix the initial quirk or two.

• Use the TS outputDir, don't lump TS and JS together in one folder as it's the default.

• Deploy only JS files and sourceMaps. Turn the native source maps on if required (they work with 14+)

• Then you use patterns to pack what you need. The !** is a trick to only pack what is explicitly mentioned.

package:
  excludeDevDependencies: false
  patterns:
    - '!**'
    - 'dist/**/*'
    - 'node_modules/**/*'
    - 'processing/*.php'
    - '*.yml'
    - '!**/*.tsbuildinfo'
    - '!node_modules/@aws-sdk/**/dist-types'
    - '!node_modules/**/{*.ts,.bin}'
    - '!node_modules/.yarn.integrity'

Note that the biggest offender these days is the 4MB binary aws-crt (at least it is if you use aws-sdk@3 which you should). You need to exclude the platforms you won't need. As it's platform dependent I do that in my CI script:

    - yarn install --frozen-lockfile
    - yarn install --frozen-lockfile --prod
    - rm -rv node_modules/aws-crt/dist/bin/{linux-arm64,{darwin,win32}-x64}
    - serverless deploy --verbose ..

The first yarn install builds the TS
The second one removes the devDependencies from node_modules.

You can get some ideas of what to remove by running your code under strace, tracing execve and openat syscalls, and comparing that with the contents of node_modules`. This is not very reliable though, but then no tree-shaking is.

I expect decent shakers to be developed one day though :) But this is a workaround that doesn't use any potentially buggy software, and it does help to reduce my tree.

Also, you should put your node_modules to a layer. I didn't try that yet though.

[doteric]

Hey @nponeccop
Big thanks for your reply. For now I'm using esbuild that is built into my package (https://www.npmjs.com/package/funpack) and it works great for what I need and will probably stick with that for now. Your approach seems pretty interesting though also so thanks 💪 Maybe someone else will take use of it.

[solid-ahlholm]

serverless-jetpack should tick all your boxes.

[doteric]

Hey @solid-ahlholm,
https://www.npmjs.com/package/funpack serves me well for now and I do not actually require the serverless package so it's win-win 😁