Huge shrinkwrap with oclif latest

[otaviojacobi]

Newer versions of oclif (v4) use aws-sdk v3 which end up generating a huge shrinkrwarp file if you have oclif as a dev dependency (and use it programatically from within ts/js code). The fingerprint of aws-sdk v3 is so large that my CI fails with too many open files (which can indeed be workaroundable) but all that for a dev dependency of a aws-sdk that I don't even use as I don't use the commands that push anything to s3.

One simple way to reproduce/check is to just run npm i --save-dev oclif@^3.0.0 and npm i --save-dev oclif@^4.0.0 and compare tha package-lock.json difference. Deduped it went from ~8k lines to ~50k lines.

I would like to propose that @aws-sdk/client-cloudfront @aws-sdk/client-s3 to become optional dependencies (and that the code is refactored in a way that initialization won't fail if these are not present) so that clients can choose wether or not these are installed & generate these huge diff from previous versions.

I can also work on a PR for it if we can agree that this is something that would be valuable for oclif.

Thanks!

[mdonnalley]

Thanks for bringing this up @otaviojacobi

There was as point in time when we required CLIs to have the aws-sdk as a devDependency so that oclif could use that instead of having to depend on it itself (see the description for upload tarballs). Unfortunately, at some point aws-sdk got moved to dependencies during a refactor and no one noticed - now it would be a breaking change to go back to the original design 😞

So, yeah, I agree that there's a better approach to this but it will have to wait until we do the next major version. Not sure exactly when that would be since we typically like to have a few breaking changes lined up so we can reduce the number of majors that we release.

[otaviojacobi]

Hello @mdonnalley why would moving these into a optional dependency (rather than a dev one) be a breaking change? It would still install by default on npm i --production (or just npm i) but just not if you explicitely install it without the optional ones. It would just probably require a few changes on how inner modules import things

Thanks!

[mdonnalley]

My thought was if the optional dependency fails to install and/or the user intentionally omits optional dependencies, then any command that depends on aws-sdk will fail, which might be unexpected for the user.

Consider a CLI that currently uses the --no-optional flag for some unrelated reason and then all of a sudden upload tarballs fails because aws-sdk isn't installed. In that case, we've broken their CI pipeline unexpectedly.

Perhaps there's a way to make the change without disrupting anyone's CI but I think it's best to operate with an abundance of caution for changes like this.

[otaviojacobi]

That makes sense, thanks for clarifying.

For anyone facing the same issue, I have minimized the impact of this by removing oclif of my dev dependencies and on my ci workflw I do "npm i oclif --no-save" in order to be able to use it without having to keep the huge dev dependencies.

[tuler]

I'm not sure if our issue is related to this, but I'm about to migrate to another CLI framework because I feel our CLI is getting extremely bloated and hard to install.
The dependencies we have, with their install sizes are:

@oclif/core: 6.38Mb
@oclif/plugin-help: 8.64Mb
@oclif/plugin-plugins: 17.7Mb
@oclif/plugin-update: 16.9Mb

This adds up to ~50Mb of download. And I have "oclif" as devDependency.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.