Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bazel OOM when using swc rules for a large repo #76

Closed
ewianda opened this issue Aug 25, 2022 · 5 comments
Closed

Bazel OOM when using swc rules for a large repo #76

ewianda opened this issue Aug 25, 2022 · 5 comments
Labels
enhancement New feature or request question This issue is a question. Close the loop with documentation?

Comments

@ewianda
Copy link

ewianda commented Aug 25, 2022

I am trying to migrate a large code base to using swc transpiler and I keep getting

FATAL: bazel ran out of memory and crashed. Printing stack trace:
java.lang.OutOfMemoryError: Java heap space

The only difference I see in my config is the use of --enable_runfiles . I have looked into the code to
understand why this flag is needed and if it is possible to turn it off.

@ewianda ewianda changed the title Bazel OOM when using --enable_runfiles Bazel OOM when using swc rules for a large repo Aug 25, 2022
@cgrindel cgrindel added bug Something isn't working need: discussion enhancement New feature or request and removed need: discussion bug Something isn't working labels Sep 27, 2022
@gregmagolan
Copy link
Member

How much memory does the machine have to work with and how many typescript files do you have in your repository?

Using swc as the transpiler creates one target per typescript file which can make for a very large graph.

@gregmagolan gregmagolan added question This issue is a question. Close the loop with documentation? and removed prioritized labels Oct 12, 2022
@ewianda
Copy link
Author

ewianda commented Oct 18, 2022

We have 32GB machine and a repository of 1200 typescript files.
I even configure the bazel to have more memory with --host_jvm_args=-Xmx16g and no luck.

I also noticed that --enable_runfiles is not not required.

@alexeagle
Copy link
Member

I wonder if the use of rules_js contributed to the large amount of starlark heap used. The next release includes #57 which might help.

I think we need the profile from bazel. @ewianda if you're still able to repro after the next release here, would you mind following https://bazel.build/rules/performance#memory-profiling to get an output of bazel dump --rules and a prof.gz that we can inspect?

@ewianda
Copy link
Author

ewianda commented Jan 10, 2023

Sorry, @alexeagle and @gregmagolan I should have reported back earlier.

This error went away after we migrated from rules_nodejs to rules_js.

@alexeagle
Copy link
Member

Awesome thank you for confirming!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question This issue is a question. Close the loop with documentation?
Projects
None yet
Development

No branches or pull requests

4 participants