Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support parallel construct requests for nodejs multi-lang components #7629

Open
EvanBoyle opened this issue Jul 24, 2021 · 5 comments
Open
Labels
area/component-packages aka multi-language components kind/enhancement Improvements or new features language/javascript language/python

Comments

@EvanBoyle
Copy link
Contributor

For stability and due to global state issues in the nodejs runtime, we serialized MLC construct requests in #6452

This has some obvious downsides like not creating cloud resources in parallel if you are instantiating multiple instances of the same component in a program.

Will require some of the same set of work as #5449 to remove global state from the nodejs runtime.

@t0yv0
Copy link
Member

t0yv0 commented Apr 1, 2024

Possibly causing pulumi/pulumi-awsx#1250 ? @justinvp is this still an issue?

@mikhailshilkov
Copy link
Member

Yes, I think this is still an issue and is indeed causing serialized deployments in awsx:

// Serialize invocations of `construct` and `call` so that each call runs one after another, avoiding concurrent
// runs. We do this because `construct` and `call` modify global state to reset the SDK's runtime options.
// This is a short-term workaround to provide correctness, but likely isn't sustainable long-term due to the
// limits it places on parallelism. We will likely want to investigate if it's possible to run each invocation
// in its own context, possibly using Node's `createContext` API to avoid modifying global state:
// https://nodejs.org/api/vm.html#vm_vm_createcontext_contextobject_options

@denis-tierno-lc
Copy link

Just for visibility's sake, this is a deal breaker for my deployment. My app spawns 15 services, and they are kinda slow, so it takes almost one hour for a deployment to complete.

@justinvp
Copy link
Member

I suspect this should be a relatively easy to fix. We already made the changes in the Node.js SDK to avoid the global state issue for Automation API, we just hadn’t gone back to change the provider code to no longer serialize Construct and Call calls.

@mikhailshilkov
Copy link
Member

@justinvp Great! Is it just a matter of removing the serialization? What is the testing that we would need to add?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/component-packages aka multi-language components kind/enhancement Improvements or new features language/javascript language/python
Projects
None yet
Development

No branches or pull requests

5 participants