[Epic] Amazing Performance Initiative

[RobbieMcKinstry]

Overview

"Amazing Performance" is an Platform Team initiative that started in Q3 2022 and continued in Q4 2022. The objective is to improve performance (primarily runtime but, to a less degree, memory consumption) for as many users as possible.

Backlog

• perf(filestate): DoesProjectExist should not list all #12547 help-wanted We'd love your contributions on this issue good-first-issue Start here if you'd like to start contributing to Pulumi
• Localize TypeScript Imports #10442
• Consider bundling/minifying Pulumi Node SDK  #10210
• [Node] Perf: Minify Providers Code without Losing Sourcemaps #10403
• Make ts-node and typescript peer dependencies #10122
• [Node] Perf: Minify Providers Code without Losing Sourcemaps #10403
• GetRequiredPlugins may hang if CWD contains large numbers of files or has circularities #2928
• [nodejs/codegen] Lazy-load leaf enum modules #10607
• Parenting to a resource chain with many alias combinations slows down SDK execution #10010
• Ensure tracing output includes annotated spans for every subprocess invocation #7311 help-wanted We'd love your contributions on this issue
• Surprising logging overhead in node #7146
• Revisit ts-node version #4876
• [Node] Perf: Minify Providers Code without Losing Sourcemaps #10403
• Increase usage of Go build caching by adding it into templates help-wanted We'd love your contributions on this issue
• Step generation is done sequentially, making providers that block in Diff extremely slow #12351
• Ensure plugin downloads and TypeScript compilation have spans in the trace

Out of scope

Related work items and ideas for the future, likely out scope but retained here in case they need to be pulled in:

Tracing improvements

• Upgrade from opentracing to opentelemetry help-wanted We'd love your contributions on this issue
• Add tracing to multi-lang provider servers #7148
• Improve startup performance #3671
  • Mitigate Service call overhead in httpstate backend
  • Improve language runtime init
    • pulumi new for Python should create and activate virtual environment #2201
    • Mitigate TS compilation (on larger projects runs into 10s of seconds eg see Use SWC to transpile typescript projects #9049 (comment))
      • Allow opt-in for faster SWC Use SWC to transpile typescript projects #9049
      • Allow user upgrades of TypeScript+ts-node Make ts-node and typescript peer dependencies #10122
  • Prewarm providers when imported
• Related see performance section of azure-native 2.0 epic https://github.com/pulumi/home/issues/2061
  • Node package causes poor performance from tsserver pulumi-azure-native#932

Misc

• [Perf] [NodeJS] Async Loading of Required Modules #10487

Benchmarks and metrics improvements

• Add a CI status check to ensure PRs don't violate performance SLAs (proactively enforce SLAs instead of reactively enforcing them)
• Add a metric for Service client network bandwidth
• Add a benchmark to measure Secret handling
• Collect data on using service vs local backend

Completed Work

M82

• Closure emit requires in global scope, to have them run in Lambda init phase #11481
• Reduce size of typescript input/output definitions files #8613
• Localize TypeScript Imports #10442

M80

• Reduce size of typescript input/output definitions files #8613 (Reverted, see Tracking Issue: Splitting Types in NodeJS #11558)

M79

• [SDK/Python] Parallelism Flag Not Respected #11116

M78

• Fix orphan register-resource traces #10636
• Use dotnet run --no-build #10661
• Add Distributed Tracing to Node Runtime #10180
• https://github.com/pulumi/home/issues/2294
• Apply prewarm when benchmarking templates#362
• Do not call exportStack twice on every update #10770
• Add summary command to pulumi-trace-tool pulumi-trace-tool#3
• Cache Go builds with rebuild: true runtime option #10473
• Allow disabling tracing payload logging #10830
• Ensure response parsing is included in HTTP client trace spans #10828
• Recalibrate performance benchmark alerting thresholds after recent measurement fixes
• Build a cost model for basic programs that use one cloud provider
• Build a client for the new efficient PATCH checkpoint protocol
• Support Service team https://github.com/pulumi/pulumi-service/issues/6497
  • Related https://github.com/pulumi/home/issues/1768

M77

• https://github.com/pulumi/home/issues/2229
• Consider explicit go build instead of go run for running Go programs #10350
• https://github.com/pulumi/home/issues/1498
• https://github.com/pulumi/home/issues/2200
• Lazily export nodejs sub-modules #7653
• Fix perf data inconsistency by prewarming examples benchmarks examples#1271
• https://github.com/pulumi/home/issues/2129

M75

• Add tracing around os.Exec in node provider #10173
• Trace files produced by pulumi --tracing do not show AWS provider calls #7571
• Fix orphan bulk decrypt traces #10037
• "Importing "typescript" is costing 300ms startup even when unused": Avoid importing typescript unless necessary #10185 && Make sure import pulumi/pulumi does not necessarily import typescript #10214
• Opt out of TypeScript in JavaScript templates templates#312