Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: make module analysis async #477

Merged
merged 5 commits into from
May 16, 2024

Conversation

dsherret
Copy link
Member

@dsherret dsherret commented May 16, 2024

This allows us to parse source files in parallel for the first run:

Benchmark 1: ../deno/deno_old run -A ts_morph.ts
  Time (mean ± σ):      2.332 s ±  0.017 s    [User: 2.193 s, System: 0.126 s]
  Range (min … max):    2.307 s …  2.366 s    10 runs
 
Benchmark 2: ../deno/target/release/deno run -A ts_morph.ts
  Time (mean ± σ):      1.052 s ±  0.006 s    [User: 2.259 s, System: 0.132 s]
  Range (min … max):    1.040 s …  1.060 s    10 runs
 
Summary
  ../deno/target/release/deno run -A ts_morph.ts ran
    2.22 ± 0.02 times faster than ../deno/deno_old run -A ts_morph.ts
scratch % cat ts_morph.ts
import "https://deno.land/x/ts_morph@22.0.0/mod.ts";
import "https://deno.land/x/ts_morph@21.0.0/mod.ts";
import "https://deno.land/x/ts_morph@20.0.0/mod.ts";
import "https://deno.land/x/ts_morph@19.0.0/mod.ts";

Same speed on second run:

Benchmark 1: ../deno/deno_old run -A ts_morph.ts
  Time (mean ± σ):     419.2 ms ±   3.7 ms    [User: 390.3 ms, System: 40.7 ms]
  Range (min … max):   415.1 ms … 428.8 ms    10 runs
 
Benchmark 2: ../deno/target/release/deno run -A ts_morph.ts
  Time (mean ± σ):     415.4 ms ±   4.6 ms    [User: 385.7 ms, System: 39.6 ms]
  Range (min … max):   412.2 ms … 425.1 ms    10 runs
 
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
 
Summary
  ../deno/target/release/deno run -A ts_morph.ts ran
    1.01 ± 0.01 times faster than ../deno/deno_old run -A ts_morph.ts

Closes #474

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code in here is pretty crazy, but it was initially crazy. We should spend some time cleaning this up in the future.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you jot down some notes in an issue about what's crazy and how it could be improved? This might be a good task in the future for someone getting to know this codebase.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By "crazy" I just mean messy. Someone needs to go through and clean it up sometime.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, feels like it should be split into multiple smaller modules.

@ry
Copy link
Member

ry commented May 16, 2024

maybe this should rather be "perf: make module analysis async"

@dsherret dsherret changed the title refactor: make module analysis async perf: make module analysis async May 16, 2024
@dsherret dsherret requested a review from bartlomieju May 16, 2024 20:28
Comment on lines +2058 to +2070
pub(crate) enum ModuleSourceAndInfo {
Json {
specifier: ModuleSpecifier,
source: Arc<str>,
},
Js {
specifier: ModuleSpecifier,
media_type: MediaType,
source: Arc<str>,
maybe_headers: Option<HashMap<String, String>>,
module_info: Box<ModuleInfo>,
},
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised you don't get a warning about enum variants big size difference

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, feels like it should be split into multiple smaller modules.

@dsherret dsherret merged commit 67cdf5d into denoland:main May 16, 2024
16 checks passed
@dsherret dsherret deleted the refactoring_module_info_to_loader branch May 16, 2024 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Perf - Look into ways of running module analysis in parallel
3 participants