Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't cache Ternary.Maybe results when recursion is encountered during variance measurement #41218

Merged
merged 8 commits into from
Oct 27, 2020
Merged

Don't cache Ternary.Maybe results when recursion is encountered during variance measurement #41218

merged 8 commits into from
Oct 27, 2020

Conversation

ahejlsberg
Copy link
Member

Fixes #39947.

@typescript-bot typescript-bot added the For Milestone Bug PRs that fix a bug with a specific milestone label Oct 23, 2020
@ahejlsberg
Copy link
Member Author

@typescript-bot test this
@typescript-bot user test this
@typescript-bot run dt
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 23, 2020 •
edited

Heya @ahejlsberg, I've started to run the parallelized Definitely Typed test suite on this PR at 39724bd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 23, 2020 •
edited

Heya @ahejlsberg, I've started to run the perf test suite on this PR at 39724bd. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 23, 2020 •
edited

Heya @ahejlsberg, I've started to run the extended test suite on this PR at 39724bd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 23, 2020 •
edited

Heya @ahejlsberg, I've started to run the parallelized community code test suite on this PR at 39724bd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric master 41218 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 349,757k (± 0.02%) 350,671k (± 0.02%) +914k (+ 0.26%) 350,530k 350,785k
Parse Time 2.01s (± 0.80%) 2.02s (± 0.45%) +0.01s (+ 0.50%) 2.00s 2.04s
Bind Time 0.82s (± 1.07%) 0.83s (± 1.13%) +0.01s (+ 1.22%) 0.82s 0.86s
Check Time 4.94s (± 0.54%) 5.42s (± 0.33%) +0.48s (+ 9.82%) 5.38s 5.47s
Emit Time 5.19s (± 0.41%) 5.25s (± 0.54%) +0.06s (+ 1.19%) 5.20s 5.33s
Total Time 12.96s (± 0.34%) 13.53s (± 0.27%) +0.57s (+ 4.37%) 13.45s 13.60s
Monaco - node (v10.16.3, x64)
Memory used 354,351k (± 0.02%) 354,544k (± 0.02%) +192k (+ 0.05%) 354,337k 354,707k
Parse Time 1.57s (± 0.57%) 1.57s (± 0.43%) +0.01s (+ 0.51%) 1.56s 1.59s
Bind Time 0.71s (± 0.56%) 0.72s (± 1.14%) +0.01s (+ 0.84%) 0.70s 0.74s
Check Time 5.07s (± 0.54%) 5.53s (± 0.75%) +0.46s (+ 9.13%) 5.44s 5.63s
Emit Time 2.76s (± 0.93%) 2.79s (± 0.72%) +0.02s (+ 0.80%) 2.74s 2.84s
Total Time 10.11s (± 0.37%) 10.61s (± 0.51%) +0.50s (+ 4.91%) 10.50s 10.77s
TFS - node (v10.16.3, x64)
Memory used 307,640k (± 0.04%) 307,844k (± 0.02%) +204k (+ 0.07%) 307,636k 308,004k
Parse Time 1.22s (± 0.76%) 1.23s (± 0.45%) +0.01s (+ 0.49%) 1.22s 1.24s
Bind Time 0.66s (± 1.37%) 0.67s (± 0.78%) +0.01s (+ 1.21%) 0.65s 0.68s
Check Time 4.57s (± 0.94%) 5.04s (± 0.76%) +0.47s (+10.33%) 4.95s 5.13s
Emit Time 2.90s (± 1.72%) 2.87s (± 0.94%) -0.03s (- 0.97%) 2.81s 2.92s
Total Time 9.35s (± 0.59%) 9.81s (± 0.36%) +0.46s (+ 4.93%) 9.73s 9.87s
material-ui - node (v10.16.3, x64)
Memory used 489,125k (± 0.02%) 489,213k (± 0.01%) +88k (+ 0.02%) 489,110k 489,335k
Parse Time 1.99s (± 0.54%) 2.00s (± 0.65%) +0.01s (+ 0.30%) 1.97s 2.03s
Bind Time 0.65s (± 0.90%) 0.65s (± 1.02%) +0.00s (+ 0.46%) 0.63s 0.66s
Check Time 13.42s (± 0.51%) 13.83s (± 0.58%) +0.41s (+ 3.02%) 13.68s 14.10s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.06s (± 0.45%) 16.48s (± 0.49%) +0.42s (+ 2.60%) 16.34s 16.73s
Angular - node (v12.1.0, x64)
Memory used 326,934k (± 0.02%) 327,583k (± 0.10%) +649k (+ 0.20%) 326,547k 327,909k
Parse Time 2.00s (± 0.53%) 2.02s (± 0.76%) +0.02s (+ 0.80%) 1.98s 2.05s
Bind Time 0.81s (± 1.12%) 0.82s (± 2.45%) +0.01s (+ 1.61%) 0.80s 0.89s
Check Time 4.83s (± 0.40%) 5.37s (± 0.67%) +0.53s (+10.98%) 5.29s 5.45s
Emit Time 5.39s (± 0.84%) 5.43s (± 1.09%) +0.03s (+ 0.63%) 5.34s 5.54s
Total Time 13.04s (± 0.52%) 13.63s (± 0.69%) +0.59s (+ 4.56%) 13.44s 13.84s
Monaco - node (v12.1.0, x64)
Memory used 336,578k (± 0.02%) 336,780k (± 0.01%) +202k (+ 0.06%) 336,735k 336,858k
Parse Time 1.54s (± 0.72%) 1.55s (± 0.61%) +0.02s (+ 1.11%) 1.53s 1.58s
Bind Time 0.69s (± 0.58%) 0.70s (± 1.21%) +0.01s (+ 1.45%) 0.69s 0.73s
Check Time 4.88s (± 0.50%) 5.38s (± 0.47%) +0.50s (+10.26%) 5.33s 5.44s
Emit Time 2.82s (± 0.91%) 2.82s (± 0.75%) +0.00s (+ 0.04%) 2.79s 2.88s
Total Time 9.93s (± 0.39%) 10.46s (± 0.40%) +0.53s (+ 5.32%) 10.41s 10.57s
TFS - node (v12.1.0, x64)
Memory used 291,933k (± 0.03%) 292,039k (± 0.02%) +105k (+ 0.04%) 291,905k 292,196k
Parse Time 1.23s (± 0.49%) 1.25s (± 0.85%) +0.02s (+ 1.71%) 1.22s 1.27s
Bind Time 0.65s (± 1.38%) 0.64s (± 0.63%) -0.01s (- 1.08%) 0.63s 0.65s
Check Time 4.47s (± 0.30%) 4.98s (± 0.60%) +0.51s (+11.40%) 4.92s 5.05s
Emit Time 2.92s (± 0.99%) 2.95s (± 0.75%) +0.03s (+ 0.99%) 2.92s 3.02s
Total Time 9.26s (± 0.36%) 9.81s (± 0.39%) +0.55s (+ 6.00%) 9.76s 9.92s
material-ui - node (v12.1.0, x64)
Memory used 467,034k (± 0.05%) 467,253k (± 0.05%) +219k (+ 0.05%) 466,303k 467,488k
Parse Time 2.01s (± 0.37%) 2.04s (± 0.52%) +0.02s (+ 1.14%) 2.02s 2.06s
Bind Time 0.64s (± 1.05%) 0.64s (± 0.78%) -0.00s (- 0.31%) 0.63s 0.65s
Check Time 12.05s (± 0.94%) 12.48s (± 0.86%) +0.43s (+ 3.56%) 12.28s 12.74s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.70s (± 0.81%) 15.15s (± 0.76%) +0.45s (+ 3.05%) 14.95s 15.44s
Angular - node (v8.9.0, x64)
Memory used 346,442k (± 0.01%) 347,337k (± 0.02%) +895k (+ 0.26%) 347,185k 347,428k
Parse Time 2.54s (± 0.51%) 2.57s (± 0.41%) +0.02s (+ 0.79%) 2.54s 2.59s
Bind Time 0.85s (± 0.43%) 0.87s (± 0.81%) +0.02s (+ 1.76%) 0.85s 0.88s
Check Time 5.56s (± 0.52%) 5.98s (± 0.34%) +0.42s (+ 7.52%) 5.93s 6.01s
Emit Time 6.15s (± 1.07%) 6.21s (± 1.47%) +0.06s (+ 1.06%) 6.07s 6.46s
Total Time 15.11s (± 0.55%) 15.62s (± 0.71%) +0.51s (+ 3.40%) 15.45s 15.90s
Monaco - node (v8.9.0, x64)
Memory used 355,704k (± 0.02%) 355,800k (± 0.01%) +95k (+ 0.03%) 355,708k 355,884k
Parse Time 1.89s (± 0.77%) 1.89s (± 0.44%) +0.01s (+ 0.42%) 1.88s 1.92s
Bind Time 0.89s (± 0.45%) 0.90s (± 0.53%) +0.01s (+ 0.67%) 0.89s 0.91s
Check Time 5.62s (± 0.37%) 6.13s (± 1.11%) +0.51s (+ 9.00%) 6.04s 6.36s
Emit Time 3.29s (± 1.17%) 3.26s (± 2.86%) -0.02s (- 0.67%) 2.89s 3.34s
Total Time 11.69s (± 0.28%) 12.19s (± 0.48%) +0.50s (+ 4.26%) 12.02s 12.30s
TFS - node (v8.9.0, x64)
Memory used 309,391k (± 0.02%) 309,499k (± 0.02%) +108k (+ 0.03%) 309,396k 309,657k
Parse Time 1.55s (± 0.64%) 1.55s (± 0.38%) +0.00s (+ 0.06%) 1.54s 1.57s
Bind Time 0.68s (± 0.73%) 0.68s (± 0.59%) +0.00s (+ 0.44%) 0.67s 0.69s
Check Time 5.31s (± 0.58%) 5.77s (± 0.42%) +0.46s (+ 8.61%) 5.73s 5.84s
Emit Time 2.94s (± 0.34%) 2.98s (± 1.00%) +0.04s (+ 1.22%) 2.89s 3.02s
Total Time 10.48s (± 0.34%) 10.98s (± 0.26%) +0.49s (+ 4.71%) 10.92s 11.05s
material-ui - node (v8.9.0, x64)
Memory used 493,431k (± 0.01%) 493,520k (± 0.01%) +89k (+ 0.02%) 493,350k 493,659k
Parse Time 2.40s (± 0.46%) 2.42s (± 0.57%) +0.02s (+ 0.67%) 2.40s 2.47s
Bind Time 0.81s (± 1.02%) 0.81s (± 1.36%) -0.00s (- 0.49%) 0.79s 0.84s
Check Time 17.95s (± 0.69%) 18.34s (± 0.75%) +0.39s (+ 2.16%) 18.06s 18.59s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.17s (± 0.56%) 21.57s (± 0.70%) +0.40s (+ 1.88%) 21.27s 21.87s
Angular - node (v8.9.0, x86)
Memory used 198,690k (± 0.01%) 199,201k (± 0.03%) +511k (+ 0.26%) 199,106k 199,331k
Parse Time 2.48s (± 1.02%) 2.50s (± 0.94%) +0.02s (+ 0.89%) 2.46s 2.55s
Bind Time 1.00s (± 1.00%) 1.02s (± 1.46%) +0.02s (+ 1.69%) 0.98s 1.04s
Check Time 5.05s (± 0.56%) 5.50s (± 0.76%) +0.45s (+ 8.91%) 5.40s 5.57s
Emit Time 5.89s (± 0.83%) 6.02s (± 1.04%) +0.14s (+ 2.29%) 5.94s 6.20s
Total Time 14.42s (± 0.58%) 15.03s (± 0.60%) +0.62s (+ 4.28%) 14.82s 15.27s
Monaco - node (v8.9.0, x86)
Memory used 201,493k (± 0.01%) 201,591k (± 0.02%) +98k (+ 0.05%) 201,467k 201,665k
Parse Time 1.94s (± 1.05%) 1.95s (± 0.91%) +0.00s (+ 0.15%) 1.91s 2.00s
Bind Time 0.71s (± 0.47%) 0.71s (± 0.71%) -0.00s (- 0.70%) 0.70s 0.72s
Check Time 5.47s (± 0.38%) 6.18s (± 0.51%) +0.71s (+12.97%) 6.12s 6.25s
Emit Time 3.06s (± 0.59%) 2.71s (± 0.78%) 🟩-0.35s (-11.35%) 2.67s 2.77s
Total Time 11.18s (± 0.25%) 11.54s (± 0.28%) +0.36s (+ 3.22%) 11.47s 11.60s
TFS - node (v8.9.0, x86)
Memory used 176,867k (± 0.03%) 176,963k (± 0.02%) +96k (+ 0.05%) 176,903k 177,022k
Parse Time 1.60s (± 1.10%) 1.60s (± 1.29%) -0.00s (- 0.25%) 1.57s 1.66s
Bind Time 0.66s (± 2.04%) 0.64s (± 0.76%) -0.01s (- 1.68%) 0.64s 0.66s
Check Time 4.82s (± 0.58%) 5.32s (± 0.67%) +0.50s (+10.33%) 5.24s 5.39s
Emit Time 2.80s (± 0.66%) 2.85s (± 1.08%) +0.04s (+ 1.60%) 2.80s 2.92s
Total Time 9.88s (± 0.51%) 10.41s (± 0.53%) +0.53s (+ 5.33%) 10.28s 10.55s
material-ui - node (v8.9.0, x86)
Memory used 277,878k (± 0.02%) 277,958k (± 0.02%) +81k (+ 0.03%) 277,801k 278,089k
Parse Time 2.46s (± 0.43%) 2.48s (± 0.62%) +0.02s (+ 1.02%) 2.45s 2.51s
Bind Time 0.69s (± 1.35%) 0.70s (± 2.22%) +0.02s (+ 2.47%) 0.67s 0.75s
Check Time 16.45s (± 0.73%) 16.83s (± 0.81%) +0.38s (+ 2.30%) 16.54s 17.20s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 19.60s (± 0.63%) 20.02s (± 0.79%) +0.42s (+ 2.14%) 19.66s 20.40s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory2 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 41218 10
Baseline master 10

@typescript-bot typescript-bot mentioned this pull request Oct 23, 2020
Open
@typescript-bot
Copy link
Collaborator

The user suite test run you requested has finished and failed. I've opened a PR with the baseline diff from master.

weswigham
weswigham reviewed Oct 23, 2020
View reviewed changes
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

recursiveVariances should be un-set by recursiveTypeRelatedTo once we want to cache a non-maybe value, no? So if there is, thereafter, a Maybe during the same comparison that is not variance related, it is cache able?

@ahejlsberg
Copy link
Member Author

@weswigham In some sense we really want to have two different Ternary.Maybe values: One that originates in the depth limiter (the original one) and one that originates in a recursive variance measurement (the new one). The new one would be "infectious" and we would never put it in the cache. What's implemented now is sort of a blunt approximation of that, but once we have a Ternary.Maybe that originated in recursive variance measurement we have to assume every Ternary.Maybe is possibly one of those and thus not cacheable.

@ahejlsberg
Copy link
Member Author

Hmm, the effect on performance is definitely unacceptable. It might be better with the two kinds of Ternary.Maybe I describe above, but I'm not sure. Unfortunately it's not a small amount of work to find out.

@ahejlsberg
Copy link
Member Author

Actually, wasn't too hard to introduce a Ternary.Unknown that acts like a signaling Ternary.Maybe. We'll see if that helps performance.

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 24, 2020 •
edited

Heya @ahejlsberg, I've started to run the perf test suite on this PR at dcfce3f. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric master 41218 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Monaco - node (v10.16.3, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
TFS - node (v10.16.3, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
material-ui - node (v10.16.3, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Angular - node (v12.1.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Monaco - node (v12.1.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
TFS - node (v12.1.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
material-ui - node (v12.1.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Angular - node (v8.9.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Monaco - node (v8.9.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
TFS - node (v8.9.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
material-ui - node (v8.9.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Angular - node (v8.9.0, x86)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Monaco - node (v8.9.0, x86)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
TFS - node (v8.9.0, x86)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
material-ui - node (v8.9.0, x86)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory2 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 41218 10
Baseline master 10

@ahejlsberg
Copy link
Member Author

@rbuckton Hmm, the reporting from that last test runs seems broken. Any ideas?

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 24, 2020 •
edited

Heya @ahejlsberg, I've started to run the perf test suite on this PR at dcfce3f. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric master 41218 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Monaco - node (v10.16.3, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
TFS - node (v10.16.3, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
material-ui - node (v10.16.3, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Angular - node (v12.1.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Monaco - node (v12.1.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
TFS - node (v12.1.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
material-ui - node (v12.1.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Angular - node (v8.9.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Monaco - node (v8.9.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
TFS - node (v8.9.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
material-ui - node (v8.9.0, x64)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Angular - node (v8.9.0, x86)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Monaco - node (v8.9.0, x86)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
TFS - node (v8.9.0, x86)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
material-ui - node (v8.9.0, x86)
Memory used 0k 0k 0k ( NaN%) 0k 0k
Parse Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Bind Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Check Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Emit Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
Total Time 0.00s 0.00s 0.00s ( NaN%) 0.00s 0.00s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory2 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 41218 10
Baseline master 10

@rbuckton
Copy link
Member

Might be related to the switch to the native performance API. I'll look into it.

@rbuckton
Copy link
Member

#41253 will fix the issue with tracking performance data.

@rbuckton
Copy link
Member

#41253 has been merged. If you merge the latest from master the perf results should be working now.

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 26, 2020 •
edited

Heya @ahejlsberg, I've started to run the perf test suite on this PR at 2b8ca38. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric master 41218 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 350,683k (± 0.02%) 350,670k (± 0.02%) -13k (- 0.00%) 350,553k 350,978k
Parse Time 2.09s (± 0.38%) 2.09s (± 0.28%) -0.00s (- 0.10%) 2.08s 2.10s
Bind Time 0.84s (± 1.03%) 0.84s (± 0.35%) +0.00s (+ 0.48%) 0.84s 0.85s
Check Time 5.44s (± 0.41%) 5.43s (± 0.37%) -0.01s (- 0.13%) 5.39s 5.47s
Emit Time 5.34s (± 0.40%) 5.36s (± 1.20%) +0.03s (+ 0.47%) 5.27s 5.54s
Total Time 13.71s (± 0.27%) 13.72s (± 0.50%) +0.02s (+ 0.12%) 13.62s 13.90s
Monaco - node (v10.16.3, x64)
Memory used 354,753k (± 0.02%) 354,735k (± 0.02%) -18k (- 0.01%) 354,559k 354,909k
Parse Time 1.62s (± 0.21%) 1.61s (± 0.60%) -0.01s (- 0.37%) 1.59s 1.64s
Bind Time 0.72s (± 0.72%) 0.71s (± 0.83%) -0.01s (- 0.70%) 0.70s 0.72s
Check Time 5.56s (± 0.48%) 5.56s (± 0.50%) -0.00s (- 0.02%) 5.49s 5.65s
Emit Time 2.81s (± 0.71%) 2.80s (± 0.56%) -0.00s (- 0.18%) 2.77s 2.84s
Total Time 10.70s (± 0.42%) 10.69s (± 0.32%) -0.01s (- 0.13%) 10.62s 10.78s
TFS - node (v10.16.3, x64)
Memory used 307,908k (± 0.02%) 307,983k (± 0.03%) +75k (+ 0.02%) 307,816k 308,252k
Parse Time 1.24s (± 0.59%) 1.24s (± 0.42%) -0.00s (- 0.24%) 1.23s 1.25s
Bind Time 0.68s (± 1.41%) 0.67s (± 0.99%) -0.00s (- 0.30%) 0.66s 0.69s
Check Time 5.05s (± 0.69%) 5.05s (± 0.55%) +0.00s (+ 0.02%) 5.00s 5.12s
Emit Time 2.94s (± 1.14%) 2.88s (± 1.08%) -0.06s (- 2.11%) 2.80s 2.95s
Total Time 9.91s (± 0.53%) 9.84s (± 0.32%) -0.07s (- 0.68%) 9.75s 9.90s
material-ui - node (v10.16.3, x64)
Memory used 489,304k (± 0.01%) 489,314k (± 0.01%) +10k (+ 0.00%) 489,188k 489,421k
Parse Time 2.08s (± 0.34%) 2.08s (± 0.50%) -0.00s (- 0.05%) 2.06s 2.11s
Bind Time 0.65s (± 0.85%) 0.65s (± 0.69%) -0.00s (- 0.61%) 0.64s 0.66s
Check Time 13.98s (± 0.70%) 13.93s (± 0.87%) -0.06s (- 0.40%) 13.75s 14.36s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.72s (± 0.60%) 16.66s (± 0.75%) -0.06s (- 0.38%) 16.46s 17.09s
Angular - node (v12.1.0, x64)
Memory used 327,887k (± 0.02%) 327,896k (± 0.02%) +9k (+ 0.00%) 327,711k 328,079k
Parse Time 2.09s (± 0.57%) 2.09s (± 0.42%) -0.01s (- 0.29%) 2.07s 2.11s
Bind Time 0.82s (± 0.94%) 0.82s (± 0.73%) -0.00s (- 0.61%) 0.81s 0.83s
Check Time 5.40s (± 0.67%) 5.39s (± 0.67%) -0.01s (- 0.19%) 5.30s 5.47s
Emit Time 5.49s (± 0.62%) 5.50s (± 0.60%) +0.00s (+ 0.07%) 5.42s 5.59s
Total Time 13.81s (± 0.45%) 13.79s (± 0.39%) -0.02s (- 0.12%) 13.62s 13.92s
Monaco - node (v12.1.0, x64)
Memory used 336,815k (± 0.01%) 336,832k (± 0.01%) +18k (+ 0.01%) 336,696k 336,919k
Parse Time 1.60s (± 0.62%) 1.60s (± 0.96%) -0.00s (- 0.06%) 1.57s 1.65s
Bind Time 0.70s (± 0.96%) 0.70s (± 0.52%) -0.00s (- 0.14%) 0.69s 0.70s
Check Time 5.41s (± 0.49%) 5.42s (± 0.71%) +0.01s (+ 0.26%) 5.37s 5.54s
Emit Time 2.89s (± 1.35%) 2.87s (± 1.13%) -0.02s (- 0.52%) 2.81s 2.96s
Total Time 10.60s (± 0.49%) 10.59s (± 0.60%) -0.01s (- 0.07%) 10.48s 10.79s
TFS - node (v12.1.0, x64)
Memory used 292,242k (± 0.02%) 292,212k (± 0.03%) -30k (- 0.01%) 292,044k 292,376k
Parse Time 1.26s (± 0.82%) 1.26s (± 0.85%) -0.00s (- 0.08%) 1.24s 1.29s
Bind Time 0.65s (± 1.08%) 0.65s (± 1.36%) -0.00s (- 0.61%) 0.63s 0.67s
Check Time 5.04s (± 0.68%) 5.02s (± 0.22%) -0.02s (- 0.40%) 4.99s 5.04s
Emit Time 2.97s (± 1.14%) 2.95s (± 0.68%) -0.01s (- 0.44%) 2.91s 3.00s
Total Time 9.92s (± 0.60%) 9.88s (± 0.33%) -0.04s (- 0.39%) 9.82s 9.97s
material-ui - node (v12.1.0, x64)
Memory used 467,310k (± 0.05%) 466,926k (± 0.10%) -384k (- 0.08%) 466,025k 467,588k
Parse Time 2.10s (± 0.63%) 2.08s (± 0.69%) -0.02s (- 0.95%) 2.05s 2.11s
Bind Time 0.64s (± 0.90%) 0.64s (± 1.18%) +0.00s (+ 0.16%) 0.63s 0.67s
Check Time 12.56s (± 1.19%) 12.50s (± 0.84%) -0.06s (- 0.49%) 12.32s 12.76s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.31s (± 1.00%) 15.23s (± 0.70%) -0.08s (- 0.49%) 15.04s 15.49s
Angular - node (v8.9.0, x64)
Memory used 353,106k (± 0.02%) 353,068k (± 0.01%) -38k (- 0.01%) 352,916k 353,156k
Parse Time 2.65s (± 0.31%) 2.66s (± 0.26%) +0.00s (+ 0.19%) 2.65s 2.67s
Bind Time 0.88s (± 1.25%) 0.88s (± 0.77%) +0.00s (+ 0.11%) 0.87s 0.90s
Check Time 6.05s (± 0.45%) 6.00s (± 0.37%) -0.05s (- 0.76%) 5.96s 6.05s
Emit Time 6.48s (± 0.82%) 6.35s (± 1.25%) -0.13s (- 1.99%) 6.16s 6.49s
Total Time 16.07s (± 0.48%) 15.90s (± 0.48%) -0.17s (- 1.05%) 15.70s 16.02s
Monaco - node (v8.9.0, x64)
Memory used 358,497k (± 0.01%) 358,537k (± 0.02%) +40k (+ 0.01%) 358,372k 358,645k
Parse Time 1.94s (± 0.30%) 1.95s (± 0.52%) +0.01s (+ 0.31%) 1.93s 1.98s
Bind Time 0.90s (± 0.54%) 0.90s (± 0.86%) -0.01s (- 0.55%) 0.89s 0.92s
Check Time 6.17s (± 0.94%) 6.14s (± 0.69%) -0.03s (- 0.53%) 6.04s 6.25s
Emit Time 3.29s (± 2.46%) 3.33s (± 0.51%) +0.04s (+ 1.12%) 3.30s 3.38s
Total Time 12.31s (± 0.43%) 12.32s (± 0.37%) +0.00s (+ 0.03%) 12.21s 12.41s
TFS - node (v8.9.0, x64)
Memory used 310,333k (± 0.02%) 310,333k (± 0.01%) +0k (+ 0.00%) 310,224k 310,420k
Parse Time 1.58s (± 0.52%) 1.58s (± 0.35%) -0.00s (- 0.13%) 1.57s 1.59s
Bind Time 0.68s (± 0.33%) 0.68s (± 0.73%) -0.00s (- 0.73%) 0.66s 0.68s
Check Time 5.80s (± 0.56%) 5.76s (± 0.62%) -0.04s (- 0.66%) 5.71s 5.88s
Emit Time 2.99s (± 0.91%) 2.96s (± 0.97%) -0.03s (- 1.07%) 2.89s 2.99s
Total Time 11.05s (± 0.47%) 10.98s (± 0.30%) -0.07s (- 0.66%) 10.87s 11.04s
material-ui - node (v8.9.0, x64)
Memory used 496,234k (± 0.01%) 496,241k (± 0.01%) +8k (+ 0.00%) 496,088k 496,472k
Parse Time 2.52s (± 0.53%) 2.52s (± 0.46%) -0.00s (- 0.12%) 2.49s 2.54s
Bind Time 0.82s (± 1.90%) 0.82s (± 1.33%) -0.00s (- 0.61%) 0.79s 0.84s
Check Time 18.44s (± 1.31%) 18.41s (± 0.71%) -0.03s (- 0.17%) 18.16s 18.65s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.79s (± 1.18%) 21.74s (± 0.61%) -0.04s (- 0.19%) 21.50s 22.01s
Angular - node (v8.9.0, x86)
Memory used 202,109k (± 0.02%) 202,104k (± 0.03%) -6k (- 0.00%) 201,990k 202,239k
Parse Time 2.57s (± 0.68%) 2.57s (± 1.10%) -0.00s (- 0.04%) 2.54s 2.65s
Bind Time 1.01s (± 1.01%) 1.00s (± 0.91%) -0.01s (- 0.99%) 0.99s 1.03s
Check Time 5.52s (± 0.44%) 5.52s (± 0.77%) 0.00s ( 0.00%) 5.43s 5.63s
Emit Time 6.06s (± 0.99%) 6.07s (± 0.79%) +0.00s (+ 0.03%) 5.95s 6.21s
Total Time 15.17s (± 0.50%) 15.16s (± 0.41%) -0.00s (- 0.02%) 15.03s 15.28s
Monaco - node (v8.9.0, x86)
Memory used 202,954k (± 0.02%) 202,947k (± 0.01%) -7k (- 0.00%) 202,895k 202,981k
Parse Time 1.98s (± 0.71%) 1.98s (± 1.03%) -0.00s (- 0.10%) 1.96s 2.06s
Bind Time 0.71s (± 0.78%) 0.71s (± 0.73%) -0.00s (- 0.14%) 0.70s 0.72s
Check Time 6.22s (± 0.64%) 6.22s (± 0.88%) +0.00s (+ 0.02%) 6.15s 6.42s
Emit Time 2.76s (± 0.69%) 2.77s (± 1.01%) +0.01s (+ 0.54%) 2.73s 2.86s
Total Time 11.67s (± 0.48%) 11.69s (± 0.57%) +0.02s (+ 0.14%) 11.56s 11.87s
TFS - node (v8.9.0, x86)
Memory used 177,392k (± 0.04%) 177,415k (± 0.01%) +22k (+ 0.01%) 177,375k 177,475k
Parse Time 1.63s (± 0.89%) 1.63s (± 0.77%) 0.00s ( 0.00%) 1.60s 1.66s
Bind Time 0.64s (± 0.90%) 0.64s (± 0.46%) -0.00s (- 0.16%) 0.64s 0.65s
Check Time 5.34s (± 1.05%) 5.36s (± 0.90%) +0.02s (+ 0.34%) 5.28s 5.50s
Emit Time 2.83s (± 1.50%) 2.83s (± 1.01%) -0.00s (- 0.18%) 2.76s 2.89s
Total Time 10.45s (± 0.90%) 10.46s (± 0.46%) +0.01s (+ 0.12%) 10.36s 10.59s
material-ui - node (v8.9.0, x86)
Memory used 279,337k (± 0.01%) 279,319k (± 0.01%) -18k (- 0.01%) 279,239k 279,400k
Parse Time 2.57s (± 0.72%) 2.57s (± 0.47%) -0.00s (- 0.12%) 2.54s 2.60s
Bind Time 0.77s (± 7.52%) 0.76s (± 5.57%) -0.01s (- 1.69%) 0.69s 0.87s
Check Time 16.90s (± 0.88%) 16.90s (± 0.86%) +0.01s (+ 0.05%) 16.63s 17.35s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 20.24s (± 0.61%) 20.23s (± 0.73%) -0.01s (- 0.04%) 19.99s 20.63s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory2 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 41218 10
Baseline master 10

@ahejlsberg
Copy link
Member Author

Performance looks good now and issues in test suites all appear to be preexisting conditions.

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Oct 27, 2020 •
edited

Heya @ahejlsberg, I've started to run the perf test suite on this PR at a132c04. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..41218

Metric master 41218 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 350,615k (± 0.02%) 350,588k (± 0.02%) -27k (- 0.01%) 350,398k 350,715k
Parse Time 2.07s (± 0.70%) 2.09s (± 0.61%) +0.02s (+ 0.82%) 2.06s 2.12s
Bind Time 0.84s (± 0.66%) 0.84s (± 0.71%) +0.00s (+ 0.36%) 0.83s 0.86s
Check Time 4.96s (± 0.46%) 4.97s (± 0.51%) +0.01s (+ 0.24%) 4.92s 5.04s
Emit Time 5.31s (± 0.69%) 5.30s (± 0.38%) -0.01s (- 0.13%) 5.26s 5.34s
Total Time 13.19s (± 0.41%) 13.21s (± 0.36%) +0.02s (+ 0.17%) 13.11s 13.33s
Monaco - node (v10.16.3, x64)
Memory used 354,587k (± 0.01%) 354,589k (± 0.01%) +1k (+ 0.00%) 354,453k 354,685k
Parse Time 1.60s (± 0.66%) 1.60s (± 0.52%) +0.00s (+ 0.12%) 1.59s 1.62s
Bind Time 0.72s (± 0.56%) 0.72s (± 0.77%) -0.00s (- 0.14%) 0.71s 0.73s
Check Time 5.13s (± 0.35%) 5.11s (± 0.56%) -0.01s (- 0.29%) 5.07s 5.20s
Emit Time 2.79s (± 0.76%) 2.80s (± 0.67%) +0.01s (+ 0.39%) 2.76s 2.83s
Total Time 10.23s (± 0.29%) 10.23s (± 0.45%) -0.00s (- 0.01%) 10.16s 10.37s
TFS - node (v10.16.3, x64)
Memory used 307,837k (± 0.02%) 307,823k (± 0.02%) -14k (- 0.00%) 307,716k 307,996k
Parse Time 1.23s (± 0.57%) 1.24s (± 0.80%) +0.01s (+ 0.57%) 1.22s 1.26s
Bind Time 0.67s (± 0.71%) 0.68s (± 0.50%) +0.00s (+ 0.59%) 0.67s 0.68s
Check Time 4.56s (± 0.58%) 4.61s (± 0.80%) +0.05s (+ 1.01%) 4.52s 4.68s
Emit Time 2.93s (± 0.70%) 2.94s (± 1.22%) +0.01s (+ 0.27%) 2.86s 3.01s
Total Time 9.39s (± 0.47%) 9.46s (± 0.67%) +0.07s (+ 0.72%) 9.34s 9.60s
material-ui - node (v10.16.3, x64)
Memory used 489,242k (± 0.01%) 489,283k (± 0.01%) +41k (+ 0.01%) 489,184k 489,407k
Parse Time 2.07s (± 0.56%) 2.08s (± 0.53%) +0.01s (+ 0.58%) 2.05s 2.09s
Bind Time 0.65s (± 0.89%) 0.65s (± 0.61%) -0.00s (- 0.31%) 0.64s 0.66s
Check Time 13.56s (± 0.80%) 13.60s (± 0.74%) +0.05s (+ 0.37%) 13.43s 13.84s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.27s (± 0.72%) 16.34s (± 0.66%) +0.06s (+ 0.39%) 16.15s 16.58s
Angular - node (v12.1.0, x64)
Memory used 327,792k (± 0.02%) 327,685k (± 0.09%) -106k (- 0.03%) 326,596k 328,013k
Parse Time 2.07s (± 0.64%) 2.08s (± 0.78%) +0.01s (+ 0.39%) 2.05s 2.12s
Bind Time 0.81s (± 1.02%) 0.82s (± 1.69%) +0.01s (+ 0.61%) 0.80s 0.87s
Check Time 4.92s (± 0.47%) 4.91s (± 0.38%) -0.01s (- 0.12%) 4.87s 4.97s
Emit Time 5.50s (± 0.57%) 5.56s (± 1.21%) +0.05s (+ 0.93%) 5.44s 5.77s
Total Time 13.31s (± 0.20%) 13.37s (± 0.63%) +0.06s (+ 0.42%) 13.21s 13.64s
Monaco - node (v12.1.0, x64)
Memory used 336,733k (± 0.02%) 336,719k (± 0.01%) -14k (- 0.00%) 336,627k 336,803k
Parse Time 1.58s (± 0.64%) 1.58s (± 0.63%) -0.00s (- 0.06%) 1.56s 1.61s
Bind Time 0.70s (± 0.49%) 0.70s (± 0.57%) +0.00s (+ 0.57%) 0.69s 0.71s
Check Time 4.90s (± 0.29%) 4.93s (± 0.32%) +0.03s (+ 0.55%) 4.88s 4.96s
Emit Time 2.87s (± 0.64%) 2.86s (± 0.86%) -0.01s (- 0.17%) 2.82s 2.94s
Total Time 10.05s (± 0.27%) 10.07s (± 0.30%) +0.02s (+ 0.20%) 9.99s 10.14s
TFS - node (v12.1.0, x64)
Memory used 292,070k (± 0.01%) 292,042k (± 0.02%) -28k (- 0.01%) 291,927k 292,125k
Parse Time 1.25s (± 0.80%) 1.26s (± 0.84%) +0.01s (+ 0.88%) 1.24s 1.28s
Bind Time 0.64s (± 1.04%) 0.65s (± 0.93%) +0.00s (+ 0.31%) 0.63s 0.66s
Check Time 4.49s (± 0.44%) 4.49s (± 0.29%) -0.00s (- 0.00%) 4.47s 4.52s
Emit Time 2.96s (± 0.49%) 2.97s (± 1.02%) +0.00s (+ 0.03%) 2.87s 3.04s
Total Time 9.35s (± 0.23%) 9.37s (± 0.44%) +0.02s (+ 0.18%) 9.28s 9.46s
material-ui - node (v12.1.0, x64)
Memory used 467,273k (± 0.05%) 467,363k (± 0.01%) +89k (+ 0.02%) 467,167k 467,459k
Parse Time 2.07s (± 0.45%) 2.09s (± 0.60%) +0.01s (+ 0.63%) 2.06s 2.11s
Bind Time 0.64s (± 0.87%) 0.64s (± 0.92%) +0.00s (+ 0.31%) 0.63s 0.66s
Check Time 11.99s (± 0.49%) 12.06s (± 0.95%) +0.07s (+ 0.58%) 11.88s 12.30s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.71s (± 0.41%) 14.79s (± 0.77%) +0.09s (+ 0.58%) 14.58s 15.03s
Angular - node (v8.9.0, x64)
Memory used 352,969k (± 0.02%) 352,981k (± 0.01%) +12k (+ 0.00%) 352,896k 353,149k
Parse Time 2.65s (± 0.47%) 2.65s (± 0.47%) 0.00s ( 0.00%) 2.63s 2.69s
Bind Time 0.88s (± 0.88%) 0.88s (± 0.63%) -0.00s (- 0.11%) 0.87s 0.89s
Check Time 5.60s (± 0.55%) 5.62s (± 0.71%) +0.02s (+ 0.41%) 5.51s 5.70s
Emit Time 6.28s (± 0.92%) 6.31s (± 1.16%) +0.03s (+ 0.51%) 6.12s 6.43s
Total Time 15.41s (± 0.38%) 15.46s (± 0.53%) +0.06s (+ 0.36%) 15.25s 15.63s
Monaco - node (v8.9.0, x64)
Memory used 358,422k (± 0.01%) 358,392k (± 0.01%) -29k (- 0.01%) 358,276k 358,476k
Parse Time 1.94s (± 0.46%) 1.93s (± 0.53%) -0.00s (- 0.26%) 1.92s 1.97s
Bind Time 0.90s (± 0.90%) 0.91s (± 0.40%) +0.01s (+ 0.67%) 0.90s 0.91s
Check Time 5.63s (± 0.61%) 5.64s (± 0.64%) +0.01s (+ 0.20%) 5.59s 5.75s
Emit Time 3.38s (± 0.90%) 3.40s (± 0.41%) +0.02s (+ 0.74%) 3.37s 3.44s
Total Time 11.84s (± 0.39%) 11.88s (± 0.43%) +0.04s (+ 0.35%) 11.79s 12.04s
TFS - node (v8.9.0, x64)
Memory used 310,267k (± 0.01%) 310,274k (± 0.01%) +6k (+ 0.00%) 310,194k 310,360k
Parse Time 1.57s (± 0.51%) 1.58s (± 0.43%) +0.01s (+ 0.51%) 1.56s 1.59s
Bind Time 0.68s (± 0.91%) 0.68s (± 0.70%) +0.00s (+ 0.59%) 0.67s 0.69s
Check Time 5.30s (± 0.50%) 5.35s (± 0.73%) +0.05s (+ 1.02%) 5.28s 5.44s
Emit Time 2.98s (± 0.98%) 2.99s (± 0.50%) +0.01s (+ 0.23%) 2.96s 3.04s
Total Time 10.53s (± 0.47%) 10.60s (± 0.47%) +0.07s (+ 0.67%) 10.53s 10.73s
material-ui - node (v8.9.0, x64)
Memory used 496,195k (± 0.02%) 496,258k (± 0.01%) +63k (+ 0.01%) 496,134k 496,378k
Parse Time 2.50s (± 0.42%) 2.50s (± 0.66%) +0.00s (+ 0.08%) 2.46s 2.55s
Bind Time 0.82s (± 1.43%) 0.82s (± 0.86%) +0.00s (+ 0.49%) 0.80s 0.83s
Check Time 17.93s (± 0.62%) 18.15s (± 0.83%) +0.22s (+ 1.24%) 17.71s 18.43s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.25s (± 0.56%) 21.47s (± 0.69%) +0.23s (+ 1.06%) 21.05s 21.75s
Angular - node (v8.9.0, x86)
Memory used 202,008k (± 0.03%) 202,092k (± 0.03%) +85k (+ 0.04%) 201,980k 202,215k
Parse Time 2.57s (± 0.84%) 2.56s (± 0.53%) -0.00s (- 0.16%) 2.53s 2.59s
Bind Time 1.02s (± 1.36%) 1.02s (± 1.17%) +0.00s (+ 0.30%) 1.00s 1.05s
Check Time 5.08s (± 0.46%) 5.08s (± 0.58%) +0.00s (+ 0.02%) 5.02s 5.17s
Emit Time 6.09s (± 0.75%) 6.08s (± 0.99%) -0.01s (- 0.15%) 5.94s 6.26s
Total Time 14.74s (± 0.39%) 14.74s (± 0.45%) -0.01s (- 0.05%) 14.60s 14.88s
Monaco - node (v8.9.0, x86)
Memory used 202,929k (± 0.02%) 202,913k (± 0.02%) -16k (- 0.01%) 202,811k 202,992k
Parse Time 1.99s (± 0.81%) 1.97s (± 0.52%) -0.02s (- 0.90%) 1.95s 1.99s
Bind Time 0.72s (± 0.95%) 0.71s (± 0.69%) -0.00s (- 0.28%) 0.70s 0.72s
Check Time 5.75s (± 0.74%) 5.74s (± 1.14%) -0.01s (- 0.09%) 5.50s 5.85s
Emit Time 2.76s (± 1.50%) 2.80s (± 2.61%) +0.04s (+ 1.56%) 2.70s 3.06s
Total Time 11.21s (± 0.40%) 11.23s (± 0.50%) +0.02s (+ 0.19%) 11.15s 11.37s
TFS - node (v8.9.0, x86)
Memory used 177,394k (± 0.03%) 177,366k (± 0.02%) -28k (- 0.02%) 177,284k 177,481k
Parse Time 1.62s (± 0.62%) 1.62s (± 1.32%) +0.01s (+ 0.31%) 1.59s 1.69s
Bind Time 0.65s (± 0.72%) 0.65s (± 1.06%) -0.01s (- 0.77%) 0.63s 0.66s
Check Time 4.86s (± 0.92%) 4.83s (± 0.58%) -0.03s (- 0.72%) 4.77s 4.89s
Emit Time 2.84s (± 1.40%) 2.82s (± 0.92%) -0.02s (- 0.78%) 2.75s 2.86s
Total Time 9.97s (± 0.83%) 9.91s (± 0.41%) -0.06s (- 0.57%) 9.84s 10.00s
material-ui - node (v8.9.0, x86)
Memory used 279,273k (± 0.01%) 279,260k (± 0.01%) -12k (- 0.00%) 279,179k 279,337k
Parse Time 2.55s (± 0.52%) 2.55s (± 0.57%) +0.00s (+ 0.16%) 2.53s 2.60s
Bind Time 0.75s (± 6.64%) 0.73s (± 6.73%) -0.02s (- 2.67%) 0.68s 0.86s
Check Time 16.41s (± 0.75%) 16.38s (± 0.65%) -0.03s (- 0.18%) 16.16s 16.68s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 19.71s (± 0.44%) 19.67s (± 0.61%) -0.04s (- 0.22%) 19.39s 19.94s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory1 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 41218 10
Baseline master 10

@ahejlsberg ahejlsberg merged commit f9dcd9e into master Oct 27, 2020
@ahejlsberg ahejlsberg deleted the fix39947 branch October 27, 2020 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weswigham weswigham weswigham approved these changes

@RyanCavanaugh RyanCavanaugh Awaiting requested review from RyanCavanaugh

Assignees

@ahejlsberg ahejlsberg

Labels
For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Assignability check not enough for class that is mutually recursive with an interface?
4 participants
@ahejlsberg @typescript-bot @rbuckton @weswigham