Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add cockroachDB 23.2 #4719

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

test: add cockroachDB 23.2 #4719

wants to merge 4 commits into from

Conversation

Jolg42
Copy link
Member

@Jolg42 Jolg42 commented Feb 12, 2024

@Jolg42 Jolg42 self-assigned this Feb 12, 2024
Copy link
Contributor

github-actions bot commented Feb 12, 2024

WASM Size

Engine This PR Base branch Diff
Postgres 2.045MiB 2.044MiB 1.436KiB
Postgres (gzip) 809.480KiB 809.367KiB 116.000B
Mysql 2.027MiB 2.025MiB 2.384KiB
Mysql (gzip) 800.664KiB 800.418KiB 251.000B
Sqlite 1.988MiB 1.987MiB 1.439KiB
Sqlite (gzip) 787.858KiB 787.783KiB 77.000B

Copy link
Contributor

github-actions bot commented Feb 12, 2024

❌ WASM query-engine performance will worsen by 1.87%

Full benchmark report
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.19.0 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - 25000)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   302.7 ms/iter (301.84 ms … 306.37 ms) 303.26 ms 306.37 ms 306.37 ms
Web Assembly: Latest    385.03 ms/iter (383.66 ms … 388.13 ms)  385.9 ms 388.13 ms 388.13 ms
Web Assembly: Current   384.96 ms/iter (383.58 ms … 388.45 ms) 385.32 ms 388.45 ms 388.45 ms
Node API: Current       226.02 ms/iter (216.66 ms … 234.96 ms) 234.35 ms 234.96 ms 234.96 ms

summary for movies.findMany() (all - 25000)
  Web Assembly: Current
   1.7x slower than Node API: Current
   1.27x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    12.7 ms/iter    (12.15 ms … 20.7 ms)  12.31 ms   20.7 ms   20.7 ms
Web Assembly: Latest     15.67 ms/iter    (15.5 ms … 16.94 ms)  15.64 ms  16.94 ms  16.94 ms
Web Assembly: Current    16.11 ms/iter   (15.55 ms … 19.68 ms)   16.2 ms  19.68 ms  19.68 ms
Node API: Current        8,683 µs/iter   (8,495 µs … 9,172 µs)  8,718 µs  9,172 µs  9,172 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   1.86x slower than Node API: Current
   1.27x slower than Web Assembly: Baseline
   1.03x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,940 µs/iter   (1,837 µs … 2,953 µs)  1,941 µs  2,621 µs  2,953 µs
Web Assembly: Latest     2,436 µs/iter   (2,368 µs … 2,899 µs)  2,441 µs  2,718 µs  2,899 µs
Web Assembly: Current    2,496 µs/iter   (2,387 µs … 3,872 µs)  2,487 µs  3,616 µs  3,872 µs
Node API: Current        1,513 µs/iter   (1,387 µs … 2,357 µs)  1,500 µs  2,296 µs  2,357 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   1.65x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.23 ms/iter   (12.13 ms … 12.36 ms)  12.26 ms  12.36 ms  12.36 ms
Web Assembly: Latest     15.81 ms/iter   (15.43 ms … 20.39 ms)  15.66 ms  20.39 ms  20.39 ms
Web Assembly: Current    15.61 ms/iter   (15.48 ms … 15.72 ms)  15.66 ms  15.72 ms  15.72 ms
Node API: Current        8,680 µs/iter   (8,487 µs … 9,077 µs)  8,726 µs  9,077 µs  9,077 µs

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.8x slower than Node API: Current
   1.28x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,933 µs/iter   (1,842 µs … 2,894 µs)  1,919 µs  2,741 µs  2,894 µs
Web Assembly: Latest     2,466 µs/iter   (2,372 µs … 3,344 µs)  2,460 µs  3,018 µs  3,344 µs
Web Assembly: Current    2,479 µs/iter   (2,385 µs … 3,259 µs)  2,480 µs  2,962 µs  3,259 µs
Node API: Current        1,476 µs/iter   (1,393 µs … 1,967 µs)  1,502 µs  1,666 µs  1,967 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.68x slower than Node API: Current
   1.28x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.47 ms/iter      (12.12 ms … 16 ms)  12.29 ms     16 ms     16 ms
Web Assembly: Latest      15.5 ms/iter   (15.41 ms … 15.59 ms)  15.54 ms  15.59 ms  15.59 ms
Web Assembly: Current    16.05 ms/iter   (15.48 ms … 23.38 ms)  15.62 ms  23.38 ms  23.38 ms
Node API: Current        8,707 µs/iter   (8,484 µs … 9,189 µs)  8,741 µs  9,189 µs  9,189 µs

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.84x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1.04x slower than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1,873 µs/iter   (1,817 µs … 2,286 µs)  1,878 µs  2,229 µs  2,286 µs
Web Assembly: Latest     2,416 µs/iter   (2,350 µs … 2,819 µs)  2,418 µs  2,769 µs  2,819 µs
Web Assembly: Current    2,436 µs/iter   (2,368 µs … 2,946 µs)  2,432 µs  2,820 µs  2,946 µs
Node API: Current        1,526 µs/iter   (1,418 µs … 2,403 µs)  1,514 µs  2,191 µs  2,403 µs

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.6x slower than Node API: Current
   1.3x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  917.33 µs/iter  (857.56 µs … 1,505 µs) 913.48 µs  1,385 µs  1,505 µs
Web Assembly: Latest     1,202 µs/iter   (1,140 µs … 1,949 µs)  1,202 µs  1,757 µs  1,949 µs
Web Assembly: Current    1,247 µs/iter   (1,146 µs … 2,109 µs)  1,218 µs  2,068 µs  2,109 µs
Node API: Current       819.29 µs/iter (769.87 µs … 979.52 µs) 831.36 µs 931.13 µs 979.52 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.52x slower than Node API: Current
   1.36x slower than Web Assembly: Baseline
   1.04x slower than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  921.68 µs/iter  (884.96 µs … 1,318 µs) 926.45 µs  1,165 µs  1,318 µs
Web Assembly: Latest     1,203 µs/iter   (1,144 µs … 2,048 µs)  1,207 µs  1,839 µs  2,048 µs
Web Assembly: Current    1,237 µs/iter   (1,157 µs … 2,082 µs)  1,224 µs  2,036 µs  2,082 µs
Node API: Current       815.88 µs/iter  (743.01 µs … 5,041 µs) 828.69 µs 927.05 µs  5,041 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.52x slower than Node API: Current
   1.34x slower than Web Assembly: Baseline
   1.03x slower than Web Assembly: Latest

After changes in 767647b

Copy link

codspeed-hq bot commented Feb 12, 2024

CodSpeed Performance Report

Merging #4719 will not alter performance

Comparing joel/crdb-23-2 (767647b) with main (45a0eb4)

Summary

✅ 11 untouched benchmarks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant