feat: enable field exclusion for the json protocol #4807

Weakky · 2024-04-03T14:13:07Z

Overview

closes https://github.com/prisma/team-orm/issues/1079

Note: Field exclusion cannot be done for the GraphQL protocol.

github-actions · 2024-04-03T14:17:10Z

WASM Query Engine file Size

Engine	This PR	Base branch	Diff
Postgres	2.127MiB	2.126MiB	1.528KiB
Postgres (gzip)	837.089KiB	836.254KiB	855.000B
Mysql	2.097MiB	2.096MiB	1.546KiB
Mysql (gzip)	824.898KiB	824.265KiB	648.000B
Sqlite	1.992MiB	1.990MiB	1.538KiB
Sqlite (gzip)	785.834KiB	784.749KiB	1.085KiB

codspeed-hq · 2024-04-03T14:22:38Z

CodSpeed Performance Report

Merging #4807 will degrade performances by 5.29%

_{Comparing feat/exclude-selection (8eed76c) with main (6d35870)}

Summary

❌ 1 regressions
✅ 10 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	`main`	`feat/exclude-selection`	Change
❌	`large_read`	7.7 ms	8.1 ms	-5.29%

github-actions · 2024-04-03T14:39:54Z

✅ WASM query-engine performance won't change substantially (1.001x)

Full benchmark report

DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.20.1 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - ~50K)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     373 ms/iter       (370 ms … 377 ms)    376 ms    377 ms    377 ms
Web Assembly: Latest       455 ms/iter       (452 ms … 461 ms)    456 ms    461 ms    461 ms
Web Assembly: Current      454 ms/iter       (452 ms … 460 ms)    456 ms    460 ms    460 ms
Node API: Current          196 ms/iter       (194 ms … 197 ms)    196 ms    197 ms    197 ms

summary for movies.findMany() (all - ~50K)
  Web Assembly: Current
   2.32x slower than Node API: Current
   1.22x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  14'973 µs/iter (14'765 µs … 16'317 µs) 14'925 µs 16'317 µs 16'317 µs
Web Assembly: Latest    18'911 µs/iter (18'153 µs … 24'556 µs) 18'561 µs 24'556 µs 24'556 µs
Web Assembly: Current   18'533 µs/iter (18'237 µs … 21'025 µs) 18'507 µs 21'025 µs 21'025 µs
Node API: Current        8'143 µs/iter   (7'901 µs … 8'422 µs)  8'239 µs  8'422 µs  8'422 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   2.28x slower than Node API: Current
   1.24x slower than Web Assembly: Baseline
   1.02x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   2'307 µs/iter   (2'197 µs … 3'283 µs)  2'283 µs  3'197 µs  3'283 µs
Web Assembly: Latest     2'858 µs/iter   (2'743 µs … 3'634 µs)  2'843 µs  3'566 µs  3'634 µs
Web Assembly: Current    2'880 µs/iter   (2'766 µs … 3'777 µs)  2'862 µs  3'587 µs  3'777 µs
Node API: Current        1'415 µs/iter   (1'312 µs … 1'776 µs)  1'410 µs  1'759 µs  1'776 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   2.03x slower than Node API: Current
   1.25x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     595 ms/iter       (588 ms … 612 ms)    601 ms    612 ms    612 ms
Web Assembly: Latest       780 ms/iter       (775 ms … 798 ms)    787 ms    798 ms    798 ms
Web Assembly: Current      790 ms/iter       (782 ms … 806 ms)    804 ms    806 ms    806 ms
Node API: Current          490 ms/iter       (479 ms … 505 ms)    497 ms    505 ms    505 ms

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.61x slower than Node API: Current
   1.33x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  84'423 µs/iter (84'012 µs … 84'772 µs) 84'738 µs 84'772 µs 84'772 µs
Web Assembly: Latest       109 ms/iter       (109 ms … 110 ms)    110 ms    110 ms    110 ms
Web Assembly: Current      111 ms/iter       (110 ms … 112 ms)    111 ms    112 ms    112 ms
Node API: Current       62'281 µs/iter (61'382 µs … 62'895 µs) 62'858 µs 62'895 µs 62'895 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.78x slower than Node API: Current
   1.31x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'069 ms/iter   (1'063 ms … 1'085 ms)  1'072 ms  1'085 ms  1'085 ms
Web Assembly: Latest     1'302 ms/iter   (1'290 ms … 1'317 ms)  1'316 ms  1'317 ms  1'317 ms
Web Assembly: Current    1'306 ms/iter   (1'298 ms … 1'321 ms)  1'314 ms  1'321 ms  1'321 ms
Node API: Current          889 ms/iter       (864 ms … 926 ms)    916 ms    926 ms    926 ms

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.47x slower than Node API: Current
   1.22x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     151 ms/iter       (150 ms … 156 ms)    153 ms    156 ms    156 ms
Web Assembly: Latest       182 ms/iter       (181 ms … 183 ms)    183 ms    183 ms    183 ms
Web Assembly: Current      182 ms/iter       (182 ms … 183 ms)    183 ms    183 ms    183 ms
Node API: Current          108 ms/iter       (106 ms … 111 ms)    109 ms    111 ms    111 ms

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.69x slower than Node API: Current
   1.2x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'042 µs/iter     (981 µs … 1'813 µs)  1'037 µs  1'706 µs  1'813 µs
Web Assembly: Latest     1'379 µs/iter   (1'315 µs … 2'182 µs)  1'374 µs  1'941 µs  2'182 µs
Web Assembly: Current    1'380 µs/iter   (1'311 µs … 2'173 µs)  1'381 µs  1'801 µs  2'173 µs
Node API: Current          781 µs/iter     (733 µs … 1'164 µs)    795 µs    889 µs  1'164 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.77x slower than Node API: Current
   1.33x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'053 µs/iter     (986 µs … 1'989 µs)  1'038 µs  1'857 µs  1'989 µs
Web Assembly: Latest     1'375 µs/iter   (1'308 µs … 2'020 µs)  1'377 µs  1'835 µs  2'020 µs
Web Assembly: Current    1'377 µs/iter   (1'315 µs … 2'271 µs)  1'378 µs  1'788 µs  2'271 µs
Node API: Current          791 µs/iter     (748 µs … 1'124 µs)    803 µs    858 µs  1'124 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.74x slower than Node API: Current
   1.31x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

After changes in 8eed76c

Counterpart to prisma/prisma-engines#4807 Can be used standalone or combined with `include`, allows to exclude the fields that normally would be included by default. Available in all methods that return actual database records. It is not useable together with `select` and attempt to do so would cause type check and validation error. When using together with result extensions, excluded dependency of a computed field will be queried from a DB, but will not be returned to the end user, unless computed field is exlucded as well (see "exclude with extensions" tests in this PR). This behaviour is equivalent to what we do if depenency of a computed field is not mentioned in explicit `select`. TODO: - [ ] preview feature - [ ] validation of non-existing fields in exclude Close prisma/team-orm#1080 Close #5042

Druue · 2024-04-10T12:47:34Z

Field exclusion cannot be done for the GraphQL protocol

Which should only be used internally anyway, right?

Weakky · 2024-04-10T12:48:48Z

Field exclusion cannot be done for the GraphQL protocol

Which should only be used internally anyway, right?

Yes, correct. External clients can technically still use GraphQL, but our client only uses the JSON protocol.

query-engine/core/src/query_document/parser.rs

query-engine/core-tests/tests/query_validation_tests/unkown_selection_field.expected.json

...ne/core-tests/tests/query_validation_tests/selection_contains_no_truthy_values.expected.json

Counterpart to prisma/prisma-engines#4807 Can be used standalone or combined with `include`, allows to exclude the fields that normally would be included by default. Available in all methods that return actual database records. It is not useable together with `select` and attempt to do so would cause type check and validation error. When using together with result extensions, excluded dependency of a computed field will be queried from a DB, but will not be returned to the end user, unless computed field is exlucded as well (see "exclude with extensions" tests in this PR). This behaviour is equivalent to what we do if depenency of a computed field is not mentioned in explicit `select`. TODO: - [ ] preview feature - [ ] validation of non-existing fields in exclude Close prisma/team-orm#1080 Close #5042

Counterpart to prisma/prisma-engines#4807 Can be used standalone or combined with `include`, allows to exclude the fields that normally would be included by default. Available in all methods that return actual database records. It is not useable together with `select` and attempt to do so would cause type check and validation error. When using together with result extensions, excluded dependency of a computed field will be queried from a DB, but will not be returned to the end user, unless computed field is exlucded as well (see "exclude with extensions" tests in this PR). This behaviour is equivalent to what we do if depenency of a computed field is not mentioned in explicit `select`. Close prisma/team-orm#1080

Weakky added this to the 5.13.0 milestone Apr 3, 2024

SevInf mentioned this pull request Apr 8, 2024

feat(client): Add omit option to the queries prisma/prisma#23770

Merged

2 tasks

Weakky force-pushed the feat/exclude-selection branch from 7e648d2 to 9ed8975 Compare April 9, 2024 13:39

Weakky marked this pull request as ready for review April 10, 2024 08:56

Weakky requested a review from a team as a code owner April 10, 2024 08:56

Weakky requested review from Druue and removed request for a team April 10, 2024 08:56

Druue reviewed Apr 10, 2024

View reviewed changes

Druue approved these changes Apr 10, 2024

View reviewed changes

Weakky added 12 commits April 12, 2024 12:38

feat: enable field exclusion for the json protocol

585e6b5

fix empty selection validation

f3dc40b

remove empty selection set check - parser validates it already

faf8a79

add parser validation for invalid field exclusion

ef89ff8

add excludeApi preview feature

2aa6df5

update unit tests

14dc90a

allow wildcard with explicit selection

7be7dbc

prettify debug snapshots

9c450bb

remove UnknownExcludedField and reuse UnknownSelectionField error

f10f03a

fix test name

10ee862

rename excludeApi to omitApi

60022fc

fix unit test

8eed76c

Weakky force-pushed the feat/exclude-selection branch from 46c1653 to 8eed76c Compare April 12, 2024 10:38

Weakky merged commit b339b9d into main Apr 12, 2024
208 of 209 checks passed

Weakky deleted the feat/exclude-selection branch April 12, 2024 12:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: enable field exclusion for the json protocol #4807

feat: enable field exclusion for the json protocol #4807

Weakky commented Apr 3, 2024 •

edited

github-actions bot commented Apr 3, 2024 •

edited

codspeed-hq bot commented Apr 3, 2024 •

edited

github-actions bot commented Apr 3, 2024 •

edited

Druue commented Apr 10, 2024 •

edited

Weakky commented Apr 10, 2024 •

edited

feat: enable field exclusion for the json protocol #4807

feat: enable field exclusion for the json protocol #4807

Conversation

Weakky commented Apr 3, 2024 • edited

Overview

github-actions bot commented Apr 3, 2024 • edited

WASM Query Engine file Size

codspeed-hq bot commented Apr 3, 2024 • edited

CodSpeed Performance Report

Merging #4807 will degrade performances by 5.29%

Summary

Benchmarks breakdown

github-actions bot commented Apr 3, 2024 • edited

✅ WASM query-engine performance won't change substantially (1.001x)

Druue commented Apr 10, 2024 • edited

Weakky commented Apr 10, 2024 • edited

Weakky commented Apr 3, 2024 •

edited

github-actions bot commented Apr 3, 2024 •

edited

codspeed-hq bot commented Apr 3, 2024 •

edited

github-actions bot commented Apr 3, 2024 •

edited

Druue commented Apr 10, 2024 •

edited

Weakky commented Apr 10, 2024 •

edited