Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(psl-core): implement parse_configuration_multi_file to correct prisma/prisma's getConfig implementation #4825

Merged
merged 5 commits into from May 7, 2024

Conversation

jkomyno
Copy link
Contributor

@jkomyno jkomyno commented Apr 11, 2024

This PR closes https://github.com/prisma/team-orm/issues/1103.

This PR changes the implementation of prisma_fmt::get_config, to actually support multiple schema files without validating them entirely. This fixes a non-retrocompatible regression introduced in #4787 and discovered in prisma/prisma#23824 (specifically, in DbPull.test.ts).

/integration

@jkomyno jkomyno changed the title feat(psl-core): implement "parse_configuration_multi_file" to correct prisma/prisma's getConfig implementation feat(psl-core): implement parse_configuration_multi_file to correct prisma/prisma's getConfig implementation Apr 11, 2024
Comment on lines +148 to +153
let asts = Files::new(files, &mut diagnostics);

for (_, _, _, ast) in asts.iter() {
let out = validate_configuration(ast, &mut diagnostics, connectors);
configuration.extend(out);
}
Copy link
Contributor Author

@jkomyno jkomyno Apr 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I purposedly avoided creating a ParserDatabase instance here, as it performs additional validations that are not retro-compatible with parse_configuration, used in prisma_fmt::get_config.

Copy link
Contributor

github-actions bot commented Apr 11, 2024

WASM Query Engine file Size

Engine This PR Base branch Diff
Postgres 2.141MiB 2.141MiB 0.000B
Postgres (gzip) 842.650KiB 842.649KiB 1.000B
Mysql 2.110MiB 2.110MiB 0.000B
Mysql (gzip) 829.014KiB 829.013KiB 1.000B
Sqlite 2.003MiB 2.003MiB 0.000B
Sqlite (gzip) 789.713KiB 789.712KiB 1.000B

Copy link

codspeed-hq bot commented Apr 11, 2024

CodSpeed Performance Report

Merging #4825 will not alter performance

Comparing feat/get-config-multi-file (48f2d45) with main (133a47f)

Summary

✅ 11 untouched benchmarks

@jkomyno
Copy link
Contributor Author

jkomyno commented Apr 11, 2024

Using prisma-fmt-wasm, consider the following:

  • ./prisma/schema.prisma is a self-standing Prisma schema that is valid on its own

    datasource db {
      provider = "postgresql"
      url = env("DBURL")
    }
    
    model A {
      id String @id
      b_id String @unique
      b B @relation(fields: [b_id], references: [id])
    }
    
    model B {
      id String @id
      a  A?
    }
  • schema-1.prisma is only valid when paired with some other Prisma

    model B {
      id String @id
      a  A?
    }
  • schema-2.prisma is only valid when paired with some other Prisma

    datasource db {
      provider = "postgresql"
      url = env("DBURL")
    }
    
    model A {
      id String @id
      b_id String @unique
      b B @relation(fields: [b_id], references: [id])
    }
  • index.cjs shows how the get_config function works:

    const fs = require('node:fs/promises')
    const path = require('node:path')
    const { get_config } = require('./prisma_schema_build')
    
    async function main() {
      const schema = await fs.readFile(path.join(__dirname, 'prisma/schema.prisma'), 'utf-8')
      const schema1 = await fs.readFile(path.join(__dirname, 'prisma/schema-1.prisma'), 'utf-8')
      const schema2 = await fs.readFile(path.join(__dirname, 'prisma/schema-2.prisma'), 'utf-8')
    
      
    const config1 = await get_config(JSON.stringify({ prismaSchema: schema1, ignoreEnvVarErrors: true }))
    console.log('[get_config] schema1.prisma config:', JSON.parse(config1))
    
    const config2 = await get_config(JSON.stringify({ prismaSchema: schema2, ignoreEnvVarErrors: true }))
    console.log('[get_config] schema2.prisma config:', JSON.parse(config2))
    
    const config = await get_config(JSON.stringify({ prismaSchema: [['schema-1.prisma', schema1], ['schema-2.prisma', schema2]], ignoreEnvVarErrors: true }))
    console.log('[get_config] schema1.prisma + schema.2.prisma config:', JSON.parse(config))
    }
    
    main()

Before this PR, the output was:

${pwd}/prisma-engines/target/prisma-schema-wasm/src/prisma_schema_build.js:418
    const ret = new Error(getStringFromWasm0(arg0, arg1));
                ^

Error: {"message":"error: Type \"A\" is neither a built-in type, nor refers to another model, custom type, or enum.\n  -->  schema.prisma:3\n   | \n 2 |   id String @id\n 3 |   a  A?\n   | \n\nValidation Error Count: 1","error_code":"P1012"}
    
Node.js v20.9.0

With this PR, the output is:

[get_config] schema1.prisma config: { generators: [], datasources: [], warnings: [] }
[get_config] schema2.prisma config: {
  generators: [],
  datasources: [
    {
      name: 'db',
      provider: 'postgresql',
      activeProvider: 'postgresql',
      url: [Object],
      schemas: []
    }
  ],
  warnings: []
}
[get_config] schema1.prisma + schema.2.prisma config: {
  generators: [],
  datasources: [
    {
      name: 'db',
      provider: 'postgresql',
      activeProvider: 'postgresql',
      url: [Object],
      schemas: []
    }
  ],
  warnings: []
}

@jkomyno jkomyno self-assigned this Apr 11, 2024
@jkomyno jkomyno marked this pull request as ready for review April 11, 2024 13:54
@jkomyno jkomyno requested a review from a team as a code owner April 11, 2024 13:54
@jkomyno jkomyno requested review from Druue and SevInf and removed request for a team and Druue April 11, 2024 13:54
@jkomyno
Copy link
Contributor Author

jkomyno commented Apr 11, 2024

I confirm that applying these changes to prisma/prisma allows us to uncomment these failing tests.

Copy link
Contributor

github-actions bot commented Apr 11, 2024

✅ WASM query-engine performance won't change substantially (1.014x)

Full benchmark report
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.20.2 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - ~50K)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     370 ms/iter       (367 ms … 377 ms)    376 ms    377 ms    377 ms
Web Assembly: Latest       455 ms/iter       (453 ms … 458 ms)    457 ms    458 ms    458 ms
Web Assembly: Current      457 ms/iter       (455 ms … 462 ms)    461 ms    462 ms    462 ms
Node API: Current          202 ms/iter       (197 ms … 213 ms)    210 ms    213 ms    213 ms

summary for movies.findMany() (all - ~50K)
  Web Assembly: Current
   2.26x slower than Node API: Current
   1.24x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  14'724 µs/iter (14'481 µs … 16'626 µs) 14'651 µs 16'626 µs 16'626 µs
Web Assembly: Latest    18'184 µs/iter (17'831 µs … 19'936 µs) 18'229 µs 19'936 µs 19'936 µs
Web Assembly: Current   18'439 µs/iter (18'252 µs … 19'368 µs) 18'462 µs 19'368 µs 19'368 µs
Node API: Current        8'084 µs/iter   (7'895 µs … 8'640 µs)  8'162 µs  8'640 µs  8'640 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   2.28x slower than Node API: Current
   1.25x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   2'340 µs/iter   (2'202 µs … 3'848 µs)  2'334 µs  3'586 µs  3'848 µs
Web Assembly: Latest     2'908 µs/iter   (2'758 µs … 4'391 µs)  2'900 µs  3'815 µs  4'391 µs
Web Assembly: Current    2'891 µs/iter   (2'786 µs … 3'617 µs)  2'875 µs  3'532 µs  3'617 µs
Node API: Current        1'411 µs/iter   (1'327 µs … 1'713 µs)  1'425 µs  1'674 µs  1'713 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   2.05x slower than Node API: Current
   1.24x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     574 ms/iter       (567 ms … 592 ms)    580 ms    592 ms    592 ms
Web Assembly: Latest       775 ms/iter       (772 ms … 783 ms)    783 ms    783 ms    783 ms
Web Assembly: Current      796 ms/iter       (789 ms … 812 ms)    800 ms    812 ms    812 ms
Node API: Current          478 ms/iter       (454 ms … 502 ms)    485 ms    502 ms    502 ms

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.67x slower than Node API: Current
   1.39x slower than Web Assembly: Baseline
   1.03x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  78'858 µs/iter (78'623 µs … 79'232 µs) 79'154 µs 79'232 µs 79'232 µs
Web Assembly: Latest       108 ms/iter       (108 ms … 109 ms)    109 ms    109 ms    109 ms
Web Assembly: Current      112 ms/iter       (112 ms … 113 ms)    113 ms    113 ms    113 ms
Node API: Current       62'496 µs/iter (60'938 µs … 69'956 µs) 62'083 µs 69'956 µs 69'956 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.8x slower than Node API: Current
   1.42x slower than Web Assembly: Baseline
   1.04x slower than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'019 ms/iter   (1'012 ms … 1'042 ms)  1'021 ms  1'042 ms  1'042 ms
Web Assembly: Latest     1'293 ms/iter   (1'283 ms … 1'309 ms)  1'299 ms  1'309 ms  1'309 ms
Web Assembly: Current    1'315 ms/iter   (1'307 ms … 1'339 ms)  1'333 ms  1'339 ms  1'339 ms
Node API: Current          872 ms/iter       (852 ms … 905 ms)    887 ms    905 ms    905 ms

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.51x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     143 ms/iter       (143 ms … 144 ms)    144 ms    144 ms    144 ms
Web Assembly: Latest       182 ms/iter       (179 ms … 184 ms)    184 ms    184 ms    184 ms
Web Assembly: Current      183 ms/iter       (182 ms … 184 ms)    184 ms    184 ms    184 ms
Node API: Current          107 ms/iter       (106 ms … 107 ms)    107 ms    107 ms    107 ms

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.72x slower than Node API: Current
   1.28x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'049 µs/iter     (978 µs … 1'869 µs)  1'041 µs  1'713 µs  1'869 µs
Web Assembly: Latest     1'390 µs/iter   (1'301 µs … 2'142 µs)  1'374 µs  2'098 µs  2'142 µs
Web Assembly: Current    1'421 µs/iter   (1'309 µs … 2'393 µs)  1'382 µs  2'322 µs  2'393 µs
Node API: Current          766 µs/iter     (715 µs … 1'110 µs)    799 µs    901 µs  1'110 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.86x slower than Node API: Current
   1.36x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'027 µs/iter     (988 µs … 1'532 µs)  1'035 µs  1'352 µs  1'532 µs
Web Assembly: Latest     1'362 µs/iter   (1'304 µs … 1'852 µs)  1'371 µs  1'744 µs  1'852 µs
Web Assembly: Current    1'380 µs/iter   (1'319 µs … 1'941 µs)  1'387 µs  1'762 µs  1'941 µs
Node API: Current          766 µs/iter     (720 µs … 1'323 µs)    783 µs    959 µs  1'323 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.8x slower than Node API: Current
   1.34x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

After changes in 48f2d45

@Jolg42 Jolg42 added this to the 5.13.0 milestone Apr 11, 2024
}

/// Iterate all parsed files.
#[allow(clippy::should_implement_trait)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not implement trait instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of

`impl Trait` in associated types is unstable
see issue #63063 <https://github.com/rust-lang/rust/issues/63063> for more information
add `#![feature(impl_trait_in_assoc_type)]` to the crate attributes to enable

which appears when trying to write e.g.

impl std::iter::IntoIterator for Files {
    type Item = (FileId, String, schema_ast::SourceFile, ast::SchemaAst);
    type IntoIter = impl Iterator<Item = (FileId, String, schema_ast::SourceFile, ast::SchemaAst)>;

    fn into_iter(self) -> Self::IntoIter {
        self.0
            .into_iter()
            .enumerate()
            .map(|(idx, (path, contents, ast))| (FileId(idx as u32), path, contents, ast))
    }
}

Also, iter and into_iter were previously defined like this by Tom, and I'd assume the reason he didn't implement the traits is the same one as mine. Do you have any explicit recommendation to work around this, or is it ok to leave the clippy warning suppressors as committed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, i see. LGTM with no comments then

Copy link
Contributor

@SevInf SevInf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. with one caveat: Files should probably implement Iter and IntoInter like clippy suggests, rather than disabling the lint

@jkomyno jkomyno merged commit d880d75 into main May 7, 2024
206 of 207 checks passed
@jkomyno jkomyno deleted the feat/get-config-multi-file branch May 7, 2024 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants