Slow performance with large schemas

[mxstbr]

Describe the bug

When generating code from large GraphQL schemas, running graphql codegen takes > 25s, which is way too slow.

To Reproduce

Generate schema types and some queries from the official GitHub GraphQL API, which is very large:

overwrite: true
schema:
  - http://api.github.com/graphql:
      headers:
        Authorization: Bearer ${GITHUB_PERSONAL_ACCESS_TOKEN}
documents: "src/**/*.graphql"
generates:
  src/graphql-schema-types.generated.ts:
    plugins:
      - typescript
    config:
      immutableTypes: false
  src/:
    preset: near-operation-file
    presetConfig:
      baseTypesPath: "graphql-schema-types.generated.ts"
      extension: ".generated.tsx"
    plugins:
      - add: |
          // eslint-disable
          // ⚠️ DO NOT EDIT ⚠️
          // This file is automatically generated, run yarn run graphql:codegen to update
      - "typescript-operations"
      - "typescript-urql"
    config:
      immutableTypes: false
      withComponent: false
      withHooks: true
  web/src/graphql-types.generated.d.ts:
      plugins:
      - add: |
          // eslint-disable
          // ⚠️ DO NOT EDIT ⚠️
          // This file is automatically generated, run yarn run graphql:codegen to update
      - "typescript-graphql-files-modules"

You are going to need to get a GitHub personal access token from your settings and insert . that where it says ${GITHUB_PERSONAL_ACCESS_TOKEN}. A potential example query could be something like

{
  viewer {
    id
  }
}

Expected behavior

Graphql codegen's performance should be related to how many queries/mutations/subscriptions it needs to generate code for, not how large the input schema is.

[dotansimha]

@mxstbr I think the issue here is related to near-operation-file. Does it work better with a single file?

[mxstbr]

You are right! Same configuration except for removing near-operation-file takes 11s. I only generate 12 files with the preset, it seems weird that going from writing one file to writing 12 files would add ~1s per file?!

11s is still too slow though, if I replace the large schema with a much smaller schema it only takes 1s, so there's still a lot of seemingly unnecessary overhead there?

[dotansimha]

I think it's related to the filtering we are doing before executing the plugin (per each file). I'll check :)

[dotansimha]

btw, @mxstbr , this PR by @mvestergaard is merged: #2073 (and available as 1.3.1-alpha-991b9d78.75).
It should avoid building the schema per each output file, so I think it might improve the performance, can you please try it?

[mxstbr]

On alpha.82, running graphql-codegen in my codebase takes ~20s (with prettify: false) compared to ~24s on latest, so it's a bit faster but not much. 😕

For reference, the schema I am working with is almost 100k LoC and the file is 1.7MB in size.

[mvestergaard]

That's a little odd. For me there was a big improvement:

With 1.3.1:

$ gql-gen
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Parse configuration
  √ Generate outputs
Done in 10.11s.

With @canary

$ gql-gen
  √ Parse configuration
  √ Generate outputs
Done in 3.87s.

You upgraded all packages related to graphql-codegen to the alpha, i assume?

Maybe you can try running a profiler to see where the slowness comes from.
I did the following

• Install 0x https://www.npmjs.com/package/0x globally npm install -g 0x
• In your project run 0x .\node_modules\@graphql-codegen\cli\dist\commonjs\cli.js

It'll output a flamegraph html file to a folder in your project

[mvestergaard]

Oh and with prettify: false:

$ gql-gen
  √ Parse configuration
  √ Generate outputs
Done in 2.27s.

[mxstbr]

You are right, I must have done something wrong with my last upgrades—using the latest @canary, We are down to 7.27s with prettify: false! 🎉 🎉 🎉

Still not super fast but acceptable.

[dotansimha]

Closing this for now, since @mvestergaard changes it works much better :)