projectReferences does not build with the specified tsconfig

[unional]

Expected Behaviour

ts-loader should build the dependent projects using the tsconfig specified in the references field.

Actual Behaviour

It seems like it is just building the dependent projects by using the tsconfig.json.

Steps to Reproduce the Problem

e.g:

- packages
  - app-1
  - pkg-a

/packages/app-1/tsconfig.json:

{
  "references: [{
    "path": "../pkg-a/tsconfig.build.json"
  }]
}

/packages/pkg-a/tsconfig.build.json:

{
  /* some config */
}

/packages/pkg-a/tsconfig.json:

{
  "compilerOptions": {
    "verbatimModuleSyntax": true
  }
}

When building app-1, I got Unknown compiler option 'verbatimModuleSyntx.

So it seems like ts-loader is using the tsconfig.json even though the references point to pkg-a/tsconfig.build.json

I want to use verbatimModuleSyntax in tsconfig.json to tighten type checking and IDE support.
But ts-loader does not recognize it and with this issue, I don't have a way to exclude it.

Location of a Minimal Repository that Demonstrates the Issue.

No repro yet. This happens on some private project atm.