Improved type relation for recursive mapped types

[sandersn]

Recursive mapped types usually lead to the error "excess stack depth comparing types" because of a new type relation rule added in #19564 which says that "A source type T is related to a target type { [P in keyof T]: X } if T[P] is related to X".

Unfortunately, with self-recursive mapped types like

D<T> = { [P in keyof T]: D<T[P]> }

we get infinite recursion when trying to assign a type parameter T to D, as T[P] is compared to D<T[P]>, T[P][P] is compared to D<T[P][P]>, and so on.

We can avoid many of these infinite recursions by replacing occurrences of D in the template type with its type argument. This works because mapped types will completely cover the tree, so checking assignability of the top level implies that checking of lower level would succeed, even if there are infinitely many levels.

For example:

D<T> = { [P in keyof T]: D<T[P]> | undefined }
<T>(t: T, dt: D<T>) => { dt = t }

would previously check that T[P] is assignable to D<T[P]> | undefined. Now, after replacement, it checks that T[P] is assignable to T[P] | undefined.

This implementation suffers from 3 limitations:

1. I use aliasSymbol to detect whether a type reference is a self-recursive one. This only works when the mapped type is at the top level of a type alias.
2. Not all instances of D are replaced with T, just those in intersections and unions. I think this covers almost all uses.
3. This doesn't fix Out of memory checking recursive mapped type with two constraint type references #21048, which tries to assign an "off-by-one" partial-deep type to itself.

Mostly fixes #21592. One repro there has a type alias to a union, and a mapped type is a member of the union. But this can be split into two aliases:

type SafeAnyMap<T> = { [K in keyof T]?: SafeAny<T[K] };
type SafeAny<T> = SafeAnyMap<T> | boolean | string | symbol | number | null | undefined;

[sandersn]

the addition of replaceRecursiveAliasReference is the only real change in this section; the rest of the code was incorrectly indented before.

[sandersn]

@DanielRosenwasser I think you were interested in this too.

[ghost]

@sandersn Could you verify that this fixes DefinitelyTyped/DefinitelyTyped#25632?

[typescript-bot]

Thanks for your contribution. This PR has not been updated in a while and cannot be automatically merged at the time being. For housekeeping purposes we are closing stale PRs. If you'd still like to continue working on this PR, please leave a message and one of the maintainers can reopen it.

[alechemy]

I believe this is still an issue..

I'm using @types/lodash 4.14.122 and TypeScript 3.2.4, and I get the error:

ERROR in error TS2321: Excessive stack depth comparing types 'any' and 'ListIterateeCustom<T, boolean>'.