Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature: Infer schema type automatically. #11563

Merged
merged 42 commits into from Jun 5, 2022
Merged

Feature: Infer schema type automatically. #11563

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
2dbe288
Add TS utilities to infer schema type automatically.
mohammad0-0ahmad Mar 24, 2022
03e34db
Import inferschematype in index.d.ts file & refactor Schema class.
mohammad0-0ahmad Mar 24, 2022
ec6eb3d
Refactor model FN type & Model interface type.
mohammad0-0ahmad Mar 24, 2022
9321fd5
Make schema type obtainable from schema its constructor parameter "de…
mohammad0-0ahmad Mar 24, 2022
2145afa
Supporting custom typeKey to infer schema type.
mohammad0-0ahmad Mar 24, 2022
34ae74c
Add statics property in schema options to supporting auto typed model…
mohammad0-0ahmad Mar 24, 2022
967bd65
Add methods property in schema options to supporting auto typed docum…
mohammad0-0ahmad Mar 24, 2022
e75d4b6
Improve create FN type to show actual schema type instead of any
mohammad0-0ahmad Mar 24, 2022
ff2bf3e
Improve ApplyBasicQueryCasting type to improve find FN suggestions
mohammad0-0ahmad Mar 24, 2022
1bf940e
Refact create FN returned type
mohammad0-0ahmad Mar 24, 2022
4a5d712
Refact Model new FN & some related tests.
mohammad0-0ahmad Mar 26, 2022
295bad2
Fix typo from the previous PR
mohammad0-0ahmad Mar 26, 2022
cd215a7
Fix typo
mohammad0-0ahmad Mar 26, 2022
a7967d9
Improve Model.insertMany FN type
mohammad0-0ahmad Mar 27, 2022
e9d4491
Add query property in schema options to support auto typed query help…
mohammad0-0ahmad Mar 28, 2022
526ff6a
Rearrang Schema class generics
mohammad0-0ahmad Mar 28, 2022
411350b
Rearrange SchemaOptions generics.
mohammad0-0ahmad Mar 28, 2022
b942ce8
refactor query type in SchemaOptions & related tests.
mohammad0-0ahmad Mar 29, 2022
6bcb7a4
refactor statics & methods properties in SchemaOptions to get better …
mohammad0-0ahmad Mar 29, 2022
56af40b
Pass DocType to statics, methods & query props in SchemaOptions to ge…
mohammad0-0ahmad Mar 29, 2022
9cdec80
Refactor statics & methods props in SchemaOptions.
mohammad0-0ahmad Mar 29, 2022
b3e9668
Add ability to infer path of mixed array type.
mohammad0-0ahmad Apr 4, 2022
a42bbde
Refactor the previous PR.
mohammad0-0ahmad Apr 4, 2022
302de56
Refactor some function names & comment lines.
mohammad0-0ahmad Apr 5, 2022
96571b1
Refactor test in query.test.js file.
mohammad0-0ahmad Apr 5, 2022
897452b
Resolve Mixed type as any type.
mohammad0-0ahmad Apr 19, 2022
503c0d1
Refactor doc.populate FN to make it accept any Model type.
mohammad0-0ahmad Apr 22, 2022
aa6497d
Refactor Model new FN
mohammad0-0ahmad Apr 28, 2022
a3b32ac
Refactor model FN to optimize TS server performance.
mohammad0-0ahmad Apr 29, 2022
569d4eb
create MergeBOntoA And implement instead of UnpackedIntersection in s…
mohammad0-0ahmad Apr 29, 2022
0fe4928
Add some details about auto typed schema.
mohammad0-0ahmad Apr 29, 2022
f4f17d7
Add some details about auto typed statics functions
mohammad0-0ahmad Apr 29, 2022
6ef2f6c
Add some details about auto typed query helpers.
mohammad0-0ahmad Apr 29, 2022
6a77f09
Add details about auto typed statics, query-helpers and instance meth…
mohammad0-0ahmad Apr 30, 2022
f21dfbe
Refactor docs.
mohammad0-0ahmad Apr 30, 2022
7093195
Undo auto docs formatting changes.
mohammad0-0ahmad May 1, 2022
555db64
Refactor docs & some comments in tests files.
mohammad0-0ahmad May 3, 2022
0131b7e
fix typo
mohammad0-0ahmad May 17, 2022
47469d6
Cover decimal128
mohammad0-0ahmad Jun 2, 2022
e0d299f
patch autotyping to handle properly population and
Uzlopak Jun 2, 2022
e137a6d
Fix linting
mohammad0-0ahmad Jun 3, 2022
6547fe2
Merge branch '6.4' into auto-typed-schema
vkarpov15 Jun 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
178 changes: 170 additions & 8 deletions test/types/create.test.ts
View file
Open in desktop
@@ -1,25 +1,187 @@
import { Schema, model, Document, Types } from 'mongoose';
import { Schema, model, Types, CallbackError } from 'mongoose';
import { expectError, expectType } from 'tsd';

const schema: Schema = new Schema({ name: { type: 'String' } });

interface ITest extends Document {
interface ITest {
_id?: Types.ObjectId;
name?: string;
}

const Test = model<ITest>('Test', schema);

Test.create({ _id: new Types.ObjectId('0'.repeat(24)), name: 'test' }).then((doc: ITest) => console.log(doc.name));
Test.create({ _id: '000000000000000000000000', name: 'test' }).then(doc => {
expectType<Types.ObjectId>(doc._id);
expectType<string>(doc.name);
expectType<boolean>(doc.isNew);
});

Test.create([{ name: 'test' }], { validateBeforeSave: false }).then((docs: ITest[]) => console.log(docs[0].name));
Test.create({ _id: new Types.ObjectId('000000000000000000000000'), name: 'test' }).then((doc) => {
expectType<Types.ObjectId>(doc._id);
expectType<string>(doc.name);
expectType<boolean>(doc.isNew);
});

Test.create({ name: 'test' }, { name: 'test2' }).then((docs: ITest[]) => console.log(docs[0].name));
Test.create([{ name: 'test' }], { validateBeforeSave: false }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ name: 'test' }).then((docs: ITest[]) => console.log(docs[0].name));
Test.create({ name: 'test' }, { name: 'test2' }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<Types.ObjectId>(docs[1]._id);
expectType<string>(docs[1].name);
});

Test.create([{ name: 'test' }], { validateBeforeSave: true }).then((docs: ITest[]) => console.log(docs[0].name));
Test.create([{ name: 'test' }], { validateBeforeSave: true }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
});

(async() => {

Test.insertMany({ name: 'test' }, {}, (err, docs) => {
expectType<CallbackError>(err);
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ name: 'test' }, { lean: true }, (err, docs) => {
expectType<CallbackError>(err);
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectError(docs[0].isNew);
});

Test.insertMany({ name: 'test' }, (err, docs) => {
expectType<CallbackError>(err);
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ name: 'test' }, {}, (err, docs) => {
expectType<CallbackError>(err);
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany([{ name: 'test' }], { rawResult: true }, (err, result) => {
expectType<CallbackError>(err);
expectType<boolean>(result.acknowledged);
expectType<number>(result.insertedCount);
expectType<{[key: number]: Types.ObjectId;}>(result.insertedIds)
});

Test.insertMany([{ name: 'test' }], { rawResult: true }, (err, result) => {
expectType<CallbackError>(err);
expectType<boolean>(result.acknowledged);
expectType<number>(result.insertedCount);
expectType<{[key: number]: Types.ObjectId;}>(result.insertedIds)
});

Test.insertMany([{ name: 'test' }], { lean: true }, (err, docs) => {
expectType<CallbackError>(err);
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectError(docs[0].isNew);
});

Test.insertMany([{ name: 'test' }], (err, docs) => {
expectType<CallbackError>(err);
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ _id: '000000000000000000000000', name: 'test' }, (err, docs) => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ _id: new Types.ObjectId('000000000000000000000000')}, (err, docs) => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string | undefined>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ name: 'test' }, {}).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ name: 'test' }, { lean: true }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectError(docs[0].isNew);
});

Test.insertMany({ name: 'test' }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ name: 'test' }, {}).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany([{ name: 'test' }], { rawResult: true }).then(result => {
expectType<boolean>(result.acknowledged);
expectType<number>(result.insertedCount);
expectType<{[key: number]: Types.ObjectId;}>(result.insertedIds)
});

Test.insertMany([{ name: 'test' }], { rawResult: true }).then(result => {
expectType<boolean>(result.acknowledged);
expectType<number>(result.insertedCount);
expectType<{[key: number]: Types.ObjectId;}>(result.insertedIds)
});

Test.insertMany([{ name: 'test' }], { lean: true }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectError(docs[0].isNew);
});

Test.insertMany([{ name: 'test' }], { lean: false }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string | undefined>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany([{ name: 'test' }], { }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string | undefined>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany([{ name: 'test' }]).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ _id: '000000000000000000000000', name: 'test' }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

Test.insertMany({ _id: new Types.ObjectId('000000000000000000000000'), name: 'test' }).then(docs => {
expectType<Types.ObjectId>(docs[0]._id);
expectType<string>(docs[0].name);
expectType<boolean>(docs[0].isNew);
});

(async () => {
const [t1] = await Test.create([{ name: 'test' }]);
const [t2, t3, t4] = await Test.create({ name: 'test' }, { name: 'test' }, { name: 'test' });
(await Test.create([{ name: 'test' }]))[0];
Expand Down
19 changes: 11 additions & 8 deletions test/types/models.test.ts
View file
Open in desktop
@@ -1,4 +1,5 @@
import { Schema, Document, Model, Types, connection, model } from 'mongoose';
import { ObjectId } from 'bson';
import { Schema, Document, Model, connection, model, Types } from 'mongoose';
import { expectError, expectType } from 'tsd';
import { AutoTypedSchemaType, autoTypedSchema } from './schema.test';

Expand Down Expand Up @@ -41,9 +42,7 @@ function rawDocSyntax(): void {

const Test = connection.model<ITest, TestModel>('Test', TestSchema);

const bar = (SomeModel: Model<any, any, any>) => console.log(SomeModel);

bar(Test);
expectType<Model<ITest, {}, ITestMethods, {}>>(Test);

const doc = new Test({ foo: '42' });
console.log(doc.foo);
Expand Down Expand Up @@ -84,7 +83,7 @@ async function insertManyTest() {
});

const res = await Test.insertMany([{ foo: 'bar' }], { rawResult: true });
const ids: Types.ObjectId[] = Object.values(res.insertedIds);
expectType<ObjectId>(res.insertedIds[0]);
}

function schemaStaticsWithoutGenerics() {
Expand Down Expand Up @@ -140,13 +139,17 @@ async function gh10359() {
lastName: string;
}

async function foo<T extends Group>(model: Model<any>): Promise<T | null> {
const doc: T | null = await model.findOne({ groupId: 'test' }).lean().exec();
async function foo(model: Model<User, {}, {}, {}>) {
const doc = await model.findOne({ groupId: 'test' }).lean().exec();
expectType<string | undefined>(doc?.firstName);
expectType<string | undefined>(doc?.lastName);
expectType<Types.ObjectId | undefined>(doc?._id);
expectType<string | undefined>(doc?.groupId);
return doc;
}

const UserModel = model<User>('gh10359', new Schema({ firstName: String, lastName: String, groupId: String }));
const u: User | null = await foo<User>(UserModel);
foo(UserModel);
}

const ExpiresSchema = new Schema({
Expand Down
1 change: 1 addition & 0 deletions test/types/populate.test.ts
View file
Open in desktop
Expand Up @@ -72,6 +72,7 @@ const Story = model<IStory>('Story', storySchema);

await story.populate('author');
await story.populate({ path: 'fans' });
await story.populate({ path: 'fans', model: Person });
await story.populate(['author']);
await story.populate([{ path: 'fans' }]);
await story.populate(['author', { path: 'fans' }]);
Expand Down
4 changes: 2 additions & 2 deletions test/types/queries.test.ts
View file
Open in desktop
Expand Up @@ -163,10 +163,10 @@ function testGenericQuery(): void {

function eachAsync(): void {
Test.find().cursor().eachAsync((doc) => {
expectType<(ITest & { _id: any; })>(doc);
expectType<(ITest & { _id: Types.ObjectId; })>(doc);
});
Test.find().cursor().eachAsync((docs) => {
expectType<(ITest & { _id: any; })[]>(docs);
expectType<(ITest & { _id: Types.ObjectId; })[]>(docs);
}, { batchSize: 2 });
}

Expand Down
13 changes: 13 additions & 0 deletions test/types/utility.test.ts
View file
Open in desktop
@@ -0,0 +1,13 @@
import { MergeType } from 'mongoose';
import { expectType } from 'tsd';

type A = { a: string, c: number};
type B = { a: number, b: string };

expectType<string>({} as MergeType<B, A>["a"]);
expectType<string>({} as MergeType<B, A>["b"]);
expectType<number>({} as MergeType<B, A>["c"]);

expectType<number>({} as MergeType<A, B>["a"]);
expectType<string>({} as MergeType<A, B>["b"]);
expectType<number>({} as MergeType<A, B>["c"]);
1 change: 1 addition & 0 deletions tsconfig.json
View file
Open in desktop
@@ -1,5 +1,6 @@
{
"compilerOptions": {
"strict": true,
"strictNullChecks": true,
"paths": {
"mongoose" : ["./types/index.d.ts"]
Expand Down
12 changes: 10 additions & 2 deletions types/index.d.ts
View file
Open in desktop
Expand Up @@ -67,7 +67,7 @@ declare module 'mongoose' {
schema?: TSchema,
collection?: string,
options?: CompileModelOptions
): Model<InferSchemaType<TSchema>, ObtainSchemaGeneric<TSchema, 'TQueryHelpers'>, ObtainSchemaGeneric<TSchema, 'TInstanceMethods'>, {}, TSchema>;
): Model<InferSchemaType<TSchema>, ObtainSchemaGeneric<TSchema, 'TQueryHelpers'>, ObtainSchemaGeneric<TSchema, 'TInstanceMethods'>, {}, TSchema> & ObtainSchemaGeneric<TSchema, 'TStaticMethods'>;

export function model<T>(name: string, schema?: Schema<T, any, any> | Schema<T & Document, any, any>, collection?: string, options?: CompileModelOptions): Model<T>;
mohammad0-0ahmad marked this conversation as resolved.
Show resolved Hide resolved

Expand Down Expand Up @@ -101,7 +101,15 @@ declare module 'mongoose' {
[k: string]: any
}

export type Require_id<T> = T extends { _id?: any } ? (T & { _id: T['_id'] }) : (T & { _id: Types.ObjectId });
export type Require_id<T> = T extends { _id?: infer U }
? U extends any
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry that I'm not excellent in Typescript, but what does U extends any here mean?

As far as I know, everything extends any? so the contiditional clause (T & { _id: Types.ObjectId }) is always used, and the clause T & Required<{ _id: U }> is never used.

Thus, _id will be always infered with type ObjectId, even if I have defined another type such as String in the Schema definition.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example:

import {model, Schema} from "mongoose";
const schema_with_string_id = new Schema({_id: String, nickname: String})
const theModel = model('test', schema_with_string_id)
const obj = new theModel()
const theId = obj._id

The type of variable theId will be string | Types.ObjectId, rather than string.
Is this designed behaviour for some reason I don't know, or just a bug?

Copy link

@Starrah Starrah Jul 7, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mohammad0-0ahmad @Uzlopak Sorry for bothering you!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @Starrah !
Firstly, you are not bothering, and you are totally right.
I think this meant to be something like:

  export type Require_id<T> = T extends { _id?: infer U }
    ? IfEquals<U, any> extends true
      ? (T & { _id: Types.ObjectId })
      : T & Required<{ _id: U }>
    : T & { _id: Types.ObjectId };

Thanks for your review, I will make sure to refactor this.
What are your thoughts @Uzlopak about that?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I did this to check if T is not never.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I did this to check if T is not never.

First, does you mean you did that to check if {_id: } in T is not never, rather than check T?
Because according to my experiment,

import {Require_id, Types} from "mongoose";

type Required_id_old<T> = T extends { _id?: any } ? (T & { _id: T['_id'] }) : (T & { _id: Types.ObjectId });
type Required_id_new<T> = T extends { _id?: infer U }
    ? U extends any
        ? (T & { _id: Types.ObjectId })
        : T & Required<{ _id: U }>
    : T & { _id: Types.ObjectId };

type t1_old = Required_id_old<{_id: never}>
type t1_new = Required_id_new<{_id: never}>

type t2_old = Required_id_old<never>
type t2_new = Required_id_new<never>

Both t2_old and t2_new is never(which I think is expected behaviour), and for t1, t1_old is {_id: never} but t1_new is never(I have no idea which is your expected behaviour).

Then, if my assumption is true and you does mean check if {_id: } in T is not never, then maybe the two conditional clause should be swapped, as the following:

type Required_id_new<T> = T extends { _id?: infer U }
    ? U extends any
        ? T & Required<{ _id: U }>
        : (T & { _id: Types.ObjectId })
    : T & { _id: Types.ObjectId };

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, my Typescript version is 4.7.2. I surprisingly found that T extends any? true: false equals to never when T is never, because the following code:

type Test<T> = T extends any ? true : false
type A = Test<never>
const a: A = false

gives error output

Error:(3, 7) TS2322: Type 'boolean' is not assignable to type 'never'.

The behaviour is the same in Typescript 4.6.3.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I guess, I should have implemented appropriate typings tests.

Copy link

@Starrah Starrah Jul 8, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mohammad0-0ahmad @Uzlopak
Though with the discussion above, we (or just I) cannot still figure out what the codes here means, specifically, what does the U extends any mean.

However, I want to point out that the following test case written by me, got failed:

// index.test-d.ts
import {model, Schema} from "mongoose";
import {expectType} from "tsd";

const schema_with_string_id = new Schema({_id: String, nickname: String})
const theModel = model('test', schema_with_string_id)
const obj = new theModel()

expectType<string>(obj._id)

when testing it with tsd, gives the following error:

 index.test-d.ts:8:0
  ✖  8:0  Parameter type string is declared too wide for argument type string & ObjectId.

According to @mohammad0-0ahmad :

Hello @Starrah ! Firstly, you are not bothering, and you are totally right. I think this meant to be something like:

  export type Require_id<T> = T extends { _id?: infer U }
    ? IfEquals<U, any> extends true
      ? (T & { _id: Types.ObjectId })
      : T & Required<{ _id: U }>
    : T & { _id: Types.ObjectId };

Thanks for your review, I will make sure to refactor this. What are your thoughts @Uzlopak about that?

I think that the test case failure above indicates a bug. So, I decide to raise a new issue (#12070) to report and/or further discuss this bug.

? (T & { _id: Types.ObjectId })
: T & Required<{ _id: U }>
: T & { _id: Types.ObjectId };

export type RequireOnlyTypedId<T> = T extends { _id?: infer U; }
? Required<{ _id: U }>
: { _id: Types.ObjectId };

export type HydratedDocument<DocType, TMethodsAndOverrides = {}, TVirtuals = {}> = DocType extends Document ? Require_id<DocType> : (Document<unknown, any, DocType> & Require_id<DocType> & TVirtuals & TMethodsAndOverrides);

Expand Down