Model with discriminatorKey throws error in aggregation pipeline when performing a search

[georgehess]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Mongoose version

6.3.1

Node.js version

14

MongoDB server version

4.4.0

Description

If you are trying to use the aggregation pipeline to search a Model that has a discriminatorKey, this error is thrown because the discriminatorKey's match phase is automatically injected into the pipeline before the search's match phase.

$match with $text is only allowed as the first pipeline stage

Steps to Reproduce

import { Schema, model } from 'mongoose';

// setup a parent schema with a discriminatorKey
const ParentSchema: Schema = new Schema(
	{
		title: String,
		description: String,
	},
	{
		discriminatorKey: 'partition',
	}
);

// create search index on the title and description fields
ParentSchema.index({
	partition: 1,
	title: 'text',
	description: 'text',
});

// create child schema to inherit from parent
const ChildSchema: Schema = new Schema(
	{
		type: String,
	},
	{
		discriminatorKey: 'partition',
	}
);

// create models
const Parent = model('Parent', ParentSchema);
Parent.discriminator('Child', ChildSchema);
const Child = model('Child');

(async function () {

	// insert a child document to search for
	await Child.create({
		title: 'Hello World',
		description: 'This is a test',
		type: 'Test',
	});

	// search for child document using the aggregation pipeline
	await Child.aggregate([
		{
			$match: {
				partition: 'Child',
				$text: {
					$search: 'test',
				},
			},
		},
	]);

	// MongoServerError: $match with $text is only allowed as the first pipeline stage
})();

Clearly I put $text in the first pipeline stage but mongoose injects its own discriminatorKey match into the pipeline ahead of mine and produces this in mongo:

db.parents.aggregate([
	{ $match: { partition: 'Child' } },
	{ $match: { partition: 'Child', $text: { $search: 'test' } } },
]);

If I remove the partition param from my $match, then mongoose's $match is merged into mine and it works:

Child.aggregate([
	{
		$match: {
			$text: {
				$search: 'test',
			},
		},
	},
])

Produces this in mongo...

db.parents.aggregate([
	{ $match: { $text: { $search: 'test' }, partition: 'Child' } }
]);

Expected Behavior

If the first pipeline stage is a $match, Mongoose should always merge its discriminatorKey params into the first stage instead of inserting an additional $match above it. If the first stage already contains a discriminatorKey param then the auto-generated params from Mongoose should be dropped.

[IslandRhythms]

MongoServerError: text index required for $text query
    at Connection.onMessage (C:\Users\Daniel Diaz\Desktop\Work\debugging\Typescript\node_modules\mongodb\lib\cmap\connection.js:207:30)
    at MessageStream.<anonymous> (C:\Users\Daniel Diaz\Desktop\Work\debugging\Typescript\node_modules\mongodb\lib\cmap\connection.js:60:60)
    at MessageStream.emit (node:events:390:28)
    at processIncomingData (C:\Users\Daniel Diaz\Desktop\Work\debugging\Typescript\node_modules\mongodb\lib\cmap\message_stream.js:132:20)
    at MessageStream._write (C:\Users\Daniel Diaz\Desktop\Work\debugging\Typescript\node_modules\mongodb\lib\cmap\message_stream.js:33:9)
    at writeOrBuffer (node:internal/streams/writable:389:12)
    at _write (node:internal/streams/writable:330:10)
    at MessageStream.Writable.write (node:internal/streams/writable:334:10)
    at Socket.ondata (node:internal/streams/readable:754:22)
    at Socket.emit (node:events:390:28) {
  ok: 0,
  code: 27,
  codeName: 'IndexNotFound',
  [Symbol(errorLabels)]: Set(0) {}

You are not encountering this error when commenting out partition?

[georgehess]

No I am not because mongoose will automatically add the param {partition: "Child"} to the query when you use the Child model since its schema has "Child" set as its discriminator's value.

Parent.discriminator('Child', ChildSchema);

So any search I perform using the Child model satisfies the search index requirements even if I don't expressly add a partition field to the query myself.

This is the case for any query. A simple find() will automatically add the partition param to the query. i.e., Child.find({type: "Test"}) will generate db.parents.find({partition: "Child", type: "Test"}).

Which is great. I'm totally fine with that. The issue arises in the aggregation pipeline if I have specifically set the partition param myself. I realize it's not necessary to set it myself but I wouldn't expect it to insert an extra $match stage above my search stage (which necessitates being first). Instead, I would expect mongoose to drop its automatically generated partition filter if it sees that one already exists in the pipeline's first stage.

[IslandRhythms]

Forgot I was dropping the db on each run and that deletes indexes.

import { Schema, model, connect, connection } from 'mongoose';

// setup a parent schema with a discriminatorKey
const ParentSchema: Schema = new Schema(
	{
		title: String,
		description: String,
	},
	{
		discriminatorKey: 'partition',
	}
);

// create search index on the title and description fields
ParentSchema.index({
	partition: 1,
	title: 'text',
	description: 'text',
});

// create child schema to inherit from parent
const ChildSchema: Schema = new Schema(
	{
		type: String,
	},
	{
		discriminatorKey: 'partition',
	}
);

// create models
const Parent = model('Parent', ParentSchema);
Parent.discriminator('Child', ChildSchema);
const Child = model('Child');

(async function () {

    await connect('mongodb://localhost:27017');
    await connection.dropDatabase();

	await Parent.init();
	// insert a child document to search for
	await Child.create({
		title: 'Hello World',
		description: 'This is a test',
		type: 'Test',
	});

	// search for child document using the aggregation pipeline
	await Child.aggregate([
		{
			$match: {
				// partition: 'Child',
				$text: {
					$search: 'test',
				},
			},
		},
	]);
    console.log('done');
	// MongoServerError: $match with $text is only allowed as the first pipeline stage
})();

[vkarpov15]

The issue is that partition: 'Child' isn't necessary, Mongoose adds that automatically for you. That being said, Mongoose shouldn't change the aggregation pipeline if you correctly set the discriminator key in the first $match stage, so we'll add a fix for this in v6.6.6.