Manually populating doesn't work for embedded discriminators in embedded array

[saveman71]

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

Manually populated as described in the docs doesn't work for discriminators.

Repro script (sorry a bit long):

// Modules
const Promise = require('bluebird');
const mongoose = require('mongoose');
mongoose.Promise = Promise;
const Schema = mongoose.Schema;

console.log(mongoose.version);

// DB
let connect = mongoose.connect('mongodb://localhost:27018/test');

// Models
const AddonSchema = new Schema({
  title: String,
});
const Addon = mongoose.model('Addon', AddonSchema);
const AddonItemSchema = new Schema({
  addon: { type: Schema.Types.ObjectId, ref: 'Addon' },
  quantity: Number,
});

const ProductSchema = new Schema({
  title: String,
});
const Product = mongoose.model('Product', ProductSchema);
const ProductItemSchema = new Schema({
  product: { type: Schema.Types.ObjectId, ref: 'Product' },
});

const OrderItemSchema = new mongoose.Schema({
  quantity: Number,
}, {discriminatorKey: '__t'});
const OrderItem = mongoose.model('OrderItem', OrderItemSchema);


const OrderSchema = new Schema({
  product: { type: Schema.Types.ObjectId, ref: 'Product' },
  noDiscriminator: {
    type: [{
      product: { type: Schema.Types.ObjectId, ref: 'Product' },
    }],
    default: [],
  },
  discriminatorNoArray: {
    type: OrderItemSchema,
  },
  items: {
    type: [OrderItemSchema],
    default: [],
  },
});

const docArray = OrderSchema.path('items');

const ProductItem = docArray
  .discriminator('ProductItem', ProductItemSchema);
const AddonItem = docArray
  .discriminator('AddonItem', AddonItemSchema);

OrderSchema.path('discriminatorNoArray').discriminator('ProductItem', ProductItemSchema);
OrderSchema.path('discriminatorNoArray').discriminator('AddonItem', AddonItemSchema);

const Order = mongoose.model('Order', OrderSchema);

// Create test data
connect.then(async () => {
  mongoose.connection.db.dropDatabase();

  const addon = await Addon({title: 'Addon title'}).save();
  const product = await Product({title: 'Product title'}).save();

  // Create order with embedded items
  const unsaved = new Order({
    product,
    noDiscriminator: [
      {
        product,
      }
    ],
    discriminatorNoArray: {
      product,
      quantity: 42,
      __t: 'ProductItem',
    },
    items: [
      {
        addon,
        quantity: 42,
        __t: 'AddonItem',
      },
      {
        product,
        quantity: 42,
        __t: 'ProductItem',
      }
    ]
  });

  // Doesn't show manually populated in items
  console.log(unsaved);

  await unsaved.save();

  // Find order, populate addon of items
  const order = await Order.findOne({}).exec();

  order.product = product;
  order.noDiscriminator[0].product = product;
  order.discriminatorNoArray.product = product;
  order.items[0].addon = addon;
  order.items[0].addon = addon;

  // Doesn't show manually populated in items
  console.log(order);
});

What are the versions of Node.js, Mongoose and MongoDB you are using?

Node.js v8.11.0
Mongoose v5.7.6

[saveman71]

I've tried to debug this a bit:

here's my unit test (i added an other working product in the root of Order just to tests things):

      it.only('with discriminators in embedded arrays (gh-8273)', function(done) {
        const ProductSchema = new Schema({
          title: String,
        });
        const Product = mongoose.model('gh8273_Product', ProductSchema);
        const ProductItemSchema = new Schema({
          product: { type: Schema.Types.ObjectId, ref: 'gh8273_Product' },
        });

        const OrderItemSchema = new Schema({}, {discriminatorKey: '__t'});

        const OrderSchema = new Schema({
          product: { type: Schema.Types.ObjectId, ref: 'gh8273_Product' },
          items: [OrderItemSchema],
        });

        OrderSchema.path('items').discriminator('ProductItem', ProductItemSchema);
        const Order = mongoose.model('Order', OrderSchema);

        const product = new Product({title: 'Product title'});

        const order = new Order({
          product,
          items: [
            {
              product,
              __t: 'ProductItem',
            },
          ]
        });
        console.log(order);
        assert.ok(order.items[0].product.title);

        done();
      });

I've isolated it to the fact that during the setting process, the subdocument is casted to the object id ref, which is supposed to check if it's populated to, i guess, either allow the document to pass through or to just keep the id.

Cast calls SchemaType._isRef, which calls populated() on the owner document, here with the path of items.product which seems (?) wrong, as this should be a more "advanced" check, as i understand it as it's an array.

The check fails and this set operation is casted to the not populated version.

[saveman71]

From what I got the populated state is stored in the "local" scope of the embedded item here

mongoose/lib/document.js

Lines 1132 to 1135 in 9549015

	if (refMatches && val instanceof Document) {
	this.populated(path, val._id, { [populateModelSymbol]: val.constructor });
	didPopulate = true;
	}

Then here

mongoose/lib/schematype.js

Lines 1266 to 1268 in 9549015

	const path = doc.$__fullPath(self.path);
	const owner = doc.ownerDocument ? doc.ownerDocument() : doc;
	ref = owner.populated(path);

The population is checked against the full path (items.product) and against the owner document that doesn't know about that population.

[saveman71]

@vkarpov15 anything I can do to help from there? I feel i'm lacking a deep understanding of how things work to submit a patch

[saveman71]

Also, this seems very similar to #3575

[vkarpov15]

@saveman71 thanks for the test script, that was very helpful. You were right about the item.populated and SchemaType._isRef() bit. I opened up a new issue #8298 to track refactoring that.

[vkarpov15]

In the meantime, the fix for this issue will be in 5.7.8

[saveman71]

Thanks for the fix! 🎉

Just a small question, regarding dfde779. Wouldn't

mongoose/lib/document.js

Lines 1136 to 1139 in dfde779

	if (refMatches && val instanceof Document) {
	this.populated(path, val._id, { [populateModelSymbol]: val.constructor });
	didPopulate = true;
	}

still be run anyway? Isn't there a side effect?

[vkarpov15]

@saveman71 yes it will be run, but this.populated(path, val._id) will set populated() on the nested subdoc, not the top-level doc.