transform option for lean()

[bhuynhdev]

Do you want to request a feature or report a bug?
Honestly, I don't know if this is a bug or a feature request

What is the current behavior?

When creating schema, I am using the toObject and toJSON options to strip fields like _id and __v before presenting it to the end-users. In some GET endpoints, I want to also use lean() to increase the performance, as I am only returning the data, not actively modifying it (except for toJSON transformation). So far, I have had no success in making these features work together, and I am honestly not sure if these can work together or not, as the toObject doc does not talk about lean and the lean doc also does not really mention toObject and toJSON

If the current behavior is a bug, please provide the steps to reproduce.

My code:

const schema = new mongoose.Schema({
  name: String
}, {
  toJSON: {transform: (doc, ret) => {
    ret.id= ret._id;
    delete ret._id;
    delete ret.__v;
    return ret;
  }}
})

const model = mongoose.model("Model", schema);

async function test() {
  await model.insertMany([{name: "First"}])
  const one = await model.findOne({name: "First"}).exec();
  const oneLean = await model.findOne({name: "First"}).lean().exec();
  
  console.log(JSON.stringify(one)) // Has field `name` and `id`
  console.log(JSON.stringify(oneLean)) // Has field `name`, `_id`, and `__v`
  console.log(JSON.stringify(one).length === JSON.stringify(oneLean).length) // false
}

What is the expected behavior?

I just want to ask how to make these feature work together, or is that an impossible thing? Are there any packages/plugins to help with my goal ?

What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.

• Node version: v14.17.2
• Mongoose: 5.13.2
• MongoDB Atlas
• Express: 4.16.1

[IslandRhythms]

const mongoose = require('mongoose');
const {Schema} = mongoose;


const schema = new mongoose.Schema({
    name: String
  }, {
    toJSON: {transform: (doc, ret) => {
      ret.id= ret._id;
      delete ret._id;
      delete ret.__v;
      return ret;
    }}
  })
  
  const model = mongoose.model("Model", schema);
  
  async function test() {
    await mongoose.connect("mongodb://localhost:27017", {
        useCreateIndex: true,
        useNewUrlParser: true,
        useUnifiedTopology: true,
        useFindAndModify: false,
      });
    await mongoose.connection.dropDatabase();
    // await model.insertMany([{name: "First"}])
    await model.create({name: 'First'});
    const one = await model.findOne({name: "First"}).exec();
    const oneLean = await model.findOne({name: "First"}).lean().exec();
    const func = await model.findOne({name: "First"}).exec(function(err, entry) {
        console.log('My Line', entry.toJSON());
    })
    console.log(JSON.stringify(one)) // Has field `name` and `id`
    console.log(JSON.stringify(oneLean)) // Has field `name`, `_id`, and `__v`
    console.log(JSON.stringify(one).length === JSON.stringify(oneLean).length) // false
  }

  test();

Read this for more clarification
https://stackoverflow.com/a/21500522

[vkarpov15]

@baohuynhlam we don't currently have support for transforms with lean(). We should add support for that.

Currently, I'd recommend structuring this as a post hook that applies your transform function if lean is set:

schema.post(['find', 'findOne', 'findOneAndUpdate'], function(res) {
  if (!this.mongooseOptions().lean) {
    return;
  }
  if (Array.isArray(res)) {
    res.forEach(transformDoc);
    return;
  }
  transformDoc(res);
});

function transformDoc(doc) {
  doc.id = doc._id;
  delete doc._id;
  delete doc.__v;
}

[mjfwebb]

@baohuynhlam we don't currently have support for transforms with lean(). We should add support for that.

Currently, I'd recommend structuring this as a post hook that applies your transform function if lean is set:

schema.post(['find', 'findOne', 'findOneAndUpdate'], function(res) {
  if (!this.mongooseOptions().lean) {
    return;
  }
  if (Array.isArray(res)) {
    res.forEach(transformDoc);
    return;
  }
  transformDoc(res);
});

function transformDoc(doc) {
  doc.id = doc._id;
  delete doc._id;
  delete doc.__v;
}

Would there be a way to do this after populating and also removing the _id and __v on populated documents @vkarpov15?

It seems that with your current example (and expanding it to include sub-arrays with Object.keys(doc).map((k) => { like in https://github.com/mongoosejs/mongoose-lean-id/blob/master/index.js), the _id values are removed before population occurs so the population cannot be performed.

[vkarpov15]

@mjfwebb I took a look and you're right, it isn't possible to do this with populate() because populate tries to match the populated docs after the transformDoc() function runs. In your case, I'd recommend recursively applying transformDoc() as shown below:

  const parentSchema = new Schema({ name: String, children: [{ type: 'ObjectId', ref: 'Child' }] });
  const childSchema = new Schema({ name: String });

  parentSchema.post(['find', 'findOne', 'findOneAndUpdate'], function(res) {
    if (!this.mongooseOptions().lean) {
      return;
    }
    if (Array.isArray(res)) {
      res.forEach(transformDoc);
      return;
    }
    transformDoc(res);
  });

  function transformDoc(doc) {
    if (doc._id != null) {
      doc.id = doc._id;
      delete doc._id;
    }
    delete doc.__v;

    for (const key of Object.keys(doc)) {
      if (doc[key] != null && doc[key].constructor.name === 'Object') {
        transformDoc(doc[key]);
      } else if (Array.isArray(doc[key])) {
        for (const el of doc[key]) {
          if (el != null && el.constructor.name === 'Object') {
            transformDoc(el);
          }
        }
      }
    }
  }

  const Child = mongoose.model('Child', childSchema);
  const Parent = mongoose.model('Parent', parentSchema);

  const c = await Child.create({ name: 'test child' });
  await Parent.create({ name: 'test parent', children: [c._id] });

  const p = await Parent.findOne().lean().populate('children');
  console.log(p);

[Akash187]

@vkarpov15 solution works perfectly, Just added check if res is not null.

schema.post(['find', 'findOne', 'findOneAndUpdate'], function(res) {
  if (!res || !this.mongooseOptions().lean) {
    return;
  }
  if (Array.isArray(res)) {
    res.forEach(transformDoc);
    return;
  }
  transformDoc(res);
});

function transformDoc(doc) {
  doc.id = doc._id;
  delete doc._id;
  delete doc.__v;
}

[rubenvereecken]

@vkarpov15 for our use case the solution above doesn't quite do it because

• we only want to transform right before sending
• we have a lot of complicated subdocuments that are very hard to track, whereas Mongoose did this beautifully for us

I'm up for writing a mongoose-lean-transform. Could you point me in the right direction — where is the current transform logic?

[vkarpov15]

@rubenvereecken can you please provide some more concrete examples where the provided solution doesn't work?

Here's where the current transform() code is called:

mongoose/lib/document.js

Lines 3587 to 3591 in 94d96b8

	if (typeof transform === 'function') {
	const xformed = transform(this, ret, options);
	if (typeof xformed !== 'undefined') {
	ret = xformed;
	}

[rubenvereecken]

@vkarpov15 First issue was transform happening at the wrong time. For example this scenario:

1. Fetch user
2. Do something based on user.achievements
3. Transform user, removing user.achievements, and return
   -> If we were to transform in step (1) (as in your solution), we wouldn't have the data required for step (2).

The second problem is that your proposed solution's transformDoc doesn't call transformDoc recursively, though I suppose there'd be ways around that.

As I replied in another thread, our work-around is implementing our own recursive toJSON method on lean objects (which does require methods to work, for which we used mongoose-lean-methods).