Slow creation of documents with 100+ fields

[shyrik13]

Question
We store mongoose documents objects in Redis as JSON.
We transform that objects to documents like this:

// const redisResponse = response from Redis
const docs = {};

for (let item in redisResponse) {
    let doc = new myModel(JSON.parse(item));
    docs[doc.id] = doc;
}

In item we have approximately 100 fields included embedded object.
The problem is that when we have 500 objects it takes 3 seconds to build all documents from that cached objects.
We have give to our NodeJs instance 500m (millicpu) CPU and 2048Mi RAM.

[Uzlopak]

Why do you need the hydrated Objects? Hydrated Object always mean Overhead. If you just want to revive e.g. ObjectIds, than you could try to store the rawResult of your mongo calls into redis and use the bson deserializer instead of JSON.parse.

Why do you use an Array to store the hydrated objects? v8 probably also slows there down. It would be probably better to store the hydrated Objects in an Object or even better in a Map. Having "holes" in the Array keys reduces the v8 performance significantly. But If you want to keep your Array, you should atleast initialize the length of your docs Array with a length.

Like:
const docs = new Array(redisResponse.length);

Cross that. You cant initialize the length.

[molarro]

We store documents in Redis as hash, where id is a key, but value as document content in JSON. Periodically we need to get all documents from Redis and transform (hydrate) to Mongoose documents, because we use built-in and custom methods of document object, as well as id generation for new embedded objects. So JSON.parse is quite fast, unlike new myModel .

[vkarpov15]

I belive this is a duplicate of #11541. We're actively working on improving this.