Possible memory leak on impressions tracking

[leeeomaaax]

lib version: 10.25.2 (but this was also happening on version 10.23.1)
node version: 18

• Our servers seems to be experiencing a memory leak on the getTreatment function. (similar to this issue)
• Memory goes up really slowly, so it's super hard to reproduce locally
• Exploring the memory profiler in prod, we found some data pointing to the track method inside the javascript-common library

• every time the lines going up drop, means we did a new deploy. deploys in this graph were done 1 week apart so I don't believe this is the regular impression queue going up before being flushed.
• lines that are constantly going up vary, but they are always one of the following: Track method, hash128x86 method, process method (all methods from javascript-commons)

• flamegraph of memory allocation

--

• is there a way I can disable impressions tracking on the node SDK?
• can you guys take a look and let me know if I'm missing something?

[radiantstatic]

@leeeomaaax - my company is seeing similar behavior. We even went so far as to set up split synchronizer to alleviate some of the pressure, but unfortunately did not resolve the issue. Would love to hear from Split team if there is something worth fixing.

[leeeomaaax]

hey @radiantstatic , we've opened a support ticket with the Split team and they suggested disabling the impressionsMode.
We've tested and it seems that the leak is gone (or at least it is at a really low level).

sync: {
  impressionsMode: 'NONE',
},

[EmilianoSanchez]

Hi @leeeomaaax,

From some performance tests running tens of millions of getTreatment evaluations, we haven't identified any patterns in heap memory usage that suggest a memory leak.

However, given the range of memory usage displayed in the screenshots (53 MiB ~ 293 MiB), it might be related to the internal cache used by the 'OPTIMIZED' and 'DEBUG' impression modes. The 'NONE' mode does not use this cache.

This cache, which is used to deduplicate impressions, has a maximum size limit of 500,000 items. A getTreatment call with a different user key, flag name, etc., represents a new item in the cache. So, eventually, the cache will reach its maximum size of 500k items, and memory usage should stabilize.

The cache is a Least Recently Used (LRU) cache, based on a Map and linked list data structures. Each item consists of a 32-byte string and a number (8 bytes). Considering the overhead from data structure references and other variables, it is reasonable to assume that the cache could reach one hundred to a few hundred MBs, though we don't have exact numbers.

We will include some comments about this in our public documentation for future reference.

[leeeomaaax]

Thank you for following up on this @EmilianoSanchez!
It makes sense now why I wan't able to reproduce it locally -> I was applying load on the getTreatment call with always the same user key/flag.

[EmilianoSanchez]

Yes. That makes sense @leeeomaaax .

I cannot 100% guarantee there is not a memory leak, but it's probably related to that cache.

Using the Chrome dev tools, it seems that the LRUCache size plus its Nodes are around 144.7 MBs when it is full (the Retained size column in the screenshot, which is in bytes).