composite crashes silently

[dekdevy]

I am trying to composite an image atlas made out of many small images. It works for up to 150 images, but it crashes every single time as soon as I try it with 151 images.

  sharp({
    create: {
      width     : size, height    : size,
      channels  : 4,
      background: { r: 0, g: 0, b: 0, alpha: 0.0 }
    }
  })
  .composite(images)
  .toFile(dest.path()+'.webp')

images is an array of buffer inputs. As soon as it reaches 151 elements, it does not work anymore.

There is no error message, nothing to catch - the callback function of "toFile" is never called - any form of nodejs process event listening for process exits / SIGINT / uncaughtExceptions catches absolutely nothing.
The process is not running out of memory, it peaks at around 80mb. and then just closes.
I have tried this with various sets of images with the same result.

[lovell]

Hi, you've not provided details of the platform in use but my best guess would be a flavour of Linux with a small stack size such as Alpine, where that stack has overflowed.

Disabling caching within libvips via sharp.cache(false) reduces stack pressure.

[lovell]

Hope this helped, please feel free to re-open with more details if further help is required.

[tgrajewski]

@lovell I have exact same problem with .composite() failing when called with 151+ images. The call sharp.cache(false) doesn't change a thing.

I'm running Windows 10 Pro, Node.js v10 (64-bit).

I've managed to discover that a stack overflow is occuring inside Glib. I've used procdump to see what happens and executed it like this:

procdump64.exe -e 1 -f "" -x c:\dumps node.exe my_script.js

The above call ends with this in command prompt:

[13:16:23] Exception: C00000FD.STACK_OVERFLOW
[13:16:23] Unhandled: C00000FD.STACK_OVERFLOW
[13:16:23] Dump 1 initiated: c:\DUMPS\node.exe_190711_131623.dmp
[13:16:24] Dump 1 complete: 5 MB written in 0.1 seconds
[13:16:24] Dump count reached.

And this is the generated crash dump which maybe will be useful: node.exe_190711_131623.dmp.zip
(you can inspect it with WinDbg).

---

Also, a side note: it seems that .composite() is very slow and takes several seconds to complete when called with less that 151 images. To compare, I've a similar script that generates atlases, written in Python and Pillow and that script generates an atlas file of 671 images in just 0.8 seconds.

[lovell]

The performance of the composite operation in libvips v8.8.0 (which the forthcoming sharp v0.23.0 will include) has much improved.

For non-overlapping inputs you may also be interested in the future possible enhancement at #1580.

[tgrajewski]

@lovell OK, thanks for good news about performance improvements, but will the stack overflow be fixed? Also, I don't think that the enhancement at #1580 is for me, I need images to be pasted into specific positions (non-overlapping), not on a grid.

[slikts]

I'm compositing 2560 small, overlapping images, and it was working earlier, but now it stopped (the code is the same, so it's very confusing). sharp.cache(false) seems to have no effect. This issue shouldn't be closed, and at the very least it shouldn't fail silently.

[ceeroover]

same, crashes with 147 images

[willemmulder]

same, with 196 images.

@lovell Can this issue be re-opened? It is definitely not fixed.

[lovell]

@willemmulder see #2286