Importing wand takes 700ms on Ubuntu (and can't be avoided via environment)

[craigds]

In Ubuntu 18.04 LTS, the libmagickwand-6.q16-3 package installs the wand libraries at:

• /usr/lib/x86_64-linux-gnu/libMagickCore-6.Q16.so.3
• /usr/lib/x86_64-linux-gnu/libMagickWand-6.Q16.so.3

import wand finds these, but it takes ~700ms to do because it runs ctypes.find_library (apparently several times?)

Looking at the source code, it seemed like we could avoid this by setting MAGICK_HOME and WAND_MAGICK_LIBRARY_SUFFIX in our containers.

However, this doesn't actually help because wand hardcodes the .so extension, and .so.3 doesn't match.

Perhaps wand should handle a couple of environment variables containing the full paths to the libraries?

[emcconville]

I'm open to suggestions. Try NOT setting MAGICK_HOME, but just WAND_MAGICK_LIBRARY_SUFFIX=-6.Q16 value.

[craigds]

While checking that I found it interesting to note that the options doesn't seem to be required in our case. If I omit both environment variables and let wand run find_library over all the various combinations I get this:

>>> for suffix in suffixes:
...     libwand = find_library('MagickWand' + suffix)
...     print(f"{suffix:13s} : {libwand}")
...
...
...
              : None
HDRI          : None
HDRI-2        : None
-7            : None
-7HDRI        : None
-7HDRI-2      : None
-7.Q8         : None
-7.Q8HDRI     : None
-7.Q8HDRI-2   : None
-7.Q16        : None
-7.Q16HDRI    : None
-7.Q16HDRI-2  : None
-6            : libMagickWand-6.Q16.so.3
-6HDRI        : None
-6HDRI-2      : None
-Q16          : None
-Q16HDRI      : None
-Q16HDRI-2    : None
-Q8           : None
-Q8HDRI       : None
-Q8HDRI-2     : None
-6.Q16        : libMagickWand-6.Q16.so.3
-6.Q16HDRI    : None
-6.Q16HDRI-2  : None

It was surprising to me that the library is found while checking the -6 suffix even though it has the Q16 option tacked on to the library name. Perhaps the options are unnecessary? I wouldn't presume to know whether it is necessary on other systems though.

So the issue for us is that find_library is called 13 times before it finds a result. As you suggested, switching to setting the suffix but not MAGICK_HOME would solve this for us - find_library would run once and find the library immediately.

[emcconville]

It was surprising to me that the library is found while checking the -6 suffix even though it has the Q16 option tacked on to the library name.

That's ABI working at its best.

So the issue for us is that find_library is called 13 times before it finds a result.

I good time to consider migrating to ImageMagick-7. If performance is an important feature, I highly recommend ImageMagick-7's Q8 without HDRI. I know it's crummy to hear that, but Wand must exhaust ImageMagick-7 & support for High Dynamic Range images before attempting ImageMagick-6.

All that said, I still believe a "short-cut" environment variable is a good idea.

[craigds]

Imagemagick 7.x hasn't made it into an ubuntu LTS any ubuntu releases yet. Not sure why, seems like it's been around for years (?)

Looks like progress is being made though so maybe 22.04 will get it.