Importing numba causes numpy to segfault in frozen application

[drewm1980]

The following python code segfaults when the frozen application is run:

#!/usr/bin/env python3
import numpy
import numba

print("Attempting matrix multiplication of two numpy arrays...")
A = numpy.zeros((3,3))
B = numpy.zeros((3,3))
C = A@B
print("done")

The segfault occurs occurs on the matrix multiplication with @.
The un-frozen code runs as expected.

I am both building and running the frozen code within the same conda environment, on the same machine.

I haven't been able to reproduce this using the git version because I'm unable to run it. I'll try again later and file a separate request for that.

$ cat /etc/issue
Ubuntu 16.04.4 LTS

awagner@tars:/home/awagner/private/deployment_testing
$ pyinstaller --version
3.3.1

I don't have a backtrace since my system gdb is screwed up.

strace output:
strace_output.txt

[htgoebel]

Please run strace using -f (follow).

I don't have a backtrace since my system gdb is screwed up.

gdb would definitively be best :-) (I can't see how what you linked is to running the exe bundled by PyiInstaller, though.)

[HennersBro98]

I am also experiencing this exact same issue. My program works as expected when run in IDE, however 3x3 or greater matrix multiplication crashes the program after it has been compiled with pyinstaller. Weirdly, it seems to work with 2x2 multiplication. I have tested with using the @ symbol, and also using np.matmul() and np.dot() and all have the same effect.

[rokm]

@HennersBro98 What OS are you using? And are you also using a conda environment, like the original report?

[HennersBro98]

Windows 10, and yes I am using a conda environment. I am not using numba, but the same issue occurs at the matrix multiplication.

[rokm]

Can you try using branch from #5168 to see if that fixes the issue with numpy under conda? I.e., uninstall the installed pyinstaller, then install the PR's branch via pip install https://github.com/bwoodsend/pyinstaller/archive/fix-numpy-hooks.zip

If that does not help, please post additional details: version of numpy, version of pyinstaller, and the error message you get from the program when it crashes. Since you're on Windows and not on linux, I doubt you are experiencing a segfault like the original poster...

[bwoodsend]

Ughh I've seen this one too many times. See here.

[HennersBro98]

Thanks @bwoodsend , worked perfectly!

[bwoodsend]

@HennersBro98 Could you also try the patch PR as requested by @rokm?

I should also warn you that although that particular dll is no longer missing, there are an extensive list of other missing dlls just like this except that, as they come with a lot of common programming tools, you probably already have them in PATH meaning that you can't see that your app is broken just by running it on your own machine.

We are in the process of moving the numpy hooks to numpy's repo where hopefully Conda will pick it up and maintain the list of required binaries. But for now, I strongly recommend that you consider Conda + numpy + PyInstaller incompatible. Using pip install numpy in a Conda environment is safe with PyInstaller.

[HennersBro98]

That patch does not seem to work, I get:
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect

[bwoodsend]

@HennersBro98 Can we have the details please? Is it runtime or buildtime? What's the full error message?

[bwoodsend]

Oh, and did you remember to use the --clean option?

[HennersBro98]

Using the version of pyinstaller as requested by @rokm: 4.1.dev0
And numpy version: 1.18.5
I used the following command: pyinstaller --onefile --clean -w myFileName.py
This resulted in an error where the last couple of lines of the output were:
File "c:\users\henry\anaconda3\lib\pathlib.py", line 1194, in stat
return self._accessor.stat(self)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\Users\henryC:\Users\henry\AppData\Local\Microsoft\WindowsApps'

I'm unsure why it is repeating the first couple of terms in the path, it did not produce this error in your workaround.

[HennersBro98]

Apologies, just updated to latest version of numpy (v1.19.2) and the above error is no longer persiting. It also fixes the original numpy matrix multiplication error, thanks!