BUG: MacOS universal2 wheels have two gfortran shared libraries, which trips up PyInstaller

[danielhrisca]

Describe your issue.

Starting with release 1.8.0 there seems to be a mismatch of shared libraries. Please see the failure message on this github actions job:
https://github.com/danielhrisca/asammdf/runs/5109724435?check_suite_focus=true

Reproducing Code Example

Try to build a stand-alone application using PyInstaller

Error message

Traceback (most recent call last):
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/bin/pyinstaller", line 8, in <module>
      sys.exit(run())
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/__main__.py", line 124, in run
      run_build(pyi_config, spec_file, **vars(args))
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/__main__.py", line 58, in run_build
      PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/building/build_main.py", line 793, in main
      build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/building/build_main.py", line 725, in build
      exec(code, spec_namespace)
    File "asammdf.spec", line 83, in <module>
      exe = EXE(
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/building/api.py", line 490, in __init__
      self.pkg = PKG(
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/building/api.py", line 205, in __init__
      self.__postinit__()
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/building/datastruct.py", line 155, in __postinit__
      self.assemble()
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/building/api.py", line 268, in assemble
      fnm = checkCache(
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/building/utils.py", line 361, in checkCache
      osxutils.binary_to_target_arch(cachedfile, target_arch, display_name=fnm)
    File "/Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/PyInstaller/utils/osx.py", line 315, in binary_to_target_arch
      assert target_arch in archs, \
  AssertionError: /Users/runner/work/asammdf/asammdf/.tox/exe/lib/python3.9/site-packages/scipy/linalg/../.dylibs/libgfortran.5.dylib is incompatible with target arch x86_64 (has arch: arm64)!

SciPy/NumPy/Python version information

1.8.0 1.22.2 CPython 3.9.10 on macOS-10.16-x86_64-i386-64bit

[emlys]

I'm also experiencing this (example github actions run).

[tupui]

Hi @danielhrisca, thank you for reporting. I am not familiar with pyinstaller so it's not clear to me what it's doing and what we could have done to break things. Can you provide something to reproduce the issue or more logs to understand what pyinstaller is trying to do?

We added two external C-submodules UNU.RAN and PROPACK. That could be linked.

[danielhrisca]

pyinstaller just bundles the python interpreter + libraries into a distributable package. It looks at the imports and sees what libraries are necessary, then copies the library from the site_packages folder

In this case I think that the macos scipy wheel for x86_64 contains the shared libray libgfortran.5.dylib which was compiled for arm

[tupui]

Ah ok, I see. The wheels are created on another repo. cc @tylerjereddy

Here is the relevant CI files
https://github.com/MacPython/scipy-wheels/blob/master/azure-pipelines.yml
https://github.com/MacPython/scipy-wheels/blob/master/azure-posix.yml

Maybe a bug in the logic when we specify PLAT: "x86_64". We have a few issues/PR there and still fixing things. Did you try with another version of Python?

[danielhrisca]

Did you try with another version of Python?

As you can see here only Python 3.9 fails. Python 3.7, 3.8 and 3.10 are ok

[rgommers]

In this case I think that the macos scipy wheel for x86_64 contains the shared libray libgfortran.5.dylib which was compiled for arm

Ah, that would be bad. Are you sure though? That would mean that the wheel would be broken for regular use on x86_64 as well, not just for PyInstaller. And we'd see many more complaints here I imagine.

Can someone with an Intel Mac confirm where the Python 3.9 wheels is importable and passes its tests (import scipy; scipy.test())?

[andyfaff]

I have an Intel Mac. I created a Python 3.9.10 conda env and installed scipy:

>>> pip install scipy
Collecting scipy
  Using cached scipy-1.8.0-cp39-cp39-macosx_12_0_universal2.macosx_10_9_x86_64.whl (55.6 MB)
Collecting numpy<1.25.0,>=1.17.3
  Using cached numpy-1.22.2-cp39-cp39-macosx_10_14_x86_64.whl (17.6 MB)
Installing collected packages: numpy, scipy
Successfully installed numpy-1.22.2 scipy-1.8.0

I then ran:

>>> import scipy
>>> scipy.test()

The tests pass:

35397 passed, 2767 skipped, 11452 deselected, 104 xfailed, 10 xpassed, 15 warnings in 617.26s (0:10:17)

I then went into the scipy/.dylibs directory:

(test) 192-168-1-100:.dylibs andrew$ lipo -info *.dylib
Architectures in the fat file: libgcc_s.1.dylib are: x86_64 i386 
Architectures in the fat file: libgcc_s.2.dylib are: arm64 
Architectures in the fat file: libopenblas.0.dylib are: x86_64 arm64 
Non-fat file: libgfortran.3.dylib is architecture: x86_64
Non-fat file: libgfortran.5.dylib is architecture: arm64
Non-fat file: libquadmath.0.dylib is architecture: x86_64

There are two libgfortrans, one that's targeting x86_64, and one targeting arm64.

[andyfaff]

So the original error message is technically correct, there is an arm64 dylib (actually libgcc_2 as well) present in the wheel, but its presence has no effect when in a normal interpreter.
I don't know how easy it would be to ensure that a fat binary of libgfortran, with x86_64 and arm64, is bundled. I could hazard a guess that it should be possible to make a FAT x86_64/arm64, but libgcc_s might be harder.

Perhaps @matthew-brett might be able to shed light, because it might be something to do with delocate?

[andyfaff]

I note that there is a dedicated Intel wheel, scipy-1.8.0-cp39-cp39-macosx_10_9_x86_64.whl. That seems to only have the right architectures bundled:

(test) P0014M:.dylibs anz$ ls
libgcc_s.1.dylib	libgfortran.3.dylib	libopenblas.0.dylib	libquadmath.0.dylib
(test) P0014M:.dylibs anz$ lipo -info *.dylib
Architectures in the fat file: libgcc_s.1.dylib are: x86_64 i386 
Non-fat file: libgfortran.3.dylib is architecture: x86_64
Non-fat file: libopenblas.0.dylib is architecture: x86_64
Non-fat file: libquadmath.0.dylib is architecture: x86_64

@danielhrisca, can I recommend that if you want to use scipy in a PyInstaller bundle that you use the scipy-1.8.0-cp39-cp39-macosx_10_9_x86_64.whl wheel?

[rgommers]

Thanks for that detailed analysis @andyfaff!

That all looks correct, at least in the sense that all the architectures one expects and no extra ones are present for both universal2 and x86_64 wheels.

I don't know how easy it would be to ensure that a fat binary of libgfortran, with x86_64 and arm64, is bundled. I could hazard a guess that it should be possible to make a FAT x86_64/arm64, but libgcc_s might be harder.

@isuruf implemented this universal2 support, so may be able to say more.

Right now it looks like the only reason we have universal2 wheels is PyInstaller, py2app etc., so if that doesn't work anyway we may as well get rid of them if it's not easily fixable.

[rgommers]

@danielhrisca do you really need to build a universal application, or would you be fine building separate Intel and arm64 installers for your users?

[rgommers]

This is actually a duplicate with gh-15539, where these issues were discussed and resolved. PyInstaller issue is on their side it looks like from that conversation. So I'll close this issue.

@danielhrisca do you really need to build a universal application, or would you be fine building separate Intel and arm64 installers for your users?

This question is still of interest.

[danielhrisca]

It was not my intention to create an universal application, just to run the build on a macos VM. I guess I'll follow the other issue for a solution. Thanks

[rgommers]

Okay thanks @danielhrisca. No idea what Tox is doing there then.

[jonahschueller]

For those who will get here through Google, etc:

INFO: This issue has been solved in PyInstaller version 4.10

https://github.com/pyinstaller/pyinstaller/releases/tag/v4.10

[danielhrisca]

For those who will get here through Google, etc:

INFO: This issue has been solved in PyInstaller version 4.10

https://github.com/pyinstaller/pyinstaller/releases/tag/v4.10

Thank you for the info, I will try it out