Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when trying to run executable built on M1 (Apple Silicon) #5886

Closed
6 tasks done
hussainweb opened this issue Jun 3, 2021 · 16 comments
Closed
6 tasks done

Error when trying to run executable built on M1 (Apple Silicon) #5886

hussainweb opened this issue Jun 3, 2021 · 16 comments
Labels
state:need info Need more information for solve or help. triage Please triage and relabel this issue

Comments

@hussainweb
Copy link

hussainweb commented Jun 3, 2021

Description of the issue

Context information (for bug reports)

  • Output of pyinstaller --version: 4.3

  • Version of Python: 3.9.5

  • Platform: macOS

  • How you installed Python: brew (but pyinstaller is installed within a virtualenv using pipenv)

  • Did you also try this on another platform? Does it work there?: No, I don't have one at the moment and I am convinced this is a ARM64 issue.

  • try the latest development version, using the following command:

pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

I get a build error when doing this.

Error when building the minimal program with 5.0.dev0

24 INFO: PyInstaller: 5.0.dev0
25 INFO: Python: 3.9.5
30 INFO: Platform: macOS-11.4-arm64-arm-64bit
31 INFO: wrote /Users/hw/work/axl-hiring/setup-test-repo/hello.spec
32 INFO: UPX is not available.
32 INFO: Removing temporary files and cleaning cache in /Users/hw/Library/Application Support/pyinstaller
40 INFO: Extending PYTHONPATH with paths
['/Users/hw/work/axl-hiring/setup-test-repo',
 '/Users/hw/work/axl-hiring/setup-test-repo']
46 INFO: checking Analysis
46 INFO: Building Analysis because Analysis-00.toc is non existent
46 INFO: Initializing module dependency graph...
47 INFO: Caching module graph hooks...
49 WARNING: Several hooks defined for module 'win32ctypes.core'. Please take care they do not conflict.
50 INFO: Analyzing base_library.zip ...
1150 INFO: Processing pre-find module path hook distutils from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
1150 INFO: distutils: retargeting to non-venv dir '/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9'
2576 INFO: Caching module dependency graph...
2644 INFO: running Analysis Analysis-00.toc
2647 INFO: Analyzing /Users/hw/work/axl-hiring/setup-test-repo/hello.py
2648 INFO: Processing module hooks...
2648 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2649 INFO: Loading module hook 'hook-lib2to3.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2657 INFO: Loading module hook 'hook-encodings.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2689 INFO: Loading module hook 'hook-distutils.util.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2691 INFO: Loading module hook 'hook-pickle.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2691 INFO: Loading module hook 'hook-heapq.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2692 INFO: Loading module hook 'hook-difflib.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2693 INFO: Loading module hook 'hook-multiprocessing.util.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2693 INFO: Loading module hook 'hook-sysconfig.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2697 INFO: Loading module hook 'hook-xml.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2727 INFO: Loading module hook 'hook-distutils.py' from '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks'...
2736 INFO: Looking for ctypes DLLs
2743 INFO: Analyzing run-time hooks ...
2745 INFO: Including run-time hook '/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
2749 INFO: Looking for dynamic libraries
2777 INFO: Looking for eggs
2777 INFO: Using Python library /opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/Python
2779 INFO: Warnings written to /Users/hw/work/axl-hiring/setup-test-repo/build/hello/warn-hello.txt
2793 INFO: Graph cross-reference written to /Users/hw/work/axl-hiring/setup-test-repo/build/hello/xref-hello.html
2798 INFO: checking PYZ
2798 INFO: Building PYZ because PYZ-00.toc is non existent
2798 INFO: Building PYZ (ZlibArchive) /Users/hw/work/axl-hiring/setup-test-repo/build/hello/PYZ-00.pyz
3018 INFO: Building PYZ (ZlibArchive) /Users/hw/work/axl-hiring/setup-test-repo/build/hello/PYZ-00.pyz completed successfully.
3021 INFO: EXE target arch: arm64
3021 INFO: Code signing identity: None
3021 INFO: checking PKG
3021 INFO: Building PKG because PKG-00.toc is non existent
3021 INFO: Building PKG (CArchive) PKG-00.pkg
3026 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
3026 INFO: Bootloader /Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/bootloader/Darwin-64bit/run
3026 INFO: checking EXE
3026 INFO: Building EXE because EXE-00.toc is non existent
3026 INFO: Building EXE from EXE-00.toc
3026 INFO: Copying bootloader exe to /Users/hw/work/axl-hiring/setup-test-repo/build/hello/hello
3027 INFO: Converting EXE to target arch (arm64)
Traceback (most recent call last):
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/bin/pyinstaller", line 8, in <module>
    sys.exit(run())
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/building/build_main.py", line 758, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/building/build_main.py", line 705, in build
    exec(code, spec_namespace)
  File "/Users/hw/work/axl-hiring/setup-test-repo/hello.spec", line 23, in <module>
    exe = EXE(pyz,
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/building/api.py", line 505, in __init__
    self.__postinit__()
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/building/datastruct.py", line 159, in __postinit__
    self.assemble()
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/building/api.py", line 695, in assemble
    osxutils.binary_to_target_arch(self.name, self.target_arch,
  File "/Users/hw/.local/share/virtualenvs/setup-test-repo-HE-9HUxx/lib/python3.9/site-packages/PyInstaller/utils/osx.py", line 312, in binary_to_target_arch
    assert target_arch in archs, \
AssertionError: Bootloader EXE is incompatible with target arch arm64 (has arch: x86_64)!

Make sure everything is packaged correctly

  • start with clean installation
  • use the latest development version
  • Run your frozen program from a command window (shell) — instead of double-clicking on it
  • Package your program in --onedir mode
  • Package without UPX, say: use the option --noupx or set upx=False in your .spec-file
  • Repackage you application in verbose/debug mode. For this, pass the option --debug to pyi-makespec or pyinstaller or use EXE(..., debug=1, ...) in your .spec file.

A minimal example program which shows the error

#!/usr/bin/env python3

print("hello")

Stacktrace / full error message

[68665] Error loading Python lib '/Users/hw/work/axl-hiring/setup-test-repo/dist/hello/Python': dlopen: dlopen(/Users/hw/work/axl-hiring/setup-test-repo/dist/hello/Python, 10): no suitable image found.  Did find:
	/Users/hw/work/axl-hiring/setup-test-repo/dist/hello/Python: mach-o, but wrong architecture
	/Users/hw/work/axl-hiring/setup-test-repo/dist/hello/Python: mach-o, but wrong architecture```

Please also see <https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs>
for more about what would use to solve the issue.
@hussainweb hussainweb added the triage Please triage and relabel this issue label Jun 3, 2021
@hussainweb
Copy link
Author

I have tried both --onefile and --onedir and get the same error (different paths). I also tried to understand the output in debug mode as well. There was nothing I could see that would point to a problem. I tried this first on 4.3 which has the fix from #5640 (I double-checked it). Like I said above, I get a build error itself when using the develop release.

@bwoodsend
Copy link
Member

I take it you didn't compile the bootloaders? The ones baked into this repository are still boring old x86_64.

@bwoodsend bwoodsend added the state:need info Need more information for solve or help. label Jun 3, 2021
@rokm
Copy link
Member

rokm commented Jun 3, 2021

Since you're on M1, you will need to use develop and you will need to rebuild the bootloaders. The build error you're getting with develop is because the pre-built bootloaders that are committed to repository are still x86_64-only, as Brénainn said.

@hussainweb
Copy link
Author

hussainweb commented Jun 3, 2021

@bwoodsend, no. I installed this using pip (both the regular release and the dev one). I found this link and I would appreciate some help as to what should be the parameters.

@hussainweb
Copy link
Author

Okay, I was trying to do this within the files downloaded by pip but there is no waf script (running the python ./waf all command obviously throws errors). So, I am assuming I have to download the source, run the command in there and copy the bootloader to the directory within my virtualenv?

@rokm
Copy link
Member

rokm commented Jun 3, 2021

The following steps should suffice:

git clone https://github.com/pyinstaller/pyinstaller.git
cd pyinstaller/bootloader
python ./waf all
pip install ..

Assuming (python and pip are the correct versions from your virtualenv).

@hussainweb
Copy link
Author

hussainweb commented Jun 3, 2021

Thanks @rokm. This works for my test (minimal) script but not for my actual application. It just exits immediately without any error or message. It is a CLI application based on click. I will try to come up with a minimal script and post here (maybe a new issue) but is there something you know that I should do when using click?

@rokm
Copy link
Member

rokm commented Jun 3, 2021

I will try to come up with a minimal script and post here (maybe a new issue) but is there something you know that I should do when using click?

Nothing comes to mind. Does the introductory example from here work for you?

@hussainweb
Copy link
Author

@rokm, yes, that works. I suppose it has to do with how I have structured the program then. This is the structure.

├── setuprepo
│   ├── __init__.py
│   ├── click_types.py
│   ├── constants.py
│   ├── repo.py
│   └── repo_test.py
├── setuprepo.py

And this is setuprepo.py (which is my entrypoint.

#!/usr/bin/env python3

import sys

from setuprepo import repo


if __name__ == "__main__":
    sys.exit(repo.main())

I am building with this command:

pyinstaller --onedir --clean --noconfirm --hidden-import click setuprepo.py

I will try variations and see what makes it fail. Please let me know if there is something obvious I am missing and if you prefer me to create another issue.

@rokm
Copy link
Member

rokm commented Jun 3, 2021

My first guess would be that the problem is that both the package and the entry-point script are named setuprepo. Can you try renaming your entry-point script?

@hussainweb
Copy link
Author

@rokm, yes, that was the problem! I was so confused. I had stripped down the entrypoint to just a print statement and it still didn't work. I'm curious. Is this a Python thing or something about PyInstaller?

After renaming the entrypoint, everything seems to be working but I am testing this further. Thank you for all your help.

@hussainweb
Copy link
Author

@rokm, one more question here. The binary this produces won't work on macOS on Intel, right? I need to build that binary on that platform. Is this correct? I saw some issues talking about supporting universal2 but I don't know if that is implemented or even planned.

@bwoodsend
Copy link
Member

See the docs on that topic...

@bwoodsend
Copy link
Member

Speaking of which, I see that it hasn't really rendered properly.

@rokm
Copy link
Member

rokm commented Jun 3, 2021

Is this a Python thing or something about PyInstaller?

I suspect this is a side-effect of PyInstalling adding entry pointt's directory to its path; then the entry point script ends up being treated as a stand-alone module and shadows the package (or vice versa). It seems to work normally under python, but that's probably because the interpreter does not care about the entry-point's name (as main module is referred to as __main__).

The binary this produces won't work on macOS on Intel, right?

That's right.

I saw some issues talking about supporting universal2 but I don't know if that is implemented or even planned.

It is now possible to attempt to create universal2 binaries as well (by passing --target-arch universal2), but you need a universal2 python for that. Homebrew python is single-arch, so trying to build universal2 application should give you a build error.

Also, as per linked docs, even with universal2 python it depends on what packages you are using.

I need to build that binary on that platform. Is this correct?

Not strictly. You could probably install x86_64-only homebrew python (not sure how much trickery that involves in brew's case, though) and run it under rosetta on your M1. Then use that to set up a x86_64-only virtual environment, and build x86_64-only version of your application.

@rokm rokm closed this as completed Jun 11, 2021
@hussainweb
Copy link
Author

Thanks for all your help here. I have not yet tried the steps to compile an x86 or a universal2 binary (mainly because I have not yet figured out how to do that with homebrew). These notes would help me to do that when I do manage it and get back to my Python module.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
state:need info Need more information for solve or help. triage Please triage and relabel this issue
Projects
None yet
Development

No branches or pull requests

3 participants