Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

win build: bump python, wine, pyinstaller #7721

Merged
merged 3 commits into from Mar 20, 2022

Conversation

SomberNight
Copy link
Member

@SomberNight SomberNight commented Mar 17, 2022

This is bumping the python version bundled inside the Windows binaries, from 3.9.10 to 3.9.11.

@SomberNight SomberNight added this to the 4.2.1 milestone Mar 17, 2022
@SomberNight
Copy link
Member Author

Looks like the wine build of the Windows binaries is failing.

The error is when trying to install python inside wine.
With wine 6.0.2 (which we use on master) and with wine 6.0.3, cpython 3.9.11 fails to install
(but note that cpython 3.9.10 worked)

010c:err:virtual:virtual_setup_exception stack overflow 1220 bytes in thread 010c addr 0x7bc6713d stack 0x440b3c (0x440000-0x441000-0x640000)
🗯 ERROR: wine msiexec failed for dev.msi
🗯 ERROR: prepare-wine failed

With wine 7.0.0.0 (latest stable atm), the error goes away. (as in second commit)

@SomberNight
Copy link
Member Author

SomberNight commented Mar 18, 2022

Re the Windows build, with wine 7.0.0.0, it is still failing but now much later with a new error.
It is now pyinstaller that errors during its runtime:

80572 INFO: Processing module hooks...
80573 INFO: Loading module hook 'hook-certifi.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
80618 INFO: Loading module hook 'hook-cryptography.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
82879 INFO: Loading module hook 'hook-dns.rdata.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84147 INFO: Loading module hook 'hook-mnemonic.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84207 INFO: Loading module hook 'hook-pycparser.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84212 INFO: Loading module hook 'hook-usb1.py' from 'C:\\python3\\lib\\site-packages\\usb1\\__pyinstaller'...
84215 INFO: --- libusb1 pyinstaller hook ---
84226 INFO: Added libusb binaries: [('C:\\python3\\lib\\site-packages\\usb1\\libusb-1.0.dll', 'usb1')]
84228 INFO: Loading module hook 'hook-difflib.py' from 'C:\\python3\\lib\\site-packages\\PyInstaller\\hooks'...
84237 INFO: Excluding import of doctest from module difflib
84237 INFO: Loading module hook 'hook-distutils.py' from 'C:\\python3\\lib\\site-packages\\PyInstaller\\hooks'...
Unable to find "C:\python3\Include\pyconfig.h" when adding binary and data files.This would mean your Python installation doesn't
come with proper library files. This usually happens by missing development
package, or unsuitable build parameters of Python installation.
* On Debian/Ubuntu, you would need to install Python development packages
  * apt-get install python3-dev
  * apt-get install python-dev
* If you're building Python by yourself, please rebuild your Python with
`--enable-shared` (or, `--enable-framework` on Darwin)


🗯 ERROR: build-electrum-git failed

Looks like this might be fixed in pyinstaller 4.3 (we are using 4.2):
pyinstaller/pyinstaller#5218

SomberNight added a commit that referenced this pull request Mar 18, 2022
This is bumping the python versions bundled inside our binaries.
For macOS and AppImage, from 3.9.10 to 3.9.11.
For Android, from 3.8.12 to 3.8.13.

Windows is left untouched as I am having issues with the wine build when using 3.9.11.
(see #7721 (comment) )
@SomberNight
Copy link
Member Author

As I am having non-trivial issues with the Windows build, but the rest of the builds look fine,
I have pushed the rest to master (1bb7ef9), and changing this PR to focus on Windows.

@SomberNight SomberNight changed the title binaries: bump python 3.9.10->3.9.11; and for android 3.8.12->3.8.13 win build: bump python 3.9.10->3.9.11 Mar 18, 2022
@SomberNight
Copy link
Member Author

SomberNight commented Mar 18, 2022

On pyinstaller 4.3-4.6, the bootloader fails to build (i.e. we cannot cross-compile from linux).

Could not find the program ['windres']
(complete log in /opt/wine64/drive_c/electrum/contrib/build-wine/.cache/win32/pyinstaller/bootloader/build/config.log)

This error (pyinstaller/pyinstaller#6339), and another (pyinstaller/pyinstaller#6338) are fixed in pyinstaller 4.7.


Using pyinstaller 4.7 (or 4.8 or 4.10), the whole build succeeds, however there is an issue at runtime.
i.e. when launching the electrum exe on Windows, the process crashes.
Log using pyinstaller 4.10 (latest stable):

[14164] LOADER: Overriding Python's sys.path
[14164] LOADER: Post-init sys.path is C:\Users\User\AppData\Local\Temp\_MEI159242\base_library.zip;C:\Users\User\AppData\Local\Temp\_MEI159242\lib-dynload;C:\Users\User\AppData\Local\Temp\_MEI159242
[14164] LOADER: Setting sys.argv
[14164] LOADER: setting sys._MEIPASS
[14164] LOADER: importing modules from CArchive
[14164] LOADER: extracted struct
[14164] LOADER: running unmarshalled code object for struct...
[14164] LOADER: extracted pyimod01_os_path
[14164] LOADER: running unmarshalled code object for pyimod01_os_path...
[14164] LOADER: extracted pyimod02_archive
[14164] LOADER: running unmarshalled code object for pyimod02_archive...
[14164] LOADER: extracted pyimod03_importers
[14164] LOADER: running unmarshalled code object for pyimod03_importers...
[14164] LOADER: extracted pyimod04_ctypes
[14164] LOADER: running unmarshalled code object for pyimod04_ctypes...
[14164] LOADER: Installing PYZ archive with Python modules.
[14164] LOADER: PYZ archive: PYZ-00.pyz
[14164] LOADER: Running pyiboot01_bootstrap.py
Traceback (most recent call last):
  File "PyInstaller\loader\pyiboot01_bootstrap.py", line 20, in <module>
  File "PyInstaller\loader\pyimod03_importers.py", line 573, in install
  File "PyInstaller\loader\pyimod03_importers.py", line 128, in __init__
ImportError: Cannot load frozen modules.
[14164] Failed to execute script 'pyiboot01_bootstrap' due to unhandled exception!
[14164] LOADER: OK.
[14164] LOADER: Manually flushing stdout and stderr
[14164] LOADER: Cleaning up Python interpreter.
[15924] LOADER: Back to parent (RC: 1)
[15924] LOADER: Doing cleanup

edit: opened pyinstaller/pyinstaller#6686 - see details there

with wine 6.0.2 and 6.0.3, cpython 3.9.11 fails to install (but cpython 3.9.10 worked)
```
010c:err:virtual:virtual_setup_exception stack overflow 1220 bytes in thread 010c addr 0x7bc6713d stack 0x440b3c (0x440000-0x441000-0x640000)
🗯 ERROR: wine msiexec failed for dev.msi
🗯 ERROR: prepare-wine failed
```

-----

btw, related note:
After changing the Dockerfile, building the docker image from cache failed. Setting ELECBUILD_NOCACHE=1 fixed it:
```
E: Could not configure 'libc6:i386'.
E: Could not perform immediate configuration on 'libgcc-s1:i386'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)

$ ELECBUILD_NOCACHE=1 ./contrib/build-wine/build.sh
```
see spesmilo#7721 (comment)

-----

pyinstaller 4.2 failed (during its runtime) to create exes (worked with cpython 3.9.10, but not with cpython 3.9.11):
```
80572 INFO: Processing module hooks...
80573 INFO: Loading module hook 'hook-certifi.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
80618 INFO: Loading module hook 'hook-cryptography.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
82879 INFO: Loading module hook 'hook-dns.rdata.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84147 INFO: Loading module hook 'hook-mnemonic.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84207 INFO: Loading module hook 'hook-pycparser.py' from 'C:\\python3\\lib\\site-packages\\_pyinstaller_hooks_contrib\\hooks\\stdhooks'...
84212 INFO: Loading module hook 'hook-usb1.py' from 'C:\\python3\\lib\\site-packages\\usb1\\__pyinstaller'...
84215 INFO: --- libusb1 pyinstaller hook ---
84226 INFO: Added libusb binaries: [('C:\\python3\\lib\\site-packages\\usb1\\libusb-1.0.dll', 'usb1')]
84228 INFO: Loading module hook 'hook-difflib.py' from 'C:\\python3\\lib\\site-packages\\PyInstaller\\hooks'...
84237 INFO: Excluding import of doctest from module difflib
84237 INFO: Loading module hook 'hook-distutils.py' from 'C:\\python3\\lib\\site-packages\\PyInstaller\\hooks'...
Unable to find "C:\python3\Include\pyconfig.h" when adding binary and data files.This would mean your Python installation doesn't
come with proper library files. This usually happens by missing development
package, or unsuitable build parameters of Python installation.
* On Debian/Ubuntu, you would need to install Python development packages
  * apt-get install python3-dev
  * apt-get install python-dev
* If you're building Python by yourself, please rebuild your Python with
`--enable-shared` (or, `--enable-framework` on Darwin)

🗯 ERROR: build-electrum-git failed
```

Looks like this might be fixed in pyinstaller 4.3 (we are using 4.2):
pyinstaller/pyinstaller#5218

-----

While trying to update pyinstaller to have that fix, several issues found with versions 4.3-4.10,
now reported upstream and already fixed:
pyinstaller/pyinstaller#6338
pyinstaller/pyinstaller#6339
pyinstaller/pyinstaller#6686

So the pyinstaller commit pinned here is from the stable "v4" branch, some commits after the 4.10 tag,
which has fixes for the above issues.
@SomberNight SomberNight changed the title win build: bump python 3.9.10->3.9.11 win build: bump python, wine, pyinstaller Mar 20, 2022
@SomberNight
Copy link
Member Author

pyinstaller/pyinstaller#6686 is now fixed upstream.
I've tried with a pyinstaller commit from the stable "v4" branch, that has the fix, and the application starts correctly and looks ok.

@SomberNight SomberNight marked this pull request as ready for review March 20, 2022 17:12
@SomberNight SomberNight merged commit 14b600b into spesmilo:master Mar 20, 2022
@ecdsa
Copy link
Member

ecdsa commented Mar 21, 2022

great!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants