Pyinstaller for PyQt5 and Pyvis requires some manual copying of files.

[ariharan-v]

Issue:
I have created a network graph using PyVis and used PyQt5's QWebEngineView to show the html output created using PyVis. I used anaconda and spyder to develop the code and graph is shown in as a windows application. When I try to use Pyinstaller to convert the python code to binaries, the library contents of PyVis is not properly getting converted into binaries (the error it shows is "<path_to_binaries>/dist/<app_name>/pyvis/template.html is not found". The template.html file is inside the PyVis, so if I manually copy that entire package from virtual environment to the dist folder created by Pyinstaller, then it works. The second problem is with PyQt5; the QtWebEngineProcess.exe has to be copied to "<path_to_binaries>/dist/<app_name>/PyQt5/qt/bin" location. Otherwise the application fails.

Note: Earlier I tried to do with conda packages, but "mkl_intel_thread_1.dll" file missing error came. Then I created new virtual environment and used pip to install all the packages, then that error was taken care (as pip does not install mkl library when installing numpy).

System configuration:

• OS: Windows 10
• Python: 3.7.10
• Anaconda Navigator: 1.9.7
• Spyder: 5.0.3
• Pyinstaller: 4.3
• PyQt5: 5.15.4
• PyVis: 0.1.9

Code Snippet:

from PyQt5.QtWebEngineWidgets import QWebEngineView
from PyQt5.QtWidgets import QApplication
from PyQt5.QtCore import QUrl
from pyvis.network import Network
import networkx as nx
import sys
import os

nx_graph = nx.cycle_graph(10)
nx_graph.nodes[1]['title'] = 'Number 1'
nx_graph.nodes[1]['group'] = 1
nx_graph.nodes[3]['title'] = 'I belong to a different group!'
nx_graph.nodes[3]['group'] = 10
nx_graph.add_node(20, size=20, title='couple', group=2)
nx_graph.add_node(21, size=15, title='couple', group=2)
nx_graph.add_edge(20, 21, weight=5)
nx_graph.add_node(25, size=25, label='lonely', title='lonely node', group=3)
nt = Network('500px', '700px')
nt.from_nx(nx_graph)
nt.save_graph('nx.html')

app = QApplication(sys.argv)
browser = QWebEngineView()
file_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "nx.html"))
local_url = QUrl.fromLocalFile(file_path)
browser.load(local_url)
browser.show()

app.exec_()

Questions:

1. Am I doing any fundamental mistake?
2. Why there is a need for copying QtWebEngineProcess.exe manually?
3. Why the PyVis library is not getting compiled properly (not every file in PyVis library get converted as pyd file)?

[rokm]

When I try to use Pyinstaller to convert the python code to binaries, the library contents of PyVis is not properly getting converted into binaries (the error it shows is "<path_to_binaries>/dist/<app_name>/pyvis/template.html is not found". The template.html file is inside the PyVis, so if I manually copy that entire package from virtual environment to the dist folder created by Pyinstaller, then it works.

This is normal behavior. Data files (such as template.html) are not automatically picked up by PyInstaller, unless someone already wrote a hook that calls collect_data_files for that package.

Since PyInstaller 4.3, you can also use --collect-data pyvis as part of your pyinstaller command-line, which adds the corresponding collect_data_files call to your .spec file, and alleviates the need for the hook.

The second problem is with PyQt5; the QtWebEngineProcess.exe has to be copied to "<path_to_binaries>/dist/<app_name>/PyQt5/qt/bin" location. Otherwise the application fails.

That's probably because PyInstaller 4.3 does not yet support PyQt5/PyQtWebEngine 5.15.4 and its modified layout (#5631). If you want to use that version, you will have to use develop branch that has pyinstaller/pyinstaller#5811 merged. Or, you could downgrade to earlier version of PyQt5/PyQtWebEngine (e.g., 5.15.2).

Questions:

1. Am I doing any fundamental mistake?

Nope, you're running into usual PyInstaller issues: the data collection not being fully automatic + the fact that the fixes for PyQt5 5.15.4 are not part of release yet.

2. Why there is a need for copying QtWebEngineProcess.exe manually?

Explained above.

3. Why the PyVis library is not getting compiled properly (not every file in PyVis library get converted as pyd file)?

The library itself (its python modules) are probably getting collected just fine, and it is just the data files data are missing.

not every file in PyVis library get converted as pyd file

This is a red herring; if there are pyd files in a package, they get collected directly into program's dist folder, and you can see them when you inspect your bundle (e.g., with File Explorer). The pure-python (.py) modules, however, are compiled into .pyc files and collected into archive that's embedded in the executable (unless you build with --debug noarchive or --debug all). So you don't see them, but that doesn't mean they're (necessarily) missing.

[bwoodsend]

I guess then that that leaves this issue as a hook request for pyvis so this can scoot over to the hooks repo.

[ariharan-v]

Thanks @rokm and @bwoodsend. I will try with PyQt5 5.15.2 and use --collect-data pyvis. If there is a problem then I will try once with the recent source of pyinstaller.

[ariharan-v]

@rokm, I have tried your suggestions. Downgrading PyQt5 to 5.15.2 and use of --collect-data pyvis helped in resolving the error messages I have mentioned.

Issue 1

Discription:
I forgot to mention another workaround I have done to manage the PyQt issue. I had to make a file named qt.conf inside the dist folder. Without this file, the application throws the same "Could not find QtWebEngineProcess.exe" error message. After including this qt.conf file, it works fine.

Content of qt.conf file:

[Paths]
Prefix = PyQt5/Qt/

Question:

1. How to include this qt.conf automatically or what is the other way to resolve this problem so that I can make a single exe file?

Issue 2

Discription
After copying the qt.conf file, I moved the entire dist folder to another windows 10 machine to check if the executable works fine. When I run the program, it creates the nx.html file and opens the QT application window. It fails to show the graph. If I open the html file in any browser, the graph is shown properly. Even by running the application as administrator, the graph is not displayed in the application window. There is no specific error message on the console.

Note: In the same machine if I change the directory and run the program, it works fine. If I change to another machine it creates the problem. I have not created as a single file exe yet, so I can check that the nx.html file is created in the other system too.

Question:

1. Is there any additional library or pyinstaller to be installed in a system before running the exe made by py installer?

[rokm]

Issue 1

Discription:
I forgot to mention another workaround I have done to manage the PyQt issue. I had to make a file named qt.conf inside the dist folder. Without this file, the application throws the same "Could not find QtWebEngineProcess.exe" error message. After including this qt.conf file, it works fine.

Content of qt.conf file:

[Paths]
Prefix = PyQt5/Qt/

Question:

1. How to include this _qt.conf_ automatically or what is the other way to resolve this problem so that I can make a single exe file?

This should work out of the box. The qt.conf file should originally reside in site-packages\PyQt5\Qt\bin and automatically collected into dist\<name>\PyQt5\Qt\bin˙. It sets prefix to ..`.

Can you check if qt.conf file is a part of your PyQt5 installation?

Issue 2

Discription
After copying the qt.conf file, I moved the entire dist folder to another windows 10 machine to check if the executable works fine. When I run the program, it creates the nx.html file and opens the QT application window. It fails to show the graph. If I open the html file in any browser, the graph is shown properly. Even by running the application as administrator, the graph is not displayed in the application window. There is no specific error message on the console.

Note: In the same machine if I change the directory and run the program, it works fine. If I change to another machine it creates the problem. I have not created as a single file exe yet, so I can check that the nx.html file is created in the other system too.

Question:

1. Is there any additional library or pyinstaller to be installed in a system before running the exe made by py installer?

This is probably path issue. Your original code snippet is creating the nx.html file in current working directory (nt.save_graph('nx.html')), while the view is assuming it is located next to the executable file_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "nx.html")).

I have not created as a single file exe yet, so I can check that the nx.html file is created in the other system too.

Where is it created?

Can add an assert os.path.isfile(file_path) before trying to load the page, to see if the file you are trying to load is indeed where you think it is?

It fails to show the graph.

What exactly does this mean? Is the view window completely blank, is it showing a file-not-found error, or something else?

[ariharan-v]

@rokm, as you suggested earlier, I have downgraded PyQt5 to 5.15.2. The installation process showed PyQt5 is successfully installed and the QtWebEngineProcess.exe file is getting copied whenever I ran pyinstaller. After some attempts when I checked PyQt5 version using pip show pyqt5 and pip show pyqtwebengine, I understood some cache was not cleared properly due to which version conflicts were there. So I removed all Qt related libraries and reinstalled them.

This reinstallation solved my issue 1, and this also solved the issue 2.

When creating the binaries as a single file, the path issue comes, otherwise it was showing blank window. Now I suppose it is due to the cache issue with library installation. Both issues are resolved now.

Thanks a ton for your kind support!