struct.error when try to add-data a .pb tensorflow model #4617
Comments
|
The problem will probably be the file size of your model, because PyInstaller only supports files up to 2 GB in an archive. The problem is already described in #3939.
|
|
Thanks @Chrisg2000 for your reply. After several attempts, I'm not still able to solve the problem. I've tried to change the As workaround, I have outsourced the .pb model. Btw, I'm very interested to the topic so I keep listening. Thanks |
|
@rorobertostring this is, as mentioned above, already known about. Maybe try compressing the |
|
Hi I have a same problem when trying to pack the exe with pytorch (model weights are small, 120 Kb) any ideas how to resolve this? |
But pytorch is some 7GB at the moment. You can try the experimental fix branch or just not use |
|
Oh, Is there an option to make it smaller/faster? Edit: I am not using --onefile |
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='app',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
console=True )That is a onefile build. Everything is being bundled into one big EXE.
You can try installing UPX which PyInstaller can use to compress everything with which will make it smaller but even slower. If you can, find an older version of pytorch which is a more sane size. |
But do note that there's also a 4 GiB executable size limit imposed by Windows itself (i.e., if executable is larger than that, it won't be recognized as executable at all). |
|
I have torch=1.1.0 and that's the lowest I can go. I have tried UPX packaging - took ages but the log says it was packed successfully, whereas I cannot open the .exe now(its 1.5 GB in total) |
|
Actually now I think about it, UPX decompresses DLLs into RAM and loads them directly from there which probably means those DLLs now eat 7-8GB of RAM which is likely more than you have room for... |
|
Just don't use onefile. It's a useless feature for anything large. |
|
should i just skip this EXE() part in my .spec and leave everything else as it is? |
|
Ahh no. Then you'll end up with nothing at all. Use: exe = EXE(pyz,
a.scripts,
[],
exclude_binaries=True,
name='app',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True )
coll = COLLECT(exe,
a.binaries,
a.zipfiles,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name='app') |
|
omg it worked, thank you so much! |
I'm getting the following error when I try to include (with
--add-data) my pb model to the executable.The command that I ran:
pyinstaller --add-data '/run/media/robert/ntfs-data/envs/equi_face/lib/python3.7/site-packages/astor/VERSION:astor' --paths '/run/media/robert/ntfs-data/Equirectangular image face detection/pkgs/' --hidden-import tensorflow --hidden-import tensorflow.lite.python.lite --hidden-import torch --hidden-import PIL.ExifTags --add-data 'yolov3/cfg/*:yolov3/cfg' --add-data 'yolov3/weights/*:yolov3/weights' --add-data 'yolov3/data/*:yolov3/data' --add-data 'tinyfaces/weights/best.pb:tiny/weights' --clean --onefile detection.pyThis error, occur if I run pyinsteller with
--one-fileoption, but it does not appear with the option--one-dir.As I said, the error occurs when i add
--add-data 'tinyfaces/weights/best.pb:tiny/weights'. If I remove this file it builds perfectly.More info about my configuration
The text was updated successfully, but these errors were encountered: