Set correct PE checksum for .exe files (helps with virus false-positives)

[The-Compiler]

Is your feature request related to a problem? Please describe.
As you've probably all seen before, virus scanner false-positives are a recurring problem for PyInstaller-generated executables, as well as PyInstaller's bootloader (see e.g. #5490, #5474, #5479, ...).

One piece of information virus scanners seem to use is the PE checksum. Quoting from that article:

• 83% of malware had invalid checksums
• 90% of legitimate files had valid checksums.

It looks like anything generated from PyInstaller (as well as the Windows bootloader in the release?) seems to have a checksum of 00 00 00 00, which is thus invalid.

Describe the solution you'd like
PyInstaller should set a correct PE checksum. This is possible using pefile, which it already seems to depend on:

import pefile
pe = pefile.PE('filename.exe')
pe.OPTIONAL_HEADER.CheckSum = pe.generate_checksum()
pe.close()
pe.write('filename.exe')

(You can use pe.verify_checksum() to verify that it is correct)

After doing this, e.g. PyInstaller's Windows-64bit/runw.exe (in the 4.2 release) goes down from 12 false-positives to only 5 - most notably, McAfee and Avast (which are the two affected engines I've actually heard of before...) now are happy with the file.

Doing this with my own project (which is built with PyInstaller), @bitraid found that:

the number of false positives for v2.0.2 goes down from 31 to 7. Similarly v2.0.0 goes down to 6 from 29 and v2.0.1 goes down to 8 from 15.

Describe alternatives you've considered
Submitting false-positive reports to antivirus vendors. A pain.

Also, doing this after PyInstaller has generated the exe - that works, but it seems to me like this should be done in PyInstaller itself.

Additional context
See qutebrowser/qutebrowser#6194

[bwoodsend]

Ah yes! Nice one. How that makes an executable any more trustworthy seeing as anyone could do it is beyond me but if it shuts up the bogus AV software then great.

[bwoodsend]

@The-Compiler Fancy making the changes or shall I?

[The-Compiler]

Feel free! I don't think I've contributed before (looks like I did 😆)

I'm not really sure what the best place for such a change would be - but I also try to avoid working on Windows stuff if I can, so if you don't mind, go ahead. 👍

[bwoodsend]

Alright then. I'll blow the cobwebs off my own Windows partition and do it...

[andyshinn]

I've been following the virus false-positive stuff for a while and struggling with it myself. Here are the before VirusTotal Windows bootloader analysis so we can compare after. The bigger one I have issues with (that I am mot sure the source for) is Windows Defender. Will be interesting to see if this helps that case.

Windows 32bit:

• run_d.exe 1/70: https://www.virustotal.com/gui/file/5c8ddb668b8fa768d2037879a22214db104e48cf26136cc311e159f007e6436e/detection
• run.exe 11/70: https://www.virustotal.com/gui/file/2e4dc3fd4cb960cdc0dd0c8abc176fe80209dbf353c789e79dd0f271ad02a785/detection
• runw_d.exe 2/70: https://www.virustotal.com/gui/file/e70c7cfb63a9ca41c55cbca60ff7acd521b67606ab33ccf2661400e876268c7a/detection
• runw.exe 14/70: https://www.virustotal.com/gui/file/ddab59c787d9673a8098a1338098effeedca938dc338e96c5fd4654ed0b76dfa/detection

Windows 64bit:

• run_d.exe 2/71: https://www.virustotal.com/gui/file/8ef748a59cd81d99b0ee3f5008c2275b5630d5bf8e1aecc2ed7b8a288bea21d7/detection
• run.exe 4/70: https://www.virustotal.com/gui/file/0367cd6d43be86912de0f30dd118087ac9e79a9b5f7440e496703b1de4c44fd7/detection
• runw_d.exe 5/70: https://www.virustotal.com/gui/file/c27d3749d3614e1b3022c84ff240fbcb3605ba6b72fcc1cf9729088fc4345ccc/detection
• runw.exe 12/71: https://www.virustotal.com/gui/file/749666ba5af516692ff3716000b992cbcf2857e825c9c1d94fac7ac8ba11759e/detection

[bwoodsend]

I've done it (#5580). EXEs built by PyInstaller will now have the checksum set. The bootloaders won't until we rebuild them so unless you rebuild the bootloaders yourself, there is no point submitting those to virustotal yet. Instead submit your built applications. I have tried rebuilding a bootloader then submitting it and it came back all green.

[sjackman]

The Internet Archive flagged Pyinstaller as malware: https://www.virustotal.com/gui/file/af5a1b117ea14224d5993b74b45a42f8aab0a3230e1c81dc5a00d1a27ebe631c/detection/f-af5a1b117ea14224d5993b74b45a42f8aab0a3230e1c81dc5a00d1a27ebe631c-1614111479

[andyshinn]

I tried with the new PE changes and it's still blocked in Defender and flagged by a bunch in Virus Total, unfortunately.

https://www.virustotal.com/gui/file/a70a7c7b5682abe70dfe7bd186e6f48b636e2a59908bcbdfc0184a4e52cc1635/detection

[Legorooj]

Interesting, using commit f078351, I get the following result:

https://www.virustotal.com/gui/file/f5587397e6ce11d659e3b4cfe1637991b9b5547a3f280873ef48948e606f7434/detection

Just a build of print('Hello world!') as a onefile was what I used.

[Legorooj]

@sjackman that's Internet Archive flag is on version 3.6... a little out of date.

[srkn]

I've done it (#5580). EXEs built by PyInstaller will now have the checksum set. The bootloaders won't until we rebuild them so unless you rebuild the bootloaders yourself, there is no point submitting those to virustotal yet. Instead submit your built applications. I have tried rebuilding a bootloader then submitting it and it came back all green.

Would you please give a little more detail regarding this? How should we rebuild bootloader to prevent being flagged as malware?

[bwoodsend]

Rebuilding the bootloader only affects people who can't install pyinstaller. If that includes you then the docs are here.