Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support AES decryption #203

Merged
merged 38 commits into from Jan 30, 2022
Merged

Support AES decryption #203

merged 38 commits into from Jan 30, 2022

Conversation

mbr
Copy link
Contributor

@mbr mbr commented Oct 19, 2020

This PR adds support for decrypting AES encrypted zip archives (compression method 99). It implements the special non-NIST variant of the CTR mode used by WinZip, but otherwise uses off-the-shelf crypto crates.

The feature itself is hidden behind an aes feature flag. We will leave it up to upstream to enable or disable it and/or mention it in the docs.

Encryption can be added if desired, as the decryption methods can be used to encryption as well.

@zamazan4ik
Copy link
Contributor

@mbr can you please fix the conflicts? :) Then, I hope, such useful feature will be merged.

@Plecra Plecra mentioned this pull request May 29, 2021
@mbr
Copy link
Contributor Author

mbr commented Jul 26, 2021

@mbr can you please fix the conflicts? :) Then, I hope, such useful feature will be merged.

I think this might be a job for @Lireer :)

@Lireer
Copy link
Contributor

Lireer commented Aug 6, 2021

All conflicts have been resolved. The MSRV is now 1.42, which shouldn't be much of a problem considering the current stable version is 1.54. 1.41 would be possible too, if we remove the two uses of the matches! macro.

@Lireer
Copy link
Contributor

Lireer commented Jan 25, 2022

The branch is up-to-date with master and could be merged. I added some simple tests which test the different AES encryption schemes (128/192/256) and whether they work with compressed and uncompressed files.

Benchmarks would be quite interesting but would require some archive to be decrypted. That would mean either implementing encryption, but I don't know enough about cryptography to give any guarantees about security, or including a benchmark file.
Here some results of local benchmarks:

encryption compression file size MB/s
none none 1 MiB 6744
none deflate 1 MiB 1340
none bzip2 1 MiB 15
AES128 none 1 MB 37
AES128 deflate 1 MB 42
AES256 none 1 MB 29
AES256 deflate 1 MB 33
AES256 none 10 MB 29
AES256 deflate 10 MB 33

Edit: Updating all aes-crypto dependencies seems to have increased the performance by quite a bit, AES128 with deflate increased from 42 MB/s to around 170 MB/s on my machine.

Cargo.toml Outdated
@@ -11,11 +11,16 @@ Library to support the reading and writing of zip files.
edition = "2018"

[dependencies]
flate2 = { version = "1.0.0", default-features = false, optional = true }
time = { version = "0.3", features = ["formatting", "macros" ], optional = true }
aes = { version = "0.6.0", optional = true }

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about using the latest aes version 0.7.5?
There seems to be breaking changes in the API.

Thank you for the hard work!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried updating aes, pbkdf2, hmac and sha-1 (now sha1) together since they all use the same dependencies. Since this required some small changes to the decryption code, I also put in some more (possibly meaningless) checks.

This increased the performance of AES-128 from 42 MB/s to around 160 - 180 MB/s.

@zamazan4ik
Copy link
Contributor

Just approved for running the CI

@Lireer
Copy link
Contributor

Lireer commented Jan 27, 2022

The new commit changes the name of sha-1 to sha1. Both point to the same crate, but sha1 recently changed ownership to RustCrypto, see here. sha1 will continue to be developed and supported, while sha-1 will be deprecated with the next minor version update.

@zamazan4ik
Copy link
Contributor

@Lireer please ping me or @Plecra when you will finish the work. By the way, maybe a more convenient way for discussing the PR will be our new Discord server - https://discord.gg/rQ7H9cSsF4 ? :)

@Lireer
Copy link
Contributor

Lireer commented Jan 27, 2022

@zamazan4ik I consider this PR finished in the sense that it does what it's supposed to do, add aes decryption support. Things like benchmarks, aes encryption, and possible performance improvements can be added in other PRs.

src/compression.rs Outdated Show resolved Hide resolved
src/compression.rs Outdated Show resolved Hide resolved
@zamazan4ik
Copy link
Contributor

@Plecra if you have no additional comments (and I am really not sure regarding my comments - maybe it's intended design to not move these values under the feature), I want to merge this PR.

@zamazan4ik
Copy link
Contributor

LGTM

Copy link
Member

@Plecra Plecra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks a ton for the work on this!

src/aes.rs Outdated
&vec![0; <Hmac<Sha1> as KeySizeUser>::KeySize::to_usize()],
)),
);
let computed_auth_code = &hmac.finalize().into_bytes()[0..AUTH_CODE_LENGTH];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use finalize_reset here to make this simpler to read

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before I realized that the Reset trait is now feature-gated, this was the only way to make it work. After I realized that finalize_reset can still be used I left it like this, because I thought the Hmac containing the password would be dropped earlier like this. But I guess it probably doesn't improve security at all, so making it more readable is more important.

src/aes_ctr.rs Outdated
/// XORs a slice in place with another slice.
#[inline]
fn xor(dest: &mut [u8], src: &[u8]) {
debug_assert_eq!(dest.len(), src.len());
Copy link
Member

@Plecra Plecra Jan 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth exploring how the compiler optimizes this if we make it an assert_eq

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't seem to affect performance all that much. I will try with criterion if that makes it more noticable. Bencher runs report 2130 - 2160 MB/s with debug_assert_eq and 2120 - 2130 MB/s with assert_eq. Note the over 2000 MB/s, which is substantially more than the previously reported 170 MB/s. Differences to previous benchmarks are: Windows instead of NixOs and a newer CPU (probably the reason). The other benchmark runs about 50% faster too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like badly written benchmark code was the real reason for the performance improvement. But the criterion benchmarks with high sample sizes don't show any significant changes when decrypting a ZipFile.

src/aes_ctr.rs Outdated
// `7z a -phelloworld -mem=AES256 -mx=0 aes256_40byte.zip 40byte_data.txt`
#[test]
fn crypt_aes_256_0_byte() {
let ciphertext = [];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The common part of these tests should be in its own function. Probably roundtrips(&[], b"", [0x0b, ...])

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

src/compression.rs Outdated Show resolved Hide resolved
@zamazan4ik
Copy link
Contributor

zamazan4ik commented Jan 30, 2022

@Lireer if you want - please resolve the comments from @Plecra in this PR, and I will merge it. If not - I think we can do it in another PR. And thanks for the so significant work!

@Lireer
Copy link
Contributor

Lireer commented Jan 30, 2022

@zamazan4ik @Plecra thanks for the reviews, if you don't have any other comments the PR should be ready to be merged.

@zamazan4ik zamazan4ik merged commit 1cd39fb into zip-rs:master Jan 30, 2022
@Plecra
Copy link
Member

Plecra commented Jan 30, 2022

ack! my bad for not being clearer, but we do need to remove CompressionMethod::Aes completely. I'll make a new issue for it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants