Batch verification for range proofs #86

oleganza · 2018-05-03T20:28:01Z

Overview

This adds batch verification to any number of range-proofs. The API allows mixing proofs with different sizes (in terms of both range size n and aggregation size m), and checks that the generators provided are enough to cover max(n*m).

Addresses #22 and replaces older PR #27.

Performance

Performance depends on Pippenger implementation (not merged yet): dalek-cryptography/curve25519-dalek#129

This PR does not have benchmarks for the batched verification yet.

The existing verify method is not implemented via generalized batch verification to avoid code duplication. This should add negligible overhead, but I haven't run the benchmarks since previous version #27 (where I did not find any statistically meaningful difference).

Add a random oracle API

Add scalar batch invert

Calculate t using l0, l1, r0, r1

@oleganza

@oleganza pointed out that when the dealer's polynomial challenge `x` is zero, the parties will leak secrets, and suggested this check. Informal analysis of the protocol flow: dealer -> party: position party -> dealer: `ValueCommitment` The `ValueCommitment` cannot leak information to the dealer, since it's blinded independently of any dealer messages dealer -> party: `ValueChallenge` Contains `y, z` challenges. party -> dealer: `PolyCommitment` Contains `T_1`, `T_2`, which are blinded independently from any dealer messages and therefore can't leak information dealer -> party: `PolyChallenge` Contains `x` challenge party -> dealer: `ProofShare` Up till now, we know that the each of the party's messages can't reveal info because they're blinded independently of the dealer messages. The paper notes that the blindings for the `l` and `r` vectors are chosen such that the prover can reveal `l(x), r(x)` for one challenge point `x \in \ZZ_p^{\times}` without revealing information, but if `x` is zero, then `\blinding t(x)` is just `z^2 \blinding v`, so the dealer could multiply by `z^2` and recover the blinding for the original commitment. However, if the party checks that `x` is nonzero, then `x \in \ZZ_p^{\times}`, the blinding factors are not annihilated, and the proof share does not leak information. In the previous (non-MPC) version of the code, the `x` was computed by the prover out of the proof transcript, and so it wasn't necessary to check that `x` is nonzero (which occurs with probability `~ 2^{-252}`): as noted by AGL, the instructions required to check this are more likely to fail than the check itself. Another concern is about replay attacks: a malicious dealer who was able to get the party to apply the same `PolyChallenge` multiple times could use polynomial interpolation to recover the coefficients of the `t` polynomial and reveal the party's secrets. (I think three points would be sufficient). However, because our encoding of the protocol flow into affine types ensures that each `Party` state can be used at most once, we are ensured that any compilable instantiation of the MPC protocol is invulnerable to replay attacks, since typechecking the program requires the compiler to prove that states are not reused.

Prevent a malicious dealer from retrieving the party's secrets

hdevalence · 2018-05-07T21:49:44Z

Instead of moving to a new verification module, I think it would be better to keep all of this as part of the rangeproof.

Currently, the Verification struct holds a bunch of temporary data. I'm wondering whether we can get away with not holding any temporaries, and have the verification struct act as a view into the proof data, computing things on the fly.

Related to the above is something we delayed thinking about earlier: whether the proof data should have compressed points, so that we don't have to decompress during deser and then recompress to hash during verification. This could impact whether we have to allocate temporaries, but I don't think we made a ticket for this.

oleganza · 2018-05-07T22:00:32Z

Related to the above is something we delayed thinking about earlier: whether the proof data should have compressed points, so that we don't have to decompress during deser and then recompress to hash during verification. This could impact whether we have to allocate temporaries, but I don't think we made a ticket for this.

@hdevalence Does it depend on upstream support for deserializing compressed points? Do we need to get creative and have DecompressedRistretto type that holds both compressed bytes (for fast compression) and the decompressed point? So that we can verify point validity during deserialization and also avoid extra work?

hdevalence · 2018-05-07T22:06:01Z

We could just use compressed points in the struct, and call .map(|p_bytes| p.decompress()), with some extra iterator combinators to handle the None case.

oleganza · 2018-05-08T22:27:55Z

I rewrote the verification so that API does not expose the helper object with temporaries.

hdevalence · 2018-05-10T18:08:31Z

Just wondering, what's the motivation for making the batch verification API mix all kinds of proofs in a single batch, instead of requiring that the batch contains only the same type of proof?

I'm worried that there's additional complexity and room for mistakes (both in the implementation and in the code that uses it) by mixing proof types, and I'm also not sure what kind of use-case that would be important for. Even supposing you had a protocol with different kinds of proofs, you could alternately create a batch of proofs of type 1, a batch of proofs of type 2, etc.

cathieyun · 2018-05-10T18:27:38Z

The code looks good, but I am also wondering the same as above ^ - if we don't mix the different proof sizes, it seems like a some of the harder-to-follow logic can be removed (eg figuring out the max generator sizes, the split of n*m into n*m, 1, padding proofs).

oleganza · 2018-05-10T19:19:27Z

I think the mix of differently-sized (m) aggregated proofs is very realistic, while a mix of differently-sized ranges (n) is very unlikely.

Why varying m is very likely: bulletproofs create an incentive to aggregate proofs both for better privacy (coinjoin) and minimized cost. The process is interactive and we can expect all sorts of differently sized aggregations in the wild. Since all we care is n*m, supporting varying ns comes at no cost if we support varying ms. Batching all proofs in one giant expression instead of a grouping them by smaller ones should give non-trivial performance advantage since Pippenger scales sub-linearly and something hypothetical like "verification on a GPU" wants large amount of work and has higher-than-trivial latency to set up (per proof).

That said, I think the PR wouldn't be noticeably simpler if we fixed m and n. We'd still have to collect verifications scalars in vecs, and verification objects in one single vector because m wouldn't be statically defined anyway and we need to iterate over per-proof data several times: (1) combining scalars on static bases and (2,3) concatenating dynamic bases and scalars.

Re: n*m,1 - maybe we should simplify the generators API to take a single usize which will be filled in by n*m in the RP protocol?

hdevalence · 2018-05-10T21:15:20Z

The point about varying aggregation sizes (m) is convincing, and that seems like something we should support in the future.

But as things are now, I think it actually violates the contract of the Generators struct, which contains generators for aggregated range proofs of size (m,n). That API isn't designed for varying m, and I think that the awkwardness about doing ad-hoc m*ns etc., is really a symptom of trying to use the Generators in a way that it wasn't designed for. If we would like to support varying m, I think we should redesign the Generators API around the idea that m can vary, and then update the rest of the code accordingly.

hdevalence · 2018-05-10T21:19:15Z

src/range_proof/mod.rs

+    /// Proofs may use different ranges (`n`) or different number of aggregated commitments (`m`).
+    /// You must provide big enough view into generators (`gens`) that covers
+    /// the biggest proof
+    pub fn verify_batch<'a,'b,I,R,P,V>(


I don't see where 'a, 'b are used?

hdevalence · 2018-05-10T21:20:50Z

src/range_proof/mod.rs

+    /// You must provide big enough view into generators (`gens`) that covers
+    /// the biggest proof
+    pub fn verify_batch<'a,'b,I,R,P,V>(
+        proofs: I,


I think it would be much simpler if this just took an iterator of (&RangeProof, &[RistrettoPoint]).

We could also consider using a type alias to make the roles more clear in the signature, or have the function take two iterators (one for proofs and the other for commitments).

hdevalence · 2018-05-10T21:22:56Z

src/range_proof/mod.rs

+
+        // First statement is used without a random factor
+        let mut pedersen_base_scalars: (Scalar, Scalar) = (Scalar::zero(), Scalar::zero());
+        let mut g_scalars: Vec<Scalar> = iter::repeat(Scalar::zero()).take(nm).collect();


let mut g_scalars = vec![Scalar::zero(), nm];

hdevalence · 2018-05-10T21:28:37Z

src/range_proof/mod.rs

+            );
+        }
+
+        // First statement is used without a random factor


IMO it is cleaner not to unroll the first loop iteration, and just multiply all statements by a random factor. The code is simpler, the additional cost is minimal (and shrinks as the batch size grows).

Ouch, that's a stale comment. The first iteration is no longer unrolled.

hdevalence · 2018-05-10T21:32:54Z

src/range_proof/mod.rs

+/// Represents a deferred computation to verify a single rangeproof.
+/// Multiple instances can be verified more efficient as a batch using
+/// `RangeProof::verify_batch` function.
+struct Verification {


In the actual batchverify code above, each of these fields are used only once in the inner loop accumulating the batched verification scalars.

So, instead of having a seperate Verification struct that contains owned copies of the verification scalars for a given RangeProof, why not just have functions on the RangeProof struct that return the iterators for each of the fields of what is currently the Verification struct?

goldenMetteyya · 2019-10-10T23:26:58Z

Hi @oleganza what's the status of this PR? One suggestion is to have two apis one for same 'm' and one for a variable one.

cathieyun and others added 30 commits February 1, 2018 17:40

create readme

98b2c9a

Fix merge conflict whoops

223bf90

Added comments, some more progress on generating t

e745b83

add helper functions

327d21b

syntax fix

f1087fc

Generate t1, t2

79a701a

fmt

d1a385d

generate x

242e5c6

Output range proof

5c096e3

generate l, r

75bd9f1

Use the crates.io version of dalek, now that it's released.

58e3b24

Add tests

cae7675

Merge branch 'master' of github.com:chain/ristretto-bulletproofs

1c25edd

Starting verification

78d6e26

messy but working proof

df128e3

implement last verification check

481ac92

Switch to efficient proof gen

e0577a3

speeding up verification

03ccaf8

it compiles now

af66f59

added fiatshamir api

84eade7

Feature: random oracle API (#2)

a57bee5

Add a random oracle API

Remove alternative ways of calculating t0, t1, t2

103e9e7

Adding scalar invert func & test

aa9cf89

move range proofs to a separate file (#3)

02dc9b5

fix merge conflict

dad7d59

added testfile

bbffac1

Merge pull request #4 from chain/scalar-batch-invert

1a17370

Add scalar batch invert

merge testfile

e30ddf0

Change calculation for t0, t1, t2

b142b05

Merge pull request #5 from chain/karatsuba

ce9dd2a

Calculate t using l0, l1, r0, r1

hdevalence added 2 commits May 4, 2018 11:52

Merge pull request #87 from chain/ensure-evaluation-point-is-nonzero

1dfa781

Prevent a malicious dealer from retrieving the party's secrets

oleganza added 5 commits May 7, 2018 14:57

generalize and factor out batch verification of range proofs

04a885a

allow rangeproofs with different n,m to be batched

ed6b0f2

refactor batch verification tests

ae0239f

make sure Verification type is not dropped w/o verification

87fc9f5

cargo fmt

44f23d9

oleganza force-pushed the oleg/batch-verify branch from ec27503 to a5bb572 Compare May 7, 2018 21:57

oleganza added 6 commits May 8, 2018 14:16

simplify impl

692cedc

move test_delta back

647a421

debugging batch logic

b7ceab5

bug fix: take only necessary number of generator points

d33dc23

remove obsolete test

7e61002

remove unnecessary vec allocations

7295a16

hdevalence reviewed May 10, 2018

View reviewed changes

cathieyun force-pushed the oleg/batch-verify branch from 6883c28 to 7295a16 Compare July 3, 2018 21:59

oleganza mentioned this pull request Jan 30, 2019

Batch PointOp verification interstellar/zkvm#11

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Batch verification for range proofs #86

Batch verification for range proofs #86

oleganza commented May 3, 2018 •

edited

hdevalence commented May 7, 2018

oleganza commented May 7, 2018

hdevalence commented May 7, 2018

oleganza commented May 8, 2018

hdevalence commented May 10, 2018

cathieyun commented May 10, 2018

oleganza commented May 10, 2018 •

edited

hdevalence commented May 10, 2018

hdevalence May 10, 2018

hdevalence May 10, 2018

hdevalence May 10, 2018

hdevalence May 10, 2018 •

edited

hdevalence May 10, 2018

oleganza May 10, 2018

hdevalence May 10, 2018

goldenMetteyya commented Oct 10, 2019

Batch verification for range proofs #86

Are you sure you want to change the base?

Batch verification for range proofs #86

Conversation

oleganza commented May 3, 2018 • edited

hdevalence commented May 7, 2018

oleganza commented May 7, 2018

hdevalence commented May 7, 2018

oleganza commented May 8, 2018

hdevalence commented May 10, 2018

cathieyun commented May 10, 2018

oleganza commented May 10, 2018 • edited

hdevalence commented May 10, 2018

hdevalence May 10, 2018

Choose a reason for hiding this comment

hdevalence May 10, 2018

Choose a reason for hiding this comment

hdevalence May 10, 2018

Choose a reason for hiding this comment

hdevalence May 10, 2018 • edited

Choose a reason for hiding this comment

hdevalence May 10, 2018

Choose a reason for hiding this comment

oleganza May 10, 2018

Choose a reason for hiding this comment

hdevalence May 10, 2018

Choose a reason for hiding this comment

goldenMetteyya commented Oct 10, 2019

oleganza commented May 3, 2018 •

edited

oleganza commented May 10, 2018 •

edited

hdevalence May 10, 2018 •

edited