Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a MerkleProof.processProof utility function #2841

Merged
merged 14 commits into from Oct 14, 2021
1 change: 1 addition & 0 deletions CHANGELOG.md
Expand Up @@ -10,6 +10,7 @@
* `Governor`: shift vote start and end by one block to better match Compound's GovernorBravo and prevent voting at the Governor level if the voting snapshot is not ready. ([#2892](https://github.com/OpenZeppelin/openzeppelin-contracts/pull/#2892))
* `PaymentSplitter`: now supports ERC20 assets in addition to Ether. ([#2858](https://github.com/OpenZeppelin/openzeppelin-contracts/pull/#2858))
* `ECDSA`: add a variant of `toEthSignedMessageHash` for arbitrary length message hashing. ([#2865](https://github.com/OpenZeppelin/openzeppelin-contracts/pull/#2865))
* `MerkleProof`: add a `processProof` function that returns the rebuilt root hash, and a unique leaf indentifier, given a leaf and a proof. ([#2841](https://github.com/OpenZeppelin/openzeppelin-contracts/pull/2841))

## 4.3.2 (2021-09-14)

Expand Down
4 changes: 4 additions & 0 deletions contracts/mocks/MerkleProofWrapper.sol
Expand Up @@ -12,4 +12,8 @@ contract MerkleProofWrapper {
) public pure returns (bool) {
return MerkleProof.verify(proof, root, leaf);
}

function processProof(bytes32[] memory proof, bytes32 leaf) public pure returns (bytes32, uint256) {
return MerkleProof.processProof(proof, leaf);
}
}
29 changes: 24 additions & 5 deletions contracts/utils/cryptography/MerkleProof.sol
Expand Up @@ -23,21 +23,40 @@ library MerkleProof {
bytes32 root,
bytes32 leaf
) internal pure returns (bool) {
bytes32 computedHash = leaf;
// Check if the computed hash (root) is equal to the provided root
(bytes32 computedHash, ) = processProof(proof, leaf);
return computedHash == root;
}

/**
* @dev Returns the rebuilt hash obtained by traversing a Merklee tree up
* from `leaf` using `proof`, and an index uniquelly identifying the leaf
Amxx marked this conversation as resolved.
Show resolved Hide resolved
* location in the tree. A `proof` is valid if and only if the rebuilt hash
* matches the root of the tree. When processing the proof, the pairs of
* leafs & pre-images are assumed to be sorted.
* The produced index is unique in the sens that processing two valid proofs
Amxx marked this conversation as resolved.
Show resolved Hide resolved
* will return the same indices if and only if the leaf at the same location
Amxx marked this conversation as resolved.
Show resolved Hide resolved
* in the tree. This helps distinguishing two leaves that have the same
* bytes32 identifier but are present in different locations in the merkle
* tree.
*
* _Available since v4.4._
*/
function processProof(bytes32[] memory proof, bytes32 leaf) internal pure returns (bytes32, uint256) {
bytes32 computedHash = leaf;
uint256 index = 0;
for (uint256 i = 0; i < proof.length; i++) {
bytes32 proofElement = proof[i];

index <<= 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do index * 2 for overflow protection?

Suggested change
index <<= 1;
index *= 2;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Amxx If this suggestion isn't relevant can you explain why?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally cared mostly about consistency between line 50 and 57, but the overflow protection is really interesting.

  • if we use "multiply", we protect against overflow that would happen if there is a right-branching at least 256 down in the tree. For that to happen, the Merkle tree would have to be more than 256 levels depth, which is only required if the tree contains more than 2**256 leaves. (which means there much be duplicates or collisions in the leaves
  • if we use "shift", we don't have this protection.

This protection is technically a breaking changes because some proofs won't be verifiable anymore. These proofs will only realistically happen if trees are not built properly. Without this protection, the returned index value becomes unreliable if trees have more then 256 levels ...

My conclusion

  • In all reasonable cases, adding the protection should have no positive impact on security, and just add some gas cost.
  • In some corner cases, the protection will prevent verifying valid proofs, that were accepted before this PR.
  • I'd go for using shift.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the Merkle tree would have to be more than 256 levels depth, which is only required if the tree contains more than 2**256 leaves

Are trees necessarily balanced?

Copy link
Collaborator Author

@Amxx Amxx Oct 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contract do not require any balancing, but effectiveness of the merkle structure is optimized if the most frequently proved leaves are higher... In practice, this means that any good tooling will try to balance the tree ... and I don't expect we would ever have production trees that are so obviously unbalanced

if (computedHash <= proofElement) {
// Hash(current computed hash + current element of the proof)
computedHash = keccak256(abi.encodePacked(computedHash, proofElement));
} else {
// Hash(current element of the proof + current computed hash)
computedHash = keccak256(abi.encodePacked(proofElement, computedHash));
index |= 1;
}
}

// Check if the computed hash (root) is equal to the provided root
return computedHash == root;
return (computedHash, index);
}
}
20 changes: 20 additions & 0 deletions test/utils/cryptography/MerkleProof.test.js
Expand Up @@ -56,4 +56,24 @@ contract('MerkleProof', function (accounts) {
expect(await this.merkleProof.verify(badProof, root, leaf)).to.equal(false);
});
});

describe('processProof', function () {
it('create unique indices', async function () {
const elements = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/='.split('');
const merkleTree = new MerkleTree(elements, keccak256, { hashLeaves: true, sortPairs: true });

const root = merkleTree.getHexRoot();

const results = await Promise.all(elements
.map(element => keccak256(element))
.map(leaf => this.merkleProof.processProof(merkleTree.getHexProof(leaf), leaf)),
);

// All reconstructed roots are correct
expect(results.map(result => result[0])).to.have.members(Array(elements.length).fill(root));

// Indices are unique
expect(results.map(result => result[1].toNumber()).every((index, i, indices) => indices.indexOf(i) === index));
});
});
});