Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a double ended queue #3153

Merged
merged 21 commits into from Feb 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Expand Up @@ -5,6 +5,7 @@
* `AccessControl`: add a virtual `_checkRole(bytes32)` function that can be overridden to alter the `onlyRole` modifier behavior. ([#3137](https://github.com/OpenZeppelin/openzeppelin-contracts/pull/3137))
* `EnumerableMap`: add new `AddressToUintMap` map type. ([#3150](https://github.com/OpenZeppelin/openzeppelin-contracts/pull/3150))
* `ERC1155`: Add a `_afterTokenTransfer` hook for improved extensibility. ([#3166](https://github.com/OpenZeppelin/openzeppelin-contracts/pull/3166))
* `DoubleEndedQueue`: a new data structure that supports efficient push and pop to both front and back, useful for FIFO and LIFO queues. ([#3153](https://github.com/OpenZeppelin/openzeppelin-contracts/pull/3153))

## 4.5.0 (2022-02-09)

Expand Down
58 changes: 58 additions & 0 deletions contracts/mocks/DoubleEndedQueueMock.sol
@@ -0,0 +1,58 @@
// SPDX-License-Identifier: MIT

pragma solidity ^0.8.0;

import "../utils/structs/DoubleEndedQueue.sol";

// Bytes32Deque
contract Bytes32DequeMock {
using DoubleEndedQueue for DoubleEndedQueue.Bytes32Deque;

event OperationResult(bytes32 value);

DoubleEndedQueue.Bytes32Deque private _vector;

function pushBack(bytes32 value) public {
_vector.pushBack(value);
}

function pushFront(bytes32 value) public {
_vector.pushFront(value);
}

function popFront() public returns (bytes32) {
bytes32 value = _vector.popFront();
emit OperationResult(value);
return value;
}

function popBack() public returns (bytes32) {
bytes32 value = _vector.popBack();
emit OperationResult(value);
return value;
}

function front() public view returns (bytes32) {
return _vector.front();
}

function back() public view returns (bytes32) {
return _vector.back();
}

function at(uint256 i) public view returns (bytes32) {
return _vector.at(i);
}

function clear() public {
_vector.clear();
}

function length() public view returns (uint256) {
return _vector.length();
}

function empty() public view returns (bool) {
return _vector.empty();
}
}
2 changes: 2 additions & 0 deletions contracts/utils/README.adoc
Expand Up @@ -88,6 +88,8 @@ Note that, in all cases, accounts simply _declare_ their interfaces, but they ar

{{EnumerableSet}}

{{DoubleEndedQueue}}

{{Checkpoints}}

== Libraries
Expand Down
165 changes: 165 additions & 0 deletions contracts/utils/structs/DoubleEndedQueue.sol
@@ -0,0 +1,165 @@
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.4;

import "../math/SafeCast.sol";

/**
* @dev A sequence of items with the ability to efficiently push and pop items (i.e. insert and remove) on both ends of
* the sequence (called front and back). Among other access patterns, it can be used to implement efficient LIFO and
* FIFO queues. Storage use is optimized, and all operations are O(1) constant time. This includes {clear}, given that
* the existing queue contents are left in storage.
*
* The struct is called `Bytes32Deque`. Other types can be cast to and from `bytes32`. This data structure can only be
* used in storage, and not in memory.
* ```
* DoubleEndedQueue.Bytes32Deque queue;
* ```
*
* _Available since v4.6._
*/
library DoubleEndedQueue {
/**
* @dev An operation (e.g. {front}) couldn't be completed due to the queue being empty.
*/
error Empty();

/**
* @dev An operation (e.g. {at}) could't be completed due to an index being out of bounds.
*/
error OutOfBounds();

/**
* @dev Indices are signed integers because the queue can grow in any direction. They are 128 bits so begin and end
* are packed in a single storage slot for efficient access. Since the items are added one at a time we can safely
* assume that these 128-bit indices will not overflow, and use unchecked arithmetic.
*
* Struct members have an underscore prefix indicating that they are "private" and should not be read or written to
* directly. Use the functions provided below instead. Modifying the struct manually may violate assumptions and
* lead to unexpected behavior.
*
* Indices are in the range [begin, end) which means the first item is at data[begin] and the last item is at
* data[end - 1].
*/
struct Bytes32Deque {
int128 _begin;
int128 _end;
mapping(int128 => bytes32) _data;
}

/**
* @dev Inserts an item at the end of the queue.
*/
function pushBack(Bytes32Deque storage deque, bytes32 value) internal {
int128 backIndex = deque._end;
deque._data[backIndex] = value;
unchecked {
deque._end = backIndex + 1;
}
Copy link
Collaborator Author

@Amxx Amxx Feb 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, the previous implementation

unchecked {
     deque.data[deque.end++] = value;
}

was way more easy to read/understand (same argument apply to all push and pop functions)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre/post inc/decrements are quick to read, but definitely not easy...

The statement deque.data[deque.end++] = value is too dense: there are two different storage updates, and it's not immediately clear what value is used to index the data mapping.

I tried a few alternatives and ended up going with the current option. I think this code is quite ok:

int128 backIndex = deque.end;
deque.data[backIndex] = value;
deque.end = backIndex + 1;

The unchecked makes it harder to read. Perhaps if we put everything in the unchecked block it would look better?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely think we want the +1 unchecked.

Putting everything in the unchecked block might make it cleaner to read (all the logic is indented at the same level)

}

/**
* @dev Removes the item at the end of the queue and returns it.
*
* Reverts with `Empty` if the queue is empty.
*/
function popBack(Bytes32Deque storage deque) internal returns (bytes32 value) {
if (empty(deque)) revert Empty();
int128 backIndex;
unchecked {
backIndex = deque._end - 1;
}
value = deque._data[backIndex];
delete deque._data[backIndex];
deque._end = backIndex;
}

/**
* @dev Inserts an item at the beginning of the queue.
*/
function pushFront(Bytes32Deque storage deque, bytes32 value) internal {
int128 frontIndex;
unchecked {
frontIndex = deque._begin - 1;
}
deque._data[frontIndex] = value;
deque._begin = frontIndex;
}

/**
* @dev Removes the item at the beginning of the queue and returns it.
*
* Reverts with `Empty` if the queue is empty.
*/
function popFront(Bytes32Deque storage deque) internal returns (bytes32 value) {
if (empty(deque)) revert Empty();
int128 frontIndex = deque._begin;
value = deque._data[frontIndex];
delete deque._data[frontIndex];
unchecked {
deque._begin = frontIndex + 1;
}
}

/**
* @dev Returns the item at the beginning of the queue.
*/
function front(Bytes32Deque storage deque) internal view returns (bytes32 value) {
if (empty(deque)) revert Empty();
int128 frontIndex = deque._begin;
return deque._data[frontIndex];
}

/**
* @dev Returns the item at the end of the queue.
*/
function back(Bytes32Deque storage deque) internal view returns (bytes32 value) {
if (empty(deque)) revert Empty();
int128 backIndex;
unchecked {
backIndex = deque._end - 1;
}
return deque._data[backIndex];
}

/**
* @dev Return the item at a position in the queue given by `index`, with the first item at 0 and last item at
* `length(deque) - 1`.
*
* Reverts with `OutOfBounds` if the index is out of bounds.
*/
function at(Bytes32Deque storage deque, uint256 index) internal view returns (bytes32 value) {
// int256(deque._begin) is a safe upcast
int128 idx = SafeCast.toInt128(int256(deque._begin) + SafeCast.toInt256(index));
if (idx >= deque._end) revert OutOfBounds();
return deque._data[idx];
}

/**
* @dev Resets the queue back to being empty.
*
* NOTE: The current items are left behind in storage. This does not affect the functioning of the queue, but misses
* out on potential gas refunds.
*/
function clear(Bytes32Deque storage deque) internal {
deque._begin = 0;
deque._end = 0;
}

/**
* @dev Returns the number of items in the queue.
*/
function length(Bytes32Deque storage deque) internal view returns (uint256) {
// The interface preserves the invariant that begin <= end so we assume this will not overflow.
// We also assume there are at most int256.max items in the queue.
unchecked {
return uint256(int256(deque._end) - int256(deque._begin));
}
}

/**
* @dev Returns true if the queue is empty.
*/
function empty(Bytes32Deque storage deque) internal view returns (bool) {
return deque._end <= deque._begin;
}
}
108 changes: 108 additions & 0 deletions test/utils/structs/DoubleEndedQueue.test.js
@@ -0,0 +1,108 @@
const { expectEvent } = require('@openzeppelin/test-helpers');
const { expect } = require('chai');

const Bytes32DequeMock = artifacts.require('Bytes32DequeMock');

/** Rebuild the content of the deque as a JS array. */
async function getContent (deque) {
const length = await deque.length().then(bn => bn.toNumber());
const values = await Promise.all(Array(length).fill().map((_, i) => deque.at(i)));
return values;
}

/** Revert handler that supports custom errors. */
async function expectRevert (promise, reason) {
try {
await promise;
expect.fail('Expected promise to throw but it didn\'t');
} catch (error) {
if (reason) {
expect(error.message).to.include(reason);
}
}
}

contract('DoubleEndedQueue', function (accounts) {
const bytesA = '0xdeadbeef'.padEnd(66, '0');
const bytesB = '0x0123456789'.padEnd(66, '0');
const bytesC = '0x42424242'.padEnd(66, '0');
const bytesD = '0x171717'.padEnd(66, '0');

beforeEach(async function () {
this.deque = await Bytes32DequeMock.new();
});

describe('when empty', function () {
it('getters', async function () {
expect(await this.deque.empty()).to.be.equal(true);
expect(await getContent(this.deque)).to.have.ordered.members([]);
});

it('reverts on accesses', async function () {
await expectRevert(this.deque.popBack(), 'Empty()');
await expectRevert(this.deque.popFront(), 'Empty()');
await expectRevert(this.deque.back(), 'Empty()');
await expectRevert(this.deque.front(), 'Empty()');
});
});

describe('when not empty', function () {
beforeEach(async function () {
await this.deque.pushBack(bytesB);
await this.deque.pushFront(bytesA);
await this.deque.pushBack(bytesC);
this.content = [ bytesA, bytesB, bytesC ];
});

it('getters', async function () {
expect(await this.deque.empty()).to.be.equal(false);
expect(await this.deque.length()).to.be.bignumber.equal(this.content.length.toString());
expect(await this.deque.front()).to.be.equal(this.content[0]);
expect(await this.deque.back()).to.be.equal(this.content[this.content.length - 1]);
expect(await getContent(this.deque)).to.have.ordered.members(this.content);
});

it('out of bounds access', async function () {
await expectRevert(this.deque.at(this.content.length), 'OutOfBounds()');
});

describe('push', function () {
it('front', async function () {
await this.deque.pushFront(bytesD);
this.content.unshift(bytesD); // add element at the begining

expect(await getContent(this.deque)).to.have.ordered.members(this.content);
});

it('back', async function () {
await this.deque.pushBack(bytesD);
this.content.push(bytesD); // add element at the end

expect(await getContent(this.deque)).to.have.ordered.members(this.content);
});
});

describe('pop', function () {
it('front', async function () {
const value = this.content.shift(); // remove first element
expectEvent(await this.deque.popFront(), 'OperationResult', { value });

expect(await getContent(this.deque)).to.have.ordered.members(this.content);
});

it('back', async function () {
const value = this.content.pop(); // remove last element
expectEvent(await this.deque.popBack(), 'OperationResult', { value });

expect(await getContent(this.deque)).to.have.ordered.members(this.content);
});
});

it('clear', async function () {
await this.deque.clear();

expect(await this.deque.empty()).to.be.equal(true);
expect(await getContent(this.deque)).to.have.ordered.members([]);
});
});
});