-
Notifications
You must be signed in to change notification settings - Fork 294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add next_array
and collect_array
#560
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
fn main() { | ||
let is_nightly = version_check::is_feature_flaggable() == Some(true); | ||
let is_at_least_1_34 = version_check::is_min_version("1.34.0").unwrap_or(false); | ||
let is_at_least_1_51 = version_check::is_min_version("1.51.0").unwrap_or(false); | ||
|
||
if !is_at_least_1_34 && !is_nightly { | ||
println!("cargo:warning=itertools requires rustc => 1.34.0"); | ||
} | ||
|
||
if is_at_least_1_51 || is_nightly { | ||
println!("cargo:rustc-cfg=has_min_const_generics"); | ||
} | ||
} | ||
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -42,7 +42,7 @@ | |
//! | ||
//! ## Rust Version | ||
//! | ||
//! This version of itertools requires Rust 1.32 or later. | ||
//! This version of itertools requires Rust 1.34 or later. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If your assessment is correct, we could possibly increment the minimum rust version in a separate commit? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you mean a pull request? It's already a separate commit. |
||
#![doc(html_root_url="https://docs.rs/itertools/0.8/")] | ||
|
||
#[cfg(not(feature = "use_std"))] | ||
|
@@ -214,6 +214,8 @@ mod merge_join; | |
mod minmax; | ||
#[cfg(feature = "use_alloc")] | ||
mod multipeek_impl; | ||
#[cfg(has_min_const_generics)] | ||
mod next_array; | ||
mod pad_tail; | ||
#[cfg(feature = "use_alloc")] | ||
mod peek_nth; | ||
|
@@ -1679,6 +1681,58 @@ pub trait Itertools : Iterator { | |
} | ||
|
||
// non-adaptor methods | ||
/// Advances the iterator and returns the next items grouped in an array of | ||
/// a specific size. | ||
/// | ||
/// If there are enough elements to be grouped in an array, then the array | ||
/// is returned inside `Some`, otherwise `None` is returned. | ||
/// | ||
/// Requires Rust version 1.51 or later. | ||
/// | ||
/// ``` | ||
/// use itertools::Itertools; | ||
/// | ||
/// let mut iter = 1..5; | ||
/// | ||
/// assert_eq!(Some([1, 2]), iter.next_array()); | ||
/// ``` | ||
#[cfg(has_min_const_generics)] | ||
fn next_array<T, const N: usize>(&mut self) -> Option<[T; N]> | ||
where | ||
Self: Sized + Iterator<Item = T>, | ||
{ | ||
next_array::next_array(self) | ||
} | ||
|
||
|
||
/// Collects all items from the iterator into an array of a specific size. | ||
/// | ||
/// If the number of elements inside the iterator is **exactly** equal to | ||
/// the array size, then the array is returned inside `Some`, otherwise | ||
/// `None` is returned. | ||
/// | ||
/// Requires Rust version 1.51 or later. | ||
/// | ||
/// ``` | ||
/// use itertools::Itertools; | ||
/// | ||
/// let iter = 1..3; | ||
/// | ||
/// if let Some([x, y]) = iter.collect_array() { | ||
/// assert_eq!([x, y], [1, 2]) | ||
/// } else { | ||
/// panic!("Expected two elements") | ||
/// } | ||
/// ``` | ||
#[cfg(has_min_const_generics)] | ||
fn collect_array<T, const N: usize>(mut self) -> Option<[T; N]> | ||
where | ||
Self: Sized + Iterator<Item = T>, | ||
{ | ||
self.next_array().filter(|_| self.next().is_none()) | ||
} | ||
|
||
|
||
/// Advances the iterator and returns the next items grouped in a tuple of | ||
/// a specific size (up to 12). | ||
/// | ||
|
@@ -1699,6 +1753,7 @@ pub trait Itertools : Iterator { | |
T::collect_from_iter_no_buf(self) | ||
} | ||
|
||
|
||
/// Collects all items from the iterator into a tuple of a specific size | ||
/// (up to 12). | ||
/// | ||
|
@@ -1721,13 +1776,7 @@ pub trait Itertools : Iterator { | |
where Self: Sized + Iterator<Item = T::Item>, | ||
T: traits::HomogeneousTuple | ||
{ | ||
match self.next_tuple() { | ||
elt @ Some(_) => match self.next() { | ||
Some(_) => None, | ||
None => elt, | ||
}, | ||
_ => None | ||
} | ||
self.next_tuple().filter(|_| self.next().is_none()) | ||
Comment on lines
-1724
to
+1779
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this really relevant to this PR? If not, could we separate it into another PR? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It was mostly to be consistent with the other implementation. As it's just a stylistic change I don't think it's worth a pull request by itself to be honest. |
||
} | ||
|
||
|
||
|
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,80 @@ | ||||||||||||||||||||||||
use core::mem::MaybeUninit; | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think there was some discussion about building arrays: |
||||||||||||||||||||||||
|
||||||||||||||||||||||||
/// Helper struct to build up an array element by element. | ||||||||||||||||||||||||
struct ArrayBuilder<T, const N: usize> { | ||||||||||||||||||||||||
arr: [MaybeUninit<T>; N], | ||||||||||||||||||||||||
i: usize | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What is the safety invariant of |
||||||||||||||||||||||||
} | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
impl<T, const N: usize> ArrayBuilder<T, N> { | ||||||||||||||||||||||||
pub fn new() -> Self { | ||||||||||||||||||||||||
Self { arr: maybe_uninit::uninit_array(), i: 0 } | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
pub unsafe fn push_unchecked(&mut self, x: T) { | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Needs a safety comment, in the format of:
Suggested change
|
||||||||||||||||||||||||
debug_assert!(self.i < N); | ||||||||||||||||||||||||
*self.arr.get_unchecked_mut(self.i) = MaybeUninit::new(x); | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Needs a safety comment in the form:
Suggested change
|
||||||||||||||||||||||||
self.i += 1; | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
pub fn take(mut self) -> Option<[T; N]> { | ||||||||||||||||||||||||
if self.i == N { | ||||||||||||||||||||||||
unsafe { | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you scope this |
||||||||||||||||||||||||
// SAFETY: prevent double drop. | ||||||||||||||||||||||||
self.i = 0; | ||||||||||||||||||||||||
// SAFETY: [MaybeUninit<T>; N] and [T; N] have the same layout. | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could this safety comment cite the standard library documentation? While it's true that these two types have the same size and alignment, it's not true that they have the same bit validity. |
||||||||||||||||||||||||
let init_arr_ptr = &self.arr as *const _ as *const [T; N]; | ||||||||||||||||||||||||
Some(core::ptr::read(init_arr_ptr)) | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This needs a SAFETY comment citing what the preconditions of |
||||||||||||||||||||||||
} | ||||||||||||||||||||||||
} else { | ||||||||||||||||||||||||
None | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
impl<T, const N: usize> Drop for ArrayBuilder<T, N> { | ||||||||||||||||||||||||
fn drop(&mut self) { | ||||||||||||||||||||||||
unsafe { | ||||||||||||||||||||||||
// SAFETY: we only loop over the initialized portion. | ||||||||||||||||||||||||
for el in &mut self.arr[..self.i] { | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shouldn't need a loop here -- it's generally better to drop-in-place a whole slice rather than items individually. |
||||||||||||||||||||||||
maybe_uninit::assume_init_drop(el) | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
/// Equivalent to `it.next_array()`. | ||||||||||||||||||||||||
pub fn next_array<I, T, const N: usize>(it: &mut I) -> Option<[T; N]> | ||||||||||||||||||||||||
where | ||||||||||||||||||||||||
I: Iterator<Item = T>, | ||||||||||||||||||||||||
{ | ||||||||||||||||||||||||
let mut builder = ArrayBuilder::new(); | ||||||||||||||||||||||||
for el in it.take(N) { | ||||||||||||||||||||||||
unsafe { | ||||||||||||||||||||||||
// SAFETY: the take(N) guarantees we never go out of bounds. | ||||||||||||||||||||||||
builder.push_unchecked(el); | ||||||||||||||||||||||||
Comment on lines
+56
to
+57
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure this is sound -- there might be a way for me to override Maybe have it be something like it.try_for_each(|x| builder.try_push(x)); with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's a nasty one, I think you're right. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There should still be a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's right on the edge of soundness. There's no easy demo that I can come up with -- if you try to override So it's possible that it's actually sound today, but there's so many nuances to that argument that I think it's probably better to consider it unsound. For example, if Rust one day added a way to "call super" -- which seems like an entirely plausible feature -- then it'd immediately be obviously-unsound as someone could implement There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Oh, right, because otherwise you'll consume an extra element. Good catch. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Seconding @scottmcm's comment: For our MVP, does |
||||||||||||||||||||||||
} | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
builder.take() | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
/// Replacements for unstable core methods, copied from stdlib. | ||||||||||||||||||||||||
mod maybe_uninit { | ||||||||||||||||||||||||
use core::mem::MaybeUninit; | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
pub fn uninit_array<T, const N: usize>() -> [MaybeUninit<T>; N] { | ||||||||||||||||||||||||
// SAFETY: an uninitialized `[MaybeUninit<_>; N]` is valid. | ||||||||||||||||||||||||
unsafe { MaybeUninit::<[MaybeUninit<T>; N]>::uninit().assume_init() } | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
pub unsafe fn assume_init_drop<T>(u: &mut MaybeUninit<T>) { | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You don't need to replicate the entire stdlib doc comment here, but could you document the safety preconditions of |
||||||||||||||||||||||||
// SAFETY: the caller must guarantee that `self` is initialized and | ||||||||||||||||||||||||
// satisfies all invariants of `T`. | ||||||||||||||||||||||||
// Dropping the value in place is safe if that is the case. | ||||||||||||||||||||||||
core::ptr::drop_in_place(u.as_mut_ptr()) | ||||||||||||||||||||||||
} | ||||||||||||||||||||||||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Usually, I like the idea of having everything automated, but I am not sure if we should go with a
build.rs
and an additional dependency. My first idea was to use a feature flag (that would probably be off by default) that the user can enable if desired.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think feature flags to enable things that are already available in the latest stable Rust and have no further compile time or dependency drawbacks makes no sense.
These drawbacks while it could be done completely and correctly automatically are unacceptable in my opinion. If you are hesitant regarding the
version-check
dependency, I'd just like to note that it's tiny, has no further downstream dependencies, and is already relied on by crates such astime
,nom
,rocket
,fd-find
among others.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Philippe-Cholet, @phimuemue, perhaps it's time we updated our MSRV to 1.51 (which is two years old at this point).
While I don't mind the version-detection approach, I would like us to adopt it in tandem with changes to our CI that ensure we are testing on all detected versions. I'd also like to perhaps avoid taking the dependency on
rust_version
. This would all be a substantial change, and outside the scope of this PR.My vote is that we increase our MSRV. We can aways decrease it in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jswrenn I sure don't mind increasing the MSRV but I would suggest we release the 0.13.0 first, and then increase the MSRV in 0.14.0 to not require the build script (in which case orlp will have enough time to work on this).